2024-01-15T17:41:43.024 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-15T17:41:43.024 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi --owner scheduled_rishabh@teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} --archive /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105 --verbose -- /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/orig.config.yaml 2024-01-15T17:41:43.065 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-15T17:41:43.138 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7517105' kernel: flavor: default kdb: true sha1: 2668949e0971849893a3fb7f177ce9032fc65780 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-clone-stats sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9779 sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-clone-stats suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvuBUE6lVmsoYjKOKpKWQ7T+l0ufb15R/3UeDf8iIWTqzvGiIbD/viXR9zGGy67YZY6WEFp9Nu7+ofrtGNcZBY= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC0TxJjXSPdKNBhbICy8dkJxph6idNeJjN3iVR7cgJk1Al0pJ2ycofFY0WlQIAvYDBAwL2oSo0c545raLcSNHVs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgress teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-15_17:35:46 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-15T17:41:43.139 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa; will attempt to use it 2024-01-15T17:41:43.139 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks 2024-01-15T17:41:43.139 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-15T17:41:43.141 INFO:teuthology.task.internal:Checking packages... 2024-01-15T17:41:43.162 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '95c90f413c4bb65e3e56909855f2308fd9bda05b' 2024-01-15T17:41:43.162 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-15T17:41:43.162 INFO:teuthology.packaging:ref: None 2024-01-15T17:41:43.162 INFO:teuthology.packaging:tag: None 2024-01-15T17:41:43.162 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-01-15T17:41:43.162 INFO:teuthology.packaging:sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T17:41:43.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=rishabh-clone-stats 2024-01-15T17:41:43.307 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7629-g95c90f41-1jammy 2024-01-15T17:41:43.308 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-15T17:41:43.314 INFO:teuthology.task.internal:no buildpackages task found 2024-01-15T17:41:43.315 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-15T17:41:43.337 INFO:teuthology.task.internal:Saving configuration 2024-01-15T17:41:43.351 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-15T17:41:43.356 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-15T17:41:43.381 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:54.432779', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHKa5Ft9gE+IHw3gBF/Jk75MI8I5lDRCNH0qmLuAss9q'} 2024-01-15T17:41:43.407 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:54.431240', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOTNSLm76qPq4gZ1Li0ibcy1EMjhmORbO/f0NofiYKlM'} 2024-01-15T17:41:43.407 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-15T17:41:43.413 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-15T17:41:43.413 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-15T17:41:43.413 INFO:teuthology.run_tasks:Running task console_log... 2024-01-15T17:41:43.485 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa6be311550>, signals=[15]) 2024-01-15T17:41:43.486 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-15T17:41:43.492 INFO:teuthology.task.internal:Opening connections... 2024-01-15T17:41:43.492 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T17:41:43.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:41:43.574 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T17:41:43.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:41:43.651 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-15T17:41:43.657 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-01-15T17:41:43.662 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-01-15T17:41:43.662 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Ubuntu" 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="22.04" 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_CODENAME=jammy 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:ID=ubuntu 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE=debian 2024-01-15T17:41:43.708 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-15T17:41:43.709 INFO:teuthology.orchestra.run.smithi143.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-15T17:41:43.709 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-15T17:41:43.709 INFO:teuthology.orchestra.run.smithi143.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-15T17:41:43.709 INFO:teuthology.orchestra.run.smithi143.stdout:UBUNTU_CODENAME=jammy 2024-01-15T17:41:43.709 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-01-15T17:41:43.747 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-01-15T17:41:43.752 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-01-15T17:41:43.752 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="Ubuntu" 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="22.04" 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_CODENAME=jammy 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:ID=ubuntu 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE=debian 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://www.ubuntu.com/" 2024-01-15T17:41:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-01-15T17:41:43.800 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-01-15T17:41:43.800 INFO:teuthology.orchestra.run.smithi204.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-01-15T17:41:43.800 INFO:teuthology.orchestra.run.smithi204.stdout:UBUNTU_CODENAME=jammy 2024-01-15T17:41:43.800 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-01-15T17:41:43.829 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-15T17:41:43.837 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-15T17:41:43.842 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-15T17:41:43.842 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-01-15T17:41:43.845 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-01-15T17:41:43.849 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-15T17:41:43.855 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-15T17:41:43.855 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-01-15T17:41:43.893 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-01-15T17:41:43.901 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-15T17:41:43.957 INFO:teuthology.run_tasks:Running task kernel... 2024-01-15T17:41:43.971 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'} 2024-01-15T17:41:43.971 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}}, timeout 300 2024-01-15T17:41:43.971 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-15T17:41:43.972 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-15T17:41:43.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:41:43.977 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:43.977 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2668949e0971849893a3fb7f177ce9032fc65780"... 2024-01-15T17:41:43.977 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-01-15T17:41:43.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:41:43.978 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:43.978 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2668949e0971849893a3fb7f177ce9032fc65780"... 2024-01-15T17:41:43.979 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-01-15T17:41:44.023 INFO:teuthology.orchestra.run.smithi204.stdout:5.15.0-56-generic 2024-01-15T17:41:44.023 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:44.023 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-15T17:41:44.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:44.027 INFO:teuthology.orchestra.run.smithi143.stdout:5.15.0-56-generic 2024-01-15T17:41:44.027 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:44.027 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-15T17:41:44.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:44.168 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-01-15T17:41:44.169 DEBUG:teuthology.orchestra.run.smithi204:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-01-15T17:41:44.180 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-01-15T17:41:44.300 INFO:teuthology.orchestra.run.smithi204.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: git config --global init.defaultBranch 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: 2024-01-15T17:41:44.301 INFO:teuthology.orchestra.run.smithi204.stderr:hint: git branch -m 2024-01-15T17:41:44.302 INFO:teuthology.orchestra.run.smithi204.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-01-15T17:41:44.304 DEBUG:teuthology.orchestra.run.smithi204:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-01-15T17:41:44.358 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-01-15T17:41:44.359 DEBUG:teuthology.orchestra.run.smithi143:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-01-15T17:41:44.373 DEBUG:teuthology.orchestra.run.smithi204:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-01-15T17:41:44.378 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-01-15T17:41:44.427 DEBUG:teuthology.orchestra.run.smithi204:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: git config --global init.defaultBranch 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: 2024-01-15T17:41:44.500 INFO:teuthology.orchestra.run.smithi143.stderr:hint: git branch -m 2024-01-15T17:41:44.501 INFO:teuthology.orchestra.run.smithi143.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-01-15T17:41:44.503 DEBUG:teuthology.orchestra.run.smithi143:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-01-15T17:41:44.567 DEBUG:teuthology.orchestra.run.smithi143:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-01-15T17:41:44.626 DEBUG:teuthology.orchestra.run.smithi143:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-01-15T17:49:54.862 INFO:teuthology.orchestra.run.smithi204.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-01-15T17:49:54.863 INFO:teuthology.orchestra.run.smithi204.stderr: * [new branch] main -> origin/main 2024-01-15T17:49:54.863 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20240115 -> 20240115 2024-01-15T17:49:54.887 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190312 -> 20190312 2024-01-15T17:49:54.887 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190416 -> 20190416 2024-01-15T17:49:54.887 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190514 -> 20190514 2024-01-15T17:49:54.887 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190618 -> 20190618 2024-01-15T17:49:54.887 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190717 -> 20190717 2024-01-15T17:49:54.887 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190815 -> 20190815 2024-01-15T17:49:54.888 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20190923 -> 20190923 2024-01-15T17:49:54.888 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20191022 -> 20191022 2024-01-15T17:49:54.888 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20191215 -> 20191215 2024-01-15T17:49:54.888 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200122 -> 20200122 2024-01-15T17:49:54.893 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200316 -> 20200316 2024-01-15T17:49:54.893 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200421 -> 20200421 2024-01-15T17:49:54.893 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200519 -> 20200519 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200619 -> 20200619 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200721 -> 20200721 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200817 -> 20200817 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20200918 -> 20200918 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20201022 -> 20201022 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20201118 -> 20201118 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20201218 -> 20201218 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20210208 -> 20210208 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20210315 -> 20210315 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20210511 -> 20210511 2024-01-15T17:49:54.894 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20210716 -> 20210716 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20210818 -> 20210818 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20210919 -> 20210919 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20211027 -> 20211027 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20211216 -> 20211216 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220209 -> 20220209 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220310 -> 20220310 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220411 -> 20220411 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220509 -> 20220509 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220610 -> 20220610 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220708 -> 20220708 2024-01-15T17:49:54.895 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220815 -> 20220815 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20220913 -> 20220913 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20221012 -> 20221012 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20221109 -> 20221109 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20221214 -> 20221214 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230117 -> 20230117 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230210 -> 20230210 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230310 -> 20230310 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230404 -> 20230404 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230515 -> 20230515 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230625 -> 20230625 2024-01-15T17:49:54.896 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230804 -> 20230804 2024-01-15T17:49:54.897 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20230919 -> 20230919 2024-01-15T17:49:54.897 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20231030 -> 20231030 2024-01-15T17:49:54.897 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20231111 -> 20231111 2024-01-15T17:49:54.897 INFO:teuthology.orchestra.run.smithi204.stderr: * [new tag] 20231211 -> 20231211 2024-01-15T17:49:58.980 INFO:teuthology.orchestra.run.smithi204.stderr:Updating files: 49% (1703/3424) Updating files: 50% (1712/3424) Updating files: 51% (1747/3424) Updating files: 52% (1781/3424) Updating files: 53% (1815/3424) Updating files: 54% (1849/3424) Updating files: 55% (1884/3424) Updating files: 56% (1918/3424) Updating files: 57% (1952/3424) Updating files: 58% (1986/3424) Updating files: 59% (2021/3424) Updating files: 60% (2055/3424) Updating files: 61% (2089/3424) Updating files: 62% (2123/3424) Updating files: 63% (2158/3424) Updating files: 64% (2192/3424) Updating files: 64% (2214/3424) Updating files: 65% (2226/3424) Updating files: 66% (2260/3424) Updating files: 67% (2295/3424) Updating files: 68% (2329/3424) Updating files: 69% (2363/3424) Updating files: 70% (2397/3424) Updating files: 71% (2432/3424) Updating files: 72% (2466/3424) Updating files: 73% (2500/3424) Updating files: 74% (2534/3424) Updating files: 75% (2568/3424) Updating files: 76% (2603/3424) Updating files: 77% (2637/3424) Updating files: 78% (2671/3424) Updating files: 79% (2705/3424) Updating files: 80% (2740/3424) Updating files: 81% (2774/3424) Updating files: 81% (2782/3424) Updating files: 82% (2808/3424) Updating files: 83% (2842/3424) Updating files: 84% (2877/3424) Updating files: 85% (2911/3424) Updating files: 86% (2945/3424) Updating files: 87% (2979/3424) Updating files: 88% (3014/3424) Updating files: 89% (3048/3424) Updating files: 90% (3082/3424) Updating files: 91% (3116/3424) Updating files: 92% (3151/3424) Updating files: 93% (3185/3424) Updating files: 94% (3219/3424) Updating files: 95% (3253/3424) Updating files: 95% (3287/3424) Updating files: 96% (3288/3424) Updating files: 97% (3322/3424) Updating files: 98% (3356/3424) Updating files: 99% (3390/3424) Updating files: 100% (3424/3424) Updating files: 100% (3424/3424), done. 2024-01-15T17:49:58.985 INFO:teuthology.orchestra.run.smithi204.stdout:HEAD is now at 9b6d0b08 Merge branch 'cirrus' into 'main' 2024-01-15T17:49:59.006 INFO:teuthology.task.kernel:Downloading kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.b... 2024-01-15T17:49:59.007 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:49:59.148 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/ubuntu/jammy/flavors/default/pool/main/l/linux-6.7.0-g2668949e0971/ 2024-01-15T17:49:59.148 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /tmp/linux-image.deb && echo linux-image-6.7.0-g2668949e0971_6.7.0-g2668949e0971-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/ubuntu/jammy/flavors/default/pool/main/l/linux-6.7.0-g2668949e0971/ --input-file=- 2024-01-15T17:49:59.151 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-01-15T17:50:00.651 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15 17:50:00 URL:https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/ubuntu/jammy/flavors/default/pool/main/l/linux-6.7.0-g2668949e0971/linux-image-6.7.0-g2668949e0971_6.7.0-g2668949e0971-1_amd64.deb [39096058/39096058] -> "/tmp/linux-image.deb" [1] 2024-01-15T17:50:00.652 INFO:teuthology.orchestra.run.smithi204.stderr:FINISHED --2024-01-15 17:50:00-- 2024-01-15T17:50:00.652 INFO:teuthology.orchestra.run.smithi204.stderr:Total wall clock time: 1.5s 2024-01-15T17:50:00.652 INFO:teuthology.orchestra.run.smithi204.stderr:Downloaded: 1 files, 37M in 1.4s (27.6 MB/s) 2024-01-15T17:50:00.657 INFO:teuthology.task.kernel:Installing kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.b... 2024-01-15T17:50:00.657 DEBUG:teuthology.orchestra.run.smithi204:> sudo dpkg -i /tmp/linux-image.deb 2024-01-15T17:50:00.898 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package linux-image-6.7.0-g2668949e0971. 2024-01-15T17:50:02.915 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-01-15T17:50:02.919 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-01-15T17:50:03.039 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking linux-image-6.7.0-g2668949e0971 (6.7.0-g2668949e0971-1) ... 2024-01-15T17:50:06.133 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up linux-image-6.7.0-g2668949e0971 (6.7.0-g2668949e0971-1) ... 2024-01-15T17:50:06.242 INFO:teuthology.orchestra.run.smithi204.stderr:update-initramfs: Generating /boot/initrd.img-6.7.0-g2668949e0971 2024-01-15T17:50:11.013 INFO:teuthology.orchestra.run.smithi143.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-01-15T17:50:11.014 INFO:teuthology.orchestra.run.smithi143.stderr: * [new branch] main -> origin/main 2024-01-15T17:50:11.014 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20240115 -> 20240115 2024-01-15T17:50:11.039 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190312 -> 20190312 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190416 -> 20190416 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190514 -> 20190514 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190618 -> 20190618 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190717 -> 20190717 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190815 -> 20190815 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20190923 -> 20190923 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20191022 -> 20191022 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20191215 -> 20191215 2024-01-15T17:50:11.040 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200122 -> 20200122 2024-01-15T17:50:11.042 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200316 -> 20200316 2024-01-15T17:50:11.042 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200421 -> 20200421 2024-01-15T17:50:11.042 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200519 -> 20200519 2024-01-15T17:50:11.042 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200619 -> 20200619 2024-01-15T17:50:11.042 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200721 -> 20200721 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200817 -> 20200817 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20200918 -> 20200918 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20201022 -> 20201022 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20201118 -> 20201118 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20201218 -> 20201218 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210208 -> 20210208 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210315 -> 20210315 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210511 -> 20210511 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210716 -> 20210716 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210818 -> 20210818 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20210919 -> 20210919 2024-01-15T17:50:11.043 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20211027 -> 20211027 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20211216 -> 20211216 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220209 -> 20220209 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220310 -> 20220310 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220411 -> 20220411 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220509 -> 20220509 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220610 -> 20220610 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220708 -> 20220708 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220815 -> 20220815 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20220913 -> 20220913 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20221012 -> 20221012 2024-01-15T17:50:11.045 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20221109 -> 20221109 2024-01-15T17:50:11.046 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20221214 -> 20221214 2024-01-15T17:50:11.046 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230117 -> 20230117 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230210 -> 20230210 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230310 -> 20230310 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230404 -> 20230404 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230515 -> 20230515 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230625 -> 20230625 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230804 -> 20230804 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20230919 -> 20230919 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20231030 -> 20231030 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20231111 -> 20231111 2024-01-15T17:50:11.047 INFO:teuthology.orchestra.run.smithi143.stderr: * [new tag] 20231211 -> 20231211 2024-01-15T17:50:15.127 INFO:teuthology.orchestra.run.smithi143.stderr:Updating files: 49% (1704/3424) Updating files: 50% (1712/3424) Updating files: 51% (1747/3424) Updating files: 52% (1781/3424) Updating files: 53% (1815/3424) Updating files: 54% (1849/3424) Updating files: 55% (1884/3424) Updating files: 56% (1918/3424) Updating files: 57% (1952/3424) Updating files: 58% (1986/3424) Updating files: 59% (2021/3424) Updating files: 60% (2055/3424) Updating files: 61% (2089/3424) Updating files: 62% (2123/3424) Updating files: 63% (2158/3424) Updating files: 64% (2192/3424) Updating files: 64% (2215/3424) Updating files: 65% (2226/3424) Updating files: 66% (2260/3424) Updating files: 67% (2295/3424) Updating files: 68% (2329/3424) Updating files: 69% (2363/3424) Updating files: 70% (2397/3424) Updating files: 71% (2432/3424) Updating files: 72% (2466/3424) Updating files: 73% (2500/3424) Updating files: 74% (2534/3424) Updating files: 75% (2568/3424) Updating files: 76% (2603/3424) Updating files: 77% (2637/3424) Updating files: 78% (2671/3424) Updating files: 79% (2705/3424) Updating files: 80% (2740/3424) Updating files: 81% (2774/3424) Updating files: 81% (2783/3424) Updating files: 82% (2808/3424) Updating files: 83% (2842/3424) Updating files: 84% (2877/3424) Updating files: 85% (2911/3424) Updating files: 86% (2945/3424) Updating files: 87% (2979/3424) Updating files: 88% (3014/3424) Updating files: 89% (3048/3424) Updating files: 90% (3082/3424) Updating files: 91% (3116/3424) Updating files: 92% (3151/3424) Updating files: 93% (3185/3424) Updating files: 94% (3219/3424) Updating files: 95% (3253/3424) Updating files: 96% (3288/3424) Updating files: 96% (3302/3424) Updating files: 97% (3322/3424) Updating files: 98% (3356/3424) Updating files: 99% (3390/3424) Updating files: 100% (3424/3424) Updating files: 100% (3424/3424), done. 2024-01-15T17:50:15.132 INFO:teuthology.orchestra.run.smithi143.stdout:HEAD is now at 9b6d0b08 Merge branch 'cirrus' into 'main' 2024-01-15T17:50:15.153 INFO:teuthology.task.kernel:Downloading kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.a... 2024-01-15T17:50:15.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:50:15.289 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/ubuntu/jammy/flavors/default/pool/main/l/linux-6.7.0-g2668949e0971/ 2024-01-15T17:50:15.289 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /tmp/linux-image.deb && echo linux-image-6.7.0-g2668949e0971_6.7.0-g2668949e0971-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/ubuntu/jammy/flavors/default/pool/main/l/linux-6.7.0-g2668949e0971/ --input-file=- 2024-01-15T17:50:15.292 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi143.front.sepia.ceph.com to complete... 2024-01-15T17:50:17.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15 17:50:17 URL:https://3.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/ubuntu/jammy/flavors/default/pool/main/l/linux-6.7.0-g2668949e0971/linux-image-6.7.0-g2668949e0971_6.7.0-g2668949e0971-1_amd64.deb [39096058/39096058] -> "/tmp/linux-image.deb" [1] 2024-01-15T17:50:17.540 INFO:teuthology.orchestra.run.smithi143.stderr:FINISHED --2024-01-15 17:50:17-- 2024-01-15T17:50:17.540 INFO:teuthology.orchestra.run.smithi143.stderr:Total wall clock time: 2.2s 2024-01-15T17:50:17.540 INFO:teuthology.orchestra.run.smithi143.stderr:Downloaded: 1 files, 37M in 1.8s (20.2 MB/s) 2024-01-15T17:50:17.544 INFO:teuthology.task.kernel:Installing kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.a... 2024-01-15T17:50:17.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo dpkg -i /tmp/linux-image.deb 2024-01-15T17:50:17.796 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-image-6.7.0-g2668949e0971. 2024-01-15T17:50:19.827 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-01-15T17:50:19.832 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-01-15T17:50:19.964 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-image-6.7.0-g2668949e0971 (6.7.0-g2668949e0971-1) ... 2024-01-15T17:50:23.168 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-image-6.7.0-g2668949e0971 (6.7.0-g2668949e0971-1) ... 2024-01-15T17:50:23.261 INFO:teuthology.orchestra.run.smithi143.stderr:update-initramfs: Generating /boot/initrd.img-6.7.0-g2668949e0971 2024-01-15T17:50:27.881 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub' 2024-01-15T17:50:27.899 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-15T17:50:27.924 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-01-15T17:50:28.255 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:50:28.283 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-6.7.0-g2668949e0971 2024-01-15T17:50:28.579 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-15T17:50:28.588 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-15T17:50:28.656 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-15T17:50:28.661 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-15T17:50:28.817 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-15T17:50:28.817 INFO:teuthology.orchestra.run.smithi204.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-01-15T17:50:28.818 INFO:teuthology.orchestra.run.smithi204.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-15T17:50:28.853 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-01-15T17:50:29.024 DEBUG:teuthology.orchestra.run.smithi204:> dpkg-deb -c /tmp/linux-image.deb 2024-01-15T17:50:29.032 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./ 2024-01-15T17:50:29.033 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./boot/ 2024-01-15T17:50:29.033 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 4444641 2024-01-15 16:53 ./boot/System.map-6.7.0-g2668949e0971 2024-01-15T17:50:29.044 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 170149 2024-01-15 16:53 ./boot/config-6.7.0-g2668949e0971 2024-01-15T17:50:29.045 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11441152 2024-01-15 16:53 ./boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/postinst.d/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/postrm.d/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/preinst.d/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/prerm.d/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/ 2024-01-15T17:50:29.068 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/ 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/ 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/ 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/ 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 490345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/aesni-intel.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 93113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 106729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 152241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-15T17:50:29.069 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 83889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/ 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/ 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/ 2024-01-15T17:50:29.070 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-15T17:50:29.071 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpuid.ko 2024-01-15T17:50:29.071 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/msr.ko 2024-01-15T17:50:29.071 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/ 2024-01-15T17:50:29.071 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 353705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-15T17:50:29.071 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 881649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-15T17:50:29.071 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 2276625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm.ko 2024-01-15T17:50:29.111 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/ 2024-01-15T17:50:29.111 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/af_alg.ko 2024-01-15T17:50:29.111 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/algif_hash.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/algif_skcipher.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/ansi_cprng.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/anubis.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/arc4.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/asymmetric_keys/ 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/ 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_memcpy.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_pq.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_tx.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_xor.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/raid6test.ko 2024-01-15T17:50:29.112 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/authenc.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/authencesn.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/blake2b_generic.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_common.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_generic.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/camellia_generic.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast5_generic.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast6_generic.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast_common.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/ccm.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cmac.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crc32_generic.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cryptd.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_engine.ko 2024-01-15T17:50:29.113 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_simd.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_user.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cts.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/des_generic.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/dh_generic.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/echainiv.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/essiv.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/fcrypt.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/khazad.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/lrw.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4hc.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/md4.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/michael_mic.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcbc.ko 2024-01-15T17:50:29.114 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcrypt.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/rmd160.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/seed.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/serpent_generic.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 149481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/tcrypt.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/tea.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_common.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_generic.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/vmac.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/wp512.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xcbc.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xor.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xts.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xxhash_generic.ko 2024-01-15T17:50:29.115 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_extlog.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_pad.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/apei/ 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/apei/einj.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ec_sys.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/platform_profile.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbs.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbshc.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 104729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/video.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/acard-ahci.ko 2024-01-15T17:50:29.116 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci_platform.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 88401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci_platform.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_acpi.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ali.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_amd.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_artop.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atiixp.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atp867x.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd640.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd64x.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cypress.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_efar.ko 2024-01-15T17:50:29.117 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt366.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt37x.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it8213.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it821x.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_jmicron.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_legacy.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_marvell.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_mpiix.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_netcell.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ninja32.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87410.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87415.ko 2024-01-15T17:50:29.118 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_oldpiix.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_opti.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_optidma.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pcmcia.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_piccolo.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_radisys.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rdc.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rz1000.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sch.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_serverworks.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sil680.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sl82c105.ko 2024-01-15T17:50:29.119 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_triflex.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_via.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pdc_adma.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_inic162x.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 83345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_mv.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_nv.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_promise.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_qstor.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil24.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sis.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_svw.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sx4.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_uli.ko 2024-01-15T17:50:29.120 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_via.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_vsc.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ks0108.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/ 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 124473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/bcma.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/ 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/ 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 113753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/aoe.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/ 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 965305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/drbd.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 173353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/floppy.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/ 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 94297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 116321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/nbd.ko 2024-01-15T17:50:29.121 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/ 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 110737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/null_blk.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 98281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/pktcdvd.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 263769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/rbd.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/ 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 98185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/ 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/zram.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/applicom.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hangcheck-timer.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/ 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-15T17:50:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/via-rng.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 148609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 139313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/lp.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/ 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 84097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/mwave.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/nvram.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ppdev.ko 2024-01-15T17:50:29.123 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tlclk.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/ 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/ 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/xillybus_class.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/xillybus_core.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/ 2024-01-15T17:50:29.124 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-palmas.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-si5351.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-twl6040.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-wm831x.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/ 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 246841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp.ko 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/ 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/ 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/ 2024-01-15T17:50:29.125 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 517033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-aes.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-sha.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/ 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/ 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/dca.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/ 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 122569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-15T17:50:29.126 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 79953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/amd64_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/e752x_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 63617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/edac_mce_amd.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3000_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3200_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5100_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5400_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7300_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7core_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i82975x_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ie31200_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/sb_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/x38_edac.ko 2024-01-15T17:50:29.127 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/ 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-gpio.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-max14577.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-max77693.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-max8997.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-palmas.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-rt8973a.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-sm5502.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/ 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/dmi-sysfs.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/efi/ 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/efi/efi-pstore.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/ 2024-01-15T17:50:29.128 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-adp5520.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-amd8111.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-arizona.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-da9052.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-da9055.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-dln2.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 77849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-f7188x.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-generic.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ich.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-kempld.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-lp3943.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-max7300.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-max730x.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-max732x.ko 2024-01-15T17:50:29.129 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-pca953x.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-pcf857x.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-rdc321x.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-sch.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-sch311x.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-tps65912.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-twl4030.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-twl6040.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-vx855.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-wm831x.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-wm8350.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-wm8994.ko 2024-01-15T17:50:29.130 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/ 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-a4tech.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-apple.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-appleir.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-aureal.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-axff.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-belkin.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-cherry.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-chicony.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-cp2112.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-cypress.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-dr.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elecom.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elo.ko 2024-01-15T17:50:29.131 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-emsff.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ezkey.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gaff.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-generic.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gt683r.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gyration.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtekff.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-hyperv.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-icade.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kensington.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-keytouch.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kye.ko 2024-01-15T17:50:29.132 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lcpower.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-led.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lenovo.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lg-g15.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 100649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 80993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-magicmouse.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-microsoft.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-monterey.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-multitouch.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ntrig.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ortek.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-penmount.ko 2024-01-15T17:50:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-petalynx.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 72937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-picolcd.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-pl.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-plantronics.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-primax.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-rmi.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-common.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-15T17:50:29.134 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-saitek.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-samsung.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sjoy.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sony.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-speedlink.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-steelseries.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sunplus.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tivo.ko 2024-01-15T17:50:29.135 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tmff.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-topseed.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-twinhan.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 71073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-uclogic.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-waltop.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 143473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-wiimote.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-xinmo.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zpff.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zydacron.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 269281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/uhid.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/ 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 125785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/usbhid.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/usbkbd.ko 2024-01-15T17:50:29.136 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/usbmouse.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 217873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/wacom.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/ 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 64201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_balloon.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_utils.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 305321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_vmbus.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru3.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7414.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7418.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adc128d818.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1025.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 97977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1026.ko 2024-01-15T17:50:29.137 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1029.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1031.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm9240.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ads7828.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7410.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7411.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 74409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7462.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7470.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7475.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7x10.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/amc6821.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 55201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/applesmc.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asb100.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asc7621.ko 2024-01-15T17:50:29.138 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/atxp1.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/coretemp.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/da9052-hwmon.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/da9055-hwmon.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 99601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dme1737.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds1621.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds620.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc1403.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc2103.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc6w201.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71805f.ko 2024-01-15T17:50:29.139 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 94689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71882fg.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f75375s.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fam15h_power.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 55449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fschmd.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g760a.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g762.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl518sm.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl520sm.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hih6130.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5500_temp.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5k_amb.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmaem.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmpex.ko 2024-01-15T17:50:29.140 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina209.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina2xx.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 130449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/it87.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/jc42.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k10temp.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k8temp.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lineage-pem.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm63.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm73.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm75.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm77.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm78.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm80.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm83.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 72833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm85.ko 2024-01-15T17:50:29.141 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm87.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm90.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm92.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 108841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm93.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95234.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95241.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95245.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc2945.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4151.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4215.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4222.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4245.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4260.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4261.ko 2024-01-15T17:50:29.142 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max16065.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1619.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1668.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max197.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6639.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6650.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6697.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mc13783-adc.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mcp3021.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6683.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 119689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775-core.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct7802.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 82337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87360.ko 2024-01-15T17:50:29.143 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87427.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pcf8591.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 82577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-15T17:50:29.144 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/powr1220.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5627.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5636.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht15.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht21.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/shtc1.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sis5595.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47b397.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m1.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m192.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/thmc50.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp102.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp103.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp401.ko 2024-01-15T17:50:29.145 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp421.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via-cputemp.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via686a.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt1211.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt8231.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 57449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627ehf.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627hf.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 71345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83781d.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 63225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83791d.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 71321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83792d.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 101473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83793.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 119505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83795.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l785ts.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l786ng.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-01-15T17:50:29.146 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/ 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/ 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/ 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 6929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-15T17:50:29.147 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 62681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-15T17:50:29.148 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-15T17:50:29.149 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-mux.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-smbus.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-stub.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/ 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 273609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 883681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_core.ko 2024-01-15T17:50:29.150 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 324329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 97921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 282601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/ 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/ 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 493217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/ 2024-01-15T17:50:29.151 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 420561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-15T17:50:29.207 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 741449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 296881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/ocrdma/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 153801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/usnic/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 168969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 263081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ 2024-01-15T17:50:29.208 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 100353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 111425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 128753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 129529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/ 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/evbug.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/ff-memless.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/ 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/gameport.ko 2024-01-15T17:50:29.209 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/joydev.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/ 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/lkkbd.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/lm8323.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/lm8333.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-01-15T17:50:29.210 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/newtonkbd.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/qt1070.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/qt2160.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/stowaway.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/sunkbd.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/xtkbd.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/matrix-keymap.ko 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/ 2024-01-15T17:50:29.211 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/appletouch.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/bcm5974.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 123417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/cyapatp.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/gpio_mouse.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 335953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/psmouse.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/sermouse.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/ 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 109873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-15T17:50:29.212 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/ 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/altera_ps2.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/arc_ps2.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/ct82c710.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/parkbd.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/pcips2.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/ps2mult.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serio_raw.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serport.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/sparse-keymap.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/ 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-88pm860x.ko 2024-01-15T17:50:29.213 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-adp5520.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-bd2802.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-blinkm.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-da903x.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-da9052.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-gpio.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3530.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3533.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm355x.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3642.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp3944.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp8788.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-max8997.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-mc13783.ko 2024-01-15T17:50:29.214 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-menf21bmc.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-pca9532.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-pca955x.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-pca963x.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-ss4200.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-tca6507.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-wm831x-status.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-wm8350.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-15T17:50:29.215 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/ 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/ 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 647193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/bcache.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bio-prison.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 93409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bufio.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache-smq.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 155897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 107257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-crypt.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-delay.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-era.ko 2024-01-15T17:50:29.216 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-flakey.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log-userspace.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-mirror.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-multipath.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-queue-length.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 71073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-raid.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-region-hash.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-round-robin.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-service-time.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 109529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-snapshot.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-switch.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 181865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-thin-pool.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-verity.ko 2024-01-15T17:50:29.217 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-zero.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/faulty.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/linear.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/multipath.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/ 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 199937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid0.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 113001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid1.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 158657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid10.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 393777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid456.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/ 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/ 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 178137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptbase.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 63321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptctl.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptfc.ko 2024-01-15T17:50:29.218 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptlan.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 123321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptsas.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 81009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptscsih.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptspi.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/ 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/88pm800.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/88pm805.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/88pm80x.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/arizona-i2c.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 123721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/arizona.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/bcm590xx.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/cros_ec_dev.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/dln2.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/janz-cmodio.ko 2024-01-15T17:50:29.219 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/kempld-core.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lm3533-core.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lp3943.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 57057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_ich.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_sch.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/max8907.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mc13xxx-core.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/menf21bmc.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/pcf50633-adc.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/pcf50633-gpio.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/pcf50633.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-01-15T17:50:29.220 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/retu-mfd.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 80425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/si476x-core.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/sm501.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/ti_am335x_tscadc.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/tps65010.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/tps6507x.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/viperboard.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/vx855.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wl1273-core.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wm8994.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ad525x_dpot.ko 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/ 2024-01-15T17:50:29.221 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds9802als.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds990x.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/bh1770glc.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/c2port/ 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/c2port/core.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/ 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/cb710.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ds1682.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/dummy-irq.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/echo/ 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/echo/echo.ko 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/ 2024-01-15T17:50:29.222 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/at24.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/max6875.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/enclosure.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/genwqe/ 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 152201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hmc6352.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hpilo.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ibmasm/ 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 72601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ics932s401.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29003.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29020.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/ 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-15T17:50:29.223 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/ 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 125945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-me.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-txe.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 313641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/phantom.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ti-st/ 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 74377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ti-st/st_drv.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_7xx1.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_core.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tsl2550.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_balloon.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/ 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 183609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-15T17:50:29.224 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 490849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/bonding.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 106033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 455065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dummy.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/eql.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/ 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-15T17:50:29.225 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 130553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 68849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 99049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/ 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/ 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/ 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-15T17:50:29.226 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 87297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 97337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 102497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 90345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/ 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 81569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-15T17:50:29.227 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/ 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 176761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1421529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 128569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/ 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 131337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 314553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/ 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/ 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 323745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/ 2024-01-15T17:50:29.228 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 193497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 347257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1044793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 166193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 204345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/ 2024-01-15T17:50:29.229 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 141225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 62817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/ 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 62833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dnet.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ec_bhf.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/ 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/ 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 281137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-15T17:50:29.230 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ethoc.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fealnx.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fujitsu/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 90369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 295481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 584497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 246209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 937673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/ 2024-01-15T17:50:29.231 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 287305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/ 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 514569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/ 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 93497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 734353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 151769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 86921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/jme.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/ 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 102289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 122601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-15T17:50:29.232 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 727473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 278961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1118201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 95665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 108649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-15T17:50:29.233 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 73441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/ 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 150905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/ 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 136721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/ 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/ 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 227897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 87105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-15T17:50:29.234 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 671937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 83985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 192353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/samsung/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 104241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1064737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/ 2024-01-15T17:50:29.235 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/ 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 63337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/ 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 73409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 68177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/ 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-01-15T17:50:29.236 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 487593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/ 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 103337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 166097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/ 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/ 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 86529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/ 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 80273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 83793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/ 2024-01-15T17:50:29.237 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/ 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/ 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/defxx.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/skfp/ 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 216985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/skfp/skfp.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 74569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/geneve.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/ 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 236225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/ 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ifb.ko 2024-01-15T17:50:29.238 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 85817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 62769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvlan.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvtap.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/ 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-gpio.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mii.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/netconsole.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/nlmon.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/ 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-15T17:50:29.239 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/amd.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm87xx.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/broadcom.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/cicada.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/davicom.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/et1011c.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/icplus.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/lxt.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 82049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/marvell.ko 2024-01-15T17:50:29.240 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 76409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/micrel.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/national.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 95561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/phylink.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/qsemi.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/realtek.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/smsc.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ste10Xp.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/vitesse.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/plip/ 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/plip/plip.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_async.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-15T17:50:29.241 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppoe.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppox.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pptp.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sb1000.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/ 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slip.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sungem_phy.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 64585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tap.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/ 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 107089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-15T17:50:29.242 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_random.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 86945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/veth.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/ 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 144361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/ 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 301281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/vxlan.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/ 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 144001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ntb.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/ 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/ 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 265777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme-core.ko 2024-01-15T17:50:29.243 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 102545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/ 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 131249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_cs.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 85377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_pc.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_serial.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/ 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/ 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pci-stub.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/xen-pcifront.ko 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/ 2024-01-15T17:50:29.244 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/i82092.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 146289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pcmcia.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 62161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pcmcia_core.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pd6729.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 112889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/ 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/broadcom/ 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/samsung/ 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/ 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/ 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/ 2024-01-15T17:50:29.245 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/ 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/ 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acerhdf.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-15T17:50:29.246 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 132025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/hdaps.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/ibm_rtl.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/ 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel_ips.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-15T17:50:29.247 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/ 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/88pm860x_battery.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/88pm860x_charger.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 55057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/bq2415x_charger.ko 2024-01-15T17:50:29.248 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/bq24190_charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/bq24735-charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/da9030_battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/da9052-battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2760_battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2780_battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2781_battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2782_battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/gpio-charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/isp1704_charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/lp8727_charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max14577_charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max17042_battery.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max8903_charger.ko 2024-01-15T17:50:29.249 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max8925_power.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/pcf50633-charger.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/sbs-battery.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/test_power.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/tps65090-charger.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/wm831x_backup.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/wm831x_power.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/wm8350_power.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/ 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/ 2024-01-15T17:50:29.250 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/ 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps_parport.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/pps_core.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp_kvm.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/ 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-88pm80x.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-88pm860x.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-da9052.ko 2024-01-15T17:50:29.251 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-da9055.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-da9063.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 68577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-em3027.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-15T17:50:29.252 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-lp8788.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t86.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max6900.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8907.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8925.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8997.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8998.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-palmas.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-15T17:50:29.253 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf50633.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rc5t583.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-s35390a.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-tps6586x.ko 2024-01-15T17:50:29.254 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-tps65910.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-wm831x.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-wm8350.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-x1205.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 77617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-9xxx.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 57665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-sas.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 55105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-xxxx.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 135185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/BusLogic.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/a100u2w.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/ 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 271113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 100617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/advansys.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/ 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 356929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-15T17:50:29.255 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 296049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic94xx/ 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 189185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/am53c974.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/ 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 106881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/atp870u.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/ 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 247009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/ 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1171945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/bfa.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/ 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 237721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/ 2024-01-15T17:50:29.256 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 115369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ch.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/ 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 308921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/ 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 76145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 101449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 146625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 75553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dc395x.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dmx3191d.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/ 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 212425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esp_scsi.ko 2024-01-15T17:50:29.257 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/ 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 158505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fdomain.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/ 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 263489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/fnic.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 201817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hpsa.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hptiop.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hv_storvsc.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/imm.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/initio.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 266537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ipr.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 101473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ips.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/ 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 317153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/isci.ko 2024-01-15T17:50:29.258 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/ 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 315209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/libfc.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 145585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/ 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 216441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/libsas.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/ 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 2197481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/ 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 73569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 310841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-15T17:50:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 84649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/ 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 659801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/ 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 119561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvumi.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/ 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 81649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/ 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 366401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 124633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pmcraid.ko 2024-01-15T17:50:29.260 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ppa.ko 2024-01-15T17:50:29.261 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 55425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla1280.ko 2024-01-15T17:50:29.261 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/ 2024-01-15T17:50:29.261 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1876633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-15T17:50:29.261 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 85289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-15T17:50:29.261 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/ 2024-01-15T17:50:29.261 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 559033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qlogicfas408.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/raid_class.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 261241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_debug.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 173937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 279313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 97729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 73969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ses.ko 2024-01-15T17:50:29.349 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 115889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/st.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/stex.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/ 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 164961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/virtio_scsi.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/wd719x.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/xen-scsifront.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 129441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ssb.ko 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/ 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/ 2024-01-15T17:50:29.350 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 736769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/ 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_file.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_iblock.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 930697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_mod.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_pscsi.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 122521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_user.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/ 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 75593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/ 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/ 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/ 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-15T17:50:29.351 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-15T17:50:29.352 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 57865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/moxa.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 50953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/mxser.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 108601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_gsm.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_hdlc.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/nozomi.ko 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/ 2024-01-15T17:50:29.353 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/ 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/altera_uart.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/arc_uart.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 72425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/ 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 103073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/rp2.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/sc16is7xx.ko 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/ 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ 2024-01-15T17:50:29.354 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 535985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/ 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_aec.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_cif.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_dmem_genirq.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_mf624.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_netx.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pci_generic.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_sercos3.ko 2024-01-15T17:50:29.355 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/ 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/c67x00/ 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 73297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/c67x00/c67x00.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 144617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/ 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 87073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-acm.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-15T17:50:29.356 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usblp.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usbtmc.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/common/ 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/common/ulpi.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc2/ 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/ 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 454385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/dwc3.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/ 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/ 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/u_ether.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/u_serial.ko 2024-01-15T17:50:29.357 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 92489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 122081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 57937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 57729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 85609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-01-15T17:50:29.358 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/ 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-01-15T17:50:29.359 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 62361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 161225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/libcomposite.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/ 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/bdc/ 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 157681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 69705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 76057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/net2272.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 75073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/net2280.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 73169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-01-15T17:50:29.360 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 65809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 81505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 156409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/ 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/bcma-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 80113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/isp116x-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 87353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811_cs.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/ssb-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/ 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/mdc800.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/microtek.ko 2024-01-15T17:50:29.361 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/adutux.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/appledisplay.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/cytherm.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ehset.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi26.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi62.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ezusb.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/idmouse.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/iowarrior.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ldusb.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/legousbtower.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/lvstest.ko 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/ 2024-01-15T17:50:29.362 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 64841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usb3503.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usblcd.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbtest.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/uss720.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/yurex.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/mon/ 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 72953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/mon/usbmon.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/musb/ 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 301697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/musb/musb_hdrc.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/ 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-generic.ko 2024-01-15T17:50:29.363 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-isp1301.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-tahvo.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/aircable.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ark3116.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ch341.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cp210x.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cyberjack.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/empeg.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/f81232.ko 2024-01-15T17:50:29.364 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 175193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 137081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 124833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_ti.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipaq.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipw.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ir-usb.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 107521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mct_u232.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/metro-usb.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7720.ko 2024-01-15T17:50:29.365 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 78217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7840.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mxuport.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/navman.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/omninet.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/opticon.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 191801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/option.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/oti6858.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/pl2303.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcaux.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcserial.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/quatech2.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/safe_serial.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 51625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/sierra.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ssu100.ko 2024-01-15T17:50:29.366 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/symbolserial.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_debug.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 125609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usbserial.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/visor.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/whiteheat.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/wishbone-serial.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 52913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/uas.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-15T17:50:29.367 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-karma.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 180225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/usb-storage.ko 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/ 2024-01-15T17:50:29.368 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 67481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 71097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 103977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/ 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/ 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 154297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 103737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 77609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/ 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 113785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_net.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_scsi.ko 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/ 2024-01-15T17:50:29.369 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/ 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/aat2870_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/adp5520_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/adp8860_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/adp8870_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/apple_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/as3711_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/bd6107.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/da903x_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/da9052_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/gpio_backlight.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/kb3886_bl.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lcd.ko 2024-01-15T17:50:29.370 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lm3533_bl.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lm3639_bl.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lv5207lp.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/max8925_bl.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/pandora_bl.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/wm831x_bl.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/ 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2482.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2490.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/matrox_w1.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/w1-gpio.ko 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/ 2024-01-15T17:50:29.371 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 102673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/wire.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/acquirewdt.ko 2024-01-15T17:50:29.372 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/advantechwdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/cpu5wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/da9052_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/da9055_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/da9063_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/dw_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/eurotechwdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/hpwdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/i6300esb.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-15T17:50:29.373 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ib700wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ibmasr.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it87_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/kempld_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/machzwd.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/mena21_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/nv_tco.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pc87413_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/retu_wdt.ko 2024-01-15T17:50:29.374 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sc1200wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/softdog.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/twl4030_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/via_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wafer5823wdt.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wdt_pci.ko 2024-01-15T17:50:29.375 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wm831x_wdt.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wm8350_wdt.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/xen_wdt.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/ 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-evtchn.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntalloc.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntdev.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/ 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 176441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-privcmd.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 71249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-scsiback.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/ 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ 2024-01-15T17:50:29.376 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 166753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/9p.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/adfs/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/adfs/adfs.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 163449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/affs.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/afs/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 935033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/afs/kafs.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/autofs/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 103401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/autofs/autofs4.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 64681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/befs.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/bfs/ 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/bfs/bfs.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/binfmt_misc.ko 2024-01-15T17:50:29.377 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 3455857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/btrfs.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 306009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/cachefiles.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1335129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ceph.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 100801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/coda.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/configfs/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 108641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/configfs/configfs.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/cramfs.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/ 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 644073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/dlm.ko 2024-01-15T17:50:29.378 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/efs/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/efs/efs.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1506641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/f2fs.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/msdos.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/freevxfs/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/freevxfs/freevxfs.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 226481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/fscache.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/cuse.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1229913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/gfs2.ko 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/ 2024-01-15T17:50:29.379 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 141985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/hfs.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 223225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/hfsplus.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hpfs/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 193569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hpfs/hpfs.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 74785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/isofs.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 419881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/jfs.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 282049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/lockd.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/minix.ko 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/ 2024-01-15T17:50:29.380 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 124649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/netfs.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/ 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/ 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 81513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/ 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/ 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 117465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1057969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfs.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv2.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 120265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv3.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 2042153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv4.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/ 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/grace.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/nfs_acl.ko 2024-01-15T17:50:29.381 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/ 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1563929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/nfsd.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/ 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 590713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/nilfs2.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/ 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-celtic.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-centeuro.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-croatian.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-cyrillic.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-gaelic.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-greek.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-iceland.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-inuit.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-roman.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-romanian.ko 2024-01-15T17:50:29.382 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-turkish.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ascii.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1250.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1251.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1255.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp737.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp775.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp850.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp852.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp855.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp857.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp860.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp861.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp862.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp863.ko 2024-01-15T17:50:29.383 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp864.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp865.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp866.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp869.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp874.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 94137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp932.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 142673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp936.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 165745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp949.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 118121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp950.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_euc-jp.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-1.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-13.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-14.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-15.ko 2024-01-15T17:50:29.384 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-2.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-3.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-4.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-5.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-6.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-7.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-9.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-r.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-ru.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-u.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_utf8.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ntfs/ 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 220553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ntfs/ntfs.ko 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ 2024-01-15T17:50:29.385 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 244737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 440737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 2593521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/omfs/ 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/omfs/omfs.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/ 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 370697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/overlay.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ramoops.ko 2024-01-15T17:50:29.386 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx4/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx4/qnx4.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx6/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 46689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx6/qnx6.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/quota_tree.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/quota_v1.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/quota_v2.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 532025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/reiserfs.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/romfs.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/ 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 2683249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/cifs.ko 2024-01-15T17:50:29.387 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_arc4.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_md4.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 134777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/squashfs.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 91249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/sysv.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 248425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/udf.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 162537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ufs.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 4014433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/xfs.ko 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/ 2024-01-15T17:50:29.388 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/locking/ 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 80585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/locking/locktorture.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/time/ 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/time/test_udelay.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/torture.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 6833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crc-itu-t.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 6529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crc7.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crc8.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/ 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libarc4.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libdes.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/interval_tree_test.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/libcrc32c.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lru_cache.ko 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/ 2024-01-15T17:50:29.389 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4_compress.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4hc_compress.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/math/ 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 6617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/math/cordic.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/memory-notifier-error-inject.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/notifier-error-inject.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/percpu_test.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/pm-notifier-error-inject.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/ 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 143353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/raid6_pq.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/rbtree_test.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/ 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test-kstrtox.ko 2024-01-15T17:50:29.390 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test-string_helpers.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 4876633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_bpf.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_firmware.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 6369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_module.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_user_copy.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_bm.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_fsm.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_kmp.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/xz/ 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/xz/xz_dec_test.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/mm/ 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/mm/hwpoison-inject.ko 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/ 2024-01-15T17:50:29.391 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 55737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/6lowpan.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_dest.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_fragment.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_hop.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 6985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_mobility.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_routing.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_udp.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/ 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/garp.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/mrp.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/p8022.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/psnap.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/stp.ko 2024-01-15T17:50:29.392 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/ 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 82593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/8021q.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/ 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 198905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_fd.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_rdma.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_virtio.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/ 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 649977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/batman-adv.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/ 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 65137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/br_netfilter.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 785817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/bridge.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-15T17:50:29.393 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-15T17:50:29.394 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 56033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtables.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/ 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1046969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/libceph.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/core/ 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 126041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/core/pktgen.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/ 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 194529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp.ko 2024-01-15T17:50:29.395 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_diag.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 58841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv4.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 60873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv6.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/ 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 390129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/dsa_core.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_brcm.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_dsa.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_none.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_trailer.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/hsr/ 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 99881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/hsr/hsr.ko 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ 2024-01-15T17:50:29.396 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 45537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 255849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 47985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ah4.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/esp4.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/fou.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/gre.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/inet_diag.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_gre.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_tunnel.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_vti.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipcomp.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 29001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipip.ko 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ 2024-01-15T17:50:29.397 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 41929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ip_tables.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-15T17:50:29.398 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_bic.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_dctcp.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_diag.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_highspeed.ko 2024-01-15T17:50:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_htcp.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_hybla.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_illinois.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_lp.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_scalable.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_vegas.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_veno.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_westwood.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_yeah.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tunnel4.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_diag.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_tunnel.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ 2024-01-15T17:50:29.400 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ah6.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/esp6.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/fou6.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 80105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_gre.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 93977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_tunnel.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_vti.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ipcomp6.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/mip6.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-15T17:50:29.401 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-15T17:50:29.402 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 74921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/sit.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/tunnel6.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/key/ 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 84433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/key/af_key.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/ 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 125305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_core.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-15T17:50:29.403 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 25953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_eth.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip6.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_netlink.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 53321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ppp.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/lapb/ 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/lapb/lapb.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/llc/ 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 268265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc2.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/ 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 290441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/mac802154.ko 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/ 2024-01-15T17:50:29.404 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/mpls_gso.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 118801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 100297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 100633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 102881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 102649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 112017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 59585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 108761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 112297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 112649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-15T17:50:29.405 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 110753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 113633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 412561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-15T17:50:29.406 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conncount.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 411489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 118193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 108105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-15T17:50:29.407 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 91737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 37297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_log_syslog.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 96545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_irc.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_sip.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 44297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 614025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_tables.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 42609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink.ko 2024-01-15T17:50:29.408 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 39785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_log.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_chain_nat.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_compat.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_ct.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fib.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_hash.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_limit.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_log.ko 2024-01-15T17:50:29.409 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_masq.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_nat.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_numgen.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_osf.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_queue.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_redir.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject_inet.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_socket.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_synproxy.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_tproxy.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 89313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/x_tables.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_AUDIT.ko 2024-01-15T17:50:29.410 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 18169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CT.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_DSCP.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HL.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HMARK.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 38873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LED.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LOG.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NETMAP.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFLOG.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-15T17:50:29.411 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_RATEEST.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_SECMARK.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TEE.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TPROXY.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TRACE.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_addrtype.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_bpf.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cgroup.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cluster.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_comment.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connbytes.ko 2024-01-15T17:50:29.412 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlabel.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlimit.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connmark.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_conntrack.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cpu.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dccp.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_devgroup.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dscp.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ecn.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_esp.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hashlimit.ko 2024-01-15T17:50:29.413 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_helper.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hl.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipcomp.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_iprange.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipvs.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_l2tp.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_length.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_limit.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mac.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mark.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_multiport.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nat.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nfacct.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_osf.ko 2024-01-15T17:50:29.414 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_owner.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_physdev.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_pkttype.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_policy.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_quota.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_rateest.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_realm.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_recent.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_sctp.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_set.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_socket.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_state.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_statistic.ko 2024-01-15T17:50:29.415 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_string.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 8553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpmss.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpudp.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_time.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_u32.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/ 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 14537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/netlink_diag.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/ 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 12425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/nsh.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/ 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 383481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/openvswitch.ko 2024-01-15T17:50:29.416 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-geneve.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-gre.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-vxlan.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/packet/ 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/packet/af_packet_diag.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/phonet/ 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 112057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/phonet/phonet.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/phonet/pn_pep.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/ 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 252953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 164937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_rdma.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 68161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_tcp.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rxrpc/ 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 818777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rxrpc/rxrpc.ko 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/ 2024-01-15T17:50:29.417 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_csum.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_gact.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_ipt.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_mirred.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_nat.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 43449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_pedit.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_police.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_simple.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_skbedit.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_vlan.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_basic.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_bpf.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 19281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_cgroup.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_flow.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 32377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_fw.ko 2024-01-15T17:50:29.418 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 40009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_route.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 61529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_u32.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_cmp.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 13409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_ipset.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_meta.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_nbyte.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_text.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_u32.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 21769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_choke.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_codel.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 34681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_drr.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq_codel.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 35545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_gred.ko 2024-01-15T17:50:29.419 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 49937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hfsc.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 22089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hhf.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 66577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_htb.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_ingress.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 30385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio_lib.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_multiq.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 36249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_netem.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 27497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_pie.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_plug.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 24841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_prio.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 48625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_qfq.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 33017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_red.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 26881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfb.ko 2024-01-15T17:50:29.420 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 31409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfq.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_tbf.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 28153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_teql.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/ 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 932169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp_diag.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/ 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/ 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 336777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 54993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 1518417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/sunrpc.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/ 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 708281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/ 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 10489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/diag.ko 2024-01-15T17:50:29.421 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 866345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/tipc.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/unix/ 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 16033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/unix/unix_diag.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/ 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 86417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 68049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 98017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 11177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 17761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/x25/ 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 125409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/x25/x25.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/ 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_algo.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 20649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 104161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_user.ko 2024-01-15T17:50:29.422 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/virt/ 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/ 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 15537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/irqbypass.ko 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 641009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.alias 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 625033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.alias.bin 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 7677 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 23634 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin.alias.bin 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 9916 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin.bin 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 70653 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin.modinfo 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 147734 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.dep 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 234045 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.dep.bin 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 210 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.devname 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 72797 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.order 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 879 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.softdep 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 255175 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.symbols 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 309826 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.symbols.bin 2024-01-15T17:50:29.423 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/ 2024-01-15T17:50:29.424 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/share/ 2024-01-15T17:50:29.424 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/share/doc/ 2024-01-15T17:50:29.424 INFO:teuthology.orchestra.run.smithi204.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/share/doc/linux-image-6.7.0-g2668949e0971/ 2024-01-15T17:50:29.424 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 171 2024-01-15 16:53 ./usr/share/doc/linux-image-6.7.0-g2668949e0971/changelog.Debian.gz 2024-01-15T17:50:29.424 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r-- root/root 691 2024-01-15 16:53 ./usr/share/doc/linux-image-6.7.0-g2668949e0971/copyright 2024-01-15T17:50:29.425 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g2668949e0971 2024-01-15T17:50:29.425 INFO:teuthology.task.kernel:searching for kernel 6.7.0-g2668949e0971 2024-01-15T17:50:29.425 DEBUG:teuthology.orchestra.run.smithi204:> egrep '(submenu|menuentry.*6.7.0-g2668949e0971).*{' /boot/grub/grub.cfg 2024-01-15T17:50:29.432 INFO:teuthology.orchestra.run.smithi204.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-01-15T17:50:29.432 INFO:teuthology.orchestra.run.smithi204.stdout: menuentry 'Ubuntu, with Linux 6.7.0-g2668949e0971' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.0-g2668949e0971-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-01-15T17:50:29.432 INFO:teuthology.orchestra.run.smithi204.stdout: menuentry 'Ubuntu, with Linux 6.7.0-g2668949e0971 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.0-g2668949e0971-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-01-15T17:50:29.433 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-01-15T17:50:29.433 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.7.0-g2668949e0971 2024-01-15T17:50:29.433 DEBUG:teuthology.orchestra.run.smithi204:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-01-15T17:50:29.477 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-01-15T17:50:29.843 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub' 2024-01-15T17:50:29.844 INFO:teuthology.orchestra.run.smithi204.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-15T17:50:29.871 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-01-15T17:50:30.053 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:50:30.059 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-6.7.0-g2668949e0971 2024-01-15T17:50:30.276 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-15T17:50:30.281 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-15T17:50:30.323 INFO:teuthology.orchestra.run.smithi204.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-15T17:50:30.329 INFO:teuthology.orchestra.run.smithi204.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-15T17:50:30.491 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-15T17:50:30.491 INFO:teuthology.orchestra.run.smithi204.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-01-15T17:50:30.491 INFO:teuthology.orchestra.run.smithi204.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-15T17:50:30.503 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-01-15T17:50:44.343 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub' 2024-01-15T17:50:44.357 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-15T17:50:44.382 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-01-15T17:50:44.667 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:50:44.687 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-6.7.0-g2668949e0971 2024-01-15T17:50:44.992 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-15T17:50:45.008 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-15T17:50:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-15T17:50:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-15T17:50:45.240 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-15T17:50:45.240 INFO:teuthology.orchestra.run.smithi143.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-01-15T17:50:45.241 INFO:teuthology.orchestra.run.smithi143.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-15T17:50:45.279 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-01-15T17:50:45.464 DEBUG:teuthology.orchestra.run.smithi143:> dpkg-deb -c /tmp/linux-image.deb 2024-01-15T17:50:45.472 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./ 2024-01-15T17:50:45.472 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./boot/ 2024-01-15T17:50:45.472 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 4444641 2024-01-15 16:53 ./boot/System.map-6.7.0-g2668949e0971 2024-01-15T17:50:45.482 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 170149 2024-01-15 16:53 ./boot/config-6.7.0-g2668949e0971 2024-01-15T17:50:45.482 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11441152 2024-01-15 16:53 ./boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/postinst.d/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/postrm.d/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/preinst.d/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./etc/kernel/prerm.d/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/ 2024-01-15T17:50:45.500 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/ 2024-01-15T17:50:45.501 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ 2024-01-15T17:50:45.501 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 490345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/aesni-intel.ko 2024-01-15T17:50:45.503 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-15T17:50:45.503 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 152241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-15T17:50:45.504 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/ 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/ 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/ 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpuid.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/msr.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/ 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 353705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 881649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-15T17:50:45.505 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2276625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm.ko 2024-01-15T17:50:45.579 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/ 2024-01-15T17:50:45.579 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/af_alg.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/algif_hash.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/algif_skcipher.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/ansi_cprng.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/anubis.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/arc4.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/asymmetric_keys/ 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/ 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_memcpy.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_pq.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_tx.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_xor.ko 2024-01-15T17:50:45.580 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/raid6test.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/authenc.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/authencesn.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/blake2b_generic.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_common.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_generic.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/camellia_generic.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast5_generic.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast6_generic.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast_common.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/ccm.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cmac.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crc32_generic.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cryptd.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_engine.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_simd.ko 2024-01-15T17:50:45.581 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_user.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/cts.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/des_generic.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/dh_generic.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/echainiv.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/essiv.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/fcrypt.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/khazad.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/lrw.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4hc.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/md4.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/michael_mic.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcbc.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcrypt.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/rmd160.ko 2024-01-15T17:50:45.582 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/seed.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/serpent_generic.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 149481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/tcrypt.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/tea.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_common.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_generic.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/vmac.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/wp512.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xcbc.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xor.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xts.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/crypto/xxhash_generic.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_extlog.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-15T17:50:45.583 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_pad.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/apei/ 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/apei/einj.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ec_sys.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/platform_profile.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbs.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbshc.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 104729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/video.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/acard-ahci.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci_platform.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 88401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci_platform.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_acpi.ko 2024-01-15T17:50:45.584 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ali.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_amd.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_artop.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atiixp.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atp867x.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd640.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd64x.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cypress.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_efar.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt366.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt37x.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it8213.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it821x.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_jmicron.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_legacy.ko 2024-01-15T17:50:45.585 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_marvell.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_mpiix.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_netcell.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ninja32.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87410.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87415.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_oldpiix.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_opti.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_optidma.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pcmcia.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_piccolo.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_radisys.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rdc.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rz1000.ko 2024-01-15T17:50:45.586 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sch.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_serverworks.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sil680.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sl82c105.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_triflex.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_via.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pdc_adma.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_inic162x.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_mv.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_nv.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_promise.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_qstor.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil24.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sis.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_svw.ko 2024-01-15T17:50:45.587 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sx4.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_uli.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_via.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_vsc.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ks0108.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/ 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/bcma.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/ 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/ 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/aoe.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/ 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 965305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/drbd.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 173353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/floppy.ko 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/ 2024-01-15T17:50:45.588 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 94297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 116321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/nbd.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/ 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 110737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/null_blk.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 98281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/pktcdvd.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 263769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/rbd.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/ 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 98185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/ 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/zram.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/applicom.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hangcheck-timer.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/ 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-15T17:50:45.589 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/via-rng.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 148609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 139313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/lp.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/ 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 84097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/mwave.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/nvram.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ppdev.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tlclk.ko 2024-01-15T17:50:45.590 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/ 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/ 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/xillybus_class.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/xillybus_core.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/ 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-palmas.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-si5351.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-twl6040.ko 2024-01-15T17:50:45.591 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/clk/clk-wm831x.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 246841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 517033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-15T17:50:45.592 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-aes.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-sha.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/ 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/ 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/dca.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/ 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 79953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/amd64_edac.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/e752x_edac.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/edac_mce_amd.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3000_edac.ko 2024-01-15T17:50:45.593 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3200_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5100_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5400_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7300_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7core_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i82975x_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ie31200_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/sb_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/x38_edac.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/ 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-gpio.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-max14577.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-max77693.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-max8997.ko 2024-01-15T17:50:45.594 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-palmas.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-rt8973a.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/extcon/extcon-sm5502.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/ 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/dmi-sysfs.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/efi/ 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/efi/efi-pstore.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/ 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-adp5520.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-amd8111.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-arizona.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-da9052.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-da9055.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-dln2.ko 2024-01-15T17:50:45.595 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-f7188x.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-generic.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ich.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-kempld.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-lp3943.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-max7300.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-max730x.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-max732x.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-pca953x.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-pcf857x.ko 2024-01-15T17:50:45.596 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-rdc321x.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-sch.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-sch311x.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-tps65912.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-twl4030.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-twl6040.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-vx855.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-wm831x.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-wm8350.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-wm8994.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/ 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-a4tech.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-apple.ko 2024-01-15T17:50:45.597 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-appleir.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-aureal.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-axff.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-belkin.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-cherry.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-chicony.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-cp2112.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-cypress.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-dr.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elecom.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elo.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-emsff.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ezkey.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gaff.ko 2024-01-15T17:50:45.598 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-generic.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gt683r.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gyration.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtekff.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-hyperv.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-icade.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kensington.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-keytouch.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kye.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lcpower.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-led.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lenovo.ko 2024-01-15T17:50:45.599 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lg-g15.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-magicmouse.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-microsoft.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-monterey.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-multitouch.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ntrig.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ortek.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-penmount.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-petalynx.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-picolcd.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-pl.ko 2024-01-15T17:50:45.600 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-plantronics.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-primax.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-rmi.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-common.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-saitek.ko 2024-01-15T17:50:45.601 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-samsung.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sjoy.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sony.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-speedlink.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-steelseries.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sunplus.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tivo.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tmff.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-topseed.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-twinhan.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-uclogic.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-waltop.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 143473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-wiimote.ko 2024-01-15T17:50:45.602 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-xinmo.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zpff.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zydacron.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 269281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/uhid.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/ 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/usbhid.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/usbkbd.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/usbhid/usbmouse.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 217873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/wacom.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/ 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_balloon.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_utils.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 305321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_vmbus.ko 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ 2024-01-15T17:50:45.603 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru3.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7414.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7418.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adc128d818.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1025.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1026.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1029.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1031.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm9240.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ads7828.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7410.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7411.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7462.ko 2024-01-15T17:50:45.604 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7470.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7475.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7x10.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/amc6821.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/applesmc.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asb100.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asc7621.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/atxp1.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/coretemp.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/da9052-hwmon.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/da9055-hwmon.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 99601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dme1737.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds1621.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds620.ko 2024-01-15T17:50:45.605 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc1403.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc2103.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc6w201.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71805f.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 94689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71882fg.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f75375s.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fam15h_power.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fschmd.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g760a.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g762.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl518sm.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl520sm.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hih6130.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5500_temp.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5k_amb.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmaem.ko 2024-01-15T17:50:45.606 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmpex.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina209.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina2xx.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 130449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/it87.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/jc42.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k10temp.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k8temp.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lineage-pem.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm63.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm73.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm75.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm77.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm78.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm80.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm83.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm85.ko 2024-01-15T17:50:45.607 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm87.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm90.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm92.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm93.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95234.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95241.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95245.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc2945.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4151.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4215.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4222.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4245.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4260.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4261.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max16065.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1619.ko 2024-01-15T17:50:45.608 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1668.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max197.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6639.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6650.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6697.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mc13783-adc.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mcp3021.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6683.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775-core.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct7802.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87360.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87427.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pcf8591.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-15T17:50:45.609 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/powr1220.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5627.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5636.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht15.ko 2024-01-15T17:50:45.610 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht21.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/shtc1.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sis5595.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47b397.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m1.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m192.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/thmc50.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp102.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp103.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp401.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp421.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via-cputemp.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via686a.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt1211.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt8231.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627ehf.ko 2024-01-15T17:50:45.611 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627hf.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83781d.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83791d.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83792d.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83793.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83795.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l785ts.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l786ng.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/ 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/ 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/ 2024-01-15T17:50:45.612 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-01-15T17:50:45.613 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-15T17:50:45.614 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-mux.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-smbus.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-stub.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/ 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 273609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-15T17:50:45.615 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 883681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_core.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 324329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 282601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/ 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/ 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 493217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/ 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 420561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/ 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 741449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 296881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/ocrdma/ 2024-01-15T17:50:45.616 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 153801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/usnic/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 168969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 263081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 111425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 128753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 129529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/ 2024-01-15T17:50:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/evbug.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/ff-memless.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/ 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/gameport.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/joydev.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/ 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/lkkbd.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/lm8323.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/lm8333.ko 2024-01-15T17:50:45.618 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/newtonkbd.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/qt1070.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/qt2160.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/stowaway.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/sunkbd.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/xtkbd.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/matrix-keymap.ko 2024-01-15T17:50:45.619 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/ 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/appletouch.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/bcm5974.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/cyapatp.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/gpio_mouse.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 335953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/psmouse.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/sermouse.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/ 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 109873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/ 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/altera_ps2.ko 2024-01-15T17:50:45.620 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/arc_ps2.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/ct82c710.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/parkbd.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/pcips2.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/ps2mult.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serio_raw.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serport.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/sparse-keymap.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/ 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-88pm860x.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-adp5520.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-bd2802.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-blinkm.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-da903x.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-da9052.ko 2024-01-15T17:50:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-gpio.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3530.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3533.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm355x.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3642.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp3944.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp8788.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-max8997.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-mc13783.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-menf21bmc.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-pca9532.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-pca955x.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-pca963x.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-ss4200.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-tca6507.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-wm831x-status.ko 2024-01-15T17:50:45.622 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-wm8350.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/ 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/ 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 647193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/bcache.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bio-prison.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bufio.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache-smq.ko 2024-01-15T17:50:45.623 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 155897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 107257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-crypt.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-delay.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-era.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-flakey.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log-userspace.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-mirror.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-multipath.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-queue-length.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-raid.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-region-hash.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-round-robin.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-service-time.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 109529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-snapshot.ko 2024-01-15T17:50:45.624 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-switch.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 181865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-thin-pool.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-verity.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-zero.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/faulty.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/linear.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/multipath.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/ 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 199937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid0.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid1.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 158657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid10.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 393777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid456.ko 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/ 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/ 2024-01-15T17:50:45.625 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 178137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptbase.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptctl.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptfc.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptlan.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptsas.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptscsih.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptspi.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/ 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/88pm800.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/88pm805.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/88pm80x.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/arizona-i2c.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 123721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/arizona.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/bcm590xx.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/cros_ec_dev.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/dln2.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/janz-cmodio.ko 2024-01-15T17:50:45.626 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/kempld-core.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lm3533-core.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lp3943.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_ich.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_sch.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/max8907.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mc13xxx-core.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/menf21bmc.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/pcf50633-adc.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/pcf50633-gpio.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/pcf50633.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/retu-mfd.ko 2024-01-15T17:50:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/si476x-core.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/sm501.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/ti_am335x_tscadc.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/tps65010.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/tps6507x.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/viperboard.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/vx855.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wl1273-core.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wm8994.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ad525x_dpot.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/ 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds9802als.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds990x.ko 2024-01-15T17:50:45.628 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/bh1770glc.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/c2port/ 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/c2port/core.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/ 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/cb710.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ds1682.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/dummy-irq.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/echo/ 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/echo/echo.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/ 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/at24.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/max6875.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/enclosure.ko 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/genwqe/ 2024-01-15T17:50:45.629 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 152201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hmc6352.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hpilo.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ibmasm/ 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ics932s401.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29003.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29020.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/ 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/ 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-me.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-txe.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 313641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/phantom.ko 2024-01-15T17:50:45.630 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ti-st/ 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/ti-st/st_drv.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_7xx1.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_core.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tsl2550.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_balloon.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/ 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 183609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/ 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 490849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/bonding.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/ 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/ 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-15T17:50:45.631 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/ 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 455065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dummy.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/eql.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/ 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 130553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 99049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/ 2024-01-15T17:50:45.632 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/ 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/ 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/ 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/ 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/ 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/ 2024-01-15T17:50:45.633 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 90345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/ 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/ 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 176761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1421529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 128569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/ 2024-01-15T17:50:45.635 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 131337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 314553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 323745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 193497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 347257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1044793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 166193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-01-15T17:50:45.636 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/ 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/ 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 204345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/ 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/ 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 141225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/ 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dnet.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ec_bhf.ko 2024-01-15T17:50:45.637 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 281137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ethoc.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fealnx.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fujitsu/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 90369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 295481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 584497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 246209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/ 2024-01-15T17:50:45.638 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 937673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/ 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 287305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/ 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 514569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/ 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 734353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-01-15T17:50:45.639 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 151769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-15T17:50:45.703 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 86921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/jme.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/ 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/ 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 727473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 278961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1118201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-01-15T17:50:45.704 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 95665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/ 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/ 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 150905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/ 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 136721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/ 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-15T17:50:45.705 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 227897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 671937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 192353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/samsung/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 104241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/ 2024-01-15T17:50:45.706 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1064737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/ 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/ 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 63337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/ 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/ 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-01-15T17:50:45.707 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 487593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/ 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 166097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/ 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/ 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 86529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/ 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 83793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/ 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-15T17:50:45.708 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/ 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/ 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/defxx.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/skfp/ 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 216985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/fddi/skfp/skfp.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/geneve.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/ 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 236225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/ 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ifb.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvlan.ko 2024-01-15T17:50:45.709 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvtap.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/ 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-gpio.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mii.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/netconsole.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/nlmon.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/ 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/amd.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-15T17:50:45.710 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm87xx.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/broadcom.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/cicada.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/davicom.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/et1011c.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/icplus.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/lxt.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/marvell.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 76409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/micrel.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/national.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 95561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/phylink.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/qsemi.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/realtek.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/smsc.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ste10Xp.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/vitesse.ko 2024-01-15T17:50:45.711 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/plip/ 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/plip/plip.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_async.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppoe.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppox.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pptp.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sb1000.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/ 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slip.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sungem_phy.ko 2024-01-15T17:50:45.712 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tap.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/ 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 107089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_random.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 86945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/veth.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/ 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 144361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/ 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 301281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/vxlan.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/ 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 144001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ 2024-01-15T17:50:45.713 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ntb.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/ 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/ 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 265777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme-core.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/ 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 131249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_cs.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_pc.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_serial.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/ 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/ 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pci-stub.ko 2024-01-15T17:50:45.714 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/xen-pcifront.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/ 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/i82092.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 146289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pcmcia.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pcmcia_core.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pd6729.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/ 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/broadcom/ 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/samsung/ 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/ 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/ 2024-01-15T17:50:45.715 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/ 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/ 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/ 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acerhdf.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-15T17:50:45.716 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 132025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/hdaps.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/ibm_rtl.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/ 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel_ips.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-15T17:50:45.717 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/ 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/88pm860x_battery.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/88pm860x_charger.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/bq2415x_charger.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/bq24190_charger.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/bq24735-charger.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/da9030_battery.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/da9052-battery.ko 2024-01-15T17:50:45.718 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2760_battery.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2780_battery.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2781_battery.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/ds2782_battery.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/gpio-charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/isp1704_charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/lp8727_charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max14577_charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max17042_battery.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max8903_charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/max8925_power.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/pcf50633-charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/sbs-battery.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/test_power.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/tps65090-charger.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/wm831x_backup.ko 2024-01-15T17:50:45.719 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/wm831x_power.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/power/supply/wm8350_power.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/ 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/ 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/ 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps_parport.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/pps_core.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp_kvm.ko 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/ 2024-01-15T17:50:45.720 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-88pm80x.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-88pm860x.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-da9052.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-da9055.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-da9063.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-15T17:50:45.721 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-em3027.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-lp8788.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t86.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max6900.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8907.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8925.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8997.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max8998.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-palmas.ko 2024-01-15T17:50:45.722 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf50633.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rc5t583.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-s35390a.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-tps6586x.ko 2024-01-15T17:50:45.723 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-tps65910.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-wm831x.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-wm8350.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-x1205.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-9xxx.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-sas.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-xxxx.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 135185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/BusLogic.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/a100u2w.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/ 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 271113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/advansys.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/ 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 356929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 296049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-15T17:50:45.724 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic94xx/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 189185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/am53c974.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 106881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/atp870u.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 247009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1171945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/bfa.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 237721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 115369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ch.ko 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/ 2024-01-15T17:50:45.725 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 308921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/ 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 76145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 146625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dc395x.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dmx3191d.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/ 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 212425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esp_scsi.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/ 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 158505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fdomain.ko 2024-01-15T17:50:45.726 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/ 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 263489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/fnic.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 201817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hpsa.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hptiop.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hv_storvsc.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/imm.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/initio.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 266537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ipr.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 101473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ips.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/ 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 317153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/isci.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/ 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 315209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/libfc.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 145585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi.ko 2024-01-15T17:50:45.727 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/ 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 216441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/libsas.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/ 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2197481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/ 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 310841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 84649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/ 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 659801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/ 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 119561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-15T17:50:45.728 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvumi.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/ 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/ 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 366401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pmcraid.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ppa.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla1280.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/ 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1876633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/ 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 559033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-15T17:50:45.729 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qlogicfas408.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/raid_class.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 261241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_debug.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 173937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 279313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 97729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ses.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 115889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/st.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/stex.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/ 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 164961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/virtio_scsi.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-15T17:50:45.730 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/wd719x.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/xen-scsifront.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 129441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ssb.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/ 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/ 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 736769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/ 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_file.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_iblock.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 930697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_mod.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_pscsi.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_user.ko 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/ 2024-01-15T17:50:45.731 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/ 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/ 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/ 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-15T17:50:45.732 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/moxa.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 50953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/mxser.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_gsm.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_hdlc.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/nozomi.ko 2024-01-15T17:50:45.733 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/ 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/ 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/altera_uart.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/arc_uart.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/ 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/rp2.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/sc16is7xx.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/ 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 535985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ 2024-01-15T17:50:45.734 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/ 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_aec.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_cif.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_dmem_genirq.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_mf624.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_netx.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pci_generic.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_sercos3.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/ 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/c67x00/ 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/c67x00/c67x00.ko 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ 2024-01-15T17:50:45.735 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 144617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/ 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-acm.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usblp.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usbtmc.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/common/ 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/common/ulpi.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc2/ 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/ 2024-01-15T17:50:45.736 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 454385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/dwc3/dwc3.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/ 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/ 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/u_ether.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/u_serial.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 92489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 122081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-01-15T17:50:45.737 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 57729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 85609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/ 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-01-15T17:50:45.738 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 62361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 161225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/libcomposite.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/ 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/bdc/ 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 157681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 69705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 76057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/net2272.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 75073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/net2280.ko 2024-01-15T17:50:45.739 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 73169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 65809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 156409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/ 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/bcma-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/isp116x-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 87353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811_cs.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/ssb-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/ 2024-01-15T17:50:45.740 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/mdc800.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/microtek.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/adutux.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/appledisplay.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/cytherm.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ehset.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi26.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi62.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ezusb.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/idmouse.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/iowarrior.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ldusb.ko 2024-01-15T17:50:45.741 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/legousbtower.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/lvstest.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/ 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usb3503.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usblcd.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbtest.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/uss720.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/yurex.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/mon/ 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/mon/usbmon.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/musb/ 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 301697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/musb/musb_hdrc.ko 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/ 2024-01-15T17:50:45.742 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-generic.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-isp1301.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-tahvo.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/aircable.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ark3116.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ch341.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cp210x.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cyberjack.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/empeg.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/f81232.ko 2024-01-15T17:50:45.743 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 175193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 137081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_ti.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipaq.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipw.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ir-usb.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 107521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mct_u232.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/metro-usb.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7720.ko 2024-01-15T17:50:45.744 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 78217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7840.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mxuport.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/navman.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/omninet.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/opticon.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 191801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/option.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/oti6858.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/pl2303.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcaux.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcserial.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/quatech2.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/safe_serial.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 51625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/sierra.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ssu100.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/symbolserial.ko 2024-01-15T17:50:45.745 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_debug.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33201 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usbserial.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/visor.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/whiteheat.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/wishbone-serial.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 52913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/uas.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-15T17:50:45.746 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-karma.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 180225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/usb-storage.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/ 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 67481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/ 2024-01-15T17:50:45.747 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/ 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 154297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 77609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/ 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_net.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_scsi.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/ 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/ 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/aat2870_bl.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/adp5520_bl.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/adp8860_bl.ko 2024-01-15T17:50:45.748 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/adp8870_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/apple_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/as3711_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/bd6107.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/da903x_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/da9052_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/gpio_backlight.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/kb3886_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lcd.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lm3533_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lm3639_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lv5207lp.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/max8925_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/pandora_bl.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-15T17:50:45.749 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/wm831x_bl.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/ 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2482.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2490.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/matrox_w1.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/w1-gpio.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/ 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-15T17:50:45.750 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/wire.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/acquirewdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/advantechwdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/cpu5wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/da9052_wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/da9055_wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/da9063_wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/dw_wdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/eurotechwdt.ko 2024-01-15T17:50:45.751 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/hpwdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/i6300esb.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ib700wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ibmasr.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it87_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/kempld_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/machzwd.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/mena21_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-01-15T17:50:45.752 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/nv_tco.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pc87413_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/retu_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sc1200wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/softdog.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/twl4030_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/via_wdt.ko 2024-01-15T17:50:45.753 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wafer5823wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wdt_pci.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wm831x_wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wm8350_wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/xen_wdt.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/ 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-evtchn.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntalloc.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntdev.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/ 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 176441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-privcmd.ko 2024-01-15T17:50:45.754 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 71249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-scsiback.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 166753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/9p.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/adfs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/adfs/adfs.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 163449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/affs.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/afs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 935033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/afs/kafs.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/autofs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 103401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/autofs/autofs4.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/ 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 64681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/befs.ko 2024-01-15T17:50:45.755 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/bfs/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/bfs/bfs.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/binfmt_misc.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 3455857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/btrfs.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 306009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/cachefiles.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1335129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ceph.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/coda.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/configfs/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/configfs/configfs.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/ 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/cramfs.ko 2024-01-15T17:50:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/ 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 644073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/dlm.ko 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/efs/ 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/efs/efs.ko 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/ 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1506641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/f2fs.ko 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/ 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/msdos.ko 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/freevxfs/ 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/freevxfs/freevxfs.ko 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/ 2024-01-15T17:50:45.757 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 226481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/fscache.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/ 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/cuse.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/ 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1229913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/gfs2.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/ 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 141985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/hfs.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/ 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 223225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/hfsplus.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hpfs/ 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 193569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/hpfs/hpfs.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/ 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/isofs.ko 2024-01-15T17:50:45.758 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 419881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/jfs.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 282049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/lockd.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/minix.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 124649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/netfs.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 81513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/ 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 117465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1057969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfs.ko 2024-01-15T17:50:45.759 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv2.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 120265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv3.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2042153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv4.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/ 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/grace.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/nfs_acl.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/ 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1563929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/nfsd.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/ 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 590713 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/nilfs2.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/ 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-celtic.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-centeuro.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-croatian.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-cyrillic.ko 2024-01-15T17:50:45.760 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-gaelic.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-greek.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-iceland.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-inuit.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-roman.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-romanian.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13793 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-turkish.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ascii.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1250.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1251.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1255.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp737.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp775.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp850.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp852.ko 2024-01-15T17:50:45.761 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp855.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp857.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp860.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp861.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp862.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp863.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp864.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp865.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp866.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp869.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp874.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 94137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp932.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 142673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp936.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 165745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp949.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 118121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp950.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_euc-jp.ko 2024-01-15T17:50:45.762 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-1.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-13.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-14.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-15.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-2.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-3.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-4.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-5.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-6.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-7.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-9.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-r.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-ru.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-u.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_utf8.ko 2024-01-15T17:50:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ntfs/ 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 220553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ntfs/ntfs.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 244737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 440737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2593521 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/omfs/ 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/omfs/omfs.ko 2024-01-15T17:50:45.764 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 370697 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/overlay.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ramoops.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx4/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx4/qnx4.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx6/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 46689 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/qnx6/qnx6.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/quota_tree.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/quota_v1.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/quota/quota_v2.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 532025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/reiserfs.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/ 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/romfs.ko 2024-01-15T17:50:45.765 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/ 2024-01-15T17:50:45.766 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/ 2024-01-15T17:50:45.766 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 2683249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/cifs.ko 2024-01-15T17:50:45.814 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/ 2024-01-15T17:50:45.814 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_arc4.ko 2024-01-15T17:50:45.814 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_md4.ko 2024-01-15T17:50:45.814 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/ 2024-01-15T17:50:45.814 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 134777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/squashfs.ko 2024-01-15T17:50:45.814 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 91249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/sysv.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 248425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/udf.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 162537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ufs.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 4014433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/xfs.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/locking/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/locking/locktorture.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/time/ 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/time/test_udelay.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/kernel/torture.ko 2024-01-15T17:50:45.815 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6833 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crc-itu-t.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crc7.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crc8.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/ 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libarc4.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libdes.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/interval_tree_test.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/libcrc32c.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lru_cache.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/ 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4_compress.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4hc_compress.ko 2024-01-15T17:50:45.816 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/math/ 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/math/cordic.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/memory-notifier-error-inject.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/notifier-error-inject.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/percpu_test.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/pm-notifier-error-inject.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/ 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 143353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/raid6_pq.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/rbtree_test.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/ 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test-kstrtox.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test-string_helpers.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 4876633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_bpf.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_firmware.ko 2024-01-15T17:50:45.817 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_module.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/test_user_copy.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_bm.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_fsm.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_kmp.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/xz/ 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/lib/xz/xz_dec_test.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/mm/ 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/mm/hwpoison-inject.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/ 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 55737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/6lowpan.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_dest.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_fragment.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_hop.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 6985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-15T17:50:45.818 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_mobility.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_routing.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_udp.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/ 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/garp.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/mrp.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/p8022.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/psnap.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/802/stp.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/ 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 82593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/8021q.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/ 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 198905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_fd.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_rdma.ko 2024-01-15T17:50:45.819 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_virtio.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/ 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 649977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/batman-adv.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/ 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 65137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/br_netfilter.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 785817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/bridge.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9145 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-15T17:50:45.820 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13217 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9593 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11641 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 56033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtables.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-15T17:50:45.821 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/ 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1046969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/libceph.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/core/ 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 126041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/core/pktgen.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/ 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 194529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_diag.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 58841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv4.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 60873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv6.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/ 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 390129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/dsa_core.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_brcm.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_dsa.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7721 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_none.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_trailer.ko 2024-01-15T17:50:45.822 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/hsr/ 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 99881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/hsr/hsr.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 45537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 255849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 47985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ah4.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/esp4.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/fou.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/gre.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/inet_diag.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_gre.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_tunnel.ko 2024-01-15T17:50:45.823 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_vti.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipcomp.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 29001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipip.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 41929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ip_tables.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-01-15T17:50:45.824 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_bic.ko 2024-01-15T17:50:45.825 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_dctcp.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_diag.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_highspeed.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_htcp.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_hybla.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_illinois.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_lp.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_scalable.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_vegas.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_veno.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_westwood.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_yeah.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tunnel4.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_diag.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_tunnel.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11361 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-15T17:50:45.826 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ah6.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39705 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/esp6.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/fou6.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 80105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_gre.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 93977 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_tunnel.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_vti.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ipcomp6.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/mip6.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14137 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14273 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-15T17:50:45.827 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-15T17:50:45.828 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25225 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 74921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/sit.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/tunnel6.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/key/ 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 84433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/key/af_key.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/ 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125305 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_core.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 25953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_eth.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip.ko 2024-01-15T17:50:45.829 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip6.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_netlink.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 53321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ppp.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/lapb/ 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48121 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/lapb/lapb.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/llc/ 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 268265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc2.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/ 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 290441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/mac802154.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/ 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9393 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/mpls_gso.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 118801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-15T17:50:45.830 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 100633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 102649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 59585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 110753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 113633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 412561 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-15T17:50:45.831 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49265 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conncount.ko 2024-01-15T17:50:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 411489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 118193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 108105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11233 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10113 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 91737 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-15T17:50:45.833 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 37297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_log_syslog.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 96545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10129 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_irc.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_sip.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 44297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 614025 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_tables.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 42609 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 39785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_log.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_chain_nat.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_compat.ko 2024-01-15T17:50:45.834 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_ct.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9601 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11513 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fib.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_hash.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_limit.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_log.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_masq.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_nat.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13673 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_numgen.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_osf.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_queue.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_redir.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10081 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject_inet.ko 2024-01-15T17:50:45.835 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_socket.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_synproxy.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_tproxy.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 89313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/x_tables.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_AUDIT.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 18169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CT.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_DSCP.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HL.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13369 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HMARK.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 38873 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-15T17:50:45.836 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13889 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LED.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LOG.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11753 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NETMAP.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFLOG.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_RATEEST.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11073 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14337 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_SECMARK.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10433 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TEE.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TPROXY.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8321 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TRACE.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13193 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_addrtype.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_bpf.ko 2024-01-15T17:50:45.837 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cgroup.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cluster.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_comment.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connbytes.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlabel.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlimit.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connmark.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_conntrack.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cpu.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10969 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dccp.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_devgroup.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8801 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dscp.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10657 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ecn.ko 2024-01-15T17:50:45.839 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_esp.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hashlimit.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_helper.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7865 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hl.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11921 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipcomp.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_iprange.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11785 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipvs.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_l2tp.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8001 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_length.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10209 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_limit.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mac.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mark.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11825 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_multiport.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nat.ko 2024-01-15T17:50:45.840 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10105 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nfacct.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_osf.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_owner.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9729 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_physdev.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7633 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_pkttype.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11985 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_policy.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_quota.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_rateest.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8257 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_realm.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36457 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_recent.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13329 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_sctp.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_set.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16913 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_socket.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9505 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_state.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9473 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_statistic.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8809 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_string.ko 2024-01-15T17:50:45.841 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 8553 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpmss.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpudp.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11065 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_time.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_u32.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/ 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 14537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/netlink_diag.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/ 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 12425 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/nsh.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/ 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 383481 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/openvswitch.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-geneve.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9585 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-gre.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11929 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-vxlan.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/packet/ 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13569 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/packet/af_packet_diag.ko 2024-01-15T17:50:45.842 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/phonet/ 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 112057 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/phonet/phonet.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54857 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/phonet/pn_pep.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/ 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 252953 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 164937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_rdma.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_tcp.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rxrpc/ 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 818777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/rxrpc/rxrpc.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/ 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32241 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_csum.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_gact.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_ipt.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34097 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_mirred.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23401 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_nat.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 43449 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_pedit.ko 2024-01-15T17:50:45.843 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31961 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_police.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20289 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_simple.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35297 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_skbedit.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36665 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_vlan.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26465 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_basic.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34353 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_bpf.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 19281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_cgroup.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_flow.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 32377 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_fw.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 40009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_route.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 61529 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_u32.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7441 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_cmp.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 13409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_ipset.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33313 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_meta.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7897 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_nbyte.ko 2024-01-15T17:50:45.844 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_text.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_u32.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 21769 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_choke.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20849 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_codel.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 34681 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_drr.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35817 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31617 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq_codel.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 35545 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_gred.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 49937 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hfsc.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 22089 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hhf.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 66577 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_htb.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_ingress.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 30385 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10905 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio_lib.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26185 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_multiq.ko 2024-01-15T17:50:45.845 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 36249 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_netem.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 27497 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_pie.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10745 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_plug.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 24841 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_prio.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 48625 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_qfq.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 33017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_red.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 26881 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfb.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 31409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfq.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28945 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_tbf.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 28153 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_teql.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/ 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 932169 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp_diag.ko 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/ 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/ 2024-01-15T17:50:45.846 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 336777 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 54993 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 1518417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/sunrpc.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/ 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 708281 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/ 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 10489 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/diag.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 866345 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/tipc.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/unix/ 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 16033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/unix/unix_diag.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/ 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 86417 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 68049 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 98017 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 11177 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-15T17:50:45.847 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 17761 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/x25/ 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 125409 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/x25/x25.ko 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/ 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23041 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_algo.ko 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 20649 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 104161 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_user.ko 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/virt/ 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/ 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 15537 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/irqbypass.ko 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 641009 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.alias 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 625033 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.alias.bin 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 7677 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin 2024-01-15T17:50:45.848 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 23634 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin.alias.bin 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 9916 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin.bin 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 70653 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.builtin.modinfo 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 147734 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.dep 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 234045 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.dep.bin 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 210 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.devname 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 72797 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.order 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 879 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.softdep 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 255175 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.symbols 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 309826 2024-01-15 16:53 ./lib/modules/6.7.0-g2668949e0971/modules.symbols.bin 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/ 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/share/ 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/share/doc/ 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:drwxr-xr-x root/root 0 2024-01-15 16:53 ./usr/share/doc/linux-image-6.7.0-g2668949e0971/ 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 171 2024-01-15 16:53 ./usr/share/doc/linux-image-6.7.0-g2668949e0971/changelog.Debian.gz 2024-01-15T17:50:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r-- root/root 691 2024-01-15 16:53 ./usr/share/doc/linux-image-6.7.0-g2668949e0971/copyright 2024-01-15T17:50:45.850 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g2668949e0971 2024-01-15T17:50:45.850 INFO:teuthology.task.kernel:searching for kernel 6.7.0-g2668949e0971 2024-01-15T17:50:45.850 DEBUG:teuthology.orchestra.run.smithi143:> egrep '(submenu|menuentry.*6.7.0-g2668949e0971).*{' /boot/grub/grub.cfg 2024-01-15T17:50:45.862 INFO:teuthology.orchestra.run.smithi143.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-01-15T17:50:45.862 INFO:teuthology.orchestra.run.smithi143.stdout: menuentry 'Ubuntu, with Linux 6.7.0-g2668949e0971' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.0-g2668949e0971-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-01-15T17:50:45.862 INFO:teuthology.orchestra.run.smithi143.stdout: menuentry 'Ubuntu, with Linux 6.7.0-g2668949e0971 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.7.0-g2668949e0971-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-01-15T17:50:45.863 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-01-15T17:50:45.863 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.7.0-g2668949e0971 2024-01-15T17:50:45.863 DEBUG:teuthology.orchestra.run.smithi143:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-01-15T17:50:45.906 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi143.front.sepia.ceph.com to complete... 2024-01-15T17:50:46.269 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub' 2024-01-15T17:50:46.270 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-01-15T17:50:46.289 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2024-01-15T17:50:46.506 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:50:46.513 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-6.7.0-g2668949e0971 2024-01-15T17:50:46.744 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-01-15T17:50:46.749 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-01-15T17:50:46.790 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-01-15T17:50:46.794 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-01-15T17:50:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-01-15T17:50:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-01-15T17:50:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-01-15T17:50:46.958 INFO:teuthology.orchestra.run.smithi143.stderr:done 2024-01-15T17:51:01.632 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-01-15T17:51:01.632 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:51:01.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:18.095 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-15T17:51:18.095 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-15T17:51:18.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:20.124 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-15T17:51:29.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:51:29.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:32.188 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-15T17:51:36.508 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-01-15T17:51:44.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:51:44.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:45.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-15T17:51:45.511 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:48.572 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2024-01-15T17:52:00.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-15T17:52:00.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:52:02.620 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-15T17:52:17.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:52:17.629 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:52:17.633 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-01-15T17:52:32.277 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-01-15T17:52:32.996 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-15T17:52:32.996 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.7.0-g2668949e0971"... 2024-01-15T17:52:32.997 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2024-01-15T17:52:33.045 INFO:teuthology.orchestra.run.smithi143.stdout:6.7.0-g2668949e0971 2024-01-15T17:52:33.045 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g2668949e0971 vs 6.7.0-g2668949e0971 2024-01-15T17:52:33.045 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-15T17:52:33.045 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-01-15T17:52:34.047 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-15T17:52:34.047 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-15T17:52:34.138 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2024-01-15T17:52:34.192 DEBUG:teuthology.parallel:result is None 2024-01-15T17:52:35.633 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:52:35.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:52:36.037 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-01-15T17:52:36.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:52:36.766 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.7.0-g2668949e0971"... 2024-01-15T17:52:36.767 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-01-15T17:52:36.817 INFO:teuthology.orchestra.run.smithi204.stdout:6.7.0-g2668949e0971 2024-01-15T17:52:36.817 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g2668949e0971 vs 6.7.0-g2668949e0971 2024-01-15T17:52:36.817 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-15T17:52:36.817 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-01-15T17:52:37.818 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-01-15T17:52:37.818 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-15T17:52:37.898 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-01-15T17:52:37.937 DEBUG:teuthology.parallel:result is None 2024-01-15T17:52:37.937 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-15T17:52:37.950 INFO:teuthology.task.internal:Creating test directory... 2024-01-15T17:52:37.950 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-15T17:52:37.954 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-15T17:52:37.963 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-15T17:52:37.972 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-15T17:52:37.981 INFO:teuthology.task.internal:Creating archive directory... 2024-01-15T17:52:37.981 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-15T17:52:38.004 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-15T17:52:38.034 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-15T17:52:38.042 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-15T17:52:38.042 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-15T17:52:38.076 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-15T17:52:38.108 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:52:38.111 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:52:38.121 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:52:38.125 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:52:38.128 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-15T17:52:38.148 INFO:teuthology.task.internal:Configuring sudo... 2024-01-15T17:52:38.148 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-15T17:52:38.176 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-15T17:52:38.197 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-15T17:52:38.207 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-15T17:52:38.207 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-15T17:52:38.240 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-15T17:52:38.254 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-15T17:52:38.298 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-15T17:52:38.350 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T17:52:38.350 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-15T17:52:38.425 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-15T17:52:38.434 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-15T17:52:38.486 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T17:52:38.486 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-15T17:52:38.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-01-15T17:52:38.561 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-01-15T17:52:38.684 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-15T17:52:38.691 INFO:teuthology.task.internal:Starting timer... 2024-01-15T17:52:38.692 INFO:teuthology.run_tasks:Running task pcp... 2024-01-15T17:52:38.703 INFO:teuthology.run_tasks:Running task selinux... 2024-01-15T17:52:38.712 DEBUG:teuthology.task.selinux:Excluding smithi143: OS 'ubuntu' does not support SELinux 2024-01-15T17:52:38.713 DEBUG:teuthology.task.selinux:Excluding smithi204: OS 'ubuntu' does not support SELinux 2024-01-15T17:52:38.713 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-15T17:52:38.713 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-01-15T17:52:38.713 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-15T17:52:38.713 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-15T17:52:38.723 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-01-15T17:52:38.730 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-01-15T17:52:38.888 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-15T17:52:38.903 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-15T17:52:38.904 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-15T17:59:56.082 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-01-15T17:59:56.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-15T17:59:56.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:59:56.166 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-01-15T17:59:56.283 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-01-15T17:59:56.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:59:56.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:59:56.363 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-01-15T17:59:56.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-01-15T17:59:56.511 INFO:teuthology.run_tasks:Running task clock... 2024-01-15T17:59:56.522 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-15T17:59:56.522 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-15T17:59:56.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T17:59:56.526 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-15T17:59:56.526 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Command line: ntpd -gq 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: ---------------------------------------------------- 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: ntp-4 is maintained by Network Time Foundation, 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: corporation. Support and training for ntp-4 are 2024-01-15T17:59:56.561 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: available at https://www.nwtime.org/support 2024-01-15T17:59:56.562 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: ---------------------------------------------------- 2024-01-15T17:59:56.562 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: proto: precision = 0.042 usec (-24) 2024-01-15T17:59:56.562 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: basedate set to 2022-02-04 2024-01-15T17:59:56.562 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: gps base set to 2022-02-06 (week 2196) 2024-01-15T17:59:56.563 INFO:teuthology.orchestra.run.smithi143.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-15T17:59:56.563 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-15T17:59:56.563 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: restrict ::: KOD does nothing without LIMITED. 2024-01-15T17:59:56.563 INFO:teuthology.orchestra.run.smithi143.stderr:restrict ::: KOD does nothing without LIMITED. 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listen and drop on 0 v6wildcard [::]:123 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listen normally on 2 lo 127.0.0.1:123 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listen normally on 3 enp3s0f1 172.21.15.143:123 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listen normally on 4 lo [::1]:123 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7783%5]:123 2024-01-15T17:59:56.565 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 17:59:56 ntpd[16830]: Listening on routing socket on fd #22 for interface updates 2024-01-15T17:59:56.589 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Command line: ntpd -gq 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: ---------------------------------------------------- 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: ntp-4 is maintained by Network Time Foundation, 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: corporation. Support and training for ntp-4 are 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: available at https://www.nwtime.org/support 2024-01-15T17:59:56.590 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: ---------------------------------------------------- 2024-01-15T17:59:56.591 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: proto: precision = 0.042 usec (-24) 2024-01-15T17:59:56.591 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: basedate set to 2022-02-04 2024-01-15T17:59:56.591 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: gps base set to 2022-02-06 (week 2196) 2024-01-15T17:59:56.591 INFO:teuthology.orchestra.run.smithi204.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-15T17:59:56.591 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-01-15T17:59:56.591 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: restrict ::: KOD does nothing without LIMITED. 2024-01-15T17:59:56.592 INFO:teuthology.orchestra.run.smithi204.stderr:restrict ::: KOD does nothing without LIMITED. 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listen and drop on 0 v6wildcard [::]:123 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listen normally on 2 lo 127.0.0.1:123 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listen normally on 3 enp3s0f1 172.21.15.204:123 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listen normally on 4 lo [::1]:123 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77ab%5]:123 2024-01-15T17:59:56.593 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 17:59:56 ntpd[16873]: Listening on routing socket on fd #22 for interface updates 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:57.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:57 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:57.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:57.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:57.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:57.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:57.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:57 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:59.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 17:59:59 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:59.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T17:59:59.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T17:59:59.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T17:59:59.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T17:59:59.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 17:59:59 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:00.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:00.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:00.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:00.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:00.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:00.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:00.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:00.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:00 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:00.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:00.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:00.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:00.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:00.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:00.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:00.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:00.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:00 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:01.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:01 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:01.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:01.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:01 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:02.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:02 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:02.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:02 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:02.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:02 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:02.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:02 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:02.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:02 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:02.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:02 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:02.590 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:02 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:02.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:02 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:02.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:02 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:02.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:02 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:02.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:02 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:02.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:02 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:03.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:03.562 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stdout:15 Jan 18:00:03 ntpd[16830]: ntpd: time slew -0.000495 s 2024-01-15T18:00:03.563 INFO:teuthology.orchestra.run.smithi143.stdout:ntpd: time slew -0.000495s 2024-01-15T18:00:03.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-01-15T18:00:03.564 INFO:teuthology.orchestra.run.smithi143.stderr:15 Jan 18:00:03 ntpd[16830]: can't open /var/log/ntpstats/loopstats.20240115: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: can't open /var/log/ntpstats/rawstats.20240115: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: can't open /var/log/ntpstats/peerstats.20240115: Permission denied 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stdout:15 Jan 18:00:03 ntpd[16873]: ntpd: time slew -0.000698 s 2024-01-15T18:00:03.591 INFO:teuthology.orchestra.run.smithi204.stdout:ntpd: time slew -0.000698s 2024-01-15T18:00:03.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-01-15T18:00:03.592 INFO:teuthology.orchestra.run.smithi204.stderr:15 Jan 18:00:03 ntpd[16873]: can't open /var/log/ntpstats/loopstats.20240115: Permission denied 2024-01-15T18:00:03.675 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2024-01-15T18:00:03.675 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2024-01-15T18:00:03.675 INFO:teuthology.orchestra.run.smithi143.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.675 INFO:teuthology.orchestra.run.smithi143.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.675 INFO:teuthology.orchestra.run.smithi143.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.676 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.696 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2024-01-15T18:00:03.696 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2024-01-15T18:00:03.697 INFO:teuthology.orchestra.run.smithi204.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.697 INFO:teuthology.orchestra.run.smithi204.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.697 INFO:teuthology.orchestra.run.smithi204.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.697 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-01-15T18:00:03.698 INFO:teuthology.run_tasks:Running task install... 2024-01-15T18:00:03.707 DEBUG:teuthology.task.install:project ceph 2024-01-15T18:00:03.707 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}} 2024-01-15T18:00:03.707 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'} 2024-01-15T18:00:03.707 INFO:teuthology.task.install:Using flavor: default 2024-01-15T18:00:03.712 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-15T18:00:03.712 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-15T18:00:03.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-key list | grep Ceph 2024-01-15T18:00:03.713 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-key list | grep Ceph 2024-01-15T18:00:03.824 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-01-15T18:00:03.841 INFO:teuthology.orchestra.run.smithi204.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-01-15T18:00:03.875 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-01-15T18:00:03.875 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph.com (release key) 2024-01-15T18:00:03.876 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-01-15T18:00:03.876 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-01-15T18:00:03.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T18:00:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-01-15T18:00:03.892 INFO:teuthology.orchestra.run.smithi204.stdout:uid [ unknown] Ceph.com (release key) 2024-01-15T18:00:03.893 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-01-15T18:00:03.893 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-01-15T18:00:03.893 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T18:00:04.032 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default/ 2024-01-15T18:00:04.032 INFO:teuthology.task.install.deb:Package version is 18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:00:04.045 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default/ 2024-01-15T18:00:04.045 INFO:teuthology.task.install.deb:Package version is 18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:00:04.318 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:00:04.318 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-01-15T18:00:04.337 DEBUG:teuthology.orchestra.run.smithi204:> sudo apt-get update 2024-01-15T18:00:04.404 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:00:04.404 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-01-15T18:00:04.423 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2024-01-15T18:00:04.826 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy InRelease 2024-01-15T18:00:04.887 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy InRelease 2024-01-15T18:00:04.891 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-01-15T18:00:04.955 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-01-15T18:00:04.967 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-01-15T18:00:05.033 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-01-15T18:00:05.055 INFO:teuthology.orchestra.run.smithi204.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-01-15T18:00:05.184 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-01-15T18:00:05.224 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-01-15T18:00:05.330 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-01-15T18:00:05.404 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-01-15T18:00:07.137 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-01-15T18:00:09.875 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-15T18:00:09.903 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-01-15T18:00:09.929 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-01-15T18:00:10.313 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-01-15T18:00:10.339 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-01-15T18:00:10.347 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 28.6 kB in 6s (5,010 B/s) 2024-01-15T18:00:10.365 INFO:teuthology.orchestra.run.smithi204.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-01-15T18:00:10.786 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 28.6 kB in 6s (4,589 B/s) 2024-01-15T18:00:11.269 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2024-01-15T18:00:11.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7629-g95c90f41-1jammy cephadm=18.0.0-7629-g95c90f41-1jammy ceph-mds=18.0.0-7629-g95c90f41-1jammy ceph-mgr=18.0.0-7629-g95c90f41-1jammy ceph-common=18.0.0-7629-g95c90f41-1jammy ceph-fuse=18.0.0-7629-g95c90f41-1jammy ceph-test=18.0.0-7629-g95c90f41-1jammy ceph-volume=18.0.0-7629-g95c90f41-1jammy radosgw=18.0.0-7629-g95c90f41-1jammy python3-rados=18.0.0-7629-g95c90f41-1jammy python3-rgw=18.0.0-7629-g95c90f41-1jammy python3-cephfs=18.0.0-7629-g95c90f41-1jammy python3-rbd=18.0.0-7629-g95c90f41-1jammy libcephfs2=18.0.0-7629-g95c90f41-1jammy libcephfs-dev=18.0.0-7629-g95c90f41-1jammy librados2=18.0.0-7629-g95c90f41-1jammy librbd1=18.0.0-7629-g95c90f41-1jammy rbd-fuse=18.0.0-7629-g95c90f41-1jammy python3-cephfs=18.0.0-7629-g95c90f41-1jammy cephfs-shell=18.0.0-7629-g95c90f41-1jammy cephfs-top=18.0.0-7629-g95c90f41-1jammy cephfs-mirror=18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:00:11.359 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2024-01-15T18:00:11.529 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2024-01-15T18:00:11.530 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-15T18:00:11.689 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-15T18:00:11.692 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2024-01-15T18:00:11.692 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-01-15T18:00:11.692 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-01-15T18:00:11.693 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-01-15T18:00:11.694 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-01-15T18:00:11.695 INFO:teuthology.orchestra.run.smithi143.stdout:Recommended packages: 2024-01-15T18:00:11.696 INFO:teuthology.orchestra.run.smithi143.stdout: btrfs-tools 2024-01-15T18:00:11.706 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-01-15T18:00:11.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-7629-g95c90f41-1jammy cephadm=18.0.0-7629-g95c90f41-1jammy ceph-mds=18.0.0-7629-g95c90f41-1jammy ceph-mgr=18.0.0-7629-g95c90f41-1jammy ceph-common=18.0.0-7629-g95c90f41-1jammy ceph-fuse=18.0.0-7629-g95c90f41-1jammy ceph-test=18.0.0-7629-g95c90f41-1jammy ceph-volume=18.0.0-7629-g95c90f41-1jammy radosgw=18.0.0-7629-g95c90f41-1jammy python3-rados=18.0.0-7629-g95c90f41-1jammy python3-rgw=18.0.0-7629-g95c90f41-1jammy python3-cephfs=18.0.0-7629-g95c90f41-1jammy python3-rbd=18.0.0-7629-g95c90f41-1jammy libcephfs2=18.0.0-7629-g95c90f41-1jammy libcephfs-dev=18.0.0-7629-g95c90f41-1jammy librados2=18.0.0-7629-g95c90f41-1jammy librbd1=18.0.0-7629-g95c90f41-1jammy rbd-fuse=18.0.0-7629-g95c90f41-1jammy python3-cephfs=18.0.0-7629-g95c90f41-1jammy cephfs-shell=18.0.0-7629-g95c90f41-1jammy cephfs-top=18.0.0-7629-g95c90f41-1jammy cephfs-mirror=18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:00:11.794 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-01-15T18:00:11.840 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2024-01-15T18:00:11.840 INFO:teuthology.orchestra.run.smithi143.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-01-15T18:00:11.840 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-01-15T18:00:11.840 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-01-15T18:00:11.841 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-01-15T18:00:11.842 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-01-15T18:00:11.843 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-01-15T18:00:11.843 INFO:teuthology.orchestra.run.smithi143.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-01-15T18:00:11.843 INFO:teuthology.orchestra.run.smithi143.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-01-15T18:00:11.843 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-01-15T18:00:11.843 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-01-15T18:00:11.843 INFO:teuthology.orchestra.run.smithi143.stdout: socat unzip xmlstarlet zip 2024-01-15T18:00:11.958 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-01-15T18:00:11.959 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-01-15T18:00:12.035 INFO:teuthology.orchestra.run.smithi143.stdout:0 upgraded, 124 newly installed, 0 to remove and 269 not upgraded. 2024-01-15T18:00:12.035 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 187 MB of archives. 2024-01-15T18:00:12.035 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 835 MB of additional disk space will be used. 2024-01-15T18:00:12.035 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-01-15T18:00:12.113 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-01-15T18:00:12.113 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-15T18:00:12.113 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-15T18:00:12.113 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-15T18:00:12.113 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-15T18:00:12.114 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-15T18:00:12.114 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-15T18:00:12.116 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-01-15T18:00:12.116 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-01-15T18:00:12.117 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-01-15T18:00:12.118 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-01-15T18:00:12.119 INFO:teuthology.orchestra.run.smithi204.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-01-15T18:00:12.120 INFO:teuthology.orchestra.run.smithi204.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-01-15T18:00:12.120 INFO:teuthology.orchestra.run.smithi204.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-01-15T18:00:12.120 INFO:teuthology.orchestra.run.smithi204.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-01-15T18:00:12.120 INFO:teuthology.orchestra.run.smithi204.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-01-15T18:00:12.120 INFO:teuthology.orchestra.run.smithi204.stdout:Recommended packages: 2024-01-15T18:00:12.120 INFO:teuthology.orchestra.run.smithi204.stdout: btrfs-tools 2024-01-15T18:00:12.121 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-01-15T18:00:12.163 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-01-15T18:00:12.265 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-01-15T18:00:12.265 INFO:teuthology.orchestra.run.smithi204.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-01-15T18:00:12.265 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-01-15T18:00:12.265 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-01-15T18:00:12.265 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-01-15T18:00:12.265 INFO:teuthology.orchestra.run.smithi204.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-01-15T18:00:12.266 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-01-15T18:00:12.267 INFO:teuthology.orchestra.run.smithi204.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-01-15T18:00:12.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-01-15T18:00:12.268 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-01-15T18:00:12.268 INFO:teuthology.orchestra.run.smithi204.stdout: socat unzip xmlstarlet zip 2024-01-15T18:00:12.308 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7629-g95c90f41-1jammy [3,468 kB] 2024-01-15T18:00:12.365 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-01-15T18:00:12.459 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-01-15T18:00:12.460 INFO:teuthology.orchestra.run.smithi204.stdout:0 upgraded, 124 newly installed, 0 to remove and 269 not upgraded. 2024-01-15T18:00:12.460 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 187 MB of archives. 2024-01-15T18:00:12.460 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 835 MB of additional disk space will be used. 2024-01-15T18:00:12.461 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-01-15T18:00:12.472 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-01-15T18:00:12.523 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-01-15T18:00:12.542 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-01-15T18:00:12.588 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-01-15T18:00:12.649 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-7629-g95c90f41-1jammy [3,468 kB] 2024-01-15T18:00:12.697 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-01-15T18:00:12.707 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-01-15T18:00:12.737 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-01-15T18:00:12.745 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-01-15T18:00:12.793 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-01-15T18:00:12.893 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-01-15T18:00:12.905 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-01-15T18:00:12.909 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-01-15T18:00:12.958 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-01-15T18:00:13.132 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-01-15T18:00:13.142 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-01-15T18:00:13.169 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-01-15T18:00:13.177 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-01-15T18:00:13.342 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-01-15T18:00:13.495 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-01-15T18:00:13.510 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-01-15T18:00:13.526 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-01-15T18:00:13.535 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-01-15T18:00:13.535 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-01-15T18:00:13.551 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2024-01-15T18:00:13.589 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-01-15T18:00:13.607 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-01-15T18:00:13.607 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-01-15T18:00:13.777 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-01-15T18:00:13.860 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-01-15T18:00:13.876 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-01-15T18:00:13.941 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-01-15T18:00:13.973 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-01-15T18:00:13.991 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-01-15T18:00:14.014 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-01-15T18:00:14.026 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-01-15T18:00:14.198 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-01-15T18:00:14.213 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-01-15T18:00:14.221 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-01-15T18:00:14.225 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-01-15T18:00:14.409 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-01-15T18:00:14.432 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-01-15T18:00:14.492 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-01-15T18:00:14.506 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-01-15T18:00:14.570 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-01-15T18:00:14.679 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-01-15T18:00:14.692 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-01-15T18:00:14.743 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-01-15T18:00:14.756 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-01-15T18:00:14.762 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-01-15T18:00:14.894 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2024-01-15T18:00:14.915 INFO:teuthology.orchestra.run.smithi143.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-01-15T18:00:14.915 INFO:teuthology.orchestra.run.smithi143.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-01-15T18:00:14.920 INFO:teuthology.orchestra.run.smithi143.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-01-15T18:00:14.925 INFO:teuthology.orchestra.run.smithi143.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-01-15T18:00:14.939 INFO:teuthology.orchestra.run.smithi143.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-01-15T18:00:15.021 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-01-15T18:00:15.124 INFO:teuthology.orchestra.run.smithi143.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-01-15T18:00:15.139 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-01-15T18:00:15.230 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-01-15T18:00:15.313 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-01-15T18:00:15.377 INFO:teuthology.orchestra.run.smithi143.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-01-15T18:00:15.413 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-01-15T18:00:15.511 INFO:teuthology.orchestra.run.smithi143.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-01-15T18:00:15.555 INFO:teuthology.orchestra.run.smithi143.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-01-15T18:00:15.578 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-01-15T18:00:15.646 INFO:teuthology.orchestra.run.smithi143.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-01-15T18:00:15.655 INFO:teuthology.orchestra.run.smithi143.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-01-15T18:00:15.660 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-01-15T18:00:15.667 INFO:teuthology.orchestra.run.smithi143.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-01-15T18:00:15.710 INFO:teuthology.orchestra.run.smithi143.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-01-15T18:00:15.762 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-01-15T18:00:15.879 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-01-15T18:00:15.974 INFO:teuthology.orchestra.run.smithi143.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-01-15T18:00:15.974 INFO:teuthology.orchestra.run.smithi143.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2024-01-15T18:00:15.976 INFO:teuthology.orchestra.run.smithi143.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-01-15T18:00:15.977 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-01-15T18:00:16.077 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-01-15T18:00:16.141 INFO:teuthology.orchestra.run.smithi143.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-01-15T18:00:16.209 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7629-g95c90f41-1jammy [3,112 kB] 2024-01-15T18:00:16.447 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-01-15T18:00:16.485 INFO:teuthology.orchestra.run.smithi143.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-01-15T18:00:16.539 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-01-15T18:00:16.624 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-01-15T18:00:16.709 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-01-15T18:00:16.808 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-01-15T18:00:16.889 INFO:teuthology.orchestra.run.smithi143.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-01-15T18:00:16.899 INFO:teuthology.orchestra.run.smithi143.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-01-15T18:00:16.899 INFO:teuthology.orchestra.run.smithi143.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-01-15T18:00:16.969 INFO:teuthology.orchestra.run.smithi143.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-01-15T18:00:16.969 INFO:teuthology.orchestra.run.smithi143.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-01-15T18:00:16.970 INFO:teuthology.orchestra.run.smithi143.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-01-15T18:00:16.971 INFO:teuthology.orchestra.run.smithi143.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-01-15T18:00:16.972 INFO:teuthology.orchestra.run.smithi143.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-01-15T18:00:16.973 INFO:teuthology.orchestra.run.smithi143.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-01-15T18:00:16.974 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-01-15T18:00:17.054 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7629-g95c90f41-1jammy [690 kB] 2024-01-15T18:00:17.056 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-01-15T18:00:17.137 INFO:teuthology.orchestra.run.smithi143.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-01-15T18:00:17.195 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7629-g95c90f41-1jammy [356 kB] 2024-01-15T18:00:17.266 INFO:teuthology.orchestra.run.smithi204.stdout:Get:41 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7629-g95c90f41-1jammy [32.8 kB] 2024-01-15T18:00:17.267 INFO:teuthology.orchestra.run.smithi204.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7629-g95c90f41-1jammy [180 kB] 2024-01-15T18:00:17.269 INFO:teuthology.orchestra.run.smithi204.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7629-g95c90f41-1jammy [63.5 kB] 2024-01-15T18:00:17.336 INFO:teuthology.orchestra.run.smithi204.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7629-g95c90f41-1jammy [330 kB] 2024-01-15T18:00:17.340 INFO:teuthology.orchestra.run.smithi204.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-01-15T18:00:17.341 INFO:teuthology.orchestra.run.smithi204.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7629-g95c90f41-1jammy [6,223 kB] 2024-01-15T18:00:17.341 INFO:teuthology.orchestra.run.smithi143.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-01-15T18:00:17.455 INFO:teuthology.orchestra.run.smithi204.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-01-15T18:00:17.491 INFO:teuthology.orchestra.run.smithi143.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-7629-g95c90f41-1jammy [3,112 kB] 2024-01-15T18:00:17.547 INFO:teuthology.orchestra.run.smithi204.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-01-15T18:00:17.661 INFO:teuthology.orchestra.run.smithi143.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-01-15T18:00:17.707 INFO:teuthology.orchestra.run.smithi143.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-01-15T18:00:17.735 INFO:teuthology.orchestra.run.smithi143.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-01-15T18:00:17.743 INFO:teuthology.orchestra.run.smithi143.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-01-15T18:00:17.789 INFO:teuthology.orchestra.run.smithi143.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-01-15T18:00:17.815 INFO:teuthology.orchestra.run.smithi143.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-01-15T18:00:17.817 INFO:teuthology.orchestra.run.smithi143.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-01-15T18:00:17.824 INFO:teuthology.orchestra.run.smithi143.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-01-15T18:00:17.827 INFO:teuthology.orchestra.run.smithi143.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-01-15T18:00:17.846 INFO:teuthology.orchestra.run.smithi204.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-01-15T18:00:17.929 INFO:teuthology.orchestra.run.smithi204.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-01-15T18:00:18.013 INFO:teuthology.orchestra.run.smithi204.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-01-15T18:00:18.023 INFO:teuthology.orchestra.run.smithi143.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-01-15T18:00:18.097 INFO:teuthology.orchestra.run.smithi204.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-01-15T18:00:18.188 INFO:teuthology.orchestra.run.smithi204.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-01-15T18:00:18.209 INFO:teuthology.orchestra.run.smithi204.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7629-g95c90f41-1jammy [112 kB] 2024-01-15T18:00:18.210 INFO:teuthology.orchestra.run.smithi204.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7629-g95c90f41-1jammy [439 kB] 2024-01-15T18:00:18.217 INFO:teuthology.orchestra.run.smithi204.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7629-g95c90f41-1jammy [22.6 MB] 2024-01-15T18:00:18.282 INFO:teuthology.orchestra.run.smithi143.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-01-15T18:00:18.286 INFO:teuthology.orchestra.run.smithi204.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-01-15T18:00:18.454 INFO:teuthology.orchestra.run.smithi204.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-01-15T18:00:18.545 INFO:teuthology.orchestra.run.smithi143.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-01-15T18:00:18.547 INFO:teuthology.orchestra.run.smithi143.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-01-15T18:00:18.563 INFO:teuthology.orchestra.run.smithi143.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-01-15T18:00:18.569 INFO:teuthology.orchestra.run.smithi143.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-01-15T18:00:18.606 INFO:teuthology.orchestra.run.smithi143.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-01-15T18:00:18.620 INFO:teuthology.orchestra.run.smithi143.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-7629-g95c90f41-1jammy [690 kB] 2024-01-15T18:00:18.620 INFO:teuthology.orchestra.run.smithi143.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-01-15T18:00:18.711 INFO:teuthology.orchestra.run.smithi143.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-01-15T18:00:18.712 INFO:teuthology.orchestra.run.smithi143.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-01-15T18:00:18.715 INFO:teuthology.orchestra.run.smithi143.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-01-15T18:00:18.797 INFO:teuthology.orchestra.run.smithi143.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-7629-g95c90f41-1jammy [356 kB] 2024-01-15T18:00:18.808 INFO:teuthology.orchestra.run.smithi204.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-01-15T18:00:18.886 INFO:teuthology.orchestra.run.smithi143.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-7629-g95c90f41-1jammy [32.8 kB] 2024-01-15T18:00:18.891 INFO:teuthology.orchestra.run.smithi143.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-7629-g95c90f41-1jammy [180 kB] 2024-01-15T18:00:18.895 INFO:teuthology.orchestra.run.smithi143.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-01-15T18:00:18.923 INFO:teuthology.orchestra.run.smithi204.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-01-15T18:00:18.930 INFO:teuthology.orchestra.run.smithi143.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-7629-g95c90f41-1jammy [63.5 kB] 2024-01-15T18:00:18.966 INFO:teuthology.orchestra.run.smithi143.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-7629-g95c90f41-1jammy [330 kB] 2024-01-15T18:00:19.011 INFO:teuthology.orchestra.run.smithi143.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-7629-g95c90f41-1jammy [6,223 kB] 2024-01-15T18:00:19.125 INFO:teuthology.orchestra.run.smithi204.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-01-15T18:00:19.169 INFO:teuthology.orchestra.run.smithi143.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-01-15T18:00:19.214 INFO:teuthology.orchestra.run.smithi204.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-01-15T18:00:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-01-15T18:00:19.430 INFO:teuthology.orchestra.run.smithi204.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-01-15T18:00:19.681 INFO:teuthology.orchestra.run.smithi204.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7629-g95c90f41-1jammy [6,900 kB] 2024-01-15T18:00:19.884 INFO:teuthology.orchestra.run.smithi204.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-01-15T18:00:19.967 INFO:teuthology.orchestra.run.smithi204.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2024-01-15T18:00:19.971 INFO:teuthology.orchestra.run.smithi204.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7629-g95c90f41-1jammy [242 kB] 2024-01-15T18:00:19.979 INFO:teuthology.orchestra.run.smithi204.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7629-g95c90f41-1jammy [124 kB] 2024-01-15T18:00:20.051 INFO:teuthology.orchestra.run.smithi204.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-01-15T18:00:20.133 INFO:teuthology.orchestra.run.smithi204.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-01-15T18:00:20.141 INFO:teuthology.orchestra.run.smithi204.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7629-g95c90f41-1jammy [1,496 kB] 2024-01-15T18:00:20.298 INFO:teuthology.orchestra.run.smithi204.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-01-15T18:00:20.311 INFO:teuthology.orchestra.run.smithi204.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7629-g95c90f41-1jammy [9,955 kB] 2024-01-15T18:00:20.720 INFO:teuthology.orchestra.run.smithi204.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7629-g95c90f41-1jammy [31.8 MB] 2024-01-15T18:00:21.007 INFO:teuthology.orchestra.run.smithi143.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-7629-g95c90f41-1jammy [112 kB] 2024-01-15T18:00:21.013 INFO:teuthology.orchestra.run.smithi143.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-7629-g95c90f41-1jammy [439 kB] 2024-01-15T18:00:21.023 INFO:teuthology.orchestra.run.smithi204.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-01-15T18:00:21.111 INFO:teuthology.orchestra.run.smithi204.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-01-15T18:00:21.151 INFO:teuthology.orchestra.run.smithi143.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-7629-g95c90f41-1jammy [22.6 MB] 2024-01-15T18:00:21.193 INFO:teuthology.orchestra.run.smithi204.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-01-15T18:00:21.327 INFO:teuthology.orchestra.run.smithi204.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-01-15T18:00:21.421 INFO:teuthology.orchestra.run.smithi204.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-01-15T18:00:21.530 INFO:teuthology.orchestra.run.smithi204.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-01-15T18:00:21.600 INFO:teuthology.orchestra.run.smithi204.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-01-15T18:00:21.649 INFO:teuthology.orchestra.run.smithi204.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-01-15T18:00:21.696 INFO:teuthology.orchestra.run.smithi204.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-01-15T18:00:21.743 INFO:teuthology.orchestra.run.smithi204.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-01-15T18:00:21.795 INFO:teuthology.orchestra.run.smithi204.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-01-15T18:00:21.924 INFO:teuthology.orchestra.run.smithi204.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7629-g95c90f41-1jammy [14.1 kB] 2024-01-15T18:00:21.925 INFO:teuthology.orchestra.run.smithi204.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7629-g95c90f41-1jammy [839 kB] 2024-01-15T18:00:21.964 INFO:teuthology.orchestra.run.smithi204.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7629-g95c90f41-1jammy [2,144 kB] 2024-01-15T18:00:21.971 INFO:teuthology.orchestra.run.smithi204.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-01-15T18:00:22.020 INFO:teuthology.orchestra.run.smithi204.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-01-15T18:00:22.064 INFO:teuthology.orchestra.run.smithi204.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-01-15T18:00:22.105 INFO:teuthology.orchestra.run.smithi204.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-01-15T18:00:22.179 INFO:teuthology.orchestra.run.smithi204.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-01-15T18:00:22.246 INFO:teuthology.orchestra.run.smithi204.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-01-15T18:00:22.295 INFO:teuthology.orchestra.run.smithi204.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-01-15T18:00:22.296 INFO:teuthology.orchestra.run.smithi204.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7629-g95c90f41-1jammy [805 kB] 2024-01-15T18:00:22.340 INFO:teuthology.orchestra.run.smithi204.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7629-g95c90f41-1jammy [139 kB] 2024-01-15T18:00:22.342 INFO:teuthology.orchestra.run.smithi204.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-01-15T18:00:22.342 INFO:teuthology.orchestra.run.smithi204.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7629-g95c90f41-1jammy [1,900 kB] 2024-01-15T18:00:22.393 INFO:teuthology.orchestra.run.smithi204.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-01-15T18:00:22.436 INFO:teuthology.orchestra.run.smithi204.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7629-g95c90f41-1jammy [8,625 kB] 2024-01-15T18:00:22.480 INFO:teuthology.orchestra.run.smithi204.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-01-15T18:00:22.533 INFO:teuthology.orchestra.run.smithi204.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-01-15T18:00:22.600 INFO:teuthology.orchestra.run.smithi204.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-01-15T18:00:22.656 INFO:teuthology.orchestra.run.smithi204.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-01-15T18:00:22.706 INFO:teuthology.orchestra.run.smithi204.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-01-15T18:00:22.752 INFO:teuthology.orchestra.run.smithi204.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-01-15T18:00:22.787 INFO:teuthology.orchestra.run.smithi204.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7629-g95c90f41-1jammy [14.2 kB] 2024-01-15T18:00:22.787 INFO:teuthology.orchestra.run.smithi204.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7629-g95c90f41-1jammy [48.6 MB] 2024-01-15T18:00:22.810 INFO:teuthology.orchestra.run.smithi204.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-01-15T18:00:22.874 INFO:teuthology.orchestra.run.smithi204.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-01-15T18:00:23.122 INFO:teuthology.orchestra.run.smithi204.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-01-15T18:00:23.158 INFO:teuthology.orchestra.run.smithi204.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-01-15T18:00:23.197 INFO:teuthology.orchestra.run.smithi204.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-01-15T18:00:23.240 INFO:teuthology.orchestra.run.smithi204.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-01-15T18:00:23.283 INFO:teuthology.orchestra.run.smithi204.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-01-15T18:00:24.797 INFO:teuthology.orchestra.run.smithi204.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7629-g95c90f41-1jammy [121 kB] 2024-01-15T18:00:24.842 INFO:teuthology.orchestra.run.smithi204.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7629-g95c90f41-1jammy [226 kB] 2024-01-15T18:00:24.883 INFO:teuthology.orchestra.run.smithi204.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7629-g95c90f41-1jammy [34.0 kB] 2024-01-15T18:00:24.924 INFO:teuthology.orchestra.run.smithi204.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7629-g95c90f41-1jammy [23.4 kB] 2024-01-15T18:00:24.924 INFO:teuthology.orchestra.run.smithi204.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7629-g95c90f41-1jammy [38.3 kB] 2024-01-15T18:00:24.925 INFO:teuthology.orchestra.run.smithi204.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7629-g95c90f41-1jammy [12.3 MB] 2024-01-15T18:00:25.349 INFO:teuthology.orchestra.run.smithi204.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7629-g95c90f41-1jammy [91.4 kB] 2024-01-15T18:00:25.983 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 187 MB in 13s (14.3 MB/s) 2024-01-15T18:00:26.108 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyperclip. 2024-01-15T18:00:26.175 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2024-01-15T18:00:26.178 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-01-15T18:00:26.223 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-01-15T18:00:26.493 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-wcwidth. 2024-01-15T18:00:26.507 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-01-15T18:00:26.551 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-15T18:00:26.873 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cmd2. 2024-01-15T18:00:26.887 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-01-15T18:00:26.973 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-01-15T18:00:27.267 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-01-15T18:00:27.282 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-01-15T18:00:27.359 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-15T18:00:27.663 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-01-15T18:00:27.678 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-01-15T18:00:27.721 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-15T18:00:28.074 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-01-15T18:00:28.089 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-01-15T18:00:28.132 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-15T18:00:28.569 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-01-15T18:00:28.584 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-15T18:00:28.627 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:00:29.006 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-01-15T18:00:29.020 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-15T18:00:29.072 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:00:29.442 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-01-15T18:00:29.457 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-15T18:00:29.500 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:00:29.845 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-01-15T18:00:29.861 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-01-15T18:00:29.895 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-15T18:00:30.198 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librados2. 2024-01-15T18:00:30.214 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../010-librados2_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:30.256 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librados2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:30.680 INFO:teuthology.orchestra.run.smithi143.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-7629-g95c90f41-1jammy [6,900 kB] 2024-01-15T18:00:30.910 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librbd1. 2024-01-15T18:00:30.925 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../011-librbd1_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:30.992 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librbd1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:31.445 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs2. 2024-01-15T18:00:31.460 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:31.504 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:31.765 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rados. 2024-01-15T18:00:31.780 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:31.814 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rados (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:32.151 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-01-15T18:00:32.166 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:00:32.209 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-argparse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:32.445 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cephfs. 2024-01-15T18:00:32.460 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:32.503 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cephfs (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:32.822 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-ceph-common. 2024-01-15T18:00:32.837 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:00:32.914 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-prettytable. 2024-01-15T18:00:33.233 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-01-15T18:00:33.267 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-01-15T18:00:33.511 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rbd. 2024-01-15T18:00:33.526 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:33.545 INFO:teuthology.orchestra.run.smithi143.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-7629-g95c90f41-1jammy [242 kB] 2024-01-15T18:00:33.570 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rbd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:33.624 INFO:teuthology.orchestra.run.smithi143.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-7629-g95c90f41-1jammy [124 kB] 2024-01-15T18:00:33.695 INFO:teuthology.orchestra.run.smithi143.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-7629-g95c90f41-1jammy [1,496 kB] 2024-01-15T18:00:33.923 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-01-15T18:00:33.938 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-01-15T18:00:33.981 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-15T18:00:34.205 INFO:teuthology.orchestra.run.smithi143.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-7629-g95c90f41-1jammy [9,955 kB] 2024-01-15T18:00:34.267 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-01-15T18:00:34.282 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-01-15T18:00:34.325 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-01-15T18:00:34.637 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-01-15T18:00:34.652 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-01-15T18:00:34.686 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-15T18:00:35.031 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua5.1. 2024-01-15T18:00:35.047 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-01-15T18:00:35.090 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-01-15T18:00:35.401 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-any. 2024-01-15T18:00:35.416 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-01-15T18:00:35.459 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-any (27ubuntu1) ... 2024-01-15T18:00:35.704 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package zip. 2024-01-15T18:00:35.719 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-01-15T18:00:35.753 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking zip (3.0-12build2) ... 2024-01-15T18:00:36.040 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package unzip. 2024-01-15T18:00:36.056 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2024-01-15T18:00:36.098 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2024-01-15T18:00:36.503 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package luarocks. 2024-01-15T18:00:36.518 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-01-15T18:00:36.577 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-01-15T18:00:36.960 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package librgw2. 2024-01-15T18:00:36.974 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../027-librgw2_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:37.018 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking librgw2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:37.471 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rgw. 2024-01-15T18:00:37.485 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../028-python3-rgw_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:37.874 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liboath0:amd64. 2024-01-15T18:00:37.889 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-01-15T18:00:37.932 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-01-15T18:00:38.235 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libradosstriper1. 2024-01-15T18:00:38.250 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../030-libradosstriper1_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:38.293 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libradosstriper1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:38.688 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-common. 2024-01-15T18:00:38.704 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../031-ceph-common_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:38.738 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:39.425 INFO:teuthology.orchestra.run.smithi143.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-7629-g95c90f41-1jammy [31.8 MB] 2024-01-15T18:00:39.819 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-base. 2024-01-15T18:00:39.834 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../032-ceph-base_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:39.886 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-base (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:40.573 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-01-15T18:00:40.587 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-01-15T18:00:40.623 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-01-15T18:00:40.935 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cheroot. 2024-01-15T18:00:40.949 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-01-15T18:00:40.993 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-15T18:00:41.321 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-01-15T18:00:41.336 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-01-15T18:00:41.379 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-01-15T18:00:41.673 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.text. 2024-01-15T18:00:41.688 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-01-15T18:00:41.723 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-01-15T18:00:42.010 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-01-15T18:00:42.025 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-01-15T18:00:42.068 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-01-15T18:00:42.371 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempora. 2024-01-15T18:00:42.386 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-01-15T18:00:42.429 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-01-15T18:00:42.715 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-portend. 2024-01-15T18:00:42.730 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-01-15T18:00:42.782 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-01-15T18:00:43.043 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-01-15T18:00:43.058 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-01-15T18:00:43.093 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-01-15T18:00:43.363 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cherrypy3. 2024-01-15T18:00:43.378 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-01-15T18:00:43.421 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-01-15T18:00:43.750 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-natsort. 2024-01-15T18:00:43.765 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-01-15T18:00:43.799 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-01-15T18:00:44.086 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-logutils. 2024-01-15T18:00:44.101 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-01-15T18:00:44.144 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-01-15T18:00:44.447 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-markupsafe. 2024-01-15T18:00:44.462 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-01-15T18:00:44.505 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-01-15T18:00:44.817 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-mako. 2024-01-15T18:00:44.831 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-01-15T18:00:44.875 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-15T18:00:45.187 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplegeneric. 2024-01-15T18:00:45.201 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-01-15T18:00:45.236 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-01-15T18:00:45.497 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-singledispatch. 2024-01-15T18:00:45.512 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-01-15T18:00:45.556 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-01-15T18:00:45.867 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webob. 2024-01-15T18:00:45.882 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-01-15T18:00:45.986 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-01-15T18:00:47.095 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-waitress. 2024-01-15T18:00:47.111 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-01-15T18:00:47.156 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-15T18:00:47.487 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-tempita. 2024-01-15T18:00:47.502 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-01-15T18:00:47.549 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-01-15T18:00:47.819 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-paste. 2024-01-15T18:00:47.833 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-01-15T18:00:47.868 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-01-15T18:00:48.171 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-01-15T18:00:48.186 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-01-15T18:00:48.221 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-01-15T18:00:48.507 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastedeploy. 2024-01-15T18:00:48.522 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-01-15T18:00:48.565 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-01-15T18:00:48.852 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-webtest. 2024-01-15T18:00:48.867 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-01-15T18:00:48.910 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-01-15T18:00:49.221 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pecan. 2024-01-15T18:00:49.236 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-01-15T18:00:49.279 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-01-15T18:00:49.659 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-certifi. 2024-01-15T18:00:49.675 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-01-15T18:00:49.717 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-01-15T18:00:49.995 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-idna. 2024-01-15T18:00:50.013 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-01-15T18:00:50.045 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-idna (3.3-1) ... 2024-01-15T18:00:50.365 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-urllib3. 2024-01-15T18:00:50.380 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-01-15T18:00:50.423 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-15T18:00:50.743 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-requests. 2024-01-15T18:00:50.758 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-01-15T18:00:50.801 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-15T18:00:51.149 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-werkzeug. 2024-01-15T18:00:51.164 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-01-15T18:00:51.212 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-15T18:00:51.507 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-01-15T18:00:51.523 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:00:51.565 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:51.929 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-01-15T18:00:51.943 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:51.978 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:51.998 INFO:teuthology.orchestra.run.smithi143.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-7629-g95c90f41-1jammy [14.1 kB] 2024-01-15T18:00:51.999 INFO:teuthology.orchestra.run.smithi143.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-7629-g95c90f41-1jammy [839 kB] 2024-01-15T18:00:52.265 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-bcrypt. 2024-01-15T18:00:52.280 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-01-15T18:00:52.323 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-01-15T18:00:52.635 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-openssl. 2024-01-15T18:00:52.650 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-01-15T18:00:52.693 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-01-15T18:00:53.062 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr. 2024-01-15T18:00:53.078 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../065-ceph-mgr_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:53.120 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:53.457 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mon. 2024-01-15T18:00:53.473 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../066-ceph-mon_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:53.480 INFO:teuthology.orchestra.run.smithi143.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-7629-g95c90f41-1jammy [2,144 kB] 2024-01-15T18:00:53.507 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mon (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:54.301 INFO:teuthology.orchestra.run.smithi143.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-7629-g95c90f41-1jammy [805 kB] 2024-01-15T18:00:54.319 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-osd. 2024-01-15T18:00:54.334 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../067-ceph-osd_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:54.377 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-osd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:54.537 INFO:teuthology.orchestra.run.smithi143.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-7629-g95c90f41-1jammy [139 kB] 2024-01-15T18:00:54.581 INFO:teuthology.orchestra.run.smithi143.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-7629-g95c90f41-1jammy [1,900 kB] 2024-01-15T18:00:55.201 INFO:teuthology.orchestra.run.smithi143.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-7629-g95c90f41-1jammy [8,625 kB] 2024-01-15T18:00:55.689 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph. 2024-01-15T18:00:55.704 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../068-ceph_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:55.747 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:56.008 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-fuse. 2024-01-15T18:00:56.023 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../069-ceph-fuse_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:56.075 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:56.620 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mds. 2024-01-15T18:00:56.635 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../070-ceph-mds_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:56.678 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mds (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:57.081 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephadm. 2024-01-15T18:00:57.096 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../071-cephadm_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:00:57.139 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:00:57.425 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-asyncssh. 2024-01-15T18:00:57.441 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-01-15T18:00:57.475 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-01-15T18:00:57.795 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-babel-localedata. 2024-01-15T18:00:57.810 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-01-15T18:00:57.844 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-15T18:00:58.833 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-babel. 2024-01-15T18:00:58.847 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-01-15T18:00:58.891 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-01-15T18:00:59.211 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-jinja2. 2024-01-15T18:00:59.225 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2024-01-15T18:00:59.269 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2024-01-15T18:00:59.622 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-01-15T18:00:59.636 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:00:59.680 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:00.009 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-repoze.lru. 2024-01-15T18:01:00.031 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-01-15T18:01:00.075 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-01-15T18:01:00.087 INFO:teuthology.orchestra.run.smithi143.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-7629-g95c90f41-1jammy [14.2 kB] 2024-01-15T18:01:00.090 INFO:teuthology.orchestra.run.smithi143.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-7629-g95c90f41-1jammy [48.6 MB] 2024-01-15T18:01:00.378 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-routes. 2024-01-15T18:01:00.392 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-01-15T18:01:00.436 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-01-15T18:01:00.798 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-01-15T18:01:00.811 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:00.856 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:01.293 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-01-15T18:01:01.307 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-01-15T18:01:01.351 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-15T18:01:01.821 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-joblib. 2024-01-15T18:01:01.835 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-01-15T18:01:01.871 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-01-15T18:01:02.191 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-01-15T18:01:02.205 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-01-15T18:01:02.343 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-01-15T18:01:02.679 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-sklearn. 2024-01-15T18:01:02.694 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-01-15T18:01:02.737 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-15T18:01:03.347 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-01-15T18:01:03.361 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:03.397 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:04.426 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-cachetools. 2024-01-15T18:01:04.441 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-01-15T18:01:04.486 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-01-15T18:01:04.764 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1. 2024-01-15T18:01:04.779 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-01-15T18:01:04.813 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-01-15T18:01:05.109 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-01-15T18:01:05.124 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-01-15T18:01:05.159 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-01-15T18:01:05.479 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-rsa. 2024-01-15T18:01:05.495 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-01-15T18:01:05.536 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-rsa (4.8-1) ... 2024-01-15T18:01:05.858 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-google-auth. 2024-01-15T18:01:05.873 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-01-15T18:01:05.915 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-01-15T18:01:06.286 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-01-15T18:01:06.301 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-01-15T18:01:06.335 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-15T18:01:06.614 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-websocket. 2024-01-15T18:01:06.629 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-01-15T18:01:06.672 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-01-15T18:01:06.967 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-kubernetes. 2024-01-15T18:01:06.983 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-01-15T18:01:07.025 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-15T18:01:07.673 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-01-15T18:01:07.687 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:07.730 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:08.076 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libonig5:amd64. 2024-01-15T18:01:08.089 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-01-15T18:01:08.133 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-15T18:01:08.454 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjq1:amd64. 2024-01-15T18:01:08.468 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-01-15T18:01:08.511 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-15T18:01:08.773 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package jq. 2024-01-15T18:01:08.788 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-01-15T18:01:08.831 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-01-15T18:01:09.084 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package socat. 2024-01-15T18:01:09.100 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-01-15T18:01:09.142 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-01-15T18:01:09.446 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package xmlstarlet. 2024-01-15T18:01:09.461 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-01-15T18:01:09.495 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-01-15T18:01:09.850 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-test. 2024-01-15T18:01:09.864 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../099-ceph-test_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:09.908 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-test (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:12.163 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ceph-volume. 2024-01-15T18:01:12.179 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../100-ceph-volume_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:12.221 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ceph-volume (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:12.575 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephfs-mirror. 2024-01-15T18:01:12.591 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../101-cephfs-mirror_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:12.633 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephfs-mirror (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:12.911 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-colorama. 2024-01-15T18:01:12.927 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-01-15T18:01:12.969 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-01-15T18:01:13.230 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephfs-shell. 2024-01-15T18:01:13.246 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../103-cephfs-shell_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:13.289 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephfs-shell (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:13.583 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package cephfs-top. 2024-01-15T18:01:13.599 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../104-cephfs-top_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:13.641 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking cephfs-top (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:13.877 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libcephfs-dev. 2024-01-15T18:01:13.893 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../105-libcephfs-dev_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:13.935 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libcephfs-dev (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:14.247 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-socket:amd64. 2024-01-15T18:01:14.263 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-01-15T18:01:14.304 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-15T18:01:14.593 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package lua-sec:amd64. 2024-01-15T18:01:14.608 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-01-15T18:01:14.651 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-01-15T18:01:15.014 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package nvme-cli. 2024-01-15T18:01:15.029 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-01-15T18:01:15.071 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-01-15T18:01:15.493 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package pkg-config. 2024-01-15T18:01:15.509 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-01-15T18:01:15.551 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-01-15T18:01:15.813 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-01-15T18:01:15.829 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-01-15T18:01:15.871 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-01-15T18:01:16.258 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-iniconfig. 2024-01-15T18:01:16.273 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-01-15T18:01:16.308 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-01-15T18:01:16.594 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pastescript. 2024-01-15T18:01:16.609 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-01-15T18:01:16.652 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-01-15T18:01:16.973 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pluggy. 2024-01-15T18:01:16.988 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-01-15T18:01:17.031 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-01-15T18:01:17.367 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-psutil. 2024-01-15T18:01:17.383 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-01-15T18:01:17.425 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-01-15T18:01:17.770 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-py. 2024-01-15T18:01:17.785 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-01-15T18:01:17.828 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-py (1.10.0-1) ... 2024-01-15T18:01:18.157 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pygments. 2024-01-15T18:01:18.172 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-01-15T18:01:18.214 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-01-15T18:01:18.644 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pyinotify. 2024-01-15T18:01:18.659 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-01-15T18:01:18.694 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-01-15T18:01:19.014 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-toml. 2024-01-15T18:01:19.028 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-01-15T18:01:19.071 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-01-15T18:01:19.375 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-pytest. 2024-01-15T18:01:19.390 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-01-15T18:01:19.432 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-01-15T18:01:19.745 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package python3-simplejson. 2024-01-15T18:01:19.760 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-01-15T18:01:19.803 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-01-15T18:01:20.073 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-01-15T18:01:20.088 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-01-15T18:01:20.131 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-01-15T18:01:20.918 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package radosgw. 2024-01-15T18:01:20.933 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../122-radosgw_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:20.976 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking radosgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:21.572 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package rbd-fuse. 2024-01-15T18:01:21.587 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../123-rbd-fuse_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:21.629 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking rbd-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:21.931 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-01-15T18:01:22.218 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-15T18:01:22.344 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-01-15T18:01:22.615 INFO:teuthology.orchestra.run.smithi143.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-7629-g95c90f41-1jammy [121 kB] 2024-01-15T18:01:22.627 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-01-15T18:01:22.696 INFO:teuthology.orchestra.run.smithi143.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-7629-g95c90f41-1jammy [226 kB] 2024-01-15T18:01:22.856 INFO:teuthology.orchestra.run.smithi143.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-7629-g95c90f41-1jammy [34.0 kB] 2024-01-15T18:01:22.862 INFO:teuthology.orchestra.run.smithi143.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-7629-g95c90f41-1jammy [23.4 kB] 2024-01-15T18:01:22.901 INFO:teuthology.orchestra.run.smithi143.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-7629-g95c90f41-1jammy [38.3 kB] 2024-01-15T18:01:22.936 INFO:teuthology.orchestra.run.smithi143.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-7629-g95c90f41-1jammy [12.3 MB] 2024-01-15T18:01:23.004 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-01-15T18:01:23.535 INFO:teuthology.orchestra.run.smithi204.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-01-15T18:01:23.574 INFO:teuthology.orchestra.run.smithi204.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-01-15T18:01:23.630 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:24.233 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user cephadm....done 2024-01-15T18:01:24.377 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-15T18:01:24.696 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-01-15T18:01:24.956 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-01-15T18:01:25.089 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-01-15T18:01:25.349 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-01-15T18:01:25.625 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-01-15T18:01:25.776 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-py (1.10.0-1) ... 2024-01-15T18:01:26.103 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-01-15T18:01:26.480 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-01-15T18:01:26.965 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-01-15T18:01:27.425 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-01-15T18:01:27.709 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2024-01-15T18:01:27.953 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-01-15T18:01:28.221 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-01-15T18:01:28.480 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-01-15T18:01:28.586 INFO:teuthology.orchestra.run.smithi143.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-7629-g95c90f41-1jammy [91.4 kB] 2024-01-15T18:01:28.723 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-argparse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:29.007 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-15T18:01:29.133 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-15T18:01:29.247 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 187 MB in 1min 17s (2,434 kB/s) 2024-01-15T18:01:29.259 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-01-15T18:01:29.391 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyperclip. 2024-01-15T18:01:29.458 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2024-01-15T18:01:29.461 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-01-15T18:01:29.506 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-01-15T18:01:29.519 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-01-15T18:01:29.620 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-01-15T18:01:29.818 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-wcwidth. 2024-01-15T18:01:29.831 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-01-15T18:01:29.863 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-01-15T18:01:29.876 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-15T18:01:30.195 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cmd2. 2024-01-15T18:01:30.209 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-01-15T18:01:30.253 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-01-15T18:01:30.280 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-01-15T18:01:30.310 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-01-15T18:01:30.391 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-15T18:01:30.517 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-01-15T18:01:30.581 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-01-15T18:01:30.596 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-01-15T18:01:30.664 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-15T18:01:30.877 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-15T18:01:30.978 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-01-15T18:01:31.001 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-01-15T18:01:31.016 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-01-15T18:01:31.059 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-15T18:01:31.229 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-01-15T18:01:31.421 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-01-15T18:01:31.435 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-01-15T18:01:31.479 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-15T18:01:31.530 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-01-15T18:01:31.823 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up zip (3.0-12build2) ... 2024-01-15T18:01:31.907 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-01-15T18:01:31.921 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-15T18:01:31.924 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-01-15T18:01:31.964 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:01:32.302 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-01-15T18:01:32.315 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-15T18:01:32.360 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:01:32.609 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-01-15T18:01:32.729 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-01-15T18:01:32.744 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-01-15T18:01:32.788 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:01:32.869 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-01-15T18:01:32.995 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-01-15T18:01:33.130 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-15T18:01:33.132 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-01-15T18:01:33.147 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-01-15T18:01:33.190 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-15T18:01:33.447 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-01-15T18:01:33.468 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librados2. 2024-01-15T18:01:33.483 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../010-librados2_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:33.518 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librados2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:33.840 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-01-15T18:01:33.912 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librbd1. 2024-01-15T18:01:33.927 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../011-librbd1_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:33.970 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librbd1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:34.100 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-01-15T18:01:34.406 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs2. 2024-01-15T18:01:34.421 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../012-libcephfs2_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:34.464 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:34.493 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-15T18:01:34.750 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rados. 2024-01-15T18:01:34.765 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../013-python3-rados_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:34.794 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-01-15T18:01:34.808 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rados (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:35.078 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-01-15T18:01:35.093 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:35.113 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-15T18:01:35.128 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-argparse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:35.347 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cephfs. 2024-01-15T18:01:35.362 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../015-python3-cephfs_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:35.405 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cephfs (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:35.464 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-01-15T18:01:35.715 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-01-15T18:01:35.749 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-common. 2024-01-15T18:01:35.764 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../016-python3-ceph-common_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:35.807 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:35.841 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:36.127 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-prettytable. 2024-01-15T18:01:36.142 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-01-15T18:01:36.143 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-idna (3.3-1) ... 2024-01-15T18:01:36.185 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-01-15T18:01:36.421 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rbd. 2024-01-15T18:01:36.436 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../018-python3-rbd_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:36.469 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-15T18:01:36.479 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rbd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:36.790 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-01-15T18:01:36.805 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-01-15T18:01:36.840 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-15T18:01:37.093 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-01-15T18:01:37.108 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-01-15T18:01:37.151 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-01-15T18:01:37.437 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-01-15T18:01:37.452 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-01-15T18:01:37.495 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-15T18:01:37.675 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-01-15T18:01:37.823 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package lua5.1. 2024-01-15T18:01:37.838 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-01-15T18:01:37.881 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-01-15T18:01:37.918 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:01:38.070 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-toml (0.10.2-1) ... 2024-01-15T18:01:38.176 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package lua-any. 2024-01-15T18:01:38.191 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-01-15T18:01:38.251 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking lua-any (27ubuntu1) ... 2024-01-15T18:01:38.338 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-15T18:01:38.464 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-01-15T18:01:38.470 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package zip. 2024-01-15T18:01:38.485 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-01-15T18:01:38.528 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking zip (3.0-12build2) ... 2024-01-15T18:01:38.590 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-15T18:01:38.839 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package unzip. 2024-01-15T18:01:38.854 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2024-01-15T18:01:38.891 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-01-15T18:01:38.897 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2024-01-15T18:01:39.159 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-01-15T18:01:39.301 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package luarocks. 2024-01-15T18:01:39.316 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-01-15T18:01:39.359 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-01-15T18:01:39.410 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:01:39.536 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-01-15T18:01:39.774 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librgw2. 2024-01-15T18:01:39.789 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../027-librgw2_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:39.824 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librgw2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:39.846 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-01-15T18:01:40.106 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-01-15T18:01:40.277 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rgw. 2024-01-15T18:01:40.292 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../028-python3-rgw_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:40.335 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:40.357 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-01-15T18:01:40.616 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-01-15T18:01:40.663 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liboath0:amd64. 2024-01-15T18:01:40.677 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-01-15T18:01:40.721 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-01-15T18:01:40.909 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-01-15T18:01:41.024 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libradosstriper1. 2024-01-15T18:01:41.039 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../030-libradosstriper1_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:41.073 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libradosstriper1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:41.169 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-15T18:01:41.294 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-01-15T18:01:41.502 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-common. 2024-01-15T18:01:41.517 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../031-ceph-common_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:41.560 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:41.579 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-15T18:01:41.697 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-15T18:01:41.973 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-01-15T18:01:42.283 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-01-15T18:01:42.665 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-base. 2024-01-15T18:01:42.680 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../032-ceph-base_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:42.732 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-base (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:42.792 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-15T18:01:42.893 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-01-15T18:01:42.995 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-01-15T18:01:43.067 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-01-15T18:01:43.322 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-15T18:01:43.427 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-01-15T18:01:43.441 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-01-15T18:01:43.448 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-01-15T18:01:43.485 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-01-15T18:01:43.796 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cheroot. 2024-01-15T18:01:43.810 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-01-15T18:01:43.841 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-01-15T18:01:43.854 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-15T18:01:44.109 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up lua-any (27ubuntu1) ... 2024-01-15T18:01:44.157 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-01-15T18:01:44.172 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-01-15T18:01:44.207 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-01-15T18:01:44.218 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-portend (3.0.0-1) ... 2024-01-15T18:01:44.469 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:01:44.518 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.text. 2024-01-15T18:01:44.533 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-01-15T18:01:44.577 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-01-15T18:01:44.612 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2024-01-15T18:01:44.863 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-01-15T18:01:44.877 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-01-15T18:01:44.921 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-01-15T18:01:44.947 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-01-15T18:01:45.248 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-tempora. 2024-01-15T18:01:45.257 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-15T18:01:45.263 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-01-15T18:01:45.298 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-01-15T18:01:45.524 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-01-15T18:01:45.559 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-portend. 2024-01-15T18:01:45.574 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-01-15T18:01:45.617 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-01-15T18:01:45.744 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rsa (4.8-1) ... 2024-01-15T18:01:45.904 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-01-15T18:01:45.919 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-01-15T18:01:45.961 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-01-15T18:01:46.003 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-01-15T18:01:46.264 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cherrypy3. 2024-01-15T18:01:46.272 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-15T18:01:46.279 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-01-15T18:01:46.322 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-01-15T18:01:46.531 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-01-15T18:01:46.726 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-natsort. 2024-01-15T18:01:46.741 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-01-15T18:01:46.784 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-01-15T18:01:46.900 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-01-15T18:01:47.095 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-logutils. 2024-01-15T18:01:47.109 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-01-15T18:01:47.153 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-01-15T18:01:47.193 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-01-15T18:01:47.440 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-markupsafe. 2024-01-15T18:01:47.454 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-01-15T18:01:47.497 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-01-15T18:01:47.529 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-15T18:01:47.655 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librados2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:47.742 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-mako. 2024-01-15T18:01:47.757 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-01-15T18:01:47.781 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:47.791 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-15T18:01:47.907 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-01-15T18:01:48.128 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplegeneric. 2024-01-15T18:01:48.143 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-01-15T18:01:48.186 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-01-15T18:01:48.260 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:48.361 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libradosstriper1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:48.462 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-01-15T18:01:48.506 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-singledispatch. 2024-01-15T18:01:48.521 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-01-15T18:01:48.564 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-01-15T18:01:48.740 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librbd1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:48.866 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webob. 2024-01-15T18:01:48.867 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-modules-core (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:48.881 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-01-15T18:01:48.924 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-01-15T18:01:48.992 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:49.201 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-15T18:01:49.201 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-15T18:01:49.302 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-waitress. 2024-01-15T18:01:49.317 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-01-15T18:01:49.355 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-15T18:01:49.638 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-tempita. 2024-01-15T18:01:49.653 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-01-15T18:01:49.688 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-01-15T18:01:49.760 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libcephfs-dev (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:49.862 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rados (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:49.962 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up librgw2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:49.991 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-paste. 2024-01-15T18:01:50.006 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-01-15T18:01:50.049 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-01-15T18:01:50.114 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephfs-top (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:50.231 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-15T18:01:50.344 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-01-15T18:01:50.358 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-01-15T18:01:50.402 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-01-15T18:01:50.679 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastedeploy. 2024-01-15T18:01:50.694 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-01-15T18:01:50.738 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-01-15T18:01:51.024 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webtest. 2024-01-15T18:01:51.038 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-01-15T18:01:51.082 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-01-15T18:01:51.410 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pecan. 2024-01-15T18:01:51.425 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-01-15T18:01:51.468 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-01-15T18:01:51.494 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rbd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:51.595 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up rbd-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:51.696 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-rgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:51.813 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up python3-cephfs (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:51.831 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-certifi. 2024-01-15T18:01:51.846 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-01-15T18:01:51.889 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-01-15T18:01:51.965 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephfs-shell (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:52.091 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:52.242 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-idna. 2024-01-15T18:01:52.257 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-01-15T18:01:52.300 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-idna (3.3-1) ... 2024-01-15T18:01:52.603 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-urllib3. 2024-01-15T18:01:52.611 INFO:teuthology.orchestra.run.smithi204.stdout:Adding group ceph....done 2024-01-15T18:01:52.618 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-01-15T18:01:52.661 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-15T18:01:52.989 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-requests. 2024-01-15T18:01:53.004 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-01-15T18:01:53.029 INFO:teuthology.orchestra.run.smithi204.stdout:Adding system user ceph....done 2024-01-15T18:01:53.047 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-15T18:01:53.215 INFO:teuthology.orchestra.run.smithi204.stdout:Setting system user ceph properties....done 2024-01-15T18:01:53.235 INFO:teuthology.orchestra.run.smithi204.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-01-15T18:01:53.358 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-werkzeug. 2024-01-15T18:01:53.373 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-01-15T18:01:53.374 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-01-15T18:01:53.441 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-15T18:01:53.744 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-01-15T18:01:53.756 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-01-15T18:01:53.759 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:01:53.802 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-modules-core (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:54.198 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-01-15T18:01:54.213 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:54.256 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:54.385 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up cephfs-mirror (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:54.542 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-bcrypt. 2024-01-15T18:01:54.557 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-01-15T18:01:54.585 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-01-15T18:01:54.585 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-01-15T18:01:54.600 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-01-15T18:01:54.937 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-openssl. 2024-01-15T18:01:54.952 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-01-15T18:01:54.994 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-01-15T18:01:55.136 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-test (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:55.237 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up radosgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:55.348 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr. 2024-01-15T18:01:55.363 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../065-ceph-mgr_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:55.405 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:55.750 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-15T18:01:55.750 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-15T18:01:55.775 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mon. 2024-01-15T18:01:55.789 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../066-ceph-mon_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:55.825 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mon (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:56.297 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-base (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:56.612 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-osd. 2024-01-15T18:01:56.626 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../067-ceph-osd_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:56.663 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-01-15T18:01:56.670 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-osd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:57.275 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mds (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:57.512 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-15T18:01:57.512 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-15T18:01:57.848 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph. 2024-01-15T18:01:57.863 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../068-ceph_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:57.906 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:58.094 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:58.208 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-fuse. 2024-01-15T18:01:58.223 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../069-ceph-fuse_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:58.266 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:58.306 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-15T18:01:58.307 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-15T18:01:58.628 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mds. 2024-01-15T18:01:58.643 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../070-ceph-mds_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:58.686 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mds (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:58.912 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-osd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:59.055 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephadm. 2024-01-15T18:01:59.070 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../071-cephadm_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:01:59.105 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:59.192 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-15T18:01:59.192 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-15T18:01:59.349 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-asyncssh. 2024-01-15T18:01:59.364 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-01-15T18:01:59.407 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-01-15T18:01:59.789 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:59.914 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:01:59.985 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-babel-localedata. 2024-01-15T18:02:00.000 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-01-15T18:02:00.043 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-15T18:02:00.057 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mon (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:00.264 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-15T18:02:00.264 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-15T18:02:00.876 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:01.043 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:01.056 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-babel. 2024-01-15T18:02:01.070 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-01-15T18:02:01.113 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-01-15T18:02:01.169 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-mgr-dashboard (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:01.303 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ceph-volume (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:01.408 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jinja2. 2024-01-15T18:02:01.423 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2024-01-15T18:02:01.466 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2024-01-15T18:02:01.656 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-01-15T18:02:01.757 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-01-15T18:02:01.844 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-01-15T18:02:01.858 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:01.902 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:01.909 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-01-15T18:02:02.247 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-repoze.lru. 2024-01-15T18:02:02.261 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-01-15T18:02:02.305 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-01-15T18:02:02.591 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-routes. 2024-01-15T18:02:02.606 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-01-15T18:02:02.650 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-01-15T18:02:03.011 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-01-15T18:02:03.026 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:03.050 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-01-15T18:02:03.069 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-dashboard (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:03.497 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-01-15T18:02:03.511 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-01-15T18:02:03.555 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-15T18:02:03.736 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-01-15T18:02:03.741 DEBUG:teuthology.orchestra.run.smithi204:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-01-15T18:02:03.815 INFO:teuthology.orchestra.run.smithi204.stdout:Reading package lists... 2024-01-15T18:02:03.979 INFO:teuthology.orchestra.run.smithi204.stdout:Building dependency tree... 2024-01-15T18:02:03.980 INFO:teuthology.orchestra.run.smithi204.stdout:Reading state information... 2024-01-15T18:02:04.066 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-joblib. 2024-01-15T18:02:04.080 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-01-15T18:02:04.125 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-01-15T18:02:04.133 INFO:teuthology.orchestra.run.smithi204.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-01-15T18:02:04.133 INFO:teuthology.orchestra.run.smithi204.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-01-15T18:02:04.133 INFO:teuthology.orchestra.run.smithi204.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-01-15T18:02:04.133 INFO:teuthology.orchestra.run.smithi204.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-01-15T18:02:04.133 INFO:teuthology.orchestra.run.smithi204.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-01-15T18:02:04.133 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages were automatically installed and are no longer required: 2024-01-15T18:02:04.134 INFO:teuthology.orchestra.run.smithi204.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-15T18:02:04.134 INFO:teuthology.orchestra.run.smithi204.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-15T18:02:04.134 INFO:teuthology.orchestra.run.smithi204.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-15T18:02:04.134 INFO:teuthology.orchestra.run.smithi204.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-15T18:02:04.134 INFO:teuthology.orchestra.run.smithi204.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-15T18:02:04.134 INFO:teuthology.orchestra.run.smithi204.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-15T18:02:04.135 INFO:teuthology.orchestra.run.smithi204.stdout:The following additional packages will be installed: 2024-01-15T18:02:04.135 INFO:teuthology.orchestra.run.smithi204.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-01-15T18:02:04.135 INFO:teuthology.orchestra.run.smithi204.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-01-15T18:02:04.135 INFO:teuthology.orchestra.run.smithi204.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-01-15T18:02:04.135 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-01-15T18:02:04.136 INFO:teuthology.orchestra.run.smithi204.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-01-15T18:02:04.136 INFO:teuthology.orchestra.run.smithi204.stdout:Suggested packages: 2024-01-15T18:02:04.136 INFO:teuthology.orchestra.run.smithi204.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-01-15T18:02:04.136 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-01-15T18:02:04.136 INFO:teuthology.orchestra.run.smithi204.stdout: libengine-pkcs11-openssl 2024-01-15T18:02:04.185 INFO:teuthology.orchestra.run.smithi204.stdout:The following NEW packages will be installed: 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-01-15T18:02:04.187 INFO:teuthology.orchestra.run.smithi204.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-01-15T18:02:04.188 INFO:teuthology.orchestra.run.smithi204.stdout:The following packages will be upgraded: 2024-01-15T18:02:04.188 INFO:teuthology.orchestra.run.smithi204.stdout: libssl3 2024-01-15T18:02:04.398 INFO:teuthology.orchestra.run.smithi204.stdout:1 upgraded, 39 newly installed, 0 to remove and 268 not upgraded. 2024-01-15T18:02:04.398 INFO:teuthology.orchestra.run.smithi204.stdout:Need to get 54.6 MB of archives. 2024-01-15T18:02:04.398 INFO:teuthology.orchestra.run.smithi204.stdout:After this operation, 202 MB of additional disk space will be used. 2024-01-15T18:02:04.398 INFO:teuthology.orchestra.run.smithi204.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-01-15T18:02:04.511 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-01-15T18:02:04.525 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-01-15T18:02:04.641 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-01-15T18:02:04.892 INFO:teuthology.orchestra.run.smithi204.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2024-01-15T18:02:04.944 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn. 2024-01-15T18:02:04.958 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-01-15T18:02:05.002 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-15T18:02:05.150 INFO:teuthology.orchestra.run.smithi204.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-01-15T18:02:05.184 INFO:teuthology.orchestra.run.smithi204.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-01-15T18:02:05.186 INFO:teuthology.orchestra.run.smithi204.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-01-15T18:02:05.191 INFO:teuthology.orchestra.run.smithi204.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-01-15T18:02:05.194 INFO:teuthology.orchestra.run.smithi204.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-01-15T18:02:05.195 INFO:teuthology.orchestra.run.smithi204.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-01-15T18:02:05.198 INFO:teuthology.orchestra.run.smithi204.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-01-15T18:02:05.200 INFO:teuthology.orchestra.run.smithi204.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-01-15T18:02:05.201 INFO:teuthology.orchestra.run.smithi204.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-01-15T18:02:05.365 INFO:teuthology.orchestra.run.smithi204.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-01-15T18:02:05.621 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-01-15T18:02:05.635 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:05.671 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:05.700 INFO:teuthology.orchestra.run.smithi204.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-01-15T18:02:06.632 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cachetools. 2024-01-15T18:02:06.648 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-01-15T18:02:06.690 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-01-15T18:02:06.764 INFO:teuthology.orchestra.run.smithi204.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-01-15T18:02:06.766 INFO:teuthology.orchestra.run.smithi204.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-01-15T18:02:06.767 INFO:teuthology.orchestra.run.smithi204.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2024-01-15T18:02:06.769 INFO:teuthology.orchestra.run.smithi204.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-01-15T18:02:06.770 INFO:teuthology.orchestra.run.smithi204.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-01-15T18:02:06.774 INFO:teuthology.orchestra.run.smithi204.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2024-01-15T18:02:06.775 INFO:teuthology.orchestra.run.smithi204.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-01-15T18:02:06.794 INFO:teuthology.orchestra.run.smithi204.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-01-15T18:02:06.794 INFO:teuthology.orchestra.run.smithi204.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2024-01-15T18:02:06.994 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1. 2024-01-15T18:02:07.009 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-01-15T18:02:07.036 INFO:teuthology.orchestra.run.smithi204.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-01-15T18:02:07.052 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-01-15T18:02:07.118 INFO:teuthology.orchestra.run.smithi204.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2024-01-15T18:02:07.406 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-01-15T18:02:07.421 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-01-15T18:02:07.464 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-01-15T18:02:07.591 INFO:teuthology.orchestra.run.smithi204.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-01-15T18:02:07.725 INFO:teuthology.orchestra.run.smithi204.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-01-15T18:02:07.735 INFO:teuthology.orchestra.run.smithi204.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-01-15T18:02:07.736 INFO:teuthology.orchestra.run.smithi204.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-01-15T18:02:07.737 INFO:teuthology.orchestra.run.smithi204.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-01-15T18:02:07.738 INFO:teuthology.orchestra.run.smithi204.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2024-01-15T18:02:07.759 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rsa. 2024-01-15T18:02:07.771 INFO:teuthology.orchestra.run.smithi204.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-01-15T18:02:07.774 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-01-15T18:02:07.778 INFO:teuthology.orchestra.run.smithi204.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2024-01-15T18:02:07.808 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rsa (4.8-1) ... 2024-01-15T18:02:08.071 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-google-auth. 2024-01-15T18:02:08.086 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-01-15T18:02:08.121 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-01-15T18:02:08.216 INFO:teuthology.orchestra.run.smithi204.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-01-15T18:02:08.380 INFO:teuthology.orchestra.run.smithi204.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-01-15T18:02:08.432 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-01-15T18:02:08.448 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-01-15T18:02:08.462 INFO:teuthology.orchestra.run.smithi204.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-01-15T18:02:08.462 INFO:teuthology.orchestra.run.smithi204.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-01-15T18:02:08.491 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-15T18:02:08.548 INFO:teuthology.orchestra.run.smithi204.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-01-15T18:02:08.558 INFO:teuthology.orchestra.run.smithi204.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-01-15T18:02:08.811 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-websocket. 2024-01-15T18:02:08.826 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-01-15T18:02:08.869 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-01-15T18:02:09.019 INFO:teuthology.orchestra.run.smithi204.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-01-15T18:02:09.021 INFO:teuthology.orchestra.run.smithi204.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-01-15T18:02:09.189 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-kubernetes. 2024-01-15T18:02:09.204 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-01-15T18:02:09.247 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-15T18:02:09.401 INFO:teuthology.orchestra.run.smithi204.stdout:Preconfiguring packages ... 2024-01-15T18:02:09.561 INFO:teuthology.orchestra.run.smithi204.stdout:Fetched 54.6 MB in 5s (11.3 MB/s) 2024-01-15T18:02:09.678 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package flex. 2024-01-15T18:02:09.748 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140566 files and directories currently installed.) 2024-01-15T18:02:09.752 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-01-15T18:02:09.802 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking flex (2.6.4-8build2) ... 2024-01-15T18:02:09.928 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-01-15T18:02:09.942 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:09.986 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:10.331 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libonig5:amd64. 2024-01-15T18:02:10.345 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-01-15T18:02:10.389 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-15T18:02:10.406 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2024-01-15T18:02:10.718 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjq1:amd64. 2024-01-15T18:02:10.732 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-01-15T18:02:10.768 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-15T18:02:10.817 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2024-01-15T18:02:11.021 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package jq. 2024-01-15T18:02:11.035 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-01-15T18:02:11.079 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-01-15T18:02:11.216 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2024-01-15T18:02:11.331 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package socat. 2024-01-15T18:02:11.347 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-01-15T18:02:11.415 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-01-15T18:02:11.726 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package xmlstarlet. 2024-01-15T18:02:11.742 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-01-15T18:02:11.743 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package bison. 2024-01-15T18:02:11.784 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-01-15T18:02:11.813 INFO:teuthology.orchestra.run.smithi204.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140639 files and directories currently installed.) 2024-01-15T18:02:11.817 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-test. 2024-01-15T18:02:12.187 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../099-ceph-test_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:02:12.239 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-test (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:12.263 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-01-15T18:02:12.277 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-01-15T18:02:12.321 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-01-15T18:02:12.608 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package dump. 2024-01-15T18:02:12.622 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-01-15T18:02:12.666 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking dump (0.4b47-3) ... 2024-01-15T18:02:12.953 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package indent. 2024-01-15T18:02:12.967 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:02:13.012 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-01-15T18:02:13.603 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-01-15T18:02:13.617 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-01-15T18:02:13.657 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-01-15T18:02:13.986 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-01-15T18:02:14.001 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-01-15T18:02:14.035 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-01-15T18:02:14.236 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-volume. 2024-01-15T18:02:14.251 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../100-ceph-volume_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:14.271 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-01-15T18:02:14.287 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-01-15T18:02:14.293 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-volume (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:14.330 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-01-15T18:02:14.665 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-mirror. 2024-01-15T18:02:14.675 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libfl2:amd64. 2024-01-15T18:02:14.681 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../101-cephfs-mirror_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:02:14.691 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-01-15T18:02:14.723 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-mirror (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:14.733 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-01-15T18:02:15.011 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-01-15T18:02:15.027 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-01-15T18:02:15.034 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-colorama. 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-01-15T18:02:15.069 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-01-15T18:02:15.093 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-01-15T18:02:15.271 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libjson-perl. 2024-01-15T18:02:15.287 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-01-15T18:02:15.321 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-01-15T18:02:15.608 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libllvm14:amd64. 2024-01-15T18:02:15.624 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-01-15T18:02:15.629 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-shell. 2024-01-15T18:02:15.644 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../103-cephfs-shell_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:15.683 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-01-15T18:02:15.687 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-shell (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:15.932 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephfs-top. 2024-01-15T18:02:15.947 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../104-cephfs-top_18.0.0-7629-g95c90f41-1jammy_all.deb ... 2024-01-15T18:02:15.981 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephfs-top (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:16.226 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs-dev. 2024-01-15T18:02:16.241 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../105-libcephfs-dev_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:02:16.284 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs-dev (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:16.587 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package lua-socket:amd64. 2024-01-15T18:02:16.602 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-01-15T18:02:16.645 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-15T18:02:17.000 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package lua-sec:amd64. 2024-01-15T18:02:17.015 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-01-15T18:02:17.052 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-01-15T18:02:17.058 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-01-15T18:02:17.069 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-01-15T18:02:17.111 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:02:17.389 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmbim-proxy. 2024-01-15T18:02:17.395 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package nvme-cli. 2024-01-15T18:02:17.405 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-01-15T18:02:17.411 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-01-15T18:02:17.462 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-01-15T18:02:17.481 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:02:17.808 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-01-15T18:02:17.824 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2024-01-15T18:02:17.866 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:02:17.876 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package pkg-config. 2024-01-15T18:02:17.891 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-01-15T18:02:17.934 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-01-15T18:02:18.152 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libndp0:amd64. 2024-01-15T18:02:18.168 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-01-15T18:02:18.196 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-01-15T18:02:18.202 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-01-15T18:02:18.211 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-01-15T18:02:18.287 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-01-15T18:02:18.538 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libnm0:amd64. 2024-01-15T18:02:18.554 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-01-15T18:02:18.640 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-01-15T18:02:18.707 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-iniconfig. 2024-01-15T18:02:18.722 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-01-15T18:02:18.766 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-01-15T18:02:18.917 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libpq5:amd64. 2024-01-15T18:02:18.933 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:02:18.967 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:02:19.068 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastescript. 2024-01-15T18:02:19.083 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-01-15T18:02:19.126 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-01-15T18:02:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-01-15T18:02:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:02:19.395 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:02:19.438 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pluggy. 2024-01-15T18:02:19.455 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-01-15T18:02:19.488 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-01-15T18:02:19.698 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libqmi-proxy. 2024-01-15T18:02:19.714 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:02:19.756 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:02:19.817 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-psutil. 2024-01-15T18:02:19.833 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-01-15T18:02:19.875 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-01-15T18:02:20.009 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-01-15T18:02:20.025 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2024-01-15T18:02:20.059 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2024-01-15T18:02:20.153 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-py. 2024-01-15T18:02:20.169 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-01-15T18:02:20.202 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-py (1.10.0-1) ... 2024-01-15T18:02:20.540 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pygments. 2024-01-15T18:02:20.555 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-01-15T18:02:20.596 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-01-15T18:02:20.598 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-01-15T18:02:20.611 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-01-15T18:02:20.654 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-01-15T18:02:21.007 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package modemmanager. 2024-01-15T18:02:21.023 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2024-01-15T18:02:21.060 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyinotify. 2024-01-15T18:02:21.065 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:02:21.075 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-01-15T18:02:21.118 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-01-15T18:02:21.455 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-toml. 2024-01-15T18:02:21.470 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-01-15T18:02:21.513 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-01-15T18:02:21.738 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package network-manager. 2024-01-15T18:02:21.754 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-01-15T18:02:21.791 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pytest. 2024-01-15T18:02:21.804 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-01-15T18:02:21.807 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-01-15T18:02:21.849 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-01-15T18:02:22.187 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplejson. 2024-01-15T18:02:22.202 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-01-15T18:02:22.244 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-01-15T18:02:22.454 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package ppp. 2024-01-15T18:02:22.470 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-01-15T18:02:22.539 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-01-15T18:02:22.555 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-01-15T18:02:22.597 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-01-15T18:02:22.783 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-01-15T18:02:23.151 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package pptp-linux. 2024-01-15T18:02:23.165 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-01-15T18:02:23.209 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-01-15T18:02:23.277 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package radosgw. 2024-01-15T18:02:23.291 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../122-radosgw_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:02:23.335 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking radosgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:23.462 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package network-manager-pptp. 2024-01-15T18:02:23.476 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-01-15T18:02:23.520 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-01-15T18:02:23.905 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package rbd-fuse. 2024-01-15T18:02:23.920 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../123-rbd-fuse_18.0.0-7629-g95c90f41-1jammy_amd64.deb ... 2024-01-15T18:02:23.949 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-client-common. 2024-01-15T18:02:23.955 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking rbd-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:23.964 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-01-15T18:02:24.008 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-client-common (238) ... 2024-01-15T18:02:24.241 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-01-15T18:02:24.270 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-client-14. 2024-01-15T18:02:24.284 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:02:24.320 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:02:24.510 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-01-15T18:02:24.636 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-01-15T18:02:24.899 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-01-15T18:02:24.941 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-common. 2024-01-15T18:02:24.956 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-01-15T18:02:25.025 INFO:teuthology.orchestra.run.smithi204.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-01-15T18:02:25.056 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-common (238) ... 2024-01-15T18:02:25.287 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-01-15T18:02:25.465 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-14. 2024-01-15T18:02:25.480 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:02:25.518 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:02:25.838 INFO:teuthology.orchestra.run.smithi143.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-01-15T18:02:25.886 INFO:teuthology.orchestra.run.smithi143.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-01-15T18:02:25.930 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:26.450 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user cephadm....done 2024-01-15T18:02:26.636 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql. 2024-01-15T18:02:26.636 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-01-15T18:02:26.651 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-01-15T18:02:26.694 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql (14+238) ... 2024-01-15T18:02:26.921 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-01-15T18:02:26.921 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-client. 2024-01-15T18:02:26.936 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-01-15T18:02:26.971 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-client (14+238) ... 2024-01-15T18:02:27.165 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package postgresql-contrib. 2024-01-15T18:02:27.172 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-01-15T18:02:27.180 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-01-15T18:02:27.215 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking postgresql-contrib (14+238) ... 2024-01-15T18:02:27.273 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-01-15T18:02:27.524 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-01-15T18:02:27.542 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package tcl8.6. 2024-01-15T18:02:27.558 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-01-15T18:02:27.601 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-01-15T18:02:27.784 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-01-15T18:02:27.845 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package tcl. 2024-01-15T18:02:27.860 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-01-15T18:02:27.900 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-01-15T18:02:27.935 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-py (1.10.0-1) ... 2024-01-15T18:02:28.223 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package wpasupplicant. 2024-01-15T18:02:28.239 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-01-15T18:02:28.254 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-01-15T18:02:28.281 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-01-15T18:02:28.606 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-01-15T18:02:28.678 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-01-15T18:02:28.693 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-01-15T18:02:28.736 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-01-15T18:02:28.857 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-01-15T18:02:29.064 INFO:teuthology.orchestra.run.smithi204.stdout:Selecting previously unselected package usb-modeswitch. 2024-01-15T18:02:29.080 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-01-15T18:02:29.122 INFO:teuthology.orchestra.run.smithi204.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-01-15T18:02:29.385 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2024-01-15T18:02:29.399 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-client-common (238) ... 2024-01-15T18:02:29.579 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up flex (2.6.4-8build2) ... 2024-01-15T18:02:29.661 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-01-15T18:02:29.705 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-01-15T18:02:29.831 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:02:29.921 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-01-15T18:02:29.957 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-01-15T18:02:30.083 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-01-15T18:02:30.227 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:02:30.339 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-01-15T18:02:30.816 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-argparse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:31.092 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-01-15T18:02:31.179 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-01-15T18:02:31.217 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-01-15T18:02:31.281 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-01-15T18:02:31.344 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-01-15T18:02:31.628 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-01-15T18:02:31.667 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-01-15T18:02:31.667 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-01-15T18:02:31.729 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-01-15T18:02:31.998 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-01-15T18:02:32.310 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-01-15T18:02:32.419 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2024-01-15T18:02:32.480 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-01-15T18:02:32.520 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-01-15T18:02:32.536 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-01-15T18:02:32.609 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-01-15T18:02:32.646 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-01-15T18:02:32.735 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-01-15T18:02:32.772 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-01-15T18:02:32.898 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-01-15T18:02:33.024 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-01-15T18:02:33.128 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-01-15T18:02:33.279 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-01-15T18:02:33.573 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-01-15T18:02:33.630 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-01-15T18:02:33.756 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up dump (0.4b47-3) ... 2024-01-15T18:02:33.851 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-01-15T18:02:33.866 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-01-15T18:02:33.932 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:02:34.217 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up zip (3.0-12build2) ... 2024-01-15T18:02:34.343 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-01-15T18:02:34.409 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-01-15T18:02:34.535 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up tcl (8.6.11+1build2) ... 2024-01-15T18:02:34.636 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libjson-perl (4.04000-1) ... 2024-01-15T18:02:34.737 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-01-15T18:02:34.811 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-01-15T18:02:34.880 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-01-15T18:02:35.006 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-01-15T18:02:35.037 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-01-15T18:02:35.183 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-01-15T18:02:35.305 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-01-15T18:02:35.309 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:02:35.435 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-01-15T18:02:35.456 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-01-15T18:02:35.578 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-client (14+238) ... 2024-01-15T18:02:35.607 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-01-15T18:02:35.679 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-01-15T18:02:35.942 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-01-15T18:02:36.254 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:02:36.254 INFO:teuthology.orchestra.run.smithi204.stdout:The following network interfaces were found in /etc/network/interfaces 2024-01-15T18:02:36.255 INFO:teuthology.orchestra.run.smithi204.stdout:which means they are currently configured by ifupdown: 2024-01-15T18:02:36.255 INFO:teuthology.orchestra.run.smithi204.stdout:- enp3s0f1 2024-01-15T18:02:36.255 INFO:teuthology.orchestra.run.smithi204.stdout:If you want to manage those interfaces with NetworkManager instead 2024-01-15T18:02:36.255 INFO:teuthology.orchestra.run.smithi204.stdout:remove their configuration from /etc/network/interfaces. 2024-01-15T18:02:36.255 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:02:36.351 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-01-15T18:02:36.387 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-01-15T18:02:36.611 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-01-15T18:02:36.866 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-01-15T18:02:37.013 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-01-15T18:02:37.266 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-01-15T18:02:37.331 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-01-15T18:02:37.691 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-01-15T18:02:37.983 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:02:38.067 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-01-15T18:02:38.195 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-01-15T18:02:38.329 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-01-15T18:02:38.335 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-01-15T18:02:38.452 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:38.455 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-common (238) ... 2024-01-15T18:02:38.746 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-idna (3.3-1) ... 2024-01-15T18:02:39.097 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-01-15T18:02:39.386 INFO:teuthology.orchestra.run.smithi204.stdout:Adding user postgres to group ssl-cert 2024-01-15T18:02:39.628 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:02:39.628 INFO:teuthology.orchestra.run.smithi204.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-01-15T18:02:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-01-15T18:02:39.882 INFO:teuthology.orchestra.run.smithi204.stdout:Removing obsolete dictionary files: 2024-01-15T18:02:40.345 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-01-15T18:02:40.579 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-01-15T18:02:40.630 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:02:40.765 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-toml (0.10.2-1) ... 2024-01-15T18:02:41.032 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:02:41.041 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-01-15T18:02:41.183 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-01-15T18:02:41.310 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-01-15T18:02:41.370 INFO:teuthology.orchestra.run.smithi204.stdout:Creating new PostgreSQL cluster 14/main ... 2024-01-15T18:02:41.393 INFO:teuthology.orchestra.run.smithi204.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-01-15T18:02:41.444 INFO:teuthology.orchestra.run.smithi204.stdout:The files belonging to this database system will be owned by user "postgres". 2024-01-15T18:02:41.444 INFO:teuthology.orchestra.run.smithi204.stdout:This user must also own the server process. 2024-01-15T18:02:41.444 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:02:41.444 INFO:teuthology.orchestra.run.smithi204.stdout:The database cluster will be initialized with locale "en_US". 2024-01-15T18:02:41.445 INFO:teuthology.orchestra.run.smithi204.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-01-15T18:02:41.445 INFO:teuthology.orchestra.run.smithi204.stdout:The default text search configuration will be set to "english". 2024-01-15T18:02:41.445 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:02:41.445 INFO:teuthology.orchestra.run.smithi204.stdout:Data page checksums are disabled. 2024-01-15T18:02:41.445 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:02:41.445 INFO:teuthology.orchestra.run.smithi204.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-01-15T18:02:41.446 INFO:teuthology.orchestra.run.smithi204.stdout:creating subdirectories ... ok 2024-01-15T18:02:41.446 INFO:teuthology.orchestra.run.smithi204.stdout:selecting dynamic shared memory implementation ... posix 2024-01-15T18:02:41.469 INFO:teuthology.orchestra.run.smithi204.stdout:selecting default max_connections ... 100 2024-01-15T18:02:41.503 INFO:teuthology.orchestra.run.smithi204.stdout:selecting default shared_buffers ... 128MB 2024-01-15T18:02:41.516 INFO:teuthology.orchestra.run.smithi204.stdout:selecting default time zone ... Etc/UTC 2024-01-15T18:02:41.518 INFO:teuthology.orchestra.run.smithi204.stdout:creating configuration files ... ok 2024-01-15T18:02:41.611 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-01-15T18:02:41.681 INFO:teuthology.orchestra.run.smithi204.stdout:running bootstrap script ... ok 2024-01-15T18:02:41.879 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-01-15T18:02:42.156 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:02:42.282 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-01-15T18:02:42.308 INFO:teuthology.orchestra.run.smithi204.stdout:performing post-bootstrap initialization ... ok 2024-01-15T18:02:42.600 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-01-15T18:02:42.863 INFO:teuthology.orchestra.run.smithi204.stdout:syncing data to disk ... ok 2024-01-15T18:02:42.868 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-01-15T18:02:43.144 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-01-15T18:02:43.362 INFO:teuthology.orchestra.run.smithi204.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-01-15T18:02:43.445 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-01-15T18:02:43.713 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-01-15T18:02:44.006 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-01-15T18:02:44.157 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-01-15T18:02:44.475 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-01-15T18:02:44.626 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-01-15T18:02:44.927 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-01-15T18:02:45.245 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-01-15T18:02:45.547 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-01-15T18:02:45.698 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-01-15T18:02:45.849 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-01-15T18:02:45.979 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-01-15T18:02:46.226 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-01-15T18:02:46.352 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-01-15T18:02:46.373 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:02:46.474 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql-contrib (14+238) ... 2024-01-15T18:02:46.762 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-01-15T18:02:46.984 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up postgresql (14+238) ... 2024-01-15T18:02:47.055 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up lua-any (27ubuntu1) ... 2024-01-15T18:02:47.110 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:02:47.206 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-portend (3.0.0-1) ... 2024-01-15T18:02:47.236 INFO:teuthology.orchestra.run.smithi204.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:02:47.494 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-01-15T18:02:47.495 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-01-15T18:02:47.524 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-01-15T18:02:47.675 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2024-01-15T18:02:48.010 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-01-15T18:02:48.247 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-01-15T18:02:48.319 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-01-15T18:02:48.590 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-01-15T18:02:48.612 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-01-15T18:02:48.758 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-01-15T18:02:48.874 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rsa (4.8-1) ... 2024-01-15T18:02:49.167 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-01-15T18:02:49.460 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-01-15T18:02:49.728 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-01-15T18:02:49.784 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-01-15T18:02:49.876 INFO:teuthology.orchestra.run.smithi204.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-01-15T18:02:50.133 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-01-15T18:02:50.384 INFO:teuthology.orchestra.run.smithi204.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-01-15T18:02:50.393 DEBUG:teuthology.parallel:result is None 2024-01-15T18:02:50.426 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-01-15T18:02:50.778 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-01-15T18:02:50.929 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librados2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:51.331 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libsqlite3-mod-ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:51.515 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-01-15T18:02:51.885 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:52.019 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libradosstriper1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:52.170 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-01-15T18:02:52.480 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librbd1 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:52.607 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-modules-core (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:52.733 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:52.975 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-15T18:02:52.976 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-01-15T18:02:53.593 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs-dev (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:53.719 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rados (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:54.226 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librgw2 (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:54.355 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-top (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:54.481 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-01-15T18:02:55.761 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rbd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:55.886 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up rbd-fuse (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:56.013 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:56.139 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cephfs (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:56.273 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-shell (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:56.399 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-common (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:56.894 INFO:teuthology.orchestra.run.smithi143.stdout:Adding group ceph....done 2024-01-15T18:02:57.288 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user ceph....done 2024-01-15T18:02:57.448 INFO:teuthology.orchestra.run.smithi143.stdout:Setting system user ceph properties....done 2024-01-15T18:02:57.465 INFO:teuthology.orchestra.run.smithi143.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-01-15T18:02:57.605 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-01-15T18:02:57.986 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-01-15T18:02:58.626 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephfs-mirror (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:58.851 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-01-15T18:02:58.851 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-01-15T18:02:59.412 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-test (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:02:59.563 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up radosgw (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:00.115 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-15T18:03:00.115 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-01-15T18:03:00.673 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-base (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:01.051 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-01-15T18:03:01.676 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mds (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:01.889 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-15T18:03:01.889 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-01-15T18:03:02.485 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:02.700 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-15T18:03:02.700 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-01-15T18:03:03.282 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-osd (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:03.615 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-15T18:03:03.615 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-01-15T18:03:04.189 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-k8sevents (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:04.341 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:04.475 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mon (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:04.685 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-15T18:03:04.685 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-01-15T18:03:05.285 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-cephadm (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:05.452 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:05.603 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-dashboard (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:05.763 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-volume (18.0.0-7629-g95c90f41-1jammy) ... 2024-01-15T18:03:06.126 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-01-15T18:03:06.242 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-01-15T18:03:06.419 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-01-15T18:03:07.555 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-01-15T18:03:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-01-15T18:03:08.290 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-01-15T18:03:08.364 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2024-01-15T18:03:08.528 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2024-01-15T18:03:08.529 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-01-15T18:03:08.681 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2024-01-15T18:03:08.682 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2024-01-15T18:03:08.682 INFO:teuthology.orchestra.run.smithi143.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-01-15T18:03:08.683 INFO:teuthology.orchestra.run.smithi143.stdout: libengine-pkcs11-openssl 2024-01-15T18:03:08.732 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2024-01-15T18:03:08.733 INFO:teuthology.orchestra.run.smithi143.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-01-15T18:03:08.733 INFO:teuthology.orchestra.run.smithi143.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-01-15T18:03:08.733 INFO:teuthology.orchestra.run.smithi143.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-01-15T18:03:08.734 INFO:teuthology.orchestra.run.smithi143.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-01-15T18:03:08.734 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-01-15T18:03:08.734 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-01-15T18:03:08.734 INFO:teuthology.orchestra.run.smithi143.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-01-15T18:03:08.734 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages will be upgraded: 2024-01-15T18:03:08.735 INFO:teuthology.orchestra.run.smithi143.stdout: libssl3 2024-01-15T18:03:08.971 INFO:teuthology.orchestra.run.smithi143.stdout:1 upgraded, 39 newly installed, 0 to remove and 268 not upgraded. 2024-01-15T18:03:08.971 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 54.6 MB of archives. 2024-01-15T18:03:08.971 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 202 MB of additional disk space will be used. 2024-01-15T18:03:08.972 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-01-15T18:03:09.467 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2024-01-15T18:03:09.725 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-01-15T18:03:09.759 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-01-15T18:03:09.761 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-01-15T18:03:09.766 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-01-15T18:03:09.770 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-01-15T18:03:09.770 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-01-15T18:03:09.773 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-01-15T18:03:09.776 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-01-15T18:03:09.776 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-01-15T18:03:09.941 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-01-15T18:03:10.277 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-01-15T18:03:11.349 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-01-15T18:03:11.351 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-01-15T18:03:11.351 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2024-01-15T18:03:11.354 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-01-15T18:03:11.354 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-01-15T18:03:11.358 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2024-01-15T18:03:11.360 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-01-15T18:03:11.379 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-01-15T18:03:11.380 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2024-01-15T18:03:11.625 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-01-15T18:03:11.707 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2024-01-15T18:03:12.186 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-01-15T18:03:12.316 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-01-15T18:03:12.327 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-01-15T18:03:12.327 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-01-15T18:03:12.329 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-01-15T18:03:12.329 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2024-01-15T18:03:12.364 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-01-15T18:03:12.367 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2024-01-15T18:03:12.809 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-01-15T18:03:12.972 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-01-15T18:03:13.054 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-01-15T18:03:13.056 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-01-15T18:03:13.136 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-01-15T18:03:13.144 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-01-15T18:03:13.612 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-01-15T18:03:13.614 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-01-15T18:03:13.995 INFO:teuthology.orchestra.run.smithi143.stdout:Preconfiguring packages ... 2024-01-15T18:03:14.153 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 54.6 MB in 5s (11.2 MB/s) 2024-01-15T18:03:14.296 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package flex. 2024-01-15T18:03:14.366 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140566 files and directories currently installed.) 2024-01-15T18:03:14.369 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-01-15T18:03:14.419 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking flex (2.6.4-8build2) ... 2024-01-15T18:03:15.057 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2024-01-15T18:03:15.227 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2024-01-15T18:03:15.608 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2024-01-15T18:03:16.137 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package bison. 2024-01-15T18:03:16.206 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140639 files and directories currently installed.) 2024-01-15T18:03:16.210 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-01-15T18:03:16.297 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-01-15T18:03:16.657 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-01-15T18:03:16.671 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-01-15T18:03:16.707 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-01-15T18:03:17.277 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package dump. 2024-01-15T18:03:17.292 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-01-15T18:03:17.335 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking dump (0.4b47-3) ... 2024-01-15T18:03:17.656 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package indent. 2024-01-15T18:03:17.672 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:03:17.714 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-01-15T18:03:17.975 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-01-15T18:03:17.991 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-01-15T18:03:18.033 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-01-15T18:03:18.304 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-01-15T18:03:18.320 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-01-15T18:03:18.362 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-01-15T18:03:18.624 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-01-15T18:03:18.639 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-01-15T18:03:18.682 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-01-15T18:03:19.052 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libfl2:amd64. 2024-01-15T18:03:19.068 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-01-15T18:03:19.110 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-01-15T18:03:19.329 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-01-15T18:03:19.345 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-01-15T18:03:19.379 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-01-15T18:03:19.632 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjson-perl. 2024-01-15T18:03:19.648 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-01-15T18:03:19.690 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-01-15T18:03:19.985 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libllvm14:amd64. 2024-01-15T18:03:20.001 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-01-15T18:03:20.043 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-01-15T18:03:21.380 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-01-15T18:03:21.395 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-01-15T18:03:21.429 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:03:21.683 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libmbim-proxy. 2024-01-15T18:03:21.699 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-01-15T18:03:21.774 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:03:22.155 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-01-15T18:03:22.171 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2024-01-15T18:03:22.213 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:03:22.508 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libndp0:amd64. 2024-01-15T18:03:22.524 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-01-15T18:03:22.566 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-01-15T18:03:22.885 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libnm0:amd64. 2024-01-15T18:03:22.902 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-01-15T18:03:22.944 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-01-15T18:03:23.313 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libpq5:amd64. 2024-01-15T18:03:23.329 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:03:23.371 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:03:23.658 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-01-15T18:03:23.674 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:03:23.716 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:03:24.011 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqmi-proxy. 2024-01-15T18:03:24.026 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:03:24.069 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:03:24.313 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-01-15T18:03:24.329 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2024-01-15T18:03:24.371 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2024-01-15T18:03:24.875 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-01-15T18:03:24.890 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-01-15T18:03:24.933 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-01-15T18:03:25.253 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package modemmanager. 2024-01-15T18:03:25.269 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2024-01-15T18:03:25.303 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:03:25.968 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package network-manager. 2024-01-15T18:03:25.984 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-01-15T18:03:26.034 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-01-15T18:03:26.651 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ppp. 2024-01-15T18:03:26.666 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-01-15T18:03:27.013 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-01-15T18:03:27.491 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package pptp-linux. 2024-01-15T18:03:27.505 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-01-15T18:03:27.541 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-01-15T18:03:27.768 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package network-manager-pptp. 2024-01-15T18:03:27.783 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-01-15T18:03:27.827 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-01-15T18:03:28.130 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-client-common. 2024-01-15T18:03:28.144 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-01-15T18:03:28.188 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-client-common (238) ... 2024-01-15T18:03:28.476 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-client-14. 2024-01-15T18:03:28.490 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:03:28.526 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:03:29.273 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-common. 2024-01-15T18:03:29.287 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-01-15T18:03:29.348 INFO:teuthology.orchestra.run.smithi143.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-01-15T18:03:29.379 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-common (238) ... 2024-01-15T18:03:29.863 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-14. 2024-01-15T18:03:29.878 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2024-01-15T18:03:29.924 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:03:30.984 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql. 2024-01-15T18:03:30.999 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-01-15T18:03:31.043 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql (14+238) ... 2024-01-15T18:03:31.270 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-client. 2024-01-15T18:03:31.285 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-01-15T18:03:31.320 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-client (14+238) ... 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package postgresql-contrib. 2024-01-15T18:03:31.587 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-01-15T18:03:31.631 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking postgresql-contrib (14+238) ... 2024-01-15T18:03:31.951 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package tcl8.6. 2024-01-15T18:03:31.966 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-01-15T18:03:32.009 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-01-15T18:03:32.287 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package tcl. 2024-01-15T18:03:32.302 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-01-15T18:03:32.341 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package wpasupplicant. 2024-01-15T18:03:32.664 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-01-15T18:03:32.698 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-01-15T18:03:33.086 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-01-15T18:03:33.102 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-01-15T18:03:33.144 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-01-15T18:03:33.523 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package usb-modeswitch. 2024-01-15T18:03:33.539 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-01-15T18:03:33.589 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-01-15T18:03:33.866 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-client-common (238) ... 2024-01-15T18:03:34.079 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up flex (2.6.4-8build2) ... 2024-01-15T18:03:34.205 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-01-15T18:03:34.356 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:03:34.482 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-01-15T18:03:34.600 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-01-15T18:03:34.701 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:03:35.655 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-01-15T18:03:35.757 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-01-15T18:03:36.126 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-01-15T18:03:36.127 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-01-15T18:03:36.769 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-01-15T18:03:36.886 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2024-01-15T18:03:37.038 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-01-15T18:03:37.164 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-01-15T18:03:37.307 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-01-15T18:03:37.433 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-01-15T18:03:37.559 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-01-15T18:03:38.190 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-01-15T18:03:38.324 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up dump (0.4b47-3) ... 2024-01-15T18:03:38.445 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-01-15T18:03:38.501 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:03:38.627 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-01-15T18:03:38.737 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up tcl (8.6.11+1build2) ... 2024-01-15T18:03:38.854 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjson-perl (4.04000-1) ... 2024-01-15T18:03:38.997 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-01-15T18:03:39.087 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-01-15T18:03:39.159 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-01-15T18:03:39.285 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-01-15T18:03:39.511 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-01-15T18:03:39.612 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:03:39.771 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-01-15T18:03:39.939 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-client (14+238) ... 2024-01-15T18:03:40.065 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-01-15T18:03:40.650 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:40.650 INFO:teuthology.orchestra.run.smithi143.stdout:The following network interfaces were found in /etc/network/interfaces 2024-01-15T18:03:40.650 INFO:teuthology.orchestra.run.smithi143.stdout:which means they are currently configured by ifupdown: 2024-01-15T18:03:40.650 INFO:teuthology.orchestra.run.smithi143.stdout:- enp3s0f1 2024-01-15T18:03:40.650 INFO:teuthology.orchestra.run.smithi143.stdout:If you want to manage those interfaces with NetworkManager instead 2024-01-15T18:03:40.651 INFO:teuthology.orchestra.run.smithi143.stdout:remove their configuration from /etc/network/interfaces. 2024-01-15T18:03:40.651 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-01-15T18:03:41.164 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-01-15T18:03:41.545 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-01-15T18:03:42.201 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-01-15T18:03:42.335 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-01-15T18:03:42.778 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-01-15T18:03:42.904 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-common (238) ... 2024-01-15T18:03:43.859 INFO:teuthology.orchestra.run.smithi143.stdout:Adding user postgres to group ssl-cert 2024-01-15T18:03:44.121 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:44.121 INFO:teuthology.orchestra.run.smithi143.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-01-15T18:03:44.386 INFO:teuthology.orchestra.run.smithi143.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-01-15T18:03:44.387 INFO:teuthology.orchestra.run.smithi143.stdout:Removing obsolete dictionary files: 2024-01-15T18:03:45.067 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-01-15T18:03:45.538 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2024-01-15T18:03:45.874 INFO:teuthology.orchestra.run.smithi143.stdout:Creating new PostgreSQL cluster 14/main ... 2024-01-15T18:03:45.896 INFO:teuthology.orchestra.run.smithi143.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-01-15T18:03:45.937 INFO:teuthology.orchestra.run.smithi143.stdout:The files belonging to this database system will be owned by user "postgres". 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout:This user must also own the server process. 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout:The database cluster will be initialized with locale "en_US". 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout:The default text search configuration will be set to "english". 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout:Data page checksums are disabled. 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:45.938 INFO:teuthology.orchestra.run.smithi143.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-01-15T18:03:45.939 INFO:teuthology.orchestra.run.smithi143.stdout:creating subdirectories ... ok 2024-01-15T18:03:45.939 INFO:teuthology.orchestra.run.smithi143.stdout:selecting dynamic shared memory implementation ... posix 2024-01-15T18:03:45.962 INFO:teuthology.orchestra.run.smithi143.stdout:selecting default max_connections ... 100 2024-01-15T18:03:45.996 INFO:teuthology.orchestra.run.smithi143.stdout:selecting default shared_buffers ... 128MB 2024-01-15T18:03:46.010 INFO:teuthology.orchestra.run.smithi143.stdout:selecting default time zone ... Etc/UTC 2024-01-15T18:03:46.011 INFO:teuthology.orchestra.run.smithi143.stdout:creating configuration files ... ok 2024-01-15T18:03:46.174 INFO:teuthology.orchestra.run.smithi143.stdout:running bootstrap script ... ok 2024-01-15T18:03:46.808 INFO:teuthology.orchestra.run.smithi143.stdout:performing post-bootstrap initialization ... ok 2024-01-15T18:03:47.402 INFO:teuthology.orchestra.run.smithi143.stdout:syncing data to disk ... ok 2024-01-15T18:03:47.914 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-01-15T18:03:51.244 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:03:51.370 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql-contrib (14+238) ... 2024-01-15T18:03:51.496 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up postgresql (14+238) ... 2024-01-15T18:03:51.614 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-01-15T18:03:51.723 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2024-01-15T18:03:51.957 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-01-15T18:03:51.957 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-01-15T18:03:52.650 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-01-15T18:03:52.969 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-01-15T18:03:53.112 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-01-15T18:03:54.447 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-01-15T18:03:54.556 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-01-15T18:03:55.088 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-01-15T18:03:55.096 DEBUG:teuthology.parallel:result is None 2024-01-15T18:03:55.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T18:03:55.369 DEBUG:teuthology.orchestra.run.smithi143:> dpkg-query -W -f '${Version}' ceph 2024-01-15T18:03:55.391 INFO:teuthology.orchestra.run.smithi143.stdout:18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:03:55.391 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:03:55.392 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629-g95c90f41-1jammy is installed. 2024-01-15T18:03:55.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T18:03:55.539 DEBUG:teuthology.orchestra.run.smithi204:> dpkg-query -W -f '${Version}' ceph 2024-01-15T18:03:55.563 INFO:teuthology.orchestra.run.smithi204.stdout:18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:03:55.564 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629-g95c90f41-1jammy 2024-01-15T18:03:55.564 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629-g95c90f41-1jammy is installed. 2024-01-15T18:03:55.565 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-15T18:03:55.565 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:55.565 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-15T18:03:55.585 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:55.586 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-15T18:03:55.625 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-15T18:03:55.625 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:55.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-01-15T18:03:55.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-15T18:03:55.710 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:55.710 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-01-15T18:03:55.731 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-15T18:03:55.792 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-15T18:03:55.792 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:55.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-15T18:03:55.811 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-15T18:03:55.878 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:55.878 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-15T18:03:55.899 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-15T18:03:55.964 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-15T18:03:55.964 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:55.965 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-01-15T18:03:55.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-15T18:03:56.050 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:56.050 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-01-15T18:03:56.071 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-15T18:03:56.133 INFO:teuthology.run_tasks:Running task ceph... 2024-01-15T18:03:56.221 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-15T18:03:56.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2024-01-15T18:03:56.225 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /var/log/ceph 2024-01-15T18:03:56.245 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-15T18:03:56.245 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-15T18:03:56.286 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-15T18:03:56.307 INFO:tasks.ceph:Creating extra log directories... 2024-01-15T18:03:56.308 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-15T18:03:56.350 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-15T18:03:56.377 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-15T18:03:56.378 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-15T18:03:56.378 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-15T18:03:56.378 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:56.378 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-15T18:03:56.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-15T18:03:56.503 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:56.504 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-15T18:03:56.513 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-15T18:03:56.610 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-15T18:03:56.610 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-15T18:03:56.610 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7517105', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-clone-stats', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9779, 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-clone-stats', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvuBUE6lVmsoYjKOKpKWQ7T+l0ufb15R/3UeDf8iIWTqzvGiIbD/viXR9zGGy67YZY6WEFp9Nu7+ofrtGNcZBY=', 'smithi204.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC0TxJjXSPdKNBhbICy8dkJxph6idNeJjN3iVR7cgJk1Al0pJ2ycofFY0WlQIAvYDBAwL2oSo0c545raLcSNHVs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgress']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-15_17:35:46', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-15T18:03:56.611 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-15T18:03:56.614 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-15T18:03:56.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2024-01-15T18:03:56.666 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m0777 -- /var/run/ceph 2024-01-15T18:03:56.688 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:56.688 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-01-15T18:03:56.735 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-15T18:03:56.736 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 902 Links: 1 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-15 17:56:27.642148030 +0000 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-15 17:56:27.390150280 +0000 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-15 17:56:27.390150280 +0000 2024-01-15T18:03:56.788 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-15 17:56:27.390150280 +0000 2024-01-15T18:03:56.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-15T18:03:56.848 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-15T18:03:56.848 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-15T18:03:56.848 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000396772 s, 1.3 MB/s 2024-01-15T18:03:56.850 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-15T18:03:56.901 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-01-15T18:03:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-15T18:03:56.955 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:56.956 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-01-15T18:03:56.956 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:56.956 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-15 17:56:28.158143421 +0000 2024-01-15T18:03:56.956 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-15 17:56:27.890145815 +0000 2024-01-15T18:03:56.956 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-15 17:56:27.890145815 +0000 2024-01-15T18:03:56.956 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-15 17:56:27.890145815 +0000 2024-01-15T18:03:56.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-15T18:03:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-15T18:03:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-15T18:03:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000338914 s, 1.5 MB/s 2024-01-15T18:03:57.018 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-15T18:03:57.069 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-01-15T18:03:57.119 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-15 17:56:28.426141027 +0000 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-15 17:56:28.422141062 +0000 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-15 17:56:28.422141062 +0000 2024-01-15T18:03:57.120 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-15 17:56:28.422141062 +0000 2024-01-15T18:03:57.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-15T18:03:57.181 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-15T18:03:57.181 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-15T18:03:57.181 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000615874 s, 831 kB/s 2024-01-15T18:03:57.183 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-15T18:03:57.233 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 953 Links: 1 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-01-15 17:56:28.950136347 +0000 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-01-15 17:56:28.946136382 +0000 2024-01-15T18:03:57.284 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-01-15 17:56:28.946136382 +0000 2024-01-15T18:03:57.285 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-01-15 17:56:28.946136382 +0000 2024-01-15T18:03:57.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-15T18:03:57.344 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-01-15T18:03:57.344 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-01-15T18:03:57.344 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000371292 s, 1.4 MB/s 2024-01-15T18:03:57.346 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-15T18:03:57.397 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T18:03:57.398 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:57.398 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-01-15T18:03:57.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-15T18:03:57.407 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-15 17:56:28.980652849 +0000 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-15 17:56:28.704655314 +0000 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-15 17:56:28.704655314 +0000 2024-01-15T18:03:57.457 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-15 17:56:28.704655314 +0000 2024-01-15T18:03:57.458 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-15T18:03:57.518 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-15T18:03:57.518 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-15T18:03:57.518 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000391884 s, 1.3 MB/s 2024-01-15T18:03:57.520 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-15T18:03:57.575 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-15 17:56:29.504648169 +0000 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-15 17:56:29.236650563 +0000 2024-01-15T18:03:57.625 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-15 17:56:29.236650563 +0000 2024-01-15T18:03:57.626 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-15 17:56:29.236650563 +0000 2024-01-15T18:03:57.626 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-15T18:03:57.691 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-15T18:03:57.691 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-15T18:03:57.692 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.00038819 s, 1.3 MB/s 2024-01-15T18:03:57.694 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-15T18:03:57.750 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 930 Links: 1 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-15 17:56:30.032643454 +0000 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-15 17:56:29.772645776 +0000 2024-01-15T18:03:57.801 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-15 17:56:29.772645776 +0000 2024-01-15T18:03:57.802 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-15 17:56:29.772645776 +0000 2024-01-15T18:03:57.802 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-15T18:03:57.867 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-15T18:03:57.867 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-15T18:03:57.867 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000385638 s, 1.3 MB/s 2024-01-15T18:03:57.869 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-15T18:03:57.922 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-01-15T18:03:57.973 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-15T18:03:57.973 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T18:03:57.973 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 947 Links: 1 2024-01-15T18:03:57.973 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T18:03:57.973 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-01-15 17:56:30.560638737 +0000 2024-01-15T18:03:57.973 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-01-15 17:56:30.304641023 +0000 2024-01-15T18:03:57.974 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-01-15 17:56:30.304641023 +0000 2024-01-15T18:03:57.974 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-01-15 17:56:30.304641023 +0000 2024-01-15T18:03:57.974 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-15T18:03:58.039 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-01-15T18:03:58.039 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-01-15T18:03:58.039 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000402428 s, 1.3 MB/s 2024-01-15T18:03:58.041 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-15T18:03:58.095 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T18:03:58.095 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-15T18:03:58.095 INFO:tasks.ceph:Generating config... 2024-01-15T18:03:58.096 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-15T18:03:58.096 INFO:tasks.ceph:[client] debug client = 20 2024-01-15T18:03:58.096 INFO:tasks.ceph:[client] debug ms = 1 2024-01-15T18:03:58.097 INFO:tasks.ceph:[client] pid file = 2024-01-15T18:03:58.097 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-15T18:03:58.097 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-15T18:03:58.097 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mgr] debug client = 20 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-15T18:03:58.098 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-15T18:03:58.098 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-15T18:03:58.099 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-15T18:03:58.100 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-15T18:03:58.100 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-15T18:03:58.100 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-15T18:03:58.100 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-15T18:03:58.100 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-15T18:03:58.100 INFO:tasks.ceph:Setting up mon.a... 2024-01-15T18:03:58.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-15T18:03:58.147 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.keyring 2024-01-15T18:03:58.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-15T18:03:58.261 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-15T18:03:58.282 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.143'), ('mon.b', '172.21.15.204'), ('mon.c', '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]')] 2024-01-15T18:03:58.282 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.143,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-15T18:03:58.283 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:58.283 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-15T18:03:58.336 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.143 --add b 172.21.15.204 --addv c '[v2:172.21.15.204:3301,v1:172.21.15.204:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:03:58.424 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: generated fsid 10305fab-1773-4952-a8b6-aa6388f5534c 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = pacific 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 0 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:fsid 10305fab-1773-4952-a8b6-aa6388f5534c 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:last_changed 2024-01-15T18:03:58.425092+0000 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:created 2024-01-15T18:03:58.425092+0000 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:min_mon_release 16 (pacific) 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:election_strategy: 1 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:1: [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon.b 2024-01-15T18:03:58.425 INFO:teuthology.orchestra.run.smithi143.stdout:2: [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] mon.c 2024-01-15T18:03:58.426 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-15T18:03:58.428 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-15T18:03:58.436 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 10305fab-1773-4952-a8b6-aa6388f5534c... 2024-01-15T18:03:58.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-15T18:03:58.486 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-15T18:03:58.525 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2024-01-15T18:03:58.525 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2024-01-15T18:03:58.526 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2024-01-15T18:03:58.527 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.528 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = 10305fab-1773-4952-a8b6-aa6388f5534c 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2024-01-15T18:03:58.529 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2024-01-15T18:03:58.530 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug trim objects = true 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2024-01-15T18:03:58.531 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: bdev async discard = True 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: bdev enable discard = True 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore allocator = bitmap 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 1/20 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 1/20 2024-01-15T18:03:58.532 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 4/10 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2024-01-15T18:03:58.533 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug finisher = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2024-01-15T18:03:58.534 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.535 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: mon op complaint time = 120 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2024-01-15T18:03:58.536 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: client mount timeout = 600 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = "" 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2024-01-15T18:03:58.537 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds = 20 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: debug mds balancer = 20 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug frag = True 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: mds debug scatterstat = True 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: mds op complaint time = 180 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: mds verify scatter = True 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: osd op complaint time = 180 2024-01-15T18:03:58.538 INFO:teuthology.orchestra.run.smithi143.stdout: rados mon op timeout = 900 2024-01-15T18:03:58.539 INFO:teuthology.orchestra.run.smithi143.stdout: rados osd op timeout = 900 2024-01-15T18:03:58.539 INFO:teuthology.orchestra.run.smithi204.stdout:[global] 2024-01-15T18:03:58.539 INFO:teuthology.orchestra.run.smithi204.stdout: chdir = "" 2024-01-15T18:03:58.539 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-15T18:03:58.539 INFO:teuthology.orchestra.run.smithi204.stdout: auth supported = cephx 2024-01-15T18:03:58.539 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: filestore xattr use omap = true 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: mon clock drift allowed = 1.000 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: osd crush chooseleaf type = 0 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: auth debug = true 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on old message = true 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: ms die on bug = true 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-15T18:03:58.540 INFO:teuthology.orchestra.run.smithi204.stdout: mon pg warn max object skew = 0 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: # disable pg_autoscaler by default for new pools 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default size = 2 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow primary affinity = true 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd allow pg remap = true 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on legacy crush tunables = false 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on crush straw calc version zero = false 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on no sortbitwise = false 2024-01-15T18:03:58.541 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on osd down out interval zero = false 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: mon warn on too few osds = false 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: mon_allow_pool_size_one = true 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: osd default data pool replay window = 5 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: mon allow pool delete = true 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: mon cluster log file level = debug 2024-01-15T18:03:58.542 INFO:teuthology.orchestra.run.smithi204.stdout: debug asserts on shutdown = true 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: mon health detail to clog = false 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: mon host = "172.21.15.143,172.21.15.204,[v2:172.21.15.204:3301,v1:172.21.15.204:6790]" 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: fsid = 10305fab-1773-4952-a8b6-aa6388f5534c 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout:[osd] 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: osd journal size = 100 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub load threshold = 5.0 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: osd scrub max interval = 600 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: osd mclock profile = high_recovery_ops 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: osd recover clone overlap = true 2024-01-15T18:03:58.543 INFO:teuthology.orchestra.run.smithi204.stdout: osd recovery max chunk = 1048576 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug shutdown = true 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug op order = true 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug verify stray on activate = true 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug trim objects = true 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd open classes on start = true 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug pg log writeout = true 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd deep scrub update digest min age = 30 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: osd map max advance = 10 2024-01-15T18:03:58.544 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: journal zero on create = true 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: filestore ondisk finisher threads = 3 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: filestore apply finisher threads = 3 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: bdev debug aio = true 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: osd debug misdirected ops = true 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: bdev async discard = True 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: bdev enable discard = True 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore allocator = bitmap 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore block size = 96636764160 2024-01-15T18:03:58.545 INFO:teuthology.orchestra.run.smithi204.stdout: bluestore fsck on mount = True 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluefs = 1/20 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: debug bluestore = 1/20 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: debug osd = 20 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: debug rocksdb = 4/10 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd backfillfull_ratio = 0.85 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd full ratio = 0.9 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd nearfull ratio = 0.8 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: osd failsafe full ratio = 0.95 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: osd objectstore = bluestore 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr] 2024-01-15T18:03:58.546 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug mgr = 20 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: mgr/telemetry/nag = false 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug finisher = 20 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout:[mon] 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug mon = 20 2024-01-15T18:03:58.547 INFO:teuthology.orchestra.run.smithi204.stdout: debug paxos = 20 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: debug auth = 20 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: mon data avail warn = 5 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: mon mgr mkfs grace = 240 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min pgs per osd = 4 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd reporter subtree level = osd 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: mon osd prime pg temp = true 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: mon reweight min bytes per osd = 10 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: auth mon ticket ttl = 660 # 11m 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: auth service ticket ttl = 240 # 4m 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.548 INFO:teuthology.orchestra.run.smithi204.stdout: # don't complain about insecure global_id in the test suite 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: # 1m isn't quite enough 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: mon_down_mkfs_grace = 2m 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: mon_warn_on_filestore_osds = false 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: mon op complaint time = 120 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout:[client] 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: rgw cache enabled = true 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable ops log = true 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: rgw enable usage log = true 2024-01-15T18:03:58.549 INFO:teuthology.orchestra.run.smithi204.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: client mount timeout = 600 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: debug client = 20 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: pid file = "" 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.a] 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.b] 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mon.c] 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-01-15T18:03:58.550 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout:[mds] 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds = 20 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: debug mds balancer = 20 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: debug ms = 1 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug frag = True 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: mds debug scatterstat = True 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: mds op complaint time = 180 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: mds verify scatter = True 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: osd op complaint time = 180 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: rados mon op timeout = 900 2024-01-15T18:03:58.551 INFO:teuthology.orchestra.run.smithi204.stdout: rados osd op timeout = 900 2024-01-15T18:03:58.553 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-15T18:03:58.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-15T18:03:58.652 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-15T18:03:58.653 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:58.653 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-15T18:03:58.662 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:58.662 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-15T18:03:58.711 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:03:58.711 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:58.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-15T18:03:58.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-15T18:03:58.787 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:03:58.787 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:03:58.835 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:03:58.836 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:58.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-15T18:03:58.836 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-15T18:03:58.871 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:03:58.871 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:03:58.921 INFO:tasks.ceph:Setting up mon nodes... 2024-01-15T18:03:58.921 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-15T18:03:58.922 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-15T18:03:58.976 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-15T18:03:58.982 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-15T18:03:59.041 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-15T18:03:59.065 INFO:tasks.ceph:Setting up mds nodes... 2024-01-15T18:03:59.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-15T18:03:59.119 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-15T18:03:59.124 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-15T18:03:59.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-15T18:03:59.245 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-15T18:03:59.250 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-15T18:03:59.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-15T18:03:59.330 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-15T18:03:59.335 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-15T18:03:59.356 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-15T18:03:59.455 INFO:teuthology.orchestra.run.smithi204.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-15T18:03:59.461 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-15T18:03:59.481 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-15T18:03:59.481 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-15T18:03:59.523 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-15T18:03:59.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-15T18:03:59.625 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-15T18:03:59.642 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-15T18:03:59.685 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-15T18:03:59.702 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-15T18:03:59.786 INFO:teuthology.orchestra.run.smithi204.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-15T18:03:59.803 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-15T18:03:59.803 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi204.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-15T18:03:59.803 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-15T18:03:59.823 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T18:03:59.823 INFO:tasks.ceph:role: osd.0 2024-01-15T18:03:59.823 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:03:59.823 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-15T18:03:59.889 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:03:59.889 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:03:59.889 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:03:59.889 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:03:59.890 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:03:59.890 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:03:59.890 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:03:59.890 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:03:59.890 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:03:59.890 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:00.374 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-15T18:04:00.377 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-15T18:04:00.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-15T18:04:00.521 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-15T18:04:00.538 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:00.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:00.539 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-15T18:04:00.603 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:00.603 INFO:tasks.ceph:role: osd.1 2024-01-15T18:04:00.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:04:00.604 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-15T18:04:00.674 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:00.674 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:00.675 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:01.113 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-15T18:04:01.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-15T18:04:01.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-15T18:04:01.230 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-15T18:04:01.247 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:01.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:01.248 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-15T18:04:01.311 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:01.311 INFO:tasks.ceph:role: osd.2 2024-01-15T18:04:01.311 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:04:01.312 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-15T18:04:01.382 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:01.383 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:01.854 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-15T18:04:01.855 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-15T18:04:01.856 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-15T18:04:02.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-15T18:04:02.040 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:02.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:02.041 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-15T18:04:02.106 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:02.107 INFO:tasks.ceph:role: osd.3 2024-01-15T18:04:02.107 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:04:02.107 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:02.179 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:02.660 INFO:teuthology.orchestra.run.smithi143.stdout:Discarding blocks...Done. 2024-01-15T18:04:02.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2024-01-15T18:04:02.663 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-15T18:04:02.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-15T18:04:02.826 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:02.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:02.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:02.956+0000 7f12d3717640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-15T18:04:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:02.956+0000 7f12d3717640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-15T18:04:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:02.956+0000 7f12d3717640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-15T18:04:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:02.956+0000 7f12d3717640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-15T18:04:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:02.956+0000 7f12d3717640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-15T18:04:02.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:02.960+0000 7f12d3717640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-15T18:04:05.981 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-15T18:04:06.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:06.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:06.099+0000 7f757d68e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-15T18:04:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:06.099+0000 7f757d68e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-15T18:04:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:06.099+0000 7f757d68e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-15T18:04:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:06.099+0000 7f757d68e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-15T18:04:06.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:06.099+0000 7f757d68e640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-15T18:04:06.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:06.099+0000 7f757d68e640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-15T18:04:09.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-15T18:04:09.115 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:09.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:09.211+0000 7f5ffdfc6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-15T18:04:09.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:09.215+0000 7f5ffdfc6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-15T18:04:09.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:09.215+0000 7f5ffdfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-15T18:04:09.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:09.215+0000 7f5ffdfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-15T18:04:09.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:09.215+0000 7f5ffdfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-15T18:04:09.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:09.215+0000 7f5ffdfc6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-15T18:04:12.198 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-15T18:04:12.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:12.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:12.315+0000 7f343a4f3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-15T18:04:12.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:12.315+0000 7f343a4f3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-15T18:04:12.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:12.315+0000 7f343a4f3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-15T18:04:12.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:12.315+0000 7f343a4f3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-15T18:04:12.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:12.315+0000 7f343a4f3640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-15T18:04:12.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:12.315+0000 7f343a4f3640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-15T18:04:15.303 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-15T18:04:15.323 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-15T18:04:15.346 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:15.346 INFO:tasks.ceph:role: osd.4 2024-01-15T18:04:15.346 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:04:15.346 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-15T18:04:15.416 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:15.416 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:15.416 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:15.417 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:15.874 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-15T18:04:15.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-15T18:04:15.877 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-15T18:04:15.983 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-15T18:04:16.000 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:16.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:16.002 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-15T18:04:16.064 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:16.064 INFO:tasks.ceph:role: osd.5 2024-01-15T18:04:16.064 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:04:16.064 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-15T18:04:16.136 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:16.136 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:16.136 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:16.136 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:16.136 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:16.137 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:16.137 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:16.137 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:16.137 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:16.137 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:16.617 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-15T18:04:16.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-15T18:04:16.621 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-15T18:04:16.785 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-15T18:04:16.802 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:16.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:16.803 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-15T18:04:16.864 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:16.864 INFO:tasks.ceph:role: osd.6 2024-01-15T18:04:16.864 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:04:16.864 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:16.936 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:17.417 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-15T18:04:17.420 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-15T18:04:17.420 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-15T18:04:17.544 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-15T18:04:17.561 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:17.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:17.562 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-15T18:04:17.627 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T18:04:17.628 INFO:tasks.ceph:role: osd.7 2024-01-15T18:04:17.628 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:04:17.628 DEBUG:teuthology.orchestra.run.smithi204:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-15T18:04:17.699 INFO:teuthology.orchestra.run.smithi204.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T18:04:17.699 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T18:04:17.699 INFO:teuthology.orchestra.run.smithi204.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout: = sunit=0 swidth=0 blks 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T18:04:17.700 INFO:teuthology.orchestra.run.smithi204.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T18:04:18.147 INFO:teuthology.orchestra.run.smithi204.stdout:Discarding blocks...Done. 2024-01-15T18:04:18.150 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi204.front.sepia.ceph.com -o noatime 2024-01-15T18:04:18.150 DEBUG:teuthology.orchestra.run.smithi204:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-15T18:04:18.252 DEBUG:teuthology.orchestra.run.smithi204:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-15T18:04:18.268 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: /sbin/restorecon: command not found 2024-01-15T18:04:18.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:04:18.270 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:18.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:18.398+0000 7f9856763640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-15T18:04:18.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:18.398+0000 7f9856763640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-15T18:04:18.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:18.398+0000 7f9856763640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-15T18:04:18.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:18.398+0000 7f9856763640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-15T18:04:18.402 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:18.398+0000 7f9856763640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-15T18:04:18.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:18.398+0000 7f9856763640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-15T18:04:21.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-15T18:04:21.404 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:21.499 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:21.494+0000 7f8f6f02f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-15T18:04:21.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:21.494+0000 7f8f6f02f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-15T18:04:21.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:21.494+0000 7f8f6f02f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-15T18:04:21.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:21.494+0000 7f8f6f02f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-15T18:04:21.500 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:21.494+0000 7f8f6f02f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-15T18:04:21.501 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:21.498+0000 7f8f6f02f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-15T18:04:24.483 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-15T18:04:24.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:24.601 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:24.598+0000 7fa62dd8f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-15T18:04:24.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:24.598+0000 7fa62dd8f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-15T18:04:24.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:24.598+0000 7fa62dd8f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-15T18:04:24.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:24.598+0000 7fa62dd8f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-15T18:04:24.602 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:24.598+0000 7fa62dd8f640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-15T18:04:24.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:24.598+0000 7fa62dd8f640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-15T18:04:26.612 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:26.616 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:27.599 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-15T18:04:27.620 DEBUG:teuthology.orchestra.run.smithi204:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:27.720 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:27.714+0000 7f610042b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-15T18:04:27.721 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:27.718+0000 7f610042b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-15T18:04:27.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:27.718+0000 7f610042b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-15T18:04:27.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:27.718+0000 7f610042b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-15T18:04:27.722 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:27.718+0000 7f610042b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-15T18:04:27.723 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:27.718+0000 7f610042b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-15T18:04:30.707 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-15T18:04:30.727 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-15T18:04:30.728 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:30.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-15T18:04:30.751 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:30.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-15T18:04:30.816 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:30.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-15T18:04:30.880 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:30.880 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-15T18:04:30.947 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:30.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-15T18:04:31.011 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:31.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-15T18:04:31.075 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:31.076 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-15T18:04:31.143 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.143 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-15T18:04:31.165 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.165 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-15T18:04:31.227 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-15T18:04:31.291 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.291 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-15T18:04:31.355 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.355 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-15T18:04:31.419 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.419 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-15T18:04:31.483 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.483 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-15T18:04:31.547 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:31.547 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-15T18:04:31.556 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:31.556 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-15T18:04:31.608 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.608 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-15T18:04:31.617 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:31.617 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-15T18:04:31.671 INFO:tasks.ceph:Adding keys to all mons... 2024-01-15T18:04:31.672 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-15T18:04:31.675 DEBUG:teuthology.orchestra.run.smithi204:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-15T18:04:31.715 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.y] 2024-01-15T18:04:31.715 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg== 2024-01-15T18:04:31.715 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA== 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg== 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ== 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q== 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg== 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.3] 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ== 2024-01-15T18:04:31.716 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA== 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ== 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.d] 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw== 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.4] 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA== 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.5] 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg== 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.6] 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ== 2024-01-15T18:04:31.717 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.7] 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA== 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout:[client.2] 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg== 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout:[client.1] 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw== 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout:[client.3] 2024-01-15T18:04:31.718 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ== 2024-01-15T18:04:31.728 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.y] 2024-01-15T18:04:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg== 2024-01-15T18:04:31.728 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.a] 2024-01-15T18:04:31.728 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.c] 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.0] 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.1] 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.2] 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.3] 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[mgr.x] 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA== 2024-01-15T18:04:31.729 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.b] 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ== 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout:[mds.d] 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw== 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.4] 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA== 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.5] 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg== 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.6] 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ== 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout:[osd.7] 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA== 2024-01-15T18:04:31.730 INFO:teuthology.orchestra.run.smithi204.stdout:[client.0] 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout:[client.2] 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg== 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout:[client.1] 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw== 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout:[client.3] 2024-01-15T18:04:31.731 INFO:teuthology.orchestra.run.smithi204.stdout: key = AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ== 2024-01-15T18:04:31.732 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T18:04:31.766 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T18:04:31.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:31.831 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:31.887 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:31.890 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:31.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:31.949 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.003 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.006 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.059 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.062 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.115 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.118 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.171 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T18:04:32.174 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T18:04:32.227 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:32.230 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:32.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:32.288 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T18:04:32.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.347 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.403 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.456 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.459 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.515 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T18:04:32.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.571 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.630 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.686 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.689 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.744 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T18:04:32.803 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-15T18:04:32.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-15T18:04:32.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-15T18:04:32.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-15T18:04:32.956 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-15T18:04:32.975 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-15T18:04:33.084 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-15T18:04:33.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-15T18:04:33.162 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-15T18:04:33.276 DEBUG:teuthology.orchestra.run.smithi204:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-15T18:04:33.295 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:33.298 DEBUG:teuthology.orchestra.run.smithi204:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-15T18:04:33.343 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-15T18:04:33.343 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-15T18:04:33.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-15T18:04:33.346 INFO:tasks.ceph.mon.a:Started 2024-01-15T18:04:33.346 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-15T18:04:33.347 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-15T18:04:33.390 INFO:tasks.ceph.mon.b:Started 2024-01-15T18:04:33.390 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-15T18:04:33.390 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-15T18:04:33.393 INFO:tasks.ceph.mon.c:Started 2024-01-15T18:04:33.393 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-15T18:04:33.393 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-15T18:04:33.394 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-15T18:04:33.397 INFO:tasks.ceph.mgr.y:Started 2024-01-15T18:04:33.397 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-15T18:04:33.397 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-15T18:04:33.400 INFO:tasks.ceph.mgr.x:Started 2024-01-15T18:04:33.400 INFO:tasks.ceph:Setting crush tunables to default 2024-01-15T18:04:33.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd crush tunables default 2024-01-15T18:04:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb81400640 1 Processor -- start 2024-01-15T18:04:33.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb81400640 1 -- start start 2024-01-15T18:04:33.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb81400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb7c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb81400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c0c4d90 con 0x7fdb7c0d19f0 2024-01-15T18:04:33.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb81400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb7c15b640 con 0x7fdb7c0ccaf0 2024-01-15T18:04:33.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb81400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0cf800 con 0x7fdb7c15b260 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb79e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb7c0d19f0 0x7fdb7c0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38764/0 (socket says 172.21.15.143:38764) 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb79e00640 1 -- 172.21.15.143:0/808702481 learned_addr learned my addr 172.21.15.143:0/808702481 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb7a800640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb7c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb7a800640 1 -- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 msgr2=0x7fdb7c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb7a800640 1 -- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 msgr2=0x7fdb7c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb7a800640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb7c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T18:04:33.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.539+0000 7fdb7a800640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb7c0cd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T18:04:33.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832836191 0 0) 0x7fdb7c0c4d90 con 0x7fdb7c0d19f0 2024-01-15T18:04:33.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb5c003530 con 0x7fdb7c0d19f0 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2098884629 0 0) 0x7fdb5c003530 con 0x7fdb7c0d19f0 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 msgr2=0x7fdb7c0cd810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb7c0cd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 >> v1:172.21.15.204:6789/0 conn(0x7fdb7c0ccaf0 legacy=0x7fdb7c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0cfae0 con 0x7fdb7c0d19f0 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fdb64001680 con 0x7fdb7c0d19f0 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 >> v1:172.21.15.143:6789/0 conn(0x7fdb7c0d19f0 legacy=0x7fdb7c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5c0039b0 0x7fdb5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb5c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb5c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c0cf800 con 0x7fdb5c0047d0 2024-01-15T18:04:33.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c00bb20 con 0x7fdb5c0039b0 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5c00bce0 con 0x7fdb7c15b260 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79e00640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb5c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb7a800640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5c0039b0 0x7fdb5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79e00640 1 -- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 msgr2=0x7fdb5c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79e00640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c15b260 0x7fdb5c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79e00640 1 -- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5c0039b0 msgr2=0x7fdb5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79e00640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb5c0039b0 0x7fdb5c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.695+0000 7fdb79e00640 1 -- 172.21.15.143:0/808702481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c003530 con 0x7fdb5c0047d0 2024-01-15T18:04:33.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.699+0000 7fdb79e00640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb5c004c50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400c0e0 tx=0x7fdb6401f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b53f2e8fbad0f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640058e0 con 0x7fdb5c0047d0 2024-01-15T18:04:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb64005a40 con 0x7fdb5c0047d0 2024-01-15T18:04:33.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb79400640 1 -- 172.21.15.143:0/808702481 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64005e30 con 0x7fdb5c0047d0 2024-01-15T18:04:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb81400640 1 -- 172.21.15.143:0/808702481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 msgr2=0x7fdb5c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb81400640 1 --2- 172.21.15.143:0/808702481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb5c004c50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400c0e0 tx=0x7fdb6401f2d0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb7b200640 1 -- 172.21.15.143:0/808702481 reap_dead start 2024-01-15T18:04:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb81400640 1 -- 172.21.15.143:0/808702481 shutdown_connections 2024-01-15T18:04:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb81400640 1 -- 172.21.15.143:0/808702481 >> 172.21.15.143:0/808702481 conn(0x7fdb7c0d0ed0 msgr2=0x7fdb7c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:33.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb81400640 1 -- 172.21.15.143:0/808702481 shutdown_connections 2024-01-15T18:04:33.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.703+0000 7fdb81400640 1 -- 172.21.15.143:0/808702481 wait complete. 2024-01-15T18:04:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 Processor -- start 2024-01-15T18:04:33.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 -- start start 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb7c09ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 0x7fdb7c1f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c1f28f0 0x7fdb7c09e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c09d3f0 con 0x7fdb5c0047d0 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c09d6d0 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb81400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c09d9b0 con 0x7fdb7c1f28f0 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb7a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb7c09ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 0x7fdb7c1f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb7b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c1f28f0 0x7fdb7c09e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 0x7fdb7c1f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:43152/0 (socket says 172.21.15.143:43152) 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 -- 172.21.15.143:0/2062276684 learned_addr learned my addr 172.21.15.143:0/2062276684 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 -- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c1f28f0 msgr2=0x7fdb7c09e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c1f28f0 0x7fdb7c09e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 -- 172.21.15.143:0/2062276684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 msgr2=0x7fdb7c09ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb7c09ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 -- 172.21.15.143:0/2062276684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c09dc90 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb7b200640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c1f28f0 0x7fdb7c09e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb79e00640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 0x7fdb7c1f23b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401f100 tx=0x7fdb5c005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.707+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb640b7070 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb64005bb0 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c09df70 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb64007090 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c09b380 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdb640071f0 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdb640b1c50 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.711+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c067500 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.715+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fdb64007c50 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.779+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdb7c0c4240 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb72a00640 1 -- 172.21.15.143:0/2062276684 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdb640ca020 con 0x7fdb7c0b65e0 2024-01-15T18:04:33.789 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2024-01-15T18:04:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 msgr2=0x7fdb7c1f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 0x7fdb7c1f23b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdb6401f100 tx=0x7fdb5c005800 comp rx=0 tx=0).stop 2024-01-15T18:04:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 shutdown_connections 2024-01-15T18:04:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb7c1f28f0 0x7fdb7c09e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb7c0b65e0 0x7fdb7c1f23b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 --2- 172.21.15.143:0/2062276684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb5c0047d0 0x7fdb7c09ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 >> 172.21.15.143:0/2062276684 conn(0x7fdb7c0d0ed0 msgr2=0x7fdb7c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 shutdown_connections 2024-01-15T18:04:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:33.787+0000 7fdb81400640 1 -- 172.21.15.143:0/2062276684 wait complete. 2024-01-15T18:04:33.810 INFO:tasks.ceph:check_enable_crimson: False 2024-01-15T18:04:33.810 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-15T18:04:33.810 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:33.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-15T18:04:33.831 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:33.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-15T18:04:33.892 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:33.892 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-15T18:04:33.901 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:33.898+0000 7f8d39f99180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-15T18:04:33.929 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:33.927+0000 7f3d2e6e0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-15T18:04:33.949 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:04:33.949 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-15T18:04:34.011 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:34.012 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-15T18:04:34.031 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:34.032 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-15T18:04:34.095 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:34.095 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-15T18:04:34.159 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:04:34.159 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-15T18:04:34.222 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 7f53e508-d995-4e4c-811b-124725163b34 0 2024-01-15T18:04:34.383 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:34.378+0000 7f8d39f99180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-15T18:04:34.397 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 Processor -- start 2024-01-15T18:04:34.398 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 -- start start 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3b40157780 0x7f3b40143400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b40140960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3b400cc7e0 0x7f3b400ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4006cbc0 con 0x7f3b40157780 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b400cd1b0 con 0x7f3b400cc7e0 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b47a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40143940 con 0x7f3b4015d7f0 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3b400cc7e0 0x7f3b400ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b40140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3b400cc7e0 0x7f3b400ccc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41650/0 (socket says 172.21.15.204:41650) 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 -- 172.21.15.204:0/3862209984 learned_addr learned my addr 172.21.15.204:0/3862209984 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45000640 1 --2- 172.21.15.204:0/3862209984 >> v2:172.21.15.143:3300/0 conn(0x7f3b40157780 0x7f3b40143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 -- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 msgr2=0x7f3b40140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:34.400 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 --2- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b40140960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 -- 172.21.15.204:0/3862209984 >> v2:172.21.15.143:3300/0 conn(0x7f3b40157780 msgr2=0x7f3b40143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 --2- 172.21.15.204:0/3862209984 >> v2:172.21.15.143:3300/0 conn(0x7f3b40157780 0x7f3b40143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b45a00640 1 -- 172.21.15.204:0/3862209984 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b40160de0 con 0x7f3b400cc7e0 2024-01-15T18:04:34.401 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.394+0000 7f3b3fe00640 1 --2- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b40140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:34.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b45a00640 1 --2- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b400cc7e0 0x7f3b400ccc70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b4015dc70 tx=0x7f3b3801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c877768a14344002 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:34.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b3f400640 1 -- 172.21.15.204:0/3862209984 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380a5820 con 0x7f3b400cc7e0 2024-01-15T18:04:34.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b3f400640 1 -- 172.21.15.204:0/3862209984 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b380a5980 con 0x7f3b400cc7e0 2024-01-15T18:04:34.403 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b3f400640 1 -- 172.21.15.204:0/3862209984 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380a5d10 con 0x7f3b400cc7e0 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 -- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b400cc7e0 msgr2=0x7f3b400ccc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 --2- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b400cc7e0 0x7f3b400ccc70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b4015dc70 tx=0x7f3b3801c9a0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 -- 172.21.15.204:0/3862209984 shutdown_connections 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 --2- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b400cc7e0 0x7f3b400ccc70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 --2- 172.21.15.204:0/3862209984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b40140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 --2- 172.21.15.204:0/3862209984 >> v2:172.21.15.143:3300/0 conn(0x7f3b40157780 0x7f3b40143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 -- 172.21.15.204:0/3862209984 >> 172.21.15.204:0/3862209984 conn(0x7f3b400b1980 msgr2=0x7f3b400cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.398+0000 7f3b47a00640 1 -- 172.21.15.204:0/3862209984 shutdown_connections 2024-01-15T18:04:34.404 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 -- 172.21.15.204:0/3862209984 wait complete. 2024-01-15T18:04:34.406 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 Processor -- start 2024-01-15T18:04:34.406 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 -- start start 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b400cc7e0 0x7f3b40154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b40157780 0x7f3b4013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b4013e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4013dd80 con 0x7f3b400cc7e0 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4013da80 con 0x7f3b40157780 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b47a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40155170 con 0x7f3b4015d7f0 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b40157780 0x7f3b4013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b4013e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b40157780 0x7f3b4013f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41662/0 (socket says 172.21.15.204:41662) 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b3fe00640 1 -- 172.21.15.204:0/693149390 learned_addr learned my addr 172.21.15.204:0/693149390 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45000640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b400cc7e0 0x7f3b40154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.408 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 -- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b40157780 msgr2=0x7f3b4013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.409 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b40157780 0x7f3b4013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.409 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 -- 172.21.15.204:0/693149390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b400cc7e0 msgr2=0x7f3b40154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.409 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b400cc7e0 0x7f3b40154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.409 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 -- 172.21.15.204:0/693149390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28009420 con 0x7f3b4015d7f0 2024-01-15T18:04:34.409 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45000640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b400cc7e0 0x7f3b40154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:34.409 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.402+0000 7f3b45a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b4013e210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b400cd3a0 tx=0x7f3b38019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380b8670 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b380b87d0 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380b8b80 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b38009b30 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4013f9a0 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3b380c3460 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b380b6440 con 0x7f3b4015d7f0 2024-01-15T18:04:34.410 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.406+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4013df30 con 0x7f3b4015d7f0 2024-01-15T18:04:34.412 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.410+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f3b4013df30 con 0x7f3b4015d7f0 2024-01-15T18:04:34.420 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:34.419+0000 7f3d2e6e0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-15T18:04:34.478 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.474+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7f53e508-d995-4e4c-811b-124725163b34", "id": 0} v 0) v1 -- 0x7f3b4015f5b0 con 0x7f3b4015d7f0 2024-01-15T18:04:34.483 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.478+0000 7f3b3cc00640 1 -- 172.21.15.204:0/693149390 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7f53e508-d995-4e4c-811b-124725163b34", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3b380c3880 con 0x7f3b4015d7f0 2024-01-15T18:04:34.483 INFO:teuthology.orchestra.run.smithi204.stdout:0 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 msgr2=0x7f3b4013e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b4013e210 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b400cd3a0 tx=0x7f3b38019740 comp rx=0 tx=0).stop 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 shutdown_connections 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4015d7f0 0x7f3b4013e210 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b40157780 0x7f3b4013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 --2- 172.21.15.204:0/693149390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b400cc7e0 0x7f3b40154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 >> 172.21.15.204:0/693149390 conn(0x7f3b400b1980 msgr2=0x7f3b4015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 shutdown_connections 2024-01-15T18:04:34.485 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.482+0000 7f3b47a00640 1 -- 172.21.15.204:0/693149390 wait complete. 2024-01-15T18:04:34.504 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 7066343c-fbd3-45f3-8f91-12ecd8dab094 1 2024-01-15T18:04:34.597 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:34.594+0000 7f8d39f99180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-15T18:04:34.639 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.634+0000 7fc095000640 1 Processor -- start 2024-01-15T18:04:34.640 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.634+0000 7fc095000640 1 -- start start 2024-01-15T18:04:34.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc095000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc09006aba0 0x7fc09015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc095000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc09015de90 0x7fc0900ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.641 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc095000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0900ae830 con 0x7fc0900ae450 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc095000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0900ad900 con 0x7fc09015de90 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc095000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09015e6c0 con 0x7fc09006aba0 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc09006aba0 0x7fc09015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08de00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc0900ae450 0x7fc09006a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:36562/0 (socket says 172.21.15.204:36562) 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08de00640 1 -- 172.21.15.204:0/4050509687 learned_addr learned my addr 172.21.15.204:0/4050509687 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08e800640 1 --2- 172.21.15.204:0/4050509687 >> v2:172.21.15.143:3300/0 conn(0x7fc09015de90 0x7fc0900ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08d400640 1 -- 172.21.15.204:0/4050509687 >> v2:172.21.15.143:3300/0 conn(0x7fc09015de90 msgr2=0x7fc0900ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:34.642 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08d400640 1 --2- 172.21.15.204:0/4050509687 >> v2:172.21.15.143:3300/0 conn(0x7fc09015de90 0x7fc0900ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.643 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08d400640 1 -- 172.21.15.204:0/4050509687 >> v1:172.21.15.204:6789/0 conn(0x7fc0900ae450 legacy=0x7fc09006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.643 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08ca00640 1 -- 172.21.15.204:0/4050509687 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383888422 0 0) 0x7fc0900ae830 con 0x7fc0900ae450 2024-01-15T18:04:34.643 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.638+0000 7fc08d400640 1 -- 172.21.15.204:0/4050509687 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc090160e20 con 0x7fc09006aba0 2024-01-15T18:04:34.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc08d400640 1 --2- 172.21.15.204:0/4050509687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc09006aba0 0x7fc09015d950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0780097c0 tx=0x7fc07801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f5b5a948fc3325c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:34.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc08ca00640 1 -- 172.21.15.204:0/4050509687 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780ab650 con 0x7fc09006aba0 2024-01-15T18:04:34.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc08ca00640 1 -- 172.21.15.204:0/4050509687 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0780ab7b0 con 0x7fc09006aba0 2024-01-15T18:04:34.645 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc08ca00640 1 -- 172.21.15.204:0/4050509687 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780abb40 con 0x7fc09006aba0 2024-01-15T18:04:34.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 -- 172.21.15.204:0/4050509687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc09006aba0 msgr2=0x7fc09015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.646 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 --2- 172.21.15.204:0/4050509687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc09006aba0 0x7fc09015d950 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc0780097c0 tx=0x7fc07801c450 comp rx=0 tx=0).stop 2024-01-15T18:04:34.647 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 -- 172.21.15.204:0/4050509687 shutdown_connections 2024-01-15T18:04:34.647 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 --2- 172.21.15.204:0/4050509687 >> v2:172.21.15.143:3300/0 conn(0x7fc09015de90 0x7fc0900ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.647 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 --2- 172.21.15.204:0/4050509687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc09006aba0 0x7fc09015d950 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.647 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 -- 172.21.15.204:0/4050509687 >> 172.21.15.204:0/4050509687 conn(0x7fc0900cfd20 msgr2=0x7fc0900d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.647 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 -- 172.21.15.204:0/4050509687 shutdown_connections 2024-01-15T18:04:34.647 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.642+0000 7fc095000640 1 -- 172.21.15.204:0/4050509687 wait complete. 2024-01-15T18:04:34.649 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 Processor -- start 2024-01-15T18:04:34.649 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 -- start start 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09006aba0 0x7fc09014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 0x7fc090154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc09015de90 0x7fc09014b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09015ef70 con 0x7fc09006aba0 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc090152ca0 con 0x7fc09015de90 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc095000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc090152f40 con 0x7fc0900ae450 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc09015de90 0x7fc09014b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 0x7fc090154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09006aba0 0x7fc09014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.652 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 0x7fc090154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:52286/0 (socket says 172.21.15.204:52286) 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 -- 172.21.15.204:0/166036791 learned_addr learned my addr 172.21.15.204:0/166036791 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 -- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc09015de90 msgr2=0x7fc09014b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc09015de90 0x7fc09014b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 -- 172.21.15.204:0/166036791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09006aba0 msgr2=0x7fc09014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09006aba0 0x7fc09014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 -- 172.21.15.204:0/166036791 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc078009420 con 0x7fc0900ae450 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08e800640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc09015de90 0x7fc09014b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:34.653 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.646+0000 7fc08d400640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 0x7fc090154860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0780ad040 tx=0x7fc07801ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780b5810 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0780b5990 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09015e920 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0780b5d40 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc090154da0 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc0780bf710 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc0780b47e0 con 0x7fc0900ae450 2024-01-15T18:04:34.654 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc090154f90 con 0x7fc0900ae450 2024-01-15T18:04:34.656 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.650+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc090154f90 con 0x7fc0900ae450 2024-01-15T18:04:34.658 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:34.655+0000 7f3d2e6e0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-15T18:04:34.724 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.722+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7066343c-fbd3-45f3-8f91-12ecd8dab094", "id": 1} v 0) v1 -- 0x7fc090067500 con 0x7fc0900ae450 2024-01-15T18:04:34.731 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.726+0000 7fc086000640 1 -- 172.21.15.204:0/166036791 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7066343c-fbd3-45f3-8f91-12ecd8dab094", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc0780cf020 con 0x7fc0900ae450 2024-01-15T18:04:34.731 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:04:34.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 msgr2=0x7fc090154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 0x7fc090154860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc0780ad040 tx=0x7fc07801ac90 comp rx=0 tx=0).stop 2024-01-15T18:04:34.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 shutdown_connections 2024-01-15T18:04:34.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc09015de90 0x7fc09014b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.732 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0900ae450 0x7fc090154860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 --2- 172.21.15.204:0/166036791 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc09006aba0 0x7fc09014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 >> 172.21.15.204:0/166036791 conn(0x7fc0900cfd20 msgr2=0x7fc0900c2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 shutdown_connections 2024-01-15T18:04:34.733 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.730+0000 7fc095000640 1 -- 172.21.15.204:0/166036791 wait complete. 2024-01-15T18:04:34.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new cfa742d1-b3c0-440a-87db-8f22f23d5e79 2 2024-01-15T18:04:34.885 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87aaa00640 1 Processor -- start 2024-01-15T18:04:34.886 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87aaa00640 1 -- start start 2024-01-15T18:04:34.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87aaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a40725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87aaa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a40aed20 con 0x7f87a40ad070 2024-01-15T18:04:34.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87aaa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a415d750 con 0x7f87a40b3d90 2024-01-15T18:04:34.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87aaa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40ae780 con 0x7f87a415d370 2024-01-15T18:04:34.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87a3400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f87a40b3d90 0x7f87a40af1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:36566/0 (socket says 172.21.15.204:36566) 2024-01-15T18:04:34.887 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87a3400640 1 -- 172.21.15.204:0/3309128753 learned_addr learned my addr 172.21.15.204:0/3309128753 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-15T18:04:34.888 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87a3e00640 1 --2- 172.21.15.204:0/3309128753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a40725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.888 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87a3e00640 1 -- 172.21.15.204:0/3309128753 >> v1:172.21.15.204:6789/0 conn(0x7f87a40b3d90 legacy=0x7f87a40af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.888 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87a3e00640 1 -- 172.21.15.204:0/3309128753 >> v1:172.21.15.143:6789/0 conn(0x7f87a40ad070 legacy=0x7f87a40ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.888 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.882+0000 7f87a3e00640 1 -- 172.21.15.204:0/3309128753 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a40adbd0 con 0x7f87a415d370 2024-01-15T18:04:34.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87a3e00640 1 --2- 172.21.15.204:0/3309128753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a40725b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87940097f0 tx=0x7f879401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72c4bc01d93ee94 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:34.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87a2a00640 1 -- 172.21.15.204:0/3309128753 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940ab600 con 0x7f87a415d370 2024-01-15T18:04:34.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87a2a00640 1 -- 172.21.15.204:0/3309128753 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87940ab760 con 0x7f87a415d370 2024-01-15T18:04:34.891 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87a2a00640 1 -- 172.21.15.204:0/3309128753 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87940abaf0 con 0x7f87a415d370 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87aaa00640 1 -- 172.21.15.204:0/3309128753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 msgr2=0x7f87a40725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87aaa00640 1 --2- 172.21.15.204:0/3309128753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a40725b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87940097f0 tx=0x7f879401c400 comp rx=0 tx=0).stop 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87aaa00640 1 -- 172.21.15.204:0/3309128753 shutdown_connections 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87aaa00640 1 --2- 172.21.15.204:0/3309128753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a40725b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.886+0000 7f87aaa00640 1 -- 172.21.15.204:0/3309128753 >> 172.21.15.204:0/3309128753 conn(0x7f87a4071a90 msgr2=0x7f87a4071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 -- 172.21.15.204:0/3309128753 shutdown_connections 2024-01-15T18:04:34.892 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 -- 172.21.15.204:0/3309128753 wait complete. 2024-01-15T18:04:34.894 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 Processor -- start 2024-01-15T18:04:34.894 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 -- start start 2024-01-15T18:04:34.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 0x7f87a4099300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a40b3d90 0x7f87a4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:34.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a414ea90 con 0x7f87a40b3d90 2024-01-15T18:04:34.895 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a415a3a0 con 0x7f87a40ad070 2024-01-15T18:04:34.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87aaa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a414de80 con 0x7f87a415d370 2024-01-15T18:04:34.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 0x7f87a4099300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 0x7f87a4099300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41682/0 (socket says 172.21.15.204:41682) 2024-01-15T18:04:34.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3e00640 1 -- 172.21.15.204:0/36376267 learned_addr learned my addr 172.21.15.204:0/36376267 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:34.896 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a40b3d90 0x7f87a4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:34.897 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 -- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 msgr2=0x7f87a414ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a414ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 -- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 msgr2=0x7f87a4099300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 0x7f87a4099300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 -- 172.21.15.204:0/36376267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794009420 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3e00640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 0x7f87a4099300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a8a00640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.890+0000 7f87a3400640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a40b3d90 0x7f87a4159e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c430 tx=0x7f878c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c005cf0 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c005e50 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0072d0 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a414e160 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a414e720 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f878c01c440 con 0x7f87a40b3d90 2024-01-15T18:04:34.898 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f878c01cb10 con 0x7f87a40b3d90 2024-01-15T18:04:34.899 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.894+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a4067c10 con 0x7f87a40b3d90 2024-01-15T18:04:34.903 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.898+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f878c021060 con 0x7f87a40b3d90 2024-01-15T18:04:34.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.966+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cfa742d1-b3c0-440a-87db-8f22f23d5e79", "id": 2} v 0) v1 -- 0x7f87a414ebd0 con 0x7f87a40b3d90 2024-01-15T18:04:34.973 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f879be00640 1 -- 172.21.15.204:0/36376267 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfa742d1-b3c0-440a-87db-8f22f23d5e79", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f878c017070 con 0x7f87a40b3d90 2024-01-15T18:04:34.973 INFO:teuthology.orchestra.run.smithi204.stdout:2 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a40b3d90 msgr2=0x7f87a4159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a40b3d90 0x7f87a4159e60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c430 tx=0x7f878c00c8f0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 shutdown_connections 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87a415d370 0x7f87a414ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87a40b3d90 0x7f87a4159e60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 --2- 172.21.15.204:0/36376267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87a40ad070 0x7f87a4099300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 >> 172.21.15.204:0/36376267 conn(0x7f87a4071a90 msgr2=0x7f87a40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.975 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 shutdown_connections 2024-01-15T18:04:34.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:34.970+0000 7f87aaa00640 1 -- 172.21.15.204:0/36376267 wait complete. 2024-01-15T18:04:34.994 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 68992798-8430-430e-a3d4-d05044dba6a1 3 2024-01-15T18:04:35.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 Processor -- start 2024-01-15T18:04:35.128 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 -- start start 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef4c06a490 0x7fef4c06a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c09dcc0 0x7fef4c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c06adb0 con 0x7fef4c071210 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c0b4140 con 0x7fef4c06a490 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef51600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0aed70 con 0x7fef4c09dcc0 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c09dcc0 0x7fef4c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.130 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4b400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fef4c071210 0x7fef4c0716c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:36572/0 (socket says 172.21.15.204:36572) 2024-01-15T18:04:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4b400640 1 -- 172.21.15.204:0/4242105868 learned_addr learned my addr 172.21.15.204:0/4242105868 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-15T18:04:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4aa00640 1 --2- 172.21.15.204:0/4242105868 >> v2:172.21.15.143:3300/0 conn(0x7fef4c06a490 0x7fef4c06a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4a000640 1 -- 172.21.15.204:0/4242105868 >> v2:172.21.15.143:3300/0 conn(0x7fef4c06a490 msgr2=0x7fef4c06a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4a000640 1 --2- 172.21.15.204:0/4242105868 >> v2:172.21.15.143:3300/0 conn(0x7fef4c06a490 0x7fef4c06a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4a000640 1 -- 172.21.15.204:0/4242105868 >> v1:172.21.15.204:6789/0 conn(0x7fef4c071210 legacy=0x7fef4c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.131 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.126+0000 7fef4a000640 1 -- 172.21.15.204:0/4242105868 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0af050 con 0x7fef4c09dcc0 2024-01-15T18:04:35.133 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef4a000640 1 --2- 172.21.15.204:0/4242105868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c09dcc0 0x7fef4c0acdb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef340097c0 tx=0x7fef3401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d151e5536adb72 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.134 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef49600640 1 -- 172.21.15.204:0/4242105868 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340ab600 con 0x7fef4c09dcc0 2024-01-15T18:04:35.134 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef49600640 1 -- 172.21.15.204:0/4242105868 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef340a7050 con 0x7fef4c09dcc0 2024-01-15T18:04:35.134 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef49600640 1 -- 172.21.15.204:0/4242105868 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef340ab980 con 0x7fef4c09dcc0 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 -- 172.21.15.204:0/4242105868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c09dcc0 msgr2=0x7fef4c0acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 --2- 172.21.15.204:0/4242105868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c09dcc0 0x7fef4c0acdb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef340097c0 tx=0x7fef3401c480 comp rx=0 tx=0).stop 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 -- 172.21.15.204:0/4242105868 shutdown_connections 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 --2- 172.21.15.204:0/4242105868 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c09dcc0 0x7fef4c0acdb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 --2- 172.21.15.204:0/4242105868 >> v2:172.21.15.143:3300/0 conn(0x7fef4c06a490 0x7fef4c06a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 -- 172.21.15.204:0/4242105868 >> 172.21.15.204:0/4242105868 conn(0x7fef4c06b5a0 msgr2=0x7fef4c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 -- 172.21.15.204:0/4242105868 shutdown_connections 2024-01-15T18:04:35.135 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.130+0000 7fef51600640 1 -- 172.21.15.204:0/4242105868 wait complete. 2024-01-15T18:04:35.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 Processor -- start 2024-01-15T18:04:35.136 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 -- start start 2024-01-15T18:04:35.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 0x7fef4c0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.138 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c071210 0x7fef4c06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef4c09dcc0 0x7fef4c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c09be10 con 0x7fef4c06a490 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c09c060 con 0x7fef4c09dcc0 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c09c340 con 0x7fef4c071210 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 0x7fef4c0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c071210 0x7fef4c06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 0x7fef4c0c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.204:36784/0 (socket says 172.21.15.204:36784) 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 -- 172.21.15.204:0/2476391838 learned_addr learned my addr 172.21.15.204:0/2476391838 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4b400640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef4c09dcc0 0x7fef4c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 -- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c071210 msgr2=0x7fef4c06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c071210 0x7fef4c06e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 -- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef4c09dcc0 msgr2=0x7fef4c15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef4c09dcc0 0x7fef4c15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.139 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 -- 172.21.15.204:0/2476391838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34009420 con 0x7fef4c06a490 2024-01-15T18:04:35.140 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef4aa00640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 0x7fef4c0c4c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e930 tx=0x7fef3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c009690 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00ca00 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c09c620 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00cd70 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c0c5180 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fef3c005ca0 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.134+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fef3c0064b0 con 0x7fef4c06a490 2024-01-15T18:04:35.141 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.138+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c0c6b60 con 0x7fef4c06a490 2024-01-15T18:04:35.144 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.142+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fef4c0c6b60 con 0x7fef4c06a490 2024-01-15T18:04:35.210 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.206+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "68992798-8430-430e-a3d4-d05044dba6a1", "id": 3} v 0) v1 -- 0x7fef4c067c10 con 0x7fef4c06a490 2024-01-15T18:04:35.215 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.210+0000 7fef42a00640 1 -- 172.21.15.204:0/2476391838 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68992798-8430-430e-a3d4-d05044dba6a1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fef3c029020 con 0x7fef4c06a490 2024-01-15T18:04:35.215 INFO:teuthology.orchestra.run.smithi204.stdout:3 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.210+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 msgr2=0x7fef4c0c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.210+0000 7fef51600640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 0x7fef4c0c4c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00e930 tx=0x7fef3c00edf0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 shutdown_connections 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef4c09dcc0 0x7fef4c15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef4c071210 0x7fef4c06e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 --2- 172.21.15.204:0/2476391838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef4c06a490 0x7fef4c0c4c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.216 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 >> 172.21.15.204:0/2476391838 conn(0x7fef4c06b5a0 msgr2=0x7fef4c0afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 shutdown_connections 2024-01-15T18:04:35.217 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.214+0000 7fef51600640 1 -- 172.21.15.204:0/2476391838 wait complete. 2024-01-15T18:04:35.235 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 597dd8ae-08f6-4609-9eee-19492d4f4354 4 2024-01-15T18:04:35.260 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.258+0000 7f8d39f99180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-15T18:04:35.290 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.287+0000 7f3d2e6e0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-15T18:04:35.324 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.318+0000 7f8d39f99180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-15T18:04:35.352 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.347+0000 7f3d2e6e0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-15T18:04:35.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf0c000640 1 Processor -- start 2024-01-15T18:04:35.370 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf0c000640 1 -- start start 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf0c000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbf040b3610 0x7fbf040cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf0c000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040cefa0 con 0x7fbf040b4080 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf0c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040d0f90 con 0x7fbf040b45d0 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf0c000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf040c9fa0 con 0x7fbf040b3610 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf09600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbf040b3610 0x7fbf040cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf09600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbf040b3610 0x7fbf040cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41712/0 (socket says 172.21.15.204:41712) 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf09600640 1 -- 172.21.15.204:0/3668666702 learned_addr learned my addr 172.21.15.204:0/3668666702 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf09600640 1 -- 172.21.15.204:0/3668666702 >> v1:172.21.15.204:6789/0 conn(0x7fbf040b45d0 legacy=0x7fbf040cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf09600640 1 -- 172.21.15.204:0/3668666702 >> v1:172.21.15.143:6789/0 conn(0x7fbf040b4080 legacy=0x7fbf040ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.366+0000 7fbf09600640 1 -- 172.21.15.204:0/3668666702 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040c7b90 con 0x7fbf040b3610 2024-01-15T18:04:35.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf09600640 1 --2- 172.21.15.204:0/3668666702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b3610 0x7fbf040cfa40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009870 tx=0x7fbef401ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2cd08b67abc9c70 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf03e00640 1 -- 172.21.15.204:0/3668666702 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef40a8a30 con 0x7fbf040b3610 2024-01-15T18:04:35.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf03e00640 1 -- 172.21.15.204:0/3668666702 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef40a8b90 con 0x7fbf040b3610 2024-01-15T18:04:35.375 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf03e00640 1 -- 172.21.15.204:0/3668666702 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef40b0440 con 0x7fbf040b3610 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf0c000640 1 -- 172.21.15.204:0/3668666702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b3610 msgr2=0x7fbf040cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf0c000640 1 --2- 172.21.15.204:0/3668666702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b3610 0x7fbf040cfa40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009870 tx=0x7fbef401ed30 comp rx=0 tx=0).stop 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf0c000640 1 -- 172.21.15.204:0/3668666702 shutdown_connections 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf0c000640 1 --2- 172.21.15.204:0/3668666702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b3610 0x7fbf040cfa40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.370+0000 7fbf0c000640 1 -- 172.21.15.204:0/3668666702 >> 172.21.15.204:0/3668666702 conn(0x7fbf040d3030 msgr2=0x7fbf040d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 -- 172.21.15.204:0/3668666702 shutdown_connections 2024-01-15T18:04:35.376 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 -- 172.21.15.204:0/3668666702 wait complete. 2024-01-15T18:04:35.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 Processor -- start 2024-01-15T18:04:35.378 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 -- start start 2024-01-15T18:04:35.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf040b3610 0x7fbf040b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 0x7fbf04155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf040b45d0 0x7fbf04155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040b3ef0 con 0x7fbf040b3610 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040cadf0 con 0x7fbf040b4080 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040cb0d0 con 0x7fbf040b45d0 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf08c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 0x7fbf04155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf040b3610 0x7fbf040b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf08c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 0x7fbf04155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41718/0 (socket says 172.21.15.204:41718) 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf08c00640 1 -- 172.21.15.204:0/4187838490 learned_addr learned my addr 172.21.15.204:0/4187838490 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:35.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf0a000640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf040b45d0 0x7fbf04155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 -- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf040b45d0 msgr2=0x7fbf04155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf040b45d0 0x7fbf04155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 -- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 msgr2=0x7fbf04155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 0x7fbf04155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 -- 172.21.15.204:0/4187838490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40094d0 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf08c00640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 0x7fbf04155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.374+0000 7fbf09600640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf040b3610 0x7fbf040b83e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbef40af000 tx=0x7fbef401bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef40be070 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef40ba400 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf040ced70 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef40ba7b0 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf040cb690 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbef40ba910 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbef40c3d80 con 0x7fbf040b3610 2024-01-15T18:04:35.382 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.378+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf04067500 con 0x7fbf040b3610 2024-01-15T18:04:35.386 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.382+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbef40b5070 con 0x7fbf040b3610 2024-01-15T18:04:35.389 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.386+0000 7f8d39f99180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-15T18:04:35.415 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.411+0000 7f3d2e6e0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-15T18:04:35.451 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.446+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd new", "uuid": "597dd8ae-08f6-4609-9eee-19492d4f4354", "id": 4} v 0) v1 -- 0x7fbf040cf530 con 0x7fbf040b3610 2024-01-15T18:04:35.454 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf01600640 1 -- 172.21.15.204:0/4187838490 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "597dd8ae-08f6-4609-9eee-19492d4f4354", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbef40c3410 con 0x7fbf040b3610 2024-01-15T18:04:35.454 INFO:teuthology.orchestra.run.smithi204.stdout:4 2024-01-15T18:04:35.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf040b3610 msgr2=0x7fbf040b83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf040b3610 0x7fbf040b83e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbef40af000 tx=0x7fbef401bad0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 shutdown_connections 2024-01-15T18:04:35.455 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf040b45d0 0x7fbf04155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf040b4080 0x7fbf04155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 --2- 172.21.15.204:0/4187838490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf040b3610 0x7fbf040b83e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 >> 172.21.15.204:0/4187838490 conn(0x7fbf040d3030 msgr2=0x7fbf040c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 shutdown_connections 2024-01-15T18:04:35.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.450+0000 7fbf0c000640 1 -- 172.21.15.204:0/4187838490 wait complete. 2024-01-15T18:04:35.474 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 644e82b5-0dbb-46af-9320-65cf68f8e9b0 5 2024-01-15T18:04:35.480 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.478+0000 7f8d39f99180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-15T18:04:35.503 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.499+0000 7f3d2e6e0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-15T18:04:35.542 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.538+0000 7f8d39f99180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-15T18:04:35.563 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.559+0000 7f3d2e6e0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-15T18:04:35.614 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 Processor -- start 2024-01-15T18:04:35.614 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 -- start start 2024-01-15T18:04:35.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.616 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f091c15da90 0x7f091c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c0ce810 con 0x7f091c071520 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c0cc220 con 0x7f091c15da90 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f0922200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c15a4a0 con 0x7f091c071140 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f091b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f091b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c0ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:52358/0 (socket says 172.21.15.204:52358) 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f091b400640 1 -- 172.21.15.204:0/3405359437 learned_addr learned my addr 172.21.15.204:0/3405359437 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.610+0000 7f091be00640 1 --2- 172.21.15.204:0/3405359437 >> v2:172.21.15.143:3300/0 conn(0x7f091c15da90 0x7f091c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091b400640 1 -- 172.21.15.204:0/3405359437 >> v2:172.21.15.143:3300/0 conn(0x7f091c15da90 msgr2=0x7f091c15df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091b400640 1 --2- 172.21.15.204:0/3405359437 >> v2:172.21.15.143:3300/0 conn(0x7f091c15da90 0x7f091c15df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091b400640 1 -- 172.21.15.204:0/3405359437 >> v1:172.21.15.143:6789/0 conn(0x7f091c071520 legacy=0x7f091c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091b400640 1 -- 172.21.15.204:0/3405359437 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0cb980 con 0x7f091c071140 2024-01-15T18:04:35.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091b400640 1 --2- 172.21.15.204:0/3405359437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c0ce2d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f090c009870 tx=0x7f090c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1146ddb541f04a57 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091a000640 1 -- 172.21.15.204:0/3405359437 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c0aa670 con 0x7f091c071140 2024-01-15T18:04:35.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091a000640 1 -- 172.21.15.204:0/3405359437 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c0aa7d0 con 0x7f091c071140 2024-01-15T18:04:35.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.614+0000 7f091a000640 1 -- 172.21.15.204:0/3405359437 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c0aab60 con 0x7f091c071140 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- 172.21.15.204:0/3405359437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 msgr2=0x7f091c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 --2- 172.21.15.204:0/3405359437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c0ce2d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f090c009870 tx=0x7f090c01c510 comp rx=0 tx=0).stop 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- 172.21.15.204:0/3405359437 shutdown_connections 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 --2- 172.21.15.204:0/3405359437 >> v2:172.21.15.143:3300/0 conn(0x7f091c15da90 0x7f091c15df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 --2- 172.21.15.204:0/3405359437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c0ce2d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- 172.21.15.204:0/3405359437 >> 172.21.15.204:0/3405359437 conn(0x7f091c0cfd20 msgr2=0x7f091c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- 172.21.15.204:0/3405359437 shutdown_connections 2024-01-15T18:04:35.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- 172.21.15.204:0/3405359437 wait complete. 2024-01-15T18:04:35.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 Processor -- start 2024-01-15T18:04:35.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- start start 2024-01-15T18:04:35.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c071520 0x7f091c0b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 0x7f091c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0b5080 con 0x7f091c071520 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0b5360 con 0x7f091c15da90 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f0922200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0b5640 con 0x7f091c071140 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f091b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f091be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 0x7f091c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f091be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 0x7f091c0c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41722/0 (socket says 172.21.15.204:41722) 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f091b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c1541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:52362/0 (socket says 172.21.15.204:52362) 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f091be00640 1 -- 172.21.15.204:0/633603681 learned_addr learned my addr 172.21.15.204:0/633603681 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.618+0000 7f091aa00640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c071520 0x7f091c0b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091be00640 1 -- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 msgr2=0x7f091c1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091be00640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091be00640 1 -- 172.21.15.204:0/633603681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c071520 msgr2=0x7f091c0b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091be00640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c071520 0x7f091c0b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091be00640 1 -- 172.21.15.204:0/633603681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0094d0 con 0x7f091c15da90 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091b400640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:35.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f091be00640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 0x7f091c0c8cf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f091400efc0 tx=0x7f091400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914014480 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400f040 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c1629a0 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914014820 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c154710 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0914014c80 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f09140133e0 con 0x7f091c15da90 2024-01-15T18:04:35.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.622+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c067500 con 0x7f091c15da90 2024-01-15T18:04:35.628 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.626+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f091403bce0 con 0x7f091c15da90 2024-01-15T18:04:35.636 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.630+0000 7f8d39f99180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-15T18:04:35.651 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.647+0000 7f3d2e6e0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-15T18:04:35.695 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.690+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "644e82b5-0dbb-46af-9320-65cf68f8e9b0", "id": 5} v 0) v1 -- 0x7f091c162e50 con 0x7f091c15da90 2024-01-15T18:04:35.697 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.694+0000 7f8d39f99180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-15T18:04:35.707 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0913400640 1 -- 172.21.15.204:0/633603681 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "644e82b5-0dbb-46af-9320-65cf68f8e9b0", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f091402a070 con 0x7f091c15da90 2024-01-15T18:04:35.707 INFO:teuthology.orchestra.run.smithi204.stdout:5 2024-01-15T18:04:35.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 msgr2=0x7f091c0c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 0x7f091c0c8cf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f091400efc0 tx=0x7f091400c490 comp rx=0 tx=0).stop 2024-01-15T18:04:35.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 shutdown_connections 2024-01-15T18:04:35.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15da90 0x7f091c0c8cf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c071520 0x7f091c0b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.708 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 --2- 172.21.15.204:0/633603681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c071140 0x7f091c1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.709 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.702+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 >> 172.21.15.204:0/633603681 conn(0x7f091c0cfd20 msgr2=0x7f091c06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.709 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.706+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 shutdown_connections 2024-01-15T18:04:35.709 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.706+0000 7f0922200640 1 -- 172.21.15.204:0/633603681 wait complete. 2024-01-15T18:04:35.712 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.711+0000 7f3d2e6e0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-15T18:04:35.727 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64 6 2024-01-15T18:04:35.860 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.854+0000 7f8d39f99180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-15T18:04:35.860 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 Processor -- start 2024-01-15T18:04:35.861 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 -- start start 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc6f00cf440 0x7fc6f00cc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6f00cc5c0 0x7fc6f00b4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f015b640 con 0x7fc6f015b260 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00ce6b0 con 0x7fc6f00cf440 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f8a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00cfa90 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc6f015b260 0x7fc6f00cb970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:36592/0 (socket says 172.21.15.204:36592) 2024-01-15T18:04:35.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6000640 1 -- 172.21.15.204:0/1279508855 learned_addr learned my addr 172.21.15.204:0/1279508855 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6a00640 1 --2- 172.21.15.204:0/1279508855 >> v2:172.21.15.204:3300/0 conn(0x7fc6f00cc5c0 0x7fc6f00b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f5600640 1 --2- 172.21.15.204:0/1279508855 >> v2:172.21.15.143:3300/0 conn(0x7fc6f00cf440 0x7fc6f00cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f4c00640 1 -- 172.21.15.204:0/1279508855 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134991248 0 0) 0x7fc6f015b640 con 0x7fc6f015b260 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f4c00640 1 -- 172.21.15.204:0/1279508855 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d0003530 con 0x7fc6f015b260 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6a00640 1 -- 172.21.15.204:0/1279508855 >> v2:172.21.15.143:3300/0 conn(0x7fc6f00cf440 msgr2=0x7fc6f00cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6a00640 1 --2- 172.21.15.204:0/1279508855 >> v2:172.21.15.143:3300/0 conn(0x7fc6f00cf440 0x7fc6f00cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6a00640 1 -- 172.21.15.204:0/1279508855 >> v1:172.21.15.204:6789/0 conn(0x7fc6f015b260 legacy=0x7fc6f00cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f6a00640 1 -- 172.21.15.204:0/1279508855 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00b3d50 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.858+0000 7fc6f5600640 1 --2- 172.21.15.204:0/1279508855 >> v2:172.21.15.143:3300/0 conn(0x7fc6f00cf440 0x7fc6f00cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:35.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f6a00640 1 --2- 172.21.15.204:0/1279508855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00b4830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00713b0 tx=0x7fc6e801c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe2b28540e20bcc7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f4c00640 1 -- 172.21.15.204:0/1279508855 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80a5940 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f4c00640 1 -- 172.21.15.204:0/1279508855 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e80a5aa0 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f4c00640 1 -- 172.21.15.204:0/1279508855 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80a5e30 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 -- 172.21.15.204:0/1279508855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 msgr2=0x7fc6f00b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/1279508855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00b4830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00713b0 tx=0x7fc6e801c880 comp rx=0 tx=0).stop 2024-01-15T18:04:35.867 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 -- 172.21.15.204:0/1279508855 shutdown_connections 2024-01-15T18:04:35.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/1279508855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00b4830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/1279508855 >> v2:172.21.15.143:3300/0 conn(0x7fc6f00cf440 0x7fc6f00cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 -- 172.21.15.204:0/1279508855 >> 172.21.15.204:0/1279508855 conn(0x7fc6f00d0e60 msgr2=0x7fc6f00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 -- 172.21.15.204:0/1279508855 shutdown_connections 2024-01-15T18:04:35.868 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.862+0000 7fc6f8a00640 1 -- 172.21.15.204:0/1279508855 wait complete. 2024-01-15T18:04:35.868 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.862+0000 7f3d2e6e0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-15T18:04:35.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 Processor -- start 2024-01-15T18:04:35.869 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 -- start start 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 0x7fc6f015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6f015b260 0x7fc6f015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0165560 con 0x7fc6f00cf440 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00cb020 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0165760 con 0x7fc6f015b260 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.871 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6f015b260 0x7fc6f015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 0x7fc6f015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:41754/0 (socket says 172.21.15.204:41754) 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 0x7fc6f015c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.204:36830/0 (socket says 172.21.15.204:36830) 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 -- 172.21.15.204:0/739670252 learned_addr learned my addr 172.21.15.204:0/739670252 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 -- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6f015b260 msgr2=0x7fc6f015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6f015b260 0x7fc6f015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 -- 172.21.15.204:0/739670252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 msgr2=0x7fc6f015c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 0x7fc6f015c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 -- 172.21.15.204:0/739670252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8009420 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6a00640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6f015b260 0x7fc6f015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f5600640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 0x7fc6f015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:35.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f6000640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00c7f30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c420 tx=0x7fc6e000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0005cf0 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.873 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0005e50 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00072d0 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e8009a70 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f00cacb0 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc6e001c440 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.866+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fc6e001cce0 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.874 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.870+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f00d0cd0 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.875 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.870+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc6e0021060 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.932 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:35.926+0000 7f8d39f99180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-15T18:04:35.935 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:35.930+0000 7f3d2e6e0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-15T18:04:35.941 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.938+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64", "id": 6} v 0) v1 -- 0x7fc6f0067c10 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.946 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6ee000640 1 -- 172.21.15.204:0/739670252 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc6e0017070 con 0x7fc6f00cc5c0 2024-01-15T18:04:35.946 INFO:teuthology.orchestra.run.smithi204.stdout:6 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 msgr2=0x7fc6f00c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00c7f30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c420 tx=0x7fc6e000c8e0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 shutdown_connections 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6f015b260 0x7fc6f015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6f00cf440 0x7fc6f015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 --2- 172.21.15.204:0/739670252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6f00cc5c0 0x7fc6f00c7f30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 >> 172.21.15.204:0/739670252 conn(0x7fc6f00d0e60 msgr2=0x7fc6f00cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 shutdown_connections 2024-01-15T18:04:35.948 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:35.942+0000 7fc6f8a00640 1 -- 172.21.15.204:0/739670252 wait complete. 2024-01-15T18:04:35.966 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --cluster ceph osd new 80584af3-3415-4da6-b726-fa846967621e 7 2024-01-15T18:04:36.100 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face3a00640 1 Processor -- start 2024-01-15T18:04:36.101 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face3a00640 1 -- start start 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face3a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7facdc0ae620 0x7facdc15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face3a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc15e650 con 0x7facdc06aba0 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc0ad900 con 0x7facdc0ae240 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face3a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc160d80 con 0x7facdc0ae620 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face1000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7facdc0ae240 0x7facdc06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.204:36596/0 (socket says 172.21.15.204:36596) 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7face1000640 1 -- 172.21.15.204:0/1785961049 learned_addr learned my addr 172.21.15.204:0/1785961049 (peer_addr_for_me v1:172.21.15.204:0/0) 2024-01-15T18:04:36.102 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7facdbe00640 1 --2- 172.21.15.204:0/1785961049 >> v2:172.21.15.143:3300/0 conn(0x7facdc0ae620 0x7facdc15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.103 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7facdb400640 1 -- 172.21.15.204:0/1785961049 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370290355 0 0) 0x7facdc0ad900 con 0x7facdc0ae240 2024-01-15T18:04:36.103 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7facdb400640 1 -- 172.21.15.204:0/1785961049 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facbc003530 con 0x7facdc0ae240 2024-01-15T18:04:36.103 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7facdbe00640 1 -- 172.21.15.204:0/1785961049 >> v1:172.21.15.204:6789/0 conn(0x7facdc0ae240 legacy=0x7facdc06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.103 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7facdbe00640 1 -- 172.21.15.204:0/1785961049 >> v1:172.21.15.143:6789/0 conn(0x7facdc06aba0 legacy=0x7facdc15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.103 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.098+0000 7facdbe00640 1 -- 172.21.15.204:0/1785961049 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc161060 con 0x7facdc0ae620 2024-01-15T18:04:36.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7facdbe00640 1 --2- 172.21.15.204:0/1785961049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc0ae620 0x7facdc15d8e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7facc40097c0 tx=0x7facc401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8d6d07bdaf6dfd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.105 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7facdb400640 1 -- 172.21.15.204:0/1785961049 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40ab650 con 0x7facdc0ae620 2024-01-15T18:04:36.106 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7facdb400640 1 -- 172.21.15.204:0/1785961049 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc40ab7b0 con 0x7facdc0ae620 2024-01-15T18:04:36.106 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7facdb400640 1 -- 172.21.15.204:0/1785961049 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40abb40 con 0x7facdc0ae620 2024-01-15T18:04:36.106 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 -- 172.21.15.204:0/1785961049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc0ae620 msgr2=0x7facdc15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 --2- 172.21.15.204:0/1785961049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc0ae620 0x7facdc15d8e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7facc40097c0 tx=0x7facc401c450 comp rx=0 tx=0).stop 2024-01-15T18:04:36.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 -- 172.21.15.204:0/1785961049 shutdown_connections 2024-01-15T18:04:36.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 --2- 172.21.15.204:0/1785961049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc0ae620 0x7facdc15d8e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 -- 172.21.15.204:0/1785961049 >> 172.21.15.204:0/1785961049 conn(0x7facdc0cfd20 msgr2=0x7facdc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:36.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 -- 172.21.15.204:0/1785961049 shutdown_connections 2024-01-15T18:04:36.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.102+0000 7face3a00640 1 -- 172.21.15.204:0/1785961049 wait complete. 2024-01-15T18:04:36.108 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 Processor -- start 2024-01-15T18:04:36.108 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- start start 2024-01-15T18:04:36.109 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facdc06aba0 0x7facdc0b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facdc0ae240 0x7facdc0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc1519c0 0x7facdc15fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c5c20 con 0x7facdc1519c0 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c46d0 con 0x7facdc06aba0 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c52f0 con 0x7facdc0ae240 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facdc06aba0 0x7facdc0b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facdbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facdc0ae240 0x7facdc0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facdbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facdc0ae240 0x7facdc0c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.204:52386/0 (socket says 172.21.15.204:52386) 2024-01-15T18:04:36.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facdbe00640 1 -- 172.21.15.204:0/4023949309 learned_addr learned my addr 172.21.15.204:0/4023949309 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-01-15T18:04:36.111 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1a00640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc1519c0 0x7facdc15fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 -- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facdc0ae240 msgr2=0x7facdc0c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facdc0ae240 0x7facdc0c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 -- 172.21.15.204:0/4023949309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc1519c0 msgr2=0x7facdc15fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc1519c0 0x7facdc15fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 -- 172.21.15.204:0/4023949309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc4009420 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face1000640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facdc06aba0 0x7facdc0b5660 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faccc00c400 tx=0x7faccc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc005cf0 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc005e50 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc0c55d0 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc0072d0 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc0c65f0 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faccc01c440 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7faccc01cd50 con 0x7facdc06aba0 2024-01-15T18:04:36.112 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.106+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc0cb740 con 0x7facdc06aba0 2024-01-15T18:04:36.114 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.110+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7faccc021060 con 0x7facdc06aba0 2024-01-15T18:04:36.183 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.178+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd new", "uuid": "80584af3-3415-4da6-b726-fa846967621e", "id": 7} v 0) v1 -- 0x7facdc067500 con 0x7facdc06aba0 2024-01-15T18:04:36.188 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.182+0000 7facd8c00640 1 -- 172.21.15.204:0/4023949309 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "80584af3-3415-4da6-b726-fa846967621e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7faccc017070 con 0x7facdc06aba0 2024-01-15T18:04:36.188 INFO:teuthology.orchestra.run.smithi204.stdout:7 2024-01-15T18:04:36.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facdc06aba0 msgr2=0x7facdc0b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facdc06aba0 0x7facdc0b5660 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faccc00c400 tx=0x7faccc00c8c0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 shutdown_connections 2024-01-15T18:04:36.189 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facdc1519c0 0x7facdc15fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facdc0ae240 0x7facdc0c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 --2- 172.21.15.204:0/4023949309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facdc06aba0 0x7facdc0b5660 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 >> 172.21.15.204:0/4023949309 conn(0x7facdc0cfd20 msgr2=0x7facdc1534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:36.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 shutdown_connections 2024-01-15T18:04:36.190 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:04:36.186+0000 7face3a00640 1 -- 172.21.15.204:0/4023949309 wait complete. 2024-01-15T18:04:36.207 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-15T18:04:36.207 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-15T18:04:36.211 INFO:tasks.ceph.osd.0:Started 2024-01-15T18:04:36.211 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-15T18:04:36.211 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-15T18:04:36.214 INFO:tasks.ceph.osd.1:Started 2024-01-15T18:04:36.214 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-15T18:04:36.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-15T18:04:36.218 INFO:tasks.ceph.osd.2:Started 2024-01-15T18:04:36.218 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-15T18:04:36.218 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-15T18:04:36.221 INFO:tasks.ceph.osd.3:Started 2024-01-15T18:04:36.221 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-15T18:04:36.221 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-15T18:04:36.225 INFO:tasks.ceph.osd.4:Started 2024-01-15T18:04:36.225 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-15T18:04:36.225 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-15T18:04:36.228 INFO:tasks.ceph.osd.5:Started 2024-01-15T18:04:36.228 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-15T18:04:36.228 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-15T18:04:36.231 INFO:tasks.ceph.osd.6:Started 2024-01-15T18:04:36.232 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-15T18:04:36.232 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-15T18:04:36.241 INFO:tasks.ceph.osd.7:Started 2024-01-15T18:04:36.241 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:36.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.446+0000 7f8a67400640 1 Processor -- start 2024-01-15T18:04:36.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.446+0000 7f8a67400640 1 -- start start 2024-01-15T18:04:36.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a67400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a600b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a67400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a6006ece0 0x7f8a6006a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a67400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600b32b0 con 0x7f8a6006a9d0 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a67400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a60161e50 con 0x7f8a6006ece0 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a67400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6006ae60 con 0x7f8a60157af0 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a65400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a6006a9d0 0x7f8a6006d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38832/0 (socket says 172.21.15.143:38832) 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a65400640 1 -- 172.21.15.143:0/1437189334 learned_addr learned my addr 172.21.15.143:0/1437189334 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a5fe00640 1 --2- 172.21.15.143:0/1437189334 >> v2:172.21.15.204:3300/0 conn(0x7f8a6006ece0 0x7f8a6006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a64a00640 1 --2- 172.21.15.143:0/1437189334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a600b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a5f400640 1 -- 172.21.15.143:0/1437189334 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078390759 0 0) 0x7f8a600b32b0 con 0x7f8a6006a9d0 2024-01-15T18:04:36.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a5f400640 1 -- 172.21.15.143:0/1437189334 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a40003530 con 0x7f8a6006a9d0 2024-01-15T18:04:36.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a64a00640 1 -- 172.21.15.143:0/1437189334 >> v2:172.21.15.204:3300/0 conn(0x7f8a6006ece0 msgr2=0x7f8a6006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:36.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a64a00640 1 --2- 172.21.15.143:0/1437189334 >> v2:172.21.15.204:3300/0 conn(0x7f8a6006ece0 0x7f8a6006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a64a00640 1 -- 172.21.15.143:0/1437189334 >> v1:172.21.15.143:6789/0 conn(0x7f8a6006a9d0 legacy=0x7f8a6006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.450+0000 7f8a64a00640 1 -- 172.21.15.143:0/1437189334 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6015dba0 con 0x7f8a60157af0 2024-01-15T18:04:36.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a64a00640 1 --2- 172.21.15.143:0/1437189334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a600b2cd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8a500097f0 tx=0x7f8a5001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9115e228231037ed server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a5f400640 1 -- 172.21.15.143:0/1437189334 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500ab600 con 0x7f8a60157af0 2024-01-15T18:04:36.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a5f400640 1 -- 172.21.15.143:0/1437189334 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a500a7050 con 0x7f8a60157af0 2024-01-15T18:04:36.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a5f400640 1 -- 172.21.15.143:0/1437189334 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500ab980 con 0x7f8a60157af0 2024-01-15T18:04:36.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 -- 172.21.15.143:0/1437189334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 msgr2=0x7f8a600b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 --2- 172.21.15.143:0/1437189334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a600b2cd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8a500097f0 tx=0x7f8a5001c480 comp rx=0 tx=0).stop 2024-01-15T18:04:36.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 -- 172.21.15.143:0/1437189334 shutdown_connections 2024-01-15T18:04:36.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 --2- 172.21.15.143:0/1437189334 >> v2:172.21.15.204:3300/0 conn(0x7f8a6006ece0 0x7f8a6006a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 --2- 172.21.15.143:0/1437189334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a600b2cd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 -- 172.21.15.143:0/1437189334 >> 172.21.15.143:0/1437189334 conn(0x7f8a6006b5a0 msgr2=0x7f8a6006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:36.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 -- 172.21.15.143:0/1437189334 shutdown_connections 2024-01-15T18:04:36.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 -- 172.21.15.143:0/1437189334 wait complete. 2024-01-15T18:04:36.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 Processor -- start 2024-01-15T18:04:36.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.454+0000 7f8a67400640 1 -- start start 2024-01-15T18:04:36.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a6006a9d0 0x7f8a6015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 0x7f8a601531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a6014e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600ceed0 con 0x7f8a6006ece0 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6014def0 con 0x7f8a6006a9d0 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600ce630 con 0x7f8a60157af0 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 0x7f8a601531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 0x7f8a601531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33146/0 (socket says 172.21.15.143:33146) 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a6006a9d0 0x7f8a6015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a65400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a6014e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 -- 172.21.15.143:0/2660095063 learned_addr learned my addr 172.21.15.143:0/2660095063 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 -- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 msgr2=0x7f8a6014e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a6014e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 -- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a6006a9d0 msgr2=0x7f8a6015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a6006a9d0 0x7f8a6015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 -- 172.21.15.143:0/2660095063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009420 con 0x7f8a6006ece0 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a64a00640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a6006a9d0 0x7f8a6015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5fe00640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 0x7f8a601531d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e930 tx=0x7f8a4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48009690 con 0x7f8a6006ece0 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800ca00 con 0x7f8a6006ece0 2024-01-15T18:04:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4800cd70 con 0x7f8a6006ece0 2024-01-15T18:04:36.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a600ce910 con 0x7f8a6006ece0 2024-01-15T18:04:36.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a600ce250 con 0x7f8a6006ece0 2024-01-15T18:04:36.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8a48005ca0 con 0x7f8a6006ece0 2024-01-15T18:04:36.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f8a48013470 con 0x7f8a6006ece0 2024-01-15T18:04:36.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.458+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6014e060 con 0x7f8a6006ece0 2024-01-15T18:04:36.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.462+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8a48021060 con 0x7f8a6006ece0 2024-01-15T18:04:36.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.518+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a60067c10 con 0x7f8a6006ece0 2024-01-15T18:04:36.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.518+0000 7f8a5cc00640 1 -- 172.21.15.143:0/2660095063 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f8a48029020 con 0x7f8a6006ece0 2024-01-15T18:04:36.523 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:36.523 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":10,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:36.184250+0000","last_up_change":"0.000000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.518+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 msgr2=0x7f8a601531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.518+0000 7f8a67400640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 0x7f8a601531d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800e930 tx=0x7f8a4800edf0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 shutdown_connections 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a60157af0 0x7f8a6014e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a6006ece0 0x7f8a601531d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 --2- 172.21.15.143:0/2660095063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a6006a9d0 0x7f8a6015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 >> 172.21.15.143:0/2660095063 conn(0x7f8a6006b5a0 msgr2=0x7f8a6006e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 shutdown_connections 2024-01-15T18:04:36.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:36.522+0000 7f8a67400640 1 -- 172.21.15.143:0/2660095063 wait complete. 2024-01-15T18:04:36.544 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-15T18:04:36.544 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-15T18:04:36.568 INFO:tasks.ceph.mgr.y.smithi143.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-01-15T18:04:36.569 INFO:tasks.ceph.mgr.y.smithi143.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-01-15T18:04:36.569 INFO:tasks.ceph.mgr.y.smithi143.stderr: from numpy import show_config as show_numpy_config 2024-01-15T18:04:36.596 INFO:tasks.ceph.mgr.x.smithi204.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-01-15T18:04:36.596 INFO:tasks.ceph.mgr.x.smithi204.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-01-15T18:04:36.597 INFO:tasks.ceph.mgr.x.smithi204.stderr: from numpy import show_config as show_numpy_config 2024-01-15T18:04:36.616 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:36.610+0000 7f3d2e6e0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-15T18:04:36.660 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:36.658+0000 7f8d39f99180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-15T18:04:36.752 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:36.750+0000 7f3d2e6e0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-15T18:04:36.798 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:36.794+0000 7f8d39f99180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-15T18:04:36.893 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-15T18:04:36.890+0000 7f35389ef640 -1 Falling back to public interface 2024-01-15T18:04:36.894 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-15T18:04:36.890+0000 7fac1f359640 -1 Falling back to public interface 2024-01-15T18:04:36.917 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-15T18:04:36.914+0000 7fa927aba640 -1 Falling back to public interface 2024-01-15T18:04:36.922 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-15T18:04:36.918+0000 7fa3c482b640 -1 Falling back to public interface 2024-01-15T18:04:36.927 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:36.922+0000 7f3d2e6e0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-15T18:04:36.929 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-15T18:04:36.926+0000 7ff6c61cf640 -1 Falling back to public interface 2024-01-15T18:04:36.961 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-15T18:04:36.958+0000 7face985d640 -1 Falling back to public interface 2024-01-15T18:04:36.966 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-15T18:04:36.962+0000 7fafc1248640 -1 Falling back to public interface 2024-01-15T18:04:36.972 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:36.966+0000 7f8d39f99180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-15T18:04:36.977 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-15T18:04:36.974+0000 7fb1a2196640 -1 Falling back to public interface 2024-01-15T18:04:37.000 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:36.994+0000 7f3d2e6e0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-15T18:04:37.046 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:37.042+0000 7f8d39f99180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-15T18:04:37.136 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:37.130+0000 7f3d2e6e0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-15T18:04:37.184 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:37.178+0000 7f8d39f99180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-15T18:04:37.350 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:37.478 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 Processor -- start 2024-01-15T18:04:37.478 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 -- start start 2024-01-15T18:04:37.480 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ae00ceee0 0x7f4ae00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.480 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4ae015b320 0x7f4ae015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.480 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae015f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae00cfcb0 con 0x7f4ae00ceee0 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae015c150 con 0x7f4ae015b320 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ae4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae015c680 con 0x7f4ae00cea30 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ae00ceee0 0x7f4ae00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ae00ceee0 0x7f4ae00cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35288/0 (socket says 172.21.15.143:35288) 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 -- 172.21.15.143:0/1579824926 learned_addr learned my addr 172.21.15.143:0/1579824926 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4ade800640 1 --2- 172.21.15.143:0/1579824926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae015f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4add400640 1 --2- 172.21.15.143:0/1579824926 >> v2:172.21.15.204:3300/0 conn(0x7f4ae015b320 0x7f4ae015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 -- 172.21.15.143:0/1579824926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 msgr2=0x7f4ae015f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:37.481 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 --2- 172.21.15.143:0/1579824926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae015f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.482 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 -- 172.21.15.143:0/1579824926 >> v2:172.21.15.204:3300/0 conn(0x7f4ae015b320 msgr2=0x7f4ae015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:37.482 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 --2- 172.21.15.143:0/1579824926 >> v2:172.21.15.204:3300/0 conn(0x7f4ae015b320 0x7f4ae015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.482 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.474+0000 7f4adde00640 1 -- 172.21.15.143:0/1579824926 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00cdff0 con 0x7f4ae00ceee0 2024-01-15T18:04:37.483 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4adde00640 1 --2- 172.21.15.143:0/1579824926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00cf2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00d1f60 tx=0x7f4ad001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15e90ed6982d467 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.483 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4adca00640 1 -- 172.21.15.143:0/1579824926 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00ac630 con 0x7f4ae00ceee0 2024-01-15T18:04:37.483 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4adca00640 1 -- 172.21.15.143:0/1579824926 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad00ac790 con 0x7f4ae00ceee0 2024-01-15T18:04:37.483 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4adca00640 1 -- 172.21.15.143:0/1579824926 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00acb20 con 0x7f4ae00ceee0 2024-01-15T18:04:37.484 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 -- 172.21.15.143:0/1579824926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae00ceee0 msgr2=0x7f4ae00cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/1579824926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00cf2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00d1f60 tx=0x7f4ad001c410 comp rx=0 tx=0).stop 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 -- 172.21.15.143:0/1579824926 shutdown_connections 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/1579824926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae015f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/1579824926 >> v2:172.21.15.204:3300/0 conn(0x7f4ae015b320 0x7f4ae015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/1579824926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00cf2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 -- 172.21.15.143:0/1579824926 >> 172.21.15.143:0/1579824926 conn(0x7f4ae00d0e60 msgr2=0x7f4ae00d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.478+0000 7f4ae4c00640 1 -- 172.21.15.143:0/1579824926 shutdown_connections 2024-01-15T18:04:37.485 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 -- 172.21.15.143:0/1579824926 wait complete. 2024-01-15T18:04:37.489 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 Processor -- start 2024-01-15T18:04:37.489 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 -- start start 2024-01-15T18:04:37.489 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae0152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.489 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 0x7f4ae00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae00b6410 con 0x7f4ae015b320 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0159080 con 0x7f4ae00ceee0 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ae4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae0159360 con 0x7f4ae00cea30 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 0x7f4ae00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 0x7f4ae00b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35296/0 (socket says 172.21.15.143:35296) 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 -- 172.21.15.143:0/953370888 learned_addr learned my addr 172.21.15.143:0/953370888 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4adde00640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae0152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4add400640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 -- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 msgr2=0x7f4ae0152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.490 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae0152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 -- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ae00ceee0 msgr2=0x7f4ae00c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 -- 172.21.15.143:0/953370888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00094d0 con 0x7f4ae015b320 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4adde00640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae0152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4add400640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ade800640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 0x7f4ae00b4fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800efc0 tx=0x7f4ad800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.482+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8014480 con 0x7f4ae015b320 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad800f040 con 0x7f4ae015b320 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae0159640 con 0x7f4ae015b320 2024-01-15T18:04:37.491 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8014820 con 0x7f4ae015b320 2024-01-15T18:04:37.492 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae0159c00 con 0x7f4ae015b320 2024-01-15T18:04:37.492 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4ad8014c80 con 0x7f4ae015b320 2024-01-15T18:04:37.492 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4ad8013ab0 con 0x7f4ae015b320 2024-01-15T18:04:37.492 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae0159e20 con 0x7f4ae015b320 2024-01-15T18:04:37.493 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.486+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4ad8018020 con 0x7f4ae015b320 2024-01-15T18:04:37.550 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.546+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ae0067500 con 0x7f4ae015b320 2024-01-15T18:04:37.551 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.546+0000 7f4ad6000640 1 -- 172.21.15.143:0/953370888 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4ad8013d90 con 0x7f4ae015b320 2024-01-15T18:04:37.551 INFO:teuthology.misc.health.smithi143.stdout: 2024-01-15T18:04:37.552 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":10,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:36.184250+0000","last_up_change":"0.000000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:37.553 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 msgr2=0x7f4ae00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.553 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 0x7f4ae00b4fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800efc0 tx=0x7f4ad800c490 comp rx=0 tx=0).stop 2024-01-15T18:04:37.553 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 shutdown_connections 2024-01-15T18:04:37.553 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ae015b320 0x7f4ae00b4fe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.554 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ae00ceee0 0x7f4ae00c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.554 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 --2- 172.21.15.143:0/953370888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ae00cea30 0x7f4ae0152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.554 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 >> 172.21.15.143:0/953370888 conn(0x7f4ae00d0e60 msgr2=0x7f4ae00ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:37.554 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 shutdown_connections 2024-01-15T18:04:37.554 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:37.550+0000 7f4ae4c00640 1 -- 172.21.15.143:0/953370888 wait complete. 2024-01-15T18:04:37.569 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-15T18:04:37.852 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:37.850+0000 7f3d2e6e0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-15T18:04:37.866 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:37.862+0000 7f8d39f99180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-15T18:04:37.939 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:37.934+0000 7f3d2e6e0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-15T18:04:37.953 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:37.950+0000 7f8d39f99180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-15T18:04:38.130 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:38.126+0000 7f3d2e6e0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-15T18:04:38.146 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:38.142+0000 7f8d39f99180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-15T18:04:38.214 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:38.210+0000 7f3d2e6e0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-15T18:04:38.230 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:38.226+0000 7f8d39f99180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-15T18:04:38.540 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:38.534+0000 7f3d2e6e0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-15T18:04:38.557 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:38.554+0000 7f8d39f99180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-15T18:04:38.606 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:38.602+0000 7f3d2e6e0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-15T18:04:38.627 INFO:tasks.ceph.mgr.x.smithi204.stderr:2024-01-15T18:04:38.622+0000 7f8d39f99180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-15T18:04:39.628 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:39.622+0000 7f3cd4a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-15T18:04:39.628 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:04:39.626+0000 7f3cd4a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-15T18:04:40.742 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-15T18:04:40.738+0000 7fac1f359640 -1 osd.0 0 log_to_monitors true 2024-01-15T18:04:40.745 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-15T18:04:40.742+0000 7ff6c61cf640 -1 osd.5 0 log_to_monitors true 2024-01-15T18:04:40.832 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-15T18:04:40.826+0000 7fb1a2196640 -1 osd.7 0 log_to_monitors true 2024-01-15T18:04:40.836 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-15T18:04:40.830+0000 7f35389ef640 -1 osd.1 0 log_to_monitors true 2024-01-15T18:04:40.909 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-15T18:04:40.906+0000 7fa927aba640 -1 osd.2 0 log_to_monitors true 2024-01-15T18:04:40.931 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-15T18:04:40.926+0000 7fafc1248640 -1 osd.4 0 log_to_monitors true 2024-01-15T18:04:40.938 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-15T18:04:40.934+0000 7fa3c482b640 -1 osd.3 0 log_to_monitors true 2024-01-15T18:04:41.060 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-15T18:04:41.054+0000 7face985d640 -1 osd.6 0 log_to_monitors true 2024-01-15T18:04:44.377 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:44.590 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 Processor -- start 2024-01-15T18:04:44.593 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 -- start start 2024-01-15T18:04:44.593 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f955c14af60 0x7f955c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:44.593 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:44.593 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0719a0 con 0x7f955c0b2eb0 2024-01-15T18:04:44.593 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c0cc8b0 con 0x7f955c14af60 2024-01-15T18:04:44.593 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.586+0000 7f9562c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c0b3550 con 0x7f955c06c7d0 2024-01-15T18:04:44.594 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f955c14af60 0x7f955c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:44.594 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f955c14af60 0x7f955c06c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35392/0 (socket says 172.21.15.143:35392) 2024-01-15T18:04:44.594 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 -- 172.21.15.143:0/1217828331 learned_addr learned my addr 172.21.15.143:0/1217828331 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:44.594 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 -- 172.21.15.143:0/1217828331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 msgr2=0x7f955c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:44.594 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 --2- 172.21.15.143:0/1217828331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.595 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 -- 172.21.15.143:0/1217828331 >> v1:172.21.15.204:6789/0 conn(0x7f955c0b2eb0 legacy=0x7f955c070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:44.598 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.590+0000 7f955be00640 1 -- 172.21.15.143:0/1217828331 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0b3320 con 0x7f955c14af60 2024-01-15T18:04:44.599 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.594+0000 7f955be00640 1 --2- 172.21.15.143:0/1217828331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c06c290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f954401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f744e84ebccf639c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:44.600 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.594+0000 7f955aa00640 1 -- 172.21.15.143:0/1217828331 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440aa5f0 con 0x7f955c14af60 2024-01-15T18:04:44.600 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.594+0000 7f955aa00640 1 -- 172.21.15.143:0/1217828331 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440aa750 con 0x7f955c14af60 2024-01-15T18:04:44.600 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.594+0000 7f955aa00640 1 -- 172.21.15.143:0/1217828331 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95440aaae0 con 0x7f955c14af60 2024-01-15T18:04:44.601 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 -- 172.21.15.143:0/1217828331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 msgr2=0x7f955c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:44.601 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 --2- 172.21.15.143:0/1217828331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c06c290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9544009870 tx=0x7f954401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.602 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 -- 172.21.15.143:0/1217828331 shutdown_connections 2024-01-15T18:04:44.602 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 --2- 172.21.15.143:0/1217828331 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.602 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 --2- 172.21.15.143:0/1217828331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c06c290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.602 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 -- 172.21.15.143:0/1217828331 >> 172.21.15.143:0/1217828331 conn(0x7f955c06dc00 msgr2=0x7f955c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:44.603 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 -- 172.21.15.143:0/1217828331 shutdown_connections 2024-01-15T18:04:44.604 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.598+0000 7f9562c00640 1 -- 172.21.15.143:0/1217828331 wait complete. 2024-01-15T18:04:44.605 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 Processor -- start 2024-01-15T18:04:44.606 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 -- start start 2024-01-15T18:04:44.606 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c166c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:44.607 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0b2eb0 0x7f955c16b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:44.607 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f955be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c166c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:44.607 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f955be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c166c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:53740/0 (socket says 172.21.15.143:53740) 2024-01-15T18:04:44.607 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f955be00640 1 -- 172.21.15.143:0/3485728282 learned_addr learned my addr 172.21.15.143:0/3485728282 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:44.607 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c162a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c163c40 con 0x7f955c14af60 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c167190 con 0x7f955c0b2eb0 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c167450 con 0x7f955c06c7d0 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f955b400640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0b2eb0 0x7f955c16b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.602+0000 7f9560c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c162a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f955b400640 1 -- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 msgr2=0x7f955c166c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f955b400640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c166c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f955b400640 1 -- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 msgr2=0x7f955c162a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f955b400640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c162a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.609 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f955b400640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440094d0 con 0x7f955c0b2eb0 2024-01-15T18:04:44.610 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f955b400640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0b2eb0 0x7f955c16b2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b3f0 tx=0x7f954c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:44.610 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c011830 con 0x7f955c0b2eb0 2024-01-15T18:04:44.610 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c011990 con 0x7f955c0b2eb0 2024-01-15T18:04:44.610 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c011d40 con 0x7f955c0b2eb0 2024-01-15T18:04:44.612 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c06cc30 con 0x7f955c0b2eb0 2024-01-15T18:04:44.612 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c167a10 con 0x7f955c0b2eb0 2024-01-15T18:04:44.612 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.606+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c067680 con 0x7f955c0b2eb0 2024-01-15T18:04:44.613 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.610+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 4) v1 ==== 86104+0+0 (secure 0 0 0) 0x7f954c038450 con 0x7f955c0b2eb0 2024-01-15T18:04:44.615 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.610+0000 7f9553e00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9534066890 0x7f9534068d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:44.615 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.610+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f954c083790 con 0x7f955c0b2eb0 2024-01-15T18:04:44.615 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.610+0000 7f955be00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9534066890 0x7f9534068d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:44.615 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.610+0000 7f955be00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9534066890 0x7f9534068d50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f954401c510 tx=0x7f95440b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:44.623 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.618+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f954c0519b0 con 0x7f955c0b2eb0 2024-01-15T18:04:44.928 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.922+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f955c13c3b0 con 0x7f955c0b2eb0 2024-01-15T18:04:44.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.934+0000 7f9553e00640 1 -- 172.21.15.143:0/3485728282 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f954c015070 con 0x7f955c0b2eb0 2024-01-15T18:04:44.939 INFO:teuthology.misc.health.smithi143.stdout: 2024-01-15T18:04:44.940 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":12,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:42.627271+0000","last_up_change":"0.000000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:44.943 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.938+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9534066890 msgr2=0x7f9534068d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:44.943 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.938+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9534066890 0x7f9534068d50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f954401c510 tx=0x7f95440b5040 comp rx=0 tx=0).stop 2024-01-15T18:04:44.943 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.938+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0b2eb0 msgr2=0x7f955c16b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:44.943 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.938+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0b2eb0 0x7f955c16b2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f954c00b3f0 tx=0x7f954c00b8b0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.945 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 shutdown_connections 2024-01-15T18:04:44.945 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9534066890 0x7f9534068d50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.945 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f955c14af60 0x7f955c162a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.945 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f955c0b2eb0 0x7f955c16b2f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.945 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 --2- 172.21.15.143:0/3485728282 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f955c06c7d0 0x7f955c166c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:44.945 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 >> 172.21.15.143:0/3485728282 conn(0x7f955c06dc00 msgr2=0x7f955c16c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:44.946 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 shutdown_connections 2024-01-15T18:04:44.946 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:44.942+0000 7f9562c00640 1 -- 172.21.15.143:0/3485728282 wait complete. 2024-01-15T18:04:44.981 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-15T18:04:45.008 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-15T18:04:45.006+0000 7face3e00640 -1 osd.6 0 waiting for initial osdmap 2024-01-15T18:04:45.033 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-15T18:04:45.030+0000 7fa3be200640 -1 osd.3 0 waiting for initial osdmap 2024-01-15T18:04:45.042 INFO:tasks.ceph.osd.6.smithi204.stderr:2024-01-15T18:04:45.038+0000 7facda800640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.046 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-15T18:04:45.042+0000 7ff6c0600640 -1 osd.5 0 waiting for initial osdmap 2024-01-15T18:04:45.050 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-15T18:04:45.046+0000 7fafbb800640 -1 osd.4 0 waiting for initial osdmap 2024-01-15T18:04:45.065 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-15T18:04:45.062+0000 7fb19c600640 -1 osd.7 0 waiting for initial osdmap 2024-01-15T18:04:45.071 INFO:tasks.ceph.osd.3.smithi143.stderr:2024-01-15T18:04:45.066+0000 7fa3b4c00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.082 INFO:tasks.ceph.osd.4.smithi204.stderr:2024-01-15T18:04:45.078+0000 7fafb2200640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.091 INFO:tasks.ceph.osd.7.smithi204.stderr:2024-01-15T18:04:45.086+0000 7fb193000640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.091 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-15T18:04:45.086+0000 7f3532e00640 -1 osd.1 0 waiting for initial osdmap 2024-01-15T18:04:45.098 INFO:tasks.ceph.osd.5.smithi204.stderr:2024-01-15T18:04:45.094+0000 7ff6b7000640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.114 INFO:tasks.ceph.osd.1.smithi143.stderr:2024-01-15T18:04:45.110+0000 7f3529800640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.115 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-15T18:04:45.110+0000 7fa922000640 -1 osd.2 0 waiting for initial osdmap 2024-01-15T18:04:45.146 INFO:tasks.ceph.osd.2.smithi143.stderr:2024-01-15T18:04:45.142+0000 7fa918a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:45.171 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-15T18:04:45.166+0000 7fac19800640 -1 osd.0 0 waiting for initial osdmap 2024-01-15T18:04:45.202 INFO:tasks.ceph.osd.0.smithi143.stderr:2024-01-15T18:04:45.198+0000 7fac10200640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T18:04:51.788 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:51.924 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.917+0000 7f41e5000640 1 Processor -- start 2024-01-15T18:04:51.924 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.917+0000 7f41e5000640 1 -- start start 2024-01-15T18:04:51.925 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41e5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 0x7f41e0071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41e5000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f41e006c1f0 0x7f41e00cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41e5000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e0161230 con 0x7f41e0160cc0 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41e5000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e00cda00 con 0x7f41e006c1f0 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41e5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0160ad0 con 0x7f41e01606f0 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 0x7f41e0071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 0x7f41e0071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60410/0 (socket says 172.21.15.143:60410) 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 -- 172.21.15.143:0/2658542966 learned_addr learned my addr 172.21.15.143:0/2658542966 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:51.926 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41de800640 1 --2- 172.21.15.143:0/2658542966 >> v2:172.21.15.204:3300/0 conn(0x7f41e006c1f0 0x7f41e00cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.927 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 -- 172.21.15.143:0/2658542966 >> v2:172.21.15.204:3300/0 conn(0x7f41e006c1f0 msgr2=0x7f41e00cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:51.927 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 --2- 172.21.15.143:0/2658542966 >> v2:172.21.15.204:3300/0 conn(0x7f41e006c1f0 0x7f41e00cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.927 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 -- 172.21.15.143:0/2658542966 >> v1:172.21.15.204:6789/0 conn(0x7f41e0160cc0 legacy=0x7f41e006bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.927 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.921+0000 7f41dde00640 1 -- 172.21.15.143:0/2658542966 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0161510 con 0x7f41e01606f0 2024-01-15T18:04:51.929 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41dde00640 1 --2- 172.21.15.143:0/2658542966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 0x7f41e0071600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009870 tx=0x7f41d001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26949b39329804b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:51.929 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41dca00640 1 -- 172.21.15.143:0/2658542966 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00ab680 con 0x7f41e01606f0 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41dca00640 1 -- 172.21.15.143:0/2658542966 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d00ab7e0 con 0x7f41e01606f0 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41dca00640 1 -- 172.21.15.143:0/2658542966 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00abb70 con 0x7f41e01606f0 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 -- 172.21.15.143:0/2658542966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 msgr2=0x7f41e0071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 --2- 172.21.15.143:0/2658542966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 0x7f41e0071600 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009870 tx=0x7f41d001c480 comp rx=0 tx=0).stop 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 -- 172.21.15.143:0/2658542966 shutdown_connections 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 --2- 172.21.15.143:0/2658542966 >> v2:172.21.15.204:3300/0 conn(0x7f41e006c1f0 0x7f41e00cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.930 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 --2- 172.21.15.143:0/2658542966 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e01606f0 0x7f41e0071600 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.931 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 -- 172.21.15.143:0/2658542966 >> 172.21.15.143:0/2658542966 conn(0x7f41e00cfd30 msgr2=0x7f41e00d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:51.931 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 -- 172.21.15.143:0/2658542966 shutdown_connections 2024-01-15T18:04:51.931 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 -- 172.21.15.143:0/2658542966 wait complete. 2024-01-15T18:04:51.932 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.925+0000 7f41e5000640 1 Processor -- start 2024-01-15T18:04:51.932 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- start start 2024-01-15T18:04:51.935 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e006c1f0 0x7f41e015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.935 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 0x7f41e0151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.935 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e0160cc0 0x7f41e00adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.935 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0151e90 con 0x7f41e01606f0 2024-01-15T18:04:51.935 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01562d0 con 0x7f41e0160cc0 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01556c0 con 0x7f41e006c1f0 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 0x7f41e0151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 0x7f41e0151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34492/0 (socket says 172.21.15.143:34492) 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 -- 172.21.15.143:0/1895280218 learned_addr learned my addr 172.21.15.143:0/1895280218 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dde00640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e006c1f0 0x7f41e015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41de800640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e0160cc0 0x7f41e00adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 -- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e006c1f0 msgr2=0x7f41e015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e006c1f0 0x7f41e015a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 -- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e0160cc0 msgr2=0x7f41e00adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e0160cc0 0x7f41e00adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 -- 172.21.15.143:0/1895280218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00094d0 con 0x7f41e01606f0 2024-01-15T18:04:51.936 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41dd400640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 0x7f41e0151950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c3c0 tx=0x7f41c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:51.937 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80052d0 con 0x7f41e01606f0 2024-01-15T18:04:51.937 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8005430 con 0x7f41e01606f0 2024-01-15T18:04:51.937 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e01559a0 con 0x7f41e01606f0 2024-01-15T18:04:51.937 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8003840 con 0x7f41e01606f0 2024-01-15T18:04:51.937 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e0155f60 con 0x7f41e01606f0 2024-01-15T18:04:51.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.929+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e0067500 con 0x7f41e01606f0 2024-01-15T18:04:51.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.933+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f41c80039a0 con 0x7f41e01606f0 2024-01-15T18:04:51.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.933+0000 7f41d6000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41b80668f0 0x7f41b8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.933+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f41c8083550 con 0x7f41e01606f0 2024-01-15T18:04:51.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.933+0000 7f41dde00640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41b80668f0 0x7f41b8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.938 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.933+0000 7f41dde00640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41b80668f0 0x7f41b8068db0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009870 tx=0x7f41d00b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:51.943 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:51.937+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41c8050b80 con 0x7f41e01606f0 2024-01-15T18:04:52.095 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.089+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41e006e2c0 con 0x7f41e01606f0 2024-01-15T18:04:52.096 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.089+0000 7f41d6000640 1 -- 172.21.15.143:0/1895280218 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f41c80541a0 con 0x7f41e01606f0 2024-01-15T18:04:52.096 INFO:teuthology.misc.health.smithi143.stdout: 2024-01-15T18:04:52.096 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":16,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:49.487198+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:52.098 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41b80668f0 msgr2=0x7f41b8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.098 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41b80668f0 0x7f41b8068db0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009870 tx=0x7f41d00b6040 comp rx=0 tx=0).stop 2024-01-15T18:04:52.098 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 msgr2=0x7f41e0151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.098 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 0x7f41e0151950 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41c800c3c0 tx=0x7f41c800c880 comp rx=0 tx=0).stop 2024-01-15T18:04:52.098 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 shutdown_connections 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41b80668f0 0x7f41b8068db0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41e0160cc0 0x7f41e00adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41e01606f0 0x7f41e0151950 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 --2- 172.21.15.143:0/1895280218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41e006c1f0 0x7f41e015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 >> 172.21.15.143:0/1895280218 conn(0x7f41e00cfd30 msgr2=0x7f41e015ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 shutdown_connections 2024-01-15T18:04:52.099 INFO:teuthology.misc.health.smithi143.stderr:2024-01-15T18:04:52.093+0000 7f41e5000640 1 -- 172.21.15.143:0/1895280218 wait complete. 2024-01-15T18:04:52.115 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-15T18:04:52.116 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-15T18:04:52.116 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:04:52.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:04:52.119 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:04:52.120 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:04:52.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:04:52.124 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:04:52.124 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:04:52.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:04:52.127 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:04:52.127 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:04:52.128 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:04:52.131 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:04:52.131 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-15T18:04:52.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:52.220 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:04:52.221 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:04:52.230 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:04:52.231 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:04:52.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52e2600640 1 Processor -- start 2024-01-15T18:04:52.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52e2600640 1 -- start start 2024-01-15T18:04:52.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52e2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52dc06aba0 0x7f52dc15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52e2600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc0ae830 con 0x7f52dc15de90 2024-01-15T18:04:52.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52e2600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc0ad900 con 0x7f52dc0ae450 2024-01-15T18:04:52.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52e2600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc15e6c0 con 0x7f52dc06aba0 2024-01-15T18:04:52.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52dbe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f52dc15de90 0x7f52dc0ca370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49562/0 (socket says 172.21.15.143:49562) 2024-01-15T18:04:52.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52dbe00640 1 -- 172.21.15.143:0/1873494246 learned_addr learned my addr 172.21.15.143:0/1873494246 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:52.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52daa00640 1 --2- 172.21.15.143:0/1873494246 >> v2:172.21.15.143:3300/0 conn(0x7f52dc06aba0 0x7f52dc15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52daa00640 1 -- 172.21.15.143:0/1873494246 >> v1:172.21.15.204:6789/0 conn(0x7f52dc0ae450 legacy=0x7f52dc06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52daa00640 1 -- 172.21.15.143:0/1873494246 >> v1:172.21.15.143:6789/0 conn(0x7f52dc15de90 legacy=0x7f52dc0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52daa00640 1 -- 172.21.15.143:0/1873494246 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc160e20 con 0x7f52dc06aba0 2024-01-15T18:04:52.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.273+0000 7f52da000640 1 -- 172.21.15.143:0/1873494246 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727306987 0 0) 0x7f52dc0ae830 con 0x7f52dc15de90 2024-01-15T18:04:52.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52daa00640 1 --2- 172.21.15.143:0/1873494246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc06aba0 0x7f52dc15d950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52c40097c0 tx=0x7f52c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac40310823a80230 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52da000640 1 -- 172.21.15.143:0/1873494246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40ab650 con 0x7f52dc06aba0 2024-01-15T18:04:52.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52da000640 1 -- 172.21.15.143:0/1873494246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40ab7b0 con 0x7f52dc06aba0 2024-01-15T18:04:52.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52da000640 1 -- 172.21.15.143:0/1873494246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40abb40 con 0x7f52dc06aba0 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 -- 172.21.15.143:0/1873494246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc06aba0 msgr2=0x7f52dc15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 --2- 172.21.15.143:0/1873494246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc06aba0 0x7f52dc15d950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52c40097c0 tx=0x7f52c401c450 comp rx=0 tx=0).stop 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 -- 172.21.15.143:0/1873494246 shutdown_connections 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 --2- 172.21.15.143:0/1873494246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc06aba0 0x7f52dc15d950 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 -- 172.21.15.143:0/1873494246 >> 172.21.15.143:0/1873494246 conn(0x7f52dc0cfd20 msgr2=0x7f52dc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 -- 172.21.15.143:0/1873494246 shutdown_connections 2024-01-15T18:04:52.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.277+0000 7f52e2600640 1 -- 172.21.15.143:0/1873494246 wait complete. 2024-01-15T18:04:52.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 Processor -- start 2024-01-15T18:04:52.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 -- start start 2024-01-15T18:04:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc06aba0 0x7f52dc150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 0x7f52dc157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc15de90 0x7f52dc0c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc150bb0 con 0x7f52dc0ae450 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc155570 con 0x7f52dc15de90 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1563d0 con 0x7f52dc06aba0 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 0x7f52dc157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 0x7f52dc157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34548/0 (socket says 172.21.15.143:34548) 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 -- 172.21.15.143:0/2292894711 learned_addr learned my addr 172.21.15.143:0/2292894711 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52dbe00640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc15de90 0x7f52dc0c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52db400640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc06aba0 0x7f52dc150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 -- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc06aba0 msgr2=0x7f52dc150670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc06aba0 0x7f52dc150670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 -- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc15de90 msgr2=0x7f52dc0c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc15de90 0x7f52dc0c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 -- 172.21.15.143:0/2292894711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009420 con 0x7f52dc0ae450 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52dbe00640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc15de90 0x7f52dc0c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52db400640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc06aba0 0x7f52dc150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52daa00640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 0x7f52dc157240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52c40ad040 tx=0x7f52c401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40b5810 con 0x7f52dc0ae450 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40b59b0 con 0x7f52dc0ae450 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40b5d60 con 0x7f52dc0ae450 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc1557c0 con 0x7f52dc0ae450 2024-01-15T18:04:52.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.281+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc155d80 con 0x7f52dc0ae450 2024-01-15T18:04:52.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.285+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc067500 con 0x7f52dc0ae450 2024-01-15T18:04:52.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.285+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f52c40b4390 con 0x7f52dc0ae450 2024-01-15T18:04:52.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.285+0000 7f52d3400640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b40668f0 0x7f52b4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.285+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52c411eae0 con 0x7f52dc0ae450 2024-01-15T18:04:52.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.285+0000 7f52db400640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b40668f0 0x7f52b4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.285+0000 7f52db400640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b40668f0 0x7f52b4068db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52dc15da90 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.289+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52c40ec120 con 0x7f52dc0ae450 2024-01-15T18:04:52.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.441+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52dc0d0bc0 con 0x7f52dc0ae450 2024-01-15T18:04:52.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.441+0000 7f52d3400640 1 -- 172.21.15.143:0/2292894711 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f52c40ef740 con 0x7f52dc0ae450 2024-01-15T18:04:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:52.447 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:49.487198+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:52.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.441+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b40668f0 msgr2=0x7f52b4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.441+0000 7f52e2600640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b40668f0 0x7f52b4068db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52dc15da90 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-01-15T18:04:52.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.441+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 msgr2=0x7f52dc157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.441+0000 7f52e2600640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 0x7f52dc157240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f52c40ad040 tx=0x7f52c401ac90 comp rx=0 tx=0).stop 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 shutdown_connections 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b40668f0 0x7f52b4068db0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc15de90 0x7f52dc0c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0ae450 0x7f52dc157240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 --2- 172.21.15.143:0/2292894711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc06aba0 0x7f52dc150670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 >> 172.21.15.143:0/2292894711 conn(0x7f52dc0cfd20 msgr2=0x7f52dc0b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 shutdown_connections 2024-01-15T18:04:52.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.445+0000 7f52e2600640 1 -- 172.21.15.143:0/2292894711 wait complete. 2024-01-15T18:04:52.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:04:52.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:04:52.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 Processor -- start 2024-01-15T18:04:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 -- start start 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f42a0071d00 0x7f42a0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a015f310 con 0x7f42a015ec80 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a015fbb0 con 0x7f42a0071d00 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a7600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a0072690 con 0x7f42a0141660 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f429fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f42a015ec80 0x7f42a0141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49566/0 (socket says 172.21.15.143:49566) 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f429fe00640 1 -- 172.21.15.143:0/3541808291 learned_addr learned my addr 172.21.15.143:0/3541808291 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a4c00640 1 --2- 172.21.15.143:0/3541808291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f42a5600640 1 --2- 172.21.15.143:0/3541808291 >> v2:172.21.15.204:3300/0 conn(0x7f42a0071d00 0x7f42a0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f429f400640 1 -- 172.21.15.143:0/3541808291 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793147730 0 0) 0x7f42a015f310 con 0x7f42a015ec80 2024-01-15T18:04:52.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.601+0000 7f429f400640 1 -- 172.21.15.143:0/3541808291 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4280003530 con 0x7f42a015ec80 2024-01-15T18:04:52.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a4c00640 1 -- 172.21.15.143:0/3541808291 >> v2:172.21.15.204:3300/0 conn(0x7f42a0071d00 msgr2=0x7f42a0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:52.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a4c00640 1 --2- 172.21.15.143:0/3541808291 >> v2:172.21.15.204:3300/0 conn(0x7f42a0071d00 0x7f42a0072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a4c00640 1 -- 172.21.15.143:0/3541808291 >> v1:172.21.15.143:6789/0 conn(0x7f42a015ec80 legacy=0x7f42a0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a4c00640 1 -- 172.21.15.143:0/3541808291 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a015f5f0 con 0x7f42a0141660 2024-01-15T18:04:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a4c00640 1 --2- 172.21.15.143:0/3541808291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00ce560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f42900097c0 tx=0x7f429001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75583abc11ce38f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f429f400640 1 -- 172.21.15.143:0/3541808291 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42900ab600 con 0x7f42a0141660 2024-01-15T18:04:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f429f400640 1 -- 172.21.15.143:0/3541808291 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42900a7050 con 0x7f42a0141660 2024-01-15T18:04:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f429f400640 1 -- 172.21.15.143:0/3541808291 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42900ab960 con 0x7f42a0141660 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a7600640 1 -- 172.21.15.143:0/3541808291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 msgr2=0x7f42a00ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.605+0000 7f42a7600640 1 --2- 172.21.15.143:0/3541808291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00ce560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f42900097c0 tx=0x7f429001c480 comp rx=0 tx=0).stop 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- 172.21.15.143:0/3541808291 shutdown_connections 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 --2- 172.21.15.143:0/3541808291 >> v2:172.21.15.204:3300/0 conn(0x7f42a0071d00 0x7f42a0072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 --2- 172.21.15.143:0/3541808291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00ce560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- 172.21.15.143:0/3541808291 >> 172.21.15.143:0/3541808291 conn(0x7f42a006c3f0 msgr2=0x7f42a006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- 172.21.15.143:0/3541808291 shutdown_connections 2024-01-15T18:04:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- 172.21.15.143:0/3541808291 wait complete. 2024-01-15T18:04:52.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 Processor -- start 2024-01-15T18:04:52.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- start start 2024-01-15T18:04:52.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a0071d00 0x7f42a013dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 0x7f42a01393a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a013f130 con 0x7f42a015ec80 2024-01-15T18:04:52.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a013f410 con 0x7f42a0071d00 2024-01-15T18:04:52.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a7600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a013f6f0 con 0x7f42a0141660 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 0x7f42a01393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 0x7f42a01393a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34562/0 (socket says 172.21.15.143:34562) 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 -- 172.21.15.143:0/3831990984 learned_addr learned my addr 172.21.15.143:0/3831990984 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a4c00640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a0071d00 0x7f42a013dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f429fe00640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 -- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 msgr2=0x7f42a00aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 -- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a0071d00 msgr2=0x7f42a013dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a0071d00 0x7f42a013dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.609+0000 7f42a5600640 1 -- 172.21.15.143:0/3831990984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4290009420 con 0x7f42a015ec80 2024-01-15T18:04:52.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f42a5600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 0x7f42a01393a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f429800ec10 tx=0x7f429800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800cd20 con 0x7f42a015ec80 2024-01-15T18:04:52.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429800f040 con 0x7f42a015ec80 2024-01-15T18:04:52.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a00cd150 con 0x7f42a015ec80 2024-01-15T18:04:52.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42980146c0 con 0x7f42a015ec80 2024-01-15T18:04:52.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a013fcb0 con 0x7f42a015ec80 2024-01-15T18:04:52.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a013fed0 con 0x7f42a015ec80 2024-01-15T18:04:52.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f4298014a30 con 0x7f42a015ec80 2024-01-15T18:04:52.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f429cc00640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42780668f0 0x7f4278068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4298084f40 con 0x7f42a015ec80 2024-01-15T18:04:52.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.613+0000 7f42a4c00640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42780668f0 0x7f4278068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.617+0000 7f42a4c00640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42780668f0 0x7f4278068db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42900ad000 tx=0x7f4290008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.617+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42980524c0 con 0x7f42a015ec80 2024-01-15T18:04:52.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.769+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42a0067500 con 0x7f42a015ec80 2024-01-15T18:04:52.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.769+0000 7f429cc00640 1 -- 172.21.15.143:0/3831990984 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4298018020 con 0x7f42a015ec80 2024-01-15T18:04:52.776 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:04:52.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42780668f0 msgr2=0x7f4278068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42780668f0 0x7f4278068db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f42900ad000 tx=0x7f4290008040 comp rx=0 tx=0).stop 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 msgr2=0x7f42a01393a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 0x7f42a01393a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f429800ec10 tx=0x7f429800c490 comp rx=0 tx=0).stop 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 shutdown_connections 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42780668f0 0x7f4278068db0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42a015ec80 0x7f42a01393a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42a0141660 0x7f42a00aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 --2- 172.21.15.143:0/3831990984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42a0071d00 0x7f42a013dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 >> 172.21.15.143:0/3831990984 conn(0x7f42a006c3f0 msgr2=0x7f42a0070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 shutdown_connections 2024-01-15T18:04:52.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.773+0000 7f42a7600640 1 -- 172.21.15.143:0/3831990984 wait complete. 2024-01-15T18:04:52.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:04:52.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 Processor -- start 2024-01-15T18:04:52.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 -- start start 2024-01-15T18:04:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f146c0b0310 0x7f146c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f146c0cf740 0x7f146c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0cbff0 con 0x7f146c06bfb0 2024-01-15T18:04:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0d0c50 con 0x7f146c0cf740 2024-01-15T18:04:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f1472000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0cb6c0 con 0x7f146c0b0310 2024-01-15T18:04:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f146c06bfb0 0x7f146c06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49568/0 (socket says 172.21.15.143:49568) 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146b400640 1 -- 172.21.15.143:0/3866850950 learned_addr learned my addr 172.21.15.143:0/3866850950 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146be00640 1 --2- 172.21.15.143:0/3866850950 >> v2:172.21.15.143:3300/0 conn(0x7f146c0cf740 0x7f146c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146aa00640 1 --2- 172.21.15.143:0/3866850950 >> v2:172.21.15.204:3300/0 conn(0x7f146c0b0310 0x7f146c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146a000640 1 -- 172.21.15.143:0/3866850950 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720317616 0 0) 0x7f146c0cbff0 con 0x7f146c06bfb0 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146a000640 1 -- 172.21.15.143:0/3866850950 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003530 con 0x7f146c06bfb0 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146be00640 1 -- 172.21.15.143:0/3866850950 >> v2:172.21.15.204:3300/0 conn(0x7f146c0b0310 msgr2=0x7f146c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146be00640 1 --2- 172.21.15.143:0/3866850950 >> v2:172.21.15.204:3300/0 conn(0x7f146c0b0310 0x7f146c0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146be00640 1 -- 172.21.15.143:0/3866850950 >> v1:172.21.15.143:6789/0 conn(0x7f146c06bfb0 legacy=0x7f146c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.929+0000 7f146be00640 1 -- 172.21.15.143:0/3866850950 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0cb9a0 con 0x7f146c0cf740 2024-01-15T18:04:52.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f146be00640 1 --2- 172.21.15.143:0/3866850950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c0cf740 0x7f146c0cfbf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f146c06bf00 tx=0x7f146401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad86c7c2a13104e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f146a000640 1 -- 172.21.15.143:0/3866850950 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14640a5910 con 0x7f146c0cf740 2024-01-15T18:04:52.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f146a000640 1 -- 172.21.15.143:0/3866850950 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14640a5a70 con 0x7f146c0cf740 2024-01-15T18:04:52.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f146a000640 1 -- 172.21.15.143:0/3866850950 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14640a5e00 con 0x7f146c0cf740 2024-01-15T18:04:52.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 -- 172.21.15.143:0/3866850950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c0cf740 msgr2=0x7f146c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 --2- 172.21.15.143:0/3866850950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c0cf740 0x7f146c0cfbf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f146c06bf00 tx=0x7f146401c850 comp rx=0 tx=0).stop 2024-01-15T18:04:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 -- 172.21.15.143:0/3866850950 shutdown_connections 2024-01-15T18:04:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 --2- 172.21.15.143:0/3866850950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c0cf740 0x7f146c0cfbf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 --2- 172.21.15.143:0/3866850950 >> v2:172.21.15.204:3300/0 conn(0x7f146c0b0310 0x7f146c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 -- 172.21.15.143:0/3866850950 >> 172.21.15.143:0/3866850950 conn(0x7f146c0d0ed0 msgr2=0x7f146c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 -- 172.21.15.143:0/3866850950 shutdown_connections 2024-01-15T18:04:52.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.933+0000 7f1472000640 1 -- 172.21.15.143:0/3866850950 wait complete. 2024-01-15T18:04:52.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 Processor -- start 2024-01-15T18:04:52.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 -- start start 2024-01-15T18:04:52.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 0x7f146c154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f146c0b0310 0x7f146c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f146c0cf740 0x7f146c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0caaf0 con 0x7f146c06bfb0 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0c78e0 con 0x7f146c0b0310 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0cb1a0 con 0x7f146c0cf740 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 0x7f146c154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 0x7f146c154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34578/0 (socket says 172.21.15.143:34578) 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 -- 172.21.15.143:0/3528877795 learned_addr learned my addr 172.21.15.143:0/3528877795 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146aa00640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f146c0b0310 0x7f146c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146be00640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f146c0cf740 0x7f146c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 -- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f146c0cf740 msgr2=0x7f146c0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f146c0cf740 0x7f146c0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 -- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f146c0b0310 msgr2=0x7f146c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f146c0b0310 0x7f146c151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 -- 172.21.15.143:0/3528877795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1464009a70 con 0x7f146c06bfb0 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f146b400640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 0x7f146c154b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f145c00c420 tx=0x7f145c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c005cf0 con 0x7f146c06bfb0 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c005e50 con 0x7f146c06bfb0 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0072d0 con 0x7f146c06bfb0 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0cb480 con 0x7f146c06bfb0 2024-01-15T18:04:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.937+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c0c7600 con 0x7f146c06bfb0 2024-01-15T18:04:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.941+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0c7180 con 0x7f146c06bfb0 2024-01-15T18:04:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.941+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f145c007430 con 0x7f146c06bfb0 2024-01-15T18:04:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.941+0000 7f1463400640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14440668f0 0x7f1444068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.941+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f145c083690 con 0x7f146c06bfb0 2024-01-15T18:04:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.941+0000 7f146aa00640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14440668f0 0x7f1444068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.941+0000 7f146aa00640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14440668f0 0x7f1444068db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f146c154220 tx=0x7f1454008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:52.945+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f146c0c7180 con 0x7f146c06bfb0 2024-01-15T18:04:53.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.125+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f146c0c7180 con 0x7f146c06bfb0 2024-01-15T18:04:53.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.125+0000 7f1463400640 1 -- 172.21.15.143:0/3528877795 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f146c0c7180 con 0x7f146c06bfb0 2024-01-15T18:04:53.131 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:53.131 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-15T18:04:53.132 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 2 2024-01-15T18:04:53.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14440668f0 msgr2=0x7f1444068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14440668f0 0x7f1444068db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f146c154220 tx=0x7f1454008040 comp rx=0 tx=0).stop 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 msgr2=0x7f146c154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 0x7f146c154b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f145c00c420 tx=0x7f145c00c8e0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 shutdown_connections 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14440668f0 0x7f1444068db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f146c0cf740 0x7f146c0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f146c0b0310 0x7f146c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 --2- 172.21.15.143:0/3528877795 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f146c06bfb0 0x7f146c154b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 >> 172.21.15.143:0/3528877795 conn(0x7f146c0d0ed0 msgr2=0x7f146c06f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:53.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 shutdown_connections 2024-01-15T18:04:53.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:53.129+0000 7f1472000640 1 -- 172.21.15.143:0/3528877795 wait complete. 2024-01-15T18:04:55.154 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:04:55.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 Processor -- start 2024-01-15T18:04:55.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 -- start start 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f446406bfb0 0x7f446406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44640b0310 0x7f44640cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44640cc9c0 0x7f44640cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44640cd3b0 con 0x7f44640b0310 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44640cbff0 con 0x7f446406bfb0 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f446a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640d0c50 con 0x7f44640cc9c0 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44640b0310 0x7f44640cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44640b0310 0x7f44640cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34588/0 (socket says 172.21.15.143:34588) 2024-01-15T18:04:55.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 -- 172.21.15.143:0/468888554 learned_addr learned my addr 172.21.15.143:0/468888554 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4463400640 1 --2- 172.21.15.143:0/468888554 >> v2:172.21.15.204:3300/0 conn(0x7f446406bfb0 0x7f446406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4463e00640 1 --2- 172.21.15.143:0/468888554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44640cc9c0 0x7f44640cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 -- 172.21.15.143:0/468888554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44640cc9c0 msgr2=0x7f44640cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 --2- 172.21.15.143:0/468888554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44640cc9c0 0x7f44640cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 -- 172.21.15.143:0/468888554 >> v2:172.21.15.204:3300/0 conn(0x7f446406bfb0 msgr2=0x7f446406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 --2- 172.21.15.143:0/468888554 >> v2:172.21.15.204:3300/0 conn(0x7f446406bfb0 0x7f446406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.297+0000 7f4462a00640 1 -- 172.21.15.143:0/468888554 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446406c8d0 con 0x7f44640b0310 2024-01-15T18:04:55.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f4462a00640 1 --2- 172.21.15.143:0/468888554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640b0310 0x7f44640cc480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f44640716c0 tx=0x7f444c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117a20138bb793b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f4462000640 1 -- 172.21.15.143:0/468888554 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0ab600 con 0x7f44640b0310 2024-01-15T18:04:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f4462000640 1 -- 172.21.15.143:0/468888554 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c0ab760 con 0x7f44640b0310 2024-01-15T18:04:55.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f4462000640 1 -- 172.21.15.143:0/468888554 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0abaf0 con 0x7f44640b0310 2024-01-15T18:04:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 -- 172.21.15.143:0/468888554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640b0310 msgr2=0x7f44640cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 --2- 172.21.15.143:0/468888554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640b0310 0x7f44640cc480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f44640716c0 tx=0x7f444c01c440 comp rx=0 tx=0).stop 2024-01-15T18:04:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 -- 172.21.15.143:0/468888554 shutdown_connections 2024-01-15T18:04:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 --2- 172.21.15.143:0/468888554 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44640cc9c0 0x7f44640cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 --2- 172.21.15.143:0/468888554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640b0310 0x7f44640cc480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 --2- 172.21.15.143:0/468888554 >> v2:172.21.15.204:3300/0 conn(0x7f446406bfb0 0x7f446406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 -- 172.21.15.143:0/468888554 >> 172.21.15.143:0/468888554 conn(0x7f44640d0ed0 msgr2=0x7f44640d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 -- 172.21.15.143:0/468888554 shutdown_connections 2024-01-15T18:04:55.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.301+0000 7f446a200640 1 -- 172.21.15.143:0/468888554 wait complete. 2024-01-15T18:04:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 Processor -- start 2024-01-15T18:04:55.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 -- start start 2024-01-15T18:04:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f446406bfb0 0x7f4464150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44640b0310 0x7f44641528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 0x7f44641566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44641514e0 con 0x7f44640cc9c0 2024-01-15T18:04:55.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464157140 con 0x7f44640b0310 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4464157420 con 0x7f446406bfb0 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 0x7f44641566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 0x7f44641566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34600/0 (socket says 172.21.15.143:34600) 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 -- 172.21.15.143:0/1661855113 learned_addr learned my addr 172.21.15.143:0/1661855113 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463400640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f446406bfb0 0x7f4464150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4462a00640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44640b0310 0x7f44641528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 -- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f446406bfb0 msgr2=0x7f4464150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f446406bfb0 0x7f4464150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 -- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44640b0310 msgr2=0x7f44641528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44640b0310 0x7f44641528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 -- 172.21.15.143:0/1661855113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c0094d0 con 0x7f44640cc9c0 2024-01-15T18:04:55.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f4463e00640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 0x7f44641566f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f445c007e60 tx=0x7f445c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c014420 con 0x7f44640cc9c0 2024-01-15T18:04:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c0145a0 con 0x7f44640cc9c0 2024-01-15T18:04:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4464151dc0 con 0x7f44640cc9c0 2024-01-15T18:04:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c014950 con 0x7f44640cc9c0 2024-01-15T18:04:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.305+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4464152380 con 0x7f44640cc9c0 2024-01-15T18:04:55.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.309+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4464067500 con 0x7f44640cc9c0 2024-01-15T18:04:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.309+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f445c014ab0 con 0x7f44640cc9c0 2024-01-15T18:04:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.309+0000 7f445b400640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f443c0668f0 0x7f443c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.309+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f445c0848b0 con 0x7f44640cc9c0 2024-01-15T18:04:55.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.309+0000 7f4463400640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f443c0668f0 0x7f443c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.309+0000 7f4463400640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f443c0668f0 0x7f443c068db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f44640afeb0 tx=0x7f4454008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.313+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f445c051e50 con 0x7f44640cc9c0 2024-01-15T18:04:55.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44640d1db0 con 0x7f44640cc9c0 2024-01-15T18:04:55.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f445b400640 1 -- 172.21.15.143:0/1661855113 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f445c055470 con 0x7f44640cc9c0 2024-01-15T18:04:55.502 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:55.502 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-15T18:04:55.502 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 3 2024-01-15T18:04:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f443c0668f0 msgr2=0x7f443c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f443c0668f0 0x7f443c068db0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f44640afeb0 tx=0x7f4454008040 comp rx=0 tx=0).stop 2024-01-15T18:04:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 msgr2=0x7f44641566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 0x7f44641566f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f445c007e60 tx=0x7f445c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 shutdown_connections 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f443c0668f0 0x7f443c068db0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44640cc9c0 0x7f44641566f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44640b0310 0x7f44641528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 --2- 172.21.15.143:0/1661855113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f446406bfb0 0x7f4464150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 >> 172.21.15.143:0/1661855113 conn(0x7f44640d0ed0 msgr2=0x7f44640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 shutdown_connections 2024-01-15T18:04:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.497+0000 7f446a200640 1 -- 172.21.15.143:0/1661855113 wait complete. 2024-01-15T18:04:55.524 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:55.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.653+0000 7fb783c00640 1 Processor -- start 2024-01-15T18:04:55.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.653+0000 7fb783c00640 1 -- start start 2024-01-15T18:04:55.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.653+0000 7fb783c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb77c143780 0x7fb77c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.653+0000 7fb783c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77c0ceb50 con 0x7fb77c0722c0 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.653+0000 7fb783c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77c15c110 con 0x7fb77c0ce520 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.653+0000 7fb783c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77c15b7e0 con 0x7fb77c143780 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb77c143780 0x7fb77c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb77c0722c0 0x7fb77c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49576/0 (socket says 172.21.15.143:49576) 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781c00640 1 -- 172.21.15.143:0/370509099 learned_addr learned my addr 172.21.15.143:0/370509099 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781200640 1 -- 172.21.15.143:0/370509099 >> v1:172.21.15.204:6789/0 conn(0x7fb77c0ce520 legacy=0x7fb77c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781200640 1 -- 172.21.15.143:0/370509099 >> v1:172.21.15.143:6789/0 conn(0x7fb77c0722c0 legacy=0x7fb77c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781200640 1 -- 172.21.15.143:0/370509099 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c15bac0 con 0x7fb77c143780 2024-01-15T18:04:55.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb77b400640 1 -- 172.21.15.143:0/370509099 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30111458 0 0) 0x7fb7740014a0 con 0x7fb77c0722c0 2024-01-15T18:04:55.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb781200640 1 --2- 172.21.15.143:0/370509099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c143780 0x7fb77c0cdfe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0097c0 tx=0x7fb76c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290078f398083e3e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb77b400640 1 -- 172.21.15.143:0/370509099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0a8050 con 0x7fb77c143780 2024-01-15T18:04:55.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb77b400640 1 -- 172.21.15.143:0/370509099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c0ac5f0 con 0x7fb77c143780 2024-01-15T18:04:55.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.657+0000 7fb77b400640 1 -- 172.21.15.143:0/370509099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0ac980 con 0x7fb77c143780 2024-01-15T18:04:55.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- 172.21.15.143:0/370509099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c143780 msgr2=0x7fb77c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 --2- 172.21.15.143:0/370509099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c143780 0x7fb77c0cdfe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0097c0 tx=0x7fb76c01c480 comp rx=0 tx=0).stop 2024-01-15T18:04:55.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- 172.21.15.143:0/370509099 shutdown_connections 2024-01-15T18:04:55.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 --2- 172.21.15.143:0/370509099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c143780 0x7fb77c0cdfe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- 172.21.15.143:0/370509099 >> 172.21.15.143:0/370509099 conn(0x7fb77c06c3f0 msgr2=0x7fb77c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:55.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- 172.21.15.143:0/370509099 shutdown_connections 2024-01-15T18:04:55.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- 172.21.15.143:0/370509099 wait complete. 2024-01-15T18:04:55.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 Processor -- start 2024-01-15T18:04:55.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- start start 2024-01-15T18:04:55.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 0x7fb77c142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb77c0ce520 0x7fb77c1f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb77c143780 0x7fb77c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c135b10 con 0x7fb77c0722c0 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1f6b00 con 0x7fb77c143780 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb783c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1f6da0 con 0x7fb77c0ce520 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb781200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 0x7fb77c142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb781200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 0x7fb77c142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34624/0 (socket says 172.21.15.143:34624) 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb781200640 1 -- 172.21.15.143:0/3467919428 learned_addr learned my addr 172.21.15.143:0/3467919428 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb77be00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb77c0ce520 0x7fb77c1f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.661+0000 7fb781c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb77c143780 0x7fb77c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781200640 1 -- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb77c0ce520 msgr2=0x7fb77c1f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781200640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb77c0ce520 0x7fb77c1f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781200640 1 -- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb77c143780 msgr2=0x7fb77c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781200640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb77c143780 0x7fb77c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781200640 1 -- 172.21.15.143:0/3467919428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c009420 con 0x7fb77c0722c0 2024-01-15T18:04:55.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb77c143780 0x7fb77c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb781200640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 0x7fb77c142780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb76c01a0b0 tx=0x7fb76c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0b8860 con 0x7fb77c0722c0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76c0b89c0 con 0x7fb77c0722c0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0b8d50 con 0x7fb77c0722c0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c1f7080 con 0x7fb77c0722c0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c0ace20 con 0x7fb77c0722c0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77c0cc390 con 0x7fb77c0722c0 2024-01-15T18:04:55.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb76c0c2410 con 0x7fb77c0722c0 2024-01-15T18:04:55.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb778c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7540668f0 0x7fb754068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb76c11e550 con 0x7fb77c0722c0 2024-01-15T18:04:55.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.665+0000 7fb77be00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7540668f0 0x7fb754068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.669+0000 7fb77be00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7540668f0 0x7fb754068db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb77c135560 tx=0x7fb764009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.669+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb77c0cc390 con 0x7fb77c0722c0 2024-01-15T18:04:55.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.821+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb77c0cc390 con 0x7fb77c0722c0 2024-01-15T18:04:55.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.821+0000 7fb778c00640 1 -- 172.21.15.143:0/3467919428 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fb77c0cc390 con 0x7fb77c0722c0 2024-01-15T18:04:55.827 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:55.827 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":16,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:49.487198+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:55.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7540668f0 msgr2=0x7fb754068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7540668f0 0x7fb754068db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb77c135560 tx=0x7fb764009040 comp rx=0 tx=0).stop 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 msgr2=0x7fb77c142780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 0x7fb77c142780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb76c01a0b0 tx=0x7fb76c01acc0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 shutdown_connections 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7540668f0 0x7fb754068db0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb77c143780 0x7fb77c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb77c0ce520 0x7fb77c1f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 --2- 172.21.15.143:0/3467919428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb77c0722c0 0x7fb77c142780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 >> 172.21.15.143:0/3467919428 conn(0x7fb77c06c3f0 msgr2=0x7fb77c0af140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 shutdown_connections 2024-01-15T18:04:55.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.825+0000 7fb783c00640 1 -- 172.21.15.143:0/3467919428 wait complete. 2024-01-15T18:04:55.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:04:55.849 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:04:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.981+0000 7eff39400640 1 Processor -- start 2024-01-15T18:04:55.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.981+0000 7eff39400640 1 -- start start 2024-01-15T18:04:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff39400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7eff34071140 0x7eff340ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff39400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340ce810 con 0x7eff34071520 2024-01-15T18:04:55.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff39400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340cb980 con 0x7eff3415da90 2024-01-15T18:04:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff39400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff340cc220 con 0x7eff34071140 2024-01-15T18:04:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7eff34071520 0x7eff3415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49580/0 (socket says 172.21.15.143:49580) 2024-01-15T18:04:55.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31e00640 1 -- 172.21.15.143:0/23064950 learned_addr learned my addr 172.21.15.143:0/23064950 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff32800640 1 --2- 172.21.15.143:0/23064950 >> v2:172.21.15.204:3300/0 conn(0x7eff34071140 0x7eff340ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832439916 0 0) 0x7eff340ce810 con 0x7eff34071520 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff14005740 con 0x7eff34071520 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3438260910 0 0) 0x7eff14005740 con 0x7eff34071520 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 >> v2:172.21.15.204:3300/0 conn(0x7eff34071140 msgr2=0x7eff340ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 --2- 172.21.15.143:0/23064950 >> v2:172.21.15.204:3300/0 conn(0x7eff34071140 0x7eff340ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 >> v1:172.21.15.204:6789/0 conn(0x7eff3415da90 legacy=0x7eff34164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340cbc60 con 0x7eff34071520 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7eff1c001680 con 0x7eff34071520 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 >> v1:172.21.15.143:6789/0 conn(0x7eff34071520 legacy=0x7eff3415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff14005b90 0x7eff14005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff140069e0 0x7eff14006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff1400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340cc220 con 0x7eff140073a0 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400e240 con 0x7eff140069e0 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400e400 con 0x7eff14005b90 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff33200640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff1400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff31e00640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff140069e0 0x7eff14006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff32800640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff14005b90 0x7eff14005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff33200640 1 -- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff14005b90 msgr2=0x7eff14005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff33200640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff14005b90 0x7eff14005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff33200640 1 -- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff140069e0 msgr2=0x7eff14006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff33200640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff140069e0 0x7eff14006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.985+0000 7eff33200640 1 -- 172.21.15.143:0/23064950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14005740 con 0x7eff140073a0 2024-01-15T18:04:55.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff33200640 1 -- 172.21.15.143:0/23064950 reap_dead start 2024-01-15T18:04:55.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff33200640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff1400dba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff2c001e70 tx=0x7eff2c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b4a218532c2f3c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c003b80 con 0x7eff140073a0 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c003d00 con 0x7eff140073a0 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff31400640 1 -- 172.21.15.143:0/23064950 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c005260 con 0x7eff140073a0 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 -- 172.21.15.143:0/23064950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 msgr2=0x7eff1400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff1400dba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7eff2c001e70 tx=0x7eff2c01c760 comp rx=0 tx=0).stop 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 -- 172.21.15.143:0/23064950 shutdown_connections 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 --2- 172.21.15.143:0/23064950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff1400dba0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 -- 172.21.15.143:0/23064950 >> 172.21.15.143:0/23064950 conn(0x7eff340cfd20 msgr2=0x7eff340d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 -- 172.21.15.143:0/23064950 shutdown_connections 2024-01-15T18:04:55.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.989+0000 7eff39400640 1 -- 172.21.15.143:0/23064950 wait complete. 2024-01-15T18:04:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 Processor -- start 2024-01-15T18:04:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- start start 2024-01-15T18:04:55.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff340c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff34149fb0 0x7eff340b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff340b4430 0x7eff340b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340c6320 con 0x7eff140073a0 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340c32a0 con 0x7eff34149fb0 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340c9a60 con 0x7eff340b4430 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff340c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff340c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34646/0 (socket says 172.21.15.143:34646) 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 -- 172.21.15.143:0/329194531 learned_addr learned my addr 172.21.15.143:0/329194531 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff31e00640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff34149fb0 0x7eff340b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff33200640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff340b4430 0x7eff340b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 -- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff340b4430 msgr2=0x7eff340b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff340b4430 0x7eff340b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 -- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff34149fb0 msgr2=0x7eff340b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff34149fb0 0x7eff340b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 -- 172.21.15.143:0/329194531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340c9d40 con 0x7eff140073a0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff32800640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff340c5de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff24011960 tx=0x7eff24011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2400faf0 con 0x7eff140073a0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2400fc50 con 0x7eff140073a0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24014440 con 0x7eff140073a0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff340c34b0 con 0x7eff140073a0 2024-01-15T18:04:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3414a430 con 0x7eff140073a0 2024-01-15T18:04:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.993+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff34067500 con 0x7eff140073a0 2024-01-15T18:04:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.997+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7eff240145a0 con 0x7eff140073a0 2024-01-15T18:04:56.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.997+0000 7eff2aa00640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff0c066920 0x7eff0c068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.997+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7eff24084840 con 0x7eff140073a0 2024-01-15T18:04:56.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.997+0000 7eff31e00640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff0c066920 0x7eff0c068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:55.997+0000 7eff31e00640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff0c066920 0x7eff0c068de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff1c001060 tx=0x7eff1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.001+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff24051dc0 con 0x7eff140073a0 2024-01-15T18:04:56.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.149+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff340c9ed0 con 0x7eff140073a0 2024-01-15T18:04:56.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.149+0000 7eff2aa00640 1 -- 172.21.15.143:0/329194531 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff240553e0 con 0x7eff140073a0 2024-01-15T18:04:56.156 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:04:56.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff0c066920 msgr2=0x7eff0c068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff0c066920 0x7eff0c068de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff1c001060 tx=0x7eff1c009040 comp rx=0 tx=0).stop 2024-01-15T18:04:56.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 msgr2=0x7eff340c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff340c5de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff24011960 tx=0x7eff24011e20 comp rx=0 tx=0).stop 2024-01-15T18:04:56.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 shutdown_connections 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff0c066920 0x7eff0c068de0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff340b4430 0x7eff340b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff34149fb0 0x7eff340b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 --2- 172.21.15.143:0/329194531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff140073a0 0x7eff340c5de0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 >> 172.21.15.143:0/329194531 conn(0x7eff340cfd20 msgr2=0x7eff3406e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 shutdown_connections 2024-01-15T18:04:56.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.153+0000 7eff39400640 1 -- 172.21.15.143:0/329194531 wait complete. 2024-01-15T18:04:56.178 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:04:56.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.309+0000 7f9429400640 1 Processor -- start 2024-01-15T18:04:56.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.309+0000 7f9429400640 1 -- start start 2024-01-15T18:04:56.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.309+0000 7f9429400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f94240c4430 0x7f94240b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.309+0000 7f9429400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942406bbf0 con 0x7f94240cc560 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.309+0000 7f9429400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94240b22f0 con 0x7f94240c4050 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.309+0000 7f9429400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94240b25d0 con 0x7f94240c4430 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94240cc560 0x7f94240b0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49588/0 (socket says 172.21.15.143:49588) 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421e00640 1 -- 172.21.15.143:0/4292216449 learned_addr learned my addr 172.21.15.143:0/4292216449 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9423200640 1 --2- 172.21.15.143:0/4292216449 >> v2:172.21.15.204:3300/0 conn(0x7f94240c4430 0x7f94240b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421400640 1 -- 172.21.15.143:0/4292216449 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034205138 0 0) 0x7f942406bbf0 con 0x7f94240cc560 2024-01-15T18:04:56.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421400640 1 -- 172.21.15.143:0/4292216449 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9404003530 con 0x7f94240cc560 2024-01-15T18:04:56.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421400640 1 -- 172.21.15.143:0/4292216449 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487411910 0 0) 0x7f9404003530 con 0x7f94240cc560 2024-01-15T18:04:56.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9423200640 1 -- 172.21.15.143:0/4292216449 >> v1:172.21.15.204:6789/0 conn(0x7f94240c4050 legacy=0x7f94240ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9423200640 1 -- 172.21.15.143:0/4292216449 >> v1:172.21.15.143:6789/0 conn(0x7f94240cc560 legacy=0x7f94240b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9423200640 1 -- 172.21.15.143:0/4292216449 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942406b5a0 con 0x7f94240c4430 2024-01-15T18:04:56.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9423200640 1 --2- 172.21.15.143:0/4292216449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4430 0x7f94240b0880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f941c009d90 tx=0x7f941c01c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb235fd5b25bc89 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421400640 1 -- 172.21.15.143:0/4292216449 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940c001680 con 0x7f94240c4430 2024-01-15T18:04:56.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421400640 1 -- 172.21.15.143:0/4292216449 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c0a55e0 con 0x7f94240c4430 2024-01-15T18:04:56.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.313+0000 7f9421400640 1 -- 172.21.15.143:0/4292216449 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0a5950 con 0x7f94240c4430 2024-01-15T18:04:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- 172.21.15.143:0/4292216449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4430 msgr2=0x7f94240b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 --2- 172.21.15.143:0/4292216449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4430 0x7f94240b0880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f941c009d90 tx=0x7f941c01c840 comp rx=0 tx=0).stop 2024-01-15T18:04:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- 172.21.15.143:0/4292216449 shutdown_connections 2024-01-15T18:04:56.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 --2- 172.21.15.143:0/4292216449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4430 0x7f94240b0880 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- 172.21.15.143:0/4292216449 >> 172.21.15.143:0/4292216449 conn(0x7f94240d0e20 msgr2=0x7f94240d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- 172.21.15.143:0/4292216449 shutdown_connections 2024-01-15T18:04:56.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- 172.21.15.143:0/4292216449 wait complete. 2024-01-15T18:04:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 Processor -- start 2024-01-15T18:04:56.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- start start 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4050 0x7f94240b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240c4430 0x7f94241533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 0x7f94240b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942415e800 con 0x7f94240cc560 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240cae10 con 0x7f94240c4050 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.317+0000 7f9429400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942415ded0 con 0x7f94240c4430 2024-01-15T18:04:56.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 0x7f94240b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 0x7f94240b2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34652/0 (socket says 172.21.15.143:34652) 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 -- 172.21.15.143:0/1263212491 learned_addr learned my addr 172.21.15.143:0/1263212491 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9422800640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4050 0x7f94240b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9421e00640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240c4430 0x7f94241533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 -- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240c4430 msgr2=0x7f94241533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240c4430 0x7f94241533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 -- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4050 msgr2=0x7f94240b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4050 0x7f94240b66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 -- 172.21.15.143:0/1263212491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009a70 con 0x7f94240cc560 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9423200640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 0x7f94240b2910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f941c009d90 tx=0x7f941c0195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0ad850 con 0x7f94240cc560 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c0ad9b0 con 0x7f94240cc560 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0add40 con 0x7f94240cc560 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942415e1b0 con 0x7f94240cc560 2024-01-15T18:04:56.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94240b4a40 con 0x7f94240cc560 2024-01-15T18:04:56.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942415e970 con 0x7f94240cc560 2024-01-15T18:04:56.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.321+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f941c0b4070 con 0x7f94240cc560 2024-01-15T18:04:56.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.325+0000 7f941aa00640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93fc066920 0x7f93fc068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.325+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94240b15d0 con 0x7f94240cc560 2024-01-15T18:04:56.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.325+0000 7f9422800640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93fc066920 0x7f93fc068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.325+0000 7f9422800640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93fc066920 0x7f93fc068de0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9414007fd0 tx=0x7f9414009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.329+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f941c0ebb80 con 0x7f94240cc560 2024-01-15T18:04:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.477+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9424067500 con 0x7f94240cc560 2024-01-15T18:04:56.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.477+0000 7f941aa00640 1 -- 172.21.15.143:0/1263212491 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f941c0b2090 con 0x7f94240cc560 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:04:56.485 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93fc066920 msgr2=0x7f93fc068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93fc066920 0x7f93fc068de0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9414007fd0 tx=0x7f9414009040 comp rx=0 tx=0).stop 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 msgr2=0x7f94240b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 0x7f94240b2910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f941c009d90 tx=0x7f941c0195e0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 shutdown_connections 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93fc066920 0x7f93fc068de0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94240cc560 0x7f94240b2910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94240c4430 0x7f94241533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 --2- 172.21.15.143:0/1263212491 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94240c4050 0x7f94240b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 >> 172.21.15.143:0/1263212491 conn(0x7f94240d0e20 msgr2=0x7f94240734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 shutdown_connections 2024-01-15T18:04:56.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.481+0000 7f9429400640 1 -- 172.21.15.143:0/1263212491 wait complete. 2024-01-15T18:04:56.506 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:04:56.506 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:04:56.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 Processor -- start 2024-01-15T18:04:56.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 -- start start 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcbec15d1f0 0x7fcbec06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbec0b3370 con 0x7fcbec06bb70 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbec15d5d0 con 0x7fcbec15d1f0 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec06a800 con 0x7fcbec071470 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcbec15d1f0 0x7fcbec06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcbec15d1f0 0x7fcbec06b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34662/0 (socket says 172.21.15.143:34662) 2024-01-15T18:04:56.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 -- 172.21.15.143:0/808981059 learned_addr learned my addr 172.21.15.143:0/808981059 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:56.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2a00640 1 --2- 172.21.15.143:0/808981059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 -- 172.21.15.143:0/808981059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 msgr2=0x7fcbec0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:56.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 --2- 172.21.15.143:0/808981059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 -- 172.21.15.143:0/808981059 >> v1:172.21.15.204:6789/0 conn(0x7fcbec06bb70 legacy=0x7fcbec070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.641+0000 7fcbf2000640 1 -- 172.21.15.143:0/808981059 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec06aae0 con 0x7fcbec15d1f0 2024-01-15T18:04:56.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf2000640 1 --2- 172.21.15.143:0/808981059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec15d1f0 0x7fcbec06b630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009870 tx=0x7fcbdc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d336a8ba063849c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf0c00640 1 -- 172.21.15.143:0/808981059 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0aa670 con 0x7fcbec15d1f0 2024-01-15T18:04:56.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf0c00640 1 -- 172.21.15.143:0/808981059 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc0aa7d0 con 0x7fcbec15d1f0 2024-01-15T18:04:56.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf0c00640 1 -- 172.21.15.143:0/808981059 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0aab60 con 0x7fcbec15d1f0 2024-01-15T18:04:56.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 -- 172.21.15.143:0/808981059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec15d1f0 msgr2=0x7fcbec06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/808981059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec15d1f0 0x7fcbec06b630 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009870 tx=0x7fcbdc01c510 comp rx=0 tx=0).stop 2024-01-15T18:04:56.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 -- 172.21.15.143:0/808981059 shutdown_connections 2024-01-15T18:04:56.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/808981059 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/808981059 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec15d1f0 0x7fcbec06b630 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 -- 172.21.15.143:0/808981059 >> 172.21.15.143:0/808981059 conn(0x7fcbec0b1610 msgr2=0x7fcbec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:56.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 -- 172.21.15.143:0/808981059 shutdown_connections 2024-01-15T18:04:56.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.645+0000 7fcbf4a00640 1 -- 172.21.15.143:0/808981059 wait complete. 2024-01-15T18:04:56.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 Processor -- start 2024-01-15T18:04:56.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 -- start start 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 0x7fcbec15e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec1f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbec15d1f0 0x7fcbec1f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec15f020 con 0x7fcbec06bb70 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0b0a80 con 0x7fcbec15d1f0 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0afb90 con 0x7fcbec071470 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 0x7fcbec15e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 0x7fcbec15e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34670/0 (socket says 172.21.15.143:34670) 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 -- 172.21.15.143:0/3705692751 learned_addr learned my addr 172.21.15.143:0/3705692751 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbec15d1f0 0x7fcbec1f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf1600640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec1f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 -- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 msgr2=0x7fcbec1f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec1f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 -- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbec15d1f0 msgr2=0x7fcbec1f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbec15d1f0 0x7fcbec1f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 -- 172.21.15.143:0/3705692751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc0094d0 con 0x7fcbec06bb70 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf1600640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec1f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf2000640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 0x7fcbec15e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0ac040 tx=0x7fcbdc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0b3800 con 0x7fcbec06bb70 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc0b3960 con 0x7fcbec06bb70 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec0afe70 con 0x7fcbec06bb70 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0b3d10 con 0x7fcbec06bb70 2024-01-15T18:04:56.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.649+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec0b0430 con 0x7fcbec06bb70 2024-01-15T18:04:56.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.653+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec074b00 con 0x7fcbec06bb70 2024-01-15T18:04:56.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.653+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fcbdc0bd3f0 con 0x7fcbec06bb70 2024-01-15T18:04:56.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.653+0000 7fcbea000640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbc4066920 0x7fcbc4068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:56.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.653+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcbdc11ec00 con 0x7fcbec06bb70 2024-01-15T18:04:56.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.653+0000 7fcbf1600640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbc4066920 0x7fcbc4068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:56.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.653+0000 7fcbf1600640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbc4066920 0x7fcbc4068de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcbec14f250 tx=0x7fcbd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:56.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.657+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcbdc0ec1a0 con 0x7fcbec06bb70 2024-01-15T18:04:56.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:56.721 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:56.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:56.809+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcbec067500 con 0x7fcbec06bb70 2024-01-15T18:04:57.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbea000640 1 -- 172.21.15.143:0/3705692751 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcbdc0ef7c0 con 0x7fcbec06bb70 2024-01-15T18:04:57.262 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbc4066920 msgr2=0x7fcbc4068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbc4066920 0x7fcbc4068de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcbec14f250 tx=0x7fcbd4009040 comp rx=0 tx=0).stop 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 msgr2=0x7fcbec15e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 0x7fcbec15e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0ac040 tx=0x7fcbdc01ad50 comp rx=0 tx=0).stop 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 shutdown_connections 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbc4066920 0x7fcbc4068de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcbec15d1f0 0x7fcbec1f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcbec071470 0x7fcbec1f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 --2- 172.21.15.143:0/3705692751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcbec06bb70 0x7fcbec15e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 >> 172.21.15.143:0/3705692751 conn(0x7fcbec0b1610 msgr2=0x7fcbec06cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:57.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.257+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 shutdown_connections 2024-01-15T18:04:57.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.261+0000 7fcbf4a00640 1 -- 172.21.15.143:0/3705692751 wait complete. 2024-01-15T18:04:57.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:04:57.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 Processor -- start 2024-01-15T18:04:57.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 -- start start 2024-01-15T18:04:57.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c13f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:57.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc20c13faf0 0x7fc20c0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:57.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c13ff50 con 0x7fc20c0cdc30 2024-01-15T18:04:57.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20c0ce470 con 0x7fc20c13faf0 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc211c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0ce750 con 0x7fc20c13cf10 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc20a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc20c13faf0 0x7fc20c0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc20a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc20c13faf0 0x7fc20c0cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56176/0 (socket says 172.21.15.143:56176) 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc20a000640 1 -- 172.21.15.143:0/3954891251 learned_addr learned my addr 172.21.15.143:0/3954891251 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.429+0000 7fc20aa00640 1 --2- 172.21.15.143:0/3954891251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c13f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc20a000640 1 -- 172.21.15.143:0/3954891251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 msgr2=0x7fc20c13f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc20a000640 1 --2- 172.21.15.143:0/3954891251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c13f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc20a000640 1 -- 172.21.15.143:0/3954891251 >> v1:172.21.15.204:6789/0 conn(0x7fc20c0cdc30 legacy=0x7fc20c15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:57.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc20a000640 1 -- 172.21.15.143:0/3954891251 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0cea30 con 0x7fc20c13faf0 2024-01-15T18:04:57.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc20a000640 1 --2- 172.21.15.143:0/3954891251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c13faf0 0x7fc20c0cd6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009870 tx=0x7fc1f401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555c6c6c78343b05 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:57.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc209600640 1 -- 172.21.15.143:0/3954891251 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40ab600 con 0x7fc20c13faf0 2024-01-15T18:04:57.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc209600640 1 -- 172.21.15.143:0/3954891251 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40a7050 con 0x7fc20c13faf0 2024-01-15T18:04:57.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc209600640 1 -- 172.21.15.143:0/3954891251 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40ab980 con 0x7fc20c13faf0 2024-01-15T18:04:57.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc211c00640 1 -- 172.21.15.143:0/3954891251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c13faf0 msgr2=0x7fc20c0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.433+0000 7fc211c00640 1 --2- 172.21.15.143:0/3954891251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c13faf0 0x7fc20c0cd6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009870 tx=0x7fc1f401c470 comp rx=0 tx=0).stop 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- 172.21.15.143:0/3954891251 shutdown_connections 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 --2- 172.21.15.143:0/3954891251 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c13faf0 0x7fc20c0cd6f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 --2- 172.21.15.143:0/3954891251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- 172.21.15.143:0/3954891251 >> 172.21.15.143:0/3954891251 conn(0x7fc20c06a890 msgr2=0x7fc20c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- 172.21.15.143:0/3954891251 shutdown_connections 2024-01-15T18:04:57.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- 172.21.15.143:0/3954891251 wait complete. 2024-01-15T18:04:57.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 Processor -- start 2024-01-15T18:04:57.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- start start 2024-01-15T18:04:57.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 0x7fc20c13ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc20c13faf0 0x7fc20c1344f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c134a30 con 0x7fc20c0cdc30 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c159140 con 0x7fc20c13faf0 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc211c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c071d80 con 0x7fc20c13cf10 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 0x7fc20c13ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 0x7fc20c13ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56192/0 (socket says 172.21.15.143:56192) 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 -- 172.21.15.143:0/3584012286 learned_addr learned my addr 172.21.15.143:0/3584012286 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:57.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20a000640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20b400640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc20c13faf0 0x7fc20c1344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 -- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 msgr2=0x7fc20c133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 -- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc20c13faf0 msgr2=0x7fc20c1344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc20c13faf0 0x7fc20c1344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.437+0000 7fc20aa00640 1 -- 172.21.15.143:0/3584012286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f40094d0 con 0x7fc20c0cdc30 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc20aa00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 0x7fc20c13ae00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00d800 tx=0x7fc1fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00b660 con 0x7fc20c0cdc30 2024-01-15T18:04:57.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc00b7c0 con 0x7fc20c0cdc30 2024-01-15T18:04:57.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c158e60 con 0x7fc20c0cdc30 2024-01-15T18:04:57.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc00bb30 con 0x7fc20c0cdc30 2024-01-15T18:04:57.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c072270 con 0x7fc20c0cdc30 2024-01-15T18:04:57.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c067500 con 0x7fc20c0cdc30 2024-01-15T18:04:57.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc1fc00bc90 con 0x7fc20c0cdc30 2024-01-15T18:04:57.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc202a00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1e40668f0 0x7fc1e4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:57.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.441+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc1fc0845f0 con 0x7fc20c0cdc30 2024-01-15T18:04:57.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.445+0000 7fc20a000640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1e40668f0 0x7fc1e4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:57.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.445+0000 7fc20a000640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1e40668f0 0x7fc1e4068db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40ad000 tx=0x7fc1f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:57.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.449+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1fc051ac0 con 0x7fc20c0cdc30 2024-01-15T18:04:57.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:57.597+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc20c139e20 con 0x7fc20c0cdc30 2024-01-15T18:04:58.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc202a00640 1 -- 172.21.15.143:0/3584012286 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc1fc0550e0 con 0x7fc20c0cdc30 2024-01-15T18:04:58.266 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:04:58.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1e40668f0 msgr2=0x7fc1e4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:58.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1e40668f0 0x7fc1e4068db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc1f40ad000 tx=0x7fc1f4008040 comp rx=0 tx=0).stop 2024-01-15T18:04:58.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 msgr2=0x7fc20c13ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 0x7fc20c13ae00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00d800 tx=0x7fc1fc00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 shutdown_connections 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1e40668f0 0x7fc1e4068db0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc20c13faf0 0x7fc20c1344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc20c13cf10 0x7fc20c133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 --2- 172.21.15.143:0/3584012286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc20c0cdc30 0x7fc20c13ae00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 >> 172.21.15.143:0/3584012286 conn(0x7fc20c06a890 msgr2=0x7fc20c1364d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 shutdown_connections 2024-01-15T18:04:58.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.261+0000 7fc211c00640 1 -- 172.21.15.143:0/3584012286 wait complete. 2024-01-15T18:04:58.306 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:04:58.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 Processor -- start 2024-01-15T18:04:58.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 -- start start 2024-01-15T18:04:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6f34071140 0x7f6f340ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f34071520 0x7f6f3415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f3415b110 0x7f6f3415da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f340cc220 con 0x7f6f34071520 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3415a4a0 con 0x7f6f34071140 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340ce810 con 0x7f6f3415b110 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f34071520 0x7f6f3415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f34071520 0x7f6f3415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56196/0 (socket says 172.21.15.143:56196) 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 -- 172.21.15.143:0/183528879 learned_addr learned my addr 172.21.15.143:0/183528879 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f39200640 1 --2- 172.21.15.143:0/183528879 >> v2:172.21.15.204:3300/0 conn(0x7f6f34071140 0x7f6f340ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f39c00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f3415b110 0x7f6f3415da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 -- 172.21.15.143:0/183528879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f3415b110 msgr2=0x7f6f3415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f3415b110 0x7f6f3415da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 -- 172.21.15.143:0/183528879 >> v2:172.21.15.204:3300/0 conn(0x7f6f34071140 msgr2=0x7f6f340ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 --2- 172.21.15.143:0/183528879 >> v2:172.21.15.204:3300/0 conn(0x7f6f34071140 0x7f6f340ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f33e00640 1 -- 172.21.15.143:0/183528879 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f340cb980 con 0x7f6f34071520 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f39c00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f3415b110 0x7f6f3415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:04:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.441+0000 7f6f39200640 1 --2- 172.21.15.143:0/183528879 >> v2:172.21.15.204:3300/0 conn(0x7f6f34071140 0x7f6f340ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:04:58.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f33e00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f34071520 0x7f6f3415abd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f340721d0 tx=0x7f6f1c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2569351ba8939b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:58.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f33400640 1 -- 172.21.15.143:0/183528879 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0ac600 con 0x7f6f34071520 2024-01-15T18:04:58.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f33400640 1 -- 172.21.15.143:0/183528879 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c0a8050 con 0x7f6f34071520 2024-01-15T18:04:58.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f33400640 1 -- 172.21.15.143:0/183528879 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0ac980 con 0x7f6f34071520 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 -- 172.21.15.143:0/183528879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f34071520 msgr2=0x7f6f3415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f34071520 0x7f6f3415abd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f340721d0 tx=0x7f6f1c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 -- 172.21.15.143:0/183528879 shutdown_connections 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f3415b110 0x7f6f3415da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/183528879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f34071520 0x7f6f3415abd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/183528879 >> v2:172.21.15.204:3300/0 conn(0x7f6f34071140 0x7f6f340ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 -- 172.21.15.143:0/183528879 >> 172.21.15.143:0/183528879 conn(0x7f6f340cfd20 msgr2=0x7f6f340d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 -- 172.21.15.143:0/183528879 shutdown_connections 2024-01-15T18:04:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.445+0000 7f6f3bc00640 1 -- 172.21.15.143:0/183528879 wait complete. 2024-01-15T18:04:58.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 Processor -- start 2024-01-15T18:04:58.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 -- start start 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f34071140 0x7f6f340c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f34071520 0x7f6f340c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 0x7f6f340c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340c6570 con 0x7f6f3415b110 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340b3a00 con 0x7f6f34071520 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340b2df0 con 0x7f6f34071140 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 0x7f6f340c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 0x7f6f340c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56208/0 (socket says 172.21.15.143:56208) 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 -- 172.21.15.143:0/2609743115 learned_addr learned my addr 172.21.15.143:0/2609743115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39200640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f34071140 0x7f6f340c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f33e00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f34071520 0x7f6f340c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 -- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f34071140 msgr2=0x7f6f340c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f34071140 0x7f6f340c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 -- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f34071520 msgr2=0x7f6f340c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f34071520 0x7f6f340c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 -- 172.21.15.143:0/2609743115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0094d0 con 0x7f6f3415b110 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f39c00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 0x7f6f340c7a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009fa0 tx=0x7f6f2c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c003ba0 con 0x7f6f3415b110 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c00f040 con 0x7f6f3415b110 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f340b30d0 con 0x7f6f3415b110 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004030 con 0x7f6f3415b110 2024-01-15T18:04:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.449+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f340b3690 con 0x7f6f3415b110 2024-01-15T18:04:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.453+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3415aad0 con 0x7f6f3415b110 2024-01-15T18:04:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.453+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f6f2c004190 con 0x7f6f3415b110 2024-01-15T18:04:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.453+0000 7f6f30c00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f0c0668f0 0x7f6f0c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.453+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6f2c084740 con 0x7f6f3415b110 2024-01-15T18:04:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.453+0000 7f6f39200640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f0c0668f0 0x7f6f0c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.453+0000 7f6f39200640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f0c0668f0 0x7f6f0c068db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f240045f0 tx=0x7f6f24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:58.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.457+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f2c0519f0 con 0x7f6f3415b110 2024-01-15T18:04:58.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:58.641+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6f340cf560 con 0x7f6f3415b110 2024-01-15T18:04:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.489+0000 7f6f30c00640 1 -- 172.21.15.143:0/2609743115 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6f2c055010 con 0x7f6f3415b110 2024-01-15T18:04:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 2 and data pool 3 2024-01-15T18:04:59.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f0c0668f0 msgr2=0x7f6f0c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f0c0668f0 0x7f6f0c068db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6f240045f0 tx=0x7f6f24008040 comp rx=0 tx=0).stop 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 msgr2=0x7f6f340c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 0x7f6f340c7a10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009fa0 tx=0x7f6f2c00d560 comp rx=0 tx=0).stop 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 shutdown_connections 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f0c0668f0 0x7f6f0c068db0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f3415b110 0x7f6f340c7a10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f34071520 0x7f6f340c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 --2- 172.21.15.143:0/2609743115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f34071140 0x7f6f340c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 >> 172.21.15.143:0/2609743115 conn(0x7f6f340cfd20 msgr2=0x7f6f3406e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:59.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 shutdown_connections 2024-01-15T18:04:59.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.493+0000 7f6f3bc00640 1 -- 172.21.15.143:0/2609743115 wait complete. 2024-01-15T18:04:59.528 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.657+0000 7fa4f8200640 1 Processor -- start 2024-01-15T18:04:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.657+0000 7fa4f8200640 1 -- start start 2024-01-15T18:04:59.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f8200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4f00ce560 0x7fa4f015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0071d00 0x7fa4f0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f8200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f015f310 con 0x7fa4f0141660 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f8200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f015fbb0 con 0x7fa4f00ce560 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0072690 con 0x7fa4f0071d00 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f4e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa4f00ce560 0x7fa4f015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f5800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa4f0141660 0x7fa4f0141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34008/0 (socket says 172.21.15.143:34008) 2024-01-15T18:04:59.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f5800640 1 -- 172.21.15.143:0/1121028959 learned_addr learned my addr 172.21.15.143:0/1121028959 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f6200640 1 --2- 172.21.15.143:0/1121028959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0071d00 0x7fa4f0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f4e00640 1 -- 172.21.15.143:0/1121028959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0071d00 msgr2=0x7fa4f0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f4e00640 1 --2- 172.21.15.143:0/1121028959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0071d00 0x7fa4f0072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f4e00640 1 -- 172.21.15.143:0/1121028959 >> v1:172.21.15.143:6789/0 conn(0x7fa4f0141660 legacy=0x7fa4f0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4f4e00640 1 -- 172.21.15.143:0/1121028959 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f015f5f0 con 0x7fa4f00ce560 2024-01-15T18:04:59.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.661+0000 7fa4efe00640 1 -- 172.21.15.143:0/1121028959 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956328699 0 0) 0x7fa4f015f310 con 0x7fa4f0141660 2024-01-15T18:04:59.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f4e00640 1 --2- 172.21.15.143:0/1121028959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f015ec80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80097c0 tx=0x7fa4d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32535d0834b4b33b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:59.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4efe00640 1 -- 172.21.15.143:0/1121028959 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80ab600 con 0x7fa4f00ce560 2024-01-15T18:04:59.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4efe00640 1 -- 172.21.15.143:0/1121028959 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d80a7050 con 0x7fa4f00ce560 2024-01-15T18:04:59.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4efe00640 1 -- 172.21.15.143:0/1121028959 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80ab980 con 0x7fa4f00ce560 2024-01-15T18:04:59.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 -- 172.21.15.143:0/1121028959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 msgr2=0x7fa4f015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 --2- 172.21.15.143:0/1121028959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f015ec80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80097c0 tx=0x7fa4d801c480 comp rx=0 tx=0).stop 2024-01-15T18:04:59.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 -- 172.21.15.143:0/1121028959 shutdown_connections 2024-01-15T18:04:59.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 --2- 172.21.15.143:0/1121028959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0071d00 0x7fa4f0072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 --2- 172.21.15.143:0/1121028959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f015ec80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 -- 172.21.15.143:0/1121028959 >> 172.21.15.143:0/1121028959 conn(0x7fa4f006c3f0 msgr2=0x7fa4f006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 -- 172.21.15.143:0/1121028959 shutdown_connections 2024-01-15T18:04:59.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 -- 172.21.15.143:0/1121028959 wait complete. 2024-01-15T18:04:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 Processor -- start 2024-01-15T18:04:59.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.665+0000 7fa4f8200640 1 -- start start 2024-01-15T18:04:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f0071d00 0x7fa4f01409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f01375c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0141660 0x7fa4f0158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0139820 con 0x7fa4f00ce560 2024-01-15T18:04:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f0139ce0 con 0x7fa4f0071d00 2024-01-15T18:04:59.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f013f5d0 con 0x7fa4f0141660 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f01375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f01375c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56224/0 (socket says 172.21.15.143:56224) 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 -- 172.21.15.143:0/3196189526 learned_addr learned my addr 172.21.15.143:0/3196189526 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f6200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0141660 0x7fa4f0158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f5800640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f0071d00 0x7fa4f01409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 -- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0141660 msgr2=0x7fa4f0158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0141660 0x7fa4f0158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 -- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f0071d00 msgr2=0x7fa4f01409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f0071d00 0x7fa4f01409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 -- 172.21.15.143:0/3196189526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8009420 con 0x7fa4f00ce560 2024-01-15T18:04:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f4e00640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f01375c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4d801ab30 tx=0x7fa4d8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8004230 con 0x7fa4f00ce560 2024-01-15T18:04:59.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d8004390 con 0x7fa4f00ce560 2024-01-15T18:04:59.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8003820 con 0x7fa4f00ce560 2024-01-15T18:04:59.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f00cd150 con 0x7fa4f00ce560 2024-01-15T18:04:59.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f0140f00 con 0x7fa4f00ce560 2024-01-15T18:04:59.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f0139a60 con 0x7fa4f00ce560 2024-01-15T18:04:59.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.669+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fa4d80b2040 con 0x7fa4f00ce560 2024-01-15T18:04:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.673+0000 7fa4ed600640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4c80668f0 0x7fa4c8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.673+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4d811d7d0 con 0x7fa4f00ce560 2024-01-15T18:04:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.673+0000 7fa4f5800640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4c80668f0 0x7fa4c8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.673+0000 7fa4f5800640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4c80668f0 0x7fa4c8068db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000f70 tx=0x7fa4e0007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:59.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.677+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa4d80ea970 con 0x7fa4f00ce560 2024-01-15T18:04:59.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.825+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4f0067500 con 0x7fa4f00ce560 2024-01-15T18:04:59.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4ed600640 1 -- 172.21.15.143:0/3196189526 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16554 (secure 0 0 0) 0x7fa4d80bc090 con 0x7fa4f00ce560 2024-01-15T18:04:59.834 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:04:59.834 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:58.645886+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:04:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4c80668f0 msgr2=0x7fa4c8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4c80668f0 0x7fa4c8068db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000f70 tx=0x7fa4e0007c10 comp rx=0 tx=0).stop 2024-01-15T18:04:59.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 msgr2=0x7fa4f01375c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f01375c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa4d801ab30 tx=0x7fa4d8019220 comp rx=0 tx=0).stop 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 shutdown_connections 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4c80668f0 0x7fa4c8068db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4f0141660 0x7fa4f0158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f00ce560 0x7fa4f01375c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 --2- 172.21.15.143:0/3196189526 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4f0071d00 0x7fa4f01409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 >> 172.21.15.143:0/3196189526 conn(0x7fa4f006c3f0 msgr2=0x7fa4f013d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 shutdown_connections 2024-01-15T18:04:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.829+0000 7fa4f8200640 1 -- 172.21.15.143:0/3196189526 wait complete. 2024-01-15T18:04:59.856 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:04:59.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.985+0000 7f1868000640 1 Processor -- start 2024-01-15T18:04:59.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.985+0000 7f1868000640 1 -- start start 2024-01-15T18:04:59.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1868000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f18600cdef0 0x7f186015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:59.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1868000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600d02d0 con 0x7f186006c540 2024-01-15T18:04:59.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1868000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186015d6e0 con 0x7f18600cf150 2024-01-15T18:04:59.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1868000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600cf6c0 con 0x7f18600cdef0 2024-01-15T18:04:59.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1865600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f186006c540 0x7f18600cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34016/0 (socket says 172.21.15.143:34016) 2024-01-15T18:04:59.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1865600640 1 -- 172.21.15.143:0/2172191347 learned_addr learned my addr 172.21.15.143:0/2172191347 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:04:59.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1864c00640 1 --2- 172.21.15.143:0/2172191347 >> v2:172.21.15.204:3300/0 conn(0x7f18600cdef0 0x7f186015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:59.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f185fe00640 1 -- 172.21.15.143:0/2172191347 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631993191 0 0) 0x7f18600d02d0 con 0x7f186006c540 2024-01-15T18:04:59.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f185fe00640 1 -- 172.21.15.143:0/2172191347 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1840003530 con 0x7f186006c540 2024-01-15T18:04:59.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1864c00640 1 -- 172.21.15.143:0/2172191347 >> v1:172.21.15.204:6789/0 conn(0x7f18600cf150 legacy=0x7f186015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1864c00640 1 -- 172.21.15.143:0/2172191347 >> v1:172.21.15.143:6789/0 conn(0x7f186006c540 legacy=0x7f18600cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.989+0000 7f1864c00640 1 -- 172.21.15.143:0/2172191347 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600cf9a0 con 0x7f18600cdef0 2024-01-15T18:04:59.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1864c00640 1 --2- 172.21.15.143:0/2172191347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18600cdef0 0x7f186015ca90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f18480097c0 tx=0x7f184801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5656c4645b9591b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:59.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f185fe00640 1 -- 172.21.15.143:0/2172191347 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480ac620 con 0x7f18600cdef0 2024-01-15T18:04:59.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f185fe00640 1 -- 172.21.15.143:0/2172191347 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18480ac780 con 0x7f18600cdef0 2024-01-15T18:04:59.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f185fe00640 1 -- 172.21.15.143:0/2172191347 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480acb10 con 0x7f18600cdef0 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 -- 172.21.15.143:0/2172191347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18600cdef0 msgr2=0x7f186015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 --2- 172.21.15.143:0/2172191347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18600cdef0 0x7f186015ca90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f18480097c0 tx=0x7f184801c3c0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 -- 172.21.15.143:0/2172191347 shutdown_connections 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 --2- 172.21.15.143:0/2172191347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18600cdef0 0x7f186015ca90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 -- 172.21.15.143:0/2172191347 >> 172.21.15.143:0/2172191347 conn(0x7f18600d0e20 msgr2=0x7f18600d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 -- 172.21.15.143:0/2172191347 shutdown_connections 2024-01-15T18:04:59.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 -- 172.21.15.143:0/2172191347 wait complete. 2024-01-15T18:05:00.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.993+0000 7f1868000640 1 Processor -- start 2024-01-15T18:05:00.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- start start 2024-01-15T18:05:00.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f186006c540 0x7f1860141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18600cdef0 0x7f18600b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 0x7f18600ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600ae610 con 0x7f18600cf150 2024-01-15T18:05:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600ae8f0 con 0x7f186006c540 2024-01-15T18:05:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600aebd0 con 0x7f18600cdef0 2024-01-15T18:05:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 0x7f18600ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 0x7f18600ca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56230/0 (socket says 172.21.15.143:56230) 2024-01-15T18:05:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 -- 172.21.15.143:0/2759770736 learned_addr learned my addr 172.21.15.143:0/2759770736 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1865600640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f186006c540 0x7f1860141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1864c00640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18600cdef0 0x7f18600b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 -- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18600cdef0 msgr2=0x7f18600b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18600cdef0 0x7f18600b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 -- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f186006c540 msgr2=0x7f1860141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f186006c540 0x7f1860141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 -- 172.21.15.143:0/2759770736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848009420 con 0x7f18600cf150 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1866000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 0x7f18600ca200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f185800c770 tx=0x7f185800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858003b80 con 0x7f18600cf150 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858003ce0 con 0x7f18600cf150 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800aa90 con 0x7f18600cf150 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18600aeeb0 con 0x7f18600cf150 2024-01-15T18:05:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1860156400 con 0x7f18600cf150 2024-01-15T18:05:00.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:04:59.997+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1860067500 con 0x7f18600cf150 2024-01-15T18:05:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.001+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f185800abf0 con 0x7f18600cf150 2024-01-15T18:05:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.001+0000 7f185d600640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18380668f0 0x7f1838068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.001+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1858084270 con 0x7f18600cf150 2024-01-15T18:05:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.001+0000 7f1865600640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18380668f0 0x7f1838068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.001+0000 7f1865600640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18380668f0 0x7f1838068db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18500011f0 tx=0x7f1850009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:00.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.005+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18580514b0 con 0x7f18600cf150 2024-01-15T18:05:00.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.157+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18600d1940 con 0x7f18600cf150 2024-01-15T18:05:00.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.157+0000 7f185d600640 1 -- 172.21.15.143:0/2759770736 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16554 (secure 0 0 0) 0x7f1858054ad0 con 0x7f18600cf150 2024-01-15T18:05:00.165 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:00.165 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:04:58.645886+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18380668f0 msgr2=0x7f1838068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18380668f0 0x7f1838068db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18500011f0 tx=0x7f1850009040 comp rx=0 tx=0).stop 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 msgr2=0x7f18600ca200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 0x7f18600ca200 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f185800c770 tx=0x7f185800cc30 comp rx=0 tx=0).stop 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 shutdown_connections 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18380668f0 0x7f1838068db0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18600cf150 0x7f18600ca200 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18600cdef0 0x7f18600b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 --2- 172.21.15.143:0/2759770736 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f186006c540 0x7f1860141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 >> 172.21.15.143:0/2759770736 conn(0x7f18600d0e20 msgr2=0x7f18601683e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:00.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 shutdown_connections 2024-01-15T18:05:00.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.161+0000 7f1868000640 1 -- 172.21.15.143:0/2759770736 wait complete. 2024-01-15T18:05:00.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:05:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.321+0000 7f57f3800640 1 Processor -- start 2024-01-15T18:05:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.321+0000 7f57f3800640 1 -- start start 2024-01-15T18:05:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57f3800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57ec0ce4a0 0x7f57ec0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57f3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec0c4ea0 con 0x7f57ec15b180 2024-01-15T18:05:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57f3800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57ec0c4b70 con 0x7f57ec0cdf80 2024-01-15T18:05:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57f3800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ec0cfdc0 con 0x7f57ec0ce4a0 2024-01-15T18:05:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57f0e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f57ec15b180 0x7f57ec0cd870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34026/0 (socket says 172.21.15.143:34026) 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57f0e00640 1 -- 172.21.15.143:0/2184384814 learned_addr learned my addr 172.21.15.143:0/2184384814 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57ebe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f57ec0ce4a0 0x7f57ec0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57eb400640 1 -- 172.21.15.143:0/2184384814 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514150449 0 0) 0x7f57ec0c4ea0 con 0x7f57ec15b180 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57eb400640 1 -- 172.21.15.143:0/2184384814 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57cc003530 con 0x7f57ec15b180 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57ebe00640 1 -- 172.21.15.143:0/2184384814 >> v1:172.21.15.204:6789/0 conn(0x7f57ec0cdf80 legacy=0x7f57ec0c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57ebe00640 1 -- 172.21.15.143:0/2184384814 >> v1:172.21.15.143:6789/0 conn(0x7f57ec15b180 legacy=0x7f57ec0cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57eb400640 1 -- 172.21.15.143:0/2184384814 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158814973 0 0) 0x7f57ec0c4b70 con 0x7f57ec0cdf80 2024-01-15T18:05:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.325+0000 7f57ebe00640 1 -- 172.21.15.143:0/2184384814 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec0cf800 con 0x7f57ec0ce4a0 2024-01-15T18:05:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57ebe00640 1 --2- 172.21.15.143:0/2184384814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0ce4a0 0x7f57ec0d19a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57d40097c0 tx=0x7f57d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa97d6a702d23e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57eb400640 1 -- 172.21.15.143:0/2184384814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40ad680 con 0x7f57ec0ce4a0 2024-01-15T18:05:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57eb400640 1 -- 172.21.15.143:0/2184384814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d40ad7e0 con 0x7f57ec0ce4a0 2024-01-15T18:05:00.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57eb400640 1 -- 172.21.15.143:0/2184384814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d40adb70 con 0x7f57ec0ce4a0 2024-01-15T18:05:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 -- 172.21.15.143:0/2184384814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0ce4a0 msgr2=0x7f57ec0d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 --2- 172.21.15.143:0/2184384814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0ce4a0 0x7f57ec0d19a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57d40097c0 tx=0x7f57d401c450 comp rx=0 tx=0).stop 2024-01-15T18:05:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 -- 172.21.15.143:0/2184384814 shutdown_connections 2024-01-15T18:05:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 --2- 172.21.15.143:0/2184384814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0ce4a0 0x7f57ec0d19a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 -- 172.21.15.143:0/2184384814 >> 172.21.15.143:0/2184384814 conn(0x7f57ec0d0e80 msgr2=0x7f57ec0d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 -- 172.21.15.143:0/2184384814 shutdown_connections 2024-01-15T18:05:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 -- 172.21.15.143:0/2184384814 wait complete. 2024-01-15T18:05:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 Processor -- start 2024-01-15T18:05:00.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.329+0000 7f57f3800640 1 -- start start 2024-01-15T18:05:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 0x7f57ec152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57ec0ce4a0 0x7f57ec157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57ec15b180 0x7f57ec157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0c9f20 con 0x7f57ec0cdf80 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec152c70 con 0x7f57ec15b180 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0c8e80 con 0x7f57ec0ce4a0 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 0x7f57ec152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 0x7f57ec152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56246/0 (socket says 172.21.15.143:56246) 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 -- 172.21.15.143:0/3883437594 learned_addr learned my addr 172.21.15.143:0/3883437594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f1800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57ec15b180 0x7f57ec157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57ebe00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57ec0ce4a0 0x7f57ec157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 -- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57ec0ce4a0 msgr2=0x7f57ec157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57ec0ce4a0 0x7f57ec157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 -- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57ec15b180 msgr2=0x7f57ec157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57ec15b180 0x7f57ec157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 -- 172.21.15.143:0/3883437594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4009420 con 0x7f57ec0cdf80 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f0e00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 0x7f57ec152730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c3f0 tx=0x7f57dc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:00.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc005cf0 con 0x7f57ec0cdf80 2024-01-15T18:05:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc005e50 con 0x7f57ec0cdf80 2024-01-15T18:05:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0072f0 con 0x7f57ec0cdf80 2024-01-15T18:05:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec0c9160 con 0x7f57ec0cdf80 2024-01-15T18:05:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec0c9720 con 0x7f57ec0cdf80 2024-01-15T18:05:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec067500 con 0x7f57ec0cdf80 2024-01-15T18:05:00.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.333+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f57dc007450 con 0x7f57ec0cdf80 2024-01-15T18:05:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.337+0000 7f57e8c00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57c4066920 0x7f57c4068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.337+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57dc083940 con 0x7f57ec0cdf80 2024-01-15T18:05:00.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.337+0000 7f57ebe00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57c4066920 0x7f57c4068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:00.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.337+0000 7f57ebe00640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57c4066920 0x7f57c4068de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57d40af000 tx=0x7f57d40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:00.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.341+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f57dc050c20 con 0x7f57ec0cdf80 2024-01-15T18:05:00.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:00.521+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57ec0cb190 con 0x7f57ec0cdf80 2024-01-15T18:05:01.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.505+0000 7f57e8c00640 1 -- 172.21.15.143:0/3883437594 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57dc017020 con 0x7f57ec0cdf80 2024-01-15T18:05:01.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.505+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57c4066920 msgr2=0x7f57c4068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.505+0000 7f57f3800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57c4066920 0x7f57c4068de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f57d40af000 tx=0x7f57d40a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.505+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 msgr2=0x7f57ec152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.505+0000 7f57f3800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 0x7f57ec152730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00c3f0 tx=0x7f57dc00c8b0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 shutdown_connections 2024-01-15T18:05:01.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57c4066920 0x7f57c4068de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f57ec15b180 0x7f57ec157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57ec0ce4a0 0x7f57ec157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 --2- 172.21.15.143:0/3883437594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57ec0cdf80 0x7f57ec152730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 >> 172.21.15.143:0/3883437594 conn(0x7f57ec0d0e80 msgr2=0x7f57ec06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 shutdown_connections 2024-01-15T18:05:01.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.509+0000 7f57f3800640 1 -- 172.21.15.143:0/3883437594 wait complete. 2024-01-15T18:05:01.538 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:05:01.538 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:05:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ec5400640 1 Processor -- start 2024-01-15T18:05:01.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ec5400640 1 -- start start 2024-01-15T18:05:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ec5400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5ec0072030 0x7f5ec015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ec5400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec00ceb50 con 0x7f5ec0071bb0 2024-01-15T18:05:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ec5400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec015c110 con 0x7f5ec0143780 2024-01-15T18:05:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ec5400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec015b7e0 con 0x7f5ec0072030 2024-01-15T18:05:01.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebde00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5ec0071bb0 0x7f5ec015c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34038/0 (socket says 172.21.15.143:34038) 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebde00640 1 -- 172.21.15.143:0/3725878398 learned_addr learned my addr 172.21.15.143:0/3725878398 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebf200640 1 --2- 172.21.15.143:0/3725878398 >> v2:172.21.15.204:3300/0 conn(0x7f5ec0072030 0x7f5ec015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebd400640 1 -- 172.21.15.143:0/3725878398 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148341055 0 0) 0x7f5ea8001120 con 0x7f5ec0071bb0 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebd400640 1 -- 172.21.15.143:0/3725878398 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea0003530 con 0x7f5ec0071bb0 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebd400640 1 -- 172.21.15.143:0/3725878398 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799992110 0 0) 0x7f5ea0003530 con 0x7f5ec0071bb0 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebf200640 1 -- 172.21.15.143:0/3725878398 >> v1:172.21.15.204:6789/0 conn(0x7f5ec0143780 legacy=0x7f5ec00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebf200640 1 -- 172.21.15.143:0/3725878398 >> v1:172.21.15.143:6789/0 conn(0x7f5ec0071bb0 legacy=0x7f5ec015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.677+0000 7f5ebf200640 1 -- 172.21.15.143:0/3725878398 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec015bac0 con 0x7f5ec0072030 2024-01-15T18:05:01.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ebf200640 1 --2- 172.21.15.143:0/3725878398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0072030 0x7f5ec015ccb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009d90 tx=0x7f5eb801c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e095e8e1a19b829 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:01.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ebd400640 1 -- 172.21.15.143:0/3725878398 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8001810 con 0x7f5ec0072030 2024-01-15T18:05:01.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ebd400640 1 -- 172.21.15.143:0/3725878398 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb80a5560 con 0x7f5ec0072030 2024-01-15T18:05:01.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ebd400640 1 -- 172.21.15.143:0/3725878398 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb80a58f0 con 0x7f5ec0072030 2024-01-15T18:05:01.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 -- 172.21.15.143:0/3725878398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0072030 msgr2=0x7f5ec015ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 --2- 172.21.15.143:0/3725878398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0072030 0x7f5ec015ccb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8009d90 tx=0x7f5eb801c720 comp rx=0 tx=0).stop 2024-01-15T18:05:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 -- 172.21.15.143:0/3725878398 shutdown_connections 2024-01-15T18:05:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 --2- 172.21.15.143:0/3725878398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0072030 0x7f5ec015ccb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 -- 172.21.15.143:0/3725878398 >> 172.21.15.143:0/3725878398 conn(0x7f5ec006c3f0 msgr2=0x7f5ec006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 -- 172.21.15.143:0/3725878398 shutdown_connections 2024-01-15T18:05:01.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.681+0000 7f5ec5400640 1 -- 172.21.15.143:0/3725878398 wait complete. 2024-01-15T18:05:01.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 Processor -- start 2024-01-15T18:05:01.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 -- start start 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0071bb0 0x7f5ec01418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ec0072030 0x7f5ec00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 0x7f5ec0158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec01405f0 con 0x7f5ec0143780 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0140800 con 0x7f5ec0071bb0 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0140ae0 con 0x7f5ec0072030 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 0x7f5ec0158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 0x7f5ec0158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56256/0 (socket says 172.21.15.143:56256) 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 -- 172.21.15.143:0/720398452 learned_addr learned my addr 172.21.15.143:0/720398452 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebde00640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ec0072030 0x7f5ec00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebe800640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0071bb0 0x7f5ec01418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:01.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 -- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ec0072030 msgr2=0x7f5ec00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ec0072030 0x7f5ec00b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 -- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0071bb0 msgr2=0x7f5ec01418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0071bb0 0x7f5ec01418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 -- 172.21.15.143:0/720398452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb8009a70 con 0x7f5ec0143780 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebe800640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0071bb0 0x7f5ec01418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ebf200640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 0x7f5ec0158f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b0040 tx=0x7f5eb80194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8003ba0 con 0x7f5ec0143780 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8003d00 con 0x7f5ec0143780 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb8004260 con 0x7f5ec0143780 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec0140dc0 con 0x7f5ec0143780 2024-01-15T18:05:01.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.685+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec0141380 con 0x7f5ec0143780 2024-01-15T18:05:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.689+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec0141570 con 0x7f5ec0143780 2024-01-15T18:05:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.689+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5eb80043c0 con 0x7f5ec0143780 2024-01-15T18:05:01.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.689+0000 7f5eb6a00640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e980668f0 0x7f5e98068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.689+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5eb811f110 con 0x7f5ec0143780 2024-01-15T18:05:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.689+0000 7f5ebe800640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e980668f0 0x7f5e98068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:01.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.689+0000 7f5ebe800640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e980668f0 0x7f5e98068db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ec006dfb0 tx=0x7f5eb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:01.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.693+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5eb80ec350 con 0x7f5ec0143780 2024-01-15T18:05:01.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:01.873+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5ec0141570 con 0x7f5ec0143780 2024-01-15T18:05:02.243 INFO:tasks.ceph.mds.c.smithi143.stderr:2024-01-15T18:05:02.237+0000 7f16dbe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:05:02.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.505+0000 7f5eb6a00640 1 -- 172.21.15.143:0/720398452 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5eb8009450 con 0x7f5ec0143780 2024-01-15T18:05:02.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e980668f0 msgr2=0x7f5e98068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e980668f0 0x7f5e98068db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ec006dfb0 tx=0x7f5eb0009040 comp rx=0 tx=0).stop 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 msgr2=0x7f5ec0158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 0x7f5ec0158f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b0040 tx=0x7f5eb80194c0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 shutdown_connections 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e980668f0 0x7f5e98068db0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ec0143780 0x7f5ec0158f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ec0072030 0x7f5ec00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 --2- 172.21.15.143:0/720398452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ec0071bb0 0x7f5ec01418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 >> 172.21.15.143:0/720398452 conn(0x7f5ec006c3f0 msgr2=0x7f5ec013bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 shutdown_connections 2024-01-15T18:05:02.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.509+0000 7f5ec5400640 1 -- 172.21.15.143:0/720398452 wait complete. 2024-01-15T18:05:02.537 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:05:02.537 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 Processor -- start 2024-01-15T18:05:02.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 -- start start 2024-01-15T18:05:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6f81607c0 0x7ff6f806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f80cbf50 0x7ff6f80cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f80cffd0 con 0x7ff6f80cfb30 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80d1ce0 con 0x7ff6f81607c0 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6fd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80d02b0 con 0x7ff6f80cbf50 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6f5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6f81607c0 0x7ff6f806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6f5400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff6f80cfb30 0x7ff6f806bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34048/0 (socket says 172.21.15.143:34048) 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.669+0000 7ff6f5400640 1 -- 172.21.15.143:0/2959790378 learned_addr learned my addr 172.21.15.143:0/2959790378 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:02.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f6800640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f80cbf50 0x7ff6f80cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f5e00640 1 -- 172.21.15.143:0/2959790378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f80cbf50 msgr2=0x7ff6f80cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f5e00640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f80cbf50 0x7ff6f80cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f5e00640 1 -- 172.21.15.143:0/2959790378 >> v1:172.21.15.143:6789/0 conn(0x7ff6f80cfb30 legacy=0x7ff6f806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f5e00640 1 -- 172.21.15.143:0/2959790378 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f80cb470 con 0x7ff6f81607c0 2024-01-15T18:05:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f4a00640 1 -- 172.21.15.143:0/2959790378 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636444269 0 0) 0x7ff6f80cffd0 con 0x7ff6f80cfb30 2024-01-15T18:05:02.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f6800640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f80cbf50 0x7ff6f80cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:02.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f5e00640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f81607c0 0x7ff6f806b7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80097c0 tx=0x7ff6e801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615d4be662d04e99 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:02.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f4a00640 1 -- 172.21.15.143:0/2959790378 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80a8970 con 0x7ff6f81607c0 2024-01-15T18:05:02.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f4a00640 1 -- 172.21.15.143:0/2959790378 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e80a8ad0 con 0x7ff6f81607c0 2024-01-15T18:05:02.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.673+0000 7ff6f4a00640 1 -- 172.21.15.143:0/2959790378 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80a8e60 con 0x7ff6f81607c0 2024-01-15T18:05:02.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- 172.21.15.143:0/2959790378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f81607c0 msgr2=0x7ff6f806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f81607c0 0x7ff6f806b7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80097c0 tx=0x7ff6e801ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:02.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- 172.21.15.143:0/2959790378 shutdown_connections 2024-01-15T18:05:02.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f80cbf50 0x7ff6f80cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 --2- 172.21.15.143:0/2959790378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f81607c0 0x7ff6f806b7b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- 172.21.15.143:0/2959790378 >> 172.21.15.143:0/2959790378 conn(0x7ff6f80d0e20 msgr2=0x7ff6f80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- 172.21.15.143:0/2959790378 shutdown_connections 2024-01-15T18:05:02.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- 172.21.15.143:0/2959790378 wait complete. 2024-01-15T18:05:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 Processor -- start 2024-01-15T18:05:02.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- start start 2024-01-15T18:05:02.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 0x7ff6f814c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6f80cfb30 0x7ff6f809da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f81607c0 0x7ff6f809df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f814d330 con 0x7ff6f80cbf50 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f814dbd0 con 0x7ff6f80cfb30 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6fd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f809b090 con 0x7ff6f81607c0 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6f5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 0x7ff6f814c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6f5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 0x7ff6f814c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56272/0 (socket says 172.21.15.143:56272) 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6f5e00640 1 -- 172.21.15.143:0/3478869124 learned_addr learned my addr 172.21.15.143:0/3478869124 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.677+0000 7ff6f5400640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6f80cfb30 0x7ff6f809da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f6800640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f81607c0 0x7ff6f809df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5e00640 1 -- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f81607c0 msgr2=0x7ff6f809df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5e00640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f81607c0 0x7ff6f809df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5e00640 1 -- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6f80cfb30 msgr2=0x7ff6f809da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5e00640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6f80cfb30 0x7ff6f809da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5e00640 1 -- 172.21.15.143:0/3478869124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8009420 con 0x7ff6f80cbf50 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5e00640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 0x7ff6f814c990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80b3040 tx=0x7ff6e801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:02.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80b9630 con 0x7ff6f80cbf50 2024-01-15T18:05:02.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e80b97b0 con 0x7ff6f80cbf50 2024-01-15T18:05:02.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80b9b60 con 0x7ff6f80cbf50 2024-01-15T18:05:02.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f80ca730 con 0x7ff6f80cbf50 2024-01-15T18:05:02.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f814d860 con 0x7ff6f80cbf50 2024-01-15T18:05:02.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f806d4b0 con 0x7ff6f80cbf50 2024-01-15T18:05:02.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff6e80b7040 con 0x7ff6f80cbf50 2024-01-15T18:05:02.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6ee000640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6d0066920 0x7ff6d0068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6e81222c0 con 0x7ff6f80cbf50 2024-01-15T18:05:02.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.681+0000 7ff6f5400640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6d0066920 0x7ff6d0068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.685+0000 7ff6f5400640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6d0066920 0x7ff6d0068de0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8153a40 tx=0x7ff6e0007b60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:02.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.685+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6e80ef4f0 con 0x7ff6f80cbf50 2024-01-15T18:05:02.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.869+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6f8067500 con 0x7ff6f80cbf50 2024-01-15T18:05:02.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.869+0000 7ff6ee000640 1 -- 172.21.15.143:0/3478869124 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7ff6e80f2b10 con 0x7ff6f80cbf50 2024-01-15T18:05:02.875 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:02.875 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:02.876 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:02.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6d0066920 msgr2=0x7ff6d0068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6d0066920 0x7ff6d0068de0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8153a40 tx=0x7ff6e0007b60 comp rx=0 tx=0).stop 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 msgr2=0x7ff6f814c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 0x7ff6f814c990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80b3040 tx=0x7ff6e801ba10 comp rx=0 tx=0).stop 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 shutdown_connections 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6d0066920 0x7ff6d0068de0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6f81607c0 0x7ff6f809df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6f80cfb30 0x7ff6f809da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 --2- 172.21.15.143:0/3478869124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6f80cbf50 0x7ff6f814c990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 >> 172.21.15.143:0/3478869124 conn(0x7ff6f80d0e20 msgr2=0x7ff6f815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 shutdown_connections 2024-01-15T18:05:02.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:02.873+0000 7ff6fd200640 1 -- 172.21.15.143:0/3478869124 wait complete. 2024-01-15T18:05:02.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:03.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 Processor -- start 2024-01-15T18:05:03.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 -- start start 2024-01-15T18:05:03.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f616406bfb0 0x7f616406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61640b0310 0x7f61640cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640cbff0 con 0x7f61640cc9c0 2024-01-15T18:05:03.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640d0c50 con 0x7f61640b0310 2024-01-15T18:05:03.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616406c8d0 con 0x7f616406bfb0 2024-01-15T18:05:03.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61640b0310 0x7f61640cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616a600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f61640cc9c0 0x7f61640cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34062/0 (socket says 172.21.15.143:34062) 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f616a600640 1 -- 172.21.15.143:0/2090879334 learned_addr learned my addr 172.21.15.143:0/2090879334 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169c00640 1 --2- 172.21.15.143:0/2090879334 >> v2:172.21.15.204:3300/0 conn(0x7f616406bfb0 0x7f616406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169200640 1 -- 172.21.15.143:0/2090879334 >> v2:172.21.15.204:3300/0 conn(0x7f616406bfb0 msgr2=0x7f616406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169200640 1 --2- 172.21.15.143:0/2090879334 >> v2:172.21.15.204:3300/0 conn(0x7f616406bfb0 0x7f616406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169200640 1 -- 172.21.15.143:0/2090879334 >> v1:172.21.15.143:6789/0 conn(0x7f61640cc9c0 legacy=0x7f61640cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6163e00640 1 -- 172.21.15.143:0/2090879334 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773358718 0 0) 0x7f61640cbff0 con 0x7f61640cc9c0 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169200640 1 -- 172.21.15.143:0/2090879334 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640cb6c0 con 0x7f61640b0310 2024-01-15T18:05:03.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.029+0000 7f6169c00640 1 --2- 172.21.15.143:0/2090879334 >> v2:172.21.15.204:3300/0 conn(0x7f616406bfb0 0x7f616406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:03.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f6169200640 1 --2- 172.21.15.143:0/2090879334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640b0310 0x7f61640cc480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61640716c0 tx=0x7f614c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aab783374137665 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f6163e00640 1 -- 172.21.15.143:0/2090879334 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c0ac600 con 0x7f61640b0310 2024-01-15T18:05:03.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f6163e00640 1 -- 172.21.15.143:0/2090879334 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c0a8050 con 0x7f61640b0310 2024-01-15T18:05:03.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f6163e00640 1 -- 172.21.15.143:0/2090879334 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c0ac980 con 0x7f61640b0310 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 -- 172.21.15.143:0/2090879334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640b0310 msgr2=0x7f61640cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 --2- 172.21.15.143:0/2090879334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640b0310 0x7f61640cc480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61640716c0 tx=0x7f614c01c3a0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 -- 172.21.15.143:0/2090879334 shutdown_connections 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 --2- 172.21.15.143:0/2090879334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640b0310 0x7f61640cc480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 --2- 172.21.15.143:0/2090879334 >> v2:172.21.15.204:3300/0 conn(0x7f616406bfb0 0x7f616406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 -- 172.21.15.143:0/2090879334 >> 172.21.15.143:0/2090879334 conn(0x7f61640d0ed0 msgr2=0x7f61640d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 -- 172.21.15.143:0/2090879334 shutdown_connections 2024-01-15T18:05:03.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.033+0000 7f616c600640 1 -- 172.21.15.143:0/2090879334 wait complete. 2024-01-15T18:05:03.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 Processor -- start 2024-01-15T18:05:03.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 -- start start 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f616406bfb0 0x7f616414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61640b0310 0x7f61640c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 0x7f6164156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61641577a0 con 0x7f61640cc9c0 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164157d60 con 0x7f616406bfb0 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c76e0 con 0x7f61640b0310 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 0x7f6164156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 0x7f6164156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56288/0 (socket says 172.21.15.143:56288) 2024-01-15T18:05:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 -- 172.21.15.143:0/557340697 learned_addr learned my addr 172.21.15.143:0/557340697 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f6169c00640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f616406bfb0 0x7f616414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f6169200640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61640b0310 0x7f61640c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 -- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61640b0310 msgr2=0x7f61640c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61640b0310 0x7f61640c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 -- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f616406bfb0 msgr2=0x7f616414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f616406bfb0 0x7f616414e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 -- 172.21.15.143:0/557340697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c009420 con 0x7f61640cc9c0 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616a600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 0x7f6164156c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f615c001570 tx=0x7f615c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00a640 con 0x7f61640cc9c0 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c00a7c0 con 0x7f61640cc9c0 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00ab70 con 0x7f61640cc9c0 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61640c79c0 con 0x7f61640cc9c0 2024-01-15T18:05:03.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.037+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640c71d0 con 0x7f61640cc9c0 2024-01-15T18:05:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.041+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6164157ed0 con 0x7f61640cc9c0 2024-01-15T18:05:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.041+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f615c00acd0 con 0x7f61640cc9c0 2024-01-15T18:05:03.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.041+0000 7f6161600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f613c0668f0 0x7f613c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.041+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f615c083a10 con 0x7f61640cc9c0 2024-01-15T18:05:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.041+0000 7f6169c00640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f613c0668f0 0x7f613c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.041+0000 7f6169c00640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f613c0668f0 0x7f613c068db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61641552a0 tx=0x7f6154008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.045+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f615c050c40 con 0x7f61640cc9c0 2024-01-15T18:05:03.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.197+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6164067500 con 0x7f61640cc9c0 2024-01-15T18:05:03.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.197+0000 7f6161600640 1 -- 172.21.15.143:0/557340697 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f615c018070 con 0x7f61640cc9c0 2024-01-15T18:05:03.203 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:03.204 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f613c0668f0 msgr2=0x7f613c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f613c0668f0 0x7f613c068db0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61641552a0 tx=0x7f6154008040 comp rx=0 tx=0).stop 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 msgr2=0x7f6164156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 0x7f6164156c10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f615c001570 tx=0x7f615c00cca0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 shutdown_connections 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f613c0668f0 0x7f613c068db0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61640cc9c0 0x7f6164156c10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61640b0310 0x7f61640c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 --2- 172.21.15.143:0/557340697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f616406bfb0 0x7f616414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 >> 172.21.15.143:0/557340697 conn(0x7f61640d0ed0 msgr2=0x7f616406ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 shutdown_connections 2024-01-15T18:05:03.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.201+0000 7f616c600640 1 -- 172.21.15.143:0/557340697 wait complete. 2024-01-15T18:05:03.225 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:05:03.226 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:05:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.353+0000 7f069fc00640 1 Processor -- start 2024-01-15T18:05:03.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.353+0000 7f069fc00640 1 -- start start 2024-01-15T18:05:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f06980b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069fc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 0x7f0698070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069fc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980b2a20 con 0x7f0698157300 2024-01-15T18:05:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069fc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06980b35a0 con 0x7f06980b48f0 2024-01-15T18:05:03.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698159fd0 con 0x7f0698159b70 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069dc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 0x7f0698070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f0697e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f06980b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069d200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0698157300 0x7f06980b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59130/0 (socket says 172.21.15.143:59130) 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069dc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 0x7f0698070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57800/0 (socket says 172.21.15.143:57800) 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069d200640 1 -- 172.21.15.143:0/1568294185 learned_addr learned my addr 172.21.15.143:0/1568294185 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f0697e00640 1 -- 172.21.15.143:0/1568294185 >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 msgr2=0x7f0698070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f0697e00640 1 --2- 172.21.15.143:0/1568294185 >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 0x7f0698070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f0697e00640 1 -- 172.21.15.143:0/1568294185 >> v1:172.21.15.204:6789/0 conn(0x7f0698157300 legacy=0x7f06980b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f0697e00640 1 -- 172.21.15.143:0/1568294185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980b2d00 con 0x7f0698159b70 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f069dc00640 1 --2- 172.21.15.143:0/1568294185 >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 0x7f0698070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:03.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.357+0000 7f0697400640 1 -- 172.21.15.143:0/1568294185 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836660582 0 0) 0x7f06980b2a20 con 0x7f0698157300 2024-01-15T18:05:03.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f0697e00640 1 --2- 172.21.15.143:0/1568294185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f06980b43b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06800097c0 tx=0x7f068001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2ce190d267cb09 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f0697400640 1 -- 172.21.15.143:0/1568294185 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06800ab600 con 0x7f0698159b70 2024-01-15T18:05:03.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f0697400640 1 -- 172.21.15.143:0/1568294185 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06800a7050 con 0x7f0698159b70 2024-01-15T18:05:03.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f0697400640 1 -- 172.21.15.143:0/1568294185 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06800ab980 con 0x7f0698159b70 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 -- 172.21.15.143:0/1568294185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 msgr2=0x7f06980b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 --2- 172.21.15.143:0/1568294185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f06980b43b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f06800097c0 tx=0x7f068001c480 comp rx=0 tx=0).stop 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 -- 172.21.15.143:0/1568294185 shutdown_connections 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 --2- 172.21.15.143:0/1568294185 >> v2:172.21.15.204:3300/0 conn(0x7f06980b48f0 0x7f0698070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 --2- 172.21.15.143:0/1568294185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f06980b43b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 -- 172.21.15.143:0/1568294185 >> 172.21.15.143:0/1568294185 conn(0x7f069806c3f0 msgr2=0x7f069806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 -- 172.21.15.143:0/1568294185 shutdown_connections 2024-01-15T18:05:03.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 -- 172.21.15.143:0/1568294185 wait complete. 2024-01-15T18:05:03.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.361+0000 7f069fc00640 1 Processor -- start 2024-01-15T18:05:03.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- start start 2024-01-15T18:05:03.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 0x7f069815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0698157300 0x7f0698156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f0698151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069815d9e0 con 0x7f06980b48f0 2024-01-15T18:05:03.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698151de0 con 0x7f0698157300 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698158780 con 0x7f0698159b70 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 0x7f069815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 0x7f069815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56296/0 (socket says 172.21.15.143:56296) 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 -- 172.21.15.143:0/3097340156 learned_addr learned my addr 172.21.15.143:0/3097340156 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069dc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f0698151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f0697e00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0698157300 0x7f0698156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 -- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 msgr2=0x7f0698151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f0698151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 -- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0698157300 msgr2=0x7f0698156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0698157300 0x7f0698156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 -- 172.21.15.143:0/3097340156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069000bc80 con 0x7f06980b48f0 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069d200640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 0x7f069815cfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06880011f0 tx=0x7f068800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880074e0 con 0x7f06980b48f0 2024-01-15T18:05:03.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688007640 con 0x7f06980b48f0 2024-01-15T18:05:03.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688004a10 con 0x7f06980b48f0 2024-01-15T18:05:03.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0680009420 con 0x7f06980b48f0 2024-01-15T18:05:03.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069815dc30 con 0x7f06980b48f0 2024-01-15T18:05:03.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.365+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698153580 con 0x7f06980b48f0 2024-01-15T18:05:03.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.369+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f0688004b70 con 0x7f06980b48f0 2024-01-15T18:05:03.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.369+0000 7f0694c00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0670066920 0x7f0670068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.369+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0688083560 con 0x7f06980b48f0 2024-01-15T18:05:03.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.369+0000 7f0697e00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0670066920 0x7f0670068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.369+0000 7f0697e00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0670066920 0x7f0670068de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06800ad000 tx=0x7f06800a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.373+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0688050850 con 0x7f06980b48f0 2024-01-15T18:05:03.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.521+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06981588f0 con 0x7f0670066920 2024-01-15T18:05:03.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.529+0000 7f0694c00640 1 -- 172.21.15.143:0/3097340156 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264659 (secure 0 0 0) 0x7f06981588f0 con 0x7f0670066920 2024-01-15T18:05:03.537 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:03.541 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0670066920 msgr2=0x7f0670068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0670066920 0x7f0670068de0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f06800ad000 tx=0x7f06800a6000 comp rx=0 tx=0).stop 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 msgr2=0x7f069815cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 0x7f069815cfd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f06880011f0 tx=0x7f068800c9b0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 shutdown_connections 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0670066920 0x7f0670068de0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0698159b70 0x7f0698151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0698157300 0x7f0698156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 --2- 172.21.15.143:0/3097340156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06980b48f0 0x7f069815cfd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 >> 172.21.15.143:0/3097340156 conn(0x7f069806c3f0 msgr2=0x7f069806c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 shutdown_connections 2024-01-15T18:05:03.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.533+0000 7f069fc00640 1 -- 172.21.15.143:0/3097340156 wait complete. 2024-01-15T18:05:03.559 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-15T18:05:02.622126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.360327"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539184+0000","last_change":"2024-01-15T18:04:59.313108+0000","last_active":"2024-01-15T18:05:00.539184+0000","last_peered":"2024-01-15T18:05:00.539184+0000","last_clean":"2024-01-15T18:05:00.539184+0000","last_became_active":"2024-01-15T18:04:59.312739+0000","last_became_peered":"2024-01-15T18:04:59.312739+0000","last_unstale":"2024-01-15T18:05:00.539184+0000","last_undegraded":"2024-01-15T18:05:00.539184+0000","last_fullsized":"2024-01-15T18:05:00.539184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740563+0000","last_change":"2024-01-15T18:04:59.303929+0000","last_active":"2024-01-15T18:05:00.740563+0000","last_peered":"2024-01-15T18:05:00.740563+0000","last_clean":"2024-01-15T18:05:00.740563+0000","last_became_active":"2024-01-15T18:04:59.303598+0000","last_became_peered":"2024-01-15T18:04:59.303598+0000","last_unstale":"2024-01-15T18:05:00.740563+0000","last_undegraded":"2024-01-15T18:05:00.740563+0000","last_fullsized":"2024-01-15T18:05:00.740563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740178+0000","last_change":"2024-01-15T18:04:59.298035+0000","last_active":"2024-01-15T18:05:00.740178+0000","last_peered":"2024-01-15T18:05:00.740178+0000","last_clean":"2024-01-15T18:05:00.740178+0000","last_became_active":"2024-01-15T18:04:59.297748+0000","last_became_peered":"2024-01-15T18:04:59.297748+0000","last_unstale":"2024-01-15T18:05:00.740178+0000","last_undegraded":"2024-01-15T18:05:00.740178+0000","last_fullsized":"2024-01-15T18:05:00.740178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.314048+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.313770+0000","last_became_peered":"2024-01-15T18:04:59.313770+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.308757+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.306380+0000","last_became_peered":"2024-01-15T18:04:59.306380+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353031+0000","last_change":"2024-01-15T18:04:59.307868+0000","last_active":"2024-01-15T18:05:00.353031+0000","last_peered":"2024-01-15T18:05:00.353031+0000","last_clean":"2024-01-15T18:05:00.353031+0000","last_became_active":"2024-01-15T18:04:59.307388+0000","last_became_peered":"2024-01-15T18:04:59.307388+0000","last_unstale":"2024-01-15T18:05:00.353031+0000","last_undegraded":"2024-01-15T18:05:00.353031+0000","last_fullsized":"2024-01-15T18:05:00.353031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538844+0000","last_change":"2024-01-15T18:04:59.316726+0000","last_active":"2024-01-15T18:05:00.538844+0000","last_peered":"2024-01-15T18:05:00.538844+0000","last_clean":"2024-01-15T18:05:00.538844+0000","last_became_active":"2024-01-15T18:04:59.311435+0000","last_became_peered":"2024-01-15T18:04:59.311435+0000","last_unstale":"2024-01-15T18:05:00.538844+0000","last_undegraded":"2024-01-15T18:05:00.538844+0000","last_fullsized":"2024-01-15T18:05:00.538844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284988+0000","last_change":"2024-01-15T18:04:59.305582+0000","last_active":"2024-01-15T18:05:00.284988+0000","last_peered":"2024-01-15T18:05:00.284988+0000","last_clean":"2024-01-15T18:05:00.284988+0000","last_became_active":"2024-01-15T18:04:59.305227+0000","last_became_peered":"2024-01-15T18:04:59.305227+0000","last_unstale":"2024-01-15T18:05:00.284988+0000","last_undegraded":"2024-01-15T18:05:00.284988+0000","last_fullsized":"2024-01-15T18:05:00.284988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539386+0000","last_change":"2024-01-15T18:04:59.310510+0000","last_active":"2024-01-15T18:05:00.539386+0000","last_peered":"2024-01-15T18:05:00.539386+0000","last_clean":"2024-01-15T18:05:00.539386+0000","last_became_active":"2024-01-15T18:04:59.310228+0000","last_became_peered":"2024-01-15T18:04:59.310228+0000","last_unstale":"2024-01-15T18:05:00.539386+0000","last_undegraded":"2024-01-15T18:05:00.539386+0000","last_fullsized":"2024-01-15T18:05:00.539386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283959+0000","last_change":"2024-01-15T18:04:59.308904+0000","last_active":"2024-01-15T18:05:00.283959+0000","last_peered":"2024-01-15T18:05:00.283959+0000","last_clean":"2024-01-15T18:05:00.283959+0000","last_became_active":"2024-01-15T18:04:59.308650+0000","last_became_peered":"2024-01-15T18:04:59.308650+0000","last_unstale":"2024-01-15T18:05:00.283959+0000","last_undegraded":"2024-01-15T18:05:00.283959+0000","last_fullsized":"2024-01-15T18:05:00.283959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283572+0000","last_change":"2024-01-15T18:04:59.311700+0000","last_active":"2024-01-15T18:05:00.283572+0000","last_peered":"2024-01-15T18:05:00.283572+0000","last_clean":"2024-01-15T18:05:00.283572+0000","last_became_active":"2024-01-15T18:04:59.311406+0000","last_became_peered":"2024-01-15T18:04:59.311406+0000","last_unstale":"2024-01-15T18:05:00.283572+0000","last_undegraded":"2024-01-15T18:05:00.283572+0000","last_fullsized":"2024-01-15T18:05:00.283572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284074+0000","last_change":"2024-01-15T18:04:59.287898+0000","last_active":"2024-01-15T18:05:00.284074+0000","last_peered":"2024-01-15T18:05:00.284074+0000","last_clean":"2024-01-15T18:05:00.284074+0000","last_became_active":"2024-01-15T18:04:59.287554+0000","last_became_peered":"2024-01-15T18:04:59.287554+0000","last_unstale":"2024-01-15T18:05:00.284074+0000","last_undegraded":"2024-01-15T18:05:00.284074+0000","last_fullsized":"2024-01-15T18:05:00.284074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:43.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285461+0000","last_change":"2024-01-15T18:04:59.303813+0000","last_active":"2024-01-15T18:05:00.285461+0000","last_peered":"2024-01-15T18:05:00.285461+0000","last_clean":"2024-01-15T18:05:00.285461+0000","last_became_active":"2024-01-15T18:04:59.303549+0000","last_became_peered":"2024-01-15T18:04:59.303549+0000","last_unstale":"2024-01-15T18:05:00.285461+0000","last_undegraded":"2024-01-15T18:05:00.285461+0000","last_fullsized":"2024-01-15T18:05:00.285461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739952+0000","last_change":"2024-01-15T18:04:59.301393+0000","last_active":"2024-01-15T18:05:00.739952+0000","last_peered":"2024-01-15T18:05:00.739952+0000","last_clean":"2024-01-15T18:05:00.739952+0000","last_became_active":"2024-01-15T18:04:59.300604+0000","last_became_peered":"2024-01-15T18:04:59.300604+0000","last_unstale":"2024-01-15T18:05:00.739952+0000","last_undegraded":"2024-01-15T18:05:00.739952+0000","last_fullsized":"2024-01-15T18:05:00.739952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740231+0000","last_change":"2024-01-15T18:04:59.311581+0000","last_active":"2024-01-15T18:05:00.740231+0000","last_peered":"2024-01-15T18:05:00.740231+0000","last_clean":"2024-01-15T18:05:00.740231+0000","last_became_active":"2024-01-15T18:04:59.311309+0000","last_became_peered":"2024-01-15T18:04:59.311309+0000","last_unstale":"2024-01-15T18:05:00.740231+0000","last_undegraded":"2024-01-15T18:05:00.740231+0000","last_fullsized":"2024-01-15T18:05:00.740231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353034+0000","last_change":"2024-01-15T18:04:59.307600+0000","last_active":"2024-01-15T18:05:00.353034+0000","last_peered":"2024-01-15T18:05:00.353034+0000","last_clean":"2024-01-15T18:05:00.353034+0000","last_became_active":"2024-01-15T18:04:59.307160+0000","last_became_peered":"2024-01-15T18:04:59.307160+0000","last_unstale":"2024-01-15T18:05:00.353034+0000","last_undegraded":"2024-01-15T18:05:00.353034+0000","last_fullsized":"2024-01-15T18:05:00.353034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353453+0000","last_change":"2024-01-15T18:04:59.316075+0000","last_active":"2024-01-15T18:05:00.353453+0000","last_peered":"2024-01-15T18:05:00.353453+0000","last_clean":"2024-01-15T18:05:00.353453+0000","last_became_active":"2024-01-15T18:04:59.315751+0000","last_became_peered":"2024-01-15T18:04:59.315751+0000","last_unstale":"2024-01-15T18:05:00.353453+0000","last_undegraded":"2024-01-15T18:05:00.353453+0000","last_fullsized":"2024-01-15T18:05:00.353453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739317+0000","last_change":"2024-01-15T18:04:59.305938+0000","last_active":"2024-01-15T18:05:00.739317+0000","last_peered":"2024-01-15T18:05:00.739317+0000","last_clean":"2024-01-15T18:05:00.739317+0000","last_became_active":"2024-01-15T18:04:59.305686+0000","last_became_peered":"2024-01-15T18:04:59.305686+0000","last_unstale":"2024-01-15T18:05:00.739317+0000","last_undegraded":"2024-01-15T18:05:00.739317+0000","last_fullsized":"2024-01-15T18:05:00.739317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:35.095703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539274+0000","last_change":"2024-01-15T18:04:59.306518+0000","last_active":"2024-01-15T18:05:00.539274+0000","last_peered":"2024-01-15T18:05:00.539274+0000","last_clean":"2024-01-15T18:05:00.539274+0000","last_became_active":"2024-01-15T18:04:59.306009+0000","last_became_peered":"2024-01-15T18:04:59.306009+0000","last_unstale":"2024-01-15T18:05:00.539274+0000","last_undegraded":"2024-01-15T18:05:00.539274+0000","last_fullsized":"2024-01-15T18:05:00.539274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283820+0000","last_change":"2024-01-15T18:04:59.296492+0000","last_active":"2024-01-15T18:05:00.283820+0000","last_peered":"2024-01-15T18:05:00.283820+0000","last_clean":"2024-01-15T18:05:00.283820+0000","last_became_active":"2024-01-15T18:04:59.296200+0000","last_became_peered":"2024-01-15T18:04:59.296200+0000","last_unstale":"2024-01-15T18:05:00.283820+0000","last_undegraded":"2024-01-15T18:05:00.283820+0000","last_fullsized":"2024-01-15T18:05:00.283820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.300916+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.300453+0000","last_became_peered":"2024-01-15T18:04:59.300453+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:50.361044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353117+0000","last_change":"2024-01-15T18:04:59.313753+0000","last_active":"2024-01-15T18:05:00.353117+0000","last_peered":"2024-01-15T18:05:00.353117+0000","last_clean":"2024-01-15T18:05:00.353117+0000","last_became_active":"2024-01-15T18:04:59.313404+0000","last_became_peered":"2024-01-15T18:04:59.313404+0000","last_unstale":"2024-01-15T18:05:00.353117+0000","last_undegraded":"2024-01-15T18:05:00.353117+0000","last_fullsized":"2024-01-15T18:05:00.353117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285121+0000","last_change":"2024-01-15T18:04:59.297113+0000","last_active":"2024-01-15T18:05:00.285121+0000","last_peered":"2024-01-15T18:05:00.285121+0000","last_clean":"2024-01-15T18:05:00.285121+0000","last_became_active":"2024-01-15T18:04:59.296920+0000","last_became_peered":"2024-01-15T18:04:59.296920+0000","last_unstale":"2024-01-15T18:05:00.285121+0000","last_undegraded":"2024-01-15T18:05:00.285121+0000","last_fullsized":"2024-01-15T18:05:00.285121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353148+0000","last_change":"2024-01-15T18:04:59.307789+0000","last_active":"2024-01-15T18:05:00.353148+0000","last_peered":"2024-01-15T18:05:00.353148+0000","last_clean":"2024-01-15T18:05:00.353148+0000","last_became_active":"2024-01-15T18:04:59.306606+0000","last_became_peered":"2024-01-15T18:04:59.306606+0000","last_unstale":"2024-01-15T18:05:00.353148+0000","last_undegraded":"2024-01-15T18:05:00.353148+0000","last_fullsized":"2024-01-15T18:05:00.353148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285732+0000","last_change":"2024-01-15T18:04:59.305088+0000","last_active":"2024-01-15T18:05:00.285732+0000","last_peered":"2024-01-15T18:05:00.285732+0000","last_clean":"2024-01-15T18:05:00.285732+0000","last_became_active":"2024-01-15T18:04:59.304735+0000","last_became_peered":"2024-01-15T18:04:59.304735+0000","last_unstale":"2024-01-15T18:05:00.285732+0000","last_undegraded":"2024-01-15T18:05:00.285732+0000","last_fullsized":"2024-01-15T18:05:00.285732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284011+0000","last_change":"2024-01-15T18:04:59.311649+0000","last_active":"2024-01-15T18:05:00.284011+0000","last_peered":"2024-01-15T18:05:00.284011+0000","last_clean":"2024-01-15T18:05:00.284011+0000","last_became_active":"2024-01-15T18:04:59.311225+0000","last_became_peered":"2024-01-15T18:04:59.311225+0000","last_unstale":"2024-01-15T18:05:00.284011+0000","last_undegraded":"2024-01-15T18:05:00.284011+0000","last_fullsized":"2024-01-15T18:05:00.284011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.306001+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.305732+0000","last_became_peered":"2024-01-15T18:04:59.305732+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285595+0000","last_change":"2024-01-15T18:04:59.306947+0000","last_active":"2024-01-15T18:05:00.285595+0000","last_peered":"2024-01-15T18:05:00.285595+0000","last_clean":"2024-01-15T18:05:00.285595+0000","last_became_active":"2024-01-15T18:04:59.306617+0000","last_became_peered":"2024-01-15T18:04:59.306617+0000","last_unstale":"2024-01-15T18:05:00.285595+0000","last_undegraded":"2024-01-15T18:05:00.285595+0000","last_fullsized":"2024-01-15T18:05:00.285595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.303640+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.303076+0000","last_became_peered":"2024-01-15T18:04:59.303076+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:45.894125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353181+0000","last_change":"2024-01-15T18:04:59.307697+0000","last_active":"2024-01-15T18:05:00.353181+0000","last_peered":"2024-01-15T18:05:00.353181+0000","last_clean":"2024-01-15T18:05:00.353181+0000","last_became_active":"2024-01-15T18:04:59.305330+0000","last_became_peered":"2024-01-15T18:04:59.305330+0000","last_unstale":"2024-01-15T18:05:00.353181+0000","last_undegraded":"2024-01-15T18:05:00.353181+0000","last_fullsized":"2024-01-15T18:05:00.353181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739155+0000","last_change":"2024-01-15T18:04:59.305380+0000","last_active":"2024-01-15T18:05:00.739155+0000","last_peered":"2024-01-15T18:05:00.739155+0000","last_clean":"2024-01-15T18:05:00.739155+0000","last_became_active":"2024-01-15T18:04:59.305090+0000","last_became_peered":"2024-01-15T18:04:59.305090+0000","last_unstale":"2024-01-15T18:05:00.739155+0000","last_undegraded":"2024-01-15T18:05:00.739155+0000","last_fullsized":"2024-01-15T18:05:00.739155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283376+0000","last_change":"2024-01-15T18:04:59.294688+0000","last_active":"2024-01-15T18:05:00.283376+0000","last_peered":"2024-01-15T18:05:00.283376+0000","last_clean":"2024-01-15T18:05:00.283376+0000","last_became_active":"2024-01-15T18:04:59.294065+0000","last_became_peered":"2024-01-15T18:04:59.294065+0000","last_unstale":"2024-01-15T18:05:00.283376+0000","last_undegraded":"2024-01-15T18:05:00.283376+0000","last_fullsized":"2024-01-15T18:05:00.283376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283129+0000","last_change":"2024-01-15T18:04:59.300834+0000","last_active":"2024-01-15T18:05:00.283129+0000","last_peered":"2024-01-15T18:05:00.283129+0000","last_clean":"2024-01-15T18:05:00.283129+0000","last_became_active":"2024-01-15T18:04:59.300257+0000","last_became_peered":"2024-01-15T18:04:59.300257+0000","last_unstale":"2024-01-15T18:05:00.283129+0000","last_undegraded":"2024-01-15T18:05:00.283129+0000","last_fullsized":"2024-01-15T18:05:00.283129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:31.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285696+0000","last_change":"2024-01-15T18:04:58.298585+0000","last_active":"2024-01-15T18:05:00.285696+0000","last_peered":"2024-01-15T18:05:00.285696+0000","last_clean":"2024-01-15T18:05:00.285696+0000","last_became_active":"2024-01-15T18:04:58.298288+0000","last_became_peered":"2024-01-15T18:04:58.298288+0000","last_unstale":"2024-01-15T18:05:00.285696+0000","last_undegraded":"2024-01-15T18:05:00.285696+0000","last_fullsized":"2024-01-15T18:05:00.285696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740106+0000","last_change":"2024-01-15T18:04:59.295504+0000","last_active":"2024-01-15T18:05:00.740106+0000","last_peered":"2024-01-15T18:05:00.740106+0000","last_clean":"2024-01-15T18:05:00.740106+0000","last_became_active":"2024-01-15T18:04:59.295205+0000","last_became_peered":"2024-01-15T18:04:59.295205+0000","last_unstale":"2024-01-15T18:05:00.740106+0000","last_undegraded":"2024-01-15T18:05:00.740106+0000","last_fullsized":"2024-01-15T18:05:00.740106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283737+0000","last_change":"2024-01-15T18:04:58.296152+0000","last_active":"2024-01-15T18:05:00.283737+0000","last_peered":"2024-01-15T18:05:00.283737+0000","last_clean":"2024-01-15T18:05:00.283737+0000","last_became_active":"2024-01-15T18:04:58.295329+0000","last_became_peered":"2024-01-15T18:04:58.295329+0000","last_unstale":"2024-01-15T18:05:00.283737+0000","last_undegraded":"2024-01-15T18:05:00.283737+0000","last_fullsized":"2024-01-15T18:05:00.283737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285163+0000","last_change":"2024-01-15T18:04:58.291634+0000","last_active":"2024-01-15T18:05:00.285163+0000","last_peered":"2024-01-15T18:05:00.285163+0000","last_clean":"2024-01-15T18:05:00.285163+0000","last_became_active":"2024-01-15T18:04:58.290715+0000","last_became_peered":"2024-01-15T18:04:58.290715+0000","last_unstale":"2024-01-15T18:05:00.285163+0000","last_undegraded":"2024-01-15T18:05:00.285163+0000","last_fullsized":"2024-01-15T18:05:00.285163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285067+0000","last_change":"2024-01-15T18:04:59.302779+0000","last_active":"2024-01-15T18:05:00.285067+0000","last_peered":"2024-01-15T18:05:00.285067+0000","last_clean":"2024-01-15T18:05:00.285067+0000","last_became_active":"2024-01-15T18:04:59.302387+0000","last_became_peered":"2024-01-15T18:04:59.302387+0000","last_unstale":"2024-01-15T18:05:00.285067+0000","last_undegraded":"2024-01-15T18:05:00.285067+0000","last_fullsized":"2024-01-15T18:05:00.285067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353124+0000","last_change":"2024-01-15T18:04:58.285848+0000","last_active":"2024-01-15T18:05:00.353124+0000","last_peered":"2024-01-15T18:05:00.353124+0000","last_clean":"2024-01-15T18:05:00.353124+0000","last_became_active":"2024-01-15T18:04:58.283781+0000","last_became_peered":"2024-01-15T18:04:58.283781+0000","last_unstale":"2024-01-15T18:05:00.353124+0000","last_undegraded":"2024-01-15T18:05:00.353124+0000","last_fullsized":"2024-01-15T18:05:00.353124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740140+0000","last_change":"2024-01-15T18:04:59.306022+0000","last_active":"2024-01-15T18:05:00.740140+0000","last_peered":"2024-01-15T18:05:00.740140+0000","last_clean":"2024-01-15T18:05:00.740140+0000","last_became_active":"2024-01-15T18:04:59.305785+0000","last_became_peered":"2024-01-15T18:04:59.305785+0000","last_unstale":"2024-01-15T18:05:00.740140+0000","last_undegraded":"2024-01-15T18:05:00.740140+0000","last_fullsized":"2024-01-15T18:05:00.740140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283482+0000","last_change":"2024-01-15T18:04:58.292333+0000","last_active":"2024-01-15T18:05:00.283482+0000","last_peered":"2024-01-15T18:05:00.283482+0000","last_clean":"2024-01-15T18:05:00.283482+0000","last_became_active":"2024-01-15T18:04:58.291543+0000","last_became_peered":"2024-01-15T18:04:58.291543+0000","last_unstale":"2024-01-15T18:05:00.283482+0000","last_undegraded":"2024-01-15T18:05:00.283482+0000","last_fullsized":"2024-01-15T18:05:00.283482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353424+0000","last_change":"2024-01-15T18:04:59.313691+0000","last_active":"2024-01-15T18:05:00.353424+0000","last_peered":"2024-01-15T18:05:00.353424+0000","last_clean":"2024-01-15T18:05:00.353424+0000","last_became_active":"2024-01-15T18:04:59.313395+0000","last_became_peered":"2024-01-15T18:04:59.313395+0000","last_unstale":"2024-01-15T18:05:00.353424+0000","last_undegraded":"2024-01-15T18:05:00.353424+0000","last_fullsized":"2024-01-15T18:05:00.353424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353418+0000","last_change":"2024-01-15T18:04:58.288981+0000","last_active":"2024-01-15T18:05:00.353418+0000","last_peered":"2024-01-15T18:05:00.353418+0000","last_clean":"2024-01-15T18:05:00.353418+0000","last_became_active":"2024-01-15T18:04:58.288667+0000","last_became_peered":"2024-01-15T18:04:58.288667+0000","last_unstale":"2024-01-15T18:05:00.353418+0000","last_undegraded":"2024-01-15T18:05:00.353418+0000","last_fullsized":"2024-01-15T18:05:00.353418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740466+0000","last_change":"2024-01-15T18:04:59.311337+0000","last_active":"2024-01-15T18:05:00.740466+0000","last_peered":"2024-01-15T18:05:00.740466+0000","last_clean":"2024-01-15T18:05:00.740466+0000","last_became_active":"2024-01-15T18:04:59.311102+0000","last_became_peered":"2024-01-15T18:04:59.311102+0000","last_unstale":"2024-01-15T18:05:00.740466+0000","last_undegraded":"2024-01-15T18:05:00.740466+0000","last_fullsized":"2024-01-15T18:05:00.740466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740475+0000","last_change":"2024-01-15T18:04:58.295173+0000","last_active":"2024-01-15T18:05:00.740475+0000","last_peered":"2024-01-15T18:05:00.740475+0000","last_clean":"2024-01-15T18:05:00.740475+0000","last_became_active":"2024-01-15T18:04:58.294883+0000","last_became_peered":"2024-01-15T18:04:58.294883+0000","last_unstale":"2024-01-15T18:05:00.740475+0000","last_undegraded":"2024-01-15T18:05:00.740475+0000","last_fullsized":"2024-01-15T18:05:00.740475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284186+0000","last_change":"2024-01-15T18:04:59.311644+0000","last_active":"2024-01-15T18:05:00.284186+0000","last_peered":"2024-01-15T18:05:00.284186+0000","last_clean":"2024-01-15T18:05:00.284186+0000","last_became_active":"2024-01-15T18:04:59.311224+0000","last_became_peered":"2024-01-15T18:04:59.311224+0000","last_unstale":"2024-01-15T18:05:00.284186+0000","last_undegraded":"2024-01-15T18:05:00.284186+0000","last_fullsized":"2024-01-15T18:05:00.284186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283776+0000","last_change":"2024-01-15T18:04:58.293217+0000","last_active":"2024-01-15T18:05:00.283776+0000","last_peered":"2024-01-15T18:05:00.283776+0000","last_clean":"2024-01-15T18:05:00.283776+0000","last_became_active":"2024-01-15T18:04:58.292902+0000","last_became_peered":"2024-01-15T18:04:58.292902+0000","last_unstale":"2024-01-15T18:05:00.283776+0000","last_undegraded":"2024-01-15T18:05:00.283776+0000","last_fullsized":"2024-01-15T18:05:00.283776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539100+0000","last_change":"2024-01-15T18:04:59.313869+0000","last_active":"2024-01-15T18:05:00.539100+0000","last_peered":"2024-01-15T18:05:00.539100+0000","last_clean":"2024-01-15T18:05:00.539100+0000","last_became_active":"2024-01-15T18:04:59.313336+0000","last_became_peered":"2024-01-15T18:04:59.313336+0000","last_unstale":"2024-01-15T18:05:00.539100+0000","last_undegraded":"2024-01-15T18:05:00.539100+0000","last_fullsized":"2024-01-15T18:05:00.539100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740054+0000","last_change":"2024-01-15T18:04:58.294176+0000","last_active":"2024-01-15T18:05:00.740054+0000","last_peered":"2024-01-15T18:05:00.740054+0000","last_clean":"2024-01-15T18:05:00.740054+0000","last_became_active":"2024-01-15T18:04:58.292524+0000","last_became_peered":"2024-01-15T18:04:58.292524+0000","last_unstale":"2024-01-15T18:05:00.740054+0000","last_undegraded":"2024-01-15T18:05:00.740054+0000","last_fullsized":"2024-01-15T18:05:00.740054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740077+0000","last_change":"2024-01-15T18:04:59.300038+0000","last_active":"2024-01-15T18:05:00.740077+0000","last_peered":"2024-01-15T18:05:00.740077+0000","last_clean":"2024-01-15T18:05:00.740077+0000","last_became_active":"2024-01-15T18:04:59.299776+0000","last_became_peered":"2024-01-15T18:04:59.299776+0000","last_unstale":"2024-01-15T18:05:00.740077+0000","last_undegraded":"2024-01-15T18:05:00.740077+0000","last_fullsized":"2024-01-15T18:05:00.740077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353198+0000","last_change":"2024-01-15T18:04:58.297644+0000","last_active":"2024-01-15T18:05:00.353198+0000","last_peered":"2024-01-15T18:05:00.353198+0000","last_clean":"2024-01-15T18:05:00.353198+0000","last_became_active":"2024-01-15T18:04:58.296922+0000","last_became_peered":"2024-01-15T18:04:58.296922+0000","last_unstale":"2024-01-15T18:05:00.353198+0000","last_undegraded":"2024-01-15T18:05:00.353198+0000","last_fullsized":"2024-01-15T18:05:00.353198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539142+0000","last_change":"2024-01-15T18:04:59.312547+0000","last_active":"2024-01-15T18:05:00.539142+0000","last_peered":"2024-01-15T18:05:00.539142+0000","last_clean":"2024-01-15T18:05:00.539142+0000","last_became_active":"2024-01-15T18:04:59.312203+0000","last_became_peered":"2024-01-15T18:04:59.312203+0000","last_unstale":"2024-01-15T18:05:00.539142+0000","last_undegraded":"2024-01-15T18:05:00.539142+0000","last_fullsized":"2024-01-15T18:05:00.539142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283454+0000","last_change":"2024-01-15T18:04:58.297011+0000","last_active":"2024-01-15T18:05:00.283454+0000","last_peered":"2024-01-15T18:05:00.283454+0000","last_clean":"2024-01-15T18:05:00.283454+0000","last_became_active":"2024-01-15T18:04:58.296476+0000","last_became_peered":"2024-01-15T18:04:58.296476+0000","last_unstale":"2024-01-15T18:05:00.283454+0000","last_undegraded":"2024-01-15T18:05:00.283454+0000","last_fullsized":"2024-01-15T18:05:00.283454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:39.535650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283371+0000","last_change":"2024-01-15T18:04:59.294840+0000","last_active":"2024-01-15T18:05:00.283371+0000","last_peered":"2024-01-15T18:05:00.283371+0000","last_clean":"2024-01-15T18:05:00.283371+0000","last_became_active":"2024-01-15T18:04:59.294449+0000","last_became_peered":"2024-01-15T18:04:59.294449+0000","last_unstale":"2024-01-15T18:05:00.283371+0000","last_undegraded":"2024-01-15T18:05:00.283371+0000","last_fullsized":"2024-01-15T18:05:00.283371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739071+0000","last_change":"2024-01-15T18:04:58.285671+0000","last_active":"2024-01-15T18:05:00.739071+0000","last_peered":"2024-01-15T18:05:00.739071+0000","last_clean":"2024-01-15T18:05:00.739071+0000","last_became_active":"2024-01-15T18:04:58.285350+0000","last_became_peered":"2024-01-15T18:04:58.285350+0000","last_unstale":"2024-01-15T18:05:00.739071+0000","last_undegraded":"2024-01-15T18:05:00.739071+0000","last_fullsized":"2024-01-15T18:05:00.739071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538843+0000","last_change":"2024-01-15T18:04:59.315031+0000","last_active":"2024-01-15T18:05:00.538843+0000","last_peered":"2024-01-15T18:05:00.538843+0000","last_clean":"2024-01-15T18:05:00.538843+0000","last_became_active":"2024-01-15T18:04:59.311772+0000","last_became_peered":"2024-01-15T18:04:59.311772+0000","last_unstale":"2024-01-15T18:05:00.538843+0000","last_undegraded":"2024-01-15T18:05:00.538843+0000","last_fullsized":"2024-01-15T18:05:00.538843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284479+0000","last_change":"2024-01-15T18:04:58.290373+0000","last_active":"2024-01-15T18:05:00.284479+0000","last_peered":"2024-01-15T18:05:00.284479+0000","last_clean":"2024-01-15T18:05:00.284479+0000","last_became_active":"2024-01-15T18:04:58.290060+0000","last_became_peered":"2024-01-15T18:04:58.290060+0000","last_unstale":"2024-01-15T18:05:00.284479+0000","last_undegraded":"2024-01-15T18:05:00.284479+0000","last_fullsized":"2024-01-15T18:05:00.284479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539224+0000","last_change":"2024-01-15T18:04:59.315975+0000","last_active":"2024-01-15T18:05:00.539224+0000","last_peered":"2024-01-15T18:05:00.539224+0000","last_clean":"2024-01-15T18:05:00.539224+0000","last_became_active":"2024-01-15T18:04:59.315667+0000","last_became_peered":"2024-01-15T18:04:59.315667+0000","last_unstale":"2024-01-15T18:05:00.539224+0000","last_undegraded":"2024-01-15T18:05:00.539224+0000","last_fullsized":"2024-01-15T18:05:00.539224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739314+0000","last_change":"2024-01-15T18:04:58.297988+0000","last_active":"2024-01-15T18:05:00.739314+0000","last_peered":"2024-01-15T18:05:00.739314+0000","last_clean":"2024-01-15T18:05:00.739314+0000","last_became_active":"2024-01-15T18:04:58.296427+0000","last_became_peered":"2024-01-15T18:04:58.296427+0000","last_unstale":"2024-01-15T18:05:00.739314+0000","last_undegraded":"2024-01-15T18:05:00.739314+0000","last_fullsized":"2024-01-15T18:05:00.739314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539392+0000","last_change":"2024-01-15T18:04:59.312511+0000","last_active":"2024-01-15T18:05:00.539392+0000","last_peered":"2024-01-15T18:05:00.539392+0000","last_clean":"2024-01-15T18:05:00.539392+0000","last_became_active":"2024-01-15T18:04:59.312195+0000","last_became_peered":"2024-01-15T18:04:59.312195+0000","last_unstale":"2024-01-15T18:05:00.539392+0000","last_undegraded":"2024-01-15T18:05:00.539392+0000","last_fullsized":"2024-01-15T18:05:00.539392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:09:03.873184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740526+0000","last_change":"2024-01-15T18:04:58.289637+0000","last_active":"2024-01-15T18:05:00.740526+0000","last_peered":"2024-01-15T18:05:00.740526+0000","last_clean":"2024-01-15T18:05:00.740526+0000","last_became_active":"2024-01-15T18:04:58.289228+0000","last_became_peered":"2024-01-15T18:04:58.289228+0000","last_unstale":"2024-01-15T18:05:00.740526+0000","last_undegraded":"2024-01-15T18:05:00.740526+0000","last_fullsized":"2024-01-15T18:05:00.740526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353542+0000","last_change":"2024-01-15T18:04:59.309405+0000","last_active":"2024-01-15T18:05:00.353542+0000","last_peered":"2024-01-15T18:05:00.353542+0000","last_clean":"2024-01-15T18:05:00.353542+0000","last_became_active":"2024-01-15T18:04:59.309119+0000","last_became_peered":"2024-01-15T18:04:59.309119+0000","last_unstale":"2024-01-15T18:05:00.353542+0000","last_undegraded":"2024-01-15T18:05:00.353542+0000","last_fullsized":"2024-01-15T18:05:00.353542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739208+0000","last_change":"2024-01-15T18:04:58.294285+0000","last_active":"2024-01-15T18:05:00.739208+0000","last_peered":"2024-01-15T18:05:00.739208+0000","last_clean":"2024-01-15T18:05:00.739208+0000","last_became_active":"2024-01-15T18:04:58.293970+0000","last_became_peered":"2024-01-15T18:04:58.293970+0000","last_unstale":"2024-01-15T18:05:00.739208+0000","last_undegraded":"2024-01-15T18:05:00.739208+0000","last_fullsized":"2024-01-15T18:05:00.739208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.314051+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.313761+0000","last_became_peered":"2024-01-15T18:04:59.313761+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-15T18:04:59.290583+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:04:59.290583+0000","last_peered":"2024-01-15T18:04:59.290583+0000","last_clean":"2024-01-15T18:04:59.290583+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:04:59.290583+0000","last_undegraded":"2024-01-15T18:04:59.290583+0000","last_fullsized":"2024-01-15T18:04:59.290583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353134+0000","last_change":"2024-01-15T18:04:58.286159+0000","last_active":"2024-01-15T18:05:00.353134+0000","last_peered":"2024-01-15T18:05:00.353134+0000","last_clean":"2024-01-15T18:05:00.353134+0000","last_became_active":"2024-01-15T18:04:58.285819+0000","last_became_peered":"2024-01-15T18:04:58.285819+0000","last_unstale":"2024-01-15T18:05:00.353134+0000","last_undegraded":"2024-01-15T18:05:00.353134+0000","last_fullsized":"2024-01-15T18:05:00.353134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285340+0000","last_change":"2024-01-15T18:04:59.305454+0000","last_active":"2024-01-15T18:05:00.285340+0000","last_peered":"2024-01-15T18:05:00.285340+0000","last_clean":"2024-01-15T18:05:00.285340+0000","last_became_active":"2024-01-15T18:04:59.305160+0000","last_became_peered":"2024-01-15T18:04:59.305160+0000","last_unstale":"2024-01-15T18:05:00.285340+0000","last_undegraded":"2024-01-15T18:05:00.285340+0000","last_fullsized":"2024-01-15T18:05:00.285340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538748+0000","last_change":"2024-01-15T18:04:58.297164+0000","last_active":"2024-01-15T18:05:00.538748+0000","last_peered":"2024-01-15T18:05:00.538748+0000","last_clean":"2024-01-15T18:05:00.538748+0000","last_became_active":"2024-01-15T18:04:58.295969+0000","last_became_peered":"2024-01-15T18:04:58.295969+0000","last_unstale":"2024-01-15T18:05:00.538748+0000","last_undegraded":"2024-01-15T18:05:00.538748+0000","last_fullsized":"2024-01-15T18:05:00.538748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739741+0000","last_change":"2024-01-15T18:04:59.297889+0000","last_active":"2024-01-15T18:05:00.739741+0000","last_peered":"2024-01-15T18:05:00.739741+0000","last_clean":"2024-01-15T18:05:00.739741+0000","last_became_active":"2024-01-15T18:04:59.296874+0000","last_became_peered":"2024-01-15T18:04:59.296874+0000","last_unstale":"2024-01-15T18:05:00.739741+0000","last_undegraded":"2024-01-15T18:05:00.739741+0000","last_fullsized":"2024-01-15T18:05:00.739741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:52.200760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739266+0000","last_change":"2024-01-15T18:04:58.293459+0000","last_active":"2024-01-15T18:05:00.739266+0000","last_peered":"2024-01-15T18:05:00.739266+0000","last_clean":"2024-01-15T18:05:00.739266+0000","last_became_active":"2024-01-15T18:04:58.292396+0000","last_became_peered":"2024-01-15T18:04:58.292396+0000","last_unstale":"2024-01-15T18:05:00.739266+0000","last_undegraded":"2024-01-15T18:05:00.739266+0000","last_fullsized":"2024-01-15T18:05:00.739266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740002+0000","last_change":"2024-01-15T18:04:59.302244+0000","last_active":"2024-01-15T18:05:00.740002+0000","last_peered":"2024-01-15T18:05:00.740002+0000","last_clean":"2024-01-15T18:05:00.740002+0000","last_became_active":"2024-01-15T18:04:59.301981+0000","last_became_peered":"2024-01-15T18:04:59.301981+0000","last_unstale":"2024-01-15T18:05:00.740002+0000","last_undegraded":"2024-01-15T18:05:00.740002+0000","last_fullsized":"2024-01-15T18:05:00.740002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353265+0000","last_change":"2024-01-15T18:04:58.288895+0000","last_active":"2024-01-15T18:05:00.353265+0000","last_peered":"2024-01-15T18:05:00.353265+0000","last_clean":"2024-01-15T18:05:00.353265+0000","last_became_active":"2024-01-15T18:04:58.288385+0000","last_became_peered":"2024-01-15T18:04:58.288385+0000","last_unstale":"2024-01-15T18:05:00.353265+0000","last_undegraded":"2024-01-15T18:05:00.353265+0000","last_fullsized":"2024-01-15T18:05:00.353265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:10.169965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283326+0000","last_change":"2024-01-15T18:04:59.289150+0000","last_active":"2024-01-15T18:05:00.283326+0000","last_peered":"2024-01-15T18:05:00.283326+0000","last_clean":"2024-01-15T18:05:00.283326+0000","last_became_active":"2024-01-15T18:04:59.288315+0000","last_became_peered":"2024-01-15T18:04:59.288315+0000","last_unstale":"2024-01-15T18:05:00.283326+0000","last_undegraded":"2024-01-15T18:05:00.283326+0000","last_fullsized":"2024-01-15T18:05:00.283326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:14.706225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739871+0000","last_change":"2024-01-15T18:04:58.280841+0000","last_active":"2024-01-15T18:05:00.739871+0000","last_peered":"2024-01-15T18:05:00.739871+0000","last_clean":"2024-01-15T18:05:00.739871+0000","last_became_active":"2024-01-15T18:04:58.280590+0000","last_became_peered":"2024-01-15T18:04:58.280590+0000","last_unstale":"2024-01-15T18:05:00.739871+0000","last_undegraded":"2024-01-15T18:05:00.739871+0000","last_fullsized":"2024-01-15T18:05:00.739871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353597+0000","last_change":"2024-01-15T18:04:59.310649+0000","last_active":"2024-01-15T18:05:00.353597+0000","last_peered":"2024-01-15T18:05:00.353597+0000","last_clean":"2024-01-15T18:05:00.353597+0000","last_became_active":"2024-01-15T18:04:59.310266+0000","last_became_peered":"2024-01-15T18:04:59.310266+0000","last_unstale":"2024-01-15T18:05:00.353597+0000","last_undegraded":"2024-01-15T18:05:00.353597+0000","last_fullsized":"2024-01-15T18:05:00.353597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539301+0000","last_change":"2024-01-15T18:04:58.300881+0000","last_active":"2024-01-15T18:05:00.539301+0000","last_peered":"2024-01-15T18:05:00.539301+0000","last_clean":"2024-01-15T18:05:00.539301+0000","last_became_active":"2024-01-15T18:04:58.300322+0000","last_became_peered":"2024-01-15T18:04:58.300322+0000","last_unstale":"2024-01-15T18:05:00.539301+0000","last_undegraded":"2024-01-15T18:05:00.539301+0000","last_fullsized":"2024-01-15T18:05:00.539301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:47.163925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.299409+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.299156+0000","last_became_peered":"2024-01-15T18:04:59.299156+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740587+0000","last_change":"2024-01-15T18:04:58.283804+0000","last_active":"2024-01-15T18:05:00.740587+0000","last_peered":"2024-01-15T18:05:00.740587+0000","last_clean":"2024-01-15T18:05:00.740587+0000","last_became_active":"2024-01-15T18:04:58.283533+0000","last_became_peered":"2024-01-15T18:04:58.283533+0000","last_unstale":"2024-01-15T18:05:00.740587+0000","last_undegraded":"2024-01-15T18:05:00.740587+0000","last_fullsized":"2024-01-15T18:05:00.740587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285921+0000","last_change":"2024-01-15T18:04:59.316822+0000","last_active":"2024-01-15T18:05:00.285921+0000","last_peered":"2024-01-15T18:05:00.285921+0000","last_clean":"2024-01-15T18:05:00.285921+0000","last_became_active":"2024-01-15T18:04:59.316513+0000","last_became_peered":"2024-01-15T18:04:59.316513+0000","last_unstale":"2024-01-15T18:05:00.285921+0000","last_undegraded":"2024-01-15T18:05:00.285921+0000","last_fullsized":"2024-01-15T18:05:00.285921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283543+0000","last_change":"2024-01-15T18:04:58.292591+0000","last_active":"2024-01-15T18:05:00.283543+0000","last_peered":"2024-01-15T18:05:00.283543+0000","last_clean":"2024-01-15T18:05:00.283543+0000","last_became_active":"2024-01-15T18:04:58.291917+0000","last_became_peered":"2024-01-15T18:04:58.291917+0000","last_unstale":"2024-01-15T18:05:00.283543+0000","last_undegraded":"2024-01-15T18:05:00.283543+0000","last_fullsized":"2024-01-15T18:05:00.283543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740270+0000","last_change":"2024-01-15T18:04:59.300764+0000","last_active":"2024-01-15T18:05:00.740270+0000","last_peered":"2024-01-15T18:05:00.740270+0000","last_clean":"2024-01-15T18:05:00.740270+0000","last_became_active":"2024-01-15T18:04:59.299980+0000","last_became_peered":"2024-01-15T18:04:59.299980+0000","last_unstale":"2024-01-15T18:05:00.740270+0000","last_undegraded":"2024-01-15T18:05:00.740270+0000","last_fullsized":"2024-01-15T18:05:00.740270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538729+0000","last_change":"2024-01-15T18:04:58.301082+0000","last_active":"2024-01-15T18:05:00.538729+0000","last_peered":"2024-01-15T18:05:00.538729+0000","last_clean":"2024-01-15T18:05:00.538729+0000","last_became_active":"2024-01-15T18:04:58.300809+0000","last_became_peered":"2024-01-15T18:04:58.300809+0000","last_unstale":"2024-01-15T18:05:00.538729+0000","last_undegraded":"2024-01-15T18:05:00.538729+0000","last_fullsized":"2024-01-15T18:05:00.538729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283231+0000","last_change":"2024-01-15T18:04:59.297502+0000","last_active":"2024-01-15T18:05:00.283231+0000","last_peered":"2024-01-15T18:05:00.283231+0000","last_clean":"2024-01-15T18:05:00.283231+0000","last_became_active":"2024-01-15T18:04:59.297211+0000","last_became_peered":"2024-01-15T18:04:59.297211+0000","last_unstale":"2024-01-15T18:05:00.283231+0000","last_undegraded":"2024-01-15T18:05:00.283231+0000","last_fullsized":"2024-01-15T18:05:00.283231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:13.025413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740211+0000","last_change":"2024-01-15T18:04:58.296402+0000","last_active":"2024-01-15T18:05:00.740211+0000","last_peered":"2024-01-15T18:05:00.740211+0000","last_clean":"2024-01-15T18:05:00.740211+0000","last_became_active":"2024-01-15T18:04:58.296101+0000","last_became_peered":"2024-01-15T18:04:58.296101+0000","last_unstale":"2024-01-15T18:05:00.740211+0000","last_undegraded":"2024-01-15T18:05:00.740211+0000","last_fullsized":"2024-01-15T18:05:00.740211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285133+0000","last_change":"2024-01-15T18:04:59.298441+0000","last_active":"2024-01-15T18:05:00.285133+0000","last_peered":"2024-01-15T18:05:00.285133+0000","last_clean":"2024-01-15T18:05:00.285133+0000","last_became_active":"2024-01-15T18:04:59.298204+0000","last_became_peered":"2024-01-15T18:04:59.298204+0000","last_unstale":"2024-01-15T18:05:00.285133+0000","last_undegraded":"2024-01-15T18:05:00.285133+0000","last_fullsized":"2024-01-15T18:05:00.285133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740408+0000","last_change":"2024-01-15T18:04:58.297952+0000","last_active":"2024-01-15T18:05:00.740408+0000","last_peered":"2024-01-15T18:05:00.740408+0000","last_clean":"2024-01-15T18:05:00.740408+0000","last_became_active":"2024-01-15T18:04:58.297621+0000","last_became_peered":"2024-01-15T18:04:58.297621+0000","last_unstale":"2024-01-15T18:05:00.740408+0000","last_undegraded":"2024-01-15T18:05:00.740408+0000","last_fullsized":"2024-01-15T18:05:00.740408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285600+0000","last_change":"2024-01-15T18:04:59.304388+0000","last_active":"2024-01-15T18:05:00.285600+0000","last_peered":"2024-01-15T18:05:00.285600+0000","last_clean":"2024-01-15T18:05:00.285600+0000","last_became_active":"2024-01-15T18:04:59.303178+0000","last_became_peered":"2024-01-15T18:04:59.303178+0000","last_unstale":"2024-01-15T18:05:00.285600+0000","last_undegraded":"2024-01-15T18:05:00.285600+0000","last_fullsized":"2024-01-15T18:05:00.285600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539246+0000","last_change":"2024-01-15T18:04:58.300443+0000","last_active":"2024-01-15T18:05:00.539246+0000","last_peered":"2024-01-15T18:05:00.539246+0000","last_clean":"2024-01-15T18:05:00.539246+0000","last_became_active":"2024-01-15T18:04:58.300157+0000","last_became_peered":"2024-01-15T18:04:58.300157+0000","last_unstale":"2024-01-15T18:05:00.539246+0000","last_undegraded":"2024-01-15T18:05:00.539246+0000","last_fullsized":"2024-01-15T18:05:00.539246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283605+0000","last_change":"2024-01-15T18:04:59.295964+0000","last_active":"2024-01-15T18:05:00.283605+0000","last_peered":"2024-01-15T18:05:00.283605+0000","last_clean":"2024-01-15T18:05:00.283605+0000","last_became_active":"2024-01-15T18:04:59.295569+0000","last_became_peered":"2024-01-15T18:04:59.295569+0000","last_unstale":"2024-01-15T18:05:00.283605+0000","last_undegraded":"2024-01-15T18:05:00.283605+0000","last_fullsized":"2024-01-15T18:05:00.283605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.316899+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.316632+0000","last_became_peered":"2024-01-15T18:04:59.316632+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538942+0000","last_change":"2024-01-15T18:04:58.301177+0000","last_active":"2024-01-15T18:05:00.538942+0000","last_peered":"2024-01-15T18:05:00.538942+0000","last_clean":"2024-01-15T18:05:00.538942+0000","last_became_active":"2024-01-15T18:04:58.300869+0000","last_became_peered":"2024-01-15T18:04:58.300869+0000","last_unstale":"2024-01-15T18:05:00.538942+0000","last_undegraded":"2024-01-15T18:05:00.538942+0000","last_fullsized":"2024-01-15T18:05:00.538942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353471+0000","last_change":"2024-01-15T18:04:59.311012+0000","last_active":"2024-01-15T18:05:00.353471+0000","last_peered":"2024-01-15T18:05:00.353471+0000","last_clean":"2024-01-15T18:05:00.353471+0000","last_became_active":"2024-01-15T18:04:59.310673+0000","last_became_peered":"2024-01-15T18:04:59.310673+0000","last_unstale":"2024-01-15T18:05:00.353471+0000","last_undegraded":"2024-01-15T18:05:00.353471+0000","last_fullsized":"2024-01-15T18:05:00.353471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539209+0000","last_change":"2024-01-15T18:04:58.303349+0000","last_active":"2024-01-15T18:05:00.539209+0000","last_peered":"2024-01-15T18:05:00.539209+0000","last_clean":"2024-01-15T18:05:00.539209+0000","last_became_active":"2024-01-15T18:04:58.303074+0000","last_became_peered":"2024-01-15T18:04:58.303074+0000","last_unstale":"2024-01-15T18:05:00.539209+0000","last_undegraded":"2024-01-15T18:05:00.539209+0000","last_fullsized":"2024-01-15T18:05:00.539209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740412+0000","last_change":"2024-01-15T18:04:59.311306+0000","last_active":"2024-01-15T18:05:00.740412+0000","last_peered":"2024-01-15T18:05:00.740412+0000","last_clean":"2024-01-15T18:05:00.740412+0000","last_became_active":"2024-01-15T18:04:59.311031+0000","last_became_peered":"2024-01-15T18:04:59.311031+0000","last_unstale":"2024-01-15T18:05:00.740412+0000","last_undegraded":"2024-01-15T18:05:00.740412+0000","last_fullsized":"2024-01-15T18:05:00.740412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285716+0000","last_change":"2024-01-15T18:04:58.299606+0000","last_active":"2024-01-15T18:05:00.285716+0000","last_peered":"2024-01-15T18:05:00.285716+0000","last_clean":"2024-01-15T18:05:00.285716+0000","last_became_active":"2024-01-15T18:04:58.299353+0000","last_became_peered":"2024-01-15T18:04:58.299353+0000","last_unstale":"2024-01-15T18:05:00.285716+0000","last_undegraded":"2024-01-15T18:05:00.285716+0000","last_fullsized":"2024-01-15T18:05:00.285716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285464+0000","last_change":"2024-01-15T18:04:59.304019+0000","last_active":"2024-01-15T18:05:00.285464+0000","last_peered":"2024-01-15T18:05:00.285464+0000","last_clean":"2024-01-15T18:05:00.285464+0000","last_became_active":"2024-01-15T18:04:59.303662+0000","last_became_peered":"2024-01-15T18:04:59.303662+0000","last_unstale":"2024-01-15T18:05:00.285464+0000","last_undegraded":"2024-01-15T18:05:00.285464+0000","last_fullsized":"2024-01-15T18:05:00.285464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740536+0000","last_change":"2024-01-15T18:04:58.303996+0000","last_active":"2024-01-15T18:05:00.740536+0000","last_peered":"2024-01-15T18:05:00.740536+0000","last_clean":"2024-01-15T18:05:00.740536+0000","last_became_active":"2024-01-15T18:04:58.303687+0000","last_became_peered":"2024-01-15T18:04:58.303687+0000","last_unstale":"2024-01-15T18:05:00.740536+0000","last_undegraded":"2024-01-15T18:05:00.740536+0000","last_fullsized":"2024-01-15T18:05:00.740536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:05:03.564 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:05:03.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.733+0000 7f4377a00640 1 Processor -- start 2024-01-15T18:05:03.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.733+0000 7f4377a00640 1 -- start start 2024-01-15T18:05:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4377a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4370157300 0x7f4370159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4377a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b4ac0 0x7f4370070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4377a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43700b2a20 con 0x7f43700b21d0 2024-01-15T18:05:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4377a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43700b35a0 con 0x7f4370157300 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4377a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43700b2d00 con 0x7f43700b4ac0 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f436fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f43700b21d0 0x7f43700b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34074/0 (socket says 172.21.15.143:34074) 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4375000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4370157300 0x7f4370159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f436fe00640 1 -- 172.21.15.143:0/3727389091 learned_addr learned my addr 172.21.15.143:0/3727389091 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4375a00640 1 --2- 172.21.15.143:0/3727389091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b4ac0 0x7f4370070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4375000640 1 -- 172.21.15.143:0/3727389091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b4ac0 msgr2=0x7f4370070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4375000640 1 --2- 172.21.15.143:0/3727389091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b4ac0 0x7f4370070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4375000640 1 -- 172.21.15.143:0/3727389091 >> v1:172.21.15.143:6789/0 conn(0x7f43700b21d0 legacy=0x7f43700b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.737+0000 7f4375000640 1 -- 172.21.15.143:0/3727389091 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700b2fe0 con 0x7f4370157300 2024-01-15T18:05:03.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4375000640 1 --2- 172.21.15.143:0/3727389091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f4370159b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43600097c0 tx=0x7f436001d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4671ed79a975207a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f436f400640 1 -- 172.21.15.143:0/3727389091 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43600aa650 con 0x7f4370157300 2024-01-15T18:05:03.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f436f400640 1 -- 172.21.15.143:0/3727389091 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43600aa7b0 con 0x7f4370157300 2024-01-15T18:05:03.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f436f400640 1 -- 172.21.15.143:0/3727389091 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43600aab40 con 0x7f4370157300 2024-01-15T18:05:03.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 -- 172.21.15.143:0/3727389091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 msgr2=0x7f4370159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 --2- 172.21.15.143:0/3727389091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f4370159b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f43600097c0 tx=0x7f436001d560 comp rx=0 tx=0).stop 2024-01-15T18:05:03.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 -- 172.21.15.143:0/3727389091 shutdown_connections 2024-01-15T18:05:03.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 --2- 172.21.15.143:0/3727389091 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b4ac0 0x7f4370070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 --2- 172.21.15.143:0/3727389091 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f4370159b70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 -- 172.21.15.143:0/3727389091 >> 172.21.15.143:0/3727389091 conn(0x7f437006c3f0 msgr2=0x7f437006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 -- 172.21.15.143:0/3727389091 shutdown_connections 2024-01-15T18:05:03.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 -- 172.21.15.143:0/3727389091 wait complete. 2024-01-15T18:05:03.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 Processor -- start 2024-01-15T18:05:03.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.741+0000 7f4377a00640 1 -- start start 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b21d0 0x7f43700ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43700b4ac0 0x7f43701576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f437014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700ac8e0 con 0x7f4370157300 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43701f6360 con 0x7f43700b4ac0 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43701f6640 con 0x7f43700b21d0 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f437014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f437014d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56316/0 (socket says 172.21.15.143:56316) 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 -- 172.21.15.143:0/814153527 learned_addr learned my addr 172.21.15.143:0/814153527 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f436fe00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43700b4ac0 0x7f43701576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375000640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b21d0 0x7f43700ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 -- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b21d0 msgr2=0x7f43700ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b21d0 0x7f43700ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 -- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43700b4ac0 msgr2=0x7f43701576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43700b4ac0 0x7f43701576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 -- 172.21.15.143:0/814153527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009420 con 0x7f4370157300 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4375a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f437014d880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f436800ef90 tx=0x7f436800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4368009bf0 con 0x7f4370157300 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800f040 con 0x7f4370157300 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43701f6920 con 0x7f4370157300 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43680146c0 con 0x7f4370157300 2024-01-15T18:05:03.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43700ae570 con 0x7f4370157300 2024-01-15T18:05:03.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.745+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43700aca50 con 0x7f4370157300 2024-01-15T18:05:03.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.749+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f4368014820 con 0x7f4370157300 2024-01-15T18:05:03.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.749+0000 7f436cc00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f43480668f0 0x7f4348068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.749+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4368085100 con 0x7f4370157300 2024-01-15T18:05:03.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.749+0000 7f4375000640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f43480668f0 0x7f4348068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.749+0000 7f4375000640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f43480668f0 0x7f4348068db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43600af000 tx=0x7f43600a8000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.753+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f43680523f0 con 0x7f4370157300 2024-01-15T18:05:03.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.901+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43700b4180 con 0x7f43480668f0 2024-01-15T18:05:03.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.905+0000 7f436cc00640 1 -- 172.21.15.143:0/814153527 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264659 (secure 0 0 0) 0x7f43700b4180 con 0x7f43480668f0 2024-01-15T18:05:03.915 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:03.916 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:05:03.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f43480668f0 msgr2=0x7f4348068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f43480668f0 0x7f4348068db0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f43600af000 tx=0x7f43600a8000 comp rx=0 tx=0).stop 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 msgr2=0x7f437014d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f437014d880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f436800ef90 tx=0x7f436800c490 comp rx=0 tx=0).stop 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 shutdown_connections 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f43480668f0 0x7f4348068db0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4370157300 0x7f437014d880 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f43700b4ac0 0x7f43701576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 --2- 172.21.15.143:0/814153527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f43700b21d0 0x7f43700ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 >> 172.21.15.143:0/814153527 conn(0x7f437006c3f0 msgr2=0x7f43700b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 shutdown_connections 2024-01-15T18:05:03.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:03.909+0000 7f4377a00640 1 -- 172.21.15.143:0/814153527 wait complete. 2024-01-15T18:05:03.935 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-15T18:05:02.622126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9],"upper_bound":16},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.360327"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539184+0000","last_change":"2024-01-15T18:04:59.313108+0000","last_active":"2024-01-15T18:05:00.539184+0000","last_peered":"2024-01-15T18:05:00.539184+0000","last_clean":"2024-01-15T18:05:00.539184+0000","last_became_active":"2024-01-15T18:04:59.312739+0000","last_became_peered":"2024-01-15T18:04:59.312739+0000","last_unstale":"2024-01-15T18:05:00.539184+0000","last_undegraded":"2024-01-15T18:05:00.539184+0000","last_fullsized":"2024-01-15T18:05:00.539184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740563+0000","last_change":"2024-01-15T18:04:59.303929+0000","last_active":"2024-01-15T18:05:00.740563+0000","last_peered":"2024-01-15T18:05:00.740563+0000","last_clean":"2024-01-15T18:05:00.740563+0000","last_became_active":"2024-01-15T18:04:59.303598+0000","last_became_peered":"2024-01-15T18:04:59.303598+0000","last_unstale":"2024-01-15T18:05:00.740563+0000","last_undegraded":"2024-01-15T18:05:00.740563+0000","last_fullsized":"2024-01-15T18:05:00.740563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740178+0000","last_change":"2024-01-15T18:04:59.298035+0000","last_active":"2024-01-15T18:05:00.740178+0000","last_peered":"2024-01-15T18:05:00.740178+0000","last_clean":"2024-01-15T18:05:00.740178+0000","last_became_active":"2024-01-15T18:04:59.297748+0000","last_became_peered":"2024-01-15T18:04:59.297748+0000","last_unstale":"2024-01-15T18:05:00.740178+0000","last_undegraded":"2024-01-15T18:05:00.740178+0000","last_fullsized":"2024-01-15T18:05:00.740178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.314048+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.313770+0000","last_became_peered":"2024-01-15T18:04:59.313770+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.308757+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.306380+0000","last_became_peered":"2024-01-15T18:04:59.306380+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353031+0000","last_change":"2024-01-15T18:04:59.307868+0000","last_active":"2024-01-15T18:05:00.353031+0000","last_peered":"2024-01-15T18:05:00.353031+0000","last_clean":"2024-01-15T18:05:00.353031+0000","last_became_active":"2024-01-15T18:04:59.307388+0000","last_became_peered":"2024-01-15T18:04:59.307388+0000","last_unstale":"2024-01-15T18:05:00.353031+0000","last_undegraded":"2024-01-15T18:05:00.353031+0000","last_fullsized":"2024-01-15T18:05:00.353031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538844+0000","last_change":"2024-01-15T18:04:59.316726+0000","last_active":"2024-01-15T18:05:00.538844+0000","last_peered":"2024-01-15T18:05:00.538844+0000","last_clean":"2024-01-15T18:05:00.538844+0000","last_became_active":"2024-01-15T18:04:59.311435+0000","last_became_peered":"2024-01-15T18:04:59.311435+0000","last_unstale":"2024-01-15T18:05:00.538844+0000","last_undegraded":"2024-01-15T18:05:00.538844+0000","last_fullsized":"2024-01-15T18:05:00.538844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284988+0000","last_change":"2024-01-15T18:04:59.305582+0000","last_active":"2024-01-15T18:05:00.284988+0000","last_peered":"2024-01-15T18:05:00.284988+0000","last_clean":"2024-01-15T18:05:00.284988+0000","last_became_active":"2024-01-15T18:04:59.305227+0000","last_became_peered":"2024-01-15T18:04:59.305227+0000","last_unstale":"2024-01-15T18:05:00.284988+0000","last_undegraded":"2024-01-15T18:05:00.284988+0000","last_fullsized":"2024-01-15T18:05:00.284988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539386+0000","last_change":"2024-01-15T18:04:59.310510+0000","last_active":"2024-01-15T18:05:00.539386+0000","last_peered":"2024-01-15T18:05:00.539386+0000","last_clean":"2024-01-15T18:05:00.539386+0000","last_became_active":"2024-01-15T18:04:59.310228+0000","last_became_peered":"2024-01-15T18:04:59.310228+0000","last_unstale":"2024-01-15T18:05:00.539386+0000","last_undegraded":"2024-01-15T18:05:00.539386+0000","last_fullsized":"2024-01-15T18:05:00.539386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283959+0000","last_change":"2024-01-15T18:04:59.308904+0000","last_active":"2024-01-15T18:05:00.283959+0000","last_peered":"2024-01-15T18:05:00.283959+0000","last_clean":"2024-01-15T18:05:00.283959+0000","last_became_active":"2024-01-15T18:04:59.308650+0000","last_became_peered":"2024-01-15T18:04:59.308650+0000","last_unstale":"2024-01-15T18:05:00.283959+0000","last_undegraded":"2024-01-15T18:05:00.283959+0000","last_fullsized":"2024-01-15T18:05:00.283959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283572+0000","last_change":"2024-01-15T18:04:59.311700+0000","last_active":"2024-01-15T18:05:00.283572+0000","last_peered":"2024-01-15T18:05:00.283572+0000","last_clean":"2024-01-15T18:05:00.283572+0000","last_became_active":"2024-01-15T18:04:59.311406+0000","last_became_peered":"2024-01-15T18:04:59.311406+0000","last_unstale":"2024-01-15T18:05:00.283572+0000","last_undegraded":"2024-01-15T18:05:00.283572+0000","last_fullsized":"2024-01-15T18:05:00.283572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284074+0000","last_change":"2024-01-15T18:04:59.287898+0000","last_active":"2024-01-15T18:05:00.284074+0000","last_peered":"2024-01-15T18:05:00.284074+0000","last_clean":"2024-01-15T18:05:00.284074+0000","last_became_active":"2024-01-15T18:04:59.287554+0000","last_became_peered":"2024-01-15T18:04:59.287554+0000","last_unstale":"2024-01-15T18:05:00.284074+0000","last_undegraded":"2024-01-15T18:05:00.284074+0000","last_fullsized":"2024-01-15T18:05:00.284074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:43.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285461+0000","last_change":"2024-01-15T18:04:59.303813+0000","last_active":"2024-01-15T18:05:00.285461+0000","last_peered":"2024-01-15T18:05:00.285461+0000","last_clean":"2024-01-15T18:05:00.285461+0000","last_became_active":"2024-01-15T18:04:59.303549+0000","last_became_peered":"2024-01-15T18:04:59.303549+0000","last_unstale":"2024-01-15T18:05:00.285461+0000","last_undegraded":"2024-01-15T18:05:00.285461+0000","last_fullsized":"2024-01-15T18:05:00.285461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739952+0000","last_change":"2024-01-15T18:04:59.301393+0000","last_active":"2024-01-15T18:05:00.739952+0000","last_peered":"2024-01-15T18:05:00.739952+0000","last_clean":"2024-01-15T18:05:00.739952+0000","last_became_active":"2024-01-15T18:04:59.300604+0000","last_became_peered":"2024-01-15T18:04:59.300604+0000","last_unstale":"2024-01-15T18:05:00.739952+0000","last_undegraded":"2024-01-15T18:05:00.739952+0000","last_fullsized":"2024-01-15T18:05:00.739952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740231+0000","last_change":"2024-01-15T18:04:59.311581+0000","last_active":"2024-01-15T18:05:00.740231+0000","last_peered":"2024-01-15T18:05:00.740231+0000","last_clean":"2024-01-15T18:05:00.740231+0000","last_became_active":"2024-01-15T18:04:59.311309+0000","last_became_peered":"2024-01-15T18:04:59.311309+0000","last_unstale":"2024-01-15T18:05:00.740231+0000","last_undegraded":"2024-01-15T18:05:00.740231+0000","last_fullsized":"2024-01-15T18:05:00.740231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353034+0000","last_change":"2024-01-15T18:04:59.307600+0000","last_active":"2024-01-15T18:05:00.353034+0000","last_peered":"2024-01-15T18:05:00.353034+0000","last_clean":"2024-01-15T18:05:00.353034+0000","last_became_active":"2024-01-15T18:04:59.307160+0000","last_became_peered":"2024-01-15T18:04:59.307160+0000","last_unstale":"2024-01-15T18:05:00.353034+0000","last_undegraded":"2024-01-15T18:05:00.353034+0000","last_fullsized":"2024-01-15T18:05:00.353034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353453+0000","last_change":"2024-01-15T18:04:59.316075+0000","last_active":"2024-01-15T18:05:00.353453+0000","last_peered":"2024-01-15T18:05:00.353453+0000","last_clean":"2024-01-15T18:05:00.353453+0000","last_became_active":"2024-01-15T18:04:59.315751+0000","last_became_peered":"2024-01-15T18:04:59.315751+0000","last_unstale":"2024-01-15T18:05:00.353453+0000","last_undegraded":"2024-01-15T18:05:00.353453+0000","last_fullsized":"2024-01-15T18:05:00.353453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739317+0000","last_change":"2024-01-15T18:04:59.305938+0000","last_active":"2024-01-15T18:05:00.739317+0000","last_peered":"2024-01-15T18:05:00.739317+0000","last_clean":"2024-01-15T18:05:00.739317+0000","last_became_active":"2024-01-15T18:04:59.305686+0000","last_became_peered":"2024-01-15T18:04:59.305686+0000","last_unstale":"2024-01-15T18:05:00.739317+0000","last_undegraded":"2024-01-15T18:05:00.739317+0000","last_fullsized":"2024-01-15T18:05:00.739317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:35.095703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539274+0000","last_change":"2024-01-15T18:04:59.306518+0000","last_active":"2024-01-15T18:05:00.539274+0000","last_peered":"2024-01-15T18:05:00.539274+0000","last_clean":"2024-01-15T18:05:00.539274+0000","last_became_active":"2024-01-15T18:04:59.306009+0000","last_became_peered":"2024-01-15T18:04:59.306009+0000","last_unstale":"2024-01-15T18:05:00.539274+0000","last_undegraded":"2024-01-15T18:05:00.539274+0000","last_fullsized":"2024-01-15T18:05:00.539274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283820+0000","last_change":"2024-01-15T18:04:59.296492+0000","last_active":"2024-01-15T18:05:00.283820+0000","last_peered":"2024-01-15T18:05:00.283820+0000","last_clean":"2024-01-15T18:05:00.283820+0000","last_became_active":"2024-01-15T18:04:59.296200+0000","last_became_peered":"2024-01-15T18:04:59.296200+0000","last_unstale":"2024-01-15T18:05:00.283820+0000","last_undegraded":"2024-01-15T18:05:00.283820+0000","last_fullsized":"2024-01-15T18:05:00.283820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.300916+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.300453+0000","last_became_peered":"2024-01-15T18:04:59.300453+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:50.361044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353117+0000","last_change":"2024-01-15T18:04:59.313753+0000","last_active":"2024-01-15T18:05:00.353117+0000","last_peered":"2024-01-15T18:05:00.353117+0000","last_clean":"2024-01-15T18:05:00.353117+0000","last_became_active":"2024-01-15T18:04:59.313404+0000","last_became_peered":"2024-01-15T18:04:59.313404+0000","last_unstale":"2024-01-15T18:05:00.353117+0000","last_undegraded":"2024-01-15T18:05:00.353117+0000","last_fullsized":"2024-01-15T18:05:00.353117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285121+0000","last_change":"2024-01-15T18:04:59.297113+0000","last_active":"2024-01-15T18:05:00.285121+0000","last_peered":"2024-01-15T18:05:00.285121+0000","last_clean":"2024-01-15T18:05:00.285121+0000","last_became_active":"2024-01-15T18:04:59.296920+0000","last_became_peered":"2024-01-15T18:04:59.296920+0000","last_unstale":"2024-01-15T18:05:00.285121+0000","last_undegraded":"2024-01-15T18:05:00.285121+0000","last_fullsized":"2024-01-15T18:05:00.285121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353148+0000","last_change":"2024-01-15T18:04:59.307789+0000","last_active":"2024-01-15T18:05:00.353148+0000","last_peered":"2024-01-15T18:05:00.353148+0000","last_clean":"2024-01-15T18:05:00.353148+0000","last_became_active":"2024-01-15T18:04:59.306606+0000","last_became_peered":"2024-01-15T18:04:59.306606+0000","last_unstale":"2024-01-15T18:05:00.353148+0000","last_undegraded":"2024-01-15T18:05:00.353148+0000","last_fullsized":"2024-01-15T18:05:00.353148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285732+0000","last_change":"2024-01-15T18:04:59.305088+0000","last_active":"2024-01-15T18:05:00.285732+0000","last_peered":"2024-01-15T18:05:00.285732+0000","last_clean":"2024-01-15T18:05:00.285732+0000","last_became_active":"2024-01-15T18:04:59.304735+0000","last_became_peered":"2024-01-15T18:04:59.304735+0000","last_unstale":"2024-01-15T18:05:00.285732+0000","last_undegraded":"2024-01-15T18:05:00.285732+0000","last_fullsized":"2024-01-15T18:05:00.285732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284011+0000","last_change":"2024-01-15T18:04:59.311649+0000","last_active":"2024-01-15T18:05:00.284011+0000","last_peered":"2024-01-15T18:05:00.284011+0000","last_clean":"2024-01-15T18:05:00.284011+0000","last_became_active":"2024-01-15T18:04:59.311225+0000","last_became_peered":"2024-01-15T18:04:59.311225+0000","last_unstale":"2024-01-15T18:05:00.284011+0000","last_undegraded":"2024-01-15T18:05:00.284011+0000","last_fullsized":"2024-01-15T18:05:00.284011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.306001+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.305732+0000","last_became_peered":"2024-01-15T18:04:59.305732+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285595+0000","last_change":"2024-01-15T18:04:59.306947+0000","last_active":"2024-01-15T18:05:00.285595+0000","last_peered":"2024-01-15T18:05:00.285595+0000","last_clean":"2024-01-15T18:05:00.285595+0000","last_became_active":"2024-01-15T18:04:59.306617+0000","last_became_peered":"2024-01-15T18:04:59.306617+0000","last_unstale":"2024-01-15T18:05:00.285595+0000","last_undegraded":"2024-01-15T18:05:00.285595+0000","last_fullsized":"2024-01-15T18:05:00.285595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.303640+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.303076+0000","last_became_peered":"2024-01-15T18:04:59.303076+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:45.894125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353181+0000","last_change":"2024-01-15T18:04:59.307697+0000","last_active":"2024-01-15T18:05:00.353181+0000","last_peered":"2024-01-15T18:05:00.353181+0000","last_clean":"2024-01-15T18:05:00.353181+0000","last_became_active":"2024-01-15T18:04:59.305330+0000","last_became_peered":"2024-01-15T18:04:59.305330+0000","last_unstale":"2024-01-15T18:05:00.353181+0000","last_undegraded":"2024-01-15T18:05:00.353181+0000","last_fullsized":"2024-01-15T18:05:00.353181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739155+0000","last_change":"2024-01-15T18:04:59.305380+0000","last_active":"2024-01-15T18:05:00.739155+0000","last_peered":"2024-01-15T18:05:00.739155+0000","last_clean":"2024-01-15T18:05:00.739155+0000","last_became_active":"2024-01-15T18:04:59.305090+0000","last_became_peered":"2024-01-15T18:04:59.305090+0000","last_unstale":"2024-01-15T18:05:00.739155+0000","last_undegraded":"2024-01-15T18:05:00.739155+0000","last_fullsized":"2024-01-15T18:05:00.739155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283376+0000","last_change":"2024-01-15T18:04:59.294688+0000","last_active":"2024-01-15T18:05:00.283376+0000","last_peered":"2024-01-15T18:05:00.283376+0000","last_clean":"2024-01-15T18:05:00.283376+0000","last_became_active":"2024-01-15T18:04:59.294065+0000","last_became_peered":"2024-01-15T18:04:59.294065+0000","last_unstale":"2024-01-15T18:05:00.283376+0000","last_undegraded":"2024-01-15T18:05:00.283376+0000","last_fullsized":"2024-01-15T18:05:00.283376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283129+0000","last_change":"2024-01-15T18:04:59.300834+0000","last_active":"2024-01-15T18:05:00.283129+0000","last_peered":"2024-01-15T18:05:00.283129+0000","last_clean":"2024-01-15T18:05:00.283129+0000","last_became_active":"2024-01-15T18:04:59.300257+0000","last_became_peered":"2024-01-15T18:04:59.300257+0000","last_unstale":"2024-01-15T18:05:00.283129+0000","last_undegraded":"2024-01-15T18:05:00.283129+0000","last_fullsized":"2024-01-15T18:05:00.283129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:31.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285696+0000","last_change":"2024-01-15T18:04:58.298585+0000","last_active":"2024-01-15T18:05:00.285696+0000","last_peered":"2024-01-15T18:05:00.285696+0000","last_clean":"2024-01-15T18:05:00.285696+0000","last_became_active":"2024-01-15T18:04:58.298288+0000","last_became_peered":"2024-01-15T18:04:58.298288+0000","last_unstale":"2024-01-15T18:05:00.285696+0000","last_undegraded":"2024-01-15T18:05:00.285696+0000","last_fullsized":"2024-01-15T18:05:00.285696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740106+0000","last_change":"2024-01-15T18:04:59.295504+0000","last_active":"2024-01-15T18:05:00.740106+0000","last_peered":"2024-01-15T18:05:00.740106+0000","last_clean":"2024-01-15T18:05:00.740106+0000","last_became_active":"2024-01-15T18:04:59.295205+0000","last_became_peered":"2024-01-15T18:04:59.295205+0000","last_unstale":"2024-01-15T18:05:00.740106+0000","last_undegraded":"2024-01-15T18:05:00.740106+0000","last_fullsized":"2024-01-15T18:05:00.740106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283737+0000","last_change":"2024-01-15T18:04:58.296152+0000","last_active":"2024-01-15T18:05:00.283737+0000","last_peered":"2024-01-15T18:05:00.283737+0000","last_clean":"2024-01-15T18:05:00.283737+0000","last_became_active":"2024-01-15T18:04:58.295329+0000","last_became_peered":"2024-01-15T18:04:58.295329+0000","last_unstale":"2024-01-15T18:05:00.283737+0000","last_undegraded":"2024-01-15T18:05:00.283737+0000","last_fullsized":"2024-01-15T18:05:00.283737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285163+0000","last_change":"2024-01-15T18:04:58.291634+0000","last_active":"2024-01-15T18:05:00.285163+0000","last_peered":"2024-01-15T18:05:00.285163+0000","last_clean":"2024-01-15T18:05:00.285163+0000","last_became_active":"2024-01-15T18:04:58.290715+0000","last_became_peered":"2024-01-15T18:04:58.290715+0000","last_unstale":"2024-01-15T18:05:00.285163+0000","last_undegraded":"2024-01-15T18:05:00.285163+0000","last_fullsized":"2024-01-15T18:05:00.285163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285067+0000","last_change":"2024-01-15T18:04:59.302779+0000","last_active":"2024-01-15T18:05:00.285067+0000","last_peered":"2024-01-15T18:05:00.285067+0000","last_clean":"2024-01-15T18:05:00.285067+0000","last_became_active":"2024-01-15T18:04:59.302387+0000","last_became_peered":"2024-01-15T18:04:59.302387+0000","last_unstale":"2024-01-15T18:05:00.285067+0000","last_undegraded":"2024-01-15T18:05:00.285067+0000","last_fullsized":"2024-01-15T18:05:00.285067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353124+0000","last_change":"2024-01-15T18:04:58.285848+0000","last_active":"2024-01-15T18:05:00.353124+0000","last_peered":"2024-01-15T18:05:00.353124+0000","last_clean":"2024-01-15T18:05:00.353124+0000","last_became_active":"2024-01-15T18:04:58.283781+0000","last_became_peered":"2024-01-15T18:04:58.283781+0000","last_unstale":"2024-01-15T18:05:00.353124+0000","last_undegraded":"2024-01-15T18:05:00.353124+0000","last_fullsized":"2024-01-15T18:05:00.353124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740140+0000","last_change":"2024-01-15T18:04:59.306022+0000","last_active":"2024-01-15T18:05:00.740140+0000","last_peered":"2024-01-15T18:05:00.740140+0000","last_clean":"2024-01-15T18:05:00.740140+0000","last_became_active":"2024-01-15T18:04:59.305785+0000","last_became_peered":"2024-01-15T18:04:59.305785+0000","last_unstale":"2024-01-15T18:05:00.740140+0000","last_undegraded":"2024-01-15T18:05:00.740140+0000","last_fullsized":"2024-01-15T18:05:00.740140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283482+0000","last_change":"2024-01-15T18:04:58.292333+0000","last_active":"2024-01-15T18:05:00.283482+0000","last_peered":"2024-01-15T18:05:00.283482+0000","last_clean":"2024-01-15T18:05:00.283482+0000","last_became_active":"2024-01-15T18:04:58.291543+0000","last_became_peered":"2024-01-15T18:04:58.291543+0000","last_unstale":"2024-01-15T18:05:00.283482+0000","last_undegraded":"2024-01-15T18:05:00.283482+0000","last_fullsized":"2024-01-15T18:05:00.283482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353424+0000","last_change":"2024-01-15T18:04:59.313691+0000","last_active":"2024-01-15T18:05:00.353424+0000","last_peered":"2024-01-15T18:05:00.353424+0000","last_clean":"2024-01-15T18:05:00.353424+0000","last_became_active":"2024-01-15T18:04:59.313395+0000","last_became_peered":"2024-01-15T18:04:59.313395+0000","last_unstale":"2024-01-15T18:05:00.353424+0000","last_undegraded":"2024-01-15T18:05:00.353424+0000","last_fullsized":"2024-01-15T18:05:00.353424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353418+0000","last_change":"2024-01-15T18:04:58.288981+0000","last_active":"2024-01-15T18:05:00.353418+0000","last_peered":"2024-01-15T18:05:00.353418+0000","last_clean":"2024-01-15T18:05:00.353418+0000","last_became_active":"2024-01-15T18:04:58.288667+0000","last_became_peered":"2024-01-15T18:04:58.288667+0000","last_unstale":"2024-01-15T18:05:00.353418+0000","last_undegraded":"2024-01-15T18:05:00.353418+0000","last_fullsized":"2024-01-15T18:05:00.353418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740466+0000","last_change":"2024-01-15T18:04:59.311337+0000","last_active":"2024-01-15T18:05:00.740466+0000","last_peered":"2024-01-15T18:05:00.740466+0000","last_clean":"2024-01-15T18:05:00.740466+0000","last_became_active":"2024-01-15T18:04:59.311102+0000","last_became_peered":"2024-01-15T18:04:59.311102+0000","last_unstale":"2024-01-15T18:05:00.740466+0000","last_undegraded":"2024-01-15T18:05:00.740466+0000","last_fullsized":"2024-01-15T18:05:00.740466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740475+0000","last_change":"2024-01-15T18:04:58.295173+0000","last_active":"2024-01-15T18:05:00.740475+0000","last_peered":"2024-01-15T18:05:00.740475+0000","last_clean":"2024-01-15T18:05:00.740475+0000","last_became_active":"2024-01-15T18:04:58.294883+0000","last_became_peered":"2024-01-15T18:04:58.294883+0000","last_unstale":"2024-01-15T18:05:00.740475+0000","last_undegraded":"2024-01-15T18:05:00.740475+0000","last_fullsized":"2024-01-15T18:05:00.740475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284186+0000","last_change":"2024-01-15T18:04:59.311644+0000","last_active":"2024-01-15T18:05:00.284186+0000","last_peered":"2024-01-15T18:05:00.284186+0000","last_clean":"2024-01-15T18:05:00.284186+0000","last_became_active":"2024-01-15T18:04:59.311224+0000","last_became_peered":"2024-01-15T18:04:59.311224+0000","last_unstale":"2024-01-15T18:05:00.284186+0000","last_undegraded":"2024-01-15T18:05:00.284186+0000","last_fullsized":"2024-01-15T18:05:00.284186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283776+0000","last_change":"2024-01-15T18:04:58.293217+0000","last_active":"2024-01-15T18:05:00.283776+0000","last_peered":"2024-01-15T18:05:00.283776+0000","last_clean":"2024-01-15T18:05:00.283776+0000","last_became_active":"2024-01-15T18:04:58.292902+0000","last_became_peered":"2024-01-15T18:04:58.292902+0000","last_unstale":"2024-01-15T18:05:00.283776+0000","last_undegraded":"2024-01-15T18:05:00.283776+0000","last_fullsized":"2024-01-15T18:05:00.283776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539100+0000","last_change":"2024-01-15T18:04:59.313869+0000","last_active":"2024-01-15T18:05:00.539100+0000","last_peered":"2024-01-15T18:05:00.539100+0000","last_clean":"2024-01-15T18:05:00.539100+0000","last_became_active":"2024-01-15T18:04:59.313336+0000","last_became_peered":"2024-01-15T18:04:59.313336+0000","last_unstale":"2024-01-15T18:05:00.539100+0000","last_undegraded":"2024-01-15T18:05:00.539100+0000","last_fullsized":"2024-01-15T18:05:00.539100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740054+0000","last_change":"2024-01-15T18:04:58.294176+0000","last_active":"2024-01-15T18:05:00.740054+0000","last_peered":"2024-01-15T18:05:00.740054+0000","last_clean":"2024-01-15T18:05:00.740054+0000","last_became_active":"2024-01-15T18:04:58.292524+0000","last_became_peered":"2024-01-15T18:04:58.292524+0000","last_unstale":"2024-01-15T18:05:00.740054+0000","last_undegraded":"2024-01-15T18:05:00.740054+0000","last_fullsized":"2024-01-15T18:05:00.740054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740077+0000","last_change":"2024-01-15T18:04:59.300038+0000","last_active":"2024-01-15T18:05:00.740077+0000","last_peered":"2024-01-15T18:05:00.740077+0000","last_clean":"2024-01-15T18:05:00.740077+0000","last_became_active":"2024-01-15T18:04:59.299776+0000","last_became_peered":"2024-01-15T18:04:59.299776+0000","last_unstale":"2024-01-15T18:05:00.740077+0000","last_undegraded":"2024-01-15T18:05:00.740077+0000","last_fullsized":"2024-01-15T18:05:00.740077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353198+0000","last_change":"2024-01-15T18:04:58.297644+0000","last_active":"2024-01-15T18:05:00.353198+0000","last_peered":"2024-01-15T18:05:00.353198+0000","last_clean":"2024-01-15T18:05:00.353198+0000","last_became_active":"2024-01-15T18:04:58.296922+0000","last_became_peered":"2024-01-15T18:04:58.296922+0000","last_unstale":"2024-01-15T18:05:00.353198+0000","last_undegraded":"2024-01-15T18:05:00.353198+0000","last_fullsized":"2024-01-15T18:05:00.353198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539142+0000","last_change":"2024-01-15T18:04:59.312547+0000","last_active":"2024-01-15T18:05:00.539142+0000","last_peered":"2024-01-15T18:05:00.539142+0000","last_clean":"2024-01-15T18:05:00.539142+0000","last_became_active":"2024-01-15T18:04:59.312203+0000","last_became_peered":"2024-01-15T18:04:59.312203+0000","last_unstale":"2024-01-15T18:05:00.539142+0000","last_undegraded":"2024-01-15T18:05:00.539142+0000","last_fullsized":"2024-01-15T18:05:00.539142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283454+0000","last_change":"2024-01-15T18:04:58.297011+0000","last_active":"2024-01-15T18:05:00.283454+0000","last_peered":"2024-01-15T18:05:00.283454+0000","last_clean":"2024-01-15T18:05:00.283454+0000","last_became_active":"2024-01-15T18:04:58.296476+0000","last_became_peered":"2024-01-15T18:04:58.296476+0000","last_unstale":"2024-01-15T18:05:00.283454+0000","last_undegraded":"2024-01-15T18:05:00.283454+0000","last_fullsized":"2024-01-15T18:05:00.283454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:39.535650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283371+0000","last_change":"2024-01-15T18:04:59.294840+0000","last_active":"2024-01-15T18:05:00.283371+0000","last_peered":"2024-01-15T18:05:00.283371+0000","last_clean":"2024-01-15T18:05:00.283371+0000","last_became_active":"2024-01-15T18:04:59.294449+0000","last_became_peered":"2024-01-15T18:04:59.294449+0000","last_unstale":"2024-01-15T18:05:00.283371+0000","last_undegraded":"2024-01-15T18:05:00.283371+0000","last_fullsized":"2024-01-15T18:05:00.283371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739071+0000","last_change":"2024-01-15T18:04:58.285671+0000","last_active":"2024-01-15T18:05:00.739071+0000","last_peered":"2024-01-15T18:05:00.739071+0000","last_clean":"2024-01-15T18:05:00.739071+0000","last_became_active":"2024-01-15T18:04:58.285350+0000","last_became_peered":"2024-01-15T18:04:58.285350+0000","last_unstale":"2024-01-15T18:05:00.739071+0000","last_undegraded":"2024-01-15T18:05:00.739071+0000","last_fullsized":"2024-01-15T18:05:00.739071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538843+0000","last_change":"2024-01-15T18:04:59.315031+0000","last_active":"2024-01-15T18:05:00.538843+0000","last_peered":"2024-01-15T18:05:00.538843+0000","last_clean":"2024-01-15T18:05:00.538843+0000","last_became_active":"2024-01-15T18:04:59.311772+0000","last_became_peered":"2024-01-15T18:04:59.311772+0000","last_unstale":"2024-01-15T18:05:00.538843+0000","last_undegraded":"2024-01-15T18:05:00.538843+0000","last_fullsized":"2024-01-15T18:05:00.538843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284479+0000","last_change":"2024-01-15T18:04:58.290373+0000","last_active":"2024-01-15T18:05:00.284479+0000","last_peered":"2024-01-15T18:05:00.284479+0000","last_clean":"2024-01-15T18:05:00.284479+0000","last_became_active":"2024-01-15T18:04:58.290060+0000","last_became_peered":"2024-01-15T18:04:58.290060+0000","last_unstale":"2024-01-15T18:05:00.284479+0000","last_undegraded":"2024-01-15T18:05:00.284479+0000","last_fullsized":"2024-01-15T18:05:00.284479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539224+0000","last_change":"2024-01-15T18:04:59.315975+0000","last_active":"2024-01-15T18:05:00.539224+0000","last_peered":"2024-01-15T18:05:00.539224+0000","last_clean":"2024-01-15T18:05:00.539224+0000","last_became_active":"2024-01-15T18:04:59.315667+0000","last_became_peered":"2024-01-15T18:04:59.315667+0000","last_unstale":"2024-01-15T18:05:00.539224+0000","last_undegraded":"2024-01-15T18:05:00.539224+0000","last_fullsized":"2024-01-15T18:05:00.539224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739314+0000","last_change":"2024-01-15T18:04:58.297988+0000","last_active":"2024-01-15T18:05:00.739314+0000","last_peered":"2024-01-15T18:05:00.739314+0000","last_clean":"2024-01-15T18:05:00.739314+0000","last_became_active":"2024-01-15T18:04:58.296427+0000","last_became_peered":"2024-01-15T18:04:58.296427+0000","last_unstale":"2024-01-15T18:05:00.739314+0000","last_undegraded":"2024-01-15T18:05:00.739314+0000","last_fullsized":"2024-01-15T18:05:00.739314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539392+0000","last_change":"2024-01-15T18:04:59.312511+0000","last_active":"2024-01-15T18:05:00.539392+0000","last_peered":"2024-01-15T18:05:00.539392+0000","last_clean":"2024-01-15T18:05:00.539392+0000","last_became_active":"2024-01-15T18:04:59.312195+0000","last_became_peered":"2024-01-15T18:04:59.312195+0000","last_unstale":"2024-01-15T18:05:00.539392+0000","last_undegraded":"2024-01-15T18:05:00.539392+0000","last_fullsized":"2024-01-15T18:05:00.539392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:09:03.873184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740526+0000","last_change":"2024-01-15T18:04:58.289637+0000","last_active":"2024-01-15T18:05:00.740526+0000","last_peered":"2024-01-15T18:05:00.740526+0000","last_clean":"2024-01-15T18:05:00.740526+0000","last_became_active":"2024-01-15T18:04:58.289228+0000","last_became_peered":"2024-01-15T18:04:58.289228+0000","last_unstale":"2024-01-15T18:05:00.740526+0000","last_undegraded":"2024-01-15T18:05:00.740526+0000","last_fullsized":"2024-01-15T18:05:00.740526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353542+0000","last_change":"2024-01-15T18:04:59.309405+0000","last_active":"2024-01-15T18:05:00.353542+0000","last_peered":"2024-01-15T18:05:00.353542+0000","last_clean":"2024-01-15T18:05:00.353542+0000","last_became_active":"2024-01-15T18:04:59.309119+0000","last_became_peered":"2024-01-15T18:04:59.309119+0000","last_unstale":"2024-01-15T18:05:00.353542+0000","last_undegraded":"2024-01-15T18:05:00.353542+0000","last_fullsized":"2024-01-15T18:05:00.353542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739208+0000","last_change":"2024-01-15T18:04:58.294285+0000","last_active":"2024-01-15T18:05:00.739208+0000","last_peered":"2024-01-15T18:05:00.739208+0000","last_clean":"2024-01-15T18:05:00.739208+0000","last_became_active":"2024-01-15T18:04:58.293970+0000","last_became_peered":"2024-01-15T18:04:58.293970+0000","last_unstale":"2024-01-15T18:05:00.739208+0000","last_undegraded":"2024-01-15T18:05:00.739208+0000","last_fullsized":"2024-01-15T18:05:00.739208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.314051+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.313761+0000","last_became_peered":"2024-01-15T18:04:59.313761+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-15T18:04:59.290583+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:04:59.290583+0000","last_peered":"2024-01-15T18:04:59.290583+0000","last_clean":"2024-01-15T18:04:59.290583+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:04:59.290583+0000","last_undegraded":"2024-01-15T18:04:59.290583+0000","last_fullsized":"2024-01-15T18:04:59.290583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353134+0000","last_change":"2024-01-15T18:04:58.286159+0000","last_active":"2024-01-15T18:05:00.353134+0000","last_peered":"2024-01-15T18:05:00.353134+0000","last_clean":"2024-01-15T18:05:00.353134+0000","last_became_active":"2024-01-15T18:04:58.285819+0000","last_became_peered":"2024-01-15T18:04:58.285819+0000","last_unstale":"2024-01-15T18:05:00.353134+0000","last_undegraded":"2024-01-15T18:05:00.353134+0000","last_fullsized":"2024-01-15T18:05:00.353134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285340+0000","last_change":"2024-01-15T18:04:59.305454+0000","last_active":"2024-01-15T18:05:00.285340+0000","last_peered":"2024-01-15T18:05:00.285340+0000","last_clean":"2024-01-15T18:05:00.285340+0000","last_became_active":"2024-01-15T18:04:59.305160+0000","last_became_peered":"2024-01-15T18:04:59.305160+0000","last_unstale":"2024-01-15T18:05:00.285340+0000","last_undegraded":"2024-01-15T18:05:00.285340+0000","last_fullsized":"2024-01-15T18:05:00.285340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538748+0000","last_change":"2024-01-15T18:04:58.297164+0000","last_active":"2024-01-15T18:05:00.538748+0000","last_peered":"2024-01-15T18:05:00.538748+0000","last_clean":"2024-01-15T18:05:00.538748+0000","last_became_active":"2024-01-15T18:04:58.295969+0000","last_became_peered":"2024-01-15T18:04:58.295969+0000","last_unstale":"2024-01-15T18:05:00.538748+0000","last_undegraded":"2024-01-15T18:05:00.538748+0000","last_fullsized":"2024-01-15T18:05:00.538748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739741+0000","last_change":"2024-01-15T18:04:59.297889+0000","last_active":"2024-01-15T18:05:00.739741+0000","last_peered":"2024-01-15T18:05:00.739741+0000","last_clean":"2024-01-15T18:05:00.739741+0000","last_became_active":"2024-01-15T18:04:59.296874+0000","last_became_peered":"2024-01-15T18:04:59.296874+0000","last_unstale":"2024-01-15T18:05:00.739741+0000","last_undegraded":"2024-01-15T18:05:00.739741+0000","last_fullsized":"2024-01-15T18:05:00.739741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:52.200760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739266+0000","last_change":"2024-01-15T18:04:58.293459+0000","last_active":"2024-01-15T18:05:00.739266+0000","last_peered":"2024-01-15T18:05:00.739266+0000","last_clean":"2024-01-15T18:05:00.739266+0000","last_became_active":"2024-01-15T18:04:58.292396+0000","last_became_peered":"2024-01-15T18:04:58.292396+0000","last_unstale":"2024-01-15T18:05:00.739266+0000","last_undegraded":"2024-01-15T18:05:00.739266+0000","last_fullsized":"2024-01-15T18:05:00.739266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740002+0000","last_change":"2024-01-15T18:04:59.302244+0000","last_active":"2024-01-15T18:05:00.740002+0000","last_peered":"2024-01-15T18:05:00.740002+0000","last_clean":"2024-01-15T18:05:00.740002+0000","last_became_active":"2024-01-15T18:04:59.301981+0000","last_became_peered":"2024-01-15T18:04:59.301981+0000","last_unstale":"2024-01-15T18:05:00.740002+0000","last_undegraded":"2024-01-15T18:05:00.740002+0000","last_fullsized":"2024-01-15T18:05:00.740002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353265+0000","last_change":"2024-01-15T18:04:58.288895+0000","last_active":"2024-01-15T18:05:00.353265+0000","last_peered":"2024-01-15T18:05:00.353265+0000","last_clean":"2024-01-15T18:05:00.353265+0000","last_became_active":"2024-01-15T18:04:58.288385+0000","last_became_peered":"2024-01-15T18:04:58.288385+0000","last_unstale":"2024-01-15T18:05:00.353265+0000","last_undegraded":"2024-01-15T18:05:00.353265+0000","last_fullsized":"2024-01-15T18:05:00.353265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:10.169965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283326+0000","last_change":"2024-01-15T18:04:59.289150+0000","last_active":"2024-01-15T18:05:00.283326+0000","last_peered":"2024-01-15T18:05:00.283326+0000","last_clean":"2024-01-15T18:05:00.283326+0000","last_became_active":"2024-01-15T18:04:59.288315+0000","last_became_peered":"2024-01-15T18:04:59.288315+0000","last_unstale":"2024-01-15T18:05:00.283326+0000","last_undegraded":"2024-01-15T18:05:00.283326+0000","last_fullsized":"2024-01-15T18:05:00.283326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:14.706225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739871+0000","last_change":"2024-01-15T18:04:58.280841+0000","last_active":"2024-01-15T18:05:00.739871+0000","last_peered":"2024-01-15T18:05:00.739871+0000","last_clean":"2024-01-15T18:05:00.739871+0000","last_became_active":"2024-01-15T18:04:58.280590+0000","last_became_peered":"2024-01-15T18:04:58.280590+0000","last_unstale":"2024-01-15T18:05:00.739871+0000","last_undegraded":"2024-01-15T18:05:00.739871+0000","last_fullsized":"2024-01-15T18:05:00.739871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353597+0000","last_change":"2024-01-15T18:04:59.310649+0000","last_active":"2024-01-15T18:05:00.353597+0000","last_peered":"2024-01-15T18:05:00.353597+0000","last_clean":"2024-01-15T18:05:00.353597+0000","last_became_active":"2024-01-15T18:04:59.310266+0000","last_became_peered":"2024-01-15T18:04:59.310266+0000","last_unstale":"2024-01-15T18:05:00.353597+0000","last_undegraded":"2024-01-15T18:05:00.353597+0000","last_fullsized":"2024-01-15T18:05:00.353597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539301+0000","last_change":"2024-01-15T18:04:58.300881+0000","last_active":"2024-01-15T18:05:00.539301+0000","last_peered":"2024-01-15T18:05:00.539301+0000","last_clean":"2024-01-15T18:05:00.539301+0000","last_became_active":"2024-01-15T18:04:58.300322+0000","last_became_peered":"2024-01-15T18:04:58.300322+0000","last_unstale":"2024-01-15T18:05:00.539301+0000","last_undegraded":"2024-01-15T18:05:00.539301+0000","last_fullsized":"2024-01-15T18:05:00.539301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:47.163925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.299409+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.299156+0000","last_became_peered":"2024-01-15T18:04:59.299156+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740587+0000","last_change":"2024-01-15T18:04:58.283804+0000","last_active":"2024-01-15T18:05:00.740587+0000","last_peered":"2024-01-15T18:05:00.740587+0000","last_clean":"2024-01-15T18:05:00.740587+0000","last_became_active":"2024-01-15T18:04:58.283533+0000","last_became_peered":"2024-01-15T18:04:58.283533+0000","last_unstale":"2024-01-15T18:05:00.740587+0000","last_undegraded":"2024-01-15T18:05:00.740587+0000","last_fullsized":"2024-01-15T18:05:00.740587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285921+0000","last_change":"2024-01-15T18:04:59.316822+0000","last_active":"2024-01-15T18:05:00.285921+0000","last_peered":"2024-01-15T18:05:00.285921+0000","last_clean":"2024-01-15T18:05:00.285921+0000","last_became_active":"2024-01-15T18:04:59.316513+0000","last_became_peered":"2024-01-15T18:04:59.316513+0000","last_unstale":"2024-01-15T18:05:00.285921+0000","last_undegraded":"2024-01-15T18:05:00.285921+0000","last_fullsized":"2024-01-15T18:05:00.285921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283543+0000","last_change":"2024-01-15T18:04:58.292591+0000","last_active":"2024-01-15T18:05:00.283543+0000","last_peered":"2024-01-15T18:05:00.283543+0000","last_clean":"2024-01-15T18:05:00.283543+0000","last_became_active":"2024-01-15T18:04:58.291917+0000","last_became_peered":"2024-01-15T18:04:58.291917+0000","last_unstale":"2024-01-15T18:05:00.283543+0000","last_undegraded":"2024-01-15T18:05:00.283543+0000","last_fullsized":"2024-01-15T18:05:00.283543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740270+0000","last_change":"2024-01-15T18:04:59.300764+0000","last_active":"2024-01-15T18:05:00.740270+0000","last_peered":"2024-01-15T18:05:00.740270+0000","last_clean":"2024-01-15T18:05:00.740270+0000","last_became_active":"2024-01-15T18:04:59.299980+0000","last_became_peered":"2024-01-15T18:04:59.299980+0000","last_unstale":"2024-01-15T18:05:00.740270+0000","last_undegraded":"2024-01-15T18:05:00.740270+0000","last_fullsized":"2024-01-15T18:05:00.740270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538729+0000","last_change":"2024-01-15T18:04:58.301082+0000","last_active":"2024-01-15T18:05:00.538729+0000","last_peered":"2024-01-15T18:05:00.538729+0000","last_clean":"2024-01-15T18:05:00.538729+0000","last_became_active":"2024-01-15T18:04:58.300809+0000","last_became_peered":"2024-01-15T18:04:58.300809+0000","last_unstale":"2024-01-15T18:05:00.538729+0000","last_undegraded":"2024-01-15T18:05:00.538729+0000","last_fullsized":"2024-01-15T18:05:00.538729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283231+0000","last_change":"2024-01-15T18:04:59.297502+0000","last_active":"2024-01-15T18:05:00.283231+0000","last_peered":"2024-01-15T18:05:00.283231+0000","last_clean":"2024-01-15T18:05:00.283231+0000","last_became_active":"2024-01-15T18:04:59.297211+0000","last_became_peered":"2024-01-15T18:04:59.297211+0000","last_unstale":"2024-01-15T18:05:00.283231+0000","last_undegraded":"2024-01-15T18:05:00.283231+0000","last_fullsized":"2024-01-15T18:05:00.283231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:13.025413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740211+0000","last_change":"2024-01-15T18:04:58.296402+0000","last_active":"2024-01-15T18:05:00.740211+0000","last_peered":"2024-01-15T18:05:00.740211+0000","last_clean":"2024-01-15T18:05:00.740211+0000","last_became_active":"2024-01-15T18:04:58.296101+0000","last_became_peered":"2024-01-15T18:04:58.296101+0000","last_unstale":"2024-01-15T18:05:00.740211+0000","last_undegraded":"2024-01-15T18:05:00.740211+0000","last_fullsized":"2024-01-15T18:05:00.740211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285133+0000","last_change":"2024-01-15T18:04:59.298441+0000","last_active":"2024-01-15T18:05:00.285133+0000","last_peered":"2024-01-15T18:05:00.285133+0000","last_clean":"2024-01-15T18:05:00.285133+0000","last_became_active":"2024-01-15T18:04:59.298204+0000","last_became_peered":"2024-01-15T18:04:59.298204+0000","last_unstale":"2024-01-15T18:05:00.285133+0000","last_undegraded":"2024-01-15T18:05:00.285133+0000","last_fullsized":"2024-01-15T18:05:00.285133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740408+0000","last_change":"2024-01-15T18:04:58.297952+0000","last_active":"2024-01-15T18:05:00.740408+0000","last_peered":"2024-01-15T18:05:00.740408+0000","last_clean":"2024-01-15T18:05:00.740408+0000","last_became_active":"2024-01-15T18:04:58.297621+0000","last_became_peered":"2024-01-15T18:04:58.297621+0000","last_unstale":"2024-01-15T18:05:00.740408+0000","last_undegraded":"2024-01-15T18:05:00.740408+0000","last_fullsized":"2024-01-15T18:05:00.740408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285600+0000","last_change":"2024-01-15T18:04:59.304388+0000","last_active":"2024-01-15T18:05:00.285600+0000","last_peered":"2024-01-15T18:05:00.285600+0000","last_clean":"2024-01-15T18:05:00.285600+0000","last_became_active":"2024-01-15T18:04:59.303178+0000","last_became_peered":"2024-01-15T18:04:59.303178+0000","last_unstale":"2024-01-15T18:05:00.285600+0000","last_undegraded":"2024-01-15T18:05:00.285600+0000","last_fullsized":"2024-01-15T18:05:00.285600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539246+0000","last_change":"2024-01-15T18:04:58.300443+0000","last_active":"2024-01-15T18:05:00.539246+0000","last_peered":"2024-01-15T18:05:00.539246+0000","last_clean":"2024-01-15T18:05:00.539246+0000","last_became_active":"2024-01-15T18:04:58.300157+0000","last_became_peered":"2024-01-15T18:04:58.300157+0000","last_unstale":"2024-01-15T18:05:00.539246+0000","last_undegraded":"2024-01-15T18:05:00.539246+0000","last_fullsized":"2024-01-15T18:05:00.539246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283605+0000","last_change":"2024-01-15T18:04:59.295964+0000","last_active":"2024-01-15T18:05:00.283605+0000","last_peered":"2024-01-15T18:05:00.283605+0000","last_clean":"2024-01-15T18:05:00.283605+0000","last_became_active":"2024-01-15T18:04:59.295569+0000","last_became_peered":"2024-01-15T18:04:59.295569+0000","last_unstale":"2024-01-15T18:05:00.283605+0000","last_undegraded":"2024-01-15T18:05:00.283605+0000","last_fullsized":"2024-01-15T18:05:00.283605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.316899+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.316632+0000","last_became_peered":"2024-01-15T18:04:59.316632+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538942+0000","last_change":"2024-01-15T18:04:58.301177+0000","last_active":"2024-01-15T18:05:00.538942+0000","last_peered":"2024-01-15T18:05:00.538942+0000","last_clean":"2024-01-15T18:05:00.538942+0000","last_became_active":"2024-01-15T18:04:58.300869+0000","last_became_peered":"2024-01-15T18:04:58.300869+0000","last_unstale":"2024-01-15T18:05:00.538942+0000","last_undegraded":"2024-01-15T18:05:00.538942+0000","last_fullsized":"2024-01-15T18:05:00.538942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353471+0000","last_change":"2024-01-15T18:04:59.311012+0000","last_active":"2024-01-15T18:05:00.353471+0000","last_peered":"2024-01-15T18:05:00.353471+0000","last_clean":"2024-01-15T18:05:00.353471+0000","last_became_active":"2024-01-15T18:04:59.310673+0000","last_became_peered":"2024-01-15T18:04:59.310673+0000","last_unstale":"2024-01-15T18:05:00.353471+0000","last_undegraded":"2024-01-15T18:05:00.353471+0000","last_fullsized":"2024-01-15T18:05:00.353471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539209+0000","last_change":"2024-01-15T18:04:58.303349+0000","last_active":"2024-01-15T18:05:00.539209+0000","last_peered":"2024-01-15T18:05:00.539209+0000","last_clean":"2024-01-15T18:05:00.539209+0000","last_became_active":"2024-01-15T18:04:58.303074+0000","last_became_peered":"2024-01-15T18:04:58.303074+0000","last_unstale":"2024-01-15T18:05:00.539209+0000","last_undegraded":"2024-01-15T18:05:00.539209+0000","last_fullsized":"2024-01-15T18:05:00.539209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740412+0000","last_change":"2024-01-15T18:04:59.311306+0000","last_active":"2024-01-15T18:05:00.740412+0000","last_peered":"2024-01-15T18:05:00.740412+0000","last_clean":"2024-01-15T18:05:00.740412+0000","last_became_active":"2024-01-15T18:04:59.311031+0000","last_became_peered":"2024-01-15T18:04:59.311031+0000","last_unstale":"2024-01-15T18:05:00.740412+0000","last_undegraded":"2024-01-15T18:05:00.740412+0000","last_fullsized":"2024-01-15T18:05:00.740412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285716+0000","last_change":"2024-01-15T18:04:58.299606+0000","last_active":"2024-01-15T18:05:00.285716+0000","last_peered":"2024-01-15T18:05:00.285716+0000","last_clean":"2024-01-15T18:05:00.285716+0000","last_became_active":"2024-01-15T18:04:58.299353+0000","last_became_peered":"2024-01-15T18:04:58.299353+0000","last_unstale":"2024-01-15T18:05:00.285716+0000","last_undegraded":"2024-01-15T18:05:00.285716+0000","last_fullsized":"2024-01-15T18:05:00.285716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285464+0000","last_change":"2024-01-15T18:04:59.304019+0000","last_active":"2024-01-15T18:05:00.285464+0000","last_peered":"2024-01-15T18:05:00.285464+0000","last_clean":"2024-01-15T18:05:00.285464+0000","last_became_active":"2024-01-15T18:04:59.303662+0000","last_became_peered":"2024-01-15T18:04:59.303662+0000","last_unstale":"2024-01-15T18:05:00.285464+0000","last_undegraded":"2024-01-15T18:05:00.285464+0000","last_fullsized":"2024-01-15T18:05:00.285464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740536+0000","last_change":"2024-01-15T18:04:58.303996+0000","last_active":"2024-01-15T18:05:00.740536+0000","last_peered":"2024-01-15T18:05:00.740536+0000","last_clean":"2024-01-15T18:05:00.740536+0000","last_became_active":"2024-01-15T18:04:58.303687+0000","last_became_peered":"2024-01-15T18:04:58.303687+0000","last_unstale":"2024-01-15T18:05:00.740536+0000","last_undegraded":"2024-01-15T18:05:00.740536+0000","last_fullsized":"2024-01-15T18:05:00.740536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:05:03.939 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:05:03.939 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-15T18:05:03.940 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-15T18:05:03.940 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-15T18:05:03.940 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-15T18:05:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.105+0000 7f6a49000640 1 Processor -- start 2024-01-15T18:05:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.105+0000 7f6a49000640 1 -- start start 2024-01-15T18:05:04.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a49000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a441607c0 0x7f6a4406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a49000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6a440cfb30 0x7f6a4406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a49000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a440cffd0 con 0x7f6a4406c230 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a49000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a440d1ce0 con 0x7f6a441607c0 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a49000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a440d02b0 con 0x7f6a440cfb30 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a441607c0 0x7f6a4406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a441607c0 0x7f6a4406b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56328/0 (socket says 172.21.15.143:56328) 2024-01-15T18:05:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 -- 172.21.15.143:0/1847901195 learned_addr learned my addr 172.21.15.143:0/1847901195 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:04.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41400640 1 --2- 172.21.15.143:0/1847901195 >> v2:172.21.15.204:3300/0 conn(0x7f6a440cfb30 0x7f6a4406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 -- 172.21.15.143:0/1847901195 >> v2:172.21.15.204:3300/0 conn(0x7f6a440cfb30 msgr2=0x7f6a4406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:04.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 --2- 172.21.15.143:0/1847901195 >> v2:172.21.15.204:3300/0 conn(0x7f6a440cfb30 0x7f6a4406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 -- 172.21.15.143:0/1847901195 >> v1:172.21.15.204:6789/0 conn(0x7f6a4406c230 legacy=0x7f6a440cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:04.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41e00640 1 -- 172.21.15.143:0/1847901195 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a440cb470 con 0x7f6a441607c0 2024-01-15T18:05:04.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.109+0000 7f6a41400640 1 --2- 172.21.15.143:0/1847901195 >> v2:172.21.15.204:3300/0 conn(0x7f6a440cfb30 0x7f6a4406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:04.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a41e00640 1 --2- 172.21.15.143:0/1847901195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4406b7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a4409a980 tx=0x7f6a3401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6daa1d93c19d19 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a40a00640 1 -- 172.21.15.143:0/1847901195 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a340a89b0 con 0x7f6a441607c0 2024-01-15T18:05:04.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a40a00640 1 -- 172.21.15.143:0/1847901195 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a340a8b10 con 0x7f6a441607c0 2024-01-15T18:05:04.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a40a00640 1 -- 172.21.15.143:0/1847901195 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a340a8ea0 con 0x7f6a441607c0 2024-01-15T18:05:04.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 -- 172.21.15.143:0/1847901195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 msgr2=0x7f6a4406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 --2- 172.21.15.143:0/1847901195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4406b7b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6a4409a980 tx=0x7f6a3401ecb0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 -- 172.21.15.143:0/1847901195 shutdown_connections 2024-01-15T18:05:04.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 --2- 172.21.15.143:0/1847901195 >> v2:172.21.15.204:3300/0 conn(0x7f6a440cfb30 0x7f6a4406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 --2- 172.21.15.143:0/1847901195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4406b7b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 -- 172.21.15.143:0/1847901195 >> 172.21.15.143:0/1847901195 conn(0x7f6a440d0e20 msgr2=0x7f6a440d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 -- 172.21.15.143:0/1847901195 shutdown_connections 2024-01-15T18:05:04.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 -- 172.21.15.143:0/1847901195 wait complete. 2024-01-15T18:05:04.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 Processor -- start 2024-01-15T18:05:04.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.113+0000 7f6a49000640 1 -- start start 2024-01-15T18:05:04.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406c230 0x7f6a440b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a440cfb30 0x7f6a4415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440c8ae0 con 0x7f6a441607c0 2024-01-15T18:05:04.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440c9e80 con 0x7f6a4406c230 2024-01-15T18:05:04.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440c7ef0 con 0x7f6a440cfb30 2024-01-15T18:05:04.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4415c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56336/0 (socket says 172.21.15.143:56336) 2024-01-15T18:05:04.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 -- 172.21.15.143:0/2789430928 learned_addr learned my addr 172.21.15.143:0/2789430928 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:04.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a41e00640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406c230 0x7f6a440b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a41400640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a440cfb30 0x7f6a4415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 -- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a440cfb30 msgr2=0x7f6a4415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a440cfb30 0x7f6a4415c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 -- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406c230 msgr2=0x7f6a440b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406c230 0x7f6a440b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 -- 172.21.15.143:0/2789430928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340094d0 con 0x7f6a441607c0 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a42800640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4415c680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c7b0 tx=0x7f6a3c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c004b70 con 0x7f6a441607c0 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c004cd0 con 0x7f6a441607c0 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00aa90 con 0x7f6a441607c0 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a440c8140 con 0x7f6a441607c0 2024-01-15T18:05:04.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a440c8700 con 0x7f6a441607c0 2024-01-15T18:05:04.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.117+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4406d4b0 con 0x7f6a441607c0 2024-01-15T18:05:04.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.121+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f6a3c00abf0 con 0x7f6a441607c0 2024-01-15T18:05:04.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.121+0000 7f6a3a000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6a1c0668f0 0x7f6a1c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.121+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a3c084460 con 0x7f6a441607c0 2024-01-15T18:05:04.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.121+0000 7f6a41e00640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6a1c0668f0 0x7f6a1c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.121+0000 7f6a41e00640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6a1c0668f0 0x7f6a1c068db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a340b2000 tx=0x7f6a34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.125+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6a3c051740 con 0x7f6a441607c0 2024-01-15T18:05:04.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.309+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6a440c8c20 con 0x7f6a441607c0 2024-01-15T18:05:04.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.309+0000 7f6a3a000640 1 -- 172.21.15.143:0/2789430928 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164963 (secure 0 0 0) 0x7f6a440c8c20 con 0x7f6a441607c0 2024-01-15T18:05:04.318 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6a1c0668f0 msgr2=0x7f6a1c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6a1c0668f0 0x7f6a1c068db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a340b2000 tx=0x7f6a34009040 comp rx=0 tx=0).stop 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 msgr2=0x7f6a4415c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4415c680 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c7b0 tx=0x7f6a3c00cc70 comp rx=0 tx=0).stop 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 shutdown_connections 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6a1c0668f0 0x7f6a1c068db0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a441607c0 0x7f6a4415c680 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a440cfb30 0x7f6a4415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 --2- 172.21.15.143:0/2789430928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a4406c230 0x7f6a440b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 >> 172.21.15.143:0/2789430928 conn(0x7f6a440d0e20 msgr2=0x7f6a440c6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 shutdown_connections 2024-01-15T18:05:04.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.313+0000 7f6a49000640 1 -- 172.21.15.143:0/2789430928 wait complete. 2024-01-15T18:05:04.339 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":3137793172},{"type":"v1","addr":"172.21.15.143:6833","nonce":3137793172}]},"active_addr":"172.21.15.143:6833/3137793172","active_change":"2024-01-15T18:04:38.608714+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1856222453}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":322294544}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2451546371}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":229841759}]}]} 2024-01-15T18:05:04.342 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-15T18:05:04.343 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-15T18:05:04.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:04.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 Processor -- start 2024-01-15T18:05:04.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 -- start start 2024-01-15T18:05:04.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1240ae450 0x7fe12406a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12406ae30 con 0x7fe12415de90 2024-01-15T18:05:04.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240ae830 con 0x7fe1240ae450 2024-01-15T18:05:04.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe12aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240ad900 con 0x7fe12406a9d0 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe123400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe123e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1240ae450 0x7fe12406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe128a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe12415de90 0x7fe124071140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59144/0 (socket says 172.21.15.143:59144) 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe128a00640 1 -- 172.21.15.143:0/294079296 learned_addr learned my addr 172.21.15.143:0/294079296 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe123400640 1 -- 172.21.15.143:0/294079296 >> v2:172.21.15.204:3300/0 conn(0x7fe1240ae450 msgr2=0x7fe12406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe123400640 1 --2- 172.21.15.143:0/294079296 >> v2:172.21.15.204:3300/0 conn(0x7fe1240ae450 0x7fe12406a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe123400640 1 -- 172.21.15.143:0/294079296 >> v1:172.21.15.204:6789/0 conn(0x7fe12415de90 legacy=0x7fe124071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.473+0000 7fe123400640 1 -- 172.21.15.143:0/294079296 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12415e6c0 con 0x7fe12406a9d0 2024-01-15T18:05:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe123400640 1 --2- 172.21.15.143:0/294079296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12415d950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0097c0 tx=0x7fe10c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2554d3e936d401 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe122a00640 1 -- 172.21.15.143:0/294079296 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0ab600 con 0x7fe12406a9d0 2024-01-15T18:05:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe122a00640 1 -- 172.21.15.143:0/294079296 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c0a7050 con 0x7fe12406a9d0 2024-01-15T18:05:04.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe122a00640 1 -- 172.21.15.143:0/294079296 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c0ab980 con 0x7fe12406a9d0 2024-01-15T18:05:04.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 -- 172.21.15.143:0/294079296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 msgr2=0x7fe12415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 --2- 172.21.15.143:0/294079296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12415d950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0097c0 tx=0x7fe10c01c450 comp rx=0 tx=0).stop 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 -- 172.21.15.143:0/294079296 shutdown_connections 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 --2- 172.21.15.143:0/294079296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12415d950 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 --2- 172.21.15.143:0/294079296 >> v2:172.21.15.204:3300/0 conn(0x7fe1240ae450 0x7fe12406a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 -- 172.21.15.143:0/294079296 >> 172.21.15.143:0/294079296 conn(0x7fe1240cfd20 msgr2=0x7fe1240d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 -- 172.21.15.143:0/294079296 shutdown_connections 2024-01-15T18:05:04.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.477+0000 7fe12aa00640 1 -- 172.21.15.143:0/294079296 wait complete. 2024-01-15T18:05:04.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 Processor -- start 2024-01-15T18:05:04.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 -- start start 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1240ae450 0x7fe124152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 0x7fe1240ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415f150 con 0x7fe12415de90 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415f430 con 0x7fe1240ae450 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe12aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415f710 con 0x7fe12406a9d0 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 0x7fe1240ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 0x7fe1240ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56338/0 (socket says 172.21.15.143:56338) 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 -- 172.21.15.143:0/965000176 learned_addr learned my addr 172.21.15.143:0/965000176 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe123400640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1240ae450 0x7fe124152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe123e00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 -- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 msgr2=0x7fe12414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12414e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 -- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1240ae450 msgr2=0x7fe124152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1240ae450 0x7fe124152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 -- 172.21.15.143:0/965000176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009420 con 0x7fe12415de90 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe128a00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 0x7fe1240ce8e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00ca80 tx=0x7fe11c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.481+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00ac30 con 0x7fe12415de90 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c00ad90 con 0x7fe12415de90 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12415f9f0 con 0x7fe12415de90 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c012600 con 0x7fe12415de90 2024-01-15T18:05:04.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12414e740 con 0x7fe12415de90 2024-01-15T18:05:04.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe124067500 con 0x7fe12415de90 2024-01-15T18:05:04.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe11c007070 con 0x7fe12415de90 2024-01-15T18:05:04.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe11be00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0fc0668f0 0x7fe0fc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe11c0842d0 con 0x7fe12415de90 2024-01-15T18:05:04.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe123e00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0fc0668f0 0x7fe0fc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.485+0000 7fe123e00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0fc0668f0 0x7fe0fc068db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe114009b60 tx=0x7fe114009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.489+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe11c051520 con 0x7fe12415de90 2024-01-15T18:05:04.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.641+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe12415fea0 con 0x7fe12415de90 2024-01-15T18:05:04.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe11be00640 1 -- 172.21.15.143:0/965000176 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe11c054b40 con 0x7fe12415de90 2024-01-15T18:05:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:04.649 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0fc0668f0 msgr2=0x7fe0fc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0fc0668f0 0x7fe0fc068db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe114009b60 tx=0x7fe114009190 comp rx=0 tx=0).stop 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 msgr2=0x7fe1240ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 0x7fe1240ce8e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00ca80 tx=0x7fe11c00cf40 comp rx=0 tx=0).stop 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 shutdown_connections 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0fc0668f0 0x7fe0fc068db0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe12415de90 0x7fe1240ce8e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1240ae450 0x7fe124152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 --2- 172.21.15.143:0/965000176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe12406a9d0 0x7fe12414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 >> 172.21.15.143:0/965000176 conn(0x7fe1240cfd20 msgr2=0x7fe1240ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 shutdown_connections 2024-01-15T18:05:04.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.645+0000 7fe12aa00640 1 -- 172.21.15.143:0/965000176 wait complete. 2024-01-15T18:05:04.671 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-15T18:05:04.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:04.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.801+0000 7f8e27400640 1 Processor -- start 2024-01-15T18:05:04.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.801+0000 7f8e27400640 1 -- start start 2024-01-15T18:05:04.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e27400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8e20143780 0x7f8e200cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e27400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e200ceb50 con 0x7f8e200722c0 2024-01-15T18:05:04.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e27400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2015c110 con 0x7f8e200ce520 2024-01-15T18:05:04.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e27400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2015b7e0 con 0x7f8e20143780 2024-01-15T18:05:04.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e25400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8e200722c0 0x7f8e2015c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34086/0 (socket says 172.21.15.143:34086) 2024-01-15T18:05:04.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e25400640 1 -- 172.21.15.143:0/1143062394 learned_addr learned my addr 172.21.15.143:0/1143062394 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:04.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e24a00640 1 --2- 172.21.15.143:0/1143062394 >> v2:172.21.15.204:3300/0 conn(0x7f8e20143780 0x7f8e200cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576715049 0 0) 0x7f8e180014a0 con 0x7f8e200722c0 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e00003530 con 0x7f8e200722c0 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983332572 0 0) 0x7f8e2015c110 con 0x7f8e200ce520 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e180014a0 con 0x7f8e200ce520 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e24a00640 1 -- 172.21.15.143:0/1143062394 >> v1:172.21.15.204:6789/0 conn(0x7f8e200ce520 legacy=0x7f8e20071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e24a00640 1 -- 172.21.15.143:0/1143062394 >> v1:172.21.15.143:6789/0 conn(0x7f8e200722c0 legacy=0x7f8e2015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.805+0000 7f8e24a00640 1 -- 172.21.15.143:0/1143062394 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2015bac0 con 0x7f8e20143780 2024-01-15T18:05:04.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e24a00640 1 --2- 172.21.15.143:0/1143062394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e20143780 0x7f8e200cdfe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e100097c0 tx=0x7f8e1001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae16ec900db2dea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100a8050 con 0x7f8e20143780 2024-01-15T18:05:04.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e100ac5f0 con 0x7f8e20143780 2024-01-15T18:05:04.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e1f400640 1 -- 172.21.15.143:0/1143062394 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100ac980 con 0x7f8e20143780 2024-01-15T18:05:04.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 -- 172.21.15.143:0/1143062394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e20143780 msgr2=0x7f8e200cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 --2- 172.21.15.143:0/1143062394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e20143780 0x7f8e200cdfe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e100097c0 tx=0x7f8e1001c480 comp rx=0 tx=0).stop 2024-01-15T18:05:04.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 -- 172.21.15.143:0/1143062394 shutdown_connections 2024-01-15T18:05:04.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 --2- 172.21.15.143:0/1143062394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e20143780 0x7f8e200cdfe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 -- 172.21.15.143:0/1143062394 >> 172.21.15.143:0/1143062394 conn(0x7f8e2006c3f0 msgr2=0x7f8e2006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 -- 172.21.15.143:0/1143062394 shutdown_connections 2024-01-15T18:05:04.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 -- 172.21.15.143:0/1143062394 wait complete. 2024-01-15T18:05:04.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 Processor -- start 2024-01-15T18:05:04.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.809+0000 7f8e27400640 1 -- start start 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e200722c0 0x7f8e20159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 0x7f8e200b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e20143780 0x7f8e200b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20135d90 con 0x7f8e200ce520 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20135fa0 con 0x7f8e200722c0 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20136280 con 0x7f8e20143780 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 0x7f8e200b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 0x7f8e200b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56346/0 (socket says 172.21.15.143:56346) 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 -- 172.21.15.143:0/2664125371 learned_addr learned my addr 172.21.15.143:0/2664125371 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e24a00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e200722c0 0x7f8e20159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e25400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e20143780 0x7f8e200b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 -- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e20143780 msgr2=0x7f8e200b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e20143780 0x7f8e200b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 -- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e200722c0 msgr2=0x7f8e20159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e200722c0 0x7f8e20159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 -- 172.21.15.143:0/2664125371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009420 con 0x7f8e200ce520 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1fe00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 0x7f8e200b2e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c490 tx=0x7f8e0800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080052d0 con 0x7f8e200ce520 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08005430 con 0x7f8e200ce520 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e20136560 con 0x7f8e200ce520 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08012870 con 0x7f8e200ce520 2024-01-15T18:05:04.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e200b38c0 con 0x7f8e200ce520 2024-01-15T18:05:04.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e20067500 con 0x7f8e200ce520 2024-01-15T18:05:04.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.813+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f8e080129d0 con 0x7f8e200ce520 2024-01-15T18:05:04.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.817+0000 7f8e1cc00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8df80668f0 0x7f8df8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:04.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.817+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e08083f20 con 0x7f8e200ce520 2024-01-15T18:05:04.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.817+0000 7f8e24a00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8df80668f0 0x7f8df8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:04.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.817+0000 7f8e24a00640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8df80668f0 0x7f8df8068db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ae000 tx=0x7f8e100a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.821+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8e08051150 con 0x7f8e200ce520 2024-01-15T18:05:04.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.969+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e2013a4e0 con 0x7f8e200ce520 2024-01-15T18:05:04.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e1cc00640 1 -- 172.21.15.143:0/2664125371 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f8e08054770 con 0x7f8e200ce520 2024-01-15T18:05:04.977 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:04.977 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8df80668f0 msgr2=0x7f8df8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8df80668f0 0x7f8df8068db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e100ae000 tx=0x7f8e100a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 msgr2=0x7f8e200b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 0x7f8e200b2e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c490 tx=0x7f8e0800c950 comp rx=0 tx=0).stop 2024-01-15T18:05:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 shutdown_connections 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8df80668f0 0x7f8df8068db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e20143780 0x7f8e200b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e200ce520 0x7f8e200b2e40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 --2- 172.21.15.143:0/2664125371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e200722c0 0x7f8e20159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 >> 172.21.15.143:0/2664125371 conn(0x7f8e2006c3f0 msgr2=0x7f8e20139930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 shutdown_connections 2024-01-15T18:05:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:04.973+0000 7f8e27400640 1 -- 172.21.15.143:0/2664125371 wait complete. 2024-01-15T18:05:04.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-15T18:05:04.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-15T18:05:04.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-15T18:05:05.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-15T18:05:05.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-15T18:05:05.001 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-15T18:05:05.001 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-15T18:05:05.001 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-15T18:05:05.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.221+0000 7fb2a6a00640 1 Processor -- start 2024-01-15T18:05:05.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.221+0000 7fb2a6a00640 1 -- start start 2024-01-15T18:05:05.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.221+0000 7fb2a6a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2a006db20 0x7fb2a00b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.221+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a015a440 0x7fb2a00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb2a6a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00cc790 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb2a6a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a00b4ae0 con 0x7fb2a006db20 2024-01-15T18:05:05.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb2a6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00b4800 con 0x7fb2a015a440 2024-01-15T18:05:05.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb2a4a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fb2a00cd9e0 0x7fb2a00b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59162/0 (socket says 172.21.15.143:59162) 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb2a4a00640 1 -- 172.21.15.143:0/563326140 learned_addr learned my addr 172.21.15.143:0/563326140 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.221+0000 7fb29fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2a006db20 0x7fb2a00b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a015a440 0x7fb2a00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29ea00640 1 -- 172.21.15.143:0/563326140 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910970306 0 0) 0x7fb2a00cc790 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29ea00640 1 -- 172.21.15.143:0/563326140 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb280003530 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29fe00640 1 -- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a015a440 msgr2=0x7fb2a00cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29fe00640 1 --2- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a015a440 0x7fb2a00cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29fe00640 1 -- 172.21.15.143:0/563326140 >> v1:172.21.15.204:6789/0 conn(0x7fb2a00cd9e0 legacy=0x7fb2a00b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.225+0000 7fb29fe00640 1 -- 172.21.15.143:0/563326140 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00cc9c0 con 0x7fb2a006db20 2024-01-15T18:05:05.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.229+0000 7fb29fe00640 1 --2- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a006db20 0x7fb2a00b3e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2900097c0 tx=0x7fb29001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5139395e42e74abb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.229+0000 7fb29ea00640 1 -- 172.21.15.143:0/563326140 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900ab670 con 0x7fb2a006db20 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.229+0000 7fb29ea00640 1 -- 172.21.15.143:0/563326140 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2900ab7d0 con 0x7fb2a006db20 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.229+0000 7fb29ea00640 1 -- 172.21.15.143:0/563326140 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900abba0 con 0x7fb2a006db20 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.229+0000 7fb2a6a00640 1 -- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a006db20 msgr2=0x7fb2a00b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.229+0000 7fb2a6a00640 1 --2- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a006db20 0x7fb2a00b3e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2900097c0 tx=0x7fb29001c470 comp rx=0 tx=0).stop 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.233+0000 7fb2a6a00640 1 -- 172.21.15.143:0/563326140 shutdown_connections 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.233+0000 7fb2a6a00640 1 --2- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a015a440 0x7fb2a00cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.233+0000 7fb2a6a00640 1 --2- 172.21.15.143:0/563326140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a006db20 0x7fb2a00b3e60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.233+0000 7fb2a6a00640 1 -- 172.21.15.143:0/563326140 >> 172.21.15.143:0/563326140 conn(0x7fb2a00aea10 msgr2=0x7fb2a00aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.233+0000 7fb2a6a00640 1 -- 172.21.15.143:0/563326140 shutdown_connections 2024-01-15T18:05:05.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.233+0000 7fb2a6a00640 1 -- 172.21.15.143:0/563326140 wait complete. 2024-01-15T18:05:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 Processor -- start 2024-01-15T18:05:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 -- start start 2024-01-15T18:05:05.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a006db20 0x7fb2a014d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a00cd9e0 0x7fb2a01fa280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a015a440 0x7fb2a01fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015c340 con 0x7fb2a015a440 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015b730 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb2a6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015ba10 con 0x7fb2a006db20 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a00cd9e0 0x7fb2a01fa280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a00cd9e0 0x7fb2a01fa280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57878/0 (socket says 172.21.15.143:57878) 2024-01-15T18:05:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 -- 172.21.15.143:0/2108726860 learned_addr learned my addr 172.21.15.143:0/2108726860 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29fe00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a006db20 0x7fb2a014d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 -- 172.21.15.143:0/2108726860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a006db20 msgr2=0x7fb2a014d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2a006db20 0x7fb2a014d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 -- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a015a440 msgr2=0x7fb2a01fa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2a015a440 0x7fb2a01fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.237+0000 7fb29f400640 1 -- 172.21.15.143:0/2108726860 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290009420 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.241+0000 7fb29f400640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a00cd9e0 0x7fb2a01fa280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb29800efc0 tx=0x7fb29800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.241+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298014480 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.241+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29800f040 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.241+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298014800 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.241+0000 7fb2a6a00640 1 -- 172.21.15.143:0/2108726860 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a015bcf0 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.241+0000 7fb2a6a00640 1 -- 172.21.15.143:0/2108726860 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a014d950 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2a00b43a0 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb298014960 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7f272b400640 1 Processor -- start 2024-01-15T18:05:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7f272b400640 1 -- start start 2024-01-15T18:05:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb0f9c00640 1 Processor -- start 2024-01-15T18:05:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb0f9c00640 1 -- start start 2024-01-15T18:05:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb297e00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb278066940 0x7fb278068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb298085320 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb297e00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7fb27806eb60 0x7fb278070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 --> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb278071630 con 0x7fb27806eb60 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fef95600640 1 Processor -- start 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f272b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27241439e0 0x7f27240adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f272b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f272406d080 0x7f27240b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f272b400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2724143dc0 con 0x7f27240ae060 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f272b400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27240ad700 con 0x7f27241439e0 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f272b400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27240aca80 con 0x7f272406d080 2024-01-15T18:05:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.245+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb298087a30 con 0x7fb2a00cd9e0 2024-01-15T18:05:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fef95600640 1 -- start start 2024-01-15T18:05:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f9c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0f41439e0 0x7fb0f40adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f406c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f9c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f4143dc0 con 0x7fb0f406ceb0 2024-01-15T18:05:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f9c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f40ad700 con 0x7fb0f41439e0 2024-01-15T18:05:05.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f9c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f40ae4a0 con 0x7fb0f40ae060 2024-01-15T18:05:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb29fe00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb278066940 0x7fb278068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2729400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f272406d080 0x7f27240b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0f41439e0 0x7fb0f40adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0f41439e0 0x7fb0f40adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57890/0 (socket says 172.21.15.143:57890) 2024-01-15T18:05:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 -- 172.21.15.143:0/2467836469 learned_addr learned my addr 172.21.15.143:0/2467836469 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2729400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f272406d080 0x7f27240b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57884/0 (socket says 172.21.15.143:57884) 2024-01-15T18:05:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2723e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f27240ae060 0x7f272406c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59168/0 (socket says 172.21.15.143:59168) 2024-01-15T18:05:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2723e00640 1 -- 172.21.15.143:0/4032448760 learned_addr learned my addr 172.21.15.143:0/4032448760 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:05.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb2a4a00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7fb27806eb60 0x7fb278070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2729400640 1 -- 172.21.15.143:0/4032448760 >> v2:172.21.15.143:3300/0 conn(0x7f27241439e0 msgr2=0x7f27240adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2729400640 1 --2- 172.21.15.143:0/4032448760 >> v2:172.21.15.143:3300/0 conn(0x7f27241439e0 0x7f27240adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2729400640 1 -- 172.21.15.143:0/4032448760 >> v1:172.21.15.204:6789/0 conn(0x7f27240ae060 legacy=0x7f272406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f2729400640 1 -- 172.21.15.143:0/4032448760 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27240acd60 con 0x7f272406d080 2024-01-15T18:05:05.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f6c28200640 1 Processor -- start 2024-01-15T18:05:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb29fe00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb278066940 0x7fb278068e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2900af000 tx=0x7fb290008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 -- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 msgr2=0x7fb0f406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f5494800640 1 Processor -- start 2024-01-15T18:05:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 --2- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f406c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 -- 172.21.15.143:0/2467836469 >> v1:172.21.15.143:6789/0 conn(0x7fb0f406ceb0 legacy=0x7fb0f40b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb2a4a00640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7fb27806eb60 0x7fb278070f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f5494800640 1 -- start start 2024-01-15T18:05:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fef95600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fef90071140 0x7fef900ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fef95600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef90071520 0x7fef9015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7fb0f2a00640 1 -- 172.21.15.143:0/2467836469 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40aca80 con 0x7fb0f41439e0 2024-01-15T18:05:05.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fef90071140 0x7fef900ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fef90071140 0x7fef900ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57900/0 (socket says 172.21.15.143:57900) 2024-01-15T18:05:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fb0f2000640 1 --2- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f406c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 -- 172.21.15.143:0/4135531603 learned_addr learned my addr 172.21.15.143:0/4135531603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef900ce810 con 0x7fef9015b110 2024-01-15T18:05:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef900cc220 con 0x7fef90071520 2024-01-15T18:05:05.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9015a4a0 con 0x7fef90071140 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.249+0000 7f6c28200640 1 -- start start 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c28200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c20159170 0x7f6c2006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c28200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2006cf30 con 0x7f6c200ce310 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c28200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c200cb2f0 con 0x7f6c20070a80 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c28200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c200cb0e0 con 0x7f6c20159170 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5494800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5494800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f548c070fd0 0x7f548c162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5494800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0cfc20 con 0x7f548c0cf7c0 2024-01-15T18:05:05.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5494800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c162080 con 0x7f548c070fd0 2024-01-15T18:05:05.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5494800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c161d70 con 0x7f548c14b1e0 2024-01-15T18:05:05.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== osd.1 v2:172.21.15.143:6800/2952112662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb278071630 con 0x7fb27806eb60 2024-01-15T18:05:05.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8e000640 1 --2- 172.21.15.143:0/4135531603 >> v2:172.21.15.143:3300/0 conn(0x7fef90071520 0x7fef9015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60298/0 (socket says 172.21.15.143:60298) 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 -- 172.21.15.143:0/1151377703 learned_addr learned my addr 172.21.15.143:0/1151377703 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5492800640 1 --2- 172.21.15.143:0/1151377703 >> v2:172.21.15.143:3300/0 conn(0x7f548c070fd0 0x7f548c162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 -- 172.21.15.143:0/1151377703 >> v2:172.21.15.143:3300/0 conn(0x7f548c070fd0 msgr2=0x7f548c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 --2- 172.21.15.143:0/1151377703 >> v2:172.21.15.143:3300/0 conn(0x7f548c070fd0 0x7f548c162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5490a00640 1 -- 172.21.15.143:0/1151377703 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456361907 0 0) 0x7f548c0cfc20 con 0x7f548c0cf7c0 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c25800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c20159170 0x7f6c2006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c26200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f6c20070a80 0x7f6c2015cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59176/0 (socket says 172.21.15.143:59176) 2024-01-15T18:05:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f6c26200640 1 -- 172.21.15.143:0/2047800355 learned_addr learned my addr 172.21.15.143:0/2047800355 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f2729400640 1 --2- 172.21.15.143:0/4032448760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272406d080 0x7f27240b2a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27240b1920 tx=0x7f271c01c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fb70f9af81b137 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 -- 172.21.15.143:0/1151377703 >> v1:172.21.15.204:6789/0 conn(0x7f548c0cf7c0 legacy=0x7f548c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7f5491e00640 1 -- 172.21.15.143:0/1151377703 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c14b5c0 con 0x7f548c14b1e0 2024-01-15T18:05:05.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f2723400640 1 -- 172.21.15.143:0/4032448760 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0a5800 con 0x7f272406d080 2024-01-15T18:05:05.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f6c25800640 1 -- 172.21.15.143:0/2047800355 >> v1:172.21.15.204:6789/0 conn(0x7f6c20070a80 legacy=0x7f6c2015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f6c25800640 1 -- 172.21.15.143:0/2047800355 >> v1:172.21.15.143:6789/0 conn(0x7f6c200ce310 legacy=0x7f6c20070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f2723400640 1 -- 172.21.15.143:0/4032448760 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c0a5960 con 0x7f272406d080 2024-01-15T18:05:05.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f2723400640 1 -- 172.21.15.143:0/4032448760 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0a5cf0 con 0x7f272406d080 2024-01-15T18:05:05.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 -- 172.21.15.143:0/4032448760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272406d080 msgr2=0x7f27240b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 --2- 172.21.15.143:0/4032448760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272406d080 0x7f27240b2a00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f27240b1920 tx=0x7f271c01c7a0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7fb0f2a00640 1 --2- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40adb20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009e80 tx=0x7fb0ec01c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb9512c4a1b55b9d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 -- 172.21.15.143:0/4032448760 shutdown_connections 2024-01-15T18:05:05.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 --2- 172.21.15.143:0/4032448760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f272406d080 0x7f27240b2a00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 --2- 172.21.15.143:0/4032448760 >> v2:172.21.15.143:3300/0 conn(0x7f27241439e0 0x7f27240adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 -- 172.21.15.143:0/4032448760 >> 172.21.15.143:0/4032448760 conn(0x7f27240b0bf0 msgr2=0x7f27240b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7fb0f1600640 1 -- 172.21.15.143:0/2467836469 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0a5720 con 0x7fb0f41439e0 2024-01-15T18:05:05.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 -- 172.21.15.143:0/4032448760 shutdown_connections 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7fb0f1600640 1 -- 172.21.15.143:0/2467836469 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ec0a5880 con 0x7fb0f41439e0 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7fb0f1600640 1 -- 172.21.15.143:0/2467836469 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec0a5c10 con 0x7fb0f41439e0 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 -- 172.21.15.143:0/4135531603 >> v2:172.21.15.143:3300/0 conn(0x7fef90071520 msgr2=0x7fef9015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 --2- 172.21.15.143:0/4135531603 >> v2:172.21.15.143:3300/0 conn(0x7fef90071520 0x7fef9015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 -- 172.21.15.143:0/4135531603 >> v1:172.21.15.143:6789/0 conn(0x7fef9015b110 legacy=0x7fef9015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.253+0000 7fef8ea00640 1 -- 172.21.15.143:0/4135531603 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef900cb980 con 0x7fef90071140 2024-01-15T18:05:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f6c25800640 1 -- 172.21.15.143:0/2047800355 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c200cb5d0 con 0x7f6c20159170 2024-01-15T18:05:05.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7fb0f9c00640 1 -- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 msgr2=0x7fb0f40adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 --2- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40adb20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009e80 tx=0x7fb0ec01c9b0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7fef8ea00640 1 --2- 172.21.15.143:0/4135531603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef900ce2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef900721d0 tx=0x7fef8001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43af5c924d10c43a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.257+0000 7f272b400640 1 -- 172.21.15.143:0/4032448760 wait complete. 2024-01-15T18:05:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 -- 172.21.15.143:0/2467836469 shutdown_connections 2024-01-15T18:05:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 --2- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f406c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 --2- 172.21.15.143:0/2467836469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40adb20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef8d600640 1 -- 172.21.15.143:0/4135531603 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800aa430 con 0x7fef90071140 2024-01-15T18:05:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 -- 172.21.15.143:0/2467836469 >> 172.21.15.143:0/2467836469 conn(0x7fb0f40b0bf0 msgr2=0x7fb0f40b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef8d600640 1 -- 172.21.15.143:0/4135531603 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef800aa590 con 0x7fef90071140 2024-01-15T18:05:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef8d600640 1 -- 172.21.15.143:0/4135531603 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800aa430 con 0x7fef90071140 2024-01-15T18:05:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f5491e00640 1 --2- 172.21.15.143:0/1151377703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c0cf280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f547c009870 tx=0x7f547c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79cfe9519a802ce6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 -- 172.21.15.143:0/2467836469 shutdown_connections 2024-01-15T18:05:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f272b400640 1 Processor -- start 2024-01-15T18:05:05.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f272b400640 1 -- start start 2024-01-15T18:05:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fb0f9c00640 1 -- 172.21.15.143:0/2467836469 wait complete. 2024-01-15T18:05:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f6c25800640 1 --2- 172.21.15.143:0/2047800355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c20159170 0x7f6c2006d680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800c2e0 tx=0x7f6c1801d820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340a18383550a283 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f3d69a00640 1 Processor -- start 2024-01-15T18:05:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f5490a00640 1 -- 172.21.15.143:0/1151377703 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0aa670 con 0x7f548c14b1e0 2024-01-15T18:05:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f5490a00640 1 -- 172.21.15.143:0/1151377703 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c0aa7d0 con 0x7f548c14b1e0 2024-01-15T18:05:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f5490a00640 1 -- 172.21.15.143:0/1151377703 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0aab60 con 0x7f548c14b1e0 2024-01-15T18:05:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f6c1fe00640 1 -- 172.21.15.143:0/2047800355 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c200cebe0 con 0x7f6c20159170 2024-01-15T18:05:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f6c1fe00640 1 -- 172.21.15.143:0/2047800355 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c180a8050 con 0x7f6c20159170 2024-01-15T18:05:05.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f6c1fe00640 1 -- 172.21.15.143:0/2047800355 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1800a4f0 con 0x7f6c20159170 2024-01-15T18:05:05.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 msgr2=0x7fef900ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef95600640 1 --2- 172.21.15.143:0/4135531603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef900ce2d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fef900721d0 tx=0x7fef8001c4f0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f3d69a00640 1 -- start start 2024-01-15T18:05:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 shutdown_connections 2024-01-15T18:05:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f5494800640 1 -- 172.21.15.143:0/1151377703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 msgr2=0x7f548c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f5494800640 1 --2- 172.21.15.143:0/1151377703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c0cf280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f547c009870 tx=0x7f547c01c510 comp rx=0 tx=0).stop 2024-01-15T18:05:05.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef95600640 1 --2- 172.21.15.143:0/4135531603 >> v2:172.21.15.143:3300/0 conn(0x7fef90071520 0x7fef9015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f6c28200640 1 -- 172.21.15.143:0/2047800355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c20159170 msgr2=0x7f6c2006d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f6c28200640 1 --2- 172.21.15.143:0/2047800355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c20159170 0x7f6c2006d680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c1800c2e0 tx=0x7f6c1801d820 comp rx=0 tx=0).stop 2024-01-15T18:05:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef95600640 1 --2- 172.21.15.143:0/4135531603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef900ce2d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 >> 172.21.15.143:0/4135531603 conn(0x7fef900cfd20 msgr2=0x7fef900d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f272b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27240ae060 0x7f27240abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.261+0000 7f272b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27241439e0 0x7f27240ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f272b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27240cd090 0x7f27240cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f272b400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27240cda80 con 0x7f27241439e0 2024-01-15T18:05:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f272b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27240cc810 con 0x7f27240ae060 2024-01-15T18:05:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f272b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272415c7e0 con 0x7f27240cd090 2024-01-15T18:05:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 shutdown_connections 2024-01-15T18:05:05.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f6c28200640 1 -- 172.21.15.143:0/2047800355 shutdown_connections 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f6c28200640 1 --2- 172.21.15.143:0/2047800355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c20159170 0x7f6c2006d680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f6c28200640 1 -- 172.21.15.143:0/2047800355 >> 172.21.15.143:0/2047800355 conn(0x7f6c200b0bf0 msgr2=0x7f6c200b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 -- 172.21.15.143:0/1151377703 shutdown_connections 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 --2- 172.21.15.143:0/1151377703 >> v2:172.21.15.143:3300/0 conn(0x7f548c070fd0 0x7f548c162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 --2- 172.21.15.143:0/1151377703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c0cf280 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 -- 172.21.15.143:0/1151377703 >> 172.21.15.143:0/1151377703 conn(0x7f548c06a890 msgr2=0x7f548c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2723e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27241439e0 0x7f27240ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2728a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27240ae060 0x7f27240abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d69a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d640cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fef95600640 1 -- 172.21.15.143:0/4135531603 wait complete. 2024-01-15T18:05:05.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d69a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640cfc20 con 0x7f3d640cf7c0 2024-01-15T18:05:05.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d69a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d64162080 con 0x7f3d64070fd0 2024-01-15T18:05:05.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d69a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d64161d70 con 0x7f3d6414b1e0 2024-01-15T18:05:05.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2728a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27240ae060 0x7f27240abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57916/0 (socket says 172.21.15.143:57916) 2024-01-15T18:05:05.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 -- 172.21.15.143:0/1151377703 shutdown_connections 2024-01-15T18:05:05.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2723e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27241439e0 0x7f27240ac350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56372/0 (socket says 172.21.15.143:56372) 2024-01-15T18:05:05.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2723e00640 1 -- 172.21.15.143:0/479111487 learned_addr learned my addr 172.21.15.143:0/479111487 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 -- 172.21.15.143:0/1151377703 wait complete. 2024-01-15T18:05:05.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27240cd090 0x7f27240cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d640cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fb0f9c00640 1 Processor -- start 2024-01-15T18:05:05.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d640cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60322/0 (socket says 172.21.15.143:60322) 2024-01-15T18:05:05.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d62a00640 1 -- 172.21.15.143:0/2510730361 learned_addr learned my addr 172.21.15.143:0/2510730361 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f6c28200640 1 -- 172.21.15.143:0/2047800355 shutdown_connections 2024-01-15T18:05:05.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fb0f9c00640 1 -- start start 2024-01-15T18:05:05.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d62a00640 1 -- 172.21.15.143:0/2510730361 >> v1:172.21.15.204:6789/0 conn(0x7f3d64070fd0 legacy=0x7f3d64162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d62a00640 1 -- 172.21.15.143:0/2510730361 >> v1:172.21.15.143:6789/0 conn(0x7f3d640cf7c0 legacy=0x7f3d640708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f3d62a00640 1 -- 172.21.15.143:0/2510730361 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6414b5c0 con 0x7f3d6414b1e0 2024-01-15T18:05:05.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 -- 172.21.15.143:0/479111487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27240ae060 msgr2=0x7f27240abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27240ae060 0x7f27240abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 -- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27241439e0 msgr2=0x7f27240ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27241439e0 0x7f27240ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 -- 172.21.15.143:0/479111487 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c009a70 con 0x7f27240cd090 2024-01-15T18:05:05.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f2729400640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27240cd090 0x7f27240cd540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ac000 tx=0x7f271c019540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fb0f9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0f406ceb0 0x7fb0f415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fb0f9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f40aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f5494800640 1 Processor -- start 2024-01-15T18:05:05.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f9c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f415f1a0 con 0x7fb0f406ceb0 2024-01-15T18:05:05.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 -- start start 2024-01-15T18:05:05.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7f6c28200640 1 -- 172.21.15.143:0/2047800355 wait complete. 2024-01-15T18:05:05.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f6c28200640 1 Processor -- start 2024-01-15T18:05:05.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57930/0 (socket says 172.21.15.143:57930) 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 -- 172.21.15.143:0/1513811948 learned_addr learned my addr 172.21.15.143:0/1513811948 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4143050 con 0x7fb0f41439e0 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f2000640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f40aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f4141fb0 con 0x7fb0f40ae060 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f2a00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0f406ceb0 0x7fb0f415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0adc00 con 0x7f27240cd090 2024-01-15T18:05:05.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c0add60 con 0x7f27240cd090 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0b7610 con 0x7f27240cd090 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27240cbc00 con 0x7f27240cd090 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.265+0000 7fef95600640 1 Processor -- start 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 -- start start 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef9014dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef90071520 0x7fef900caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef9015b110 0x7fef90149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900ca1b0 con 0x7fef90071520 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef900b45e0 con 0x7fef90071140 2024-01-15T18:05:05.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef95600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef900caf50 con 0x7fef9015b110 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f548c070fd0 0x7f548c16b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f548c0cf7c0 0x7f548c15f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c160d30 con 0x7f548c0cf7c0 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c160490 con 0x7f548c070fd0 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5494800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c160770 con 0x7f548c14b1e0 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f548c070fd0 0x7f548c16b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f548c070fd0 0x7f548c16b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57946/0 (socket says 172.21.15.143:57946) 2024-01-15T18:05:05.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 -- 172.21.15.143:0/3050764861 learned_addr learned my addr 172.21.15.143:0/3050764861 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f6c28200640 1 -- start start 2024-01-15T18:05:05.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 -- 172.21.15.143:0/1513811948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 msgr2=0x7fb0f40aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0f40ae060 0x7fb0f40aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 -- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0f406ceb0 msgr2=0x7fb0f415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0f406ceb0 0x7fb0f415c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fb0f3400640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec009b30 con 0x7fb0f41439e0 2024-01-15T18:05:05.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5492800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef9014dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef9014dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57956/0 (socket says 172.21.15.143:57956) 2024-01-15T18:05:05.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 -- 172.21.15.143:0/93028965 learned_addr learned my addr 172.21.15.143:0/93028965 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27240cc1c0 con 0x7f27240cd090 2024-01-15T18:05:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 -- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 msgr2=0x7f548c149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 -- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f548c0cf7c0 msgr2=0x7f548c15f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f548c0cf7c0 0x7f548c15f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f37c6e00640 1 Processor -- start 2024-01-15T18:05:05.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 -- 172.21.15.143:0/93028965 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef9015b110 msgr2=0x7fef90149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef9015b110 0x7fef90149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 -- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef90071520 msgr2=0x7fef900caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7fef8ea00640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef90071520 0x7fef900caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5491e00640 1 -- 172.21.15.143:0/3050764861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0094d0 con 0x7f548c070fd0 2024-01-15T18:05:05.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f5492800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f548c14b1e0 0x7f548c149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c28200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c20070a80 0x7f6c202039a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c28200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c200ce310 0x7f6c200ace90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c28200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c20159170 0x7f6c200ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c28200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200ad910 con 0x7f6c20159170 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c28200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c200a9d00 con 0x7f6c200ce310 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c28200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c20158480 con 0x7f6c20070a80 2024-01-15T18:05:05.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef8ea00640 1 -- 172.21.15.143:0/93028965 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800094d0 con 0x7fef90071140 2024-01-15T18:05:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0f3400640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40af420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00b410 tx=0x7fb0dc00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f271c0b4050 con 0x7f27240cd090 2024-01-15T18:05:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f5491e00640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f548c070fd0 0x7f548c16b820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f547c0ac040 tx=0x7f547c01ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f3d62a00640 1 --2- 172.21.15.143:0/2510730361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d640cf280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009820 tx=0x7f3d4c01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48665ac0ce080e01 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c26200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c20159170 0x7f6c200ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c200ce310 0x7f6c200ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c200ce310 0x7f6c200ace90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57962/0 (socket says 172.21.15.143:57962) 2024-01-15T18:05:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 -- 172.21.15.143:0/1239821848 learned_addr learned my addr 172.21.15.143:0/1239821848 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c25800640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c20070a80 0x7f6c202039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef8ea00640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef9014dcc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef800ac040 tx=0x7fef8001ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011830 con 0x7fb0f41439e0 2024-01-15T18:05:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc011990 con 0x7fb0f41439e0 2024-01-15T18:05:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011d40 con 0x7fb0f41439e0 2024-01-15T18:05:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0bd5f0 con 0x7f548c070fd0 2024-01-15T18:05:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c0bd770 con 0x7f548c070fd0 2024-01-15T18:05:05.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0bdb20 con 0x7f548c070fd0 2024-01-15T18:05:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f3d61600640 1 -- 172.21.15.143:0/2510730361 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c004230 con 0x7f3d6414b1e0 2024-01-15T18:05:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f3d61600640 1 -- 172.21.15.143:0/2510730361 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0a7050 con 0x7f3d6414b1e0 2024-01-15T18:05:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f3d61600640 1 -- 172.21.15.143:0/2510730361 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c003620 con 0x7f3d6414b1e0 2024-01-15T18:05:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800aa430 con 0x7fef90071140 2024-01-15T18:05:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef800b3600 con 0x7fef90071140 2024-01-15T18:05:05.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef800b3970 con 0x7fef90071140 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef900c3a70 con 0x7fef90071140 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef900c9ed0 con 0x7fef90071140 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f4142290 con 0x7fb0f41439e0 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f4142850 con 0x7fb0f41439e0 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27240ad320 con 0x7f27240cd090 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2720c00640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f26fc0669c0 0x7f26fc068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f271c11f500 con 0x7f27240cd090 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2720c00640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7f26fc06eb60 0x7f26fc070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 --> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26fc071630 con 0x7f26fc06eb60 2024-01-15T18:05:05.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2728a00640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f26fc0669c0 0x7f26fc068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c160a50 con 0x7f548c070fd0 2024-01-15T18:05:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c16bd60 con 0x7f548c070fd0 2024-01-15T18:05:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.269+0000 7f37c6e00640 1 -- start start 2024-01-15T18:05:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37c6e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37c0153ad0 0x7f37c015a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37c6e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37c006d5f0 0x7f37c006da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 -- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c20070a80 msgr2=0x7f6c202039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c20070a80 0x7f6c202039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 -- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c20159170 msgr2=0x7f6c200ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c20159170 0x7f6c200ad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 -- 172.21.15.143:0/1239821848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1800c050 con 0x7f6c200ce310 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c25800640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c20070a80 0x7f6c202039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f6c24e00640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c200ce310 0x7f6c200ace90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c7b0 tx=0x7f6c0800cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f3d69a00640 1 -- 172.21.15.143:0/2510730361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 msgr2=0x7f3d640cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f3d69a00640 1 --2- 172.21.15.143:0/2510730361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d640cf280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009820 tx=0x7f3d4c01c330 comp rx=0 tx=0).stop 2024-01-15T18:05:05.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f2723e00640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7f26fc06eb60 0x7f26fc070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f548c06b240 con 0x7f548c070fd0 2024-01-15T18:05:05.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08005270 con 0x7f6c200ce310 2024-01-15T18:05:05.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f3d69a00640 1 -- 172.21.15.143:0/2510730361 shutdown_connections 2024-01-15T18:05:05.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f3d69a00640 1 --2- 172.21.15.143:0/2510730361 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d640cf280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f3d69a00640 1 -- 172.21.15.143:0/2510730361 >> 172.21.15.143:0/2510730361 conn(0x7f3d6406a890 msgr2=0x7f3d6406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef90157de0 con 0x7fef90071140 2024-01-15T18:05:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f2723e00640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7f26fc06eb60 0x7f26fc070f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c080053f0 con 0x7f6c200ce310 2024-01-15T18:05:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0800a880 con 0x7f6c200ce310 2024-01-15T18:05:05.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2015aa40 con 0x7f6c200ce310 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f2728a00640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f26fc0669c0 0x7f26fc068e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2714006fd0 tx=0x7f2714008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37bfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37c0153ad0 0x7f37c015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37c6e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c0153eb0 con 0x7f37c00b1730 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37c6e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c00b2230 con 0x7f37c006d5f0 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37c6e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c006d160 con 0x7f37c0153ad0 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37bfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37c0153ad0 0x7f37c015a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57968/0 (socket says 172.21.15.143:57968) 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7f37bfe00640 1 -- 172.21.15.143:0/1701903121 learned_addr learned my addr 172.21.15.143:0/1701903121 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f37bf400640 1 --2- 172.21.15.143:0/1701903121 >> v2:172.21.15.143:3300/0 conn(0x7f37c006d5f0 0x7f37c006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f37bfe00640 1 -- 172.21.15.143:0/1701903121 >> v2:172.21.15.143:3300/0 conn(0x7f37c006d5f0 msgr2=0x7f37c006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f37bfe00640 1 --2- 172.21.15.143:0/1701903121 >> v2:172.21.15.143:3300/0 conn(0x7f37c006d5f0 0x7f37c006da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f37bfe00640 1 -- 172.21.15.143:0/1701903121 >> v1:172.21.15.143:6789/0 conn(0x7f37c00b1730 legacy=0x7f37c015cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f37bfe00640 1 -- 172.21.15.143:0/1701903121 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c006c5c0 con 0x7f37c0153ad0 2024-01-15T18:05:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2015b000 con 0x7f6c200ce310 2024-01-15T18:05:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.273+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0f40b3110 con 0x7fb0f41439e0 2024-01-15T18:05:05.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f271c0ec800 con 0x7f27240cd090 2024-01-15T18:05:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fef800b3ad0 con 0x7fef90071140 2024-01-15T18:05:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== osd.3 v2:172.21.15.143:6822/254985013 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f26fc071630 con 0x7f26fc06eb60 2024-01-15T18:05:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f3d69a00640 1 -- 172.21.15.143:0/2510730361 shutdown_connections 2024-01-15T18:05:05.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb0dc01a400 con 0x7fb0f41439e0 2024-01-15T18:05:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f3d69a00640 1 -- 172.21.15.143:0/2510730361 wait complete. 2024-01-15T18:05:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7fb0eaa00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0cc0669c0 0x7fb0cc068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0dc0844a0 con 0x7fb0f41439e0 2024-01-15T18:05:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c20159760 con 0x7f6c200ce310 2024-01-15T18:05:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0eaa00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb0cc06ebe0 0x7fb0cc071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0cc0716b0 con 0x7fb0cc06ebe0 2024-01-15T18:05:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0dc086ac0 con 0x7fb0f41439e0 2024-01-15T18:05:05.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f6c0800a9e0 con 0x7f6c200ce310 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef86a00640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef6806ae00 0x7fef6806d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef8011ee50 con 0x7fef90071140 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef86a00640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fef68073020 0x7fef68075440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 --> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef68075af0 con 0x7fef68073020 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef80121470 con 0x7fef90071140 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.277+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f547c0b3600 con 0x7f548c070fd0 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f548a000640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5464066940 0x7f5464068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef8e000640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef6806ae00 0x7fef6806d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0f2a00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0cc0669c0 0x7fb0cc068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef8f400640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fef68073020 0x7fef68075440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef8e000640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef6806ae00 0x7fef6806d2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef78006e90 tx=0x7fef78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0f2a00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0cc0669c0 0x7fb0cc068e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0ae000 tx=0x7fb0ec0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f547c0f11f0 con 0x7f548c070fd0 2024-01-15T18:05:05.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f548a000640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f546406f840 0x7f5464071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 --> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f54640723b0 con 0x7f546406f840 2024-01-15T18:05:05.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 Processor -- start 2024-01-15T18:05:05.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 -- start start 2024-01-15T18:05:05.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f547c0f08c0 con 0x7f548c070fd0 2024-01-15T18:05:05.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fb0f2000640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb0cc06ebe0 0x7fb0cc071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7fef8f400640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fef68073020 0x7fef68075440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c1d600640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bf80669c0 0x7f6bf8068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f5491400640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5464066940 0x7f5464068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c08083ad0 con 0x7f6c200ce310 2024-01-15T18:05:05.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c1d600640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f6bf806ebe0 0x7f6bf8071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 --> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6bf80716b0 con 0x7f6bf806ebe0 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c080882a0 con 0x7f6c200ce310 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d64070fd0 0x7f3d640a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d640cf7c0 0x7f3d641f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d64161150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641f8030 con 0x7f3d640cf7c0 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640a8ab0 con 0x7f3d64070fd0 2024-01-15T18:05:05.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f3d69a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d641f82d0 con 0x7f3d6414b1e0 2024-01-15T18:05:05.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f6c25800640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bf80669c0 0x7f6bf8068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f6c26200640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f6bf806ebe0 0x7f6bf8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d64070fd0 0x7f3d640a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d64070fd0 0x7f3d640a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57974/0 (socket says 172.21.15.143:57974) 2024-01-15T18:05:05.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 -- 172.21.15.143:0/742676726 learned_addr learned my addr 172.21.15.143:0/742676726 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d63400640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d64161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62000640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d640cf7c0 0x7f3d641f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.281+0000 7f5492800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f546406f840 0x7f5464071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f5492800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f546406f840 0x7f5464071ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== osd.0 v2:172.21.15.143:6802/2045350344 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fef68075af0 con 0x7fef68073020 2024-01-15T18:05:05.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7fb0f2000640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb0cc06ebe0 0x7fb0cc071000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 -- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 msgr2=0x7f3d64161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d6414b1e0 0x7f3d64161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 -- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d640cf7c0 msgr2=0x7f3d641f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f6c25800640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bf80669c0 0x7f6bf8068e80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c180ae000 tx=0x7f6c180a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d640cf7c0 0x7f3d641f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 -- 172.21.15.143:0/742676726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c0094d0 con 0x7f3d64070fd0 2024-01-15T18:05:05.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62000640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d640cf7c0 0x7f3d641f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:05.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f5491400640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5464066940 0x7f5464068e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5474007fd0 tx=0x7f5474007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f6c26200640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f6bf806ebe0 0x7f6bf8071000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d62a00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d64070fd0 0x7f3d640a84d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c019f60 tx=0x7f3d4c0190d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== osd.4 v2:172.21.15.204:6816/1717170274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f54640723b0 con 0x7f546406f840 2024-01-15T18:05:05.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== osd.2 v2:172.21.15.143:6816/761026363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb0cc0716b0 con 0x7fb0cc06ebe0 2024-01-15T18:05:05.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== osd.6 v2:172.21.15.204:6808/1677254362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6bf80716b0 con 0x7f6bf806ebe0 2024-01-15T18:05:05.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0ab640 con 0x7f3d64070fd0 2024-01-15T18:05:05.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c0ab7a0 con 0x7f3d64070fd0 2024-01-15T18:05:05.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0abb50 con 0x7f3d64070fd0 2024-01-15T18:05:05.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d641f85b0 con 0x7f3d64070fd0 2024-01-15T18:05:05.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.285+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d641f8b70 con 0x7f3d64070fd0 2024-01-15T18:05:05.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37bfe00640 1 --2- 172.21.15.143:0/1701903121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0153ad0 0x7f37c015a410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009870 tx=0x7f37a802d4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a52e49286ed6680 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d64071b40 con 0x7f3d64070fd0 2024-01-15T18:05:05.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37bea00640 1 -- 172.21.15.143:0/1701903121 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80bc600 con 0x7f37c0153ad0 2024-01-15T18:05:05.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37bea00640 1 -- 172.21.15.143:0/1701903121 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a80b8050 con 0x7f37c0153ad0 2024-01-15T18:05:05.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37bea00640 1 -- 172.21.15.143:0/1701903121 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80bc980 con 0x7f37c0153ad0 2024-01-15T18:05:05.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3d4c0b2050 con 0x7f3d64070fd0 2024-01-15T18:05:05.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 --> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2a015c1a0 con 0x7fb27806eb60 2024-01-15T18:05:05.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37c6e00640 1 -- 172.21.15.143:0/1701903121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0153ad0 msgr2=0x7f37c015a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7fb297e00640 1 -- 172.21.15.143:0/2108726860 <== osd.1 v2:172.21.15.143:6800/2952112662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2880058a0 con 0x7fb27806eb60 2024-01-15T18:05:05.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37c6e00640 1 --2- 172.21.15.143:0/1701903121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0153ad0 0x7f37c015a410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009870 tx=0x7f37a802d4d0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37c6e00640 1 -- 172.21.15.143:0/1701903121 shutdown_connections 2024-01-15T18:05:05.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37c6e00640 1 --2- 172.21.15.143:0/1701903121 >> v2:172.21.15.143:3300/0 conn(0x7f37c006d5f0 0x7f37c006da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37c6e00640 1 --2- 172.21.15.143:0/1701903121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c0153ad0 0x7f37c015a410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f37c6e00640 1 -- 172.21.15.143:0/1701903121 >> 172.21.15.143:0/1701903121 conn(0x7f37c00b0c10 msgr2=0x7f37c00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.289+0000 7f3d5aa00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d3c066940 0x7f3d3c068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7fb27806eb60 msgr2=0x7fb278070f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7fb295600640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7fb27806eb60 0x7fb278070f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d4c11e7a0 con 0x7f3d64070fd0 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d5aa00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f3d3c06eb60 0x7f3d3c070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 --> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d3c071630 con 0x7f3d3c06eb60 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d4c120d80 con 0x7f3d64070fd0 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d62000640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d3c066940 0x7f3d3c068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d63400640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f3d3c06eb60 0x7f3d3c070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f37c6e00640 1 -- 172.21.15.143:0/1701903121 shutdown_connections 2024-01-15T18:05:05.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f37c6e00640 1 -- 172.21.15.143:0/1701903121 wait complete. 2024-01-15T18:05:05.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d63400640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f3d3c06eb60 0x7f3d3c070f80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f37c6e00640 1 Processor -- start 2024-01-15T18:05:05.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d62000640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d3c066940 0x7f3d3c068e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d640b1620 tx=0x7f3d54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f37c6e00640 1 -- start start 2024-01-15T18:05:05.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.293+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== osd.7 v2:172.21.15.204:6824/2462757405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3d3c071630 con 0x7f3d3c06eb60 2024-01-15T18:05:05.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb278066940 msgr2=0x7fb278068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb295600640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb278066940 0x7fb278068e00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2900af000 tx=0x7fb290008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a00cd9e0 msgr2=0x7fb2a01fa280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb295600640 1 --2- 172.21.15.143:0/2108726860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2a00cd9e0 0x7fb2a01fa280 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb29800efc0 tx=0x7fb29800c490 comp rx=0 tx=0).stop 2024-01-15T18:05:05.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb2a4a00640 1 -- 172.21.15.143:0/2108726860 reap_dead start 2024-01-15T18:05:05.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c006d5f0 0x7f37c015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c00b1730 0x7f37c014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c6e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37c0153ad0 0x7f37c0151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37bfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c006d5f0 0x7f37c015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37c0153ad0 0x7f37c0151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37bf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c00b1730 0x7f37c014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37bf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c00b1730 0x7f37c014c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60366/0 (socket says 172.21.15.143:60366) 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37c0153ad0 0x7f37c0151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56440/0 (socket says 172.21.15.143:56440) 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c6e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015b960 con 0x7f37c0153ad0 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c009af90 con 0x7f37c006d5f0 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37c6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c009b250 con 0x7f37c00b1730 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7f37bf400640 1 -- 172.21.15.143:0/1618897272 learned_addr learned my addr 172.21.15.143:0/1618897272 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 shutdown_connections 2024-01-15T18:05:05.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.297+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 >> 172.21.15.143:0/2108726860 conn(0x7fb2a00aea10 msgr2=0x7fb2a0166030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 shutdown_connections 2024-01-15T18:05:05.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7fb295600640 1 -- 172.21.15.143:0/2108726860 wait complete. 2024-01-15T18:05:05.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c4e00640 1 -- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c00b1730 msgr2=0x7f37c014c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c4e00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37c00b1730 0x7f37c014c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c4e00640 1 -- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c006d5f0 msgr2=0x7f37c015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c4e00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c006d5f0 0x7f37c015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c4e00640 1 -- 172.21.15.143:0/1618897272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80094d0 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37bfe00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37c006d5f0 0x7f37c015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c4e00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37c0153ad0 0x7f37c0151ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c780 tx=0x7f37b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800aa50 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c6e00640 1 -- 172.21.15.143:0/1618897272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c009b530 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.301+0000 7f37c6e00640 1 -- 172.21.15.143:0/1618897272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c009baf0 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.305+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b8003e60 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.305+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800adb0 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.309+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3790000fd0 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.309+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f37b8012420 con 0x7f37c0153ad0 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.309+0000 7f37b7e00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37980668f0 0x7f3798068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.309+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37b800ce30 con 0x7f37c0153ad0 2024-01-15T18:05:05.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.309+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 --> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0f4067680 con 0x7fb0cc06ebe0 2024-01-15T18:05:05.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.309+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 --> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2724067680 con 0x7f26fc06eb60 2024-01-15T18:05:05.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37b7e00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f379806ea90 0x7f3798070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 --> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3798071560 con 0x7f379806ea90 2024-01-15T18:05:05.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b8051a30 con 0x7f37c0153ad0 2024-01-15T18:05:05.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f2720c00640 1 -- 172.21.15.143:0/479111487 <== osd.3 v2:172.21.15.143:6822/254985013 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2724067680 con 0x7f26fc06eb60 2024-01-15T18:05:05.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37bf400640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f379806ea90 0x7f3798070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0eaa00640 1 -- 172.21.15.143:0/1513811948 <== osd.2 v2:172.21.15.143:6816/761026363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb0f4067680 con 0x7fb0cc06ebe0 2024-01-15T18:05:05.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37bfe00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37980668f0 0x7f3798068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7f26fc06eb60 msgr2=0x7f26fc070f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7f26fc06eb60 0x7f26fc070f80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37bf400640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f379806ea90 0x7f3798070eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f26fc0669c0 msgr2=0x7f26fc068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb0cc06ebe0 msgr2=0x7fb0cc071000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb0cc06ebe0 0x7fb0cc071000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f26fc0669c0 0x7f26fc068e80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2714006fd0 tx=0x7f2714008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27240cd090 msgr2=0x7f27240cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 --2- 172.21.15.143:0/479111487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27240cd090 0x7f27240cd540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f271c0ac000 tx=0x7f271c019540 comp rx=0 tx=0).stop 2024-01-15T18:05:05.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f2729400640 1 -- 172.21.15.143:0/479111487 reap_dead start 2024-01-15T18:05:05.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 --> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f548c067680 con 0x7f546406f840 2024-01-15T18:05:05.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0cc0669c0 msgr2=0x7fb0cc068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0cc0669c0 0x7fb0cc068e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec0ae000 tx=0x7fb0ec0a7070 comp rx=0 tx=0).stop 2024-01-15T18:05:05.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 msgr2=0x7fb0f40af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 --2- 172.21.15.143:0/1513811948 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0f41439e0 0x7fb0f40af420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00b410 tx=0x7fb0dc00b8d0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 shutdown_connections 2024-01-15T18:05:05.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 >> 172.21.15.143:0/479111487 conn(0x7f27240b0bf0 msgr2=0x7f27241416f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f3400640 1 -- 172.21.15.143:0/1513811948 reap_dead start 2024-01-15T18:05:05.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== osd.5 v2:172.21.15.204:6800/1020370322 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3798071560 con 0x7f379806ea90 2024-01-15T18:05:05.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 shutdown_connections 2024-01-15T18:05:05.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f272b400640 1 -- 172.21.15.143:0/479111487 wait complete. 2024-01-15T18:05:05.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 shutdown_connections 2024-01-15T18:05:05.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 >> 172.21.15.143:0/1513811948 conn(0x7fb0f40b0bf0 msgr2=0x7fb0f40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.313+0000 7f548a000640 1 -- 172.21.15.143:0/3050764861 <== osd.4 v2:172.21.15.204:6816/1717170274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f548c067680 con 0x7f546406f840 2024-01-15T18:05:05.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f546406f840 msgr2=0x7f5464071ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f546406f840 0x7f5464071ce0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 shutdown_connections 2024-01-15T18:05:05.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f37bfe00640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37980668f0 0x7f3798068db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f37a80be000 tx=0x7f37a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7fb0f9c00640 1 -- 172.21.15.143:0/1513811948 wait complete. 2024-01-15T18:05:05.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 --> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c20067680 con 0x7f6bf806ebe0 2024-01-15T18:05:05.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 --> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef900cf560 con 0x7fef68073020 2024-01-15T18:05:05.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5464066940 msgr2=0x7f5464068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5464066940 0x7f5464068e00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5474007fd0 tx=0x7f5474007900 comp rx=0 tx=0).stop 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f548c070fd0 msgr2=0x7f548c16b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 --2- 172.21.15.143:0/3050764861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f548c070fd0 0x7f548c16b820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f547c0ac040 tx=0x7f547c01ad50 comp rx=0 tx=0).stop 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5492800640 1 -- 172.21.15.143:0/3050764861 reap_dead start 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f6c1d600640 1 -- 172.21.15.143:0/1239821848 <== osd.6 v2:172.21.15.204:6808/1677254362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6c20067680 con 0x7f6bf806ebe0 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 shutdown_connections 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 >> 172.21.15.143:0/3050764861 conn(0x7f548c06a890 msgr2=0x7f548c0b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 shutdown_connections 2024-01-15T18:05:05.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f5494800640 1 -- 172.21.15.143:0/3050764861 wait complete. 2024-01-15T18:05:05.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f6bf806ebe0 msgr2=0x7f6bf8071000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7f6c28200640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f6bf806ebe0 0x7f6bf8071000 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.317+0000 7fef86a00640 1 -- 172.21.15.143:0/93028965 <== osd.0 v2:172.21.15.143:6802/2045350344 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef880017d0 con 0x7fef68073020 2024-01-15T18:05:05.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bf80669c0 msgr2=0x7f6bf8068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bf80669c0 0x7f6bf8068e80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c180ae000 tx=0x7f6c180a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:05.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c200ce310 msgr2=0x7f6c200ace90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 --2- 172.21.15.143:0/1239821848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c200ce310 0x7f6c200ace90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c0800c7b0 tx=0x7f6c0800cc70 comp rx=0 tx=0).stop 2024-01-15T18:05:05.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fef68073020 msgr2=0x7fef68075440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7fef95600640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fef68073020 0x7fef68075440 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef6806ae00 msgr2=0x7fef6806d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7fef95600640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef6806ae00 0x7fef6806d2c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fef78006e90 tx=0x7fef78008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c26200640 1 -- 172.21.15.143:0/1239821848 reap_dead start 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 shutdown_connections 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 >> 172.21.15.143:0/1239821848 conn(0x7f6c200b0bf0 msgr2=0x7f6c200cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 msgr2=0x7fef9014dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7fef95600640 1 --2- 172.21.15.143:0/93028965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef90071140 0x7fef9014dcc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fef800ac040 tx=0x7fef8001ad30 comp rx=0 tx=0).stop 2024-01-15T18:05:05.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 shutdown_connections 2024-01-15T18:05:05.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.321+0000 7f6c28200640 1 -- 172.21.15.143:0/1239821848 wait complete. 2024-01-15T18:05:05.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7fef8f400640 1 -- 172.21.15.143:0/93028965 reap_dead start 2024-01-15T18:05:05.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 shutdown_connections 2024-01-15T18:05:05.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 >> 172.21.15.143:0/93028965 conn(0x7fef900cfd20 msgr2=0x7fef90159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 --> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d64071860 con 0x7f3d3c06eb60 2024-01-15T18:05:05.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 shutdown_connections 2024-01-15T18:05:05.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7fef95600640 1 -- 172.21.15.143:0/93028965 wait complete. 2024-01-15T18:05:05.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d5aa00640 1 -- 172.21.15.143:0/742676726 <== osd.7 v2:172.21.15.204:6824/2462757405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d5c00f350 con 0x7f3d3c06eb60 2024-01-15T18:05:05.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f3d3c06eb60 msgr2=0x7f3d3c070f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f3d3c06eb60 0x7f3d3c070f80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d3c066940 msgr2=0x7f3d3c068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d3c066940 0x7f3d3c068e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d640b1620 tx=0x7f3d54009040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d64070fd0 msgr2=0x7f3d640a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 --2- 172.21.15.143:0/742676726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d64070fd0 0x7f3d640a84d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c019f60 tx=0x7f3d4c0190d0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d63400640 1 -- 172.21.15.143:0/742676726 reap_dead start 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 shutdown_connections 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.325+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 >> 172.21.15.143:0/742676726 conn(0x7f3d6406a890 msgr2=0x7f3d640b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.329+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 shutdown_connections 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.333+0000 7f3d69a00640 1 -- 172.21.15.143:0/742676726 wait complete. 2024-01-15T18:05:05.354 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:05.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-15T18:05:05.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.345+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 --> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3790002cc0 con 0x7f379806ea90 2024-01-15T18:05:05.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.349+0000 7f37b7e00640 1 -- 172.21.15.143:0/1618897272 <== osd.5 v2:172.21.15.204:6800/1020370322 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3790002cc0 con 0x7f379806ea90 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.349+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f379806ea90 msgr2=0x7f3798070eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.349+0000 7f37b5600640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f379806ea90 0x7f3798070eb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37980668f0 msgr2=0x7f3798068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37b5600640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37980668f0 0x7f3798068db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f37a80be000 tx=0x7f37a8008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37c0153ad0 msgr2=0x7f37c0151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37b5600640 1 --2- 172.21.15.143:0/1618897272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37c0153ad0 0x7f37c0151ae0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c780 tx=0x7f37b800cc40 comp rx=0 tx=0).stop 2024-01-15T18:05:05.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37c4e00640 1 -- 172.21.15.143:0/1618897272 reap_dead start 2024-01-15T18:05:05.361 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:05.361 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-15T18:05:05.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 shutdown_connections 2024-01-15T18:05:05.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.353+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 >> 172.21.15.143:0/1618897272 conn(0x7f37c00b0c10 msgr2=0x7f37c00b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.357+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 shutdown_connections 2024-01-15T18:05:05.362 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:05.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.357+0000 7f37b5600640 1 -- 172.21.15.143:0/1618897272 wait complete. 2024-01-15T18:05:05.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-15T18:05:05.366 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:05.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-15T18:05:05.373 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:05.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-15T18:05:05.377 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:05.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-15T18:05:05.382 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:05.382 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-15T18:05:05.396 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:05.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-15T18:05:05.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.581+0000 7f7a39200640 1 Processor -- start 2024-01-15T18:05:05.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a39200640 1 -- start start 2024-01-15T18:05:05.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a39200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a3415a440 0x7f7a3406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a39200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a340b0540 0x7f7a340b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a32800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f7a340cd4a0 0x7f7a340b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59214/0 (socket says 172.21.15.143:59214) 2024-01-15T18:05:05.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a31400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a340b0540 0x7f7a340b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a31400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a340b0540 0x7f7a340b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60372/0 (socket says 172.21.15.143:60372) 2024-01-15T18:05:05.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a31e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a3415a440 0x7f7a3406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a31e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7a3415a440 0x7f7a3406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57994/0 (socket says 172.21.15.143:57994) 2024-01-15T18:05:05.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.585+0000 7f7a32800640 1 -- 172.21.15.143:0/161641226 learned_addr learned my addr 172.21.15.143:0/161641226 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a340cc790 con 0x7f7a340cd4a0 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b4ae0 con 0x7f7a3415a440 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340b4800 con 0x7f7a340b0540 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a30a00640 1 -- 172.21.15.143:0/161641226 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438615456 0 0) 0x7f7a340cc790 con 0x7f7a340cd4a0 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a30a00640 1 -- 172.21.15.143:0/161641226 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a14003530 con 0x7f7a340cd4a0 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a31e00640 1 -- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a340b0540 msgr2=0x7f7a340b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a31e00640 1 --2- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a340b0540 0x7f7a340b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a31e00640 1 -- 172.21.15.143:0/161641226 >> v1:172.21.15.204:6789/0 conn(0x7f7a340cd4a0 legacy=0x7f7a340b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.589+0000 7f7a31e00640 1 -- 172.21.15.143:0/161641226 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340cc9c0 con 0x7f7a3415a440 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.593+0000 7f7a31e00640 1 --2- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a3415a440 0x7f7a3406db20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009d90 tx=0x7f7a2c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7720d1fdbaf36fdd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.593+0000 7f7a30a00640 1 -- 172.21.15.143:0/161641226 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0a58f0 con 0x7f7a3415a440 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.593+0000 7f7a30a00640 1 -- 172.21.15.143:0/161641226 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c0a5a50 con 0x7f7a3415a440 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.593+0000 7f7a30a00640 1 -- 172.21.15.143:0/161641226 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0a5de0 con 0x7f7a3415a440 2024-01-15T18:05:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.593+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a3415a440 msgr2=0x7f7a3406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.593+0000 7f7a39200640 1 --2- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a3415a440 0x7f7a3406db20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009d90 tx=0x7f7a2c01c920 comp rx=0 tx=0).stop 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 shutdown_connections 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 --2- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a340b0540 0x7f7a340b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 --2- 172.21.15.143:0/161641226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a3415a440 0x7f7a3406db20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 >> 172.21.15.143:0/161641226 conn(0x7f7a340aea10 msgr2=0x7f7a340aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 shutdown_connections 2024-01-15T18:05:05.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 -- 172.21.15.143:0/161641226 wait complete. 2024-01-15T18:05:05.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 Processor -- start 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f3a21400640 1 Processor -- start 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f7a39200640 1 -- start start 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a39200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a340b0540 0x7f7a3414b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a39200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 0x7f7a340abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a39200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 0x7f7a3414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a39200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340a8570 con 0x7f7a340b0540 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a39200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3414be80 con 0x7f7a340cd4a0 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a39200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34152160 con 0x7f7a3415a440 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.597+0000 7f3a21400640 1 -- start start 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a21400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a1c1613e0 0x7f3a1c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a21400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3a1c0b2290 0x7f3a1c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a21400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c0b4140 con 0x7f3a1c06a690 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a21400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b2c40 con 0x7f3a1c1613e0 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a21400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0b4420 con 0x7f3a1c0b2290 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a32800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 0x7f7a3414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 0x7f7a340abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 0x7f7a340abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58002/0 (socket says 172.21.15.143:58002) 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 -- 172.21.15.143:0/2081329943 learned_addr learned my addr 172.21.15.143:0/2081329943 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31e00640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a340b0540 0x7f7a3414b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a32800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 0x7f7a3414f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60376/0 (socket says 172.21.15.143:60376) 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3a1c0b2290 0x7f3a1c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3a1c0b2290 0x7f3a1c0b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58014/0 (socket says 172.21.15.143:58014) 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 -- 172.21.15.143:0/17730004 learned_addr learned my addr 172.21.15.143:0/17730004 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 -- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 msgr2=0x7f7a3414f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 0x7f7a3414f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 -- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a340b0540 msgr2=0x7f7a3414b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a340b0540 0x7f7a3414b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31400640 1 -- 172.21.15.143:0/2081329943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009a70 con 0x7f7a340cd4a0 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a31e00640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a340b0540 0x7f7a3414b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f7a32800640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 0x7f7a3414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a1a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a1c1613e0 0x7f3a1c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.605+0000 7f7a31400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 0x7f7a340abb00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e840 tx=0x7f7a1c00ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.605+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c009510 con 0x7f7a340cd4a0 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.605+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c00ee90 con 0x7f7a340cd4a0 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.605+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00cb80 con 0x7f7a340cd4a0 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19400640 1 -- 172.21.15.143:0/17730004 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784708838 0 0) 0x7f3a1c0b4140 con 0x7f3a1c06a690 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19400640 1 -- 172.21.15.143:0/17730004 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39fc003530 con 0x7f3a1c06a690 2024-01-15T18:05:05.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 -- 172.21.15.143:0/17730004 >> v2:172.21.15.143:3300/0 conn(0x7f3a1c1613e0 msgr2=0x7f3a1c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 --2- 172.21.15.143:0/17730004 >> v2:172.21.15.143:3300/0 conn(0x7f3a1c1613e0 0x7f3a1c0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 -- 172.21.15.143:0/17730004 >> v1:172.21.15.204:6789/0 conn(0x7f3a1c06a690 legacy=0x7f3a1c06ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.601+0000 7f3a19e00640 1 -- 172.21.15.143:0/17730004 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0b4700 con 0x7f3a1c0b2290 2024-01-15T18:05:05.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.605+0000 7f7a39200640 1 -- 172.21.15.143:0/2081329943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34152440 con 0x7f7a340cd4a0 2024-01-15T18:05:05.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.605+0000 7f7a39200640 1 -- 172.21.15.143:0/2081329943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a34152a00 con 0x7f7a340cd4a0 2024-01-15T18:05:05.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f7a1c00cce0 con 0x7f7a340cd4a0 2024-01-15T18:05:05.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34067500 con 0x7f7a340cd4a0 2024-01-15T18:05:05.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7fbaeda00640 1 Processor -- start 2024-01-15T18:05:05.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7fbaeda00640 1 -- start start 2024-01-15T18:05:05.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7f3a19e00640 1 --2- 172.21.15.143:0/17730004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c0b2700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0b3320 tx=0x7f3a0401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=996757c2b9ca9044 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7f7a2a000640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a0c066940 0x7f7a0c068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.609+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a1c084410 con 0x7f7a340cd4a0 2024-01-15T18:05:05.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbaeda00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbae80b1710 0x7fbae806d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f7a31e00640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a0c066940 0x7f7a0c068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbaeda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8159170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbaeda00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae806cf30 con 0x7fbae8070370 2024-01-15T18:05:05.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbaeda00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae80cb0e0 con 0x7fbae80b1710 2024-01-15T18:05:05.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbaeda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae80cb2f0 con 0x7fbae80ce310 2024-01-15T18:05:05.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbae80b1710 0x7fbae806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 Processor -- start 2024-01-15T18:05:05.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbae80b1710 0x7fbae806d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58020/0 (socket says 172.21.15.143:58020) 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 -- 172.21.15.143:0/423084474 learned_addr learned my addr 172.21.15.143:0/423084474 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6000640 1 --2- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 -- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 msgr2=0x7fbae8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 --2- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 -- 172.21.15.143:0/423084474 >> v1:172.21.15.204:6789/0 conn(0x7fbae8070370 legacy=0x7fbae8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7fbae6a00640 1 -- 172.21.15.143:0/423084474 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae80cb5d0 con 0x7fbae80b1710 2024-01-15T18:05:05.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 -- start start 2024-01-15T18:05:05.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f7a31e00640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a0c066940 0x7f7a0c068e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0af000 tx=0x7f7a2c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f201015b260 0x7f20100cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f20100d19f0 0x7f20100cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f20100c5220 0x7f20100c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f200f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f201015b260 0x7f20100cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f200f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f201015b260 0x7f20100cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60390/0 (socket says 172.21.15.143:60390) 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f200f400640 1 -- 172.21.15.143:0/157999341 learned_addr learned my addr 172.21.15.143:0/157999341 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f200ea00640 1 --2- 172.21.15.143:0/157999341 >> v2:172.21.15.143:3300/0 conn(0x7f20100d19f0 0x7f20100cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20100c4d90 con 0x7f20100d19f0 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201015b640 con 0x7f20100c5220 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.613+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20100cf800 con 0x7f201015b260 2024-01-15T18:05:05.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f200fe00640 1 --2- 172.21.15.143:0/157999341 >> v2:172.21.15.204:3300/0 conn(0x7f20100c5220 0x7f20100c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f200ea00640 1 -- 172.21.15.143:0/157999341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f201015b260 msgr2=0x7f20100cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f200ea00640 1 --2- 172.21.15.143:0/157999341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f201015b260 0x7f20100cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f200ea00640 1 -- 172.21.15.143:0/157999341 >> v2:172.21.15.204:3300/0 conn(0x7f20100c5220 msgr2=0x7f20100c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f200ea00640 1 --2- 172.21.15.143:0/157999341 >> v2:172.21.15.204:3300/0 conn(0x7f20100c5220 0x7f20100c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f200ea00640 1 -- 172.21.15.143:0/157999341 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100cfae0 con 0x7f20100d19f0 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a19400640 1 -- 172.21.15.143:0/17730004 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040ab640 con 0x7f3a1c0b2290 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a19400640 1 -- 172.21.15.143:0/17730004 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a040a7070 con 0x7f3a1c0b2290 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a19400640 1 -- 172.21.15.143:0/17730004 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040ab9c0 con 0x7f3a1c0b2290 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 -- 172.21.15.143:0/17730004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0b2290 msgr2=0x7f3a1c0b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 --2- 172.21.15.143:0/17730004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c0b2700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0b3320 tx=0x7f3a0401c460 comp rx=0 tx=0).stop 2024-01-15T18:05:05.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 -- 172.21.15.143:0/17730004 shutdown_connections 2024-01-15T18:05:05.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 --2- 172.21.15.143:0/17730004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c0b2700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 --2- 172.21.15.143:0/17730004 >> v2:172.21.15.143:3300/0 conn(0x7f3a1c1613e0 0x7f3a1c0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 -- 172.21.15.143:0/17730004 >> 172.21.15.143:0/17730004 conn(0x7f3a1c066720 msgr2=0x7f3a1c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7a1c051730 con 0x7f7a340cd4a0 2024-01-15T18:05:05.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7fbae6a00640 1 --2- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae806d680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009870 tx=0x7fbad0020890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44ade529d689c78a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 -- 172.21.15.143:0/17730004 shutdown_connections 2024-01-15T18:05:05.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7f3a21400640 1 -- 172.21.15.143:0/17730004 wait complete. 2024-01-15T18:05:05.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7fbae5600640 1 -- 172.21.15.143:0/423084474 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a9760 con 0x7fbae80b1710 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7fbae5600640 1 -- 172.21.15.143:0/423084474 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad00a98c0 con 0x7fbae80b1710 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.617+0000 7fbae5600640 1 -- 172.21.15.143:0/423084474 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad00a9c50 con 0x7fbae80b1710 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 -- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 msgr2=0x7fbae806d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 --2- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae806d680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009870 tx=0x7fbad0020890 comp rx=0 tx=0).stop 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 -- 172.21.15.143:0/423084474 shutdown_connections 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 --2- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 --2- 172.21.15.143:0/423084474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae806d680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 -- 172.21.15.143:0/423084474 >> 172.21.15.143:0/423084474 conn(0x7fbae80b0bf0 msgr2=0x7fbae80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fe00b800640 1 Processor -- start 2024-01-15T18:05:05.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 -- 172.21.15.143:0/423084474 shutdown_connections 2024-01-15T18:05:05.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fe00b800640 1 -- start start 2024-01-15T18:05:05.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7f3a21400640 1 Processor -- start 2024-01-15T18:05:05.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7f3a21400640 1 -- start start 2024-01-15T18:05:05.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7f200ea00640 1 --2- 172.21.15.143:0/157999341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20100d19f0 0x7f20100cdd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2010071260 tx=0x7f200001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe3b6575e2543ff server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7f200e000640 1 -- 172.21.15.143:0/157999341 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20000ab600 con 0x7f20100d19f0 2024-01-15T18:05:05.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.621+0000 7fbaeda00640 1 -- 172.21.15.143:0/423084474 wait complete. 2024-01-15T18:05:05.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f200e000640 1 -- 172.21.15.143:0/157999341 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20000a7050 con 0x7f20100d19f0 2024-01-15T18:05:05.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f200e000640 1 -- 172.21.15.143:0/157999341 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20000ab980 con 0x7f20100d19f0 2024-01-15T18:05:05.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a21400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a1c06a690 0x7f3a1c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a21400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe00b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe00415a440 0x7fe00406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe00b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0040cd4a0 0x7fe0040cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe00b800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0040cc790 con 0x7fe0040b0540 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe00b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0040b4ae0 con 0x7fe00415a440 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe00b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0040b4570 con 0x7fe0040cd4a0 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20100d19f0 msgr2=0x7f20100cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 --2- 172.21.15.143:0/157999341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20100d19f0 0x7f20100cdd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2010071260 tx=0x7f200001c450 comp rx=0 tx=0).stop 2024-01-15T18:05:05.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fbaeda00640 1 Processor -- start 2024-01-15T18:05:05.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 shutdown_connections 2024-01-15T18:05:05.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 --2- 172.21.15.143:0/157999341 >> v2:172.21.15.204:3300/0 conn(0x7f20100c5220 0x7f20100c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe003e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe0040b0540 0x7fe0040b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:59240/0 (socket says 172.21.15.143:59240) 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fe003e00640 1 -- 172.21.15.143:0/2414042963 learned_addr learned my addr 172.21.15.143:0/2414042963 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 --2- 172.21.15.143:0/157999341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20100d19f0 0x7f20100cdd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a21400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c1613e0 0x7f3a1c1386b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a1a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a1c06a690 0x7f3a1c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a1a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a1c06a690 0x7f3a1c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60400/0 (socket says 172.21.15.143:60400) 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a19e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a19e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c142850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56478/0 (socket says 172.21.15.143:56478) 2024-01-15T18:05:05.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7fbaeda00640 1 -- start start 2024-01-15T18:05:05.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 --2- 172.21.15.143:0/157999341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f201015b260 0x7f20100cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 >> 172.21.15.143:0/157999341 conn(0x7f20100d0ed0 msgr2=0x7f20100d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a1a800640 1 -- 172.21.15.143:0/1303334926 learned_addr learned my addr 172.21.15.143:0/1303334926 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c143280 con 0x7f3a1c0b2290 2024-01-15T18:05:05.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c143560 con 0x7f3a1c1613e0 2024-01-15T18:05:05.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c143840 con 0x7f3a1c06a690 2024-01-15T18:05:05.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 shutdown_connections 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3a20a00640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c1613e0 0x7f3a1c1386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbaeda00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae8070370 0x7fbae81f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbaeda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbaeda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbaeda00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae815e810 con 0x7fbae8070370 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbaeda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae815eaf0 con 0x7fbae80b1710 2024-01-15T18:05:05.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbaeda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae815edd0 con 0x7fbae80ce310 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3dd4a00640 1 Processor -- start 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.625+0000 7f3dd4a00640 1 -- start start 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a20a00640 1 -- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a1c06a690 msgr2=0x7f3a1c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a20a00640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a1c06a690 0x7f3a1c15eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a20a00640 1 -- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a1c0b2290 msgr2=0x7f3a1c142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a20a00640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c142850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a20a00640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040094d0 con 0x7f3a1c1613e0 2024-01-15T18:05:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60422/0 (socket says 172.21.15.143:60422) 2024-01-15T18:05:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 -- 172.21.15.143:0/3680811999 learned_addr learned my addr 172.21.15.143:0/3680811999 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dd4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 0x7f3dd006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dd4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3dd006e300 0x7f3dd00cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a20a00640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c1613e0 0x7f3a1c1386b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400c700 tx=0x7f3a1400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae6000640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dd4a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd0159c00 con 0x7f3dd01538a0 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dd4a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd00b1e10 con 0x7f3dd006e300 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dd4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0159ee0 con 0x7f3dd01534c0 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe008e00640 1 --2- 172.21.15.143:0/2414042963 >> v2:172.21.15.204:3300/0 conn(0x7fe00415a440 0x7fe00406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe009800640 1 --2- 172.21.15.143:0/2414042963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0040cd4a0 0x7fe0040cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe003400640 1 -- 172.21.15.143:0/2414042963 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731297409 0 0) 0x7fe0040cc790 con 0x7fe0040b0540 2024-01-15T18:05:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe003400640 1 -- 172.21.15.143:0/2414042963 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe400bd40 con 0x7fe0040b0540 2024-01-15T18:05:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 -- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 msgr2=0x7fbae815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 -- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae8070370 msgr2=0x7fbae81f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f2016800640 1 -- 172.21.15.143:0/157999341 wait complete. 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe009800640 1 -- 172.21.15.143:0/2414042963 >> v2:172.21.15.204:3300/0 conn(0x7fe00415a440 msgr2=0x7fe00406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe009800640 1 --2- 172.21.15.143:0/2414042963 >> v2:172.21.15.204:3300/0 conn(0x7fe00415a440 0x7fe00406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe009800640 1 -- 172.21.15.143:0/2414042963 >> v1:172.21.15.204:6789/0 conn(0x7fe0040b0540 legacy=0x7fe0040b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fe009800640 1 -- 172.21.15.143:0/2414042963 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0040b4800 con 0x7fe0040cd4a0 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a14003b80 con 0x7f3a1c1613e0 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a14003ce0 con 0x7f3a1c1613e0 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a14005310 con 0x7f3a1c1613e0 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c143b20 con 0x7f3a1c1613e0 2024-01-15T18:05:05.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c147830 con 0x7f3a1c1613e0 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae6a00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae8070370 0x7fbae81f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae8070370 0x7fbae81f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 -- 172.21.15.143:0/3680811999 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad00094d0 con 0x7fbae80ce310 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae6a00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae8070370 0x7fbae81f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7fbae7400640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8144f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c730 tx=0x7fbae000cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0003b80 con 0x7fbae80ce310 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae815f0b0 con 0x7fbae80ce310 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae81f9d10 con 0x7fbae80ce310 2024-01-15T18:05:05.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0003ce0 con 0x7fbae80ce310 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 0x7f3dd006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 0x7f3dd006cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60438/0 (socket says 172.21.15.143:60438) 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 -- 172.21.15.143:0/2238319790 learned_addr learned my addr 172.21.15.143:0/2238319790 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 -- 172.21.15.143:0/2238319790 >> v2:172.21.15.204:3300/0 conn(0x7f3dd006e300 msgr2=0x7f3dd00cc800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 --2- 172.21.15.143:0/2238319790 >> v2:172.21.15.204:3300/0 conn(0x7f3dd006e300 0x7f3dd00cc800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 -- 172.21.15.143:0/2238319790 >> v1:172.21.15.143:6789/0 conn(0x7f3dd01538a0 legacy=0x7f3dd00b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f3dce800640 1 -- 172.21.15.143:0/2238319790 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd015a1c0 con 0x7f3dd01534c0 2024-01-15T18:05:05.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.629+0000 7f2016800640 1 Processor -- start 2024-01-15T18:05:05.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae000aa30 con 0x7fbae80ce310 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 -- start start 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 0x7f201014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 0x7f201006f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201015b260 0x7f201006f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201015b260 0x7f201006f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 0x7f201006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 0x7f201006f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60448/0 (socket says 172.21.15.143:60448) 2024-01-15T18:05:05.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab8005260 con 0x7fbae80ce310 2024-01-15T18:05:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201015b260 0x7f201006f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56496/0 (socket says 172.21.15.143:56496) 2024-01-15T18:05:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 0x7f201014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f4ab6e00640 1 Processor -- start 2024-01-15T18:05:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 0x7f201014d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58086/0 (socket says 172.21.15.143:58086) 2024-01-15T18:05:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f200ea00640 1 -- 172.21.15.143:0/397029596 learned_addr learned my addr 172.21.15.143:0/397029596 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201014d8b0 con 0x7f201015b260 2024-01-15T18:05:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010157c40 con 0x7f20100c5220 2024-01-15T18:05:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201009aca0 con 0x7f20100d19f0 2024-01-15T18:05:05.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f4ab6e00640 1 -- start start 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3a14005470 con 0x7f3a1c1613e0 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c067680 con 0x7f3a1c1613e0 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.633+0000 7f3a12a00640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39f40669f0 0x7f39f4068eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200ea00640 1 -- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 msgr2=0x7f201014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200ea00640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 0x7f201014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200ea00640 1 -- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201015b260 msgr2=0x7f201006f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200ea00640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201015b260 0x7f201006f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200ea00640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000009420 con 0x7f20100d19f0 2024-01-15T18:05:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a14084020 con 0x7f3a1c1613e0 2024-01-15T18:05:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fbae000ab90 con 0x7fbae80ce310 2024-01-15T18:05:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200f400640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 0x7f201014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:05.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f200ea00640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 0x7f201006f360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20000ae040 tx=0x7f20000191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f3dce800640 1 --2- 172.21.15.143:0/2238319790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 0x7f3dd006cda0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009870 tx=0x7f3db801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd1d30631a6d9dd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f3a1a800640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39f40669f0 0x7f39f4068eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab6e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ab0159170 0x7f4ab006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00ce310 0x7f4ab0070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab6e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4ab00708b0 0x7f4ab015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7fbadea00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbac00668f0 0x7fbac0068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbae0084220 con 0x7fbae80ce310 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab6e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab00cb0e0 con 0x7f4ab0159170 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4aaf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00ce310 0x7f4ab0070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4aafe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4ab0159170 0x7f4ab006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4aaf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00ce310 0x7f4ab0070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60462/0 (socket says 172.21.15.143:60462) 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4aaf400640 1 -- 172.21.15.143:0/1650071687 learned_addr learned my addr 172.21.15.143:0/1650071687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab6e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab006cf30 con 0x7f4ab00708b0 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab6e00640 1 -- 172.21.15.143:0/1650071687 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab015dc70 con 0x7f4ab00ce310 2024-01-15T18:05:05.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/1650071687 >> v2:172.21.15.204:3300/0 conn(0x7f4ab00708b0 0x7f4ab015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f3a1a800640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39f40669f0 0x7f39f4068eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c16bf50 tx=0x7f3a0c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7fbae6a00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbac00668f0 0x7fbac0068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab4e00640 1 -- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00ce310 msgr2=0x7f4ab0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00ce310 0x7f4ab0070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab4e00640 1 -- 172.21.15.143:0/1650071687 >> v2:172.21.15.143:3300/0 conn(0x7f4ab0159170 msgr2=0x7f4ab006d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/1650071687 >> v2:172.21.15.143:3300/0 conn(0x7f4ab0159170 0x7f4ab006d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7f4ab4e00640 1 -- 172.21.15.143:0/1650071687 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00cb2f0 con 0x7f4ab00708b0 2024-01-15T18:05:05.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7fbae6a00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbac00668f0 0x7fbac0068db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b3000 tx=0x7fbad0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.637+0000 7fe009800640 1 --2- 172.21.15.143:0/2414042963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0040cd4a0 0x7fe0040cd950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdff40097c0 tx=0x7fdff401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4002b057e17519 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3a14051380 con 0x7f3a1c1613e0 2024-01-15T18:05:05.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7fe003400640 1 -- 172.21.15.143:0/2414042963 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff40ab600 con 0x7fe0040cd4a0 2024-01-15T18:05:05.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20000b4430 con 0x7f20100d19f0 2024-01-15T18:05:05.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20000b4590 con 0x7f20100d19f0 2024-01-15T18:05:05.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20000b4900 con 0x7f20100d19f0 2024-01-15T18:05:05.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f3dcd400640 1 -- 172.21.15.143:0/2238319790 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80ad640 con 0x7f3dd01534c0 2024-01-15T18:05:05.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f3dcd400640 1 -- 172.21.15.143:0/2238319790 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db80ad7a0 con 0x7f3dd01534c0 2024-01-15T18:05:05.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f3dcd400640 1 -- 172.21.15.143:0/2238319790 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80adb30 con 0x7f3dd01534c0 2024-01-15T18:05:05.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201009af80 con 0x7f20100d19f0 2024-01-15T18:05:05.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbae00513c0 con 0x7fbae80ce310 2024-01-15T18:05:05.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201009b540 con 0x7f20100d19f0 2024-01-15T18:05:05.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.641+0000 7fe003400640 1 -- 172.21.15.143:0/2414042963 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff40a7050 con 0x7fe0040cd4a0 2024-01-15T18:05:05.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 -- 172.21.15.143:0/2414042963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0040cd4a0 msgr2=0x7fe0040cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 --2- 172.21.15.143:0/2414042963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0040cd4a0 0x7fe0040cd950 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdff40097c0 tx=0x7fdff401c410 comp rx=0 tx=0).stop 2024-01-15T18:05:05.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f3dd4a00640 1 -- 172.21.15.143:0/2238319790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 msgr2=0x7f3dd006cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00708b0 0x7f4ab015cea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00aeb30 tx=0x7f4aa8020c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9024af9c707a383 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/2238319790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 0x7f3dd006cda0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009870 tx=0x7f3db801c520 comp rx=0 tx=0).stop 2024-01-15T18:05:05.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2010067500 con 0x7f20100d19f0 2024-01-15T18:05:05.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4aaea00640 1 -- 172.21.15.143:0/1650071687 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80a9d00 con 0x7f4ab00708b0 2024-01-15T18:05:05.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f20000b2040 con 0x7f20100d19f0 2024-01-15T18:05:05.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4aaea00640 1 -- 172.21.15.143:0/1650071687 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa80a9e60 con 0x7f4ab00708b0 2024-01-15T18:05:05.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4aaea00640 1 -- 172.21.15.143:0/1650071687 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80b1660 con 0x7f4ab00708b0 2024-01-15T18:05:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 -- 172.21.15.143:0/2414042963 shutdown_connections 2024-01-15T18:05:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 --2- 172.21.15.143:0/2414042963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0040cd4a0 0x7fe0040cd950 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 --2- 172.21.15.143:0/2414042963 >> v2:172.21.15.204:3300/0 conn(0x7fe00415a440 0x7fe00406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 -- 172.21.15.143:0/2414042963 >> 172.21.15.143:0/2414042963 conn(0x7fe0040aea10 msgr2=0x7fe0040aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f2007400640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1fe8066940 0x7f1fe8068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f200011e220 con 0x7f20100d19f0 2024-01-15T18:05:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 -- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00708b0 msgr2=0x7f4ab015cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00708b0 0x7f4ab015cea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00aeb30 tx=0x7f4aa8020c70 comp rx=0 tx=0).stop 2024-01-15T18:05:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f200f400640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1fe8066940 0x7f1fe8068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 -- 172.21.15.143:0/1650071687 shutdown_connections 2024-01-15T18:05:05.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00708b0 0x7f4ab015cea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/1650071687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab00ce310 0x7f4ab0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/1650071687 >> v2:172.21.15.143:3300/0 conn(0x7f4ab0159170 0x7f4ab006d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7f4ab6e00640 1 -- 172.21.15.143:0/1650071687 >> 172.21.15.143:0/1650071687 conn(0x7f4ab00b0bf0 msgr2=0x7f4ab00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 -- 172.21.15.143:0/2414042963 shutdown_connections 2024-01-15T18:05:05.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.645+0000 7fe00b800640 1 -- 172.21.15.143:0/2414042963 wait complete. 2024-01-15T18:05:05.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 -- 172.21.15.143:0/2238319790 shutdown_connections 2024-01-15T18:05:05.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/2238319790 >> v2:172.21.15.204:3300/0 conn(0x7f3dd006e300 0x7f3dd00cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/2238319790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd01534c0 0x7f3dd006cda0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 -- 172.21.15.143:0/2238319790 >> 172.21.15.143:0/2238319790 conn(0x7f3dd006a890 msgr2=0x7f3dd006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f4ab6e00640 1 -- 172.21.15.143:0/1650071687 shutdown_connections 2024-01-15T18:05:05.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f200f400640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1fe8066940 0x7f1fe8068e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009b00 tx=0x7f1ff8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 -- 172.21.15.143:0/2238319790 shutdown_connections 2024-01-15T18:05:05.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 -- 172.21.15.143:0/2238319790 wait complete. 2024-01-15T18:05:05.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f4ab6e00640 1 -- 172.21.15.143:0/1650071687 wait complete. 2024-01-15T18:05:05.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7fe00b800640 1 Processor -- start 2024-01-15T18:05:05.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f4ab6e00640 1 Processor -- start 2024-01-15T18:05:05.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20000eb4d0 con 0x7f20100d19f0 2024-01-15T18:05:05.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 -- start start 2024-01-15T18:05:05.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f4ab6e00640 1 -- start start 2024-01-15T18:05:05.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab6e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ab00708b0 0x7f4ab0141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00ce310 0x7f4ab015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0159170 0x7f4ab014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0040b0540 0x7fe0041f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0040cd4a0 0x7fe004157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe00415a440 0x7fe00415cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe004157b80 con 0x7fe0040b0540 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0041f5150 con 0x7fe0040cd4a0 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe00b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0041f5430 con 0x7fe00415a440 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe009800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe00415a440 0x7fe00415cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0040cd4a0 0x7fe004157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe009800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe00415a440 0x7fe00415cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60472/0 (socket says 172.21.15.143:60472) 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe009800640 1 -- 172.21.15.143:0/3352287304 learned_addr learned my addr 172.21.15.143:0/3352287304 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab6e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab015c400 con 0x7f4ab00708b0 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab01fb6f0 con 0x7f4ab00ce310 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab015b780 con 0x7f4ab0159170 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4aaf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00ce310 0x7f4ab015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4aaf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00ce310 0x7f4ab015acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58116/0 (socket says 172.21.15.143:58116) 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4aaf400640 1 -- 172.21.15.143:0/310276608 learned_addr learned my addr 172.21.15.143:0/310276608 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 -- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe00415a440 msgr2=0x7fe00415cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe00415a440 0x7fe00415cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 -- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0040b0540 msgr2=0x7fe0041f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0040b0540 0x7fe0041f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 -- 172.21.15.143:0/3352287304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4009420 con 0x7fe0040cd4a0 2024-01-15T18:05:05.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0159170 0x7f4ab014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.649+0000 7f3dd4a00640 1 Processor -- start 2024-01-15T18:05:05.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f3dd4a00640 1 -- start start 2024-01-15T18:05:05.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe003e00640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0040cd4a0 0x7fe004157350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0040b0960 tx=0x7fdfec00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab4e00640 1 -- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00ce310 msgr2=0x7f4ab015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00ce310 0x7f4ab015acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab4e00640 1 -- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ab00708b0 msgr2=0x7f4ab0141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ab00708b0 0x7f4ab0141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7f4ab4e00640 1 -- 172.21.15.143:0/310276608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa8009a70 con 0x7f4ab0159170 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.653+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0074e0 con 0x7fe0040cd4a0 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec007640 con 0x7fe0040cd4a0 2024-01-15T18:05:05.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0049f0 con 0x7fe0040cd4a0 2024-01-15T18:05:05.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f4ab4e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0159170 0x7f4ab014dfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00cb220 tx=0x7f4aa801da10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0041f5710 con 0x7fe0040cd4a0 2024-01-15T18:05:05.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dd4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 0x7f3dd0147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80b1d20 con 0x7f4ab0159170 2024-01-15T18:05:05.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa80b1e80 con 0x7f4ab0159170 2024-01-15T18:05:05.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dd4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd01534c0 0x7f3dd0149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8012290 con 0x7f4ab0159170 2024-01-15T18:05:05.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab015ba60 con 0x7f4ab0159170 2024-01-15T18:05:05.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe004157e60 con 0x7fe0040cd4a0 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 0x7f3dd0147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 0x7f3dd0147230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60482/0 (socket says 172.21.15.143:60482) 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 -- 172.21.15.143:0/1164829268 learned_addr learned my addr 172.21.15.143:0/1164829268 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd01538a0 0x7f3dd0148030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0163a30 con 0x7f3dd01534c0 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0163d10 con 0x7f3dd01538a0 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd0163ff0 con 0x7f3dd006e300 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dcf200640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd01538a0 0x7f3dd0148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 -- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd01538a0 msgr2=0x7f3dd0148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd01538a0 0x7f3dd0148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 -- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd01534c0 msgr2=0x7f3dd0149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd01534c0 0x7f3dd0149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80094d0 con 0x7f3dd006e300 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dcf200640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd01538a0 0x7f3dd0148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:05.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dce800640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 0x7f3dd0147230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3db80b0040 tx=0x7f3db80192c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80b6830 con 0x7f3dd006e300 2024-01-15T18:05:05.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db80a6040 con 0x7f3dd006e300 2024-01-15T18:05:05.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80b6bf0 con 0x7f3dd006e300 2024-01-15T18:05:05.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.657+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab015c020 con 0x7f4ab0159170 2024-01-15T18:05:05.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd01642d0 con 0x7f3dd006e300 2024-01-15T18:05:05.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0041f5930 con 0x7fe0040cd4a0 2024-01-15T18:05:05.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd014a480 con 0x7f3dd006e300 2024-01-15T18:05:05.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f4aa80123f0 con 0x7f4ab0159170 2024-01-15T18:05:05.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab0067680 con 0x7f4ab0159170 2024-01-15T18:05:05.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f4aa7e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a880669c0 0x7f4a88068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fdfec004bb0 con 0x7fe0040cd4a0 2024-01-15T18:05:05.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4aa811fb90 con 0x7f4ab0159170 2024-01-15T18:05:05.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.661+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd01647f0 con 0x7f3dd006e300 2024-01-15T18:05:05.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7f4aafe00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a880669c0 0x7f4a88068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7fe000c00640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfdc0669c0 0x7fdfdc068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7f4aafe00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a880669c0 0x7f4a88068e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0006aa0 tx=0x7f4aa0006a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfec0842e0 con 0x7fe0040cd4a0 2024-01-15T18:05:05.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7fe008e00640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfdc0669c0 0x7fdfdc068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3db80b4070 con 0x7f3dd006e300 2024-01-15T18:05:05.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.665+0000 7fe008e00640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfdc0669c0 0x7fdfdc068e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe004158af0 tx=0x7fdffc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.669+0000 7f3dc6a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3da80669c0 0x7f3da8068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.669+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db811ee00 con 0x7f3dd006e300 2024-01-15T18:05:05.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.669+0000 7f3dcde00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3da80669c0 0x7f3da8068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.669+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4aa80ecf00 con 0x7f4ab0159170 2024-01-15T18:05:05.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.669+0000 7f3dcde00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3da80669c0 0x7f3da8068e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3dd01fd780 tx=0x7f3dc0006c50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.669+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdfec0515c0 con 0x7fe0040cd4a0 2024-01-15T18:05:05.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.673+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3dd01647f0 con 0x7f3dd006e300 2024-01-15T18:05:05.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.681+0000 7fe58a800640 1 Processor -- start 2024-01-15T18:05:05.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.681+0000 7fe58a800640 1 -- start start 2024-01-15T18:05:05.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.681+0000 7fe58a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe5840d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.681+0000 7fe589400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe5840d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.681+0000 7fe589400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe5840d19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60498/0 (socket says 172.21.15.143:60498) 2024-01-15T18:05:05.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.681+0000 7fe589400640 1 -- 172.21.15.143:0/1529640304 learned_addr learned my addr 172.21.15.143:0/1529640304 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe58a800640 1 --2- 172.21.15.143:0/1529640304 >> v2:172.21.15.143:3300/0 conn(0x7fe58409a180 0x7fe5840cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58406c570 con 0x7fe5840b2240 2024-01-15T18:05:05.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe589e00640 1 --2- 172.21.15.143:0/1529640304 >> v2:172.21.15.143:3300/0 conn(0x7fe58409a180 0x7fe5840cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58406ce10 con 0x7fe58409a180 2024-01-15T18:05:05.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840b2740 con 0x7fe5840b17f0 2024-01-15T18:05:05.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe589e00640 1 -- 172.21.15.143:0/1529640304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 msgr2=0x7fe5840d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe589e00640 1 --2- 172.21.15.143:0/1529640304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe5840d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe589e00640 1 -- 172.21.15.143:0/1529640304 >> v1:172.21.15.143:6789/0 conn(0x7fe5840b2240 legacy=0x7fe584099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe589e00640 1 -- 172.21.15.143:0/1529640304 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58406c7c0 con 0x7fe58409a180 2024-01-15T18:05:05.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.685+0000 7fe589400640 1 --2- 172.21.15.143:0/1529640304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe5840d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:05.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.689+0000 7fe589e00640 1 --2- 172.21.15.143:0/1529640304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe58409a180 0x7fe5840cec10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009d90 tx=0x7fe57c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa654470334f2d00 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe583e00640 1 -- 172.21.15.143:0/1529640304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0a5870 con 0x7fe58409a180 2024-01-15T18:05:05.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe583e00640 1 -- 172.21.15.143:0/1529640304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c0a59d0 con 0x7fe58409a180 2024-01-15T18:05:05.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe58409a180 msgr2=0x7fe5840cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe583e00640 1 -- 172.21.15.143:0/1529640304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0a5870 con 0x7fe58409a180 2024-01-15T18:05:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 --2- 172.21.15.143:0/1529640304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe58409a180 0x7fe5840cec10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009d90 tx=0x7fe57c01c890 comp rx=0 tx=0).stop 2024-01-15T18:05:05.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 shutdown_connections 2024-01-15T18:05:05.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 --2- 172.21.15.143:0/1529640304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe58409a180 0x7fe5840cec10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 --2- 172.21.15.143:0/1529640304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe5840d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 >> 172.21.15.143:0/1529640304 conn(0x7fe5840d0e90 msgr2=0x7fe5840d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 shutdown_connections 2024-01-15T18:05:05.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.693+0000 7fe58a800640 1 -- 172.21.15.143:0/1529640304 wait complete. 2024-01-15T18:05:05.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.697+0000 7fe58a800640 1 Processor -- start 2024-01-15T18:05:05.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 -- start start 2024-01-15T18:05:05.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 0x7fe5840cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe58414f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5840b2240 0x7fe584150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58414fc30 con 0x7fe5840b2240 2024-01-15T18:05:05.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5841510d0 con 0x7fe58409a180 2024-01-15T18:05:05.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe58a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5840aecd0 con 0x7fe5840b17f0 2024-01-15T18:05:05.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 0x7fe5840cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 0x7fe5840cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58158/0 (socket says 172.21.15.143:58158) 2024-01-15T18:05:05.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 -- 172.21.15.143:0/1543133602 learned_addr learned my addr 172.21.15.143:0/1543133602 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:05.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 -- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 msgr2=0x7fe58414f6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T18:05:05.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe58414f6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 -- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5840b2240 msgr2=0x7fe584150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5840b2240 0x7fe584150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.701+0000 7fe589400640 1 -- 172.21.15.143:0/1543133602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c0094d0 con 0x7fe58409a180 2024-01-15T18:05:05.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.705+0000 7fe589400640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 0x7fe5840cbc40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e930 tx=0x7fe56c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.705+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00ca00 con 0x7fe58409a180 2024-01-15T18:05:05.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.705+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c00cb60 con 0x7fe58409a180 2024-01-15T18:05:05.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.705+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c013470 con 0x7fe58409a180 2024-01-15T18:05:05.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.705+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c009a70 con 0x7fe58409a180 2024-01-15T18:05:05.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.705+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5840af290 con 0x7fe58409a180 2024-01-15T18:05:05.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.709+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe56c0135d0 con 0x7fe58409a180 2024-01-15T18:05:05.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.709+0000 7fe581600640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe55c0668f0 0x7fe55c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.709+0000 7fe588a00640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe55c0668f0 0x7fe55c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.713+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe584067c10 con 0x7fe58409a180 2024-01-15T18:05:05.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.713+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe56c084760 con 0x7fe58409a180 2024-01-15T18:05:05.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.713+0000 7fe588a00640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe55c0668f0 0x7fe55c068db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe574000f70 tx=0x7fe5740079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.717+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe56c0519c0 con 0x7fe58409a180 2024-01-15T18:05:05.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.953+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7a340b01b0 con 0x7f7a340cd4a0 2024-01-15T18:05:05.962 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:05.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.953+0000 7f7a2a000640 1 -- 172.21.15.143:0/2081329943 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a1c054d50 con 0x7f7a340cd4a0 2024-01-15T18:05:05.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbab8005510 con 0x7fbae80ce310 2024-01-15T18:05:05.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7fbadea00640 1 -- 172.21.15.143:0/3680811999 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbae00549e0 con 0x7fbae80ce310 2024-01-15T18:05:05.963 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-15T18:05:05.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a0c066940 msgr2=0x7f7a0c068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a0c066940 0x7f7a0c068e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c0af000 tx=0x7f7a2c0a7070 comp rx=0 tx=0).stop 2024-01-15T18:05:05.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 msgr2=0x7f7a340abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 0x7f7a340abb00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e840 tx=0x7f7a1c00ec40 comp rx=0 tx=0).stop 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 shutdown_connections 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a0c066940 0x7f7a0c068e00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a3415a440 0x7f7a3414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a340cd4a0 0x7f7a340abb00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 --2- 172.21.15.143:0/2081329943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a340b0540 0x7f7a3414b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.957+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 >> 172.21.15.143:0/2081329943 conn(0x7f7a340aea10 msgr2=0x7f7a34071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 shutdown_connections 2024-01-15T18:05:05.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7f7a23400640 1 -- 172.21.15.143:0/2081329943 wait complete. 2024-01-15T18:05:05.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbac00668f0 msgr2=0x7fbac0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbac00668f0 0x7fbac0068db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbad00b3000 tx=0x7fbad0008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 msgr2=0x7fbae8144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8144f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbae000c730 tx=0x7fbae000cbf0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 shutdown_connections 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbac00668f0 0x7fbac0068db0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbae80ce310 0x7fbae8144f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbae80b1710 0x7fbae815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 --2- 172.21.15.143:0/3680811999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbae8070370 0x7fbae81f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 >> 172.21.15.143:0/3680811999 conn(0x7fbae80b0bf0 msgr2=0x7fbae80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 shutdown_connections 2024-01-15T18:05:05.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.961+0000 7fbaeda00640 1 -- 172.21.15.143:0/3680811999 wait complete. 2024-01-15T18:05:05.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.969+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a1c1436a0 con 0x7f3a1c1613e0 2024-01-15T18:05:05.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.969+0000 7f3a12a00640 1 -- 172.21.15.143:0/1303334926 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a1c1436a0 con 0x7f3a1c1613e0 2024-01-15T18:05:05.974 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-15T18:05:05.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39f40669f0 msgr2=0x7f39f4068eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39f40669f0 0x7f39f4068eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c16bf50 tx=0x7f3a0c009190 comp rx=0 tx=0).stop 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c1613e0 msgr2=0x7f3a1c1386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c1613e0 0x7f3a1c1386b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a1400c700 tx=0x7f3a1400cbc0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 shutdown_connections 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39f40669f0 0x7f39f4068eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a1c1613e0 0x7f3a1c1386b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a1c0b2290 0x7f3a1c142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 --2- 172.21.15.143:0/1303334926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a1c06a690 0x7f3a1c15eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.973+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 >> 172.21.15.143:0/1303334926 conn(0x7f3a1c066720 msgr2=0x7f3a1c0b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.977+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 shutdown_connections 2024-01-15T18:05:05.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.977+0000 7f3a21400640 1 -- 172.21.15.143:0/1303334926 wait complete. 2024-01-15T18:05:05.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.977+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20100b6d50 con 0x7f20100d19f0 2024-01-15T18:05:05.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.981+0000 7f2007400640 1 -- 172.21.15.143:0/397029596 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20000eeaf0 con 0x7f20100d19f0 2024-01-15T18:05:05.989 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:05.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1fe8066940 msgr2=0x7f1fe8068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1fe8066940 0x7f1fe8068e00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009b00 tx=0x7f1ff8008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 msgr2=0x7f201006f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 0x7f201006f360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f20000ae040 tx=0x7f20000191f0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 shutdown_connections 2024-01-15T18:05:05.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1fe8066940 0x7f1fe8068e00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201015b260 0x7f201006f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20100d19f0 0x7f201006f360 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 --2- 172.21.15.143:0/397029596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20100c5220 0x7f201014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 >> 172.21.15.143:0/397029596 conn(0x7f20100d0ed0 msgr2=0x7f20100cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 shutdown_connections 2024-01-15T18:05:05.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.985+0000 7f2016800640 1 -- 172.21.15.143:0/397029596 wait complete. 2024-01-15T18:05:06.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.993+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4ab015c540 con 0x7f4ab0159170 2024-01-15T18:05:06.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.993+0000 7f4aa7e00640 1 -- 172.21.15.143:0/310276608 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ab015c540 con 0x7f4ab0159170 2024-01-15T18:05:06.002 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:06.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-01-15T18:05:06.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.997+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3dd01647f0 con 0x7f3dd006e300 2024-01-15T18:05:06.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.997+0000 7f3dc6a00640 1 -- 172.21.15.143:0/1164829268 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3dd01647f0 con 0x7f3dd006e300 2024-01-15T18:05:06.004 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-15T18:05:06.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.997+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a880669c0 msgr2=0x7f4a88068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.997+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a880669c0 0x7f4a88068e80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0006aa0 tx=0x7f4aa0006a30 comp rx=0 tx=0).stop 2024-01-15T18:05:06.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.997+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0159170 msgr2=0x7f4ab014dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:05.997+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0159170 0x7f4ab014dfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00cb220 tx=0x7f4aa801da10 comp rx=0 tx=0).stop 2024-01-15T18:05:06.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 shutdown_connections 2024-01-15T18:05:06.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a880669c0 0x7f4a88068e80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4ab0159170 0x7f4ab014dfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ab00ce310 0x7f4ab015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 --2- 172.21.15.143:0/310276608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ab00708b0 0x7f4ab0141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 >> 172.21.15.143:0/310276608 conn(0x7f4ab00b0bf0 msgr2=0x7f4ab00aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:06.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 shutdown_connections 2024-01-15T18:05:06.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f4ab6e00640 1 -- 172.21.15.143:0/310276608 wait complete. 2024-01-15T18:05:06.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3da80669c0 msgr2=0x7f3da8068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3da80669c0 0x7f3da8068e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3dd01fd780 tx=0x7f3dc0006c50 comp rx=0 tx=0).stop 2024-01-15T18:05:06.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 msgr2=0x7f3dd0147230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 0x7f3dd0147230 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3db80b0040 tx=0x7f3db80192c0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 shutdown_connections 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3da80669c0 0x7f3da8068e80 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3dd01fd780 tx=0x7f3dc0006c50 comp rx=0 tx=0).stop 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd01538a0 0x7f3dd0148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd01534c0 0x7f3dd0149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 --2- 172.21.15.143:0/1164829268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3dd006e300 0x7f3dd0147230 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 >> 172.21.15.143:0/1164829268 conn(0x7f3dd006a890 msgr2=0x7f3dd00cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 shutdown_connections 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7f3dd4a00640 1 -- 172.21.15.143:0/1164829268 wait complete. 2024-01-15T18:05:06.009 INFO:teuthology.orchestra.run.smithi143.stdout:55834574853 2024-01-15T18:05:06.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe0041f5290 con 0x7fe0040cd4a0 2024-01-15T18:05:06.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.001+0000 7fe000c00640 1 -- 172.21.15.143:0/3352287304 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0041f5290 con 0x7fe0040cd4a0 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.005+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfdc0669c0 msgr2=0x7fdfdc068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.005+0000 7fe00b800640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfdc0669c0 0x7fdfdc068e80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe004158af0 tx=0x7fdffc00b040 comp rx=0 tx=0).stop 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.005+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0040cd4a0 msgr2=0x7fe004157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.005+0000 7fe00b800640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0040cd4a0 0x7fe004157350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0040b0960 tx=0x7fdfec00c8e0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 shutdown_connections 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfdc0669c0 0x7fdfdc068e80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe00415a440 0x7fe00415cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0040cd4a0 0x7fe004157350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 --2- 172.21.15.143:0/3352287304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0040b0540 0x7fe0041f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 >> 172.21.15.143:0/3352287304 conn(0x7fe0040aea10 msgr2=0x7fe0040713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:06.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 shutdown_connections 2024-01-15T18:05:06.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.009+0000 7fe00b800640 1 -- 172.21.15.143:0/3352287304 wait complete. 2024-01-15T18:05:06.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-01-15T18:05:06.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-01-15T18:05:06.035 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-01-15T18:05:06.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-01-15T18:05:06.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.037+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe584099df0 con 0x7fe58409a180 2024-01-15T18:05:06.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.037+0000 7fe581600640 1 -- 172.21.15.143:0/1543133602 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe56c021070 con 0x7fe58409a180 2024-01-15T18:05:06.044 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe55c0668f0 msgr2=0x7fe55c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe55c0668f0 0x7fe55c068db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe574000f70 tx=0x7fe5740079c0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 msgr2=0x7fe5840cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:06.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 0x7fe5840cbc40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00e930 tx=0x7fe56c00edf0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 shutdown_connections 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe55c0668f0 0x7fe55c068db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5840b2240 0x7fe584150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5840b17f0 0x7fe58414f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 --2- 172.21.15.143:0/1543133602 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe58409a180 0x7fe5840cbc40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 >> 172.21.15.143:0/1543133602 conn(0x7fe5840d0e90 msgr2=0x7fe5840c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 shutdown_connections 2024-01-15T18:05:06.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:06.041+0000 7fe58a800640 1 -- 172.21.15.143:0/1543133602 wait complete. 2024-01-15T18:05:06.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-01-15T18:05:06.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.5 2024-01-15T18:05:07.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-15T18:05:07.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-15T18:05:07.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-15T18:05:07.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-15T18:05:07.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-15T18:05:07.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-15T18:05:07.050 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-15T18:05:07.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-15T18:05:07.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.213+0000 7f21c1000640 1 Processor -- start 2024-01-15T18:05:07.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.213+0000 7f21c1000640 1 -- start start 2024-01-15T18:05:07.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21c1000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21bc06db20 0x7f21bc0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21c1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21bc15a440 0x7f21bc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21b9e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21bc06db20 0x7f21bc0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21b9e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21bc06db20 0x7f21bc0b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58098/0 (socket says 172.21.15.143:58098) 2024-01-15T18:05:07.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21ba800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f21bc0cd9e0 0x7f21bc0b20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33204/0 (socket says 172.21.15.143:33204) 2024-01-15T18:05:07.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21c1000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc0cc790 con 0x7f21bc0cd9e0 2024-01-15T18:05:07.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21b9400640 1 --2- 172.21.15.143:0/3006592147 >> v2:172.21.15.143:3300/0 conn(0x7f21bc15a440 0x7f21bc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21b9e00640 1 -- 172.21.15.143:0/3006592147 learned_addr learned my addr 172.21.15.143:0/3006592147 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0b4ae0 con 0x7f21bc15a440 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0b4800 con 0x7f21bc06db20 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21b8a00640 1 -- 172.21.15.143:0/3006592147 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909458589 0 0) 0x7f21bc0cc790 con 0x7f21bc0cd9e0 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.217+0000 7f21b8a00640 1 -- 172.21.15.143:0/3006592147 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f219c003530 con 0x7f21bc0cd9e0 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.221+0000 7f21b8a00640 1 -- 172.21.15.143:0/3006592147 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1003584150 0 0) 0x7f219c003530 con 0x7f21bc0cd9e0 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.221+0000 7f21b9e00640 1 -- 172.21.15.143:0/3006592147 >> v2:172.21.15.143:3300/0 conn(0x7f21bc15a440 msgr2=0x7f21bc0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.221+0000 7f21b9e00640 1 --2- 172.21.15.143:0/3006592147 >> v2:172.21.15.143:3300/0 conn(0x7f21bc15a440 0x7f21bc0cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.221+0000 7f21b9e00640 1 -- 172.21.15.143:0/3006592147 >> v1:172.21.15.204:6789/0 conn(0x7f21bc0cd9e0 legacy=0x7f21bc0b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.221+0000 7f21b9e00640 1 -- 172.21.15.143:0/3006592147 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0cc9c0 con 0x7f21bc06db20 2024-01-15T18:05:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.225+0000 7f21b9e00640 1 --2- 172.21.15.143:0/3006592147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc06db20 0x7f21bc0b3e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0b5060 tx=0x7f21b401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83288d07aa0f77d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.225+0000 7f21b8a00640 1 -- 172.21.15.143:0/3006592147 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4001680 con 0x7f21bc06db20 2024-01-15T18:05:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.225+0000 7f21b8a00640 1 -- 172.21.15.143:0/3006592147 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4007b60 con 0x7f21bc06db20 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21b8a00640 1 -- 172.21.15.143:0/3006592147 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b40a5700 con 0x7f21bc06db20 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc06db20 msgr2=0x7f21bc0b3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 --2- 172.21.15.143:0/3006592147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc06db20 0x7f21bc0b3e60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0b5060 tx=0x7f21b401c8c0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 shutdown_connections 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 --2- 172.21.15.143:0/3006592147 >> v2:172.21.15.143:3300/0 conn(0x7f21bc15a440 0x7f21bc0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 --2- 172.21.15.143:0/3006592147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc06db20 0x7f21bc0b3e60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 >> 172.21.15.143:0/3006592147 conn(0x7f21bc0aea10 msgr2=0x7f21bc0aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 shutdown_connections 2024-01-15T18:05:07.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.229+0000 7f21c1000640 1 -- 172.21.15.143:0/3006592147 wait complete. 2024-01-15T18:05:07.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 Processor -- start 2024-01-15T18:05:07.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 -- start start 2024-01-15T18:05:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc06db20 0x7f21bc163aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 0x7f21bc14f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21bc15a440 0x7f21bc15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc06db20 0x7f21bc163aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21b9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 0x7f21bc14f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21b9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc06db20 0x7f21bc163aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48304/0 (socket says 172.21.15.143:48304) 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21b9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 0x7f21bc14f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58110/0 (socket says 172.21.15.143:58110) 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21b9e00640 1 -- 172.21.15.143:0/3026577066 learned_addr learned my addr 172.21.15.143:0/3026577066 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc14fd10 con 0x7f21bc06db20 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc158100 con 0x7f21bc0cd9e0 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1541f0 con 0x7f21bc15a440 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.233+0000 7f21ba800640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21bc15a440 0x7f21bc15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21ba800640 1 -- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 msgr2=0x7f21bc14f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21ba800640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 0x7f21bc14f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21ba800640 1 -- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc06db20 msgr2=0x7f21bc163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21ba800640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc06db20 0x7f21bc163aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21ba800640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b40075e0 con 0x7f21bc15a440 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21b9400640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 0x7f21bc14f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21ba800640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21bc15a440 0x7f21bc15c9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21a40017b0 tx=0x7f21a400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4007480 con 0x7f21bc15a440 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc1634c0 con 0x7f21bc15a440 2024-01-15T18:05:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a40075e0 con 0x7f21bc15a440 2024-01-15T18:05:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4007480 con 0x7f21bc15a440 2024-01-15T18:05:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.237+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc1567c0 con 0x7f21bc15a440 2024-01-15T18:05:07.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.241+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f21a4005820 con 0x7f21bc15a440 2024-01-15T18:05:07.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.241+0000 7f21b2000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21940669e0 0x7f2194068ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.241+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21a4083520 con 0x7f21bc15a440 2024-01-15T18:05:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.245+0000 7f21b9e00640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21940669e0 0x7f2194068ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.245+0000 7f21ab400640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc067500 con 0x7f21bc15a440 2024-01-15T18:05:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.245+0000 7f21b9e00640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21940669e0 0x7f2194068ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21bc15d580 tx=0x7f21b40a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.249+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21a4050770 con 0x7f21bc15a440 2024-01-15T18:05:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.261+0000 7fb1a8c00640 1 Processor -- start 2024-01-15T18:05:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a8c00640 1 -- start start 2024-01-15T18:05:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 0x7fb1a40adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a8c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a4143dc0 con 0x7fb1a406d080 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a8c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40aca80 con 0x7fb1a40ae060 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40ad700 con 0x7fb1a41439e0 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a2800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb1a406d080 0x7fb1a40b2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36028/0 (socket says 172.21.15.143:36028) 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.265+0000 7fb1a2800640 1 -- 172.21.15.143:0/1268895981 learned_addr learned my addr 172.21.15.143:0/1268895981 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 0x7fb1a40adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984354160 0 0) 0x7fb1a4143dc0 con 0x7fb1a406d080 2024-01-15T18:05:07.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb184003530 con 0x7fb1a406d080 2024-01-15T18:05:07.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107266094 0 0) 0x7fb184003530 con 0x7fb1a406d080 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 msgr2=0x7fb1a40adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 0x7fb1a40adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 >> v1:172.21.15.204:6789/0 conn(0x7fb1a40ae060 legacy=0x7fb1a406c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40acd60 con 0x7fb1a406d080 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fb19c001a00 con 0x7fb1a406d080 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 >> v1:172.21.15.143:6789/0 conn(0x7fb1a406d080 legacy=0x7fb1a40b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840039b0 0x7fb184003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 0x7fb1840047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840039b0 0x7fb184003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb184004d10 0x7fb184005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40ad700 con 0x7fb184004d10 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18400ba40 con 0x7fb1840039b0 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18400bbb0 con 0x7fb1a41439e0 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 -- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 msgr2=0x7fb1840047d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a41439e0 0x7fb1840047d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 -- 172.21.15.143:0/1268895981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb184004d10 msgr2=0x7fb184005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb184004d10 0x7fb184005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.269+0000 7fb1a1e00640 1 -- 172.21.15.143:0/1268895981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40acd60 con 0x7fb1840039b0 2024-01-15T18:05:07.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840039b0 0x7fb184003df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ae0 tx=0x7fb19401ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741550b8f679bfcb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7f5007200640 1 Processor -- start 2024-01-15T18:05:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940a5640 con 0x7fb1840039b0 2024-01-15T18:05:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1940a57a0 con 0x7fb1840039b0 2024-01-15T18:05:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a0a00640 1 -- 172.21.15.143:0/1268895981 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940a5b50 con 0x7fb1840039b0 2024-01-15T18:05:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7f5007200640 1 -- start start 2024-01-15T18:05:07.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a8c00640 1 -- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840039b0 msgr2=0x7fb184003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/1268895981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1840039b0 0x7fb184003df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb194009ae0 tx=0x7fb19401ca50 comp rx=0 tx=0).stop 2024-01-15T18:05:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.277+0000 7fb1a2800640 1 -- 172.21.15.143:0/1268895981 reap_dead start 2024-01-15T18:05:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7fb1a8c00640 1 -- 172.21.15.143:0/1268895981 shutdown_connections 2024-01-15T18:05:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7fb1a8c00640 1 -- 172.21.15.143:0/1268895981 >> 172.21.15.143:0/1268895981 conn(0x7fb1a40b0bf0 msgr2=0x7fb1a40b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7fb1a8c00640 1 -- 172.21.15.143:0/1268895981 shutdown_connections 2024-01-15T18:05:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5007200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5007200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 0x7f50081610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5007200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50080ac8e0 con 0x7f50080b2f20 2024-01-15T18:05:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5007200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5008146d90 con 0x7f5008160c60 2024-01-15T18:05:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5007200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5008161720 con 0x7f50081469b0 2024-01-15T18:05:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7fb1a8c00640 1 -- 172.21.15.143:0/1268895981 wait complete. 2024-01-15T18:05:07.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7fb1a8c00640 1 Processor -- start 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 -- start start 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 0x7f50081610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 0x7f50081610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58122/0 (socket says 172.21.15.143:58122) 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005400640 1 -- 172.21.15.143:0/623088313 learned_addr learned my addr 172.21.15.143:0/623088313 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005e00640 1 -- 172.21.15.143:0/623088313 >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 msgr2=0x7f50081610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005e00640 1 --2- 172.21.15.143:0/623088313 >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 0x7f50081610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005e00640 1 -- 172.21.15.143:0/623088313 >> v1:172.21.15.143:6789/0 conn(0x7f50080b2f20 legacy=0x7f500806d8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:05:07.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.281+0000 7f5005e00640 1 -- 172.21.15.143:0/623088313 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008161a00 con 0x7f50081469b0 2024-01-15T18:05:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7f5005400640 1 --2- 172.21.15.143:0/623088313 >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 0x7f50081610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 0x7fb1a40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1a40cd410 0x7fb1a40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a41f48c0 0x7fb1a41f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40ae4a0 con 0x7fb1a40cd410 2024-01-15T18:05:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4160430 con 0x7fb1a41f48c0 2024-01-15T18:05:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a415dba0 con 0x7fb1a4202d70 2024-01-15T18:05:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 0x7fb1a40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 0x7fb1a40cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55388/0 (socket says 172.21.15.143:55388) 2024-01-15T18:05:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 -- 172.21.15.143:0/2767401618 learned_addr learned my addr 172.21.15.143:0/2767401618 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a2800640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a41f48c0 0x7fb1a41f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 -- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a41f48c0 msgr2=0x7fb1a41f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a41f48c0 0x7fb1a41f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 -- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1a40cd410 msgr2=0x7fb1a40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1400640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1a40cd410 0x7fb1a40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1a40cd410 0x7fb1a40cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 -- 172.21.15.143:0/2767401618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4203150 con 0x7fb1a4202d70 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.285+0000 7fb1a1e00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 0x7fb1a40cb8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb19401c880 tx=0x7fb1940197f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7f5005e00640 1 --2- 172.21.15.143:0/623088313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080ad380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80097c0 tx=0x7f500806e7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c533d1d68577a63 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940a5e90 con 0x7fb1a4202d70 2024-01-15T18:05:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb194004020 con 0x7fb1a4202d70 2024-01-15T18:05:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1940043d0 con 0x7fb1a4202d70 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7f5004a00640 1 -- 172.21.15.143:0/623088313 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80ac630 con 0x7f50081469b0 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7f5004a00640 1 -- 172.21.15.143:0/623088313 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff80ac790 con 0x7f50081469b0 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7f5004a00640 1 -- 172.21.15.143:0/623088313 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80acb20 con 0x7f50081469b0 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a41f5b20 con 0x7fb1a4202d70 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a41f60e0 con 0x7fb1a4202d70 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7f5007200640 1 -- 172.21.15.143:0/623088313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 msgr2=0x7f50080ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.289+0000 7f5007200640 1 --2- 172.21.15.143:0/623088313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080ad380 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80097c0 tx=0x7f500806e7c0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7f5007200640 1 -- 172.21.15.143:0/623088313 shutdown_connections 2024-01-15T18:05:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7f5007200640 1 --2- 172.21.15.143:0/623088313 >> v2:172.21.15.204:3300/0 conn(0x7f5008160c60 0x7f50081610c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7f5007200640 1 --2- 172.21.15.143:0/623088313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080ad380 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7f5007200640 1 -- 172.21.15.143:0/623088313 >> 172.21.15.143:0/623088313 conn(0x7f50080b0b10 msgr2=0x7f50080b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a4067680 con 0x7fb1a4202d70 2024-01-15T18:05:07.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7f5007200640 1 -- 172.21.15.143:0/623088313 shutdown_connections 2024-01-15T18:05:07.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7f5007200640 1 -- 172.21.15.143:0/623088313 wait complete. 2024-01-15T18:05:07.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.293+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb1940b4070 con 0x7fb1a4202d70 2024-01-15T18:05:07.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.297+0000 7f5007200640 1 Processor -- start 2024-01-15T18:05:07.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.297+0000 7f1212400640 1 Processor -- start 2024-01-15T18:05:07.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.297+0000 7f1212400640 1 -- start start 2024-01-15T18:05:07.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.297+0000 7fb19a000640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb17c066940 0x7fb17c068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f9741a00640 1 Processor -- start 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.297+0000 7f5007200640 1 -- start start 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5007200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50080b2f20 0x7f50080cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5007200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5007200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5008160c60 0x7f500815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5007200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008164950 con 0x7f50080b2f20 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5007200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008163d80 con 0x7f5008160c60 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5007200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5008164020 con 0x7f50081469b0 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55394/0 (socket says 172.21.15.143:55394) 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 -- 172.21.15.143:0/3746756366 learned_addr learned my addr 172.21.15.143:0/3746756366 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5006800640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5008160c60 0x7f500815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f1212400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f120c0cf990 0x7f120c15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f120b400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f120c13e290 0x7f120c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33214/0 (socket says 172.21.15.143:33214) 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f120b400640 1 -- 172.21.15.143:0/3267146408 learned_addr learned my addr 172.21.15.143:0/3267146408 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c06c520 con 0x7f120c15f4e0 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 -- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5008160c60 msgr2=0x7f500815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5008160c60 0x7f500815b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 -- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50080b2f20 msgr2=0x7f50080cd5e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:05:07.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50080b2f20 0x7f50080cd5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 -- 172.21.15.143:0/3746756366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8009420 con 0x7f50081469b0 2024-01-15T18:05:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f5005400640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080cc7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000c7f0 tx=0x7f4ff000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c06c800 con 0x7f120c13e290 2024-01-15T18:05:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120c06cae0 con 0x7f120c0cf990 2024-01-15T18:05:07.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f120aa00640 1 --2- 172.21.15.143:0/3267146408 >> v2:172.21.15.143:3300/0 conn(0x7f120c0cf990 0x7f120c15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f9741a00640 1 -- start start 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f9741a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f973c14b1e0 0x7f973c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f9741a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 0x7f973c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f9741a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0cfc20 con 0x7f973c070e00 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f9741a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c162080 con 0x7f973c14b1e0 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f9741a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c161d70 con 0x7f973c0cf7c0 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 0x7f973c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb19411f150 con 0x7fb1a4202d70 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb19411f510 con 0x7fb1a4202d70 2024-01-15T18:05:07.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7fb1a1400640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb17c066940 0x7fb17c068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0007820 con 0x7f50081469b0 2024-01-15T18:05:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff0007980 con 0x7f50081469b0 2024-01-15T18:05:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0007d10 con 0x7f50081469b0 2024-01-15T18:05:07.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7fb5dce00640 1 Processor -- start 2024-01-15T18:05:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 0x7f973c0708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55406/0 (socket says 172.21.15.143:55406) 2024-01-15T18:05:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.301+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313367440 0 0) 0x7f120c06c800 con 0x7f120c13e290 2024-01-15T18:05:07.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 -- 172.21.15.143:0/3071788955 learned_addr learned my addr 172.21.15.143:0/3071788955 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5008164300 con 0x7f50081469b0 2024-01-15T18:05:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50080cdb20 con 0x7f50081469b0 2024-01-15T18:05:07.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ec003530 con 0x7f120c13e290 2024-01-15T18:05:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 -- 172.21.15.143:0/3071788955 >> v2:172.21.15.204:3300/0 conn(0x7f973c14b1e0 msgr2=0x7f973c0cf280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:05:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 --2- 172.21.15.143:0/3071788955 >> v2:172.21.15.204:3300/0 conn(0x7f973c14b1e0 0x7f973c0cf280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 -- 172.21.15.143:0/3071788955 >> v1:172.21.15.143:6789/0 conn(0x7f973c070e00 legacy=0x7f973c0b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f973a000640 1 -- 172.21.15.143:0/3071788955 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c14b5c0 con 0x7f973c0cf7c0 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281664589 0 0) 0x7f120c06c520 con 0x7f120c15f4e0 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120c06c800 con 0x7f120c15f4e0 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243754381 0 0) 0x7f11ec003530 con 0x7f120c13e290 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 >> v2:172.21.15.143:3300/0 conn(0x7f120c0cf990 msgr2=0x7f120c15efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 --2- 172.21.15.143:0/3267146408 >> v2:172.21.15.143:3300/0 conn(0x7f120c0cf990 0x7f120c15efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 >> v1:172.21.15.143:6789/0 conn(0x7f120c15f4e0 legacy=0x7f120c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c06cdc0 con 0x7f120c13e290 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120aa00640 1 --2- 172.21.15.143:0/3267146408 >> v2:172.21.15.143:3300/0 conn(0x7f120c0cf990 0x7f120c15efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f11fc001680 con 0x7f120c13e290 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 >> v1:172.21.15.204:6789/0 conn(0x7f120c13e290 legacy=0x7f120c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120a000640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec003b60 0x7f11ec003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.305+0000 7fb5dce00640 1 -- start start 2024-01-15T18:05:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5dce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb5d80cb430 0x7fb5d80b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5dce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5d8162490 0x7fb5d8071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5dce00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d8165930 con 0x7fb5d80717c0 2024-01-15T18:05:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5dce00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80b38a0 con 0x7fb5d8162490 2024-01-15T18:05:07.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5dce00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d8071bd0 con 0x7fb5d80cb430 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120a000640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11ec0047d0 0x7f11ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120a000640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ec005190 0x7f11ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c06cae0 con 0x7f11ec005190 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec00c060 con 0x7f11ec003b60 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec00c220 con 0x7f11ec0047d0 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120be00640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ec005190 0x7f11ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120be00640 1 -- 172.21.15.143:0/3267146408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11ec0047d0 msgr2=0x7f11ec004c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f4ff00113f0 con 0x7f50081469b0 2024-01-15T18:05:07.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5008067500 con 0x7f50081469b0 2024-01-15T18:05:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120be00640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11ec0047d0 0x7f11ec004c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120be00640 1 -- 172.21.15.143:0/3267146408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec003b60 msgr2=0x7f11ec003f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120be00640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec003b60 0x7f11ec003f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7f120be00640 1 -- 172.21.15.143:0/3267146408 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec003530 con 0x7f11ec005190 2024-01-15T18:05:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb5d80717c0 0x7fb5d8070210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36056/0 (socket says 172.21.15.143:36056) 2024-01-15T18:05:07.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7e00640 1 -- 172.21.15.143:0/2685167659 learned_addr learned my addr 172.21.15.143:0/2685167659 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb1a1400640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb17c066940 0x7fb17c068e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40ac8b0 tx=0x7fb18c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7400640 1 --2- 172.21.15.143:0/2685167659 >> v2:172.21.15.204:3300/0 conn(0x7fb5d80cb430 0x7fb5d80b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7400640 1 -- 172.21.15.143:0/2685167659 >> v2:172.21.15.143:3300/0 conn(0x7fb5d8162490 msgr2=0x7fb5d8071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7400640 1 --2- 172.21.15.143:0/2685167659 >> v2:172.21.15.143:3300/0 conn(0x7fb5d8162490 0x7fb5d8071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f4ffe000640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe00669c0 0x7f4fe0068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f973a000640 1 --2- 172.21.15.143:0/3071788955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 0x7f973c0708c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9724009870 tx=0x7f972401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68bdb59d5b0450c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f5005e00640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe00669c0 0x7f4fe0068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ff0084530 con 0x7f50081469b0 2024-01-15T18:05:07.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f120be00640 1 -- 172.21.15.143:0/3267146408 reap_dead start 2024-01-15T18:05:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f5005e00640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe00669c0 0x7f4fe0068e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80ae000 tx=0x7f4ff8004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f120be00640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ec005190 0x7f11ec00b990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1204001fd0 tx=0x7f1204020cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2440ba4b9a0b24cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7400640 1 -- 172.21.15.143:0/2685167659 >> v1:172.21.15.143:6789/0 conn(0x7fb5d80717c0 legacy=0x7fb5d8070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.309+0000 7fb5d7400640 1 -- 172.21.15.143:0/2685167659 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d81628d0 con 0x7fb5d80cb430 2024-01-15T18:05:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.313+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040a9bf0 con 0x7f11ec005190 2024-01-15T18:05:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f120a000640 1 -- 172.21.15.143:0/3267146408 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12040a9d50 con 0x7f11ec005190 2024-01-15T18:05:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ec005190 msgr2=0x7f11ec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ec005190 0x7f11ec00b990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1204001fd0 tx=0x7f1204020cf0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4ff0051750 con 0x7f50081469b0 2024-01-15T18:05:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9739600640 1 -- 172.21.15.143:0/3071788955 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240aa610 con 0x7f973c0cf7c0 2024-01-15T18:05:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9739600640 1 -- 172.21.15.143:0/3071788955 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240a6040 con 0x7f973c0cf7c0 2024-01-15T18:05:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9739600640 1 -- 172.21.15.143:0/3071788955 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240aa990 con 0x7f973c0cf7c0 2024-01-15T18:05:07.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7fb5d7400640 1 --2- 172.21.15.143:0/2685167659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d80cb430 0x7fb5d80b2b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80b2af0 tx=0x7fb5d001c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cdddc6effef3668 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 shutdown_connections 2024-01-15T18:05:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 --2- 172.21.15.143:0/3267146408 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11ec005190 0x7f11ec00b990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 >> 172.21.15.143:0/3267146408 conn(0x7f120c06a890 msgr2=0x7f120c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7fb5d6000640 1 -- 172.21.15.143:0/2685167659 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00a5800 con 0x7fb5d80cb430 2024-01-15T18:05:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 shutdown_connections 2024-01-15T18:05:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f1212400640 1 -- 172.21.15.143:0/3267146408 wait complete. 2024-01-15T18:05:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7fb5d6000640 1 -- 172.21.15.143:0/2685167659 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d00a5960 con 0x7fb5d80cb430 2024-01-15T18:05:07.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7fb5d6000640 1 -- 172.21.15.143:0/2685167659 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00a5cf0 con 0x7fb5d80cb430 2024-01-15T18:05:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9741a00640 1 -- 172.21.15.143:0/3071788955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 msgr2=0x7f973c0708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9741a00640 1 --2- 172.21.15.143:0/3071788955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 0x7f973c0708c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9724009870 tx=0x7f972401c4f0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9741a00640 1 -- 172.21.15.143:0/3071788955 shutdown_connections 2024-01-15T18:05:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7fb5dce00640 1 -- 172.21.15.143:0/2685167659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d80cb430 msgr2=0x7fb5d80b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9741a00640 1 --2- 172.21.15.143:0/3071788955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c0cf7c0 0x7f973c0708c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9741a00640 1 --2- 172.21.15.143:0/3071788955 >> v2:172.21.15.204:3300/0 conn(0x7f973c14b1e0 0x7f973c0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7f9741a00640 1 -- 172.21.15.143:0/3071788955 >> 172.21.15.143:0/3071788955 conn(0x7f973c06a890 msgr2=0x7f973c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.317+0000 7fb5dce00640 1 --2- 172.21.15.143:0/2685167659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d80cb430 0x7fb5d80b2b60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80b2af0 tx=0x7fb5d001c7a0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 -- 172.21.15.143:0/2685167659 shutdown_connections 2024-01-15T18:05:07.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7f9741a00640 1 -- 172.21.15.143:0/3071788955 shutdown_connections 2024-01-15T18:05:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 --2- 172.21.15.143:0/2685167659 >> v2:172.21.15.143:3300/0 conn(0x7fb5d8162490 0x7fb5d8071280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 --2- 172.21.15.143:0/2685167659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d80cb430 0x7fb5d80b2b60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 -- 172.21.15.143:0/2685167659 >> 172.21.15.143:0/2685167659 conn(0x7fb5d806a890 msgr2=0x7fb5d806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 -- 172.21.15.143:0/2685167659 shutdown_connections 2024-01-15T18:05:07.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 -- 172.21.15.143:0/2685167659 wait complete. 2024-01-15T18:05:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7f1212400640 1 Processor -- start 2024-01-15T18:05:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7f1212400640 1 -- start start 2024-01-15T18:05:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7f9741a00640 1 -- 172.21.15.143:0/3071788955 wait complete. 2024-01-15T18:05:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7f9741a00640 1 Processor -- start 2024-01-15T18:05:07.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7fb5dce00640 1 Processor -- start 2024-01-15T18:05:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.321+0000 7f9741a00640 1 -- start start 2024-01-15T18:05:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1212400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec005190 0x7f120c154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1212400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120c15c660 0x7f120c0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7fb5dce00640 1 -- start start 2024-01-15T18:05:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1212400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f120c1f2750 0x7f120c1f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1212400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c1f3120 con 0x7f120c15c660 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1212400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c154cb0 con 0x7f11ec005190 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1212400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0afb80 con 0x7f120c1f2750 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120c15c660 0x7f120c0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120c15c660 0x7f120c0b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48334/0 (socket says 172.21.15.143:48334) 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120aa00640 1 -- 172.21.15.143:0/2422754649 learned_addr learned my addr 172.21.15.143:0/2422754649 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f120c1f2750 0x7f120c1f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f9741a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c070e00 0x7f973c13f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f9741a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 0x7f973c144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f9741a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c14b1e0 0x7f973c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 -- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec005190 msgr2=0x7f120c154770 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec005190 0x7f120c154770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 -- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120c15c660 msgr2=0x7f120c0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120c15c660 0x7f120c0b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 -- 172.21.15.143:0/2422754649 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c135d40 con 0x7f120c1f2750 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f120be00640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f120c1f2750 0x7f120c1f2be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f120c0aeb30 tx=0x7f11ec005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204005860 con 0x7f120c1f2750 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f973a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 0x7f973c144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f973a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 0x7f973c144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58184/0 (socket says 172.21.15.143:58184) 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f9741a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1fa890 con 0x7f973c14b1e0 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f9741a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c140d30 con 0x7f973c0cf7c0 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f9741a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c140120 con 0x7f973c070e00 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f973aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c070e00 0x7f973c13f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f973a000640 1 -- 172.21.15.143:0/1305937599 learned_addr learned my addr 172.21.15.143:0/1305937599 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12040b1430 con 0x7f120c1f2750 2024-01-15T18:05:07.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.325+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040b17a0 con 0x7f120c1f2750 2024-01-15T18:05:07.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973b400640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c14b1e0 0x7f973c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973a000640 1 -- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c070e00 msgr2=0x7f973c13f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973a000640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c070e00 0x7f973c13f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973a000640 1 -- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c14b1e0 msgr2=0x7f973c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c136020 con 0x7f120c1f2750 2024-01-15T18:05:07.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973a000640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c14b1e0 0x7f973c0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973a000640 1 -- 172.21.15.143:0/1305937599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240094d0 con 0x7f973c0cf7c0 2024-01-15T18:05:07.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0b1980 con 0x7f120c1f2750 2024-01-15T18:05:07.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973b400640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c14b1e0 0x7f973c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:07.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5dce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 0x7fb5d814bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5dce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d80cb430 0x7fb5d815edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f973a000640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 0x7f973c144850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97240ac040 tx=0x7f972401ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 0x7fb5d814bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 0x7fb5d814bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55444/0 (socket says 172.21.15.143:55444) 2024-01-15T18:05:07.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240b3800 con 0x7f973c0cf7c0 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5dce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d8162490 0x7fb5d814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5dce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8148bb0 con 0x7fb5d80cb430 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5dce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8148e90 con 0x7fb5d8162490 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5dce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8149170 con 0x7fb5d80717c0 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d80cb430 0x7fb5d815edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d7400640 1 -- 172.21.15.143:0/2322579885 learned_addr learned my addr 172.21.15.143:0/2322579885 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240b3980 con 0x7f973c0cf7c0 2024-01-15T18:05:07.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240b3d30 con 0x7f973c0cf7c0 2024-01-15T18:05:07.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d7e00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d8162490 0x7fb5d814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c140400 con 0x7f973c0cf7c0 2024-01-15T18:05:07.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d7400640 1 -- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d8162490 msgr2=0x7fb5d814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c1409c0 con 0x7f973c0cf7c0 2024-01-15T18:05:07.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.329+0000 7fb5d7400640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d8162490 0x7fb5d814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5d7400640 1 -- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d80cb430 msgr2=0x7fb5d815edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5d7400640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d80cb430 0x7fb5d815edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5d7400640 1 -- 172.21.15.143:0/2322579885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0009a70 con 0x7fb5d80717c0 2024-01-15T18:05:07.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5d7e00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d8162490 0x7fb5d814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:07.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c136540 con 0x7f120c1f2750 2024-01-15T18:05:07.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5d7400640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 0x7fb5d814bc70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00b0040 tx=0x7fb5d0019540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c067680 con 0x7f973c0cf7c0 2024-01-15T18:05:07.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00adc80 con 0x7fb5d80717c0 2024-01-15T18:05:07.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d00ade00 con 0x7fb5d80717c0 2024-01-15T18:05:07.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00b7650 con 0x7fb5d80717c0 2024-01-15T18:05:07.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5dce00640 1 -- 172.21.15.143:0/2322579885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d8149450 con 0x7fb5d80717c0 2024-01-15T18:05:07.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7fb5dce00640 1 -- 172.21.15.143:0/2322579885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d815f2f0 con 0x7fb5d80717c0 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f12040b93b0 con 0x7f120c1f2750 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f1203400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f11e4066940 0x7f11e4068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f120411dc70 con 0x7f120c1f2750 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f120b400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f11e4066940 0x7f11e4068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.333+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f97240c6390 con 0x7f973c0cf7c0 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f9732a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97140669c0 0x7f9714068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f972411edd0 con 0x7f973c0cf7c0 2024-01-15T18:05:07.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f973aa00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97140669c0 0x7f9714068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb5d00b4050 con 0x7fb5d80717c0 2024-01-15T18:05:07.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f120b400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f11e4066940 0x7f11e4068e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11fc004880 tx=0x7f11fc001120 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7f973aa00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97140669c0 0x7f9714068e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f973400a510 tx=0x7f973400d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7fb5cf400640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5b0066940 0x7fb5b0068e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.337+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5d00f0820 con 0x7fb5d80717c0 2024-01-15T18:05:07.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.341+0000 7fb5d6a00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5b0066940 0x7fb5b0068e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.341+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9724123070 con 0x7f973c0cf7c0 2024-01-15T18:05:07.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.341+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d8067500 con 0x7fb5d80717c0 2024-01-15T18:05:07.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.341+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f120c136540 con 0x7f120c1f2750 2024-01-15T18:05:07.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.341+0000 7fb5d6a00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5b0066940 0x7fb5b0068e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0006fd0 tx=0x7fb5c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.349+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5d00c7080 con 0x7fb5d80717c0 2024-01-15T18:05:07.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.365+0000 7f86ad800640 1 Processor -- start 2024-01-15T18:05:07.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.365+0000 7f86ad800640 1 -- start start 2024-01-15T18:05:07.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.365+0000 7f6b72c00640 1 Processor -- start 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f86ad800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f86a814b110 0x7f86a80cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f86ad800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a80cc8f0 0x7f86a80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f86ad800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a80b30a0 con 0x7f86a8071f70 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f86ad800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a8070340 con 0x7f86a814b110 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f86ad800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8070620 con 0x7f86a80cc8f0 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.365+0000 7f6b72c00640 1 -- start start 2024-01-15T18:05:07.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b72c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6b6c071e90 0x7f6b6c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b72c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c157650 con 0x7f6b6c159b70 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b72c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0b2a20 con 0x7f6b6c157270 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b72c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0b35a0 con 0x7f6b6c071e90 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b6be00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f6b6c157270 0x7f6b6c0b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33238/0 (socket says 172.21.15.143:33238) 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b6be00640 1 -- 172.21.15.143:0/1682270342 learned_addr learned my addr 172.21.15.143:0/1682270342 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b70c00640 1 --2- 172.21.15.143:0/1682270342 >> v2:172.21.15.143:3300/0 conn(0x7f6b6c071e90 0x7f6b6c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b70c00640 1 -- 172.21.15.143:0/1682270342 >> v1:172.21.15.204:6789/0 conn(0x7f6b6c157270 legacy=0x7f6b6c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b6aa00640 1 -- 172.21.15.143:0/1682270342 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322115519 0 0) 0x7f6b6c0b2a20 con 0x7f6b6c157270 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b70c00640 1 -- 172.21.15.143:0/1682270342 >> v1:172.21.15.143:6789/0 conn(0x7f6b6c159b70 legacy=0x7f6b6c070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.369+0000 7f6b70c00640 1 -- 172.21.15.143:0/1682270342 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c160530 con 0x7f6b6c071e90 2024-01-15T18:05:07.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b70c00640 1 --2- 172.21.15.143:0/1682270342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c071e90 0x7f6b6c0b43b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009d90 tx=0x7f6b64030140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e060da646bd8cac server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b6aa00640 1 -- 172.21.15.143:0/1682270342 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b640c0600 con 0x7f6b6c071e90 2024-01-15T18:05:07.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a7400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f86a8071f70 0x7f86a8071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33224/0 (socket says 172.21.15.143:33224) 2024-01-15T18:05:07.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a7400640 1 -- 172.21.15.143:0/66007897 learned_addr learned my addr 172.21.15.143:0/66007897 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:07.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b6aa00640 1 -- 172.21.15.143:0/1682270342 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b640bc050 con 0x7f6b6c071e90 2024-01-15T18:05:07.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b6aa00640 1 -- 172.21.15.143:0/1682270342 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b640c0980 con 0x7f6b6c071e90 2024-01-15T18:05:07.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a6000640 1 --2- 172.21.15.143:0/66007897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a80cc8f0 0x7f86a80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a6a00640 1 --2- 172.21.15.143:0/66007897 >> v2:172.21.15.204:3300/0 conn(0x7f86a814b110 0x7f86a80cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a6000640 1 -- 172.21.15.143:0/66007897 >> v2:172.21.15.204:3300/0 conn(0x7f86a814b110 msgr2=0x7f86a80cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a6000640 1 --2- 172.21.15.143:0/66007897 >> v2:172.21.15.204:3300/0 conn(0x7f86a814b110 0x7f86a80cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a6000640 1 -- 172.21.15.143:0/66007897 >> v1:172.21.15.204:6789/0 conn(0x7f86a8071f70 legacy=0x7f86a8071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.373+0000 7f86a6000640 1 -- 172.21.15.143:0/66007897 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a8070900 con 0x7f86a80cc8f0 2024-01-15T18:05:07.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f86a6000640 1 --2- 172.21.15.143:0/66007897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a80cc8f0 0x7f86a80b3a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009d90 tx=0x7f86a001c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5591533899995633 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 -- 172.21.15.143:0/1682270342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c071e90 msgr2=0x7f6b6c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f86a5600640 1 -- 172.21.15.143:0/66007897 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a00a5770 con 0x7f86a80cc8f0 2024-01-15T18:05:07.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f86a5600640 1 -- 172.21.15.143:0/66007897 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a00a58d0 con 0x7f86a80cc8f0 2024-01-15T18:05:07.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 --2- 172.21.15.143:0/1682270342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c071e90 0x7f6b6c0b43b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009d90 tx=0x7f6b64030140 comp rx=0 tx=0).stop 2024-01-15T18:05:07.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 -- 172.21.15.143:0/1682270342 shutdown_connections 2024-01-15T18:05:07.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 --2- 172.21.15.143:0/1682270342 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c071e90 0x7f6b6c0b43b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f86ad800640 1 -- 172.21.15.143:0/66007897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a80cc8f0 msgr2=0x7f86a80b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f86a5600640 1 -- 172.21.15.143:0/66007897 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a00a5770 con 0x7f86a80cc8f0 2024-01-15T18:05:07.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f86ad800640 1 --2- 172.21.15.143:0/66007897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a80cc8f0 0x7f86a80b3a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009d90 tx=0x7f86a001c770 comp rx=0 tx=0).stop 2024-01-15T18:05:07.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 -- 172.21.15.143:0/1682270342 >> 172.21.15.143:0/1682270342 conn(0x7f6b6c06c3f0 msgr2=0x7f6b6c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 -- 172.21.15.143:0/1682270342 shutdown_connections 2024-01-15T18:05:07.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.377+0000 7f6b72c00640 1 -- 172.21.15.143:0/1682270342 wait complete. 2024-01-15T18:05:07.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f86ad800640 1 -- 172.21.15.143:0/66007897 shutdown_connections 2024-01-15T18:05:07.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f86ad800640 1 --2- 172.21.15.143:0/66007897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a80cc8f0 0x7f86a80b3a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f86ad800640 1 --2- 172.21.15.143:0/66007897 >> v2:172.21.15.204:3300/0 conn(0x7f86a814b110 0x7f86a80cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f86ad800640 1 -- 172.21.15.143:0/66007897 >> 172.21.15.143:0/66007897 conn(0x7f86a806c3f0 msgr2=0x7f86a806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f86ad800640 1 -- 172.21.15.143:0/66007897 shutdown_connections 2024-01-15T18:05:07.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f86ad800640 1 -- 172.21.15.143:0/66007897 wait complete. 2024-01-15T18:05:07.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.381+0000 7f6b72c00640 1 Processor -- start 2024-01-15T18:05:07.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 -- start start 2024-01-15T18:05:07.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f86ad800640 1 Processor -- start 2024-01-15T18:05:07.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f86ad800640 1 -- start start 2024-01-15T18:05:07.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b6c071e90 0x7f6b6c13e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c157270 0x7f6b6c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b6c159b70 0x7f6b6c144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c159fd0 con 0x7f6b6c157270 2024-01-15T18:05:07.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c15e310 con 0x7f6b6c071e90 2024-01-15T18:05:07.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b6c159b70 0x7f6b6c144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b6c159b70 0x7f6b6c144320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55466/0 (socket says 172.21.15.143:55466) 2024-01-15T18:05:07.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b70c00640 1 -- 172.21.15.143:0/493933099 learned_addr learned my addr 172.21.15.143:0/493933099 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c15e5f0 con 0x7f6b6c159b70 2024-01-15T18:05:07.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.385+0000 7f86ad800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a8071f70 0x7f86a8162a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6b400640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c157270 0x7f6b6c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86ad800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86a80cc8f0 0x7f86a8156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6be00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b6c071e90 0x7f6b6c13e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a8071f70 0x7f86a8162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a8071f70 0x7f86a8162a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55478/0 (socket says 172.21.15.143:55478) 2024-01-15T18:05:07.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6a00640 1 -- 172.21.15.143:0/857511214 learned_addr learned my addr 172.21.15.143:0/857511214 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6b400640 1 -- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b6c159b70 msgr2=0x7f6b6c144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6b400640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b6c159b70 0x7f6b6c144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6b400640 1 -- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b6c071e90 msgr2=0x7f6b6c13e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6b400640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b6c071e90 0x7f6b6c13e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6b400640 1 -- 172.21.15.143:0/493933099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64009a70 con 0x7f6b6c157270 2024-01-15T18:05:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f6b6be00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b6c071e90 0x7f6b6c13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:07.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86a80cc8f0 0x7f86a8156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a814b110 0x7f86a8071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8161c40 con 0x7f86a814b110 2024-01-15T18:05:07.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8161f20 con 0x7f86a80cc8f0 2024-01-15T18:05:07.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a8162200 con 0x7f86a8071f70 2024-01-15T18:05:07.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 -- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a8071f70 msgr2=0x7f86a8162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a8071f70 0x7f86a8162a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 -- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a814b110 msgr2=0x7f86a8071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a814b110 0x7f86a8071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0009a70 con 0x7f86a80cc8f0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.389+0000 7f86a6000640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86a80cc8f0 0x7f86a8156bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86a00ac000 tx=0x7f86a0019510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0004250 con 0x7f86a80cc8f0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a00043b0 con 0x7f86a80cc8f0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a0004740 con 0x7f86a80cc8f0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a81624e0 con 0x7f86a80cc8f0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a8157300 con 0x7f86a80cc8f0 2024-01-15T18:05:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a8067500 con 0x7f86a80cc8f0 2024-01-15T18:05:07.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f6b6b400640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c157270 0x7f6b6c15d9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c390 tx=0x7f6b5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f86a00b9450 con 0x7f86a80cc8f0 2024-01-15T18:05:07.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.397+0000 7f869ea00640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86800668f0 0x7f8680068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.397+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a011f9a0 con 0x7f86a80cc8f0 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54005220 con 0x7f6b6c157270 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c15e8d0 con 0x7f6b6c157270 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c1425f0 con 0x7f6b6c157270 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.393+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54005380 con 0x7f6b6c157270 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.397+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54005220 con 0x7f6b6c157270 2024-01-15T18:05:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.397+0000 7f86a6a00640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86800668f0 0x7f8680068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.397+0000 7f86a6a00640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86800668f0 0x7f8680068db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f869800aaf0 tx=0x7f8698008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.397+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c005260 con 0x7f6b6c157270 2024-01-15T18:05:07.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.401+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86a00ecbd0 con 0x7f86a80cc8f0 2024-01-15T18:05:07.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.401+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f6b54007030 con 0x7f6b6c157270 2024-01-15T18:05:07.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.401+0000 7f6b63e00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6b440669b0 0x7f6b44068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.401+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b54083180 con 0x7f6b6c157270 2024-01-15T18:05:07.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.401+0000 7f6b6be00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6b440669b0 0x7f6b44068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.405+0000 7f6b6be00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6b440669b0 0x7f6b44068e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c007fd0 tx=0x7f6b5c007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.413+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6b54050480 con 0x7f6b6c157270 2024-01-15T18:05:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.581+0000 7f21ab400640 1 -- 172.21.15.143:0/3026577066 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f21bc150ab0 con 0x7f21bc15a440 2024-01-15T18:05:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.585+0000 7f21b2000640 1 -- 172.21.15.143:0/3026577066 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f21a4017070 con 0x7f21bc15a440 2024-01-15T18:05:07.589 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21940669e0 msgr2=0x7f2194068ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21940669e0 0x7f2194068ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f21bc15d580 tx=0x7f21b40a7070 comp rx=0 tx=0).stop 2024-01-15T18:05:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21bc15a440 msgr2=0x7f21bc15c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21bc15a440 0x7f21bc15c9f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f21a40017b0 tx=0x7f21a400c960 comp rx=0 tx=0).stop 2024-01-15T18:05:07.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 shutdown_connections 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21940669e0 0x7f2194068ea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21bc15a440 0x7f21bc15c9f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21bc0cd9e0 0x7f21bc14f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 --2- 172.21.15.143:0/3026577066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21bc06db20 0x7f21bc163aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 >> 172.21.15.143:0/3026577066 conn(0x7f21bc0aea10 msgr2=0x7f21bc0b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 shutdown_connections 2024-01-15T18:05:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.589+0000 7f21c1000640 1 -- 172.21.15.143:0/3026577066 wait complete. 2024-01-15T18:05:07.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-01-15T18:05:07.629 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.633+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb1a415d430 con 0x7fb1a4202d70 2024-01-15T18:05:07.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.633+0000 7fb19a000640 1 -- 172.21.15.143:0/2767401618 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1940ec400 con 0x7fb1a4202d70 2024-01-15T18:05:07.641 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:07.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb17c066940 msgr2=0x7fb17c068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb17c066940 0x7fb17c068e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40ac8b0 tx=0x7fb18c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:07.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 msgr2=0x7fb1a40cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 0x7fb1a40cb8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb19401c880 tx=0x7fb1940197f0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 shutdown_connections 2024-01-15T18:05:07.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb17c066940 0x7fb17c068e00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb1a41f48c0 0x7fb1a41f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1a40cd410 0x7fb1a40cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 --2- 172.21.15.143:0/2767401618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb1a4202d70 0x7fb1a40cb8b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 >> 172.21.15.143:0/2767401618 conn(0x7fb1a40b0bf0 msgr2=0x7fb1a4071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 shutdown_connections 2024-01-15T18:05:07.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.637+0000 7fb1a8c00640 1 -- 172.21.15.143:0/2767401618 wait complete. 2024-01-15T18:05:07.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.653+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f973c06ae80 con 0x7f973c0cf7c0 2024-01-15T18:05:07.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.653+0000 7f9732a00640 1 -- 172.21.15.143:0/1305937599 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97240ec0a0 con 0x7f973c0cf7c0 2024-01-15T18:05:07.661 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f500806e860 con 0x7f50081469b0 2024-01-15T18:05:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97140669c0 msgr2=0x7f9714068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97140669c0 0x7f9714068e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f973400a510 tx=0x7f973400d040 comp rx=0 tx=0).stop 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 msgr2=0x7f973c144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 0x7f973c144850 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97240ac040 tx=0x7f972401ad30 comp rx=0 tx=0).stop 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 shutdown_connections 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97140669c0 0x7f9714068e80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f973c14b1e0 0x7f973c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f973c0cf7c0 0x7f973c144850 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 --2- 172.21.15.143:0/1305937599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f973c070e00 0x7f973c13f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 >> 172.21.15.143:0/1305937599 conn(0x7f973c06a890 msgr2=0x7f973c148710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 shutdown_connections 2024-01-15T18:05:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f9741a00640 1 -- 172.21.15.143:0/1305937599 wait complete. 2024-01-15T18:05:07.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.657+0000 7f4ffe000640 1 -- 172.21.15.143:0/3746756366 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ff0054d70 con 0x7f50081469b0 2024-01-15T18:05:07.665 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe00669c0 msgr2=0x7f4fe0068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe00669c0 0x7f4fe0068e80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ff80ae000 tx=0x7f4ff8004740 comp rx=0 tx=0).stop 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 msgr2=0x7f50080cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080cc7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000c7f0 tx=0x7f4ff000ccb0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 shutdown_connections 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe00669c0 0x7f4fe0068e80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5008160c60 0x7f500815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50081469b0 0x7f50080cc7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 --2- 172.21.15.143:0/3746756366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50080b2f20 0x7f50080cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 >> 172.21.15.143:0/3746756366 conn(0x7f50080b0b10 msgr2=0x7f500815e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 shutdown_connections 2024-01-15T18:05:07.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.661+0000 7f5007200640 1 -- 172.21.15.143:0/3746756366 wait complete. 2024-01-15T18:05:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.665+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f120c136540 con 0x7f120c1f2750 2024-01-15T18:05:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.665+0000 7f1203400640 1 -- 172.21.15.143:0/2422754649 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f120c136540 con 0x7f120c1f2750 2024-01-15T18:05:07.671 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f11e4066940 msgr2=0x7f11e4068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f11e4066940 0x7f11e4068e00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11fc004880 tx=0x7f11fc001120 comp rx=0 tx=0).stop 2024-01-15T18:05:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f120c1f2750 msgr2=0x7f120c1f2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f120c1f2750 0x7f120c1f2be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f120c0aeb30 tx=0x7f11ec005800 comp rx=0 tx=0).stop 2024-01-15T18:05:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 shutdown_connections 2024-01-15T18:05:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f11e4066940 0x7f11e4068e00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f120c1f2750 0x7f120c1f2be0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f120c15c660 0x7f120c0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 --2- 172.21.15.143:0/2422754649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11ec005190 0x7f120c154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 >> 172.21.15.143:0/2422754649 conn(0x7f120c06a890 msgr2=0x7f120c13be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 shutdown_connections 2024-01-15T18:05:07.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.669+0000 7f1212400640 1 -- 172.21.15.143:0/2422754649 wait complete. 2024-01-15T18:05:07.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-01-15T18:05:07.678 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.677+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb5d8149900 con 0x7fb5d80717c0 2024-01-15T18:05:07.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.677+0000 7fb5cf400640 1 -- 172.21.15.143:0/2322579885 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5d00b2020 con 0x7fb5d80717c0 2024-01-15T18:05:07.685 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:07.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5b0066940 msgr2=0x7fb5b0068e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5b0066940 0x7fb5b0068e00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0006fd0 tx=0x7fb5c0008040 comp rx=0 tx=0).stop 2024-01-15T18:05:07.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 msgr2=0x7fb5d814bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 0x7fb5d814bc70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00b0040 tx=0x7fb5d0019540 comp rx=0 tx=0).stop 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 shutdown_connections 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5b0066940 0x7fb5b0068e00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb5d8162490 0x7fb5d814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d80cb430 0x7fb5d815edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 --2- 172.21.15.143:0/2322579885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5d80717c0 0x7fb5d814bc70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.681+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 >> 172.21.15.143:0/2322579885 conn(0x7fb5d806a890 msgr2=0x7fb5d8163ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.685+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 shutdown_connections 2024-01-15T18:05:07.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.685+0000 7fb5ccc00640 1 -- 172.21.15.143:0/2322579885 wait complete. 2024-01-15T18:05:07.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.689+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f86a8071d50 con 0x7f86a80cc8f0 2024-01-15T18:05:07.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.689+0000 7f869ea00640 1 -- 172.21.15.143:0/857511214 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f86a00f01f0 con 0x7f86a80cc8f0 2024-01-15T18:05:07.696 INFO:teuthology.orchestra.run.smithi143.stdout:55834574854 2024-01-15T18:05:07.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86800668f0 msgr2=0x7f8680068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86800668f0 0x7f8680068db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f869800aaf0 tx=0x7f8698008040 comp rx=0 tx=0).stop 2024-01-15T18:05:07.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86a80cc8f0 msgr2=0x7f86a8156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86a80cc8f0 0x7f86a8156bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86a00ac000 tx=0x7f86a0019510 comp rx=0 tx=0).stop 2024-01-15T18:05:07.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 shutdown_connections 2024-01-15T18:05:07.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86800668f0 0x7f8680068db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a814b110 0x7f86a8071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86a80cc8f0 0x7f86a8156bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 --2- 172.21.15.143:0/857511214 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86a8071f70 0x7f86a8162a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 >> 172.21.15.143:0/857511214 conn(0x7f86a806c3f0 msgr2=0x7f86a80cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 shutdown_connections 2024-01-15T18:05:07.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.693+0000 7f86ad800640 1 -- 172.21.15.143:0/857511214 wait complete. 2024-01-15T18:05:07.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-01-15T18:05:07.706 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-01-15T18:05:07.707 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.707 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-01-15T18:05:07.707 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-01-15T18:05:07.723 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-01-15T18:05:07.724 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6b3c005510 con 0x7f6b6c157270 2024-01-15T18:05:07.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b63e00640 1 -- 172.21.15.143:0/493933099 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6b54050480 con 0x7f6b6c157270 2024-01-15T18:05:07.735 INFO:teuthology.orchestra.run.smithi143.stdout:55834574855 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6b440669b0 msgr2=0x7f6b44068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6b440669b0 0x7f6b44068e70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c007fd0 tx=0x7f6b5c007d60 comp rx=0 tx=0).stop 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c157270 msgr2=0x7f6b6c15d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c157270 0x7f6b6c15d9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c390 tx=0x7f6b5400c850 comp rx=0 tx=0).stop 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 shutdown_connections 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6b440669b0 0x7f6b44068e70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6b6c159b70 0x7f6b6c144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6b6c157270 0x7f6b6c15d9e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 --2- 172.21.15.143:0/493933099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6b6c071e90 0x7f6b6c13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.729+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 >> 172.21.15.143:0/493933099 conn(0x7f6b6c06c3f0 msgr2=0x7f6b6c152120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.733+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 shutdown_connections 2024-01-15T18:05:07.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.733+0000 7f6b72c00640 1 -- 172.21.15.143:0/493933099 wait complete. 2024-01-15T18:05:07.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2024-01-15T18:05:07.756 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:07.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-15T18:05:07.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:05:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 Processor -- start 2024-01-15T18:05:07.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 -- start start 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40a8157300 0x7f40a8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40a80b21d0 0x7f40a80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a80b35a0 con 0x7f40a80b21d0 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a80b4d60 con 0x7f40a8157300 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.925+0000 7f40b0400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a80b2a20 con 0x7f40a80b48f0 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40a80b21d0 0x7f40a80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40a80b21d0 0x7f40a80b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48400/0 (socket says 172.21.15.143:48400) 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 -- 172.21.15.143:0/265043497 learned_addr learned my addr 172.21.15.143:0/265043497 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 -- 172.21.15.143:0/265043497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 msgr2=0x7f40a815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ae400640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ada00640 1 --2- 172.21.15.143:0/265043497 >> v2:172.21.15.204:3300/0 conn(0x7f40a8157300 0x7f40a8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 -- 172.21.15.143:0/265043497 >> v2:172.21.15.204:3300/0 conn(0x7f40a8157300 msgr2=0x7f40a8159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 --2- 172.21.15.143:0/265043497 >> v2:172.21.15.204:3300/0 conn(0x7f40a8157300 0x7f40a8159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 -- 172.21.15.143:0/265043497 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a80b2d00 con 0x7f40a80b21d0 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ada00640 1 --2- 172.21.15.143:0/265043497 >> v2:172.21.15.204:3300/0 conn(0x7f40a8157300 0x7f40a8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:07.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ae400640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:07.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40ad000640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a80b43b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40a80b1aa0 tx=0x7f409001d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5995c1e487835c0a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40a7e00640 1 -- 172.21.15.143:0/265043497 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900a6910 con 0x7f40a80b21d0 2024-01-15T18:05:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40a7e00640 1 -- 172.21.15.143:0/265043497 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40900a6a70 con 0x7f40a80b21d0 2024-01-15T18:05:07.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.929+0000 7f40a7e00640 1 -- 172.21.15.143:0/265043497 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900a6e00 con 0x7f40a80b21d0 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- 172.21.15.143:0/265043497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 msgr2=0x7f40a80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a80b43b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40a80b1aa0 tx=0x7f409001d8a0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- 172.21.15.143:0/265043497 shutdown_connections 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- 172.21.15.143:0/265043497 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a80b43b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- 172.21.15.143:0/265043497 >> v2:172.21.15.204:3300/0 conn(0x7f40a8157300 0x7f40a8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- 172.21.15.143:0/265043497 >> 172.21.15.143:0/265043497 conn(0x7f40a806c3f0 msgr2=0x7f40a806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- 172.21.15.143:0/265043497 shutdown_connections 2024-01-15T18:05:07.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- 172.21.15.143:0/265043497 wait complete. 2024-01-15T18:05:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 Processor -- start 2024-01-15T18:05:07.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- start start 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40a8157300 0x7f40a814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8155b70 con 0x7f40a80b21d0 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8155e50 con 0x7f40a8157300 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40b0400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8156130 con 0x7f40a80b48f0 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40ada00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40ada00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48402/0 (socket says 172.21.15.143:48402) 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.933+0000 7f40ada00640 1 -- 172.21.15.143:0/409692637 learned_addr learned my addr 172.21.15.143:0/409692637 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ad000640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ae400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40a8157300 0x7f40a814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ada00640 1 -- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 msgr2=0x7f40a8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ada00640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a8155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ada00640 1 -- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40a8157300 msgr2=0x7f40a814ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ada00640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40a8157300 0x7f40a814ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ada00640 1 -- 172.21.15.143:0/409692637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409000a670 con 0x7f40a80b21d0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ae400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40a8157300 0x7f40a814ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40ada00640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a815b930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f409800c7f0 tx=0x7f409800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098007820 con 0x7f40a80b21d0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098007980 con 0x7f40a80b21d0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098007d30 con 0x7f40a80b21d0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a8156410 con 0x7f40a80b21d0 2024-01-15T18:05:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a814e460 con 0x7f40a80b21d0 2024-01-15T18:05:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a8067500 con 0x7f40a80b21d0 2024-01-15T18:05:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f40980113f0 con 0x7f40a80b21d0 2024-01-15T18:05:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40a5600640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40800668f0 0x7f4080068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.937+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4098084690 con 0x7f40a80b21d0 2024-01-15T18:05:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.941+0000 7f40ad000640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40800668f0 0x7f4080068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.941+0000 7f40ad000640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40800668f0 0x7f4080068db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f40900b0000 tx=0x7f4090008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:07.945+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40980518e0 con 0x7f40a80b21d0 2024-01-15T18:05:08.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.089+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40a806e4e0 con 0x7f40800668f0 2024-01-15T18:05:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.097+0000 7f40a5600640 1 -- 172.21.15.143:0/409692637 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264536 (secure 0 0 0) 0x7f40a806e4e0 con 0x7f40800668f0 2024-01-15T18:05:08.104 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:08.106 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.097+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40800668f0 msgr2=0x7f4080068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.097+0000 7f40b0400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40800668f0 0x7f4080068db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f40900b0000 tx=0x7f4090008040 comp rx=0 tx=0).stop 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.097+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 msgr2=0x7f40a815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.097+0000 7f40b0400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a815b930 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f409800c7f0 tx=0x7f409800ccb0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 shutdown_connections 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40800668f0 0x7f4080068db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40a8157300 0x7f40a814ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40a80b48f0 0x7f40a8155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 --2- 172.21.15.143:0/409692637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40a80b21d0 0x7f40a815b930 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 >> 172.21.15.143:0/409692637 conn(0x7f40a806c3f0 msgr2=0x7f40a814ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 shutdown_connections 2024-01-15T18:05:08.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.101+0000 7f40b0400640 1 -- 172.21.15.143:0/409692637 wait complete. 2024-01-15T18:05:08.125 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-01-15T18:05:06.622971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001634"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539184+0000","last_change":"2024-01-15T18:04:59.313108+0000","last_active":"2024-01-15T18:05:00.539184+0000","last_peered":"2024-01-15T18:05:00.539184+0000","last_clean":"2024-01-15T18:05:00.539184+0000","last_became_active":"2024-01-15T18:04:59.312739+0000","last_became_peered":"2024-01-15T18:04:59.312739+0000","last_unstale":"2024-01-15T18:05:00.539184+0000","last_undegraded":"2024-01-15T18:05:00.539184+0000","last_fullsized":"2024-01-15T18:05:00.539184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740563+0000","last_change":"2024-01-15T18:04:59.303929+0000","last_active":"2024-01-15T18:05:00.740563+0000","last_peered":"2024-01-15T18:05:00.740563+0000","last_clean":"2024-01-15T18:05:00.740563+0000","last_became_active":"2024-01-15T18:04:59.303598+0000","last_became_peered":"2024-01-15T18:04:59.303598+0000","last_unstale":"2024-01-15T18:05:00.740563+0000","last_undegraded":"2024-01-15T18:05:00.740563+0000","last_fullsized":"2024-01-15T18:05:00.740563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740178+0000","last_change":"2024-01-15T18:04:59.298035+0000","last_active":"2024-01-15T18:05:00.740178+0000","last_peered":"2024-01-15T18:05:00.740178+0000","last_clean":"2024-01-15T18:05:00.740178+0000","last_became_active":"2024-01-15T18:04:59.297748+0000","last_became_peered":"2024-01-15T18:04:59.297748+0000","last_unstale":"2024-01-15T18:05:00.740178+0000","last_undegraded":"2024-01-15T18:05:00.740178+0000","last_fullsized":"2024-01-15T18:05:00.740178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.314048+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.313770+0000","last_became_peered":"2024-01-15T18:04:59.313770+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.308757+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.306380+0000","last_became_peered":"2024-01-15T18:04:59.306380+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353031+0000","last_change":"2024-01-15T18:04:59.307868+0000","last_active":"2024-01-15T18:05:00.353031+0000","last_peered":"2024-01-15T18:05:00.353031+0000","last_clean":"2024-01-15T18:05:00.353031+0000","last_became_active":"2024-01-15T18:04:59.307388+0000","last_became_peered":"2024-01-15T18:04:59.307388+0000","last_unstale":"2024-01-15T18:05:00.353031+0000","last_undegraded":"2024-01-15T18:05:00.353031+0000","last_fullsized":"2024-01-15T18:05:00.353031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538844+0000","last_change":"2024-01-15T18:04:59.316726+0000","last_active":"2024-01-15T18:05:00.538844+0000","last_peered":"2024-01-15T18:05:00.538844+0000","last_clean":"2024-01-15T18:05:00.538844+0000","last_became_active":"2024-01-15T18:04:59.311435+0000","last_became_peered":"2024-01-15T18:04:59.311435+0000","last_unstale":"2024-01-15T18:05:00.538844+0000","last_undegraded":"2024-01-15T18:05:00.538844+0000","last_fullsized":"2024-01-15T18:05:00.538844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284988+0000","last_change":"2024-01-15T18:04:59.305582+0000","last_active":"2024-01-15T18:05:00.284988+0000","last_peered":"2024-01-15T18:05:00.284988+0000","last_clean":"2024-01-15T18:05:00.284988+0000","last_became_active":"2024-01-15T18:04:59.305227+0000","last_became_peered":"2024-01-15T18:04:59.305227+0000","last_unstale":"2024-01-15T18:05:00.284988+0000","last_undegraded":"2024-01-15T18:05:00.284988+0000","last_fullsized":"2024-01-15T18:05:00.284988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539386+0000","last_change":"2024-01-15T18:04:59.310510+0000","last_active":"2024-01-15T18:05:00.539386+0000","last_peered":"2024-01-15T18:05:00.539386+0000","last_clean":"2024-01-15T18:05:00.539386+0000","last_became_active":"2024-01-15T18:04:59.310228+0000","last_became_peered":"2024-01-15T18:04:59.310228+0000","last_unstale":"2024-01-15T18:05:00.539386+0000","last_undegraded":"2024-01-15T18:05:00.539386+0000","last_fullsized":"2024-01-15T18:05:00.539386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283959+0000","last_change":"2024-01-15T18:04:59.308904+0000","last_active":"2024-01-15T18:05:00.283959+0000","last_peered":"2024-01-15T18:05:00.283959+0000","last_clean":"2024-01-15T18:05:00.283959+0000","last_became_active":"2024-01-15T18:04:59.308650+0000","last_became_peered":"2024-01-15T18:04:59.308650+0000","last_unstale":"2024-01-15T18:05:00.283959+0000","last_undegraded":"2024-01-15T18:05:00.283959+0000","last_fullsized":"2024-01-15T18:05:00.283959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283572+0000","last_change":"2024-01-15T18:04:59.311700+0000","last_active":"2024-01-15T18:05:00.283572+0000","last_peered":"2024-01-15T18:05:00.283572+0000","last_clean":"2024-01-15T18:05:00.283572+0000","last_became_active":"2024-01-15T18:04:59.311406+0000","last_became_peered":"2024-01-15T18:04:59.311406+0000","last_unstale":"2024-01-15T18:05:00.283572+0000","last_undegraded":"2024-01-15T18:05:00.283572+0000","last_fullsized":"2024-01-15T18:05:00.283572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284074+0000","last_change":"2024-01-15T18:04:59.287898+0000","last_active":"2024-01-15T18:05:00.284074+0000","last_peered":"2024-01-15T18:05:00.284074+0000","last_clean":"2024-01-15T18:05:00.284074+0000","last_became_active":"2024-01-15T18:04:59.287554+0000","last_became_peered":"2024-01-15T18:04:59.287554+0000","last_unstale":"2024-01-15T18:05:00.284074+0000","last_undegraded":"2024-01-15T18:05:00.284074+0000","last_fullsized":"2024-01-15T18:05:00.284074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:43.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285461+0000","last_change":"2024-01-15T18:04:59.303813+0000","last_active":"2024-01-15T18:05:00.285461+0000","last_peered":"2024-01-15T18:05:00.285461+0000","last_clean":"2024-01-15T18:05:00.285461+0000","last_became_active":"2024-01-15T18:04:59.303549+0000","last_became_peered":"2024-01-15T18:04:59.303549+0000","last_unstale":"2024-01-15T18:05:00.285461+0000","last_undegraded":"2024-01-15T18:05:00.285461+0000","last_fullsized":"2024-01-15T18:05:00.285461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739952+0000","last_change":"2024-01-15T18:04:59.301393+0000","last_active":"2024-01-15T18:05:00.739952+0000","last_peered":"2024-01-15T18:05:00.739952+0000","last_clean":"2024-01-15T18:05:00.739952+0000","last_became_active":"2024-01-15T18:04:59.300604+0000","last_became_peered":"2024-01-15T18:04:59.300604+0000","last_unstale":"2024-01-15T18:05:00.739952+0000","last_undegraded":"2024-01-15T18:05:00.739952+0000","last_fullsized":"2024-01-15T18:05:00.739952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740231+0000","last_change":"2024-01-15T18:04:59.311581+0000","last_active":"2024-01-15T18:05:00.740231+0000","last_peered":"2024-01-15T18:05:00.740231+0000","last_clean":"2024-01-15T18:05:00.740231+0000","last_became_active":"2024-01-15T18:04:59.311309+0000","last_became_peered":"2024-01-15T18:04:59.311309+0000","last_unstale":"2024-01-15T18:05:00.740231+0000","last_undegraded":"2024-01-15T18:05:00.740231+0000","last_fullsized":"2024-01-15T18:05:00.740231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353034+0000","last_change":"2024-01-15T18:04:59.307600+0000","last_active":"2024-01-15T18:05:00.353034+0000","last_peered":"2024-01-15T18:05:00.353034+0000","last_clean":"2024-01-15T18:05:00.353034+0000","last_became_active":"2024-01-15T18:04:59.307160+0000","last_became_peered":"2024-01-15T18:04:59.307160+0000","last_unstale":"2024-01-15T18:05:00.353034+0000","last_undegraded":"2024-01-15T18:05:00.353034+0000","last_fullsized":"2024-01-15T18:05:00.353034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353453+0000","last_change":"2024-01-15T18:04:59.316075+0000","last_active":"2024-01-15T18:05:00.353453+0000","last_peered":"2024-01-15T18:05:00.353453+0000","last_clean":"2024-01-15T18:05:00.353453+0000","last_became_active":"2024-01-15T18:04:59.315751+0000","last_became_peered":"2024-01-15T18:04:59.315751+0000","last_unstale":"2024-01-15T18:05:00.353453+0000","last_undegraded":"2024-01-15T18:05:00.353453+0000","last_fullsized":"2024-01-15T18:05:00.353453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739317+0000","last_change":"2024-01-15T18:04:59.305938+0000","last_active":"2024-01-15T18:05:00.739317+0000","last_peered":"2024-01-15T18:05:00.739317+0000","last_clean":"2024-01-15T18:05:00.739317+0000","last_became_active":"2024-01-15T18:04:59.305686+0000","last_became_peered":"2024-01-15T18:04:59.305686+0000","last_unstale":"2024-01-15T18:05:00.739317+0000","last_undegraded":"2024-01-15T18:05:00.739317+0000","last_fullsized":"2024-01-15T18:05:00.739317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:35.095703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539274+0000","last_change":"2024-01-15T18:04:59.306518+0000","last_active":"2024-01-15T18:05:00.539274+0000","last_peered":"2024-01-15T18:05:00.539274+0000","last_clean":"2024-01-15T18:05:00.539274+0000","last_became_active":"2024-01-15T18:04:59.306009+0000","last_became_peered":"2024-01-15T18:04:59.306009+0000","last_unstale":"2024-01-15T18:05:00.539274+0000","last_undegraded":"2024-01-15T18:05:00.539274+0000","last_fullsized":"2024-01-15T18:05:00.539274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283820+0000","last_change":"2024-01-15T18:04:59.296492+0000","last_active":"2024-01-15T18:05:00.283820+0000","last_peered":"2024-01-15T18:05:00.283820+0000","last_clean":"2024-01-15T18:05:00.283820+0000","last_became_active":"2024-01-15T18:04:59.296200+0000","last_became_peered":"2024-01-15T18:04:59.296200+0000","last_unstale":"2024-01-15T18:05:00.283820+0000","last_undegraded":"2024-01-15T18:05:00.283820+0000","last_fullsized":"2024-01-15T18:05:00.283820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.300916+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.300453+0000","last_became_peered":"2024-01-15T18:04:59.300453+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:50.361044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353117+0000","last_change":"2024-01-15T18:04:59.313753+0000","last_active":"2024-01-15T18:05:00.353117+0000","last_peered":"2024-01-15T18:05:00.353117+0000","last_clean":"2024-01-15T18:05:00.353117+0000","last_became_active":"2024-01-15T18:04:59.313404+0000","last_became_peered":"2024-01-15T18:04:59.313404+0000","last_unstale":"2024-01-15T18:05:00.353117+0000","last_undegraded":"2024-01-15T18:05:00.353117+0000","last_fullsized":"2024-01-15T18:05:00.353117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285121+0000","last_change":"2024-01-15T18:04:59.297113+0000","last_active":"2024-01-15T18:05:00.285121+0000","last_peered":"2024-01-15T18:05:00.285121+0000","last_clean":"2024-01-15T18:05:00.285121+0000","last_became_active":"2024-01-15T18:04:59.296920+0000","last_became_peered":"2024-01-15T18:04:59.296920+0000","last_unstale":"2024-01-15T18:05:00.285121+0000","last_undegraded":"2024-01-15T18:05:00.285121+0000","last_fullsized":"2024-01-15T18:05:00.285121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353148+0000","last_change":"2024-01-15T18:04:59.307789+0000","last_active":"2024-01-15T18:05:00.353148+0000","last_peered":"2024-01-15T18:05:00.353148+0000","last_clean":"2024-01-15T18:05:00.353148+0000","last_became_active":"2024-01-15T18:04:59.306606+0000","last_became_peered":"2024-01-15T18:04:59.306606+0000","last_unstale":"2024-01-15T18:05:00.353148+0000","last_undegraded":"2024-01-15T18:05:00.353148+0000","last_fullsized":"2024-01-15T18:05:00.353148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285732+0000","last_change":"2024-01-15T18:04:59.305088+0000","last_active":"2024-01-15T18:05:00.285732+0000","last_peered":"2024-01-15T18:05:00.285732+0000","last_clean":"2024-01-15T18:05:00.285732+0000","last_became_active":"2024-01-15T18:04:59.304735+0000","last_became_peered":"2024-01-15T18:04:59.304735+0000","last_unstale":"2024-01-15T18:05:00.285732+0000","last_undegraded":"2024-01-15T18:05:00.285732+0000","last_fullsized":"2024-01-15T18:05:00.285732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284011+0000","last_change":"2024-01-15T18:04:59.311649+0000","last_active":"2024-01-15T18:05:00.284011+0000","last_peered":"2024-01-15T18:05:00.284011+0000","last_clean":"2024-01-15T18:05:00.284011+0000","last_became_active":"2024-01-15T18:04:59.311225+0000","last_became_peered":"2024-01-15T18:04:59.311225+0000","last_unstale":"2024-01-15T18:05:00.284011+0000","last_undegraded":"2024-01-15T18:05:00.284011+0000","last_fullsized":"2024-01-15T18:05:00.284011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.306001+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.305732+0000","last_became_peered":"2024-01-15T18:04:59.305732+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285595+0000","last_change":"2024-01-15T18:04:59.306947+0000","last_active":"2024-01-15T18:05:00.285595+0000","last_peered":"2024-01-15T18:05:00.285595+0000","last_clean":"2024-01-15T18:05:00.285595+0000","last_became_active":"2024-01-15T18:04:59.306617+0000","last_became_peered":"2024-01-15T18:04:59.306617+0000","last_unstale":"2024-01-15T18:05:00.285595+0000","last_undegraded":"2024-01-15T18:05:00.285595+0000","last_fullsized":"2024-01-15T18:05:00.285595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.303640+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.303076+0000","last_became_peered":"2024-01-15T18:04:59.303076+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:45.894125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353181+0000","last_change":"2024-01-15T18:04:59.307697+0000","last_active":"2024-01-15T18:05:00.353181+0000","last_peered":"2024-01-15T18:05:00.353181+0000","last_clean":"2024-01-15T18:05:00.353181+0000","last_became_active":"2024-01-15T18:04:59.305330+0000","last_became_peered":"2024-01-15T18:04:59.305330+0000","last_unstale":"2024-01-15T18:05:00.353181+0000","last_undegraded":"2024-01-15T18:05:00.353181+0000","last_fullsized":"2024-01-15T18:05:00.353181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739155+0000","last_change":"2024-01-15T18:04:59.305380+0000","last_active":"2024-01-15T18:05:00.739155+0000","last_peered":"2024-01-15T18:05:00.739155+0000","last_clean":"2024-01-15T18:05:00.739155+0000","last_became_active":"2024-01-15T18:04:59.305090+0000","last_became_peered":"2024-01-15T18:04:59.305090+0000","last_unstale":"2024-01-15T18:05:00.739155+0000","last_undegraded":"2024-01-15T18:05:00.739155+0000","last_fullsized":"2024-01-15T18:05:00.739155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283376+0000","last_change":"2024-01-15T18:04:59.294688+0000","last_active":"2024-01-15T18:05:00.283376+0000","last_peered":"2024-01-15T18:05:00.283376+0000","last_clean":"2024-01-15T18:05:00.283376+0000","last_became_active":"2024-01-15T18:04:59.294065+0000","last_became_peered":"2024-01-15T18:04:59.294065+0000","last_unstale":"2024-01-15T18:05:00.283376+0000","last_undegraded":"2024-01-15T18:05:00.283376+0000","last_fullsized":"2024-01-15T18:05:00.283376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283129+0000","last_change":"2024-01-15T18:04:59.300834+0000","last_active":"2024-01-15T18:05:00.283129+0000","last_peered":"2024-01-15T18:05:00.283129+0000","last_clean":"2024-01-15T18:05:00.283129+0000","last_became_active":"2024-01-15T18:04:59.300257+0000","last_became_peered":"2024-01-15T18:04:59.300257+0000","last_unstale":"2024-01-15T18:05:00.283129+0000","last_undegraded":"2024-01-15T18:05:00.283129+0000","last_fullsized":"2024-01-15T18:05:00.283129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:31.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285696+0000","last_change":"2024-01-15T18:04:58.298585+0000","last_active":"2024-01-15T18:05:00.285696+0000","last_peered":"2024-01-15T18:05:00.285696+0000","last_clean":"2024-01-15T18:05:00.285696+0000","last_became_active":"2024-01-15T18:04:58.298288+0000","last_became_peered":"2024-01-15T18:04:58.298288+0000","last_unstale":"2024-01-15T18:05:00.285696+0000","last_undegraded":"2024-01-15T18:05:00.285696+0000","last_fullsized":"2024-01-15T18:05:00.285696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740106+0000","last_change":"2024-01-15T18:04:59.295504+0000","last_active":"2024-01-15T18:05:00.740106+0000","last_peered":"2024-01-15T18:05:00.740106+0000","last_clean":"2024-01-15T18:05:00.740106+0000","last_became_active":"2024-01-15T18:04:59.295205+0000","last_became_peered":"2024-01-15T18:04:59.295205+0000","last_unstale":"2024-01-15T18:05:00.740106+0000","last_undegraded":"2024-01-15T18:05:00.740106+0000","last_fullsized":"2024-01-15T18:05:00.740106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283737+0000","last_change":"2024-01-15T18:04:58.296152+0000","last_active":"2024-01-15T18:05:00.283737+0000","last_peered":"2024-01-15T18:05:00.283737+0000","last_clean":"2024-01-15T18:05:00.283737+0000","last_became_active":"2024-01-15T18:04:58.295329+0000","last_became_peered":"2024-01-15T18:04:58.295329+0000","last_unstale":"2024-01-15T18:05:00.283737+0000","last_undegraded":"2024-01-15T18:05:00.283737+0000","last_fullsized":"2024-01-15T18:05:00.283737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285163+0000","last_change":"2024-01-15T18:04:58.291634+0000","last_active":"2024-01-15T18:05:00.285163+0000","last_peered":"2024-01-15T18:05:00.285163+0000","last_clean":"2024-01-15T18:05:00.285163+0000","last_became_active":"2024-01-15T18:04:58.290715+0000","last_became_peered":"2024-01-15T18:04:58.290715+0000","last_unstale":"2024-01-15T18:05:00.285163+0000","last_undegraded":"2024-01-15T18:05:00.285163+0000","last_fullsized":"2024-01-15T18:05:00.285163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285067+0000","last_change":"2024-01-15T18:04:59.302779+0000","last_active":"2024-01-15T18:05:00.285067+0000","last_peered":"2024-01-15T18:05:00.285067+0000","last_clean":"2024-01-15T18:05:00.285067+0000","last_became_active":"2024-01-15T18:04:59.302387+0000","last_became_peered":"2024-01-15T18:04:59.302387+0000","last_unstale":"2024-01-15T18:05:00.285067+0000","last_undegraded":"2024-01-15T18:05:00.285067+0000","last_fullsized":"2024-01-15T18:05:00.285067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353124+0000","last_change":"2024-01-15T18:04:58.285848+0000","last_active":"2024-01-15T18:05:00.353124+0000","last_peered":"2024-01-15T18:05:00.353124+0000","last_clean":"2024-01-15T18:05:00.353124+0000","last_became_active":"2024-01-15T18:04:58.283781+0000","last_became_peered":"2024-01-15T18:04:58.283781+0000","last_unstale":"2024-01-15T18:05:00.353124+0000","last_undegraded":"2024-01-15T18:05:00.353124+0000","last_fullsized":"2024-01-15T18:05:00.353124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740140+0000","last_change":"2024-01-15T18:04:59.306022+0000","last_active":"2024-01-15T18:05:00.740140+0000","last_peered":"2024-01-15T18:05:00.740140+0000","last_clean":"2024-01-15T18:05:00.740140+0000","last_became_active":"2024-01-15T18:04:59.305785+0000","last_became_peered":"2024-01-15T18:04:59.305785+0000","last_unstale":"2024-01-15T18:05:00.740140+0000","last_undegraded":"2024-01-15T18:05:00.740140+0000","last_fullsized":"2024-01-15T18:05:00.740140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283482+0000","last_change":"2024-01-15T18:04:58.292333+0000","last_active":"2024-01-15T18:05:00.283482+0000","last_peered":"2024-01-15T18:05:00.283482+0000","last_clean":"2024-01-15T18:05:00.283482+0000","last_became_active":"2024-01-15T18:04:58.291543+0000","last_became_peered":"2024-01-15T18:04:58.291543+0000","last_unstale":"2024-01-15T18:05:00.283482+0000","last_undegraded":"2024-01-15T18:05:00.283482+0000","last_fullsized":"2024-01-15T18:05:00.283482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353424+0000","last_change":"2024-01-15T18:04:59.313691+0000","last_active":"2024-01-15T18:05:00.353424+0000","last_peered":"2024-01-15T18:05:00.353424+0000","last_clean":"2024-01-15T18:05:00.353424+0000","last_became_active":"2024-01-15T18:04:59.313395+0000","last_became_peered":"2024-01-15T18:04:59.313395+0000","last_unstale":"2024-01-15T18:05:00.353424+0000","last_undegraded":"2024-01-15T18:05:00.353424+0000","last_fullsized":"2024-01-15T18:05:00.353424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353418+0000","last_change":"2024-01-15T18:04:58.288981+0000","last_active":"2024-01-15T18:05:00.353418+0000","last_peered":"2024-01-15T18:05:00.353418+0000","last_clean":"2024-01-15T18:05:00.353418+0000","last_became_active":"2024-01-15T18:04:58.288667+0000","last_became_peered":"2024-01-15T18:04:58.288667+0000","last_unstale":"2024-01-15T18:05:00.353418+0000","last_undegraded":"2024-01-15T18:05:00.353418+0000","last_fullsized":"2024-01-15T18:05:00.353418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740466+0000","last_change":"2024-01-15T18:04:59.311337+0000","last_active":"2024-01-15T18:05:00.740466+0000","last_peered":"2024-01-15T18:05:00.740466+0000","last_clean":"2024-01-15T18:05:00.740466+0000","last_became_active":"2024-01-15T18:04:59.311102+0000","last_became_peered":"2024-01-15T18:04:59.311102+0000","last_unstale":"2024-01-15T18:05:00.740466+0000","last_undegraded":"2024-01-15T18:05:00.740466+0000","last_fullsized":"2024-01-15T18:05:00.740466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740475+0000","last_change":"2024-01-15T18:04:58.295173+0000","last_active":"2024-01-15T18:05:00.740475+0000","last_peered":"2024-01-15T18:05:00.740475+0000","last_clean":"2024-01-15T18:05:00.740475+0000","last_became_active":"2024-01-15T18:04:58.294883+0000","last_became_peered":"2024-01-15T18:04:58.294883+0000","last_unstale":"2024-01-15T18:05:00.740475+0000","last_undegraded":"2024-01-15T18:05:00.740475+0000","last_fullsized":"2024-01-15T18:05:00.740475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284186+0000","last_change":"2024-01-15T18:04:59.311644+0000","last_active":"2024-01-15T18:05:00.284186+0000","last_peered":"2024-01-15T18:05:00.284186+0000","last_clean":"2024-01-15T18:05:00.284186+0000","last_became_active":"2024-01-15T18:04:59.311224+0000","last_became_peered":"2024-01-15T18:04:59.311224+0000","last_unstale":"2024-01-15T18:05:00.284186+0000","last_undegraded":"2024-01-15T18:05:00.284186+0000","last_fullsized":"2024-01-15T18:05:00.284186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283776+0000","last_change":"2024-01-15T18:04:58.293217+0000","last_active":"2024-01-15T18:05:00.283776+0000","last_peered":"2024-01-15T18:05:00.283776+0000","last_clean":"2024-01-15T18:05:00.283776+0000","last_became_active":"2024-01-15T18:04:58.292902+0000","last_became_peered":"2024-01-15T18:04:58.292902+0000","last_unstale":"2024-01-15T18:05:00.283776+0000","last_undegraded":"2024-01-15T18:05:00.283776+0000","last_fullsized":"2024-01-15T18:05:00.283776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539100+0000","last_change":"2024-01-15T18:04:59.313869+0000","last_active":"2024-01-15T18:05:00.539100+0000","last_peered":"2024-01-15T18:05:00.539100+0000","last_clean":"2024-01-15T18:05:00.539100+0000","last_became_active":"2024-01-15T18:04:59.313336+0000","last_became_peered":"2024-01-15T18:04:59.313336+0000","last_unstale":"2024-01-15T18:05:00.539100+0000","last_undegraded":"2024-01-15T18:05:00.539100+0000","last_fullsized":"2024-01-15T18:05:00.539100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740054+0000","last_change":"2024-01-15T18:04:58.294176+0000","last_active":"2024-01-15T18:05:00.740054+0000","last_peered":"2024-01-15T18:05:00.740054+0000","last_clean":"2024-01-15T18:05:00.740054+0000","last_became_active":"2024-01-15T18:04:58.292524+0000","last_became_peered":"2024-01-15T18:04:58.292524+0000","last_unstale":"2024-01-15T18:05:00.740054+0000","last_undegraded":"2024-01-15T18:05:00.740054+0000","last_fullsized":"2024-01-15T18:05:00.740054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740077+0000","last_change":"2024-01-15T18:04:59.300038+0000","last_active":"2024-01-15T18:05:00.740077+0000","last_peered":"2024-01-15T18:05:00.740077+0000","last_clean":"2024-01-15T18:05:00.740077+0000","last_became_active":"2024-01-15T18:04:59.299776+0000","last_became_peered":"2024-01-15T18:04:59.299776+0000","last_unstale":"2024-01-15T18:05:00.740077+0000","last_undegraded":"2024-01-15T18:05:00.740077+0000","last_fullsized":"2024-01-15T18:05:00.740077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353198+0000","last_change":"2024-01-15T18:04:58.297644+0000","last_active":"2024-01-15T18:05:00.353198+0000","last_peered":"2024-01-15T18:05:00.353198+0000","last_clean":"2024-01-15T18:05:00.353198+0000","last_became_active":"2024-01-15T18:04:58.296922+0000","last_became_peered":"2024-01-15T18:04:58.296922+0000","last_unstale":"2024-01-15T18:05:00.353198+0000","last_undegraded":"2024-01-15T18:05:00.353198+0000","last_fullsized":"2024-01-15T18:05:00.353198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539142+0000","last_change":"2024-01-15T18:04:59.312547+0000","last_active":"2024-01-15T18:05:00.539142+0000","last_peered":"2024-01-15T18:05:00.539142+0000","last_clean":"2024-01-15T18:05:00.539142+0000","last_became_active":"2024-01-15T18:04:59.312203+0000","last_became_peered":"2024-01-15T18:04:59.312203+0000","last_unstale":"2024-01-15T18:05:00.539142+0000","last_undegraded":"2024-01-15T18:05:00.539142+0000","last_fullsized":"2024-01-15T18:05:00.539142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283454+0000","last_change":"2024-01-15T18:04:58.297011+0000","last_active":"2024-01-15T18:05:00.283454+0000","last_peered":"2024-01-15T18:05:00.283454+0000","last_clean":"2024-01-15T18:05:00.283454+0000","last_became_active":"2024-01-15T18:04:58.296476+0000","last_became_peered":"2024-01-15T18:04:58.296476+0000","last_unstale":"2024-01-15T18:05:00.283454+0000","last_undegraded":"2024-01-15T18:05:00.283454+0000","last_fullsized":"2024-01-15T18:05:00.283454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:39.535650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283371+0000","last_change":"2024-01-15T18:04:59.294840+0000","last_active":"2024-01-15T18:05:00.283371+0000","last_peered":"2024-01-15T18:05:00.283371+0000","last_clean":"2024-01-15T18:05:00.283371+0000","last_became_active":"2024-01-15T18:04:59.294449+0000","last_became_peered":"2024-01-15T18:04:59.294449+0000","last_unstale":"2024-01-15T18:05:00.283371+0000","last_undegraded":"2024-01-15T18:05:00.283371+0000","last_fullsized":"2024-01-15T18:05:00.283371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739071+0000","last_change":"2024-01-15T18:04:58.285671+0000","last_active":"2024-01-15T18:05:00.739071+0000","last_peered":"2024-01-15T18:05:00.739071+0000","last_clean":"2024-01-15T18:05:00.739071+0000","last_became_active":"2024-01-15T18:04:58.285350+0000","last_became_peered":"2024-01-15T18:04:58.285350+0000","last_unstale":"2024-01-15T18:05:00.739071+0000","last_undegraded":"2024-01-15T18:05:00.739071+0000","last_fullsized":"2024-01-15T18:05:00.739071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538843+0000","last_change":"2024-01-15T18:04:59.315031+0000","last_active":"2024-01-15T18:05:00.538843+0000","last_peered":"2024-01-15T18:05:00.538843+0000","last_clean":"2024-01-15T18:05:00.538843+0000","last_became_active":"2024-01-15T18:04:59.311772+0000","last_became_peered":"2024-01-15T18:04:59.311772+0000","last_unstale":"2024-01-15T18:05:00.538843+0000","last_undegraded":"2024-01-15T18:05:00.538843+0000","last_fullsized":"2024-01-15T18:05:00.538843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284479+0000","last_change":"2024-01-15T18:04:58.290373+0000","last_active":"2024-01-15T18:05:00.284479+0000","last_peered":"2024-01-15T18:05:00.284479+0000","last_clean":"2024-01-15T18:05:00.284479+0000","last_became_active":"2024-01-15T18:04:58.290060+0000","last_became_peered":"2024-01-15T18:04:58.290060+0000","last_unstale":"2024-01-15T18:05:00.284479+0000","last_undegraded":"2024-01-15T18:05:00.284479+0000","last_fullsized":"2024-01-15T18:05:00.284479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539224+0000","last_change":"2024-01-15T18:04:59.315975+0000","last_active":"2024-01-15T18:05:00.539224+0000","last_peered":"2024-01-15T18:05:00.539224+0000","last_clean":"2024-01-15T18:05:00.539224+0000","last_became_active":"2024-01-15T18:04:59.315667+0000","last_became_peered":"2024-01-15T18:04:59.315667+0000","last_unstale":"2024-01-15T18:05:00.539224+0000","last_undegraded":"2024-01-15T18:05:00.539224+0000","last_fullsized":"2024-01-15T18:05:00.539224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739314+0000","last_change":"2024-01-15T18:04:58.297988+0000","last_active":"2024-01-15T18:05:00.739314+0000","last_peered":"2024-01-15T18:05:00.739314+0000","last_clean":"2024-01-15T18:05:00.739314+0000","last_became_active":"2024-01-15T18:04:58.296427+0000","last_became_peered":"2024-01-15T18:04:58.296427+0000","last_unstale":"2024-01-15T18:05:00.739314+0000","last_undegraded":"2024-01-15T18:05:00.739314+0000","last_fullsized":"2024-01-15T18:05:00.739314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539392+0000","last_change":"2024-01-15T18:04:59.312511+0000","last_active":"2024-01-15T18:05:00.539392+0000","last_peered":"2024-01-15T18:05:00.539392+0000","last_clean":"2024-01-15T18:05:00.539392+0000","last_became_active":"2024-01-15T18:04:59.312195+0000","last_became_peered":"2024-01-15T18:04:59.312195+0000","last_unstale":"2024-01-15T18:05:00.539392+0000","last_undegraded":"2024-01-15T18:05:00.539392+0000","last_fullsized":"2024-01-15T18:05:00.539392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:09:03.873184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740526+0000","last_change":"2024-01-15T18:04:58.289637+0000","last_active":"2024-01-15T18:05:00.740526+0000","last_peered":"2024-01-15T18:05:00.740526+0000","last_clean":"2024-01-15T18:05:00.740526+0000","last_became_active":"2024-01-15T18:04:58.289228+0000","last_became_peered":"2024-01-15T18:04:58.289228+0000","last_unstale":"2024-01-15T18:05:00.740526+0000","last_undegraded":"2024-01-15T18:05:00.740526+0000","last_fullsized":"2024-01-15T18:05:00.740526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353542+0000","last_change":"2024-01-15T18:04:59.309405+0000","last_active":"2024-01-15T18:05:00.353542+0000","last_peered":"2024-01-15T18:05:00.353542+0000","last_clean":"2024-01-15T18:05:00.353542+0000","last_became_active":"2024-01-15T18:04:59.309119+0000","last_became_peered":"2024-01-15T18:04:59.309119+0000","last_unstale":"2024-01-15T18:05:00.353542+0000","last_undegraded":"2024-01-15T18:05:00.353542+0000","last_fullsized":"2024-01-15T18:05:00.353542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739208+0000","last_change":"2024-01-15T18:04:58.294285+0000","last_active":"2024-01-15T18:05:00.739208+0000","last_peered":"2024-01-15T18:05:00.739208+0000","last_clean":"2024-01-15T18:05:00.739208+0000","last_became_active":"2024-01-15T18:04:58.293970+0000","last_became_peered":"2024-01-15T18:04:58.293970+0000","last_unstale":"2024-01-15T18:05:00.739208+0000","last_undegraded":"2024-01-15T18:05:00.739208+0000","last_fullsized":"2024-01-15T18:05:00.739208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.314051+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.313761+0000","last_became_peered":"2024-01-15T18:04:59.313761+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-15T18:04:59.290583+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:04:59.290583+0000","last_peered":"2024-01-15T18:04:59.290583+0000","last_clean":"2024-01-15T18:04:59.290583+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:04:59.290583+0000","last_undegraded":"2024-01-15T18:04:59.290583+0000","last_fullsized":"2024-01-15T18:04:59.290583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353134+0000","last_change":"2024-01-15T18:04:58.286159+0000","last_active":"2024-01-15T18:05:00.353134+0000","last_peered":"2024-01-15T18:05:00.353134+0000","last_clean":"2024-01-15T18:05:00.353134+0000","last_became_active":"2024-01-15T18:04:58.285819+0000","last_became_peered":"2024-01-15T18:04:58.285819+0000","last_unstale":"2024-01-15T18:05:00.353134+0000","last_undegraded":"2024-01-15T18:05:00.353134+0000","last_fullsized":"2024-01-15T18:05:00.353134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285340+0000","last_change":"2024-01-15T18:04:59.305454+0000","last_active":"2024-01-15T18:05:00.285340+0000","last_peered":"2024-01-15T18:05:00.285340+0000","last_clean":"2024-01-15T18:05:00.285340+0000","last_became_active":"2024-01-15T18:04:59.305160+0000","last_became_peered":"2024-01-15T18:04:59.305160+0000","last_unstale":"2024-01-15T18:05:00.285340+0000","last_undegraded":"2024-01-15T18:05:00.285340+0000","last_fullsized":"2024-01-15T18:05:00.285340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538748+0000","last_change":"2024-01-15T18:04:58.297164+0000","last_active":"2024-01-15T18:05:00.538748+0000","last_peered":"2024-01-15T18:05:00.538748+0000","last_clean":"2024-01-15T18:05:00.538748+0000","last_became_active":"2024-01-15T18:04:58.295969+0000","last_became_peered":"2024-01-15T18:04:58.295969+0000","last_unstale":"2024-01-15T18:05:00.538748+0000","last_undegraded":"2024-01-15T18:05:00.538748+0000","last_fullsized":"2024-01-15T18:05:00.538748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739741+0000","last_change":"2024-01-15T18:04:59.297889+0000","last_active":"2024-01-15T18:05:00.739741+0000","last_peered":"2024-01-15T18:05:00.739741+0000","last_clean":"2024-01-15T18:05:00.739741+0000","last_became_active":"2024-01-15T18:04:59.296874+0000","last_became_peered":"2024-01-15T18:04:59.296874+0000","last_unstale":"2024-01-15T18:05:00.739741+0000","last_undegraded":"2024-01-15T18:05:00.739741+0000","last_fullsized":"2024-01-15T18:05:00.739741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:52.200760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739266+0000","last_change":"2024-01-15T18:04:58.293459+0000","last_active":"2024-01-15T18:05:00.739266+0000","last_peered":"2024-01-15T18:05:00.739266+0000","last_clean":"2024-01-15T18:05:00.739266+0000","last_became_active":"2024-01-15T18:04:58.292396+0000","last_became_peered":"2024-01-15T18:04:58.292396+0000","last_unstale":"2024-01-15T18:05:00.739266+0000","last_undegraded":"2024-01-15T18:05:00.739266+0000","last_fullsized":"2024-01-15T18:05:00.739266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740002+0000","last_change":"2024-01-15T18:04:59.302244+0000","last_active":"2024-01-15T18:05:00.740002+0000","last_peered":"2024-01-15T18:05:00.740002+0000","last_clean":"2024-01-15T18:05:00.740002+0000","last_became_active":"2024-01-15T18:04:59.301981+0000","last_became_peered":"2024-01-15T18:04:59.301981+0000","last_unstale":"2024-01-15T18:05:00.740002+0000","last_undegraded":"2024-01-15T18:05:00.740002+0000","last_fullsized":"2024-01-15T18:05:00.740002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353265+0000","last_change":"2024-01-15T18:04:58.288895+0000","last_active":"2024-01-15T18:05:00.353265+0000","last_peered":"2024-01-15T18:05:00.353265+0000","last_clean":"2024-01-15T18:05:00.353265+0000","last_became_active":"2024-01-15T18:04:58.288385+0000","last_became_peered":"2024-01-15T18:04:58.288385+0000","last_unstale":"2024-01-15T18:05:00.353265+0000","last_undegraded":"2024-01-15T18:05:00.353265+0000","last_fullsized":"2024-01-15T18:05:00.353265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:10.169965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283326+0000","last_change":"2024-01-15T18:04:59.289150+0000","last_active":"2024-01-15T18:05:00.283326+0000","last_peered":"2024-01-15T18:05:00.283326+0000","last_clean":"2024-01-15T18:05:00.283326+0000","last_became_active":"2024-01-15T18:04:59.288315+0000","last_became_peered":"2024-01-15T18:04:59.288315+0000","last_unstale":"2024-01-15T18:05:00.283326+0000","last_undegraded":"2024-01-15T18:05:00.283326+0000","last_fullsized":"2024-01-15T18:05:00.283326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:14.706225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739871+0000","last_change":"2024-01-15T18:04:58.280841+0000","last_active":"2024-01-15T18:05:00.739871+0000","last_peered":"2024-01-15T18:05:00.739871+0000","last_clean":"2024-01-15T18:05:00.739871+0000","last_became_active":"2024-01-15T18:04:58.280590+0000","last_became_peered":"2024-01-15T18:04:58.280590+0000","last_unstale":"2024-01-15T18:05:00.739871+0000","last_undegraded":"2024-01-15T18:05:00.739871+0000","last_fullsized":"2024-01-15T18:05:00.739871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353597+0000","last_change":"2024-01-15T18:04:59.310649+0000","last_active":"2024-01-15T18:05:00.353597+0000","last_peered":"2024-01-15T18:05:00.353597+0000","last_clean":"2024-01-15T18:05:00.353597+0000","last_became_active":"2024-01-15T18:04:59.310266+0000","last_became_peered":"2024-01-15T18:04:59.310266+0000","last_unstale":"2024-01-15T18:05:00.353597+0000","last_undegraded":"2024-01-15T18:05:00.353597+0000","last_fullsized":"2024-01-15T18:05:00.353597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539301+0000","last_change":"2024-01-15T18:04:58.300881+0000","last_active":"2024-01-15T18:05:00.539301+0000","last_peered":"2024-01-15T18:05:00.539301+0000","last_clean":"2024-01-15T18:05:00.539301+0000","last_became_active":"2024-01-15T18:04:58.300322+0000","last_became_peered":"2024-01-15T18:04:58.300322+0000","last_unstale":"2024-01-15T18:05:00.539301+0000","last_undegraded":"2024-01-15T18:05:00.539301+0000","last_fullsized":"2024-01-15T18:05:00.539301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:47.163925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.299409+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.299156+0000","last_became_peered":"2024-01-15T18:04:59.299156+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:01.654927+0000","last_change":"2024-01-15T18:04:58.283804+0000","last_active":"2024-01-15T18:05:01.654927+0000","last_peered":"2024-01-15T18:05:01.654927+0000","last_clean":"2024-01-15T18:05:01.654927+0000","last_became_active":"2024-01-15T18:04:58.283533+0000","last_became_peered":"2024-01-15T18:04:58.283533+0000","last_unstale":"2024-01-15T18:05:01.654927+0000","last_undegraded":"2024-01-15T18:05:01.654927+0000","last_fullsized":"2024-01-15T18:05:01.654927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285921+0000","last_change":"2024-01-15T18:04:59.316822+0000","last_active":"2024-01-15T18:05:00.285921+0000","last_peered":"2024-01-15T18:05:00.285921+0000","last_clean":"2024-01-15T18:05:00.285921+0000","last_became_active":"2024-01-15T18:04:59.316513+0000","last_became_peered":"2024-01-15T18:04:59.316513+0000","last_unstale":"2024-01-15T18:05:00.285921+0000","last_undegraded":"2024-01-15T18:05:00.285921+0000","last_fullsized":"2024-01-15T18:05:00.285921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283543+0000","last_change":"2024-01-15T18:04:58.292591+0000","last_active":"2024-01-15T18:05:00.283543+0000","last_peered":"2024-01-15T18:05:00.283543+0000","last_clean":"2024-01-15T18:05:00.283543+0000","last_became_active":"2024-01-15T18:04:58.291917+0000","last_became_peered":"2024-01-15T18:04:58.291917+0000","last_unstale":"2024-01-15T18:05:00.283543+0000","last_undegraded":"2024-01-15T18:05:00.283543+0000","last_fullsized":"2024-01-15T18:05:00.283543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740270+0000","last_change":"2024-01-15T18:04:59.300764+0000","last_active":"2024-01-15T18:05:00.740270+0000","last_peered":"2024-01-15T18:05:00.740270+0000","last_clean":"2024-01-15T18:05:00.740270+0000","last_became_active":"2024-01-15T18:04:59.299980+0000","last_became_peered":"2024-01-15T18:04:59.299980+0000","last_unstale":"2024-01-15T18:05:00.740270+0000","last_undegraded":"2024-01-15T18:05:00.740270+0000","last_fullsized":"2024-01-15T18:05:00.740270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538729+0000","last_change":"2024-01-15T18:04:58.301082+0000","last_active":"2024-01-15T18:05:00.538729+0000","last_peered":"2024-01-15T18:05:00.538729+0000","last_clean":"2024-01-15T18:05:00.538729+0000","last_became_active":"2024-01-15T18:04:58.300809+0000","last_became_peered":"2024-01-15T18:04:58.300809+0000","last_unstale":"2024-01-15T18:05:00.538729+0000","last_undegraded":"2024-01-15T18:05:00.538729+0000","last_fullsized":"2024-01-15T18:05:00.538729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283231+0000","last_change":"2024-01-15T18:04:59.297502+0000","last_active":"2024-01-15T18:05:00.283231+0000","last_peered":"2024-01-15T18:05:00.283231+0000","last_clean":"2024-01-15T18:05:00.283231+0000","last_became_active":"2024-01-15T18:04:59.297211+0000","last_became_peered":"2024-01-15T18:04:59.297211+0000","last_unstale":"2024-01-15T18:05:00.283231+0000","last_undegraded":"2024-01-15T18:05:00.283231+0000","last_fullsized":"2024-01-15T18:05:00.283231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:13.025413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740211+0000","last_change":"2024-01-15T18:04:58.296402+0000","last_active":"2024-01-15T18:05:00.740211+0000","last_peered":"2024-01-15T18:05:00.740211+0000","last_clean":"2024-01-15T18:05:00.740211+0000","last_became_active":"2024-01-15T18:04:58.296101+0000","last_became_peered":"2024-01-15T18:04:58.296101+0000","last_unstale":"2024-01-15T18:05:00.740211+0000","last_undegraded":"2024-01-15T18:05:00.740211+0000","last_fullsized":"2024-01-15T18:05:00.740211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285133+0000","last_change":"2024-01-15T18:04:59.298441+0000","last_active":"2024-01-15T18:05:00.285133+0000","last_peered":"2024-01-15T18:05:00.285133+0000","last_clean":"2024-01-15T18:05:00.285133+0000","last_became_active":"2024-01-15T18:04:59.298204+0000","last_became_peered":"2024-01-15T18:04:59.298204+0000","last_unstale":"2024-01-15T18:05:00.285133+0000","last_undegraded":"2024-01-15T18:05:00.285133+0000","last_fullsized":"2024-01-15T18:05:00.285133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740408+0000","last_change":"2024-01-15T18:04:58.297952+0000","last_active":"2024-01-15T18:05:00.740408+0000","last_peered":"2024-01-15T18:05:00.740408+0000","last_clean":"2024-01-15T18:05:00.740408+0000","last_became_active":"2024-01-15T18:04:58.297621+0000","last_became_peered":"2024-01-15T18:04:58.297621+0000","last_unstale":"2024-01-15T18:05:00.740408+0000","last_undegraded":"2024-01-15T18:05:00.740408+0000","last_fullsized":"2024-01-15T18:05:00.740408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285600+0000","last_change":"2024-01-15T18:04:59.304388+0000","last_active":"2024-01-15T18:05:00.285600+0000","last_peered":"2024-01-15T18:05:00.285600+0000","last_clean":"2024-01-15T18:05:00.285600+0000","last_became_active":"2024-01-15T18:04:59.303178+0000","last_became_peered":"2024-01-15T18:04:59.303178+0000","last_unstale":"2024-01-15T18:05:00.285600+0000","last_undegraded":"2024-01-15T18:05:00.285600+0000","last_fullsized":"2024-01-15T18:05:00.285600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539246+0000","last_change":"2024-01-15T18:04:58.300443+0000","last_active":"2024-01-15T18:05:00.539246+0000","last_peered":"2024-01-15T18:05:00.539246+0000","last_clean":"2024-01-15T18:05:00.539246+0000","last_became_active":"2024-01-15T18:04:58.300157+0000","last_became_peered":"2024-01-15T18:04:58.300157+0000","last_unstale":"2024-01-15T18:05:00.539246+0000","last_undegraded":"2024-01-15T18:05:00.539246+0000","last_fullsized":"2024-01-15T18:05:00.539246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283605+0000","last_change":"2024-01-15T18:04:59.295964+0000","last_active":"2024-01-15T18:05:00.283605+0000","last_peered":"2024-01-15T18:05:00.283605+0000","last_clean":"2024-01-15T18:05:00.283605+0000","last_became_active":"2024-01-15T18:04:59.295569+0000","last_became_peered":"2024-01-15T18:04:59.295569+0000","last_unstale":"2024-01-15T18:05:00.283605+0000","last_undegraded":"2024-01-15T18:05:00.283605+0000","last_fullsized":"2024-01-15T18:05:00.283605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.316899+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.316632+0000","last_became_peered":"2024-01-15T18:04:59.316632+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538942+0000","last_change":"2024-01-15T18:04:58.301177+0000","last_active":"2024-01-15T18:05:00.538942+0000","last_peered":"2024-01-15T18:05:00.538942+0000","last_clean":"2024-01-15T18:05:00.538942+0000","last_became_active":"2024-01-15T18:04:58.300869+0000","last_became_peered":"2024-01-15T18:04:58.300869+0000","last_unstale":"2024-01-15T18:05:00.538942+0000","last_undegraded":"2024-01-15T18:05:00.538942+0000","last_fullsized":"2024-01-15T18:05:00.538942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353471+0000","last_change":"2024-01-15T18:04:59.311012+0000","last_active":"2024-01-15T18:05:00.353471+0000","last_peered":"2024-01-15T18:05:00.353471+0000","last_clean":"2024-01-15T18:05:00.353471+0000","last_became_active":"2024-01-15T18:04:59.310673+0000","last_became_peered":"2024-01-15T18:04:59.310673+0000","last_unstale":"2024-01-15T18:05:00.353471+0000","last_undegraded":"2024-01-15T18:05:00.353471+0000","last_fullsized":"2024-01-15T18:05:00.353471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539209+0000","last_change":"2024-01-15T18:04:58.303349+0000","last_active":"2024-01-15T18:05:00.539209+0000","last_peered":"2024-01-15T18:05:00.539209+0000","last_clean":"2024-01-15T18:05:00.539209+0000","last_became_active":"2024-01-15T18:04:58.303074+0000","last_became_peered":"2024-01-15T18:04:58.303074+0000","last_unstale":"2024-01-15T18:05:00.539209+0000","last_undegraded":"2024-01-15T18:05:00.539209+0000","last_fullsized":"2024-01-15T18:05:00.539209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740412+0000","last_change":"2024-01-15T18:04:59.311306+0000","last_active":"2024-01-15T18:05:00.740412+0000","last_peered":"2024-01-15T18:05:00.740412+0000","last_clean":"2024-01-15T18:05:00.740412+0000","last_became_active":"2024-01-15T18:04:59.311031+0000","last_became_peered":"2024-01-15T18:04:59.311031+0000","last_unstale":"2024-01-15T18:05:00.740412+0000","last_undegraded":"2024-01-15T18:05:00.740412+0000","last_fullsized":"2024-01-15T18:05:00.740412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285716+0000","last_change":"2024-01-15T18:04:58.299606+0000","last_active":"2024-01-15T18:05:00.285716+0000","last_peered":"2024-01-15T18:05:00.285716+0000","last_clean":"2024-01-15T18:05:00.285716+0000","last_became_active":"2024-01-15T18:04:58.299353+0000","last_became_peered":"2024-01-15T18:04:58.299353+0000","last_unstale":"2024-01-15T18:05:00.285716+0000","last_undegraded":"2024-01-15T18:05:00.285716+0000","last_fullsized":"2024-01-15T18:05:00.285716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285464+0000","last_change":"2024-01-15T18:04:59.304019+0000","last_active":"2024-01-15T18:05:00.285464+0000","last_peered":"2024-01-15T18:05:00.285464+0000","last_clean":"2024-01-15T18:05:00.285464+0000","last_became_active":"2024-01-15T18:04:59.303662+0000","last_became_peered":"2024-01-15T18:04:59.303662+0000","last_unstale":"2024-01-15T18:05:00.285464+0000","last_undegraded":"2024-01-15T18:05:00.285464+0000","last_fullsized":"2024-01-15T18:05:00.285464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740536+0000","last_change":"2024-01-15T18:04:58.303996+0000","last_active":"2024-01-15T18:05:00.740536+0000","last_peered":"2024-01-15T18:05:00.740536+0000","last_clean":"2024-01-15T18:05:00.740536+0000","last_became_active":"2024-01-15T18:04:58.303687+0000","last_became_peered":"2024-01-15T18:04:58.303687+0000","last_unstale":"2024-01-15T18:05:00.740536+0000","last_undegraded":"2024-01-15T18:05:00.740536+0000","last_fullsized":"2024-01-15T18:05:00.740536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7759999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.532}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6840000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8259999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6909999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7309999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:05:08.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:05:08.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 Processor -- start 2024-01-15T18:05:08.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 -- start start 2024-01-15T18:05:08.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0850143780 0x7f08500cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08500ce520 0x7f0850071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08500ceb50 con 0x7f08500720f0 2024-01-15T18:05:08.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085015c110 con 0x7f08500ce520 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0857600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0850072530 con 0x7f0850143780 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f084fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f08500ce520 0x7f0850071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0855600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f08500720f0 0x7f085015a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36070/0 (socket says 172.21.15.143:36070) 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0855600640 1 -- 172.21.15.143:0/2804712358 learned_addr learned my addr 172.21.15.143:0/2804712358 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f0854c00640 1 --2- 172.21.15.143:0/2804712358 >> v2:172.21.15.204:3300/0 conn(0x7f0850143780 0x7f08500cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f084fe00640 1 -- 172.21.15.143:0/2804712358 >> v2:172.21.15.204:3300/0 conn(0x7f0850143780 msgr2=0x7f08500cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f084fe00640 1 --2- 172.21.15.143:0/2804712358 >> v2:172.21.15.204:3300/0 conn(0x7f0850143780 0x7f08500cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f084fe00640 1 -- 172.21.15.143:0/2804712358 >> v1:172.21.15.143:6789/0 conn(0x7f08500720f0 legacy=0x7f085015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f084fe00640 1 -- 172.21.15.143:0/2804712358 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085015b7e0 con 0x7f08500ce520 2024-01-15T18:05:08.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.305+0000 7f084f400640 1 -- 172.21.15.143:0/2804712358 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877668401 0 0) 0x7f08480014a0 con 0x7f08500720f0 2024-01-15T18:05:08.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f084fe00640 1 --2- 172.21.15.143:0/2804712358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f0850071bb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f085006be30 tx=0x7f083801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3017ffcfaed7d51c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f084f400640 1 -- 172.21.15.143:0/2804712358 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380ab600 con 0x7f08500ce520 2024-01-15T18:05:08.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f084f400640 1 -- 172.21.15.143:0/2804712358 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08380a7050 con 0x7f08500ce520 2024-01-15T18:05:08.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f084f400640 1 -- 172.21.15.143:0/2804712358 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380ab980 con 0x7f08500ce520 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 -- 172.21.15.143:0/2804712358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 msgr2=0x7f0850071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 --2- 172.21.15.143:0/2804712358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f0850071bb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f085006be30 tx=0x7f083801c460 comp rx=0 tx=0).stop 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 -- 172.21.15.143:0/2804712358 shutdown_connections 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 --2- 172.21.15.143:0/2804712358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f0850071bb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 --2- 172.21.15.143:0/2804712358 >> v2:172.21.15.204:3300/0 conn(0x7f0850143780 0x7f08500cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 -- 172.21.15.143:0/2804712358 >> 172.21.15.143:0/2804712358 conn(0x7f085006c3f0 msgr2=0x7f085006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 -- 172.21.15.143:0/2804712358 shutdown_connections 2024-01-15T18:05:08.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.309+0000 7f0857600640 1 -- 172.21.15.143:0/2804712358 wait complete. 2024-01-15T18:05:08.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 Processor -- start 2024-01-15T18:05:08.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 -- start start 2024-01-15T18:05:08.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08500720f0 0x7f0850142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f085013bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0850143780 0x7f08500af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0850156fe0 con 0x7f08500ce520 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08501412c0 con 0x7f08500720f0 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08501415a0 con 0x7f0850143780 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f085013bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f085013bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48418/0 (socket says 172.21.15.143:48418) 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 -- 172.21.15.143:0/1281836309 learned_addr learned my addr 172.21.15.143:0/1281836309 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0854c00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08500720f0 0x7f0850142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0855600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0850143780 0x7f08500af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 -- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0850143780 msgr2=0x7f08500af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0850143780 0x7f08500af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 -- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08500720f0 msgr2=0x7f0850142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08500720f0 0x7f0850142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 -- 172.21.15.143:0/1281836309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838009420 con 0x7f08500ce520 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084fe00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f085013bb50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08380ae040 tx=0x7f0838019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380b4410 con 0x7f08500ce520 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08380b4570 con 0x7f08500ce520 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0850141880 con 0x7f08500ce520 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380b4920 con 0x7f08500ce520 2024-01-15T18:05:08.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.313+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085013c090 con 0x7f08500ce520 2024-01-15T18:05:08.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.317+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085013c280 con 0x7f08500ce520 2024-01-15T18:05:08.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.317+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f08380b2040 con 0x7f08500ce520 2024-01-15T18:05:08.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.317+0000 7f084cc00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08280668f0 0x7f0828068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.317+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f083811e1f0 con 0x7f08500ce520 2024-01-15T18:05:08.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.317+0000 7f0854c00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08280668f0 0x7f0828068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.317+0000 7f0854c00640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08280668f0 0x7f0828068db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0840009ac0 tx=0x7f0840009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.321+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08380eb3b0 con 0x7f08500ce520 2024-01-15T18:05:08.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.469+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0850141430 con 0x7f08280668f0 2024-01-15T18:05:08.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.473+0000 7f084cc00640 1 -- 172.21.15.143:0/1281836309 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264536 (secure 0 0 0) 0x7f0850141430 con 0x7f08280668f0 2024-01-15T18:05:08.483 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:08.485 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:05:08.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08280668f0 msgr2=0x7f0828068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08280668f0 0x7f0828068db0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0840009ac0 tx=0x7f0840009190 comp rx=0 tx=0).stop 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 msgr2=0x7f085013bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f085013bb50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08380ae040 tx=0x7f0838019200 comp rx=0 tx=0).stop 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 shutdown_connections 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08280668f0 0x7f0828068db0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0850143780 0x7f08500af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f08500ce520 0x7f085013bb50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 --2- 172.21.15.143:0/1281836309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08500720f0 0x7f0850142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 >> 172.21.15.143:0/1281836309 conn(0x7f085006c3f0 msgr2=0x7f08500aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 shutdown_connections 2024-01-15T18:05:08.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.477+0000 7f0857600640 1 -- 172.21.15.143:0/1281836309 wait complete. 2024-01-15T18:05:08.505 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-01-15T18:05:06.622971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001634"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539184+0000","last_change":"2024-01-15T18:04:59.313108+0000","last_active":"2024-01-15T18:05:00.539184+0000","last_peered":"2024-01-15T18:05:00.539184+0000","last_clean":"2024-01-15T18:05:00.539184+0000","last_became_active":"2024-01-15T18:04:59.312739+0000","last_became_peered":"2024-01-15T18:04:59.312739+0000","last_unstale":"2024-01-15T18:05:00.539184+0000","last_undegraded":"2024-01-15T18:05:00.539184+0000","last_fullsized":"2024-01-15T18:05:00.539184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740563+0000","last_change":"2024-01-15T18:04:59.303929+0000","last_active":"2024-01-15T18:05:00.740563+0000","last_peered":"2024-01-15T18:05:00.740563+0000","last_clean":"2024-01-15T18:05:00.740563+0000","last_became_active":"2024-01-15T18:04:59.303598+0000","last_became_peered":"2024-01-15T18:04:59.303598+0000","last_unstale":"2024-01-15T18:05:00.740563+0000","last_undegraded":"2024-01-15T18:05:00.740563+0000","last_fullsized":"2024-01-15T18:05:00.740563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740178+0000","last_change":"2024-01-15T18:04:59.298035+0000","last_active":"2024-01-15T18:05:00.740178+0000","last_peered":"2024-01-15T18:05:00.740178+0000","last_clean":"2024-01-15T18:05:00.740178+0000","last_became_active":"2024-01-15T18:04:59.297748+0000","last_became_peered":"2024-01-15T18:04:59.297748+0000","last_unstale":"2024-01-15T18:05:00.740178+0000","last_undegraded":"2024-01-15T18:05:00.740178+0000","last_fullsized":"2024-01-15T18:05:00.740178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.314048+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.313770+0000","last_became_peered":"2024-01-15T18:04:59.313770+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:19.659551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.308757+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.306380+0000","last_became_peered":"2024-01-15T18:04:59.306380+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353031+0000","last_change":"2024-01-15T18:04:59.307868+0000","last_active":"2024-01-15T18:05:00.353031+0000","last_peered":"2024-01-15T18:05:00.353031+0000","last_clean":"2024-01-15T18:05:00.353031+0000","last_became_active":"2024-01-15T18:04:59.307388+0000","last_became_peered":"2024-01-15T18:04:59.307388+0000","last_unstale":"2024-01-15T18:05:00.353031+0000","last_undegraded":"2024-01-15T18:05:00.353031+0000","last_fullsized":"2024-01-15T18:05:00.353031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538844+0000","last_change":"2024-01-15T18:04:59.316726+0000","last_active":"2024-01-15T18:05:00.538844+0000","last_peered":"2024-01-15T18:05:00.538844+0000","last_clean":"2024-01-15T18:05:00.538844+0000","last_became_active":"2024-01-15T18:04:59.311435+0000","last_became_peered":"2024-01-15T18:04:59.311435+0000","last_unstale":"2024-01-15T18:05:00.538844+0000","last_undegraded":"2024-01-15T18:05:00.538844+0000","last_fullsized":"2024-01-15T18:05:00.538844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284988+0000","last_change":"2024-01-15T18:04:59.305582+0000","last_active":"2024-01-15T18:05:00.284988+0000","last_peered":"2024-01-15T18:05:00.284988+0000","last_clean":"2024-01-15T18:05:00.284988+0000","last_became_active":"2024-01-15T18:04:59.305227+0000","last_became_peered":"2024-01-15T18:04:59.305227+0000","last_unstale":"2024-01-15T18:05:00.284988+0000","last_undegraded":"2024-01-15T18:05:00.284988+0000","last_fullsized":"2024-01-15T18:05:00.284988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539386+0000","last_change":"2024-01-15T18:04:59.310510+0000","last_active":"2024-01-15T18:05:00.539386+0000","last_peered":"2024-01-15T18:05:00.539386+0000","last_clean":"2024-01-15T18:05:00.539386+0000","last_became_active":"2024-01-15T18:04:59.310228+0000","last_became_peered":"2024-01-15T18:04:59.310228+0000","last_unstale":"2024-01-15T18:05:00.539386+0000","last_undegraded":"2024-01-15T18:05:00.539386+0000","last_fullsized":"2024-01-15T18:05:00.539386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283959+0000","last_change":"2024-01-15T18:04:59.308904+0000","last_active":"2024-01-15T18:05:00.283959+0000","last_peered":"2024-01-15T18:05:00.283959+0000","last_clean":"2024-01-15T18:05:00.283959+0000","last_became_active":"2024-01-15T18:04:59.308650+0000","last_became_peered":"2024-01-15T18:04:59.308650+0000","last_unstale":"2024-01-15T18:05:00.283959+0000","last_undegraded":"2024-01-15T18:05:00.283959+0000","last_fullsized":"2024-01-15T18:05:00.283959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283572+0000","last_change":"2024-01-15T18:04:59.311700+0000","last_active":"2024-01-15T18:05:00.283572+0000","last_peered":"2024-01-15T18:05:00.283572+0000","last_clean":"2024-01-15T18:05:00.283572+0000","last_became_active":"2024-01-15T18:04:59.311406+0000","last_became_peered":"2024-01-15T18:04:59.311406+0000","last_unstale":"2024-01-15T18:05:00.283572+0000","last_undegraded":"2024-01-15T18:05:00.283572+0000","last_fullsized":"2024-01-15T18:05:00.283572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284074+0000","last_change":"2024-01-15T18:04:59.287898+0000","last_active":"2024-01-15T18:05:00.284074+0000","last_peered":"2024-01-15T18:05:00.284074+0000","last_clean":"2024-01-15T18:05:00.284074+0000","last_became_active":"2024-01-15T18:04:59.287554+0000","last_became_peered":"2024-01-15T18:04:59.287554+0000","last_unstale":"2024-01-15T18:05:00.284074+0000","last_undegraded":"2024-01-15T18:05:00.284074+0000","last_fullsized":"2024-01-15T18:05:00.284074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:43.736312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285461+0000","last_change":"2024-01-15T18:04:59.303813+0000","last_active":"2024-01-15T18:05:00.285461+0000","last_peered":"2024-01-15T18:05:00.285461+0000","last_clean":"2024-01-15T18:05:00.285461+0000","last_became_active":"2024-01-15T18:04:59.303549+0000","last_became_peered":"2024-01-15T18:04:59.303549+0000","last_unstale":"2024-01-15T18:05:00.285461+0000","last_undegraded":"2024-01-15T18:05:00.285461+0000","last_fullsized":"2024-01-15T18:05:00.285461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739952+0000","last_change":"2024-01-15T18:04:59.301393+0000","last_active":"2024-01-15T18:05:00.739952+0000","last_peered":"2024-01-15T18:05:00.739952+0000","last_clean":"2024-01-15T18:05:00.739952+0000","last_became_active":"2024-01-15T18:04:59.300604+0000","last_became_peered":"2024-01-15T18:04:59.300604+0000","last_unstale":"2024-01-15T18:05:00.739952+0000","last_undegraded":"2024-01-15T18:05:00.739952+0000","last_fullsized":"2024-01-15T18:05:00.739952+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740231+0000","last_change":"2024-01-15T18:04:59.311581+0000","last_active":"2024-01-15T18:05:00.740231+0000","last_peered":"2024-01-15T18:05:00.740231+0000","last_clean":"2024-01-15T18:05:00.740231+0000","last_became_active":"2024-01-15T18:04:59.311309+0000","last_became_peered":"2024-01-15T18:04:59.311309+0000","last_unstale":"2024-01-15T18:05:00.740231+0000","last_undegraded":"2024-01-15T18:05:00.740231+0000","last_fullsized":"2024-01-15T18:05:00.740231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353034+0000","last_change":"2024-01-15T18:04:59.307600+0000","last_active":"2024-01-15T18:05:00.353034+0000","last_peered":"2024-01-15T18:05:00.353034+0000","last_clean":"2024-01-15T18:05:00.353034+0000","last_became_active":"2024-01-15T18:04:59.307160+0000","last_became_peered":"2024-01-15T18:04:59.307160+0000","last_unstale":"2024-01-15T18:05:00.353034+0000","last_undegraded":"2024-01-15T18:05:00.353034+0000","last_fullsized":"2024-01-15T18:05:00.353034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353453+0000","last_change":"2024-01-15T18:04:59.316075+0000","last_active":"2024-01-15T18:05:00.353453+0000","last_peered":"2024-01-15T18:05:00.353453+0000","last_clean":"2024-01-15T18:05:00.353453+0000","last_became_active":"2024-01-15T18:04:59.315751+0000","last_became_peered":"2024-01-15T18:04:59.315751+0000","last_unstale":"2024-01-15T18:05:00.353453+0000","last_undegraded":"2024-01-15T18:05:00.353453+0000","last_fullsized":"2024-01-15T18:05:00.353453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:12.915871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739317+0000","last_change":"2024-01-15T18:04:59.305938+0000","last_active":"2024-01-15T18:05:00.739317+0000","last_peered":"2024-01-15T18:05:00.739317+0000","last_clean":"2024-01-15T18:05:00.739317+0000","last_became_active":"2024-01-15T18:04:59.305686+0000","last_became_peered":"2024-01-15T18:04:59.305686+0000","last_unstale":"2024-01-15T18:05:00.739317+0000","last_undegraded":"2024-01-15T18:05:00.739317+0000","last_fullsized":"2024-01-15T18:05:00.739317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:35.095703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539274+0000","last_change":"2024-01-15T18:04:59.306518+0000","last_active":"2024-01-15T18:05:00.539274+0000","last_peered":"2024-01-15T18:05:00.539274+0000","last_clean":"2024-01-15T18:05:00.539274+0000","last_became_active":"2024-01-15T18:04:59.306009+0000","last_became_peered":"2024-01-15T18:04:59.306009+0000","last_unstale":"2024-01-15T18:05:00.539274+0000","last_undegraded":"2024-01-15T18:05:00.539274+0000","last_fullsized":"2024-01-15T18:05:00.539274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283820+0000","last_change":"2024-01-15T18:04:59.296492+0000","last_active":"2024-01-15T18:05:00.283820+0000","last_peered":"2024-01-15T18:05:00.283820+0000","last_clean":"2024-01-15T18:05:00.283820+0000","last_became_active":"2024-01-15T18:04:59.296200+0000","last_became_peered":"2024-01-15T18:04:59.296200+0000","last_unstale":"2024-01-15T18:05:00.283820+0000","last_undegraded":"2024-01-15T18:05:00.283820+0000","last_fullsized":"2024-01-15T18:05:00.283820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.300916+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.300453+0000","last_became_peered":"2024-01-15T18:04:59.300453+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:50.361044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353117+0000","last_change":"2024-01-15T18:04:59.313753+0000","last_active":"2024-01-15T18:05:00.353117+0000","last_peered":"2024-01-15T18:05:00.353117+0000","last_clean":"2024-01-15T18:05:00.353117+0000","last_became_active":"2024-01-15T18:04:59.313404+0000","last_became_peered":"2024-01-15T18:04:59.313404+0000","last_unstale":"2024-01-15T18:05:00.353117+0000","last_undegraded":"2024-01-15T18:05:00.353117+0000","last_fullsized":"2024-01-15T18:05:00.353117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285121+0000","last_change":"2024-01-15T18:04:59.297113+0000","last_active":"2024-01-15T18:05:00.285121+0000","last_peered":"2024-01-15T18:05:00.285121+0000","last_clean":"2024-01-15T18:05:00.285121+0000","last_became_active":"2024-01-15T18:04:59.296920+0000","last_became_peered":"2024-01-15T18:04:59.296920+0000","last_unstale":"2024-01-15T18:05:00.285121+0000","last_undegraded":"2024-01-15T18:05:00.285121+0000","last_fullsized":"2024-01-15T18:05:00.285121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353148+0000","last_change":"2024-01-15T18:04:59.307789+0000","last_active":"2024-01-15T18:05:00.353148+0000","last_peered":"2024-01-15T18:05:00.353148+0000","last_clean":"2024-01-15T18:05:00.353148+0000","last_became_active":"2024-01-15T18:04:59.306606+0000","last_became_peered":"2024-01-15T18:04:59.306606+0000","last_unstale":"2024-01-15T18:05:00.353148+0000","last_undegraded":"2024-01-15T18:05:00.353148+0000","last_fullsized":"2024-01-15T18:05:00.353148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285732+0000","last_change":"2024-01-15T18:04:59.305088+0000","last_active":"2024-01-15T18:05:00.285732+0000","last_peered":"2024-01-15T18:05:00.285732+0000","last_clean":"2024-01-15T18:05:00.285732+0000","last_became_active":"2024-01-15T18:04:59.304735+0000","last_became_peered":"2024-01-15T18:04:59.304735+0000","last_unstale":"2024-01-15T18:05:00.285732+0000","last_undegraded":"2024-01-15T18:05:00.285732+0000","last_fullsized":"2024-01-15T18:05:00.285732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284011+0000","last_change":"2024-01-15T18:04:59.311649+0000","last_active":"2024-01-15T18:05:00.284011+0000","last_peered":"2024-01-15T18:05:00.284011+0000","last_clean":"2024-01-15T18:05:00.284011+0000","last_became_active":"2024-01-15T18:04:59.311225+0000","last_became_peered":"2024-01-15T18:04:59.311225+0000","last_unstale":"2024-01-15T18:05:00.284011+0000","last_undegraded":"2024-01-15T18:05:00.284011+0000","last_fullsized":"2024-01-15T18:05:00.284011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.306001+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.305732+0000","last_became_peered":"2024-01-15T18:04:59.305732+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285595+0000","last_change":"2024-01-15T18:04:59.306947+0000","last_active":"2024-01-15T18:05:00.285595+0000","last_peered":"2024-01-15T18:05:00.285595+0000","last_clean":"2024-01-15T18:05:00.285595+0000","last_became_active":"2024-01-15T18:04:59.306617+0000","last_became_peered":"2024-01-15T18:04:59.306617+0000","last_unstale":"2024-01-15T18:05:00.285595+0000","last_undegraded":"2024-01-15T18:05:00.285595+0000","last_fullsized":"2024-01-15T18:05:00.285595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:33:53.093223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739949+0000","last_change":"2024-01-15T18:04:59.303640+0000","last_active":"2024-01-15T18:05:00.739949+0000","last_peered":"2024-01-15T18:05:00.739949+0000","last_clean":"2024-01-15T18:05:00.739949+0000","last_became_active":"2024-01-15T18:04:59.303076+0000","last_became_peered":"2024-01-15T18:04:59.303076+0000","last_unstale":"2024-01-15T18:05:00.739949+0000","last_undegraded":"2024-01-15T18:05:00.739949+0000","last_fullsized":"2024-01-15T18:05:00.739949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:45.894125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353181+0000","last_change":"2024-01-15T18:04:59.307697+0000","last_active":"2024-01-15T18:05:00.353181+0000","last_peered":"2024-01-15T18:05:00.353181+0000","last_clean":"2024-01-15T18:05:00.353181+0000","last_became_active":"2024-01-15T18:04:59.305330+0000","last_became_peered":"2024-01-15T18:04:59.305330+0000","last_unstale":"2024-01-15T18:05:00.353181+0000","last_undegraded":"2024-01-15T18:05:00.353181+0000","last_fullsized":"2024-01-15T18:05:00.353181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739155+0000","last_change":"2024-01-15T18:04:59.305380+0000","last_active":"2024-01-15T18:05:00.739155+0000","last_peered":"2024-01-15T18:05:00.739155+0000","last_clean":"2024-01-15T18:05:00.739155+0000","last_became_active":"2024-01-15T18:04:59.305090+0000","last_became_peered":"2024-01-15T18:04:59.305090+0000","last_unstale":"2024-01-15T18:05:00.739155+0000","last_undegraded":"2024-01-15T18:05:00.739155+0000","last_fullsized":"2024-01-15T18:05:00.739155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283376+0000","last_change":"2024-01-15T18:04:59.294688+0000","last_active":"2024-01-15T18:05:00.283376+0000","last_peered":"2024-01-15T18:05:00.283376+0000","last_clean":"2024-01-15T18:05:00.283376+0000","last_became_active":"2024-01-15T18:04:59.294065+0000","last_became_peered":"2024-01-15T18:04:59.294065+0000","last_unstale":"2024-01-15T18:05:00.283376+0000","last_undegraded":"2024-01-15T18:05:00.283376+0000","last_fullsized":"2024-01-15T18:05:00.283376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283129+0000","last_change":"2024-01-15T18:04:59.300834+0000","last_active":"2024-01-15T18:05:00.283129+0000","last_peered":"2024-01-15T18:05:00.283129+0000","last_clean":"2024-01-15T18:05:00.283129+0000","last_became_active":"2024-01-15T18:04:59.300257+0000","last_became_peered":"2024-01-15T18:04:59.300257+0000","last_unstale":"2024-01-15T18:05:00.283129+0000","last_undegraded":"2024-01-15T18:05:00.283129+0000","last_fullsized":"2024-01-15T18:05:00.283129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:31.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285696+0000","last_change":"2024-01-15T18:04:58.298585+0000","last_active":"2024-01-15T18:05:00.285696+0000","last_peered":"2024-01-15T18:05:00.285696+0000","last_clean":"2024-01-15T18:05:00.285696+0000","last_became_active":"2024-01-15T18:04:58.298288+0000","last_became_peered":"2024-01-15T18:04:58.298288+0000","last_unstale":"2024-01-15T18:05:00.285696+0000","last_undegraded":"2024-01-15T18:05:00.285696+0000","last_fullsized":"2024-01-15T18:05:00.285696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740106+0000","last_change":"2024-01-15T18:04:59.295504+0000","last_active":"2024-01-15T18:05:00.740106+0000","last_peered":"2024-01-15T18:05:00.740106+0000","last_clean":"2024-01-15T18:05:00.740106+0000","last_became_active":"2024-01-15T18:04:59.295205+0000","last_became_peered":"2024-01-15T18:04:59.295205+0000","last_unstale":"2024-01-15T18:05:00.740106+0000","last_undegraded":"2024-01-15T18:05:00.740106+0000","last_fullsized":"2024-01-15T18:05:00.740106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283737+0000","last_change":"2024-01-15T18:04:58.296152+0000","last_active":"2024-01-15T18:05:00.283737+0000","last_peered":"2024-01-15T18:05:00.283737+0000","last_clean":"2024-01-15T18:05:00.283737+0000","last_became_active":"2024-01-15T18:04:58.295329+0000","last_became_peered":"2024-01-15T18:04:58.295329+0000","last_unstale":"2024-01-15T18:05:00.283737+0000","last_undegraded":"2024-01-15T18:05:00.283737+0000","last_fullsized":"2024-01-15T18:05:00.283737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285163+0000","last_change":"2024-01-15T18:04:58.291634+0000","last_active":"2024-01-15T18:05:00.285163+0000","last_peered":"2024-01-15T18:05:00.285163+0000","last_clean":"2024-01-15T18:05:00.285163+0000","last_became_active":"2024-01-15T18:04:58.290715+0000","last_became_peered":"2024-01-15T18:04:58.290715+0000","last_unstale":"2024-01-15T18:05:00.285163+0000","last_undegraded":"2024-01-15T18:05:00.285163+0000","last_fullsized":"2024-01-15T18:05:00.285163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285067+0000","last_change":"2024-01-15T18:04:59.302779+0000","last_active":"2024-01-15T18:05:00.285067+0000","last_peered":"2024-01-15T18:05:00.285067+0000","last_clean":"2024-01-15T18:05:00.285067+0000","last_became_active":"2024-01-15T18:04:59.302387+0000","last_became_peered":"2024-01-15T18:04:59.302387+0000","last_unstale":"2024-01-15T18:05:00.285067+0000","last_undegraded":"2024-01-15T18:05:00.285067+0000","last_fullsized":"2024-01-15T18:05:00.285067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353124+0000","last_change":"2024-01-15T18:04:58.285848+0000","last_active":"2024-01-15T18:05:00.353124+0000","last_peered":"2024-01-15T18:05:00.353124+0000","last_clean":"2024-01-15T18:05:00.353124+0000","last_became_active":"2024-01-15T18:04:58.283781+0000","last_became_peered":"2024-01-15T18:04:58.283781+0000","last_unstale":"2024-01-15T18:05:00.353124+0000","last_undegraded":"2024-01-15T18:05:00.353124+0000","last_fullsized":"2024-01-15T18:05:00.353124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740140+0000","last_change":"2024-01-15T18:04:59.306022+0000","last_active":"2024-01-15T18:05:00.740140+0000","last_peered":"2024-01-15T18:05:00.740140+0000","last_clean":"2024-01-15T18:05:00.740140+0000","last_became_active":"2024-01-15T18:04:59.305785+0000","last_became_peered":"2024-01-15T18:04:59.305785+0000","last_unstale":"2024-01-15T18:05:00.740140+0000","last_undegraded":"2024-01-15T18:05:00.740140+0000","last_fullsized":"2024-01-15T18:05:00.740140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283482+0000","last_change":"2024-01-15T18:04:58.292333+0000","last_active":"2024-01-15T18:05:00.283482+0000","last_peered":"2024-01-15T18:05:00.283482+0000","last_clean":"2024-01-15T18:05:00.283482+0000","last_became_active":"2024-01-15T18:04:58.291543+0000","last_became_peered":"2024-01-15T18:04:58.291543+0000","last_unstale":"2024-01-15T18:05:00.283482+0000","last_undegraded":"2024-01-15T18:05:00.283482+0000","last_fullsized":"2024-01-15T18:05:00.283482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353424+0000","last_change":"2024-01-15T18:04:59.313691+0000","last_active":"2024-01-15T18:05:00.353424+0000","last_peered":"2024-01-15T18:05:00.353424+0000","last_clean":"2024-01-15T18:05:00.353424+0000","last_became_active":"2024-01-15T18:04:59.313395+0000","last_became_peered":"2024-01-15T18:04:59.313395+0000","last_unstale":"2024-01-15T18:05:00.353424+0000","last_undegraded":"2024-01-15T18:05:00.353424+0000","last_fullsized":"2024-01-15T18:05:00.353424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353418+0000","last_change":"2024-01-15T18:04:58.288981+0000","last_active":"2024-01-15T18:05:00.353418+0000","last_peered":"2024-01-15T18:05:00.353418+0000","last_clean":"2024-01-15T18:05:00.353418+0000","last_became_active":"2024-01-15T18:04:58.288667+0000","last_became_peered":"2024-01-15T18:04:58.288667+0000","last_unstale":"2024-01-15T18:05:00.353418+0000","last_undegraded":"2024-01-15T18:05:00.353418+0000","last_fullsized":"2024-01-15T18:05:00.353418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740466+0000","last_change":"2024-01-15T18:04:59.311337+0000","last_active":"2024-01-15T18:05:00.740466+0000","last_peered":"2024-01-15T18:05:00.740466+0000","last_clean":"2024-01-15T18:05:00.740466+0000","last_became_active":"2024-01-15T18:04:59.311102+0000","last_became_peered":"2024-01-15T18:04:59.311102+0000","last_unstale":"2024-01-15T18:05:00.740466+0000","last_undegraded":"2024-01-15T18:05:00.740466+0000","last_fullsized":"2024-01-15T18:05:00.740466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740475+0000","last_change":"2024-01-15T18:04:58.295173+0000","last_active":"2024-01-15T18:05:00.740475+0000","last_peered":"2024-01-15T18:05:00.740475+0000","last_clean":"2024-01-15T18:05:00.740475+0000","last_became_active":"2024-01-15T18:04:58.294883+0000","last_became_peered":"2024-01-15T18:04:58.294883+0000","last_unstale":"2024-01-15T18:05:00.740475+0000","last_undegraded":"2024-01-15T18:05:00.740475+0000","last_fullsized":"2024-01-15T18:05:00.740475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284186+0000","last_change":"2024-01-15T18:04:59.311644+0000","last_active":"2024-01-15T18:05:00.284186+0000","last_peered":"2024-01-15T18:05:00.284186+0000","last_clean":"2024-01-15T18:05:00.284186+0000","last_became_active":"2024-01-15T18:04:59.311224+0000","last_became_peered":"2024-01-15T18:04:59.311224+0000","last_unstale":"2024-01-15T18:05:00.284186+0000","last_undegraded":"2024-01-15T18:05:00.284186+0000","last_fullsized":"2024-01-15T18:05:00.284186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:30.904066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283776+0000","last_change":"2024-01-15T18:04:58.293217+0000","last_active":"2024-01-15T18:05:00.283776+0000","last_peered":"2024-01-15T18:05:00.283776+0000","last_clean":"2024-01-15T18:05:00.283776+0000","last_became_active":"2024-01-15T18:04:58.292902+0000","last_became_peered":"2024-01-15T18:04:58.292902+0000","last_unstale":"2024-01-15T18:05:00.283776+0000","last_undegraded":"2024-01-15T18:05:00.283776+0000","last_fullsized":"2024-01-15T18:05:00.283776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539100+0000","last_change":"2024-01-15T18:04:59.313869+0000","last_active":"2024-01-15T18:05:00.539100+0000","last_peered":"2024-01-15T18:05:00.539100+0000","last_clean":"2024-01-15T18:05:00.539100+0000","last_became_active":"2024-01-15T18:04:59.313336+0000","last_became_peered":"2024-01-15T18:04:59.313336+0000","last_unstale":"2024-01-15T18:05:00.539100+0000","last_undegraded":"2024-01-15T18:05:00.539100+0000","last_fullsized":"2024-01-15T18:05:00.539100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740054+0000","last_change":"2024-01-15T18:04:58.294176+0000","last_active":"2024-01-15T18:05:00.740054+0000","last_peered":"2024-01-15T18:05:00.740054+0000","last_clean":"2024-01-15T18:05:00.740054+0000","last_became_active":"2024-01-15T18:04:58.292524+0000","last_became_peered":"2024-01-15T18:04:58.292524+0000","last_unstale":"2024-01-15T18:05:00.740054+0000","last_undegraded":"2024-01-15T18:05:00.740054+0000","last_fullsized":"2024-01-15T18:05:00.740054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740077+0000","last_change":"2024-01-15T18:04:59.300038+0000","last_active":"2024-01-15T18:05:00.740077+0000","last_peered":"2024-01-15T18:05:00.740077+0000","last_clean":"2024-01-15T18:05:00.740077+0000","last_became_active":"2024-01-15T18:04:59.299776+0000","last_became_peered":"2024-01-15T18:04:59.299776+0000","last_unstale":"2024-01-15T18:05:00.740077+0000","last_undegraded":"2024-01-15T18:05:00.740077+0000","last_fullsized":"2024-01-15T18:05:00.740077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:48.170362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353198+0000","last_change":"2024-01-15T18:04:58.297644+0000","last_active":"2024-01-15T18:05:00.353198+0000","last_peered":"2024-01-15T18:05:00.353198+0000","last_clean":"2024-01-15T18:05:00.353198+0000","last_became_active":"2024-01-15T18:04:58.296922+0000","last_became_peered":"2024-01-15T18:04:58.296922+0000","last_unstale":"2024-01-15T18:05:00.353198+0000","last_undegraded":"2024-01-15T18:05:00.353198+0000","last_fullsized":"2024-01-15T18:05:00.353198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539142+0000","last_change":"2024-01-15T18:04:59.312547+0000","last_active":"2024-01-15T18:05:00.539142+0000","last_peered":"2024-01-15T18:05:00.539142+0000","last_clean":"2024-01-15T18:05:00.539142+0000","last_became_active":"2024-01-15T18:04:59.312203+0000","last_became_peered":"2024-01-15T18:04:59.312203+0000","last_unstale":"2024-01-15T18:05:00.539142+0000","last_undegraded":"2024-01-15T18:05:00.539142+0000","last_fullsized":"2024-01-15T18:05:00.539142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:03.541828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283454+0000","last_change":"2024-01-15T18:04:58.297011+0000","last_active":"2024-01-15T18:05:00.283454+0000","last_peered":"2024-01-15T18:05:00.283454+0000","last_clean":"2024-01-15T18:05:00.283454+0000","last_became_active":"2024-01-15T18:04:58.296476+0000","last_became_peered":"2024-01-15T18:04:58.296476+0000","last_unstale":"2024-01-15T18:05:00.283454+0000","last_undegraded":"2024-01-15T18:05:00.283454+0000","last_fullsized":"2024-01-15T18:05:00.283454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:39.535650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283371+0000","last_change":"2024-01-15T18:04:59.294840+0000","last_active":"2024-01-15T18:05:00.283371+0000","last_peered":"2024-01-15T18:05:00.283371+0000","last_clean":"2024-01-15T18:05:00.283371+0000","last_became_active":"2024-01-15T18:04:59.294449+0000","last_became_peered":"2024-01-15T18:04:59.294449+0000","last_unstale":"2024-01-15T18:05:00.283371+0000","last_undegraded":"2024-01-15T18:05:00.283371+0000","last_fullsized":"2024-01-15T18:05:00.283371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:11.176403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739071+0000","last_change":"2024-01-15T18:04:58.285671+0000","last_active":"2024-01-15T18:05:00.739071+0000","last_peered":"2024-01-15T18:05:00.739071+0000","last_clean":"2024-01-15T18:05:00.739071+0000","last_became_active":"2024-01-15T18:04:58.285350+0000","last_became_peered":"2024-01-15T18:04:58.285350+0000","last_unstale":"2024-01-15T18:05:00.739071+0000","last_undegraded":"2024-01-15T18:05:00.739071+0000","last_fullsized":"2024-01-15T18:05:00.739071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538843+0000","last_change":"2024-01-15T18:04:59.315031+0000","last_active":"2024-01-15T18:05:00.538843+0000","last_peered":"2024-01-15T18:05:00.538843+0000","last_clean":"2024-01-15T18:05:00.538843+0000","last_became_active":"2024-01-15T18:04:59.311772+0000","last_became_peered":"2024-01-15T18:04:59.311772+0000","last_unstale":"2024-01-15T18:05:00.538843+0000","last_undegraded":"2024-01-15T18:05:00.538843+0000","last_fullsized":"2024-01-15T18:05:00.538843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:44:18.390119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.284479+0000","last_change":"2024-01-15T18:04:58.290373+0000","last_active":"2024-01-15T18:05:00.284479+0000","last_peered":"2024-01-15T18:05:00.284479+0000","last_clean":"2024-01-15T18:05:00.284479+0000","last_became_active":"2024-01-15T18:04:58.290060+0000","last_became_peered":"2024-01-15T18:04:58.290060+0000","last_unstale":"2024-01-15T18:05:00.284479+0000","last_undegraded":"2024-01-15T18:05:00.284479+0000","last_fullsized":"2024-01-15T18:05:00.284479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539224+0000","last_change":"2024-01-15T18:04:59.315975+0000","last_active":"2024-01-15T18:05:00.539224+0000","last_peered":"2024-01-15T18:05:00.539224+0000","last_clean":"2024-01-15T18:05:00.539224+0000","last_became_active":"2024-01-15T18:04:59.315667+0000","last_became_peered":"2024-01-15T18:04:59.315667+0000","last_unstale":"2024-01-15T18:05:00.539224+0000","last_undegraded":"2024-01-15T18:05:00.539224+0000","last_fullsized":"2024-01-15T18:05:00.539224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739314+0000","last_change":"2024-01-15T18:04:58.297988+0000","last_active":"2024-01-15T18:05:00.739314+0000","last_peered":"2024-01-15T18:05:00.739314+0000","last_clean":"2024-01-15T18:05:00.739314+0000","last_became_active":"2024-01-15T18:04:58.296427+0000","last_became_peered":"2024-01-15T18:04:58.296427+0000","last_unstale":"2024-01-15T18:05:00.739314+0000","last_undegraded":"2024-01-15T18:05:00.739314+0000","last_fullsized":"2024-01-15T18:05:00.739314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:30.756932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539392+0000","last_change":"2024-01-15T18:04:59.312511+0000","last_active":"2024-01-15T18:05:00.539392+0000","last_peered":"2024-01-15T18:05:00.539392+0000","last_clean":"2024-01-15T18:05:00.539392+0000","last_became_active":"2024-01-15T18:04:59.312195+0000","last_became_peered":"2024-01-15T18:04:59.312195+0000","last_unstale":"2024-01-15T18:05:00.539392+0000","last_undegraded":"2024-01-15T18:05:00.539392+0000","last_fullsized":"2024-01-15T18:05:00.539392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:09:03.873184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740526+0000","last_change":"2024-01-15T18:04:58.289637+0000","last_active":"2024-01-15T18:05:00.740526+0000","last_peered":"2024-01-15T18:05:00.740526+0000","last_clean":"2024-01-15T18:05:00.740526+0000","last_became_active":"2024-01-15T18:04:58.289228+0000","last_became_peered":"2024-01-15T18:04:58.289228+0000","last_unstale":"2024-01-15T18:05:00.740526+0000","last_undegraded":"2024-01-15T18:05:00.740526+0000","last_fullsized":"2024-01-15T18:05:00.740526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353542+0000","last_change":"2024-01-15T18:04:59.309405+0000","last_active":"2024-01-15T18:05:00.353542+0000","last_peered":"2024-01-15T18:05:00.353542+0000","last_clean":"2024-01-15T18:05:00.353542+0000","last_became_active":"2024-01-15T18:04:59.309119+0000","last_became_peered":"2024-01-15T18:04:59.309119+0000","last_unstale":"2024-01-15T18:05:00.353542+0000","last_undegraded":"2024-01-15T18:05:00.353542+0000","last_fullsized":"2024-01-15T18:05:00.353542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:51.704287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739208+0000","last_change":"2024-01-15T18:04:58.294285+0000","last_active":"2024-01-15T18:05:00.739208+0000","last_peered":"2024-01-15T18:05:00.739208+0000","last_clean":"2024-01-15T18:05:00.739208+0000","last_became_active":"2024-01-15T18:04:58.293970+0000","last_became_peered":"2024-01-15T18:04:58.293970+0000","last_unstale":"2024-01-15T18:05:00.739208+0000","last_undegraded":"2024-01-15T18:05:00.739208+0000","last_fullsized":"2024-01-15T18:05:00.739208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539148+0000","last_change":"2024-01-15T18:04:59.314051+0000","last_active":"2024-01-15T18:05:00.539148+0000","last_peered":"2024-01-15T18:05:00.539148+0000","last_clean":"2024-01-15T18:05:00.539148+0000","last_became_active":"2024-01-15T18:04:59.313761+0000","last_became_peered":"2024-01-15T18:04:59.313761+0000","last_unstale":"2024-01-15T18:05:00.539148+0000","last_undegraded":"2024-01-15T18:05:00.539148+0000","last_fullsized":"2024-01-15T18:05:00.539148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-01-15T18:04:59.290583+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:04:59.290583+0000","last_peered":"2024-01-15T18:04:59.290583+0000","last_clean":"2024-01-15T18:04:59.290583+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:04:59.290583+0000","last_undegraded":"2024-01-15T18:04:59.290583+0000","last_fullsized":"2024-01-15T18:04:59.290583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353134+0000","last_change":"2024-01-15T18:04:58.286159+0000","last_active":"2024-01-15T18:05:00.353134+0000","last_peered":"2024-01-15T18:05:00.353134+0000","last_clean":"2024-01-15T18:05:00.353134+0000","last_became_active":"2024-01-15T18:04:58.285819+0000","last_became_peered":"2024-01-15T18:04:58.285819+0000","last_unstale":"2024-01-15T18:05:00.353134+0000","last_undegraded":"2024-01-15T18:05:00.353134+0000","last_fullsized":"2024-01-15T18:05:00.353134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:44.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285340+0000","last_change":"2024-01-15T18:04:59.305454+0000","last_active":"2024-01-15T18:05:00.285340+0000","last_peered":"2024-01-15T18:05:00.285340+0000","last_clean":"2024-01-15T18:05:00.285340+0000","last_became_active":"2024-01-15T18:04:59.305160+0000","last_became_peered":"2024-01-15T18:04:59.305160+0000","last_unstale":"2024-01-15T18:05:00.285340+0000","last_undegraded":"2024-01-15T18:05:00.285340+0000","last_fullsized":"2024-01-15T18:05:00.285340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538748+0000","last_change":"2024-01-15T18:04:58.297164+0000","last_active":"2024-01-15T18:05:00.538748+0000","last_peered":"2024-01-15T18:05:00.538748+0000","last_clean":"2024-01-15T18:05:00.538748+0000","last_became_active":"2024-01-15T18:04:58.295969+0000","last_became_peered":"2024-01-15T18:04:58.295969+0000","last_unstale":"2024-01-15T18:05:00.538748+0000","last_undegraded":"2024-01-15T18:05:00.538748+0000","last_fullsized":"2024-01-15T18:05:00.538748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:13.699787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739741+0000","last_change":"2024-01-15T18:04:59.297889+0000","last_active":"2024-01-15T18:05:00.739741+0000","last_peered":"2024-01-15T18:05:00.739741+0000","last_clean":"2024-01-15T18:05:00.739741+0000","last_became_active":"2024-01-15T18:04:59.296874+0000","last_became_peered":"2024-01-15T18:04:59.296874+0000","last_unstale":"2024-01-15T18:05:00.739741+0000","last_undegraded":"2024-01-15T18:05:00.739741+0000","last_fullsized":"2024-01-15T18:05:00.739741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:52.200760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739266+0000","last_change":"2024-01-15T18:04:58.293459+0000","last_active":"2024-01-15T18:05:00.739266+0000","last_peered":"2024-01-15T18:05:00.739266+0000","last_clean":"2024-01-15T18:05:00.739266+0000","last_became_active":"2024-01-15T18:04:58.292396+0000","last_became_peered":"2024-01-15T18:04:58.292396+0000","last_unstale":"2024-01-15T18:05:00.739266+0000","last_undegraded":"2024-01-15T18:05:00.739266+0000","last_fullsized":"2024-01-15T18:05:00.739266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:14.183342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740002+0000","last_change":"2024-01-15T18:04:59.302244+0000","last_active":"2024-01-15T18:05:00.740002+0000","last_peered":"2024-01-15T18:05:00.740002+0000","last_clean":"2024-01-15T18:05:00.740002+0000","last_became_active":"2024-01-15T18:04:59.301981+0000","last_became_peered":"2024-01-15T18:04:59.301981+0000","last_unstale":"2024-01-15T18:05:00.740002+0000","last_undegraded":"2024-01-15T18:05:00.740002+0000","last_fullsized":"2024-01-15T18:05:00.740002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353265+0000","last_change":"2024-01-15T18:04:58.288895+0000","last_active":"2024-01-15T18:05:00.353265+0000","last_peered":"2024-01-15T18:05:00.353265+0000","last_clean":"2024-01-15T18:05:00.353265+0000","last_became_active":"2024-01-15T18:04:58.288385+0000","last_became_peered":"2024-01-15T18:04:58.288385+0000","last_unstale":"2024-01-15T18:05:00.353265+0000","last_undegraded":"2024-01-15T18:05:00.353265+0000","last_fullsized":"2024-01-15T18:05:00.353265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:10.169965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283326+0000","last_change":"2024-01-15T18:04:59.289150+0000","last_active":"2024-01-15T18:05:00.283326+0000","last_peered":"2024-01-15T18:05:00.283326+0000","last_clean":"2024-01-15T18:05:00.283326+0000","last_became_active":"2024-01-15T18:04:59.288315+0000","last_became_peered":"2024-01-15T18:04:59.288315+0000","last_unstale":"2024-01-15T18:05:00.283326+0000","last_undegraded":"2024-01-15T18:05:00.283326+0000","last_fullsized":"2024-01-15T18:05:00.283326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:14.706225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739871+0000","last_change":"2024-01-15T18:04:58.280841+0000","last_active":"2024-01-15T18:05:00.739871+0000","last_peered":"2024-01-15T18:05:00.739871+0000","last_clean":"2024-01-15T18:05:00.739871+0000","last_became_active":"2024-01-15T18:04:58.280590+0000","last_became_peered":"2024-01-15T18:04:58.280590+0000","last_unstale":"2024-01-15T18:05:00.739871+0000","last_undegraded":"2024-01-15T18:05:00.739871+0000","last_fullsized":"2024-01-15T18:05:00.739871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.414931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353597+0000","last_change":"2024-01-15T18:04:59.310649+0000","last_active":"2024-01-15T18:05:00.353597+0000","last_peered":"2024-01-15T18:05:00.353597+0000","last_clean":"2024-01-15T18:05:00.353597+0000","last_became_active":"2024-01-15T18:04:59.310266+0000","last_became_peered":"2024-01-15T18:04:59.310266+0000","last_unstale":"2024-01-15T18:05:00.353597+0000","last_undegraded":"2024-01-15T18:05:00.353597+0000","last_fullsized":"2024-01-15T18:05:00.353597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:29.781101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539301+0000","last_change":"2024-01-15T18:04:58.300881+0000","last_active":"2024-01-15T18:05:00.539301+0000","last_peered":"2024-01-15T18:05:00.539301+0000","last_clean":"2024-01-15T18:05:00.539301+0000","last_became_active":"2024-01-15T18:04:58.300322+0000","last_became_peered":"2024-01-15T18:04:58.300322+0000","last_unstale":"2024-01-15T18:05:00.539301+0000","last_undegraded":"2024-01-15T18:05:00.539301+0000","last_fullsized":"2024-01-15T18:05:00.539301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:15:47.163925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.739810+0000","last_change":"2024-01-15T18:04:59.299409+0000","last_active":"2024-01-15T18:05:00.739810+0000","last_peered":"2024-01-15T18:05:00.739810+0000","last_clean":"2024-01-15T18:05:00.739810+0000","last_became_active":"2024-01-15T18:04:59.299156+0000","last_became_peered":"2024-01-15T18:04:59.299156+0000","last_unstale":"2024-01-15T18:05:00.739810+0000","last_undegraded":"2024-01-15T18:05:00.739810+0000","last_fullsized":"2024-01-15T18:05:00.739810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:12:01.153755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:01.654927+0000","last_change":"2024-01-15T18:04:58.283804+0000","last_active":"2024-01-15T18:05:01.654927+0000","last_peered":"2024-01-15T18:05:01.654927+0000","last_clean":"2024-01-15T18:05:01.654927+0000","last_became_active":"2024-01-15T18:04:58.283533+0000","last_became_peered":"2024-01-15T18:04:58.283533+0000","last_unstale":"2024-01-15T18:05:01.654927+0000","last_undegraded":"2024-01-15T18:05:01.654927+0000","last_fullsized":"2024-01-15T18:05:01.654927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:49.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285921+0000","last_change":"2024-01-15T18:04:59.316822+0000","last_active":"2024-01-15T18:05:00.285921+0000","last_peered":"2024-01-15T18:05:00.285921+0000","last_clean":"2024-01-15T18:05:00.285921+0000","last_became_active":"2024-01-15T18:04:59.316513+0000","last_became_peered":"2024-01-15T18:04:59.316513+0000","last_unstale":"2024-01-15T18:05:00.285921+0000","last_undegraded":"2024-01-15T18:05:00.285921+0000","last_fullsized":"2024-01-15T18:05:00.285921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:23.928149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283543+0000","last_change":"2024-01-15T18:04:58.292591+0000","last_active":"2024-01-15T18:05:00.283543+0000","last_peered":"2024-01-15T18:05:00.283543+0000","last_clean":"2024-01-15T18:05:00.283543+0000","last_became_active":"2024-01-15T18:04:58.291917+0000","last_became_peered":"2024-01-15T18:04:58.291917+0000","last_unstale":"2024-01-15T18:05:00.283543+0000","last_undegraded":"2024-01-15T18:05:00.283543+0000","last_fullsized":"2024-01-15T18:05:00.283543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740270+0000","last_change":"2024-01-15T18:04:59.300764+0000","last_active":"2024-01-15T18:05:00.740270+0000","last_peered":"2024-01-15T18:05:00.740270+0000","last_clean":"2024-01-15T18:05:00.740270+0000","last_became_active":"2024-01-15T18:04:59.299980+0000","last_became_peered":"2024-01-15T18:04:59.299980+0000","last_unstale":"2024-01-15T18:05:00.740270+0000","last_undegraded":"2024-01-15T18:05:00.740270+0000","last_fullsized":"2024-01-15T18:05:00.740270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:37.137487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538729+0000","last_change":"2024-01-15T18:04:58.301082+0000","last_active":"2024-01-15T18:05:00.538729+0000","last_peered":"2024-01-15T18:05:00.538729+0000","last_clean":"2024-01-15T18:05:00.538729+0000","last_became_active":"2024-01-15T18:04:58.300809+0000","last_became_peered":"2024-01-15T18:04:58.300809+0000","last_unstale":"2024-01-15T18:05:00.538729+0000","last_undegraded":"2024-01-15T18:05:00.538729+0000","last_fullsized":"2024-01-15T18:05:00.538729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:12.018976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283231+0000","last_change":"2024-01-15T18:04:59.297502+0000","last_active":"2024-01-15T18:05:00.283231+0000","last_peered":"2024-01-15T18:05:00.283231+0000","last_clean":"2024-01-15T18:05:00.283231+0000","last_became_active":"2024-01-15T18:04:59.297211+0000","last_became_peered":"2024-01-15T18:04:59.297211+0000","last_unstale":"2024-01-15T18:05:00.283231+0000","last_undegraded":"2024-01-15T18:05:00.283231+0000","last_fullsized":"2024-01-15T18:05:00.283231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:13.025413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740211+0000","last_change":"2024-01-15T18:04:58.296402+0000","last_active":"2024-01-15T18:05:00.740211+0000","last_peered":"2024-01-15T18:05:00.740211+0000","last_clean":"2024-01-15T18:05:00.740211+0000","last_became_active":"2024-01-15T18:04:58.296101+0000","last_became_peered":"2024-01-15T18:04:58.296101+0000","last_unstale":"2024-01-15T18:05:00.740211+0000","last_undegraded":"2024-01-15T18:05:00.740211+0000","last_fullsized":"2024-01-15T18:05:00.740211+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285133+0000","last_change":"2024-01-15T18:04:59.298441+0000","last_active":"2024-01-15T18:05:00.285133+0000","last_peered":"2024-01-15T18:05:00.285133+0000","last_clean":"2024-01-15T18:05:00.285133+0000","last_became_active":"2024-01-15T18:04:59.298204+0000","last_became_peered":"2024-01-15T18:04:59.298204+0000","last_unstale":"2024-01-15T18:05:00.285133+0000","last_undegraded":"2024-01-15T18:05:00.285133+0000","last_fullsized":"2024-01-15T18:05:00.285133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740408+0000","last_change":"2024-01-15T18:04:58.297952+0000","last_active":"2024-01-15T18:05:00.740408+0000","last_peered":"2024-01-15T18:05:00.740408+0000","last_clean":"2024-01-15T18:05:00.740408+0000","last_became_active":"2024-01-15T18:04:58.297621+0000","last_became_peered":"2024-01-15T18:04:58.297621+0000","last_unstale":"2024-01-15T18:05:00.740408+0000","last_undegraded":"2024-01-15T18:05:00.740408+0000","last_fullsized":"2024-01-15T18:05:00.740408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285600+0000","last_change":"2024-01-15T18:04:59.304388+0000","last_active":"2024-01-15T18:05:00.285600+0000","last_peered":"2024-01-15T18:05:00.285600+0000","last_clean":"2024-01-15T18:05:00.285600+0000","last_became_active":"2024-01-15T18:04:59.303178+0000","last_became_peered":"2024-01-15T18:04:59.303178+0000","last_unstale":"2024-01-15T18:05:00.285600+0000","last_undegraded":"2024-01-15T18:05:00.285600+0000","last_fullsized":"2024-01-15T18:05:00.285600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:27.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539246+0000","last_change":"2024-01-15T18:04:58.300443+0000","last_active":"2024-01-15T18:05:00.539246+0000","last_peered":"2024-01-15T18:05:00.539246+0000","last_clean":"2024-01-15T18:05:00.539246+0000","last_became_active":"2024-01-15T18:04:58.300157+0000","last_became_peered":"2024-01-15T18:04:58.300157+0000","last_unstale":"2024-01-15T18:05:00.539246+0000","last_undegraded":"2024-01-15T18:05:00.539246+0000","last_fullsized":"2024-01-15T18:05:00.539246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:36.131049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.283605+0000","last_change":"2024-01-15T18:04:59.295964+0000","last_active":"2024-01-15T18:05:00.283605+0000","last_peered":"2024-01-15T18:05:00.283605+0000","last_clean":"2024-01-15T18:05:00.283605+0000","last_became_active":"2024-01-15T18:04:59.295569+0000","last_became_peered":"2024-01-15T18:04:59.295569+0000","last_unstale":"2024-01-15T18:05:00.283605+0000","last_undegraded":"2024-01-15T18:05:00.283605+0000","last_fullsized":"2024-01-15T18:05:00.283605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:34.833272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353512+0000","last_change":"2024-01-15T18:04:59.316899+0000","last_active":"2024-01-15T18:05:00.353512+0000","last_peered":"2024-01-15T18:05:00.353512+0000","last_clean":"2024-01-15T18:05:00.353512+0000","last_became_active":"2024-01-15T18:04:59.316632+0000","last_became_peered":"2024-01-15T18:04:59.316632+0000","last_unstale":"2024-01-15T18:05:00.353512+0000","last_undegraded":"2024-01-15T18:05:00.353512+0000","last_fullsized":"2024-01-15T18:05:00.353512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:21.592179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.538942+0000","last_change":"2024-01-15T18:04:58.301177+0000","last_active":"2024-01-15T18:05:00.538942+0000","last_peered":"2024-01-15T18:05:00.538942+0000","last_clean":"2024-01-15T18:05:00.538942+0000","last_became_active":"2024-01-15T18:04:58.300869+0000","last_became_peered":"2024-01-15T18:04:58.300869+0000","last_unstale":"2024-01-15T18:05:00.538942+0000","last_undegraded":"2024-01-15T18:05:00.538942+0000","last_fullsized":"2024-01-15T18:05:00.538942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.089266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.353471+0000","last_change":"2024-01-15T18:04:59.311012+0000","last_active":"2024-01-15T18:05:00.353471+0000","last_peered":"2024-01-15T18:05:00.353471+0000","last_clean":"2024-01-15T18:05:00.353471+0000","last_became_active":"2024-01-15T18:04:59.310673+0000","last_became_peered":"2024-01-15T18:04:59.310673+0000","last_unstale":"2024-01-15T18:05:00.353471+0000","last_undegraded":"2024-01-15T18:05:00.353471+0000","last_fullsized":"2024-01-15T18:05:00.353471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:03.491064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.539209+0000","last_change":"2024-01-15T18:04:58.303349+0000","last_active":"2024-01-15T18:05:00.539209+0000","last_peered":"2024-01-15T18:05:00.539209+0000","last_clean":"2024-01-15T18:05:00.539209+0000","last_became_active":"2024-01-15T18:04:58.303074+0000","last_became_peered":"2024-01-15T18:04:58.303074+0000","last_unstale":"2024-01-15T18:05:00.539209+0000","last_undegraded":"2024-01-15T18:05:00.539209+0000","last_fullsized":"2024-01-15T18:05:00.539209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740412+0000","last_change":"2024-01-15T18:04:59.311306+0000","last_active":"2024-01-15T18:05:00.740412+0000","last_peered":"2024-01-15T18:05:00.740412+0000","last_clean":"2024-01-15T18:05:00.740412+0000","last_became_active":"2024-01-15T18:04:59.311031+0000","last_became_peered":"2024-01-15T18:04:59.311031+0000","last_unstale":"2024-01-15T18:05:00.740412+0000","last_undegraded":"2024-01-15T18:05:00.740412+0000","last_fullsized":"2024-01-15T18:05:00.740412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:24.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285716+0000","last_change":"2024-01-15T18:04:58.299606+0000","last_active":"2024-01-15T18:05:00.285716+0000","last_peered":"2024-01-15T18:05:00.285716+0000","last_clean":"2024-01-15T18:05:00.285716+0000","last_became_active":"2024-01-15T18:04:58.299353+0000","last_became_peered":"2024-01-15T18:04:58.299353+0000","last_unstale":"2024-01-15T18:05:00.285716+0000","last_undegraded":"2024-01-15T18:05:00.285716+0000","last_fullsized":"2024-01-15T18:05:00.285716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:17:51.194322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.285464+0000","last_change":"2024-01-15T18:04:59.304019+0000","last_active":"2024-01-15T18:05:00.285464+0000","last_peered":"2024-01-15T18:05:00.285464+0000","last_clean":"2024-01-15T18:05:00.285464+0000","last_became_active":"2024-01-15T18:04:59.303662+0000","last_became_peered":"2024-01-15T18:04:59.303662+0000","last_unstale":"2024-01-15T18:05:00.285464+0000","last_undegraded":"2024-01-15T18:05:00.285464+0000","last_fullsized":"2024-01-15T18:05:00.285464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:58.261083+0000","last_clean_scrub_stamp":"2024-01-15T18:04:58.261083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:04:28.226375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-01-15T18:05:00.740536+0000","last_change":"2024-01-15T18:04:58.303996+0000","last_active":"2024-01-15T18:05:00.740536+0000","last_peered":"2024-01-15T18:05:00.740536+0000","last_clean":"2024-01-15T18:05:00.740536+0000","last_became_active":"2024-01-15T18:04:58.303687+0000","last_became_peered":"2024-01-15T18:04:58.303687+0000","last_unstale":"2024-01-15T18:05:00.740536+0000","last_undegraded":"2024-01-15T18:05:00.740536+0000","last_fullsized":"2024-01-15T18:05:00.740536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:57.254645+0000","last_clean_scrub_stamp":"2024-01-15T18:04:57.254645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:16.102238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7759999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.532}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6840000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.8259999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7629999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6909999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7309999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6120000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:05:08.509 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-15T18:05:08.509 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-15T18:05:08.509 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-15T18:05:08.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-15T18:05:08.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 Processor -- start 2024-01-15T18:05:08.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 -- start start 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b24b0 0x7f597c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f597c0b3bf0 0x7f597c0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c070bf0 con 0x7f597c0ce690 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597c15dba0 con 0x7f597c0b3bf0 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5983000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c0b4920 con 0x7f597c0b24b0 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f597be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f597c0ce690 0x7f597c0b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36084/0 (socket says 172.21.15.143:36084) 2024-01-15T18:05:08.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f597be00640 1 -- 172.21.15.143:0/3422923459 learned_addr learned my addr 172.21.15.143:0/3422923459 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5981000640 1 --2- 172.21.15.143:0/3422923459 >> v2:172.21.15.204:3300/0 conn(0x7f597c0b3bf0 0x7f597c0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f597b400640 1 --2- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b24b0 0x7f597c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f597aa00640 1 -- 172.21.15.143:0/3422923459 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508412058 0 0) 0x7f597c070bf0 con 0x7f597c0ce690 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f597aa00640 1 -- 172.21.15.143:0/3422923459 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f595c003530 con 0x7f597c0ce690 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5981000640 1 -- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b24b0 msgr2=0x7f597c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f597aa00640 1 -- 172.21.15.143:0/3422923459 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1335727274 0 0) 0x7f595c003530 con 0x7f597c0ce690 2024-01-15T18:05:08.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5981000640 1 --2- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b24b0 0x7f597c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5981000640 1 -- 172.21.15.143:0/3422923459 >> v1:172.21.15.143:6789/0 conn(0x7f597c0ce690 legacy=0x7f597c0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.685+0000 7f5981000640 1 -- 172.21.15.143:0/3422923459 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c0b4c00 con 0x7f597c0b3bf0 2024-01-15T18:05:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5981000640 1 --2- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0b3bf0 0x7f597c0b4080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5974009d90 tx=0x7f597401d2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd650103eeba685 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f597aa00640 1 -- 172.21.15.143:0/3422923459 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c001680 con 0x7f597c0b3bf0 2024-01-15T18:05:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f597aa00640 1 -- 172.21.15.143:0/3422923459 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740a5660 con 0x7f597c0b3bf0 2024-01-15T18:05:08.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f597aa00640 1 -- 172.21.15.143:0/3422923459 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740a59f0 con 0x7f597c0b3bf0 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 -- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0b3bf0 msgr2=0x7f597c0b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 --2- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0b3bf0 0x7f597c0b4080 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5974009d90 tx=0x7f597401d2a0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 -- 172.21.15.143:0/3422923459 shutdown_connections 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 --2- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0b3bf0 0x7f597c0b4080 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 --2- 172.21.15.143:0/3422923459 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b24b0 0x7f597c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 -- 172.21.15.143:0/3422923459 >> 172.21.15.143:0/3422923459 conn(0x7f597c06c3f0 msgr2=0x7f597c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 -- 172.21.15.143:0/3422923459 shutdown_connections 2024-01-15T18:05:08.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.689+0000 7f5983000640 1 -- 172.21.15.143:0/3422923459 wait complete. 2024-01-15T18:05:08.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 Processor -- start 2024-01-15T18:05:08.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 -- start start 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 0x7f597c1679c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b3bf0 0x7f597c0af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0ce690 0x7f597c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c13ef90 con 0x7f597c0b24b0 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c146780 con 0x7f597c0ce690 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c146a60 con 0x7f597c0b3bf0 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 0x7f597c1679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 0x7f597c1679c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48434/0 (socket says 172.21.15.143:48434) 2024-01-15T18:05:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 -- 172.21.15.143:0/4281368424 learned_addr learned my addr 172.21.15.143:0/4281368424 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597b400640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b3bf0 0x7f597c0af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5981000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0ce690 0x7f597c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 -- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b3bf0 msgr2=0x7f597c0af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b3bf0 0x7f597c0af010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 -- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0ce690 msgr2=0x7f597c0ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0ce690 0x7f597c0ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 -- 172.21.15.143:0/4281368424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974009a70 con 0x7f597c0b24b0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f597be00640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 0x7f597c1679c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c750 tx=0x7f596c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c005ae0 con 0x7f597c0b24b0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c005c40 con 0x7f597c0b24b0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c007070 con 0x7f597c0b24b0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.693+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c146d40 con 0x7f597c0b24b0 2024-01-15T18:05:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c145130 con 0x7f597c0b24b0 2024-01-15T18:05:08.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c067500 con 0x7f597c0b24b0 2024-01-15T18:05:08.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f596c0071f0 con 0x7f597c0b24b0 2024-01-15T18:05:08.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f5973e00640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59540668f0 0x7f5954068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f596c0841d0 con 0x7f597c0b24b0 2024-01-15T18:05:08.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f597b400640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59540668f0 0x7f5954068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.697+0000 7f597b400640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59540668f0 0x7f5954068db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f597c067120 tx=0x7f5964009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.701+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f596c051420 con 0x7f597c0b24b0 2024-01-15T18:05:08.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.885+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f597c06e260 con 0x7f597c0b24b0 2024-01-15T18:05:08.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.885+0000 7f5973e00640 1 -- 172.21.15.143:0/4281368424 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f596c054a40 con 0x7f597c0b24b0 2024-01-15T18:05:08.892 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:08.892 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-15T18:05:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59540668f0 msgr2=0x7f5954068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59540668f0 0x7f5954068db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f597c067120 tx=0x7f5964009190 comp rx=0 tx=0).stop 2024-01-15T18:05:08.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 msgr2=0x7f597c1679c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 0x7f597c1679c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f596c00c750 tx=0x7f596c00cc10 comp rx=0 tx=0).stop 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 shutdown_connections 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59540668f0 0x7f5954068db0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597c0ce690 0x7f597c0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597c0b3bf0 0x7f597c0af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 --2- 172.21.15.143:0/4281368424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f597c0b24b0 0x7f597c1679c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 >> 172.21.15.143:0/4281368424 conn(0x7f597c06c3f0 msgr2=0x7f597c15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 shutdown_connections 2024-01-15T18:05:08.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:08.889+0000 7f5983000640 1 -- 172.21.15.143:0/4281368424 wait complete. 2024-01-15T18:05:08.915 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-15T18:05:08.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:09.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.045+0000 7ff6ae800640 1 Processor -- start 2024-01-15T18:05:09.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.045+0000 7ff6ae800640 1 -- start start 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6ae800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6ae800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff6a8161aa0 0x7ff6a80c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6ae800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6a80cf8a0 0x7ff6a80cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6ae800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a806be70 con 0x7ff6a80cf8a0 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6ae800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a80cf410 con 0x7ff6a8161aa0 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6ae800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a806b640 con 0x7ff6a80c3e00 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6a80cf8a0 0x7ff6a80cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6a80cf8a0 0x7ff6a80cfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48448/0 (socket says 172.21.15.143:48448) 2024-01-15T18:05:09.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 -- 172.21.15.143:0/355959192 learned_addr learned my addr 172.21.15.143:0/355959192 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/355959192 >> v2:172.21.15.204:3300/0 conn(0x7ff6a8161aa0 0x7ff6a80c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7400640 1 --2- 172.21.15.143:0/355959192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 -- 172.21.15.143:0/355959192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 msgr2=0x7ff6a80b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 --2- 172.21.15.143:0/355959192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a80b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 -- 172.21.15.143:0/355959192 >> v2:172.21.15.204:3300/0 conn(0x7ff6a8161aa0 msgr2=0x7ff6a80c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 --2- 172.21.15.143:0/355959192 >> v2:172.21.15.204:3300/0 conn(0x7ff6a8161aa0 0x7ff6a80c41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 -- 172.21.15.143:0/355959192 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80b0130 con 0x7ff6a80cf8a0 2024-01-15T18:05:09.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a7e00640 1 --2- 172.21.15.143:0/355959192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80cfd30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8071a90 tx=0x7ff6a001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02122687e327fa0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a6000640 1 -- 172.21.15.143:0/355959192 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00a5810 con 0x7ff6a80cf8a0 2024-01-15T18:05:09.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a6000640 1 -- 172.21.15.143:0/355959192 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a00a5970 con 0x7ff6a80cf8a0 2024-01-15T18:05:09.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.049+0000 7ff6a6000640 1 -- 172.21.15.143:0/355959192 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00a5d00 con 0x7ff6a80cf8a0 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- 172.21.15.143:0/355959192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a80cf8a0 msgr2=0x7ff6a80cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- 172.21.15.143:0/355959192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80cfd30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8071a90 tx=0x7ff6a001c990 comp rx=0 tx=0).stop 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- 172.21.15.143:0/355959192 shutdown_connections 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- 172.21.15.143:0/355959192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80cfd30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- 172.21.15.143:0/355959192 >> v2:172.21.15.204:3300/0 conn(0x7ff6a8161aa0 0x7ff6a80c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- 172.21.15.143:0/355959192 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a80b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- 172.21.15.143:0/355959192 >> 172.21.15.143:0/355959192 conn(0x7ff6a80d0e30 msgr2=0x7ff6a80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- 172.21.15.143:0/355959192 shutdown_connections 2024-01-15T18:05:09.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- 172.21.15.143:0/355959192 wait complete. 2024-01-15T18:05:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 Processor -- start 2024-01-15T18:05:09.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- start start 2024-01-15T18:05:09.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a8156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 0x7ff6a80b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8156f80 con 0x7ff6a815e8f0 2024-01-15T18:05:09.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b5560 con 0x7ff6a80cf8a0 2024-01-15T18:05:09.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6ae800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a81571e0 con 0x7ff6a80c3e00 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6a7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 0x7ff6a80b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6a7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 0x7ff6a80b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48462/0 (socket says 172.21.15.143:48462) 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6a7e00640 1 -- 172.21.15.143:0/3110056805 learned_addr learned my addr 172.21.15.143:0/3110056805 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.053+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7400640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a8156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7e00640 1 -- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 msgr2=0x7ff6a8156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7e00640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a8156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7e00640 1 -- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80cf8a0 msgr2=0x7ff6a80b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7e00640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7e00640 1 -- 172.21.15.143:0/3110056805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0009b30 con 0x7ff6a815e8f0 2024-01-15T18:05:09.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6a7e00640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 0x7ff6a80b5020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00af040 tx=0x7ff6a0019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00b8670 con 0x7ff6a815e8f0 2024-01-15T18:05:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a00b8810 con 0x7ff6a815e8f0 2024-01-15T18:05:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00b8bc0 con 0x7ff6a815e8f0 2024-01-15T18:05:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a815e370 con 0x7ff6a815e8f0 2024-01-15T18:05:09.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a8157760 con 0x7ff6a815e8f0 2024-01-15T18:05:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a8157950 con 0x7ff6a815e8f0 2024-01-15T18:05:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff6a00b8d20 con 0x7ff6a815e8f0 2024-01-15T18:05:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff69f400640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.057+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6a011fa30 con 0x7ff6a815e8f0 2024-01-15T18:05:09.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.061+0000 7ff6a7400640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.061+0000 7ff6a7400640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff6980045f0 tx=0x7ff698008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.061+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6a00ece60 con 0x7ff6a815e8f0 2024-01-15T18:05:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.213+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6a8067500 con 0x7ff6a815e8f0 2024-01-15T18:05:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.213+0000 7ff69f400640 1 -- 172.21.15.143:0/3110056805 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ff6a00f0480 con 0x7ff6a815e8f0 2024-01-15T18:05:09.221 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:09.221 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 msgr2=0x7ff680068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff6980045f0 tx=0x7ff698008040 comp rx=0 tx=0).stop 2024-01-15T18:05:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 msgr2=0x7ff6a80b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 0x7ff6a80b5020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00af040 tx=0x7ff6a0019730 comp rx=0 tx=0).stop 2024-01-15T18:05:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 shutdown_connections 2024-01-15T18:05:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815e8f0 0x7ff6a80b5020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80cf8a0 0x7ff6a80b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 --2- 172.21.15.143:0/3110056805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a80c3e00 0x7ff6a8156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 >> 172.21.15.143:0/3110056805 conn(0x7ff6a80d0e30 msgr2=0x7ff6a815c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:09.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 shutdown_connections 2024-01-15T18:05:09.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.217+0000 7ff6ae800640 1 -- 172.21.15.143:0/3110056805 wait complete. 2024-01-15T18:05:09.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:09.244 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 Processor -- start 2024-01-15T18:05:09.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 -- start start 2024-01-15T18:05:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f503415b260 0x7f50340cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f50340d19f0 0x7f50340cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50340c4d90 con 0x7f50340c5220 2024-01-15T18:05:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f503415b640 con 0x7f50340d19f0 2024-01-15T18:05:09.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5039e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50340cf800 con 0x7f503415b260 2024-01-15T18:05:09.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f50340d19f0 0x7f50340cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f50340d19f0 0x7f50340cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58294/0 (socket says 172.21.15.143:58294) 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 -- 172.21.15.143:0/937436294 learned_addr learned my addr 172.21.15.143:0/937436294 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032a00640 1 --2- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f503415b260 0x7f50340cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 -- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f503415b260 msgr2=0x7f50340cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 --2- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f503415b260 0x7f50340cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 -- 172.21.15.143:0/937436294 >> v1:172.21.15.204:6789/0 conn(0x7f50340c5220 legacy=0x7f50340c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.377+0000 7f5032000640 1 -- 172.21.15.143:0/937436294 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50340cfae0 con 0x7f50340d19f0 2024-01-15T18:05:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.381+0000 7f5032000640 1 --2- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50340cdd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f501c009870 tx=0x7f501c01c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2be7eb9e5ed386d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.381+0000 7f5031600640 1 -- 172.21.15.143:0/937436294 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c0ab620 con 0x7f50340d19f0 2024-01-15T18:05:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.381+0000 7f5031600640 1 -- 172.21.15.143:0/937436294 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c0ab780 con 0x7f50340d19f0 2024-01-15T18:05:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.381+0000 7f5031600640 1 -- 172.21.15.143:0/937436294 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c0abb10 con 0x7f50340d19f0 2024-01-15T18:05:09.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.381+0000 7f5039e00640 1 -- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 msgr2=0x7f50340cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.381+0000 7f5039e00640 1 --2- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50340cdd50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f501c009870 tx=0x7f501c01c510 comp rx=0 tx=0).stop 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- 172.21.15.143:0/937436294 shutdown_connections 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 --2- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50340cdd50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 --2- 172.21.15.143:0/937436294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f503415b260 0x7f50340cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- 172.21.15.143:0/937436294 >> 172.21.15.143:0/937436294 conn(0x7f50340d0ed0 msgr2=0x7f50340d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- 172.21.15.143:0/937436294 shutdown_connections 2024-01-15T18:05:09.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- 172.21.15.143:0/937436294 wait complete. 2024-01-15T18:05:09.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 Processor -- start 2024-01-15T18:05:09.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- start start 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50340c5220 0x7f5034150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50341572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 0x7f5034159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503415a0d0 con 0x7f503415b260 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5034157970 con 0x7f50340d19f0 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5039e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5034157c50 con 0x7f50340c5220 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 0x7f5034159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 0x7f5034159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48476/0 (socket says 172.21.15.143:48476) 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 -- 172.21.15.143:0/3550927246 learned_addr learned my addr 172.21.15.143:0/3550927246 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5032000640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50341572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5032a00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50340c5220 0x7f5034150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 -- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50340c5220 msgr2=0x7f5034150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50340c5220 0x7f5034150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 -- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 msgr2=0x7f50341572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50341572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 -- 172.21.15.143:0/3550927246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c0094d0 con 0x7f503415b260 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.385+0000 7f5033400640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 0x7f5034159b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c730 tx=0x7f502c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c003b80 con 0x7f503415b260 2024-01-15T18:05:09.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c003ce0 con 0x7f503415b260 2024-01-15T18:05:09.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5034157f30 con 0x7f503415b260 2024-01-15T18:05:09.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c00aa30 con 0x7f503415b260 2024-01-15T18:05:09.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50341584f0 con 0x7f503415b260 2024-01-15T18:05:09.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503409bc10 con 0x7f503415b260 2024-01-15T18:05:09.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f502c00ab90 con 0x7f503415b260 2024-01-15T18:05:09.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f502b400640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f500c0668f0 0x7f500c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f502c084290 con 0x7f503415b260 2024-01-15T18:05:09.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.389+0000 7f5032a00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f500c0668f0 0x7f500c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.393+0000 7f5032a00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f500c0668f0 0x7f500c068db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5024009a50 tx=0x7f5024008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.393+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f502c051430 con 0x7f503415b260 2024-01-15T18:05:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.545+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5034067500 con 0x7f503415b260 2024-01-15T18:05:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.545+0000 7f502b400640 1 -- 172.21.15.143:0/3550927246 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f502c054a50 con 0x7f503415b260 2024-01-15T18:05:09.551 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.545+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f500c0668f0 msgr2=0x7f500c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.545+0000 7f5039e00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f500c0668f0 0x7f500c068db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5024009a50 tx=0x7f5024008040 comp rx=0 tx=0).stop 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.545+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 msgr2=0x7f5034159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.545+0000 7f5039e00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 0x7f5034159b90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c730 tx=0x7f502c00cbf0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 shutdown_connections 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f500c0668f0 0x7f500c068db0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f503415b260 0x7f5034159b90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50340d19f0 0x7f50341572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 --2- 172.21.15.143:0/3550927246 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50340c5220 0x7f5034150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 >> 172.21.15.143:0/3550927246 conn(0x7f50340d0ed0 msgr2=0x7f50340cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 shutdown_connections 2024-01-15T18:05:09.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.549+0000 7f5039e00640 1 -- 172.21.15.143:0/3550927246 wait complete. 2024-01-15T18:05:09.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:09.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.705+0000 7f9281400640 1 Processor -- start 2024-01-15T18:05:09.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.705+0000 7f9281400640 1 -- start start 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f9281400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c143780 0x7f927c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f9281400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f927c0722c0 0x7f927c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f9281400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c0ceb50 con 0x7f927c0ce520 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f9281400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c15c110 con 0x7f927c0722c0 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f9281400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c15b7e0 con 0x7f927c143780 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f927c0722c0 0x7f927c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f927c0722c0 0x7f927c15c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48482/0 (socket says 172.21.15.143:48482) 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 -- 172.21.15.143:0/3689124518 learned_addr learned my addr 172.21.15.143:0/3689124518 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927a800640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c143780 0x7f927c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 -- 172.21.15.143:0/3689124518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c143780 msgr2=0x7f927c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c143780 0x7f927c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 -- 172.21.15.143:0/3689124518 >> v1:172.21.15.204:6789/0 conn(0x7f927c0ce520 legacy=0x7f927c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927b200640 1 -- 172.21.15.143:0/3689124518 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c15bac0 con 0x7f927c0722c0 2024-01-15T18:05:09.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.709+0000 7f927a800640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c143780 0x7f927c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:09.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f927b200640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0722c0 0x7f927c15c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9274009e50 tx=0x7f927401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed4da9615660a89 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9279400640 1 -- 172.21.15.143:0/3689124518 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740a57f0 con 0x7f927c0722c0 2024-01-15T18:05:09.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9279400640 1 -- 172.21.15.143:0/3689124518 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92740a5950 con 0x7f927c0722c0 2024-01-15T18:05:09.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9279400640 1 -- 172.21.15.143:0/3689124518 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92740a5ce0 con 0x7f927c0722c0 2024-01-15T18:05:09.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 -- 172.21.15.143:0/3689124518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0722c0 msgr2=0x7f927c15c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0722c0 0x7f927c15c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9274009e50 tx=0x7f927401ca10 comp rx=0 tx=0).stop 2024-01-15T18:05:09.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 -- 172.21.15.143:0/3689124518 shutdown_connections 2024-01-15T18:05:09.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0722c0 0x7f927c15c5a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 --2- 172.21.15.143:0/3689124518 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c143780 0x7f927c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 -- 172.21.15.143:0/3689124518 >> 172.21.15.143:0/3689124518 conn(0x7f927c06c3f0 msgr2=0x7f927c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:09.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 -- 172.21.15.143:0/3689124518 shutdown_connections 2024-01-15T18:05:09.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 -- 172.21.15.143:0/3689124518 wait complete. 2024-01-15T18:05:09.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.713+0000 7f9281400640 1 Processor -- start 2024-01-15T18:05:09.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- start start 2024-01-15T18:05:09.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c0722c0 0x7f927c152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 0x7f927c159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c143780 0x7f927c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c1529a0 con 0x7f927c0ce520 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c13a000 con 0x7f927c143780 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927c152bb0 con 0x7f927c0722c0 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 0x7f927c159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 0x7f927c159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48494/0 (socket says 172.21.15.143:48494) 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 -- 172.21.15.143:0/1566220195 learned_addr learned my addr 172.21.15.143:0/1566220195 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f927a800640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c0722c0 0x7f927c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f927b200640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c143780 0x7f927c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 -- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c0722c0 msgr2=0x7f927c152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c0722c0 0x7f927c152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 -- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c143780 msgr2=0x7f927c158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c143780 0x7f927c158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 -- 172.21.15.143:0/1566220195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274009b30 con 0x7f927c0ce520 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9279e00640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 0x7f927c159bb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f926400c500 tx=0x7f926400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264005220 con 0x7f927c0ce520 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264005380 con 0x7f927c0ce520 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c152e90 con 0x7f927c0ce520 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264007270 con 0x7f927c0ce520 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c13a560 con 0x7f927c0ce520 2024-01-15T18:05:09.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.717+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c13a780 con 0x7f927c0ce520 2024-01-15T18:05:09.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.721+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f92640073d0 con 0x7f927c0ce520 2024-01-15T18:05:09.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.721+0000 7f9272a00640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9254066920 0x7f9254068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:09.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.721+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9264083a30 con 0x7f927c0ce520 2024-01-15T18:05:09.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.721+0000 7f927a800640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9254066920 0x7f9254068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:09.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.721+0000 7f927a800640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9254066920 0x7f9254068de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f927c159b40 tx=0x7f926c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:09.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.725+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9264050d30 con 0x7f927c0ce520 2024-01-15T18:05:09.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.873+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f927c0ceb50 con 0x7f927c0ce520 2024-01-15T18:05:09.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.873+0000 7f9272a00640 1 -- 172.21.15.143:0/1566220195 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f927c0ceb50 con 0x7f927c0ce520 2024-01-15T18:05:09.881 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9254066920 msgr2=0x7f9254068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9254066920 0x7f9254068de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f927c159b40 tx=0x7f926c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 msgr2=0x7f927c159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 0x7f927c159bb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f926400c500 tx=0x7f926400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 shutdown_connections 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9254066920 0x7f9254068de0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f927c143780 0x7f927c158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f927c0ce520 0x7f927c159bb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 --2- 172.21.15.143:0/1566220195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f927c0722c0 0x7f927c152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:09.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 >> 172.21.15.143:0/1566220195 conn(0x7f927c06c3f0 msgr2=0x7f927c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:09.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 shutdown_connections 2024-01-15T18:05:09.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:09.877+0000 7f9281400640 1 -- 172.21.15.143:0/1566220195 wait complete. 2024-01-15T18:05:09.903 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:10.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03e0e00640 1 Processor -- start 2024-01-15T18:05:10.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03e0e00640 1 -- start start 2024-01-15T18:05:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03e0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03dc0cdf20 0x7f03dc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03e0e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc0c4d90 con 0x7f03dc0c6230 2024-01-15T18:05:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03e0e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc15b640 con 0x7f03dc15b260 2024-01-15T18:05:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03e0e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0cf800 con 0x7f03dc0cdf20 2024-01-15T18:05:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03da800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f03dc0c6230 0x7f03dc0cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36100/0 (socket says 172.21.15.143:36100) 2024-01-15T18:05:10.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03da800640 1 -- 172.21.15.143:0/2774177021 learned_addr learned my addr 172.21.15.143:0/2774177021 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03d9400640 1 --2- 172.21.15.143:0/2774177021 >> v2:172.21.15.143:3300/0 conn(0x7f03dc0cdf20 0x7f03dc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03d9400640 1 -- 172.21.15.143:0/2774177021 >> v1:172.21.15.204:6789/0 conn(0x7f03dc15b260 legacy=0x7f03dc0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03d9400640 1 -- 172.21.15.143:0/2774177021 >> v1:172.21.15.143:6789/0 conn(0x7f03dc0c6230 legacy=0x7f03dc0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03d8a00640 1 -- 172.21.15.143:0/2774177021 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281141837 0 0) 0x7f03dc0c4d90 con 0x7f03dc0c6230 2024-01-15T18:05:10.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.037+0000 7f03d9400640 1 -- 172.21.15.143:0/2774177021 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0cfae0 con 0x7f03dc0cdf20 2024-01-15T18:05:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03d9400640 1 --2- 172.21.15.143:0/2774177021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc0cbae0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03c40097c0 tx=0x7f03c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66885fb86c68c640 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03d8a00640 1 -- 172.21.15.143:0/2774177021 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40ab650 con 0x7f03dc0cdf20 2024-01-15T18:05:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03d8a00640 1 -- 172.21.15.143:0/2774177021 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0ccf40 con 0x7f03dc0cdf20 2024-01-15T18:05:10.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03d8a00640 1 -- 172.21.15.143:0/2774177021 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40ab9d0 con 0x7f03dc0cdf20 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 -- 172.21.15.143:0/2774177021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 msgr2=0x7f03dc0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 --2- 172.21.15.143:0/2774177021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc0cbae0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03c40097c0 tx=0x7f03c401c450 comp rx=0 tx=0).stop 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 -- 172.21.15.143:0/2774177021 shutdown_connections 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 --2- 172.21.15.143:0/2774177021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc0cbae0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 -- 172.21.15.143:0/2774177021 >> 172.21.15.143:0/2774177021 conn(0x7f03dc0d0ed0 msgr2=0x7f03dc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 -- 172.21.15.143:0/2774177021 shutdown_connections 2024-01-15T18:05:10.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.041+0000 7f03e0e00640 1 -- 172.21.15.143:0/2774177021 wait complete. 2024-01-15T18:05:10.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 Processor -- start 2024-01-15T18:05:10.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 -- start start 2024-01-15T18:05:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03dc0c6230 0x7f03dc151580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03dc15b260 0x7f03dc1f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc152630 con 0x7f03dc0cdf20 2024-01-15T18:05:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc153e70 con 0x7f03dc15b260 2024-01-15T18:05:10.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc1540e0 con 0x7f03dc0c6230 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48506/0 (socket says 172.21.15.143:48506) 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 -- 172.21.15.143:0/3771713957 learned_addr learned my addr 172.21.15.143:0/3771713957 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03da800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03dc15b260 0x7f03dc1f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03dc0c6230 0x7f03dc151580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 -- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03dc0c6230 msgr2=0x7f03dc151580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03dc0c6230 0x7f03dc151580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 -- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03dc15b260 msgr2=0x7f03dc1f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03dc15b260 0x7f03dc1f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 -- 172.21.15.143:0/3771713957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4009420 con 0x7f03dc0cdf20 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d9400640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc159360 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f03c4019060 tx=0x7f03c401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40bf610 con 0x7f03dc0cdf20 2024-01-15T18:05:10.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c40bf790 con 0x7f03dc0cdf20 2024-01-15T18:05:10.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c40bfb40 con 0x7f03dc0cdf20 2024-01-15T18:05:10.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.045+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc1543c0 con 0x7f03dc0cdf20 2024-01-15T18:05:10.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc154980 con 0x7f03dc0cdf20 2024-01-15T18:05:10.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc067500 con 0x7f03dc0cdf20 2024-01-15T18:05:10.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f03c40b5400 con 0x7f03dc0cdf20 2024-01-15T18:05:10.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03d2000640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03b40668f0 0x7f03b4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03c411ef50 con 0x7f03dc0cdf20 2024-01-15T18:05:10.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03d9e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03b40668f0 0x7f03b4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.049+0000 7f03d9e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03b40668f0 0x7f03b4068db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03cc007fd0 tx=0x7f03cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.053+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03c40ec110 con 0x7f03dc0cdf20 2024-01-15T18:05:10.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.205+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03dc0b6d50 con 0x7f03dc0cdf20 2024-01-15T18:05:10.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.205+0000 7f03d2000640 1 -- 172.21.15.143:0/3771713957 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03c40ef730 con 0x7f03dc0cdf20 2024-01-15T18:05:10.211 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.205+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03b40668f0 msgr2=0x7f03b4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.205+0000 7f03e0e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03b40668f0 0x7f03b4068db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f03cc007fd0 tx=0x7f03cc009040 comp rx=0 tx=0).stop 2024-01-15T18:05:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.205+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 msgr2=0x7f03dc159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.205+0000 7f03e0e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc159360 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f03c4019060 tx=0x7f03c401ac90 comp rx=0 tx=0).stop 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 shutdown_connections 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03b40668f0 0x7f03b4068db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03dc15b260 0x7f03dc1f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03dc0cdf20 0x7f03dc159360 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 --2- 172.21.15.143:0/3771713957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03dc0c6230 0x7f03dc151580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 >> 172.21.15.143:0/3771713957 conn(0x7f03dc0d0ed0 msgr2=0x7f03dc0c87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 shutdown_connections 2024-01-15T18:05:10.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.209+0000 7f03e0e00640 1 -- 172.21.15.143:0/3771713957 wait complete. 2024-01-15T18:05:10.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:10.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.361+0000 7f424fc00640 1 Processor -- start 2024-01-15T18:05:10.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424fc00640 1 -- start start 2024-01-15T18:05:10.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424fc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f42480b0490 0x7f424806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424fc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480cb480 con 0x7f424806c460 2024-01-15T18:05:10.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424fc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42480cb6d0 con 0x7f424815d3f0 2024-01-15T18:05:10.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424fc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424806c8d0 con 0x7f42480b0490 2024-01-15T18:05:10.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f424806c460 0x7f42480b1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36108/0 (socket says 172.21.15.143:36108) 2024-01-15T18:05:10.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424dc00640 1 -- 172.21.15.143:0/2082169805 learned_addr learned my addr 172.21.15.143:0/2082169805 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 --2- 172.21.15.143:0/2082169805 >> v2:172.21.15.204:3300/0 conn(0x7f42480b0490 0x7f424806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227581787 0 0) 0x7f42480cb480 con 0x7f424806c460 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4228003530 con 0x7f424806c460 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4036209812 0 0) 0x7f4228003530 con 0x7f424806c460 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 >> v2:172.21.15.204:3300/0 conn(0x7f42480b0490 msgr2=0x7f424806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 --2- 172.21.15.143:0/2082169805 >> v2:172.21.15.204:3300/0 conn(0x7f42480b0490 0x7f424806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 >> v1:172.21.15.204:6789/0 conn(0x7f424815d3f0 legacy=0x7f42480cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480cb9b0 con 0x7f424806c460 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4240001a00 con 0x7f424806c460 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 >> v1:172.21.15.143:6789/0 conn(0x7f424806c460 legacy=0x7f42480b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4228003980 0x7f4228003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42280047d0 0x7f4228004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228005190 0x7f422800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424806c8d0 con 0x7f42280047d0 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422800c030 con 0x7f4228005190 2024-01-15T18:05:10.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422800c1f0 con 0x7f4228003980 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42280047d0 0x7f4228004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424dc00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228005190 0x7f422800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424d200640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4228003980 0x7f4228003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 -- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4228003980 msgr2=0x7f4228003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4228003980 0x7f4228003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 -- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228005190 msgr2=0x7f422800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4228005190 0x7f422800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f4247e00640 1 -- 172.21.15.143:0/2082169805 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228003530 con 0x7f42280047d0 2024-01-15T18:05:10.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.365+0000 7f424dc00640 1 -- 172.21.15.143:0/2082169805 reap_dead start 2024-01-15T18:05:10.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f4247e00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42280047d0 0x7f4228004c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4230009ae0 tx=0x7f423001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484d25a76092e9ae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42300a5530 con 0x7f42280047d0 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42300a5690 con 0x7f42280047d0 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f4247400640 1 -- 172.21.15.143:0/2082169805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42300a5a40 con 0x7f42280047d0 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 -- 172.21.15.143:0/2082169805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42280047d0 msgr2=0x7f4228004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42280047d0 0x7f4228004c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4230009ae0 tx=0x7f423001ca50 comp rx=0 tx=0).stop 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 -- 172.21.15.143:0/2082169805 shutdown_connections 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 --2- 172.21.15.143:0/2082169805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42280047d0 0x7f4228004c50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 -- 172.21.15.143:0/2082169805 >> 172.21.15.143:0/2082169805 conn(0x7f42480d0e30 msgr2=0x7f42480d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 -- 172.21.15.143:0/2082169805 shutdown_connections 2024-01-15T18:05:10.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.369+0000 7f424fc00640 1 -- 172.21.15.143:0/2082169805 wait complete. 2024-01-15T18:05:10.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 Processor -- start 2024-01-15T18:05:10.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 -- start start 2024-01-15T18:05:10.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280047d0 0x7f42481526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 0x7f42480c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424815ddd0 0x7f424806ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4248152c30 con 0x7f424815d950 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480c53b0 con 0x7f424815ddd0 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42480b4d30 con 0x7f42280047d0 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 0x7f42480c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 0x7f42480c4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48514/0 (socket says 172.21.15.143:48514) 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 -- 172.21.15.143:0/2276624646 learned_addr learned my addr 172.21.15.143:0/2276624646 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:10.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424dc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424815ddd0 0x7f424806ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424d200640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280047d0 0x7f42481526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 -- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280047d0 msgr2=0x7f42481526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280047d0 0x7f42481526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 -- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424815ddd0 msgr2=0x7f424806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424815ddd0 0x7f424806ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 -- 172.21.15.143:0/2276624646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42480b5010 con 0x7f424815d950 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424d200640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280047d0 0x7f42481526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4247e00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 0x7f42480c4960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f423001c880 tx=0x7f42300197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230004020 con 0x7f424815d950 2024-01-15T18:05:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42300041c0 con 0x7f424815d950 2024-01-15T18:05:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42480b52f0 con 0x7f424815d950 2024-01-15T18:05:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42300b8660 con 0x7f424815d950 2024-01-15T18:05:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.373+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480cabb0 con 0x7f424815d950 2024-01-15T18:05:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.377+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42480d1a90 con 0x7f424815d950 2024-01-15T18:05:10.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.377+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f42300b4070 con 0x7f424815d950 2024-01-15T18:05:10.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.377+0000 7f4244c00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4220066920 0x7f4220068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.377+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f423011edb0 con 0x7f424815d950 2024-01-15T18:05:10.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.377+0000 7f424d200640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4220066920 0x7f4220068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.377+0000 7f424d200640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4220066920 0x7f4220068de0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c10 tx=0x7f4238009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.381+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42300ec090 con 0x7f424815d950 2024-01-15T18:05:10.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.561+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42480b5480 con 0x7f424815d950 2024-01-15T18:05:10.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.561+0000 7f4244c00640 1 -- 172.21.15.143:0/2276624646 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f42300ef6b0 con 0x7f424815d950 2024-01-15T18:05:10.567 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:10.567 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:10.568 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4220066920 msgr2=0x7f4220068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4220066920 0x7f4220068de0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4238000c10 tx=0x7f4238009040 comp rx=0 tx=0).stop 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 msgr2=0x7f42480c4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 0x7f42480c4960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f423001c880 tx=0x7f42300197f0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 shutdown_connections 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4220066920 0x7f4220068de0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f424815ddd0 0x7f424806ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f424815d950 0x7f42480c4960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 --2- 172.21.15.143:0/2276624646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f42280047d0 0x7f42481526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 >> 172.21.15.143:0/2276624646 conn(0x7f42480d0e30 msgr2=0x7f42480d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 shutdown_connections 2024-01-15T18:05:10.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.565+0000 7f424fc00640 1 -- 172.21.15.143:0/2276624646 wait complete. 2024-01-15T18:05:10.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe193000640 1 Processor -- start 2024-01-15T18:05:10.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe193000640 1 -- start start 2024-01-15T18:05:10.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe193000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 0x7fe18c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe193000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c15f310 con 0x7fe18c141660 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe193000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c15f5f0 con 0x7fe18c071d00 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe193000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15fbb0 con 0x7fe18c0ce560 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe18be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe18c141660 0x7fe18c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36118/0 (socket says 172.21.15.143:36118) 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe18be00640 1 -- 172.21.15.143:0/1546684183 learned_addr learned my addr 172.21.15.143:0/1546684183 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe18b400640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 0x7fe18c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508435490 0 0) 0x7fe18c15f310 con 0x7fe18c141660 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.721+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe16c003530 con 0x7fe18c141660 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207103434 0 0) 0x7fe16c003530 con 0x7fe18c141660 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 msgr2=0x7fe18c15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 0x7fe18c15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 >> v1:172.21.15.204:6789/0 conn(0x7fe18c071d00 legacy=0x7fe18c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c15f8d0 con 0x7fe18c141660 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fe17c001680 con 0x7fe18c141660 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 >> v1:172.21.15.143:6789/0 conn(0x7fe18c141660 legacy=0x7fe18c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c0039b0 0x7fe16c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 0x7fe16c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c004d10 0x7fe16c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c15fbb0 con 0x7fe16c0039b0 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c00ba40 con 0x7fe16c004d10 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c00bbb0 con 0x7fe18c0ce560 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c0039b0 0x7fe16c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18b400640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 0x7fe16c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe191000640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c004d10 0x7fe16c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 -- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 msgr2=0x7fe16c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ce560 0x7fe16c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 -- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c004d10 msgr2=0x7fe16c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c004d10 0x7fe16c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 -- 172.21.15.143:0/1546684183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c003530 con 0x7fe16c0039b0 2024-01-15T18:05:10.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.725+0000 7fe18be00640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c0039b0 0x7fe16c003d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c190 tx=0x7fe17c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af55e619a3a2e1b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c005ae0 con 0x7fe16c0039b0 2024-01-15T18:05:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17c005c40 con 0x7fe16c0039b0 2024-01-15T18:05:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe18aa00640 1 -- 172.21.15.143:0/1546684183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17c0a5860 con 0x7fe16c0039b0 2024-01-15T18:05:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- 172.21.15.143:0/1546684183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c0039b0 msgr2=0x7fe16c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 --2- 172.21.15.143:0/1546684183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe16c0039b0 0x7fe16c003d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00c190 tx=0x7fe17c01cbf0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe191000640 1 -- 172.21.15.143:0/1546684183 reap_dead start 2024-01-15T18:05:10.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- 172.21.15.143:0/1546684183 shutdown_connections 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- 172.21.15.143:0/1546684183 >> 172.21.15.143:0/1546684183 conn(0x7fe18c06c3f0 msgr2=0x7fe18c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- 172.21.15.143:0/1546684183 shutdown_connections 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- 172.21.15.143:0/1546684183 wait complete. 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 Processor -- start 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- start start 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c0039b0 0x7fe18c1406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 0x7fe18c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ac8e0 0x7fe18c0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0ce9e0 con 0x7fe18c0b37b0 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0ad2d0 con 0x7fe16c0039b0 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.729+0000 7fe193000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c140c20 con 0x7fe18c0ac8e0 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 0x7fe18c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 0x7fe18c0cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48532/0 (socket says 172.21.15.143:48532) 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 -- 172.21.15.143:0/3975426517 learned_addr learned my addr 172.21.15.143:0/3975426517 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18be00640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c0039b0 0x7fe18c1406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe191000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ac8e0 0x7fe18c0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 -- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ac8e0 msgr2=0x7fe18c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ac8e0 0x7fe18c0acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 -- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c0039b0 msgr2=0x7fe18c1406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c0039b0 0x7fe18c1406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 -- 172.21.15.143:0/3975426517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c0cd150 con 0x7fe18c0b37b0 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe18b400640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 0x7fe18c0cb690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe174011610 tx=0x7fe174011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400f8f0 con 0x7fe18c0b37b0 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17400fa50 con 0x7fe18c0b37b0 2024-01-15T18:05:10.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17400fdc0 con 0x7fe18c0b37b0 2024-01-15T18:05:10.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c0b3320 con 0x7fe18c0b37b0 2024-01-15T18:05:10.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c0b2710 con 0x7fe18c0b37b0 2024-01-15T18:05:10.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c0b34b0 con 0x7fe18c0b37b0 2024-01-15T18:05:10.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.733+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe174015460 con 0x7fe18c0b37b0 2024-01-15T18:05:10.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.737+0000 7fe183e00640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1640668f0 0x7fe164068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.737+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe174084870 con 0x7fe18c0b37b0 2024-01-15T18:05:10.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.737+0000 7fe18be00640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1640668f0 0x7fe164068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.737+0000 7fe18be00640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1640668f0 0x7fe164068db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0ab000 tx=0x7fe17c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.741+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe174051aa0 con 0x7fe18c0b37b0 2024-01-15T18:05:10.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.893+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe18c067500 con 0x7fe18c0b37b0 2024-01-15T18:05:10.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.893+0000 7fe183e00640 1 -- 172.21.15.143:0/3975426517 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fe1740550c0 con 0x7fe18c0b37b0 2024-01-15T18:05:10.900 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:10.900 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:10.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1640668f0 msgr2=0x7fe164068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1640668f0 0x7fe164068db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0ab000 tx=0x7fe17c0a7070 comp rx=0 tx=0).stop 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 msgr2=0x7fe18c0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 0x7fe18c0cb690 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe174011610 tx=0x7fe174011ad0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 shutdown_connections 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1640668f0 0x7fe164068db0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe18c0ac8e0 0x7fe18c0acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe18c0b37b0 0x7fe18c0cb690 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 --2- 172.21.15.143:0/3975426517 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe16c0039b0 0x7fe18c1406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 >> 172.21.15.143:0/3975426517 conn(0x7fe18c06c3f0 msgr2=0x7fe18c166610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 shutdown_connections 2024-01-15T18:05:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:10.897+0000 7fe193000640 1 -- 172.21.15.143:0/3975426517 wait complete. 2024-01-15T18:05:10.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:10.922 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:11.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.053+0000 7f0090400640 1 Processor -- start 2024-01-15T18:05:11.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.053+0000 7f0090400640 1 -- start start 2024-01-15T18:05:11.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f0090400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 0x7f0088072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f0090400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f008815c5a0 0x7f008815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f0090400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00880ceb50 con 0x7f0088143780 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f0090400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0088072570 con 0x7f0088071bb0 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f0090400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008815c110 con 0x7f008815c5a0 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 0x7f0088072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f008815c5a0 0x7f008815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 0x7f0088072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58366/0 (socket says 172.21.15.143:58366) 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008d000640 1 -- 172.21.15.143:0/4184414615 learned_addr learned my addr 172.21.15.143:0/4184414615 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:11.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008da00640 1 --1- 172.21.15.143:0/4184414615 >> v1:172.21.15.204:6789/0 conn(0x7f0088143780 0x7f00880cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33296/0 (socket says 172.21.15.143:33296) 2024-01-15T18:05:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008e400640 1 -- 172.21.15.143:0/4184414615 >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 msgr2=0x7f0088072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008e400640 1 --2- 172.21.15.143:0/4184414615 >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 0x7f0088072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008e400640 1 -- 172.21.15.143:0/4184414615 >> v1:172.21.15.204:6789/0 conn(0x7f0088143780 legacy=0x7f00880cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008e400640 1 -- 172.21.15.143:0/4184414615 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008815b7e0 con 0x7f008815c5a0 2024-01-15T18:05:11.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.057+0000 7f008d000640 1 --2- 172.21.15.143:0/4184414615 >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 0x7f0088072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:11.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f008e400640 1 --2- 172.21.15.143:0/4184414615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f008815c5a0 0x7f008815ca30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0080009d90 tx=0x7f008001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d48f72c0d95011f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0087e00640 1 -- 172.21.15.143:0/4184414615 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800a5a10 con 0x7f008815c5a0 2024-01-15T18:05:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0087e00640 1 -- 172.21.15.143:0/4184414615 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00800a5b70 con 0x7f008815c5a0 2024-01-15T18:05:11.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0087e00640 1 -- 172.21.15.143:0/4184414615 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800ad440 con 0x7f008815c5a0 2024-01-15T18:05:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 -- 172.21.15.143:0/4184414615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f008815c5a0 msgr2=0x7f008815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 --2- 172.21.15.143:0/4184414615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f008815c5a0 0x7f008815ca30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0080009d90 tx=0x7f008001c950 comp rx=0 tx=0).stop 2024-01-15T18:05:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 -- 172.21.15.143:0/4184414615 shutdown_connections 2024-01-15T18:05:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 --2- 172.21.15.143:0/4184414615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f008815c5a0 0x7f008815ca30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 --2- 172.21.15.143:0/4184414615 >> v2:172.21.15.204:3300/0 conn(0x7f0088071bb0 0x7f0088072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 -- 172.21.15.143:0/4184414615 >> 172.21.15.143:0/4184414615 conn(0x7f008806c3f0 msgr2=0x7f008806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 -- 172.21.15.143:0/4184414615 shutdown_connections 2024-01-15T18:05:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.061+0000 7f0090400640 1 -- 172.21.15.143:0/4184414615 wait complete. 2024-01-15T18:05:11.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 Processor -- start 2024-01-15T18:05:11.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- start start 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088071bb0 0x7f0088158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0088143780 0x7f008813a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 0x7f008813f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008813ad80 con 0x7f008815c5a0 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0088159a20 con 0x7f0088071bb0 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008813b040 con 0x7f0088143780 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 0x7f008813f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 0x7f008813f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48534/0 (socket says 172.21.15.143:48534) 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 -- 172.21.15.143:0/162014033 learned_addr learned my addr 172.21.15.143:0/162014033 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008d000640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0088143780 0x7f008813a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008da00640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088071bb0 0x7f0088158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 -- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0088143780 msgr2=0x7f008813a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0088143780 0x7f008813a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 -- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088071bb0 msgr2=0x7f0088158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088071bb0 0x7f0088158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 -- 172.21.15.143:0/162014033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700094d0 con 0x7f008815c5a0 2024-01-15T18:05:11.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f008e400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 0x7f008813f1d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f008001b000 tx=0x7f00800196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800adc60 con 0x7f008815c5a0 2024-01-15T18:05:11.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00800addc0 con 0x7f008815c5a0 2024-01-15T18:05:11.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0080009a70 con 0x7f008815c5a0 2024-01-15T18:05:11.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800b9650 con 0x7f008815c5a0 2024-01-15T18:05:11.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008813b600 con 0x7f008815c5a0 2024-01-15T18:05:11.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.065+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0088067500 con 0x7f008815c5a0 2024-01-15T18:05:11.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.069+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f00800b97b0 con 0x7f008815c5a0 2024-01-15T18:05:11.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.069+0000 7f0085600640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00600668f0 0x7f0060068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.069+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f008011f910 con 0x7f008815c5a0 2024-01-15T18:05:11.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.069+0000 7f008da00640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00600668f0 0x7f0060068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.069+0000 7f008da00640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00600668f0 0x7f0060068db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0078000f70 tx=0x7f0078007e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.073+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00800ecb60 con 0x7f008815c5a0 2024-01-15T18:05:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.221+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0088159cc0 con 0x7f008815c5a0 2024-01-15T18:05:11.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.221+0000 7f0085600640 1 -- 172.21.15.143:0/162014033 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0080009450 con 0x7f008815c5a0 2024-01-15T18:05:11.229 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00600668f0 msgr2=0x7f0060068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00600668f0 0x7f0060068db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0078000f70 tx=0x7f0078007e10 comp rx=0 tx=0).stop 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 msgr2=0x7f008813f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 0x7f008813f1d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f008001b000 tx=0x7f00800196f0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 shutdown_connections 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00600668f0 0x7f0060068db0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f008815c5a0 0x7f008813f1d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0088143780 0x7f008813a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 --2- 172.21.15.143:0/162014033 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0088071bb0 0x7f0088158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 >> 172.21.15.143:0/162014033 conn(0x7f008806c3f0 msgr2=0x7f008806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 shutdown_connections 2024-01-15T18:05:11.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.225+0000 7f0090400640 1 -- 172.21.15.143:0/162014033 wait complete. 2024-01-15T18:05:11.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:11.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f61c00640 1 Processor -- start 2024-01-15T18:05:11.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f61c00640 1 -- start start 2024-01-15T18:05:11.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f61c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f5c072030 0x7f6f5c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f61c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c0ceb50 con 0x7f6f5c071bb0 2024-01-15T18:05:11.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f61c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c15c110 con 0x7f6f5c143780 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f61c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c15b7e0 con 0x7f6f5c072030 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f5b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6f5c072030 0x7f6f5c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f5a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6f5c071bb0 0x7f6f5c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36128/0 (socket says 172.21.15.143:36128) 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f5a000640 1 -- 172.21.15.143:0/3927083994 learned_addr learned my addr 172.21.15.143:0/3927083994 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f5b400640 1 -- 172.21.15.143:0/3927083994 >> v1:172.21.15.204:6789/0 conn(0x7f6f5c143780 legacy=0x7f6f5c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f5b400640 1 -- 172.21.15.143:0/3927083994 >> v1:172.21.15.143:6789/0 conn(0x7f6f5c071bb0 legacy=0x7f6f5c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f5b400640 1 -- 172.21.15.143:0/3927083994 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c15bac0 con 0x7f6f5c072030 2024-01-15T18:05:11.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.385+0000 7f6f59600640 1 -- 172.21.15.143:0/3927083994 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323122502 0 0) 0x7f6f44001120 con 0x7f6f5c071bb0 2024-01-15T18:05:11.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f5b400640 1 --2- 172.21.15.143:0/3927083994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c072030 0x7f6f5c15ccb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009d90 tx=0x7f6f5401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b282d1c88b455e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f59600640 1 -- 172.21.15.143:0/3927083994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540a59d0 con 0x7f6f5c072030 2024-01-15T18:05:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f59600640 1 -- 172.21.15.143:0/3927083994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f540a5b30 con 0x7f6f5c072030 2024-01-15T18:05:11.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f59600640 1 -- 172.21.15.143:0/3927083994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540ad440 con 0x7f6f5c072030 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 -- 172.21.15.143:0/3927083994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c072030 msgr2=0x7f6f5c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 --2- 172.21.15.143:0/3927083994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c072030 0x7f6f5c15ccb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009d90 tx=0x7f6f5401c950 comp rx=0 tx=0).stop 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 -- 172.21.15.143:0/3927083994 shutdown_connections 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 --2- 172.21.15.143:0/3927083994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c072030 0x7f6f5c15ccb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 -- 172.21.15.143:0/3927083994 >> 172.21.15.143:0/3927083994 conn(0x7f6f5c06c3f0 msgr2=0x7f6f5c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 -- 172.21.15.143:0/3927083994 shutdown_connections 2024-01-15T18:05:11.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.389+0000 7f6f61c00640 1 -- 172.21.15.143:0/3927083994 wait complete. 2024-01-15T18:05:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 Processor -- start 2024-01-15T18:05:11.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- start start 2024-01-15T18:05:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c071bb0 0x7f6f5c137230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f5c072030 0x7f6f5c1599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 0x7f6f5c130ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c0accc0 con 0x7f6f5c143780 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c135740 con 0x7f6f5c071bb0 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c135460 con 0x7f6f5c072030 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 0x7f6f5c130ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 0x7f6f5c130ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48558/0 (socket says 172.21.15.143:48558) 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 -- 172.21.15.143:0/658229446 learned_addr learned my addr 172.21.15.143:0/658229446 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5aa00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c071bb0 0x7f6f5c137230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5a000640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f5c072030 0x7f6f5c1599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 -- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f5c072030 msgr2=0x7f6f5c1599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f5c072030 0x7f6f5c1599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 -- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c071bb0 msgr2=0x7f6f5c137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c071bb0 0x7f6f5c137230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 -- 172.21.15.143:0/658229446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f54009a70 con 0x7f6f5c143780 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f5b400640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 0x7f6f5c130ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f540b0040 tx=0x7f6f540196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540add90 con 0x7f6f5c143780 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f540b9450 con 0x7f6f5c143780 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540b9800 con 0x7f6f5c143780 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c0acf60 con 0x7f6f5c143780 2024-01-15T18:05:11.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c134c30 con 0x7f6f5c143780 2024-01-15T18:05:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.393+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c0acb20 con 0x7f6f5c143780 2024-01-15T18:05:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.397+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f6f540b9960 con 0x7f6f5c143780 2024-01-15T18:05:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.397+0000 7f6f52a00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f340668f0 0x7f6f34068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.397+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f5411fad0 con 0x7f6f5c143780 2024-01-15T18:05:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.397+0000 7f6f5aa00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f340668f0 0x7f6f34068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.397+0000 7f6f5aa00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f340668f0 0x7f6f34068db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c007fd0 tx=0x7f6f4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.401+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6f5c0acb20 con 0x7f6f5c143780 2024-01-15T18:05:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.549+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f5c0acb20 con 0x7f6f5c143780 2024-01-15T18:05:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.549+0000 7f6f52a00640 1 -- 172.21.15.143:0/658229446 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f5c0acb20 con 0x7f6f5c143780 2024-01-15T18:05:11.556 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f340668f0 msgr2=0x7f6f34068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f340668f0 0x7f6f34068db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c007fd0 tx=0x7f6f4c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 msgr2=0x7f6f5c130ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 0x7f6f5c130ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f540b0040 tx=0x7f6f540196f0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 shutdown_connections 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6f340668f0 0x7f6f34068db0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f5c143780 0x7f6f5c130ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6f5c072030 0x7f6f5c1599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 --2- 172.21.15.143:0/658229446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f5c071bb0 0x7f6f5c137230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 >> 172.21.15.143:0/658229446 conn(0x7f6f5c06c3f0 msgr2=0x7f6f5c0afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 shutdown_connections 2024-01-15T18:05:11.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.553+0000 7f6f61c00640 1 -- 172.21.15.143:0/658229446 wait complete. 2024-01-15T18:05:11.578 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:11.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 Processor -- start 2024-01-15T18:05:11.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 -- start start 2024-01-15T18:05:11.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f64e01607c0 0x7f64e006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64e00cfb30 0x7f64e006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e006c230 0x7f64e00cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e006c670 con 0x7f64e00cfb30 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e00d1ce0 con 0x7f64e01607c0 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00cffd0 con 0x7f64e006c230 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64e00cfb30 0x7f64e006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64e00cfb30 0x7f64e006bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48570/0 (socket says 172.21.15.143:48570) 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 -- 172.21.15.143:0/1552909848 learned_addr learned my addr 172.21.15.143:0/1552909848 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5c00640 1 --2- 172.21.15.143:0/1552909848 >> v2:172.21.15.204:3300/0 conn(0x7f64e01607c0 0x7f64e006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e6600640 1 --2- 172.21.15.143:0/1552909848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e006c230 0x7f64e00cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 -- 172.21.15.143:0/1552909848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e006c230 msgr2=0x7f64e00cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 --2- 172.21.15.143:0/1552909848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e006c230 0x7f64e00cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 -- 172.21.15.143:0/1552909848 >> v2:172.21.15.204:3300/0 conn(0x7f64e01607c0 msgr2=0x7f64e006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:11.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 --2- 172.21.15.143:0/1552909848 >> v2:172.21.15.204:3300/0 conn(0x7f64e01607c0 0x7f64e006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.713+0000 7f64e5200640 1 -- 172.21.15.143:0/1552909848 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00d02b0 con 0x7f64e00cfb30 2024-01-15T18:05:11.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e5200640 1 --2- 172.21.15.143:0/1552909848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e00cfb30 0x7f64e006bcf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64e0071a00 tx=0x7f64c801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73647d79c76c380 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64dfe00640 1 -- 172.21.15.143:0/1552909848 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80a89b0 con 0x7f64e00cfb30 2024-01-15T18:05:11.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64dfe00640 1 -- 172.21.15.143:0/1552909848 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c80a8b10 con 0x7f64e00cfb30 2024-01-15T18:05:11.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64dfe00640 1 -- 172.21.15.143:0/1552909848 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80a8ea0 con 0x7f64e00cfb30 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 -- 172.21.15.143:0/1552909848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e00cfb30 msgr2=0x7f64e006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 --2- 172.21.15.143:0/1552909848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e00cfb30 0x7f64e006bcf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f64e0071a00 tx=0x7f64c801ecf0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 -- 172.21.15.143:0/1552909848 shutdown_connections 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 --2- 172.21.15.143:0/1552909848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e006c230 0x7f64e00cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 --2- 172.21.15.143:0/1552909848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e00cfb30 0x7f64e006bcf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 --2- 172.21.15.143:0/1552909848 >> v2:172.21.15.204:3300/0 conn(0x7f64e01607c0 0x7f64e006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 -- 172.21.15.143:0/1552909848 >> 172.21.15.143:0/1552909848 conn(0x7f64e00d0e20 msgr2=0x7f64e00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 -- 172.21.15.143:0/1552909848 shutdown_connections 2024-01-15T18:05:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.717+0000 7f64e8600640 1 -- 172.21.15.143:0/1552909848 wait complete. 2024-01-15T18:05:11.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 Processor -- start 2024-01-15T18:05:11.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 -- start start 2024-01-15T18:05:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e006c230 0x7f64e015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e00cfb30 0x7f64e00cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 0x7f64e00cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0158ba0 con 0x7f64e01607c0 2024-01-15T18:05:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01f60c0 con 0x7f64e006c230 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01f63a0 con 0x7f64e00cfb30 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 0x7f64e00cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 0x7f64e00cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48584/0 (socket says 172.21.15.143:48584) 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 -- 172.21.15.143:0/509881719 learned_addr learned my addr 172.21.15.143:0/509881719 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e5c00640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e006c230 0x7f64e015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e5200640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e00cfb30 0x7f64e00cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 -- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e00cfb30 msgr2=0x7f64e00cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e00cfb30 0x7f64e00cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 -- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e006c230 msgr2=0x7f64e015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e006c230 0x7f64e015acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 -- 172.21.15.143:0/509881719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80094d0 con 0x7f64e01607c0 2024-01-15T18:05:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e6600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 0x7f64e00cd360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f64d8007e60 tx=0x7f64d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d8014620 con 0x7f64e01607c0 2024-01-15T18:05:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d800ee90 con 0x7f64e01607c0 2024-01-15T18:05:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e01f6680 con 0x7f64e01607c0 2024-01-15T18:05:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80149e0 con 0x7f64e01607c0 2024-01-15T18:05:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.721+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e0157f90 con 0x7f64e01607c0 2024-01-15T18:05:11.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.725+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e006d4b0 con 0x7f64e01607c0 2024-01-15T18:05:11.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.725+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f64d8014b40 con 0x7f64e01607c0 2024-01-15T18:05:11.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.725+0000 7f64dd600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f64b80668f0 0x7f64b8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.725+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64d8084bf0 con 0x7f64e01607c0 2024-01-15T18:05:11.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.725+0000 7f64e5c00640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f64b80668f0 0x7f64b8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.725+0000 7f64e5c00640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f64b80668f0 0x7f64b8068db0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64d00079c0 tx=0x7f64d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.729+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f64d8051ed0 con 0x7f64e01607c0 2024-01-15T18:05:11.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.877+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64e01f6ba0 con 0x7f64e01607c0 2024-01-15T18:05:11.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.877+0000 7f64dd600640 1 -- 172.21.15.143:0/509881719 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64e01f6ba0 con 0x7f64e01607c0 2024-01-15T18:05:11.884 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:11.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f64b80668f0 msgr2=0x7f64b8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f64b80668f0 0x7f64b8068db0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64d00079c0 tx=0x7f64d0008040 comp rx=0 tx=0).stop 2024-01-15T18:05:11.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 msgr2=0x7f64e00cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 0x7f64e00cd360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f64d8007e60 tx=0x7f64d800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 shutdown_connections 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f64b80668f0 0x7f64b8068db0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64e01607c0 0x7f64e00cd360 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64e00cfb30 0x7f64e00cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 --2- 172.21.15.143:0/509881719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64e006c230 0x7f64e015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 >> 172.21.15.143:0/509881719 conn(0x7f64e00d0e20 msgr2=0x7f64e006fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 shutdown_connections 2024-01-15T18:05:11.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:11.881+0000 7f64e8600640 1 -- 172.21.15.143:0/509881719 wait complete. 2024-01-15T18:05:11.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:12.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 Processor -- start 2024-01-15T18:05:12.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 -- start start 2024-01-15T18:05:12.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49d4071bb0 0x7f49d4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49d415c5a0 0x7f49d415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d40ceb50 con 0x7f49d4143780 2024-01-15T18:05:12.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d4072570 con 0x7f49d415c5a0 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d9c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d415c110 con 0x7f49d4071bb0 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49d415c5a0 0x7f49d415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49d415c5a0 0x7f49d415c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48596/0 (socket says 172.21.15.143:48596) 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 -- 172.21.15.143:0/690542953 learned_addr learned my addr 172.21.15.143:0/690542953 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d2000640 1 --2- 172.21.15.143:0/690542953 >> v2:172.21.15.204:3300/0 conn(0x7f49d4071bb0 0x7f49d4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 -- 172.21.15.143:0/690542953 >> v2:172.21.15.204:3300/0 conn(0x7f49d4071bb0 msgr2=0x7f49d4072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 --2- 172.21.15.143:0/690542953 >> v2:172.21.15.204:3300/0 conn(0x7f49d4071bb0 0x7f49d4072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 -- 172.21.15.143:0/690542953 >> v1:172.21.15.204:6789/0 conn(0x7f49d4143780 legacy=0x7f49d40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d3400640 1 -- 172.21.15.143:0/690542953 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d415b7e0 con 0x7f49d415c5a0 2024-01-15T18:05:12.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.037+0000 7f49d2000640 1 --2- 172.21.15.143:0/690542953 >> v2:172.21.15.204:3300/0 conn(0x7f49d4071bb0 0x7f49d4072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:12.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d3400640 1 --2- 172.21.15.143:0/690542953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415c5a0 0x7f49d415c9e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f49d406beb0 tx=0x7f49cc01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7178a18b078a8fd1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d1600640 1 -- 172.21.15.143:0/690542953 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0a5890 con 0x7f49d415c5a0 2024-01-15T18:05:12.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d1600640 1 -- 172.21.15.143:0/690542953 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc0a59f0 con 0x7f49d415c5a0 2024-01-15T18:05:12.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d1600640 1 -- 172.21.15.143:0/690542953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0a5d80 con 0x7f49d415c5a0 2024-01-15T18:05:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 -- 172.21.15.143:0/690542953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415c5a0 msgr2=0x7f49d415c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 --2- 172.21.15.143:0/690542953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415c5a0 0x7f49d415c9e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f49d406beb0 tx=0x7f49cc01c9d0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 -- 172.21.15.143:0/690542953 shutdown_connections 2024-01-15T18:05:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 --2- 172.21.15.143:0/690542953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d415c5a0 0x7f49d415c9e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 --2- 172.21.15.143:0/690542953 >> v2:172.21.15.204:3300/0 conn(0x7f49d4071bb0 0x7f49d4072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 -- 172.21.15.143:0/690542953 >> 172.21.15.143:0/690542953 conn(0x7f49d406c3f0 msgr2=0x7f49d406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 -- 172.21.15.143:0/690542953 shutdown_connections 2024-01-15T18:05:12.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.041+0000 7f49d9c00640 1 -- 172.21.15.143:0/690542953 wait complete. 2024-01-15T18:05:12.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 Processor -- start 2024-01-15T18:05:12.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 -- start start 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 0x7f49d40b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d4143780 0x7f49d4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49d415c5a0 0x7f49d40ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40b3d10 con 0x7f49d4071bb0 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d413c010 con 0x7f49d4143780 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40b2db0 con 0x7f49d415c5a0 2024-01-15T18:05:12.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 0x7f49d40b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 0x7f49d40b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48612/0 (socket says 172.21.15.143:48612) 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 -- 172.21.15.143:0/2007943320 learned_addr learned my addr 172.21.15.143:0/2007943320 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2000640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d4143780 0x7f49d4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d3400640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49d415c5a0 0x7f49d40ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 -- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49d415c5a0 msgr2=0x7f49d40ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49d415c5a0 0x7f49d40ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 -- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d4143780 msgr2=0x7f49d4158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d4143780 0x7f49d4158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 -- 172.21.15.143:0/2007943320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc009b30 con 0x7f49d4071bb0 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d2a00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 0x7f49d40b1e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c500 tx=0x7f49c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4005220 con 0x7f49d4071bb0 2024-01-15T18:05:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4005380 con 0x7f49d4071bb0 2024-01-15T18:05:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d40b2510 con 0x7f49d4071bb0 2024-01-15T18:05:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4007270 con 0x7f49d4071bb0 2024-01-15T18:05:12.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.045+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d40b2ad0 con 0x7f49d4071bb0 2024-01-15T18:05:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.049+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d415b5e0 con 0x7f49d4071bb0 2024-01-15T18:05:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.049+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f49c40073d0 con 0x7f49d4071bb0 2024-01-15T18:05:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.049+0000 7f49caa00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f49ac066920 0x7f49ac068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.049+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49c4083550 con 0x7f49d4071bb0 2024-01-15T18:05:12.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.049+0000 7f49d2000640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f49ac066920 0x7f49ac068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.049+0000 7f49d2000640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f49ac066920 0x7f49ac068de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49d413f4f0 tx=0x7f49bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.053+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49c40507b0 con 0x7f49d4071bb0 2024-01-15T18:05:12.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.233+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49d40ceb50 con 0x7f49d4071bb0 2024-01-15T18:05:12.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.233+0000 7f49caa00640 1 -- 172.21.15.143:0/2007943320 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f49d40ceb50 con 0x7f49d4071bb0 2024-01-15T18:05:12.240 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:12.240 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:12.240 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:12.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f49ac066920 msgr2=0x7f49ac068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f49ac066920 0x7f49ac068de0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49d413f4f0 tx=0x7f49bc008040 comp rx=0 tx=0).stop 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 msgr2=0x7f49d40b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 0x7f49d40b1e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f49c400c500 tx=0x7f49c400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 shutdown_connections 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f49ac066920 0x7f49ac068de0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49d415c5a0 0x7f49d40ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49d4143780 0x7f49d4158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 --2- 172.21.15.143:0/2007943320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49d4071bb0 0x7f49d40b1e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 >> 172.21.15.143:0/2007943320 conn(0x7f49d406c3f0 msgr2=0x7f49d413ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 shutdown_connections 2024-01-15T18:05:12.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.237+0000 7f49d9c00640 1 -- 172.21.15.143:0/2007943320 wait complete. 2024-01-15T18:05:12.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:12.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.393+0000 7f3e9f600640 1 Processor -- start 2024-01-15T18:05:12.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.393+0000 7f3e9f600640 1 -- start start 2024-01-15T18:05:12.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cda70 0x7f3e980cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9f600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e980cf400 0x7f3e980cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9f600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9815b5b0 con 0x7f3e980cf800 2024-01-15T18:05:12.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9f600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e980ccd30 con 0x7f3e980cf400 2024-01-15T18:05:12.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980cba70 con 0x7f3e980cda70 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9d600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e980cf800 0x7f3e980b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36142/0 (socket says 172.21.15.143:36142) 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e980cf400 0x7f3e980cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9d600640 1 -- 172.21.15.143:0/2912392779 learned_addr learned my addr 172.21.15.143:0/2912392779 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e9cc00640 1 --2- 172.21.15.143:0/2912392779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cda70 0x7f3e980cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97e00640 1 -- 172.21.15.143:0/2912392779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cda70 msgr2=0x7f3e980cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97e00640 1 --2- 172.21.15.143:0/2912392779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cda70 0x7f3e980cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97e00640 1 -- 172.21.15.143:0/2912392779 >> v1:172.21.15.143:6789/0 conn(0x7f3e980cf800 legacy=0x7f3e980b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97e00640 1 -- 172.21.15.143:0/2912392779 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980cbd10 con 0x7f3e980cf400 2024-01-15T18:05:12.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97400640 1 -- 172.21.15.143:0/2912392779 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392980577 0 0) 0x7f3e9815b5b0 con 0x7f3e980cf800 2024-01-15T18:05:12.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.397+0000 7f3e97e00640 1 --2- 172.21.15.143:0/2912392779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf400 0x7f3e980cde50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3e800097c0 tx=0x7f3e8001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82646e9715a18df1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e97400640 1 -- 172.21.15.143:0/2912392779 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800ab650 con 0x7f3e980cf400 2024-01-15T18:05:12.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e97400640 1 -- 172.21.15.143:0/2912392779 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e800ab7b0 con 0x7f3e980cf400 2024-01-15T18:05:12.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e97400640 1 -- 172.21.15.143:0/2912392779 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800abb40 con 0x7f3e980cf400 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 -- 172.21.15.143:0/2912392779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf400 msgr2=0x7f3e980cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 --2- 172.21.15.143:0/2912392779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf400 0x7f3e980cde50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3e800097c0 tx=0x7f3e8001c450 comp rx=0 tx=0).stop 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 -- 172.21.15.143:0/2912392779 shutdown_connections 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 --2- 172.21.15.143:0/2912392779 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf400 0x7f3e980cde50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 --2- 172.21.15.143:0/2912392779 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cda70 0x7f3e980cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 -- 172.21.15.143:0/2912392779 >> 172.21.15.143:0/2912392779 conn(0x7f3e980d0e20 msgr2=0x7f3e980d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 -- 172.21.15.143:0/2912392779 shutdown_connections 2024-01-15T18:05:12.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 -- 172.21.15.143:0/2912392779 wait complete. 2024-01-15T18:05:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 Processor -- start 2024-01-15T18:05:12.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 -- start start 2024-01-15T18:05:12.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.401+0000 7f3e9f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e980cda70 0x7f3e98157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cf400 0x7f3e98164af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 0x7f3e9806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c8ae0 con 0x7f3e980cf800 2024-01-15T18:05:12.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c7ef0 con 0x7f3e980cda70 2024-01-15T18:05:12.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e980c81b0 con 0x7f3e980cf400 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 0x7f3e9806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 0x7f3e9806e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48626/0 (socket says 172.21.15.143:48626) 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 -- 172.21.15.143:0/3438446170 learned_addr learned my addr 172.21.15.143:0/3438446170 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9cc00640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e980cda70 0x7f3e98157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e97e00640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cf400 0x7f3e98164af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 -- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cf400 msgr2=0x7f3e98164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cf400 0x7f3e98164af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 -- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e980cda70 msgr2=0x7f3e98157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e980cda70 0x7f3e98157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 -- 172.21.15.143:0/3438446170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009420 con 0x7f3e980cf800 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9cc00640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e980cda70 0x7f3e98157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9d600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 0x7f3e9806e420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c7a0 tx=0x7f3e9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000a640 con 0x7f3e980cf800 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000a7a0 con 0x7f3e980cf800 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e980c8490 con 0x7f3e980cf800 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000abd0 con 0x7f3e980cf800 2024-01-15T18:05:12.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e981577a0 con 0x7f3e980cf800 2024-01-15T18:05:12.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e980c8c50 con 0x7f3e980cf800 2024-01-15T18:05:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.405+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3e9000ad30 con 0x7f3e980cf800 2024-01-15T18:05:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.409+0000 7f3e94c00640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e70066920 0x7f3e70068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.409+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e90083bb0 con 0x7f3e980cf800 2024-01-15T18:05:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.409+0000 7f3e9cc00640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e70066920 0x7f3e70068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.409+0000 7f3e9cc00640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e70066920 0x7f3e70068de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e88007db0 tx=0x7f3e88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.413+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e90050e10 con 0x7f3e980cf800 2024-01-15T18:05:12.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.561+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e98067500 con 0x7f3e980cf800 2024-01-15T18:05:12.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e94c00640 1 -- 172.21.15.143:0/3438446170 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f3e90018070 con 0x7f3e980cf800 2024-01-15T18:05:12.570 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:12.570 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e70066920 msgr2=0x7f3e70068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e70066920 0x7f3e70068de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e88007db0 tx=0x7f3e88008040 comp rx=0 tx=0).stop 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 msgr2=0x7f3e9806e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 0x7f3e9806e420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000c7a0 tx=0x7f3e9000cc60 comp rx=0 tx=0).stop 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 shutdown_connections 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e70066920 0x7f3e70068de0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e980cf800 0x7f3e9806e420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e980cf400 0x7f3e98164af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 --2- 172.21.15.143:0/3438446170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e980cda70 0x7f3e98157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 >> 172.21.15.143:0/3438446170 conn(0x7f3e980d0e20 msgr2=0x7f3e980b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 shutdown_connections 2024-01-15T18:05:12.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.565+0000 7f3e9f600640 1 -- 172.21.15.143:0/3438446170 wait complete. 2024-01-15T18:05:12.592 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f456b600640 1 Processor -- start 2024-01-15T18:05:12.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f456b600640 1 -- start start 2024-01-15T18:05:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f456b600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45640ae240 0x7f456406a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f456b600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45640ad900 con 0x7f45640ae620 2024-01-15T18:05:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f456b600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456415e650 con 0x7f456415dff0 2024-01-15T18:05:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f456b600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4564160d80 con 0x7f45640ae240 2024-01-15T18:05:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f45640ae620 0x7f456415d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36154/0 (socket says 172.21.15.143:36154) 2024-01-15T18:05:12.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 -- 172.21.15.143:0/2993232549 learned_addr learned my addr 172.21.15.143:0/2993232549 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4568c00640 1 --2- 172.21.15.143:0/2993232549 >> v2:172.21.15.204:3300/0 conn(0x7f45640ae240 0x7f456406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707091019 0 0) 0x7f45640ad900 con 0x7f45640ae620 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4544003530 con 0x7f45640ae620 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 886867760 0 0) 0x7f4544003530 con 0x7f45640ae620 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 >> v2:172.21.15.204:3300/0 conn(0x7f45640ae240 msgr2=0x7f456406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 --2- 172.21.15.143:0/2993232549 >> v2:172.21.15.204:3300/0 conn(0x7f45640ae240 0x7f456406a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 >> v1:172.21.15.204:6789/0 conn(0x7f456415dff0 legacy=0x7f45640ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4564161060 con 0x7f45640ae620 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f454c001680 con 0x7f45640ae620 2024-01-15T18:05:12.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 >> v1:172.21.15.143:6789/0 conn(0x7f45640ae620 legacy=0x7f456415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4544003980 0x7f4544003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4544004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544005190 0x7f454400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4564160d80 con 0x7f45440047d0 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454400c030 con 0x7f4544005190 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454400c1f0 con 0x7f4544003980 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4544004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4568c00640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4544003980 0x7f4544003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4569600640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544005190 0x7f454400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 -- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4544003980 msgr2=0x7f4544003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4544003980 0x7f4544003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 -- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544005190 msgr2=0x7f454400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:12.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4544005190 0x7f454400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4563e00640 1 -- 172.21.15.143:0/2993232549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544003530 con 0x7f45440047d0 2024-01-15T18:05:12.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.725+0000 7f4569600640 1 -- 172.21.15.143:0/2993232549 reap_dead start 2024-01-15T18:05:12.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.729+0000 7f4563e00640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4544004c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45640b5e90 tx=0x7f454c01f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e03b07f309f394 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.729+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c005ae0 con 0x7f45440047d0 2024-01-15T18:05:12.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.729+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c0ab070 con 0x7f45440047d0 2024-01-15T18:05:12.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.729+0000 7f4563400640 1 -- 172.21.15.143:0/2993232549 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c005ea0 con 0x7f45440047d0 2024-01-15T18:05:12.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- 172.21.15.143:0/2993232549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 msgr2=0x7f4544004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4544004c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45640b5e90 tx=0x7f454c01f350 comp rx=0 tx=0).stop 2024-01-15T18:05:12.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- 172.21.15.143:0/2993232549 shutdown_connections 2024-01-15T18:05:12.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 --2- 172.21.15.143:0/2993232549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4544004c50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- 172.21.15.143:0/2993232549 >> 172.21.15.143:0/2993232549 conn(0x7f45640cfd20 msgr2=0x7f45640d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- 172.21.15.143:0/2993232549 shutdown_connections 2024-01-15T18:05:12.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- 172.21.15.143:0/2993232549 wait complete. 2024-01-15T18:05:12.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 Processor -- start 2024-01-15T18:05:12.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- start start 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4564155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45640c3370 0x7f45640c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45640c2db0 0x7f4564071040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45640c4a70 con 0x7f45440047d0 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45640c4d50 con 0x7f45640c3370 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f456b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45640c5030 con 0x7f45640c2db0 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f4568c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4564155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f4568c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4564155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48638/0 (socket says 172.21.15.143:48638) 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f4568c00640 1 -- 172.21.15.143:0/553461842 learned_addr learned my addr 172.21.15.143:0/553461842 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f4563e00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45640c3370 0x7f45640c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.733+0000 7f4569600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45640c2db0 0x7f4564071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4568c00640 1 -- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45640c2db0 msgr2=0x7f4564071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4568c00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45640c2db0 0x7f4564071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4568c00640 1 -- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45640c3370 msgr2=0x7f45640c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4568c00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45640c3370 0x7f45640c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4568c00640 1 -- 172.21.15.143:0/553461842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4564161cc0 con 0x7f45440047d0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4568c00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4564155370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4554011aa0 tx=0x7f4554011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400fb80 con 0x7f45440047d0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400fce0 con 0x7f45440047d0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554014680 con 0x7f45440047d0 2024-01-15T18:05:12.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45640c5310 con 0x7f45440047d0 2024-01-15T18:05:12.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45640c5d70 con 0x7f45440047d0 2024-01-15T18:05:12.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640c54a0 con 0x7f45440047d0 2024-01-15T18:05:12.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f45540147e0 con 0x7f45440047d0 2024-01-15T18:05:12.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4560c00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f453c066920 0x7f453c068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:12.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4554084cc0 con 0x7f45440047d0 2024-01-15T18:05:12.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.737+0000 7f4563e00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f453c066920 0x7f453c068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:12.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.741+0000 7f4563e00640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f453c066920 0x7f453c068de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f454c0af000 tx=0x7f454c0aa000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.741+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f455408b080 con 0x7f45440047d0 2024-01-15T18:05:12.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.921+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45640c4bb0 con 0x7f45440047d0 2024-01-15T18:05:12.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.921+0000 7f4560c00640 1 -- 172.21.15.143:0/553461842 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f45640c4bb0 con 0x7f45440047d0 2024-01-15T18:05:12.929 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:12.929 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:12.929 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f453c066920 msgr2=0x7f453c068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f453c066920 0x7f453c068de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f454c0af000 tx=0x7f454c0aa000 comp rx=0 tx=0).stop 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 msgr2=0x7f4564155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4564155370 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4554011aa0 tx=0x7f4554011f60 comp rx=0 tx=0).stop 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 shutdown_connections 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f453c066920 0x7f453c068de0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45640c2db0 0x7f4564071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45640c3370 0x7f45640c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 --2- 172.21.15.143:0/553461842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45440047d0 0x7f4564155370 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 >> 172.21.15.143:0/553461842 conn(0x7f45640cfd20 msgr2=0x7f45641559a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 shutdown_connections 2024-01-15T18:05:12.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:12.925+0000 7f456b600640 1 -- 172.21.15.143:0/553461842 wait complete. 2024-01-15T18:05:13.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:04:58.646333+0000', 'modified': '2024-01-15T18:05:01.878628+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6835/1204727493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 1204727493}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 1204727493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:05:13.354 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:05:13.354 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-15T18:05:13.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.485+0000 7fb69a200640 1 Processor -- start 2024-01-15T18:05:13.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.485+0000 7fb69a200640 1 -- start start 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb69a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb69415b180 0x7fb6940d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb69a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6940ce4a0 0x7fb6940cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb69a200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6940ce200 con 0x7fb6940cddb0 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb69a200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6940c4ea0 con 0x7fb69415b180 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb69a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940c4b70 con 0x7fb6940ce4a0 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb69415b180 0x7fb6940d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb69415b180 0x7fb6940d19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48644/0 (socket says 172.21.15.143:48644) 2024-01-15T18:05:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 -- 172.21.15.143:0/592701188 learned_addr learned my addr 172.21.15.143:0/592701188 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb692a00640 1 --2- 172.21.15.143:0/592701188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6940ce4a0 0x7fb6940cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 -- 172.21.15.143:0/592701188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6940ce4a0 msgr2=0x7fb6940cd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 --2- 172.21.15.143:0/592701188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6940ce4a0 0x7fb6940cd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 -- 172.21.15.143:0/592701188 >> v1:172.21.15.204:6789/0 conn(0x7fb6940cddb0 legacy=0x7fb6940cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.489+0000 7fb693400640 1 -- 172.21.15.143:0/592701188 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940cfdc0 con 0x7fb69415b180 2024-01-15T18:05:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb693400640 1 --2- 172.21.15.143:0/592701188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb69415b180 0x7fb6940d19a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb684009870 tx=0x7fb68401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0bff0a9cfabbae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb692000640 1 -- 172.21.15.143:0/592701188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840ab620 con 0x7fb69415b180 2024-01-15T18:05:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb692000640 1 -- 172.21.15.143:0/592701188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840ab780 con 0x7fb69415b180 2024-01-15T18:05:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb692000640 1 -- 172.21.15.143:0/592701188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840abb10 con 0x7fb69415b180 2024-01-15T18:05:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 -- 172.21.15.143:0/592701188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb69415b180 msgr2=0x7fb6940d19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 --2- 172.21.15.143:0/592701188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb69415b180 0x7fb6940d19a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb684009870 tx=0x7fb68401c510 comp rx=0 tx=0).stop 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 -- 172.21.15.143:0/592701188 shutdown_connections 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 --2- 172.21.15.143:0/592701188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6940ce4a0 0x7fb6940cd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 --2- 172.21.15.143:0/592701188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb69415b180 0x7fb6940d19a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 -- 172.21.15.143:0/592701188 >> 172.21.15.143:0/592701188 conn(0x7fb6940d0e80 msgr2=0x7fb6940d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 -- 172.21.15.143:0/592701188 shutdown_connections 2024-01-15T18:05:13.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 -- 172.21.15.143:0/592701188 wait complete. 2024-01-15T18:05:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 Processor -- start 2024-01-15T18:05:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.493+0000 7fb69a200640 1 -- start start 2024-01-15T18:05:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 0x7fb6940c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6940ce4a0 0x7fb694150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69415b180 0x7fb6940ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940c8a60 con 0x7fb6940cddb0 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940cadb0 con 0x7fb6940ce4a0 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb694151a00 con 0x7fb69415b180 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 0x7fb6940c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 0x7fb6940c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48658/0 (socket says 172.21.15.143:48658) 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 -- 172.21.15.143:0/2794412612 learned_addr learned my addr 172.21.15.143:0/2794412612 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb692a00640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6940ce4a0 0x7fb694150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693e00640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69415b180 0x7fb6940ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 -- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69415b180 msgr2=0x7fb6940ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69415b180 0x7fb6940ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 -- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6940ce4a0 msgr2=0x7fb694150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6940ce4a0 0x7fb694150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 -- 172.21.15.143:0/2794412612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6840094d0 con 0x7fb6940cddb0 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb693400640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 0x7fb6940c9a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb68401abc0 tx=0x7fb68401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb68b400640 1 -- 172.21.15.143:0/2794412612 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840abe40 con 0x7fb6940cddb0 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb68b400640 1 -- 172.21.15.143:0/2794412612 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840be3f0 con 0x7fb6940cddb0 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6940c8c70 con 0x7fb6940cddb0 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb68b400640 1 -- 172.21.15.143:0/2794412612 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840be7a0 con 0x7fb6940cddb0 2024-01-15T18:05:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.497+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6940c9230 con 0x7fb6940cddb0 2024-01-15T18:05:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb68b400640 1 -- 172.21.15.143:0/2794412612 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb6840b43f0 con 0x7fb6940cddb0 2024-01-15T18:05:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb68b400640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb66c0668f0 0x7fb66c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb68b400640 1 -- 172.21.15.143:0/2794412612 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb68411df60 con 0x7fb6940cddb0 2024-01-15T18:05:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb692a00640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb66c0668f0 0x7fb66c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 Processor -- start 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 -- start start 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb692a00640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb66c0668f0 0x7fb66c068db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb694151c60 tx=0x7fb67c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a3832c90 0x55a1a3833070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a37e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a3653130 con 0x55a1a37e9320 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a36409a0 con 0x55a1a3832c90 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a363ded0 con 0x55a1a38335b0 2024-01-15T18:05:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37e9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48670/0 (socket says 172.21.15.143:48670) 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 -- 172.21.15.143:0/1706305455 learned_addr learned my addr 172.21.15.143:0/1706305455 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693400640 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a3832c90 0x55a1a3833070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb692a00640 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a37e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 -- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 msgr2=0x55a1a37e8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a37e8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 -- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a3832c90 msgr2=0x55a1a3833070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a3832c90 0x55a1a3833070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 -- 172.21.15.143:0/1706305455 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a3331890 con 0x55a1a37e9320 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb692a00640 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a37e8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.501+0000 7fb693e00640 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37e9700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001f20 tx=0x7fb68c0016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8de929fb560bb03 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb683e00640 1 -- 172.21.15.143:0/1706305455 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c001a70 con 0x55a1a37e9320 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb683e00640 1 -- 172.21.15.143:0/1706305455 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c00fe50 con 0x55a1a37e9320 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb683e00640 1 -- 172.21.15.143:0/1706305455 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c003b80 con 0x55a1a37e9320 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1706305455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 msgr2=0x55a1a37e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37e9700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001f20 tx=0x7fb68c0016b0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1706305455 shutdown_connections 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37e9700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a37e8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1706305455 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a3832c90 0x55a1a3833070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1706305455 >> 172.21.15.143:0/1706305455 conn(0x55a1a37e52d0 msgr2=0x55a1a37e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1706305455 shutdown_connections 2024-01-15T18:05:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1706305455 wait complete. 2024-01-15T18:05:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- start start 2024-01-15T18:05:13.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 Processor -- start 2024-01-15T18:05:13.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a37ee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a38335b0 0x55a1a3801910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a36397c0 con 0x55a1a37e9320 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a35f1bf0 con 0x55a1a38335b0 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a35f7a90 con 0x55a1a3832c90 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb69d5ac000 10 client.0 _ll_register_callbacks cb 0x55a1a37d43c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb693400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb693400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37edd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48684/0 (socket says 172.21.15.143:48684) 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb693400640 1 -- 172.21.15.143:0/1239542864 learned_addr learned my addr 172.21.15.143:0/1239542864 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb692a00640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a37ee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.505+0000 7fb693e00640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a38335b0 0x55a1a3801910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb693400640 1 -- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 msgr2=0x55a1a37ee280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb693400640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a37ee280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb693400640 1 -- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a38335b0 msgr2=0x55a1a3801910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb693400640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a38335b0 0x55a1a3801910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb693400640 1 -- 172.21.15.143:0/1239542864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c001bd0 con 0x55a1a37e9320 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb693400640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37edd40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6840f6c20 tx=0x7fb6840f5170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df8681847140792 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 1 -- 172.21.15.143:0/1239542864 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840f7150 con 0x55a1a37e9320 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 1 -- 172.21.15.143:0/1239542864 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6840f6cc0 con 0x55a1a37e9320 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1a35fb470 con 0x55a1a37e9320 2024-01-15T18:05:13.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 1 -- 172.21.15.143:0/1239542864 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6840f6e80 con 0x55a1a37e9320 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 10 client.4632 fetch_fsmap learned FSMap version 8 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a1a35fa190 con 0x55a1a37e9320 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 1 -- 172.21.15.143:0/1239542864 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1a35fb470 con 0x55a1a37e9320 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 1 -- 172.21.15.143:0/1239542864 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb684102ba0 con 0x55a1a37e9320 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 10 client.4632 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 20 client.4632 populate_metadata read hostname 'smithi143' 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3802320 0x55a1a3844f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 4 client.4632 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 --> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1a35feb50 con 0x55a1a3802320 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb692a00640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3802320 0x55a1a3844f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb692a00640 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3802320 0x55a1a3844f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7799da664d3d580e server_cookie=88e1a4b4888451a0 in_seq=0 out_seq=0 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 10 client.4632 ms_handle_connect on v2:172.21.15.143:6834/1204727493 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 1 -- 172.21.15.143:0/1239542864 <== mds.0 v2:172.21.15.143:6834/1204727493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a1a35feb50 con 0x55a1a3802320 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 10 client.4632 handle_command_reply: tid=0 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 client.4632 shutdown 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 msgr2=0x55a1a37edd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37edd40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6840f6c20 tx=0x7fb6840f5170 comp rx=0 tx=0).stop 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 shutdown_connections 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3802320 0x55a1a3844f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a38335b0 0x55a1a3801910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a37ee280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb680c00640 0 client.4632 ms_handle_reset on v2:172.21.15.143:6834/1204727493 2024-01-15T18:05:13.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1239542864 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a37e9320 0x55a1a37edd40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.509+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 >> 172.21.15.143:0/1239542864 conn(0x55a1a37e52d0 msgr2=0x55a1a37e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.513+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 shutdown_connections 2024-01-15T18:05:13.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.513+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1239542864 wait complete. 2024-01-15T18:05:13.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.513+0000 7fb69d5ac000 20 client.4632 trim_cache size 0 max 16384 2024-01-15T18:05:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.525+0000 7fb69d5ac000 1 Processor -- start 2024-01-15T18:05:13.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.525+0000 7fb69d5ac000 1 -- start start 2024-01-15T18:05:13.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a3831c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a3830b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 0x55a1a3831070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a32f2940 con 0x55a1a38335b0 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a37f0890 con 0x55a1a37e9320 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a3637790 con 0x55a1a3832c90 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 0x55a1a3831070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 0x55a1a3831070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48686/0 (socket says 172.21.15.143:48686) 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 -- 172.21.15.143:0/3470470183 learned_addr learned my addr 172.21.15.143:0/3470470183 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb692a00640 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a3830b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693400640 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a3831c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 -- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 msgr2=0x55a1a3830b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a3830b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 -- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 msgr2=0x55a1a3831c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a3831c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 -- 172.21.15.143:0/3470470183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a35e0490 con 0x55a1a38335b0 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb693e00640 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 0x55a1a3831070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb68c014000 tx=0x7fb68c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518828c63e3ab71c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb688c00640 1 -- 172.21.15.143:0/3470470183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c003980 con 0x55a1a38335b0 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb688c00640 1 -- 172.21.15.143:0/3470470183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c003ae0 con 0x55a1a38335b0 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb688c00640 1 -- 172.21.15.143:0/3470470183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c003e70 con 0x55a1a38335b0 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- 172.21.15.143:0/3470470183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 msgr2=0x55a1a3831070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 0x55a1a3831070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb68c014000 tx=0x7fb68c00a000 comp rx=0 tx=0).stop 2024-01-15T18:05:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- 172.21.15.143:0/3470470183 shutdown_connections 2024-01-15T18:05:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a38335b0 0x55a1a3831070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a3832c90 0x55a1a3830b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/3470470183 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a3831c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- 172.21.15.143:0/3470470183 >> 172.21.15.143:0/3470470183 conn(0x55a1a37e52d0 msgr2=0x55a1a3840280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- 172.21.15.143:0/3470470183 shutdown_connections 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- 172.21.15.143:0/3470470183 wait complete. 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 -- start start 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.529+0000 7fb69d5ac000 1 Processor -- start 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a37eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 0x55a1a3845fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a38464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a3632d40 con 0x55a1a3832c90 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a3612e10 con 0x55a1a37e9320 2024-01-15T18:05:13.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a36036c0 con 0x55a1a38335b0 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 10 client.0 _ll_register_callbacks cb 0x55a1a3687c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 0x55a1a3845fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 0x55a1a3845fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48698/0 (socket says 172.21.15.143:48698) 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 -- 172.21.15.143:0/1035757954 learned_addr learned my addr 172.21.15.143:0/1035757954 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb693400640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a37eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb693e00640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a38464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 -- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 msgr2=0x55a1a38464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a38464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 -- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 msgr2=0x55a1a37eaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a37eaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 -- 172.21.15.143:0/1035757954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1a3331890 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb692a00640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 0x55a1a3845fa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb6941f95e0 tx=0x7fb67c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d61b6addef6ee6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 1 -- 172.21.15.143:0/1035757954 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c029070 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 1 -- 172.21.15.143:0/1035757954 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb67c024ac0 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1a3638110 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 1 -- 172.21.15.143:0/1035757954 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb67c024c80 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 10 client.4638 fetch_fsmap learned FSMap version 8 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55a1a3331cc0 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 1 -- 172.21.15.143:0/1035757954 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1a3638110 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 1 -- 172.21.15.143:0/1035757954 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb67c004020 con 0x55a1a3832c90 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2024-01-15T18:05:13.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 10 client.4638 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 20 client.4638 populate_metadata read hostname 'smithi143' 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3893010 0x55a1a38933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 4 client.4638 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 --> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a1a36679a0 con 0x55a1a3893010 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb693400640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3893010 0x55a1a38933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb693400640 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3893010 0x55a1a38933f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa71427e5a023ef server_cookie=ff7a378963c34270 in_seq=0 out_seq=0 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 10 client.4638 ms_handle_connect on v2:172.21.15.143:6834/1204727493 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 1 -- 172.21.15.143:0/1035757954 <== mds.0 v2:172.21.15.143:6834/1204727493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+317 (crc 0 0 0) 0x55a1a36679a0 con 0x55a1a3893010 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb680c00640 10 client.4638 handle_command_reply: tid=0 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.533+0000 7fb69d5ac000 1 client.4638 shutdown 2024-01-15T18:05:13.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 msgr2=0x55a1a3845fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 0x55a1a3845fa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb6941f95e0 tx=0x7fb67c004740 comp rx=0 tx=0).stop 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 shutdown_connections 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:6834/1204727493,v1:172.21.15.143:6835/1204727493] conn(0x55a1a3893010 0x55a1a38933f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55a1a38335b0 0x55a1a38464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a1a3832c90 0x55a1a3845fa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb680c00640 0 client.4638 ms_handle_reset on v2:172.21.15.143:6834/1204727493 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 --2- 172.21.15.143:0/1035757954 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a1a37e9320 0x55a1a37eaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 >> 172.21.15.143:0/1035757954 conn(0x55a1a37e52d0 msgr2=0x55a1a37e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 shutdown_connections 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 1 -- 172.21.15.143:0/1035757954 wait complete. 2024-01-15T18:05:13.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69d5ac000 20 client.4638 trim_cache size 0 max 16384 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4281, 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 14.04373043, 2024-01-15T18:05:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 8, 2024-01-15T18:05:13.550 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 20, 2024-01-15T18:05:13.550 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 20, 2024-01-15T18:05:13.550 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 21.316647044 2024-01-15T18:05:13.550 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb66c0668f0 msgr2=0x7fb66c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb66c0668f0 0x7fb66c068db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb694151c60 tx=0x7fb67c009190 comp rx=0 tx=0).stop 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 msgr2=0x7fb6940c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 0x7fb6940c9a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb68401abc0 tx=0x7fb68401ad50 comp rx=0 tx=0).stop 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 shutdown_connections 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb66c0668f0 0x7fb66c068db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb69415b180 0x7fb6940ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6940ce4a0 0x7fb694150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 --2- 172.21.15.143:0/2794412612 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6940cddb0 0x7fb6940c9a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 >> 172.21.15.143:0/2794412612 conn(0x7fb6940d0e80 msgr2=0x7fb694166300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 shutdown_connections 2024-01-15T18:05:13.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.537+0000 7fb69a200640 1 -- 172.21.15.143:0/2794412612 wait complete. 2024-01-15T18:05:13.563 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-01-15T18:05:13.592 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-01-15T18:05:13.592 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-15T18:05:13.593 INFO:tasks.ceph_fuse:client.0 config is {} 2024-01-15T18:05:13.593 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T18:05:13.593 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:13.593 INFO:tasks.ceph_fuse:client.2 config is {} 2024-01-15T18:05:13.593 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T18:05:13.593 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:13.593 INFO:tasks.ceph_fuse:client.1 config is {} 2024-01-15T18:05:13.593 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T18:05:13.593 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:13.594 INFO:tasks.ceph_fuse:client.3 config is {} 2024-01-15T18:05:13.594 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T18:05:13.594 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:13.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:13.594 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:05:13.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:13.606 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-01-15T18:05:13.671 INFO:teuthology.orchestra.run.smithi204.stderr:Cannot find device "ceph-brx" 2024-01-15T18:05:13.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:05:13.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:13.674 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:05:13.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:13.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-01-15T18:05:13.746 INFO:teuthology.orchestra.run.smithi143.stderr:Cannot find device "ceph-brx" 2024-01-15T18:05:13.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:05:13.749 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-01-15T18:05:13.749 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:05:13.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 Processor -- start 2024-01-15T18:05:13.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 -- start start 2024-01-15T18:05:13.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcb840cf400 0x7fcb840cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840ce390 0x7fcb840b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8415b5b0 con 0x7fcb840cda70 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb840cba70 con 0x7fcb840cf400 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb8b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840cbd10 con 0x7fcb840ce390 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb88a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcb840cda70 0x7fcb840cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36168/0 (socket says 172.21.15.143:36168) 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb88a00640 1 -- 172.21.15.143:0/2107773009 learned_addr learned my addr 172.21.15.143:0/2107773009 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb83e00640 1 --2- 172.21.15.143:0/2107773009 >> v2:172.21.15.143:3300/0 conn(0x7fcb840cf400 0x7fcb840cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.921+0000 7fcb89400640 1 --2- 172.21.15.143:0/2107773009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840ce390 0x7fcb840b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83400640 1 -- 172.21.15.143:0/2107773009 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704461360 0 0) 0x7fcb8415b5b0 con 0x7fcb840cda70 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83400640 1 -- 172.21.15.143:0/2107773009 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb64003530 con 0x7fcb840cda70 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83e00640 1 -- 172.21.15.143:0/2107773009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840ce390 msgr2=0x7fcb840b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83e00640 1 --2- 172.21.15.143:0/2107773009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840ce390 0x7fcb840b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83e00640 1 -- 172.21.15.143:0/2107773009 >> v1:172.21.15.143:6789/0 conn(0x7fcb840cda70 legacy=0x7fcb840cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83e00640 1 -- 172.21.15.143:0/2107773009 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840cbff0 con 0x7fcb840cf400 2024-01-15T18:05:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83e00640 1 --2- 172.21.15.143:0/2107773009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840cf400 0x7fcb840cde50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0097c0 tx=0x7fcb6c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffac365a079bde0e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83400640 1 -- 172.21.15.143:0/2107773009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0ab670 con 0x7fcb840cf400 2024-01-15T18:05:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83400640 1 -- 172.21.15.143:0/2107773009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c0ab7d0 con 0x7fcb840cf400 2024-01-15T18:05:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb83400640 1 -- 172.21.15.143:0/2107773009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0abb60 con 0x7fcb840cf400 2024-01-15T18:05:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb8b400640 1 -- 172.21.15.143:0/2107773009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840cf400 msgr2=0x7fcb840cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.925+0000 7fcb8b400640 1 --2- 172.21.15.143:0/2107773009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840cf400 0x7fcb840cde50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0097c0 tx=0x7fcb6c01c450 comp rx=0 tx=0).stop 2024-01-15T18:05:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- 172.21.15.143:0/2107773009 shutdown_connections 2024-01-15T18:05:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 --2- 172.21.15.143:0/2107773009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840ce390 0x7fcb840b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 --2- 172.21.15.143:0/2107773009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840cf400 0x7fcb840cde50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- 172.21.15.143:0/2107773009 >> 172.21.15.143:0/2107773009 conn(0x7fcb840d0e20 msgr2=0x7fcb840d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- 172.21.15.143:0/2107773009 shutdown_connections 2024-01-15T18:05:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- 172.21.15.143:0/2107773009 wait complete. 2024-01-15T18:05:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 Processor -- start 2024-01-15T18:05:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- start start 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb840cda70 0x7fcb840c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 0x7fcb840c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840cf400 0x7fcb840c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841631b0 con 0x7fcb840ce390 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840b5840 con 0x7fcb840cda70 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb8b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840b5b20 con 0x7fcb840cf400 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 0x7fcb840c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 0x7fcb840c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48706/0 (socket says 172.21.15.143:48706) 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 -- 172.21.15.143:0/575055595 learned_addr learned my addr 172.21.15.143:0/575055595 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:13.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb89400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840cf400 0x7fcb840c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb88a00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb840cda70 0x7fcb840c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 -- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840cf400 msgr2=0x7fcb840c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840cf400 0x7fcb840c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 -- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb840cda70 msgr2=0x7fcb840c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb840cda70 0x7fcb840c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.929+0000 7fcb83e00640 1 -- 172.21.15.143:0/575055595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c009420 con 0x7fcb840ce390 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb83e00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 0x7fcb840c8430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0ad040 tx=0x7fcb6c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0bf610 con 0x7fcb840ce390 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6c0bf7b0 con 0x7fcb840ce390 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb84163470 con 0x7fcb840ce390 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c0bfb60 con 0x7fcb840ce390 2024-01-15T18:05:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb840c95d0 con 0x7fcb840ce390 2024-01-15T18:05:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb841636b0 con 0x7fcb840ce390 2024-01-15T18:05:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fcb6c0b5400 con 0x7fcb840ce390 2024-01-15T18:05:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb80c00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb5c0668f0 0x7fcb5c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb6c11f0a0 con 0x7fcb840ce390 2024-01-15T18:05:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.933+0000 7fcb88a00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb5c0668f0 0x7fcb5c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.937+0000 7fcb88a00640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb5c0668f0 0x7fcb5c068db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb740011f0 tx=0x7fcb74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:13.937+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcb841636b0 con 0x7fcb840ce390 2024-01-15T18:05:14.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.121+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb841636b0 con 0x7fcb840ce390 2024-01-15T18:05:14.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.121+0000 7fcb80c00640 1 -- 172.21.15.143:0/575055595 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb841636b0 con 0x7fcb840ce390 2024-01-15T18:05:14.127 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:05:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb5c0668f0 msgr2=0x7fcb5c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb5c0668f0 0x7fcb5c068db0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcb740011f0 tx=0x7fcb74009040 comp rx=0 tx=0).stop 2024-01-15T18:05:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 msgr2=0x7fcb840c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 0x7fcb840c8430 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c0ad040 tx=0x7fcb6c01ac90 comp rx=0 tx=0).stop 2024-01-15T18:05:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 shutdown_connections 2024-01-15T18:05:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb5c0668f0 0x7fcb5c068db0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb840cf400 0x7fcb840c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb840ce390 0x7fcb840c8430 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 --2- 172.21.15.143:0/575055595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb840cda70 0x7fcb840c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 >> 172.21.15.143:0/575055595 conn(0x7fcb840d0e20 msgr2=0x7fcb8406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:14.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 shutdown_connections 2024-01-15T18:05:14.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:14.125+0000 7fcb8b400640 1 -- 172.21.15.143:0/575055595 wait complete. 2024-01-15T18:05:14.149 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:05:14.149 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:05:14.150 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:05:14.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.150 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:05:14.160 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:14.161 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42429sec preferred_lft 42429sec 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:05:14.162 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:14.163 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:05:14.163 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:14.163 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-15T18:05:14.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.163 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:14.164 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:14.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add name ceph-brx type bridge 2024-01-15T18:05:14.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr flush dev ceph-brx 2024-01-15T18:05:14.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set ceph-brx up 2024-01-15T18:05:14.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-15T18:05:14.164 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:14.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.271 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:05:14.290 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:05:14.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.293 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:05:14.369 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:05:14.369 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:14.370 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:05:14.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.371 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:14.371 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:14.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:05:14.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:05:14.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:05:14.371 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:14.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.461 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:05:14.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.471 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-01-15T18:05:14.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.522 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:14.522 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:14.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:14.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-15T18:05:14.522 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:14.606 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-15T18:05:14.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.606 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-15T18:05:14.607 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:14.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:14.765 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:14.765 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:14.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:05:14.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.0 master ceph-brx 2024-01-15T18:05:14.765 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:14.804 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:05:14.805 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:14.805 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:14.861 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:05:14.861 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:14.862 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:14.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:05:14.977 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:15.030 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:05:15.030 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:05:15.030 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:15.031 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:05:15.032 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:15.033 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:15.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:15.034 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:05:15.085 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:05:15.085 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:05:15.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:05:15.152 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:05:15.207 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:05:15.208 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:05:15.209 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:05:15.210 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:05:15.210 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:05:15.210 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:05:15.210 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:15.210 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:15.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:15.211 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:05:15.234 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:05:15.228+0000 7f02afde8580 -1 init, newargv = 0x56403383a1e0 newargc=15 2024-01-15T18:05:15.234 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[34663]: starting ceph client 2024-01-15T18:05:15.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[34663]: starting fuse 2024-01-15T18:05:15.243 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:05:15.243 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:05:15.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:15.244 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:05:15.245 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:05:15.246 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:05:15.246 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:15.246 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:05:15.246 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:05:15.367 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.34663.asok 2024-01-15T18:05:15.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:05:15.378 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34663.asok 2024-01-15T18:05:15.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:15.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34663.asok status 2024-01-15T18:05:15.547 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:05:15.547 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:05:15.547 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "34663", 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:15.548 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4653, 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 4653 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:05:15.549 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 922998743 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 922998743 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:15.550 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.4653 192.168.0.1:0/922998743", 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/922998743", 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 8, 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 20, 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:05:15.551 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:05:15.562 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:05:15.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 Processor -- start 2024-01-15T18:05:15.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 -- start start 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c15d1f0 0x7f681c06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f681c071470 0x7f681c0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f681c0b3370 con 0x7f681c06bb70 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681c15d5d0 con 0x7f681c071470 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f6822c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c06a800 con 0x7f681c15d1f0 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.692+0000 7f681b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f681c06bb70 0x7f681c070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36174/0 (socket says 172.21.15.143:36174) 2024-01-15T18:05:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681b400640 1 -- 172.21.15.143:0/755581408 learned_addr learned my addr 172.21.15.143:0/755581408 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681be00640 1 --2- 172.21.15.143:0/755581408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c15d1f0 0x7f681c06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f6820c00640 1 --2- 172.21.15.143:0/755581408 >> v2:172.21.15.204:3300/0 conn(0x7f681c071470 0x7f681c0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681aa00640 1 -- 172.21.15.143:0/755581408 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499261419 0 0) 0x7f681c0b3370 con 0x7f681c06bb70 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681aa00640 1 -- 172.21.15.143:0/755581408 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67fc003530 con 0x7f681c06bb70 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681be00640 1 -- 172.21.15.143:0/755581408 >> v2:172.21.15.204:3300/0 conn(0x7f681c071470 msgr2=0x7f681c0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681be00640 1 --2- 172.21.15.143:0/755581408 >> v2:172.21.15.204:3300/0 conn(0x7f681c071470 0x7f681c0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681be00640 1 -- 172.21.15.143:0/755581408 >> v1:172.21.15.143:6789/0 conn(0x7f681c06bb70 legacy=0x7f681c070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681aa00640 1 -- 172.21.15.143:0/755581408 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 593911001 0 0) 0x7f67fc003530 con 0x7f681c06bb70 2024-01-15T18:05:15.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681be00640 1 -- 172.21.15.143:0/755581408 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c06aae0 con 0x7f681c15d1f0 2024-01-15T18:05:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681be00640 1 --2- 172.21.15.143:0/755581408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c15d1f0 0x7f681c06b630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097c0 tx=0x7f680c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b1ed2cc9cf0769 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681aa00640 1 -- 172.21.15.143:0/755581408 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804001680 con 0x7f681c15d1f0 2024-01-15T18:05:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681aa00640 1 -- 172.21.15.143:0/755581408 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c0a7050 con 0x7f681c15d1f0 2024-01-15T18:05:15.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.696+0000 7f681aa00640 1 -- 172.21.15.143:0/755581408 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0ab620 con 0x7f681c15d1f0 2024-01-15T18:05:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- 172.21.15.143:0/755581408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c15d1f0 msgr2=0x7f681c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 --2- 172.21.15.143:0/755581408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c15d1f0 0x7f681c06b630 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097c0 tx=0x7f680c01c370 comp rx=0 tx=0).stop 2024-01-15T18:05:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- 172.21.15.143:0/755581408 shutdown_connections 2024-01-15T18:05:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 --2- 172.21.15.143:0/755581408 >> v2:172.21.15.204:3300/0 conn(0x7f681c071470 0x7f681c0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 --2- 172.21.15.143:0/755581408 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c15d1f0 0x7f681c06b630 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- 172.21.15.143:0/755581408 >> 172.21.15.143:0/755581408 conn(0x7f681c0b1610 msgr2=0x7f681c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- 172.21.15.143:0/755581408 shutdown_connections 2024-01-15T18:05:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- 172.21.15.143:0/755581408 wait complete. 2024-01-15T18:05:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 Processor -- start 2024-01-15T18:05:15.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- start start 2024-01-15T18:05:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 0x7f681c14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c071470 0x7f681c14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f681c15d1f0 0x7f681c14ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c15ffd0 con 0x7f681c06bb70 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681c14e5e0 con 0x7f681c15d1f0 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f6822c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681c160220 con 0x7f681c071470 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f681be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 0x7f681c14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f681be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 0x7f681c14cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48718/0 (socket says 172.21.15.143:48718) 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.700+0000 7f681be00640 1 -- 172.21.15.143:0/2260193735 learned_addr learned my addr 172.21.15.143:0/2260193735 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6820c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f681c15d1f0 0x7f681c14ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681b400640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c071470 0x7f681c14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681be00640 1 -- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c071470 msgr2=0x7f681c14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681be00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c071470 0x7f681c14b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681be00640 1 -- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f681c15d1f0 msgr2=0x7f681c14ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681be00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f681c15d1f0 0x7f681c14ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681be00640 1 -- 172.21.15.143:0/2260193735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009420 con 0x7f681c06bb70 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681b400640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c071470 0x7f681c14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f681be00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 0x7f681c14cf50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097c0 tx=0x7f680c01abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0b7450 con 0x7f681c06bb70 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c0b75d0 con 0x7f681c06bb70 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0b7980 con 0x7f681c06bb70 2024-01-15T18:05:15.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f681c160500 con 0x7f681c06bb70 2024-01-15T18:05:15.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681c15fcf0 con 0x7f681c06bb70 2024-01-15T18:05:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681c074b00 con 0x7f681c06bb70 2024-01-15T18:05:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f680c0b7ae0 con 0x7f681c06bb70 2024-01-15T18:05:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6813e00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67f40668f0 0x7f67f4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.704+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f680c11ec60 con 0x7f681c06bb70 2024-01-15T18:05:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.708+0000 7f681b400640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67f40668f0 0x7f67f4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.708+0000 7f681b400640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67f40668f0 0x7f67f4068db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f681c15c2e0 tx=0x7f6804009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:15.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.708+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f680c0ebf30 con 0x7f681c06bb70 2024-01-15T18:05:15.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.892+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f681c0aed50 con 0x7f681c06bb70 2024-01-15T18:05:15.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.892+0000 7f6813e00640 1 -- 172.21.15.143:0/2260193735 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f680c0b0070 con 0x7f681c06bb70 2024-01-15T18:05:15.901 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:05:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67f40668f0 msgr2=0x7f67f4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67f40668f0 0x7f67f4068db0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f681c15c2e0 tx=0x7f6804009040 comp rx=0 tx=0).stop 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 msgr2=0x7f681c14cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 0x7f681c14cf50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f680c0097c0 tx=0x7f680c01abb0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 shutdown_connections 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67f40668f0 0x7f67f4068db0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f681c15d1f0 0x7f681c14ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f681c071470 0x7f681c14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 --2- 172.21.15.143:0/2260193735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f681c06bb70 0x7f681c14cf50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 >> 172.21.15.143:0/2260193735 conn(0x7f681c0b1610 msgr2=0x7f681c0b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 shutdown_connections 2024-01-15T18:05:15.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:15.896+0000 7f6822c00640 1 -- 172.21.15.143:0/2260193735 wait complete. 2024-01-15T18:05:15.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:05:15.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:05:15.923 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-15T18:05:15.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:15.923 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:15.934 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42427sec preferred_lft 42427sec 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:05:15.935 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:05:15.936 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:15.937 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-01-15T18:05:15.937 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:15.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:15.937 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:05:15.994 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:05:15.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:15.997 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:05:16.051 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:16.052 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:16.053 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:05:16.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.053 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:16.053 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:16.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:05:16.053 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:05:16.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:05:16.054 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:16.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.142 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:05:16.152 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:05:16.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.152 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list-id 2024-01-15T18:05:16.201 INFO:teuthology.orchestra.run.smithi143.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-15T18:05:16.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.202 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:16.202 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:16.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:05:16.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-15T18:05:16.203 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:16.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.286 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:05:16.353 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 2a:04:e9:79:66:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T18:05:16.354 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T18:05:16.354 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.354 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::2804:e9ff:fe79:66d1/64 scope link tentative 2024-01-15T18:05:16.354 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 2a:04:e9:79:66:d1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T18:05:16.417 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T18:05:16.418 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.418 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::2804:e9ff:fe79:66d1/64 scope link tentative 2024-01-15T18:05:16.418 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:16.419 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-15T18:05:16.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.419 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-15T18:05:16.420 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:16.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.557 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:05:16.557 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:05:16.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.1 up 2024-01-15T18:05:16.557 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set dev brx.1 master ceph-brx 2024-01-15T18:05:16.557 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:05:16.595 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-01-15T18:05:16.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:16.595 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:16.646 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-15T18:05:16.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:16.646 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:16.697 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:05:16.762 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:16.814 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:05:16.814 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:05:16.814 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:05:16.814 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:05:16.814 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:05:16.815 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:05:16.816 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:05:16.817 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:05:16.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.819 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:05:16.869 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:05:16.869 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-01-15T18:05:16.869 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-01-15T18:05:16.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:05:16.937 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:16.991 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:05:16.991 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:05:16.991 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:05:16.991 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:05:16.992 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:05:16.993 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:05:16.994 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:05:16.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:16.996 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:05:17.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:2024-01-15T18:05:17.004+0000 7f63f3d7e580 -1 init, newargv = 0x55981d9cf1f0 newargc=15 2024-01-15T18:05:17.010 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:ceph-fuse[34797]: starting ceph client 2024-01-15T18:05:17.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi143.stderr:ceph-fuse[34797]: starting fuse 2024-01-15T18:05:17.020 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:05:17.021 INFO:teuthology.orchestra.run.smithi143.stdout:44 2024-01-15T18:05:17.021 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-01-15T18:05:17.021 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:05:17.021 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:05:17.021 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:05:17.021 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:05:17.021 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:17.022 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.2")) 2024-01-15T18:05:17.023 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:05:17.144 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.2.34797.asok 2024-01-15T18:05:17.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:05:17.156 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.34797.asok 2024-01-15T18:05:17.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:17.156 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.34797.asok status 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "2", 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "34797", 2024-01-15T18:05:17.328 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4665, 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 4665 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:05:17.329 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.2:0", 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1174834017 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.2:0", 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1174834017 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.4665 192.168.0.2:0/1174834017", 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.2:0/1174834017", 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:05:17.330 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 8, 2024-01-15T18:05:17.331 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 20, 2024-01-15T18:05:17.331 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:05:17.331 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:05:17.331 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:05:17.331 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:05:17.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:05:17.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 Processor -- start 2024-01-15T18:05:17.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 -- start start 2024-01-15T18:05:17.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f678c0cdf20 0x7f678c0c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0cf400 0x7f678c0c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678c0cf970 con 0x7f678c15b120 2024-01-15T18:05:17.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678c0cfc50 con 0x7f678c0cdf20 2024-01-15T18:05:17.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6793000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0c4b10 con 0x7f678c0cf400 2024-01-15T18:05:17.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f678c0cdf20 0x7f678c0c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6791000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0cf400 0x7f678c0c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f678c0cdf20 0x7f678c0c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:40782/0 (socket says 172.21.15.143:40782) 2024-01-15T18:05:17.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 -- 172.21.15.143:0/1450689173 learned_addr learned my addr 172.21.15.143:0/1450689173 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:17.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 -- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0cf400 msgr2=0x7f678c0c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:17.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 --2- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0cf400 0x7f678c0c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 -- 172.21.15.143:0/1450689173 >> v1:172.21.15.204:6789/0 conn(0x7f678c15b120 legacy=0x7f678c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f678b400640 1 -- 172.21.15.143:0/1450689173 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c0c4df0 con 0x7f678c0cdf20 2024-01-15T18:05:17.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.472+0000 7f6791000640 1 --2- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0cf400 0x7f678c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:17.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.476+0000 7f678b400640 1 --2- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0c5310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6774009870 tx=0x7f677401c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899debb69746def5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:17.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.476+0000 7f678aa00640 1 -- 172.21.15.143:0/1450689173 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740ab600 con 0x7f678c0cdf20 2024-01-15T18:05:17.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.476+0000 7f678aa00640 1 -- 172.21.15.143:0/1450689173 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740a7050 con 0x7f678c0cdf20 2024-01-15T18:05:17.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.476+0000 7f678aa00640 1 -- 172.21.15.143:0/1450689173 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740ab980 con 0x7f678c0cdf20 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.476+0000 7f6793000640 1 -- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 msgr2=0x7f678c0c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.476+0000 7f6793000640 1 --2- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0c5310 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6774009870 tx=0x7f677401c4b0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- 172.21.15.143:0/1450689173 shutdown_connections 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 --2- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c0cf400 0x7f678c0c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 --2- 172.21.15.143:0/1450689173 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0c5310 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- 172.21.15.143:0/1450689173 >> 172.21.15.143:0/1450689173 conn(0x7f678c0d0e20 msgr2=0x7f678c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:17.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- 172.21.15.143:0/1450689173 shutdown_connections 2024-01-15T18:05:17.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- 172.21.15.143:0/1450689173 wait complete. 2024-01-15T18:05:17.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 Processor -- start 2024-01-15T18:05:17.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- start start 2024-01-15T18:05:17.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 0x7f678c1526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15b120 0x7f678c152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c06d0d0 con 0x7f678c0cf400 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c06cdf0 con 0x7f678c0cdf20 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6793000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c0b3da0 con 0x7f678c15b120 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 0x7f678c1526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 0x7f678c1526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33112/0 (socket says 172.21.15.143:33112) 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 -- 172.21.15.143:0/3656510385 learned_addr learned my addr 172.21.15.143:0/3656510385 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678be00640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f6791000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15b120 0x7f678c152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 -- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15b120 msgr2=0x7f678c152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15b120 0x7f678c152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 -- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 msgr2=0x7f678c0b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.480+0000 7f678b400640 1 -- 172.21.15.143:0/3656510385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784009a70 con 0x7f678c0cf400 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f678b400640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 0x7f678c1526e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67740ae040 tx=0x7f6774019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740b4420 con 0x7f678c0cf400 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740b45a0 con 0x7f678c0cf400 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67740094d0 con 0x7f678c0cf400 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740b4950 con 0x7f678c0cf400 2024-01-15T18:05:17.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678c0b4360 con 0x7f678c0cf400 2024-01-15T18:05:17.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678c0c5c00 con 0x7f678c0cf400 2024-01-15T18:05:17.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.484+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f67740b2070 con 0x7f678c0cf400 2024-01-15T18:05:17.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.488+0000 7f6783e00640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67640668f0 0x7f6764068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.488+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f677411dd60 con 0x7f678c0cf400 2024-01-15T18:05:17.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.488+0000 7f678be00640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67640668f0 0x7f6764068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.488+0000 7f678be00640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67640668f0 0x7f6764068db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f678c15a450 tx=0x7f677c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.492+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f67740eb030 con 0x7f678c0cf400 2024-01-15T18:05:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.676+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f678c0b4550 con 0x7f678c0cf400 2024-01-15T18:05:17.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.676+0000 7f6783e00640 1 -- 172.21.15.143:0/3656510385 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f67740b0060 con 0x7f678c0cf400 2024-01-15T18:05:17.683 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:05:17.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.676+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67640668f0 msgr2=0x7f6764068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.676+0000 7f6793000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67640668f0 0x7f6764068db0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f678c15a450 tx=0x7f677c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:17.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.676+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 msgr2=0x7f678c1526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.676+0000 7f6793000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 0x7f678c1526e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f67740ae040 tx=0x7f6774019250 comp rx=0 tx=0).stop 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 shutdown_connections 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f67640668f0 0x7f6764068db0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f678c15b120 0x7f678c152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f678c0cf400 0x7f678c1526e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 --2- 172.21.15.143:0/3656510385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f678c0cdf20 0x7f678c0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 >> 172.21.15.143:0/3656510385 conn(0x7f678c0d0e20 msgr2=0x7f678c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 shutdown_connections 2024-01-15T18:05:17.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:17.680+0000 7f6793000640 1 -- 172.21.15.143:0/3656510385 wait complete. 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:05:17.706 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:05:17.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:17.707 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:05:17.716 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:17.716 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:17.716 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:17.716 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:17.717 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:05:17.718 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:17.719 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-15T18:05:17.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:17.719 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:17.719 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:17.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add name ceph-brx type bridge 2024-01-15T18:05:17.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr flush dev ceph-brx 2024-01-15T18:05:17.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set ceph-brx up 2024-01-15T18:05:17.720 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-15T18:05:17.720 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:17.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:17.832 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:05:17.850 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:05:17.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:17.852 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:05:17.934 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:05:17.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:17.935 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:17.936 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:17.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:05:17.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:05:17.936 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:05:17.936 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:18.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.029 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:05:18.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.039 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-01-15T18:05:18.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.088 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:18.089 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:18.089 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:18.089 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-15T18:05:18.089 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:18.170 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-15T18:05:18.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.170 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:18.170 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:18.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-15T18:05:18.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-15T18:05:18.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-15T18:05:18.170 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-15T18:05:18.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-15T18:05:18.171 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:18.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.331 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:18.332 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:18.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:05:18.332 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.0 master ceph-brx 2024-01-15T18:05:18.332 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:18.372 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:05:18.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:18.372 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:18.424 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:05:18.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:18.425 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:18.475 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:05:18.536 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:18.589 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:05:18.590 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:05:18.591 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:05:18.592 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:05:18.593 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:18.593 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:18.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.594 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:05:18.645 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:05:18.645 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:05:18.691 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:05:18.712 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:18.765 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:05:18.765 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:05:18.765 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:05:18.765 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:05:18.767 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:05:18.768 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:05:18.768 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:05:18.768 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:05:18.768 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:05:18.768 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:18.768 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:18.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:18.769 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:05:18.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:05:18.802+0000 7f6e7929b580 -1 init, newargv = 0x55fa480a6350 newargc=15 2024-01-15T18:05:18.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33172]: starting ceph client 2024-01-15T18:05:18.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33172]: starting fuse 2024-01-15T18:05:19.917 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:05:19.941 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:19.998 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:05:19.998 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:05:19.998 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:05:19.999 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:20.000 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:20.001 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:20.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.003 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:05:20.057 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:05:20.057 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:05:20.057 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:05:20.057 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:05:20.057 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:05:20.058 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:05:20.059 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:05:20.060 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:20.060 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:05:20.060 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:05:20.187 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.33172.asok 2024-01-15T18:05:20.191 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:05:20.200 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33172.asok 2024-01-15T18:05:20.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33172.asok status 2024-01-15T18:05:20.375 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:05:20.375 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:05:20.375 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "33172", 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:05:20.376 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 4556, 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 4556 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 1332076488 2024-01-15T18:05:20.377 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 1332076488 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.4556 192.168.0.1:0/1332076488", 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/1332076488", 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:05:20.378 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 8, 2024-01-15T18:05:20.379 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 20, 2024-01-15T18:05:20.379 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:05:20.379 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:05:20.379 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:05:20.379 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:05:20.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:05:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 Processor -- start 2024-01-15T18:05:20.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 -- start start 2024-01-15T18:05:20.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f54141660 0x7f4f540ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f5415ec80 0x7f4f54141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5415f310 con 0x7f4f54142470 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5415fbb0 con 0x7f4f5415ec80 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f5a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415f100 con 0x7f4f54141660 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f52a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f5415ec80 0x7f4f54141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f52a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f5415ec80 0x7f4f54141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33120/0 (socket says 172.21.15.143:33120) 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.528+0000 7f4f52a00640 1 -- 172.21.15.143:0/483427377 learned_addr learned my addr 172.21.15.143:0/483427377 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f53400640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f54141660 0x7f4f540ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52a00640 1 -- 172.21.15.143:0/483427377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f54141660 msgr2=0x7f4f540ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52a00640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f54141660 0x7f4f540ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52a00640 1 -- 172.21.15.143:0/483427377 >> v1:172.21.15.204:6789/0 conn(0x7f4f54142470 legacy=0x7f4f54071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52a00640 1 -- 172.21.15.143:0/483427377 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5415f5f0 con 0x7f4f5415ec80 2024-01-15T18:05:20.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f53400640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f54141660 0x7f4f540ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52a00640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f5415ec80 0x7f4f54141f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009870 tx=0x7f4f3c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10573d2add8501d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52000640 1 -- 172.21.15.143:0/483427377 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0aa5f0 con 0x7f4f5415ec80 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52000640 1 -- 172.21.15.143:0/483427377 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c0aa750 con 0x7f4f5415ec80 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.532+0000 7f4f52000640 1 -- 172.21.15.143:0/483427377 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0aaae0 con 0x7f4f5415ec80 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- 172.21.15.143:0/483427377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f5415ec80 msgr2=0x7f4f54141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f5415ec80 0x7f4f54141f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009870 tx=0x7f4f3c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- 172.21.15.143:0/483427377 shutdown_connections 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f5415ec80 0x7f4f54141f30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 --2- 172.21.15.143:0/483427377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f54141660 0x7f4f540ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- 172.21.15.143:0/483427377 >> 172.21.15.143:0/483427377 conn(0x7f4f5406c3f0 msgr2=0x7f4f5406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- 172.21.15.143:0/483427377 shutdown_connections 2024-01-15T18:05:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- 172.21.15.143:0/483427377 wait complete. 2024-01-15T18:05:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 Processor -- start 2024-01-15T18:05:20.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- start start 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f54141660 0x7f4f5415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 0x7f4f54159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f5415ec80 0x7f4f541438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415d8c0 con 0x7f4f54142470 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415a330 con 0x7f4f54141660 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f5a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f5415ccb0 con 0x7f4f5415ec80 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f52a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 0x7f4f54159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f52a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 0x7f4f54159df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33122/0 (socket says 172.21.15.143:33122) 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f52a00640 1 -- 172.21.15.143:0/446090655 learned_addr learned my addr 172.21.15.143:0/446090655 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f53e00640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f5415ec80 0x7f4f541438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.536+0000 7f4f53400640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f54141660 0x7f4f5415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f52a00640 1 -- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f5415ec80 msgr2=0x7f4f541438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f52a00640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f5415ec80 0x7f4f541438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f52a00640 1 -- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f54141660 msgr2=0x7f4f5415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f52a00640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f54141660 0x7f4f5415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f52a00640 1 -- 172.21.15.143:0/446090655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c0094d0 con 0x7f4f54142470 2024-01-15T18:05:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f52a00640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 0x7f4f54159df0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0ac040 tx=0x7f4f3c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0b3800 con 0x7f4f54142470 2024-01-15T18:05:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c0b3980 con 0x7f4f54142470 2024-01-15T18:05:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0b3d30 con 0x7f4f54142470 2024-01-15T18:05:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f540cd150 con 0x7f4f54142470 2024-01-15T18:05:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5415d270 con 0x7f4f54142470 2024-01-15T18:05:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f54067500 con 0x7f4f54142470 2024-01-15T18:05:20.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f4f3c0b2380 con 0x7f4f54142470 2024-01-15T18:05:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f4b400640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f2c0668f0 0x7f4f2c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f3c0a7070 con 0x7f4f54142470 2024-01-15T18:05:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.540+0000 7f4f53400640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f2c0668f0 0x7f4f2c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.544+0000 7f4f53400640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f2c0668f0 0x7f4f2c068db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400ab60 tx=0x7f4f44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.548+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f3c0ebe10 con 0x7f4f54142470 2024-01-15T18:05:20.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.732+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f5415da00 con 0x7f4f54142470 2024-01-15T18:05:20.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.732+0000 7f4f4b400640 1 -- 172.21.15.143:0/446090655 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f5415da00 con 0x7f4f54142470 2024-01-15T18:05:20.741 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:05:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f2c0668f0 msgr2=0x7f4f2c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f2c0668f0 0x7f4f2c068db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400ab60 tx=0x7f4f44008040 comp rx=0 tx=0).stop 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 msgr2=0x7f4f54159df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 0x7f4f54159df0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0ac040 tx=0x7f4f3c01ad10 comp rx=0 tx=0).stop 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 shutdown_connections 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f2c0668f0 0x7f4f2c068db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f5415ec80 0x7f4f541438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f54142470 0x7f4f54159df0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 --2- 172.21.15.143:0/446090655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f54141660 0x7f4f5415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 >> 172.21.15.143:0/446090655 conn(0x7f4f5406c3f0 msgr2=0x7f4f54070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:20.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 shutdown_connections 2024-01-15T18:05:20.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:20.736+0000 7f4f5a200640 1 -- 172.21.15.143:0/446090655 wait complete. 2024-01-15T18:05:20.763 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:05:20.763 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:05:20.764 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-15T18:05:20.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.764 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:05:20.775 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:20.775 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:20.775 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:20.775 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.775 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:20.775 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:20.776 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42417sec preferred_lft 42417sec 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.777 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:05:20.778 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:20.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.779 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:05:20.834 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:05:20.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.836 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:05:20.890 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:05:20.890 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:05:20.890 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:05:20.891 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:05:20.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.892 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:20.892 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:20.892 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:05:20.892 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:05:20.892 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:05:20.892 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:20.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.980 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:05:20.990 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:05:20.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:20.991 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list-id 2024-01-15T18:05:21.040 INFO:teuthology.orchestra.run.smithi204.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-15T18:05:21.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.041 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:21.041 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:21.041 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:05:21.041 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-15T18:05:21.042 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:21.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.121 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-15T18:05:21.184 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:21.184 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:21.184 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 6e:32:17:f9:2f:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::6c32:17ff:fef9:2f2c/64 scope link 2024-01-15T18:05:21.185 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.187 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-15T18:05:21.247 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:05:21.247 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:05:21.247 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:05:21.247 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.247 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 6e:32:17:f9:2f:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::6c32:17ff:fef9:2f2c/64 scope link 2024-01-15T18:05:21.248 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:05:21.249 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-15T18:05:21.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-15T18:05:21.250 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:21.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.384 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:05:21.384 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:05:21.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.1 up 2024-01-15T18:05:21.384 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set dev brx.1 master ceph-brx 2024-01-15T18:05:21.384 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:05:21.422 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-01-15T18:05:21.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:21.422 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:21.472 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-15T18:05:21.472 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:05:21.472 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:21.523 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:05:21.584 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:21.641 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:05:21.641 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:05:21.642 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:21.643 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:05:21.644 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:05:21.645 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:21.645 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:21.645 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:21.645 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:05:21.645 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:05:21.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.646 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:05:21.696 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:05:21.696 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-01-15T18:05:21.696 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-01-15T18:05:21.742 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:05:21.764 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:05:21.817 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:05:21.817 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:05:21.817 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:05:21.817 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:05:21.817 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:05:21.817 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:05:21.818 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:05:21.819 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:21.820 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:05:21.820 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:21.820 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:05:21.820 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:05:21.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.821 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:05:21.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi204.stderr:2024-01-15T18:05:21.829+0000 7f6cf7676580 -1 init, newargv = 0x55a7e0e23380 newargc=15 2024-01-15T18:05:21.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi204.stderr:ceph-fuse[33284]: starting ceph client 2024-01-15T18:05:21.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi204.stderr:ceph-fuse[33284]: starting fuse 2024-01-15T18:05:21.845 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:05:21.845 INFO:teuthology.orchestra.run.smithi204.stdout:44 2024-01-15T18:05:21.845 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:05:21.846 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.3")) 2024-01-15T18:05:21.847 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:05:21.966 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.3.33284.asok 2024-01-15T18:05:21.969 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:05:21.978 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33284.asok 2024-01-15T18:05:21.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:21.978 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33284.asok status 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "3", 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "33284", 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:05:22.145 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 4562, 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 4562 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.2:0", 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 886009187 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:05:22.146 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.2:0", 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 886009187 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.4562 192.168.0.2:0/886009187", 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.2:0/886009187", 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 8, 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 20, 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:05:22.147 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:05:22.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.160 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:05:22.160 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:22.170 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:05:22.170 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:22.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.170 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:22.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.236 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:05:22.236 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:22.287 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:05:22.287 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:22.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.287 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:22.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.353 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:05:22.354 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:22.363 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:05:22.363 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:22.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.363 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:22.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.423 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:05:22.424 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:22.476 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:05:22.476 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:22.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:22.477 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:22.538 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-15T18:05:22.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:22.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 Processor -- start 2024-01-15T18:05:22.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 -- start start 2024-01-15T18:05:22.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa918157300 0x7fa918159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9180b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9180b2a20 con 0x7fa9180b48f0 2024-01-15T18:05:22.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9180b35a0 con 0x7fa918157300 2024-01-15T18:05:22.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa91cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9180b4d80 con 0x7fa9180b21d0 2024-01-15T18:05:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.696+0000 7fa915400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9180b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa918157300 0x7fa918159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9180b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41706/0 (socket says 172.21.15.143:41706) 2024-01-15T18:05:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915400640 1 -- 172.21.15.143:0/479696976 learned_addr learned my addr 172.21.15.143:0/479696976 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa918157300 0x7fa918159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:40822/0 (socket says 172.21.15.143:40822) 2024-01-15T18:05:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa914a00640 1 -- 172.21.15.143:0/479696976 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361036748 0 0) 0x7fa9180b2a20 con 0x7fa9180b48f0 2024-01-15T18:05:22.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa914a00640 1 -- 172.21.15.143:0/479696976 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f8003530 con 0x7fa9180b48f0 2024-01-15T18:05:22.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 -- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 msgr2=0x7fa9180b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:22.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 --2- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9180b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 -- 172.21.15.143:0/479696976 >> v1:172.21.15.204:6789/0 conn(0x7fa9180b48f0 legacy=0x7fa91815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 -- 172.21.15.143:0/479696976 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9180b2d00 con 0x7fa918157300 2024-01-15T18:05:22.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915400640 1 --2- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9180b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:22.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa915e00640 1 --2- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa918157300 0x7fa918159b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa9080097c0 tx=0x7fa90801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1ca5191cc79caa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:22.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa914a00640 1 -- 172.21.15.143:0/479696976 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080ab650 con 0x7fa918157300 2024-01-15T18:05:22.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa914a00640 1 -- 172.21.15.143:0/479696976 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9080ab7b0 con 0x7fa918157300 2024-01-15T18:05:22.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.700+0000 7fa914a00640 1 -- 172.21.15.143:0/479696976 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080abb40 con 0x7fa918157300 2024-01-15T18:05:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa918157300 msgr2=0x7fa918159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 --2- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa918157300 0x7fa918159b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa9080097c0 tx=0x7fa90801c450 comp rx=0 tx=0).stop 2024-01-15T18:05:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- 172.21.15.143:0/479696976 shutdown_connections 2024-01-15T18:05:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 --2- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9180b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 --2- 172.21.15.143:0/479696976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa918157300 0x7fa918159b70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- 172.21.15.143:0/479696976 >> 172.21.15.143:0/479696976 conn(0x7fa91806c3f0 msgr2=0x7fa91806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:22.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- 172.21.15.143:0/479696976 shutdown_connections 2024-01-15T18:05:22.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- 172.21.15.143:0/479696976 wait complete. 2024-01-15T18:05:22.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 Processor -- start 2024-01-15T18:05:22.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- start start 2024-01-15T18:05:22.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9181363f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9180b48f0 0x7fa9181555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 0x7fa918155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91815b730 con 0x7fa918157300 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91815aab0 con 0x7fa9180b48f0 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.704+0000 7fa91cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91815ad90 con 0x7fa9180b21d0 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 0x7fa918155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 0x7fa918155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33132/0 (socket says 172.21.15.143:33132) 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 -- 172.21.15.143:0/1754979193 learned_addr learned my addr 172.21.15.143:0/1754979193 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa915400640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9180b48f0 0x7fa9181555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa915e00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9181363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 -- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 msgr2=0x7fa9181363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9181363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 -- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9180b48f0 msgr2=0x7fa9181555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9180b48f0 0x7fa9181555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 -- 172.21.15.143:0/1754979193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90000b770 con 0x7fa918157300 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa916800640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 0x7fa918155b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c730 tx=0x7fa91000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910003b80 con 0x7fa918157300 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910003ce0 con 0x7fa918157300 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa908009420 con 0x7fa918157300 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91000aa90 con 0x7fa918157300 2024-01-15T18:05:22.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91815b350 con 0x7fa918157300 2024-01-15T18:05:22.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa918067500 con 0x7fa918157300 2024-01-15T18:05:22.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.708+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fa91000abf0 con 0x7fa918157300 2024-01-15T18:05:22.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.712+0000 7fa90e000640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8f00668f0 0x7fa8f0068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.712+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9100840f0 con 0x7fa918157300 2024-01-15T18:05:22.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.712+0000 7fa915e00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8f00668f0 0x7fa8f0068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.712+0000 7fa915e00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8f00668f0 0x7fa8f0068db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9080af000 tx=0x7fa9080a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:22.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.716+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa9100513d0 con 0x7fa918157300 2024-01-15T18:05:22.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.864+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa91806e4e0 con 0x7fa918157300 2024-01-15T18:05:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.864+0000 7fa90e000640 1 -- 172.21.15.143:0/1754979193 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7fa9100549f0 con 0x7fa918157300 2024-01-15T18:05:22.873 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:22.873 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8f00668f0 msgr2=0x7fa8f0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8f00668f0 0x7fa8f0068db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9080af000 tx=0x7fa9080a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 msgr2=0x7fa918155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 0x7fa918155b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa91000c730 tx=0x7fa91000cbf0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 shutdown_connections 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8f00668f0 0x7fa8f0068db0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa918157300 0x7fa918155b30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9180b48f0 0x7fa9181555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 --2- 172.21.15.143:0/1754979193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa9180b21d0 0x7fa9181363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 >> 172.21.15.143:0/1754979193 conn(0x7fa91806c3f0 msgr2=0x7fa918156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:22.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 shutdown_connections 2024-01-15T18:05:22.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:22.868+0000 7fa91cc00640 1 -- 172.21.15.143:0/1754979193 wait complete. 2024-01-15T18:05:22.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:22.896 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:23.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 Processor -- start 2024-01-15T18:05:23.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 -- start start 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd4814b110 0x7fdd480cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd48071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd480b30a0 con 0x7fdd480cc8f0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd48070340 con 0x7fdd4814b110 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd4d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48070620 con 0x7fdd480b4150 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd46800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdd4814b110 0x7fdd480cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd45e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdd480cc8f0 0x7fdd480b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57414/0 (socket says 172.21.15.143:57414) 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd45e00640 1 -- 172.21.15.143:0/3262329757 learned_addr learned my addr 172.21.15.143:0/3262329757 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd47200640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd48071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd46800640 1 -- 172.21.15.143:0/3262329757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 msgr2=0x7fdd48071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd46800640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd48071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd46800640 1 -- 172.21.15.143:0/3262329757 >> v1:172.21.15.143:6789/0 conn(0x7fdd480cc8f0 legacy=0x7fdd480b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd46800640 1 -- 172.21.15.143:0/3262329757 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48070900 con 0x7fdd4814b110 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd45400640 1 -- 172.21.15.143:0/3262329757 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607580846 0 0) 0x7fdd480b30a0 con 0x7fdd480cc8f0 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.028+0000 7fdd47200640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd48071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd46800640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd4814b110 0x7fdd480cc3b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd380097c0 tx=0x7fdd3801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ecfb26fb0ab56 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd45400640 1 -- 172.21.15.143:0/3262329757 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd380ac640 con 0x7fdd4814b110 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd45400640 1 -- 172.21.15.143:0/3262329757 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd480b2e90 con 0x7fdd4814b110 2024-01-15T18:05:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd45400640 1 -- 172.21.15.143:0/3262329757 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd380ac9c0 con 0x7fdd4814b110 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 -- 172.21.15.143:0/3262329757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd4814b110 msgr2=0x7fdd480cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd4814b110 0x7fdd480cc3b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd380097c0 tx=0x7fdd3801c3c0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 -- 172.21.15.143:0/3262329757 shutdown_connections 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd48071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 --2- 172.21.15.143:0/3262329757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd4814b110 0x7fdd480cc3b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 -- 172.21.15.143:0/3262329757 >> 172.21.15.143:0/3262329757 conn(0x7fdd4806c3f0 msgr2=0x7fdd4806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 -- 172.21.15.143:0/3262329757 shutdown_connections 2024-01-15T18:05:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.032+0000 7fdd4d400640 1 -- 172.21.15.143:0/3262329757 wait complete. 2024-01-15T18:05:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 Processor -- start 2024-01-15T18:05:23.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 -- start start 2024-01-15T18:05:23.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd481606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 0x7fdd4814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd4814b110 0x7fdd480ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd480ae3a0 con 0x7fdd480cc8f0 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd48160be0 con 0x7fdd4814b110 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd481501a0 con 0x7fdd480b4150 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 0x7fdd4814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 0x7fdd4814f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33144/0 (socket says 172.21.15.143:33144) 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 -- 172.21.15.143:0/1215378997 learned_addr learned my addr 172.21.15.143:0/1215378997 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd46800640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd481606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 -- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 msgr2=0x7fdd481606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd481606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd47200640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd4814b110 0x7fdd480ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 -- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd4814b110 msgr2=0x7fdd480ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd4814b110 0x7fdd480ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 -- 172.21.15.143:0/1215378997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38009420 con 0x7fdd480cc8f0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd46800640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd481606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd45e00640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 0x7fdd4814f6b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd300011f0 tx=0x7fdd3000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd300074e0 con 0x7fdd480cc8f0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd30007640 con 0x7fdd480cc8f0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd48150480 con 0x7fdd480cc8f0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30004a10 con 0x7fdd480cc8f0 2024-01-15T18:05:23.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.036+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd48150a40 con 0x7fdd480cc8f0 2024-01-15T18:05:23.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.040+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd48067500 con 0x7fdd480cc8f0 2024-01-15T18:05:23.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.040+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fdd30004b70 con 0x7fdd480cc8f0 2024-01-15T18:05:23.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.040+0000 7fdd3ea00640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd200668f0 0x7fdd20068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.040+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd30083a50 con 0x7fdd480cc8f0 2024-01-15T18:05:23.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.040+0000 7fdd46800640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd200668f0 0x7fdd20068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.040+0000 7fdd46800640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd200668f0 0x7fdd20068db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd380ae000 tx=0x7fdd38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.044+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd30050bf0 con 0x7fdd480cc8f0 2024-01-15T18:05:23.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.196+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd48071d50 con 0x7fdd480cc8f0 2024-01-15T18:05:23.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.196+0000 7fdd3ea00640 1 -- 172.21.15.143:0/1215378997 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd30018020 con 0x7fdd480cc8f0 2024-01-15T18:05:23.203 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.196+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd200668f0 msgr2=0x7fdd20068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.196+0000 7fdd4d400640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd200668f0 0x7fdd20068db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdd380ae000 tx=0x7fdd38008040 comp rx=0 tx=0).stop 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.196+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 msgr2=0x7fdd4814f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.196+0000 7fdd4d400640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 0x7fdd4814f6b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd300011f0 tx=0x7fdd3000c9b0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 shutdown_connections 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd200668f0 0x7fdd20068db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd4814b110 0x7fdd480ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd480cc8f0 0x7fdd4814f6b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 --2- 172.21.15.143:0/1215378997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd480b4150 0x7fdd481606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 >> 172.21.15.143:0/1215378997 conn(0x7fdd4806c3f0 msgr2=0x7fdd480aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 shutdown_connections 2024-01-15T18:05:23.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.200+0000 7fdd4d400640 1 -- 172.21.15.143:0/1215378997 wait complete. 2024-01-15T18:05:23.225 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff0bc00640 1 Processor -- start 2024-01-15T18:05:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff0bc00640 1 -- start start 2024-01-15T18:05:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff0bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 0x7eff040cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff0bc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff040ceb50 con 0x7eff040ce520 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff0bc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0415c110 con 0x7eff040722c0 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff0bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0415b7e0 con 0x7eff04143780 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff03e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7eff040ce520 0x7eff04071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57422/0 (socket says 172.21.15.143:57422) 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.360+0000 7eff03e00640 1 -- 172.21.15.143:0/2224761162 learned_addr learned my addr 172.21.15.143:0/2224761162 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff09200640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 0x7eff040cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386747726 0 0) 0x7efeec001120 con 0x7eff040ce520 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee4003530 con 0x7eff040ce520 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540016615 0 0) 0x7eff0415c110 con 0x7eff040722c0 2024-01-15T18:05:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeec001120 con 0x7eff040722c0 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 520105123 0 0) 0x7efee4003530 con 0x7eff040ce520 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 msgr2=0x7eff040cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 0x7eff040cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 >> v1:172.21.15.204:6789/0 conn(0x7eff040722c0 legacy=0x7eff0415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0415bac0 con 0x7eff040ce520 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7efeec001810 con 0x7eff040ce520 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 >> v1:172.21.15.143:6789/0 conn(0x7eff040ce520 legacy=0x7eff04071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee4003b40 0x7efee4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee40047d0 0x7efee4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 0x7efee4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0415b7e0 con 0x7efee40047d0 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee400ba40 con 0x7efee4003b40 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee400bbd0 con 0x7eff04143780 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03e00640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee40047d0 0x7efee4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff09200640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee4003b40 0x7efee4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff09c00640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 0x7efee4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03e00640 1 -- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 msgr2=0x7efee4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03e00640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff04143780 0x7efee4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03e00640 1 -- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee4003b40 msgr2=0x7efee4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03e00640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee4003b40 0x7efee4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.364+0000 7eff03e00640 1 -- 172.21.15.143:0/2224761162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4003530 con 0x7efee40047d0 2024-01-15T18:05:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff03e00640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee40047d0 0x7efee4004c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efeec00c160 tx=0x7efeec01cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6227272c6bab7c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0058e0 con 0x7efee40047d0 2024-01-15T18:05:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec005a60 con 0x7efee40047d0 2024-01-15T18:05:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff03400640 1 -- 172.21.15.143:0/2224761162 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec005e10 con 0x7efee40047d0 2024-01-15T18:05:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 -- 172.21.15.143:0/2224761162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee40047d0 msgr2=0x7efee4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 --2- 172.21.15.143:0/2224761162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee40047d0 0x7efee4004c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efeec00c160 tx=0x7efeec01cb40 comp rx=0 tx=0).stop 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff09c00640 1 -- 172.21.15.143:0/2224761162 reap_dead start 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 -- 172.21.15.143:0/2224761162 shutdown_connections 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 -- 172.21.15.143:0/2224761162 >> 172.21.15.143:0/2224761162 conn(0x7eff0406c3f0 msgr2=0x7eff0406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 -- 172.21.15.143:0/2224761162 shutdown_connections 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 -- 172.21.15.143:0/2224761162 wait complete. 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 Processor -- start 2024-01-15T18:05:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.368+0000 7eff0bc00640 1 -- start start 2024-01-15T18:05:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40047d0 0x7eff0413f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0412e0a0 0x7eff04141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 0x7eff04159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0412e8b0 con 0x7eff04159510 2024-01-15T18:05:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0412eb90 con 0x7efee40047d0 2024-01-15T18:05:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0412ee70 con 0x7eff0412e0a0 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 0x7eff04159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 0x7eff04159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33156/0 (socket says 172.21.15.143:33156) 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 -- 172.21.15.143:0/185488011 learned_addr learned my addr 172.21.15.143:0/185488011 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09200640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40047d0 0x7eff0413f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff03e00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0412e0a0 0x7eff04141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 -- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0412e0a0 msgr2=0x7eff04141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0412e0a0 0x7eff04141e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 -- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40047d0 msgr2=0x7eff0413f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40047d0 0x7eff0413f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 -- 172.21.15.143:0/185488011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040ca190 con 0x7eff04159510 2024-01-15T18:05:23.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09200640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40047d0 0x7eff0413f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff09c00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 0x7eff04159950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff04143d80 tx=0x7efefc012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc010910 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efefc010a90 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efefc010e40 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff04159e90 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0412e500 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.372+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff040ca380 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.376+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7efefc016480 con 0x7eff04159510 2024-01-15T18:05:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.376+0000 7eff00c00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efedc0668f0 0x7efedc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.376+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efefc087a60 con 0x7eff04159510 2024-01-15T18:05:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.376+0000 7eff09200640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efedc0668f0 0x7efedc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.376+0000 7eff09200640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efedc0668f0 0x7efedc068db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efef4009a80 tx=0x7efef4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.380+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efefc054c00 con 0x7eff04159510 2024-01-15T18:05:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.532+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff04067500 con 0x7eff04159510 2024-01-15T18:05:23.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.532+0000 7eff00c00640 1 -- 172.21.15.143:0/185488011 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7efefc058220 con 0x7eff04159510 2024-01-15T18:05:23.540 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efedc0668f0 msgr2=0x7efedc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efedc0668f0 0x7efedc068db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efef4009a80 tx=0x7efef4009190 comp rx=0 tx=0).stop 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 msgr2=0x7eff04159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 0x7eff04159950 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7eff04143d80 tx=0x7efefc012c90 comp rx=0 tx=0).stop 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 shutdown_connections 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efedc0668f0 0x7efedc068db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff04159510 0x7eff04159950 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff0412e0a0 0x7eff04141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 --2- 172.21.15.143:0/185488011 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efee40047d0 0x7eff0413f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 >> 172.21.15.143:0/185488011 conn(0x7eff0406c3f0 msgr2=0x7eff040add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 shutdown_connections 2024-01-15T18:05:23.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.536+0000 7eff0bc00640 1 -- 172.21.15.143:0/185488011 wait complete. 2024-01-15T18:05:23.562 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:23.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd04a800640 1 Processor -- start 2024-01-15T18:05:23.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd04a800640 1 -- start start 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd04a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0440cf400 0x7fd0440cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd04a800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04415b5b0 con 0x7fd0440ce390 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd04a800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0440cba70 con 0x7fd0440cda70 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd04a800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0440cbd10 con 0x7fd0440cf400 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd042a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0440cf400 0x7fd0440cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd043e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd0440ce390 0x7fd0440b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57436/0 (socket says 172.21.15.143:57436) 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd042a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0440cf400 0x7fd0440cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33168/0 (socket says 172.21.15.143:33168) 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd043e00640 1 -- 172.21.15.143:0/1342475670 learned_addr learned my addr 172.21.15.143:0/1342475670 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd042a00640 1 -- 172.21.15.143:0/1342475670 >> v1:172.21.15.204:6789/0 conn(0x7fd0440cda70 legacy=0x7fd0440cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd042a00640 1 -- 172.21.15.143:0/1342475670 >> v1:172.21.15.143:6789/0 conn(0x7fd0440ce390 legacy=0x7fd0440b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd042a00640 1 -- 172.21.15.143:0/1342475670 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440cbff0 con 0x7fd0440cf400 2024-01-15T18:05:23.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.696+0000 7fd042000640 1 -- 172.21.15.143:0/1342475670 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2258056647 0 0) 0x7fd04415b5b0 con 0x7fd0440ce390 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.700+0000 7fd042a00640 1 --2- 172.21.15.143:0/1342475670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cf400 0x7fd0440cde50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0097c0 tx=0x7fd02c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4765896598db75e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.700+0000 7fd042000640 1 -- 172.21.15.143:0/1342475670 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0ab670 con 0x7fd0440cf400 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.700+0000 7fd042000640 1 -- 172.21.15.143:0/1342475670 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c0ab7d0 con 0x7fd0440cf400 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.700+0000 7fd042000640 1 -- 172.21.15.143:0/1342475670 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0abb60 con 0x7fd0440cf400 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- 172.21.15.143:0/1342475670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cf400 msgr2=0x7fd0440cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 --2- 172.21.15.143:0/1342475670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cf400 0x7fd0440cde50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0097c0 tx=0x7fd02c01c450 comp rx=0 tx=0).stop 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- 172.21.15.143:0/1342475670 shutdown_connections 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 --2- 172.21.15.143:0/1342475670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cf400 0x7fd0440cde50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- 172.21.15.143:0/1342475670 >> 172.21.15.143:0/1342475670 conn(0x7fd0440d0e20 msgr2=0x7fd0440d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- 172.21.15.143:0/1342475670 shutdown_connections 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- 172.21.15.143:0/1342475670 wait complete. 2024-01-15T18:05:23.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 Processor -- start 2024-01-15T18:05:23.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- start start 2024-01-15T18:05:23.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 0x7fd044153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0440ce390 0x7fd0441525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0440cf400 0x7fd0440c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044155960 con 0x7fd0440cda70 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd044152b20 con 0x7fd0440cf400 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd04a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0440afd20 con 0x7fd0440ce390 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 0x7fd044153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 0x7fd044153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33176/0 (socket says 172.21.15.143:33176) 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 -- 172.21.15.143:0/1317029990 learned_addr learned my addr 172.21.15.143:0/1317029990 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd042a00640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0440ce390 0x7fd0441525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043e00640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0440cf400 0x7fd0440c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 -- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0440ce390 msgr2=0x7fd0441525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0440ce390 0x7fd0441525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 -- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0440cf400 msgr2=0x7fd0440c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0440cf400 0x7fd0440c80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.704+0000 7fd043400640 1 -- 172.21.15.143:0/1317029990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c009420 con 0x7fd0440cda70 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd043400640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 0x7fd044153220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c3c0 tx=0x7fd03400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0340052d0 con 0x7fd0440cda70 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034005430 con 0x7fd0440cda70 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd044155bb0 con 0x7fd0440cda70 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034003840 con 0x7fd0440cda70 2024-01-15T18:05:23.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0441548f0 con 0x7fd0440cda70 2024-01-15T18:05:23.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd044067500 con 0x7fd0440cda70 2024-01-15T18:05:23.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fd0340039a0 con 0x7fd0440cda70 2024-01-15T18:05:23.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd03b400640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd01c0668f0 0x7fd01c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0340839d0 con 0x7fd0440cda70 2024-01-15T18:05:23.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.708+0000 7fd042a00640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd01c0668f0 0x7fd01c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.712+0000 7fd042a00640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd01c0668f0 0x7fd01c068db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0af000 tx=0x7fd02c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.712+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd034050ba0 con 0x7fd0440cda70 2024-01-15T18:05:23.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.864+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0440b0b10 con 0x7fd0440cda70 2024-01-15T18:05:23.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.864+0000 7fd03b400640 1 -- 172.21.15.143:0/1317029990 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd034017020 con 0x7fd0440cda70 2024-01-15T18:05:23.872 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd01c0668f0 msgr2=0x7fd01c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd01c0668f0 0x7fd01c068db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd02c0af000 tx=0x7fd02c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 msgr2=0x7fd044153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 0x7fd044153220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c3c0 tx=0x7fd03400c880 comp rx=0 tx=0).stop 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 shutdown_connections 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd01c0668f0 0x7fd01c068db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0440cf400 0x7fd0440c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0440ce390 0x7fd0441525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 --2- 172.21.15.143:0/1317029990 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0440cda70 0x7fd044153220 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 >> 172.21.15.143:0/1317029990 conn(0x7fd0440d0e20 msgr2=0x7fd04406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 shutdown_connections 2024-01-15T18:05:23.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:23.868+0000 7fd04a800640 1 -- 172.21.15.143:0/1317029990 wait complete. 2024-01-15T18:05:23.894 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:24.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 Processor -- start 2024-01-15T18:05:24.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 -- start start 2024-01-15T18:05:24.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a7406bfb0 0x7f1a7406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1a740d0210 0x7f1a740ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a740cbff0 con 0x7f1a740b0310 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a740d0c50 con 0x7f1a740d0210 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a79400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7406c8d0 con 0x7f1a7406bfb0 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a73200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1a740d0210 0x7f1a740ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a71e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1a740b0310 0x7f1a740cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57438/0 (socket says 172.21.15.143:57438) 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a73200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1a740d0210 0x7f1a740ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33190/0 (socket says 172.21.15.143:33190) 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a71e00640 1 -- 172.21.15.143:0/1554933889 learned_addr learned my addr 172.21.15.143:0/1554933889 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:24.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a72800640 1 --2- 172.21.15.143:0/1554933889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a7406bfb0 0x7f1a7406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a73200640 1 -- 172.21.15.143:0/1554933889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a7406bfb0 msgr2=0x7f1a7406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:24.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a73200640 1 --2- 172.21.15.143:0/1554933889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a7406bfb0 0x7f1a7406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a73200640 1 -- 172.21.15.143:0/1554933889 >> v1:172.21.15.143:6789/0 conn(0x7f1a740b0310 legacy=0x7f1a740cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.024+0000 7f1a73200640 1 -- 172.21.15.143:0/1554933889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740cb6c0 con 0x7f1a740d0210 2024-01-15T18:05:24.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a73200640 1 --2- 172.21.15.143:0/1554933889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a740d0210 0x7f1a740ccb90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009d90 tx=0x7f1a6c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7114078befd56f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a71400640 1 -- 172.21.15.143:0/1554933889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0a5950 con 0x7f1a740d0210 2024-01-15T18:05:24.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a71400640 1 -- 172.21.15.143:0/1554933889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6c0a5ab0 con 0x7f1a740d0210 2024-01-15T18:05:24.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a71400640 1 -- 172.21.15.143:0/1554933889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0a5e40 con 0x7f1a740d0210 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 -- 172.21.15.143:0/1554933889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a740d0210 msgr2=0x7f1a740ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 --2- 172.21.15.143:0/1554933889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a740d0210 0x7f1a740ccb90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009d90 tx=0x7f1a6c01c890 comp rx=0 tx=0).stop 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 -- 172.21.15.143:0/1554933889 shutdown_connections 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 --2- 172.21.15.143:0/1554933889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a740d0210 0x7f1a740ccb90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 --2- 172.21.15.143:0/1554933889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a7406bfb0 0x7f1a7406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 -- 172.21.15.143:0/1554933889 >> 172.21.15.143:0/1554933889 conn(0x7f1a740d0ed0 msgr2=0x7f1a740d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 -- 172.21.15.143:0/1554933889 shutdown_connections 2024-01-15T18:05:24.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.028+0000 7f1a79400640 1 -- 172.21.15.143:0/1554933889 wait complete. 2024-01-15T18:05:24.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 Processor -- start 2024-01-15T18:05:24.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 -- start start 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 0x7f1a74153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a740b0310 0x7f1a7414b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a740d0210 0x7f1a74160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74155970 con 0x7f1a7406bfb0 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74155c10 con 0x7f1a740d0210 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74155ef0 con 0x7f1a740b0310 2024-01-15T18:05:24.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 0x7f1a74153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 0x7f1a74153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33202/0 (socket says 172.21.15.143:33202) 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 -- 172.21.15.143:0/3985173886 learned_addr learned my addr 172.21.15.143:0/3985173886 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a71e00640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a740b0310 0x7f1a7414b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a73200640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a740d0210 0x7f1a74160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 -- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a740b0310 msgr2=0x7f1a7414b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a740b0310 0x7f1a7414b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 -- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a740d0210 msgr2=0x7f1a74160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a740d0210 0x7f1a74160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 -- 172.21.15.143:0/3985173886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c009a70 con 0x7f1a7406bfb0 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a73200640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a740d0210 0x7f1a74160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:24.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a71e00640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a740b0310 0x7f1a7414b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:24.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a72800640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 0x7f1a74153a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d5f0 tx=0x7f1a6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a640125f0 con 0x7f1a7406bfb0 2024-01-15T18:05:24.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64012750 con 0x7f1a7406bfb0 2024-01-15T18:05:24.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a741561d0 con 0x7f1a7406bfb0 2024-01-15T18:05:24.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64012ae0 con 0x7f1a7406bfb0 2024-01-15T18:05:24.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.032+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7415e620 con 0x7f1a7406bfb0 2024-01-15T18:05:24.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.036+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a741563f0 con 0x7f1a7406bfb0 2024-01-15T18:05:24.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.036+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f1a64012c40 con 0x7f1a7406bfb0 2024-01-15T18:05:24.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.036+0000 7f1a6aa00640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a4c0668f0 0x7f1a4c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.036+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a64084a20 con 0x7f1a7406bfb0 2024-01-15T18:05:24.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.036+0000 7f1a71e00640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a4c0668f0 0x7f1a4c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.036+0000 7f1a71e00640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a4c0668f0 0x7f1a4c068db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c005890 tx=0x7f1a5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.040+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1a64051b40 con 0x7f1a7406bfb0 2024-01-15T18:05:24.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.192+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a7415e870 con 0x7f1a7406bfb0 2024-01-15T18:05:24.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a6aa00640 1 -- 172.21.15.143:0/3985173886 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f1a64007380 con 0x7f1a7406bfb0 2024-01-15T18:05:24.202 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:24.202 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:24.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a4c0668f0 msgr2=0x7f1a4c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a4c0668f0 0x7f1a4c068db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c005890 tx=0x7f1a5c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:24.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 msgr2=0x7f1a74153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 0x7f1a74153a00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400d5f0 tx=0x7f1a6400dab0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 shutdown_connections 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a4c0668f0 0x7f1a4c068db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1a740d0210 0x7f1a74160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1a740b0310 0x7f1a7414b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 --2- 172.21.15.143:0/3985173886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1a7406bfb0 0x7f1a74153a00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 >> 172.21.15.143:0/3985173886 conn(0x7f1a740d0ed0 msgr2=0x7f1a740b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 shutdown_connections 2024-01-15T18:05:24.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.196+0000 7f1a79400640 1 -- 172.21.15.143:0/3985173886 wait complete. 2024-01-15T18:05:24.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:24.225 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:24.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 Processor -- start 2024-01-15T18:05:24.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 -- start start 2024-01-15T18:05:24.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff6a815d1f0 0x7ff6a806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a8070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a80b3370 con 0x7ff6a80712a0 2024-01-15T18:05:24.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a80716f0 con 0x7ff6a815d1f0 2024-01-15T18:05:24.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.356+0000 7ff6adc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a815d5d0 con 0x7ff6a806bb70 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a7400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff6a80712a0 0x7ff6a806cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57454/0 (socket says 172.21.15.143:57454) 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a7400640 1 -- 172.21.15.143:0/3779069474 learned_addr learned my addr 172.21.15.143:0/3779069474 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6000640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a8070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/3779069474 >> v2:172.21.15.204:3300/0 conn(0x7ff6a815d1f0 0x7ff6a806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a5600640 1 -- 172.21.15.143:0/3779069474 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418151808 0 0) 0x7ff6a80b3370 con 0x7ff6a80712a0 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a5600640 1 -- 172.21.15.143:0/3779069474 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff688003530 con 0x7ff6a80712a0 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6a00640 1 -- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 msgr2=0x7ff6a8070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a8070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6a00640 1 -- 172.21.15.143:0/3779069474 >> v1:172.21.15.143:6789/0 conn(0x7ff6a80712a0 legacy=0x7ff6a806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6a00640 1 -- 172.21.15.143:0/3779069474 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a806a800 con 0x7ff6a815d1f0 2024-01-15T18:05:24.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6000640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a8070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:24.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.360+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a815d1f0 0x7ff6a806b630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff6980097c0 tx=0x7ff69801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d576a76d6c05d35 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6a5600640 1 -- 172.21.15.143:0/3779069474 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980ab650 con 0x7ff6a815d1f0 2024-01-15T18:05:24.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6a5600640 1 -- 172.21.15.143:0/3779069474 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6980ab7b0 con 0x7ff6a815d1f0 2024-01-15T18:05:24.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6a5600640 1 -- 172.21.15.143:0/3779069474 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6980abb20 con 0x7ff6a815d1f0 2024-01-15T18:05:24.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 -- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a815d1f0 msgr2=0x7ff6a806b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a815d1f0 0x7ff6a806b630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff6980097c0 tx=0x7ff69801c450 comp rx=0 tx=0).stop 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 -- 172.21.15.143:0/3779069474 shutdown_connections 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a8070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 --2- 172.21.15.143:0/3779069474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a815d1f0 0x7ff6a806b630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 -- 172.21.15.143:0/3779069474 >> 172.21.15.143:0/3779069474 conn(0x7ff6a80b1610 msgr2=0x7ff6a80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 -- 172.21.15.143:0/3779069474 shutdown_connections 2024-01-15T18:05:24.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 -- 172.21.15.143:0/3779069474 wait complete. 2024-01-15T18:05:24.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 Processor -- start 2024-01-15T18:05:24.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 -- start start 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a81544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80712a0 0x7ff6a8149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.364+0000 7ff6adc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 0x7ff6a8149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6adc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8161390 con 0x7ff6a815d1f0 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6adc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8153ea0 con 0x7ff6a80712a0 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6adc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a815dfa0 con 0x7ff6a806bb70 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 0x7ff6a8149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 0x7ff6a8149c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33210/0 (socket says 172.21.15.143:33210) 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 -- 172.21.15.143:0/879393682 learned_addr learned my addr 172.21.15.143:0/879393682 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a6000640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80712a0 0x7ff6a8149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a81544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 -- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 msgr2=0x7ff6a81544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a81544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 -- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80712a0 msgr2=0x7ff6a8149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80712a0 0x7ff6a8149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 -- 172.21.15.143:0/879393682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff690009420 con 0x7ff6a815d1f0 2024-01-15T18:05:24.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6a7400640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 0x7ff6a8149c60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000cb40 tx=0x7ff6a000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000aca0 con 0x7ff6a815d1f0 2024-01-15T18:05:24.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a000ae00 con 0x7ff6a815d1f0 2024-01-15T18:05:24.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0012620 con 0x7ff6a815d1f0 2024-01-15T18:05:24.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff698009420 con 0x7ff6a815d1f0 2024-01-15T18:05:24.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a815d950 con 0x7ff6a815d1f0 2024-01-15T18:05:24.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a8074b00 con 0x7ff6a815d1f0 2024-01-15T18:05:24.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.368+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff6a0007070 con 0x7ff6a815d1f0 2024-01-15T18:05:24.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.372+0000 7ff69ea00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.372+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6a0084400 con 0x7ff6a815d1f0 2024-01-15T18:05:24.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.372+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.372+0000 7ff6a6a00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff6980af000 tx=0x7ff6980a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.376+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6a00516e0 con 0x7ff6a815d1f0 2024-01-15T18:05:24.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.524+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6a815d7b0 con 0x7ff6a815d1f0 2024-01-15T18:05:24.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff69ea00640 1 -- 172.21.15.143:0/879393682 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6a0017020 con 0x7ff6a815d1f0 2024-01-15T18:05:24.533 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 msgr2=0x7ff680068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff6980af000 tx=0x7ff6980a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 msgr2=0x7ff6a8149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 0x7ff6a8149c60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff6a000cb40 tx=0x7ff6a000a060 comp rx=0 tx=0).stop 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 shutdown_connections 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6800668f0 0x7ff680068db0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6a815d1f0 0x7ff6a8149c60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff6a80712a0 0x7ff6a8149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 --2- 172.21.15.143:0/879393682 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6a806bb70 0x7ff6a81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 >> 172.21.15.143:0/879393682 conn(0x7ff6a80b1610 msgr2=0x7ff6a8151b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 shutdown_connections 2024-01-15T18:05:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.528+0000 7ff6adc00640 1 -- 172.21.15.143:0/879393682 wait complete. 2024-01-15T18:05:24.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:24.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.688+0000 7f95b1600640 1 Processor -- start 2024-01-15T18:05:24.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.688+0000 7f95b1600640 1 -- start start 2024-01-15T18:05:24.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95b1600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95ac0cf410 0x7f95ac0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95b1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac0cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95b1600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0ce7c0 con 0x7f95ac0cb920 2024-01-15T18:05:24.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95b1600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0cef50 con 0x7f95ac0cf410 2024-01-15T18:05:24.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95b1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0d1d90 con 0x7f95ac0d1950 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac0cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f95ac0cf410 0x7f95ac0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95ab400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f95ac0cb920 0x7f95ac0cbdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43150/0 (socket says 172.21.15.143:43150) 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac0cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41828/0 (socket says 172.21.15.143:41828) 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95ab400640 1 -- 172.21.15.143:0/3516070196 learned_addr learned my addr 172.21.15.143:0/3516070196 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aaa00640 1 -- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 msgr2=0x7f95ac0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aaa00640 1 --2- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac0cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aaa00640 1 -- 172.21.15.143:0/3516070196 >> v1:172.21.15.204:6789/0 conn(0x7f95ac0cb920 legacy=0x7f95ac0cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95aaa00640 1 -- 172.21.15.143:0/3516070196 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0ce4e0 con 0x7f95ac0cf410 2024-01-15T18:05:24.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.692+0000 7f95a9600640 1 -- 172.21.15.143:0/3516070196 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388936189 0 0) 0x7f95a40014a0 con 0x7f95ac0cb920 2024-01-15T18:05:24.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95aaa00640 1 --2- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac0cde50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f959c0097c0 tx=0x7f959c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df5d1bf4b63be96d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95a9600640 1 -- 172.21.15.143:0/3516070196 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c0ab600 con 0x7f95ac0cf410 2024-01-15T18:05:24.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95a9600640 1 -- 172.21.15.143:0/3516070196 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0a7050 con 0x7f95ac0cf410 2024-01-15T18:05:24.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95a9600640 1 -- 172.21.15.143:0/3516070196 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c0ab980 con 0x7f95ac0cf410 2024-01-15T18:05:24.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 -- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 msgr2=0x7f95ac0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 --2- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac0cde50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f959c0097c0 tx=0x7f959c01c480 comp rx=0 tx=0).stop 2024-01-15T18:05:24.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 -- 172.21.15.143:0/3516070196 shutdown_connections 2024-01-15T18:05:24.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 --2- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac0cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 --2- 172.21.15.143:0/3516070196 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac0cde50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 -- 172.21.15.143:0/3516070196 >> 172.21.15.143:0/3516070196 conn(0x7f95ac0d0e30 msgr2=0x7f95ac0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 -- 172.21.15.143:0/3516070196 shutdown_connections 2024-01-15T18:05:24.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 -- 172.21.15.143:0/3516070196 wait complete. 2024-01-15T18:05:24.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 Processor -- start 2024-01-15T18:05:24.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.696+0000 7f95b1600640 1 -- start start 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 0x7f95ac163120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac06c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac06bfb0 con 0x7f95ac0cb920 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac163af0 con 0x7f95ac0cf410 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac163dd0 con 0x7f95ac0d1950 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 0x7f95ac163120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 0x7f95ac163120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33222/0 (socket says 172.21.15.143:33222) 2024-01-15T18:05:24.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 -- 172.21.15.143:0/2189781303 learned_addr learned my addr 172.21.15.143:0/2189781303 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aa000640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95ab400640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac06c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 -- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 msgr2=0x7f95ac06c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac06c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 -- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 msgr2=0x7f95ac15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac15aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 -- 172.21.15.143:0/2189781303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009420 con 0x7f95ac0cb920 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aa000640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95aaa00640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 0x7f95ac163120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f959c0ae040 tx=0x7f959c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c0b7450 con 0x7f95ac0cb920 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0b75b0 con 0x7f95ac0cb920 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c0b7960 con 0x7f95ac0cb920 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ac1640b0 con 0x7f95ac0cb920 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac163660 con 0x7f95ac0cb920 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.700+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac1637f0 con 0x7f95ac0cb920 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.704+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f959c0b7ac0 con 0x7f95ac0cb920 2024-01-15T18:05:24.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.704+0000 7f95a2a00640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95840668f0 0x7f9584068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:24.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.704+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f959c11ec70 con 0x7f95ac0cb920 2024-01-15T18:05:24.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.704+0000 7f95aa000640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95840668f0 0x7f9584068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:24.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.704+0000 7f95aa000640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95840668f0 0x7f9584068db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f95ac165240 tx=0x7f9594008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:24.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.708+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f959c0ebea0 con 0x7f95ac0cb920 2024-01-15T18:05:24.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.856+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95ac067500 con 0x7f95ac0cb920 2024-01-15T18:05:24.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.856+0000 7f95a2a00640 1 -- 172.21.15.143:0/2189781303 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f959c0b0020 con 0x7f95ac0cb920 2024-01-15T18:05:24.864 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:24.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95840668f0 msgr2=0x7f9584068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95840668f0 0x7f9584068db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f95ac165240 tx=0x7f9594008040 comp rx=0 tx=0).stop 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 msgr2=0x7f95ac163120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 0x7f95ac163120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f959c0ae040 tx=0x7f959c01acc0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 shutdown_connections 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95840668f0 0x7f9584068db0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f95ac0d1950 0x7f95ac06c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f95ac0cf410 0x7f95ac15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 --2- 172.21.15.143:0/2189781303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95ac0cb920 0x7f95ac163120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 >> 172.21.15.143:0/2189781303 conn(0x7f95ac0d0e30 msgr2=0x7f95ac06eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 shutdown_connections 2024-01-15T18:05:24.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:24.860+0000 7f95b1600640 1 -- 172.21.15.143:0/2189781303 wait complete. 2024-01-15T18:05:24.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:25.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c90600640 1 Processor -- start 2024-01-15T18:05:25.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c90600640 1 -- start start 2024-01-15T18:05:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c90600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c88072410 0x7f5c8815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c90600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8815f310 con 0x7f5c88141660 2024-01-15T18:05:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c90600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8815f5f0 con 0x7f5c880ce560 2024-01-15T18:05:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c90600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815fbb0 con 0x7f5c88072410 2024-01-15T18:05:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5c88141660 0x7f5c88141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57470/0 (socket says 172.21.15.143:57470) 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8dc00640 1 -- 172.21.15.143:0/10396215 learned_addr learned my addr 172.21.15.143:0/10396215 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8e600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c88072410 0x7f5c8815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909457108 0 0) 0x7f5c8815f310 con 0x7f5c88141660 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c68003530 con 0x7f5c88141660 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253138586 0 0) 0x7f5c8815f5f0 con 0x7f5c880ce560 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c8815f310 con 0x7f5c880ce560 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2360419001 0 0) 0x7f5c68003530 con 0x7f5c88141660 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c88072410 msgr2=0x7f5c8815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c88072410 0x7f5c8815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 >> v1:172.21.15.204:6789/0 conn(0x7f5c880ce560 legacy=0x7f5c88071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8815f8d0 con 0x7f5c88141660 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f5c78001680 con 0x7f5c88141660 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8e600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c88072410 0x7f5c8815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:25.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 >> v1:172.21.15.143:6789/0 conn(0x7f5c88141660 legacy=0x7f5c88141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c68003b40 0x7f5c68003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c680047d0 0x7f5c68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c68005190 0x7f5c6800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815fbb0 con 0x7f5c68005190 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6800bfb0 con 0x7f5c680047d0 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6800c170 con 0x7f5c68003b40 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8e600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c68005190 0x7f5c6800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8dc00640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c68003b40 0x7f5c68003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.020+0000 7f5c8d200640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c680047d0 0x7f5c68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c8e600640 1 -- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c68003b40 msgr2=0x7f5c68003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c8e600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c68003b40 0x7f5c68003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c8e600640 1 -- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c680047d0 msgr2=0x7f5c68004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c8e600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c680047d0 0x7f5c68004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c8e600640 1 -- 172.21.15.143:0/10396215 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68003530 con 0x7f5c68005190 2024-01-15T18:05:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c8e600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c68005190 0x7f5c6800b990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c8806b430 tx=0x7f5c8001cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc65ea953ca85da server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80003c60 con 0x7f5c68005190 2024-01-15T18:05:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8000f040 con 0x7f5c68005190 2024-01-15T18:05:25.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.024+0000 7f5c87e00640 1 -- 172.21.15.143:0/10396215 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800a5ba0 con 0x7f5c68005190 2024-01-15T18:05:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- 172.21.15.143:0/10396215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c68005190 msgr2=0x7f5c6800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 --2- 172.21.15.143:0/10396215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c68005190 0x7f5c6800b990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c8806b430 tx=0x7f5c8001cd00 comp rx=0 tx=0).stop 2024-01-15T18:05:25.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 -- 172.21.15.143:0/10396215 reap_dead start 2024-01-15T18:05:25.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- 172.21.15.143:0/10396215 shutdown_connections 2024-01-15T18:05:25.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- 172.21.15.143:0/10396215 >> 172.21.15.143:0/10396215 conn(0x7f5c8806c3f0 msgr2=0x7f5c8806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- 172.21.15.143:0/10396215 shutdown_connections 2024-01-15T18:05:25.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- 172.21.15.143:0/10396215 wait complete. 2024-01-15T18:05:25.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 Processor -- start 2024-01-15T18:05:25.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- start start 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c68005190 0x7f5c88137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c881f3dd0 0x7f5c881f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 0x7f5c880b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881381c0 con 0x7f5c881f3120 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881f36d0 con 0x7f5c68005190 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c90600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881f39b0 con 0x7f5c881f3dd0 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 0x7f5c880b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 0x7f5c880b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33250/0 (socket says 172.21.15.143:33250) 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 -- 172.21.15.143:0/1254637131 learned_addr learned my addr 172.21.15.143:0/1254637131 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c68005190 0x7f5c88137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8d200640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c881f3dd0 0x7f5c881f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 -- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c881f3dd0 msgr2=0x7f5c881f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c881f3dd0 0x7f5c881f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 -- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c68005190 msgr2=0x7f5c88137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c68005190 0x7f5c88137c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8e600640 1 -- 172.21.15.143:0/1254637131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c881384e0 con 0x7f5c881f3120 2024-01-15T18:05:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.028+0000 7f5c8dc00640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c68005190 0x7f5c88137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:25.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c8e600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 0x7f5c880b3a40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c800af040 tx=0x7f5c80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800a5d90 con 0x7f5c881f3120 2024-01-15T18:05:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80004030 con 0x7f5c881f3120 2024-01-15T18:05:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800043c0 con 0x7f5c881f3120 2024-01-15T18:05:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c881387c0 con 0x7f5c881f3120 2024-01-15T18:05:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8815e040 con 0x7f5c881f3120 2024-01-15T18:05:25.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88067500 con 0x7f5c881f3120 2024-01-15T18:05:25.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5c80004520 con 0x7f5c881f3120 2024-01-15T18:05:25.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c85600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c60066920 0x7f5c60068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c8011f430 con 0x7f5c881f3120 2024-01-15T18:05:25.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.032+0000 7f5c8dc00640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c60066920 0x7f5c60068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.036+0000 7f5c8dc00640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c60066920 0x7f5c60068de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5c78000c10 tx=0x7f5c78000f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.036+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c800ec680 con 0x7f5c881f3120 2024-01-15T18:05:25.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.184+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c88067500 con 0x7f5c881f3120 2024-01-15T18:05:25.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.184+0000 7f5c85600640 1 -- 172.21.15.143:0/1254637131 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c800efca0 con 0x7f5c881f3120 2024-01-15T18:05:25.193 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:25.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c60066920 msgr2=0x7f5c60068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c60066920 0x7f5c60068de0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5c78000c10 tx=0x7f5c78000f90 comp rx=0 tx=0).stop 2024-01-15T18:05:25.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 msgr2=0x7f5c880b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 0x7f5c880b3a40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5c800af040 tx=0x7f5c80009750 comp rx=0 tx=0).stop 2024-01-15T18:05:25.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 shutdown_connections 2024-01-15T18:05:25.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c60066920 0x7f5c60068de0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c881f3120 0x7f5c880b3a40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c881f3dd0 0x7f5c881f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 --2- 172.21.15.143:0/1254637131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c68005190 0x7f5c88137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 >> 172.21.15.143:0/1254637131 conn(0x7f5c8806c3f0 msgr2=0x7f5c8813ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 shutdown_connections 2024-01-15T18:05:25.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.188+0000 7f5c90600640 1 -- 172.21.15.143:0/1254637131 wait complete. 2024-01-15T18:05:25.216 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 Processor -- start 2024-01-15T18:05:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 -- start start 2024-01-15T18:05:25.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937006bfb0 0x7f937006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93700b0310 0x7f93700cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93700cbff0 con 0x7f93700cc9c0 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93700d0c50 con 0x7f93700b0310 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f9376800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937006c8d0 con 0x7f937006bfb0 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93700b0310 0x7f93700cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93700b0310 0x7f93700cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33252/0 (socket says 172.21.15.143:33252) 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 -- 172.21.15.143:0/281559859 learned_addr learned my addr 172.21.15.143:0/281559859 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936f400640 1 --2- 172.21.15.143:0/281559859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937006bfb0 0x7f937006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 -- 172.21.15.143:0/281559859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937006bfb0 msgr2=0x7f937006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 --2- 172.21.15.143:0/281559859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937006bfb0 0x7f937006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 -- 172.21.15.143:0/281559859 >> v1:172.21.15.204:6789/0 conn(0x7f93700cc9c0 legacy=0x7f93700cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.348+0000 7f936ea00640 1 -- 172.21.15.143:0/281559859 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700cb6c0 con 0x7f93700b0310 2024-01-15T18:05:25.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f936ea00640 1 --2- 172.21.15.143:0/281559859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93700b0310 0x7f93700cc480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9358009870 tx=0x7f935801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634199c242a66622 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f936e000640 1 -- 172.21.15.143:0/281559859 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580ab600 con 0x7f93700b0310 2024-01-15T18:05:25.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f936e000640 1 -- 172.21.15.143:0/281559859 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580ab760 con 0x7f93700b0310 2024-01-15T18:05:25.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f936e000640 1 -- 172.21.15.143:0/281559859 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580abaf0 con 0x7f93700b0310 2024-01-15T18:05:25.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 -- 172.21.15.143:0/281559859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93700b0310 msgr2=0x7f93700cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 --2- 172.21.15.143:0/281559859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93700b0310 0x7f93700cc480 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9358009870 tx=0x7f935801c440 comp rx=0 tx=0).stop 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 -- 172.21.15.143:0/281559859 shutdown_connections 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 --2- 172.21.15.143:0/281559859 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93700b0310 0x7f93700cc480 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 --2- 172.21.15.143:0/281559859 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f937006bfb0 0x7f937006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 -- 172.21.15.143:0/281559859 >> 172.21.15.143:0/281559859 conn(0x7f93700d0ed0 msgr2=0x7f93700d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 -- 172.21.15.143:0/281559859 shutdown_connections 2024-01-15T18:05:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.352+0000 7f9376800640 1 -- 172.21.15.143:0/281559859 wait complete. 2024-01-15T18:05:25.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 Processor -- start 2024-01-15T18:05:25.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- start start 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 0x7f9370152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93700b0310 0x7f9370158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93700cc9c0 0x7f93700c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9370152b40 con 0x7f937006bfb0 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9370152e20 con 0x7f93700cc9c0 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9370157e90 con 0x7f93700b0310 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 0x7f9370152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 0x7f9370152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33264/0 (socket says 172.21.15.143:33264) 2024-01-15T18:05:25.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 -- 172.21.15.143:0/545635503 learned_addr learned my addr 172.21.15.143:0/545635503 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936ea00640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93700b0310 0x7f9370158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936fe00640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93700cc9c0 0x7f93700c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 -- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93700b0310 msgr2=0x7f9370158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93700b0310 0x7f9370158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 -- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93700cc9c0 msgr2=0x7f93700c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93700cc9c0 0x7f93700c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 -- 172.21.15.143:0/545635503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93580094d0 con 0x7f937006bfb0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936fe00640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93700cc9c0 0x7f93700c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f936f400640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 0x7f9370152510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f936000d5f0 tx=0x7f936000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600125f0 con 0x7f937006bfb0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9360012750 con 0x7f937006bfb0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9370158170 con 0x7f937006bfb0 2024-01-15T18:05:25.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360012ae0 con 0x7f937006bfb0 2024-01-15T18:05:25.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93700c5100 con 0x7f937006bfb0 2024-01-15T18:05:25.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.356+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9370067500 con 0x7f937006bfb0 2024-01-15T18:05:25.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.360+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f9360012c40 con 0x7f937006bfb0 2024-01-15T18:05:25.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.360+0000 7f9367400640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93480668f0 0x7f9348068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.360+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9360084910 con 0x7f937006bfb0 2024-01-15T18:05:25.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.360+0000 7f936ea00640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93480668f0 0x7f9348068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.360+0000 7f936ea00640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93480668f0 0x7f9348068db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93580af000 tx=0x7f93580b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.364+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9360051ab0 con 0x7f937006bfb0 2024-01-15T18:05:25.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.512+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93701530b0 con 0x7f937006bfb0 2024-01-15T18:05:25.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9367400640 1 -- 172.21.15.143:0/545635503 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f9360007380 con 0x7f937006bfb0 2024-01-15T18:05:25.522 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:25.523 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93480668f0 msgr2=0x7f9348068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93480668f0 0x7f9348068db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f93580af000 tx=0x7f93580b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 msgr2=0x7f9370152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 0x7f9370152510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f936000d5f0 tx=0x7f936000dab0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 shutdown_connections 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93480668f0 0x7f9348068db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93700cc9c0 0x7f93700c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93700b0310 0x7f9370158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 --2- 172.21.15.143:0/545635503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f937006bfb0 0x7f9370152510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 >> 172.21.15.143:0/545635503 conn(0x7f93700d0ed0 msgr2=0x7f93700ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 shutdown_connections 2024-01-15T18:05:25.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.516+0000 7f9376800640 1 -- 172.21.15.143:0/545635503 wait complete. 2024-01-15T18:05:25.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:25.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 Processor -- start 2024-01-15T18:05:25.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 -- start start 2024-01-15T18:05:25.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd00415d3f0 0x7fd0040cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0040d0050 0x7fd00406bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0040cb480 con 0x7fd0040b0490 2024-01-15T18:05:25.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040cb6d0 con 0x7fd0040d0050 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd00ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040cb9b0 con 0x7fd00415d3f0 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0040d0050 0x7fd00406bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0040d0050 0x7fd00406bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33268/0 (socket says 172.21.15.143:33268) 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 -- 172.21.15.143:0/1340296703 learned_addr learned my addr 172.21.15.143:0/1340296703 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd009000640 1 --2- 172.21.15.143:0/1340296703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd00415d3f0 0x7fd0040cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 -- 172.21.15.143:0/1340296703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd00415d3f0 msgr2=0x7fd0040cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 --2- 172.21.15.143:0/1340296703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd00415d3f0 0x7fd0040cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 -- 172.21.15.143:0/1340296703 >> v1:172.21.15.143:6789/0 conn(0x7fd0040b0490 legacy=0x7fd0040b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003e00640 1 -- 172.21.15.143:0/1340296703 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040cbc90 con 0x7fd0040d0050 2024-01-15T18:05:25.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.680+0000 7fd003400640 1 -- 172.21.15.143:0/1340296703 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173465327 0 0) 0x7fd0040cb480 con 0x7fd0040b0490 2024-01-15T18:05:25.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd003e00640 1 --2- 172.21.15.143:0/1340296703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0040d0050 0x7fd00406bf20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009870 tx=0x7fcfec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161f1a66cf80af1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd003400640 1 -- 172.21.15.143:0/1340296703 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0ab620 con 0x7fd0040d0050 2024-01-15T18:05:25.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd003400640 1 -- 172.21.15.143:0/1340296703 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec0ab780 con 0x7fd0040d0050 2024-01-15T18:05:25.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd003400640 1 -- 172.21.15.143:0/1340296703 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0abb10 con 0x7fd0040d0050 2024-01-15T18:05:25.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 -- 172.21.15.143:0/1340296703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0040d0050 msgr2=0x7fd00406bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 --2- 172.21.15.143:0/1340296703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0040d0050 0x7fd00406bf20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009870 tx=0x7fcfec01c510 comp rx=0 tx=0).stop 2024-01-15T18:05:25.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 -- 172.21.15.143:0/1340296703 shutdown_connections 2024-01-15T18:05:25.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 --2- 172.21.15.143:0/1340296703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0040d0050 0x7fd00406bf20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 --2- 172.21.15.143:0/1340296703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd00415d3f0 0x7fd0040cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 -- 172.21.15.143:0/1340296703 >> 172.21.15.143:0/1340296703 conn(0x7fd0040d0e30 msgr2=0x7fd0040d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 -- 172.21.15.143:0/1340296703 shutdown_connections 2024-01-15T18:05:25.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.684+0000 7fd00ba00640 1 -- 172.21.15.143:0/1340296703 wait complete. 2024-01-15T18:05:25.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 Processor -- start 2024-01-15T18:05:25.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 -- start start 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 0x7fd0041520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0040d0050 0x7fd00415fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00415d3f0 0x7fd0040c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040b52a0 con 0x7fd00415d3f0 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040b5580 con 0x7fd0040d0050 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040b5860 con 0x7fd0040b0490 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00415d3f0 0x7fd0040c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 0x7fd0041520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00415d3f0 0x7fd0040c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33274/0 (socket says 172.21.15.143:33274) 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 0x7fd0041520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41922/0 (socket says 172.21.15.143:41922) 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009a00640 1 -- 172.21.15.143:0/2806348508 learned_addr learned my addr 172.21.15.143:0/2806348508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd003e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0040d0050 0x7fd00415fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 -- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0040d0050 msgr2=0x7fd00415fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0040d0050 0x7fd00415fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 -- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00415d3f0 msgr2=0x7fd0040c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00415d3f0 0x7fd0040c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 -- 172.21.15.143:0/2806348508 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0094d0 con 0x7fd0040b0490 2024-01-15T18:05:25.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd003e00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0040d0050 0x7fd00415fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:25.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd009000640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 0x7fd0041520a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcff400d5d0 tx=0x7fcff400da90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff40125f0 con 0x7fd0040b0490 2024-01-15T18:05:25.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff4012750 con 0x7fd0040b0490 2024-01-15T18:05:25.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0040b5b40 con 0x7fd0040b0490 2024-01-15T18:05:25.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4012ae0 con 0x7fd0040b0490 2024-01-15T18:05:25.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.688+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00415ca80 con 0x7fd0040b0490 2024-01-15T18:05:25.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.692+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00415cc70 con 0x7fd0040b0490 2024-01-15T18:05:25.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.692+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fcff4012c40 con 0x7fd0040b0490 2024-01-15T18:05:25.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.692+0000 7fd000c00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcfdc0668f0 0x7fcfdc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.692+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcff4084770 con 0x7fd0040b0490 2024-01-15T18:05:25.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.692+0000 7fd003e00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcfdc0668f0 0x7fcfdc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.692+0000 7fd003e00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcfdc0668f0 0x7fcfdc068db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0af000 tx=0x7fcfec0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.696+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcff40519d0 con 0x7fd0040b0490 2024-01-15T18:05:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.848+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd004067500 con 0x7fd0040b0490 2024-01-15T18:05:25.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.848+0000 7fd000c00640 1 -- 172.21.15.143:0/2806348508 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcff4054ff0 con 0x7fd0040b0490 2024-01-15T18:05:25.856 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcfdc0668f0 msgr2=0x7fcfdc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcfdc0668f0 0x7fcfdc068db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0af000 tx=0x7fcfec0b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 msgr2=0x7fd0041520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 0x7fd0041520a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcff400d5d0 tx=0x7fcff400da90 comp rx=0 tx=0).stop 2024-01-15T18:05:25.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 shutdown_connections 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcfdc0668f0 0x7fcfdc068db0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd00415d3f0 0x7fd0040c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0040d0050 0x7fd00415fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 --2- 172.21.15.143:0/2806348508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0040b0490 0x7fd0041520a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 >> 172.21.15.143:0/2806348508 conn(0x7fd0040d0e30 msgr2=0x7fd004158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 shutdown_connections 2024-01-15T18:05:25.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:25.852+0000 7fd00ba00640 1 -- 172.21.15.143:0/2806348508 wait complete. 2024-01-15T18:05:25.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:26.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.008+0000 7f90b2a00640 1 Processor -- start 2024-01-15T18:05:26.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.008+0000 7f90b2a00640 1 -- start start 2024-01-15T18:05:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90b2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90ac15d130 0x7f90ac0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90b2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac15d510 0x7f90ac06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90b2a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ac160eb0 con 0x7f90ac06a490 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90b2a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ac161190 con 0x7f90ac15d130 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90b2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac161470 con 0x7f90ac15d510 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90ac15d130 0x7f90ac0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90ac15d130 0x7f90ac0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33280/0 (socket says 172.21.15.143:33280) 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 -- 172.21.15.143:0/3010627957 learned_addr learned my addr 172.21.15.143:0/3010627957 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90ab400640 1 --2- 172.21.15.143:0/3010627957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac15d510 0x7f90ac06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 -- 172.21.15.143:0/3010627957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac15d510 msgr2=0x7f90ac06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 --2- 172.21.15.143:0/3010627957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac15d510 0x7f90ac06e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 -- 172.21.15.143:0/3010627957 >> v1:172.21.15.143:6789/0 conn(0x7f90ac06a490 legacy=0x7f90ac071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 -- 172.21.15.143:0/3010627957 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac161750 con 0x7f90ac15d130 2024-01-15T18:05:26.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90abe00640 1 --2- 172.21.15.143:0/3010627957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac0aeed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f909c009870 tx=0x7f909c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9898c08bc6b1b48a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.012+0000 7f90aaa00640 1 -- 172.21.15.143:0/3010627957 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0a9620 con 0x7f90ac15d130 2024-01-15T18:05:26.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90aaa00640 1 -- 172.21.15.143:0/3010627957 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c0a9780 con 0x7f90ac15d130 2024-01-15T18:05:26.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90aaa00640 1 -- 172.21.15.143:0/3010627957 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0a9b10 con 0x7f90ac15d130 2024-01-15T18:05:26.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- 172.21.15.143:0/3010627957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 msgr2=0x7f90ac0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 --2- 172.21.15.143:0/3010627957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac0aeed0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f909c009870 tx=0x7f909c01c540 comp rx=0 tx=0).stop 2024-01-15T18:05:26.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- 172.21.15.143:0/3010627957 shutdown_connections 2024-01-15T18:05:26.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 --2- 172.21.15.143:0/3010627957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac15d510 0x7f90ac06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 --2- 172.21.15.143:0/3010627957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac0aeed0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- 172.21.15.143:0/3010627957 >> 172.21.15.143:0/3010627957 conn(0x7f90ac0b1b90 msgr2=0x7f90ac0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- 172.21.15.143:0/3010627957 shutdown_connections 2024-01-15T18:05:26.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- 172.21.15.143:0/3010627957 wait complete. 2024-01-15T18:05:26.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 Processor -- start 2024-01-15T18:05:26.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- start start 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac06a490 0x7f90ac147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac15d510 0x7f90ac146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac1472a0 con 0x7f90ac15d130 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac153d70 con 0x7f90ac15d510 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90b2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac148ca0 con 0x7f90ac06a490 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.016+0000 7f90ab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33284/0 (socket says 172.21.15.143:33284) 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 -- 172.21.15.143:0/4085171937 learned_addr learned my addr 172.21.15.143:0/4085171937 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90abe00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac06a490 0x7f90ac147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90b0a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac15d510 0x7f90ac146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 -- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac06a490 msgr2=0x7f90ac147680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac06a490 0x7f90ac147680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 -- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac15d510 msgr2=0x7f90ac146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac15d510 0x7f90ac146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 -- 172.21.15.143:0/4085171937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0094d0 con 0x7f90ac15d130 2024-01-15T18:05:26.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90ab400640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac146820 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f909400d620 tx=0x7f909400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400b620 con 0x7f90ac15d130 2024-01-15T18:05:26.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400b780 con 0x7f90ac15d130 2024-01-15T18:05:26.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac1499c0 con 0x7f90ac15d130 2024-01-15T18:05:26.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400bb30 con 0x7f90ac15d130 2024-01-15T18:05:26.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac153a00 con 0x7f90ac15d130 2024-01-15T18:05:26.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac067500 con 0x7f90ac15d130 2024-01-15T18:05:26.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f909400bc90 con 0x7f90ac15d130 2024-01-15T18:05:26.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90a3e00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90840668f0 0x7f9084068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.020+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90940847c0 con 0x7f90ac15d130 2024-01-15T18:05:26.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.024+0000 7f90abe00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90840668f0 0x7f9084068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.024+0000 7f90abe00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90840668f0 0x7f9084068db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f909c0ae000 tx=0x7f909c0b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.024+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9094051a10 con 0x7f90ac15d130 2024-01-15T18:05:26.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.172+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90ac06ada0 con 0x7f90ac15d130 2024-01-15T18:05:26.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90a3e00640 1 -- 172.21.15.143:0/4085171937 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9094007380 con 0x7f90ac15d130 2024-01-15T18:05:26.181 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90840668f0 msgr2=0x7f9084068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90840668f0 0x7f9084068db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f909c0ae000 tx=0x7f909c0b3040 comp rx=0 tx=0).stop 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 msgr2=0x7f90ac146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac146820 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f909400d620 tx=0x7f909400dae0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 shutdown_connections 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90840668f0 0x7f9084068db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90ac15d510 0x7f90ac146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac15d130 0x7f90ac146820 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 --2- 172.21.15.143:0/4085171937 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90ac06a490 0x7f90ac147680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 >> 172.21.15.143:0/4085171937 conn(0x7f90ac0b1b90 msgr2=0x7f90ac152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 shutdown_connections 2024-01-15T18:05:26.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.176+0000 7f90b2a00640 1 -- 172.21.15.143:0/4085171937 wait complete. 2024-01-15T18:05:26.203 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:26.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 Processor -- start 2024-01-15T18:05:26.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 -- start start 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2854143780 0x7f28540cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f28540ce520 0x7f2854071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28540ceb50 con 0x7f28540720f0 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285415c110 con 0x7f28540ce520 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f285a400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2854072530 con 0x7f2854143780 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f28540ce520 0x7f2854071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f28540ce520 0x7f2854071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33292/0 (socket says 172.21.15.143:33292) 2024-01-15T18:05:26.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 -- 172.21.15.143:0/4193372428 learned_addr learned my addr 172.21.15.143:0/4193372428 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2853400640 1 --2- 172.21.15.143:0/4193372428 >> v2:172.21.15.204:3300/0 conn(0x7f2854143780 0x7f28540cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 -- 172.21.15.143:0/4193372428 >> v2:172.21.15.204:3300/0 conn(0x7f2854143780 msgr2=0x7f28540cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 --2- 172.21.15.143:0/4193372428 >> v2:172.21.15.204:3300/0 conn(0x7f2854143780 0x7f28540cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 -- 172.21.15.143:0/4193372428 >> v1:172.21.15.204:6789/0 conn(0x7f28540720f0 legacy=0x7f285415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:26.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.336+0000 7f2852a00640 1 -- 172.21.15.143:0/4193372428 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285415b7e0 con 0x7f28540ce520 2024-01-15T18:05:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f2852a00640 1 --2- 172.21.15.143:0/4193372428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854071bb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f285406be30 tx=0x7f283c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f8af2493850162 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f2852000640 1 -- 172.21.15.143:0/4193372428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0ac630 con 0x7f28540ce520 2024-01-15T18:05:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f2852000640 1 -- 172.21.15.143:0/4193372428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0ac790 con 0x7f28540ce520 2024-01-15T18:05:26.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f2852000640 1 -- 172.21.15.143:0/4193372428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0acb20 con 0x7f28540ce520 2024-01-15T18:05:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 -- 172.21.15.143:0/4193372428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 msgr2=0x7f2854071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 --2- 172.21.15.143:0/4193372428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854071bb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f285406be30 tx=0x7f283c01c4c0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 -- 172.21.15.143:0/4193372428 shutdown_connections 2024-01-15T18:05:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 --2- 172.21.15.143:0/4193372428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854071bb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 --2- 172.21.15.143:0/4193372428 >> v2:172.21.15.204:3300/0 conn(0x7f2854143780 0x7f28540cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 -- 172.21.15.143:0/4193372428 >> 172.21.15.143:0/4193372428 conn(0x7f285406c3f0 msgr2=0x7f285406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 -- 172.21.15.143:0/4193372428 shutdown_connections 2024-01-15T18:05:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.340+0000 7f285a400640 1 -- 172.21.15.143:0/4193372428 wait complete. 2024-01-15T18:05:26.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 Processor -- start 2024-01-15T18:05:26.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 -- start start 2024-01-15T18:05:26.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28540720f0 0x7f2854156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2854143780 0x7f2854159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540ad660 con 0x7f28540ce520 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28540ad940 con 0x7f2854143780 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28540adc20 con 0x7f28540720f0 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33306/0 (socket says 172.21.15.143:33306) 2024-01-15T18:05:26.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 -- 172.21.15.143:0/3120344474 learned_addr learned my addr 172.21.15.143:0/3120344474 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2853e00640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2854143780 0x7f2854159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2853400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28540720f0 0x7f2854156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 -- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28540720f0 msgr2=0x7f2854156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28540720f0 0x7f2854156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 -- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2854143780 msgr2=0x7f2854159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2854143780 0x7f2854159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 -- 172.21.15.143:0/3120344474 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c0094d0 con 0x7f28540ce520 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2853400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28540720f0 0x7f2854156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:26.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f2852a00640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854158bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f283c01a0f0 tx=0x7f283c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b5640 con 0x7f28540ce520 2024-01-15T18:05:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0b57a0 con 0x7f28540ce520 2024-01-15T18:05:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28540adf00 con 0x7f28540ce520 2024-01-15T18:05:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b5b10 con 0x7f28540ce520 2024-01-15T18:05:26.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.344+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28540ad2f0 con 0x7f28540ce520 2024-01-15T18:05:26.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.348+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28540add60 con 0x7f28540ce520 2024-01-15T18:05:26.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.348+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f283c0b3070 con 0x7f28540ce520 2024-01-15T18:05:26.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.348+0000 7f284b400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f282c066920 0x7f282c068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.348+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f283c11e1e0 con 0x7f28540ce520 2024-01-15T18:05:26.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.348+0000 7f2853400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f282c066920 0x7f282c068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.348+0000 7f2853400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f282c066920 0x7f282c068de0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28540b2190 tx=0x7f2844008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.352+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f28540add60 con 0x7f28540ce520 2024-01-15T18:05:26.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.504+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28540b2200 con 0x7f28540ce520 2024-01-15T18:05:26.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.504+0000 7f284b400640 1 -- 172.21.15.143:0/3120344474 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28540b2200 con 0x7f28540ce520 2024-01-15T18:05:26.513 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:26.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f282c066920 msgr2=0x7f282c068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f282c066920 0x7f282c068de0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f28540b2190 tx=0x7f2844008040 comp rx=0 tx=0).stop 2024-01-15T18:05:26.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 msgr2=0x7f2854158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854158bf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f283c01a0f0 tx=0x7f283c019260 comp rx=0 tx=0).stop 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 shutdown_connections 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f282c066920 0x7f282c068de0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2854143780 0x7f2854159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28540ce520 0x7f2854158bf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 --2- 172.21.15.143:0/3120344474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f28540720f0 0x7f2854156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 >> 172.21.15.143:0/3120344474 conn(0x7f285406c3f0 msgr2=0x7f28540aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 shutdown_connections 2024-01-15T18:05:26.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.508+0000 7f285a400640 1 -- 172.21.15.143:0/3120344474 wait complete. 2024-01-15T18:05:26.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:26.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 Processor -- start 2024-01-15T18:05:26.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 -- start start 2024-01-15T18:05:26.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0c406c540 0x7ff0c40cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0c40cf150 0x7ff0c40cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c40d02d0 con 0x7ff0c406b7b0 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c40cf6c0 con 0x7ff0c406c540 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c8e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c40cf9a0 con 0x7ff0c40cf150 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c1e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0c406c540 0x7ff0c40cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c2800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff0c406b7b0 0x7ff0c415ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57498/0 (socket says 172.21.15.143:57498) 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c1e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff0c406c540 0x7ff0c40cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33312/0 (socket says 172.21.15.143:33312) 2024-01-15T18:05:26.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.668+0000 7ff0c2800640 1 -- 172.21.15.143:0/2245636972 learned_addr learned my addr 172.21.15.143:0/2245636972 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:26.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c1400640 1 --2- 172.21.15.143:0/2245636972 >> v2:172.21.15.204:3300/0 conn(0x7ff0c40cf150 0x7ff0c40cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2245636972 >> v2:172.21.15.204:3300/0 conn(0x7ff0c40cf150 msgr2=0x7ff0c40cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:26.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c1e00640 1 --2- 172.21.15.143:0/2245636972 >> v2:172.21.15.204:3300/0 conn(0x7ff0c40cf150 0x7ff0c40cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2245636972 >> v1:172.21.15.143:6789/0 conn(0x7ff0c406b7b0 legacy=0x7ff0c415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2245636972 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40cfc80 con 0x7ff0c406c540 2024-01-15T18:05:26.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c0a00640 1 -- 172.21.15.143:0/2245636972 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930318144 0 0) 0x7ff0c40d02d0 con 0x7ff0c406b7b0 2024-01-15T18:05:26.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c1e00640 1 --2- 172.21.15.143:0/2245636972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406c540 0x7ff0c40cd7e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4072810 tx=0x7ff0b401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb53331f3938b2f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c0a00640 1 -- 172.21.15.143:0/2245636972 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40ab600 con 0x7ff0c406c540 2024-01-15T18:05:26.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c0a00640 1 -- 172.21.15.143:0/2245636972 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40a7050 con 0x7ff0c406c540 2024-01-15T18:05:26.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.672+0000 7ff0c0a00640 1 -- 172.21.15.143:0/2245636972 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40ab980 con 0x7ff0c406c540 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2245636972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406c540 msgr2=0x7ff0c40cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2245636972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406c540 0x7ff0c40cd7e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4072810 tx=0x7ff0b401c310 comp rx=0 tx=0).stop 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2245636972 shutdown_connections 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2245636972 >> v2:172.21.15.204:3300/0 conn(0x7ff0c40cf150 0x7ff0c40cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2245636972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406c540 0x7ff0c40cd7e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2245636972 >> 172.21.15.143:0/2245636972 conn(0x7ff0c40d0e20 msgr2=0x7ff0c40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2245636972 shutdown_connections 2024-01-15T18:05:26.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2245636972 wait complete. 2024-01-15T18:05:26.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 Processor -- start 2024-01-15T18:05:26.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- start start 2024-01-15T18:05:26.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 0x7ff0c415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0c406c540 0x7ff0c41f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0c40cf150 0x7ff0c41f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c4159640 con 0x7ff0c406b7b0 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40c63b0 con 0x7ff0c40cf150 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c8e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40c6690 con 0x7ff0c406c540 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 0x7ff0c415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 0x7ff0c415d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33316/0 (socket says 172.21.15.143:33316) 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2587901603 learned_addr learned my addr 172.21.15.143:0/2587901603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c2800640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0c40cf150 0x7ff0c41f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.676+0000 7ff0c1400640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0c406c540 0x7ff0c41f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0c406c540 msgr2=0x7ff0c41f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c1e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0c406c540 0x7ff0c41f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0c40cf150 msgr2=0x7ff0c41f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c1e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0c40cf150 0x7ff0c41f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c1e00640 1 -- 172.21.15.143:0/2587901603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4009420 con 0x7ff0c406b7b0 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c1e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 0x7ff0c415d890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40ae040 tx=0x7ff0b401ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40b7450 con 0x7ff0c406b7b0 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40b75b0 con 0x7ff0c406b7b0 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40b7960 con 0x7ff0c406b7b0 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c40c6970 con 0x7ff0c406b7b0 2024-01-15T18:05:26.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c40c6f30 con 0x7ff0c406b7b0 2024-01-15T18:05:26.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c40c8010 con 0x7ff0c406b7b0 2024-01-15T18:05:26.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff0b40b7ac0 con 0x7ff0c406b7b0 2024-01-15T18:05:26.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.680+0000 7ff0ba000640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff09c0668f0 0x7ff09c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.684+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0b411ec70 con 0x7ff0c406b7b0 2024-01-15T18:05:26.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.684+0000 7ff0c1400640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff09c0668f0 0x7ff09c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.684+0000 7ff0c1400640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff09c0668f0 0x7ff09c068db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ca160 tx=0x7ff0ac009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.688+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0c40c8010 con 0x7ff0c406b7b0 2024-01-15T18:05:26.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.836+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0c4067500 con 0x7ff0c406b7b0 2024-01-15T18:05:26.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0ba000640 1 -- 172.21.15.143:0/2587901603 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ff0b40bb090 con 0x7ff0c406b7b0 2024-01-15T18:05:26.846 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:26.846 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff09c0668f0 msgr2=0x7ff09c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff09c0668f0 0x7ff09c068db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ca160 tx=0x7ff0ac009190 comp rx=0 tx=0).stop 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 msgr2=0x7ff0c415d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 0x7ff0c415d890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40ae040 tx=0x7ff0b401ab50 comp rx=0 tx=0).stop 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 shutdown_connections 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff09c0668f0 0x7ff09c068db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0c40cf150 0x7ff0c41f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0c406c540 0x7ff0c41f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 --2- 172.21.15.143:0/2587901603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0c406b7b0 0x7ff0c415d890 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 >> 172.21.15.143:0/2587901603 conn(0x7ff0c40d0e20 msgr2=0x7ff0c40aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 shutdown_connections 2024-01-15T18:05:26.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:26.840+0000 7ff0c8e00640 1 -- 172.21.15.143:0/2587901603 wait complete. 2024-01-15T18:05:26.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:26.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:27.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.000+0000 7fd88da00640 1 Processor -- start 2024-01-15T18:05:27.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.000+0000 7fd88da00640 1 -- start start 2024-01-15T18:05:27.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd88da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd888157300 0x7fd888159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd88da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd8880b21d0 0x7fd8880b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd88da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b48f0 0x7fd888070210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd88da00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8880b35a0 con 0x7fd888157300 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd88da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8880b4d60 con 0x7fd8880b21d0 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd88da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880b2a20 con 0x7fd8880b48f0 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd888157300 0x7fd888159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd888157300 0x7fd888159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33328/0 (socket says 172.21.15.143:33328) 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 -- 172.21.15.143:0/2164664659 learned_addr learned my addr 172.21.15.143:0/2164664659 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd887400640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b48f0 0x7fd888070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 -- 172.21.15.143:0/2164664659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b48f0 msgr2=0x7fd888070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b48f0 0x7fd888070210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 -- 172.21.15.143:0/2164664659 >> v2:172.21.15.204:3300/0 conn(0x7fd8880b21d0 msgr2=0x7fd8880b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886000640 1 --2- 172.21.15.143:0/2164664659 >> v2:172.21.15.204:3300/0 conn(0x7fd8880b21d0 0x7fd8880b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 --2- 172.21.15.143:0/2164664659 >> v2:172.21.15.204:3300/0 conn(0x7fd8880b21d0 0x7fd8880b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 -- 172.21.15.143:0/2164664659 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880b2d00 con 0x7fd888157300 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd887400640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b48f0 0x7fd888070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:27.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886000640 1 --2- 172.21.15.143:0/2164664659 >> v2:172.21.15.204:3300/0 conn(0x7fd8880b21d0 0x7fd8880b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:27.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd886a00640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd888159b70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd8880b4fe0 tx=0x7fd87801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c658713d97869038 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.004+0000 7fd885600640 1 -- 172.21.15.143:0/2164664659 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8780aa5f0 con 0x7fd888157300 2024-01-15T18:05:27.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd885600640 1 -- 172.21.15.143:0/2164664659 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780aa750 con 0x7fd888157300 2024-01-15T18:05:27.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd885600640 1 -- 172.21.15.143:0/2164664659 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8780aaae0 con 0x7fd888157300 2024-01-15T18:05:27.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- 172.21.15.143:0/2164664659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 msgr2=0x7fd888159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd888159b70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd8880b4fe0 tx=0x7fd87801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- 172.21.15.143:0/2164664659 shutdown_connections 2024-01-15T18:05:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b48f0 0x7fd888070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- 172.21.15.143:0/2164664659 >> v2:172.21.15.204:3300/0 conn(0x7fd8880b21d0 0x7fd8880b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- 172.21.15.143:0/2164664659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd888159b70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- 172.21.15.143:0/2164664659 >> 172.21.15.143:0/2164664659 conn(0x7fd88806c3f0 msgr2=0x7fd88806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- 172.21.15.143:0/2164664659 shutdown_connections 2024-01-15T18:05:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- 172.21.15.143:0/2164664659 wait complete. 2024-01-15T18:05:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 Processor -- start 2024-01-15T18:05:27.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- start start 2024-01-15T18:05:27.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b21d0 0x7fd888156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8880b48f0 0x7fd8881f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd88814da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.008+0000 7fd88da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881fa110 con 0x7fd888157300 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd88da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881fa3f0 con 0x7fd8880b48f0 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd88da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880aca60 con 0x7fd8880b21d0 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd88814da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd88814da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33344/0 (socket says 172.21.15.143:33344) 2024-01-15T18:05:27.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 -- 172.21.15.143:0/1806207201 learned_addr learned my addr 172.21.15.143:0/1806207201 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd886000640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8880b48f0 0x7fd8881f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd886a00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b21d0 0x7fd888156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 -- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b21d0 msgr2=0x7fd888156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b21d0 0x7fd888156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 -- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8880b48f0 msgr2=0x7fd8881f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8880b48f0 0x7fd8881f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 -- 172.21.15.143:0/1806207201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780094d0 con 0x7fd888157300 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd887400640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd88814da20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd88000ce40 tx=0x7fd88000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000d840 con 0x7fd888157300 2024-01-15T18:05:27.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000d9a0 con 0x7fd888157300 2024-01-15T18:05:27.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000dd50 con 0x7fd888157300 2024-01-15T18:05:27.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8880acd40 con 0x7fd888157300 2024-01-15T18:05:27.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888156b50 con 0x7fd888157300 2024-01-15T18:05:27.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd888156dd0 con 0x7fd888157300 2024-01-15T18:05:27.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.012+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fd88001e3b0 con 0x7fd888157300 2024-01-15T18:05:27.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.016+0000 7fd87ea00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8600668f0 0x7fd860068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.016+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd880084da0 con 0x7fd888157300 2024-01-15T18:05:27.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.016+0000 7fd886a00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8600668f0 0x7fd860068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.016+0000 7fd886a00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8600668f0 0x7fd860068db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd8780ae000 tx=0x7fd8780b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.020+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd888156dd0 con 0x7fd888157300 2024-01-15T18:05:27.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:27.068 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:27.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd888156dd0 con 0x7fd888157300 2024-01-15T18:05:27.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd87ea00640 1 -- 172.21.15.143:0/1806207201 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd888156dd0 con 0x7fd888157300 2024-01-15T18:05:27.178 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:27.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8600668f0 msgr2=0x7fd860068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8600668f0 0x7fd860068db0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd8780ae000 tx=0x7fd8780b5040 comp rx=0 tx=0).stop 2024-01-15T18:05:27.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 msgr2=0x7fd88814da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd88814da20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd88000ce40 tx=0x7fd88000a710 comp rx=0 tx=0).stop 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 shutdown_connections 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8600668f0 0x7fd860068db0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd888157300 0x7fd88814da20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8880b48f0 0x7fd8881f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 --2- 172.21.15.143:0/1806207201 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8880b21d0 0x7fd888156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 >> 172.21.15.143:0/1806207201 conn(0x7fd88806c3f0 msgr2=0x7fd888070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 shutdown_connections 2024-01-15T18:05:27.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.172+0000 7fd88da00640 1 -- 172.21.15.143:0/1806207201 wait complete. 2024-01-15T18:05:27.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:27.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 Processor -- start 2024-01-15T18:05:27.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 -- start start 2024-01-15T18:05:27.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3f40b2320 0x7fe3f40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f40cb670 con 0x7fe3f40d0110 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f40cb950 con 0x7fe3f40b2320 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3fc400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40cbc30 con 0x7fe3f40d1b30 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3f9a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3f40b2320 0x7fe3f40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3f9a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3f40b2320 0x7fe3f40cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37528/0 (socket says 172.21.15.143:37528) 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.332+0000 7fe3f9a00640 1 -- 172.21.15.143:0/2043167319 learned_addr learned my addr 172.21.15.143:0/2043167319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f9000640 1 --2- 172.21.15.143:0/2043167319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f9a00640 1 -- 172.21.15.143:0/2043167319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 msgr2=0x7fe3f406beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f9a00640 1 --2- 172.21.15.143:0/2043167319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f406beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f9a00640 1 -- 172.21.15.143:0/2043167319 >> v1:172.21.15.204:6789/0 conn(0x7fe3f40d0110 legacy=0x7fe3f40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f9a00640 1 -- 172.21.15.143:0/2043167319 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40cbf10 con 0x7fe3f40b2320 2024-01-15T18:05:27.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f9a00640 1 --2- 172.21.15.143:0/2043167319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40b2320 0x7fe3f40cfbd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009870 tx=0x7fe3e401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3942ac9c2b9ab8c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f3e00640 1 -- 172.21.15.143:0/2043167319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40ab680 con 0x7fe3f40b2320 2024-01-15T18:05:27.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f3e00640 1 -- 172.21.15.143:0/2043167319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e40ab7e0 con 0x7fe3f40b2320 2024-01-15T18:05:27.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.336+0000 7fe3f3e00640 1 -- 172.21.15.143:0/2043167319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40abb70 con 0x7fe3f40b2320 2024-01-15T18:05:27.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- 172.21.15.143:0/2043167319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40b2320 msgr2=0x7fe3f40cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 --2- 172.21.15.143:0/2043167319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40b2320 0x7fe3f40cfbd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009870 tx=0x7fe3e401c4a0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- 172.21.15.143:0/2043167319 shutdown_connections 2024-01-15T18:05:27.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 --2- 172.21.15.143:0/2043167319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f406beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 --2- 172.21.15.143:0/2043167319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40b2320 0x7fe3f40cfbd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- 172.21.15.143:0/2043167319 >> 172.21.15.143:0/2043167319 conn(0x7fe3f40d0e80 msgr2=0x7fe3f40d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- 172.21.15.143:0/2043167319 shutdown_connections 2024-01-15T18:05:27.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- 172.21.15.143:0/2043167319 wait complete. 2024-01-15T18:05:27.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 Processor -- start 2024-01-15T18:05:27.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- start start 2024-01-15T18:05:27.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f40b2320 0x7fe3f4153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 0x7fe3f41521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f41526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41589f0 con 0x7fe3f40d0110 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f41572c0 con 0x7fe3f40b2320 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fc400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4158c70 con 0x7fe3f40d1b30 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3f9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 0x7fe3f41521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3f9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 0x7fe3f41521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37542/0 (socket says 172.21.15.143:37542) 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3f9000640 1 -- 172.21.15.143:0/176351325 learned_addr learned my addr 172.21.15.143:0/176351325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3f9a00640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f40b2320 0x7fe3f4153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.340+0000 7fe3fa400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f41526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9000640 1 -- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 msgr2=0x7fe3f41526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9000640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f41526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9000640 1 -- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f40b2320 msgr2=0x7fe3f4153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9000640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f40b2320 0x7fe3f4153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9000640 1 -- 172.21.15.143:0/176351325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40094d0 con 0x7fe3f40d0110 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9000640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 0x7fe3f41521b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00d5f0 tx=0x7fe3dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0125f0 con 0x7fe3f40d0110 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc012750 con 0x7fe3f40d0110 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc012ae0 con 0x7fe3f40d0110 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f4158f50 con 0x7fe3f40d0110 2024-01-15T18:05:27.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f41540c0 con 0x7fe3f40d0110 2024-01-15T18:05:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4159140 con 0x7fe3f40d0110 2024-01-15T18:05:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe3dc012c40 con 0x7fe3f40d0110 2024-01-15T18:05:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f1600640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3cc0668f0 0x7fe3cc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3dc0847a0 con 0x7fe3f40d0110 2024-01-15T18:05:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.344+0000 7fe3f9a00640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3cc0668f0 0x7fe3cc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.348+0000 7fe3f9a00640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3cc0668f0 0x7fe3cc068db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40af000 tx=0x7fe3e40b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.348+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3dc051a80 con 0x7fe3f40d0110 2024-01-15T18:05:27.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.500+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3f4067500 con 0x7fe3f40d0110 2024-01-15T18:05:27.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.500+0000 7fe3f1600640 1 -- 172.21.15.143:0/176351325 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3dc0550a0 con 0x7fe3f40d0110 2024-01-15T18:05:27.507 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:27.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.500+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3cc0668f0 msgr2=0x7fe3cc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.500+0000 7fe3fc400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3cc0668f0 0x7fe3cc068db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40af000 tx=0x7fe3e40b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.500+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 msgr2=0x7fe3f41521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.500+0000 7fe3fc400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 0x7fe3f41521b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00d5f0 tx=0x7fe3dc00dab0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 shutdown_connections 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3cc0668f0 0x7fe3cc068db0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3f40d1b30 0x7fe3f41526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3f40d0110 0x7fe3f41521b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 --2- 172.21.15.143:0/176351325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3f40b2320 0x7fe3f4153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 >> 172.21.15.143:0/176351325 conn(0x7fe3f40d0e80 msgr2=0x7fe3f40cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 shutdown_connections 2024-01-15T18:05:27.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.504+0000 7fe3fc400640 1 -- 172.21.15.143:0/176351325 wait complete. 2024-01-15T18:05:27.529 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:27.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 Processor -- start 2024-01-15T18:05:27.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 -- start start 2024-01-15T18:05:27.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f091c15b260 0x7f091c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f091c0ccaf0 0x7f091c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c0c4d90 con 0x7f091c0d19f0 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c15b640 con 0x7f091c0ccaf0 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0923000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c0cf800 con 0x7f091c15b260 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0921000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f091c0ccaf0 0x7f091c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f091b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f091c0d19f0 0x7f091c0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51332/0 (socket says 172.21.15.143:51332) 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f091be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f091c15b260 0x7f091c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f091b400640 1 -- 172.21.15.143:0/1055543457 learned_addr learned my addr 172.21.15.143:0/1055543457 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0921000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f091c0ccaf0 0x7f091c0cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37544/0 (socket says 172.21.15.143:37544) 2024-01-15T18:05:27.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0921000640 1 -- 172.21.15.143:0/1055543457 >> v2:172.21.15.204:3300/0 conn(0x7f091c15b260 msgr2=0x7f091c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0921000640 1 --2- 172.21.15.143:0/1055543457 >> v2:172.21.15.204:3300/0 conn(0x7f091c15b260 0x7f091c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0921000640 1 -- 172.21.15.143:0/1055543457 >> v1:172.21.15.143:6789/0 conn(0x7f091c0d19f0 legacy=0x7f091c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.664+0000 7f0921000640 1 -- 172.21.15.143:0/1055543457 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0cfae0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0921000640 1 --2- 172.21.15.143:0/1055543457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c0cdd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f091c0681e0 tx=0x7f091401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca60b187fb093df server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f091aa00640 1 -- 172.21.15.143:0/1055543457 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140a58c0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f091aa00640 1 -- 172.21.15.143:0/1055543457 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140a5a20 con 0x7f091c0ccaf0 2024-01-15T18:05:27.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f091aa00640 1 -- 172.21.15.143:0/1055543457 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140a5db0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 -- 172.21.15.143:0/1055543457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 msgr2=0x7f091c0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 --2- 172.21.15.143:0/1055543457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c0cdd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f091c0681e0 tx=0x7f091401c910 comp rx=0 tx=0).stop 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 -- 172.21.15.143:0/1055543457 shutdown_connections 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 --2- 172.21.15.143:0/1055543457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c0cdd50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 --2- 172.21.15.143:0/1055543457 >> v2:172.21.15.204:3300/0 conn(0x7f091c15b260 0x7f091c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 -- 172.21.15.143:0/1055543457 >> 172.21.15.143:0/1055543457 conn(0x7f091c0d0ed0 msgr2=0x7f091c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 -- 172.21.15.143:0/1055543457 shutdown_connections 2024-01-15T18:05:27.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.668+0000 7f0923000640 1 -- 172.21.15.143:0/1055543457 wait complete. 2024-01-15T18:05:27.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 Processor -- start 2024-01-15T18:05:27.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- start start 2024-01-15T18:05:27.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c0d19f0 0x7f091c09ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15b260 0x7f091c09c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c09ad10 con 0x7f091c0ccaf0 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c148780 con 0x7f091c15b260 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c09af80 con 0x7f091c0d19f0 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37560/0 (socket says 172.21.15.143:37560) 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 -- 172.21.15.143:0/2352867935 learned_addr learned my addr 172.21.15.143:0/2352867935 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:27.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091b400640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c0d19f0 0x7f091c09ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0921000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15b260 0x7f091c09c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 -- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c0d19f0 msgr2=0x7f091c09ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c0d19f0 0x7f091c09ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 -- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15b260 msgr2=0x7f091c09c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15b260 0x7f091c09c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 -- 172.21.15.143:0/2352867935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009a70 con 0x7f091c0ccaf0 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f091be00640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c148240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e870 tx=0x7f090c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c009510 con 0x7f091c0ccaf0 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00eec0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c09b260 con 0x7f091c0ccaf0 2024-01-15T18:05:27.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00cb40 con 0x7f091c0ccaf0 2024-01-15T18:05:27.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c1557c0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.672+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c067500 con 0x7f091c0ccaf0 2024-01-15T18:05:27.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.676+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f090c00cca0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.676+0000 7f0913e00640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08f40668f0 0x7f08f4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.676+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f090c0845f0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.676+0000 7f091b400640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08f40668f0 0x7f08f4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.676+0000 7f091b400640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08f40668f0 0x7f08f4068db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f091c158fb0 tx=0x7f0904009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.680+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f090c051790 con 0x7f091c0ccaf0 2024-01-15T18:05:27.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.828+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f091c0b6d50 con 0x7f091c0ccaf0 2024-01-15T18:05:27.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.828+0000 7f0913e00640 1 -- 172.21.15.143:0/2352867935 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f090c054db0 con 0x7f091c0ccaf0 2024-01-15T18:05:27.836 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:27.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08f40668f0 msgr2=0x7f08f4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08f40668f0 0x7f08f4068db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f091c158fb0 tx=0x7f0904009040 comp rx=0 tx=0).stop 2024-01-15T18:05:27.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 msgr2=0x7f091c148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c148240 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e870 tx=0x7f090c00ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:27.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 shutdown_connections 2024-01-15T18:05:27.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08f40668f0 0x7f08f4068db0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f091c15b260 0x7f091c09c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f091c0d19f0 0x7f091c09ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 --2- 172.21.15.143:0/2352867935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f091c0ccaf0 0x7f091c148240 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 >> 172.21.15.143:0/2352867935 conn(0x7f091c0d0ed0 msgr2=0x7f091c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 shutdown_connections 2024-01-15T18:05:27.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:27.832+0000 7f0923000640 1 -- 172.21.15.143:0/2352867935 wait complete. 2024-01-15T18:05:27.934 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) Test that when one clone is in progress, one progress bar is printed 2024-01-15T18:05:27.934 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:05:27.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:28.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 Processor -- start 2024-01-15T18:05:28.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 -- start start 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f44c0143780 0x7f44c00cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44c00ce520 0x7f44c0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00720f0 0x7f44c015a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c0072580 con 0x7f44c00ce520 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c00ceb50 con 0x7f44c0143780 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c015c110 con 0x7f44c00720f0 2024-01-15T18:05:28.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44c00ce520 0x7f44c0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44c00ce520 0x7f44c0071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37570/0 (socket says 172.21.15.143:37570) 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 -- 172.21.15.143:0/976900000 learned_addr learned my addr 172.21.15.143:0/976900000 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44c4e00640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00720f0 0x7f44c015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bfe00640 1 --2- 172.21.15.143:0/976900000 >> v2:172.21.15.204:3300/0 conn(0x7f44c0143780 0x7f44c00cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 -- 172.21.15.143:0/976900000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00720f0 msgr2=0x7f44c015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00720f0 0x7f44c015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 -- 172.21.15.143:0/976900000 >> v2:172.21.15.204:3300/0 conn(0x7f44c0143780 msgr2=0x7f44c00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.064+0000 7f44bf400640 1 --2- 172.21.15.143:0/976900000 >> v2:172.21.15.204:3300/0 conn(0x7f44c0143780 0x7f44c00cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44bf400640 1 -- 172.21.15.143:0/976900000 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c015b7e0 con 0x7f44c00ce520 2024-01-15T18:05:28.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44c4e00640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00720f0 0x7f44c015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:28.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44bf400640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00ce520 0x7f44c0071bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44c006be30 tx=0x7f44a801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6362ce2f4a1354c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44bea00640 1 -- 172.21.15.143:0/976900000 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a80ac630 con 0x7f44c00ce520 2024-01-15T18:05:28.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44bea00640 1 -- 172.21.15.143:0/976900000 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a80ac790 con 0x7f44c00ce520 2024-01-15T18:05:28.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44bea00640 1 -- 172.21.15.143:0/976900000 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a80acb20 con 0x7f44c00ce520 2024-01-15T18:05:28.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44c6e00640 1 -- 172.21.15.143:0/976900000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00ce520 msgr2=0x7f44c0071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44c6e00640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00ce520 0x7f44c0071bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f44c006be30 tx=0x7f44a801c500 comp rx=0 tx=0).stop 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.068+0000 7f44c6e00640 1 -- 172.21.15.143:0/976900000 shutdown_connections 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00720f0 0x7f44c015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 --2- 172.21.15.143:0/976900000 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00ce520 0x7f44c0071bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 --2- 172.21.15.143:0/976900000 >> v2:172.21.15.204:3300/0 conn(0x7f44c0143780 0x7f44c00cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- 172.21.15.143:0/976900000 >> 172.21.15.143:0/976900000 conn(0x7f44c006c3f0 msgr2=0x7f44c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- 172.21.15.143:0/976900000 shutdown_connections 2024-01-15T18:05:28.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- 172.21.15.143:0/976900000 wait complete. 2024-01-15T18:05:28.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 Processor -- start 2024-01-15T18:05:28.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- start start 2024-01-15T18:05:28.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 0x7f44c0137770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00ce520 0x7f44c00b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c0143780 0x7f44c00b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c01390a0 con 0x7f44c00720f0 2024-01-15T18:05:28.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0147910 con 0x7f44c0143780 2024-01-15T18:05:28.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0159bb0 con 0x7f44c00ce520 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 0x7f44c0137770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 0x7f44c0137770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37586/0 (socket says 172.21.15.143:37586) 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 -- 172.21.15.143:0/922255690 learned_addr learned my addr 172.21.15.143:0/922255690 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bf400640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00ce520 0x7f44c00b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44c4e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c0143780 0x7f44c00b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 -- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00ce520 msgr2=0x7f44c00b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00ce520 0x7f44c00b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 -- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c0143780 msgr2=0x7f44c00b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c0143780 0x7f44c00b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 -- 172.21.15.143:0/922255690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a80094d0 con 0x7f44c00720f0 2024-01-15T18:05:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.072+0000 7f44bfe00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 0x7f44c0137770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e870 tx=0x7f44b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0009510 con 0x7f44c00720f0 2024-01-15T18:05:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b000eec0 con 0x7f44c00720f0 2024-01-15T18:05:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c00ad4f0 con 0x7f44c00720f0 2024-01-15T18:05:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b000cb40 con 0x7f44c00720f0 2024-01-15T18:05:28.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c01472c0 con 0x7f44c00720f0 2024-01-15T18:05:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c0067500 con 0x7f44c00720f0 2024-01-15T18:05:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.076+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f44b000cca0 con 0x7f44c00720f0 2024-01-15T18:05:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.080+0000 7f44b7e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44980668f0 0x7f4498068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.080+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44b0084540 con 0x7f44c00720f0 2024-01-15T18:05:28.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.080+0000 7f44bf400640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44980668f0 0x7f4498068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.080+0000 7f44bf400640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44980668f0 0x7f4498068db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44a80ae000 tx=0x7f44a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.084+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44b0051790 con 0x7f44c00720f0 2024-01-15T18:05:28.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.232+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44c0071d90 con 0x7f44c00720f0 2024-01-15T18:05:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44b7e00640 1 -- 172.21.15.143:0/922255690 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f44b0054db0 con 0x7f44c00720f0 2024-01-15T18:05:28.242 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:28.242 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:28.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44980668f0 msgr2=0x7f4498068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44980668f0 0x7f4498068db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44a80ae000 tx=0x7f44a8008040 comp rx=0 tx=0).stop 2024-01-15T18:05:28.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 msgr2=0x7f44c0137770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 0x7f44c0137770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44b000e870 tx=0x7f44b000ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:28.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 shutdown_connections 2024-01-15T18:05:28.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44980668f0 0x7f4498068db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c0143780 0x7f44c00b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c00ce520 0x7f44c00b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 --2- 172.21.15.143:0/922255690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c00720f0 0x7f44c0137770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 >> 172.21.15.143:0/922255690 conn(0x7f44c006c3f0 msgr2=0x7f44c00b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 shutdown_connections 2024-01-15T18:05:28.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.236+0000 7f44c6e00640 1 -- 172.21.15.143:0/922255690 wait complete. 2024-01-15T18:05:28.263 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:28.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:28.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 Processor -- start 2024-01-15T18:05:28.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 -- start start 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7e2815e750 0x7f7e280cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e280d1960 0x7f7e280cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280cfd40 0x7f7e280cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e280cf3e0 con 0x7f7e280d1960 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e280ceab0 con 0x7f7e2815e750 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e30400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280ced20 con 0x7f7e280cfd40 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e280d1960 0x7f7e280cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7e280d1960 0x7f7e280cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37602/0 (socket says 172.21.15.143:37602) 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 -- 172.21.15.143:0/1061907380 learned_addr learned my addr 172.21.15.143:0/1061907380 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 -- 172.21.15.143:0/1061907380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280cfd40 msgr2=0x7f7e280cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2e400640 1 --2- 172.21.15.143:0/1061907380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280cfd40 0x7f7e280cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 --2- 172.21.15.143:0/1061907380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280cfd40 0x7f7e280cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 -- 172.21.15.143:0/1061907380 >> v2:172.21.15.204:3300/0 conn(0x7f7e2815e750 msgr2=0x7f7e280cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2da00640 1 --2- 172.21.15.143:0/1061907380 >> v2:172.21.15.204:3300/0 conn(0x7f7e2815e750 0x7f7e280cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 --2- 172.21.15.143:0/1061907380 >> v2:172.21.15.204:3300/0 conn(0x7f7e2815e750 0x7f7e280cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2d000640 1 -- 172.21.15.143:0/1061907380 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280cf000 con 0x7f7e280d1960 2024-01-15T18:05:28.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.396+0000 7f7e2da00640 1 --2- 172.21.15.143:0/1061907380 >> v2:172.21.15.204:3300/0 conn(0x7f7e2815e750 0x7f7e280cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:28.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e2d000640 1 --2- 172.21.15.143:0/1061907380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280d1960 0x7f7e280cf800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e28071020 tx=0x7f7e1001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23da39fb88c11d46 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e27e00640 1 -- 172.21.15.143:0/1061907380 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e100ad640 con 0x7f7e280d1960 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e27e00640 1 -- 172.21.15.143:0/1061907380 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e100ad7a0 con 0x7f7e280d1960 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e27e00640 1 -- 172.21.15.143:0/1061907380 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e100adb30 con 0x7f7e280d1960 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 -- 172.21.15.143:0/1061907380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280d1960 msgr2=0x7f7e280cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 --2- 172.21.15.143:0/1061907380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280d1960 0x7f7e280cf800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e28071020 tx=0x7f7e1001c500 comp rx=0 tx=0).stop 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 -- 172.21.15.143:0/1061907380 shutdown_connections 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 --2- 172.21.15.143:0/1061907380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280cfd40 0x7f7e280cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 --2- 172.21.15.143:0/1061907380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280d1960 0x7f7e280cf800 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 --2- 172.21.15.143:0/1061907380 >> v2:172.21.15.204:3300/0 conn(0x7f7e2815e750 0x7f7e280cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 -- 172.21.15.143:0/1061907380 >> 172.21.15.143:0/1061907380 conn(0x7f7e280d0e40 msgr2=0x7f7e280d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 -- 172.21.15.143:0/1061907380 shutdown_connections 2024-01-15T18:05:28.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.400+0000 7f7e30400640 1 -- 172.21.15.143:0/1061907380 wait complete. 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 Processor -- start 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 -- start start 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cfd40 0x7f7e2815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280d1960 0x7f7e28150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 0x7f7e2814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2815d290 con 0x7f7e2815e750 2024-01-15T18:05:28.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2815c420 con 0x7f7e280cfd40 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2815c610 con 0x7f7e280d1960 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 0x7f7e2814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 0x7f7e2814e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37616/0 (socket says 172.21.15.143:37616) 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 -- 172.21.15.143:0/918498248 learned_addr learned my addr 172.21.15.143:0/918498248 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2da00640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cfd40 0x7f7e2815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2d000640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280d1960 0x7f7e28150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 -- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280d1960 msgr2=0x7f7e28150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280d1960 0x7f7e28150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 -- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cfd40 msgr2=0x7f7e2815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cfd40 0x7f7e2815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 -- 172.21.15.143:0/918498248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100094d0 con 0x7f7e2815e750 2024-01-15T18:05:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e2e400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 0x7f7e2814e710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000ccf0 tx=0x7f7e2000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20003ba0 con 0x7f7e2815e750 2024-01-15T18:05:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2000ae90 con 0x7f7e2815e750 2024-01-15T18:05:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2815c8f0 con 0x7f7e2815e750 2024-01-15T18:05:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20004020 con 0x7f7e2815e750 2024-01-15T18:05:28.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.404+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2815ceb0 con 0x7f7e2815e750 2024-01-15T18:05:28.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.408+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2815d3d0 con 0x7f7e2815e750 2024-01-15T18:05:28.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.408+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f7e20004180 con 0x7f7e2815e750 2024-01-15T18:05:28.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.408+0000 7f7e25600640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e000668f0 0x7f7e00068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.408+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e20012070 con 0x7f7e2815e750 2024-01-15T18:05:28.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.408+0000 7f7e2da00640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e000668f0 0x7f7e00068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.408+0000 7f7e2da00640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e000668f0 0x7f7e00068db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006fd0 tx=0x7f7e18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.412+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7e2815d3d0 con 0x7f7e2815e750 2024-01-15T18:05:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.564+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e2815d3d0 con 0x7f7e2815e750 2024-01-15T18:05:28.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.564+0000 7f7e25600640 1 -- 172.21.15.143:0/918498248 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e2815d3d0 con 0x7f7e2815e750 2024-01-15T18:05:28.571 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.564+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e000668f0 msgr2=0x7f7e00068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.564+0000 7f7e30400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e000668f0 0x7f7e00068db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e18006fd0 tx=0x7f7e18008040 comp rx=0 tx=0).stop 2024-01-15T18:05:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.564+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 msgr2=0x7f7e2814e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.564+0000 7f7e30400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 0x7f7e2814e710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e2000ccf0 tx=0x7f7e2000a660 comp rx=0 tx=0).stop 2024-01-15T18:05:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 shutdown_connections 2024-01-15T18:05:28.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e000668f0 0x7f7e00068db0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e2815e750 0x7f7e2814e710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e280d1960 0x7f7e28150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 --2- 172.21.15.143:0/918498248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cfd40 0x7f7e2815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 >> 172.21.15.143:0/918498248 conn(0x7f7e280d0e40 msgr2=0x7f7e280cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 shutdown_connections 2024-01-15T18:05:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.568+0000 7f7e30400640 1 -- 172.21.15.143:0/918498248 wait complete. 2024-01-15T18:05:28.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:28.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 Processor -- start 2024-01-15T18:05:28.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 -- start start 2024-01-15T18:05:28.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f62680ce560 0x7f626815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6268071d00 0x7f6268072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6268072650 con 0x7f6268141660 2024-01-15T18:05:28.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626815fbb0 con 0x7f6268071d00 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f626d200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f626815f310 con 0x7f62680ce560 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6268071d00 0x7f6268072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6268071d00 0x7f6268072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37618/0 (socket says 172.21.15.143:37618) 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 -- 172.21.15.143:0/1507242732 learned_addr learned my addr 172.21.15.143:0/1507242732 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6265400640 1 --2- 172.21.15.143:0/1507242732 >> v2:172.21.15.204:3300/0 conn(0x7f62680ce560 0x7f626815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 -- 172.21.15.143:0/1507242732 >> v2:172.21.15.204:3300/0 conn(0x7f62680ce560 msgr2=0x7f626815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 --2- 172.21.15.143:0/1507242732 >> v2:172.21.15.204:3300/0 conn(0x7f62680ce560 0x7f626815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 -- 172.21.15.143:0/1507242732 >> v1:172.21.15.204:6789/0 conn(0x7f6268141660 legacy=0x7f6268141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.728+0000 7f6266800640 1 -- 172.21.15.143:0/1507242732 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f626815f5f0 con 0x7f6268071d00 2024-01-15T18:05:28.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f6266800640 1 --2- 172.21.15.143:0/1507242732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268071d00 0x7f6268072110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f626806f940 tx=0x7f626001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8137f65cfd40dc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f6264a00640 1 -- 172.21.15.143:0/1507242732 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600a5780 con 0x7f6268071d00 2024-01-15T18:05:28.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f6264a00640 1 -- 172.21.15.143:0/1507242732 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62600a58e0 con 0x7f6268071d00 2024-01-15T18:05:28.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f6264a00640 1 -- 172.21.15.143:0/1507242732 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600a5c70 con 0x7f6268071d00 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 -- 172.21.15.143:0/1507242732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268071d00 msgr2=0x7f6268072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 --2- 172.21.15.143:0/1507242732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268071d00 0x7f6268072110 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f626806f940 tx=0x7f626001c9d0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 -- 172.21.15.143:0/1507242732 shutdown_connections 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 --2- 172.21.15.143:0/1507242732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268071d00 0x7f6268072110 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 --2- 172.21.15.143:0/1507242732 >> v2:172.21.15.204:3300/0 conn(0x7f62680ce560 0x7f626815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 -- 172.21.15.143:0/1507242732 >> 172.21.15.143:0/1507242732 conn(0x7f626806c3f0 msgr2=0x7f626806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 -- 172.21.15.143:0/1507242732 shutdown_connections 2024-01-15T18:05:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.732+0000 7f626d200640 1 -- 172.21.15.143:0/1507242732 wait complete. 2024-01-15T18:05:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 Processor -- start 2024-01-15T18:05:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 -- start start 2024-01-15T18:05:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6268071d00 0x7f6268158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62680ce560 0x7f62681405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 0x7f626813b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6268159290 con 0x7f6268141660 2024-01-15T18:05:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6268140b20 con 0x7f62680ce560 2024-01-15T18:05:28.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62680b37a0 con 0x7f6268071d00 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 0x7f626813b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 0x7f626813b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37634/0 (socket says 172.21.15.143:37634) 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 -- 172.21.15.143:0/3351164584 learned_addr learned my addr 172.21.15.143:0/3351164584 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6265400640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62680ce560 0x7f62681405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6265e00640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6268071d00 0x7f6268158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 -- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6268071d00 msgr2=0x7f6268158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6268071d00 0x7f6268158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 -- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62680ce560 msgr2=0x7f62681405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62680ce560 0x7f62681405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 -- 172.21.15.143:0/3351164584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260009b30 con 0x7f6268141660 2024-01-15T18:05:28.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f6266800640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 0x7f626813b490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62600195e0 tx=0x7f6260019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600b8670 con 0x7f6268141660 2024-01-15T18:05:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62600b87f0 con 0x7f6268141660 2024-01-15T18:05:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62600b8ba0 con 0x7f6268141660 2024-01-15T18:05:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62680cd150 con 0x7f6268141660 2024-01-15T18:05:28.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.736+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626813fb00 con 0x7f6268141660 2024-01-15T18:05:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.740+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6268067500 con 0x7f6268141660 2024-01-15T18:05:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.740+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f62600b8d00 con 0x7f6268141660 2024-01-15T18:05:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.740+0000 7f625e000640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f62400668f0 0x7f6240068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.740+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f626011f770 con 0x7f6268141660 2024-01-15T18:05:28.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.740+0000 7f6265e00640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f62400668f0 0x7f6240068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.740+0000 7f6265e00640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f62400668f0 0x7f6240068db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6258007fd0 tx=0x7f6258009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.744+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f62600ecb70 con 0x7f6268141660 2024-01-15T18:05:28.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.896+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6268159c60 con 0x7f6268141660 2024-01-15T18:05:28.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.896+0000 7f625e000640 1 -- 172.21.15.143:0/3351164584 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6260009510 con 0x7f6268141660 2024-01-15T18:05:28.904 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f62400668f0 msgr2=0x7f6240068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f62400668f0 0x7f6240068db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6258007fd0 tx=0x7f6258009040 comp rx=0 tx=0).stop 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 msgr2=0x7f626813b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 0x7f626813b490 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f62600195e0 tx=0x7f6260019770 comp rx=0 tx=0).stop 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 shutdown_connections 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f62400668f0 0x7f6240068db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6268141660 0x7f626813b490 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62680ce560 0x7f62681405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 --2- 172.21.15.143:0/3351164584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6268071d00 0x7f6268158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 >> 172.21.15.143:0/3351164584 conn(0x7f626806c3f0 msgr2=0x7f62680afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 shutdown_connections 2024-01-15T18:05:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:28.900+0000 7f626d200640 1 -- 172.21.15.143:0/3351164584 wait complete. 2024-01-15T18:05:28.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:29.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 Processor -- start 2024-01-15T18:05:29.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 -- start start 2024-01-15T18:05:29.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb59415b120 0x7fb5940cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb5940cf400 0x7fb5940cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5940d1940 0x7fb5940cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940cf970 con 0x7fb5940d1940 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5940cfc50 con 0x7fb5940cf400 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.056+0000 7fb59b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940c4b10 con 0x7fb59415b120 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5940d1940 0x7fb5940cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5940d1940 0x7fb5940cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37640/0 (socket says 172.21.15.143:37640) 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 -- 172.21.15.143:0/664028951 learned_addr learned my addr 172.21.15.143:0/664028951 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb598e00640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb59415b120 0x7fb5940cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 -- 172.21.15.143:0/664028951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb59415b120 msgr2=0x7fb5940cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb593e00640 1 --2- 172.21.15.143:0/664028951 >> v2:172.21.15.204:3300/0 conn(0x7fb5940cf400 0x7fb5940cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb59415b120 0x7fb5940cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 -- 172.21.15.143:0/664028951 >> v2:172.21.15.204:3300/0 conn(0x7fb5940cf400 msgr2=0x7fb5940cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:29.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 --2- 172.21.15.143:0/664028951 >> v2:172.21.15.204:3300/0 conn(0x7fb5940cf400 0x7fb5940cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 -- 172.21.15.143:0/664028951 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940c4df0 con 0x7fb5940d1940 2024-01-15T18:05:29.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb598e00640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb59415b120 0x7fb5940cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:29.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb593e00640 1 --2- 172.21.15.143:0/664028951 >> v2:172.21.15.204:3300/0 conn(0x7fb5940cf400 0x7fb5940cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:29.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb599800640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5940cba30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb594071040 tx=0x7fb58c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd69c4dde088a89 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb593400640 1 -- 172.21.15.143:0/664028951 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0a55f0 con 0x7fb5940d1940 2024-01-15T18:05:29.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb593400640 1 -- 172.21.15.143:0/664028951 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c0a5750 con 0x7fb5940d1940 2024-01-15T18:05:29.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.060+0000 7fb593400640 1 -- 172.21.15.143:0/664028951 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c0a5ae0 con 0x7fb5940d1940 2024-01-15T18:05:29.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- 172.21.15.143:0/664028951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 msgr2=0x7fb5940cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5940cba30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb594071040 tx=0x7fb58c01c940 comp rx=0 tx=0).stop 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- 172.21.15.143:0/664028951 shutdown_connections 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5940cba30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- 172.21.15.143:0/664028951 >> v2:172.21.15.204:3300/0 conn(0x7fb5940cf400 0x7fb5940cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- 172.21.15.143:0/664028951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb59415b120 0x7fb5940cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- 172.21.15.143:0/664028951 >> 172.21.15.143:0/664028951 conn(0x7fb5940d0e20 msgr2=0x7fb5940d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- 172.21.15.143:0/664028951 shutdown_connections 2024-01-15T18:05:29.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- 172.21.15.143:0/664028951 wait complete. 2024-01-15T18:05:29.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 Processor -- start 2024-01-15T18:05:29.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- start start 2024-01-15T18:05:29.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5940cf400 0x7fb5940ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5941546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb59415b120 0x7fb5940c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941f7940 con 0x7fb5940d1940 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5940c9320 con 0x7fb59415b120 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb59b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941f71a0 con 0x7fb5940cf400 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.064+0000 7fb593e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5941546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5941546b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37646/0 (socket says 172.21.15.143:37646) 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 -- 172.21.15.143:0/3675589224 learned_addr learned my addr 172.21.15.143:0/3675589224 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb599800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb59415b120 0x7fb5940c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb598e00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5940cf400 0x7fb5940ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 -- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5940cf400 msgr2=0x7fb5940ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5940cf400 0x7fb5940ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 -- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb59415b120 msgr2=0x7fb5940c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb59415b120 0x7fb5940c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 -- 172.21.15.143:0/3675589224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c009b30 con 0x7fb5940d1940 2024-01-15T18:05:29.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb599800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb59415b120 0x7fb5940c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:29.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb593e00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5941546b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c7c0 tx=0x7fb57c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c007a30 con 0x7fb5940d1940 2024-01-15T18:05:29.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c007b90 con 0x7fb5940d1940 2024-01-15T18:05:29.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5941f6be0 con 0x7fb5940d1940 2024-01-15T18:05:29.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00ce70 con 0x7fb5940d1940 2024-01-15T18:05:29.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5941f75d0 con 0x7fb5940d1940 2024-01-15T18:05:29.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5941f7af0 con 0x7fb5940d1940 2024-01-15T18:05:29.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb57c0113f0 con 0x7fb5940d1940 2024-01-15T18:05:29.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb590c00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb56c0668f0 0x7fb56c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.068+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb57c0848d0 con 0x7fb5940d1940 2024-01-15T18:05:29.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.072+0000 7fb598e00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb56c0668f0 0x7fb56c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.072+0000 7fb598e00640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb56c0668f0 0x7fb56c068db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb584006fd0 tx=0x7fb584008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.072+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb5941f7af0 con 0x7fb5940d1940 2024-01-15T18:05:29.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.220+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5940c4460 con 0x7fb5940d1940 2024-01-15T18:05:29.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb590c00640 1 -- 172.21.15.143:0/3675589224 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb57c051b00 con 0x7fb5940d1940 2024-01-15T18:05:29.229 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:29.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb56c0668f0 msgr2=0x7fb56c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb56c0668f0 0x7fb56c068db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb584006fd0 tx=0x7fb584008040 comp rx=0 tx=0).stop 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 msgr2=0x7fb5941546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5941546b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c7c0 tx=0x7fb57c00cc80 comp rx=0 tx=0).stop 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 shutdown_connections 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb56c0668f0 0x7fb56c068db0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb59415b120 0x7fb5940c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5940d1940 0x7fb5941546b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 --2- 172.21.15.143:0/3675589224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb5940cf400 0x7fb5940ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 >> 172.21.15.143:0/3675589224 conn(0x7fb5940d0e20 msgr2=0x7fb59415a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 shutdown_connections 2024-01-15T18:05:29.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.224+0000 7fb59b800640 1 -- 172.21.15.143:0/3675589224 wait complete. 2024-01-15T18:05:29.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone' 2024-01-15T18:05:29.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 Processor -- start 2024-01-15T18:05:29.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 -- start start 2024-01-15T18:05:29.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb8c0725b0 0x7ffb8c162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb8c15c830 0x7ffb8c165210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c15b540 con 0x7ffb8c0ade20 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b03d0 con 0x7ffb8c15c830 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb92a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c0af950 con 0x7ffb8c0725b0 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb90a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb8c15c830 0x7ffb8c165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb8be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffb8c0ade20 0x7ffb8c0ae200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51346/0 (socket says 172.21.15.143:51346) 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb90a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb8c15c830 0x7ffb8c165210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37658/0 (socket says 172.21.15.143:37658) 2024-01-15T18:05:29.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.380+0000 7ffb8be00640 1 -- 172.21.15.143:0/3246148294 learned_addr learned my addr 172.21.15.143:0/3246148294 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:29.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb8b400640 1 --2- 172.21.15.143:0/3246148294 >> v2:172.21.15.204:3300/0 conn(0x7ffb8c0725b0 0x7ffb8c162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb90a00640 1 -- 172.21.15.143:0/3246148294 >> v2:172.21.15.204:3300/0 conn(0x7ffb8c0725b0 msgr2=0x7ffb8c162cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:29.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb90a00640 1 --2- 172.21.15.143:0/3246148294 >> v2:172.21.15.204:3300/0 conn(0x7ffb8c0725b0 0x7ffb8c162cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb90a00640 1 -- 172.21.15.143:0/3246148294 >> v1:172.21.15.143:6789/0 conn(0x7ffb8c0ade20 legacy=0x7ffb8c0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb90a00640 1 -- 172.21.15.143:0/3246148294 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0afc30 con 0x7ffb8c15c830 2024-01-15T18:05:29.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb8aa00640 1 -- 172.21.15.143:0/3246148294 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202913611 0 0) 0x7ffb8c15b540 con 0x7ffb8c0ade20 2024-01-15T18:05:29.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb90a00640 1 --2- 172.21.15.143:0/3246148294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c165210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0accb0 tx=0x7ffb8401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f3092b37f4ffe3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb8aa00640 1 -- 172.21.15.143:0/3246148294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840a5910 con 0x7ffb8c15c830 2024-01-15T18:05:29.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb8aa00640 1 -- 172.21.15.143:0/3246148294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb840a5a70 con 0x7ffb8c15c830 2024-01-15T18:05:29.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.384+0000 7ffb8aa00640 1 -- 172.21.15.143:0/3246148294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840a5e00 con 0x7ffb8c15c830 2024-01-15T18:05:29.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- 172.21.15.143:0/3246148294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 msgr2=0x7ffb8c165210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 --2- 172.21.15.143:0/3246148294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c165210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0accb0 tx=0x7ffb8401c850 comp rx=0 tx=0).stop 2024-01-15T18:05:29.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- 172.21.15.143:0/3246148294 shutdown_connections 2024-01-15T18:05:29.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 --2- 172.21.15.143:0/3246148294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c165210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 --2- 172.21.15.143:0/3246148294 >> v2:172.21.15.204:3300/0 conn(0x7ffb8c0725b0 0x7ffb8c162cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- 172.21.15.143:0/3246148294 >> 172.21.15.143:0/3246148294 conn(0x7ffb8c071a90 msgr2=0x7ffb8c071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- 172.21.15.143:0/3246148294 shutdown_connections 2024-01-15T18:05:29.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- 172.21.15.143:0/3246148294 wait complete. 2024-01-15T18:05:29.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 Processor -- start 2024-01-15T18:05:29.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- start start 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb8c0725b0 0x7ffb8c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb8c0ade20 0x7ffb8c1529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c154600 con 0x7ffb8c15c830 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c1548e0 con 0x7ffb8c0725b0 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb92a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c154bc0 con 0x7ffb8c0ade20 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37672/0 (socket says 172.21.15.143:37672) 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 -- 172.21.15.143:0/52987493 learned_addr learned my addr 172.21.15.143:0/52987493 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:29.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb8be00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb8c0725b0 0x7ffb8c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb8b400640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb8c0ade20 0x7ffb8c1529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 -- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb8c0ade20 msgr2=0x7ffb8c1529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb8c0ade20 0x7ffb8c1529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 -- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb8c0725b0 msgr2=0x7ffb8c14c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb8c0725b0 0x7ffb8c14c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb90a00640 1 -- 172.21.15.143:0/52987493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84009a70 con 0x7ffb8c15c830 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb8b400640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb8c0ade20 0x7ffb8c1529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.388+0000 7ffb8be00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb8c0725b0 0x7ffb8c14c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:29.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb90a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c152ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb8401cd30 tx=0x7ffb840195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840bd070 con 0x7ffb8c15c830 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb840ad650 con 0x7ffb8c15c830 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c154ea0 con 0x7ffb8c15c830 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840ad9c0 con 0x7ffb8c15c830 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c14cdc0 con 0x7ffb8c15c830 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c0ae740 con 0x7ffb8c15c830 2024-01-15T18:05:29.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ffb840b4070 con 0x7ffb8c15c830 2024-01-15T18:05:29.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb83e00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffb640668f0 0x7ffb64068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb8411ec50 con 0x7ffb8c15c830 2024-01-15T18:05:29.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.392+0000 7ffb8be00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffb640668f0 0x7ffb64068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.396+0000 7ffb8be00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffb640668f0 0x7ffb64068db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c007fd0 tx=0x7ffb7c007c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.396+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb840ebd60 con 0x7ffb8c15c830 2024-01-15T18:05:29.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.600+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]} v 0) v1 -- 0x7ffb8c15b500 con 0x7ffb8c15c830 2024-01-15T18:05:29.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.660+0000 7ffb83e00640 1 -- 172.21.15.143:0/52987493 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]}]=0 v54) v1 ==== 155+0+0 (secure 0 0 0) 0x7ffb840b2070 con 0x7ffb8c15c830 2024-01-15T18:05:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffb640668f0 msgr2=0x7ffb64068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffb640668f0 0x7ffb64068db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c007fd0 tx=0x7ffb7c007c20 comp rx=0 tx=0).stop 2024-01-15T18:05:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 msgr2=0x7ffb8c152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c152ef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb8401cd30 tx=0x7ffb840195f0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 shutdown_connections 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffb640668f0 0x7ffb64068db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb8c15c830 0x7ffb8c152ef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb8c0ade20 0x7ffb8c1529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 --2- 172.21.15.143:0/52987493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb8c0725b0 0x7ffb8c14c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 >> 172.21.15.143:0/52987493 conn(0x7ffb8c071a90 msgr2=0x7ffb8c159470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 shutdown_connections 2024-01-15T18:05:29.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.664+0000 7ffb92a00640 1 -- 172.21.15.143:0/52987493 wait complete. 2024-01-15T18:05:29.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:05:29.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 Processor -- start 2024-01-15T18:05:29.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 -- start start 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f99a0141660 0x7f99a00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99a015ec80 0x7f99a0141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a015f310 con 0x7f99a0142470 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a015fbb0 con 0x7f99a015ec80 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f99a5000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a015f130 con 0x7f99a0141660 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99a015ec80 0x7f99a0141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999e800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f99a0142470 0x7f99a0071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51354/0 (socket says 172.21.15.143:51354) 2024-01-15T18:05:29.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f99a015ec80 0x7f99a0141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37676/0 (socket says 172.21.15.143:37676) 2024-01-15T18:05:29.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999e800640 1 -- 172.21.15.143:0/2768140018 learned_addr learned my addr 172.21.15.143:0/2768140018 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:29.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999de00640 1 --2- 172.21.15.143:0/2768140018 >> v2:172.21.15.204:3300/0 conn(0x7f99a0141660 0x7f99a00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999d400640 1 -- 172.21.15.143:0/2768140018 >> v2:172.21.15.204:3300/0 conn(0x7f99a0141660 msgr2=0x7f99a00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:29.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999d400640 1 --2- 172.21.15.143:0/2768140018 >> v2:172.21.15.204:3300/0 conn(0x7f99a0141660 0x7f99a00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999d400640 1 -- 172.21.15.143:0/2768140018 >> v1:172.21.15.143:6789/0 conn(0x7f99a0142470 legacy=0x7f99a0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.820+0000 7f999d400640 1 -- 172.21.15.143:0/2768140018 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a015f5f0 con 0x7f99a015ec80 2024-01-15T18:05:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f999d400640 1 --2- 172.21.15.143:0/2768140018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a015ec80 0x7f99a0141f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f99a00b1aa0 tx=0x7f998801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5178d6f5e2ef5f47 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f999ca00640 1 -- 172.21.15.143:0/2768140018 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880a7070 con 0x7f99a015ec80 2024-01-15T18:05:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f999ca00640 1 -- 172.21.15.143:0/2768140018 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880ab430 con 0x7f99a015ec80 2024-01-15T18:05:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f999ca00640 1 -- 172.21.15.143:0/2768140018 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880ab7c0 con 0x7f99a015ec80 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 -- 172.21.15.143:0/2768140018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a015ec80 msgr2=0x7f99a0141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 --2- 172.21.15.143:0/2768140018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a015ec80 0x7f99a0141f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f99a00b1aa0 tx=0x7f998801c430 comp rx=0 tx=0).stop 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 -- 172.21.15.143:0/2768140018 shutdown_connections 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 --2- 172.21.15.143:0/2768140018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a015ec80 0x7f99a0141f30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 --2- 172.21.15.143:0/2768140018 >> v2:172.21.15.204:3300/0 conn(0x7f99a0141660 0x7f99a00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 -- 172.21.15.143:0/2768140018 >> 172.21.15.143:0/2768140018 conn(0x7f99a006c3f0 msgr2=0x7f99a006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 -- 172.21.15.143:0/2768140018 shutdown_connections 2024-01-15T18:05:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.824+0000 7f99a5000640 1 -- 172.21.15.143:0/2768140018 wait complete. 2024-01-15T18:05:29.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 Processor -- start 2024-01-15T18:05:29.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 -- start start 2024-01-15T18:05:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a0141660 0x7f99a013f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 0x7f99a015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99a015ec80 0x7f99a013fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0140b90 con 0x7f99a0142470 2024-01-15T18:05:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a015de90 con 0x7f99a0141660 2024-01-15T18:05:29.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a013efe0 con 0x7f99a015ec80 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 0x7f99a015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 0x7f99a015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37686/0 (socket says 172.21.15.143:37686) 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 -- 172.21.15.143:0/1868732835 learned_addr learned my addr 172.21.15.143:0/1868732835 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999de00640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a0141660 0x7f99a013f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999e800640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99a015ec80 0x7f99a013fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 -- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99a015ec80 msgr2=0x7f99a013fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99a015ec80 0x7f99a013fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 -- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a0141660 msgr2=0x7f99a013f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a0141660 0x7f99a013f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 -- 172.21.15.143:0/1868732835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009420 con 0x7f99a0142470 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999e800640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99a015ec80 0x7f99a013fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:29.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f999d400640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 0x7f99a015d950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9988019040 tx=0x7f998801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880b5810 con 0x7f99a0142470 2024-01-15T18:05:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880b5990 con 0x7f99a0142470 2024-01-15T18:05:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880b5d40 con 0x7f99a0142470 2024-01-15T18:05:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.828+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a00cd150 con 0x7f99a0142470 2024-01-15T18:05:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a013e990 con 0x7f99a0142470 2024-01-15T18:05:29.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a013e7f0 con 0x7f99a0142470 2024-01-15T18:05:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f99880b4390 con 0x7f99a0142470 2024-01-15T18:05:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f9996000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9978066920 0x7f9978068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f998811ec80 con 0x7f99a0142470 2024-01-15T18:05:29.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f999de00640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9978066920 0x7f9978068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.832+0000 7f999de00640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9978066920 0x7f9978068de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9990007a60 tx=0x7f9990008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.836+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f99a013e7f0 con 0x7f99a0142470 2024-01-15T18:05:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.988+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f99a013e7f0 con 0x7f99a0142470 2024-01-15T18:05:29.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.992+0000 7f9996000640 1 -- 172.21.15.143:0/1868732835 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99a013e7f0 con 0x7f99a0142470 2024-01-15T18:05:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9978066920 msgr2=0x7f9978068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9978066920 0x7f9978068de0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9990007a60 tx=0x7f9990008040 comp rx=0 tx=0).stop 2024-01-15T18:05:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 msgr2=0x7f99a015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 0x7f99a015d950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9988019040 tx=0x7f998801ac70 comp rx=0 tx=0).stop 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 shutdown_connections 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9978066920 0x7f9978068de0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f99a015ec80 0x7f99a013fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f99a0142470 0x7f99a015d950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 --2- 172.21.15.143:0/1868732835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f99a0141660 0x7f99a013f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 >> 172.21.15.143:0/1868732835 conn(0x7f99a006c3f0 msgr2=0x7f99a013a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 shutdown_connections 2024-01-15T18:05:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:29.996+0000 7f99a5000640 1 -- 172.21.15.143:0/1868732835 wait complete. 2024-01-15T18:05:30.022 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:05:30.022 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:05:30.103 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:05:30.167 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:05:30.167 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:05:30.167 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:30.176 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:05:30.176 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:30.224 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:30.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 Processor -- start 2024-01-15T18:05:30.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 -- start start 2024-01-15T18:05:30.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f90e0071d00 0x7f90e0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e015f310 con 0x7f90e015ec80 2024-01-15T18:05:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e015fbb0 con 0x7f90e0071d00 2024-01-15T18:05:30.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.396+0000 7f90e6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0072690 con 0x7f90e0141660 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f90e0071d00 0x7f90e0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f90e0071d00 0x7f90e0072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:56942/0 (socket says 172.21.15.143:56942) 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90df400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f90e015ec80 0x7f90e0141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:52422/0 (socket says 172.21.15.143:52422) 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 -- 172.21.15.143:0/2557392161 learned_addr learned my addr 172.21.15.143:0/2557392161 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 -- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 msgr2=0x7f90e00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 --2- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 -- 172.21.15.143:0/2557392161 >> v1:172.21.15.204:6789/0 conn(0x7f90e015ec80 legacy=0x7f90e0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 -- 172.21.15.143:0/2557392161 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e015f5f0 con 0x7f90e0071d00 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90dea00640 1 -- 172.21.15.143:0/2557392161 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426453863 0 0) 0x7f90e015f310 con 0x7f90e015ec80 2024-01-15T18:05:30.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90dfe00640 1 --2- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:30.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90e4a00640 1 --2- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e0072150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009e50 tx=0x7f90d801ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276b614dd7d574a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.400+0000 7f90dea00640 1 -- 172.21.15.143:0/2557392161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80a57d0 con 0x7f90e0071d00 2024-01-15T18:05:30.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90dea00640 1 -- 172.21.15.143:0/2557392161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80a5930 con 0x7f90e0071d00 2024-01-15T18:05:30.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90dea00640 1 -- 172.21.15.143:0/2557392161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80a5cc0 con 0x7f90e0071d00 2024-01-15T18:05:30.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 msgr2=0x7f90e0072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 --2- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e0072150 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009e50 tx=0x7f90d801ca10 comp rx=0 tx=0).stop 2024-01-15T18:05:30.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- 172.21.15.143:0/2557392161 shutdown_connections 2024-01-15T18:05:30.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 --2- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e0072150 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 --2- 172.21.15.143:0/2557392161 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- 172.21.15.143:0/2557392161 >> 172.21.15.143:0/2557392161 conn(0x7f90e006c3f0 msgr2=0x7f90e006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:30.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- 172.21.15.143:0/2557392161 shutdown_connections 2024-01-15T18:05:30.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- 172.21.15.143:0/2557392161 wait complete. 2024-01-15T18:05:30.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 Processor -- start 2024-01-15T18:05:30.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- start start 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e015d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e0165750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 0x7f90e013e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e0147670 con 0x7f90e015ec80 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e015d760 con 0x7f90e0071d00 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.404+0000 7f90e6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e015da40 con 0x7f90e0141660 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 0x7f90e013e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 0x7f90e013e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37696/0 (socket says 172.21.15.143:37696) 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 -- 172.21.15.143:0/1651492234 learned_addr learned my addr 172.21.15.143:0/1651492234 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90dfe00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90df400640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e0165750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 -- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 msgr2=0x7f90e0165750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e0165750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 -- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 msgr2=0x7f90e015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e015d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 -- 172.21.15.143:0/1651492234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009420 con 0x7f90e015ec80 2024-01-15T18:05:30.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e4a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 0x7f90e013e8a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90d80af040 tx=0x7f90d80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80b8670 con 0x7f90e015ec80 2024-01-15T18:05:30.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d80b87d0 con 0x7f90e015ec80 2024-01-15T18:05:30.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d80b8b80 con 0x7f90e015ec80 2024-01-15T18:05:30.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d8009b30 con 0x7f90e015ec80 2024-01-15T18:05:30.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e00cd150 con 0x7f90e015ec80 2024-01-15T18:05:30.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e01477e0 con 0x7f90e015ec80 2024-01-15T18:05:30.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.408+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f90d80b8ce0 con 0x7f90e015ec80 2024-01-15T18:05:30.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.412+0000 7f90d7e00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90b80668f0 0x7f90b8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.412+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90d811f730 con 0x7f90e015ec80 2024-01-15T18:05:30.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.412+0000 7f90dfe00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90b80668f0 0x7f90b8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.412+0000 7f90dfe00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90b80668f0 0x7f90b8068db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009c50 tx=0x7f90d0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.416+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90d80ecb40 con 0x7f90e015ec80 2024-01-15T18:05:30.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.572+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e0067500 con 0x7f90e015ec80 2024-01-15T18:05:30.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.572+0000 7f90d7e00640 1 -- 172.21.15.143:0/1651492234 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f90d8009510 con 0x7f90e015ec80 2024-01-15T18:05:30.580 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:30.580 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90b80668f0 msgr2=0x7f90b8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90b80668f0 0x7f90b8068db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009c50 tx=0x7f90d0009190 comp rx=0 tx=0).stop 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 msgr2=0x7f90e013e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 0x7f90e013e8a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f90d80af040 tx=0x7f90d80197b0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 shutdown_connections 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90b80668f0 0x7f90b8068db0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90e015ec80 0x7f90e013e8a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90e0141660 0x7f90e0165750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 --2- 172.21.15.143:0/1651492234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90e0071d00 0x7f90e015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 >> 172.21.15.143:0/1651492234 conn(0x7f90e006c3f0 msgr2=0x7f90e00afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 shutdown_connections 2024-01-15T18:05:30.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.576+0000 7f90e6a00640 1 -- 172.21.15.143:0/1651492234 wait complete. 2024-01-15T18:05:30.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:30.603 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:30.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 Processor -- start 2024-01-15T18:05:30.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 -- start start 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0b0310 0x7f423c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f423c0cf740 0x7f423c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c0cbff0 con 0x7f423c06bfb0 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c0d0c50 con 0x7f423c0cf740 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4244400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0cb6c0 con 0x7f423c0b0310 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f423c0cf740 0x7f423c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f423c0cf740 0x7f423c0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37708/0 (socket says 172.21.15.143:37708) 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 -- 172.21.15.143:0/12924313 learned_addr learned my addr 172.21.15.143:0/12924313 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:30.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4241000640 1 --2- 172.21.15.143:0/12924313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0b0310 0x7f423c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 -- 172.21.15.143:0/12924313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0b0310 msgr2=0x7f423c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:30.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 --2- 172.21.15.143:0/12924313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0b0310 0x7f423c0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 -- 172.21.15.143:0/12924313 >> v1:172.21.15.204:6789/0 conn(0x7f423c06bfb0 legacy=0x7f423c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.736+0000 7f4242400640 1 -- 172.21.15.143:0/12924313 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0cb9a0 con 0x7f423c0cf740 2024-01-15T18:05:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4242400640 1 --2- 172.21.15.143:0/12924313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c0cf740 0x7f423c0cfbf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4234009e50 tx=0x7f423401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4227bad975f6cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f423be00640 1 -- 172.21.15.143:0/12924313 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340a5810 con 0x7f423c0cf740 2024-01-15T18:05:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f423be00640 1 -- 172.21.15.143:0/12924313 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42340a5970 con 0x7f423c0cf740 2024-01-15T18:05:30.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f423be00640 1 -- 172.21.15.143:0/12924313 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42340a5d00 con 0x7f423c0cf740 2024-01-15T18:05:30.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 -- 172.21.15.143:0/12924313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c0cf740 msgr2=0x7f423c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 --2- 172.21.15.143:0/12924313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c0cf740 0x7f423c0cfbf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4234009e50 tx=0x7f423401c950 comp rx=0 tx=0).stop 2024-01-15T18:05:30.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 -- 172.21.15.143:0/12924313 shutdown_connections 2024-01-15T18:05:30.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 --2- 172.21.15.143:0/12924313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c0cf740 0x7f423c0cfbf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 --2- 172.21.15.143:0/12924313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0b0310 0x7f423c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 -- 172.21.15.143:0/12924313 >> 172.21.15.143:0/12924313 conn(0x7f423c0d0ed0 msgr2=0x7f423c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:30.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 -- 172.21.15.143:0/12924313 shutdown_connections 2024-01-15T18:05:30.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.740+0000 7f4244400640 1 -- 172.21.15.143:0/12924313 wait complete. 2024-01-15T18:05:30.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 Processor -- start 2024-01-15T18:05:30.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 -- start start 2024-01-15T18:05:30.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 0x7f423c0ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c0b0310 0x7f423c0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0cf740 0x7f423c0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0caee0 con 0x7f423c06bfb0 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1552d0 con 0x7f423c0b0310 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1555b0 con 0x7f423c0cf740 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 0x7f423c0ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 0x7f423c0ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37712/0 (socket says 172.21.15.143:37712) 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 -- 172.21.15.143:0/1858959063 learned_addr learned my addr 172.21.15.143:0/1858959063 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241000640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c0b0310 0x7f423c0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4242400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0cf740 0x7f423c0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 -- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0cf740 msgr2=0x7f423c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0cf740 0x7f423c0ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 -- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c0b0310 msgr2=0x7f423c0ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c0b0310 0x7f423c0ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 -- 172.21.15.143:0/1858959063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234009b30 con 0x7f423c06bfb0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4241a00640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 0x7f423c0ca5b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f422c00c3c0 tx=0x7f422c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c0052d0 con 0x7f423c06bfb0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c005430 con 0x7f423c06bfb0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c155890 con 0x7f423c06bfb0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c003840 con 0x7f423c06bfb0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.744+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c0caaf0 con 0x7f423c06bfb0 2024-01-15T18:05:30.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.748+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c067500 con 0x7f423c06bfb0 2024-01-15T18:05:30.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.748+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f422c0039a0 con 0x7f423c06bfb0 2024-01-15T18:05:30.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.748+0000 7f4239600640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42140668f0 0x7f4214068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.748+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f422c0838c0 con 0x7f423c06bfb0 2024-01-15T18:05:30.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.748+0000 7f4241000640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42140668f0 0x7f4214068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.748+0000 7f4241000640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42140668f0 0x7f4214068db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f423c0cec20 tx=0x7f4224009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.752+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f422c050ba0 con 0x7f423c06bfb0 2024-01-15T18:05:30.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.904+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f423c0d1db0 con 0x7f423c06bfb0 2024-01-15T18:05:30.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.904+0000 7f4239600640 1 -- 172.21.15.143:0/1858959063 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f422c0170a0 con 0x7f423c06bfb0 2024-01-15T18:05:30.911 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42140668f0 msgr2=0x7f4214068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42140668f0 0x7f4214068db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f423c0cec20 tx=0x7f4224009190 comp rx=0 tx=0).stop 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 msgr2=0x7f423c0ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 0x7f423c0ca5b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f422c00c3c0 tx=0x7f422c00c880 comp rx=0 tx=0).stop 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 shutdown_connections 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42140668f0 0x7f4214068db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f423c0cf740 0x7f423c0ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f423c0b0310 0x7f423c0ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 --2- 172.21.15.143:0/1858959063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f423c06bfb0 0x7f423c0ca5b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 >> 172.21.15.143:0/1858959063 conn(0x7f423c0d0ed0 msgr2=0x7f423c06edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 shutdown_connections 2024-01-15T18:05:30.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:30.908+0000 7f4244400640 1 -- 172.21.15.143:0/1858959063 wait complete. 2024-01-15T18:05:30.933 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:31.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.064+0000 7f21ea400640 1 Processor -- start 2024-01-15T18:05:31.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.064+0000 7f21ea400640 1 -- start start 2024-01-15T18:05:31.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e4157300 0x7f21e4159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21ea400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21e40b21d0 0x7f21e40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21ea400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e40b2a20 con 0x7f21e40b48f0 2024-01-15T18:05:31.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21ea400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40b35a0 con 0x7f21e40b21d0 2024-01-15T18:05:31.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21ea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40b4d80 con 0x7f21e4157300 2024-01-15T18:05:31.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21e40b21d0 0x7f21e40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f21e40b48f0 0x7f21e415d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51368/0 (socket says 172.21.15.143:51368) 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e3e00640 1 -- 172.21.15.143:0/462860820 learned_addr learned my addr 172.21.15.143:0/462860820 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e3400640 1 --2- 172.21.15.143:0/462860820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e4157300 0x7f21e4159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2a00640 1 -- 172.21.15.143:0/462860820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e4157300 msgr2=0x7f21e4159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2a00640 1 --2- 172.21.15.143:0/462860820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e4157300 0x7f21e4159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2a00640 1 -- 172.21.15.143:0/462860820 >> v1:172.21.15.143:6789/0 conn(0x7f21e40b48f0 legacy=0x7f21e415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2a00640 1 -- 172.21.15.143:0/462860820 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40b2d00 con 0x7f21e40b21d0 2024-01-15T18:05:31.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2000640 1 -- 172.21.15.143:0/462860820 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566353668 0 0) 0x7f21e40b2a20 con 0x7f21e40b48f0 2024-01-15T18:05:31.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.068+0000 7f21e2a00640 1 --2- 172.21.15.143:0/462860820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e40b43b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097c0 tx=0x7f21cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cedd0f12eab70a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21e2000640 1 -- 172.21.15.143:0/462860820 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0ab600 con 0x7f21e40b21d0 2024-01-15T18:05:31.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21e2000640 1 -- 172.21.15.143:0/462860820 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc0a7050 con 0x7f21e40b21d0 2024-01-15T18:05:31.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21e2000640 1 -- 172.21.15.143:0/462860820 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0ab980 con 0x7f21e40b21d0 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- 172.21.15.143:0/462860820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 msgr2=0x7f21e40b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 --2- 172.21.15.143:0/462860820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e40b43b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0097c0 tx=0x7f21cc01c450 comp rx=0 tx=0).stop 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- 172.21.15.143:0/462860820 shutdown_connections 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 --2- 172.21.15.143:0/462860820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e40b43b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 --2- 172.21.15.143:0/462860820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e4157300 0x7f21e4159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- 172.21.15.143:0/462860820 >> 172.21.15.143:0/462860820 conn(0x7f21e406c3f0 msgr2=0x7f21e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- 172.21.15.143:0/462860820 shutdown_connections 2024-01-15T18:05:31.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- 172.21.15.143:0/462860820 wait complete. 2024-01-15T18:05:31.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 Processor -- start 2024-01-15T18:05:31.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- start start 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e4155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e40b48f0 0x7f21e4155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4157300 0x7f21e4138320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41576e0 con 0x7f21e40b21d0 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4136430 con 0x7f21e4157300 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.072+0000 7f21ea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4157930 con 0x7f21e40b48f0 2024-01-15T18:05:31.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e4155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e4155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37734/0 (socket says 172.21.15.143:37734) 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 -- 172.21.15.143:0/439564341 learned_addr learned my addr 172.21.15.143:0/439564341 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e2a00640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e40b48f0 0x7f21e4155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3e00640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4157300 0x7f21e4138320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 -- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e40b48f0 msgr2=0x7f21e4155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e40b48f0 0x7f21e4155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 -- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4157300 msgr2=0x7f21e4138320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4157300 0x7f21e4138320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 -- 172.21.15.143:0/439564341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009420 con 0x7f21e40b21d0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21e3400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e4155330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f21d400e870 tx=0x7f21d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4009510 con 0x7f21e40b21d0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d400eec0 con 0x7f21e40b21d0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d400cb40 con 0x7f21e40b21d0 2024-01-15T18:05:31.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e4157c10 con 0x7f21e40b21d0 2024-01-15T18:05:31.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e4157ef0 con 0x7f21e40b21d0 2024-01-15T18:05:31.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e4067500 con 0x7f21e40b21d0 2024-01-15T18:05:31.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.076+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f21d400cca0 con 0x7f21e40b21d0 2024-01-15T18:05:31.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.080+0000 7f21db400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21bc0668f0 0x7f21bc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.080+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21d4084450 con 0x7f21e40b21d0 2024-01-15T18:05:31.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.080+0000 7f21e2a00640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21bc0668f0 0x7f21bc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.080+0000 7f21e2a00640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21bc0668f0 0x7f21bc068db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e415a0b0 tx=0x7f21cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.084+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21d40517d0 con 0x7f21e40b21d0 2024-01-15T18:05:31.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21e406e4e0 con 0x7f21e40b21d0 2024-01-15T18:05:31.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21db400640 1 -- 172.21.15.143:0/439564341 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21d4054df0 con 0x7f21e40b21d0 2024-01-15T18:05:31.238 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:31.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21bc0668f0 msgr2=0x7f21bc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21bc0668f0 0x7f21bc068db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21e415a0b0 tx=0x7f21cc008040 comp rx=0 tx=0).stop 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 msgr2=0x7f21e4155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e4155330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f21d400e870 tx=0x7f21d400ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 shutdown_connections 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21bc0668f0 0x7f21bc068db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21e4157300 0x7f21e4138320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21e40b48f0 0x7f21e4155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 --2- 172.21.15.143:0/439564341 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21e40b21d0 0x7f21e4155330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 >> 172.21.15.143:0/439564341 conn(0x7f21e406c3f0 msgr2=0x7f21e40ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 shutdown_connections 2024-01-15T18:05:31.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.232+0000 7f21ea400640 1 -- 172.21.15.143:0/439564341 wait complete. 2024-01-15T18:05:31.260 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7730000640 1 Processor -- start 2024-01-15T18:05:31.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7730000640 1 -- start start 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7730000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77281607c0 0x7f772806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7730000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77280cffd0 con 0x7f77280cfb30 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7730000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77280d02b0 con 0x7f77280cbf50 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7730000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77280d1ce0 con 0x7f77281607c0 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f772cc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f77280cfb30 0x7f772806bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51380/0 (socket says 172.21.15.143:51380) 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f772cc00640 1 -- 172.21.15.143:0/2102032127 learned_addr learned my addr 172.21.15.143:0/2102032127 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:31.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f772d600640 1 --2- 172.21.15.143:0/2102032127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77281607c0 0x7f772806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7727e00640 1 -- 172.21.15.143:0/2102032127 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342171733 0 0) 0x7f77280cffd0 con 0x7f77280cfb30 2024-01-15T18:05:31.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7727e00640 1 -- 172.21.15.143:0/2102032127 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7708003530 con 0x7f77280cfb30 2024-01-15T18:05:31.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f772d600640 1 -- 172.21.15.143:0/2102032127 >> v1:172.21.15.204:6789/0 conn(0x7f77280cbf50 legacy=0x7f77280cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f7727e00640 1 -- 172.21.15.143:0/2102032127 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650649448 0 0) 0x7f77280d02b0 con 0x7f77280cbf50 2024-01-15T18:05:31.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f772d600640 1 -- 172.21.15.143:0/2102032127 >> v1:172.21.15.143:6789/0 conn(0x7f77280cfb30 legacy=0x7f772806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.396+0000 7f772d600640 1 -- 172.21.15.143:0/2102032127 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77280cb470 con 0x7f77281607c0 2024-01-15T18:05:31.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f772d600640 1 --2- 172.21.15.143:0/2102032127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77281607c0 0x7f772806b7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77180097f0 tx=0x7f771801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22ba2d93cf244e0e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7727e00640 1 -- 172.21.15.143:0/2102032127 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180a89b0 con 0x7f77281607c0 2024-01-15T18:05:31.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7727e00640 1 -- 172.21.15.143:0/2102032127 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77180a8b10 con 0x7f77281607c0 2024-01-15T18:05:31.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7727e00640 1 -- 172.21.15.143:0/2102032127 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180a8ea0 con 0x7f77281607c0 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 -- 172.21.15.143:0/2102032127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77281607c0 msgr2=0x7f772806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 --2- 172.21.15.143:0/2102032127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77281607c0 0x7f772806b7b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77180097f0 tx=0x7f771801ecb0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 -- 172.21.15.143:0/2102032127 shutdown_connections 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 --2- 172.21.15.143:0/2102032127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77281607c0 0x7f772806b7b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 -- 172.21.15.143:0/2102032127 >> 172.21.15.143:0/2102032127 conn(0x7f77280d0e20 msgr2=0x7f77280d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 -- 172.21.15.143:0/2102032127 shutdown_connections 2024-01-15T18:05:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.400+0000 7f7730000640 1 -- 172.21.15.143:0/2102032127 wait complete. 2024-01-15T18:05:31.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 Processor -- start 2024-01-15T18:05:31.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 -- start start 2024-01-15T18:05:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77280cbf50 0x7f77280c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77280cfb30 0x7f7728156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 0x7f772814a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c6b30 con 0x7f77281607c0 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728156e70 con 0x7f77280cbf50 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77280c5eb0 con 0x7f77280cfb30 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 0x7f772814a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 0x7f772814a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37736/0 (socket says 172.21.15.143:37736) 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 -- 172.21.15.143:0/2308406108 learned_addr learned my addr 172.21.15.143:0/2308406108 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772cc00640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77280cfb30 0x7f7728156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772d600640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77280cbf50 0x7f77280c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 -- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77280cfb30 msgr2=0x7f7728156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77280cfb30 0x7f7728156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 -- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77280cbf50 msgr2=0x7f77280c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77280cbf50 0x7f77280c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 -- 172.21.15.143:0/2308406108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718009420 con 0x7f77281607c0 2024-01-15T18:05:31.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f772e000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 0x7f772814a340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7720001570 tx=0x7f772000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720003db0 con 0x7f77281607c0 2024-01-15T18:05:31.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720005070 con 0x7f77281607c0 2024-01-15T18:05:31.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720005420 con 0x7f77281607c0 2024-01-15T18:05:31.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77280c6190 con 0x7f77281607c0 2024-01-15T18:05:31.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.404+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77280c6750 con 0x7f77281607c0 2024-01-15T18:05:31.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.408+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772806d4b0 con 0x7f77281607c0 2024-01-15T18:05:31.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.408+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f7720005580 con 0x7f77281607c0 2024-01-15T18:05:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.408+0000 7f7725600640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77000668f0 0x7f7700068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.408+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7720083c60 con 0x7f77281607c0 2024-01-15T18:05:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.408+0000 7f772d600640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77000668f0 0x7f7700068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.408+0000 7f772d600640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77000668f0 0x7f7700068db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77180b2000 tx=0x7f77180aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.412+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7720050f40 con 0x7f77281607c0 2024-01-15T18:05:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.560+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77280d0280 con 0x7f77281607c0 2024-01-15T18:05:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.560+0000 7f7725600640 1 -- 172.21.15.143:0/2308406108 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7720017070 con 0x7f77281607c0 2024-01-15T18:05:31.569 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77000668f0 msgr2=0x7f7700068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77000668f0 0x7f7700068db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f77180b2000 tx=0x7f77180aa070 comp rx=0 tx=0).stop 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 msgr2=0x7f772814a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 0x7f772814a340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7720001570 tx=0x7f772000cd20 comp rx=0 tx=0).stop 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 shutdown_connections 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77000668f0 0x7f7700068db0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77281607c0 0x7f772814a340 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77280cfb30 0x7f7728156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 --2- 172.21.15.143:0/2308406108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f77280cbf50 0x7f77280c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 >> 172.21.15.143:0/2308406108 conn(0x7f77280d0e20 msgr2=0x7f772815d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 shutdown_connections 2024-01-15T18:05:31.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.564+0000 7f7730000640 1 -- 172.21.15.143:0/2308406108 wait complete. 2024-01-15T18:05:31.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:31.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 Processor -- start 2024-01-15T18:05:31.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 -- start start 2024-01-15T18:05:31.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdff415c5a0 0x7fdff415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff40ceb50 con 0x7fdff4143780 2024-01-15T18:05:31.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff4072570 con 0x7fdff415c5a0 2024-01-15T18:05:31.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdffa800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff415c110 con 0x7fdff4071bb0 2024-01-15T18:05:31.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdff415c5a0 0x7fdff415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdff415c5a0 0x7fdff415ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:56972/0 (socket says 172.21.15.143:56972) 2024-01-15T18:05:31.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 -- 172.21.15.143:0/3584229070 learned_addr learned my addr 172.21.15.143:0/3584229070 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:31.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff4072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:44032/0 (socket says 172.21.15.143:44032) 2024-01-15T18:05:31.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 -- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 msgr2=0x7fdff4072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:31.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 --2- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff4072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 -- 172.21.15.143:0/3584229070 >> v1:172.21.15.204:6789/0 conn(0x7fdff4143780 legacy=0x7fdff40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.724+0000 7fdff3e00640 1 -- 172.21.15.143:0/3584229070 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff415b7e0 con 0x7fdff415c5a0 2024-01-15T18:05:31.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdff3e00640 1 --2- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff415c5a0 0x7fdff415ca30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29de98b9827fc05e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdff2000640 1 -- 172.21.15.143:0/3584229070 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0a58e0 con 0x7fdff415c5a0 2024-01-15T18:05:31.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdff2000640 1 -- 172.21.15.143:0/3584229070 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec0a5a40 con 0x7fdff415c5a0 2024-01-15T18:05:31.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdff2000640 1 -- 172.21.15.143:0/3584229070 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0a5dd0 con 0x7fdff415c5a0 2024-01-15T18:05:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 -- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff415c5a0 msgr2=0x7fdff415ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 --2- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff415c5a0 0x7fdff415ca30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009e50 tx=0x7fdfec01ca10 comp rx=0 tx=0).stop 2024-01-15T18:05:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 -- 172.21.15.143:0/3584229070 shutdown_connections 2024-01-15T18:05:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 --2- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff415c5a0 0x7fdff415ca30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 --2- 172.21.15.143:0/3584229070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff4072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 -- 172.21.15.143:0/3584229070 >> 172.21.15.143:0/3584229070 conn(0x7fdff406c3f0 msgr2=0x7fdff406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 -- 172.21.15.143:0/3584229070 shutdown_connections 2024-01-15T18:05:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.728+0000 7fdffa800640 1 -- 172.21.15.143:0/3584229070 wait complete. 2024-01-15T18:05:31.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 Processor -- start 2024-01-15T18:05:31.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 -- start start 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff413cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4143780 0x7fdff40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 0x7fdff40b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4142e40 con 0x7fdff415c5a0 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4140530 con 0x7fdff4143780 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff4159900 con 0x7fdff4071bb0 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 0x7fdff40b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 0x7fdff40b3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37744/0 (socket says 172.21.15.143:37744) 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 -- 172.21.15.143:0/2032584166 learned_addr learned my addr 172.21.15.143:0/2032584166 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:31.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff2a00640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4143780 0x7fdff40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3400640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff413cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 -- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 msgr2=0x7fdff413cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff413cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 -- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4143780 msgr2=0x7fdff40b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4143780 0x7fdff40b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 -- 172.21.15.143:0/2032584166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec009b30 con 0x7fdff415c5a0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdff3e00640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 0x7fdff40b3250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfec01a640 tx=0x7fdfec0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0b8640 con 0x7fdff415c5a0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfec0b87a0 con 0x7fdff415c5a0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0b8b30 con 0x7fdff415c5a0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.732+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff4140740 con 0x7fdff415c5a0 2024-01-15T18:05:31.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff4140d00 con 0x7fdff415c5a0 2024-01-15T18:05:31.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff40705d0 con 0x7fdff415c5a0 2024-01-15T18:05:31.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fdfec0b8c90 con 0x7fdff415c5a0 2024-01-15T18:05:31.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdfeb400640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfcc0668f0 0x7fdfcc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:31.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfec11f530 con 0x7fdff415c5a0 2024-01-15T18:05:31.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdff3400640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfcc0668f0 0x7fdfcc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:31.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.736+0000 7fdff3400640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfcc0668f0 0x7fdfcc068db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4001650 tx=0x7fdfe4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:31.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.740+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdfec0ec800 con 0x7fdff415c5a0 2024-01-15T18:05:31.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdff4067500 con 0x7fdff415c5a0 2024-01-15T18:05:31.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdfeb400640 1 -- 172.21.15.143:0/2032584166 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdfec0efe20 con 0x7fdff415c5a0 2024-01-15T18:05:31.910 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:31.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfcc0668f0 msgr2=0x7fdfcc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfcc0668f0 0x7fdfcc068db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4001650 tx=0x7fdfe4009040 comp rx=0 tx=0).stop 2024-01-15T18:05:31.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 msgr2=0x7fdff40b3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:31.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 0x7fdff40b3250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdfec01a640 tx=0x7fdfec0197b0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 shutdown_connections 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfcc0668f0 0x7fdfcc068db0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdff415c5a0 0x7fdff40b3250 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdff4143780 0x7fdff40b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 --2- 172.21.15.143:0/2032584166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdff4071bb0 0x7fdff413cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 >> 172.21.15.143:0/2032584166 conn(0x7fdff406c3f0 msgr2=0x7fdff40ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 shutdown_connections 2024-01-15T18:05:31.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:31.904+0000 7fdffa800640 1 -- 172.21.15.143:0/2032584166 wait complete. 2024-01-15T18:05:31.932 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:05:31.933 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:31.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:32.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 Processor -- start 2024-01-15T18:05:32.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 -- start start 2024-01-15T18:05:32.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65e00b24b0 0x7f65e00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f65e00b3bf0 0x7f65e00b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e0070bf0 con 0x7f65e00ce690 2024-01-15T18:05:32.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e015dba0 con 0x7f65e00b24b0 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65e5a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e00b4920 con 0x7f65e00b3bf0 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65de000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65e00b24b0 0x7f65e00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65dea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f65e00ce690 0x7f65e00b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51392/0 (socket says 172.21.15.143:51392) 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65dea00640 1 -- 172.21.15.143:0/4281614111 learned_addr learned my addr 172.21.15.143:0/4281614111 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.116+0000 7f65df400640 1 --2- 172.21.15.143:0/4281614111 >> v2:172.21.15.204:3300/0 conn(0x7f65e00b3bf0 0x7f65e00b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65de000640 1 -- 172.21.15.143:0/4281614111 >> v2:172.21.15.204:3300/0 conn(0x7f65e00b3bf0 msgr2=0x7f65e00b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65de000640 1 --2- 172.21.15.143:0/4281614111 >> v2:172.21.15.204:3300/0 conn(0x7f65e00b3bf0 0x7f65e00b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65de000640 1 -- 172.21.15.143:0/4281614111 >> v1:172.21.15.143:6789/0 conn(0x7f65e00ce690 legacy=0x7f65e00b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65de000640 1 -- 172.21.15.143:0/4281614111 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e00b4c00 con 0x7f65e00b24b0 2024-01-15T18:05:32.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65df400640 1 --2- 172.21.15.143:0/4281614111 >> v2:172.21.15.204:3300/0 conn(0x7f65e00b3bf0 0x7f65e00b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:32.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65de000640 1 --2- 172.21.15.143:0/4281614111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b24b0 0x7f65e00ac8e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f65e015e870 tx=0x7f65c801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac06a6b8a13f587a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65dd600640 1 -- 172.21.15.143:0/4281614111 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80ab650 con 0x7f65e00b24b0 2024-01-15T18:05:32.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65dd600640 1 -- 172.21.15.143:0/4281614111 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80ab7b0 con 0x7f65e00b24b0 2024-01-15T18:05:32.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65dd600640 1 -- 172.21.15.143:0/4281614111 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80abb40 con 0x7f65e00b24b0 2024-01-15T18:05:32.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65e5a00640 1 -- 172.21.15.143:0/4281614111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b24b0 msgr2=0x7f65e00ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.120+0000 7f65e5a00640 1 --2- 172.21.15.143:0/4281614111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b24b0 0x7f65e00ac8e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f65e015e870 tx=0x7f65c801ce90 comp rx=0 tx=0).stop 2024-01-15T18:05:32.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- 172.21.15.143:0/4281614111 shutdown_connections 2024-01-15T18:05:32.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 --2- 172.21.15.143:0/4281614111 >> v2:172.21.15.204:3300/0 conn(0x7f65e00b3bf0 0x7f65e00b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 --2- 172.21.15.143:0/4281614111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b24b0 0x7f65e00ac8e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- 172.21.15.143:0/4281614111 >> 172.21.15.143:0/4281614111 conn(0x7f65e006c3f0 msgr2=0x7f65e006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:32.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- 172.21.15.143:0/4281614111 shutdown_connections 2024-01-15T18:05:32.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- 172.21.15.143:0/4281614111 wait complete. 2024-01-15T18:05:32.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 Processor -- start 2024-01-15T18:05:32.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- start start 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65e00b24b0 0x7f65e0159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 0x7f65e00af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e00ce690 0x7f65e0142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e01404a0 con 0x7f65e00b3bf0 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e0140130 con 0x7f65e00ce690 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65e5a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e015a330 con 0x7f65e00b24b0 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 0x7f65e00af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 0x7f65e00af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37762/0 (socket says 172.21.15.143:37762) 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 -- 172.21.15.143:0/3891483440 learned_addr learned my addr 172.21.15.143:0/3891483440 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65df400640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e00ce690 0x7f65e0142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65dea00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65e00b24b0 0x7f65e0159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 -- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65e00b24b0 msgr2=0x7f65e0159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65e00b24b0 0x7f65e0159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 -- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e00ce690 msgr2=0x7f65e0142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e00ce690 0x7f65e0142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65de000640 1 -- 172.21.15.143:0/3891483440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009420 con 0x7f65e00b3bf0 2024-01-15T18:05:32.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.124+0000 7f65df400640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e00ce690 0x7f65e0142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:32.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65de000640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 0x7f65e00af2e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ad040 tx=0x7f65c801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80ab650 con 0x7f65e00b3bf0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c80b5830 con 0x7f65e00b3bf0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80b5be0 con 0x7f65e00b3bf0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e00afc10 con 0x7f65e00b3bf0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e00b01d0 con 0x7f65e00b3bf0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e00ae6c0 con 0x7f65e00b3bf0 2024-01-15T18:05:32.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f65c80b5d40 con 0x7f65e00b3bf0 2024-01-15T18:05:32.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65d6a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f65b80668f0 0x7f65b8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65c811f270 con 0x7f65e00b3bf0 2024-01-15T18:05:32.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65dea00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f65b80668f0 0x7f65b8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.128+0000 7f65dea00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f65b80668f0 0x7f65b8068db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65d0005890 tx=0x7f65d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.132+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f65c80ec5c0 con 0x7f65e00b3bf0 2024-01-15T18:05:32.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.284+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65e0067500 con 0x7f65e00b3bf0 2024-01-15T18:05:32.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65d6a00640 1 -- 172.21.15.143:0/3891483440 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f65c80efbe0 con 0x7f65e00b3bf0 2024-01-15T18:05:32.294 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:32.294 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:32.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f65b80668f0 msgr2=0x7f65b8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f65b80668f0 0x7f65b8068db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65d0005890 tx=0x7f65d0009040 comp rx=0 tx=0).stop 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 msgr2=0x7f65e00af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 0x7f65e00af2e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f65c80ad040 tx=0x7f65c801ac70 comp rx=0 tx=0).stop 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 shutdown_connections 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f65b80668f0 0x7f65b8068db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65e00ce690 0x7f65e0142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65e00b3bf0 0x7f65e00af2e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 --2- 172.21.15.143:0/3891483440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f65e00b24b0 0x7f65e0159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 >> 172.21.15.143:0/3891483440 conn(0x7f65e006c3f0 msgr2=0x7f65e015dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 shutdown_connections 2024-01-15T18:05:32.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.288+0000 7f65e5a00640 1 -- 172.21.15.143:0/3891483440 wait complete. 2024-01-15T18:05:32.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:32.317 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:32.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.448+0000 7f481fe00640 1 Processor -- start 2024-01-15T18:05:32.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.448+0000 7f481fe00640 1 -- start start 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.448+0000 7f481fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4818143780 0x7f48180cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.448+0000 7f481fe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180ceb50 con 0x7f48180ce520 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.448+0000 7f481fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481815c110 con 0x7f48180722c0 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.448+0000 7f481fe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481815b7e0 con 0x7f4818143780 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481ca00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f48180ce520 0x7f4818071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51394/0 (socket says 172.21.15.143:51394) 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4818143780 0x7f48180cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481ca00640 1 -- 172.21.15.143:0/2965800150 learned_addr learned my addr 172.21.15.143:0/2965800150 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481d400640 1 -- 172.21.15.143:0/2965800150 >> v1:172.21.15.204:6789/0 conn(0x7f48180722c0 legacy=0x7f481815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481d400640 1 -- 172.21.15.143:0/2965800150 >> v1:172.21.15.143:6789/0 conn(0x7f48180ce520 legacy=0x7f4818071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481d400640 1 -- 172.21.15.143:0/2965800150 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481815bac0 con 0x7f4818143780 2024-01-15T18:05:32.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f4817e00640 1 -- 172.21.15.143:0/2965800150 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058143263 0 0) 0x7f4800001120 con 0x7f48180ce520 2024-01-15T18:05:32.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f481d400640 1 --2- 172.21.15.143:0/2965800150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4818143780 0x7f48180cdfe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48080097c0 tx=0x7f480801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1ca58a53018f4d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f4817e00640 1 -- 172.21.15.143:0/2965800150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080a8050 con 0x7f4818143780 2024-01-15T18:05:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f4817e00640 1 -- 172.21.15.143:0/2965800150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48080ac5f0 con 0x7f4818143780 2024-01-15T18:05:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.452+0000 7f4817e00640 1 -- 172.21.15.143:0/2965800150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080ac980 con 0x7f4818143780 2024-01-15T18:05:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- 172.21.15.143:0/2965800150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4818143780 msgr2=0x7f48180cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 --2- 172.21.15.143:0/2965800150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4818143780 0x7f48180cdfe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48080097c0 tx=0x7f480801c480 comp rx=0 tx=0).stop 2024-01-15T18:05:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- 172.21.15.143:0/2965800150 shutdown_connections 2024-01-15T18:05:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 --2- 172.21.15.143:0/2965800150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4818143780 0x7f48180cdfe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- 172.21.15.143:0/2965800150 >> 172.21.15.143:0/2965800150 conn(0x7f481806c3f0 msgr2=0x7f481806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- 172.21.15.143:0/2965800150 shutdown_connections 2024-01-15T18:05:32.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- 172.21.15.143:0/2965800150 wait complete. 2024-01-15T18:05:32.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 Processor -- start 2024-01-15T18:05:32.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- start start 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 0x7f481813c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48180ce520 0x7f48181413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818143780 0x7f4818141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48181607f0 con 0x7f48180722c0 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180b2810 con 0x7f4818143780 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180b2af0 con 0x7f48180ce520 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 0x7f481813c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 0x7f481813c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37780/0 (socket says 172.21.15.143:37780) 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481d400640 1 -- 172.21.15.143:0/3523137039 learned_addr learned my addr 172.21.15.143:0/3523137039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.456+0000 7f481de00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818143780 0x7f4818141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481ca00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48180ce520 0x7f48181413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481d400640 1 -- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48180ce520 msgr2=0x7f48181413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481d400640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48180ce520 0x7f48181413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481d400640 1 -- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818143780 msgr2=0x7f4818141900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481d400640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818143780 0x7f4818141900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481d400640 1 -- 172.21.15.143:0/3523137039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808009420 con 0x7f48180722c0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481d400640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 0x7f481813c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48080af040 tx=0x7f480801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080b8860 con 0x7f48180722c0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48080b89e0 con 0x7f48180722c0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080b8d90 con 0x7f48180722c0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48180b2dd0 con 0x7f48180722c0 2024-01-15T18:05:32.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48180b1b70 con 0x7f48180722c0 2024-01-15T18:05:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4818160650 con 0x7f48180722c0 2024-01-15T18:05:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f48080c2410 con 0x7f48180722c0 2024-01-15T18:05:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.460+0000 7f4815600640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47f00668f0 0x7f47f0068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.464+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f480811e810 con 0x7f48180722c0 2024-01-15T18:05:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.464+0000 7f481ca00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47f00668f0 0x7f47f0068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.464+0000 7f481ca00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47f00668f0 0x7f47f0068db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4800007e00 tx=0x7f4800007d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.468+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4818160650 con 0x7f48180722c0 2024-01-15T18:05:32.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.616+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48180cc390 con 0x7f48180722c0 2024-01-15T18:05:32.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f4815600640 1 -- 172.21.15.143:0/3523137039 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48180cc390 con 0x7f48180722c0 2024-01-15T18:05:32.625 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47f00668f0 msgr2=0x7f47f0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47f00668f0 0x7f47f0068db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4800007e00 tx=0x7f4800007d90 comp rx=0 tx=0).stop 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 msgr2=0x7f481813c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 0x7f481813c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f48080af040 tx=0x7f480801acc0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 shutdown_connections 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47f00668f0 0x7f47f0068db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4818143780 0x7f4818141900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f48180ce520 0x7f48181413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 --2- 172.21.15.143:0/3523137039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48180722c0 0x7f481813c510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 >> 172.21.15.143:0/3523137039 conn(0x7f481806c3f0 msgr2=0x7f48180aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:32.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 shutdown_connections 2024-01-15T18:05:32.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.620+0000 7f481fe00640 1 -- 172.21.15.143:0/3523137039 wait complete. 2024-01-15T18:05:32.648 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:32.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 Processor -- start 2024-01-15T18:05:32.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 -- start start 2024-01-15T18:05:32.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2bf0143780 0x7f2bf00cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2bf00722c0 0x7f2bf015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf00ceb50 con 0x7f2bf00ce520 2024-01-15T18:05:32.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf015c110 con 0x7f2bf00722c0 2024-01-15T18:05:32.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bf4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf015b7e0 con 0x7f2bf0143780 2024-01-15T18:05:32.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2bf00722c0 0x7f2bf015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2bf00722c0 0x7f2bf015c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37790/0 (socket says 172.21.15.143:37790) 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 -- 172.21.15.143:0/851582809 learned_addr learned my addr 172.21.15.143:0/851582809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bede00640 1 --2- 172.21.15.143:0/851582809 >> v2:172.21.15.204:3300/0 conn(0x7f2bf0143780 0x7f2bf00cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 -- 172.21.15.143:0/851582809 >> v2:172.21.15.204:3300/0 conn(0x7f2bf0143780 msgr2=0x7f2bf00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 --2- 172.21.15.143:0/851582809 >> v2:172.21.15.204:3300/0 conn(0x7f2bf0143780 0x7f2bf00cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 -- 172.21.15.143:0/851582809 >> v1:172.21.15.204:6789/0 conn(0x7f2bf00ce520 legacy=0x7f2bf0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bee800640 1 -- 172.21.15.143:0/851582809 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf015bac0 con 0x7f2bf00722c0 2024-01-15T18:05:32.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.784+0000 7f2bede00640 1 --2- 172.21.15.143:0/851582809 >> v2:172.21.15.204:3300/0 conn(0x7f2bf0143780 0x7f2bf00cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:32.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bee800640 1 --2- 172.21.15.143:0/851582809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf00722c0 0x7f2bf015c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2bf006be30 tx=0x7f2be801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ec60a479ca8da9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2beca00640 1 -- 172.21.15.143:0/851582809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80a57a0 con 0x7f2bf00722c0 2024-01-15T18:05:32.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2beca00640 1 -- 172.21.15.143:0/851582809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be80a5900 con 0x7f2bf00722c0 2024-01-15T18:05:32.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2beca00640 1 -- 172.21.15.143:0/851582809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80a5c90 con 0x7f2bf00722c0 2024-01-15T18:05:32.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 -- 172.21.15.143:0/851582809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf00722c0 msgr2=0x7f2bf015c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/851582809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf00722c0 0x7f2bf015c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2bf006be30 tx=0x7f2be801c9d0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 -- 172.21.15.143:0/851582809 shutdown_connections 2024-01-15T18:05:32.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/851582809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf00722c0 0x7f2bf015c5a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/851582809 >> v2:172.21.15.204:3300/0 conn(0x7f2bf0143780 0x7f2bf00cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 -- 172.21.15.143:0/851582809 >> 172.21.15.143:0/851582809 conn(0x7f2bf006c3f0 msgr2=0x7f2bf006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:32.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 -- 172.21.15.143:0/851582809 shutdown_connections 2024-01-15T18:05:32.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.788+0000 7f2bf4c00640 1 -- 172.21.15.143:0/851582809 wait complete. 2024-01-15T18:05:32.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 Processor -- start 2024-01-15T18:05:32.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 -- start start 2024-01-15T18:05:32.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf00722c0 0x7f2bf0158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf00ce520 0x7f2bf01f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 0x7f2bf00b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf01592c0 con 0x7f2bf0143780 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf013f230 con 0x7f2bf00ce520 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0133850 con 0x7f2bf00722c0 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 0x7f2bf00b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bede00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf00722c0 0x7f2bf0158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bed400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf00ce520 0x7f2bf01f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 0x7f2bf00b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37806/0 (socket says 172.21.15.143:37806) 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 -- 172.21.15.143:0/2703651423 learned_addr learned my addr 172.21.15.143:0/2703651423 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 -- 172.21.15.143:0/2703651423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf00722c0 msgr2=0x7f2bf0158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf00722c0 0x7f2bf0158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 -- 172.21.15.143:0/2703651423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf00ce520 msgr2=0x7f2bf01f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf00ce520 0x7f2bf01f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 -- 172.21.15.143:0/2703651423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8009b30 con 0x7f2bf0143780 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bee800640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 0x7f2bf00b2e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2be80195e0 tx=0x7f2be8019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80b8670 con 0x7f2bf0143780 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be80b87d0 con 0x7f2bf0143780 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be80b8b60 con 0x7f2bf0143780 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf0133b30 con 0x7f2bf0143780 2024-01-15T18:05:32.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.792+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf01340f0 con 0x7f2bf0143780 2024-01-15T18:05:32.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.796+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf0067500 con 0x7f2bf0143780 2024-01-15T18:05:32.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.796+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f2be80b8cc0 con 0x7f2bf0143780 2024-01-15T18:05:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.796+0000 7f2be6000640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc80668f0 0x7f2bc8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.796+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2be811f7c0 con 0x7f2bf0143780 2024-01-15T18:05:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.796+0000 7f2bede00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc80668f0 0x7f2bc8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:32.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.796+0000 7f2bede00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc80668f0 0x7f2bc8068db0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00af5f0 tx=0x7f2be0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:32.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.800+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2be80eca10 con 0x7f2bf0143780 2024-01-15T18:05:32.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.948+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bf015c610 con 0x7f2bf0143780 2024-01-15T18:05:32.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2be6000640 1 -- 172.21.15.143:0/2703651423 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2be80f0030 con 0x7f2bf0143780 2024-01-15T18:05:32.957 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc80668f0 msgr2=0x7f2bc8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc80668f0 0x7f2bc8068db0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00af5f0 tx=0x7f2be0008040 comp rx=0 tx=0).stop 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 msgr2=0x7f2bf00b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 0x7f2bf00b2e30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2be80195e0 tx=0x7f2be8019770 comp rx=0 tx=0).stop 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 shutdown_connections 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc80668f0 0x7f2bc8068db0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bf0143780 0x7f2bf00b2e30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bf00ce520 0x7f2bf01f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 --2- 172.21.15.143:0/2703651423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bf00722c0 0x7f2bf0158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:32.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 >> 172.21.15.143:0/2703651423 conn(0x7f2bf006c3f0 msgr2=0x7f2bf013ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:32.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 shutdown_connections 2024-01-15T18:05:32.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:32.952+0000 7f2bf4c00640 1 -- 172.21.15.143:0/2703651423 wait complete. 2024-01-15T18:05:32.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:33.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 Processor -- start 2024-01-15T18:05:33.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 -- start start 2024-01-15T18:05:33.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7bc806c540 0x7f7bc80cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc80cf150 0x7f7bc80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bc806b7b0 0x7f7bc815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc80d02d0 con 0x7f7bc806b7b0 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc80cf6c0 con 0x7f7bc806c540 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcfc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80cf9a0 con 0x7f7bc80cf150 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bc806b7b0 0x7f7bc815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7bc806b7b0 0x7f7bc815ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37812/0 (socket says 172.21.15.143:37812) 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 -- 172.21.15.143:0/84827323 learned_addr learned my addr 172.21.15.143:0/84827323 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bc7e00640 1 --2- 172.21.15.143:0/84827323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc80cf150 0x7f7bc80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcd200640 1 --2- 172.21.15.143:0/84827323 >> v2:172.21.15.204:3300/0 conn(0x7f7bc806c540 0x7f7bc80cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 -- 172.21.15.143:0/84827323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc80cf150 msgr2=0x7f7bc80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:33.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 --2- 172.21.15.143:0/84827323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc80cf150 0x7f7bc80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 -- 172.21.15.143:0/84827323 >> v2:172.21.15.204:3300/0 conn(0x7f7bc806c540 msgr2=0x7f7bc80cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:33.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 --2- 172.21.15.143:0/84827323 >> v2:172.21.15.204:3300/0 conn(0x7f7bc806c540 0x7f7bc80cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcdc00640 1 -- 172.21.15.143:0/84827323 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80cfc80 con 0x7f7bc806b7b0 2024-01-15T18:05:33.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.112+0000 7f7bcd200640 1 --2- 172.21.15.143:0/84827323 >> v2:172.21.15.204:3300/0 conn(0x7f7bc806c540 0x7f7bc80cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:33.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcdc00640 1 --2- 172.21.15.143:0/84827323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc815ca90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bc806be70 tx=0x7f7bc001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc53a08efb5908d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bc7400640 1 -- 172.21.15.143:0/84827323 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc806cb30 con 0x7f7bc806b7b0 2024-01-15T18:05:33.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bc7400640 1 -- 172.21.15.143:0/84827323 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc00a57b0 con 0x7f7bc806b7b0 2024-01-15T18:05:33.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bc7400640 1 -- 172.21.15.143:0/84827323 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00a5b40 con 0x7f7bc806b7b0 2024-01-15T18:05:33.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 -- 172.21.15.143:0/84827323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 msgr2=0x7f7bc815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/84827323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc815ca90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bc806be70 tx=0x7f7bc001c910 comp rx=0 tx=0).stop 2024-01-15T18:05:33.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 -- 172.21.15.143:0/84827323 shutdown_connections 2024-01-15T18:05:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/84827323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc815ca90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/84827323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc80cf150 0x7f7bc80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/84827323 >> v2:172.21.15.204:3300/0 conn(0x7f7bc806c540 0x7f7bc80cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 -- 172.21.15.143:0/84827323 >> 172.21.15.143:0/84827323 conn(0x7f7bc80d0e20 msgr2=0x7f7bc80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 -- 172.21.15.143:0/84827323 shutdown_connections 2024-01-15T18:05:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.116+0000 7f7bcfc00640 1 -- 172.21.15.143:0/84827323 wait complete. 2024-01-15T18:05:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 Processor -- start 2024-01-15T18:05:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 -- start start 2024-01-15T18:05:33.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc8153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc806c540 0x7f7bc81550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc80cf150 0x7f7bc815f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8155600 con 0x7f7bc806b7b0 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc81600b0 con 0x7f7bc80cf150 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8160390 con 0x7f7bc806c540 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc8153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc8153430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37816/0 (socket says 172.21.15.143:37816) 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 -- 172.21.15.143:0/4087709279 learned_addr learned my addr 172.21.15.143:0/4087709279 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcdc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc80cf150 0x7f7bc815f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bc7e00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc806c540 0x7f7bc81550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 -- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc806c540 msgr2=0x7f7bc81550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc806c540 0x7f7bc81550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 -- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc80cf150 msgr2=0x7f7bc815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc80cf150 0x7f7bc815f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 -- 172.21.15.143:0/4087709279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0009b30 con 0x7f7bc806b7b0 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcdc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc80cf150 0x7f7bc815f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bc7e00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc806c540 0x7f7bc81550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcd200640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc8153430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e810 tx=0x7f7bb800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800c8d0 con 0x7f7bc806b7b0 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb800ca30 con 0x7f7bc806b7b0 2024-01-15T18:05:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb800cde0 con 0x7f7bc806b7b0 2024-01-15T18:05:33.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc8150f80 con 0x7f7bc806b7b0 2024-01-15T18:05:33.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.120+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc8160670 con 0x7f7bc806b7b0 2024-01-15T18:05:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.124+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc8067500 con 0x7f7bc806b7b0 2024-01-15T18:05:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.124+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f7bb8013410 con 0x7f7bc806b7b0 2024-01-15T18:05:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.124+0000 7f7bc4c00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7ba00668f0 0x7f7ba0068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.124+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bb8084200 con 0x7f7bc806b7b0 2024-01-15T18:05:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.124+0000 7f7bc7e00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7ba00668f0 0x7f7ba0068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.124+0000 7f7bc7e00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7ba00668f0 0x7f7ba0068db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8156040 tx=0x7f7bb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.128+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7bb8051440 con 0x7f7bc806b7b0 2024-01-15T18:05:33.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.276+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7bc80cb1e0 con 0x7f7bc806b7b0 2024-01-15T18:05:33.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.276+0000 7f7bc4c00640 1 -- 172.21.15.143:0/4087709279 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7bb8054a60 con 0x7f7bc806b7b0 2024-01-15T18:05:33.284 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7ba00668f0 msgr2=0x7f7ba0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7ba00668f0 0x7f7ba0068db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8156040 tx=0x7f7bb0008040 comp rx=0 tx=0).stop 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 msgr2=0x7f7bc8153430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc8153430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800e810 tx=0x7f7bb800ecd0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 shutdown_connections 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7ba00668f0 0x7f7ba0068db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bc80cf150 0x7f7bc815f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7bc806c540 0x7f7bc81550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 --2- 172.21.15.143:0/4087709279 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7bc806b7b0 0x7f7bc8153430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 >> 172.21.15.143:0/4087709279 conn(0x7f7bc80d0e20 msgr2=0x7f7bc80cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:33.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 shutdown_connections 2024-01-15T18:05:33.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.280+0000 7f7bcfc00640 1 -- 172.21.15.143:0/4087709279 wait complete. 2024-01-15T18:05:33.307 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:33.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 Processor -- start 2024-01-15T18:05:33.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 -- start start 2024-01-15T18:05:33.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f535c143780 0x7f535c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c0722c0 0x7f535c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0ceb50 con 0x7f535c0ce520 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535c15c110 con 0x7f535c143780 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f5362800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15b7e0 con 0x7f535c0722c0 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f535aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f535c0ce520 0x7f535c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51396/0 (socket says 172.21.15.143:51396) 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f535b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f535c143780 0x7f535c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.436+0000 7f535aa00640 1 -- 172.21.15.143:0/1733230093 learned_addr learned my addr 172.21.15.143:0/1733230093 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535be00640 1 --2- 172.21.15.143:0/1733230093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c0722c0 0x7f535c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535b400640 1 -- 172.21.15.143:0/1733230093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c0722c0 msgr2=0x7f535c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535b400640 1 --2- 172.21.15.143:0/1733230093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c0722c0 0x7f535c15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535b400640 1 -- 172.21.15.143:0/1733230093 >> v1:172.21.15.143:6789/0 conn(0x7f535c0ce520 legacy=0x7f535c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535b400640 1 -- 172.21.15.143:0/1733230093 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c15bac0 con 0x7f535c143780 2024-01-15T18:05:33.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535b400640 1 --2- 172.21.15.143:0/1733230093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c143780 0x7f535c0cdfe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f534c0097c0 tx=0x7f534c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cb2077c8558631 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535a000640 1 -- 172.21.15.143:0/1733230093 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0ad640 con 0x7f535c143780 2024-01-15T18:05:33.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535a000640 1 -- 172.21.15.143:0/1733230093 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c0ad7a0 con 0x7f535c143780 2024-01-15T18:05:33.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.440+0000 7f535a000640 1 -- 172.21.15.143:0/1733230093 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0adb30 con 0x7f535c143780 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- 172.21.15.143:0/1733230093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c143780 msgr2=0x7f535c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 --2- 172.21.15.143:0/1733230093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c143780 0x7f535c0cdfe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f534c0097c0 tx=0x7f534c01c480 comp rx=0 tx=0).stop 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- 172.21.15.143:0/1733230093 shutdown_connections 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 --2- 172.21.15.143:0/1733230093 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c0722c0 0x7f535c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 --2- 172.21.15.143:0/1733230093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c143780 0x7f535c0cdfe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- 172.21.15.143:0/1733230093 >> 172.21.15.143:0/1733230093 conn(0x7f535c06c3f0 msgr2=0x7f535c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:33.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- 172.21.15.143:0/1733230093 shutdown_connections 2024-01-15T18:05:33.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- 172.21.15.143:0/1733230093 wait complete. 2024-01-15T18:05:33.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 Processor -- start 2024-01-15T18:05:33.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- start start 2024-01-15T18:05:33.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f535c0722c0 0x7f535c159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 0x7f535c13b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c143780 0x7f535c0ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c13bfe0 con 0x7f535c0ce520 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c13c2c0 con 0x7f535c0722c0 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f5362800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c13c5a0 con 0x7f535c143780 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f535aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 0x7f535c13b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f535aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 0x7f535c13b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37834/0 (socket says 172.21.15.143:37834) 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.444+0000 7f535aa00640 1 -- 172.21.15.143:0/3371882508 learned_addr learned my addr 172.21.15.143:0/3371882508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:33.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535be00640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c143780 0x7f535c0ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535b400640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f535c0722c0 0x7f535c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535aa00640 1 -- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c143780 msgr2=0x7f535c0ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535aa00640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c143780 0x7f535c0ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535aa00640 1 -- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f535c0722c0 msgr2=0x7f535c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535aa00640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f535c0722c0 0x7f535c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535aa00640 1 -- 172.21.15.143:0/3371882508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009420 con 0x7f535c0ce520 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f535aa00640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 0x7f535c13b6e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f534400c500 tx=0x7f534400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344005cf0 con 0x7f535c0ce520 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344005e50 con 0x7f535c0ce520 2024-01-15T18:05:33.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c13c880 con 0x7f535c0ce520 2024-01-15T18:05:33.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53440072f0 con 0x7f535c0ce520 2024-01-15T18:05:33.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c13bc20 con 0x7f535c0ce520 2024-01-15T18:05:33.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c13e810 con 0x7f535c0ce520 2024-01-15T18:05:33.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5344007450 con 0x7f535c0ce520 2024-01-15T18:05:33.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5353400640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5334066920 0x7f5334068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.448+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53440836c0 con 0x7f535c0ce520 2024-01-15T18:05:33.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.452+0000 7f535b400640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5334066920 0x7f5334068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.452+0000 7f535b400640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5334066920 0x7f5334068de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f534c0af000 tx=0x7f534c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.456+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f535c13e810 con 0x7f535c0ce520 2024-01-15T18:05:33.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.612+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f535c0ceb50 con 0x7f535c0ce520 2024-01-15T18:05:33.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5353400640 1 -- 172.21.15.143:0/3371882508 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f535c0ceb50 con 0x7f535c0ce520 2024-01-15T18:05:33.621 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5334066920 msgr2=0x7f5334068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5334066920 0x7f5334068de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f534c0af000 tx=0x7f534c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 msgr2=0x7f535c13b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 0x7f535c13b6e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f534400c500 tx=0x7f534400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 shutdown_connections 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5334066920 0x7f5334068de0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f535c143780 0x7f535c0ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f535c0ce520 0x7f535c13b6e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 --2- 172.21.15.143:0/3371882508 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f535c0722c0 0x7f535c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 >> 172.21.15.143:0/3371882508 conn(0x7f535c06c3f0 msgr2=0x7f535c13a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:33.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 shutdown_connections 2024-01-15T18:05:33.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.616+0000 7f5362800640 1 -- 172.21.15.143:0/3371882508 wait complete. 2024-01-15T18:05:33.644 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:05:33.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:33.683 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:05:33.718 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:33.719 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:05:33.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:33.719 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:05:33.780 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:05:33.781 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:33.791 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:05:33.791 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:33.841 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.968+0000 7fa000600640 1 Processor -- start 2024-01-15T18:05:33.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.968+0000 7fa000600640 1 -- start start 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7fa000600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80ce690 0x7f9ff80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7fa000600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9ff80b1da0 0x7f9ff80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7fa000600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff8070bf0 con 0x7f9ff80b3bf0 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7fa000600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff815dba0 con 0x7f9ff80b1da0 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7fa000600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80b2740 con 0x7f9ff80ce690 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffd200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9ff80b1da0 0x7f9ff80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffe600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9ff80b3bf0 0x7f9ff80b4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51402/0 (socket says 172.21.15.143:51402) 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffe600640 1 -- 172.21.15.143:0/2630222701 learned_addr learned my addr 172.21.15.143:0/2630222701 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffdc00640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80ce690 0x7f9ff80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffd200640 1 -- 172.21.15.143:0/2630222701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80ce690 msgr2=0x7f9ff80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffd200640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80ce690 0x7f9ff80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffd200640 1 -- 172.21.15.143:0/2630222701 >> v1:172.21.15.143:6789/0 conn(0x7f9ff80b3bf0 legacy=0x7f9ff80b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffd200640 1 -- 172.21.15.143:0/2630222701 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80b4920 con 0x7f9ff80b1da0 2024-01-15T18:05:33.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ff7e00640 1 -- 172.21.15.143:0/2630222701 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244590275 0 0) 0x7f9ff8070bf0 con 0x7f9ff80b3bf0 2024-01-15T18:05:33.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.972+0000 7f9ffdc00640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80ce690 0x7f9ff80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:33.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7f9ffd200640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80b1da0 0x7f9ff80b2200 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00097c0 tx=0x7f9fe001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cb915160367b3c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7f9ff7e00640 1 -- 172.21.15.143:0/2630222701 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00ab650 con 0x7f9ff80b1da0 2024-01-15T18:05:33.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7f9ff7e00640 1 -- 172.21.15.143:0/2630222701 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe00ab7b0 con 0x7f9ff80b1da0 2024-01-15T18:05:33.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7f9ff7e00640 1 -- 172.21.15.143:0/2630222701 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00abb40 con 0x7f9ff80b1da0 2024-01-15T18:05:33.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 -- 172.21.15.143:0/2630222701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80b1da0 msgr2=0x7f9ff80b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80b1da0 0x7f9ff80b2200 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00097c0 tx=0x7f9fe001c450 comp rx=0 tx=0).stop 2024-01-15T18:05:33.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 -- 172.21.15.143:0/2630222701 shutdown_connections 2024-01-15T18:05:33.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80b1da0 0x7f9ff80b2200 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 --2- 172.21.15.143:0/2630222701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80ce690 0x7f9ff80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 -- 172.21.15.143:0/2630222701 >> 172.21.15.143:0/2630222701 conn(0x7f9ff806c3f0 msgr2=0x7f9ff806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:33.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 -- 172.21.15.143:0/2630222701 shutdown_connections 2024-01-15T18:05:33.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 -- 172.21.15.143:0/2630222701 wait complete. 2024-01-15T18:05:33.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.976+0000 7fa000600640 1 Processor -- start 2024-01-15T18:05:33.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- start start 2024-01-15T18:05:33.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff80b1da0 0x7f9ff813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80b3bf0 0x7f9ff80aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 0x7f9ff8140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff813fb10 con 0x7f9ff80ce690 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80ae150 con 0x7f9ff80b1da0 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80ad8b0 con 0x7f9ff80b3bf0 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 0x7f9ff8140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 0x7f9ff8140050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37842/0 (socket says 172.21.15.143:37842) 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 -- 172.21.15.143:0/3984271590 learned_addr learned my addr 172.21.15.143:0/3984271590 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffd200640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80b3bf0 0x7f9ff80aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffdc00640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff80b1da0 0x7f9ff813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 -- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80b3bf0 msgr2=0x7f9ff80aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80b3bf0 0x7f9ff80aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 -- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff80b1da0 msgr2=0x7f9ff813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:33.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff80b1da0 0x7f9ff813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 -- 172.21.15.143:0/3984271590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0009420 con 0x7f9ff80ce690 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffdc00640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff80b1da0 0x7f9ff813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ffe600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 0x7f9ff8140050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0001570 tx=0x7f9ff000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000cea0 con 0x7f9ff80ce690 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff000a5e0 con 0x7f9ff80ce690 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000a990 con 0x7f9ff80ce690 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff80adb90 con 0x7f9ff80ce690 2024-01-15T18:05:33.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff81f6020 con 0x7f9ff80ce690 2024-01-15T18:05:33.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.980+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff81f6210 con 0x7f9ff80ce690 2024-01-15T18:05:33.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.984+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f9ff000aaf0 con 0x7f9ff80ce690 2024-01-15T18:05:33.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.984+0000 7f9ff5600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fd00668f0 0x7f9fd0068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:33.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.984+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ff0083880 con 0x7f9ff80ce690 2024-01-15T18:05:33.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.984+0000 7f9ffdc00640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fd00668f0 0x7f9fd0068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:33.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.984+0000 7f9ffdc00640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fd00668f0 0x7f9fd0068db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800a9b0 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:33.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:33.988+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9ff0050b70 con 0x7f9ff80ce690 2024-01-15T18:05:34.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.140+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ff8067500 con 0x7f9ff80ce690 2024-01-15T18:05:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.140+0000 7f9ff5600640 1 -- 172.21.15.143:0/3984271590 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f9ff0018070 con 0x7f9ff80ce690 2024-01-15T18:05:34.147 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:34.148 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fd00668f0 msgr2=0x7f9fd0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fd00668f0 0x7f9fd0068db0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9fe800a9b0 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 msgr2=0x7f9ff8140050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 0x7f9ff8140050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0001570 tx=0x7f9ff000cc50 comp rx=0 tx=0).stop 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 shutdown_connections 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fd00668f0 0x7f9fd0068db0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ff80ce690 0x7f9ff8140050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9ff80b3bf0 0x7f9ff80aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 --2- 172.21.15.143:0/3984271590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ff80b1da0 0x7f9ff813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 >> 172.21.15.143:0/3984271590 conn(0x7f9ff806c3f0 msgr2=0x7f9ff815a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 shutdown_connections 2024-01-15T18:05:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.144+0000 7fa000600640 1 -- 172.21.15.143:0/3984271590 wait complete. 2024-01-15T18:05:34.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:34.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:34.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.300+0000 7f5030800640 1 Processor -- start 2024-01-15T18:05:34.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.300+0000 7f5030800640 1 -- start start 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f5030800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5028157300 0x7f5028159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f5030800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50280b4ac0 0x7f5028070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f5030800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50280b2a20 con 0x7f50280b21d0 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f5030800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280b35a0 con 0x7f50280b4ac0 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f5030800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280b2d00 con 0x7f5028157300 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50280b4ac0 0x7f5028070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50280b4ac0 0x7f5028070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37856/0 (socket says 172.21.15.143:37856) 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 -- 172.21.15.143:0/428223349 learned_addr learned my addr 172.21.15.143:0/428223349 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502de00640 1 --2- 172.21.15.143:0/428223349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5028157300 0x7f5028159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 -- 172.21.15.143:0/428223349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5028157300 msgr2=0x7f5028159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:34.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 --2- 172.21.15.143:0/428223349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5028157300 0x7f5028159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 -- 172.21.15.143:0/428223349 >> v1:172.21.15.204:6789/0 conn(0x7f50280b21d0 legacy=0x7f50280b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:34.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 -- 172.21.15.143:0/428223349 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280b2fe0 con 0x7f50280b4ac0 2024-01-15T18:05:34.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.304+0000 7f502e800640 1 --2- 172.21.15.143:0/428223349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f5028070e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5020009e50 tx=0x7f502001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602349b4d8ec5688 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f502ca00640 1 -- 172.21.15.143:0/428223349 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200a5840 con 0x7f50280b4ac0 2024-01-15T18:05:34.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f502ca00640 1 -- 172.21.15.143:0/428223349 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50200a59a0 con 0x7f50280b4ac0 2024-01-15T18:05:34.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f502ca00640 1 -- 172.21.15.143:0/428223349 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200a5d30 con 0x7f50280b4ac0 2024-01-15T18:05:34.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 -- 172.21.15.143:0/428223349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 msgr2=0x7f5028070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 --2- 172.21.15.143:0/428223349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f5028070e80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5020009e50 tx=0x7f502001c9a0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 -- 172.21.15.143:0/428223349 shutdown_connections 2024-01-15T18:05:34.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 --2- 172.21.15.143:0/428223349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f5028070e80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 --2- 172.21.15.143:0/428223349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5028157300 0x7f5028159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 -- 172.21.15.143:0/428223349 >> 172.21.15.143:0/428223349 conn(0x7f502806c3f0 msgr2=0x7f502806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:34.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 -- 172.21.15.143:0/428223349 shutdown_connections 2024-01-15T18:05:34.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 -- 172.21.15.143:0/428223349 wait complete. 2024-01-15T18:05:34.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 Processor -- start 2024-01-15T18:05:34.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.308+0000 7f5030800640 1 -- start start 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50280b21d0 0x7f5028143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f50281509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5028157300 0x7f5028150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5028156e50 con 0x7f50280b4ac0 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5028151420 con 0x7f5028157300 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5028158e80 con 0x7f50280b21d0 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f50281509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50280b21d0 0x7f5028143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f50281509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37862/0 (socket says 172.21.15.143:37862) 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 -- 172.21.15.143:0/126592505 learned_addr learned my addr 172.21.15.143:0/126592505 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:34.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502e800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5028157300 0x7f5028150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 -- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50280b21d0 msgr2=0x7f5028143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50280b21d0 0x7f5028143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 -- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5028157300 msgr2=0x7f5028150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5028157300 0x7f5028150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 -- 172.21.15.143:0/126592505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020009b30 con 0x7f50280b4ac0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f502d400640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f50281509a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f501000c450 tx=0x7f501000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010012830 con 0x7f50280b4ac0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100129b0 con 0x7f50280b4ac0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010012d60 con 0x7f50280b4ac0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5028147710 con 0x7f50280b4ac0 2024-01-15T18:05:34.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50281593b0 con 0x7f50280b4ac0 2024-01-15T18:05:34.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50281595d0 con 0x7f50280b4ac0 2024-01-15T18:05:34.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.312+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f501001c460 con 0x7f50280b4ac0 2024-01-15T18:05:34.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.316+0000 7f5026000640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50000668f0 0x7f5000068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.316+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50100841e0 con 0x7f50280b4ac0 2024-01-15T18:05:34.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.316+0000 7f502de00640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50000668f0 0x7f5000068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.316+0000 7f502de00640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50000668f0 0x7f5000068db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f50180079c0 tx=0x7f5018008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.320+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50281595d0 con 0x7f50280b4ac0 2024-01-15T18:05:34.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50281595d0 con 0x7f50280b4ac0 2024-01-15T18:05:34.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5026000640 1 -- 172.21.15.143:0/126592505 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50281595d0 con 0x7f50280b4ac0 2024-01-15T18:05:34.474 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50000668f0 msgr2=0x7f5000068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50000668f0 0x7f5000068db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f50180079c0 tx=0x7f5018008040 comp rx=0 tx=0).stop 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 msgr2=0x7f50281509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f50281509a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f501000c450 tx=0x7f501000c910 comp rx=0 tx=0).stop 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 shutdown_connections 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50000668f0 0x7f5000068db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5028157300 0x7f5028150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50280b4ac0 0x7f50281509a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 --2- 172.21.15.143:0/126592505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50280b21d0 0x7f5028143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 >> 172.21.15.143:0/126592505 conn(0x7f502806c3f0 msgr2=0x7f50280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 shutdown_connections 2024-01-15T18:05:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.468+0000 7f5030800640 1 -- 172.21.15.143:0/126592505 wait complete. 2024-01-15T18:05:34.495 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:34.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 Processor -- start 2024-01-15T18:05:34.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 -- start start 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6dc15ec80 0x7fa6dc141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa6dc142470 0x7fa6dc071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc15fbb0 con 0x7fa6dc15ec80 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc15f100 con 0x7fa6dc142470 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e4000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc15f310 con 0x7fa6dc141660 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6dc15ec80 0x7fa6dc141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6dc15ec80 0x7fa6dc141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37870/0 (socket says 172.21.15.143:37870) 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1794655603 learned_addr learned my addr 172.21.15.143:0/1794655603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:34.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e2000640 1 --2- 172.21.15.143:0/1794655603 >> v2:172.21.15.204:3300/0 conn(0x7fa6dc142470 0x7fa6dc071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e1600640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1794655603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 msgr2=0x7fa6dc0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1794655603 >> v2:172.21.15.204:3300/0 conn(0x7fa6dc142470 msgr2=0x7fa6dc071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 --2- 172.21.15.143:0/1794655603 >> v2:172.21.15.204:3300/0 conn(0x7fa6dc142470 0x7fa6dc071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1794655603 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc15f5f0 con 0x7fa6dc15ec80 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e1600640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:34.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.628+0000 7fa6e2000640 1 --2- 172.21.15.143:0/1794655603 >> v2:172.21.15.204:3300/0 conn(0x7fa6dc142470 0x7fa6dc071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:34.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e0c00640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc141f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc06ffd0 tx=0x7fa6c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c85e4a6a2d7322 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6dbe00640 1 -- 172.21.15.143:0/1794655603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40ac600 con 0x7fa6dc15ec80 2024-01-15T18:05:34.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6dbe00640 1 -- 172.21.15.143:0/1794655603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40a8050 con 0x7fa6dc15ec80 2024-01-15T18:05:34.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6dbe00640 1 -- 172.21.15.143:0/1794655603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40ac980 con 0x7fa6dc15ec80 2024-01-15T18:05:34.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 -- 172.21.15.143:0/1794655603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc15ec80 msgr2=0x7fa6dc141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc141f30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc06ffd0 tx=0x7fa6c401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 -- 172.21.15.143:0/1794655603 shutdown_connections 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1794655603 >> v2:172.21.15.204:3300/0 conn(0x7fa6dc142470 0x7fa6dc071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc141f30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1794655603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 -- 172.21.15.143:0/1794655603 >> 172.21.15.143:0/1794655603 conn(0x7fa6dc06c3f0 msgr2=0x7fa6dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 -- 172.21.15.143:0/1794655603 shutdown_connections 2024-01-15T18:05:34.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.632+0000 7fa6e4000640 1 -- 172.21.15.143:0/1794655603 wait complete. 2024-01-15T18:05:34.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 Processor -- start 2024-01-15T18:05:34.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 -- start start 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 0x7fa6dc13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc137800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc13de20 con 0x7fa6dc142470 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc13c530 con 0x7fa6dc15ec80 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc13d1a0 con 0x7fa6dc141660 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 0x7fa6dc13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 0x7fa6dc13bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37882/0 (socket says 172.21.15.143:37882) 2024-01-15T18:05:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1862127481 learned_addr learned my addr 172.21.15.143:0/1862127481 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e1600640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e2000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc137800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 msgr2=0x7fa6dc13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc15ec80 msgr2=0x7fa6dc137800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc137800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 -- 172.21.15.143:0/1862127481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40094d0 con 0x7fa6dc142470 2024-01-15T18:05:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e0c00640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 0x7fa6dc13bff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6c401cd70 tx=0x7fa6c401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40b7650 con 0x7fa6dc142470 2024-01-15T18:05:34.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40b77d0 con 0x7fa6dc142470 2024-01-15T18:05:34.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40b7b80 con 0x7fa6dc142470 2024-01-15T18:05:34.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc0cd150 con 0x7fa6dc142470 2024-01-15T18:05:34.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.636+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc13d760 con 0x7fa6dc142470 2024-01-15T18:05:34.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.640+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc13df60 con 0x7fa6dc142470 2024-01-15T18:05:34.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.640+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fa6c40b3070 con 0x7fa6dc142470 2024-01-15T18:05:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.640+0000 7fa6d9600640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa6b40668f0 0x7fa6b4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.640+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6c411e840 con 0x7fa6dc142470 2024-01-15T18:05:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.640+0000 7fa6e1600640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa6b40668f0 0x7fa6b4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.640+0000 7fa6e1600640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa6b40668f0 0x7fa6b4068db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0098d0 tx=0x7fa6cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.644+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6dc13df60 con 0x7fa6dc142470 2024-01-15T18:05:34.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.796+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6dc13df60 con 0x7fa6dc142470 2024-01-15T18:05:34.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.796+0000 7fa6d9600640 1 -- 172.21.15.143:0/1862127481 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6dc13df60 con 0x7fa6dc142470 2024-01-15T18:05:34.805 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa6b40668f0 msgr2=0x7fa6b4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa6b40668f0 0x7fa6b4068db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0098d0 tx=0x7fa6cc008040 comp rx=0 tx=0).stop 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 msgr2=0x7fa6dc13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 0x7fa6dc13bff0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa6c401cd70 tx=0x7fa6c401ad10 comp rx=0 tx=0).stop 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 shutdown_connections 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa6b40668f0 0x7fa6b4068db0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6dc15ec80 0x7fa6dc137800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6dc142470 0x7fa6dc13bff0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 --2- 172.21.15.143:0/1862127481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6dc141660 0x7fa6dc13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 >> 172.21.15.143:0/1862127481 conn(0x7fa6dc06c3f0 msgr2=0x7fa6dc143460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:34.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 shutdown_connections 2024-01-15T18:05:34.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.800+0000 7fa6e4000640 1 -- 172.21.15.143:0/1862127481 wait complete. 2024-01-15T18:05:34.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.956+0000 7f9091800640 1 Processor -- start 2024-01-15T18:05:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.956+0000 7f9091800640 1 -- start start 2024-01-15T18:05:34.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f9091800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f908c0ce690 0x7f908c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f9091800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f908c0b3bf0 0x7f908c0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f9091800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c070bf0 con 0x7f908c0b1da0 2024-01-15T18:05:34.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f9091800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c15dba0 con 0x7f908c0ce690 2024-01-15T18:05:34.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f9091800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908c0b4920 con 0x7f908c0b3bf0 2024-01-15T18:05:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f908c0b1da0 0x7f908c0b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51412/0 (socket says 172.21.15.143:51412) 2024-01-15T18:05:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908a000640 1 -- 172.21.15.143:0/745546364 learned_addr learned my addr 172.21.15.143:0/745546364 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908aa00640 1 --2- 172.21.15.143:0/745546364 >> v2:172.21.15.143:3300/0 conn(0x7f908c0ce690 0x7f908c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908b400640 1 --2- 172.21.15.143:0/745546364 >> v2:172.21.15.204:3300/0 conn(0x7f908c0b3bf0 0x7f908c0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908aa00640 1 -- 172.21.15.143:0/745546364 >> v2:172.21.15.204:3300/0 conn(0x7f908c0b3bf0 msgr2=0x7f908c0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908aa00640 1 --2- 172.21.15.143:0/745546364 >> v2:172.21.15.204:3300/0 conn(0x7f908c0b3bf0 0x7f908c0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908aa00640 1 -- 172.21.15.143:0/745546364 >> v1:172.21.15.143:6789/0 conn(0x7f908c0b1da0 legacy=0x7f908c0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f9089600640 1 -- 172.21.15.143:0/745546364 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525399426 0 0) 0x7f908c070bf0 con 0x7f908c0b1da0 2024-01-15T18:05:34.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908aa00640 1 -- 172.21.15.143:0/745546364 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c0b4c00 con 0x7f908c0ce690 2024-01-15T18:05:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.960+0000 7f908aa00640 1 --2- 172.21.15.143:0/745546364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0ce690 0x7f908c0ac8e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f908c15e870 tx=0x7f907c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8ac93d65195544 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9089600640 1 -- 172.21.15.143:0/745546364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0ab650 con 0x7f908c0ce690 2024-01-15T18:05:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9089600640 1 -- 172.21.15.143:0/745546364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c0ab7b0 con 0x7f908c0ce690 2024-01-15T18:05:34.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9089600640 1 -- 172.21.15.143:0/745546364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0abb40 con 0x7f908c0ce690 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 -- 172.21.15.143:0/745546364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0ce690 msgr2=0x7f908c0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 --2- 172.21.15.143:0/745546364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0ce690 0x7f908c0ac8e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f908c15e870 tx=0x7f907c01ce90 comp rx=0 tx=0).stop 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 -- 172.21.15.143:0/745546364 shutdown_connections 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 --2- 172.21.15.143:0/745546364 >> v2:172.21.15.204:3300/0 conn(0x7f908c0b3bf0 0x7f908c0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 --2- 172.21.15.143:0/745546364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0ce690 0x7f908c0ac8e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 -- 172.21.15.143:0/745546364 >> 172.21.15.143:0/745546364 conn(0x7f908c06c3f0 msgr2=0x7f908c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 -- 172.21.15.143:0/745546364 shutdown_connections 2024-01-15T18:05:34.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 -- 172.21.15.143:0/745546364 wait complete. 2024-01-15T18:05:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 Processor -- start 2024-01-15T18:05:34.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 -- start start 2024-01-15T18:05:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 0x7f908c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.964+0000 7f9091800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f908c0b3bf0 0x7f908c140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c0ce690 0x7f908c13fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c15af70 con 0x7f908c0b1da0 2024-01-15T18:05:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c13eec0 con 0x7f908c0b3bf0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c13f5c0 con 0x7f908c0ce690 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 0x7f908c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 0x7f908c15aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37900/0 (socket says 172.21.15.143:37900) 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 -- 172.21.15.143:0/382682231 learned_addr learned my addr 172.21.15.143:0/382682231 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908a000640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f908c0b3bf0 0x7f908c140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908b400640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c0ce690 0x7f908c13fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 -- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c0ce690 msgr2=0x7f908c13fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c0ce690 0x7f908c13fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 -- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f908c0b3bf0 msgr2=0x7f908c140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f908c0b3bf0 0x7f908c140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 -- 172.21.15.143:0/382682231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c009420 con 0x7f908c0b1da0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908b400640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c0ce690 0x7f908c13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f908aa00640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 0x7f908c15aa30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f907c0ad040 tx=0x7f907c01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b5810 con 0x7f908c0b1da0 2024-01-15T18:05:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c0b5990 con 0x7f908c0b1da0 2024-01-15T18:05:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0b5d40 con 0x7f908c0b1da0 2024-01-15T18:05:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c13daa0 con 0x7f908c0b1da0 2024-01-15T18:05:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c13e060 con 0x7f908c0b1da0 2024-01-15T18:05:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c13d900 con 0x7f908c0b1da0 2024-01-15T18:05:34.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.968+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f907c0bf410 con 0x7f908c0b1da0 2024-01-15T18:05:34.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.972+0000 7f9082a00640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90640668f0 0x7f9064068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:34.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.972+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f907c11f070 con 0x7f908c0b1da0 2024-01-15T18:05:34.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.972+0000 7f908a000640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90640668f0 0x7f9064068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.972+0000 7f908a000640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90640668f0 0x7f9064068db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f908c140910 tx=0x7f9074007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:34.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:34.976+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f908c13d900 con 0x7f908c0b1da0 2024-01-15T18:05:35.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f908c13d900 con 0x7f908c0b1da0 2024-01-15T18:05:35.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9082a00640 1 -- 172.21.15.143:0/382682231 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f908c13d900 con 0x7f908c0b1da0 2024-01-15T18:05:35.134 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90640668f0 msgr2=0x7f9064068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90640668f0 0x7f9064068db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f908c140910 tx=0x7f9074007d60 comp rx=0 tx=0).stop 2024-01-15T18:05:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 msgr2=0x7f908c15aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 0x7f908c15aa30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f907c0ad040 tx=0x7f907c01ac70 comp rx=0 tx=0).stop 2024-01-15T18:05:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 shutdown_connections 2024-01-15T18:05:35.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90640668f0 0x7f9064068db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c0ce690 0x7f908c13fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f908c0b3bf0 0x7f908c140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 --2- 172.21.15.143:0/382682231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c0b1da0 0x7f908c15aa30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 >> 172.21.15.143:0/382682231 conn(0x7f908c06c3f0 msgr2=0x7f908c0b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 shutdown_connections 2024-01-15T18:05:35.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.128+0000 7f9091800640 1 -- 172.21.15.143:0/382682231 wait complete. 2024-01-15T18:05:35.156 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:35.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.284+0000 7f2c38400640 1 Processor -- start 2024-01-15T18:05:35.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c38400640 1 -- start start 2024-01-15T18:05:35.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c38400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c30143780 0x7f2c300cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c38400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300ceb50 con 0x7f2c300ce520 2024-01-15T18:05:35.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c38400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3015c110 con 0x7f2c300722c0 2024-01-15T18:05:35.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c38400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3015b7e0 con 0x7f2c30143780 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c35a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c30143780 0x7f2c300cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c35000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2c300ce520 0x7f2c30071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51426/0 (socket says 172.21.15.143:51426) 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c35000640 1 -- 172.21.15.143:0/2710197811 learned_addr learned my addr 172.21.15.143:0/2710197811 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c2fe00640 1 -- 172.21.15.143:0/2710197811 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665766363 0 0) 0x7f2c18001120 con 0x7f2c300ce520 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c35a00640 1 -- 172.21.15.143:0/2710197811 >> v1:172.21.15.204:6789/0 conn(0x7f2c300722c0 legacy=0x7f2c3015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c35a00640 1 -- 172.21.15.143:0/2710197811 >> v1:172.21.15.143:6789/0 conn(0x7f2c300ce520 legacy=0x7f2c30071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.288+0000 7f2c35a00640 1 -- 172.21.15.143:0/2710197811 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3015bac0 con 0x7f2c30143780 2024-01-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c35a00640 1 --2- 172.21.15.143:0/2710197811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c30143780 0x7f2c300cdfe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c200097c0 tx=0x7f2c2001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1a69322d7b15db server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c2fe00640 1 -- 172.21.15.143:0/2710197811 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20004230 con 0x7f2c30143780 2024-01-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c2fe00640 1 -- 172.21.15.143:0/2710197811 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c200a7050 con 0x7f2c30143780 2024-01-15T18:05:35.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c2fe00640 1 -- 172.21.15.143:0/2710197811 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20003620 con 0x7f2c30143780 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 -- 172.21.15.143:0/2710197811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c30143780 msgr2=0x7f2c300cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 --2- 172.21.15.143:0/2710197811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c30143780 0x7f2c300cdfe0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2c200097c0 tx=0x7f2c2001c440 comp rx=0 tx=0).stop 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 -- 172.21.15.143:0/2710197811 shutdown_connections 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 --2- 172.21.15.143:0/2710197811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c30143780 0x7f2c300cdfe0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 -- 172.21.15.143:0/2710197811 >> 172.21.15.143:0/2710197811 conn(0x7f2c3006c3f0 msgr2=0x7f2c3006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 -- 172.21.15.143:0/2710197811 shutdown_connections 2024-01-15T18:05:35.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.292+0000 7f2c38400640 1 -- 172.21.15.143:0/2710197811 wait complete. 2024-01-15T18:05:35.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 Processor -- start 2024-01-15T18:05:35.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- start start 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 0x7f2c30071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300ce520 0x7f2c30140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30143780 0x7f2c30155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30141670 con 0x7f2c300722c0 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30153f10 con 0x7f2c30143780 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c301fa570 con 0x7f2c300ce520 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 0x7f2c30071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 0x7f2c30071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37914/0 (socket says 172.21.15.143:37914) 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 -- 172.21.15.143:0/3563312829 learned_addr learned my addr 172.21.15.143:0/3563312829 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35000640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300ce520 0x7f2c30140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c36400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30143780 0x7f2c30155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 -- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300ce520 msgr2=0x7f2c30140500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300ce520 0x7f2c30140500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 -- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30143780 msgr2=0x7f2c30155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30143780 0x7f2c30155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 -- 172.21.15.143:0/3563312829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c20009420 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c35a00640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 0x7f2c30071130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c20019050 tx=0x7f2c200191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20003c70 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20003df0 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200ab680 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c30140fb0 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3013bb40 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.296+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c30067500 con 0x7f2c300722c0 2024-01-15T18:05:35.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.300+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f2c200b2040 con 0x7f2c300722c0 2024-01-15T18:05:35.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.300+0000 7f2c2d600640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c080668f0 0x7f2c08068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.300+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c2011e370 con 0x7f2c300722c0 2024-01-15T18:05:35.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.300+0000 7f2c35000640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c080668f0 0x7f2c08068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.300+0000 7f2c35000640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c080668f0 0x7f2c08068db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c18007fd0 tx=0x7f2c180079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.304+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c200eb500 con 0x7f2c300722c0 2024-01-15T18:05:35.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.464+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2c300720a0 con 0x7f2c300722c0 2024-01-15T18:05:35.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.464+0000 7f2c2d600640 1 -- 172.21.15.143:0/3563312829 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2c200b0080 con 0x7f2c300722c0 2024-01-15T18:05:35.471 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c080668f0 msgr2=0x7f2c08068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c080668f0 0x7f2c08068db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2c18007fd0 tx=0x7f2c180079c0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 msgr2=0x7f2c30071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 0x7f2c30071130 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2c20019050 tx=0x7f2c200191e0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 shutdown_connections 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c080668f0 0x7f2c08068db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c30143780 0x7f2c30155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c300ce520 0x7f2c30140500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 --2- 172.21.15.143:0/3563312829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c300722c0 0x7f2c30071130 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 >> 172.21.15.143:0/3563312829 conn(0x7f2c3006c3f0 msgr2=0x7f2c300adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 shutdown_connections 2024-01-15T18:05:35.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.468+0000 7f2c38400640 1 -- 172.21.15.143:0/3563312829 wait complete. 2024-01-15T18:05:35.493 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:05:35.493 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:35.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff52de00640 1 Processor -- start 2024-01-15T18:05:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff52de00640 1 -- start start 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff52de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff5280cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff52de00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5280cbff0 con 0x7ff5280cf740 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff52de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5280d0c50 con 0x7ff52806bfb0 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff52de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280cb6c0 con 0x7ff5280b0310 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff527400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff5280cf740 0x7ff5280cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51434/0 (socket says 172.21.15.143:51434) 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff527400640 1 -- 172.21.15.143:0/387274671 learned_addr learned my addr 172.21.15.143:0/387274671 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff526000640 1 --2- 172.21.15.143:0/387274671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff5280cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537646173 0 0) 0x7ff5280cbff0 con 0x7ff5280cf740 2024-01-15T18:05:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff508003530 con 0x7ff5280cf740 2024-01-15T18:05:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791516160 0 0) 0x7ff5280d0c50 con 0x7ff52806bfb0 2024-01-15T18:05:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5280cbff0 con 0x7ff52806bfb0 2024-01-15T18:05:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff526000640 1 -- 172.21.15.143:0/387274671 >> v1:172.21.15.204:6789/0 conn(0x7ff52806bfb0 legacy=0x7ff52806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff526000640 1 -- 172.21.15.143:0/387274671 >> v1:172.21.15.143:6789/0 conn(0x7ff5280cf740 legacy=0x7ff5280cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.632+0000 7ff526000640 1 -- 172.21.15.143:0/387274671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280cb9a0 con 0x7ff5280b0310 2024-01-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff526000640 1 --2- 172.21.15.143:0/387274671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff5280cf200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5100097f0 tx=0x7ff51001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637e1d294b621726 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100ab600 con 0x7ff5280b0310 2024-01-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5100a7050 con 0x7ff5280b0310 2024-01-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff525600640 1 -- 172.21.15.143:0/387274671 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100ab980 con 0x7ff5280b0310 2024-01-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 -- 172.21.15.143:0/387274671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 msgr2=0x7ff5280cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 --2- 172.21.15.143:0/387274671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff5280cf200 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff5100097f0 tx=0x7ff51001c400 comp rx=0 tx=0).stop 2024-01-15T18:05:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 -- 172.21.15.143:0/387274671 shutdown_connections 2024-01-15T18:05:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 --2- 172.21.15.143:0/387274671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff5280cf200 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 -- 172.21.15.143:0/387274671 >> 172.21.15.143:0/387274671 conn(0x7ff5280d0ed0 msgr2=0x7ff5280d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 -- 172.21.15.143:0/387274671 shutdown_connections 2024-01-15T18:05:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.636+0000 7ff52de00640 1 -- 172.21.15.143:0/387274671 wait complete. 2024-01-15T18:05:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 Processor -- start 2024-01-15T18:05:35.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 -- start start 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff52806bfb0 0x7ff5280c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff528140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 0x7ff5280aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280c5910 con 0x7ff5280cf740 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280c78b0 con 0x7ff52806bfb0 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280c7b90 con 0x7ff5280b0310 2024-01-15T18:05:35.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 0x7ff5280aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 0x7ff5280aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37928/0 (socket says 172.21.15.143:37928) 2024-01-15T18:05:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 -- 172.21.15.143:0/2153662542 learned_addr learned my addr 172.21.15.143:0/2153662542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff526000640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff528140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff526a00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff52806bfb0 0x7ff5280c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 -- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 msgr2=0x7ff528140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff528140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 -- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff52806bfb0 msgr2=0x7ff5280c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff52806bfb0 0x7ff5280c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 -- 172.21.15.143:0/2153662542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff510009420 con 0x7ff5280cf740 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff526a00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff52806bfb0 0x7ff5280c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff527400640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 0x7ff5280aef50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff52000cba0 tx=0x7ff52000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52000ace0 con 0x7ff5280cf740 2024-01-15T18:05:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52000ae40 con 0x7ff5280cf740 2024-01-15T18:05:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5280c5350 con 0x7ff5280cf740 2024-01-15T18:05:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520012620 con 0x7ff5280cf740 2024-01-15T18:05:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.640+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5281528f0 con 0x7ff5280cf740 2024-01-15T18:05:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.644+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff528152ae0 con 0x7ff5280cf740 2024-01-15T18:05:35.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.644+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff520007070 con 0x7ff5280cf740 2024-01-15T18:05:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.644+0000 7ff51f400640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5000668f0 0x7ff500068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.644+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff520084410 con 0x7ff5280cf740 2024-01-15T18:05:35.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.644+0000 7ff526a00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5000668f0 0x7ff500068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.644+0000 7ff526a00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5000668f0 0x7ff500068db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5180011f0 tx=0x7ff518009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.648+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5200550f0 con 0x7ff5280cf740 2024-01-15T18:05:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.800+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff528067500 con 0x7ff5280cf740 2024-01-15T18:05:35.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.800+0000 7ff51f400640 1 -- 172.21.15.143:0/2153662542 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7ff520054c80 con 0x7ff5280cf740 2024-01-15T18:05:35.807 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:35.807 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5000668f0 msgr2=0x7ff500068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5000668f0 0x7ff500068db0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff5180011f0 tx=0x7ff518009040 comp rx=0 tx=0).stop 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 msgr2=0x7ff5280aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 0x7ff5280aef50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff52000cba0 tx=0x7ff52000a430 comp rx=0 tx=0).stop 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 shutdown_connections 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5000668f0 0x7ff500068db0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5280cf740 0x7ff5280aef50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5280b0310 0x7ff528140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 --2- 172.21.15.143:0/2153662542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff52806bfb0 0x7ff5280c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 >> 172.21.15.143:0/2153662542 conn(0x7ff5280d0ed0 msgr2=0x7ff52806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 shutdown_connections 2024-01-15T18:05:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.804+0000 7ff52de00640 1 -- 172.21.15.143:0/2153662542 wait complete. 2024-01-15T18:05:35.829 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:35.830 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:35.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 Processor -- start 2024-01-15T18:05:35.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 -- start start 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f622415b120 0x7f62240cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f62240cf400 0x7f62240cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f62240cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62240cf970 con 0x7f622415b120 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62240cfc50 con 0x7f62240cf400 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f622b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62240c4b10 con 0x7f62240d1940 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f622415b120 0x7f62240cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f622415b120 0x7f62240cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37932/0 (socket says 172.21.15.143:37932) 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 -- 172.21.15.143:0/1898793566 learned_addr learned my addr 172.21.15.143:0/1898793566 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6229800640 1 --2- 172.21.15.143:0/1898793566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f62240cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6223e00640 1 --2- 172.21.15.143:0/1898793566 >> v2:172.21.15.204:3300/0 conn(0x7f62240cf400 0x7f62240cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 -- 172.21.15.143:0/1898793566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 msgr2=0x7f62240cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 --2- 172.21.15.143:0/1898793566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f62240cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 -- 172.21.15.143:0/1898793566 >> v2:172.21.15.204:3300/0 conn(0x7f62240cf400 msgr2=0x7f62240cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 --2- 172.21.15.143:0/1898793566 >> v2:172.21.15.204:3300/0 conn(0x7f62240cf400 0x7f62240cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.964+0000 7f6228e00640 1 -- 172.21.15.143:0/1898793566 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62240c4df0 con 0x7f622415b120 2024-01-15T18:05:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f6228e00640 1 --2- 172.21.15.143:0/1898793566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f62240cd810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f622406b3f0 tx=0x7f621401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade5988e4171829c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f6223400640 1 -- 172.21.15.143:0/1898793566 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62140ab600 con 0x7f622415b120 2024-01-15T18:05:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f6223400640 1 -- 172.21.15.143:0/1898793566 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62140a7050 con 0x7f622415b120 2024-01-15T18:05:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f6223400640 1 -- 172.21.15.143:0/1898793566 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62140ab980 con 0x7f622415b120 2024-01-15T18:05:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 -- 172.21.15.143:0/1898793566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 msgr2=0x7f62240cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 --2- 172.21.15.143:0/1898793566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f62240cd810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f622406b3f0 tx=0x7f621401c470 comp rx=0 tx=0).stop 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 -- 172.21.15.143:0/1898793566 shutdown_connections 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 --2- 172.21.15.143:0/1898793566 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f62240cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 --2- 172.21.15.143:0/1898793566 >> v2:172.21.15.204:3300/0 conn(0x7f62240cf400 0x7f62240cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 --2- 172.21.15.143:0/1898793566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f62240cd810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 -- 172.21.15.143:0/1898793566 >> 172.21.15.143:0/1898793566 conn(0x7f62240d0e20 msgr2=0x7f62240d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 -- 172.21.15.143:0/1898793566 shutdown_connections 2024-01-15T18:05:35.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.968+0000 7f622b800640 1 -- 172.21.15.143:0/1898793566 wait complete. 2024-01-15T18:05:35.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 Processor -- start 2024-01-15T18:05:35.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- start start 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62240cf400 0x7f6224150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f6224152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f622415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62241548a0 con 0x7f622415b120 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6224153800 con 0x7f62240cf400 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6224153ae0 con 0x7f62240d1940 2024-01-15T18:05:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f622415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f622415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37934/0 (socket says 172.21.15.143:37934) 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 -- 172.21.15.143:0/1472431101 learned_addr learned my addr 172.21.15.143:0/1472431101 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6228e00640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62240cf400 0x7f6224150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6223e00640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f6224152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 -- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 msgr2=0x7f6224152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f6224152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 -- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62240cf400 msgr2=0x7f6224150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62240cf400 0x7f6224150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 -- 172.21.15.143:0/1472431101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62140094d0 con 0x7f622415b120 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6228e00640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62240cf400 0x7f6224150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6229800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f622415d9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f621c00efc0 tx=0x7f621c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c014480 con 0x7f622415b120 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c00f040 con 0x7f622415b120 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c014820 con 0x7f622415b120 2024-01-15T18:05:35.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6224153dc0 con 0x7f622415b120 2024-01-15T18:05:35.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6224154380 con 0x7f622415b120 2024-01-15T18:05:35.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.972+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62241545a0 con 0x7f622415b120 2024-01-15T18:05:35.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.976+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f621c014980 con 0x7f622415b120 2024-01-15T18:05:35.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.976+0000 7f6220c00640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61fc066920 0x7f61fc068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:35.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.976+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f621c084f80 con 0x7f622415b120 2024-01-15T18:05:35.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.976+0000 7f6228e00640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61fc066920 0x7f61fc068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:35.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.976+0000 7f6228e00640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61fc066920 0x7f61fc068de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62140ad000 tx=0x7f62140a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:35.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:35.980+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f621c052120 con 0x7f622415b120 2024-01-15T18:05:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.128+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6224067500 con 0x7f622415b120 2024-01-15T18:05:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.128+0000 7f6220c00640 1 -- 172.21.15.143:0/1472431101 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f621c0180a0 con 0x7f622415b120 2024-01-15T18:05:36.137 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61fc066920 msgr2=0x7f61fc068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61fc066920 0x7f61fc068de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f62140ad000 tx=0x7f62140a6000 comp rx=0 tx=0).stop 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 msgr2=0x7f622415d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f622415d9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f621c00efc0 tx=0x7f621c00c490 comp rx=0 tx=0).stop 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 shutdown_connections 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61fc066920 0x7f61fc068de0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622415b120 0x7f622415d9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62240d1940 0x7f6224152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 --2- 172.21.15.143:0/1472431101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62240cf400 0x7f6224150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 >> 172.21.15.143:0/1472431101 conn(0x7f62240d0e20 msgr2=0x7f622406f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 shutdown_connections 2024-01-15T18:05:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.132+0000 7f622b800640 1 -- 172.21.15.143:0/1472431101 wait complete. 2024-01-15T18:05:36.158 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.288+0000 7efc1b200640 1 Processor -- start 2024-01-15T18:05:36.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.288+0000 7efc1b200640 1 -- start start 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc1b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efc14141660 0x7efc140ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc1b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc1415ec80 0x7efc14141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc1b200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc1415f310 con 0x7efc14142470 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc1b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc1415fbb0 con 0x7efc14141660 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc1b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1415f100 con 0x7efc1415ec80 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efc14141660 0x7efc140ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc1415ec80 0x7efc14141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc19200640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7efc14142470 0x7efc14071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:52478/0 (socket says 172.21.15.143:52478) 2024-01-15T18:05:36.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc19200640 1 -- 172.21.15.143:0/834759476 learned_addr learned my addr 172.21.15.143:0/834759476 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc12a00640 1 -- 172.21.15.143:0/834759476 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124805221 0 0) 0x7efc1415f310 con 0x7efc14142470 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc12a00640 1 -- 172.21.15.143:0/834759476 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf4003530 con 0x7efc14142470 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13400640 1 -- 172.21.15.143:0/834759476 >> v2:172.21.15.204:3300/0 conn(0x7efc14141660 msgr2=0x7efc140ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13400640 1 --2- 172.21.15.143:0/834759476 >> v2:172.21.15.204:3300/0 conn(0x7efc14141660 0x7efc140ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13400640 1 -- 172.21.15.143:0/834759476 >> v1:172.21.15.204:6789/0 conn(0x7efc14142470 legacy=0x7efc14071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13400640 1 -- 172.21.15.143:0/834759476 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1415f5f0 con 0x7efc1415ec80 2024-01-15T18:05:36.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.292+0000 7efc13e00640 1 --2- 172.21.15.143:0/834759476 >> v2:172.21.15.204:3300/0 conn(0x7efc14141660 0x7efc140ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc13400640 1 --2- 172.21.15.143:0/834759476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc1415ec80 0x7efc14141f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0097c0 tx=0x7efbfc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3285e17a1d155263 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc12a00640 1 -- 172.21.15.143:0/834759476 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0ab650 con 0x7efc1415ec80 2024-01-15T18:05:36.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc12a00640 1 -- 172.21.15.143:0/834759476 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0ab7b0 con 0x7efc1415ec80 2024-01-15T18:05:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc12a00640 1 -- 172.21.15.143:0/834759476 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0abb40 con 0x7efc1415ec80 2024-01-15T18:05:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 -- 172.21.15.143:0/834759476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc1415ec80 msgr2=0x7efc14141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 --2- 172.21.15.143:0/834759476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc1415ec80 0x7efc14141f30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0097c0 tx=0x7efbfc01c450 comp rx=0 tx=0).stop 2024-01-15T18:05:36.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 -- 172.21.15.143:0/834759476 shutdown_connections 2024-01-15T18:05:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 --2- 172.21.15.143:0/834759476 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc1415ec80 0x7efc14141f30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 --2- 172.21.15.143:0/834759476 >> v2:172.21.15.204:3300/0 conn(0x7efc14141660 0x7efc140ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 -- 172.21.15.143:0/834759476 >> 172.21.15.143:0/834759476 conn(0x7efc1406c3f0 msgr2=0x7efc1406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 -- 172.21.15.143:0/834759476 shutdown_connections 2024-01-15T18:05:36.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.296+0000 7efc1b200640 1 -- 172.21.15.143:0/834759476 wait complete. 2024-01-15T18:05:36.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 Processor -- start 2024-01-15T18:05:36.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- start start 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc14141660 0x7efc140b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc14142470 0x7efc1415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1415ec80 0x7efc1413d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b2b30 con 0x7efc1415ec80 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b2200 con 0x7efc14141660 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc140b2470 con 0x7efc14142470 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc19200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1415ec80 0x7efc1413d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc19200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1415ec80 0x7efc1413d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37950/0 (socket says 172.21.15.143:37950) 2024-01-15T18:05:36.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc19200640 1 -- 172.21.15.143:0/611399710 learned_addr learned my addr 172.21.15.143:0/611399710 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc14142470 0x7efc1415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13e00640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc14141660 0x7efc140b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 -- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc14141660 msgr2=0x7efc140b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc14141660 0x7efc140b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 -- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1415ec80 msgr2=0x7efc1413d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1415ec80 0x7efc1413d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 -- 172.21.15.143:0/611399710 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc04009420 con 0x7efc14142470 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc13400640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc14142470 0x7efc1415ace0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0ac000 tx=0x7efbfc01ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0bf610 con 0x7efc14142470 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0bf790 con 0x7efc14142470 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0bfb40 con 0x7efc14142470 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc009420 con 0x7efc14142470 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc140cd150 con 0x7efc14142470 2024-01-15T18:05:36.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.300+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc140b2c70 con 0x7efc14142470 2024-01-15T18:05:36.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.304+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7efbfc0b5400 con 0x7efc14142470 2024-01-15T18:05:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.304+0000 7efc0be00640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efbec0668f0 0x7efbec068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.304+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efbfc11f7e0 con 0x7efc14142470 2024-01-15T18:05:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.304+0000 7efc13e00640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efbec0668f0 0x7efbec068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.304+0000 7efc13e00640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efbec0668f0 0x7efbec068db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efc04009cf0 tx=0x7efc04009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.308+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc140b2c70 con 0x7efc14142470 2024-01-15T18:05:36.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.456+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc140b2c70 con 0x7efc14142470 2024-01-15T18:05:36.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.456+0000 7efc0be00640 1 -- 172.21.15.143:0/611399710 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7efbfc0c4020 con 0x7efc14142470 2024-01-15T18:05:36.465 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efbec0668f0 msgr2=0x7efbec068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efbec0668f0 0x7efbec068db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efc04009cf0 tx=0x7efc04009190 comp rx=0 tx=0).stop 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc14142470 msgr2=0x7efc1415ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc14142470 0x7efc1415ace0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0ac000 tx=0x7efbfc01ac90 comp rx=0 tx=0).stop 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 shutdown_connections 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efbec0668f0 0x7efbec068db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc1415ec80 0x7efc1413d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc14142470 0x7efc1415ace0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 --2- 172.21.15.143:0/611399710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc14141660 0x7efc140b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 >> 172.21.15.143:0/611399710 conn(0x7efc1406c3f0 msgr2=0x7efc1406c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:36.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 shutdown_connections 2024-01-15T18:05:36.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.460+0000 7efc1b200640 1 -- 172.21.15.143:0/611399710 wait complete. 2024-01-15T18:05:36.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:36.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.616+0000 7f5e0ca00640 1 Processor -- start 2024-01-15T18:05:36.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0ca00640 1 -- start start 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5e040cda70 0x7f5e040cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf400 0x7f5e040cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0ca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0415b5b0 con 0x7f5e040cf800 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0ca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e040ccd30 con 0x7f5e040cda70 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040cba70 con 0x7f5e040cf400 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5e040cda70 0x7f5e040cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5e040cda70 0x7f5e040cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37962/0 (socket says 172.21.15.143:37962) 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 -- 172.21.15.143:0/2858630858 learned_addr learned my addr 172.21.15.143:0/2858630858 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e09600640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf400 0x7f5e040cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 -- 172.21.15.143:0/2858630858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf400 msgr2=0x7f5e040cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:36.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf400 0x7f5e040cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 -- 172.21.15.143:0/2858630858 >> v1:172.21.15.204:6789/0 conn(0x7f5e040cf800 legacy=0x7f5e040b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e0a000640 1 -- 172.21.15.143:0/2858630858 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040cbd10 con 0x7f5e040cda70 2024-01-15T18:05:36.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.620+0000 7f5e09600640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf400 0x7f5e040cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:36.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0a000640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e040cc7f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009870 tx=0x7f5dfc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f11dd423f647540 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e08c00640 1 -- 172.21.15.143:0/2858630858 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0aa5f0 con 0x7f5e040cda70 2024-01-15T18:05:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e08c00640 1 -- 172.21.15.143:0/2858630858 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc0aa750 con 0x7f5e040cda70 2024-01-15T18:05:36.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e08c00640 1 -- 172.21.15.143:0/2858630858 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0aaae0 con 0x7f5e040cda70 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 -- 172.21.15.143:0/2858630858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 msgr2=0x7f5e040cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e040cc7f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc009870 tx=0x7f5dfc01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 -- 172.21.15.143:0/2858630858 shutdown_connections 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf400 0x7f5e040cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/2858630858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e040cc7f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 -- 172.21.15.143:0/2858630858 >> 172.21.15.143:0/2858630858 conn(0x7f5e040d0e20 msgr2=0x7f5e040d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 -- 172.21.15.143:0/2858630858 shutdown_connections 2024-01-15T18:05:36.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 -- 172.21.15.143:0/2858630858 wait complete. 2024-01-15T18:05:36.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 Processor -- start 2024-01-15T18:05:36.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.624+0000 7f5e0ca00640 1 -- start start 2024-01-15T18:05:36.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e0415e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e040cf400 0x7f5e0415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf800 0x7f5e04157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c75c0 con 0x7f5e040cda70 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c72e0 con 0x7f5e040cf400 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e040c8f40 con 0x7f5e040cf800 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e0415e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e0415e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37964/0 (socket says 172.21.15.143:37964) 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 -- 172.21.15.143:0/3163958506 learned_addr learned my addr 172.21.15.143:0/3163958506 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e09600640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e040cf400 0x7f5e0415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0aa00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf800 0x7f5e04157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 -- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf800 msgr2=0x7f5e04157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf800 0x7f5e04157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 -- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e040cf400 msgr2=0x7f5e0415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e040cf400 0x7f5e0415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 -- 172.21.15.143:0/3163958506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc0094d0 con 0x7f5e040cda70 2024-01-15T18:05:36.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0a000640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e0415e180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01ab80 tx=0x7f5dfc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0b3800 con 0x7f5e040cda70 2024-01-15T18:05:36.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dfc0b3980 con 0x7f5e040cda70 2024-01-15T18:05:36.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dfc0b3d30 con 0x7f5e040cda70 2024-01-15T18:05:36.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e040c9220 con 0x7f5e040cda70 2024-01-15T18:05:36.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0415e6c0 con 0x7f5e040cda70 2024-01-15T18:05:36.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.628+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e040c8da0 con 0x7f5e040cda70 2024-01-15T18:05:36.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.632+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5dfc0c6440 con 0x7f5e040cda70 2024-01-15T18:05:36.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.632+0000 7f5e02000640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ddc0668f0 0x7f5ddc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.632+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dfc0a7070 con 0x7f5e040cda70 2024-01-15T18:05:36.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.632+0000 7f5e09600640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ddc0668f0 0x7f5ddc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.632+0000 7f5e09600640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ddc0668f0 0x7f5ddc068db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e040c5170 tx=0x7f5dec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.636+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e040c8da0 con 0x7f5e040cda70 2024-01-15T18:05:36.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e040c8da0 con 0x7f5e040cda70 2024-01-15T18:05:36.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e02000640 1 -- 172.21.15.143:0/3163958506 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e040c8da0 con 0x7f5e040cda70 2024-01-15T18:05:36.795 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ddc0668f0 msgr2=0x7f5ddc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ddc0668f0 0x7f5ddc068db0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5e040c5170 tx=0x7f5dec008040 comp rx=0 tx=0).stop 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 msgr2=0x7f5e0415e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e0415e180 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc01ab80 tx=0x7f5dfc01ad10 comp rx=0 tx=0).stop 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 shutdown_connections 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ddc0668f0 0x7f5ddc068db0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e040cf800 0x7f5e04157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e040cf400 0x7f5e0415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 --2- 172.21.15.143:0/3163958506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e040cda70 0x7f5e0415e180 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.788+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 >> 172.21.15.143:0/3163958506 conn(0x7f5e040d0e20 msgr2=0x7f5e040ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.792+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 shutdown_connections 2024-01-15T18:05:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.792+0000 7f5e0ca00640 1 -- 172.21.15.143:0/3163958506 wait complete. 2024-01-15T18:05:36.817 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 Processor -- start 2024-01-15T18:05:36.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 -- start start 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe090159b70 0x7fe0900b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0900b48f0 0x7fe090070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900b4d20 con 0x7fe090157300 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0900b35a0 con 0x7fe0900b48f0 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe098200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe090159fd0 con 0x7fe090159b70 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0900b48f0 0x7fe090070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0900b48f0 0x7fe090070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37974/0 (socket says 172.21.15.143:37974) 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 -- 172.21.15.143:0/3265044661 learned_addr learned my addr 172.21.15.143:0/3265044661 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:36.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe094e00640 1 --2- 172.21.15.143:0/3265044661 >> v2:172.21.15.204:3300/0 conn(0x7fe090159b70 0x7fe0900b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 -- 172.21.15.143:0/3265044661 >> v2:172.21.15.204:3300/0 conn(0x7fe090159b70 msgr2=0x7fe0900b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:36.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 --2- 172.21.15.143:0/3265044661 >> v2:172.21.15.204:3300/0 conn(0x7fe090159b70 0x7fe0900b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 -- 172.21.15.143:0/3265044661 >> v1:172.21.15.204:6789/0 conn(0x7fe090157300 legacy=0x7fe0900b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.948+0000 7fe096200640 1 -- 172.21.15.143:0/3265044661 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0900b2a20 con 0x7fe0900b48f0 2024-01-15T18:05:36.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe096200640 1 --2- 172.21.15.143:0/3265044661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe090070e80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b4fe0 tx=0x7fe08801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1510cc31b29ebe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe08fe00640 1 -- 172.21.15.143:0/3265044661 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0880a5860 con 0x7fe0900b48f0 2024-01-15T18:05:36.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe08fe00640 1 -- 172.21.15.143:0/3265044661 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0880a59c0 con 0x7fe0900b48f0 2024-01-15T18:05:36.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe08fe00640 1 -- 172.21.15.143:0/3265044661 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0880a5d50 con 0x7fe0900b48f0 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 -- 172.21.15.143:0/3265044661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 msgr2=0x7fe090070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 --2- 172.21.15.143:0/3265044661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe090070e80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b4fe0 tx=0x7fe08801c9a0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 -- 172.21.15.143:0/3265044661 shutdown_connections 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 --2- 172.21.15.143:0/3265044661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe090070e80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 --2- 172.21.15.143:0/3265044661 >> v2:172.21.15.204:3300/0 conn(0x7fe090159b70 0x7fe0900b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 -- 172.21.15.143:0/3265044661 >> 172.21.15.143:0/3265044661 conn(0x7fe09006c3f0 msgr2=0x7fe09006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:36.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 -- 172.21.15.143:0/3265044661 shutdown_connections 2024-01-15T18:05:36.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.952+0000 7fe098200640 1 -- 172.21.15.143:0/3265044661 wait complete. 2024-01-15T18:05:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 Processor -- start 2024-01-15T18:05:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 -- start start 2024-01-15T18:05:36.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe0901f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe090157300 0x7fe090071a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe090159b70 0x7fe090071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901f5e30 con 0x7fe0900b48f0 2024-01-15T18:05:36.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901fa3d0 con 0x7fe090157300 2024-01-15T18:05:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09014fd30 con 0x7fe090159b70 2024-01-15T18:05:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe0901f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe0901f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37976/0 (socket says 172.21.15.143:37976) 2024-01-15T18:05:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 -- 172.21.15.143:0/1646778045 learned_addr learned my addr 172.21.15.143:0/1646778045 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe096200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe090159b70 0x7fe090071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe094e00640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe090157300 0x7fe090071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 -- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe090159b70 msgr2=0x7fe090071f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe090159b70 0x7fe090071f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 -- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe090157300 msgr2=0x7fe090071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe090157300 0x7fe090071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 -- 172.21.15.143:0/1646778045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088009b30 con 0x7fe0900b48f0 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe095800640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe0901f9e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe08000c530 tx=0x7fe08000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080005220 con 0x7fe0900b48f0 2024-01-15T18:05:36.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe080005380 con 0x7fe0900b48f0 2024-01-15T18:05:36.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080007270 con 0x7fe0900b48f0 2024-01-15T18:05:36.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09014f720 con 0x7fe0900b48f0 2024-01-15T18:05:36.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.956+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09014f100 con 0x7fe0900b48f0 2024-01-15T18:05:36.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.960+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe090067500 con 0x7fe0900b48f0 2024-01-15T18:05:36.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.960+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe0800073d0 con 0x7fe0900b48f0 2024-01-15T18:05:36.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.960+0000 7fe08d600640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe068066920 0x7fe068068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:36.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.960+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe080083b70 con 0x7fe0900b48f0 2024-01-15T18:05:36.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.960+0000 7fe094e00640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe068066920 0x7fe068068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.960+0000 7fe094e00640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe068066920 0x7fe068068de0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0901597c0 tx=0x7fe078008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:36.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:36.964+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe080050e50 con 0x7fe0900b48f0 2024-01-15T18:05:37.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.124+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe09014bba0 con 0x7fe0900b48f0 2024-01-15T18:05:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.124+0000 7fe08d600640 1 -- 172.21.15.143:0/1646778045 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe080054470 con 0x7fe0900b48f0 2024-01-15T18:05:37.131 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:37.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe068066920 msgr2=0x7fe068068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe068066920 0x7fe068068de0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0901597c0 tx=0x7fe078008040 comp rx=0 tx=0).stop 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 msgr2=0x7fe0901f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe0901f9e90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe08000c530 tx=0x7fe08000c9f0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 shutdown_connections 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe068066920 0x7fe068068de0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe090159b70 0x7fe090071f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe090157300 0x7fe090071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 --2- 172.21.15.143:0/1646778045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0900b48f0 0x7fe0901f9e90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 >> 172.21.15.143:0/1646778045 conn(0x7fe09006c3f0 msgr2=0x7fe090156220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 shutdown_connections 2024-01-15T18:05:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.128+0000 7fe098200640 1 -- 172.21.15.143:0/1646778045 wait complete. 2024-01-15T18:05:37.153 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:05:37.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:37.153 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:05:37.197 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:37.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:05:37.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:37.197 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:05:37.255 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:05:37.255 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:37.263 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:05:37.263 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:37.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:37.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.484+0000 7f3dbfe00640 1 Processor -- start 2024-01-15T18:05:37.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.484+0000 7f3dbfe00640 1 -- start start 2024-01-15T18:05:37.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db8157300 0x7f3db8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3db80b21d0 0x7f3db80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbfe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db80b2a20 con 0x7f3db80b48f0 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbfe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db80b35a0 con 0x7f3db80b21d0 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db80b4d80 con 0x7f3db8157300 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbde00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3db80b48f0 0x7f3db815d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57070/0 (socket says 172.21.15.143:57070) 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbde00640 1 -- 172.21.15.143:0/2965979765 learned_addr learned my addr 172.21.15.143:0/2965979765 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbd400640 1 --2- 172.21.15.143:0/2965979765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db8157300 0x7f3db8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbca00640 1 --2- 172.21.15.143:0/2965979765 >> v2:172.21.15.204:3300/0 conn(0x7f3db80b21d0 0x7f3db80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3db7e00640 1 -- 172.21.15.143:0/2965979765 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853735682 0 0) 0x7f3db80b2a20 con 0x7f3db80b48f0 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3db7e00640 1 -- 172.21.15.143:0/2965979765 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d98003530 con 0x7f3db80b48f0 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbd400640 1 -- 172.21.15.143:0/2965979765 >> v2:172.21.15.204:3300/0 conn(0x7f3db80b21d0 msgr2=0x7f3db80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbd400640 1 --2- 172.21.15.143:0/2965979765 >> v2:172.21.15.204:3300/0 conn(0x7f3db80b21d0 0x7f3db80b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbd400640 1 -- 172.21.15.143:0/2965979765 >> v1:172.21.15.143:6789/0 conn(0x7f3db80b48f0 legacy=0x7f3db815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3db7e00640 1 -- 172.21.15.143:0/2965979765 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603871225 0 0) 0x7f3d98003530 con 0x7f3db80b48f0 2024-01-15T18:05:37.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbd400640 1 -- 172.21.15.143:0/2965979765 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db80b2d00 con 0x7f3db8157300 2024-01-15T18:05:37.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.488+0000 7f3dbca00640 1 --2- 172.21.15.143:0/2965979765 >> v2:172.21.15.204:3300/0 conn(0x7f3db80b21d0 0x7f3db80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:37.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbd400640 1 --2- 172.21.15.143:0/2965979765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db8157300 0x7f3db8159b70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097c0 tx=0x7f3da801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d138b7a28060a392 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3db7e00640 1 -- 172.21.15.143:0/2965979765 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0001a00 con 0x7f3db8157300 2024-01-15T18:05:37.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3db7e00640 1 -- 172.21.15.143:0/2965979765 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da80a7050 con 0x7f3db8157300 2024-01-15T18:05:37.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3db7e00640 1 -- 172.21.15.143:0/2965979765 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da80ab620 con 0x7f3db8157300 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 -- 172.21.15.143:0/2965979765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db8157300 msgr2=0x7f3db8159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/2965979765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db8157300 0x7f3db8159b70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3da80097c0 tx=0x7f3da801c370 comp rx=0 tx=0).stop 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 -- 172.21.15.143:0/2965979765 shutdown_connections 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/2965979765 >> v2:172.21.15.204:3300/0 conn(0x7f3db80b21d0 0x7f3db80b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/2965979765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db8157300 0x7f3db8159b70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 -- 172.21.15.143:0/2965979765 >> 172.21.15.143:0/2965979765 conn(0x7f3db806c3f0 msgr2=0x7f3db806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 -- 172.21.15.143:0/2965979765 shutdown_connections 2024-01-15T18:05:37.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 -- 172.21.15.143:0/2965979765 wait complete. 2024-01-15T18:05:37.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 Processor -- start 2024-01-15T18:05:37.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.492+0000 7f3dbfe00640 1 -- start start 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db80b21d0 0x7f3db8151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db80b48f0 0x7f3db8153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 0x7f3db815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81518c0 con 0x7f3db8157300 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8152220 con 0x7f3db80b21d0 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8150f20 con 0x7f3db80b48f0 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 0x7f3db815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db80b48f0 0x7f3db8153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 0x7f3db815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59144/0 (socket says 172.21.15.143:59144) 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 -- 172.21.15.143:0/1481673461 learned_addr learned my addr 172.21.15.143:0/1481673461 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbd400640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db80b21d0 0x7f3db8151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 -- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db80b48f0 msgr2=0x7f3db8153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db80b48f0 0x7f3db8153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 -- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db80b21d0 msgr2=0x7f3db8151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db80b21d0 0x7f3db8151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 -- 172.21.15.143:0/1481673461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da0009420 con 0x7f3db8157300 2024-01-15T18:05:37.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbde00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 0x7f3db815bcd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3db000cca0 tx=0x7f3db000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0012400 con 0x7f3db8157300 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db0012560 con 0x7f3db8157300 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0012910 con 0x7f3db8157300 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da8009420 con 0x7f3db8157300 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db81514e0 con 0x7f3db8157300 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.496+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db8067500 con 0x7f3db8157300 2024-01-15T18:05:37.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.500+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3db0012a70 con 0x7f3db8157300 2024-01-15T18:05:37.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.500+0000 7f3db5600640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d900668f0 0x7f3d90068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.500+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db0084ae0 con 0x7f3db8157300 2024-01-15T18:05:37.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.500+0000 7f3dbd400640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d900668f0 0x7f3d90068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.500+0000 7f3dbd400640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d900668f0 0x7f3d90068db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ad000 tx=0x7f3da80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.504+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3db0051dc0 con 0x7f3db8157300 2024-01-15T18:05:37.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.652+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3db806e4e0 con 0x7f3db8157300 2024-01-15T18:05:37.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.652+0000 7f3db5600640 1 -- 172.21.15.143:0/1481673461 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f3db00553e0 con 0x7f3db8157300 2024-01-15T18:05:37.660 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:37.660 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:37.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d900668f0 msgr2=0x7f3d90068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d900668f0 0x7f3d90068db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3da80ad000 tx=0x7f3da80a6000 comp rx=0 tx=0).stop 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 msgr2=0x7f3db815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 0x7f3db815bcd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3db000cca0 tx=0x7f3db000a430 comp rx=0 tx=0).stop 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 shutdown_connections 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d900668f0 0x7f3d90068db0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3db8157300 0x7f3db815bcd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3db80b48f0 0x7f3db8153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 --2- 172.21.15.143:0/1481673461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3db80b21d0 0x7f3db8151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 >> 172.21.15.143:0/1481673461 conn(0x7f3db806c3f0 msgr2=0x7f3db80af430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 shutdown_connections 2024-01-15T18:05:37.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.656+0000 7f3dbfe00640 1 -- 172.21.15.143:0/1481673461 wait complete. 2024-01-15T18:05:37.682 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:37.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 Processor -- start 2024-01-15T18:05:37.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 -- start start 2024-01-15T18:05:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3878070cc0 0x7f387806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f387806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38780cc810 con 0x7f387815bf40 2024-01-15T18:05:37.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f387806e9f0 con 0x7f3878070cc0 2024-01-15T18:05:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387815ea80 con 0x7f387815f1b0 2024-01-15T18:05:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3877400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f387815bf40 0x7f38780710a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57080/0 (socket says 172.21.15.143:57080) 2024-01-15T18:05:37.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3877400640 1 -- 172.21.15.143:0/2536587030 learned_addr learned my addr 172.21.15.143:0/2536587030 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:37.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ca00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f387806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3877e00640 1 --2- 172.21.15.143:0/2536587030 >> v2:172.21.15.204:3300/0 conn(0x7f3878070cc0 0x7f387806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251621485 0 0) 0x7f3860001120 con 0x7f387815bf40 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3858003530 con 0x7f387815bf40 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2081888294 0 0) 0x7f3858003530 con 0x7f387815bf40 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 msgr2=0x7f387806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f387806a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 >> v2:172.21.15.204:3300/0 conn(0x7f3878070cc0 msgr2=0x7f387806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 --2- 172.21.15.143:0/2536587030 >> v2:172.21.15.204:3300/0 conn(0x7f3878070cc0 0x7f387806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780cbf70 con 0x7f387815bf40 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3860001810 con 0x7f387815bf40 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ca00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f387806a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 >> v1:172.21.15.143:6789/0 conn(0x7f387815bf40 legacy=0x7f38780710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3858003980 0x7f3858003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38580047d0 0x7f3858004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f3858005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387806e9f0 con 0x7f38580047d0 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387815ea80 con 0x7f3858003980 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385800b990 con 0x7f387815f1b0 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3877400640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38580047d0 0x7f3858004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f3877e00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3858003980 0x7f3858003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.816+0000 7f387ca00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f3858005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3877400640 1 -- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 msgr2=0x7f3858005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3877400640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f387815f1b0 0x7f3858005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3877400640 1 -- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3858003980 msgr2=0x7f3858003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3877400640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3858003980 0x7f3858003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3877400640 1 -- 172.21.15.143:0/2536587030 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3870009a70 con 0x7f38580047d0 2024-01-15T18:05:37.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3877400640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38580047d0 0x7f3858004c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f386000c210 tx=0x7f386001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1783f34751a2430 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600a8ce0 con 0x7f38580047d0 2024-01-15T18:05:37.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38600a8e60 con 0x7f38580047d0 2024-01-15T18:05:37.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.820+0000 7f3876a00640 1 -- 172.21.15.143:0/2536587030 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860005b20 con 0x7f38580047d0 2024-01-15T18:05:37.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- 172.21.15.143:0/2536587030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38580047d0 msgr2=0x7f3858004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 --2- 172.21.15.143:0/2536587030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38580047d0 0x7f3858004c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f386000c210 tx=0x7f386001f3b0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ca00640 1 -- 172.21.15.143:0/2536587030 reap_dead start 2024-01-15T18:05:37.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- 172.21.15.143:0/2536587030 shutdown_connections 2024-01-15T18:05:37.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- 172.21.15.143:0/2536587030 >> 172.21.15.143:0/2536587030 conn(0x7f38780b1610 msgr2=0x7f38780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- 172.21.15.143:0/2536587030 shutdown_connections 2024-01-15T18:05:37.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- 172.21.15.143:0/2536587030 wait complete. 2024-01-15T18:05:37.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 Processor -- start 2024-01-15T18:05:37.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- start start 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38580047d0 0x7f387815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38780b43b0 0x7f38781545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 0x7f387815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38780b3f20 con 0x7f387814c570 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387815a710 con 0x7f38780b43b0 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38780b38d0 con 0x7f38580047d0 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 0x7f387815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 0x7f387815dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59166/0 (socket says 172.21.15.143:59166) 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ca00640 1 -- 172.21.15.143:0/3843255204 learned_addr learned my addr 172.21.15.143:0/3843255204 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f3877400640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38780b43b0 0x7f38781545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f3877e00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38580047d0 0x7f387815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ca00640 1 -- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38580047d0 msgr2=0x7f387815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.824+0000 7f387ca00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38580047d0 0x7f387815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ca00640 1 -- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38780b43b0 msgr2=0x7f38781545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ca00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38780b43b0 0x7f38781545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ca00640 1 -- 172.21.15.143:0/3843255204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780b3bb0 con 0x7f387814c570 2024-01-15T18:05:37.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ca00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 0x7f387815dc30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f387815ebc0 tx=0x7f38700117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387000e070 con 0x7f387814c570 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38700039a0 con 0x7f387814c570 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870003b50 con 0x7f387814c570 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3878154b30 con 0x7f387814c570 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387814ca00 con 0x7f387814c570 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3878074b00 con 0x7f387814c570 2024-01-15T18:05:37.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3870003cb0 con 0x7f387814c570 2024-01-15T18:05:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f386fe00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38500668f0 0x7f3850068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3870084cd0 con 0x7f387814c570 2024-01-15T18:05:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.828+0000 7f3877e00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38500668f0 0x7f3850068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.832+0000 7f3877e00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38500668f0 0x7f3850068db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3868009b60 tx=0x7f3868009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.832+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3870051e80 con 0x7f387814c570 2024-01-15T18:05:37.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.984+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38780b3d40 con 0x7f387814c570 2024-01-15T18:05:37.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.984+0000 7f386fe00640 1 -- 172.21.15.143:0/3843255204 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38700554a0 con 0x7f387814c570 2024-01-15T18:05:37.991 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:37.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38500668f0 msgr2=0x7f3850068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38500668f0 0x7f3850068db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3868009b60 tx=0x7f3868009190 comp rx=0 tx=0).stop 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 msgr2=0x7f387815dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 0x7f387815dc30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f387815ebc0 tx=0x7f38700117d0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 shutdown_connections 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38500668f0 0x7f3850068db0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f387814c570 0x7f387815dc30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38780b43b0 0x7f38781545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 --2- 172.21.15.143:0/3843255204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38580047d0 0x7f387815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 >> 172.21.15.143:0/3843255204 conn(0x7f38780b1610 msgr2=0x7f38780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 shutdown_connections 2024-01-15T18:05:37.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:37.988+0000 7f387ea00640 1 -- 172.21.15.143:0/3843255204 wait complete. 2024-01-15T18:05:38.013 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:38.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 Processor -- start 2024-01-15T18:05:38.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 -- start start 2024-01-15T18:05:38.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 0x7f786c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f786c0b48f0 0x7f786c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0b2a20 con 0x7f786c157300 2024-01-15T18:05:38.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0b35a0 con 0x7f786c0b48f0 2024-01-15T18:05:38.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7871400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c159fd0 con 0x7f786c159b70 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7869e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 0x7f786c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f786b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f786c0b48f0 0x7f786c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7869e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 0x7f786c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:36156/0 (socket says 172.21.15.143:36156) 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.148+0000 7f7869e00640 1 -- 172.21.15.143:0/1127588297 learned_addr learned my addr 172.21.15.143:0/1127588297 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869e00640 1 -- 172.21.15.143:0/1127588297 >> v2:172.21.15.204:3300/0 conn(0x7f786c0b48f0 msgr2=0x7f786c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869e00640 1 --2- 172.21.15.143:0/1127588297 >> v2:172.21.15.204:3300/0 conn(0x7f786c0b48f0 0x7f786c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869e00640 1 -- 172.21.15.143:0/1127588297 >> v1:172.21.15.204:6789/0 conn(0x7f786c157300 legacy=0x7f786c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869e00640 1 -- 172.21.15.143:0/1127588297 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0b2d00 con 0x7f786c159b70 2024-01-15T18:05:38.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869e00640 1 --2- 172.21.15.143:0/1127588297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 0x7f786c0b43b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7854009870 tx=0x7f785401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e233abcad3dc0232 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869400640 1 -- 172.21.15.143:0/1127588297 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78540a7070 con 0x7f786c159b70 2024-01-15T18:05:38.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869400640 1 -- 172.21.15.143:0/1127588297 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78540ab430 con 0x7f786c159b70 2024-01-15T18:05:38.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7869400640 1 -- 172.21.15.143:0/1127588297 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78540ab7c0 con 0x7f786c159b70 2024-01-15T18:05:38.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7871400640 1 -- 172.21.15.143:0/1127588297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 msgr2=0x7f786c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.152+0000 7f7871400640 1 --2- 172.21.15.143:0/1127588297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 0x7f786c0b43b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7854009870 tx=0x7f785401c540 comp rx=0 tx=0).stop 2024-01-15T18:05:38.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- 172.21.15.143:0/1127588297 shutdown_connections 2024-01-15T18:05:38.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 --2- 172.21.15.143:0/1127588297 >> v2:172.21.15.204:3300/0 conn(0x7f786c0b48f0 0x7f786c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 --2- 172.21.15.143:0/1127588297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c159b70 0x7f786c0b43b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- 172.21.15.143:0/1127588297 >> 172.21.15.143:0/1127588297 conn(0x7f786c06c3f0 msgr2=0x7f786c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- 172.21.15.143:0/1127588297 shutdown_connections 2024-01-15T18:05:38.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- 172.21.15.143:0/1127588297 wait complete. 2024-01-15T18:05:38.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 Processor -- start 2024-01-15T18:05:38.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- start start 2024-01-15T18:05:38.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c0b48f0 0x7f786c14a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 0x7f786c14a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c159b70 0x7f786c1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c14afb0 con 0x7f786c157300 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c154de0 con 0x7f786c159b70 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7871400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c1550c0 con 0x7f786c0b48f0 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 0x7f786c14a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 0x7f786c14a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59174/0 (socket says 172.21.15.143:59174) 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 -- 172.21.15.143:0/314338522 learned_addr learned my addr 172.21.15.143:0/314338522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f786a800640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c0b48f0 0x7f786c14a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f786b200640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c159b70 0x7f786c1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 -- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c0b48f0 msgr2=0x7f786c14a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c0b48f0 0x7f786c14a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 -- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c159b70 msgr2=0x7f786c1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c159b70 0x7f786c1540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f7869e00640 1 -- 172.21.15.143:0/314338522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540094d0 con 0x7f786c157300 2024-01-15T18:05:38.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.156+0000 7f786a800640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c0b48f0 0x7f786c14a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:38.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7869e00640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 0x7f786c14a570 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f78540ad040 tx=0x7f78540192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78540b4620 con 0x7f786c157300 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78540b4780 con 0x7f786c157300 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c1553a0 con 0x7f786c157300 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78540b4b10 con 0x7f786c157300 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c155960 con 0x7f786c157300 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c155b50 con 0x7f786c157300 2024-01-15T18:05:38.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f78540b4c70 con 0x7f786c157300 2024-01-15T18:05:38.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7862a00640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78440668f0 0x7f7844068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f785411e600 con 0x7f786c157300 2024-01-15T18:05:38.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.160+0000 7f786a800640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78440668f0 0x7f7844068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.164+0000 7f786a800640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78440668f0 0x7f7844068db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f786c1f7170 tx=0x7f785c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.164+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78540eb850 con 0x7f786c157300 2024-01-15T18:05:38.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.316+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f786c067500 con 0x7f786c157300 2024-01-15T18:05:38.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.316+0000 7f7862a00640 1 -- 172.21.15.143:0/314338522 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78540eee70 con 0x7f786c157300 2024-01-15T18:05:38.320 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78440668f0 msgr2=0x7f7844068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78440668f0 0x7f7844068db0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f786c1f7170 tx=0x7f785c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 msgr2=0x7f786c14a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 0x7f786c14a570 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f78540ad040 tx=0x7f78540192e0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 shutdown_connections 2024-01-15T18:05:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78440668f0 0x7f7844068db0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c159b70 0x7f786c1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f786c157300 0x7f786c14a570 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 --2- 172.21.15.143:0/314338522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f786c0b48f0 0x7f786c14a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 >> 172.21.15.143:0/314338522 conn(0x7f786c06c3f0 msgr2=0x7f786c156620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 shutdown_connections 2024-01-15T18:05:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.320+0000 7f7871400640 1 -- 172.21.15.143:0/314338522 wait complete. 2024-01-15T18:05:38.342 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:38.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.476+0000 7fe070c00640 1 Processor -- start 2024-01-15T18:05:38.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.476+0000 7fe070c00640 1 -- start start 2024-01-15T18:05:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe070c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe06c141660 0x7fe06c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe070c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe06c071d00 0x7fe06c072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe070c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06c072650 con 0x7fe06c15ec80 2024-01-15T18:05:38.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe070c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06c15fbb0 con 0x7fe06c141660 2024-01-15T18:05:38.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe070c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe06c15f130 con 0x7fe06c071d00 2024-01-15T18:05:38.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe06c15ec80 0x7fe06c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57090/0 (socket says 172.21.15.143:57090) 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069400640 1 -- 172.21.15.143:0/1049977153 learned_addr learned my addr 172.21.15.143:0/1049977153 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe06c141660 0x7fe06c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe06a800640 1 --2- 172.21.15.143:0/1049977153 >> v2:172.21.15.204:3300/0 conn(0x7fe06c071d00 0x7fe06c072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe068a00640 1 -- 172.21.15.143:0/1049977153 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476130964 0 0) 0x7fe054001120 con 0x7fe06c15ec80 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069e00640 1 -- 172.21.15.143:0/1049977153 >> v2:172.21.15.204:3300/0 conn(0x7fe06c071d00 msgr2=0x7fe06c072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069e00640 1 --2- 172.21.15.143:0/1049977153 >> v2:172.21.15.204:3300/0 conn(0x7fe06c071d00 0x7fe06c072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069e00640 1 -- 172.21.15.143:0/1049977153 >> v1:172.21.15.143:6789/0 conn(0x7fe06c15ec80 legacy=0x7fe06c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.480+0000 7fe069e00640 1 -- 172.21.15.143:0/1049977153 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c15f310 con 0x7fe06c141660 2024-01-15T18:05:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe069e00640 1 --2- 172.21.15.143:0/1049977153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c141660 0x7fe06c0ce560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe06c06e250 tx=0x7fe05c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a252e223d5b03c74 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe068a00640 1 -- 172.21.15.143:0/1049977153 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0a7070 con 0x7fe06c141660 2024-01-15T18:05:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe068a00640 1 -- 172.21.15.143:0/1049977153 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05c0ab430 con 0x7fe06c141660 2024-01-15T18:05:38.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe068a00640 1 -- 172.21.15.143:0/1049977153 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05c0ab7c0 con 0x7fe06c141660 2024-01-15T18:05:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 -- 172.21.15.143:0/1049977153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c141660 msgr2=0x7fe06c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 --2- 172.21.15.143:0/1049977153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c141660 0x7fe06c0ce560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe06c06e250 tx=0x7fe05c01c430 comp rx=0 tx=0).stop 2024-01-15T18:05:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 -- 172.21.15.143:0/1049977153 shutdown_connections 2024-01-15T18:05:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 --2- 172.21.15.143:0/1049977153 >> v2:172.21.15.204:3300/0 conn(0x7fe06c071d00 0x7fe06c072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 --2- 172.21.15.143:0/1049977153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c141660 0x7fe06c0ce560 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 -- 172.21.15.143:0/1049977153 >> 172.21.15.143:0/1049977153 conn(0x7fe06c06c3f0 msgr2=0x7fe06c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 -- 172.21.15.143:0/1049977153 shutdown_connections 2024-01-15T18:05:38.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 -- 172.21.15.143:0/1049977153 wait complete. 2024-01-15T18:05:38.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 Processor -- start 2024-01-15T18:05:38.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.484+0000 7fe070c00640 1 -- start start 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe06c071d00 0x7fe06c15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c141660 0x7fe06c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 0x7fe06c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c15bca0 con 0x7fe06c15ec80 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c15a440 con 0x7fe06c071d00 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c15bf10 con 0x7fe06c141660 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 0x7fe06c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 0x7fe06c0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59190/0 (socket says 172.21.15.143:59190) 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 -- 172.21.15.143:0/3556037221 learned_addr learned my addr 172.21.15.143:0/3556037221 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:38.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe069400640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c141660 0x7fe06c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe069e00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe06c071d00 0x7fe06c15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 -- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c141660 msgr2=0x7fe06c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c141660 0x7fe06c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 -- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe06c071d00 msgr2=0x7fe06c15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe06c071d00 0x7fe06c15e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 -- 172.21.15.143:0/3556037221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05c009420 con 0x7fe06c15ec80 2024-01-15T18:05:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe069e00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe06c071d00 0x7fe06c15e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe06a800640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 0x7fe06c0b2fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe064007d90 tx=0x7fe06400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064014420 con 0x7fe06c15ec80 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0640145a0 con 0x7fe06c15ec80 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064014950 con 0x7fe06c15ec80 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe06c15c1f0 con 0x7fe06c15ec80 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe06c15add0 con 0x7fe06c15ec80 2024-01-15T18:05:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06c067500 con 0x7fe06c15ec80 2024-01-15T18:05:38.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.488+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe064014ab0 con 0x7fe06c15ec80 2024-01-15T18:05:38.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.492+0000 7fe062000640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0440668f0 0x7fe044068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.492+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe064084df0 con 0x7fe06c15ec80 2024-01-15T18:05:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.492+0000 7fe069e00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0440668f0 0x7fe044068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.492+0000 7fe069e00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0440668f0 0x7fe044068db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0af000 tx=0x7fe05c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.496+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe064051fa0 con 0x7fe06c15ec80 2024-01-15T18:05:38.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe06c15c6a0 con 0x7fe06c15ec80 2024-01-15T18:05:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe062000640 1 -- 172.21.15.143:0/3556037221 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0640555c0 con 0x7fe06c15ec80 2024-01-15T18:05:38.654 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:38.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0440668f0 msgr2=0x7fe044068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0440668f0 0x7fe044068db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0af000 tx=0x7fe05c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:38.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 msgr2=0x7fe06c0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 0x7fe06c0b2fc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe064007d90 tx=0x7fe06400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 shutdown_connections 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe0440668f0 0x7fe044068db0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe06c15ec80 0x7fe06c0b2fc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe06c141660 0x7fe06c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 --2- 172.21.15.143:0/3556037221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe06c071d00 0x7fe06c15e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 >> 172.21.15.143:0/3556037221 conn(0x7fe06c06c3f0 msgr2=0x7fe06c070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 shutdown_connections 2024-01-15T18:05:38.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.652+0000 7fe070c00640 1 -- 172.21.15.143:0/3556037221 wait complete. 2024-01-15T18:05:38.677 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:38.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 Processor -- start 2024-01-15T18:05:38.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 -- start start 2024-01-15T18:05:38.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc9681607c0 0x7fc96806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc9680cfb30 0x7fc96806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9680cffd0 con 0x7fc96806c230 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680d1ce0 con 0x7fc9680cfb30 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc96e800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9680d02b0 con 0x7fc9681607c0 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc967e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc96806c230 0x7fc9680cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57098/0 (socket says 172.21.15.143:57098) 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc967e00640 1 -- 172.21.15.143:0/1224845989 learned_addr learned my addr 172.21.15.143:0/1224845989 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:38.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc966a00640 1 --2- 172.21.15.143:0/1224845989 >> v2:172.21.15.143:3300/0 conn(0x7fc9680cfb30 0x7fc96806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc967400640 1 --2- 172.21.15.143:0/1224845989 >> v2:172.21.15.204:3300/0 conn(0x7fc9681607c0 0x7fc96806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc966a00640 1 -- 172.21.15.143:0/1224845989 >> v2:172.21.15.204:3300/0 conn(0x7fc9681607c0 msgr2=0x7fc96806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc966a00640 1 --2- 172.21.15.143:0/1224845989 >> v2:172.21.15.204:3300/0 conn(0x7fc9681607c0 0x7fc96806b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc966a00640 1 -- 172.21.15.143:0/1224845989 >> v1:172.21.15.143:6789/0 conn(0x7fc96806c230 legacy=0x7fc9680cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc966000640 1 -- 172.21.15.143:0/1224845989 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457473664 0 0) 0x7fc9680cffd0 con 0x7fc96806c230 2024-01-15T18:05:38.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.812+0000 7fc966a00640 1 -- 172.21.15.143:0/1224845989 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9680cb470 con 0x7fc9680cfb30 2024-01-15T18:05:38.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc966a00640 1 --2- 172.21.15.143:0/1224845989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc96806bcf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc968071a00 tx=0x7fc95001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79784bec2ba1bcb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc966000640 1 -- 172.21.15.143:0/1224845989 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500a8950 con 0x7fc9680cfb30 2024-01-15T18:05:38.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc966000640 1 -- 172.21.15.143:0/1224845989 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500a8ab0 con 0x7fc9680cfb30 2024-01-15T18:05:38.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc966000640 1 -- 172.21.15.143:0/1224845989 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500a8e40 con 0x7fc9680cfb30 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 -- 172.21.15.143:0/1224845989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 msgr2=0x7fc96806bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 --2- 172.21.15.143:0/1224845989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc96806bcf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc968071a00 tx=0x7fc95001ec50 comp rx=0 tx=0).stop 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 -- 172.21.15.143:0/1224845989 shutdown_connections 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 --2- 172.21.15.143:0/1224845989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc96806bcf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 --2- 172.21.15.143:0/1224845989 >> v2:172.21.15.204:3300/0 conn(0x7fc9681607c0 0x7fc96806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 -- 172.21.15.143:0/1224845989 >> 172.21.15.143:0/1224845989 conn(0x7fc9680d0e20 msgr2=0x7fc9680d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 -- 172.21.15.143:0/1224845989 shutdown_connections 2024-01-15T18:05:38.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.816+0000 7fc96e800640 1 -- 172.21.15.143:0/1224845989 wait complete. 2024-01-15T18:05:38.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 Processor -- start 2024-01-15T18:05:38.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 -- start start 2024-01-15T18:05:38.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96806c230 0x7fc96815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc9680c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9681607c0 0x7fc9680ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680c98d0 con 0x7fc9680cfb30 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96815c550 con 0x7fc96806c230 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc96e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9680c9320 con 0x7fc9681607c0 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc9680c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc9680c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59214/0 (socket says 172.21.15.143:59214) 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 -- 172.21.15.143:0/3964213115 learned_addr learned my addr 172.21.15.143:0/3964213115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc967400640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96806c230 0x7fc96815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc967e00640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9681607c0 0x7fc9680ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 -- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9681607c0 msgr2=0x7fc9680ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9681607c0 0x7fc9680ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 -- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96806c230 msgr2=0x7fc96815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96806c230 0x7fc96815c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 -- 172.21.15.143:0/3964213115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950009420 con 0x7fc9680cfb30 2024-01-15T18:05:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc967e00640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9681607c0 0x7fc9680ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc966a00640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc9680c8de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc950009880 tx=0x7fc95001d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.820+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500a8950 con 0x7fc9680cfb30 2024-01-15T18:05:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950006e20 con 0x7fc9680cfb30 2024-01-15T18:05:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950004230 con 0x7fc9680cfb30 2024-01-15T18:05:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9680c95f0 con 0x7fc9680cfb30 2024-01-15T18:05:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9680b2b30 con 0x7fc9680cfb30 2024-01-15T18:05:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96806d4b0 con 0x7fc9680cfb30 2024-01-15T18:05:38.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc950004390 con 0x7fc9680cfb30 2024-01-15T18:05:38.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc95f400640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc9400668f0 0x7fc940068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc950122710 con 0x7fc9680cfb30 2024-01-15T18:05:38.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc967400640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc9400668f0 0x7fc940068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.824+0000 7fc967400640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc9400668f0 0x7fc940068db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9580079c0 tx=0x7fc958008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.828+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc9500efa60 con 0x7fc9680cfb30 2024-01-15T18:05:38.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc96806d4b0 con 0x7fc9680cfb30 2024-01-15T18:05:38.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc95f400640 1 -- 172.21.15.143:0/3964213115 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc9500f3080 con 0x7fc9680cfb30 2024-01-15T18:05:38.994 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc9400668f0 msgr2=0x7fc940068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc9400668f0 0x7fc940068db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc9580079c0 tx=0x7fc958008040 comp rx=0 tx=0).stop 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 msgr2=0x7fc9680c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc9680c8de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc950009880 tx=0x7fc95001d490 comp rx=0 tx=0).stop 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 shutdown_connections 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc9400668f0 0x7fc940068db0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9681607c0 0x7fc9680ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9680cfb30 0x7fc9680c8de0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 --2- 172.21.15.143:0/3964213115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc96806c230 0x7fc96815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 >> 172.21.15.143:0/3964213115 conn(0x7fc9680d0e20 msgr2=0x7fc9680ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 shutdown_connections 2024-01-15T18:05:38.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:38.992+0000 7fc96e800640 1 -- 172.21.15.143:0/3964213115 wait complete. 2024-01-15T18:05:39.015 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:05:39.015 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:39.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:39.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 Processor -- start 2024-01-15T18:05:39.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 -- start start 2024-01-15T18:05:39.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7415aef0 0x7f8a7406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a740b29a0 0x7f8a740ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7415f7c0 con 0x7f8a7406c560 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7406c310 con 0x7f8a740b29a0 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7415ebb0 con 0x7f8a7415aef0 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a740b29a0 0x7f8a740ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a740b29a0 0x7f8a740ceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59216/0 (socket says 172.21.15.143:59216) 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 -- 172.21.15.143:0/2058638495 learned_addr learned my addr 172.21.15.143:0/2058638495 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a79c00640 1 --2- 172.21.15.143:0/2058638495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7415aef0 0x7f8a7406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 -- 172.21.15.143:0/2058638495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7415aef0 msgr2=0x7f8a7406b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 --2- 172.21.15.143:0/2058638495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7415aef0 0x7f8a7406b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 -- 172.21.15.143:0/2058638495 >> v1:172.21.15.204:6789/0 conn(0x7f8a7406c560 legacy=0x7f8a740b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.196+0000 7f8a7b000640 1 -- 172.21.15.143:0/2058638495 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7415ee90 con 0x7f8a740b29a0 2024-01-15T18:05:39.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7b000640 1 --2- 172.21.15.143:0/2058638495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a740b29a0 0x7f8a740ceca0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009e50 tx=0x7f8a6c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6662002e3e6d488f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a73e00640 1 -- 172.21.15.143:0/2058638495 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0a5820 con 0x7f8a740b29a0 2024-01-15T18:05:39.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a73e00640 1 -- 172.21.15.143:0/2058638495 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c0a5980 con 0x7f8a740b29a0 2024-01-15T18:05:39.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a73e00640 1 -- 172.21.15.143:0/2058638495 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0a5d10 con 0x7f8a740b29a0 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 -- 172.21.15.143:0/2058638495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a740b29a0 msgr2=0x7f8a740ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 --2- 172.21.15.143:0/2058638495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a740b29a0 0x7f8a740ceca0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009e50 tx=0x7f8a6c01c9e0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 -- 172.21.15.143:0/2058638495 shutdown_connections 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 --2- 172.21.15.143:0/2058638495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a740b29a0 0x7f8a740ceca0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 --2- 172.21.15.143:0/2058638495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7415aef0 0x7f8a7406b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 -- 172.21.15.143:0/2058638495 >> 172.21.15.143:0/2058638495 conn(0x7f8a74066720 msgr2=0x7f8a7406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 -- 172.21.15.143:0/2058638495 shutdown_connections 2024-01-15T18:05:39.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.200+0000 7f8a7c600640 1 -- 172.21.15.143:0/2058638495 wait complete. 2024-01-15T18:05:39.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 Processor -- start 2024-01-15T18:05:39.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 -- start start 2024-01-15T18:05:39.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 0x7f8a74154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a740b29a0 0x7f8a74156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7415aef0 0x7f8a7415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a74158f70 con 0x7f8a7406c560 2024-01-15T18:05:39.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a741583f0 con 0x7f8a7415aef0 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a741586d0 con 0x7f8a740b29a0 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 0x7f8a74154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 0x7f8a74154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59222/0 (socket says 172.21.15.143:59222) 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 -- 172.21.15.143:0/1544252896 learned_addr learned my addr 172.21.15.143:0/1544252896 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7b000640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7415aef0 0x7f8a7415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79200640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a740b29a0 0x7f8a74156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 -- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a740b29a0 msgr2=0x7f8a74156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a740b29a0 0x7f8a74156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 -- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7415aef0 msgr2=0x7f8a7415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7415aef0 0x7f8a7415c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 -- 172.21.15.143:0/1544252896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c009b30 con 0x7f8a7406c560 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7b000640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7415aef0 0x7f8a7415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79200640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a740b29a0 0x7f8a74156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:39.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a79c00640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 0x7f8a74154480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400e870 tx=0x7f8a6400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a64009510 con 0x7f8a7406c560 2024-01-15T18:05:39.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6400eec0 con 0x7f8a7406c560 2024-01-15T18:05:39.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400cb40 con 0x7f8a7406c560 2024-01-15T18:05:39.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a741589b0 con 0x7f8a7406c560 2024-01-15T18:05:39.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.204+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a74156b70 con 0x7f8a7406c560 2024-01-15T18:05:39.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.208+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a74067680 con 0x7f8a7406c560 2024-01-15T18:05:39.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.208+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f8a6400cca0 con 0x7f8a7406c560 2024-01-15T18:05:39.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.208+0000 7f8a71600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a4c0668f0 0x7f8a4c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.208+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a640844c0 con 0x7f8a7406c560 2024-01-15T18:05:39.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.208+0000 7f8a79200640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a4c0668f0 0x7f8a4c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.208+0000 7f8a79200640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a4c0668f0 0x7f8a4c068db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a7406de30 tx=0x7f8a5c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.212+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a64051790 con 0x7f8a7406c560 2024-01-15T18:05:39.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.368+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a74067e60 con 0x7f8a7406c560 2024-01-15T18:05:39.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.368+0000 7f8a71600640 1 -- 172.21.15.143:0/1544252896 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16541 (secure 0 0 0) 0x7f8a64054db0 con 0x7f8a7406c560 2024-01-15T18:05:39.371 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:39.372 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:00.278306+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a4c0668f0 msgr2=0x7f8a4c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a4c0668f0 0x7f8a4c068db0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a7406de30 tx=0x7f8a5c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 msgr2=0x7f8a74154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 0x7f8a74154480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400e870 tx=0x7f8a6400ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 shutdown_connections 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a4c0668f0 0x7f8a4c068db0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7415aef0 0x7f8a7415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a740b29a0 0x7f8a74156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 --2- 172.21.15.143:0/1544252896 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7406c560 0x7f8a74154480 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 >> 172.21.15.143:0/1544252896 conn(0x7f8a74066720 msgr2=0x7f8a7415b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 shutdown_connections 2024-01-15T18:05:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.372+0000 7f8a7c600640 1 -- 172.21.15.143:0/1544252896 wait complete. 2024-01-15T18:05:39.394 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:39.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:39.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 Processor -- start 2024-01-15T18:05:39.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 -- start start 2024-01-15T18:05:39.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 0x7f15dc072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 0x7f15dc15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15dc0ceb50 con 0x7f15dc143780 2024-01-15T18:05:39.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15dc072570 con 0x7f15dc071bb0 2024-01-15T18:05:39.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.532+0000 7f15e3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc15c110 con 0x7f15dc15c5a0 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 0x7f15dc15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15dbe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 0x7f15dc072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 0x7f15dc15ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:36214/0 (socket says 172.21.15.143:36214) 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15dbe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 0x7f15dc072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:33778/0 (socket says 172.21.15.143:33778) 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 -- 172.21.15.143:0/2681324745 learned_addr learned my addr 172.21.15.143:0/2681324745 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 -- 172.21.15.143:0/2681324745 >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 msgr2=0x7f15dc072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 --2- 172.21.15.143:0/2681324745 >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 0x7f15dc072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 -- 172.21.15.143:0/2681324745 >> v1:172.21.15.204:6789/0 conn(0x7f15dc143780 legacy=0x7f15dc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 -- 172.21.15.143:0/2681324745 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc15b7e0 con 0x7f15dc15c5a0 2024-01-15T18:05:39.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15dbe00640 1 --2- 172.21.15.143:0/2681324745 >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 0x7f15dc072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:39.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15e1a00640 1 --2- 172.21.15.143:0/2681324745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 0x7f15dc15ca30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009e50 tx=0x7f15d401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc3b376d367ddc6c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.536+0000 7f15db400640 1 -- 172.21.15.143:0/2681324745 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d40a58e0 con 0x7f15dc15c5a0 2024-01-15T18:05:39.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15db400640 1 -- 172.21.15.143:0/2681324745 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d40a5a40 con 0x7f15dc15c5a0 2024-01-15T18:05:39.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15db400640 1 -- 172.21.15.143:0/2681324745 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d40a5dd0 con 0x7f15dc15c5a0 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 -- 172.21.15.143:0/2681324745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 msgr2=0x7f15dc15ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 --2- 172.21.15.143:0/2681324745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 0x7f15dc15ca30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009e50 tx=0x7f15d401ca10 comp rx=0 tx=0).stop 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 -- 172.21.15.143:0/2681324745 shutdown_connections 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 --2- 172.21.15.143:0/2681324745 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc15c5a0 0x7f15dc15ca30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 --2- 172.21.15.143:0/2681324745 >> v2:172.21.15.204:3300/0 conn(0x7f15dc071bb0 0x7f15dc072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 -- 172.21.15.143:0/2681324745 >> 172.21.15.143:0/2681324745 conn(0x7f15dc06c3f0 msgr2=0x7f15dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 -- 172.21.15.143:0/2681324745 shutdown_connections 2024-01-15T18:05:39.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 -- 172.21.15.143:0/2681324745 wait complete. 2024-01-15T18:05:39.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 Processor -- start 2024-01-15T18:05:39.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.540+0000 7f15e3a00640 1 -- start start 2024-01-15T18:05:39.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc071bb0 0x7f15dc0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15dc143780 0x7f15dc1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15dc15c5a0 0x7f15dc1373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc13dc70 con 0x7f15dc143780 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc13d0f0 con 0x7f15dc15c5a0 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc13d3d0 con 0x7f15dc071bb0 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15dc143780 0x7f15dc1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15dc143780 0x7f15dc1573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59228/0 (socket says 172.21.15.143:59228) 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15dbe00640 1 -- 172.21.15.143:0/1263631654 learned_addr learned my addr 172.21.15.143:0/1263631654 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15dc15c5a0 0x7f15dc1373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc071bb0 0x7f15dc0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 -- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15dc15c5a0 msgr2=0x7f15dc1373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15dc15c5a0 0x7f15dc1373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 -- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15dc143780 msgr2=0x7f15dc1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15dc143780 0x7f15dc1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 -- 172.21.15.143:0/1263631654 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c4009420 con 0x7f15dc071bb0 2024-01-15T18:05:39.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e1000640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc071bb0 0x7f15dc0afb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15cc000f70 tx=0x7f15cc00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0052d0 con 0x7f15dc071bb0 2024-01-15T18:05:39.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc005430 con 0x7f15dc071bb0 2024-01-15T18:05:39.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc0052d0 con 0x7f15dc071bb0 2024-01-15T18:05:39.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d4009b30 con 0x7f15dc071bb0 2024-01-15T18:05:39.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.544+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15dc13d990 con 0x7f15dc071bb0 2024-01-15T18:05:39.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.548+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15dc1559e0 con 0x7f15dc071bb0 2024-01-15T18:05:39.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.548+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f15cc003600 con 0x7f15dc071bb0 2024-01-15T18:05:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.548+0000 7f15d8c00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15b4066950 0x7f15b4068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.548+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f15cc083260 con 0x7f15dc071bb0 2024-01-15T18:05:39.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.548+0000 7f15dbe00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15b4066950 0x7f15b4068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.552+0000 7f15dbe00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15b4066950 0x7f15b4068e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15dc13eeb0 tx=0x7f15c4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.552+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f15cc050560 con 0x7f15dc071bb0 2024-01-15T18:05:39.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.708+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15dc0ceb50 con 0x7f15dc071bb0 2024-01-15T18:05:39.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.708+0000 7f15d8c00640 1 -- 172.21.15.143:0/1263631654 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15dc0ceb50 con 0x7f15dc071bb0 2024-01-15T18:05:39.713 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15b4066950 msgr2=0x7f15b4068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15b4066950 0x7f15b4068e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15dc13eeb0 tx=0x7f15c4009190 comp rx=0 tx=0).stop 2024-01-15T18:05:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc071bb0 msgr2=0x7f15dc0afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc071bb0 0x7f15dc0afb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f15cc000f70 tx=0x7f15cc00c820 comp rx=0 tx=0).stop 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 shutdown_connections 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15b4066950 0x7f15b4068e10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15dc15c5a0 0x7f15dc1373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15dc143780 0x7f15dc1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 --2- 172.21.15.143:0/1263631654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15dc071bb0 0x7f15dc0afb30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 >> 172.21.15.143:0/1263631654 conn(0x7f15dc06c3f0 msgr2=0x7f15dc1428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 shutdown_connections 2024-01-15T18:05:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.712+0000 7f15e3a00640 1 -- 172.21.15.143:0/1263631654 wait complete. 2024-01-15T18:05:39.735 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 Processor -- start 2024-01-15T18:05:39.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 -- start start 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3650143780 0x7f36500cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36500ce520 0x7f3650071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500ceb50 con 0x7f36500720f0 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365015c110 con 0x7f36500ce520 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3658200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3650072530 con 0x7f3650143780 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36500ce520 0x7f3650071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f36500ce520 0x7f3650071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59238/0 (socket says 172.21.15.143:59238) 2024-01-15T18:05:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 -- 172.21.15.143:0/257159850 learned_addr learned my addr 172.21.15.143:0/257159850 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:39.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3655800640 1 --2- 172.21.15.143:0/257159850 >> v2:172.21.15.204:3300/0 conn(0x7f3650143780 0x7f36500cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 -- 172.21.15.143:0/257159850 >> v2:172.21.15.204:3300/0 conn(0x7f3650143780 msgr2=0x7f36500cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:39.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 --2- 172.21.15.143:0/257159850 >> v2:172.21.15.204:3300/0 conn(0x7f3650143780 0x7f36500cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 -- 172.21.15.143:0/257159850 >> v1:172.21.15.204:6789/0 conn(0x7f36500720f0 legacy=0x7f365015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:39.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3654e00640 1 -- 172.21.15.143:0/257159850 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365015b7e0 con 0x7f36500ce520 2024-01-15T18:05:39.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.872+0000 7f3655800640 1 --2- 172.21.15.143:0/257159850 >> v2:172.21.15.204:3300/0 conn(0x7f3650143780 0x7f36500cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3654e00640 1 --2- 172.21.15.143:0/257159850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36500ce520 0x7f3650071bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f365006be30 tx=0x7f363801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398aac0f182227a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f364fe00640 1 -- 172.21.15.143:0/257159850 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380ac630 con 0x7f36500ce520 2024-01-15T18:05:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f364fe00640 1 -- 172.21.15.143:0/257159850 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36380ac790 con 0x7f36500ce520 2024-01-15T18:05:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f364fe00640 1 -- 172.21.15.143:0/257159850 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36380acb20 con 0x7f36500ce520 2024-01-15T18:05:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 -- 172.21.15.143:0/257159850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36500ce520 msgr2=0x7f3650071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 --2- 172.21.15.143:0/257159850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36500ce520 0x7f3650071bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f365006be30 tx=0x7f363801c4c0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 -- 172.21.15.143:0/257159850 shutdown_connections 2024-01-15T18:05:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 --2- 172.21.15.143:0/257159850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36500ce520 0x7f3650071bb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 --2- 172.21.15.143:0/257159850 >> v2:172.21.15.204:3300/0 conn(0x7f3650143780 0x7f36500cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 -- 172.21.15.143:0/257159850 >> 172.21.15.143:0/257159850 conn(0x7f365006c3f0 msgr2=0x7f365006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 -- 172.21.15.143:0/257159850 shutdown_connections 2024-01-15T18:05:39.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.876+0000 7f3658200640 1 -- 172.21.15.143:0/257159850 wait complete. 2024-01-15T18:05:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 Processor -- start 2024-01-15T18:05:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 -- start start 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36500720f0 0x7f3650158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500ce520 0x7f3650070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 0x7f3650070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650159450 con 0x7f3650143780 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3650159900 con 0x7f36500ce520 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3650136370 con 0x7f36500720f0 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 0x7f3650070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 0x7f3650070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59240/0 (socket says 172.21.15.143:59240) 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 -- 172.21.15.143:0/1201025541 learned_addr learned my addr 172.21.15.143:0/1201025541 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3655800640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36500720f0 0x7f3650158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3654e00640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500ce520 0x7f3650070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 -- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36500720f0 msgr2=0x7f3650158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36500720f0 0x7f3650158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 -- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500ce520 msgr2=0x7f3650070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500ce520 0x7f3650070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 -- 172.21.15.143:0/1201025541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380094d0 con 0x7f3650143780 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3656200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 0x7f3650070fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f364800c7b0 tx=0x7f364800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648004b20 con 0x7f3650143780 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648004c80 con 0x7f3650143780 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364800ce60 con 0x7f3650143780 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3650136650 con 0x7f3650143780 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.880+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3650136c10 con 0x7f3650143780 2024-01-15T18:05:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.884+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3650067500 con 0x7f3650143780 2024-01-15T18:05:39.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.884+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f364800a850 con 0x7f3650143780 2024-01-15T18:05:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.884+0000 7f364d600640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36280668f0 0x7f3628068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.884+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f36480844b0 con 0x7f3650143780 2024-01-15T18:05:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.884+0000 7f3655800640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36280668f0 0x7f3628068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.884+0000 7f3655800640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36280668f0 0x7f3628068db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3640006fd0 tx=0x7f3640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:39.888+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3648051710 con 0x7f3650143780 2024-01-15T18:05:40.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f365006e020 con 0x7f3650143780 2024-01-15T18:05:40.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f364d600640 1 -- 172.21.15.143:0/1201025541 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3648054d30 con 0x7f3650143780 2024-01-15T18:05:40.043 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36280668f0 msgr2=0x7f3628068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36280668f0 0x7f3628068db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3640006fd0 tx=0x7f3640008040 comp rx=0 tx=0).stop 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 msgr2=0x7f3650070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 0x7f3650070fe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f364800c7b0 tx=0x7f364800cc70 comp rx=0 tx=0).stop 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 shutdown_connections 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36280668f0 0x7f3628068db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3650143780 0x7f3650070fe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36500ce520 0x7f3650070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 --2- 172.21.15.143:0/1201025541 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36500720f0 0x7f3650158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.040+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 >> 172.21.15.143:0/1201025541 conn(0x7f365006c3f0 msgr2=0x7f36500ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:40.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.044+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 shutdown_connections 2024-01-15T18:05:40.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.044+0000 7f3658200640 1 -- 172.21.15.143:0/1201025541 wait complete. 2024-01-15T18:05:40.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:40.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 Processor -- start 2024-01-15T18:05:40.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 -- start start 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd83c0b3610 0x7fd83c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b45d0 0x7fd83c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c0cefa0 con 0x7fd83c0b4080 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c0c9fa0 con 0x7fd83c0b3610 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd841a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b3ef0 con 0x7fd83c0b45d0 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd83c0b4080 0x7fd83c0cddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57114/0 (socket says 172.21.15.143:57114) 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83b400640 1 -- 172.21.15.143:0/2163681506 learned_addr learned my addr 172.21.15.143:0/2163681506 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83aa00640 1 --2- 172.21.15.143:0/2163681506 >> v2:172.21.15.204:3300/0 conn(0x7fd83c0b3610 0x7fd83c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83a000640 1 --2- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b45d0 0x7fd83c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd839600640 1 -- 172.21.15.143:0/2163681506 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718544651 0 0) 0x7fd83c0cefa0 con 0x7fd83c0b4080 2024-01-15T18:05:40.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd839600640 1 -- 172.21.15.143:0/2163681506 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd81c003530 con 0x7fd83c0b4080 2024-01-15T18:05:40.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83aa00640 1 -- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b45d0 msgr2=0x7fd83c0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:40.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83aa00640 1 --2- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b45d0 0x7fd83c0cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83aa00640 1 -- 172.21.15.143:0/2163681506 >> v1:172.21.15.143:6789/0 conn(0x7fd83c0b4080 legacy=0x7fd83c0cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.200+0000 7fd83aa00640 1 -- 172.21.15.143:0/2163681506 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0d0f90 con 0x7fd83c0b3610 2024-01-15T18:05:40.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.204+0000 7fd83aa00640 1 --2- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b3610 0x7fd83c0cfa40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0097c0 tx=0x7fd82c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29d2fb7b0de09a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:40.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.204+0000 7fd839600640 1 -- 172.21.15.143:0/2163681506 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0a8970 con 0x7fd83c0b3610 2024-01-15T18:05:40.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.204+0000 7fd839600640 1 -- 172.21.15.143:0/2163681506 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c0a8ad0 con 0x7fd83c0b3610 2024-01-15T18:05:40.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.204+0000 7fd839600640 1 -- 172.21.15.143:0/2163681506 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0a8e60 con 0x7fd83c0b3610 2024-01-15T18:05:40.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.204+0000 7fd841a00640 1 -- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b3610 msgr2=0x7fd83c0cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.204+0000 7fd841a00640 1 --2- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b3610 0x7fd83c0cfa40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0097c0 tx=0x7fd82c01ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:40.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- 172.21.15.143:0/2163681506 shutdown_connections 2024-01-15T18:05:40.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 --2- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b45d0 0x7fd83c0cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 --2- 172.21.15.143:0/2163681506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b3610 0x7fd83c0cfa40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- 172.21.15.143:0/2163681506 >> 172.21.15.143:0/2163681506 conn(0x7fd83c0d3030 msgr2=0x7fd83c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:40.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- 172.21.15.143:0/2163681506 shutdown_connections 2024-01-15T18:05:40.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- 172.21.15.143:0/2163681506 wait complete. 2024-01-15T18:05:40.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 Processor -- start 2024-01-15T18:05:40.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- start start 2024-01-15T18:05:40.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 0x7fd83c1f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b4080 0x7fd83c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b45d0 0x7fd83c1f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b2870 con 0x7fd83c0b3610 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b1a00 con 0x7fd83c0b45d0 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd841a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c1f0a30 con 0x7fd83c0b4080 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 0x7fd83c1f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 0x7fd83c1f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59242/0 (socket says 172.21.15.143:59242) 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 -- 172.21.15.143:0/4155236522 learned_addr learned my addr 172.21.15.143:0/4155236522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83a000640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b4080 0x7fd83c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83b400640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b45d0 0x7fd83c1f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 -- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b4080 msgr2=0x7fd83c0afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b4080 0x7fd83c0afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 -- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b45d0 msgr2=0x7fd83c1f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b45d0 0x7fd83c1f66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 -- 172.21.15.143:0/4155236522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c009420 con 0x7fd83c0b3610 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83b400640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b45d0 0x7fd83c1f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.208+0000 7fd83aa00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 0x7fd83c1f04f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0b3040 tx=0x7fd82c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0b0d60 con 0x7fd83c0b3610 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c0b9430 con 0x7fd83c0b3610 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0b97e0 con 0x7fd83c0b3610 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c0b1c60 con 0x7fd83c0b3610 2024-01-15T18:05:40.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c0b2220 con 0x7fd83c0b3610 2024-01-15T18:05:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c067500 con 0x7fd83c0b3610 2024-01-15T18:05:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fd82c0b7040 con 0x7fd83c0b3610 2024-01-15T18:05:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd832a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd814066920 0x7fd814068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd82c121fe0 con 0x7fd83c0b3610 2024-01-15T18:05:40.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd83a000640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd814066920 0x7fd814068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.212+0000 7fd83a000640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd814066920 0x7fd814068de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0b6140 tx=0x7fd824008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:40.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.216+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd82c0ef120 con 0x7fd83c0b3610 2024-01-15T18:05:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd83c0b8eb0 con 0x7fd83c0b3610 2024-01-15T18:05:40.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd832a00640 1 -- 172.21.15.143:0/4155236522 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd82c0b50a0 con 0x7fd83c0b3610 2024-01-15T18:05:40.370 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd814066920 msgr2=0x7fd814068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd814066920 0x7fd814068de0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0b6140 tx=0x7fd824008040 comp rx=0 tx=0).stop 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 msgr2=0x7fd83c1f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 0x7fd83c1f04f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0b3040 tx=0x7fd82c01ba10 comp rx=0 tx=0).stop 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 shutdown_connections 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd814066920 0x7fd814068de0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd83c0b45d0 0x7fd83c1f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd83c0b4080 0x7fd83c0afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 --2- 172.21.15.143:0/4155236522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd83c0b3610 0x7fd83c1f04f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 >> 172.21.15.143:0/4155236522 conn(0x7fd83c0d3030 msgr2=0x7fd83c0ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 shutdown_connections 2024-01-15T18:05:40.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.368+0000 7fd841a00640 1 -- 172.21.15.143:0/4155236522 wait complete. 2024-01-15T18:05:40.392 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.528+0000 7f5acf400640 1 Processor -- start 2024-01-15T18:05:40.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.528+0000 7f5acf400640 1 -- start start 2024-01-15T18:05:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acf400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5ac806bee0 0x7f5ac815a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acf400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac806c2c0 con 0x7f5ac815b5d0 2024-01-15T18:05:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acf400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac80b3780 con 0x7f5ac815a950 2024-01-15T18:05:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acf400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac815aed0 con 0x7f5ac806bee0 2024-01-15T18:05:40.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acde00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5ac815b5d0 0x7f5ac80b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57122/0 (socket says 172.21.15.143:57122) 2024-01-15T18:05:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acde00640 1 -- 172.21.15.143:0/642624010 learned_addr learned my addr 172.21.15.143:0/642624010 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acca00640 1 --2- 172.21.15.143:0/642624010 >> v2:172.21.15.143:3300/0 conn(0x7f5ac806bee0 0x7f5ac815a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acca00640 1 -- 172.21.15.143:0/642624010 >> v1:172.21.15.204:6789/0 conn(0x7f5ac815a950 legacy=0x7f5ac80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acca00640 1 -- 172.21.15.143:0/642624010 >> v1:172.21.15.143:6789/0 conn(0x7f5ac815b5d0 legacy=0x7f5ac80b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acca00640 1 -- 172.21.15.143:0/642624010 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac815b1b0 con 0x7f5ac806bee0 2024-01-15T18:05:40.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5ac7e00640 1 -- 172.21.15.143:0/642624010 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419267149 0 0) 0x7f5ac00014a0 con 0x7f5ac815b5d0 2024-01-15T18:05:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5acca00640 1 --2- 172.21.15.143:0/642624010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac815a410 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80097c0 tx=0x7f5ab801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c0ea4ca9a97d54 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.532+0000 7f5ac7e00640 1 -- 172.21.15.143:0/642624010 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80a6040 con 0x7f5ac806bee0 2024-01-15T18:05:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5ac7e00640 1 -- 172.21.15.143:0/642624010 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab80aa600 con 0x7f5ac806bee0 2024-01-15T18:05:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5ac7e00640 1 -- 172.21.15.143:0/642624010 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80aa990 con 0x7f5ac806bee0 2024-01-15T18:05:40.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- 172.21.15.143:0/642624010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 msgr2=0x7f5ac815a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 --2- 172.21.15.143:0/642624010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac815a410 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5ab80097c0 tx=0x7f5ab801c480 comp rx=0 tx=0).stop 2024-01-15T18:05:40.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- 172.21.15.143:0/642624010 shutdown_connections 2024-01-15T18:05:40.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 --2- 172.21.15.143:0/642624010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac815a410 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- 172.21.15.143:0/642624010 >> 172.21.15.143:0/642624010 conn(0x7f5ac8066720 msgr2=0x7f5ac806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:40.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- 172.21.15.143:0/642624010 shutdown_connections 2024-01-15T18:05:40.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- 172.21.15.143:0/642624010 wait complete. 2024-01-15T18:05:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 Processor -- start 2024-01-15T18:05:40.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- start start 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac80cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ac815a950 0x7f5ac80b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ac815b5d0 0x7f5ac80b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac813c130 con 0x7f5ac806bee0 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80ac8e0 con 0x7f5ac815b5d0 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.536+0000 7f5acf400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac80cfed0 con 0x7f5ac815a950 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac80cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac80cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59264/0 (socket says 172.21.15.143:59264) 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 -- 172.21.15.143:0/1224626245 learned_addr learned my addr 172.21.15.143:0/1224626245 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acde00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ac815b5d0 0x7f5ac80b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5abfe00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ac815a950 0x7f5ac80b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 -- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ac815a950 msgr2=0x7f5ac80b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ac815a950 0x7f5ac80b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 -- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ac815b5d0 msgr2=0x7f5ac80b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ac815b5d0 0x7f5ac80b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 -- 172.21.15.143:0/1224626245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab8009420 con 0x7f5ac806bee0 2024-01-15T18:05:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acca00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac80cf5a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8019090 tx=0x7f5ab801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:40.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80b3800 con 0x7f5ac806bee0 2024-01-15T18:05:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab80b3960 con 0x7f5ac806bee0 2024-01-15T18:05:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab80b3d10 con 0x7f5ac806bee0 2024-01-15T18:05:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acf400640 1 -- 172.21.15.143:0/1224626245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac813c380 con 0x7f5ac806bee0 2024-01-15T18:05:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5acf400640 1 -- 172.21.15.143:0/1224626245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac813c940 con 0x7f5ac806bee0 2024-01-15T18:05:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac813cb30 con 0x7f5ac806bee0 2024-01-15T18:05:40.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5ab80b2380 con 0x7f5ac806bee0 2024-01-15T18:05:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.540+0000 7f5ac5600640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5aa00668f0 0x7f5aa0068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.544+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5ab811ead0 con 0x7f5ac806bee0 2024-01-15T18:05:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.544+0000 7f5abfe00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5aa00668f0 0x7f5aa0068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:40.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.544+0000 7f5abfe00640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5aa00668f0 0x7f5aa0068db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8157ad0 tx=0x7f5ab0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:40.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.544+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ab80ebda0 con 0x7f5ac806bee0 2024-01-15T18:05:40.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.708+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5ac8067680 con 0x7f5ac806bee0 2024-01-15T18:05:40.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.708+0000 7f5ac5600640 1 -- 172.21.15.143:0/1224626245 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ab80ef3c0 con 0x7f5ac806bee0 2024-01-15T18:05:40.711 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.708+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5aa00668f0 msgr2=0x7f5aa0068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.708+0000 7f5abe000640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5aa00668f0 0x7f5aa0068db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8157ad0 tx=0x7f5ab0009040 comp rx=0 tx=0).stop 2024-01-15T18:05:40.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.708+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 msgr2=0x7f5ac80cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.708+0000 7f5abe000640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac80cf5a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8019090 tx=0x7f5ab801acc0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 shutdown_connections 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5aa00668f0 0x7f5aa0068db0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ac815b5d0 0x7f5ac80b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ac815a950 0x7f5ac80b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 --2- 172.21.15.143:0/1224626245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ac806bee0 0x7f5ac80cf5a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 >> 172.21.15.143:0/1224626245 conn(0x7f5ac8066720 msgr2=0x7f5ac806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 shutdown_connections 2024-01-15T18:05:40.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.712+0000 7f5abe000640 1 -- 172.21.15.143:0/1224626245 wait complete. 2024-01-15T18:05:40.732 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:05:40.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:40.732 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-01-15T18:05:40.752 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:40.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:05:40.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:40.753 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:05:40.801 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:05:40.801 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:40.811 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:05:40.811 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:40.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:41.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.996+0000 7f0829c00640 1 Processor -- start 2024-01-15T18:05:41.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:40.996+0000 7f0829c00640 1 -- start start 2024-01-15T18:05:41.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0829c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0824141660 0x7f08240ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0829c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f082415ec80 0x7f0824141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0829c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082415f310 con 0x7f0824142470 2024-01-15T18:05:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0829c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082415fbb0 con 0x7f0824141660 2024-01-15T18:05:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0829c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082415f100 con 0x7f082415ec80 2024-01-15T18:05:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0824141660 0x7f08240ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0823400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0824142470 0x7f0824071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57128/0 (socket says 172.21.15.143:57128) 2024-01-15T18:05:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0823400640 1 -- 172.21.15.143:0/3808197171 learned_addr learned my addr 172.21.15.143:0/3808197171 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822000640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f082415ec80 0x7f0824141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822a00640 1 -- 172.21.15.143:0/3808197171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f082415ec80 msgr2=0x7f0824141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822a00640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f082415ec80 0x7f0824141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0821600640 1 -- 172.21.15.143:0/3808197171 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715034148 0 0) 0x7f082415f310 con 0x7f0824142470 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822a00640 1 -- 172.21.15.143:0/3808197171 >> v1:172.21.15.143:6789/0 conn(0x7f0824142470 legacy=0x7f0824071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822a00640 1 -- 172.21.15.143:0/3808197171 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082415f5f0 con 0x7f0824141660 2024-01-15T18:05:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.000+0000 7f0822000640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f082415ec80 0x7f0824141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:41.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0822a00640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f08240ce560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08140097c0 tx=0x7f081401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a0c8cedd3f73d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0821600640 1 -- 172.21.15.143:0/3808197171 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140ab650 con 0x7f0824141660 2024-01-15T18:05:41.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0821600640 1 -- 172.21.15.143:0/3808197171 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08140ab7b0 con 0x7f0824141660 2024-01-15T18:05:41.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0821600640 1 -- 172.21.15.143:0/3808197171 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140abb40 con 0x7f0824141660 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 -- 172.21.15.143:0/3808197171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 msgr2=0x7f08240ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f08240ce560 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08140097c0 tx=0x7f081401c450 comp rx=0 tx=0).stop 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 -- 172.21.15.143:0/3808197171 shutdown_connections 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f082415ec80 0x7f0824141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 --2- 172.21.15.143:0/3808197171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f08240ce560 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 -- 172.21.15.143:0/3808197171 >> 172.21.15.143:0/3808197171 conn(0x7f082406c3f0 msgr2=0x7f082406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 -- 172.21.15.143:0/3808197171 shutdown_connections 2024-01-15T18:05:41.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 -- 172.21.15.143:0/3808197171 wait complete. 2024-01-15T18:05:41.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 Processor -- start 2024-01-15T18:05:41.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.004+0000 7f0829c00640 1 -- start start 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f082406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0824142470 0x7f0824159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f082415ec80 0x7f082415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082413d730 con 0x7f0824141660 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082415a230 con 0x7f082415ec80 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082413d980 con 0x7f0824142470 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f082406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f082406d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59290/0 (socket says 172.21.15.143:59290) 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 -- 172.21.15.143:0/2857928082 learned_addr learned my addr 172.21.15.143:0/2857928082 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0823400640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f082415ec80 0x7f082415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822000640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0824142470 0x7f0824159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 -- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0824142470 msgr2=0x7f0824159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0824142470 0x7f0824159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 -- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f082415ec80 msgr2=0x7f082415c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f082415ec80 0x7f082415c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 -- 172.21.15.143:0/2857928082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009420 con 0x7f0824141660 2024-01-15T18:05:41.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0822a00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f082406d7e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f081401ab00 tx=0x7f081401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140b5810 con 0x7f0824141660 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08140b5990 con 0x7f0824141660 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08240cd150 con 0x7f0824141660 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140b5d40 con 0x7f0824141660 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082413df40 con 0x7f0824141660 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.008+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0824067500 con 0x7f0824141660 2024-01-15T18:05:41.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.012+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f08140bf410 con 0x7f0824141660 2024-01-15T18:05:41.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.012+0000 7f081aa00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07fc0668f0 0x7f07fc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.012+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f081411f480 con 0x7f0824141660 2024-01-15T18:05:41.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.012+0000 7f0822000640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07fc0668f0 0x7f07fc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.012+0000 7f0822000640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07fc0668f0 0x7f07fc068db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f082415b1e0 tx=0x7f080c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.016+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f08140ec6e0 con 0x7f0824141660 2024-01-15T18:05:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.164+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f082413dac0 con 0x7f0824141660 2024-01-15T18:05:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f081aa00640 1 -- 172.21.15.143:0/2857928082 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f082413dac0 con 0x7f0824141660 2024-01-15T18:05:41.169 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:41.170 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:40.506528+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07fc0668f0 msgr2=0x7f07fc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07fc0668f0 0x7f07fc068db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f082415b1e0 tx=0x7f080c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 msgr2=0x7f082406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f082406d7e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f081401ab00 tx=0x7f081401ac90 comp rx=0 tx=0).stop 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 shutdown_connections 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07fc0668f0 0x7f07fc068db0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f082415ec80 0x7f082415c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0824142470 0x7f0824159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 --2- 172.21.15.143:0/2857928082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0824141660 0x7f082406d7e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 >> 172.21.15.143:0/2857928082 conn(0x7f082406c3f0 msgr2=0x7f08240ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 shutdown_connections 2024-01-15T18:05:41.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.168+0000 7f0829c00640 1 -- 172.21.15.143:0/2857928082 wait complete. 2024-01-15T18:05:41.192 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:41.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 Processor -- start 2024-01-15T18:05:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 -- start start 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa56c141660 0x7fa56c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa56c15ec80 0x7fa56c141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c15f310 con 0x7fa56c142470 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c15fbb0 con 0x7fa56c141660 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa574800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c15f130 con 0x7fa56c15ec80 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa56c141660 0x7fa56c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa56c141660 0x7fa56c0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59294/0 (socket says 172.21.15.143:59294) 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 -- 172.21.15.143:0/1698941369 learned_addr learned my addr 172.21.15.143:0/1698941369 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571400640 1 --2- 172.21.15.143:0/1698941369 >> v2:172.21.15.204:3300/0 conn(0x7fa56c15ec80 0x7fa56c141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 -- 172.21.15.143:0/1698941369 >> v2:172.21.15.204:3300/0 conn(0x7fa56c15ec80 msgr2=0x7fa56c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 --2- 172.21.15.143:0/1698941369 >> v2:172.21.15.204:3300/0 conn(0x7fa56c15ec80 0x7fa56c141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 -- 172.21.15.143:0/1698941369 >> v1:172.21.15.204:6789/0 conn(0x7fa56c142470 legacy=0x7fa56c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.328+0000 7fa571e00640 1 -- 172.21.15.143:0/1698941369 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c15f5f0 con 0x7fa56c141660 2024-01-15T18:05:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa571e00640 1 --2- 172.21.15.143:0/1698941369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c141660 0x7fa56c0ce560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa56c06f940 tx=0x7fa55c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ff9b91287ec4c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa570a00640 1 -- 172.21.15.143:0/1698941369 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0aa430 con 0x7fa56c141660 2024-01-15T18:05:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa570a00640 1 -- 172.21.15.143:0/1698941369 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0aa590 con 0x7fa56c141660 2024-01-15T18:05:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa570a00640 1 -- 172.21.15.143:0/1698941369 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0aa920 con 0x7fa56c141660 2024-01-15T18:05:41.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 -- 172.21.15.143:0/1698941369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c141660 msgr2=0x7fa56c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 --2- 172.21.15.143:0/1698941369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c141660 0x7fa56c0ce560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa56c06f940 tx=0x7fa55c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 -- 172.21.15.143:0/1698941369 shutdown_connections 2024-01-15T18:05:41.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 --2- 172.21.15.143:0/1698941369 >> v2:172.21.15.204:3300/0 conn(0x7fa56c15ec80 0x7fa56c141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 --2- 172.21.15.143:0/1698941369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c141660 0x7fa56c0ce560 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 -- 172.21.15.143:0/1698941369 >> 172.21.15.143:0/1698941369 conn(0x7fa56c06c3f0 msgr2=0x7fa56c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 -- 172.21.15.143:0/1698941369 shutdown_connections 2024-01-15T18:05:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.332+0000 7fa574800640 1 -- 172.21.15.143:0/1698941369 wait complete. 2024-01-15T18:05:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 Processor -- start 2024-01-15T18:05:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 -- start start 2024-01-15T18:05:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c141660 0x7fa56c1f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c142470 0x7fa56c1f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 0x7fa56c143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1f6a40 con 0x7fa56c15ec80 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1f9fb0 con 0x7fa56c142470 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1f5ec0 con 0x7fa56c141660 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 0x7fa56c143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 0x7fa56c143b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59308/0 (socket says 172.21.15.143:59308) 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 -- 172.21.15.143:0/4155899102 learned_addr learned my addr 172.21.15.143:0/4155899102 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa571400640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c142470 0x7fa56c1f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa571e00640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c141660 0x7fa56c1f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 -- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c141660 msgr2=0x7fa56c1f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c141660 0x7fa56c1f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 -- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c142470 msgr2=0x7fa56c1f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c142470 0x7fa56c1f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 -- 172.21.15.143:0/4155899102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0094d0 con 0x7fa56c15ec80 2024-01-15T18:05:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa572800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 0x7fa56c143b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c6c0 tx=0x7fa56400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564003b80 con 0x7fa56c15ec80 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564003ce0 con 0x7fa56c15ec80 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c0cd150 con 0x7fa56c15ec80 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640052f0 con 0x7fa56c15ec80 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.336+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c1f6480 con 0x7fa56c15ec80 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.340+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c15d470 con 0x7fa56c15ec80 2024-01-15T18:05:41.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.340+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fa564005450 con 0x7fa56c15ec80 2024-01-15T18:05:41.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.340+0000 7fa56a000640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5440668f0 0x7fa544068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.340+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa564084160 con 0x7fa56c15ec80 2024-01-15T18:05:41.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.340+0000 7fa571e00640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5440668f0 0x7fa544068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.340+0000 7fa571e00640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5440668f0 0x7fa544068db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ae000 tx=0x7fa55c0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.344+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa56c15d470 con 0x7fa56c15ec80 2024-01-15T18:05:41.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.492+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa56c15d470 con 0x7fa56c15ec80 2024-01-15T18:05:41.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.492+0000 7fa56a000640 1 -- 172.21.15.143:0/4155899102 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa56c15d470 con 0x7fa56c15ec80 2024-01-15T18:05:41.497 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5440668f0 msgr2=0x7fa544068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5440668f0 0x7fa544068db0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ae000 tx=0x7fa55c0b5040 comp rx=0 tx=0).stop 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 msgr2=0x7fa56c143b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 0x7fa56c143b00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa56400c6c0 tx=0x7fa56400cb80 comp rx=0 tx=0).stop 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 shutdown_connections 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5440668f0 0x7fa544068db0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa56c15ec80 0x7fa56c143b00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa56c142470 0x7fa56c1f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 --2- 172.21.15.143:0/4155899102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa56c141660 0x7fa56c1f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 >> 172.21.15.143:0/4155899102 conn(0x7fa56c06c3f0 msgr2=0x7fa56c0af120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 shutdown_connections 2024-01-15T18:05:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.496+0000 7fa574800640 1 -- 172.21.15.143:0/4155899102 wait complete. 2024-01-15T18:05:41.519 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:41.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 Processor -- start 2024-01-15T18:05:41.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 -- start start 2024-01-15T18:05:41.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa41006bfb0 0x7fa41006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa4100b0310 0x7fa4100cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4100cbff0 con 0x7fa4100cc9c0 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4100d0c50 con 0x7fa41006bfb0 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa416800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41006c8d0 con 0x7fa4100b0310 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa41006bfb0 0x7fa41006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa41006bfb0 0x7fa41006c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59322/0 (socket says 172.21.15.143:59322) 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 -- 172.21.15.143:0/2517206531 learned_addr learned my addr 172.21.15.143:0/2517206531 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40ea00640 1 --2- 172.21.15.143:0/2517206531 >> v2:172.21.15.204:3300/0 conn(0x7fa4100b0310 0x7fa4100cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 -- 172.21.15.143:0/2517206531 >> v2:172.21.15.204:3300/0 conn(0x7fa4100b0310 msgr2=0x7fa4100cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 --2- 172.21.15.143:0/2517206531 >> v2:172.21.15.204:3300/0 conn(0x7fa4100b0310 0x7fa4100cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 -- 172.21.15.143:0/2517206531 >> v1:172.21.15.204:6789/0 conn(0x7fa4100cc9c0 legacy=0x7fa4100cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.655+0000 7fa40f400640 1 -- 172.21.15.143:0/2517206531 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4100cb6c0 con 0x7fa41006bfb0 2024-01-15T18:05:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa40f400640 1 --2- 172.21.15.143:0/2517206531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa41006bfb0 0x7fa41006c390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa41006bf00 tx=0x7fa40001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd6275056064116 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa40e000640 1 -- 172.21.15.143:0/2517206531 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000ab640 con 0x7fa41006bfb0 2024-01-15T18:05:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa40e000640 1 -- 172.21.15.143:0/2517206531 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4000a7070 con 0x7fa41006bfb0 2024-01-15T18:05:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa40e000640 1 -- 172.21.15.143:0/2517206531 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4000ab9c0 con 0x7fa41006bfb0 2024-01-15T18:05:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa416800640 1 -- 172.21.15.143:0/2517206531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa41006bfb0 msgr2=0x7fa41006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa416800640 1 --2- 172.21.15.143:0/2517206531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa41006bfb0 0x7fa41006c390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa41006bf00 tx=0x7fa40001c440 comp rx=0 tx=0).stop 2024-01-15T18:05:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa416800640 1 -- 172.21.15.143:0/2517206531 shutdown_connections 2024-01-15T18:05:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa416800640 1 --2- 172.21.15.143:0/2517206531 >> v2:172.21.15.204:3300/0 conn(0x7fa4100b0310 0x7fa4100cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa416800640 1 --2- 172.21.15.143:0/2517206531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa41006bfb0 0x7fa41006c390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.659+0000 7fa416800640 1 -- 172.21.15.143:0/2517206531 >> 172.21.15.143:0/2517206531 conn(0x7fa4100d0ed0 msgr2=0x7fa4100d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 -- 172.21.15.143:0/2517206531 shutdown_connections 2024-01-15T18:05:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 -- 172.21.15.143:0/2517206531 wait complete. 2024-01-15T18:05:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 Processor -- start 2024-01-15T18:05:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 -- start start 2024-01-15T18:05:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41006bfb0 0x7fa4100c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 0x7fa41015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4100cc9c0 0x7fa4100ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410153320 con 0x7fa4100b0310 2024-01-15T18:05:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410158450 con 0x7fa41006bfb0 2024-01-15T18:05:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa416800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41015ca90 con 0x7fa4100cc9c0 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 0x7fa41015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 0x7fa41015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59334/0 (socket says 172.21.15.143:59334) 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 -- 172.21.15.143:0/808498073 learned_addr learned my addr 172.21.15.143:0/808498073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40f400640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41006bfb0 0x7fa4100c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40fe00640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4100cc9c0 0x7fa4100ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 -- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4100cc9c0 msgr2=0x7fa4100ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4100cc9c0 0x7fa4100ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 -- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41006bfb0 msgr2=0x7fa4100c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41006bfb0 0x7fa4100c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 -- 172.21.15.143:0/808498073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4000094d0 con 0x7fa4100b0310 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa40ea00640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 0x7fa41015c550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e870 tx=0x7fa3f800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.663+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8009510 con 0x7fa4100b0310 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f800eec0 con 0x7fa4100b0310 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4101529b0 con 0x7fa4100b0310 2024-01-15T18:05:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800cb80 con 0x7fa4100b0310 2024-01-15T18:05:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101580e0 con 0x7fa4100b0310 2024-01-15T18:05:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa410157c60 con 0x7fa4100b0310 2024-01-15T18:05:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fa3f800cce0 con 0x7fa4100b0310 2024-01-15T18:05:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa407400640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3e80668f0 0x7fa3e8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.667+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa3f80844d0 con 0x7fa4100b0310 2024-01-15T18:05:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.671+0000 7fa40f400640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3e80668f0 0x7fa3e8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.671+0000 7fa40f400640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3e80668f0 0x7fa3e8068db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4000af000 tx=0x7fa4000b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.671+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa410157c60 con 0x7fa4100b0310 2024-01-15T18:05:41.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa410158600 con 0x7fa4100b0310 2024-01-15T18:05:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa407400640 1 -- 172.21.15.143:0/808498073 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa410158600 con 0x7fa4100b0310 2024-01-15T18:05:41.827 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3e80668f0 msgr2=0x7fa3e8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3e80668f0 0x7fa3e8068db0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa4000af000 tx=0x7fa4000b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 msgr2=0x7fa41015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 0x7fa41015c550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800e870 tx=0x7fa3f800ec70 comp rx=0 tx=0).stop 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 shutdown_connections 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3e80668f0 0x7fa3e8068db0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa4100cc9c0 0x7fa4100ca880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4100b0310 0x7fa41015c550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 --2- 172.21.15.143:0/808498073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa41006bfb0 0x7fa4100c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 >> 172.21.15.143:0/808498073 conn(0x7fa4100d0ed0 msgr2=0x7fa41006f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 shutdown_connections 2024-01-15T18:05:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.823+0000 7fa416800640 1 -- 172.21.15.143:0/808498073 wait complete. 2024-01-15T18:05:41.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:41.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e82c00640 1 Processor -- start 2024-01-15T18:05:41.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e82c00640 1 -- start start 2024-01-15T18:05:41.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e82c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e7c157300 0x7f3e7c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e82c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c0b2a20 con 0x7f3e7c0b4ac0 2024-01-15T18:05:41.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e82c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c0b2d00 con 0x7f3e7c0b21d0 2024-01-15T18:05:41.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e82c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b35a0 con 0x7f3e7c157300 2024-01-15T18:05:41.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e7be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e7c157300 0x7f3e7c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e80c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e7c0b4ac0 0x7f3e7c070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57140/0 (socket says 172.21.15.143:57140) 2024-01-15T18:05:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e80c00640 1 -- 172.21.15.143:0/1799737798 learned_addr learned my addr 172.21.15.143:0/1799737798 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e7be00640 1 -- 172.21.15.143:0/1799737798 >> v1:172.21.15.204:6789/0 conn(0x7f3e7c0b21d0 legacy=0x7f3e7c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e7be00640 1 -- 172.21.15.143:0/1799737798 >> v1:172.21.15.143:6789/0 conn(0x7f3e7c0b4ac0 legacy=0x7f3e7c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e7be00640 1 -- 172.21.15.143:0/1799737798 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0b2fe0 con 0x7f3e7c157300 2024-01-15T18:05:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.983+0000 7f3e7aa00640 1 -- 172.21.15.143:0/1799737798 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114571151 0 0) 0x7f3e7c0b2a20 con 0x7f3e7c0b4ac0 2024-01-15T18:05:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e7be00640 1 --2- 172.21.15.143:0/1799737798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c159b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0097c0 tx=0x7f3e6c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9436dbf0ea9416f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e7aa00640 1 -- 172.21.15.143:0/1799737798 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0a7070 con 0x7f3e7c157300 2024-01-15T18:05:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e7aa00640 1 -- 172.21.15.143:0/1799737798 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0ab430 con 0x7f3e7c157300 2024-01-15T18:05:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e7aa00640 1 -- 172.21.15.143:0/1799737798 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0ab7c0 con 0x7f3e7c157300 2024-01-15T18:05:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e82c00640 1 -- 172.21.15.143:0/1799737798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 msgr2=0x7f3e7c159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e82c00640 1 --2- 172.21.15.143:0/1799737798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c159b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0097c0 tx=0x7f3e6c01c410 comp rx=0 tx=0).stop 2024-01-15T18:05:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e82c00640 1 -- 172.21.15.143:0/1799737798 shutdown_connections 2024-01-15T18:05:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e82c00640 1 --2- 172.21.15.143:0/1799737798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c159b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.987+0000 7f3e82c00640 1 -- 172.21.15.143:0/1799737798 >> 172.21.15.143:0/1799737798 conn(0x7f3e7c06c3f0 msgr2=0x7f3e7c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 -- 172.21.15.143:0/1799737798 shutdown_connections 2024-01-15T18:05:41.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 -- 172.21.15.143:0/1799737798 wait complete. 2024-01-15T18:05:41.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 Processor -- start 2024-01-15T18:05:41.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 -- start start 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e7c0b21d0 0x7f3e7c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e7c0b4ac0 0x7f3e7c15c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14e7c0 con 0x7f3e7c157300 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c157990 con 0x7f3e7c0b4ac0 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e82c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0ae390 con 0x7f3e7c0b21d0 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59360/0 (socket says 172.21.15.143:59360) 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 -- 172.21.15.143:0/313307386 learned_addr learned my addr 172.21.15.143:0/313307386 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:41.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e7be00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e7c0b21d0 0x7f3e7c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e7b400640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e7c0b4ac0 0x7f3e7c15c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 -- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e7c0b21d0 msgr2=0x7f3e7c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e7c0b21d0 0x7f3e7c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 -- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e7c0b4ac0 msgr2=0x7f3e7c15c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e7c0b4ac0 0x7f3e7c15c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 -- 172.21.15.143:0/313307386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c009420 con 0x7f3e7c157300 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e7be00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e7c0b21d0 0x7f3e7c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:41.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.991+0000 7f3e80c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c14ec50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e74001570 tx=0x7f3e7400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400cea0 con 0x7f3e7c157300 2024-01-15T18:05:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7400a5e0 con 0x7f3e7c157300 2024-01-15T18:05:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7400a990 con 0x7f3e7c157300 2024-01-15T18:05:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c0adaf0 con 0x7f3e7c157300 2024-01-15T18:05:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c0ae0b0 con 0x7f3e7c157300 2024-01-15T18:05:41.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c14e970 con 0x7f3e7c157300 2024-01-15T18:05:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3e7400aaf0 con 0x7f3e7c157300 2024-01-15T18:05:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e73e00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e540668f0 0x7f3e54068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3e740839d0 con 0x7f3e7c157300 2024-01-15T18:05:42.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.995+0000 7f3e7be00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e540668f0 0x7f3e54068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.999+0000 7f3e7be00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e540668f0 0x7f3e54068db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0af000 tx=0x7f3e6c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:41.999+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e7c14e970 con 0x7f3e7c157300 2024-01-15T18:05:42.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e7c067500 con 0x7f3e7c157300 2024-01-15T18:05:42.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e73e00640 1 -- 172.21.15.143:0/313307386 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e74018070 con 0x7f3e7c157300 2024-01-15T18:05:42.155 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:42.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e540668f0 msgr2=0x7f3e54068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e540668f0 0x7f3e54068db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0af000 tx=0x7f3e6c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:42.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 msgr2=0x7f3e7c14ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c14ec50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3e74001570 tx=0x7f3e7400cc50 comp rx=0 tx=0).stop 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 shutdown_connections 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e540668f0 0x7f3e54068db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e7c157300 0x7f3e7c14ec50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e7c0b4ac0 0x7f3e7c15c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 --2- 172.21.15.143:0/313307386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e7c0b21d0 0x7f3e7c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.151+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 >> 172.21.15.143:0/313307386 conn(0x7f3e7c06c3f0 msgr2=0x7f3e7c0aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.155+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 shutdown_connections 2024-01-15T18:05:42.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.155+0000 7f3e82c00640 1 -- 172.21.15.143:0/313307386 wait complete. 2024-01-15T18:05:42.177 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c3800640 1 Processor -- start 2024-01-15T18:05:42.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c3800640 1 -- start start 2024-01-15T18:05:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc06aba0 0x7fc6bc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc160eb0 con 0x7fc6bc15d130 2024-01-15T18:05:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c3800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc161190 con 0x7fc6bc15d510 2024-01-15T18:05:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc161470 con 0x7fc6bc06aba0 2024-01-15T18:05:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c0e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc6bc15d130 0x7fc6bc06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57154/0 (socket says 172.21.15.143:57154) 2024-01-15T18:05:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6c0e00640 1 -- 172.21.15.143:0/507180231 learned_addr learned my addr 172.21.15.143:0/507180231 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:42.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bbe00640 1 --2- 172.21.15.143:0/507180231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc06aba0 0x7fc6bc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897748495 0 0) 0x7fc6bc160eb0 con 0x7fc6bc15d130 2024-01-15T18:05:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc69c003530 con 0x7fc6bc15d130 2024-01-15T18:05:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853059888 0 0) 0x7fc6bc161190 con 0x7fc6bc15d510 2024-01-15T18:05:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc160eb0 con 0x7fc6bc15d510 2024-01-15T18:05:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bbe00640 1 -- 172.21.15.143:0/507180231 >> v1:172.21.15.204:6789/0 conn(0x7fc6bc15d510 legacy=0x7fc6bc0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bbe00640 1 -- 172.21.15.143:0/507180231 >> v1:172.21.15.143:6789/0 conn(0x7fc6bc15d130 legacy=0x7fc6bc06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.311+0000 7fc6bbe00640 1 -- 172.21.15.143:0/507180231 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc161750 con 0x7fc6bc06aba0 2024-01-15T18:05:42.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6bbe00640 1 --2- 172.21.15.143:0/507180231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc06aba0 0x7fc6bc0aeed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40097f0 tx=0x7fc6a401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9bec7349118e284 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40a6040 con 0x7fc6bc06aba0 2024-01-15T18:05:42.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a40aa600 con 0x7fc6bc06aba0 2024-01-15T18:05:42.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6bb400640 1 -- 172.21.15.143:0/507180231 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40aa990 con 0x7fc6bc06aba0 2024-01-15T18:05:42.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 -- 172.21.15.143:0/507180231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc06aba0 msgr2=0x7fc6bc0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 --2- 172.21.15.143:0/507180231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc06aba0 0x7fc6bc0aeed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40097f0 tx=0x7fc6a401c4c0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 -- 172.21.15.143:0/507180231 shutdown_connections 2024-01-15T18:05:42.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 --2- 172.21.15.143:0/507180231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc06aba0 0x7fc6bc0aeed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 -- 172.21.15.143:0/507180231 >> 172.21.15.143:0/507180231 conn(0x7fc6bc0b1b90 msgr2=0x7fc6bc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 -- 172.21.15.143:0/507180231 shutdown_connections 2024-01-15T18:05:42.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.315+0000 7fc6c3800640 1 -- 172.21.15.143:0/507180231 wait complete. 2024-01-15T18:05:42.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 Processor -- start 2024-01-15T18:05:42.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 -- start start 2024-01-15T18:05:42.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 0x7fc6bc150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc15d130 0x7fc6bc158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6bc15d510 0x7fc6bc1f9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1516d0 con 0x7fc6bc06aba0 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc153350 con 0x7fc6bc15d510 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc153070 con 0x7fc6bc15d130 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 0x7fc6bc150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 0x7fc6bc150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59370/0 (socket says 172.21.15.143:59370) 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 -- 172.21.15.143:0/2742834562 learned_addr learned my addr 172.21.15.143:0/2742834562 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6bbe00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc15d130 0x7fc6bc158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c1800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6bc15d510 0x7fc6bc1f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 -- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc15d130 msgr2=0x7fc6bc158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc15d130 0x7fc6bc158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 -- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6bc15d510 msgr2=0x7fc6bc1f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6bc15d510 0x7fc6bc1f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 -- 172.21.15.143:0/2742834562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4009420 con 0x7fc6bc06aba0 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c0e00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 0x7fc6bc150140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c4d0 tx=0x7fc6ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac005cf0 con 0x7fc6bc06aba0 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac005e50 con 0x7fc6bc06aba0 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc150bf0 con 0x7fc6bc06aba0 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0072b0 con 0x7fc6bc06aba0 2024-01-15T18:05:42.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.319+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc1511b0 con 0x7fc6bc06aba0 2024-01-15T18:05:42.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.323+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc067500 con 0x7fc6bc06aba0 2024-01-15T18:05:42.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.323+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc6ac007410 con 0x7fc6bc06aba0 2024-01-15T18:05:42.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.323+0000 7fc6b8c00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6940668f0 0x7fc694068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.323+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc6ac083b20 con 0x7fc6bc06aba0 2024-01-15T18:05:42.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.323+0000 7fc6bbe00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6940668f0 0x7fc694068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.323+0000 7fc6bbe00640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6940668f0 0x7fc694068db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ae000 tx=0x7fc6a40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.327+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6ac050c40 con 0x7fc6bc06aba0 2024-01-15T18:05:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc6bc0b2ff0 con 0x7fc6bc06aba0 2024-01-15T18:05:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6b8c00640 1 -- 172.21.15.143:0/2742834562 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc6ac017020 con 0x7fc6bc06aba0 2024-01-15T18:05:42.490 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:42.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6940668f0 msgr2=0x7fc694068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6940668f0 0x7fc694068db0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40ae000 tx=0x7fc6a40a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 msgr2=0x7fc6bc150140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 0x7fc6bc150140 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00c4d0 tx=0x7fc6ac00c990 comp rx=0 tx=0).stop 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 shutdown_connections 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6940668f0 0x7fc694068db0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6bc15d510 0x7fc6bc1f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6bc15d130 0x7fc6bc158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 --2- 172.21.15.143:0/2742834562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6bc06aba0 0x7fc6bc150140 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 >> 172.21.15.143:0/2742834562 conn(0x7fc6bc0b1b90 msgr2=0x7fc6bc0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 shutdown_connections 2024-01-15T18:05:42.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.487+0000 7fc6c3800640 1 -- 172.21.15.143:0/2742834562 wait complete. 2024-01-15T18:05:42.513 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:05:42.513 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:05:42.522 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:42.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef7a400640 1 Processor -- start 2024-01-15T18:05:42.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef7a400640 1 -- start start 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef7a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef7a400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7415f310 con 0x7fef740ce560 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef7a400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7415f5f0 con 0x7fef74141660 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef7a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7415fbb0 con 0x7fef74072410 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef72a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fef740ce560 0x7fef74071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57168/0 (socket says 172.21.15.143:57168) 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef72a00640 1 -- 172.21.15.143:0/446517581 learned_addr learned my addr 172.21.15.143:0/446517581 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef73e00640 1 --2- 172.21.15.143:0/446517581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089622243 0 0) 0x7fef7415f310 con 0x7fef740ce560 2024-01-15T18:05:42.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef54003530 con 0x7fef740ce560 2024-01-15T18:05:42.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 893854390 0 0) 0x7fef7415f5f0 con 0x7fef74141660 2024-01-15T18:05:42.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.655+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef7415f310 con 0x7fef74141660 2024-01-15T18:05:42.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef73e00640 1 -- 172.21.15.143:0/446517581 >> v1:172.21.15.204:6789/0 conn(0x7fef74141660 legacy=0x7fef74141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef73e00640 1 -- 172.21.15.143:0/446517581 >> v1:172.21.15.143:6789/0 conn(0x7fef740ce560 legacy=0x7fef74071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 880926334 0 0) 0x7fef54003530 con 0x7fef740ce560 2024-01-15T18:05:42.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef73e00640 1 -- 172.21.15.143:0/446517581 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7415f8d0 con 0x7fef74072410 2024-01-15T18:05:42.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef73e00640 1 --2- 172.21.15.143:0/446517581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7415ec80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009dc0 tx=0x7fef6c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea936a953e5edf8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c001680 con 0x7fef74072410 2024-01-15T18:05:42.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c0a55e0 con 0x7fef74072410 2024-01-15T18:05:42.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.659+0000 7fef72000640 1 -- 172.21.15.143:0/446517581 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c0a5970 con 0x7fef74072410 2024-01-15T18:05:42.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- 172.21.15.143:0/446517581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 msgr2=0x7fef7415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 --2- 172.21.15.143:0/446517581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7415ec80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009dc0 tx=0x7fef6c01c890 comp rx=0 tx=0).stop 2024-01-15T18:05:42.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- 172.21.15.143:0/446517581 shutdown_connections 2024-01-15T18:05:42.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 --2- 172.21.15.143:0/446517581 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7415ec80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- 172.21.15.143:0/446517581 >> 172.21.15.143:0/446517581 conn(0x7fef7406c3f0 msgr2=0x7fef7406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- 172.21.15.143:0/446517581 shutdown_connections 2024-01-15T18:05:42.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- 172.21.15.143:0/446517581 wait complete. 2024-01-15T18:05:42.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 Processor -- start 2024-01-15T18:05:42.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- start start 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7413d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 0x7fef7413c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef74141660 0x7fef7415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7413eab0 con 0x7fef740ce560 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7413dc00 con 0x7fef74141660 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef7a400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7413dea0 con 0x7fef74072410 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 0x7fef7413c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 0x7fef7413c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59374/0 (socket says 172.21.15.143:59374) 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 -- 172.21.15.143:0/313621107 learned_addr learned my addr 172.21.15.143:0/313621107 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef73e00640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef74141660 0x7fef7415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef73400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7413d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 -- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 msgr2=0x7fef7413d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7413d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 -- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef74141660 msgr2=0x7fef7415c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef74141660 0x7fef7415c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.663+0000 7fef72a00640 1 -- 172.21.15.143:0/313621107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c009a70 con 0x7fef740ce560 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef72a00640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 0x7fef7413c450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c750 tx=0x7fef5c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c005af0 con 0x7fef740ce560 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c005c50 con 0x7fef740ce560 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c007070 con 0x7fef740ce560 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef740cd150 con 0x7fef740ce560 2024-01-15T18:05:42.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7413e460 con 0x7fef740ce560 2024-01-15T18:05:42.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef74067500 con 0x7fef740ce560 2024-01-15T18:05:42.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fef5c0071d0 con 0x7fef740ce560 2024-01-15T18:05:42.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef6b400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef4c0668f0 0x7fef4c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fef5c084120 con 0x7fef740ce560 2024-01-15T18:05:42.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.667+0000 7fef73400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef4c0668f0 0x7fef4c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.671+0000 7fef73400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef4c0668f0 0x7fef4c068db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef640011f0 tx=0x7fef64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.671+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fef5c051390 con 0x7fef740ce560 2024-01-15T18:05:42.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.823+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef740714d0 con 0x7fef740ce560 2024-01-15T18:05:42.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.823+0000 7fef6b400640 1 -- 172.21.15.143:0/313621107 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fef5c0549b0 con 0x7fef740ce560 2024-01-15T18:05:42.829 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:42.830 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:40.506528+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:42.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef4c0668f0 msgr2=0x7fef4c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef4c0668f0 0x7fef4c068db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef640011f0 tx=0x7fef64009040 comp rx=0 tx=0).stop 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 msgr2=0x7fef7413c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 0x7fef7413c450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00c750 tx=0x7fef5c00cc10 comp rx=0 tx=0).stop 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 shutdown_connections 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fef4c0668f0 0x7fef4c068db0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef74141660 0x7fef7415c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef740ce560 0x7fef7413c450 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 --2- 172.21.15.143:0/313621107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef74072410 0x7fef7413d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 >> 172.21.15.143:0/313621107 conn(0x7fef7406c3f0 msgr2=0x7fef740af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 shutdown_connections 2024-01-15T18:05:42.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.827+0000 7fef7a400640 1 -- 172.21.15.143:0/313621107 wait complete. 2024-01-15T18:05:42.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:42.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:42.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 Processor -- start 2024-01-15T18:05:42.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 -- start start 2024-01-15T18:05:42.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5d94071bb0 0x7f5d94072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5d9415c5a0 0x7f5d9415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940ceb50 con 0x7f5d94143780 2024-01-15T18:05:42.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d94072570 con 0x7f5d9415c5a0 2024-01-15T18:05:42.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d99c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9415c110 con 0x7f5d94071bb0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d93400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5d9415c5a0 0x7f5d9415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d92a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5d94143780 0x7f5d940cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57176/0 (socket says 172.21.15.143:57176) 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d92a00640 1 -- 172.21.15.143:0/1165677889 learned_addr learned my addr 172.21.15.143:0/1165677889 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d92000640 1 --2- 172.21.15.143:0/1165677889 >> v2:172.21.15.204:3300/0 conn(0x7f5d94071bb0 0x7f5d94072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d93400640 1 -- 172.21.15.143:0/1165677889 >> v2:172.21.15.204:3300/0 conn(0x7f5d94071bb0 msgr2=0x7f5d94072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d93400640 1 --2- 172.21.15.143:0/1165677889 >> v2:172.21.15.204:3300/0 conn(0x7f5d94071bb0 0x7f5d94072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d93400640 1 -- 172.21.15.143:0/1165677889 >> v1:172.21.15.143:6789/0 conn(0x7f5d94143780 legacy=0x7f5d940cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d93400640 1 -- 172.21.15.143:0/1165677889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9415b7e0 con 0x7f5d9415c5a0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.987+0000 7f5d91600640 1 -- 172.21.15.143:0/1165677889 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189010032 0 0) 0x7f5d84001120 con 0x7f5d94143780 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d93400640 1 --2- 172.21.15.143:0/1165677889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d9415c9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d9406beb0 tx=0x7f5d8c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6ce424103c230e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d91600640 1 -- 172.21.15.143:0/1165677889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0a59d0 con 0x7f5d9415c5a0 2024-01-15T18:05:42.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d91600640 1 -- 172.21.15.143:0/1165677889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c0a5b30 con 0x7f5d9415c5a0 2024-01-15T18:05:42.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d91600640 1 -- 172.21.15.143:0/1165677889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0ad440 con 0x7f5d9415c5a0 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 -- 172.21.15.143:0/1165677889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 msgr2=0x7f5d9415c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 --2- 172.21.15.143:0/1165677889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d9415c9e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5d9406beb0 tx=0x7f5d8c01c910 comp rx=0 tx=0).stop 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 -- 172.21.15.143:0/1165677889 shutdown_connections 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 --2- 172.21.15.143:0/1165677889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d9415c9e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 --2- 172.21.15.143:0/1165677889 >> v2:172.21.15.204:3300/0 conn(0x7f5d94071bb0 0x7f5d94072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 -- 172.21.15.143:0/1165677889 >> 172.21.15.143:0/1165677889 conn(0x7f5d9406c3f0 msgr2=0x7f5d9406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 -- 172.21.15.143:0/1165677889 shutdown_connections 2024-01-15T18:05:42.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.991+0000 7f5d99c00640 1 -- 172.21.15.143:0/1165677889 wait complete. 2024-01-15T18:05:42.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 Processor -- start 2024-01-15T18:05:42.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 -- start start 2024-01-15T18:05:42.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d94071bb0 0x7f5d940b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d94143780 0x7f5d940b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d941f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940b3550 con 0x7f5d9415c5a0 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9413be30 con 0x7f5d94071bb0 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940b28d0 con 0x7f5d94143780 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d941f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d941f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59400/0 (socket says 172.21.15.143:59400) 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 -- 172.21.15.143:0/2920232548 learned_addr learned my addr 172.21.15.143:0/2920232548 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d92a00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d94071bb0 0x7f5d940b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d92000640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d94143780 0x7f5d940b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 -- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d94143780 msgr2=0x7f5d940b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d94143780 0x7f5d940b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 -- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d94071bb0 msgr2=0x7f5d940b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d94071bb0 0x7f5d940b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 -- 172.21.15.143:0/2920232548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c009a70 con 0x7f5d9415c5a0 2024-01-15T18:05:43.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d93400640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d941f5d20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b0040 tx=0x7f5d8c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0add70 con 0x7f5d9415c5a0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c009450 con 0x7f5d9415c5a0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0b9690 con 0x7f5d9415c5a0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d940b2bb0 con 0x7f5d9415c5a0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.995+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d940b3170 con 0x7f5d9415c5a0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.999+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940b3690 con 0x7f5d9415c5a0 2024-01-15T18:05:43.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.999+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5d8c0b97f0 con 0x7f5d9415c5a0 2024-01-15T18:05:43.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.999+0000 7f5d8aa00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d6c0668f0 0x7f5d6c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.999+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5d8c11f900 con 0x7f5d9415c5a0 2024-01-15T18:05:43.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.999+0000 7f5d92a00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d6c0668f0 0x7f5d6c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:42.999+0000 7f5d92a00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d6c0668f0 0x7f5d6c068db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000f70 tx=0x7f5d84007ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.003+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5d8c0ecc00 con 0x7f5d9415c5a0 2024-01-15T18:05:43.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.151+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d94067500 con 0x7f5d9415c5a0 2024-01-15T18:05:43.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d8aa00640 1 -- 172.21.15.143:0/2920232548 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d8c0f0220 con 0x7f5d9415c5a0 2024-01-15T18:05:43.158 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d6c0668f0 msgr2=0x7f5d6c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d6c0668f0 0x7f5d6c068db0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000f70 tx=0x7f5d84007ba0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 msgr2=0x7f5d941f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d941f5d20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0b0040 tx=0x7f5d8c0196b0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 shutdown_connections 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d6c0668f0 0x7f5d6c068db0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9415c5a0 0x7f5d941f5d20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d94143780 0x7f5d940b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 --2- 172.21.15.143:0/2920232548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d94071bb0 0x7f5d940b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 >> 172.21.15.143:0/2920232548 conn(0x7f5d9406c3f0 msgr2=0x7f5d94070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 shutdown_connections 2024-01-15T18:05:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.155+0000 7f5d99c00640 1 -- 172.21.15.143:0/2920232548 wait complete. 2024-01-15T18:05:43.180 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:43.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc06e000640 1 Processor -- start 2024-01-15T18:05:43.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc06e000640 1 -- start start 2024-01-15T18:05:43.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc06e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0680b29a0 0x7fc06806de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc06e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06815e8a0 con 0x7fc0680b2510 2024-01-15T18:05:43.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc06e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680b3270 con 0x7fc068070d80 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc06e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b2f90 con 0x7fc0680b29a0 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc066a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc0680b2510 0x7fc06806d730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57180/0 (socket says 172.21.15.143:57180) 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc066a00640 1 -- 172.21.15.143:0/942890158 learned_addr learned my addr 172.21.15.143:0/942890158 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc067e00640 1 --2- 172.21.15.143:0/942890158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0680b29a0 0x7fc06806de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686680625 0 0) 0x7fc06815e8a0 con 0x7fc0680b2510 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc048003530 con 0x7fc0680b2510 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246111828 0 0) 0x7fc0680b3270 con 0x7fc068070d80 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc06815e8a0 con 0x7fc068070d80 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc067e00640 1 -- 172.21.15.143:0/942890158 >> v1:172.21.15.204:6789/0 conn(0x7fc068070d80 legacy=0x7fc068071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc067e00640 1 -- 172.21.15.143:0/942890158 >> v1:172.21.15.143:6789/0 conn(0x7fc0680b2510 legacy=0x7fc06806d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.315+0000 7fc067e00640 1 -- 172.21.15.143:0/942890158 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06806cc50 con 0x7fc0680b29a0 2024-01-15T18:05:43.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc067e00640 1 --2- 172.21.15.143:0/942890158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0680b29a0 0x7fc06806de40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc060009dc0 tx=0x7fc06001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72983f210851e054 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600a5970 con 0x7fc0680b29a0 2024-01-15T18:05:43.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0600a5ad0 con 0x7fc0680b29a0 2024-01-15T18:05:43.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc066000640 1 -- 172.21.15.143:0/942890158 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600a5e60 con 0x7fc0680b29a0 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 -- 172.21.15.143:0/942890158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0680b29a0 msgr2=0x7fc06806de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 --2- 172.21.15.143:0/942890158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0680b29a0 0x7fc06806de40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc060009dc0 tx=0x7fc06001c8b0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 -- 172.21.15.143:0/942890158 shutdown_connections 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 --2- 172.21.15.143:0/942890158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0680b29a0 0x7fc06806de40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 -- 172.21.15.143:0/942890158 >> 172.21.15.143:0/942890158 conn(0x7fc0680b1610 msgr2=0x7fc0680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 -- 172.21.15.143:0/942890158 shutdown_connections 2024-01-15T18:05:43.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.319+0000 7fc06e000640 1 -- 172.21.15.143:0/942890158 wait complete. 2024-01-15T18:05:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 Processor -- start 2024-01-15T18:05:43.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 -- start start 2024-01-15T18:05:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc068070d80 0x7fc068156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 0x7fc06815fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0680b29a0 0x7fc06815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068156880 con 0x7fc0680b2510 2024-01-15T18:05:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680accc0 con 0x7fc0680b29a0 2024-01-15T18:05:43.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680acfa0 con 0x7fc068070d80 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 0x7fc06815fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 0x7fc06815fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59414/0 (socket says 172.21.15.143:59414) 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 -- 172.21.15.143:0/780180431 learned_addr learned my addr 172.21.15.143:0/780180431 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc067e00640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0680b29a0 0x7fc06815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc067400640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc068070d80 0x7fc068156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 -- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc068070d80 msgr2=0x7fc068156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc068070d80 0x7fc068156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 -- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0680b29a0 msgr2=0x7fc06815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0680b29a0 0x7fc06815ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 -- 172.21.15.143:0/780180431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009a70 con 0x7fc0680b2510 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc066a00640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 0x7fc06815fa80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c4d0 tx=0x7fc05000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050005cf0 con 0x7fc0680b2510 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc050005e50 con 0x7fc0680b2510 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0680ad280 con 0x7fc0680b2510 2024-01-15T18:05:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0500072b0 con 0x7fc0680b2510 2024-01-15T18:05:43.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.323+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680ad840 con 0x7fc0680b2510 2024-01-15T18:05:43.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.327+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc068074b00 con 0x7fc0680b2510 2024-01-15T18:05:43.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.327+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc050007410 con 0x7fc0680b2510 2024-01-15T18:05:43.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.327+0000 7fc05f400640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0400668f0 0x7fc040068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.327+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc050083a10 con 0x7fc0680b2510 2024-01-15T18:05:43.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.327+0000 7fc067400640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0400668f0 0x7fc040068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.327+0000 7fc067400640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0400668f0 0x7fc040068db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0580011f0 tx=0x7fc058009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.331+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc050050c80 con 0x7fc0680b2510 2024-01-15T18:05:43.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.479+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc068146e20 con 0x7fc0680b2510 2024-01-15T18:05:43.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.479+0000 7fc05f400640 1 -- 172.21.15.143:0/780180431 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc0500170a0 con 0x7fc0680b2510 2024-01-15T18:05:43.485 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:43.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0400668f0 msgr2=0x7fc040068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0400668f0 0x7fc040068db0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0580011f0 tx=0x7fc058009040 comp rx=0 tx=0).stop 2024-01-15T18:05:43.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 msgr2=0x7fc06815fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 0x7fc06815fa80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c4d0 tx=0x7fc05000c990 comp rx=0 tx=0).stop 2024-01-15T18:05:43.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 shutdown_connections 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0400668f0 0x7fc040068db0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0680b29a0 0x7fc06815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0680b2510 0x7fc06815fa80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 --2- 172.21.15.143:0/780180431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc068070d80 0x7fc068156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 >> 172.21.15.143:0/780180431 conn(0x7fc0680b1610 msgr2=0x7fc0680af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 shutdown_connections 2024-01-15T18:05:43.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.483+0000 7fc06e000640 1 -- 172.21.15.143:0/780180431 wait complete. 2024-01-15T18:05:43.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:43.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5628e00640 1 Processor -- start 2024-01-15T18:05:43.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5628e00640 1 -- start start 2024-01-15T18:05:43.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5628e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5624072410 0x7f562415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5628e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562415f310 con 0x7f5624141660 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5628e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562415f5f0 con 0x7f56240ce560 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5628e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562415fbb0 con 0x7f5624072410 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5621e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5624141660 0x7f5624141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57184/0 (socket says 172.21.15.143:57184) 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5621e00640 1 -- 172.21.15.143:0/253973596 learned_addr learned my addr 172.21.15.143:0/253973596 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5622800640 1 --2- 172.21.15.143:0/253973596 >> v2:172.21.15.204:3300/0 conn(0x7f5624072410 0x7f562415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5620a00640 1 -- 172.21.15.143:0/253973596 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152420054 0 0) 0x7f562415f310 con 0x7f5624141660 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5620a00640 1 -- 172.21.15.143:0/253973596 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5604003530 con 0x7f5624141660 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5622800640 1 -- 172.21.15.143:0/253973596 >> v1:172.21.15.204:6789/0 conn(0x7f56240ce560 legacy=0x7f5624071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5620a00640 1 -- 172.21.15.143:0/253973596 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 286042831 0 0) 0x7f5604003530 con 0x7f5624141660 2024-01-15T18:05:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5622800640 1 -- 172.21.15.143:0/253973596 >> v1:172.21.15.143:6789/0 conn(0x7f5624141660 legacy=0x7f5624141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.643+0000 7f5622800640 1 -- 172.21.15.143:0/253973596 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562415f8d0 con 0x7f5624072410 2024-01-15T18:05:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5622800640 1 --2- 172.21.15.143:0/253973596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624072410 0x7f562415ec80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f561c009d90 tx=0x7f561c01c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96b39ffdfacd9f8c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5620a00640 1 -- 172.21.15.143:0/253973596 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140016a0 con 0x7f5624072410 2024-01-15T18:05:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5620a00640 1 -- 172.21.15.143:0/253973596 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c0a64c0 con 0x7f5624072410 2024-01-15T18:05:43.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5620a00640 1 -- 172.21.15.143:0/253973596 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0a6850 con 0x7f5624072410 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 -- 172.21.15.143:0/253973596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624072410 msgr2=0x7f562415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 --2- 172.21.15.143:0/253973596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624072410 0x7f562415ec80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f561c009d90 tx=0x7f561c01c720 comp rx=0 tx=0).stop 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 -- 172.21.15.143:0/253973596 shutdown_connections 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 --2- 172.21.15.143:0/253973596 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624072410 0x7f562415ec80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 -- 172.21.15.143:0/253973596 >> 172.21.15.143:0/253973596 conn(0x7f562406c3f0 msgr2=0x7f562406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 -- 172.21.15.143:0/253973596 shutdown_connections 2024-01-15T18:05:43.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.647+0000 7f5628e00640 1 -- 172.21.15.143:0/253973596 wait complete. 2024-01-15T18:05:43.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 Processor -- start 2024-01-15T18:05:43.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 -- start start 2024-01-15T18:05:43.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5624072410 0x7f562415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 0x7f5624156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624141660 0x7f5624138e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5624157250 con 0x7f56240ce560 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240b3580 con 0x7f5624141660 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56240b2970 con 0x7f5624072410 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 0x7f5624156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 0x7f5624156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59430/0 (socket says 172.21.15.143:59430) 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 -- 172.21.15.143:0/2841981067 learned_addr learned my addr 172.21.15.143:0/2841981067 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5622800640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624141660 0x7f5624138e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5624072410 0x7f562415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 -- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5624072410 msgr2=0x7f562415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5624072410 0x7f562415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 -- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624141660 msgr2=0x7f5624138e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624141660 0x7f5624138e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 -- 172.21.15.143:0/2841981067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c009a70 con 0x7f56240ce560 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5621400640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 0x7f5624156560 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c570 tx=0x7f560c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c005480 con 0x7f56240ce560 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560c003600 con 0x7f56240ce560 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560c0039b0 con 0x7f56240ce560 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.651+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56240cd150 con 0x7f56240ce560 2024-01-15T18:05:43.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56240b2f30 con 0x7f56240ce560 2024-01-15T18:05:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5624067500 con 0x7f56240ce560 2024-01-15T18:05:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f560c003b10 con 0x7f56240ce560 2024-01-15T18:05:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f561a000640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55fc0668f0 0x7f55fc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f560c0836e0 con 0x7f56240ce560 2024-01-15T18:05:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f5621e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55fc0668f0 0x7f55fc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.655+0000 7f5621e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55fc0668f0 0x7f55fc068db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5614001ec0 tx=0x7f5614001d10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.659+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f560c050950 con 0x7f56240ce560 2024-01-15T18:05:43.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f562406e2a0 con 0x7f56240ce560 2024-01-15T18:05:43.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f561a000640 1 -- 172.21.15.143:0/2841981067 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f560c017020 con 0x7f56240ce560 2024-01-15T18:05:43.815 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55fc0668f0 msgr2=0x7f55fc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55fc0668f0 0x7f55fc068db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5614001ec0 tx=0x7f5614001d10 comp rx=0 tx=0).stop 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 msgr2=0x7f5624156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 0x7f5624156560 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f560c00c570 tx=0x7f560c00ca30 comp rx=0 tx=0).stop 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 shutdown_connections 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55fc0668f0 0x7f55fc068db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5624141660 0x7f5624138e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56240ce560 0x7f5624156560 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 --2- 172.21.15.143:0/2841981067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5624072410 0x7f562415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 >> 172.21.15.143:0/2841981067 conn(0x7f562406c3f0 msgr2=0x7f56240acfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:43.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 shutdown_connections 2024-01-15T18:05:43.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.811+0000 7f5628e00640 1 -- 172.21.15.143:0/2841981067 wait complete. 2024-01-15T18:05:43.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:05:43.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 Processor -- start 2024-01-15T18:05:43.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 -- start start 2024-01-15T18:05:43.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93a8157300 0x7f93a8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a80b2a20 con 0x7f93a80b48f0 2024-01-15T18:05:43.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80b35a0 con 0x7f93a8157300 2024-01-15T18:05:43.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93b0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80b4d80 con 0x7f93a80b21d0 2024-01-15T18:05:43.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93aea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f93a80b48f0 0x7f93a815d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57192/0 (socket says 172.21.15.143:57192) 2024-01-15T18:05:43.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93aea00640 1 -- 172.21.15.143:0/3698897591 learned_addr learned my addr 172.21.15.143:0/3698897591 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:43.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93ae000640 1 --2- 172.21.15.143:0/3698897591 >> v2:172.21.15.204:3300/0 conn(0x7f93a8157300 0x7f93a8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93ad600640 1 --2- 172.21.15.143:0/3698897591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93acc00640 1 -- 172.21.15.143:0/3698897591 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631740146 0 0) 0x7f93a80b2a20 con 0x7f93a80b48f0 2024-01-15T18:05:43.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93acc00640 1 -- 172.21.15.143:0/3698897591 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9388005740 con 0x7f93a80b48f0 2024-01-15T18:05:43.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93ad600640 1 -- 172.21.15.143:0/3698897591 >> v2:172.21.15.204:3300/0 conn(0x7f93a8157300 msgr2=0x7f93a8159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:43.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93ad600640 1 --2- 172.21.15.143:0/3698897591 >> v2:172.21.15.204:3300/0 conn(0x7f93a8157300 0x7f93a8159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93ad600640 1 -- 172.21.15.143:0/3698897591 >> v1:172.21.15.143:6789/0 conn(0x7f93a80b48f0 legacy=0x7f93a815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.971+0000 7f93ad600640 1 -- 172.21.15.143:0/3698897591 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80b2d00 con 0x7f93a80b21d0 2024-01-15T18:05:43.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93ad600640 1 --2- 172.21.15.143:0/3698897591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a80b43b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f939000a960 tx=0x7f939001c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5fcd00cd992619 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93acc00640 1 -- 172.21.15.143:0/3698897591 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900a57e0 con 0x7f93a80b21d0 2024-01-15T18:05:43.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93acc00640 1 -- 172.21.15.143:0/3698897591 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93900a5940 con 0x7f93a80b21d0 2024-01-15T18:05:43.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93acc00640 1 -- 172.21.15.143:0/3698897591 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900a5cd0 con 0x7f93a80b21d0 2024-01-15T18:05:43.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 -- 172.21.15.143:0/3698897591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 msgr2=0x7f93a80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3698897591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a80b43b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f939000a960 tx=0x7f939001c720 comp rx=0 tx=0).stop 2024-01-15T18:05:43.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 -- 172.21.15.143:0/3698897591 shutdown_connections 2024-01-15T18:05:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3698897591 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a80b43b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3698897591 >> v2:172.21.15.204:3300/0 conn(0x7f93a8157300 0x7f93a8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 -- 172.21.15.143:0/3698897591 >> 172.21.15.143:0/3698897591 conn(0x7f93a806c3f0 msgr2=0x7f93a806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 -- 172.21.15.143:0/3698897591 shutdown_connections 2024-01-15T18:05:43.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.975+0000 7f93b0a00640 1 -- 172.21.15.143:0/3698897591 wait complete. 2024-01-15T18:05:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 Processor -- start 2024-01-15T18:05:43.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 -- start start 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a81fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93a80b48f0 0x7f93a814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 0x7f93a814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a815b580 con 0x7f93a8157300 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a814edd0 con 0x7f93a80b48f0 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93b0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81fa6f0 con 0x7f93a80b21d0 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 0x7f93a814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93ae000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a81fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 0x7f93a814adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59438/0 (socket says 172.21.15.143:59438) 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 -- 172.21.15.143:0/3228359430 learned_addr learned my addr 172.21.15.143:0/3228359430 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93ad600640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93a80b48f0 0x7f93a814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 -- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 msgr2=0x7f93a81fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a81fa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 -- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93a80b48f0 msgr2=0x7f93a814ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93a80b48f0 0x7f93a814ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 -- 172.21.15.143:0/3228359430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939000a5c0 con 0x7f93a8157300 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93ae000640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a81fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93aea00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 0x7f93a814adb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93a000caa0 tx=0x7f93a000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.979+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000ac00 con 0x7f93a8157300 2024-01-15T18:05:43.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a000ad60 con 0x7f93a8157300 2024-01-15T18:05:43.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0012620 con 0x7f93a8157300 2024-01-15T18:05:43.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a815ace0 con 0x7f93a8157300 2024-01-15T18:05:43.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a815b2a0 con 0x7f93a8157300 2024-01-15T18:05:43.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a814eba0 con 0x7f93a8157300 2024-01-15T18:05:43.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f93a0007070 con 0x7f93a8157300 2024-01-15T18:05:43.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93a6000640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93800668f0 0x7f9380068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:43.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f93a00843e0 con 0x7f93a8157300 2024-01-15T18:05:43.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93ae000640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93800668f0 0x7f9380068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:43.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.983+0000 7f93ae000640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93800668f0 0x7f9380068db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9398009eb0 tx=0x7f9398009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:43.987+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f93a814eba0 con 0x7f93a8157300 2024-01-15T18:05:44.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.147+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93a814eba0 con 0x7f93a8157300 2024-01-15T18:05:44.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.147+0000 7f93a6000640 1 -- 172.21.15.143:0/3228359430 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f93a814eba0 con 0x7f93a8157300 2024-01-15T18:05:44.152 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:05:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93800668f0 msgr2=0x7f9380068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93800668f0 0x7f9380068db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9398009eb0 tx=0x7f9398009190 comp rx=0 tx=0).stop 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 msgr2=0x7f93a814adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 0x7f93a814adb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f93a000caa0 tx=0x7f93a000cf60 comp rx=0 tx=0).stop 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 shutdown_connections 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93800668f0 0x7f9380068db0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93a8157300 0x7f93a814adb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93a80b48f0 0x7f93a814ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 --2- 172.21.15.143:0/3228359430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93a80b21d0 0x7f93a81fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 >> 172.21.15.143:0/3228359430 conn(0x7f93a806c3f0 msgr2=0x7f93a80ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 shutdown_connections 2024-01-15T18:05:44.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.151+0000 7f93b0a00640 1 -- 172.21.15.143:0/3228359430 wait complete. 2024-01-15T18:05:44.174 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:05:44.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:44.174 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-01-15T18:05:44.197 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:44.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:05:44.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:05:44.197 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:05:44.253 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:05:44.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:44.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff511000640 1 Processor -- start 2024-01-15T18:05:44.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff511000640 1 -- start start 2024-01-15T18:05:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff511000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff50c0cdf20 0x7ff50c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff511000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0c4d90 con 0x7ff50c15b260 2024-01-15T18:05:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff511000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c15b640 con 0x7ff50c0c6230 2024-01-15T18:05:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff511000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c0cf800 con 0x7ff50c0cdf20 2024-01-15T18:05:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff50c15b260 0x7ff50c0cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57206/0 (socket says 172.21.15.143:57206) 2024-01-15T18:05:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509e00640 1 -- 172.21.15.143:0/3934348875 learned_addr learned my addr 172.21.15.143:0/3934348875 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 --2- 172.21.15.143:0/3934348875 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0cdf20 0x7ff50c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777671408 0 0) 0x7ff50c0c4d90 con 0x7ff50c15b260 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ec003530 con 0x7ff50c15b260 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1011534135 0 0) 0x7ff4ec003530 con 0x7ff50c15b260 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0cdf20 msgr2=0x7ff50c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 --2- 172.21.15.143:0/3934348875 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0cdf20 0x7ff50c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 >> v1:172.21.15.204:6789/0 conn(0x7ff50c0c6230 legacy=0x7ff50c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0cfae0 con 0x7ff50c15b260 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff4fc001680 con 0x7ff50c15b260 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 --2- 172.21.15.143:0/3934348875 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0cdf20 0x7ff50c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:44.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 >> v1:172.21.15.143:6789/0 conn(0x7ff50c15b260 legacy=0x7ff50c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4ec003980 0x7ff4ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff4ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4ec005190 0x7ff4ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0cf800 con 0x7ff4ec0047d0 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec00c030 con 0x7ff4ec005190 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec00c1f0 con 0x7ff4ec003980 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff4ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff50a800640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4ec005190 0x7ff4ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509e00640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4ec003980 0x7ff4ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 -- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4ec003980 msgr2=0x7ff4ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff4ec003980 0x7ff4ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 -- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4ec005190 msgr2=0x7ff4ec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff4ec005190 0x7ff4ec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff509400640 1 -- 172.21.15.143:0/3934348875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec003530 con 0x7ff4ec0047d0 2024-01-15T18:05:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.387+0000 7ff50a800640 1 -- 172.21.15.143:0/3934348875 reap_dead start 2024-01-15T18:05:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff509400640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff4ec004c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009cb0 tx=0x7ff4f401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ed5547dd41ecca server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40a57a0 con 0x7ff4ec0047d0 2024-01-15T18:05:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f40a5900 con 0x7ff4ec0047d0 2024-01-15T18:05:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff508a00640 1 -- 172.21.15.143:0/3934348875 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f40a5cf0 con 0x7ff4ec0047d0 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 -- 172.21.15.143:0/3934348875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 msgr2=0x7ff4ec004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff4ec004c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009cb0 tx=0x7ff4f401cbf0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 -- 172.21.15.143:0/3934348875 shutdown_connections 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 --2- 172.21.15.143:0/3934348875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff4ec004c50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 -- 172.21.15.143:0/3934348875 >> 172.21.15.143:0/3934348875 conn(0x7ff50c0d0ed0 msgr2=0x7ff50c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 -- 172.21.15.143:0/3934348875 shutdown_connections 2024-01-15T18:05:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.391+0000 7ff511000640 1 -- 172.21.15.143:0/3934348875 wait complete. 2024-01-15T18:05:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 Processor -- start 2024-01-15T18:05:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 -- start start 2024-01-15T18:05:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff50c1f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c155f60 0x7ff50c06c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c06c7a0 0x7ff50c09bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c06cc20 con 0x7ff4ec0047d0 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0b4f30 con 0x7ff50c155f60 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0b5210 con 0x7ff50c06c7a0 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff50c1f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff50c1f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59452/0 (socket says 172.21.15.143:59452) 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 -- 172.21.15.143:0/3792430976 learned_addr learned my addr 172.21.15.143:0/3792430976 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff50a800640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c06c7a0 0x7ff50c09bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509400640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c155f60 0x7ff50c06c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 -- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c06c7a0 msgr2=0x7ff50c09bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c06c7a0 0x7ff50c09bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 -- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c155f60 msgr2=0x7ff50c06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c155f60 0x7ff50c06c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 -- 172.21.15.143:0/3792430976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0b54f0 con 0x7ff4ec0047d0 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff509e00640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff50c1f2ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0117f0 tx=0x7ff4fc011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00fb60 con 0x7ff4ec0047d0 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc00fcc0 con 0x7ff4ec0047d0 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c1563c0 con 0x7ff4ec0047d0 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc015610 con 0x7ff4ec0047d0 2024-01-15T18:05:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.395+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c156980 con 0x7ff4ec0047d0 2024-01-15T18:05:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.399+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c0b4d90 con 0x7ff4ec0047d0 2024-01-15T18:05:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.399+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff4fc015770 con 0x7ff4ec0047d0 2024-01-15T18:05:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.399+0000 7ff502000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40668f0 0x7ff4e4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.399+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff4fc087740 con 0x7ff4ec0047d0 2024-01-15T18:05:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.399+0000 7ff509400640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40668f0 0x7ff4e4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.399+0000 7ff509400640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40668f0 0x7ff4e4068db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40af000 tx=0x7ff4f40046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:44.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.403+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff50c0b4d90 con 0x7ff4ec0047d0 2024-01-15T18:05:44.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.583+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50c0b4d90 con 0x7ff4ec0047d0 2024-01-15T18:05:44.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.583+0000 7ff502000640 1 -- 172.21.15.143:0/3792430976 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7ff50c0b4d90 con 0x7ff4ec0047d0 2024-01-15T18:05:44.588 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:44.588 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:44.589 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40668f0 msgr2=0x7ff4e4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40668f0 0x7ff4e4068db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40af000 tx=0x7ff4f40046d0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 msgr2=0x7ff50c1f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff50c1f2ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0117f0 tx=0x7ff4fc011cb0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 shutdown_connections 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40668f0 0x7ff4e4068db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c06c7a0 0x7ff50c09bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c155f60 0x7ff50c06c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 --2- 172.21.15.143:0/3792430976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff4ec0047d0 0x7ff50c1f2ef0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 >> 172.21.15.143:0/3792430976 conn(0x7ff50c0d0ed0 msgr2=0x7ff50c06f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 shutdown_connections 2024-01-15T18:05:44.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.587+0000 7ff511000640 1 -- 172.21.15.143:0/3792430976 wait complete. 2024-01-15T18:05:44.610 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:44.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 Processor -- start 2024-01-15T18:05:44.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 -- start start 2024-01-15T18:05:44.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe68006c540 0x7fe6800cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe6800cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6800d02d0 con 0x7fe68006b7b0 2024-01-15T18:05:44.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6800cf6c0 con 0x7fe68006c540 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe685000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800cf9a0 con 0x7fe6800cf150 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe67de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe68006c540 0x7fe6800cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe67e800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe68006b7b0 0x7fe68015ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57208/0 (socket says 172.21.15.143:57208) 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.743+0000 7fe67e800640 1 -- 172.21.15.143:0/3099181198 learned_addr learned my addr 172.21.15.143:0/3099181198 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67ca00640 1 -- 172.21.15.143:0/3099181198 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326787693 0 0) 0x7fe6800d02d0 con 0x7fe68006b7b0 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67d400640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe6800cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67ca00640 1 -- 172.21.15.143:0/3099181198 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe660003530 con 0x7fe68006b7b0 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67de00640 1 -- 172.21.15.143:0/3099181198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 msgr2=0x7fe6800cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67de00640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe6800cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67de00640 1 -- 172.21.15.143:0/3099181198 >> v1:172.21.15.143:6789/0 conn(0x7fe68006b7b0 legacy=0x7fe68015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67de00640 1 -- 172.21.15.143:0/3099181198 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6800cfc80 con 0x7fe68006c540 2024-01-15T18:05:44.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67d400640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe6800cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:44.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67de00640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006c540 0x7fe6800cd7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe6700097c0 tx=0x7fe67001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3879ac7f9671d895 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:44.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67ca00640 1 -- 172.21.15.143:0/3099181198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700ac640 con 0x7fe68006c540 2024-01-15T18:05:44.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67ca00640 1 -- 172.21.15.143:0/3099181198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700ac7a0 con 0x7fe68006c540 2024-01-15T18:05:44.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.747+0000 7fe67ca00640 1 -- 172.21.15.143:0/3099181198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700acb30 con 0x7fe68006c540 2024-01-15T18:05:44.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- 172.21.15.143:0/3099181198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006c540 msgr2=0x7fe6800cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006c540 0x7fe6800cd7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe6700097c0 tx=0x7fe67001c3c0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- 172.21.15.143:0/3099181198 shutdown_connections 2024-01-15T18:05:44.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe6800cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 --2- 172.21.15.143:0/3099181198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006c540 0x7fe6800cd7e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- 172.21.15.143:0/3099181198 >> 172.21.15.143:0/3099181198 conn(0x7fe6800d0e20 msgr2=0x7fe6800d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:44.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- 172.21.15.143:0/3099181198 shutdown_connections 2024-01-15T18:05:44.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- 172.21.15.143:0/3099181198 wait complete. 2024-01-15T18:05:44.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 Processor -- start 2024-01-15T18:05:44.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- start start 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 0x7fe680155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68006c540 0x7fe680155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe680155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680150b60 con 0x7fe68006b7b0 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe680154ac0 con 0x7fe68006c540 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe685000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe680154fb0 con 0x7fe6800cf150 2024-01-15T18:05:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe67de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 0x7fe680155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe67de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 0x7fe680155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59476/0 (socket says 172.21.15.143:59476) 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe67de00640 1 -- 172.21.15.143:0/2837897998 learned_addr learned my addr 172.21.15.143:0/2837897998 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe67d400640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68006c540 0x7fe680155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.751+0000 7fe67e800640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe680155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67de00640 1 -- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 msgr2=0x7fe680155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67de00640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe680155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67de00640 1 -- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68006c540 msgr2=0x7fe680155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67de00640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68006c540 0x7fe680155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67de00640 1 -- 172.21.15.143:0/2837897998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670009420 con 0x7fe68006b7b0 2024-01-15T18:05:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67d400640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68006c540 0x7fe680155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe67de00640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 0x7fe680155300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe670019ff0 tx=0x7fe670019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b5610 con 0x7fe68006b7b0 2024-01-15T18:05:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6700b5770 con 0x7fe68006b7b0 2024-01-15T18:05:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6800c6180 con 0x7fe68006b7b0 2024-01-15T18:05:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700b5b20 con 0x7fe68006b7b0 2024-01-15T18:05:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6800c56a0 con 0x7fe68006b7b0 2024-01-15T18:05:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe680067500 con 0x7fe68006b7b0 2024-01-15T18:05:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.755+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fe6700b3040 con 0x7fe68006b7b0 2024-01-15T18:05:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.759+0000 7fe676000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe6580668f0 0x7fe658068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.759+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe67011e760 con 0x7fe68006b7b0 2024-01-15T18:05:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.759+0000 7fe67d400640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe6580668f0 0x7fe658068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.759+0000 7fe67d400640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe6580668f0 0x7fe658068db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6800b6e20 tx=0x7fe668008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.763+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6700eba30 con 0x7fe68006b7b0 2024-01-15T18:05:44.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.911+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe680167f70 con 0x7fe68006b7b0 2024-01-15T18:05:44.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.911+0000 7fe676000640 1 -- 172.21.15.143:0/2837897998 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16790 (secure 0 0 0) 0x7fe6700ef050 con 0x7fe68006b7b0 2024-01-15T18:05:44.917 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:44.917 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:40.506528+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:44.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe6580668f0 msgr2=0x7fe658068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe6580668f0 0x7fe658068db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe6800b6e20 tx=0x7fe668008040 comp rx=0 tx=0).stop 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 msgr2=0x7fe680155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 0x7fe680155300 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe670019ff0 tx=0x7fe670019160 comp rx=0 tx=0).stop 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 shutdown_connections 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe6580668f0 0x7fe658068db0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6800cf150 0x7fe680155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe68006c540 0x7fe680155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 --2- 172.21.15.143:0/2837897998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe68006b7b0 0x7fe680155300 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:44.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 >> 172.21.15.143:0/2837897998 conn(0x7fe6800d0e20 msgr2=0x7fe6800cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:44.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 shutdown_connections 2024-01-15T18:05:44.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:44.915+0000 7fe685000640 1 -- 172.21.15.143:0/2837897998 wait complete. 2024-01-15T18:05:44.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:04:56.857956+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:04:57.645977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:44.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 Processor -- start 2024-01-15T18:05:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 -- start start 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb99c143780 0x7fb99c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c0ce520 0x7fb99c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c0ceb50 con 0x7fb99c0720f0 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c072580 con 0x7fb99c143780 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15c110 con 0x7fb99c0ce520 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb99c143780 0x7fb99c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a1200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb99c0720f0 0x7fb99c15a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57218/0 (socket says 172.21.15.143:57218) 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb9a1200640 1 -- 172.21.15.143:0/993106088 learned_addr learned my addr 172.21.15.143:0/993106088 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99b400640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c0ce520 0x7fb99c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99aa00640 1 -- 172.21.15.143:0/993106088 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239117971 0 0) 0x7fb9940014a0 con 0x7fb99c0720f0 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99aa00640 1 -- 172.21.15.143:0/993106088 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb97c005740 con 0x7fb99c0720f0 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99be00640 1 -- 172.21.15.143:0/993106088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c0ce520 msgr2=0x7fb99c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99be00640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c0ce520 0x7fb99c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99be00640 1 -- 172.21.15.143:0/993106088 >> v1:172.21.15.143:6789/0 conn(0x7fb99c0720f0 legacy=0x7fb99c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99be00640 1 -- 172.21.15.143:0/993106088 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c15b7e0 con 0x7fb99c143780 2024-01-15T18:05:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.075+0000 7fb99b400640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c0ce520 0x7fb99c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb99be00640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c143780 0x7fb99c0cdfe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0097c0 tx=0x7fb98c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33630e7f0b0de5b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb99aa00640 1 -- 172.21.15.143:0/993106088 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0ad640 con 0x7fb99c143780 2024-01-15T18:05:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb99aa00640 1 -- 172.21.15.143:0/993106088 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c0ad7a0 con 0x7fb99c143780 2024-01-15T18:05:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb99aa00640 1 -- 172.21.15.143:0/993106088 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0adb30 con 0x7fb99c143780 2024-01-15T18:05:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb9a3200640 1 -- 172.21.15.143:0/993106088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c143780 msgr2=0x7fb99c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb9a3200640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c143780 0x7fb99c0cdfe0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0097c0 tx=0x7fb98c01c480 comp rx=0 tx=0).stop 2024-01-15T18:05:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb9a3200640 1 -- 172.21.15.143:0/993106088 shutdown_connections 2024-01-15T18:05:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb9a3200640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c0ce520 0x7fb99c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb9a3200640 1 --2- 172.21.15.143:0/993106088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c143780 0x7fb99c0cdfe0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.079+0000 7fb9a3200640 1 -- 172.21.15.143:0/993106088 >> 172.21.15.143:0/993106088 conn(0x7fb99c06c3f0 msgr2=0x7fb99c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:45.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 -- 172.21.15.143:0/993106088 shutdown_connections 2024-01-15T18:05:45.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 -- 172.21.15.143:0/993106088 wait complete. 2024-01-15T18:05:45.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 Processor -- start 2024-01-15T18:05:45.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 -- start start 2024-01-15T18:05:45.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb99c0720f0 0x7fb99c13ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 0x7fb99c0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c143780 0x7fb99c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0ae320 con 0x7fb99c0ce520 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c154730 con 0x7fb99c0720f0 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0ad7a0 con 0x7fb99c143780 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 0x7fb99c0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 0x7fb99c0b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59500/0 (socket says 172.21.15.143:59500) 2024-01-15T18:05:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 -- 172.21.15.143:0/2235607362 learned_addr learned my addr 172.21.15.143:0/2235607362 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99be00640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb99c0720f0 0x7fb99c13ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb9a1200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c143780 0x7fb99c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 -- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c143780 msgr2=0x7fb99c0b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c143780 0x7fb99c0b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 -- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb99c0720f0 msgr2=0x7fb99c13ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb99c0720f0 0x7fb99c13ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.083+0000 7fb99b400640 1 -- 172.21.15.143:0/2235607362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c009420 con 0x7fb99c0ce520 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb99b400640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 0x7fb99c0b36d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e810 tx=0x7fb98400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400c8d0 con 0x7fb99c0ce520 2024-01-15T18:05:45.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98400ca30 con 0x7fb99c0ce520 2024-01-15T18:05:45.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c0ada80 con 0x7fb99c0ce520 2024-01-15T18:05:45.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98400cdc0 con 0x7fb99c0ce520 2024-01-15T18:05:45.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c0ae040 con 0x7fb99c0ce520 2024-01-15T18:05:45.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c067500 con 0x7fb99c0ce520 2024-01-15T18:05:45.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb984013410 con 0x7fb99c0ce520 2024-01-15T18:05:45.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb993e00640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb9740668f0 0x7fb974068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb9840842b0 con 0x7fb99c0ce520 2024-01-15T18:05:45.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.087+0000 7fb99be00640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb9740668f0 0x7fb974068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.091+0000 7fb99be00640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb9740668f0 0x7fb974068db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0af000 tx=0x7fb98c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.091+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb984051400 con 0x7fb99c0ce520 2024-01-15T18:05:45.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb99c06cfc0 con 0x7fb99c0ce520 2024-01-15T18:05:45.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb993e00640 1 -- 172.21.15.143:0/2235607362 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb984054a20 con 0x7fb99c0ce520 2024-01-15T18:05:45.246 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb9740668f0 msgr2=0x7fb974068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb9740668f0 0x7fb974068db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb98c0af000 tx=0x7fb98c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 msgr2=0x7fb99c0b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 0x7fb99c0b36d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb98400e810 tx=0x7fb98400ecd0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 shutdown_connections 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb9740668f0 0x7fb974068db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb99c143780 0x7fb99c0b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb99c0ce520 0x7fb99c0b36d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 --2- 172.21.15.143:0/2235607362 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb99c0720f0 0x7fb99c13ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 >> 172.21.15.143:0/2235607362 conn(0x7fb99c06c3f0 msgr2=0x7fb99c06db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:45.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 shutdown_connections 2024-01-15T18:05:45.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.243+0000 7fb9a3200640 1 -- 172.21.15.143:0/2235607362 wait complete. 2024-01-15T18:05:45.268 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:05:45.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 Processor -- start 2024-01-15T18:05:45.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 -- start start 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f320c157300 0x7f320c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f320c0b4ac0 0x7f320c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320c0b2a20 con 0x7f320c0b21d0 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c0b35a0 con 0x7f320c157300 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f3212000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f320c0b2d00 con 0x7f320c0b4ac0 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f320c0b21d0 0x7f320c0b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57220/0 (socket says 172.21.15.143:57220) 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320aa00640 1 -- 172.21.15.143:0/3925173094 learned_addr learned my addr 172.21.15.143:0/3925173094 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:45.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320b400640 1 --2- 172.21.15.143:0/3925173094 >> v2:172.21.15.143:3300/0 conn(0x7f320c157300 0x7f320c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320be00640 1 --2- 172.21.15.143:0/3925173094 >> v2:172.21.15.204:3300/0 conn(0x7f320c0b4ac0 0x7f320c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320a000640 1 -- 172.21.15.143:0/3925173094 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087650877 0 0) 0x7f320c0b2a20 con 0x7f320c0b21d0 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320a000640 1 -- 172.21.15.143:0/3925173094 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31ec003530 con 0x7f320c0b21d0 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320b400640 1 -- 172.21.15.143:0/3925173094 >> v2:172.21.15.204:3300/0 conn(0x7f320c0b4ac0 msgr2=0x7f320c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320b400640 1 --2- 172.21.15.143:0/3925173094 >> v2:172.21.15.204:3300/0 conn(0x7f320c0b4ac0 0x7f320c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320b400640 1 -- 172.21.15.143:0/3925173094 >> v1:172.21.15.143:6789/0 conn(0x7f320c0b21d0 legacy=0x7f320c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.403+0000 7f320b400640 1 -- 172.21.15.143:0/3925173094 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c0b2fe0 con 0x7f320c157300 2024-01-15T18:05:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f320b400640 1 --2- 172.21.15.143:0/3925173094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c159b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f320c0b1f70 tx=0x7f31fc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47acd8322a9b71d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f320a000640 1 -- 172.21.15.143:0/3925173094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0ab650 con 0x7f320c157300 2024-01-15T18:05:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f320a000640 1 -- 172.21.15.143:0/3925173094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc0ab7b0 con 0x7f320c157300 2024-01-15T18:05:45.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f320a000640 1 -- 172.21.15.143:0/3925173094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0abb40 con 0x7f320c157300 2024-01-15T18:05:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 -- 172.21.15.143:0/3925173094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 msgr2=0x7f320c159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 --2- 172.21.15.143:0/3925173094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c159b70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f320c0b1f70 tx=0x7f31fc01c430 comp rx=0 tx=0).stop 2024-01-15T18:05:45.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 -- 172.21.15.143:0/3925173094 shutdown_connections 2024-01-15T18:05:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 --2- 172.21.15.143:0/3925173094 >> v2:172.21.15.204:3300/0 conn(0x7f320c0b4ac0 0x7f320c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 --2- 172.21.15.143:0/3925173094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c159b70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 -- 172.21.15.143:0/3925173094 >> 172.21.15.143:0/3925173094 conn(0x7f320c06c3f0 msgr2=0x7f320c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 -- 172.21.15.143:0/3925173094 shutdown_connections 2024-01-15T18:05:45.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.407+0000 7f3212000640 1 -- 172.21.15.143:0/3925173094 wait complete. 2024-01-15T18:05:45.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 Processor -- start 2024-01-15T18:05:45.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 -- start start 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f320c0b21d0 0x7f320c155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f320c0b4ac0 0x7f320c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c1f9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0ad3b0 con 0x7f320c157300 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c151a00 con 0x7f320c0b4ac0 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0aca10 con 0x7f320c0b21d0 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c1f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c1f9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59518/0 (socket says 172.21.15.143:59518) 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 -- 172.21.15.143:0/1125040629 learned_addr learned my addr 172.21.15.143:0/1125040629 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320b400640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f320c0b21d0 0x7f320c155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320aa00640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f320c0b4ac0 0x7f320c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 -- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f320c0b21d0 msgr2=0x7f320c155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f320c0b21d0 0x7f320c155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 -- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f320c0b4ac0 msgr2=0x7f320c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f320c0b4ac0 0x7f320c153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 -- 172.21.15.143:0/1125040629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31fc009420 con 0x7f320c157300 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f320be00640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c1f9590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3204007d90 tx=0x7f320400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204014420 con 0x7f320c157300 2024-01-15T18:05:45.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32040145a0 con 0x7f320c157300 2024-01-15T18:05:45.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204014950 con 0x7f320c157300 2024-01-15T18:05:45.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c0accf0 con 0x7f320c157300 2024-01-15T18:05:45.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.411+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1540f0 con 0x7f320c157300 2024-01-15T18:05:45.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.415+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c0ad4f0 con 0x7f320c157300 2024-01-15T18:05:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.415+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3204014ab0 con 0x7f320c157300 2024-01-15T18:05:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.415+0000 7f3203400640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31e40668f0 0x7f31e4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.415+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3204084df0 con 0x7f320c157300 2024-01-15T18:05:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.415+0000 7f320b400640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31e40668f0 0x7f31e4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.415+0000 7f320b400640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31e40668f0 0x7f31e4068db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0af000 tx=0x7f31fc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.419+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f320c0ad4f0 con 0x7f320c157300 2024-01-15T18:05:45.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.567+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f320c0ad4f0 con 0x7f320c157300 2024-01-15T18:05:45.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.567+0000 7f3203400640 1 -- 172.21.15.143:0/1125040629 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f320c0ad4f0 con 0x7f320c157300 2024-01-15T18:05:45.573 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31e40668f0 msgr2=0x7f31e4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31e40668f0 0x7f31e4068db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31fc0af000 tx=0x7f31fc0a7000 comp rx=0 tx=0).stop 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 msgr2=0x7f320c1f9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c1f9590 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3204007d90 tx=0x7f320400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 shutdown_connections 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31e40668f0 0x7f31e4068db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f320c157300 0x7f320c1f9590 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f320c0b4ac0 0x7f320c153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 --2- 172.21.15.143:0/1125040629 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f320c0b21d0 0x7f320c155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 >> 172.21.15.143:0/1125040629 conn(0x7f320c06c3f0 msgr2=0x7f320c0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:45.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 shutdown_connections 2024-01-15T18:05:45.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.571+0000 7f3212000640 1 -- 172.21.15.143:0/1125040629 wait complete. 2024-01-15T18:05:45.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:05:45.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 Processor -- start 2024-01-15T18:05:45.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 -- start start 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ebc0cdef0 0x7f0ebc15ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0ebc0cf150 0x7f0ebc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ebc0d02d0 con 0x7f0ebc06c540 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc0cf6c0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0ec0e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ebc0cf9a0 con 0x7f0ebc0cf150 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ebc0cdef0 0x7f0ebc15ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0ebc0cdef0 0x7f0ebc15ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59528/0 (socket says 172.21.15.143:59528) 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 -- 172.21.15.143:0/2270268660 learned_addr learned my addr 172.21.15.143:0/2270268660 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eba800640 1 --2- 172.21.15.143:0/2270268660 >> v2:172.21.15.204:3300/0 conn(0x7f0ebc0cf150 0x7f0ebc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 -- 172.21.15.143:0/2270268660 >> v2:172.21.15.204:3300/0 conn(0x7f0ebc0cf150 msgr2=0x7f0ebc15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 --2- 172.21.15.143:0/2270268660 >> v2:172.21.15.204:3300/0 conn(0x7f0ebc0cf150 0x7f0ebc15cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 -- 172.21.15.143:0/2270268660 >> v1:172.21.15.204:6789/0 conn(0x7f0ebc06c540 legacy=0x7f0ebc0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.731+0000 7f0eb9400640 1 -- 172.21.15.143:0/2270268660 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ebc0cfc80 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0eb9400640 1 --2- 172.21.15.143:0/2270268660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc15ca90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0718f0 tx=0x7f0ea401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890dd6f233005014 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0eb8a00640 1 -- 172.21.15.143:0/2270268660 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc06cb30 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0eb8a00640 1 -- 172.21.15.143:0/2270268660 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40ab660 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0eb8a00640 1 -- 172.21.15.143:0/2270268660 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40ab9f0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2270268660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 msgr2=0x7f0ebc15ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2270268660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc15ca90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0718f0 tx=0x7f0ea401c440 comp rx=0 tx=0).stop 2024-01-15T18:05:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2270268660 shutdown_connections 2024-01-15T18:05:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2270268660 >> v2:172.21.15.204:3300/0 conn(0x7f0ebc0cf150 0x7f0ebc15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2270268660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc15ca90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2270268660 >> 172.21.15.143:0/2270268660 conn(0x7f0ebc0d0e20 msgr2=0x7f0ebc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2270268660 shutdown_connections 2024-01-15T18:05:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.735+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2270268660 wait complete. 2024-01-15T18:05:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 Processor -- start 2024-01-15T18:05:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 -- start start 2024-01-15T18:05:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ebc06c540 0x7f0ebc0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ebc0cf150 0x7f0ebc0c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0c6d40 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0b4e60 con 0x7f0ebc06c540 2024-01-15T18:05:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ebc0c7710 con 0x7f0ebc0cf150 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc0b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59530/0 (socket says 172.21.15.143:59530) 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 -- 172.21.15.143:0/2783106419 learned_addr learned my addr 172.21.15.143:0/2783106419 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ebc06c540 0x7f0ebc0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eba800640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ebc0cf150 0x7f0ebc0c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 -- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ebc0cf150 msgr2=0x7f0ebc0c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ebc0cf150 0x7f0ebc0c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 -- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ebc06c540 msgr2=0x7f0ebc0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ebc06c540 0x7f0ebc0c5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 -- 172.21.15.143:0/2783106419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40094d0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb9400640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc0b4920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4019050 tx=0x7f0ea401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b4800 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40b4960 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ebc0c6410 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40b4d10 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.739+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ebc0c69d0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.743+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ebc067500 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.743+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f0ea40a9070 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.743+0000 7f0eb2000640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e940668f0 0x7f0e94068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.743+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0ea411ecb0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.743+0000 7f0eb9e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e940668f0 0x7f0e94068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.743+0000 7f0eb9e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e940668f0 0x7f0e94068db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007fd0 tx=0x7f0eac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.747+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ea40ebfa0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.895+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ebc0d1940 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.895+0000 7f0eb2000640 1 -- 172.21.15.143:0/2783106419 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ea40ef5c0 con 0x7f0ebc0cdef0 2024-01-15T18:05:45.901 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:05:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e940668f0 msgr2=0x7f0e94068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e940668f0 0x7f0e94068db0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0eac007fd0 tx=0x7f0eac009040 comp rx=0 tx=0).stop 2024-01-15T18:05:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 msgr2=0x7f0ebc0b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc0b4920 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4019050 tx=0x7f0ea401ac80 comp rx=0 tx=0).stop 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 shutdown_connections 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e940668f0 0x7f0e94068db0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0ebc0cf150 0x7f0ebc0c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ebc0cdef0 0x7f0ebc0b4920 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 --2- 172.21.15.143:0/2783106419 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0ebc06c540 0x7f0ebc0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 >> 172.21.15.143:0/2783106419 conn(0x7f0ebc0d0e20 msgr2=0x7f0ebc161530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 shutdown_connections 2024-01-15T18:05:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:45.899+0000 7f0ec0e00640 1 -- 172.21.15.143:0/2783106419 wait complete. 2024-01-15T18:05:45.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:46.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 Processor -- start 2024-01-15T18:05:46.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 -- start start 2024-01-15T18:05:46.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8358071bb0 0x7f8358072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835815c5a0 0x7f835815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83580ceb50 con 0x7f8358143780 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8358072570 con 0x7f8358071bb0 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.055+0000 7f835fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835815c110 con 0x7f835815c5a0 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f835d200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8358143780 0x7f83580cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57234/0 (socket says 172.21.15.143:57234) 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f835d200640 1 -- 172.21.15.143:0/3163465726 learned_addr learned my addr 172.21.15.143:0/3163465726 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357e00640 1 --2- 172.21.15.143:0/3163465726 >> v2:172.21.15.143:3300/0 conn(0x7f8358071bb0 0x7f8358072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f835dc00640 1 --2- 172.21.15.143:0/3163465726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835815c5a0 0x7f835815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357400640 1 -- 172.21.15.143:0/3163465726 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201692473 0 0) 0x7f8348001120 con 0x7f8358143780 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357400640 1 -- 172.21.15.143:0/3163465726 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8338003530 con 0x7f8358143780 2024-01-15T18:05:46.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357e00640 1 -- 172.21.15.143:0/3163465726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835815c5a0 msgr2=0x7f835815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:46.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357e00640 1 --2- 172.21.15.143:0/3163465726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835815c5a0 0x7f835815ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357e00640 1 -- 172.21.15.143:0/3163465726 >> v1:172.21.15.143:6789/0 conn(0x7f8358143780 legacy=0x7f83580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357e00640 1 -- 172.21.15.143:0/3163465726 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835815b7e0 con 0x7f8358071bb0 2024-01-15T18:05:46.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.059+0000 7f8357e00640 1 --2- 172.21.15.143:0/3163465726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8358071bb0 0x7f8358072030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83400097c0 tx=0x7f834001e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc099ee047179c77 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f8357400640 1 -- 172.21.15.143:0/3163465726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400a8660 con 0x7f8358071bb0 2024-01-15T18:05:46.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f8357400640 1 -- 172.21.15.143:0/3163465726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83400a87c0 con 0x7f8358071bb0 2024-01-15T18:05:46.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f8357400640 1 -- 172.21.15.143:0/3163465726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400a8b50 con 0x7f8358071bb0 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- 172.21.15.143:0/3163465726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8358071bb0 msgr2=0x7f8358072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 --2- 172.21.15.143:0/3163465726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8358071bb0 0x7f8358072030 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83400097c0 tx=0x7f834001e690 comp rx=0 tx=0).stop 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- 172.21.15.143:0/3163465726 shutdown_connections 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 --2- 172.21.15.143:0/3163465726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f835815c5a0 0x7f835815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 --2- 172.21.15.143:0/3163465726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8358071bb0 0x7f8358072030 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- 172.21.15.143:0/3163465726 >> 172.21.15.143:0/3163465726 conn(0x7f835806c3f0 msgr2=0x7f835806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:46.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- 172.21.15.143:0/3163465726 shutdown_connections 2024-01-15T18:05:46.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- 172.21.15.143:0/3163465726 wait complete. 2024-01-15T18:05:46.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 Processor -- start 2024-01-15T18:05:46.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- start start 2024-01-15T18:05:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8358071bb0 0x7f8358141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8358143780 0x7f83581340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 0x7f8358135180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8358139980 con 0x7f835815c5a0 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83581425e0 con 0x7f8358143780 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.063+0000 7f835fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8358142030 con 0x7f8358071bb0 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 0x7f8358135180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 0x7f8358135180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59558/0 (socket says 172.21.15.143:59558) 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 -- 172.21.15.143:0/1746107947 learned_addr learned my addr 172.21.15.143:0/1746107947 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835d200640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8358071bb0 0x7f8358141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f8357e00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8358143780 0x7f83581340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 -- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8358071bb0 msgr2=0x7f8358141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8358071bb0 0x7f8358141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 -- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8358143780 msgr2=0x7f83581340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8358143780 0x7f83581340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 -- 172.21.15.143:0/1746107947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009420 con 0x7f835815c5a0 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835dc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 0x7f8358135180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f835000ec10 tx=0x7f835000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835000cd20 con 0x7f835815c5a0 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835000f040 con 0x7f835815c5a0 2024-01-15T18:05:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500146c0 con 0x7f835815c5a0 2024-01-15T18:05:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8358142300 con 0x7f835815c5a0 2024-01-15T18:05:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8358139610 con 0x7f835815c5a0 2024-01-15T18:05:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8358139af0 con 0x7f835815c5a0 2024-01-15T18:05:46.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.067+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f8350014a30 con 0x7f835815c5a0 2024-01-15T18:05:46.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.071+0000 7f8354c00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8330066920 0x7f8330068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.071+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8350085120 con 0x7f835815c5a0 2024-01-15T18:05:46.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.071+0000 7f835d200640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8330066920 0x7f8330068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.071+0000 7f835d200640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8330066920 0x7f8330068de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8348001280 tx=0x7f8348009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.075+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83500522d0 con 0x7f835815c5a0 2024-01-15T18:05:46.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8358067500 con 0x7f835815c5a0 2024-01-15T18:05:46.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f8354c00640 1 -- 172.21.15.143:0/1746107947 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f83500558f0 con 0x7f835815c5a0 2024-01-15T18:05:46.263 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:46.263 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:46.263 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:46.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8330066920 msgr2=0x7f8330068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8330066920 0x7f8330068de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8348001280 tx=0x7f8348009040 comp rx=0 tx=0).stop 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 msgr2=0x7f8358135180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 0x7f8358135180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f835000ec10 tx=0x7f835000c490 comp rx=0 tx=0).stop 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 shutdown_connections 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8330066920 0x7f8330068de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f835815c5a0 0x7f8358135180 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8358143780 0x7f83581340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 --2- 172.21.15.143:0/1746107947 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8358071bb0 0x7f8358141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 >> 172.21.15.143:0/1746107947 conn(0x7f835806c3f0 msgr2=0x7f83580aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:46.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 shutdown_connections 2024-01-15T18:05:46.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.259+0000 7f835fc00640 1 -- 172.21.15.143:0/1746107947 wait complete. 2024-01-15T18:05:46.284 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 Processor -- start 2024-01-15T18:05:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 -- start start 2024-01-15T18:05:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f501006c540 0x7f50100cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f50100cf150 0x7f50100cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f501015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50100d02d0 con 0x7f501006c540 2024-01-15T18:05:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50100cf6c0 con 0x7f50100cf150 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5017e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50100cf9a0 con 0x7f501006b7b0 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f501006c540 0x7f50100cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f501006c540 0x7f50100cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59560/0 (socket says 172.21.15.143:59560) 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 -- 172.21.15.143:0/1120618409 learned_addr learned my addr 172.21.15.143:0/1120618409 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5014a00640 1 --2- 172.21.15.143:0/1120618409 >> v2:172.21.15.204:3300/0 conn(0x7f50100cf150 0x7f50100cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015e00640 1 --2- 172.21.15.143:0/1120618409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f501015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 -- 172.21.15.143:0/1120618409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 msgr2=0x7f501015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 --2- 172.21.15.143:0/1120618409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f501015ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 -- 172.21.15.143:0/1120618409 >> v2:172.21.15.204:3300/0 conn(0x7f50100cf150 msgr2=0x7f50100cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 --2- 172.21.15.143:0/1120618409 >> v2:172.21.15.204:3300/0 conn(0x7f50100cf150 0x7f50100cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.419+0000 7f5015400640 1 -- 172.21.15.143:0/1120618409 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50100cfc80 con 0x7f501006c540 2024-01-15T18:05:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5015400640 1 --2- 172.21.15.143:0/1120618409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f501006c540 0x7f50100cd7e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5010072810 tx=0x7f500001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7d318ad68d42bd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f500fe00640 1 -- 172.21.15.143:0/1120618409 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501006cb30 con 0x7f501006c540 2024-01-15T18:05:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f500fe00640 1 -- 172.21.15.143:0/1120618409 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50000ab620 con 0x7f501006c540 2024-01-15T18:05:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f500fe00640 1 -- 172.21.15.143:0/1120618409 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000ab9b0 con 0x7f501006c540 2024-01-15T18:05:46.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 -- 172.21.15.143:0/1120618409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f501006c540 msgr2=0x7f50100cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 --2- 172.21.15.143:0/1120618409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f501006c540 0x7f50100cd7e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5010072810 tx=0x7f500001c440 comp rx=0 tx=0).stop 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 -- 172.21.15.143:0/1120618409 shutdown_connections 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 --2- 172.21.15.143:0/1120618409 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f501015ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 --2- 172.21.15.143:0/1120618409 >> v2:172.21.15.204:3300/0 conn(0x7f50100cf150 0x7f50100cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 --2- 172.21.15.143:0/1120618409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f501006c540 0x7f50100cd7e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 -- 172.21.15.143:0/1120618409 >> 172.21.15.143:0/1120618409 conn(0x7f50100d0e20 msgr2=0x7f50100d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 -- 172.21.15.143:0/1120618409 shutdown_connections 2024-01-15T18:05:46.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.423+0000 7f5017e00640 1 -- 172.21.15.143:0/1120618409 wait complete. 2024-01-15T18:05:46.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 Processor -- start 2024-01-15T18:05:46.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 -- start start 2024-01-15T18:05:46.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f50100c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501006c540 0x7f501015ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 0x7f501015fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50100b4680 con 0x7f50100cf150 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50100b4ee0 con 0x7f501006c540 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501006f7d0 con 0x7f501006b7b0 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 0x7f501015fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 0x7f501015fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59566/0 (socket says 172.21.15.143:59566) 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 -- 172.21.15.143:0/1028472613 learned_addr learned my addr 172.21.15.143:0/1028472613 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5014a00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501006c540 0x7f501015ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015400640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f50100c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 -- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 msgr2=0x7f50100c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f50100c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 -- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501006c540 msgr2=0x7f501015ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501006c540 0x7f501015ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 -- 172.21.15.143:0/1028472613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50000094d0 con 0x7f50100cf150 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5015e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 0x7f501015fa60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f500800efc0 tx=0x7f500800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008014480 con 0x7f50100cf150 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500800f040 con 0x7f50100cf150 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50100b4890 con 0x7f50100cf150 2024-01-15T18:05:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5008014820 con 0x7f50100cf150 2024-01-15T18:05:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.427+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501006f430 con 0x7f50100cf150 2024-01-15T18:05:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.431+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50100b5050 con 0x7f50100cf150 2024-01-15T18:05:46.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.431+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f5008014980 con 0x7f50100cf150 2024-01-15T18:05:46.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.431+0000 7f500d600640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe80668f0 0x7f4fe8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.431+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f50080851e0 con 0x7f50100cf150 2024-01-15T18:05:46.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.431+0000 7f5015400640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe80668f0 0x7f4fe8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.431+0000 7f5015400640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe80668f0 0x7f4fe8068db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50000af000 tx=0x7f50000b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.435+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50080523c0 con 0x7f50100cf150 2024-01-15T18:05:46.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.587+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5010067500 con 0x7f50100cf150 2024-01-15T18:05:46.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.587+0000 7f500d600640 1 -- 172.21.15.143:0/1028472613 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f500805ae60 con 0x7f50100cf150 2024-01-15T18:05:46.593 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:46.593 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:40.506528+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe80668f0 msgr2=0x7f4fe8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe80668f0 0x7f4fe8068db0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f50000af000 tx=0x7f50000b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 msgr2=0x7f501015fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 0x7f501015fa60 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f500800efc0 tx=0x7f500800c490 comp rx=0 tx=0).stop 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 shutdown_connections 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fe80668f0 0x7f4fe8068db0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50100cf150 0x7f501015fa60 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f501006c540 0x7f501015ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 --2- 172.21.15.143:0/1028472613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f501006b7b0 0x7f50100c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 >> 172.21.15.143:0/1028472613 conn(0x7f50100d0e20 msgr2=0x7f5010168190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 shutdown_connections 2024-01-15T18:05:46.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.591+0000 7f5017e00640 1 -- 172.21.15.143:0/1028472613 wait complete. 2024-01-15T18:05:46.615 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-15T18:05:46.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff96e000640 1 Processor -- start 2024-01-15T18:05:46.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff96e000640 1 -- start start 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff96e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff9680cf5e0 0x7ff96806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff96e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff968071400 con 0x7ff96806a9d0 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff96e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9680716e0 con 0x7ff96806dc10 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff96e000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96806ae60 con 0x7ff9680cf5e0 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff96806a9d0 0x7ff96806d500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57238/0 (socket says 172.21.15.143:57238) 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966a00640 1 -- 172.21.15.143:0/1580133291 learned_addr learned my addr 172.21.15.143:0/1580133291 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:46.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967400640 1 --2- 172.21.15.143:0/1580133291 >> v2:172.21.15.204:3300/0 conn(0x7ff9680cf5e0 0x7ff96806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529638383 0 0) 0x7ff968071400 con 0x7ff96806a9d0 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff948003530 con 0x7ff96806a9d0 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077985343 0 0) 0x7ff948003530 con 0x7ff96806a9d0 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 >> v2:172.21.15.204:3300/0 conn(0x7ff9680cf5e0 msgr2=0x7ff96806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 --2- 172.21.15.143:0/1580133291 >> v2:172.21.15.204:3300/0 conn(0x7ff9680cf5e0 0x7ff96806a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 >> v1:172.21.15.204:6789/0 conn(0x7ff96806dc10 legacy=0x7ff968164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9680719c0 con 0x7ff96806a9d0 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff950001680 con 0x7ff96806a9d0 2024-01-15T18:05:46.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 >> v1:172.21.15.143:6789/0 conn(0x7ff96806a9d0 legacy=0x7ff96806d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff948003980 0x7ff948003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9480047d0 0x7ff948004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff948005190 0x7ff94800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96806ae60 con 0x7ff948005190 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94800c130 con 0x7ff948003980 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94800c2f0 con 0x7ff9480047d0 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967e00640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff948005190 0x7ff94800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967400640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff948003980 0x7ff948003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff966a00640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9480047d0 0x7ff948004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967e00640 1 -- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9480047d0 msgr2=0x7ff948004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:46.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967e00640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9480047d0 0x7ff948004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967e00640 1 -- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff948003980 msgr2=0x7ff948003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:46.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967e00640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff948003980 0x7ff948003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967e00640 1 -- 172.21.15.143:0/1580133291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948003530 con 0x7ff948005190 2024-01-15T18:05:46.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.751+0000 7ff967400640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff948003980 0x7ff948003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:46.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff967e00640 1 -- 172.21.15.143:0/1580133291 reap_dead start 2024-01-15T18:05:46.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff967e00640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff948005190 0x7ff94800b990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff960001e40 tx=0x7ff96001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c711c266a1a001d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960003b80 con 0x7ff948005190 2024-01-15T18:05:46.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff960003ce0 con 0x7ff948005190 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff966000640 1 -- 172.21.15.143:0/1580133291 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960005310 con 0x7ff948005190 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 -- 172.21.15.143:0/1580133291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff948005190 msgr2=0x7ff94800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff948005190 0x7ff94800b990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff960001e40 tx=0x7ff96001c740 comp rx=0 tx=0).stop 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 -- 172.21.15.143:0/1580133291 shutdown_connections 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 --2- 172.21.15.143:0/1580133291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff948005190 0x7ff94800b990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 -- 172.21.15.143:0/1580133291 >> 172.21.15.143:0/1580133291 conn(0x7ff9680cc7d0 msgr2=0x7ff9680ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 -- 172.21.15.143:0/1580133291 shutdown_connections 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.755+0000 7ff96e000640 1 -- 172.21.15.143:0/1580133291 wait complete. 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 Processor -- start 2024-01-15T18:05:46.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 -- start start 2024-01-15T18:05:46.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff948005190 0x7ff968157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680c8340 0x7ff9680c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 0x7ff9680c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681f3f30 con 0x7ff9680c49d0 2024-01-15T18:05:46.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680c5390 con 0x7ff9680c8340 2024-01-15T18:05:46.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff96e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680c3c10 con 0x7ff948005190 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 0x7ff9680c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 0x7ff9680c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59588/0 (socket says 172.21.15.143:59588) 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 -- 172.21.15.143:0/360596753 learned_addr learned my addr 172.21.15.143:0/360596753 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff966a00640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680c8340 0x7ff9680c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967400640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff948005190 0x7ff968157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 -- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff948005190 msgr2=0x7ff968157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff948005190 0x7ff968157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 -- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680c8340 msgr2=0x7ff9680c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680c8340 0x7ff9680c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 -- 172.21.15.143:0/360596753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9680c3ef0 con 0x7ff9680c49d0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff966a00640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680c8340 0x7ff9680c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff967e00640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 0x7ff9680c4e50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff96001cfe0 tx=0x7ff948005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.759+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960003b80 con 0x7ff9680c49d0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff960007dc0 con 0x7ff9680c49d0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960006da0 con 0x7ff9680c49d0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9680c41d0 con 0x7ff9680c49d0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681591b0 con 0x7ff9680c49d0 2024-01-15T18:05:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff968067500 con 0x7ff9680c49d0 2024-01-15T18:05:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff9600a7070 con 0x7ff9680c49d0 2024-01-15T18:05:46.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff95f400640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff940066920 0x7ff940068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff9600b0070 con 0x7ff9680c49d0 2024-01-15T18:05:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.763+0000 7ff967400640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff940066920 0x7ff940068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.767+0000 7ff967400640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff940066920 0x7ff940068de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff958009b60 tx=0x7ff958009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:46.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.767+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9600ea4a0 con 0x7ff9680c49d0 2024-01-15T18:05:46.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.951+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9680c4680 con 0x7ff9680c49d0 2024-01-15T18:05:46.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.951+0000 7ff95f400640 1 -- 172.21.15.143:0/360596753 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7ff9600edac0 con 0x7ff9680c49d0 2024-01-15T18:05:46.956 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:46.956 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:46.956 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:46.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff940066920 msgr2=0x7ff940068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff940066920 0x7ff940068de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff958009b60 tx=0x7ff958009190 comp rx=0 tx=0).stop 2024-01-15T18:05:46.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 msgr2=0x7ff9680c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:46.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 0x7ff9680c4e50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff96001cfe0 tx=0x7ff948005800 comp rx=0 tx=0).stop 2024-01-15T18:05:46.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 shutdown_connections 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff940066920 0x7ff940068de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9680c49d0 0x7ff9680c4e50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9680c8340 0x7ff9680c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 --2- 172.21.15.143:0/360596753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff948005190 0x7ff968157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 >> 172.21.15.143:0/360596753 conn(0x7ff9680cc7d0 msgr2=0x7ff9680ca430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 shutdown_connections 2024-01-15T18:05:46.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:46.955+0000 7ff96e000640 1 -- 172.21.15.143:0/360596753 wait complete. 2024-01-15T18:05:46.978 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:05:47.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 Processor -- start 2024-01-15T18:05:47.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 -- start start 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf015b260 0x7fdcf00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcf00d19f0 0x7fdcf00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf00c4d90 con 0x7fdcf00c5220 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf015b640 con 0x7fdcf00d19f0 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00cf800 con 0x7fdcf015b260 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcf00d19f0 0x7fdcf00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdcf00d19f0 0x7fdcf00cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59594/0 (socket says 172.21.15.143:59594) 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 -- 172.21.15.143:0/901885185 learned_addr learned my addr 172.21.15.143:0/901885185 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf5600640 1 --2- 172.21.15.143:0/901885185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf015b260 0x7fdcf00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 -- 172.21.15.143:0/901885185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf015b260 msgr2=0x7fdcf00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 --2- 172.21.15.143:0/901885185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf015b260 0x7fdcf00cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 -- 172.21.15.143:0/901885185 >> v1:172.21.15.204:6789/0 conn(0x7fdcf00c5220 legacy=0x7fdcf00c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.111+0000 7fdcf4c00640 1 -- 172.21.15.143:0/901885185 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00cfae0 con 0x7fdcf00d19f0 2024-01-15T18:05:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf4c00640 1 --2- 172.21.15.143:0/901885185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00d19f0 0x7fdcf00cdd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009870 tx=0x7fdcd801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd93828448715e0b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcefe00640 1 -- 172.21.15.143:0/901885185 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80ab620 con 0x7fdcf00d19f0 2024-01-15T18:05:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcefe00640 1 -- 172.21.15.143:0/901885185 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd80ab780 con 0x7fdcf00d19f0 2024-01-15T18:05:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcefe00640 1 -- 172.21.15.143:0/901885185 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80abb10 con 0x7fdcf00d19f0 2024-01-15T18:05:47.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 -- 172.21.15.143:0/901885185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00d19f0 msgr2=0x7fdcf00cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 --2- 172.21.15.143:0/901885185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00d19f0 0x7fdcf00cdd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009870 tx=0x7fdcd801c510 comp rx=0 tx=0).stop 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 -- 172.21.15.143:0/901885185 shutdown_connections 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 --2- 172.21.15.143:0/901885185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00d19f0 0x7fdcf00cdd50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 --2- 172.21.15.143:0/901885185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf015b260 0x7fdcf00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 -- 172.21.15.143:0/901885185 >> 172.21.15.143:0/901885185 conn(0x7fdcf00d0ed0 msgr2=0x7fdcf00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 -- 172.21.15.143:0/901885185 shutdown_connections 2024-01-15T18:05:47.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.115+0000 7fdcf8000640 1 -- 172.21.15.143:0/901885185 wait complete. 2024-01-15T18:05:47.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 Processor -- start 2024-01-15T18:05:47.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- start start 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 0x7fdcf0158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf00d19f0 0x7fdcf0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf015b260 0x7fdcf009bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01fa120 con 0x7fdcf00c5220 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01586c0 con 0x7fdcf015b260 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf01f95a0 con 0x7fdcf00d19f0 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 0x7fdcf0158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 0x7fdcf0158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59600/0 (socket says 172.21.15.143:59600) 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 -- 172.21.15.143:0/84281926 learned_addr learned my addr 172.21.15.143:0/84281926 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf6000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf015b260 0x7fdcf009bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf4c00640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf00d19f0 0x7fdcf0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 -- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf00d19f0 msgr2=0x7fdcf0153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf00d19f0 0x7fdcf0153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 -- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf015b260 msgr2=0x7fdcf009bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf015b260 0x7fdcf009bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 -- 172.21.15.143:0/84281926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80094d0 con 0x7fdcf00c5220 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf5600640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 0x7fdcf0158180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdce000d5f0 tx=0x7fdce000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce00125f0 con 0x7fdcf00c5220 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce0012750 con 0x7fdcf00c5220 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf01f9880 con 0x7fdcf00c5220 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0012ae0 con 0x7fdcf00c5220 2024-01-15T18:05:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf01f9e40 con 0x7fdcf00c5220 2024-01-15T18:05:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.119+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf0067500 con 0x7fdcf00c5220 2024-01-15T18:05:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.123+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fdce0012c40 con 0x7fdcf00c5220 2024-01-15T18:05:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.123+0000 7fdced600640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdcc8066920 0x7fdcc8068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.123+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fdce0084890 con 0x7fdcf00c5220 2024-01-15T18:05:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.123+0000 7fdcf4c00640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdcc8066920 0x7fdcc8068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.123+0000 7fdcf4c00640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdcc8066920 0x7fdcc8068de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80af000 tx=0x7fdcd80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.127+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdce0051a80 con 0x7fdcf00c5220 2024-01-15T18:05:47.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.311+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcf01590f0 con 0x7fdcf00c5220 2024-01-15T18:05:47.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.311+0000 7fdced600640 1 -- 172.21.15.143:0/84281926 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fdce00550a0 con 0x7fdcf00c5220 2024-01-15T18:05:47.317 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:47.317 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/757062356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":757062356},{"type":"v1","addr":"172.21.15.143:6837","nonce":757062356}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/4237848180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":4237848180},{"type":"v1","addr":"172.21.15.204:6834","nonce":4237848180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2302534188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2302534188},{"type":"v1","addr":"172.21.15.204:6835","nonce":2302534188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:04:58.646333+0000","modified":"2024-01-15T18:05:01.878628+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/1204727493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1204727493},{"type":"v1","addr":"172.21.15.143:6835","nonce":1204727493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T18:05:47.317 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 8 2024-01-15T18:05:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdcc8066920 msgr2=0x7fdcc8068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdcc8066920 0x7fdcc8068de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80af000 tx=0x7fdcd80b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:47.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 msgr2=0x7fdcf0158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 0x7fdcf0158180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdce000d5f0 tx=0x7fdce000dab0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 shutdown_connections 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdcc8066920 0x7fdcc8068de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdcf015b260 0x7fdcf009bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdcf00d19f0 0x7fdcf0153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 --2- 172.21.15.143:0/84281926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdcf00c5220 0x7fdcf0158180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 >> 172.21.15.143:0/84281926 conn(0x7fdcf00d0ed0 msgr2=0x7fdcf00cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 shutdown_connections 2024-01-15T18:05:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.315+0000 7fdcf8000640 1 -- 172.21.15.143:0/84281926 wait complete. 2024-01-15T18:05:47.339 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:47.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a59200640 1 Processor -- start 2024-01-15T18:05:47.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a59200640 1 -- start start 2024-01-15T18:05:47.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a59200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 0x7f8a540cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a59200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540c4d90 con 0x7f8a540ccaf0 2024-01-15T18:05:47.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a59200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5415b640 con 0x7f8a540d19f0 2024-01-15T18:05:47.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a59200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540cf800 con 0x7f8a5415b260 2024-01-15T18:05:47.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a52800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a540ccaf0 0x7f8a540c5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33332/0 (socket says 172.21.15.143:33332) 2024-01-15T18:05:47.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a52800640 1 -- 172.21.15.143:0/3912079532 learned_addr learned my addr 172.21.15.143:0/3912079532 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 0x7f8a540cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005421560 0 0) 0x7f8a540c4d90 con 0x7f8a540ccaf0 2024-01-15T18:05:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a34003530 con 0x7f8a540ccaf0 2024-01-15T18:05:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2881406701 0 0) 0x7f8a34003530 con 0x7f8a540ccaf0 2024-01-15T18:05:47.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 msgr2=0x7f8a540cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 0x7f8a540cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 >> v1:172.21.15.204:6789/0 conn(0x7f8a540d19f0 legacy=0x7f8a540c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540cfae0 con 0x7f8a540ccaf0 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8a4c001a00 con 0x7f8a540ccaf0 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 >> v1:172.21.15.143:6789/0 conn(0x7f8a540ccaf0 legacy=0x7f8a540c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a340039b0 0x7f8a34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 0x7f8a340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a34004d10 0x7f8a34005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a540cf800 con 0x7f8a340039b0 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3400b990 con 0x7f8a34004d10 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3400bb50 con 0x7f8a5415b260 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a340039b0 0x7f8a34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51400640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 0x7f8a340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a52800640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a34004d10 0x7f8a34005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 -- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 msgr2=0x7f8a340047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a5415b260 0x7f8a340047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 -- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a34004d10 msgr2=0x7f8a34005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a34004d10 0x7f8a34005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.471+0000 7f8a51e00640 1 -- 172.21.15.143:0/3912079532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34003530 con 0x7f8a340039b0 2024-01-15T18:05:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a51e00640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a340039b0 0x7f8a34003d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009ae0 tx=0x7f8a4401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e007c096b9e1e44 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a440a5580 con 0x7f8a340039b0 2024-01-15T18:05:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a440a56e0 con 0x7f8a340039b0 2024-01-15T18:05:47.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a50a00640 1 -- 172.21.15.143:0/3912079532 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a440a5a90 con 0x7f8a340039b0 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a59200640 1 -- 172.21.15.143:0/3912079532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a340039b0 msgr2=0x7f8a34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a59200640 1 --2- 172.21.15.143:0/3912079532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a340039b0 0x7f8a34003d90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009ae0 tx=0x7f8a4401c990 comp rx=0 tx=0).stop 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a52800640 1 -- 172.21.15.143:0/3912079532 reap_dead start 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a59200640 1 -- 172.21.15.143:0/3912079532 shutdown_connections 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a59200640 1 -- 172.21.15.143:0/3912079532 >> 172.21.15.143:0/3912079532 conn(0x7f8a540d0ed0 msgr2=0x7f8a540d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a59200640 1 -- 172.21.15.143:0/3912079532 shutdown_connections 2024-01-15T18:05:47.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.475+0000 7f8a59200640 1 -- 172.21.15.143:0/3912079532 wait complete. 2024-01-15T18:05:47.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 Processor -- start 2024-01-15T18:05:47.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 -- start start 2024-01-15T18:05:47.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a340039b0 0x7f8a5414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 0x7f8a5409be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5409c3b0 0x7f8a540b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5409c830 con 0x7f8a541f6010 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5409bbb0 con 0x7f8a5409c3b0 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a59200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5414d200 con 0x7f8a340039b0 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 0x7f8a5409be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a340039b0 0x7f8a5414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a52800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5409c3b0 0x7f8a540b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 0x7f8a5409be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36226/0 (socket says 172.21.15.143:36226) 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a340039b0 0x7f8a5414ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:51356/0 (socket says 172.21.15.143:51356) 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 -- 172.21.15.143:0/1922803103 learned_addr learned my addr 172.21.15.143:0/1922803103 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 -- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a340039b0 msgr2=0x7f8a5414ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a340039b0 0x7f8a5414ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 -- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5409c3b0 msgr2=0x7f8a540b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5409c3b0 0x7f8a540b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 -- 172.21.15.143:0/1922803103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5409afa0 con 0x7f8a541f6010 2024-01-15T18:05:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a52800640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5409c3b0 0x7f8a540b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a51400640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 0x7f8a5409be70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c011c60 tx=0x7f8a3c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.479+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00fcb0 con 0x7f8a541f6010 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c00fe30 con 0x7f8a541f6010 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5409b280 con 0x7f8a541f6010 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c005310 con 0x7f8a541f6010 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5409b840 con 0x7f8a541f6010 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a540b5260 con 0x7f8a541f6010 2024-01-15T18:05:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f8a3c003600 con 0x7f8a541f6010 2024-01-15T18:05:47.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a4a000640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a2c0668f0 0x7f8a2c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8a3c087ce0 con 0x7f8a541f6010 2024-01-15T18:05:47.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.483+0000 7f8a51e00640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a2c0668f0 0x7f8a2c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.487+0000 7f8a51e00640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a2c0668f0 0x7f8a2c068db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a440af000 tx=0x7f8a440046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.487+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a3c054f40 con 0x7f8a541f6010 2024-01-15T18:05:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.639+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a54067500 con 0x7f8a541f6010 2024-01-15T18:05:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.639+0000 7f8a4a000640 1 -- 172.21.15.143:0/1922803103 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16790 (secure 0 0 0) 0x7f8a3c07fe60 con 0x7f8a541f6010 2024-01-15T18:05:47.645 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:47.645 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":22,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:40.506528+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:04:56.857956+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T18:04:57.645977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a2c0668f0 msgr2=0x7f8a2c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a2c0668f0 0x7f8a2c068db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a440af000 tx=0x7f8a440046d0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 msgr2=0x7f8a5409be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 0x7f8a5409be70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c011c60 tx=0x7f8a3c00f5a0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 shutdown_connections 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a2c0668f0 0x7f8a2c068db0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a5409c3b0 0x7f8a540b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a541f6010 0x7f8a5409be70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 --2- 172.21.15.143:0/1922803103 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a340039b0 0x7f8a5414ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 >> 172.21.15.143:0/1922803103 conn(0x7f8a540d0ed0 msgr2=0x7f8a540cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 shutdown_connections 2024-01-15T18:05:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.643+0000 7f8a59200640 1 -- 172.21.15.143:0/1922803103 wait complete. 2024-01-15T18:05:47.667 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-15T18:05:47.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.799+0000 7f54c6400640 1 Processor -- start 2024-01-15T18:05:47.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.799+0000 7f54c6400640 1 -- start start 2024-01-15T18:05:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54c6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54c015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54c6400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c015f310 con 0x7f54c0141660 2024-01-15T18:05:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54c6400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c015f5f0 con 0x7f54c0071d00 2024-01-15T18:05:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54c6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c015fbb0 con 0x7f54c00ce560 2024-01-15T18:05:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bf400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f54c0141660 0x7f54c0141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33342/0 (socket says 172.21.15.143:33342) 2024-01-15T18:05:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bf400640 1 -- 172.21.15.143:0/1309730813 learned_addr learned my addr 172.21.15.143:0/1309730813 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bea00640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54c015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494223654 0 0) 0x7f54c015f310 con 0x7f54c0141660 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54a0003530 con 0x7f54c0141660 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171464237 0 0) 0x7f54c015f5f0 con 0x7f54c0071d00 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c015f310 con 0x7f54c0071d00 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3488985843 0 0) 0x7f54a0003530 con 0x7f54c0141660 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 msgr2=0x7f54c015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54c015ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 >> v1:172.21.15.204:6789/0 conn(0x7f54c0071d00 legacy=0x7f54c00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c015f8d0 con 0x7f54c0141660 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f54b0001680 con 0x7f54c0141660 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 >> v1:172.21.15.143:6789/0 conn(0x7f54c0141660 legacy=0x7f54c0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54a0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a00040f0 0x7f54a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54a0004d10 0x7f54a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c015fbb0 con 0x7f54a0004d10 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a000bad0 con 0x7f54a00040f0 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a000bc90 con 0x7f54c00ce560 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bf400640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54a0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bfe00640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54a0004d10 0x7f54a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bea00640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a00040f0 0x7f54a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bfe00640 1 -- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 msgr2=0x7f54a0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bfe00640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54a0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bfe00640 1 -- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a00040f0 msgr2=0x7f54a00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bfe00640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a00040f0 0x7f54a00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bfe00640 1 -- 172.21.15.143:0/1309730813 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a0003530 con 0x7f54a0004d10 2024-01-15T18:05:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.803+0000 7f54bf400640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c00ce560 0x7f54a0003b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:47.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54bfe00640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54a0004d10 0x7f54a0005190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001fd0 tx=0x7f54b801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aea2ff41e444a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8003b80 con 0x7f54a0004d10 2024-01-15T18:05:47.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8003d00 con 0x7f54a0004d10 2024-01-15T18:05:47.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54be000640 1 -- 172.21.15.143:0/1309730813 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8005310 con 0x7f54a0004d10 2024-01-15T18:05:47.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54c6400640 1 -- 172.21.15.143:0/1309730813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54a0004d10 msgr2=0x7f54a0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54c6400640 1 --2- 172.21.15.143:0/1309730813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54a0004d10 0x7f54a0005190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001fd0 tx=0x7f54b801c8e0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54bfe00640 1 -- 172.21.15.143:0/1309730813 reap_dead start 2024-01-15T18:05:47.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54c6400640 1 -- 172.21.15.143:0/1309730813 shutdown_connections 2024-01-15T18:05:47.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54c6400640 1 -- 172.21.15.143:0/1309730813 >> 172.21.15.143:0/1309730813 conn(0x7f54c006c3f0 msgr2=0x7f54c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:47.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54c6400640 1 -- 172.21.15.143:0/1309730813 shutdown_connections 2024-01-15T18:05:47.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.807+0000 7f54c6400640 1 -- 172.21.15.143:0/1309730813 wait complete. 2024-01-15T18:05:47.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 Processor -- start 2024-01-15T18:05:47.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 -- start start 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a0004d10 0x7f54c015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c015b760 0x7f54c01376f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 0x7f54c0143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c00ce9e0 con 0x7f54c0137c30 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c015c6e0 con 0x7f54a0004d10 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c015c120 con 0x7f54c015b760 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 0x7f54c0143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 0x7f54c0143c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36246/0 (socket says 172.21.15.143:36246) 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 -- 172.21.15.143:0/2984088713 learned_addr learned my addr 172.21.15.143:0/2984088713 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bea00640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c015b760 0x7f54c01376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bf400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a0004d10 0x7f54c015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 -- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c015b760 msgr2=0x7f54c01376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c015b760 0x7f54c01376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 -- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a0004d10 msgr2=0x7f54c015aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a0004d10 0x7f54c015aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 -- 172.21.15.143:0/2984088713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c015c400 con 0x7f54c0137c30 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54bfe00640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 0x7f54c0143c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001fd0 tx=0x7f54a0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80057b0 con 0x7f54c0137c30 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b8003b90 con 0x7f54c0137c30 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8006da0 con 0x7f54c0137c30 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c0136ef0 con 0x7f54c0137c30 2024-01-15T18:05:47.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.811+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c015b000 con 0x7f54c0137c30 2024-01-15T18:05:47.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.815+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c0067500 con 0x7f54c0137c30 2024-01-15T18:05:47.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.815+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f54b80a6450 con 0x7f54c0137c30 2024-01-15T18:05:47.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.815+0000 7f54b7400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54980668f0 0x7f5498068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:47.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.815+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f54b811e550 con 0x7f54c0137c30 2024-01-15T18:05:47.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.815+0000 7f54bf400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54980668f0 0x7f5498068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:47.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.815+0000 7f54bf400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54980668f0 0x7f5498068db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54b0000c10 tx=0x7f54b00011d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:47.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:47.819+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54b80eb720 con 0x7f54c0137c30 2024-01-15T18:05:48.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.003+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f54c01373a0 con 0x7f54c0137c30 2024-01-15T18:05:48.006 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:05:48.003+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:05:48.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.003+0000 7f54b7400640 1 -- 172.21.15.143:0/2984088713 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f54b80eed40 con 0x7f54c0137c30 2024-01-15T18:05:48.008 INFO:teuthology.orchestra.run.smithi143.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-15T18:05:48.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54980668f0 msgr2=0x7f5498068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54980668f0 0x7f5498068db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54b0000c10 tx=0x7f54b00011d0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 msgr2=0x7f54c0143c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 0x7f54c0143c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54b8001fd0 tx=0x7f54a0005800 comp rx=0 tx=0).stop 2024-01-15T18:05:48.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 shutdown_connections 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54980668f0 0x7f5498068db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54c0137c30 0x7f54c0143c70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54c015b760 0x7f54c01376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 --2- 172.21.15.143:0/2984088713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54a0004d10 0x7f54c015aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 >> 172.21.15.143:0/2984088713 conn(0x7f54c006c3f0 msgr2=0x7f54c00b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 shutdown_connections 2024-01-15T18:05:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.007+0000 7f54c6400640 1 -- 172.21.15.143:0/2984088713 wait complete. 2024-01-15T18:05:48.018 INFO:tasks.ceph.mds.c.smithi143.stderr: -2967> 2024-01-15T18:05:02.237+0000 7f16dbe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:05:48.033 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-15T18:05:48.065 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:05:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.167+0000 7fb6f6a00640 1 Processor -- start 2024-01-15T18:05:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.167+0000 7fb6f6a00640 1 -- start start 2024-01-15T18:05:48.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0072410 0x7fb6f015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:48.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f6a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f015f310 con 0x7fb6f00ce560 2024-01-15T18:05:48.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f6a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f015f5f0 con 0x7fb6f0141660 2024-01-15T18:05:48.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f015fbb0 con 0x7fb6f0072410 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6ef400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb6f00ce560 0x7fb6f0071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33356/0 (socket says 172.21.15.143:33356) 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6ef400640 1 -- 172.21.15.143:0/3263826297 learned_addr learned my addr 172.21.15.143:0/3263826297 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f4a00640 1 --2- 172.21.15.143:0/3263826297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0072410 0x7fb6f015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6eea00640 1 -- 172.21.15.143:0/3263826297 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923486634 0 0) 0x7fb6f015f310 con 0x7fb6f00ce560 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6eea00640 1 -- 172.21.15.143:0/3263826297 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d0003530 con 0x7fb6f00ce560 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f4a00640 1 -- 172.21.15.143:0/3263826297 >> v1:172.21.15.204:6789/0 conn(0x7fb6f0141660 legacy=0x7fb6f0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f4a00640 1 -- 172.21.15.143:0/3263826297 >> v1:172.21.15.143:6789/0 conn(0x7fb6f00ce560 legacy=0x7fb6f0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6eea00640 1 -- 172.21.15.143:0/3263826297 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3456238108 0 0) 0x7fb6d0003530 con 0x7fb6f00ce560 2024-01-15T18:05:48.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.171+0000 7fb6f4a00640 1 -- 172.21.15.143:0/3263826297 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f015f8d0 con 0x7fb6f0072410 2024-01-15T18:05:48.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f4a00640 1 --2- 172.21.15.143:0/3263826297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0072410 0x7fb6f015ec80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009dc0 tx=0x7fb6e801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1062689f1aa964 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:48.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6eea00640 1 -- 172.21.15.143:0/3263826297 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8001680 con 0x7fb6f0072410 2024-01-15T18:05:48.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6eea00640 1 -- 172.21.15.143:0/3263826297 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e80a55e0 con 0x7fb6f0072410 2024-01-15T18:05:48.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6eea00640 1 -- 172.21.15.143:0/3263826297 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80a5950 con 0x7fb6f0072410 2024-01-15T18:05:48.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 -- 172.21.15.143:0/3263826297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0072410 msgr2=0x7fb6f015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/3263826297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0072410 0x7fb6f015ec80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8009dc0 tx=0x7fb6e801c890 comp rx=0 tx=0).stop 2024-01-15T18:05:48.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 -- 172.21.15.143:0/3263826297 shutdown_connections 2024-01-15T18:05:48.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/3263826297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0072410 0x7fb6f015ec80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 -- 172.21.15.143:0/3263826297 >> 172.21.15.143:0/3263826297 conn(0x7fb6f006c3f0 msgr2=0x7fb6f006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:48.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 -- 172.21.15.143:0/3263826297 shutdown_connections 2024-01-15T18:05:48.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.175+0000 7fb6f6a00640 1 -- 172.21.15.143:0/3263826297 wait complete. 2024-01-15T18:05:48.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 Processor -- start 2024-01-15T18:05:48.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 -- start start 2024-01-15T18:05:48.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f0072410 0x7fb6f013fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:48.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 0x7fb6f0138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0141660 0x7fb6f00af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f013daa0 con 0x7fb6f00ce560 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f013dd80 con 0x7fb6f0072410 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6f6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f013e060 con 0x7fb6f0141660 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 0x7fb6f0138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 0x7fb6f0138780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36270/0 (socket says 172.21.15.143:36270) 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 -- 172.21.15.143:0/1001782319 learned_addr learned my addr 172.21.15.143:0/1001782319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 -- 172.21.15.143:0/1001782319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0141660 msgr2=0x7fb6f00af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0141660 0x7fb6f00af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 -- 172.21.15.143:0/1001782319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f0072410 msgr2=0x7fb6f013fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f0072410 0x7fb6f013fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 -- 172.21.15.143:0/1001782319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8009a70 con 0x7fb6f00ce560 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6ef400640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 0x7fb6f0138780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c750 tx=0x7fb6d800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.179+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8005ae0 con 0x7fb6f00ce560 2024-01-15T18:05:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8005c40 con 0x7fb6f00ce560 2024-01-15T18:05:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f00cd150 con 0x7fb6f00ce560 2024-01-15T18:05:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8007070 con 0x7fb6f00ce560 2024-01-15T18:05:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f013e620 con 0x7fb6f00ce560 2024-01-15T18:05:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f013e840 con 0x7fb6f00ce560 2024-01-15T18:05:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb6d80071d0 con 0x7fb6f00ce560 2024-01-15T18:05:48.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6e7e00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb6c80668f0 0x7fb6c8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:48.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fb6d8084390 con 0x7fb6f00ce560 2024-01-15T18:05:48.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.183+0000 7fb6efe00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb6c80668f0 0x7fb6c8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:48.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.187+0000 7fb6efe00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb6c80668f0 0x7fb6c8068db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000f70 tx=0x7fb6e00079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:48.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.187+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb6d8051500 con 0x7fb6f00ce560 2024-01-15T18:05:48.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:48.371+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6f0067500 con 0x7fb6f00ce560 2024-01-15T18:05:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6e7e00640 1 -- 172.21.15.143:0/1001782319 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6d8054b20 con 0x7fb6f00ce560 2024-01-15T18:05:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb6c80668f0 msgr2=0x7fb6c8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb6c80668f0 0x7fb6c8068db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0000f70 tx=0x7fb6e00079c0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 msgr2=0x7fb6f0138780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 0x7fb6f0138780 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c750 tx=0x7fb6d800cc10 comp rx=0 tx=0).stop 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 shutdown_connections 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb6c80668f0 0x7fb6c8068db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb6f0141660 0x7fb6f00af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb6f00ce560 0x7fb6f0138780 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 --2- 172.21.15.143:0/1001782319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb6f0072410 0x7fb6f013fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 >> 172.21.15.143:0/1001782319 conn(0x7fb6f006c3f0 msgr2=0x7fb6f013ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 shutdown_connections 2024-01-15T18:05:49.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.007+0000 7fb6f6a00640 1 -- 172.21.15.143:0/1001782319 wait complete. 2024-01-15T18:05:49.034 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-15T18:05:49.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 Processor -- start 2024-01-15T18:05:49.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 -- start start 2024-01-15T18:05:49.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8628070f20 0x7f862806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:49.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86281578d0 0x7f862815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:49.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862806a490 con 0x7f8628071300 2024-01-15T18:05:49.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862806a770 con 0x7f8628070f20 2024-01-15T18:05:49.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862806aa50 con 0x7f86281578d0 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f8627e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8628071300 0x7f86281571c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33358/0 (socket says 172.21.15.143:33358) 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f8627e00640 1 -- 172.21.15.143:0/3017517704 learned_addr learned my addr 172.21.15.143:0/3017517704 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862d600640 1 --2- 172.21.15.143:0/3017517704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86281578d0 0x7f862815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862cc00640 1 --2- 172.21.15.143:0/3017517704 >> v2:172.21.15.204:3300/0 conn(0x7f8628070f20 0x7f862806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f8627400640 1 -- 172.21.15.143:0/3017517704 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967678399 0 0) 0x7f862806a490 con 0x7f8628071300 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f8627400640 1 -- 172.21.15.143:0/3017517704 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8608003530 con 0x7f8628071300 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862d600640 1 -- 172.21.15.143:0/3017517704 >> v2:172.21.15.204:3300/0 conn(0x7f8628070f20 msgr2=0x7f862806b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862d600640 1 --2- 172.21.15.143:0/3017517704 >> v2:172.21.15.204:3300/0 conn(0x7f8628070f20 0x7f862806b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862d600640 1 -- 172.21.15.143:0/3017517704 >> v1:172.21.15.143:6789/0 conn(0x7f8628071300 legacy=0x7f86281571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:49.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.215+0000 7f862d600640 1 -- 172.21.15.143:0/3017517704 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862806ad30 con 0x7f86281578d0 2024-01-15T18:05:49.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862d600640 1 --2- 172.21.15.143:0/3017517704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86281578d0 0x7f862815d6b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8620009d90 tx=0x7f862001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e2951d61cd32e91 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:49.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f8627400640 1 -- 172.21.15.143:0/3017517704 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200a5900 con 0x7f86281578d0 2024-01-15T18:05:49.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f8627400640 1 -- 172.21.15.143:0/3017517704 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86200a5a60 con 0x7f86281578d0 2024-01-15T18:05:49.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f8627400640 1 -- 172.21.15.143:0/3017517704 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200a5df0 con 0x7f86281578d0 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 -- 172.21.15.143:0/3017517704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86281578d0 msgr2=0x7f862815d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 --2- 172.21.15.143:0/3017517704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86281578d0 0x7f862815d6b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8620009d90 tx=0x7f862001c920 comp rx=0 tx=0).stop 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 -- 172.21.15.143:0/3017517704 shutdown_connections 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 --2- 172.21.15.143:0/3017517704 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86281578d0 0x7f862815d6b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 --2- 172.21.15.143:0/3017517704 >> v2:172.21.15.204:3300/0 conn(0x7f8628070f20 0x7f862806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 -- 172.21.15.143:0/3017517704 >> 172.21.15.143:0/3017517704 conn(0x7f86280b1610 msgr2=0x7f86280b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 -- 172.21.15.143:0/3017517704 shutdown_connections 2024-01-15T18:05:49.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.219+0000 7f862f600640 1 -- 172.21.15.143:0/3017517704 wait complete. 2024-01-15T18:05:49.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 Processor -- start 2024-01-15T18:05:49.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 -- start start 2024-01-15T18:05:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8628070f20 0x7f862814b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8628071300 0x7f8628147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 0x7f86280cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862814bf10 con 0x7f86281578d0 2024-01-15T18:05:49.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86281480b0 con 0x7f8628070f20 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8628146db0 con 0x7f8628071300 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 0x7f86280cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 0x7f86280cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36284/0 (socket says 172.21.15.143:36284) 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 -- 172.21.15.143:0/1392596675 learned_addr learned my addr 172.21.15.143:0/1392596675 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862cc00640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8628070f20 0x7f862814b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f8627e00640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8628071300 0x7f8628147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 -- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8628071300 msgr2=0x7f8628147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8628071300 0x7f8628147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 -- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8628070f20 msgr2=0x7f862814b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8628070f20 0x7f862814b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 -- 172.21.15.143:0/1392596675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8620009a70 con 0x7f86281578d0 2024-01-15T18:05:49.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862d600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 0x7f86280cf420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f862001afd0 tx=0x7f86200196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200b9650 con 0x7f86281578d0 2024-01-15T18:05:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86200b97b0 con 0x7f86281578d0 2024-01-15T18:05:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200b9b20 con 0x7f86281578d0 2024-01-15T18:05:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8628147090 con 0x7f86281578d0 2024-01-15T18:05:49.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.223+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8628147650 con 0x7f86281578d0 2024-01-15T18:05:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.227+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8628074b00 con 0x7f86281578d0 2024-01-15T18:05:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.227+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f86200b9c80 con 0x7f86281578d0 2024-01-15T18:05:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.227+0000 7f8624c00640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86000668f0 0x7f8600068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.227+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f862011f680 con 0x7f86281578d0 2024-01-15T18:05:49.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.227+0000 7f862cc00640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86000668f0 0x7f8600068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:49.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.227+0000 7f862cc00640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86000668f0 0x7f8600068db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f862815d530 tx=0x7f8618009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:49.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.231+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86200ec920 con 0x7f86281578d0 2024-01-15T18:05:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:49.383+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8628147840 con 0x7f86281578d0 2024-01-15T18:05:50.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.007+0000 7f8624c00640 1 -- 172.21.15.143:0/1392596675 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f862011de60 con 0x7f86281578d0 2024-01-15T18:05:50.013 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' removed 2024-01-15T18:05:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86000668f0 msgr2=0x7f8600068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86000668f0 0x7f8600068db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f862815d530 tx=0x7f8618009190 comp rx=0 tx=0).stop 2024-01-15T18:05:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 msgr2=0x7f86280cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 0x7f86280cf420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f862001afd0 tx=0x7f86200196c0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 shutdown_connections 2024-01-15T18:05:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86000668f0 0x7f8600068db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86281578d0 0x7f86280cf420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8628071300 0x7f8628147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 --2- 172.21.15.143:0/1392596675 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8628070f20 0x7f862814b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.011+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 >> 172.21.15.143:0/1392596675 conn(0x7f86280b1610 msgr2=0x7f86280ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.015+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 shutdown_connections 2024-01-15T18:05:50.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.015+0000 7f862f600640 1 -- 172.21.15.143:0/1392596675 wait complete. 2024-01-15T18:05:50.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-15T18:05:50.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f636e200640 1 Processor -- start 2024-01-15T18:05:50.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f636e200640 1 -- start start 2024-01-15T18:05:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f636e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f63680cdf20 0x7f63680cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f636e200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63680c4d90 con 0x7f63680c6230 2024-01-15T18:05:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f636e200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636815b640 con 0x7f636815b260 2024-01-15T18:05:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f636e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63680cf800 con 0x7f63680cdf20 2024-01-15T18:05:50.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6367e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f63680c6230 0x7f63680cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33374/0 (socket says 172.21.15.143:33374) 2024-01-15T18:05:50.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6367e00640 1 -- 172.21.15.143:0/2983675686 learned_addr learned my addr 172.21.15.143:0/2983675686 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:50.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366a00640 1 --2- 172.21.15.143:0/2983675686 >> v2:172.21.15.204:3300/0 conn(0x7f63680cdf20 0x7f63680cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378955004 0 0) 0x7f63680c4d90 con 0x7f63680c6230 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6348003530 con 0x7f63680c6230 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500325842 0 0) 0x7f6348003530 con 0x7f63680c6230 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 >> v2:172.21.15.204:3300/0 conn(0x7f63680cdf20 msgr2=0x7f63680cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 --2- 172.21.15.143:0/2983675686 >> v2:172.21.15.204:3300/0 conn(0x7f63680cdf20 0x7f63680cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 >> v1:172.21.15.204:6789/0 conn(0x7f636815b260 legacy=0x7f63680cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680cfae0 con 0x7f63680c6230 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f6360001a00 con 0x7f63680c6230 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 >> v1:172.21.15.143:6789/0 conn(0x7f63680c6230 legacy=0x7f63680cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6348003980 0x7f6348003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63480047d0 0x7f6348004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6348005190 0x7f634800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680cf800 con 0x7f63480047d0 2024-01-15T18:05:50.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634800c030 con 0x7f6348003980 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.183+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634800c1f0 con 0x7f6348005190 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63480047d0 0x7f6348004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6367400640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6348003980 0x7f6348003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6367e00640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6348005190 0x7f634800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 -- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6348005190 msgr2=0x7f634800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6348005190 0x7f634800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 -- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6348003980 msgr2=0x7f6348003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6348003980 0x7f6348003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 -- 172.21.15.143:0/2983675686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348003530 con 0x7f63480047d0 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6367400640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6348003980 0x7f6348003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:50.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6367e00640 1 -- 172.21.15.143:0/2983675686 reap_dead start 2024-01-15T18:05:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366a00640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63480047d0 0x7f6348004c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6350009ae0 tx=0x7f635001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78723d2a46543194 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500a5530 con 0x7f63480047d0 2024-01-15T18:05:50.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63500a5690 con 0x7f63480047d0 2024-01-15T18:05:50.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.187+0000 7f6366000640 1 -- 172.21.15.143:0/2983675686 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500a5a40 con 0x7f63480047d0 2024-01-15T18:05:50.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- 172.21.15.143:0/2983675686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63480047d0 msgr2=0x7f6348004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63480047d0 0x7f6348004c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6350009ae0 tx=0x7f635001ca50 comp rx=0 tx=0).stop 2024-01-15T18:05:50.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- 172.21.15.143:0/2983675686 shutdown_connections 2024-01-15T18:05:50.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 --2- 172.21.15.143:0/2983675686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63480047d0 0x7f6348004c50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- 172.21.15.143:0/2983675686 >> 172.21.15.143:0/2983675686 conn(0x7f63680d0ed0 msgr2=0x7f63680d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:50.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- 172.21.15.143:0/2983675686 shutdown_connections 2024-01-15T18:05:50.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- 172.21.15.143:0/2983675686 wait complete. 2024-01-15T18:05:50.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 Processor -- start 2024-01-15T18:05:50.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- start start 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63480047d0 0x7f63680b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 0x7f63680b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63681465e0 0x7f63680c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b6b90 con 0x7f6368146180 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b58d0 con 0x7f63480047d0 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.191+0000 7f636e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b3410 con 0x7f63681465e0 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 0x7f63680b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 0x7f63680b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36306/0 (socket says 172.21.15.143:36306) 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 -- 172.21.15.143:0/3150449315 learned_addr learned my addr 172.21.15.143:0/3150449315 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6367400640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63480047d0 0x7f63680b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6367e00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63681465e0 0x7f63680c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 -- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63681465e0 msgr2=0x7f63680c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63681465e0 0x7f63680c6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 -- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63480047d0 msgr2=0x7f63680b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63480047d0 0x7f63680b6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 -- 172.21.15.143:0/3150449315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680b5b70 con 0x7f6368146180 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6367e00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63681465e0 0x7f63680c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f6366a00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 0x7f63680b29c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f635001c880 tx=0x7f63500197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500bd070 con 0x7f6368146180 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6350004020 con 0x7f6368146180 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63500043d0 con 0x7f6368146180 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f636e200640 1 -- 172.21.15.143:0/3150449315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63680b5e50 con 0x7f6368146180 2024-01-15T18:05:50.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f636e200640 1 -- 172.21.15.143:0/3150449315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636815a2a0 con 0x7f6368146180 2024-01-15T18:05:50.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6368067500 con 0x7f6368146180 2024-01-15T18:05:50.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.195+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f63500b4070 con 0x7f6368146180 2024-01-15T18:05:50.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.199+0000 7f635f400640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63400668f0 0x7f6340068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:50.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.199+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f635011e3d0 con 0x7f6368146180 2024-01-15T18:05:50.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.199+0000 7f6367400640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63400668f0 0x7f6340068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:50.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.199+0000 7f6367400640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63400668f0 0x7f6340068db0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c10 tx=0x7f6358009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:50.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.203+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63500eb950 con 0x7f6368146180 2024-01-15T18:05:50.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:50.351+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63680b6d50 con 0x7f6368146180 2024-01-15T18:05:51.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.015+0000 7f635f400640 1 -- 172.21.15.143:0/3150449315 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f63500eef70 con 0x7f6368146180 2024-01-15T18:05:51.021 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' removed 2024-01-15T18:05:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63400668f0 msgr2=0x7f6340068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63400668f0 0x7f6340068db0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6358000c10 tx=0x7f6358009040 comp rx=0 tx=0).stop 2024-01-15T18:05:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 msgr2=0x7f63680b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 0x7f63680b29c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f635001c880 tx=0x7f63500197f0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 shutdown_connections 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63400668f0 0x7f6340068db0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63681465e0 0x7f63680c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6368146180 0x7f63680b29c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 --2- 172.21.15.143:0/3150449315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63480047d0 0x7f63680b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 >> 172.21.15.143:0/3150449315 conn(0x7f63680d0ed0 msgr2=0x7f63680ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 shutdown_connections 2024-01-15T18:05:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.019+0000 7f635cc00640 1 -- 172.21.15.143:0/3150449315 wait complete. 2024-01-15T18:05:51.046 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:05:51.046 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:05:51.046 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:05:51.046 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:51.047 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:05:51.047 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:05:51.047 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:05:51.047 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:51.047 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:05:51.048 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:05:51.048 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:05:51.048 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:51.048 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:05:51.048 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:05:51.049 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:05:51.049 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:05:51.060 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:05:51.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:05:51.061 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:05:51.062 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:05:51.068 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:05:51.068 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:05:51.069 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:05:51.070 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:05:51.107 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:05:51.108 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:51.110 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:05:51.111 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:51.119 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:05:51.119 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:51.121 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:05:51.121 DEBUG:teuthology.parallel:result is None 2024-01-15T18:05:51.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:05:51.196 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:05:51.197 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:05:51.208 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:05:51.208 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:05:51.209 INFO:tasks.ceph.mds.d.smithi204.stderr:2024-01-15T18:05:51.205+0000 7fb5ccc00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-01-15T18:05:51.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.259+0000 7f9f25600640 1 Processor -- start 2024-01-15T18:05:51.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.259+0000 7f9f25600640 1 -- start start 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.259+0000 7f9f25600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f2015d130 0x7f9f200aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.259+0000 7f9f25600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2015d510 0x7f9f2006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f25600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f20160eb0 con 0x7f9f2006a490 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f25600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f20161190 con 0x7f9f2015d130 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f25600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20161470 con 0x7f9f2015d510 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f2015d130 0x7f9f200aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9f2006a490 0x7f9f20071260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33408/0 (socket says 172.21.15.143:33408) 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1f400640 1 -- 172.21.15.143:0/3231086711 learned_addr learned my addr 172.21.15.143:0/3231086711 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1e000640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2015d510 0x7f9f2006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1ea00640 1 -- 172.21.15.143:0/3231086711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2015d510 msgr2=0x7f9f2006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1ea00640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2015d510 0x7f9f2006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1ea00640 1 -- 172.21.15.143:0/3231086711 >> v1:172.21.15.143:6789/0 conn(0x7f9f2006a490 legacy=0x7f9f20071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1ea00640 1 -- 172.21.15.143:0/3231086711 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20161750 con 0x7f9f2015d130 2024-01-15T18:05:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1d600640 1 -- 172.21.15.143:0/3231086711 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699286979 0 0) 0x7f9f20160eb0 con 0x7f9f2006a490 2024-01-15T18:05:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1e000640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2015d510 0x7f9f2006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1ea00640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f200aeed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f1001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8cb2d1f62f91cd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1d600640 1 -- 172.21.15.143:0/3231086711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100a6040 con 0x7f9f2015d130 2024-01-15T18:05:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1d600640 1 -- 172.21.15.143:0/3231086711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f100aa600 con 0x7f9f2015d130 2024-01-15T18:05:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.263+0000 7f9f1d600640 1 -- 172.21.15.143:0/3231086711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100aa990 con 0x7f9f2015d130 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- 172.21.15.143:0/3231086711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 msgr2=0x7f9f200aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f200aeed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097c0 tx=0x7f9f1001c480 comp rx=0 tx=0).stop 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- 172.21.15.143:0/3231086711 shutdown_connections 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2015d510 0x7f9f2006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 --2- 172.21.15.143:0/3231086711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f200aeed0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- 172.21.15.143:0/3231086711 >> 172.21.15.143:0/3231086711 conn(0x7f9f200b1b90 msgr2=0x7f9f200b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- 172.21.15.143:0/3231086711 shutdown_connections 2024-01-15T18:05:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- 172.21.15.143:0/3231086711 wait complete. 2024-01-15T18:05:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 Processor -- start 2024-01-15T18:05:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- start start 2024-01-15T18:05:51.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2006a490 0x7f9f2014f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f20160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2015d510 0x7f9f2014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20160890 con 0x7f9f2015d130 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2014fd40 con 0x7f9f2015d510 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f25600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20150020 con 0x7f9f2006a490 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f1e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f20160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f1e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f20160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36356/0 (socket says 172.21.15.143:36356) 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.267+0000 7f9f1e000640 1 -- 172.21.15.143:0/4233045430 learned_addr learned my addr 172.21.15.143:0/4233045430 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1ea00640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2006a490 0x7f9f2014f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1f400640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2015d510 0x7f9f2014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1e000640 1 -- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2006a490 msgr2=0x7f9f2014f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1e000640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2006a490 0x7f9f2014f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1e000640 1 -- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2015d510 msgr2=0x7f9f2014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1e000640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2015d510 0x7f9f2014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1e000640 1 -- 172.21.15.143:0/4233045430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f10009420 con 0x7f9f2015d130 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f1e000640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f20160350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800c7f0 tx=0x7f9f0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08007820 con 0x7f9f2015d130 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f08007980 con 0x7f9f2015d130 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f20150300 con 0x7f9f2015d130 2024-01-15T18:05:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08007d30 con 0x7f9f2015d130 2024-01-15T18:05:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f201508c0 con 0x7f9f2015d130 2024-01-15T18:05:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f200723a0 con 0x7f9f2015d130 2024-01-15T18:05:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f9f080113f0 con 0x7f9f2015d130 2024-01-15T18:05:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.271+0000 7f9f16a00640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ef80668f0 0x7f9ef8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.275+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f08084400 con 0x7f9f2015d130 2024-01-15T18:05:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.275+0000 7f9f1ea00640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ef80668f0 0x7f9ef8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.275+0000 7f9f1ea00640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ef80668f0 0x7f9ef8068db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f100ae000 tx=0x7f9f100b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:51.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.279+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f08051950 con 0x7f9f2015d130 2024-01-15T18:05:51.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:51.443+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9f20067500 con 0x7f9f2015d130 2024-01-15T18:05:52.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.035+0000 7f9f16a00640 1 -- 172.21.15.143:0/4233045430 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9f08054f70 con 0x7f9f2015d130 2024-01-15T18:05:52.038 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:05:52.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.035+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ef80668f0 msgr2=0x7f9ef8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.035+0000 7f9f25600640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ef80668f0 0x7f9ef8068db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9f100ae000 tx=0x7f9f100b5040 comp rx=0 tx=0).stop 2024-01-15T18:05:52.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.035+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 msgr2=0x7f9f20160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.035+0000 7f9f25600640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f20160350 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800c7f0 tx=0x7f9f0800ccb0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 shutdown_connections 2024-01-15T18:05:52.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ef80668f0 0x7f9ef8068db0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2015d510 0x7f9f2014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2015d130 0x7f9f20160350 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 --2- 172.21.15.143:0/4233045430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2006a490 0x7f9f2014f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 >> 172.21.15.143:0/4233045430 conn(0x7f9f200b1b90 msgr2=0x7f9f200b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:52.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 shutdown_connections 2024-01-15T18:05:52.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.039+0000 7f9f25600640 1 -- 172.21.15.143:0/4233045430 wait complete. 2024-01-15T18:05:52.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:05:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.207+0000 7f336fe00640 1 Processor -- start 2024-01-15T18:05:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336fe00640 1 -- start start 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f33680aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336fe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3368160eb0 con 0x7f336815d130 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3368161190 con 0x7f336806aba0 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3368161470 con 0x7f336815d510 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336d400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f336815d130 0x7f336806a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33418/0 (socket says 172.21.15.143:33418) 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336d400640 1 -- 172.21.15.143:0/554675949 learned_addr learned my addr 172.21.15.143:0/554675949 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336de00640 1 --2- 172.21.15.143:0/554675949 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f33680aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631806884 0 0) 0x7f3368160eb0 con 0x7f336815d130 2024-01-15T18:05:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3348003530 con 0x7f336815d130 2024-01-15T18:05:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553902938 0 0) 0x7f3368161190 con 0x7f336806aba0 2024-01-15T18:05:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3368160eb0 con 0x7f336806aba0 2024-01-15T18:05:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336de00640 1 -- 172.21.15.143:0/554675949 >> v1:172.21.15.204:6789/0 conn(0x7f336806aba0 legacy=0x7f33680b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336de00640 1 -- 172.21.15.143:0/554675949 >> v1:172.21.15.143:6789/0 conn(0x7f336815d130 legacy=0x7f336806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930602465 0 0) 0x7f3348003530 con 0x7f336815d130 2024-01-15T18:05:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.211+0000 7f336de00640 1 -- 172.21.15.143:0/554675949 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3368161750 con 0x7f336815d510 2024-01-15T18:05:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336de00640 1 --2- 172.21.15.143:0/554675949 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f33680aeed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3360009dc0 tx=0x7f336001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abb45b0cecd7e4d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358001680 con 0x7f336815d510 2024-01-15T18:05:52.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33600a56b0 con 0x7f336815d510 2024-01-15T18:05:52.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f3367e00640 1 -- 172.21.15.143:0/554675949 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600a5a40 con 0x7f336815d510 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 -- 172.21.15.143:0/554675949 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 msgr2=0x7f33680aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 --2- 172.21.15.143:0/554675949 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f33680aeed0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3360009dc0 tx=0x7f336001c890 comp rx=0 tx=0).stop 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 -- 172.21.15.143:0/554675949 shutdown_connections 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 --2- 172.21.15.143:0/554675949 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f33680aeed0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 -- 172.21.15.143:0/554675949 >> 172.21.15.143:0/554675949 conn(0x7f33680b1b90 msgr2=0x7f33680b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 -- 172.21.15.143:0/554675949 shutdown_connections 2024-01-15T18:05:52.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.215+0000 7f336fe00640 1 -- 172.21.15.143:0/554675949 wait complete. 2024-01-15T18:05:52.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 Processor -- start 2024-01-15T18:05:52.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 -- start start 2024-01-15T18:05:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 0x7f3368158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f336815d130 0x7f336815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f336814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336815c980 con 0x7f336815d130 2024-01-15T18:05:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336815c170 con 0x7f336806aba0 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336815b500 con 0x7f336815d510 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 0x7f3368158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f336815d130 0x7f336815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 0x7f3368158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:46104/0 (socket says 172.21.15.143:46104) 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 -- 172.21.15.143:0/2867130395 learned_addr learned my addr 172.21.15.143:0/2867130395 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336de00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f336814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 -- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 msgr2=0x7f336814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f336814de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 -- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f336815d130 msgr2=0x7f336815bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f336815d130 0x7f336815bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 -- 172.21.15.143:0/2867130395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360009a70 con 0x7f336806aba0 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336d400640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 0x7f3368158840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f335800c780 tx=0x7f335800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358005cf0 con 0x7f336806aba0 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3358005e70 con 0x7f336806aba0 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358007290 con 0x7f336806aba0 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336815c3c0 con 0x7f336806aba0 2024-01-15T18:05:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.219+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3368145750 con 0x7f336806aba0 2024-01-15T18:05:52.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.223+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3368067500 con 0x7f336806aba0 2024-01-15T18:05:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.223+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f33580073f0 con 0x7f336806aba0 2024-01-15T18:05:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.223+0000 7f3365600640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33400668f0 0x7f3340068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.223+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3358083890 con 0x7f336806aba0 2024-01-15T18:05:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.223+0000 7f336ca00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33400668f0 0x7f3340068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.223+0000 7f336ca00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33400668f0 0x7f3340068db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3368145f60 tx=0x7f3350009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.227+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3358050e30 con 0x7f336806aba0 2024-01-15T18:05:52.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.431+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33680b2ff0 con 0x7f336806aba0 2024-01-15T18:05:52.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.431+0000 7f3365600640 1 -- 172.21.15.143:0/2867130395 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f3358017020 con 0x7f336806aba0 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:05:52.437 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.438 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.439 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:05:52.440 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.441 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.442 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.443 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.444 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:05:52.445 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.446 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:05:52.447 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:05:52.448 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.449 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.450 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:05:52.451 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mds", 2024-01-15T18:05:52.452 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxc6VlhRx7KBAAJCfSNEc1oy6GI6X8lnHtag==", 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mds" 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-mgr", 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxc6VlzCZ7KBAAQAcvBCeJMiOFibSgRjLI/A==", 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.453 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-osd", 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxc6VlKC97KBAA36+g/32YmdSjG16QyBF+6w==", 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-osd" 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd", 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxc6VlYDd7KBAA/VEWinr4BihQwBqMyWm0/g==", 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.454 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxc6Vl0j97KBAAK8BAe6VkntNe8euUMf2mgQ==", 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.bootstrap-rgw", 2024-01-15T18:05:52.455 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCxc6Vlf0h7KBAAGEynku/NZDm+BtEhiCqyWQ==", 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:05:52.456 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:05:52.457 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.458 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:52.458 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:05:52.458 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:05:52.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33400668f0 msgr2=0x7f3340068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33400668f0 0x7f3340068db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3368145f60 tx=0x7f3350009040 comp rx=0 tx=0).stop 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 msgr2=0x7f3368158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 0x7f3368158840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f335800c780 tx=0x7f335800cc40 comp rx=0 tx=0).stop 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 shutdown_connections 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33400668f0 0x7f3340068db0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f336815d510 0x7f336814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f336815d130 0x7f336815bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 --2- 172.21.15.143:0/2867130395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f336806aba0 0x7f3368158840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 >> 172.21.15.143:0/2867130395 conn(0x7f33680b1b90 msgr2=0x7f33680cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:52.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 shutdown_connections 2024-01-15T18:05:52.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.435+0000 7f336fe00640 1 -- 172.21.15.143:0/2867130395 wait complete. 2024-01-15T18:05:52.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-15T18:05:52.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f65600640 1 Processor -- start 2024-01-15T18:05:52.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f65600640 1 -- start start 2024-01-15T18:05:52.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f65600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8f60143780 0x7f8f600cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f65600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f600ceb50 con 0x7f8f600722c0 2024-01-15T18:05:52.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f65600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6015c110 con 0x7f8f600ce520 2024-01-15T18:05:52.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f65600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6015b7e0 con 0x7f8f60143780 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f600722c0 0x7f8f6015c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33432/0 (socket says 172.21.15.143:33432) 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5f400640 1 -- 172.21.15.143:0/1803922901 learned_addr learned my addr 172.21.15.143:0/1803922901 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/1803922901 >> v2:172.21.15.204:3300/0 conn(0x7f8f60143780 0x7f8f600cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268019065 0 0) 0x7f8f580014a0 con 0x7f8f600722c0 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f40003530 con 0x7f8f600722c0 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1473729301 0 0) 0x7f8f40003530 con 0x7f8f600722c0 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 >> v2:172.21.15.204:3300/0 conn(0x7f8f60143780 msgr2=0x7f8f600cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 --2- 172.21.15.143:0/1803922901 >> v2:172.21.15.204:3300/0 conn(0x7f8f60143780 0x7f8f600cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 >> v1:172.21.15.204:6789/0 conn(0x7f8f600ce520 legacy=0x7f8f60071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6015bac0 con 0x7f8f600722c0 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8f58001b90 con 0x7f8f600722c0 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 >> v1:172.21.15.143:6789/0 conn(0x7f8f600722c0 legacy=0x7f8f6015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f400039d0 0x7f8f40003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f400047d0 0x7f8f40004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f40005190 0x7f8f4000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6015b7e0 con 0x7f8f400039d0 2024-01-15T18:05:52.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4000e2a0 con 0x7f8f400047d0 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4000e460 con 0x7f8f40005190 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f400039d0 0x7f8f40003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5e000640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f400047d0 0x7f8f40004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5f400640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f40005190 0x7f8f4000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 -- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f40005190 msgr2=0x7f8f4000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f40005190 0x7f8f4000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 -- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f400047d0 msgr2=0x7f8f40004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f400047d0 0x7f8f40004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5ea00640 1 -- 172.21.15.143:0/1803922901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40003530 con 0x7f8f400039d0 2024-01-15T18:05:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5e000640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f400047d0 0x7f8f40004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:52.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.595+0000 7f8f5f400640 1 -- 172.21.15.143:0/1803922901 reap_dead start 2024-01-15T18:05:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f400039d0 0x7f8f40003df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009ae0 tx=0x7f8f5001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7272e249e188fe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500a9440 con 0x7f8f400039d0 2024-01-15T18:05:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f500a95a0 con 0x7f8f400039d0 2024-01-15T18:05:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f5d600640 1 -- 172.21.15.143:0/1803922901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500a9950 con 0x7f8f400039d0 2024-01-15T18:05:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 -- 172.21.15.143:0/1803922901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f400039d0 msgr2=0x7f8f40003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f400039d0 0x7f8f40003df0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009ae0 tx=0x7f8f5001c990 comp rx=0 tx=0).stop 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 -- 172.21.15.143:0/1803922901 shutdown_connections 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 --2- 172.21.15.143:0/1803922901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f400039d0 0x7f8f40003df0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 -- 172.21.15.143:0/1803922901 >> 172.21.15.143:0/1803922901 conn(0x7f8f6006c3f0 msgr2=0x7f8f6006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 -- 172.21.15.143:0/1803922901 shutdown_connections 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.599+0000 7f8f65600640 1 -- 172.21.15.143:0/1803922901 wait complete. 2024-01-15T18:05:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 Processor -- start 2024-01-15T18:05:52.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 -- start start 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 0x7f8f60140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f601395a0 0x7f8f601f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f600b3080 0x7f8f600b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60137490 con 0x7f8f601395a0 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600b3a50 con 0x7f8f600b3080 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60141770 con 0x7f8f400039d0 2024-01-15T18:05:52.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f601395a0 0x7f8f601f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 0x7f8f60140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f600b3080 0x7f8f600b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 0x7f8f60140de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:51576/0 (socket says 172.21.15.143:51576) 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 -- 172.21.15.143:0/3201950085 learned_addr learned my addr 172.21.15.143:0/3201950085 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 -- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f600b3080 msgr2=0x7f8f600b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f600b3080 0x7f8f600b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 -- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f601395a0 msgr2=0x7f8f601f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f601395a0 0x7f8f601f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 -- 172.21.15.143:0/3201950085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60141cf0 con 0x7f8f400039d0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5e000640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f601395a0 0x7f8f601f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f5ea00640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 0x7f8f60140de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009ae0 tx=0x7f8f5001b1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500a9d90 con 0x7f8f400039d0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f500b9420 con 0x7f8f400039d0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f500b9790 con 0x7f8f400039d0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f60141320 con 0x7f8f400039d0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.603+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601371b0 con 0x7f8f400039d0 2024-01-15T18:05:52.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.607+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60067500 con 0x7f8f400039d0 2024-01-15T18:05:52.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.607+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f8f500a7070 con 0x7f8f400039d0 2024-01-15T18:05:52.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.607+0000 7f8f56a00640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f380668f0 0x7f8f38068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.607+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f5011f0e0 con 0x7f8f400039d0 2024-01-15T18:05:52.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.607+0000 7f8f5e000640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f380668f0 0x7f8f38068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.607+0000 7f8f5e000640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f380668f0 0x7f8f38068db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f600b5270 tx=0x7f8f48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.611+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f500ec700 con 0x7f8f400039d0 2024-01-15T18:05:52.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.811+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f8f6015da70 con 0x7f8f400039d0 2024-01-15T18:05:52.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.819+0000 7f8f56a00640 1 -- 172.21.15.143:0/3201950085 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8f500efd20 con 0x7f8f400039d0 2024-01-15T18:05:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f380668f0 msgr2=0x7f8f38068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f380668f0 0x7f8f38068db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f600b5270 tx=0x7f8f48009040 comp rx=0 tx=0).stop 2024-01-15T18:05:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 msgr2=0x7f8f60140de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 0x7f8f60140de0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009ae0 tx=0x7f8f5001b1d0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 shutdown_connections 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f380668f0 0x7f8f38068db0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f600b3080 0x7f8f600b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f601395a0 0x7f8f601f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 --2- 172.21.15.143:0/3201950085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f400039d0 0x7f8f60140de0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 >> 172.21.15.143:0/3201950085 conn(0x7f8f6006c3f0 msgr2=0x7f8f6006d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 shutdown_connections 2024-01-15T18:05:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.823+0000 7f8f65600640 1 -- 172.21.15.143:0/3201950085 wait complete. 2024-01-15T18:05:52.846 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-15T18:05:52.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee74800640 1 Processor -- start 2024-01-15T18:05:52.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee74800640 1 -- start start 2024-01-15T18:05:52.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee74800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fee6c143780 0x7fee6c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee74800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c0ceb50 con 0x7fee6c0ce520 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee74800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c15c110 con 0x7fee6c0722c0 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee74800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6c15b7e0 con 0x7fee6c143780 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fee6c0ce520 0x7fee6c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33438/0 (socket says 172.21.15.143:33438) 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71400640 1 -- 172.21.15.143:0/2646313237 learned_addr learned my addr 172.21.15.143:0/2646313237 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 --2- 172.21.15.143:0/2646313237 >> v2:172.21.15.204:3300/0 conn(0x7fee6c143780 0x7fee6c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911006853 0 0) 0x7fee54001120 con 0x7fee6c0ce520 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee4c003530 con 0x7fee6c0ce520 2024-01-15T18:05:52.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891851297 0 0) 0x7fee6c15c110 con 0x7fee6c0722c0 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee54001120 con 0x7fee6c0722c0 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071521609 0 0) 0x7fee4c003530 con 0x7fee6c0ce520 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 >> v2:172.21.15.204:3300/0 conn(0x7fee6c143780 msgr2=0x7fee6c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 --2- 172.21.15.143:0/2646313237 >> v2:172.21.15.204:3300/0 conn(0x7fee6c143780 0x7fee6c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 >> v1:172.21.15.204:6789/0 conn(0x7fee6c0722c0 legacy=0x7fee6c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c15bac0 con 0x7fee6c0ce520 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fee54001810 con 0x7fee6c0ce520 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 >> v1:172.21.15.143:6789/0 conn(0x7fee6c0ce520 legacy=0x7fee6c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee4c003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee4c0047d0 0x7fee4c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee4c005190 0x7fee4c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c15b7e0 con 0x7fee4c003b60 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c00c060 con 0x7fee4c0047d0 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c00c220 con 0x7fee4c005190 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee4c003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71400640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee4c0047d0 0x7fee4c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee72800640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee4c005190 0x7fee4c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 -- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee4c005190 msgr2=0x7fee4c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:52.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee4c005190 0x7fee4c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 -- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee4c0047d0 msgr2=0x7fee4c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:52.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee4c0047d0 0x7fee4c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee71e00640 1 -- 172.21.15.143:0/2646313237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c003530 con 0x7fee4c003b60 2024-01-15T18:05:52.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.979+0000 7fee72800640 1 -- 172.21.15.143:0/2646313237 reap_dead start 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee71e00640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee4c003f60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009670 tx=0x7fee5c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff664148a6fb1237 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0a7570 con 0x7fee4c003b60 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c0a76d0 con 0x7fee4c003b60 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee70a00640 1 -- 172.21.15.143:0/2646313237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0a7a80 con 0x7fee4c003b60 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee74800640 1 -- 172.21.15.143:0/2646313237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 msgr2=0x7fee4c003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee74800640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee4c003f60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009670 tx=0x7fee5c01ebc0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee74800640 1 -- 172.21.15.143:0/2646313237 shutdown_connections 2024-01-15T18:05:52.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee74800640 1 --2- 172.21.15.143:0/2646313237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee4c003f60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.983+0000 7fee74800640 1 -- 172.21.15.143:0/2646313237 >> 172.21.15.143:0/2646313237 conn(0x7fee6c06c3f0 msgr2=0x7fee6c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:52.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 -- 172.21.15.143:0/2646313237 shutdown_connections 2024-01-15T18:05:52.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 -- 172.21.15.143:0/2646313237 wait complete. 2024-01-15T18:05:52.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 Processor -- start 2024-01-15T18:05:52.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 -- start start 2024-01-15T18:05:52.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee6c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee6c141bd0 0x7fee6c142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c1f6280 0x7fee6c1f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1f7bf0 con 0x7fee4c003b60 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c142590 con 0x7fee6c1f6280 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee74800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c1f7290 con 0x7fee6c141bd0 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee6c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee6c1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36392/0 (socket says 172.21.15.143:36392) 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 -- 172.21.15.143:0/11689453 learned_addr learned my addr 172.21.15.143:0/11689453 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee72800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c1f6280 0x7fee6c1f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71400640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee6c141bd0 0x7fee6c142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 -- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee6c141bd0 msgr2=0x7fee6c142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee6c141bd0 0x7fee6c142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 -- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c1f6280 msgr2=0x7fee6c1f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c1f6280 0x7fee6c1f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 -- 172.21.15.143:0/11689453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c159f20 con 0x7fee4c003b60 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee72800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c1f6280 0x7fee6c1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee71e00640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee6c1599e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0250f0 tx=0x7fee5c01b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.987+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0a7e20 con 0x7fee4c003b60 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c004030 con 0x7fee4c003b60 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0043a0 con 0x7fee4c003b60 2024-01-15T18:05:52.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c1f6fb0 con 0x7fee4c003b60 2024-01-15T18:05:52.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c1f7810 con 0x7fee4c003b60 2024-01-15T18:05:52.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c067500 con 0x7fee4c003b60 2024-01-15T18:05:52.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fee5c003620 con 0x7fee4c003b60 2024-01-15T18:05:52.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee6a000640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee44066920 0x7fee44068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:52.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fee5c11d650 con 0x7fee4c003b60 2024-01-15T18:05:52.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee71400640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee44066920 0x7fee44068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:52.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.991+0000 7fee71400640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee44066920 0x7fee44068de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee54001d20 tx=0x7fee54001bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:52.995+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee5c0eac80 con 0x7fee4c003b60 2024-01-15T18:05:53.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.211+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fee6c0b3840 con 0x7fee4c003b60 2024-01-15T18:05:53.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.215+0000 7fee6a000640 1 -- 172.21.15.143:0/11689453 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fee5c0b40a0 con 0x7fee4c003b60 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee44066920 msgr2=0x7fee44068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee44066920 0x7fee44068de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fee54001d20 tx=0x7fee54001bd0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 msgr2=0x7fee6c1599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee6c1599e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0250f0 tx=0x7fee5c01b960 comp rx=0 tx=0).stop 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 shutdown_connections 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee44066920 0x7fee44068de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c1f6280 0x7fee6c1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee6c141bd0 0x7fee6c142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 --2- 172.21.15.143:0/11689453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee4c003b60 0x7fee6c1599e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 >> 172.21.15.143:0/11689453 conn(0x7fee6c06c3f0 msgr2=0x7fee6c0afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:53.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 shutdown_connections 2024-01-15T18:05:53.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.219+0000 7fee74800640 1 -- 172.21.15.143:0/11689453 wait complete. 2024-01-15T18:05:53.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-15T18:05:53.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 Processor -- start 2024-01-15T18:05:53.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 -- start start 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f1415d130 0x7f3f140aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d510 0x7f3f1406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f14160eb0 con 0x7f3f1406a490 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f14161190 con 0x7f3f1415d130 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f1be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14161470 con 0x7f3f1415d510 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f19400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f1415d130 0x7f3f140aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f19400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3f1415d130 0x7f3f140aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36400/0 (socket says 172.21.15.143:36400) 2024-01-15T18:05:53.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f19400640 1 -- 172.21.15.143:0/2088673841 learned_addr learned my addr 172.21.15.143:0/2088673841 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:53.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f19e00640 1 --1- 172.21.15.143:0/2088673841 >> v1:172.21.15.143:6789/0 conn(0x7f3f1406a490 0x7f3f14071260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33452/0 (socket says 172.21.15.143:33452) 2024-01-15T18:05:53.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.375+0000 7f3f18a00640 1 --2- 172.21.15.143:0/2088673841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d510 0x7f3f1406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f19400640 1 -- 172.21.15.143:0/2088673841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d510 msgr2=0x7f3f1406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:53.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f19400640 1 --2- 172.21.15.143:0/2088673841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d510 0x7f3f1406e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f19400640 1 -- 172.21.15.143:0/2088673841 >> v1:172.21.15.143:6789/0 conn(0x7f3f1406a490 legacy=0x7f3f14071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f19400640 1 -- 172.21.15.143:0/2088673841 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14161750 con 0x7f3f1415d130 2024-01-15T18:05:53.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f19400640 1 --2- 172.21.15.143:0/2088673841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1415d130 0x7f3f140aeed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009870 tx=0x7f3f0401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf58f19e535cc4c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f13e00640 1 -- 172.21.15.143:0/2088673841 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040a9620 con 0x7f3f1415d130 2024-01-15T18:05:53.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f13e00640 1 -- 172.21.15.143:0/2088673841 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040a9780 con 0x7f3f1415d130 2024-01-15T18:05:53.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f13e00640 1 -- 172.21.15.143:0/2088673841 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040a9b10 con 0x7f3f1415d130 2024-01-15T18:05:53.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f1be00640 1 -- 172.21.15.143:0/2088673841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1415d130 msgr2=0x7f3f140aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.379+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2088673841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1415d130 0x7f3f140aeed0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009870 tx=0x7f3f0401c540 comp rx=0 tx=0).stop 2024-01-15T18:05:53.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- 172.21.15.143:0/2088673841 shutdown_connections 2024-01-15T18:05:53.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2088673841 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d510 0x7f3f1406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2088673841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1415d130 0x7f3f140aeed0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- 172.21.15.143:0/2088673841 >> 172.21.15.143:0/2088673841 conn(0x7f3f140b1b90 msgr2=0x7f3f140b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:53.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- 172.21.15.143:0/2088673841 shutdown_connections 2024-01-15T18:05:53.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- 172.21.15.143:0/2088673841 wait complete. 2024-01-15T18:05:53.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 Processor -- start 2024-01-15T18:05:53.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- start start 2024-01-15T18:05:53.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 0x7f3f14147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d130 0x7f3f14146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f1415d510 0x7f3f14146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f141472a0 con 0x7f3f1406a490 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14153d70 con 0x7f3f1415d510 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f1be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f14148ca0 con 0x7f3f1415d130 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 0x7f3f14147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 0x7f3f14147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36402/0 (socket says 172.21.15.143:36402) 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 -- 172.21.15.143:0/2218542560 learned_addr learned my addr 172.21.15.143:0/2218542560 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f18a00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d130 0x7f3f14146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19e00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f1415d510 0x7f3f14146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 -- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d130 msgr2=0x7f3f14146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d130 0x7f3f14146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 -- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f1415d510 msgr2=0x7f3f14146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f1415d510 0x7f3f14146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 -- 172.21.15.143:0/2218542560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040094d0 con 0x7f3f1406a490 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.383+0000 7f3f19400640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 0x7f3f14147680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3f0401a170 tx=0x7f3f0401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040bc650 con 0x7f3f1406a490 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040bc7d0 con 0x7f3f1406a490 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f040bcb80 con 0x7f3f1406a490 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f141499c0 con 0x7f3f1406a490 2024-01-15T18:05:53.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f14153a00 con 0x7f3f1406a490 2024-01-15T18:05:53.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f14067500 con 0x7f3f1406a490 2024-01-15T18:05:53.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3f040bcce0 con 0x7f3f1406a490 2024-01-15T18:05:53.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f11600640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eec0668f0 0x7f3eec068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3f0411e840 con 0x7f3f1406a490 2024-01-15T18:05:53.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.387+0000 7f3f18a00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eec0668f0 0x7f3eec068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.391+0000 7f3f18a00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eec0668f0 0x7f3eec068db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f1414aff0 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.391+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3f040ebfa0 con 0x7f3f1406a490 2024-01-15T18:05:53.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.595+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f3f1406ada0 con 0x7f3f1406a490 2024-01-15T18:05:53.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.599+0000 7f3f11600640 1 -- 172.21.15.143:0/2218542560 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3f040ef5c0 con 0x7f3f1406a490 2024-01-15T18:05:53.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eec0668f0 msgr2=0x7f3eec068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eec0668f0 0x7f3eec068db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f1414aff0 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 msgr2=0x7f3f14147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 0x7f3f14147680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3f0401a170 tx=0x7f3f0401ad80 comp rx=0 tx=0).stop 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 shutdown_connections 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eec0668f0 0x7f3eec068db0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f1415d510 0x7f3f14146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3f1415d130 0x7f3f14146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 --2- 172.21.15.143:0/2218542560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3f1406a490 0x7f3f14147680 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 >> 172.21.15.143:0/2218542560 conn(0x7f3f140b1b90 msgr2=0x7f3f14152550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:53.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 shutdown_connections 2024-01-15T18:05:53.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.603+0000 7f3f1be00640 1 -- 172.21.15.143:0/2218542560 wait complete. 2024-01-15T18:05:53.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-15T18:05:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.759+0000 7ff3b8c00640 1 Processor -- start 2024-01-15T18:05:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.759+0000 7ff3b8c00640 1 -- start start 2024-01-15T18:05:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b4141660 0x7ff3b40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b8c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3b4071d00 0x7ff3b4072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b8c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b415f310 con 0x7ff3b415ec80 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b8c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b415fbb0 con 0x7ff3b4071d00 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4072690 con 0x7ff3b4141660 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3b4071d00 0x7ff3b4072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff3b4071d00 0x7ff3b4072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36406/0 (socket says 172.21.15.143:36406) 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 -- 172.21.15.143:0/1023403914 learned_addr learned my addr 172.21.15.143:0/1023403914 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b1e00640 1 --2- 172.21.15.143:0/1023403914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b4141660 0x7ff3b40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 -- 172.21.15.143:0/1023403914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b4141660 msgr2=0x7ff3b40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 --2- 172.21.15.143:0/1023403914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b4141660 0x7ff3b40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 -- 172.21.15.143:0/1023403914 >> v1:172.21.15.204:6789/0 conn(0x7ff3b415ec80 legacy=0x7ff3b4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.763+0000 7ff3b2800640 1 -- 172.21.15.143:0/1023403914 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b415f5f0 con 0x7ff3b4071d00 2024-01-15T18:05:53.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b2800640 1 --2- 172.21.15.143:0/1023403914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b4072150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009e50 tx=0x7ff3ac01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80f360edd12cbdc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b0a00640 1 -- 172.21.15.143:0/1023403914 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0a57d0 con 0x7ff3b4071d00 2024-01-15T18:05:53.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b0a00640 1 -- 172.21.15.143:0/1023403914 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac0a5930 con 0x7ff3b4071d00 2024-01-15T18:05:53.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b0a00640 1 -- 172.21.15.143:0/1023403914 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0a5cc0 con 0x7ff3b4071d00 2024-01-15T18:05:53.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1023403914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 msgr2=0x7ff3b4072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1023403914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b4072150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009e50 tx=0x7ff3ac01ca10 comp rx=0 tx=0).stop 2024-01-15T18:05:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1023403914 shutdown_connections 2024-01-15T18:05:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1023403914 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b4072150 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1023403914 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b4141660 0x7ff3b40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1023403914 >> 172.21.15.143:0/1023403914 conn(0x7ff3b406c3f0 msgr2=0x7ff3b406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1023403914 shutdown_connections 2024-01-15T18:05:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1023403914 wait complete. 2024-01-15T18:05:53.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 Processor -- start 2024-01-15T18:05:53.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.767+0000 7ff3b8c00640 1 -- start start 2024-01-15T18:05:53.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b415ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4141660 0x7ff3b40a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b415ec80 0x7ff3b41598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b415bf40 con 0x7ff3b4071d00 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b415e330 con 0x7ff3b4141660 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b413c4a0 con 0x7ff3b415ec80 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b415ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b415ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36414/0 (socket says 172.21.15.143:36414) 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 -- 172.21.15.143:0/1374381253 learned_addr learned my addr 172.21.15.143:0/1374381253 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1400640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4141660 0x7ff3b40a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b2800640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b415ec80 0x7ff3b41598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 -- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b415ec80 msgr2=0x7ff3b41598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b415ec80 0x7ff3b41598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 -- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4141660 msgr2=0x7ff3b40a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4141660 0x7ff3b40a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 -- 172.21.15.143:0/1374381253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac009b30 con 0x7ff3b4071d00 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b1e00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b415ddf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400d5f0 tx=0x7ff3a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40125f0 con 0x7ff3b4071d00 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b40cd150 con 0x7ff3b4071d00 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b415bb60 con 0x7ff3b4071d00 2024-01-15T18:05:53.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4012750 con 0x7ff3b4071d00 2024-01-15T18:05:53.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4012ac0 con 0x7ff3b4071d00 2024-01-15T18:05:53.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b4067500 con 0x7ff3b4071d00 2024-01-15T18:05:53.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.771+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff3a4012c20 con 0x7ff3b4071d00 2024-01-15T18:05:53.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.775+0000 7ff3aa000640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff38c066920 0x7ff38c068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:53.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.775+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3a40844d0 con 0x7ff3b4071d00 2024-01-15T18:05:53.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.775+0000 7ff3b1400640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff38c066920 0x7ff38c068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:53.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.775+0000 7ff3b1400640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff38c066920 0x7ff38c068de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff3b413d3e0 tx=0x7ff39c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:53.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.779+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff3a4051b10 con 0x7ff3b4071d00 2024-01-15T18:05:53.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.979+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7ff3b41438c0 con 0x7ff3b4071d00 2024-01-15T18:05:53.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.983+0000 7ff3aa000640 1 -- 172.21.15.143:0/1374381253 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff3a4084030 con 0x7ff3b4071d00 2024-01-15T18:05:53.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff38c066920 msgr2=0x7ff38c068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff38c066920 0x7ff38c068de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff3b413d3e0 tx=0x7ff39c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 msgr2=0x7ff3b415ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b415ddf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400d5f0 tx=0x7ff3a400dab0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 shutdown_connections 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff38c066920 0x7ff38c068de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3b415ec80 0x7ff3b41598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3b4141660 0x7ff3b40a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 --2- 172.21.15.143:0/1374381253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3b4071d00 0x7ff3b415ddf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 >> 172.21.15.143:0/1374381253 conn(0x7ff3b406c3f0 msgr2=0x7ff3b4140790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 shutdown_connections 2024-01-15T18:05:53.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:53.987+0000 7ff3b8c00640 1 -- 172.21.15.143:0/1374381253 wait complete. 2024-01-15T18:05:54.011 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-15T18:05:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 Processor -- start 2024-01-15T18:05:54.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 -- start start 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd8cc143780 0x7fd8cc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 0x7fd8cc15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8cc0ceb50 con 0x7fd8cc0ce520 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8cc15c110 con 0x7fd8cc143780 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8d1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15b7e0 con 0x7fd8cc0722c0 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8ca000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd8cc0ce520 0x7fd8cc071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33456/0 (socket says 172.21.15.143:33456) 2024-01-15T18:05:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8ca000640 1 -- 172.21.15.143:0/3202034921 learned_addr learned my addr 172.21.15.143:0/3202034921 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3202034921 >> v2:172.21.15.204:3300/0 conn(0x7fd8cc143780 0x7fd8cc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8cb400640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 0x7fd8cc15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565918497 0 0) 0x7fd8b4001120 con 0x7fd8cc0ce520 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8ac003530 con 0x7fd8cc0ce520 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996814235 0 0) 0x7fd8ac003530 con 0x7fd8cc0ce520 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 msgr2=0x7fd8cc15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 0x7fd8cc15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 >> v2:172.21.15.204:3300/0 conn(0x7fd8cc143780 msgr2=0x7fd8cc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 --2- 172.21.15.143:0/3202034921 >> v2:172.21.15.204:3300/0 conn(0x7fd8cc143780 0x7fd8cc0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc15bac0 con 0x7fd8cc0ce520 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fd8b4001810 con 0x7fd8cc0ce520 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3202034921 >> v2:172.21.15.204:3300/0 conn(0x7fd8cc143780 0x7fd8cc0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 >> v1:172.21.15.143:6789/0 conn(0x7fd8cc0ce520 legacy=0x7fd8cc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8ac0047d0 0x7fd8ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 0x7fd8ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15c110 con 0x7fd8ac003980 2024-01-15T18:05:54.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc15b7e0 con 0x7fd8ac0047d0 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac00b990 con 0x7fd8cc0722c0 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8ca000640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8ac0047d0 0x7fd8ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8cb400640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 0x7fd8ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 -- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 msgr2=0x7fd8ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc0722c0 0x7fd8ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 -- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8ac0047d0 msgr2=0x7fd8ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8ac0047d0 0x7fd8ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8caa00640 1 -- 172.21.15.143:0/3202034921 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac003530 con 0x7fd8ac003980 2024-01-15T18:05:54.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.147+0000 7fd8ca000640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8ac0047d0 0x7fd8ac004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:54.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8ac003da0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009cb0 tx=0x7fd8bc01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b526462472f7bb0e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0a5740 con 0x7fd8ac003980 2024-01-15T18:05:54.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0a58a0 con 0x7fd8ac003980 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8c9600640 1 -- 172.21.15.143:0/3202034921 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0a5c10 con 0x7fd8ac003980 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3202034921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 msgr2=0x7fd8ac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3202034921 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8ac003da0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009cb0 tx=0x7fd8bc01cc70 comp rx=0 tx=0).stop 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8cb400640 1 -- 172.21.15.143:0/3202034921 reap_dead start 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3202034921 shutdown_connections 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3202034921 >> 172.21.15.143:0/3202034921 conn(0x7fd8cc06c3f0 msgr2=0x7fd8cc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3202034921 shutdown_connections 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.151+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3202034921 wait complete. 2024-01-15T18:05:54.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 Processor -- start 2024-01-15T18:05:54.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 -- start start 2024-01-15T18:05:54.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8cc137020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc135850 0x7fd8cc071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8cc0717a0 0x7fd8cc137d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc071c20 con 0x7fd8ac003980 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc136010 con 0x7fd8cc0717a0 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc136260 con 0x7fd8cc135850 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8cc137020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8cc137020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36428/0 (socket says 172.21.15.143:36428) 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 -- 172.21.15.143:0/3361363099 learned_addr learned my addr 172.21.15.143:0/3361363099 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8cb400640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8cc0717a0 0x7fd8cc137d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8ca000640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc135850 0x7fd8cc071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 -- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc135850 msgr2=0x7fd8cc071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc135850 0x7fd8cc071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 -- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8cc0717a0 msgr2=0x7fd8cc137d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8cc0717a0 0x7fd8cc137d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 -- 172.21.15.143:0/3361363099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc136540 con 0x7fd8ac003980 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8caa00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8cc137020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc01caa0 tx=0x7fd8ac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc004020 con 0x7fd8ac003980 2024-01-15T18:05:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0041a0 con 0x7fd8ac003980 2024-01-15T18:05:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.155+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8cc136820 con 0x7fd8ac003980 2024-01-15T18:05:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc003620 con 0x7fd8ac003980 2024-01-15T18:05:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8cc135cb0 con 0x7fd8ac003980 2024-01-15T18:05:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8cc067500 con 0x7fd8ac003980 2024-01-15T18:05:54.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fd8bc0b4040 con 0x7fd8ac003980 2024-01-15T18:05:54.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8c3400640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8a40668f0 0x7fd8a4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8bc11d830 con 0x7fd8ac003980 2024-01-15T18:05:54.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8ca000640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8a40668f0 0x7fd8a4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.159+0000 7fd8ca000640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8a40668f0 0x7fd8a4068db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40019c0 tx=0x7fd8b4007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.163+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd8bc0eae70 con 0x7fd8ac003980 2024-01-15T18:05:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.367+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd8cc0b0150 con 0x7fd8ac003980 2024-01-15T18:05:54.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8c3400640 1 -- 172.21.15.143:0/3361363099 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd8bc0be0a0 con 0x7fd8ac003980 2024-01-15T18:05:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8a40668f0 msgr2=0x7fd8a4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8a40668f0 0x7fd8a4068db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40019c0 tx=0x7fd8b4007040 comp rx=0 tx=0).stop 2024-01-15T18:05:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 msgr2=0x7fd8cc137020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8cc137020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc01caa0 tx=0x7fd8ac005800 comp rx=0 tx=0).stop 2024-01-15T18:05:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 shutdown_connections 2024-01-15T18:05:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8a40668f0 0x7fd8a4068db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8cc0717a0 0x7fd8cc137d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8cc135850 0x7fd8cc071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 --2- 172.21.15.143:0/3361363099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8ac003980 0x7fd8cc137020 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 >> 172.21.15.143:0/3361363099 conn(0x7fd8cc06c3f0 msgr2=0x7fd8cc13ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 shutdown_connections 2024-01-15T18:05:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.371+0000 7fd8d1e00640 1 -- 172.21.15.143:0/3361363099 wait complete. 2024-01-15T18:05:54.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-15T18:05:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.527+0000 7ff10ee00640 1 Processor -- start 2024-01-15T18:05:54.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.527+0000 7ff10ee00640 1 -- start start 2024-01-15T18:05:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b24b0 0x7ff1080ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ee00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff1080b3bf0 0x7ff1080b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ee00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff108070bf0 con 0x7ff1080ce690 2024-01-15T18:05:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ee00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10815dba0 con 0x7ff1080b3bf0 2024-01-15T18:05:54.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080b4920 con 0x7ff1080b24b0 2024-01-15T18:05:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b24b0 0x7ff1080ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff1080b3bf0 0x7ff1080b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ff1080ce690 0x7ff1080b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33816/0 (socket says 172.21.15.143:33816) 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107e00640 1 -- 172.21.15.143:0/2633459157 learned_addr learned my addr 172.21.15.143:0/2633459157 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107400640 1 -- 172.21.15.143:0/2633459157 >> v2:172.21.15.204:3300/0 conn(0x7ff1080b3bf0 msgr2=0x7ff1080b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107400640 1 --2- 172.21.15.143:0/2633459157 >> v2:172.21.15.204:3300/0 conn(0x7ff1080b3bf0 0x7ff1080b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107400640 1 -- 172.21.15.143:0/2633459157 >> v1:172.21.15.204:6789/0 conn(0x7ff1080ce690 legacy=0x7ff1080b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff107400640 1 -- 172.21.15.143:0/2633459157 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080b4c00 con 0x7ff1080b24b0 2024-01-15T18:05:54.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.531+0000 7ff10ce00640 1 --2- 172.21.15.143:0/2633459157 >> v2:172.21.15.204:3300/0 conn(0x7ff1080b3bf0 0x7ff1080b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:54.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff107400640 1 --2- 172.21.15.143:0/2633459157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b24b0 0x7ff1080ac8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00097c0 tx=0x7ff0f001ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3d206da5472ea9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff106a00640 1 -- 172.21.15.143:0/2633459157 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00ab670 con 0x7ff1080b24b0 2024-01-15T18:05:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff106a00640 1 -- 172.21.15.143:0/2633459157 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f00ab7d0 con 0x7ff1080b24b0 2024-01-15T18:05:54.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff106a00640 1 -- 172.21.15.143:0/2633459157 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f00abb60 con 0x7ff1080b24b0 2024-01-15T18:05:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 -- 172.21.15.143:0/2633459157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b24b0 msgr2=0x7ff1080ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 --2- 172.21.15.143:0/2633459157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b24b0 0x7ff1080ac8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00097c0 tx=0x7ff0f001ceb0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 -- 172.21.15.143:0/2633459157 shutdown_connections 2024-01-15T18:05:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 --2- 172.21.15.143:0/2633459157 >> v2:172.21.15.204:3300/0 conn(0x7ff1080b3bf0 0x7ff1080b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 --2- 172.21.15.143:0/2633459157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b24b0 0x7ff1080ac8e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 -- 172.21.15.143:0/2633459157 >> 172.21.15.143:0/2633459157 conn(0x7ff10806c3f0 msgr2=0x7ff10806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 -- 172.21.15.143:0/2633459157 shutdown_connections 2024-01-15T18:05:54.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 -- 172.21.15.143:0/2633459157 wait complete. 2024-01-15T18:05:54.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.535+0000 7ff10ee00640 1 Processor -- start 2024-01-15T18:05:54.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- start start 2024-01-15T18:05:54.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 0x7ff108145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b3bf0 0x7ff10815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1080ce690 0x7ff1080af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10813f1d0 con 0x7ff1080b24b0 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1081464a0 con 0x7ff1080ce690 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10815a880 con 0x7ff1080b3bf0 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff107e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 0x7ff108145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1080ce690 0x7ff1080af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff107e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 0x7ff108145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36436/0 (socket says 172.21.15.143:36436) 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff107e00640 1 -- 172.21.15.143:0/393632373 learned_addr learned my addr 172.21.15.143:0/393632373 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff107400640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b3bf0 0x7ff10815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 -- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b3bf0 msgr2=0x7ff10815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b3bf0 0x7ff10815a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 -- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 msgr2=0x7ff108145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 0x7ff108145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 -- 172.21.15.143:0/393632373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100009a70 con 0x7ff1080ce690 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff107e00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 0x7ff108145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ce00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1080ce690 0x7ff1080af790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff100009d90 tx=0x7ff10000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10000cd30 con 0x7ff1080ce690 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10000f040 con 0x7ff1080ce690 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f0009420 con 0x7ff1080ce690 2024-01-15T18:05:54.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100003be0 con 0x7ff1080ce690 2024-01-15T18:05:54.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1080b08a0 con 0x7ff1080ce690 2024-01-15T18:05:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.539+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff108067500 con 0x7ff1080ce690 2024-01-15T18:05:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.543+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff100014480 con 0x7ff1080ce690 2024-01-15T18:05:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.543+0000 7ff0ffe00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0e0066920 0x7ff0e0068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.543+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff100084b30 con 0x7ff1080ce690 2024-01-15T18:05:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.543+0000 7ff107e00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0e0066920 0x7ff0e0068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.543+0000 7ff107e00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0e0066920 0x7ff0e0068de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8007fd0 tx=0x7ff0f8007f20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.547+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff100052040 con 0x7ff1080ce690 2024-01-15T18:05:54.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.751+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7ff10806e260 con 0x7ff1080ce690 2024-01-15T18:05:54.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff0ffe00640 1 -- 172.21.15.143:0/393632373 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff1000180a0 con 0x7ff1080ce690 2024-01-15T18:05:54.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0e0066920 msgr2=0x7ff0e0068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0e0066920 0x7ff0e0068de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8007fd0 tx=0x7ff0f8007f20 comp rx=0 tx=0).stop 2024-01-15T18:05:54.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1080ce690 msgr2=0x7ff1080af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1080ce690 0x7ff1080af790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff100009d90 tx=0x7ff10000c490 comp rx=0 tx=0).stop 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 shutdown_connections 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0e0066920 0x7ff0e0068de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1080ce690 0x7ff1080af790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1080b3bf0 0x7ff10815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 --2- 172.21.15.143:0/393632373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1080b24b0 0x7ff108145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 >> 172.21.15.143:0/393632373 conn(0x7ff10806c3f0 msgr2=0x7ff108140430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 shutdown_connections 2024-01-15T18:05:54.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.759+0000 7ff10ee00640 1 -- 172.21.15.143:0/393632373 wait complete. 2024-01-15T18:05:54.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:54.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3287e00640 1 Processor -- start 2024-01-15T18:05:54.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3287e00640 1 -- start start 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3287e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3280072030 0x7f328015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3287e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32800ceb50 con 0x7f3280071bb0 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3287e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328015c110 con 0x7f3280143780 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3287e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328015b7e0 con 0x7f3280072030 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3280071bb0 0x7f328015c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33466/0 (socket says 172.21.15.143:33466) 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 -- 172.21.15.143:0/279405671 learned_addr learned my addr 172.21.15.143:0/279405671 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3285e00640 1 --2- 172.21.15.143:0/279405671 >> v2:172.21.15.204:3300/0 conn(0x7f3280072030 0x7f328015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 730979923 0 0) 0x7f3268001120 con 0x7f3280071bb0 2024-01-15T18:05:54.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3260003530 con 0x7f3280071bb0 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970527949 0 0) 0x7f328015c110 con 0x7f3280143780 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3268001120 con 0x7f3280143780 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2606431093 0 0) 0x7f3260003530 con 0x7f3280071bb0 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 >> v2:172.21.15.204:3300/0 conn(0x7f3280072030 msgr2=0x7f328015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 --2- 172.21.15.143:0/279405671 >> v2:172.21.15.204:3300/0 conn(0x7f3280072030 0x7f328015ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 >> v1:172.21.15.204:6789/0 conn(0x7f3280143780 legacy=0x7f32800cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328015bac0 con 0x7f3280071bb0 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3268001810 con 0x7f3280071bb0 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 >> v1:172.21.15.143:6789/0 conn(0x7f3280071bb0 legacy=0x7f328015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3260003b10 0x7f3260003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32600047d0 0x7f3260004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3260005190 0x7f326000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328015b7e0 con 0x7f32600047d0 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326000c010 con 0x7f3260005190 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326000c1d0 con 0x7f3260003b10 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32600047d0 0x7f3260004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3285400640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3260003b10 0x7f3260003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3285e00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3260005190 0x7f326000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 -- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3260003b10 msgr2=0x7f3260003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3260003b10 0x7f3260003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 -- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3260005190 msgr2=0x7f326000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:54.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3260005190 0x7f326000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3285e00640 1 -- 172.21.15.143:0/279405671 reap_dead start 2024-01-15T18:05:54.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.919+0000 7f3284a00640 1 -- 172.21.15.143:0/279405671 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260003530 con 0x7f32600047d0 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f3284a00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32600047d0 0x7f3260004c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f326800c160 tx=0x7f326801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cee9d6e77896cc5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32680058e0 con 0x7f32600047d0 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3268005a60 con 0x7f32600047d0 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f327fe00640 1 -- 172.21.15.143:0/279405671 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3268005e50 con 0x7f32600047d0 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f3287e00640 1 -- 172.21.15.143:0/279405671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32600047d0 msgr2=0x7f3260004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f3287e00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32600047d0 0x7f3260004c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f326800c160 tx=0x7f326801cb60 comp rx=0 tx=0).stop 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f3287e00640 1 -- 172.21.15.143:0/279405671 shutdown_connections 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f3287e00640 1 --2- 172.21.15.143:0/279405671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32600047d0 0x7f3260004c50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.923+0000 7f3287e00640 1 -- 172.21.15.143:0/279405671 >> 172.21.15.143:0/279405671 conn(0x7f328006c3f0 msgr2=0x7f328006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:54.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 -- 172.21.15.143:0/279405671 shutdown_connections 2024-01-15T18:05:54.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 -- 172.21.15.143:0/279405671 wait complete. 2024-01-15T18:05:54.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 Processor -- start 2024-01-15T18:05:54.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 -- start start 2024-01-15T18:05:54.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32600047d0 0x7f32801595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3280141990 0x7f3280158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 0x7f3280158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32800ae2b0 con 0x7f3280141df0 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3280159af0 con 0x7f3280141990 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3287e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328013bff0 con 0x7f32600047d0 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 0x7f3280158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 0x7f3280158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36468/0 (socket says 172.21.15.143:36468) 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 -- 172.21.15.143:0/1080229875 learned_addr learned my addr 172.21.15.143:0/1080229875 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3284a00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3280141990 0x7f3280158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285400640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32600047d0 0x7f32801595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 -- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32600047d0 msgr2=0x7f32801595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32600047d0 0x7f32801595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 -- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3280141990 msgr2=0x7f3280158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3280141990 0x7f3280158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 -- 172.21.15.143:0/1080229875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32801f2a80 con 0x7f3280141df0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f3285e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 0x7f3280158b70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3260004750 tx=0x7f3278011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.927+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800e040 con 0x7f3280141df0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327800f040 con 0x7f3280141df0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327800fe60 con 0x7f3280141df0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32800adfd0 con 0x7f3280141df0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f328013bc10 con 0x7f3280141df0 2024-01-15T18:05:54.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328013b790 con 0x7f3280141df0 2024-01-15T18:05:54.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f3278022460 con 0x7f3280141df0 2024-01-15T18:05:54.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f327d600640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f32580668f0 0x7f3258068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:54.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f32780847a0 con 0x7f3280141df0 2024-01-15T18:05:54.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f3285400640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f32580668f0 0x7f3258068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:54.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.931+0000 7f3285400640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f32580668f0 0x7f3258068db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3270005630 tx=0x7f3270009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:54.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:54.935+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f328013b790 con 0x7f3280141df0 2024-01-15T18:05:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.087+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f328013c130 con 0x7f3280141df0 2024-01-15T18:05:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.087+0000 7f327d600640 1 -- 172.21.15.143:0/1080229875 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12893 (secure 0 0 0) 0x7f328013c130 con 0x7f3280141df0 2024-01-15T18:05:55.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:55.094 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:52.046636+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f32580668f0 msgr2=0x7f3258068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f32580668f0 0x7f3258068db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3270005630 tx=0x7f3270009040 comp rx=0 tx=0).stop 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 msgr2=0x7f3280158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 0x7f3280158b70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3260004750 tx=0x7f3278011f10 comp rx=0 tx=0).stop 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 shutdown_connections 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f32580668f0 0x7f3258068db0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3280141df0 0x7f3280158b70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3280141990 0x7f3280158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 --2- 172.21.15.143:0/1080229875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32600047d0 0x7f32801595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 >> 172.21.15.143:0/1080229875 conn(0x7f328006c3f0 msgr2=0x7f32800afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:55.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 shutdown_connections 2024-01-15T18:05:55.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.091+0000 7f3287e00640 1 -- 172.21.15.143:0/1080229875 wait complete. 2024-01-15T18:05:55.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:05:55.116 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:05:55.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 Processor -- start 2024-01-15T18:05:55.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 -- start start 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb0280cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0280722c0 0x7fb02815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0280ceb50 con 0x7fb0280ce520 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02815c110 con 0x7fb0280722c0 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02fa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02815b7e0 con 0x7fb028143780 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0280722c0 0x7fb02815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb027e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fb0280ce520 0x7fb028071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:33838/0 (socket says 172.21.15.143:33838) 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb0280cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb027e00640 1 -- 172.21.15.143:0/3233399193 learned_addr learned my addr 172.21.15.143:0/3233399193 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:55.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb027400640 1 -- 172.21.15.143:0/3233399193 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136442419 0 0) 0x7fb010001120 con 0x7fb0280ce520 2024-01-15T18:05:55.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb027400640 1 -- 172.21.15.143:0/3233399193 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb008003530 con 0x7fb0280ce520 2024-01-15T18:05:55.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02da00640 1 -- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 msgr2=0x7fb0280cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:55.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02da00640 1 --2- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb0280cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02da00640 1 -- 172.21.15.143:0/3233399193 >> v1:172.21.15.204:6789/0 conn(0x7fb0280ce520 legacy=0x7fb028071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02da00640 1 -- 172.21.15.143:0/3233399193 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02815bac0 con 0x7fb0280722c0 2024-01-15T18:05:55.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.251+0000 7fb02d000640 1 --2- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb0280cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:55.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb02da00640 1 --2- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280722c0 0x7fb02815c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb020009d90 tx=0x7fb02001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d0571ab8cd791f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb027400640 1 -- 172.21.15.143:0/3233399193 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200a5920 con 0x7fb0280722c0 2024-01-15T18:05:55.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb027400640 1 -- 172.21.15.143:0/3233399193 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0200a5a80 con 0x7fb0280722c0 2024-01-15T18:05:55.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb027400640 1 -- 172.21.15.143:0/3233399193 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200a5e10 con 0x7fb0280722c0 2024-01-15T18:05:55.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb02fa00640 1 -- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280722c0 msgr2=0x7fb02815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb02fa00640 1 --2- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280722c0 0x7fb02815c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb020009d90 tx=0x7fb02001c950 comp rx=0 tx=0).stop 2024-01-15T18:05:55.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb02fa00640 1 -- 172.21.15.143:0/3233399193 shutdown_connections 2024-01-15T18:05:55.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.255+0000 7fb02fa00640 1 --2- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280722c0 0x7fb02815c5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 --2- 172.21.15.143:0/3233399193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb0280cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- 172.21.15.143:0/3233399193 >> 172.21.15.143:0/3233399193 conn(0x7fb02806c3f0 msgr2=0x7fb02806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:55.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- 172.21.15.143:0/3233399193 shutdown_connections 2024-01-15T18:05:55.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- 172.21.15.143:0/3233399193 wait complete. 2024-01-15T18:05:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 Processor -- start 2024-01-15T18:05:55.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- start start 2024-01-15T18:05:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 0x7fb0281599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280ce520 0x7fb0280a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb028070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb028159f20 con 0x7fb0280722c0 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280a9980 con 0x7fb0280ce520 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02fa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0280a9c60 con 0x7fb028143780 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 0x7fb0281599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 0x7fb0281599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36484/0 (socket says 172.21.15.143:36484) 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 -- 172.21.15.143:0/4272427603 learned_addr learned my addr 172.21.15.143:0/4272427603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb027e00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280ce520 0x7fb0280a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02da00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb028070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 -- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 msgr2=0x7fb028070aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb028070aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 -- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280ce520 msgr2=0x7fb0280a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280ce520 0x7fb0280a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb02d000640 1 -- 172.21.15.143:0/4272427603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb018009420 con 0x7fb0280722c0 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.259+0000 7fb027e00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280ce520 0x7fb0280a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb02d000640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 0x7fb0281599e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e980 tx=0x7fb01800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01800ca00 con 0x7fb0280722c0 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb01800cb60 con 0x7fb0280722c0 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018013470 con 0x7fb0280722c0 2024-01-15T18:05:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb020009a70 con 0x7fb0280722c0 2024-01-15T18:05:55.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0280aa220 con 0x7fb0280722c0 2024-01-15T18:05:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb028067500 con 0x7fb0280722c0 2024-01-15T18:05:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb0180135d0 con 0x7fb0280722c0 2024-01-15T18:05:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb024c00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb000066920 0x7fb000068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb018084380 con 0x7fb0280722c0 2024-01-15T18:05:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.263+0000 7fb027e00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb000066920 0x7fb000068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.267+0000 7fb027e00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb000066920 0x7fb000068de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb02813a640 tx=0x7fb010009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.267+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb018051870 con 0x7fb0280722c0 2024-01-15T18:05:55.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.423+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0280aef40 con 0x7fb0280722c0 2024-01-15T18:05:55.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.423+0000 7fb024c00640 1 -- 172.21.15.143:0/4272427603 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb018054e90 con 0x7fb0280722c0 2024-01-15T18:05:55.429 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:05:55.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb000066920 msgr2=0x7fb000068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb000066920 0x7fb000068de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb02813a640 tx=0x7fb010009040 comp rx=0 tx=0).stop 2024-01-15T18:05:55.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 msgr2=0x7fb0281599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 0x7fb0281599e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb01800e980 tx=0x7fb01800ee40 comp rx=0 tx=0).stop 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 shutdown_connections 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb000066920 0x7fb000068de0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb028143780 0x7fb028070aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0280ce520 0x7fb0280a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 --2- 172.21.15.143:0/4272427603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0280722c0 0x7fb0281599e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 >> 172.21.15.143:0/4272427603 conn(0x7fb02806c3f0 msgr2=0x7fb0280aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 shutdown_connections 2024-01-15T18:05:55.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.427+0000 7fb02fa00640 1 -- 172.21.15.143:0/4272427603 wait complete. 2024-01-15T18:05:55.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:05:55.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9367c00640 1 Processor -- start 2024-01-15T18:05:55.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9367c00640 1 -- start start 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9367c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93600b3610 0x7f93600cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9367c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93600cefa0 con 0x7f93600b4080 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9367c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93600d0f90 con 0x7f93600b45d0 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9367c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600c9fa0 con 0x7f93600b3610 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93600b3610 0x7f93600cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93600b3610 0x7f93600cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36492/0 (socket says 172.21.15.143:36492) 2024-01-15T18:05:55.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365200640 1 -- 172.21.15.143:0/1277637726 learned_addr learned my addr 172.21.15.143:0/1277637726 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:55.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f93600b4080 0x7f93600ce4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33470/0 (socket says 172.21.15.143:33470) 2024-01-15T18:05:55.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365200640 1 -- 172.21.15.143:0/1277637726 >> v1:172.21.15.204:6789/0 conn(0x7f93600b45d0 legacy=0x7f93600cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:55.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365200640 1 -- 172.21.15.143:0/1277637726 >> v1:172.21.15.143:6789/0 conn(0x7f93600b4080 legacy=0x7f93600ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.583+0000 7f9365200640 1 -- 172.21.15.143:0/1277637726 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600c7b90 con 0x7f93600b3610 2024-01-15T18:05:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9365200640 1 --2- 172.21.15.143:0/1277637726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b3610 0x7f93600cfa40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9350009870 tx=0x7f935001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25a709f55c9f756 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f935f400640 1 -- 172.21.15.143:0/1277637726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500a8a30 con 0x7f93600b3610 2024-01-15T18:05:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f935f400640 1 -- 172.21.15.143:0/1277637726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500a8b90 con 0x7f93600b3610 2024-01-15T18:05:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f935f400640 1 -- 172.21.15.143:0/1277637726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500b0440 con 0x7f93600b3610 2024-01-15T18:05:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 -- 172.21.15.143:0/1277637726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b3610 msgr2=0x7f93600cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 --2- 172.21.15.143:0/1277637726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b3610 0x7f93600cfa40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9350009870 tx=0x7f935001ed30 comp rx=0 tx=0).stop 2024-01-15T18:05:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 -- 172.21.15.143:0/1277637726 shutdown_connections 2024-01-15T18:05:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 --2- 172.21.15.143:0/1277637726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b3610 0x7f93600cfa40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 -- 172.21.15.143:0/1277637726 >> 172.21.15.143:0/1277637726 conn(0x7f93600d3030 msgr2=0x7f93600d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 -- 172.21.15.143:0/1277637726 shutdown_connections 2024-01-15T18:05:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.587+0000 7f9367c00640 1 -- 172.21.15.143:0/1277637726 wait complete. 2024-01-15T18:05:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 Processor -- start 2024-01-15T18:05:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 -- start start 2024-01-15T18:05:55.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93600b3610 0x7f936015c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 0x7f93600cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93600b45d0 0x7f93600b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b3ef0 con 0x7f93600b4080 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b1610 con 0x7f93600b3610 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b1b90 con 0x7f93600b45d0 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 0x7f93600cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 0x7f93600cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36498/0 (socket says 172.21.15.143:36498) 2024-01-15T18:05:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 -- 172.21.15.143:0/196599846 learned_addr learned my addr 172.21.15.143:0/196599846 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9365c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93600b45d0 0x7f93600b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9365200640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93600b3610 0x7f936015c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 -- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93600b45d0 msgr2=0x7f93600b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93600b45d0 0x7f93600b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 -- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93600b3610 msgr2=0x7f936015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93600b3610 0x7f936015c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 -- 172.21.15.143:0/196599846 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500094d0 con 0x7f93600b4080 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9365c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93600b45d0 0x7f93600b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935fe00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 0x7f93600cab10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f934800c3c0 tx=0x7f934800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93480052d0 con 0x7f93600b4080 2024-01-15T18:05:55.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348005430 con 0x7f93600b4080 2024-01-15T18:05:55.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93600ced70 con 0x7f93600b4080 2024-01-15T18:05:55.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348003820 con 0x7f93600b4080 2024-01-15T18:05:55.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.591+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93600cb050 con 0x7f93600b4080 2024-01-15T18:05:55.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.595+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9360067500 con 0x7f93600b4080 2024-01-15T18:05:55.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.595+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f9348003980 con 0x7f93600b4080 2024-01-15T18:05:55.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.595+0000 7f935cc00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93380668f0 0x7f9338068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.595+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9348083520 con 0x7f93600b4080 2024-01-15T18:05:55.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.595+0000 7f9365200640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93380668f0 0x7f9338068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.595+0000 7f9365200640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93380668f0 0x7f9338068db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93500b2000 tx=0x7f9350009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.599+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9348050b60 con 0x7f93600b4080 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f93600b8eb0 con 0x7f93600b4080 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f935cc00640 1 -- 172.21.15.143:0/196599846 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9348017060 con 0x7f93600b4080 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:05:55.755 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:05:55.756 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:05:55.756 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:05:55.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93380668f0 msgr2=0x7f9338068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93380668f0 0x7f9338068db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f93500b2000 tx=0x7f9350009040 comp rx=0 tx=0).stop 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 msgr2=0x7f93600cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 0x7f93600cab10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f934800c3c0 tx=0x7f934800c880 comp rx=0 tx=0).stop 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 shutdown_connections 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f93380668f0 0x7f9338068db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93600b45d0 0x7f93600b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93600b4080 0x7f93600cab10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 --2- 172.21.15.143:0/196599846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93600b3610 0x7f936015c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 >> 172.21.15.143:0/196599846 conn(0x7f93600d3030 msgr2=0x7f93600c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 shutdown_connections 2024-01-15T18:05:55.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.751+0000 7f9367c00640 1 -- 172.21.15.143:0/196599846 wait complete. 2024-01-15T18:05:55.776 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:05:55.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:05:55.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.911+0000 7fb75ca00640 1 Processor -- start 2024-01-15T18:05:55.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.911+0000 7fb75ca00640 1 -- start start 2024-01-15T18:05:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb754143780 0x7fb7540cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb7540722c0 0x7fb75415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75ca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7540ceb50 con 0x7fb7540ce520 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75ca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75415c110 con 0x7fb754143780 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75ca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75415b7e0 con 0x7fb7540722c0 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb754143780 0x7fb7540cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb759600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb7540ce520 0x7fb754071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33476/0 (socket says 172.21.15.143:33476) 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb759600640 1 -- 172.21.15.143:0/735035037 learned_addr learned my addr 172.21.15.143:0/735035037 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75aa00640 1 --2- 172.21.15.143:0/735035037 >> v2:172.21.15.204:3300/0 conn(0x7fb7540722c0 0x7fb75415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75a000640 1 -- 172.21.15.143:0/735035037 >> v2:172.21.15.204:3300/0 conn(0x7fb7540722c0 msgr2=0x7fb75415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75a000640 1 --2- 172.21.15.143:0/735035037 >> v2:172.21.15.204:3300/0 conn(0x7fb7540722c0 0x7fb75415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75a000640 1 -- 172.21.15.143:0/735035037 >> v1:172.21.15.143:6789/0 conn(0x7fb7540ce520 legacy=0x7fb754071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75a000640 1 -- 172.21.15.143:0/735035037 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75415bac0 con 0x7fb754143780 2024-01-15T18:05:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb758c00640 1 -- 172.21.15.143:0/735035037 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700671956 0 0) 0x7fb73c001120 con 0x7fb7540ce520 2024-01-15T18:05:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.915+0000 7fb75a000640 1 --2- 172.21.15.143:0/735035037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb754143780 0x7fb7540cdfe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb754070a20 tx=0x7fb74401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49aaa75b1ef94377 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb758c00640 1 -- 172.21.15.143:0/735035037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440ab600 con 0x7fb754143780 2024-01-15T18:05:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb758c00640 1 -- 172.21.15.143:0/735035037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7440a7050 con 0x7fb754143780 2024-01-15T18:05:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb758c00640 1 -- 172.21.15.143:0/735035037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440ab980 con 0x7fb754143780 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- 172.21.15.143:0/735035037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb754143780 msgr2=0x7fb7540cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 --2- 172.21.15.143:0/735035037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb754143780 0x7fb7540cdfe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb754070a20 tx=0x7fb74401c460 comp rx=0 tx=0).stop 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- 172.21.15.143:0/735035037 shutdown_connections 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 --2- 172.21.15.143:0/735035037 >> v2:172.21.15.204:3300/0 conn(0x7fb7540722c0 0x7fb75415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 --2- 172.21.15.143:0/735035037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb754143780 0x7fb7540cdfe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- 172.21.15.143:0/735035037 >> 172.21.15.143:0/735035037 conn(0x7fb75406c3f0 msgr2=0x7fb75406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- 172.21.15.143:0/735035037 shutdown_connections 2024-01-15T18:05:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- 172.21.15.143:0/735035037 wait complete. 2024-01-15T18:05:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 Processor -- start 2024-01-15T18:05:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- start start 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 0x7fb754157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7540ce520 0x7fb75413fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb754143780 0x7fb7541378a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540ad4d0 con 0x7fb7540722c0 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.919+0000 7fb75ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb754158330 con 0x7fb7540ce520 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540ac8e0 con 0x7fb754143780 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 0x7fb754157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 0x7fb754157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36522/0 (socket says 172.21.15.143:36522) 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 -- 172.21.15.143:0/957375113 learned_addr learned my addr 172.21.15.143:0/957375113 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb759600640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7540ce520 0x7fb75413fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75aa00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb754143780 0x7fb7541378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 -- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb754143780 msgr2=0x7fb7541378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb754143780 0x7fb7541378a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 -- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7540ce520 msgr2=0x7fb75413fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7540ce520 0x7fb75413fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 -- 172.21.15.143:0/957375113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744009420 con 0x7fb7540722c0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75aa00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb754143780 0x7fb7541378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb759600640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7540ce520 0x7fb75413fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75a000640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 0x7fb754157df0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb74401a090 tx=0x7fb74401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440b7650 con 0x7fb7540722c0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7440b77d0 con 0x7fb7540722c0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440b7b80 con 0x7fb7540722c0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7540acb30 con 0x7fb7540722c0 2024-01-15T18:05:55.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7540ad0f0 con 0x7fb7540722c0 2024-01-15T18:05:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754067500 con 0x7fb7540722c0 2024-01-15T18:05:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.923+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb7440b7ce0 con 0x7fb7540722c0 2024-01-15T18:05:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.927+0000 7fb752000640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb72c0668f0 0x7fb72c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.927+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb74411e8c0 con 0x7fb7540722c0 2024-01-15T18:05:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.927+0000 7fb759600640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb72c0668f0 0x7fb72c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.927+0000 7fb759600640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb72c0668f0 0x7fb72c068db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb73c007fd0 tx=0x7fb73c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:55.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:55.931+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb7440ebe40 con 0x7fb7540722c0 2024-01-15T18:05:56.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:56.079+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb754071d50 con 0x7fb7540722c0 2024-01-15T18:05:57.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.059+0000 7fb752000640 1 -- 172.21.15.143:0/957375113 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb7440bb0b0 con 0x7fb7540722c0 2024-01-15T18:05:57.063 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:05:57.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.059+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb72c0668f0 msgr2=0x7fb72c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:57.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.059+0000 7fb75ca00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb72c0668f0 0x7fb72c068db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb73c007fd0 tx=0x7fb73c009040 comp rx=0 tx=0).stop 2024-01-15T18:05:57.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.059+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 msgr2=0x7fb754157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.059+0000 7fb75ca00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 0x7fb754157df0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb74401a090 tx=0x7fb74401aca0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 shutdown_connections 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb72c0668f0 0x7fb72c068db0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb754143780 0x7fb7541378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7540ce520 0x7fb75413fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 --2- 172.21.15.143:0/957375113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7540722c0 0x7fb754157df0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 >> 172.21.15.143:0/957375113 conn(0x7fb75406c3f0 msgr2=0x7fb754140ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 shutdown_connections 2024-01-15T18:05:57.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.063+0000 7fb75ca00640 1 -- 172.21.15.143:0/957375113 wait complete. 2024-01-15T18:05:57.094 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:05:57.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 Processor -- start 2024-01-15T18:05:57.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 -- start start 2024-01-15T18:05:57.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff734158a00 0x7ff7340ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff73415bf20 0x7ff7340cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734070b50 0x7ff7340cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73415f980 con 0x7ff734158a00 2024-01-15T18:05:57.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73415c3b0 con 0x7ff73415bf20 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.235+0000 7ff739c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73415ebc0 con 0x7ff734070b50 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff734158a00 0x7ff7340ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff734158a00 0x7ff7340ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57274/0 (socket says 172.21.15.143:57274) 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 -- 172.21.15.143:0/1802957648 learned_addr learned my addr 172.21.15.143:0/1802957648 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732000640 1 --2- 172.21.15.143:0/1802957648 >> v2:172.21.15.204:3300/0 conn(0x7ff73415bf20 0x7ff7340cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff733400640 1 --2- 172.21.15.143:0/1802957648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734070b50 0x7ff7340cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 -- 172.21.15.143:0/1802957648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734070b50 msgr2=0x7ff7340cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 --2- 172.21.15.143:0/1802957648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734070b50 0x7ff7340cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 -- 172.21.15.143:0/1802957648 >> v2:172.21.15.204:3300/0 conn(0x7ff73415bf20 msgr2=0x7ff7340cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 --2- 172.21.15.143:0/1802957648 >> v2:172.21.15.204:3300/0 conn(0x7ff73415bf20 0x7ff7340cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 -- 172.21.15.143:0/1802957648 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73415eea0 con 0x7ff734158a00 2024-01-15T18:05:57.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732000640 1 --2- 172.21.15.143:0/1802957648 >> v2:172.21.15.204:3300/0 conn(0x7ff73415bf20 0x7ff7340cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.239+0000 7ff732a00640 1 --2- 172.21.15.143:0/1802957648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff734158a00 0x7ff7340ce560 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff73415b4a0 tx=0x7ff72401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ac9e80f1032eaa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff731600640 1 -- 172.21.15.143:0/1802957648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240ab600 con 0x7ff734158a00 2024-01-15T18:05:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff731600640 1 -- 172.21.15.143:0/1802957648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7240ab760 con 0x7ff734158a00 2024-01-15T18:05:57.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff731600640 1 -- 172.21.15.143:0/1802957648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240abaf0 con 0x7ff734158a00 2024-01-15T18:05:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- 172.21.15.143:0/1802957648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff734158a00 msgr2=0x7ff7340ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- 172.21.15.143:0/1802957648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff734158a00 0x7ff7340ce560 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff73415b4a0 tx=0x7ff72401c440 comp rx=0 tx=0).stop 2024-01-15T18:05:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- 172.21.15.143:0/1802957648 shutdown_connections 2024-01-15T18:05:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- 172.21.15.143:0/1802957648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734070b50 0x7ff7340cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- 172.21.15.143:0/1802957648 >> v2:172.21.15.204:3300/0 conn(0x7ff73415bf20 0x7ff7340cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- 172.21.15.143:0/1802957648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff734158a00 0x7ff7340ce560 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- 172.21.15.143:0/1802957648 >> 172.21.15.143:0/1802957648 conn(0x7ff73406a890 msgr2=0x7ff73406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- 172.21.15.143:0/1802957648 shutdown_connections 2024-01-15T18:05:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- 172.21.15.143:0/1802957648 wait complete. 2024-01-15T18:05:57.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 Processor -- start 2024-01-15T18:05:57.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- start start 2024-01-15T18:05:57.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff734070b50 0x7ff734156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734158a00 0x7ff734156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 0x7ff734141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73413fac0 con 0x7ff7341413f0 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340b3430 con 0x7ff734070b50 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.243+0000 7ff739c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340b3710 con 0x7ff734158a00 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 0x7ff734141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 0x7ff734141880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57280/0 (socket says 172.21.15.143:57280) 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 -- 172.21.15.143:0/449641764 learned_addr learned my addr 172.21.15.143:0/449641764 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff732000640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734158a00 0x7ff734156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff732a00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff734070b50 0x7ff734156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 -- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734158a00 msgr2=0x7ff734156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:57.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734158a00 0x7ff734156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 -- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff734070b50 msgr2=0x7ff734156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff734070b50 0x7ff734156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 -- 172.21.15.143:0/449641764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240094d0 con 0x7ff7341413f0 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff733400640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 0x7ff734141880 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007e60 tx=0x7ff72c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c014620 con 0x7ff7341413f0 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c00ee90 con 0x7ff7341413f0 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73413f400 con 0x7ff7341413f0 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c0149e0 con 0x7ff7341413f0 2024-01-15T18:05:57.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff734141dc0 con 0x7ff7341413f0 2024-01-15T18:05:57.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73413fc00 con 0x7ff7341413f0 2024-01-15T18:05:57.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.247+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff72c014b40 con 0x7ff7341413f0 2024-01-15T18:05:57.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.251+0000 7ff72aa00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff70c066920 0x7ff70c068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:57.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.251+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff72c0849e0 con 0x7ff7341413f0 2024-01-15T18:05:57.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.251+0000 7ff732a00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff70c066920 0x7ff70c068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:57.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.251+0000 7ff732a00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff70c066920 0x7ff70c068de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7240af000 tx=0x7ff7240b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:57.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.255+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff73413fc00 con 0x7ff7341413f0 2024-01-15T18:05:57.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:57.392 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:57.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:57.407+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff73413fc00 con 0x7ff7341413f0 2024-01-15T18:05:58.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.067+0000 7ff72aa00640 1 -- 172.21.15.143:0/449641764 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff73413fc00 con 0x7ff7341413f0 2024-01-15T18:05:58.072 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff70c066920 msgr2=0x7ff70c068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff70c066920 0x7ff70c068de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff7240af000 tx=0x7ff7240b6040 comp rx=0 tx=0).stop 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 msgr2=0x7ff734141880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 0x7ff734141880 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff72c007e60 tx=0x7ff72c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 shutdown_connections 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff70c066920 0x7ff70c068de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7341413f0 0x7ff734141880 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff734158a00 0x7ff734156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 --2- 172.21.15.143:0/449641764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff734070b50 0x7ff734156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 >> 172.21.15.143:0/449641764 conn(0x7ff73406a890 msgr2=0x7ff7340b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:58.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 shutdown_connections 2024-01-15T18:05:58.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.071+0000 7ff739c00640 1 -- 172.21.15.143:0/449641764 wait complete. 2024-01-15T18:05:58.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:05:58.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63eb800640 1 Processor -- start 2024-01-15T18:05:58.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63eb800640 1 -- start start 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63eb800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e40b4ac0 0x7f63e4164540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63eb800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e40b2a20 con 0x7f63e4157300 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63eb800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e40b2d00 con 0x7f63e4159b70 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63eb800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40b35a0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63e8e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f63e4157300 0x7f63e40b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40900/0 (socket says 172.21.15.143:40900) 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.247+0000 7f63e8e00640 1 -- 172.21.15.143:0/240697311 learned_addr learned my addr 172.21.15.143:0/240697311 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e9800640 1 --2- 172.21.15.143:0/240697311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e40b4ac0 0x7f63e4164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765630076 0 0) 0x7f63e40b2a20 con 0x7f63e4157300 2024-01-15T18:05:58.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c4003530 con 0x7f63e4157300 2024-01-15T18:05:58.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147951175 0 0) 0x7f63e40b2d00 con 0x7f63e4159b70 2024-01-15T18:05:58.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63e40b2a20 con 0x7f63e4159b70 2024-01-15T18:05:58.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e9800640 1 -- 172.21.15.143:0/240697311 >> v1:172.21.15.204:6789/0 conn(0x7f63e4159b70 legacy=0x7f63e40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e9800640 1 -- 172.21.15.143:0/240697311 >> v1:172.21.15.143:6789/0 conn(0x7f63e4157300 legacy=0x7f63e40b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e9800640 1 -- 172.21.15.143:0/240697311 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e40b2fe0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e9800640 1 --2- 172.21.15.143:0/240697311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e40b4ac0 0x7f63e4164540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00aed0 tx=0x7f63dc01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55735bbcbb4b99e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:58.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.251+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc00a2d0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc0a5820 con 0x7f63e40b4ac0 2024-01-15T18:05:58.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63e3400640 1 -- 172.21.15.143:0/240697311 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0a5bb0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- 172.21.15.143:0/240697311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e40b4ac0 msgr2=0x7f63e4164540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 --2- 172.21.15.143:0/240697311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e40b4ac0 0x7f63e4164540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00aed0 tx=0x7f63dc01c960 comp rx=0 tx=0).stop 2024-01-15T18:05:58.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- 172.21.15.143:0/240697311 shutdown_connections 2024-01-15T18:05:58.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 --2- 172.21.15.143:0/240697311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e40b4ac0 0x7f63e4164540 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- 172.21.15.143:0/240697311 >> 172.21.15.143:0/240697311 conn(0x7f63e406c3f0 msgr2=0x7f63e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:58.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- 172.21.15.143:0/240697311 shutdown_connections 2024-01-15T18:05:58.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- 172.21.15.143:0/240697311 wait complete. 2024-01-15T18:05:58.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 Processor -- start 2024-01-15T18:05:58.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- start start 2024-01-15T18:05:58.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 0x7f63e413c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:58.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e4157300 0x7f63e40ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e4159b70 0x7f63e40ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4159fd0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e4154810 con 0x7f63e4159b70 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.255+0000 7f63eb800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e40ae4b0 con 0x7f63e4157300 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 0x7f63e413c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 0x7f63e413c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57286/0 (socket says 172.21.15.143:57286) 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 -- 172.21.15.143:0/3069151883 learned_addr learned my addr 172.21.15.143:0/3069151883 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e9800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e4159b70 0x7f63e40ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 -- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e4157300 msgr2=0x7f63e40ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e3e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e4157300 0x7f63e40ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e4157300 0x7f63e40ab190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 -- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e4159b70 msgr2=0x7f63e40ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e4159b70 0x7f63e40ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 -- 172.21.15.143:0/3069151883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc00ab80 con 0x7f63e40b4ac0 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e9800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e4159b70 0x7f63e40ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e3e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e4157300 0x7f63e40ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e8e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 0x7f63e413c1a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c4f0 tx=0x7f63d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4005cf0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d4005e50 con 0x7f63e40b4ac0 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4007270 con 0x7f63e40b4ac0 2024-01-15T18:05:58.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e4154a20 con 0x7f63e40b4ac0 2024-01-15T18:05:58.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e4154fe0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e4155200 con 0x7f63e40b4ac0 2024-01-15T18:05:58.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.259+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f63d40073d0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.263+0000 7f63e0c00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63bc0668f0 0x7f63bc068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:58.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.263+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f63d4083de0 con 0x7f63e40b4ac0 2024-01-15T18:05:58.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.263+0000 7f63e3e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63bc0668f0 0x7f63bc068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:58.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.263+0000 7f63e3e00640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63bc0668f0 0x7f63bc068db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63e4157070 tx=0x7f63cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:58.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.267+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63d4051110 con 0x7f63e40b4ac0 2024-01-15T18:05:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:58.446+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f63e4067500 con 0x7f63e40b4ac0 2024-01-15T18:05:59.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.066+0000 7f63e0c00640 1 -- 172.21.15.143:0/3069151883 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f63d4054730 con 0x7f63e40b4ac0 2024-01-15T18:05:59.074 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 4 and data pool 5 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63bc0668f0 msgr2=0x7f63bc068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63bc0668f0 0x7f63bc068db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f63e4157070 tx=0x7f63cc009040 comp rx=0 tx=0).stop 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 msgr2=0x7f63e413c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 0x7f63e413c1a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c4f0 tx=0x7f63d400c9b0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 shutdown_connections 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63bc0668f0 0x7f63bc068db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63e4159b70 0x7f63e40ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63e4157300 0x7f63e40ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 --2- 172.21.15.143:0/3069151883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63e40b4ac0 0x7f63e413c1a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 >> 172.21.15.143:0/3069151883 conn(0x7f63e406c3f0 msgr2=0x7f63e40b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 shutdown_connections 2024-01-15T18:05:59.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.070+0000 7f63eb800640 1 -- 172.21.15.143:0/3069151883 wait complete. 2024-01-15T18:05:59.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:59.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.238+0000 7fc574e00640 1 Processor -- start 2024-01-15T18:05:59.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.238+0000 7fc574e00640 1 -- start start 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.238+0000 7fc574e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57006c630 0x7fc5700b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.238+0000 7fc574e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700cb480 con 0x7fc5700b0490 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.238+0000 7fc574e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5700cb6d0 con 0x7fc57015d3f0 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.238+0000 7fc574e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5700cb9b0 con 0x7fc57006c630 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56d400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc5700b0490 0x7fc57006bf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40904/0 (socket says 172.21.15.143:40904) 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56d400640 1 -- 172.21.15.143:0/90825587 learned_addr learned my addr 172.21.15.143:0/90825587 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56e800640 1 --2- 172.21.15.143:0/90825587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57006c630 0x7fc5700b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231191382 0 0) 0x7fc5700cb480 con 0x7fc5700b0490 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc550003530 con 0x7fc5700b0490 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403401327 0 0) 0x7fc5700cb6d0 con 0x7fc57015d3f0 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5700cb480 con 0x7fc57015d3f0 2024-01-15T18:05:59.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56e800640 1 -- 172.21.15.143:0/90825587 >> v1:172.21.15.204:6789/0 conn(0x7fc57015d3f0 legacy=0x7fc5700cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56e800640 1 -- 172.21.15.143:0/90825587 >> v1:172.21.15.143:6789/0 conn(0x7fc5700b0490 legacy=0x7fc57006bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56e800640 1 -- 172.21.15.143:0/90825587 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700cbc90 con 0x7fc57006c630 2024-01-15T18:05:59.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.242+0000 7fc56e800640 1 --2- 172.21.15.143:0/90825587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57006c630 0x7fc5700b1960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc568009dc0 tx=0x7fc56801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7aaf80054575f5b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680a5a20 con 0x7fc57006c630 2024-01-15T18:05:59.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5680a5b80 con 0x7fc57006c630 2024-01-15T18:05:59.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc56ca00640 1 -- 172.21.15.143:0/90825587 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680ad440 con 0x7fc57006c630 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- 172.21.15.143:0/90825587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57006c630 msgr2=0x7fc5700b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 --2- 172.21.15.143:0/90825587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57006c630 0x7fc5700b1960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc568009dc0 tx=0x7fc56801c940 comp rx=0 tx=0).stop 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- 172.21.15.143:0/90825587 shutdown_connections 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 --2- 172.21.15.143:0/90825587 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57006c630 0x7fc5700b1960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- 172.21.15.143:0/90825587 >> 172.21.15.143:0/90825587 conn(0x7fc5700d0e30 msgr2=0x7fc5700d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- 172.21.15.143:0/90825587 shutdown_connections 2024-01-15T18:05:59.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- 172.21.15.143:0/90825587 wait complete. 2024-01-15T18:05:59.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 Processor -- start 2024-01-15T18:05:59.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- start start 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 0x7fc57015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 0x7fc57014ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57015d3f0 0x7fc57014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570156700 con 0x7fc57006c630 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc570154400 con 0x7fc5700b0490 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.246+0000 7fc574e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc570154120 con 0x7fc57015d3f0 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 0x7fc57015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 0x7fc57014ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57015d3f0 0x7fc57014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 0x7fc57014ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:55518/0 (socket says 172.21.15.143:55518) 2024-01-15T18:05:59.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 0x7fc57015e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57298/0 (socket says 172.21.15.143:57298) 2024-01-15T18:05:59.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56d400640 1 -- 172.21.15.143:0/800227143 learned_addr learned my addr 172.21.15.143:0/800227143 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 -- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 msgr2=0x7fc57014ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 0x7fc57014ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 -- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 msgr2=0x7fc57015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 0x7fc57015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 -- 172.21.15.143:0/800227143 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc568009a70 con 0x7fc57015d3f0 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56d400640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 0x7fc57014ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:59.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56de00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 0x7fc57015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc56e800640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57015d3f0 0x7fc57014b380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5680ac000 tx=0x7fc5680196e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680add00 con 0x7fc57015d3f0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5680ade60 con 0x7fc57015d3f0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5701560b0 con 0x7fc57015d3f0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5680b9690 con 0x7fc57015d3f0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57014b8c0 con 0x7fc57015d3f0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc570156870 con 0x7fc57015d3f0 2024-01-15T18:05:59.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.250+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc5680b97f0 con 0x7fc57015d3f0 2024-01-15T18:05:59.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.254+0000 7fc566000640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5480668f0 0x7fc548068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.254+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc56811f380 con 0x7fc57015d3f0 2024-01-15T18:05:59.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.254+0000 7fc56de00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5480668f0 0x7fc548068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.254+0000 7fc56de00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5480668f0 0x7fc548068db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc5600011f0 tx=0x7fc560009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.254+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc5680ec6e0 con 0x7fc57015d3f0 2024-01-15T18:05:59.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.406+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc570067500 con 0x7fc57015d3f0 2024-01-15T18:05:59.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc566000640 1 -- 172.21.15.143:0/800227143 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16618 (secure 0 0 0) 0x7fc568009450 con 0x7fc57015d3f0 2024-01-15T18:05:59.415 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:59.415 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":32,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:58.450768+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:05:56.126959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-15T18:05:57.453388+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5480668f0 msgr2=0x7fc548068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5480668f0 0x7fc548068db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc5600011f0 tx=0x7fc560009040 comp rx=0 tx=0).stop 2024-01-15T18:05:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57015d3f0 msgr2=0x7fc57014b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57015d3f0 0x7fc57014b380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5680ac000 tx=0x7fc5680196e0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 shutdown_connections 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5480668f0 0x7fc548068db0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc57015d3f0 0x7fc57014b380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5700b0490 0x7fc57014ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 --2- 172.21.15.143:0/800227143 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc57006c630 0x7fc57015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 >> 172.21.15.143:0/800227143 conn(0x7fc5700d0e30 msgr2=0x7fc5700cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 shutdown_connections 2024-01-15T18:05:59.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.410+0000 7fc574e00640 1 -- 172.21.15.143:0/800227143 wait complete. 2024-01-15T18:05:59.437 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:05:59.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.566+0000 7fc6e2a00640 1 Processor -- start 2024-01-15T18:05:59.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.566+0000 7fc6e2a00640 1 -- start start 2024-01-15T18:05:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6e2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc6dc141660 0x7fc6dc0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6e2a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6dc071d00 0x7fc6dc072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6e2a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc072650 con 0x7fc6dc15ec80 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6e2a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc15fbb0 con 0x7fc6dc141660 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6e2a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc15f130 con 0x7fc6dc071d00 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6dbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc6dc141660 0x7fc6dc0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6db400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc6dc15ec80 0x7fc6dc141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40912/0 (socket says 172.21.15.143:40912) 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6dbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc6dc141660 0x7fc6dc0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57314/0 (socket says 172.21.15.143:57314) 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6db400640 1 -- 172.21.15.143:0/2661565600 learned_addr learned my addr 172.21.15.143:0/2661565600 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6e0a00640 1 --2- 172.21.15.143:0/2661565600 >> v2:172.21.15.204:3300/0 conn(0x7fc6dc071d00 0x7fc6dc072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2661565600 >> v2:172.21.15.204:3300/0 conn(0x7fc6dc071d00 msgr2=0x7fc6dc072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6dbe00640 1 --2- 172.21.15.143:0/2661565600 >> v2:172.21.15.204:3300/0 conn(0x7fc6dc071d00 0x7fc6dc072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2661565600 >> v1:172.21.15.143:6789/0 conn(0x7fc6dc15ec80 legacy=0x7fc6dc141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2661565600 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc15f310 con 0x7fc6dc141660 2024-01-15T18:05:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.570+0000 7fc6daa00640 1 -- 172.21.15.143:0/2661565600 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221800779 0 0) 0x7fc6c4001120 con 0x7fc6dc15ec80 2024-01-15T18:05:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6dbe00640 1 --2- 172.21.15.143:0/2661565600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc141660 0x7fc6dc0ce560 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc06e250 tx=0x7fc6cc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7157c484ad38aa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6daa00640 1 -- 172.21.15.143:0/2661565600 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0a7070 con 0x7fc6dc141660 2024-01-15T18:05:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6daa00640 1 -- 172.21.15.143:0/2661565600 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc0ab430 con 0x7fc6dc141660 2024-01-15T18:05:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6daa00640 1 -- 172.21.15.143:0/2661565600 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0ab7c0 con 0x7fc6dc141660 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2661565600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc141660 msgr2=0x7fc6dc0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2661565600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc141660 0x7fc6dc0ce560 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc06e250 tx=0x7fc6cc01c430 comp rx=0 tx=0).stop 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2661565600 shutdown_connections 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2661565600 >> v2:172.21.15.204:3300/0 conn(0x7fc6dc071d00 0x7fc6dc072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2661565600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc141660 0x7fc6dc0ce560 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2661565600 >> 172.21.15.143:0/2661565600 conn(0x7fc6dc06c3f0 msgr2=0x7fc6dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2661565600 shutdown_connections 2024-01-15T18:05:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2661565600 wait complete. 2024-01-15T18:05:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 Processor -- start 2024-01-15T18:05:59.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.574+0000 7fc6e2a00640 1 -- start start 2024-01-15T18:05:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 0x7fc6dc0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc141660 0x7fc6dc13ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6dc15ec80 0x7fc6dc158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc13cfb0 con 0x7fc6dc071d00 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0b3360 con 0x7fc6dc141660 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0b2750 con 0x7fc6dc15ec80 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 0x7fc6dc0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 0x7fc6dc0b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57328/0 (socket says 172.21.15.143:57328) 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2475495506 learned_addr learned my addr 172.21.15.143:0/2475495506 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6db400640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc141660 0x7fc6dc13ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e0a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6dc15ec80 0x7fc6dc158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6dc15ec80 msgr2=0x7fc6dc158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6dc15ec80 0x7fc6dc158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc141660 msgr2=0x7fc6dc13ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc141660 0x7fc6dc13ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 -- 172.21.15.143:0/2475495506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc009420 con 0x7fc6dc071d00 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6dbe00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 0x7fc6dc0b2050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc019040 tx=0x7fc6cc01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0b5810 con 0x7fc6dc071d00 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc0b5990 con 0x7fc6dc071d00 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0b5d40 con 0x7fc6dc071d00 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc0b2a30 con 0x7fc6dc071d00 2024-01-15T18:05:59.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc0b2ff0 con 0x7fc6dc071d00 2024-01-15T18:05:59.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.578+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc067500 con 0x7fc6dc071d00 2024-01-15T18:05:59.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.582+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc6cc0b4390 con 0x7fc6dc071d00 2024-01-15T18:05:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.582+0000 7fc6d3e00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6b40668f0 0x7fc6b4068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.582+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6cc11e860 con 0x7fc6dc071d00 2024-01-15T18:05:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.582+0000 7fc6db400640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6b40668f0 0x7fc6b4068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.582+0000 7fc6db400640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6b40668f0 0x7fc6b4068db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc139ea0 tx=0x7fc6c4007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.586+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6cc0ebbd0 con 0x7fc6dc071d00 2024-01-15T18:05:59.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.734+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6dc06e2a0 con 0x7fc6dc071d00 2024-01-15T18:05:59.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6d3e00640 1 -- 172.21.15.143:0/2475495506 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7fc6cc006e20 con 0x7fc6dc071d00 2024-01-15T18:05:59.742 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:05:59.742 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:59.504321+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:05:56.126959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-15T18:05:57.453388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6b40668f0 msgr2=0x7fc6b4068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6b40668f0 0x7fc6b4068db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc139ea0 tx=0x7fc6c4007df0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 msgr2=0x7fc6dc0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 0x7fc6dc0b2050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc019040 tx=0x7fc6cc01ac70 comp rx=0 tx=0).stop 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 shutdown_connections 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc6b40668f0 0x7fc6b4068db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6dc15ec80 0x7fc6dc158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc6dc141660 0x7fc6dc13ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 --2- 172.21.15.143:0/2475495506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6dc071d00 0x7fc6dc0b2050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 >> 172.21.15.143:0/2475495506 conn(0x7fc6dc06c3f0 msgr2=0x7fc6dc1407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:59.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 shutdown_connections 2024-01-15T18:05:59.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.738+0000 7fc6e2a00640 1 -- 172.21.15.143:0/2475495506 wait complete. 2024-01-15T18:05:59.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:05:59.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 Processor -- start 2024-01-15T18:05:59.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 -- start start 2024-01-15T18:05:59.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2f0159360 0x7fb2f0071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f00b3a10 con 0x7fb2f00afdb0 2024-01-15T18:05:59.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f006a490 con 0x7fb2f0159360 2024-01-15T18:05:59.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2f5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f006a750 con 0x7fb2f00af860 2024-01-15T18:05:59.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2eea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2f0159360 0x7fb2f0071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2eea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2f0159360 0x7fb2f0071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57344/0 (socket says 172.21.15.143:57344) 2024-01-15T18:05:59.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2eea00640 1 -- 172.21.15.143:0/1756289666 learned_addr learned my addr 172.21.15.143:0/1756289666 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:59.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.898+0000 7fb2e7e00640 1 --2- 172.21.15.143:0/1756289666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2eea00640 1 -- 172.21.15.143:0/1756289666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 msgr2=0x7fb2f0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:59.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2eea00640 1 --2- 172.21.15.143:0/1756289666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2eea00640 1 -- 172.21.15.143:0/1756289666 >> v1:172.21.15.204:6789/0 conn(0x7fb2f00afdb0 legacy=0x7fb2f015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2eea00640 1 -- 172.21.15.143:0/1756289666 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f006aa30 con 0x7fb2f0159360 2024-01-15T18:05:59.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2eea00640 1 --2- 172.21.15.143:0/1756289666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f0071aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009870 tx=0x7fb2e001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c185be456109a5f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2ee000640 1 -- 172.21.15.143:0/1756289666 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00a95f0 con 0x7fb2f0159360 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2ee000640 1 -- 172.21.15.143:0/1756289666 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e00a9750 con 0x7fb2f0159360 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.902+0000 7fb2ee000640 1 -- 172.21.15.143:0/1756289666 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00a9ae0 con 0x7fb2f0159360 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- 172.21.15.143:0/1756289666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 msgr2=0x7fb2f0071aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 --2- 172.21.15.143:0/1756289666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f0071aa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009870 tx=0x7fb2e001c510 comp rx=0 tx=0).stop 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- 172.21.15.143:0/1756289666 shutdown_connections 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 --2- 172.21.15.143:0/1756289666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f0071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 --2- 172.21.15.143:0/1756289666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f0071aa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- 172.21.15.143:0/1756289666 >> 172.21.15.143:0/1756289666 conn(0x7fb2f0066720 msgr2=0x7fb2f006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:59.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- 172.21.15.143:0/1756289666 shutdown_connections 2024-01-15T18:05:59.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- 172.21.15.143:0/1756289666 wait complete. 2024-01-15T18:05:59.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 Processor -- start 2024-01-15T18:05:59.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- start start 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f014bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f00afdb0 0x7fb2f013a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f014f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f013a6b0 con 0x7fb2f0159360 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f014fd40 con 0x7fb2f00afdb0 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2f5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f014e760 con 0x7fb2f00af860 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2efe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f014f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57348/0 (socket says 172.21.15.143:57348) 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2efe00640 1 -- 172.21.15.143:0/2459490250 learned_addr learned my addr 172.21.15.143:0/2459490250 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:05:59.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2e7e00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f00afdb0 0x7fb2f013a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.906+0000 7fb2eea00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f014bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2efe00640 1 -- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 msgr2=0x7fb2f014bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2efe00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f014bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2efe00640 1 -- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f00afdb0 msgr2=0x7fb2f013a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2efe00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f00afdb0 0x7fb2f013a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2efe00640 1 -- 172.21.15.143:0/2459490250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e00094d0 con 0x7fb2f0159360 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2efe00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f014f800 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c770 tx=0x7fb2e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8003b80 con 0x7fb2f0159360 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8003ce0 con 0x7fb2f0159360 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f014ea40 con 0x7fb2f0159360 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800aa90 con 0x7fb2f0159360 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f014f000 con 0x7fb2f0159360 2024-01-15T18:05:59.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f0067680 con 0x7fb2f0159360 2024-01-15T18:05:59.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fb2e800abf0 con 0x7fb2f0159360 2024-01-15T18:05:59.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2e6a00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2c80668f0 0x7fb2c8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:59.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2e8084140 con 0x7fb2f0159360 2024-01-15T18:05:59.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.910+0000 7fb2eea00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2c80668f0 0x7fb2c8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:59.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.914+0000 7fb2eea00640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2c80668f0 0x7fb2c8068db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00ae000 tx=0x7fb2e00b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:59.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:05:59.914+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2e8051430 con 0x7fb2f0159360 2024-01-15T18:06:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:00.098+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb2f00d0f50 con 0x7fb2f0159360 2024-01-15T18:06:01.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.090+0000 7fb2e6a00640 1 -- 172.21.15.143:0/2459490250 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb2e8054a50 con 0x7fb2f0159360 2024-01-15T18:06:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2c80668f0 msgr2=0x7fb2c8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2c80668f0 0x7fb2c8068db0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00ae000 tx=0x7fb2e00b3040 comp rx=0 tx=0).stop 2024-01-15T18:06:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 msgr2=0x7fb2f014f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f014f800 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c770 tx=0x7fb2e800cc30 comp rx=0 tx=0).stop 2024-01-15T18:06:01.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 shutdown_connections 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2c80668f0 0x7fb2c8068db0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2f0159360 0x7fb2f014f800 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f00afdb0 0x7fb2f013a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 --2- 172.21.15.143:0/2459490250 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2f00af860 0x7fb2f014bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 >> 172.21.15.143:0/2459490250 conn(0x7fb2f0066720 msgr2=0x7fb2f00cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 shutdown_connections 2024-01-15T18:06:01.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.094+0000 7fb2f5800640 1 -- 172.21.15.143:0/2459490250 wait complete. 2024-01-15T18:06:01.120 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:06:01.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:06:01.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.254+0000 7fc225800640 1 Processor -- start 2024-01-15T18:06:01.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.254+0000 7fc225800640 1 -- start start 2024-01-15T18:06:01.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc225800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc220144700 0x7fc2200705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc225800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc22015b4e0 0x7fc22006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc225800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc2200b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc225800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22015b950 con 0x7fc22015b4e0 2024-01-15T18:06:01.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc225800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22006ca60 con 0x7fc220144700 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc225800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200b1650 con 0x7fc2200b0cd0 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc22015b4e0 0x7fc22006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc22015b4e0 0x7fc22006c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57350/0 (socket says 172.21.15.143:57350) 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 -- 172.21.15.143:0/4229844639 learned_addr learned my addr 172.21.15.143:0/4229844639 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21f400640 1 --2- 172.21.15.143:0/4229844639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc2200b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21ea00640 1 --2- 172.21.15.143:0/4229844639 >> v2:172.21.15.204:3300/0 conn(0x7fc220144700 0x7fc2200705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 -- 172.21.15.143:0/4229844639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 msgr2=0x7fc2200b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 --2- 172.21.15.143:0/4229844639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc2200b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 -- 172.21.15.143:0/4229844639 >> v2:172.21.15.204:3300/0 conn(0x7fc220144700 msgr2=0x7fc2200705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:01.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 --2- 172.21.15.143:0/4229844639 >> v2:172.21.15.204:3300/0 conn(0x7fc220144700 0x7fc2200705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.258+0000 7fc21e000640 1 -- 172.21.15.143:0/4229844639 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22006cc90 con 0x7fc22015b4e0 2024-01-15T18:06:01.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc21e000640 1 --2- 172.21.15.143:0/4229844639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc22015b4e0 0x7fc22006c520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2200b1910 tx=0x7fc20801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36825cd07758910 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:01.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc21d600640 1 -- 172.21.15.143:0/4229844639 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080a7070 con 0x7fc22015b4e0 2024-01-15T18:06:01.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc21d600640 1 -- 172.21.15.143:0/4229844639 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080ab430 con 0x7fc22015b4e0 2024-01-15T18:06:01.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc21d600640 1 -- 172.21.15.143:0/4229844639 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ab7c0 con 0x7fc22015b4e0 2024-01-15T18:06:01.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 -- 172.21.15.143:0/4229844639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc22015b4e0 msgr2=0x7fc22006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 --2- 172.21.15.143:0/4229844639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc22015b4e0 0x7fc22006c520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc2200b1910 tx=0x7fc20801c470 comp rx=0 tx=0).stop 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 -- 172.21.15.143:0/4229844639 shutdown_connections 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 --2- 172.21.15.143:0/4229844639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc2200b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 --2- 172.21.15.143:0/4229844639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc22015b4e0 0x7fc22006c520 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 --2- 172.21.15.143:0/4229844639 >> v2:172.21.15.204:3300/0 conn(0x7fc220144700 0x7fc2200705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 -- 172.21.15.143:0/4229844639 >> 172.21.15.143:0/4229844639 conn(0x7fc22006a660 msgr2=0x7fc22006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 -- 172.21.15.143:0/4229844639 shutdown_connections 2024-01-15T18:06:01.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 -- 172.21.15.143:0/4229844639 wait complete. 2024-01-15T18:06:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 Processor -- start 2024-01-15T18:06:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.262+0000 7fc225800640 1 -- start start 2024-01-15T18:06:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc220138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 0x7fc2201533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22015b4e0 0x7fc220159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201393f0 con 0x7fc220144700 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220139640 con 0x7fc22015b4e0 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc220139920 con 0x7fc2200b0cd0 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 0x7fc2201533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 0x7fc2201533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57352/0 (socket says 172.21.15.143:57352) 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 -- 172.21.15.143:0/42333449 learned_addr learned my addr 172.21.15.143:0/42333449 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21f400640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22015b4e0 0x7fc220159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21ea00640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc220138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 -- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 msgr2=0x7fc220138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc220138c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 -- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22015b4e0 msgr2=0x7fc220159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22015b4e0 0x7fc220159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 -- 172.21.15.143:0/42333449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080094d0 con 0x7fc220144700 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc21e000640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 0x7fc2201533a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ad040 tx=0x7fc208019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080be610 con 0x7fc220144700 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080be7b0 con 0x7fc220144700 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080beb60 con 0x7fc220144700 2024-01-15T18:06:01.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc220139c00 con 0x7fc220144700 2024-01-15T18:06:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22013a1c0 con 0x7fc220144700 2024-01-15T18:06:01.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.266+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22013a3b0 con 0x7fc220144700 2024-01-15T18:06:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.270+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fc2080b43f0 con 0x7fc220144700 2024-01-15T18:06:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.270+0000 7fc216a00640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1f80668f0 0x7fc1f8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.270+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc20811e680 con 0x7fc220144700 2024-01-15T18:06:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.270+0000 7fc21ea00640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1f80668f0 0x7fc1f8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.270+0000 7fc21ea00640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1f80668f0 0x7fc1f8068db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc210009b60 tx=0x7fc210009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:01.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.274+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc2080eba80 con 0x7fc220144700 2024-01-15T18:06:01.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:01.454+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc220067500 con 0x7fc220144700 2024-01-15T18:06:02.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc216a00640 1 -- 172.21.15.143:0/42333449 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2080ef0a0 con 0x7fc220144700 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1f80668f0 msgr2=0x7fc1f8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1f80668f0 0x7fc1f8068db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc210009b60 tx=0x7fc210009190 comp rx=0 tx=0).stop 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 msgr2=0x7fc2201533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 0x7fc2201533a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ad040 tx=0x7fc208019210 comp rx=0 tx=0).stop 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 shutdown_connections 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1f80668f0 0x7fc1f8068db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc22015b4e0 0x7fc220159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc220144700 0x7fc2201533a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 --2- 172.21.15.143:0/42333449 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc2200b0cd0 0x7fc220138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 >> 172.21.15.143:0/42333449 conn(0x7fc22006a660 msgr2=0x7fc220143a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 shutdown_connections 2024-01-15T18:06:02.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.094+0000 7fc225800640 1 -- 172.21.15.143:0/42333449 wait complete. 2024-01-15T18:06:02.123 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:06:02.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:06:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f7772600640 1 Processor -- start 2024-01-15T18:06:02.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f7772600640 1 -- start start 2024-01-15T18:06:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f7772600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f776c143780 0x7f776c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f7772600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c0ceb50 con 0x7f776c0722c0 2024-01-15T18:06:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f7772600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c15c110 con 0x7f776c0ce520 2024-01-15T18:06:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f7772600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c15b7e0 con 0x7f776c143780 2024-01-15T18:06:02.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f776c143780 0x7f776c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f776c143780 0x7f776c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57368/0 (socket says 172.21.15.143:57368) 2024-01-15T18:06:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f776c0722c0 0x7f776c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40914/0 (socket says 172.21.15.143:40914) 2024-01-15T18:06:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776b400640 1 -- 172.21.15.143:0/1115356727 learned_addr learned my addr 172.21.15.143:0/1115356727 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776b400640 1 -- 172.21.15.143:0/1115356727 >> v1:172.21.15.204:6789/0 conn(0x7f776c0ce520 legacy=0x7f776c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776b400640 1 -- 172.21.15.143:0/1115356727 >> v1:172.21.15.143:6789/0 conn(0x7f776c0722c0 legacy=0x7f776c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.254+0000 7f776b400640 1 -- 172.21.15.143:0/1115356727 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c15bac0 con 0x7f776c143780 2024-01-15T18:06:02.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f776b400640 1 --2- 172.21.15.143:0/1115356727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c143780 0x7f776c0cdfe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f775c009870 tx=0x7f775c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f11cbfa5687f07f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f776a000640 1 -- 172.21.15.143:0/1115356727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c0ab650 con 0x7f776c143780 2024-01-15T18:06:02.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f776a000640 1 -- 172.21.15.143:0/1115356727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c0ab7b0 con 0x7f776c143780 2024-01-15T18:06:02.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f776a000640 1 -- 172.21.15.143:0/1115356727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c0abb40 con 0x7f776c143780 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 -- 172.21.15.143:0/1115356727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c143780 msgr2=0x7f776c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 --2- 172.21.15.143:0/1115356727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c143780 0x7f776c0cdfe0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f775c009870 tx=0x7f775c01c540 comp rx=0 tx=0).stop 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 -- 172.21.15.143:0/1115356727 shutdown_connections 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 --2- 172.21.15.143:0/1115356727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c143780 0x7f776c0cdfe0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 -- 172.21.15.143:0/1115356727 >> 172.21.15.143:0/1115356727 conn(0x7f776c06c3f0 msgr2=0x7f776c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 -- 172.21.15.143:0/1115356727 shutdown_connections 2024-01-15T18:06:02.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.258+0000 7f7772600640 1 -- 172.21.15.143:0/1115356727 wait complete. 2024-01-15T18:06:02.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 Processor -- start 2024-01-15T18:06:02.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 -- start start 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 0x7f776c143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c0ce520 0x7f776c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c143780 0x7f776c0c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c146e50 con 0x7f776c0722c0 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c146240 con 0x7f776c143780 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c146520 con 0x7f776c0ce520 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 0x7f776c143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 0x7f776c143c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57372/0 (socket says 172.21.15.143:57372) 2024-01-15T18:06:02.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 -- 172.21.15.143:0/1670732535 learned_addr learned my addr 172.21.15.143:0/1670732535 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776be00640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c143780 0x7f776c0c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776aa00640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c0ce520 0x7f776c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 -- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c0ce520 msgr2=0x7f776c0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c0ce520 0x7f776c0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 -- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c143780 msgr2=0x7f776c0c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c143780 0x7f776c0c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 -- 172.21.15.143:0/1670732535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c0094d0 con 0x7f776c0722c0 2024-01-15T18:06:02.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776be00640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c143780 0x7f776c0c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776aa00640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c0ce520 0x7f776c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f776b400640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 0x7f776c143c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f775c0ad040 tx=0x7f775c01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c0b4800 con 0x7f776c0722c0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c0b4980 con 0x7f776c0722c0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c146800 con 0x7f776c0722c0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c0b4d30 con 0x7f776c0722c0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.262+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c0c4250 con 0x7f776c0722c0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.266+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c067500 con 0x7f776c0722c0 2024-01-15T18:06:02.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.266+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f775c0b3380 con 0x7f776c0722c0 2024-01-15T18:06:02.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.266+0000 7f7763400640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77440668f0 0x7f7744068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.266+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f775c11eb50 con 0x7f776c0722c0 2024-01-15T18:06:02.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.266+0000 7f776aa00640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77440668f0 0x7f7744068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.266+0000 7f776aa00640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77440668f0 0x7f7744068db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f776c070400 tx=0x7f7754009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.270+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f775c0ebec0 con 0x7f776c0722c0 2024-01-15T18:06:02.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.450+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f776c146660 con 0x7f776c0722c0 2024-01-15T18:06:02.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.450+0000 7f7763400640 1 -- 172.21.15.143:0/1670732535 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f776c146660 con 0x7f776c0722c0 2024-01-15T18:06:02.457 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:02.457 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3272757327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3272757327},{"type":"v1","addr":"172.21.15.204:6834","nonce":3272757327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/518983608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":518983608},{"type":"v1","addr":"172.21.15.143:6836","nonce":518983608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/434311355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":434311355},{"type":"v1","addr":"172.21.15.143:6837","nonce":434311355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:05:58.451230+0000","modified":"2024-01-15T18:06:01.458502+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4126176885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":4126176885},{"type":"v1","addr":"172.21.15.204:6835","nonce":4126176885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-15T18:06:02.458 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2024-01-15T18:06:02.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77440668f0 msgr2=0x7f7744068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77440668f0 0x7f7744068db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f776c070400 tx=0x7f7754009040 comp rx=0 tx=0).stop 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 msgr2=0x7f776c143c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 0x7f776c143c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f775c0ad040 tx=0x7f775c01ad80 comp rx=0 tx=0).stop 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 shutdown_connections 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f77440668f0 0x7f7744068db0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c143780 0x7f776c0c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c0ce520 0x7f776c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 --2- 172.21.15.143:0/1670732535 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c0722c0 0x7f776c143c80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 >> 172.21.15.143:0/1670732535 conn(0x7f776c06c3f0 msgr2=0x7f776c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 shutdown_connections 2024-01-15T18:06:02.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.454+0000 7f7772600640 1 -- 172.21.15.143:0/1670732535 wait complete. 2024-01-15T18:06:02.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:06:02.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 Processor -- start 2024-01-15T18:06:02.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 -- start start 2024-01-15T18:06:02.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f86c0141660 0x7f86c00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c015f310 con 0x7f86c015ec80 2024-01-15T18:06:02.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c015fbb0 con 0x7f86c0141660 2024-01-15T18:06:02.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c7800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c015f130 con 0x7f86c0071d00 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c0072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:37878/0 (socket says 172.21.15.143:37878) 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 -- 172.21.15.143:0/1457083517 learned_addr learned my addr 172.21.15.143:0/1457083517 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c4e00640 1 --2- 172.21.15.143:0/1457083517 >> v2:172.21.15.204:3300/0 conn(0x7f86c0141660 0x7f86c00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 -- 172.21.15.143:0/1457083517 >> v2:172.21.15.204:3300/0 conn(0x7f86c0141660 msgr2=0x7f86c00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 --2- 172.21.15.143:0/1457083517 >> v2:172.21.15.204:3300/0 conn(0x7f86c0141660 0x7f86c00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 -- 172.21.15.143:0/1457083517 >> v1:172.21.15.204:6789/0 conn(0x7f86c015ec80 legacy=0x7f86c0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c5800640 1 -- 172.21.15.143:0/1457083517 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c015f5f0 con 0x7f86c0071d00 2024-01-15T18:06:02.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.610+0000 7f86c4e00640 1 --2- 172.21.15.143:0/1457083517 >> v2:172.21.15.204:3300/0 conn(0x7f86c0141660 0x7f86c00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:02.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c5800640 1 --2- 172.21.15.143:0/1457083517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c0072150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009e50 tx=0x7f86b801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5372b961d6d727 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86bf400640 1 -- 172.21.15.143:0/1457083517 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80a57d0 con 0x7f86c0071d00 2024-01-15T18:06:02.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86bf400640 1 -- 172.21.15.143:0/1457083517 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80a5930 con 0x7f86c0071d00 2024-01-15T18:06:02.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86bf400640 1 -- 172.21.15.143:0/1457083517 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80a5cc0 con 0x7f86c0071d00 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 -- 172.21.15.143:0/1457083517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 msgr2=0x7f86c0072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 --2- 172.21.15.143:0/1457083517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c0072150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009e50 tx=0x7f86b801ca10 comp rx=0 tx=0).stop 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 -- 172.21.15.143:0/1457083517 shutdown_connections 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 --2- 172.21.15.143:0/1457083517 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c0072150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 --2- 172.21.15.143:0/1457083517 >> v2:172.21.15.204:3300/0 conn(0x7f86c0141660 0x7f86c00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 -- 172.21.15.143:0/1457083517 >> 172.21.15.143:0/1457083517 conn(0x7f86c006c3f0 msgr2=0x7f86c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 -- 172.21.15.143:0/1457083517 shutdown_connections 2024-01-15T18:06:02.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.614+0000 7f86c7800640 1 -- 172.21.15.143:0/1457083517 wait complete. 2024-01-15T18:06:02.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 Processor -- start 2024-01-15T18:06:02.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 -- start start 2024-01-15T18:06:02.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c00acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 0x7f86c0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c015ec80 0x7f86c0071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c013bc40 con 0x7f86c0141660 2024-01-15T18:06:02.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c015ab70 con 0x7f86c015ec80 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c015e550 con 0x7f86c0071d00 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 0x7f86c0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 0x7f86c0070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57386/0 (socket says 172.21.15.143:57386) 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 -- 172.21.15.143:0/2134120600 learned_addr learned my addr 172.21.15.143:0/2134120600 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c4e00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c00acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c5800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c015ec80 0x7f86c0071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 -- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 msgr2=0x7f86c00acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c00acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 -- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c015ec80 msgr2=0x7f86c0071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c015ec80 0x7f86c0071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 -- 172.21.15.143:0/2134120600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0009420 con 0x7f86c0141660 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c5800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c015ec80 0x7f86c0071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bfe00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 0x7f86c0070b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f86a800c500 tx=0x7f86a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8005220 con 0x7f86c0141660 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a8005380 con 0x7f86c0141660 2024-01-15T18:06:02.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b8009b30 con 0x7f86c0141660 2024-01-15T18:06:02.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a8007270 con 0x7f86c0141660 2024-01-15T18:06:02.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.618+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c00cd150 con 0x7f86c0141660 2024-01-15T18:06:02.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.622+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c0067500 con 0x7f86c0141660 2024-01-15T18:06:02.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.622+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f86a80073d0 con 0x7f86c0141660 2024-01-15T18:06:02.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.622+0000 7f86bcc00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8698066920 0x7f8698068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.622+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86a8083c20 con 0x7f86c0141660 2024-01-15T18:06:02.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.622+0000 7f86c4e00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8698066920 0x7f8698068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.622+0000 7f86c4e00640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8698066920 0x7f8698068de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f86c0141030 tx=0x7f86b0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.626+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86a8050e70 con 0x7f86c0141660 2024-01-15T18:06:02.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.778+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c013e580 con 0x7f86c0141660 2024-01-15T18:06:02.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.778+0000 7f86bcc00640 1 -- 172.21.15.143:0/2134120600 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f86a8054490 con 0x7f86c0141660 2024-01-15T18:06:02.786 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:02.786 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":33,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:05:59.504321+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:05:56.126959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-15T18:05:57.453388+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:06:02.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8698066920 msgr2=0x7f8698068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8698066920 0x7f8698068de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f86c0141030 tx=0x7f86b0009190 comp rx=0 tx=0).stop 2024-01-15T18:06:02.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 msgr2=0x7f86c0070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 0x7f86c0070b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f86a800c500 tx=0x7f86a800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 shutdown_connections 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8698066920 0x7f8698068de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86c015ec80 0x7f86c0071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86c0141660 0x7f86c0070b40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 --2- 172.21.15.143:0/2134120600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86c0071d00 0x7f86c00acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 >> 172.21.15.143:0/2134120600 conn(0x7f86c006c3f0 msgr2=0x7f86c00afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 shutdown_connections 2024-01-15T18:06:02.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.782+0000 7f86c7800640 1 -- 172.21.15.143:0/2134120600 wait complete. 2024-01-15T18:06:02.808 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:06:02.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:06:02.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 Processor -- start 2024-01-15T18:06:02.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 -- start start 2024-01-15T18:06:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa78c0c3e50 0x7fa78c0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0d1a80 0x7fa78c0c4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0af150 con 0x7fa78c0adbd0 2024-01-15T18:06:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78c0c4770 con 0x7fa78c0c3e50 2024-01-15T18:06:02.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.938+0000 7fa793c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0ae540 con 0x7fa78c0d1a80 2024-01-15T18:06:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0d1a80 0x7fa78c0c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa791c00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa78c0adbd0 0x7fa78c0cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58168/0 (socket says 172.21.15.143:58168) 2024-01-15T18:06:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa791c00640 1 -- 172.21.15.143:0/2229504187 learned_addr learned my addr 172.21.15.143:0/2229504187 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa791200640 1 --2- 172.21.15.143:0/2229504187 >> v2:172.21.15.204:3300/0 conn(0x7fa78c0c3e50 0x7fa78c0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78be00640 1 -- 172.21.15.143:0/2229504187 >> v2:172.21.15.204:3300/0 conn(0x7fa78c0c3e50 msgr2=0x7fa78c0af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:02.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78be00640 1 --2- 172.21.15.143:0/2229504187 >> v2:172.21.15.204:3300/0 conn(0x7fa78c0c3e50 0x7fa78c0af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78be00640 1 -- 172.21.15.143:0/2229504187 >> v1:172.21.15.204:6789/0 conn(0x7fa78c0adbd0 legacy=0x7fa78c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78b400640 1 -- 172.21.15.143:0/2229504187 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126993962 0 0) 0x7fa78c0af150 con 0x7fa78c0adbd0 2024-01-15T18:06:02.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78be00640 1 -- 172.21.15.143:0/2229504187 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c0ae820 con 0x7fa78c0d1a80 2024-01-15T18:06:02.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa791200640 1 --2- 172.21.15.143:0/2229504187 >> v2:172.21.15.204:3300/0 conn(0x7fa78c0c3e50 0x7fa78c0af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:02.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.942+0000 7fa78be00640 1 --2- 172.21.15.143:0/2229504187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0d1a80 0x7fa78c0c4230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7740097c0 tx=0x7fa77401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c188ff1b8c282775 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa78b400640 1 -- 172.21.15.143:0/2229504187 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7740ab650 con 0x7fa78c0d1a80 2024-01-15T18:06:02.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa78b400640 1 -- 172.21.15.143:0/2229504187 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7740ab7b0 con 0x7fa78c0d1a80 2024-01-15T18:06:02.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa78b400640 1 -- 172.21.15.143:0/2229504187 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7740abb40 con 0x7fa78c0d1a80 2024-01-15T18:06:02.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 -- 172.21.15.143:0/2229504187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0d1a80 msgr2=0x7fa78c0c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 --2- 172.21.15.143:0/2229504187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0d1a80 0x7fa78c0c4230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa7740097c0 tx=0x7fa77401c450 comp rx=0 tx=0).stop 2024-01-15T18:06:02.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 -- 172.21.15.143:0/2229504187 shutdown_connections 2024-01-15T18:06:02.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 --2- 172.21.15.143:0/2229504187 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0d1a80 0x7fa78c0c4230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 --2- 172.21.15.143:0/2229504187 >> v2:172.21.15.204:3300/0 conn(0x7fa78c0c3e50 0x7fa78c0af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 -- 172.21.15.143:0/2229504187 >> 172.21.15.143:0/2229504187 conn(0x7fa78c0d0e80 msgr2=0x7fa78c0d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 -- 172.21.15.143:0/2229504187 shutdown_connections 2024-01-15T18:06:02.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 -- 172.21.15.143:0/2229504187 wait complete. 2024-01-15T18:06:02.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 Processor -- start 2024-01-15T18:06:02.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 -- start start 2024-01-15T18:06:02.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0adbd0 0x7fa78c15e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.946+0000 7fa793c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c0c3e50 0x7fa78c0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 0x7fa78c0c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0cbf30 con 0x7fa78c0d1a80 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0cc210 con 0x7fa78c0c3e50 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c0cc4f0 con 0x7fa78c0adbd0 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 0x7fa78c0c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 0x7fa78c0c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57388/0 (socket says 172.21.15.143:57388) 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 -- 172.21.15.143:0/2666427523 learned_addr learned my addr 172.21.15.143:0/2666427523 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791200640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0adbd0 0x7fa78c15e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa78be00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c0c3e50 0x7fa78c0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 -- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0adbd0 msgr2=0x7fa78c15e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0adbd0 0x7fa78c15e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 -- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c0c3e50 msgr2=0x7fa78c0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c0c3e50 0x7fa78c0cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 -- 172.21.15.143:0/2666427523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009420 con 0x7fa78c0d1a80 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791200640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0adbd0 0x7fa78c15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa791c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 0x7fa78c0c5ca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa784001570 tx=0x7fa78400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400a640 con 0x7fa78c0d1a80 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78400a7c0 con 0x7fa78c0d1a80 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78400ab70 con 0x7fa78c0d1a80 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa774009420 con 0x7fa78c0d1a80 2024-01-15T18:06:02.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c161c70 con 0x7fa78c0d1a80 2024-01-15T18:06:02.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c067500 con 0x7fa78c0d1a80 2024-01-15T18:06:02.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.950+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7fa78400acd0 con 0x7fa78c0d1a80 2024-01-15T18:06:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.954+0000 7fa788c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7640668f0 0x7fa764068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.954+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7840839f0 con 0x7fa78c0d1a80 2024-01-15T18:06:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.954+0000 7fa791200640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7640668f0 0x7fa764068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.954+0000 7fa791200640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7640668f0 0x7fa764068db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0c76a0 tx=0x7fa77c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:02.958+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa784050c40 con 0x7fa78c0d1a80 2024-01-15T18:06:03.054 INFO:tasks.ceph.mds.d.smithi204.stderr:2024-01-15T18:06:03.049+0000 7fb5c5400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:06:03.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.110+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa78c0c5a00 con 0x7fa7640668f0 2024-01-15T18:06:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.118+0000 7fa788c00640 1 -- 172.21.15.143:0/2666427523 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289058 (secure 0 0 0) 0x7fa78c0c5a00 con 0x7fa7640668f0 2024-01-15T18:06:03.125 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:06:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7640668f0 msgr2=0x7fa764068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7640668f0 0x7fa764068db0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0c76a0 tx=0x7fa77c009190 comp rx=0 tx=0).stop 2024-01-15T18:06:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 msgr2=0x7fa78c0c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 0x7fa78c0c5ca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa784001570 tx=0x7fa78400cca0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 shutdown_connections 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7640668f0 0x7fa764068db0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa78c0d1a80 0x7fa78c0c5ca0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa78c0c3e50 0x7fa78c0cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 --2- 172.21.15.143:0/2666427523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa78c0adbd0 0x7fa78c15e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 >> 172.21.15.143:0/2666427523 conn(0x7fa78c0d0e80 msgr2=0x7fa78c160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 shutdown_connections 2024-01-15T18:06:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.122+0000 7fa793c00640 1 -- 172.21.15.143:0/2666427523 wait complete. 2024-01-15T18:06:03.149 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-01-15T18:06:02.634299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4235264,"data_stored":2358292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570623"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514473+0000","last_change":"2024-01-15T18:05:58.526638+0000","last_active":"2024-01-15T18:05:59.514473+0000","last_peered":"2024-01-15T18:05:59.514473+0000","last_clean":"2024-01-15T18:05:59.514473+0000","last_became_active":"2024-01-15T18:05:58.526147+0000","last_became_peered":"2024-01-15T18:05:58.526147+0000","last_unstale":"2024-01-15T18:05:59.514473+0000","last_undegraded":"2024-01-15T18:05:59.514473+0000","last_fullsized":"2024-01-15T18:05:59.514473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:18.657632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246936+0000","last_change":"2024-01-15T18:05:58.524452+0000","last_active":"2024-01-15T18:06:00.246936+0000","last_peered":"2024-01-15T18:06:00.246936+0000","last_clean":"2024-01-15T18:06:00.246936+0000","last_became_active":"2024-01-15T18:05:58.524128+0000","last_became_peered":"2024-01-15T18:05:58.524128+0000","last_unstale":"2024-01-15T18:06:00.246936+0000","last_undegraded":"2024-01-15T18:06:00.246936+0000","last_fullsized":"2024-01-15T18:06:00.246936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:23.838470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763385+0000","last_change":"2024-01-15T18:05:58.523682+0000","last_active":"2024-01-15T18:05:59.763385+0000","last_peered":"2024-01-15T18:05:59.763385+0000","last_clean":"2024-01-15T18:05:59.763385+0000","last_became_active":"2024-01-15T18:05:58.523394+0000","last_became_peered":"2024-01-15T18:05:58.523394+0000","last_unstale":"2024-01-15T18:05:59.763385+0000","last_undegraded":"2024-01-15T18:05:59.763385+0000","last_fullsized":"2024-01-15T18:05:59.763385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:23.838470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153345+0000","last_change":"2024-01-15T18:05:58.533697+0000","last_active":"2024-01-15T18:06:00.153345+0000","last_peered":"2024-01-15T18:06:00.153345+0000","last_clean":"2024-01-15T18:06:00.153345+0000","last_became_active":"2024-01-15T18:05:58.533394+0000","last_became_peered":"2024-01-15T18:05:58.533394+0000","last_unstale":"2024-01-15T18:06:00.153345+0000","last_undegraded":"2024-01-15T18:06:00.153345+0000","last_fullsized":"2024-01-15T18:06:00.153345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:49.554679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512686+0000","last_change":"2024-01-15T18:05:58.519538+0000","last_active":"2024-01-15T18:05:59.512686+0000","last_peered":"2024-01-15T18:05:59.512686+0000","last_clean":"2024-01-15T18:05:59.512686+0000","last_became_active":"2024-01-15T18:05:58.519262+0000","last_became_peered":"2024-01-15T18:05:58.519262+0000","last_unstale":"2024-01-15T18:05:59.512686+0000","last_undegraded":"2024-01-15T18:05:59.512686+0000","last_fullsized":"2024-01-15T18:05:59.512686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:11.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246752+0000","last_change":"2024-01-15T18:05:58.524414+0000","last_active":"2024-01-15T18:06:00.246752+0000","last_peered":"2024-01-15T18:06:00.246752+0000","last_clean":"2024-01-15T18:06:00.246752+0000","last_became_active":"2024-01-15T18:05:58.524128+0000","last_became_peered":"2024-01-15T18:05:58.524128+0000","last_unstale":"2024-01-15T18:06:00.246752+0000","last_undegraded":"2024-01-15T18:06:00.246752+0000","last_fullsized":"2024-01-15T18:06:00.246752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:16:20.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849325+0000","last_change":"2024-01-15T18:05:58.532952+0000","last_active":"2024-01-15T18:05:59.849325+0000","last_peered":"2024-01-15T18:05:59.849325+0000","last_clean":"2024-01-15T18:05:59.849325+0000","last_became_active":"2024-01-15T18:05:58.532408+0000","last_became_peered":"2024-01-15T18:05:58.532408+0000","last_unstale":"2024-01-15T18:05:59.849325+0000","last_undegraded":"2024-01-15T18:05:59.849325+0000","last_fullsized":"2024-01-15T18:05:59.849325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.408307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.848774+0000","last_change":"2024-01-15T18:05:58.533350+0000","last_active":"2024-01-15T18:05:59.848774+0000","last_peered":"2024-01-15T18:05:59.848774+0000","last_clean":"2024-01-15T18:05:59.848774+0000","last_became_active":"2024-01-15T18:05:58.533043+0000","last_became_peered":"2024-01-15T18:05:58.533043+0000","last_unstale":"2024-01-15T18:05:59.848774+0000","last_undegraded":"2024-01-15T18:05:59.848774+0000","last_fullsized":"2024-01-15T18:05:59.848774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:18.832803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517314+0000","last_change":"2024-01-15T18:05:58.529404+0000","last_active":"2024-01-15T18:05:59.517314+0000","last_peered":"2024-01-15T18:05:59.517314+0000","last_clean":"2024-01-15T18:05:59.517314+0000","last_became_active":"2024-01-15T18:05:58.529051+0000","last_became_peered":"2024-01-15T18:05:58.529051+0000","last_unstale":"2024-01-15T18:05:59.517314+0000","last_undegraded":"2024-01-15T18:05:59.517314+0000","last_fullsized":"2024-01-15T18:05:59.517314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:49.554679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153500+0000","last_change":"2024-01-15T18:05:58.526133+0000","last_active":"2024-01-15T18:06:00.153500+0000","last_peered":"2024-01-15T18:06:00.153500+0000","last_clean":"2024-01-15T18:06:00.153500+0000","last_became_active":"2024-01-15T18:05:58.525887+0000","last_became_peered":"2024-01-15T18:05:58.525887+0000","last_unstale":"2024-01-15T18:06:00.153500+0000","last_undegraded":"2024-01-15T18:06:00.153500+0000","last_fullsized":"2024-01-15T18:06:00.153500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:42.997863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849163+0000","last_change":"2024-01-15T18:05:58.530013+0000","last_active":"2024-01-15T18:05:59.849163+0000","last_peered":"2024-01-15T18:05:59.849163+0000","last_clean":"2024-01-15T18:05:59.849163+0000","last_became_active":"2024-01-15T18:05:58.529712+0000","last_became_peered":"2024-01-15T18:05:58.529712+0000","last_unstale":"2024-01-15T18:05:59.849163+0000","last_undegraded":"2024-01-15T18:05:59.849163+0000","last_fullsized":"2024-01-15T18:05:59.849163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:32.228628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763223+0000","last_change":"2024-01-15T18:05:58.519538+0000","last_active":"2024-01-15T18:05:59.763223+0000","last_peered":"2024-01-15T18:05:59.763223+0000","last_clean":"2024-01-15T18:05:59.763223+0000","last_became_active":"2024-01-15T18:05:58.519273+0000","last_became_peered":"2024-01-15T18:05:58.519273+0000","last_unstale":"2024-01-15T18:05:59.763223+0000","last_undegraded":"2024-01-15T18:05:59.763223+0000","last_fullsized":"2024-01-15T18:05:59.763223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:16:20.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512722+0000","last_change":"2024-01-15T18:05:58.517201+0000","last_active":"2024-01-15T18:05:59.512722+0000","last_peered":"2024-01-15T18:05:59.512722+0000","last_clean":"2024-01-15T18:05:59.512722+0000","last_became_active":"2024-01-15T18:05:58.516934+0000","last_became_peered":"2024-01-15T18:05:58.516934+0000","last_unstale":"2024-01-15T18:05:59.512722+0000","last_undegraded":"2024-01-15T18:05:59.512722+0000","last_fullsized":"2024-01-15T18:05:59.512722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:55.653021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246721+0000","last_change":"2024-01-15T18:05:58.518928+0000","last_active":"2024-01-15T18:06:00.246721+0000","last_peered":"2024-01-15T18:06:00.246721+0000","last_clean":"2024-01-15T18:06:00.246721+0000","last_became_active":"2024-01-15T18:05:58.518782+0000","last_became_peered":"2024-01-15T18:05:58.518782+0000","last_unstale":"2024-01-15T18:06:00.246721+0000","last_undegraded":"2024-01-15T18:06:00.246721+0000","last_fullsized":"2024-01-15T18:06:00.246721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:39.209214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763181+0000","last_change":"2024-01-15T18:05:58.529081+0000","last_active":"2024-01-15T18:05:59.763181+0000","last_peered":"2024-01-15T18:05:59.763181+0000","last_clean":"2024-01-15T18:05:59.763181+0000","last_became_active":"2024-01-15T18:05:58.528798+0000","last_became_peered":"2024-01-15T18:05:58.528798+0000","last_unstale":"2024-01-15T18:05:59.763181+0000","last_undegraded":"2024-01-15T18:05:59.763181+0000","last_fullsized":"2024-01-15T18:05:59.763181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:39.209214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849269+0000","last_change":"2024-01-15T18:05:58.535727+0000","last_active":"2024-01-15T18:05:59.849269+0000","last_peered":"2024-01-15T18:05:59.849269+0000","last_clean":"2024-01-15T18:05:59.849269+0000","last_became_active":"2024-01-15T18:05:58.534827+0000","last_became_peered":"2024-01-15T18:05:58.534827+0000","last_unstale":"2024-01-15T18:05:59.849269+0000","last_undegraded":"2024-01-15T18:05:59.849269+0000","last_fullsized":"2024-01-15T18:05:59.849269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:05.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849361+0000","last_change":"2024-01-15T18:05:58.535784+0000","last_active":"2024-01-15T18:05:59.849361+0000","last_peered":"2024-01-15T18:05:59.849361+0000","last_clean":"2024-01-15T18:05:59.849361+0000","last_became_active":"2024-01-15T18:05:58.535387+0000","last_became_peered":"2024-01-15T18:05:58.535387+0000","last_unstale":"2024-01-15T18:05:59.849361+0000","last_undegraded":"2024-01-15T18:05:59.849361+0000","last_fullsized":"2024-01-15T18:05:59.849361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:21.429830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246983+0000","last_change":"2024-01-15T18:05:58.522105+0000","last_active":"2024-01-15T18:06:00.246983+0000","last_peered":"2024-01-15T18:06:00.246983+0000","last_clean":"2024-01-15T18:06:00.246983+0000","last_became_active":"2024-01-15T18:05:58.521810+0000","last_became_peered":"2024-01-15T18:05:58.521810+0000","last_unstale":"2024-01-15T18:06:00.246983+0000","last_undegraded":"2024-01-15T18:06:00.246983+0000","last_fullsized":"2024-01-15T18:06:00.246983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:21.429830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246783+0000","last_change":"2024-01-15T18:05:58.527401+0000","last_active":"2024-01-15T18:06:00.246783+0000","last_peered":"2024-01-15T18:06:00.246783+0000","last_clean":"2024-01-15T18:06:00.246783+0000","last_became_active":"2024-01-15T18:05:58.527115+0000","last_became_peered":"2024-01-15T18:05:58.527115+0000","last_unstale":"2024-01-15T18:06:00.246783+0000","last_undegraded":"2024-01-15T18:06:00.246783+0000","last_fullsized":"2024-01-15T18:06:00.246783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:56.364492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512889+0000","last_change":"2024-01-15T18:05:58.523443+0000","last_active":"2024-01-15T18:05:59.512889+0000","last_peered":"2024-01-15T18:05:59.512889+0000","last_clean":"2024-01-15T18:05:59.512889+0000","last_became_active":"2024-01-15T18:05:58.523167+0000","last_became_peered":"2024-01-15T18:05:58.523167+0000","last_unstale":"2024-01-15T18:05:59.512889+0000","last_undegraded":"2024-01-15T18:05:59.512889+0000","last_fullsized":"2024-01-15T18:05:59.512889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:44.949108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516699+0000","last_change":"2024-01-15T18:05:58.529091+0000","last_active":"2024-01-15T18:05:59.516699+0000","last_peered":"2024-01-15T18:05:59.516699+0000","last_clean":"2024-01-15T18:05:59.516699+0000","last_became_active":"2024-01-15T18:05:58.528793+0000","last_became_peered":"2024-01-15T18:05:58.528793+0000","last_unstale":"2024-01-15T18:05:59.516699+0000","last_undegraded":"2024-01-15T18:05:59.516699+0000","last_fullsized":"2024-01-15T18:05:59.516699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:18.657632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517104+0000","last_change":"2024-01-15T18:05:58.519531+0000","last_active":"2024-01-15T18:05:59.517104+0000","last_peered":"2024-01-15T18:05:59.517104+0000","last_clean":"2024-01-15T18:05:59.517104+0000","last_became_active":"2024-01-15T18:05:58.519186+0000","last_became_peered":"2024-01-15T18:05:58.519186+0000","last_unstale":"2024-01-15T18:05:59.517104+0000","last_undegraded":"2024-01-15T18:05:59.517104+0000","last_fullsized":"2024-01-15T18:05:59.517104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:44.324719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160057+0000","last_change":"2024-01-15T18:05:58.526896+0000","last_active":"2024-01-15T18:06:00.160057+0000","last_peered":"2024-01-15T18:06:00.160057+0000","last_clean":"2024-01-15T18:06:00.160057+0000","last_became_active":"2024-01-15T18:05:58.526636+0000","last_became_peered":"2024-01-15T18:05:58.526636+0000","last_unstale":"2024-01-15T18:06:00.160057+0000","last_undegraded":"2024-01-15T18:06:00.160057+0000","last_fullsized":"2024-01-15T18:06:00.160057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:18.657632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.762914+0000","last_change":"2024-01-15T18:05:58.519931+0000","last_active":"2024-01-15T18:05:59.762914+0000","last_peered":"2024-01-15T18:05:59.762914+0000","last_clean":"2024-01-15T18:05:59.762914+0000","last_became_active":"2024-01-15T18:05:58.519034+0000","last_became_peered":"2024-01-15T18:05:58.519034+0000","last_unstale":"2024-01-15T18:05:59.762914+0000","last_undegraded":"2024-01-15T18:05:59.762914+0000","last_fullsized":"2024-01-15T18:05:59.762914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:42:53.369765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153606+0000","last_change":"2024-01-15T18:05:58.530043+0000","last_active":"2024-01-15T18:06:00.153606+0000","last_peered":"2024-01-15T18:06:00.153606+0000","last_clean":"2024-01-15T18:06:00.153606+0000","last_became_active":"2024-01-15T18:05:58.529742+0000","last_became_peered":"2024-01-15T18:05:58.529742+0000","last_unstale":"2024-01-15T18:06:00.153606+0000","last_undegraded":"2024-01-15T18:06:00.153606+0000","last_fullsized":"2024-01-15T18:06:00.153606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:18.832803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.247003+0000","last_change":"2024-01-15T18:05:58.530430+0000","last_active":"2024-01-15T18:06:00.247003+0000","last_peered":"2024-01-15T18:06:00.247003+0000","last_clean":"2024-01-15T18:06:00.247003+0000","last_became_active":"2024-01-15T18:05:58.530142+0000","last_became_peered":"2024-01-15T18:05:58.530142+0000","last_unstale":"2024-01-15T18:06:00.247003+0000","last_undegraded":"2024-01-15T18:06:00.247003+0000","last_fullsized":"2024-01-15T18:06:00.247003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:58:02.687786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153492+0000","last_change":"2024-01-15T18:05:58.531848+0000","last_active":"2024-01-15T18:06:00.153492+0000","last_peered":"2024-01-15T18:06:00.153492+0000","last_clean":"2024-01-15T18:06:00.153492+0000","last_became_active":"2024-01-15T18:05:58.531580+0000","last_became_peered":"2024-01-15T18:05:58.531580+0000","last_unstale":"2024-01-15T18:06:00.153492+0000","last_undegraded":"2024-01-15T18:06:00.153492+0000","last_fullsized":"2024-01-15T18:06:00.153492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:13.220702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763222+0000","last_change":"2024-01-15T18:05:58.525064+0000","last_active":"2024-01-15T18:05:59.763222+0000","last_peered":"2024-01-15T18:05:59.763222+0000","last_clean":"2024-01-15T18:05:59.763222+0000","last_became_active":"2024-01-15T18:05:58.524751+0000","last_became_peered":"2024-01-15T18:05:58.524751+0000","last_unstale":"2024-01-15T18:05:59.763222+0000","last_undegraded":"2024-01-15T18:05:59.763222+0000","last_fullsized":"2024-01-15T18:05:59.763222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:56.364492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516768+0000","last_change":"2024-01-15T18:05:58.528901+0000","last_active":"2024-01-15T18:05:59.516768+0000","last_peered":"2024-01-15T18:05:59.516768+0000","last_clean":"2024-01-15T18:05:59.516768+0000","last_became_active":"2024-01-15T18:05:58.528351+0000","last_became_peered":"2024-01-15T18:05:58.528351+0000","last_unstale":"2024-01-15T18:05:59.516768+0000","last_undegraded":"2024-01-15T18:05:59.516768+0000","last_fullsized":"2024-01-15T18:05:59.516768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:35.234818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.848915+0000","last_change":"2024-01-15T18:05:58.535973+0000","last_active":"2024-01-15T18:05:59.848915+0000","last_peered":"2024-01-15T18:05:59.848915+0000","last_clean":"2024-01-15T18:05:59.848915+0000","last_became_active":"2024-01-15T18:05:58.535687+0000","last_became_peered":"2024-01-15T18:05:58.535687+0000","last_unstale":"2024-01-15T18:05:59.848915+0000","last_undegraded":"2024-01-15T18:05:59.848915+0000","last_fullsized":"2024-01-15T18:05:59.848915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:12:05.866938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514252+0000","last_change":"2024-01-15T18:05:58.527076+0000","last_active":"2024-01-15T18:05:59.514252+0000","last_peered":"2024-01-15T18:05:59.514252+0000","last_clean":"2024-01-15T18:05:59.514252+0000","last_became_active":"2024-01-15T18:05:58.524498+0000","last_became_peered":"2024-01-15T18:05:58.524498+0000","last_unstale":"2024-01-15T18:05:59.514252+0000","last_undegraded":"2024-01-15T18:05:59.514252+0000","last_fullsized":"2024-01-15T18:05:59.514252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:23.825492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.848831+0000","last_change":"2024-01-15T18:05:58.538112+0000","last_active":"2024-01-15T18:05:59.848831+0000","last_peered":"2024-01-15T18:05:59.848831+0000","last_clean":"2024-01-15T18:05:59.848831+0000","last_became_active":"2024-01-15T18:05:58.537774+0000","last_became_peered":"2024-01-15T18:05:58.537774+0000","last_unstale":"2024-01-15T18:05:59.848831+0000","last_undegraded":"2024-01-15T18:05:59.848831+0000","last_fullsized":"2024-01-15T18:05:59.848831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:42:53.369765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160335+0000","last_change":"2024-01-15T18:05:58.528250+0000","last_active":"2024-01-15T18:06:00.160335+0000","last_peered":"2024-01-15T18:06:00.160335+0000","last_clean":"2024-01-15T18:06:00.160335+0000","last_became_active":"2024-01-15T18:05:58.527996+0000","last_became_peered":"2024-01-15T18:05:58.527996+0000","last_unstale":"2024-01-15T18:06:00.160335+0000","last_undegraded":"2024-01-15T18:06:00.160335+0000","last_fullsized":"2024-01-15T18:06:00.160335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:44.324719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514566+0000","last_change":"2024-01-15T18:05:58.090063+0000","last_active":"2024-01-15T18:05:59.514566+0000","last_peered":"2024-01-15T18:05:59.514566+0000","last_clean":"2024-01-15T18:05:59.514566+0000","last_became_active":"2024-01-15T18:05:58.089758+0000","last_became_peered":"2024-01-15T18:05:58.089758+0000","last_unstale":"2024-01-15T18:05:59.514566+0000","last_undegraded":"2024-01-15T18:05:59.514566+0000","last_fullsized":"2024-01-15T18:05:59.514566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:05.223309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514532+0000","last_change":"2024-01-15T18:05:58.526406+0000","last_active":"2024-01-15T18:05:59.514532+0000","last_peered":"2024-01-15T18:05:59.514532+0000","last_clean":"2024-01-15T18:05:59.514532+0000","last_became_active":"2024-01-15T18:05:58.525812+0000","last_became_peered":"2024-01-15T18:05:58.525812+0000","last_unstale":"2024-01-15T18:05:59.514532+0000","last_undegraded":"2024-01-15T18:05:59.514532+0000","last_fullsized":"2024-01-15T18:05:59.514532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:02.645403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160322+0000","last_change":"2024-01-15T18:05:58.090182+0000","last_active":"2024-01-15T18:06:00.160322+0000","last_peered":"2024-01-15T18:06:00.160322+0000","last_clean":"2024-01-15T18:06:00.160322+0000","last_became_active":"2024-01-15T18:05:58.089936+0000","last_became_peered":"2024-01-15T18:05:58.089936+0000","last_unstale":"2024-01-15T18:06:00.160322+0000","last_undegraded":"2024-01-15T18:06:00.160322+0000","last_fullsized":"2024-01-15T18:06:00.160322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517108+0000","last_change":"2024-01-15T18:05:58.527205+0000","last_active":"2024-01-15T18:05:59.517108+0000","last_peered":"2024-01-15T18:05:59.517108+0000","last_clean":"2024-01-15T18:05:59.517108+0000","last_became_active":"2024-01-15T18:05:58.526930+0000","last_became_peered":"2024-01-15T18:05:58.526930+0000","last_unstale":"2024-01-15T18:05:59.517108+0000","last_undegraded":"2024-01-15T18:05:59.517108+0000","last_fullsized":"2024-01-15T18:05:59.517108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:44:07.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246815+0000","last_change":"2024-01-15T18:05:58.097847+0000","last_active":"2024-01-15T18:06:00.246815+0000","last_peered":"2024-01-15T18:06:00.246815+0000","last_clean":"2024-01-15T18:06:00.246815+0000","last_became_active":"2024-01-15T18:05:58.095203+0000","last_became_peered":"2024-01-15T18:05:58.095203+0000","last_unstale":"2024-01-15T18:06:00.246815+0000","last_undegraded":"2024-01-15T18:06:00.246815+0000","last_fullsized":"2024-01-15T18:06:00.246815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:48.544035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153510+0000","last_change":"2024-01-15T18:05:58.531973+0000","last_active":"2024-01-15T18:06:00.153510+0000","last_peered":"2024-01-15T18:06:00.153510+0000","last_clean":"2024-01-15T18:06:00.153510+0000","last_became_active":"2024-01-15T18:05:58.531673+0000","last_became_peered":"2024-01-15T18:05:58.531673+0000","last_unstale":"2024-01-15T18:06:00.153510+0000","last_undegraded":"2024-01-15T18:06:00.153510+0000","last_fullsized":"2024-01-15T18:06:00.153510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:07:00.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512800+0000","last_change":"2024-01-15T18:05:58.091581+0000","last_active":"2024-01-15T18:05:59.512800+0000","last_peered":"2024-01-15T18:05:59.512800+0000","last_clean":"2024-01-15T18:05:59.512800+0000","last_became_active":"2024-01-15T18:05:58.090297+0000","last_became_peered":"2024-01-15T18:05:58.090297+0000","last_unstale":"2024-01-15T18:05:59.512800+0000","last_undegraded":"2024-01-15T18:05:59.512800+0000","last_fullsized":"2024-01-15T18:05:59.512800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:14:47.548551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849141+0000","last_change":"2024-01-15T18:05:58.534773+0000","last_active":"2024-01-15T18:05:59.849141+0000","last_peered":"2024-01-15T18:05:59.849141+0000","last_clean":"2024-01-15T18:05:59.849141+0000","last_became_active":"2024-01-15T18:05:58.534430+0000","last_became_peered":"2024-01-15T18:05:58.534430+0000","last_unstale":"2024-01-15T18:05:59.849141+0000","last_undegraded":"2024-01-15T18:05:59.849141+0000","last_fullsized":"2024-01-15T18:05:59.849141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:56.364492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516827+0000","last_change":"2024-01-15T18:05:58.101502+0000","last_active":"2024-01-15T18:05:59.516827+0000","last_peered":"2024-01-15T18:05:59.516827+0000","last_clean":"2024-01-15T18:05:59.516827+0000","last_became_active":"2024-01-15T18:05:58.101169+0000","last_became_peered":"2024-01-15T18:05:58.101169+0000","last_unstale":"2024-01-15T18:05:59.516827+0000","last_undegraded":"2024-01-15T18:05:59.516827+0000","last_fullsized":"2024-01-15T18:05:59.516827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:54.642377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763054+0000","last_change":"2024-01-15T18:05:58.526495+0000","last_active":"2024-01-15T18:05:59.763054+0000","last_peered":"2024-01-15T18:05:59.763054+0000","last_clean":"2024-01-15T18:05:59.763054+0000","last_became_active":"2024-01-15T18:05:58.526160+0000","last_became_peered":"2024-01-15T18:05:58.526160+0000","last_unstale":"2024-01-15T18:05:59.763054+0000","last_undegraded":"2024-01-15T18:05:59.763054+0000","last_fullsized":"2024-01-15T18:05:59.763054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:12:05.866938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517049+0000","last_change":"2024-01-15T18:05:58.101507+0000","last_active":"2024-01-15T18:05:59.517049+0000","last_peered":"2024-01-15T18:05:59.517049+0000","last_clean":"2024-01-15T18:05:59.517049+0000","last_became_active":"2024-01-15T18:05:58.101125+0000","last_became_peered":"2024-01-15T18:05:58.101125+0000","last_unstale":"2024-01-15T18:05:59.517049+0000","last_undegraded":"2024-01-15T18:05:59.517049+0000","last_fullsized":"2024-01-15T18:05:59.517049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:22.814848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516733+0000","last_change":"2024-01-15T18:05:58.525816+0000","last_active":"2024-01-15T18:05:59.516733+0000","last_peered":"2024-01-15T18:05:59.516733+0000","last_clean":"2024-01-15T18:05:59.516733+0000","last_became_active":"2024-01-15T18:05:58.525508+0000","last_became_peered":"2024-01-15T18:05:58.525508+0000","last_unstale":"2024-01-15T18:05:59.516733+0000","last_undegraded":"2024-01-15T18:05:59.516733+0000","last_fullsized":"2024-01-15T18:05:59.516733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:02.645403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246524+0000","last_change":"2024-01-15T18:05:58.094847+0000","last_active":"2024-01-15T18:06:00.246524+0000","last_peered":"2024-01-15T18:06:00.246524+0000","last_clean":"2024-01-15T18:06:00.246524+0000","last_became_active":"2024-01-15T18:05:58.094443+0000","last_became_peered":"2024-01-15T18:05:58.094443+0000","last_unstale":"2024-01-15T18:06:00.246524+0000","last_undegraded":"2024-01-15T18:06:00.246524+0000","last_fullsized":"2024-01-15T18:06:00.246524+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153134+0000","last_change":"2024-01-15T18:05:58.525223+0000","last_active":"2024-01-15T18:06:00.153134+0000","last_peered":"2024-01-15T18:06:00.153134+0000","last_clean":"2024-01-15T18:06:00.153134+0000","last_became_active":"2024-01-15T18:05:58.524942+0000","last_became_peered":"2024-01-15T18:05:58.524942+0000","last_unstale":"2024-01-15T18:06:00.153134+0000","last_undegraded":"2024-01-15T18:06:00.153134+0000","last_fullsized":"2024-01-15T18:06:00.153134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:52:49.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516663+0000","last_change":"2024-01-15T18:05:58.089206+0000","last_active":"2024-01-15T18:05:59.516663+0000","last_peered":"2024-01-15T18:05:59.516663+0000","last_clean":"2024-01-15T18:05:59.516663+0000","last_became_active":"2024-01-15T18:05:58.088797+0000","last_became_peered":"2024-01-15T18:05:58.088797+0000","last_unstale":"2024-01-15T18:05:59.516663+0000","last_undegraded":"2024-01-15T18:05:59.516663+0000","last_fullsized":"2024-01-15T18:05:59.516663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:07.557677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763466+0000","last_change":"2024-01-15T18:05:58.531213+0000","last_active":"2024-01-15T18:05:59.763466+0000","last_peered":"2024-01-15T18:05:59.763466+0000","last_clean":"2024-01-15T18:05:59.763466+0000","last_became_active":"2024-01-15T18:05:58.530956+0000","last_became_peered":"2024-01-15T18:05:58.530956+0000","last_unstale":"2024-01-15T18:05:59.763466+0000","last_undegraded":"2024-01-15T18:05:59.763466+0000","last_fullsized":"2024-01-15T18:05:59.763466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:21.429830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.247026+0000","last_change":"2024-01-15T18:05:58.099183+0000","last_active":"2024-01-15T18:06:00.247026+0000","last_peered":"2024-01-15T18:06:00.247026+0000","last_clean":"2024-01-15T18:06:00.247026+0000","last_became_active":"2024-01-15T18:05:58.098974+0000","last_became_peered":"2024-01-15T18:05:58.098974+0000","last_unstale":"2024-01-15T18:06:00.247026+0000","last_undegraded":"2024-01-15T18:06:00.247026+0000","last_fullsized":"2024-01-15T18:06:00.247026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:12.210058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246955+0000","last_change":"2024-01-15T18:05:58.525755+0000","last_active":"2024-01-15T18:06:00.246955+0000","last_peered":"2024-01-15T18:06:00.246955+0000","last_clean":"2024-01-15T18:06:00.246955+0000","last_became_active":"2024-01-15T18:05:58.525537+0000","last_became_peered":"2024-01-15T18:05:58.525537+0000","last_unstale":"2024-01-15T18:06:00.246955+0000","last_undegraded":"2024-01-15T18:06:00.246955+0000","last_fullsized":"2024-01-15T18:06:00.246955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:05.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160424+0000","last_change":"2024-01-15T18:05:58.100616+0000","last_active":"2024-01-15T18:06:00.160424+0000","last_peered":"2024-01-15T18:06:00.160424+0000","last_clean":"2024-01-15T18:06:00.160424+0000","last_became_active":"2024-01-15T18:05:58.100152+0000","last_became_peered":"2024-01-15T18:05:58.100152+0000","last_unstale":"2024-01-15T18:06:00.160424+0000","last_undegraded":"2024-01-15T18:06:00.160424+0000","last_fullsized":"2024-01-15T18:06:00.160424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:22.814848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246965+0000","last_change":"2024-01-15T18:05:58.533034+0000","last_active":"2024-01-15T18:06:00.246965+0000","last_peered":"2024-01-15T18:06:00.246965+0000","last_clean":"2024-01-15T18:06:00.246965+0000","last_became_active":"2024-01-15T18:05:58.532600+0000","last_became_peered":"2024-01-15T18:05:58.532600+0000","last_unstale":"2024-01-15T18:06:00.246965+0000","last_undegraded":"2024-01-15T18:06:00.246965+0000","last_fullsized":"2024-01-15T18:06:00.246965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.408307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763345+0000","last_change":"2024-01-15T18:05:58.093705+0000","last_active":"2024-01-15T18:05:59.763345+0000","last_peered":"2024-01-15T18:05:59.763345+0000","last_clean":"2024-01-15T18:05:59.763345+0000","last_became_active":"2024-01-15T18:05:58.093273+0000","last_became_peered":"2024-01-15T18:05:58.093273+0000","last_unstale":"2024-01-15T18:05:59.763345+0000","last_undegraded":"2024-01-15T18:05:59.763345+0000","last_fullsized":"2024-01-15T18:05:59.763345+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.833990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763259+0000","last_change":"2024-01-15T18:05:58.527097+0000","last_active":"2024-01-15T18:05:59.763259+0000","last_peered":"2024-01-15T18:05:59.763259+0000","last_clean":"2024-01-15T18:05:59.763259+0000","last_became_active":"2024-01-15T18:05:58.526332+0000","last_became_peered":"2024-01-15T18:05:58.526332+0000","last_unstale":"2024-01-15T18:05:59.763259+0000","last_undegraded":"2024-01-15T18:05:59.763259+0000","last_fullsized":"2024-01-15T18:05:59.763259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:32.228628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246917+0000","last_change":"2024-01-15T18:05:58.095806+0000","last_active":"2024-01-15T18:06:00.246917+0000","last_peered":"2024-01-15T18:06:00.246917+0000","last_clean":"2024-01-15T18:06:00.246917+0000","last_became_active":"2024-01-15T18:05:58.095542+0000","last_became_peered":"2024-01-15T18:05:58.095542+0000","last_unstale":"2024-01-15T18:06:00.246917+0000","last_undegraded":"2024-01-15T18:06:00.246917+0000","last_fullsized":"2024-01-15T18:06:00.246917+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:17.822159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153116+0000","last_change":"2024-01-15T18:05:58.532494+0000","last_active":"2024-01-15T18:06:00.153116+0000","last_peered":"2024-01-15T18:06:00.153116+0000","last_clean":"2024-01-15T18:06:00.153116+0000","last_became_active":"2024-01-15T18:05:58.530694+0000","last_became_peered":"2024-01-15T18:05:58.530694+0000","last_unstale":"2024-01-15T18:06:00.153116+0000","last_undegraded":"2024-01-15T18:06:00.153116+0000","last_fullsized":"2024-01-15T18:06:00.153116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:44:07.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516869+0000","last_change":"2024-01-15T18:05:58.089048+0000","last_active":"2024-01-15T18:05:59.516869+0000","last_peered":"2024-01-15T18:05:59.516869+0000","last_clean":"2024-01-15T18:05:59.516869+0000","last_became_active":"2024-01-15T18:05:58.088713+0000","last_became_peered":"2024-01-15T18:05:58.088713+0000","last_unstale":"2024-01-15T18:05:59.516869+0000","last_undegraded":"2024-01-15T18:05:59.516869+0000","last_fullsized":"2024-01-15T18:05:59.516869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:54.029185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246893+0000","last_change":"2024-01-15T18:05:58.525053+0000","last_active":"2024-01-15T18:06:00.246893+0000","last_peered":"2024-01-15T18:06:00.246893+0000","last_clean":"2024-01-15T18:06:00.246893+0000","last_became_active":"2024-01-15T18:05:58.524703+0000","last_became_peered":"2024-01-15T18:05:58.524703+0000","last_unstale":"2024-01-15T18:06:00.246893+0000","last_undegraded":"2024-01-15T18:06:00.246893+0000","last_fullsized":"2024-01-15T18:06:00.246893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:32.228628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153136+0000","last_change":"2024-01-15T18:05:58.088554+0000","last_active":"2024-01-15T18:06:00.153136+0000","last_peered":"2024-01-15T18:06:00.153136+0000","last_clean":"2024-01-15T18:06:00.153136+0000","last_became_active":"2024-01-15T18:05:58.088258+0000","last_became_peered":"2024-01-15T18:05:58.088258+0000","last_unstale":"2024-01-15T18:06:00.153136+0000","last_undegraded":"2024-01-15T18:06:00.153136+0000","last_fullsized":"2024-01-15T18:06:00.153136+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849105+0000","last_change":"2024-01-15T18:05:58.535158+0000","last_active":"2024-01-15T18:05:59.849105+0000","last_peered":"2024-01-15T18:05:59.849105+0000","last_clean":"2024-01-15T18:05:59.849105+0000","last_became_active":"2024-01-15T18:05:58.534876+0000","last_became_peered":"2024-01-15T18:05:58.534876+0000","last_unstale":"2024-01-15T18:05:59.849105+0000","last_undegraded":"2024-01-15T18:05:59.849105+0000","last_fullsized":"2024-01-15T18:05:59.849105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:16:20.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246456+0000","last_change":"2024-01-15T18:05:58.100011+0000","last_active":"2024-01-15T18:06:00.246456+0000","last_peered":"2024-01-15T18:06:00.246456+0000","last_clean":"2024-01-15T18:06:00.246456+0000","last_became_active":"2024-01-15T18:05:58.099711+0000","last_became_peered":"2024-01-15T18:05:58.099711+0000","last_unstale":"2024-01-15T18:06:00.246456+0000","last_undegraded":"2024-01-15T18:06:00.246456+0000","last_fullsized":"2024-01-15T18:06:00.246456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514426+0000","last_change":"2024-01-15T18:05:58.521804+0000","last_active":"2024-01-15T18:05:59.514426+0000","last_peered":"2024-01-15T18:05:59.514426+0000","last_clean":"2024-01-15T18:05:59.514426+0000","last_became_active":"2024-01-15T18:05:58.521514+0000","last_became_peered":"2024-01-15T18:05:58.521514+0000","last_unstale":"2024-01-15T18:05:59.514426+0000","last_undegraded":"2024-01-15T18:05:59.514426+0000","last_fullsized":"2024-01-15T18:05:59.514426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:15.105803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246514+0000","last_change":"2024-01-15T18:05:58.096932+0000","last_active":"2024-01-15T18:06:00.246514+0000","last_peered":"2024-01-15T18:06:00.246514+0000","last_clean":"2024-01-15T18:06:00.246514+0000","last_became_active":"2024-01-15T18:05:58.096550+0000","last_became_peered":"2024-01-15T18:05:58.096550+0000","last_unstale":"2024-01-15T18:06:00.246514+0000","last_undegraded":"2024-01-15T18:06:00.246514+0000","last_fullsized":"2024-01-15T18:06:00.246514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:34.224174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517149+0000","last_change":"2024-01-15T18:05:58.528027+0000","last_active":"2024-01-15T18:05:59.517149+0000","last_peered":"2024-01-15T18:05:59.517149+0000","last_clean":"2024-01-15T18:05:59.517149+0000","last_became_active":"2024-01-15T18:05:58.527673+0000","last_became_peered":"2024-01-15T18:05:58.527673+0000","last_unstale":"2024-01-15T18:05:59.517149+0000","last_undegraded":"2024-01-15T18:05:59.517149+0000","last_fullsized":"2024-01-15T18:05:59.517149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:52:49.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153686+0000","last_change":"2024-01-15T18:05:58.098672+0000","last_active":"2024-01-15T18:06:00.153686+0000","last_peered":"2024-01-15T18:06:00.153686+0000","last_clean":"2024-01-15T18:06:00.153686+0000","last_became_active":"2024-01-15T18:05:58.098332+0000","last_became_peered":"2024-01-15T18:05:58.098332+0000","last_unstale":"2024-01-15T18:06:00.153686+0000","last_undegraded":"2024-01-15T18:06:00.153686+0000","last_fullsized":"2024-01-15T18:06:00.153686+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:17.646988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763391+0000","last_change":"2024-01-15T18:05:58.519891+0000","last_active":"2024-01-15T18:05:59.763391+0000","last_peered":"2024-01-15T18:05:59.763391+0000","last_clean":"2024-01-15T18:05:59.763391+0000","last_became_active":"2024-01-15T18:05:58.518714+0000","last_became_peered":"2024-01-15T18:05:58.518714+0000","last_unstale":"2024-01-15T18:05:59.763391+0000","last_undegraded":"2024-01-15T18:05:59.763391+0000","last_fullsized":"2024-01-15T18:05:59.763391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:05.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517214+0000","last_change":"2024-01-15T18:05:58.089201+0000","last_active":"2024-01-15T18:05:59.517214+0000","last_peered":"2024-01-15T18:05:59.517214+0000","last_clean":"2024-01-15T18:05:59.517214+0000","last_became_active":"2024-01-15T18:05:58.088786+0000","last_became_peered":"2024-01-15T18:05:58.088786+0000","last_unstale":"2024-01-15T18:05:59.517214+0000","last_undegraded":"2024-01-15T18:05:59.517214+0000","last_fullsized":"2024-01-15T18:05:59.517214+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153378+0000","last_change":"2024-01-15T18:05:58.533723+0000","last_active":"2024-01-15T18:06:00.153378+0000","last_peered":"2024-01-15T18:06:00.153378+0000","last_clean":"2024-01-15T18:06:00.153378+0000","last_became_active":"2024-01-15T18:05:58.533389+0000","last_became_peered":"2024-01-15T18:05:58.533389+0000","last_unstale":"2024-01-15T18:06:00.153378+0000","last_undegraded":"2024-01-15T18:06:00.153378+0000","last_fullsized":"2024-01-15T18:06:00.153378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:29.324616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517085+0000","last_change":"2024-01-15T18:05:58.091057+0000","last_active":"2024-01-15T18:05:59.517085+0000","last_peered":"2024-01-15T18:05:59.517085+0000","last_clean":"2024-01-15T18:05:59.517085+0000","last_became_active":"2024-01-15T18:05:58.090687+0000","last_became_peered":"2024-01-15T18:05:58.090687+0000","last_unstale":"2024-01-15T18:05:59.517085+0000","last_undegraded":"2024-01-15T18:05:59.517085+0000","last_fullsized":"2024-01-15T18:05:59.517085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:31:02.330657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512852+0000","last_change":"2024-01-15T18:05:58.531553+0000","last_active":"2024-01-15T18:05:59.512852+0000","last_peered":"2024-01-15T18:05:59.512852+0000","last_clean":"2024-01-15T18:05:59.512852+0000","last_became_active":"2024-01-15T18:05:58.531297+0000","last_became_peered":"2024-01-15T18:05:58.531297+0000","last_unstale":"2024-01-15T18:05:59.512852+0000","last_undegraded":"2024-01-15T18:05:59.512852+0000","last_fullsized":"2024-01-15T18:05:59.512852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:08.568321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517135+0000","last_change":"2024-01-15T18:05:58.099072+0000","last_active":"2024-01-15T18:05:59.517135+0000","last_peered":"2024-01-15T18:05:59.517135+0000","last_clean":"2024-01-15T18:05:59.517135+0000","last_became_active":"2024-01-15T18:05:58.098747+0000","last_became_peered":"2024-01-15T18:05:58.098747+0000","last_unstale":"2024-01-15T18:05:59.517135+0000","last_undegraded":"2024-01-15T18:05:59.517135+0000","last_fullsized":"2024-01-15T18:05:59.517135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512620+0000","last_change":"2024-01-15T18:05:58.526920+0000","last_active":"2024-01-15T18:05:59.512620+0000","last_peered":"2024-01-15T18:05:59.512620+0000","last_clean":"2024-01-15T18:05:59.512620+0000","last_became_active":"2024-01-15T18:05:58.526672+0000","last_became_peered":"2024-01-15T18:05:58.526672+0000","last_unstale":"2024-01-15T18:05:59.512620+0000","last_undegraded":"2024-01-15T18:05:59.512620+0000","last_fullsized":"2024-01-15T18:05:59.512620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:07.406787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160172+0000","last_change":"2024-01-15T18:05:58.089592+0000","last_active":"2024-01-15T18:06:00.160172+0000","last_peered":"2024-01-15T18:06:00.160172+0000","last_clean":"2024-01-15T18:06:00.160172+0000","last_became_active":"2024-01-15T18:05:58.089308+0000","last_became_peered":"2024-01-15T18:05:58.089308+0000","last_unstale":"2024-01-15T18:06:00.160172+0000","last_undegraded":"2024-01-15T18:06:00.160172+0000","last_fullsized":"2024-01-15T18:06:00.160172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512815+0000","last_change":"2024-01-15T18:05:58.524151+0000","last_active":"2024-01-15T18:05:59.512815+0000","last_peered":"2024-01-15T18:05:59.512815+0000","last_clean":"2024-01-15T18:05:59.512815+0000","last_became_active":"2024-01-15T18:05:58.523828+0000","last_became_peered":"2024-01-15T18:05:58.523828+0000","last_unstale":"2024-01-15T18:05:59.512815+0000","last_undegraded":"2024-01-15T18:05:59.512815+0000","last_fullsized":"2024-01-15T18:05:59.512815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:55.039829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514310+0000","last_change":"2024-01-15T18:05:58.090143+0000","last_active":"2024-01-15T18:05:59.514310+0000","last_peered":"2024-01-15T18:05:59.514310+0000","last_clean":"2024-01-15T18:05:59.514310+0000","last_became_active":"2024-01-15T18:05:58.089793+0000","last_became_peered":"2024-01-15T18:05:58.089793+0000","last_unstale":"2024-01-15T18:05:59.514310+0000","last_undegraded":"2024-01-15T18:05:59.514310+0000","last_fullsized":"2024-01-15T18:05:59.514310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512492+0000","last_change":"2024-01-15T18:05:58.520203+0000","last_active":"2024-01-15T18:05:59.512492+0000","last_peered":"2024-01-15T18:05:59.512492+0000","last_clean":"2024-01-15T18:05:59.512492+0000","last_became_active":"2024-01-15T18:05:58.519935+0000","last_became_peered":"2024-01-15T18:05:58.519935+0000","last_unstale":"2024-01-15T18:05:59.512492+0000","last_undegraded":"2024-01-15T18:05:59.512492+0000","last_fullsized":"2024-01-15T18:05:59.512492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:01.713324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516640+0000","last_change":"2024-01-15T18:05:58.089185+0000","last_active":"2024-01-15T18:05:59.516640+0000","last_peered":"2024-01-15T18:05:59.516640+0000","last_clean":"2024-01-15T18:05:59.516640+0000","last_became_active":"2024-01-15T18:05:58.088876+0000","last_became_peered":"2024-01-15T18:05:58.088876+0000","last_unstale":"2024-01-15T18:05:59.516640+0000","last_undegraded":"2024-01-15T18:05:59.516640+0000","last_fullsized":"2024-01-15T18:05:59.516640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:05.223309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849017+0000","last_change":"2024-01-15T18:05:58.532686+0000","last_active":"2024-01-15T18:05:59.849017+0000","last_peered":"2024-01-15T18:05:59.849017+0000","last_clean":"2024-01-15T18:05:59.849017+0000","last_became_active":"2024-01-15T18:05:58.531997+0000","last_became_peered":"2024-01-15T18:05:58.531997+0000","last_unstale":"2024-01-15T18:05:59.849017+0000","last_undegraded":"2024-01-15T18:05:59.849017+0000","last_fullsized":"2024-01-15T18:05:59.849017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:39.209214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.762811+0000","last_change":"2024-01-15T18:05:58.091112+0000","last_active":"2024-01-15T18:05:59.762811+0000","last_peered":"2024-01-15T18:05:59.762811+0000","last_clean":"2024-01-15T18:05:59.762811+0000","last_became_active":"2024-01-15T18:05:58.090824+0000","last_became_peered":"2024-01-15T18:05:58.090824+0000","last_unstale":"2024-01-15T18:05:59.762811+0000","last_undegraded":"2024-01-15T18:05:59.762811+0000","last_fullsized":"2024-01-15T18:05:59.762811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:41.987219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512816+0000","last_change":"2024-01-15T18:05:58.523797+0000","last_active":"2024-01-15T18:05:59.512816+0000","last_peered":"2024-01-15T18:05:59.512816+0000","last_clean":"2024-01-15T18:05:59.512816+0000","last_became_active":"2024-01-15T18:05:58.523541+0000","last_became_peered":"2024-01-15T18:05:58.523541+0000","last_unstale":"2024-01-15T18:05:59.512816+0000","last_undegraded":"2024-01-15T18:05:59.512816+0000","last_fullsized":"2024-01-15T18:05:59.512816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:31:03.341301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153665+0000","last_change":"2024-01-15T18:05:58.090681+0000","last_active":"2024-01-15T18:06:00.153665+0000","last_peered":"2024-01-15T18:06:00.153665+0000","last_clean":"2024-01-15T18:06:00.153665+0000","last_became_active":"2024-01-15T18:05:58.090415+0000","last_became_peered":"2024-01-15T18:05:58.090415+0000","last_unstale":"2024-01-15T18:06:00.153665+0000","last_undegraded":"2024-01-15T18:06:00.153665+0000","last_fullsized":"2024-01-15T18:06:00.153665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:01.634759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763412+0000","last_change":"2024-01-15T18:05:58.527992+0000","last_active":"2024-01-15T18:05:59.763412+0000","last_peered":"2024-01-15T18:05:59.763412+0000","last_clean":"2024-01-15T18:05:59.763412+0000","last_became_active":"2024-01-15T18:05:58.527648+0000","last_became_peered":"2024-01-15T18:05:58.527648+0000","last_unstale":"2024-01-15T18:05:59.763412+0000","last_undegraded":"2024-01-15T18:05:59.763412+0000","last_fullsized":"2024-01-15T18:05:59.763412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.408307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:01.214936+0000","last_change":"2024-01-15T18:05:58.089651+0000","last_active":"2024-01-15T18:06:01.214936+0000","last_peered":"2024-01-15T18:06:01.214936+0000","last_clean":"2024-01-15T18:06:01.214936+0000","last_became_active":"2024-01-15T18:05:58.089376+0000","last_became_peered":"2024-01-15T18:05:58.089376+0000","last_unstale":"2024-01-15T18:06:01.214936+0000","last_undegraded":"2024-01-15T18:06:01.214936+0000","last_fullsized":"2024-01-15T18:06:01.214936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160192+0000","last_change":"2024-01-15T18:05:58.522824+0000","last_active":"2024-01-15T18:06:00.160192+0000","last_peered":"2024-01-15T18:06:00.160192+0000","last_clean":"2024-01-15T18:06:00.160192+0000","last_became_active":"2024-01-15T18:05:58.522453+0000","last_became_peered":"2024-01-15T18:05:58.522453+0000","last_unstale":"2024-01-15T18:06:00.160192+0000","last_undegraded":"2024-01-15T18:06:00.160192+0000","last_fullsized":"2024-01-15T18:06:00.160192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:02.645403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153418+0000","last_change":"2024-01-15T18:05:58.086488+0000","last_active":"2024-01-15T18:06:00.153418+0000","last_peered":"2024-01-15T18:06:00.153418+0000","last_clean":"2024-01-15T18:06:00.153418+0000","last_became_active":"2024-01-15T18:05:58.086252+0000","last_became_peered":"2024-01-15T18:05:58.086252+0000","last_unstale":"2024-01-15T18:06:00.153418+0000","last_undegraded":"2024-01-15T18:06:00.153418+0000","last_fullsized":"2024-01-15T18:06:00.153418+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514265+0000","last_change":"2024-01-15T18:05:58.531200+0000","last_active":"2024-01-15T18:05:59.514265+0000","last_peered":"2024-01-15T18:05:59.514265+0000","last_clean":"2024-01-15T18:05:59.514265+0000","last_became_active":"2024-01-15T18:05:58.530654+0000","last_became_peered":"2024-01-15T18:05:58.530654+0000","last_unstale":"2024-01-15T18:05:59.514265+0000","last_undegraded":"2024-01-15T18:05:59.514265+0000","last_fullsized":"2024-01-15T18:05:59.514265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:32.949718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160304+0000","last_change":"2024-01-15T18:05:58.087272+0000","last_active":"2024-01-15T18:06:00.160304+0000","last_peered":"2024-01-15T18:06:00.160304+0000","last_clean":"2024-01-15T18:06:00.160304+0000","last_became_active":"2024-01-15T18:05:58.086833+0000","last_became_peered":"2024-01-15T18:05:58.086833+0000","last_unstale":"2024-01-15T18:06:00.160304+0000","last_undegraded":"2024-01-15T18:06:00.160304+0000","last_fullsized":"2024-01-15T18:06:00.160304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:05.223309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160209+0000","last_change":"2024-01-15T18:05:58.522790+0000","last_active":"2024-01-15T18:06:00.160209+0000","last_peered":"2024-01-15T18:06:00.160209+0000","last_clean":"2024-01-15T18:06:00.160209+0000","last_became_active":"2024-01-15T18:05:58.522188+0000","last_became_peered":"2024-01-15T18:05:58.522188+0000","last_unstale":"2024-01-15T18:06:00.160209+0000","last_undegraded":"2024-01-15T18:06:00.160209+0000","last_fullsized":"2024-01-15T18:06:00.160209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:35.234818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153224+0000","last_change":"2024-01-15T18:05:58.093409+0000","last_active":"2024-01-15T18:06:00.153224+0000","last_peered":"2024-01-15T18:06:00.153224+0000","last_clean":"2024-01-15T18:06:00.153224+0000","last_became_active":"2024-01-15T18:05:58.092978+0000","last_became_peered":"2024-01-15T18:05:58.092978+0000","last_unstale":"2024-01-15T18:06:00.153224+0000","last_undegraded":"2024-01-15T18:06:00.153224+0000","last_fullsized":"2024-01-15T18:06:00.153224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246415+0000","last_change":"2024-01-15T18:05:58.517491+0000","last_active":"2024-01-15T18:06:00.246415+0000","last_peered":"2024-01-15T18:06:00.246415+0000","last_clean":"2024-01-15T18:06:00.246415+0000","last_became_active":"2024-01-15T18:05:58.517181+0000","last_became_peered":"2024-01-15T18:05:58.517181+0000","last_unstale":"2024-01-15T18:06:00.246415+0000","last_undegraded":"2024-01-15T18:06:00.246415+0000","last_fullsized":"2024-01-15T18:06:00.246415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:12:05.866938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516706+0000","last_change":"2024-01-15T18:05:58.092520+0000","last_active":"2024-01-15T18:05:59.516706+0000","last_peered":"2024-01-15T18:05:59.516706+0000","last_clean":"2024-01-15T18:05:59.516706+0000","last_became_active":"2024-01-15T18:05:58.092236+0000","last_became_peered":"2024-01-15T18:05:58.092236+0000","last_unstale":"2024-01-15T18:05:59.516706+0000","last_undegraded":"2024-01-15T18:05:59.516706+0000","last_fullsized":"2024-01-15T18:05:59.516706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516595+0000","last_change":"2024-01-15T18:05:58.523903+0000","last_active":"2024-01-15T18:05:59.516595+0000","last_peered":"2024-01-15T18:05:59.516595+0000","last_clean":"2024-01-15T18:05:59.516595+0000","last_became_active":"2024-01-15T18:05:58.523410+0000","last_became_peered":"2024-01-15T18:05:58.523410+0000","last_unstale":"2024-01-15T18:05:59.516595+0000","last_undegraded":"2024-01-15T18:05:59.516595+0000","last_fullsized":"2024-01-15T18:05:59.516595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:15.105803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246442+0000","last_change":"2024-01-15T18:05:58.101671+0000","last_active":"2024-01-15T18:06:00.246442+0000","last_peered":"2024-01-15T18:06:00.246442+0000","last_clean":"2024-01-15T18:06:00.246442+0000","last_became_active":"2024-01-15T18:05:58.101363+0000","last_became_peered":"2024-01-15T18:05:58.101363+0000","last_unstale":"2024-01-15T18:06:00.246442+0000","last_undegraded":"2024-01-15T18:06:00.246442+0000","last_fullsized":"2024-01-15T18:06:00.246442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:01.634759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849211+0000","last_change":"2024-01-15T18:05:58.531818+0000","last_active":"2024-01-15T18:05:59.849211+0000","last_peered":"2024-01-15T18:05:59.849211+0000","last_clean":"2024-01-15T18:05:59.849211+0000","last_became_active":"2024-01-15T18:05:58.531459+0000","last_became_peered":"2024-01-15T18:05:58.531459+0000","last_unstale":"2024-01-15T18:05:59.849211+0000","last_undegraded":"2024-01-15T18:05:59.849211+0000","last_fullsized":"2024-01-15T18:05:59.849211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:23.838470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514292+0000","last_change":"2024-01-15T18:05:58.090778+0000","last_active":"2024-01-15T18:05:59.514292+0000","last_peered":"2024-01-15T18:05:59.514292+0000","last_clean":"2024-01-15T18:05:59.514292+0000","last_became_active":"2024-01-15T18:05:58.087855+0000","last_became_peered":"2024-01-15T18:05:58.087855+0000","last_unstale":"2024-01-15T18:05:59.514292+0000","last_undegraded":"2024-01-15T18:05:59.514292+0000","last_fullsized":"2024-01-15T18:05:59.514292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:07.557677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-15T18:05:58.522325+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:05:58.522325+0000","last_peered":"2024-01-15T18:05:58.522325+0000","last_clean":"2024-01-15T18:05:58.522325+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:05:58.522325+0000","last_undegraded":"2024-01-15T18:05:58.522325+0000","last_fullsized":"2024-01-15T18:05:58.522325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":737050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.754}]},{"osd":1,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78800000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.85599999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.89200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.623},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.495}]},{"osd":5,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.72199999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.752}]},{"osd":1,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82299999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.621}]},{"osd":6,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.878}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.436}]},{"osd":1,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.53900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.503},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.73299999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.438},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.32600000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47099999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.379},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.68899999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.66200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.749}]},{"osd":6,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.5890000000000004,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.57999999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.8259999999999996,"5min":4.8259999999999996,"15min":4.8259999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.5599999999999996,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.7629999999999999,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4480000000000004,"5min":4.4480000000000004,"15min":4.4480000000000004},"last":0.69699999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.7309999999999999,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.754}]},{"osd":7,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":4.6120000000000001,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":736960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":8.1050000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.40600000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.436}]},{"osd":3,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":8.0579999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.746},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.622}]},{"osd":5,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":0.69399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:06:03.155 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:06:03.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.322+0000 7f409bc00640 1 Processor -- start 2024-01-15T18:06:03.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.322+0000 7f409bc00640 1 -- start start 2024-01-15T18:06:03.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f409bc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40941607c0 0x7f409406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f409bc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940cffd0 con 0x7f40940cbf50 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f409bc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940d02b0 con 0x7f40940cfb30 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f409bc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40940d1ce0 con 0x7f40941607c0 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40940cbf50 0x7f40940cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40918/0 (socket says 172.21.15.143:40918) 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099c00640 1 -- 172.21.15.143:0/4282538428 learned_addr learned my addr 172.21.15.143:0/4282538428 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099200640 1 --2- 172.21.15.143:0/4282538428 >> v2:172.21.15.204:3300/0 conn(0x7f40941607c0 0x7f409406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422046868 0 0) 0x7f40940cffd0 con 0x7f40940cbf50 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4074003530 con 0x7f40940cbf50 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142869262 0 0) 0x7f4074003530 con 0x7f40940cbf50 2024-01-15T18:06:03.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 >> v2:172.21.15.204:3300/0 conn(0x7f40941607c0 msgr2=0x7f409406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 --2- 172.21.15.143:0/4282538428 >> v2:172.21.15.204:3300/0 conn(0x7f40941607c0 0x7f409406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 >> v1:172.21.15.204:6789/0 conn(0x7f40940cfb30 legacy=0x7f409406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940cb470 con 0x7f40940cbf50 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f408c001a00 con 0x7f40940cbf50 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 >> v1:172.21.15.143:6789/0 conn(0x7f40940cbf50 legacy=0x7f40940cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074003980 0x7f4074003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740047d0 0x7f4074004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4074005190 0x7f407400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40940d1ce0 con 0x7f40740047d0 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407400c040 con 0x7f4074003980 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407400c200 con 0x7f4074005190 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093e00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740047d0 0x7f4074004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099200640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074003980 0x7f4074003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099c00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4074005190 0x7f407400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093e00640 1 -- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4074005190 msgr2=0x7f407400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093e00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4074005190 0x7f407400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093e00640 1 -- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074003980 msgr2=0x7f4074003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:03.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093e00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4074003980 0x7f4074003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4093e00640 1 -- 172.21.15.143:0/4282538428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4074003530 con 0x7f40740047d0 2024-01-15T18:06:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099c00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4074005190 0x7f407400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:03.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.326+0000 7f4099c00640 1 -- 172.21.15.143:0/4282538428 reap_dead start 2024-01-15T18:06:03.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f4093e00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740047d0 0x7f4074004c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f407c001780 tx=0x7f407c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d798cb4185181e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0052d0 con 0x7f40740047d0 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c0a7050 con 0x7f40740047d0 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f4093400640 1 -- 172.21.15.143:0/4282538428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c003600 con 0x7f40740047d0 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 -- 172.21.15.143:0/4282538428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740047d0 msgr2=0x7f4074004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740047d0 0x7f4074004c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f407c001780 tx=0x7f407c01c3e0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 -- 172.21.15.143:0/4282538428 shutdown_connections 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 --2- 172.21.15.143:0/4282538428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740047d0 0x7f4074004c50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 -- 172.21.15.143:0/4282538428 >> 172.21.15.143:0/4282538428 conn(0x7f40940d0e20 msgr2=0x7f40940d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 -- 172.21.15.143:0/4282538428 shutdown_connections 2024-01-15T18:06:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.330+0000 7f409bc00640 1 -- 172.21.15.143:0/4282538428 wait complete. 2024-01-15T18:06:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 Processor -- start 2024-01-15T18:06:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- start start 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740047d0 0x7f4094151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f409409cf40 0x7f409409d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 0x7f40940c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409409d900 con 0x7f40940c4ab0 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409409ac50 con 0x7f40740047d0 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c5820 con 0x7f409409cf40 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 0x7f40940c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 0x7f40940c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57400/0 (socket says 172.21.15.143:57400) 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099c00640 1 -- 172.21.15.143:0/1313972129 learned_addr learned my addr 172.21.15.143:0/1313972129 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740047d0 0x7f4094151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4093e00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f409409cf40 0x7f409409d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 -- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f409409cf40 msgr2=0x7f409409d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f409409cf40 0x7f409409d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 -- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 msgr2=0x7f40940c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 0x7f40940c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 -- 172.21.15.143:0/1313972129 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940c5b00 con 0x7f40740047d0 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099c00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 0x7f40940c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4099200640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740047d0 0x7f4094151660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4084011960 tx=0x7f4084011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4084004240 con 0x7f40740047d0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40840043a0 con 0x7f40740047d0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408400fa80 con 0x7f40740047d0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40940c5de0 con 0x7f40740047d0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4094151ba0 con 0x7f40740047d0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.334+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409406d4b0 con 0x7f40740047d0 2024-01-15T18:06:03.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.338+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f408400fbe0 con 0x7f40740047d0 2024-01-15T18:06:03.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.338+0000 7f4090c00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f406c0668f0 0x7f406c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.338+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4084084a50 con 0x7f40740047d0 2024-01-15T18:06:03.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.338+0000 7f4093e00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f406c0668f0 0x7f406c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.338+0000 7f4093e00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f406c0668f0 0x7f406c068db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f407c001750 tx=0x7f407c0011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.342+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4084051d30 con 0x7f40740047d0 2024-01-15T18:06:03.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.490+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f409406d190 con 0x7f406c0668f0 2024-01-15T18:06:03.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.498+0000 7f4090c00640 1 -- 172.21.15.143:0/1313972129 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289058 (secure 0 0 0) 0x7f409406d190 con 0x7f406c0668f0 2024-01-15T18:06:03.508 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:03.510 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:06:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f406c0668f0 msgr2=0x7f406c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f406c0668f0 0x7f406c068db0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f407c001750 tx=0x7f407c0011a0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740047d0 msgr2=0x7f4094151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740047d0 0x7f4094151660 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4084011960 tx=0x7f4084011e20 comp rx=0 tx=0).stop 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 shutdown_connections 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f406c0668f0 0x7f406c068db0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40940c4ab0 0x7f40940c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f409409cf40 0x7f409409d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 --2- 172.21.15.143:0/1313972129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740047d0 0x7f4094151660 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 >> 172.21.15.143:0/1313972129 conn(0x7f40940d0e20 msgr2=0x7f409415fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 shutdown_connections 2024-01-15T18:06:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.502+0000 7f409bc00640 1 -- 172.21.15.143:0/1313972129 wait complete. 2024-01-15T18:06:03.530 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-01-15T18:06:02.634299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220312,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754408,"statfs":{"total":773094113280,"available":772868513792,"internally_reserved":0,"allocated":4235264,"data_stored":2358292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570623"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514473+0000","last_change":"2024-01-15T18:05:58.526638+0000","last_active":"2024-01-15T18:05:59.514473+0000","last_peered":"2024-01-15T18:05:59.514473+0000","last_clean":"2024-01-15T18:05:59.514473+0000","last_became_active":"2024-01-15T18:05:58.526147+0000","last_became_peered":"2024-01-15T18:05:58.526147+0000","last_unstale":"2024-01-15T18:05:59.514473+0000","last_undegraded":"2024-01-15T18:05:59.514473+0000","last_fullsized":"2024-01-15T18:05:59.514473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:18.657632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246936+0000","last_change":"2024-01-15T18:05:58.524452+0000","last_active":"2024-01-15T18:06:00.246936+0000","last_peered":"2024-01-15T18:06:00.246936+0000","last_clean":"2024-01-15T18:06:00.246936+0000","last_became_active":"2024-01-15T18:05:58.524128+0000","last_became_peered":"2024-01-15T18:05:58.524128+0000","last_unstale":"2024-01-15T18:06:00.246936+0000","last_undegraded":"2024-01-15T18:06:00.246936+0000","last_fullsized":"2024-01-15T18:06:00.246936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:23.838470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763385+0000","last_change":"2024-01-15T18:05:58.523682+0000","last_active":"2024-01-15T18:05:59.763385+0000","last_peered":"2024-01-15T18:05:59.763385+0000","last_clean":"2024-01-15T18:05:59.763385+0000","last_became_active":"2024-01-15T18:05:58.523394+0000","last_became_peered":"2024-01-15T18:05:58.523394+0000","last_unstale":"2024-01-15T18:05:59.763385+0000","last_undegraded":"2024-01-15T18:05:59.763385+0000","last_fullsized":"2024-01-15T18:05:59.763385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:23.838470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153345+0000","last_change":"2024-01-15T18:05:58.533697+0000","last_active":"2024-01-15T18:06:00.153345+0000","last_peered":"2024-01-15T18:06:00.153345+0000","last_clean":"2024-01-15T18:06:00.153345+0000","last_became_active":"2024-01-15T18:05:58.533394+0000","last_became_peered":"2024-01-15T18:05:58.533394+0000","last_unstale":"2024-01-15T18:06:00.153345+0000","last_undegraded":"2024-01-15T18:06:00.153345+0000","last_fullsized":"2024-01-15T18:06:00.153345+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:49.554679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512686+0000","last_change":"2024-01-15T18:05:58.519538+0000","last_active":"2024-01-15T18:05:59.512686+0000","last_peered":"2024-01-15T18:05:59.512686+0000","last_clean":"2024-01-15T18:05:59.512686+0000","last_became_active":"2024-01-15T18:05:58.519262+0000","last_became_peered":"2024-01-15T18:05:58.519262+0000","last_unstale":"2024-01-15T18:05:59.512686+0000","last_undegraded":"2024-01-15T18:05:59.512686+0000","last_fullsized":"2024-01-15T18:05:59.512686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:11.241360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246752+0000","last_change":"2024-01-15T18:05:58.524414+0000","last_active":"2024-01-15T18:06:00.246752+0000","last_peered":"2024-01-15T18:06:00.246752+0000","last_clean":"2024-01-15T18:06:00.246752+0000","last_became_active":"2024-01-15T18:05:58.524128+0000","last_became_peered":"2024-01-15T18:05:58.524128+0000","last_unstale":"2024-01-15T18:06:00.246752+0000","last_undegraded":"2024-01-15T18:06:00.246752+0000","last_fullsized":"2024-01-15T18:06:00.246752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:16:20.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849325+0000","last_change":"2024-01-15T18:05:58.532952+0000","last_active":"2024-01-15T18:05:59.849325+0000","last_peered":"2024-01-15T18:05:59.849325+0000","last_clean":"2024-01-15T18:05:59.849325+0000","last_became_active":"2024-01-15T18:05:58.532408+0000","last_became_peered":"2024-01-15T18:05:58.532408+0000","last_unstale":"2024-01-15T18:05:59.849325+0000","last_undegraded":"2024-01-15T18:05:59.849325+0000","last_fullsized":"2024-01-15T18:05:59.849325+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.408307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.848774+0000","last_change":"2024-01-15T18:05:58.533350+0000","last_active":"2024-01-15T18:05:59.848774+0000","last_peered":"2024-01-15T18:05:59.848774+0000","last_clean":"2024-01-15T18:05:59.848774+0000","last_became_active":"2024-01-15T18:05:58.533043+0000","last_became_peered":"2024-01-15T18:05:58.533043+0000","last_unstale":"2024-01-15T18:05:59.848774+0000","last_undegraded":"2024-01-15T18:05:59.848774+0000","last_fullsized":"2024-01-15T18:05:59.848774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:18.832803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517314+0000","last_change":"2024-01-15T18:05:58.529404+0000","last_active":"2024-01-15T18:05:59.517314+0000","last_peered":"2024-01-15T18:05:59.517314+0000","last_clean":"2024-01-15T18:05:59.517314+0000","last_became_active":"2024-01-15T18:05:58.529051+0000","last_became_peered":"2024-01-15T18:05:58.529051+0000","last_unstale":"2024-01-15T18:05:59.517314+0000","last_undegraded":"2024-01-15T18:05:59.517314+0000","last_fullsized":"2024-01-15T18:05:59.517314+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:49.554679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153500+0000","last_change":"2024-01-15T18:05:58.526133+0000","last_active":"2024-01-15T18:06:00.153500+0000","last_peered":"2024-01-15T18:06:00.153500+0000","last_clean":"2024-01-15T18:06:00.153500+0000","last_became_active":"2024-01-15T18:05:58.525887+0000","last_became_peered":"2024-01-15T18:05:58.525887+0000","last_unstale":"2024-01-15T18:06:00.153500+0000","last_undegraded":"2024-01-15T18:06:00.153500+0000","last_fullsized":"2024-01-15T18:06:00.153500+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:42.997863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849163+0000","last_change":"2024-01-15T18:05:58.530013+0000","last_active":"2024-01-15T18:05:59.849163+0000","last_peered":"2024-01-15T18:05:59.849163+0000","last_clean":"2024-01-15T18:05:59.849163+0000","last_became_active":"2024-01-15T18:05:58.529712+0000","last_became_peered":"2024-01-15T18:05:58.529712+0000","last_unstale":"2024-01-15T18:05:59.849163+0000","last_undegraded":"2024-01-15T18:05:59.849163+0000","last_fullsized":"2024-01-15T18:05:59.849163+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:32.228628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763223+0000","last_change":"2024-01-15T18:05:58.519538+0000","last_active":"2024-01-15T18:05:59.763223+0000","last_peered":"2024-01-15T18:05:59.763223+0000","last_clean":"2024-01-15T18:05:59.763223+0000","last_became_active":"2024-01-15T18:05:58.519273+0000","last_became_peered":"2024-01-15T18:05:58.519273+0000","last_unstale":"2024-01-15T18:05:59.763223+0000","last_undegraded":"2024-01-15T18:05:59.763223+0000","last_fullsized":"2024-01-15T18:05:59.763223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:16:20.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512722+0000","last_change":"2024-01-15T18:05:58.517201+0000","last_active":"2024-01-15T18:05:59.512722+0000","last_peered":"2024-01-15T18:05:59.512722+0000","last_clean":"2024-01-15T18:05:59.512722+0000","last_became_active":"2024-01-15T18:05:58.516934+0000","last_became_peered":"2024-01-15T18:05:58.516934+0000","last_unstale":"2024-01-15T18:05:59.512722+0000","last_undegraded":"2024-01-15T18:05:59.512722+0000","last_fullsized":"2024-01-15T18:05:59.512722+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:55.653021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246721+0000","last_change":"2024-01-15T18:05:58.518928+0000","last_active":"2024-01-15T18:06:00.246721+0000","last_peered":"2024-01-15T18:06:00.246721+0000","last_clean":"2024-01-15T18:06:00.246721+0000","last_became_active":"2024-01-15T18:05:58.518782+0000","last_became_peered":"2024-01-15T18:05:58.518782+0000","last_unstale":"2024-01-15T18:06:00.246721+0000","last_undegraded":"2024-01-15T18:06:00.246721+0000","last_fullsized":"2024-01-15T18:06:00.246721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:39.209214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763181+0000","last_change":"2024-01-15T18:05:58.529081+0000","last_active":"2024-01-15T18:05:59.763181+0000","last_peered":"2024-01-15T18:05:59.763181+0000","last_clean":"2024-01-15T18:05:59.763181+0000","last_became_active":"2024-01-15T18:05:58.528798+0000","last_became_peered":"2024-01-15T18:05:58.528798+0000","last_unstale":"2024-01-15T18:05:59.763181+0000","last_undegraded":"2024-01-15T18:05:59.763181+0000","last_fullsized":"2024-01-15T18:05:59.763181+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:39.209214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849269+0000","last_change":"2024-01-15T18:05:58.535727+0000","last_active":"2024-01-15T18:05:59.849269+0000","last_peered":"2024-01-15T18:05:59.849269+0000","last_clean":"2024-01-15T18:05:59.849269+0000","last_became_active":"2024-01-15T18:05:58.534827+0000","last_became_peered":"2024-01-15T18:05:58.534827+0000","last_unstale":"2024-01-15T18:05:59.849269+0000","last_undegraded":"2024-01-15T18:05:59.849269+0000","last_fullsized":"2024-01-15T18:05:59.849269+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:05.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849361+0000","last_change":"2024-01-15T18:05:58.535784+0000","last_active":"2024-01-15T18:05:59.849361+0000","last_peered":"2024-01-15T18:05:59.849361+0000","last_clean":"2024-01-15T18:05:59.849361+0000","last_became_active":"2024-01-15T18:05:58.535387+0000","last_became_peered":"2024-01-15T18:05:58.535387+0000","last_unstale":"2024-01-15T18:05:59.849361+0000","last_undegraded":"2024-01-15T18:05:59.849361+0000","last_fullsized":"2024-01-15T18:05:59.849361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:21.429830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246983+0000","last_change":"2024-01-15T18:05:58.522105+0000","last_active":"2024-01-15T18:06:00.246983+0000","last_peered":"2024-01-15T18:06:00.246983+0000","last_clean":"2024-01-15T18:06:00.246983+0000","last_became_active":"2024-01-15T18:05:58.521810+0000","last_became_peered":"2024-01-15T18:05:58.521810+0000","last_unstale":"2024-01-15T18:06:00.246983+0000","last_undegraded":"2024-01-15T18:06:00.246983+0000","last_fullsized":"2024-01-15T18:06:00.246983+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:21.429830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246783+0000","last_change":"2024-01-15T18:05:58.527401+0000","last_active":"2024-01-15T18:06:00.246783+0000","last_peered":"2024-01-15T18:06:00.246783+0000","last_clean":"2024-01-15T18:06:00.246783+0000","last_became_active":"2024-01-15T18:05:58.527115+0000","last_became_peered":"2024-01-15T18:05:58.527115+0000","last_unstale":"2024-01-15T18:06:00.246783+0000","last_undegraded":"2024-01-15T18:06:00.246783+0000","last_fullsized":"2024-01-15T18:06:00.246783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:56.364492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512889+0000","last_change":"2024-01-15T18:05:58.523443+0000","last_active":"2024-01-15T18:05:59.512889+0000","last_peered":"2024-01-15T18:05:59.512889+0000","last_clean":"2024-01-15T18:05:59.512889+0000","last_became_active":"2024-01-15T18:05:58.523167+0000","last_became_peered":"2024-01-15T18:05:58.523167+0000","last_unstale":"2024-01-15T18:05:59.512889+0000","last_undegraded":"2024-01-15T18:05:59.512889+0000","last_fullsized":"2024-01-15T18:05:59.512889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:44.949108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516699+0000","last_change":"2024-01-15T18:05:58.529091+0000","last_active":"2024-01-15T18:05:59.516699+0000","last_peered":"2024-01-15T18:05:59.516699+0000","last_clean":"2024-01-15T18:05:59.516699+0000","last_became_active":"2024-01-15T18:05:58.528793+0000","last_became_peered":"2024-01-15T18:05:58.528793+0000","last_unstale":"2024-01-15T18:05:59.516699+0000","last_undegraded":"2024-01-15T18:05:59.516699+0000","last_fullsized":"2024-01-15T18:05:59.516699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:18.657632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517104+0000","last_change":"2024-01-15T18:05:58.519531+0000","last_active":"2024-01-15T18:05:59.517104+0000","last_peered":"2024-01-15T18:05:59.517104+0000","last_clean":"2024-01-15T18:05:59.517104+0000","last_became_active":"2024-01-15T18:05:58.519186+0000","last_became_peered":"2024-01-15T18:05:58.519186+0000","last_unstale":"2024-01-15T18:05:59.517104+0000","last_undegraded":"2024-01-15T18:05:59.517104+0000","last_fullsized":"2024-01-15T18:05:59.517104+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:44.324719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160057+0000","last_change":"2024-01-15T18:05:58.526896+0000","last_active":"2024-01-15T18:06:00.160057+0000","last_peered":"2024-01-15T18:06:00.160057+0000","last_clean":"2024-01-15T18:06:00.160057+0000","last_became_active":"2024-01-15T18:05:58.526636+0000","last_became_peered":"2024-01-15T18:05:58.526636+0000","last_unstale":"2024-01-15T18:06:00.160057+0000","last_undegraded":"2024-01-15T18:06:00.160057+0000","last_fullsized":"2024-01-15T18:06:00.160057+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:18.657632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.762914+0000","last_change":"2024-01-15T18:05:58.519931+0000","last_active":"2024-01-15T18:05:59.762914+0000","last_peered":"2024-01-15T18:05:59.762914+0000","last_clean":"2024-01-15T18:05:59.762914+0000","last_became_active":"2024-01-15T18:05:58.519034+0000","last_became_peered":"2024-01-15T18:05:58.519034+0000","last_unstale":"2024-01-15T18:05:59.762914+0000","last_undegraded":"2024-01-15T18:05:59.762914+0000","last_fullsized":"2024-01-15T18:05:59.762914+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:42:53.369765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153606+0000","last_change":"2024-01-15T18:05:58.530043+0000","last_active":"2024-01-15T18:06:00.153606+0000","last_peered":"2024-01-15T18:06:00.153606+0000","last_clean":"2024-01-15T18:06:00.153606+0000","last_became_active":"2024-01-15T18:05:58.529742+0000","last_became_peered":"2024-01-15T18:05:58.529742+0000","last_unstale":"2024-01-15T18:06:00.153606+0000","last_undegraded":"2024-01-15T18:06:00.153606+0000","last_fullsized":"2024-01-15T18:06:00.153606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:18.832803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.247003+0000","last_change":"2024-01-15T18:05:58.530430+0000","last_active":"2024-01-15T18:06:00.247003+0000","last_peered":"2024-01-15T18:06:00.247003+0000","last_clean":"2024-01-15T18:06:00.247003+0000","last_became_active":"2024-01-15T18:05:58.530142+0000","last_became_peered":"2024-01-15T18:05:58.530142+0000","last_unstale":"2024-01-15T18:06:00.247003+0000","last_undegraded":"2024-01-15T18:06:00.247003+0000","last_fullsized":"2024-01-15T18:06:00.247003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:58:02.687786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153492+0000","last_change":"2024-01-15T18:05:58.531848+0000","last_active":"2024-01-15T18:06:00.153492+0000","last_peered":"2024-01-15T18:06:00.153492+0000","last_clean":"2024-01-15T18:06:00.153492+0000","last_became_active":"2024-01-15T18:05:58.531580+0000","last_became_peered":"2024-01-15T18:05:58.531580+0000","last_unstale":"2024-01-15T18:06:00.153492+0000","last_undegraded":"2024-01-15T18:06:00.153492+0000","last_fullsized":"2024-01-15T18:06:00.153492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:13.220702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763222+0000","last_change":"2024-01-15T18:05:58.525064+0000","last_active":"2024-01-15T18:05:59.763222+0000","last_peered":"2024-01-15T18:05:59.763222+0000","last_clean":"2024-01-15T18:05:59.763222+0000","last_became_active":"2024-01-15T18:05:58.524751+0000","last_became_peered":"2024-01-15T18:05:58.524751+0000","last_unstale":"2024-01-15T18:05:59.763222+0000","last_undegraded":"2024-01-15T18:05:59.763222+0000","last_fullsized":"2024-01-15T18:05:59.763222+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:56.364492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516768+0000","last_change":"2024-01-15T18:05:58.528901+0000","last_active":"2024-01-15T18:05:59.516768+0000","last_peered":"2024-01-15T18:05:59.516768+0000","last_clean":"2024-01-15T18:05:59.516768+0000","last_became_active":"2024-01-15T18:05:58.528351+0000","last_became_peered":"2024-01-15T18:05:58.528351+0000","last_unstale":"2024-01-15T18:05:59.516768+0000","last_undegraded":"2024-01-15T18:05:59.516768+0000","last_fullsized":"2024-01-15T18:05:59.516768+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:35.234818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.848915+0000","last_change":"2024-01-15T18:05:58.535973+0000","last_active":"2024-01-15T18:05:59.848915+0000","last_peered":"2024-01-15T18:05:59.848915+0000","last_clean":"2024-01-15T18:05:59.848915+0000","last_became_active":"2024-01-15T18:05:58.535687+0000","last_became_peered":"2024-01-15T18:05:58.535687+0000","last_unstale":"2024-01-15T18:05:59.848915+0000","last_undegraded":"2024-01-15T18:05:59.848915+0000","last_fullsized":"2024-01-15T18:05:59.848915+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:12:05.866938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514252+0000","last_change":"2024-01-15T18:05:58.527076+0000","last_active":"2024-01-15T18:05:59.514252+0000","last_peered":"2024-01-15T18:05:59.514252+0000","last_clean":"2024-01-15T18:05:59.514252+0000","last_became_active":"2024-01-15T18:05:58.524498+0000","last_became_peered":"2024-01-15T18:05:58.524498+0000","last_unstale":"2024-01-15T18:05:59.514252+0000","last_undegraded":"2024-01-15T18:05:59.514252+0000","last_fullsized":"2024-01-15T18:05:59.514252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:23.825492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.848831+0000","last_change":"2024-01-15T18:05:58.538112+0000","last_active":"2024-01-15T18:05:59.848831+0000","last_peered":"2024-01-15T18:05:59.848831+0000","last_clean":"2024-01-15T18:05:59.848831+0000","last_became_active":"2024-01-15T18:05:58.537774+0000","last_became_peered":"2024-01-15T18:05:58.537774+0000","last_unstale":"2024-01-15T18:05:59.848831+0000","last_undegraded":"2024-01-15T18:05:59.848831+0000","last_fullsized":"2024-01-15T18:05:59.848831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:42:53.369765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160335+0000","last_change":"2024-01-15T18:05:58.528250+0000","last_active":"2024-01-15T18:06:00.160335+0000","last_peered":"2024-01-15T18:06:00.160335+0000","last_clean":"2024-01-15T18:06:00.160335+0000","last_became_active":"2024-01-15T18:05:58.527996+0000","last_became_peered":"2024-01-15T18:05:58.527996+0000","last_unstale":"2024-01-15T18:06:00.160335+0000","last_undegraded":"2024-01-15T18:06:00.160335+0000","last_fullsized":"2024-01-15T18:06:00.160335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:44.324719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514566+0000","last_change":"2024-01-15T18:05:58.090063+0000","last_active":"2024-01-15T18:05:59.514566+0000","last_peered":"2024-01-15T18:05:59.514566+0000","last_clean":"2024-01-15T18:05:59.514566+0000","last_became_active":"2024-01-15T18:05:58.089758+0000","last_became_peered":"2024-01-15T18:05:58.089758+0000","last_unstale":"2024-01-15T18:05:59.514566+0000","last_undegraded":"2024-01-15T18:05:59.514566+0000","last_fullsized":"2024-01-15T18:05:59.514566+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:05.223309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514532+0000","last_change":"2024-01-15T18:05:58.526406+0000","last_active":"2024-01-15T18:05:59.514532+0000","last_peered":"2024-01-15T18:05:59.514532+0000","last_clean":"2024-01-15T18:05:59.514532+0000","last_became_active":"2024-01-15T18:05:58.525812+0000","last_became_peered":"2024-01-15T18:05:58.525812+0000","last_unstale":"2024-01-15T18:05:59.514532+0000","last_undegraded":"2024-01-15T18:05:59.514532+0000","last_fullsized":"2024-01-15T18:05:59.514532+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:02.645403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160322+0000","last_change":"2024-01-15T18:05:58.090182+0000","last_active":"2024-01-15T18:06:00.160322+0000","last_peered":"2024-01-15T18:06:00.160322+0000","last_clean":"2024-01-15T18:06:00.160322+0000","last_became_active":"2024-01-15T18:05:58.089936+0000","last_became_peered":"2024-01-15T18:05:58.089936+0000","last_unstale":"2024-01-15T18:06:00.160322+0000","last_undegraded":"2024-01-15T18:06:00.160322+0000","last_fullsized":"2024-01-15T18:06:00.160322+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517108+0000","last_change":"2024-01-15T18:05:58.527205+0000","last_active":"2024-01-15T18:05:59.517108+0000","last_peered":"2024-01-15T18:05:59.517108+0000","last_clean":"2024-01-15T18:05:59.517108+0000","last_became_active":"2024-01-15T18:05:58.526930+0000","last_became_peered":"2024-01-15T18:05:58.526930+0000","last_unstale":"2024-01-15T18:05:59.517108+0000","last_undegraded":"2024-01-15T18:05:59.517108+0000","last_fullsized":"2024-01-15T18:05:59.517108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:44:07.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246815+0000","last_change":"2024-01-15T18:05:58.097847+0000","last_active":"2024-01-15T18:06:00.246815+0000","last_peered":"2024-01-15T18:06:00.246815+0000","last_clean":"2024-01-15T18:06:00.246815+0000","last_became_active":"2024-01-15T18:05:58.095203+0000","last_became_peered":"2024-01-15T18:05:58.095203+0000","last_unstale":"2024-01-15T18:06:00.246815+0000","last_undegraded":"2024-01-15T18:06:00.246815+0000","last_fullsized":"2024-01-15T18:06:00.246815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:48.544035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153510+0000","last_change":"2024-01-15T18:05:58.531973+0000","last_active":"2024-01-15T18:06:00.153510+0000","last_peered":"2024-01-15T18:06:00.153510+0000","last_clean":"2024-01-15T18:06:00.153510+0000","last_became_active":"2024-01-15T18:05:58.531673+0000","last_became_peered":"2024-01-15T18:05:58.531673+0000","last_unstale":"2024-01-15T18:06:00.153510+0000","last_undegraded":"2024-01-15T18:06:00.153510+0000","last_fullsized":"2024-01-15T18:06:00.153510+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:07:00.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512800+0000","last_change":"2024-01-15T18:05:58.091581+0000","last_active":"2024-01-15T18:05:59.512800+0000","last_peered":"2024-01-15T18:05:59.512800+0000","last_clean":"2024-01-15T18:05:59.512800+0000","last_became_active":"2024-01-15T18:05:58.090297+0000","last_became_peered":"2024-01-15T18:05:58.090297+0000","last_unstale":"2024-01-15T18:05:59.512800+0000","last_undegraded":"2024-01-15T18:05:59.512800+0000","last_fullsized":"2024-01-15T18:05:59.512800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:14:47.548551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849141+0000","last_change":"2024-01-15T18:05:58.534773+0000","last_active":"2024-01-15T18:05:59.849141+0000","last_peered":"2024-01-15T18:05:59.849141+0000","last_clean":"2024-01-15T18:05:59.849141+0000","last_became_active":"2024-01-15T18:05:58.534430+0000","last_became_peered":"2024-01-15T18:05:58.534430+0000","last_unstale":"2024-01-15T18:05:59.849141+0000","last_undegraded":"2024-01-15T18:05:59.849141+0000","last_fullsized":"2024-01-15T18:05:59.849141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:56.364492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516827+0000","last_change":"2024-01-15T18:05:58.101502+0000","last_active":"2024-01-15T18:05:59.516827+0000","last_peered":"2024-01-15T18:05:59.516827+0000","last_clean":"2024-01-15T18:05:59.516827+0000","last_became_active":"2024-01-15T18:05:58.101169+0000","last_became_peered":"2024-01-15T18:05:58.101169+0000","last_unstale":"2024-01-15T18:05:59.516827+0000","last_undegraded":"2024-01-15T18:05:59.516827+0000","last_fullsized":"2024-01-15T18:05:59.516827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:54.642377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763054+0000","last_change":"2024-01-15T18:05:58.526495+0000","last_active":"2024-01-15T18:05:59.763054+0000","last_peered":"2024-01-15T18:05:59.763054+0000","last_clean":"2024-01-15T18:05:59.763054+0000","last_became_active":"2024-01-15T18:05:58.526160+0000","last_became_peered":"2024-01-15T18:05:58.526160+0000","last_unstale":"2024-01-15T18:05:59.763054+0000","last_undegraded":"2024-01-15T18:05:59.763054+0000","last_fullsized":"2024-01-15T18:05:59.763054+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:12:05.866938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517049+0000","last_change":"2024-01-15T18:05:58.101507+0000","last_active":"2024-01-15T18:05:59.517049+0000","last_peered":"2024-01-15T18:05:59.517049+0000","last_clean":"2024-01-15T18:05:59.517049+0000","last_became_active":"2024-01-15T18:05:58.101125+0000","last_became_peered":"2024-01-15T18:05:58.101125+0000","last_unstale":"2024-01-15T18:05:59.517049+0000","last_undegraded":"2024-01-15T18:05:59.517049+0000","last_fullsized":"2024-01-15T18:05:59.517049+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:22.814848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516733+0000","last_change":"2024-01-15T18:05:58.525816+0000","last_active":"2024-01-15T18:05:59.516733+0000","last_peered":"2024-01-15T18:05:59.516733+0000","last_clean":"2024-01-15T18:05:59.516733+0000","last_became_active":"2024-01-15T18:05:58.525508+0000","last_became_peered":"2024-01-15T18:05:58.525508+0000","last_unstale":"2024-01-15T18:05:59.516733+0000","last_undegraded":"2024-01-15T18:05:59.516733+0000","last_fullsized":"2024-01-15T18:05:59.516733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:02.645403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246524+0000","last_change":"2024-01-15T18:05:58.094847+0000","last_active":"2024-01-15T18:06:00.246524+0000","last_peered":"2024-01-15T18:06:00.246524+0000","last_clean":"2024-01-15T18:06:00.246524+0000","last_became_active":"2024-01-15T18:05:58.094443+0000","last_became_peered":"2024-01-15T18:05:58.094443+0000","last_unstale":"2024-01-15T18:06:00.246524+0000","last_undegraded":"2024-01-15T18:06:00.246524+0000","last_fullsized":"2024-01-15T18:06:00.246524+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153134+0000","last_change":"2024-01-15T18:05:58.525223+0000","last_active":"2024-01-15T18:06:00.153134+0000","last_peered":"2024-01-15T18:06:00.153134+0000","last_clean":"2024-01-15T18:06:00.153134+0000","last_became_active":"2024-01-15T18:05:58.524942+0000","last_became_peered":"2024-01-15T18:05:58.524942+0000","last_unstale":"2024-01-15T18:06:00.153134+0000","last_undegraded":"2024-01-15T18:06:00.153134+0000","last_fullsized":"2024-01-15T18:06:00.153134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:52:49.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516663+0000","last_change":"2024-01-15T18:05:58.089206+0000","last_active":"2024-01-15T18:05:59.516663+0000","last_peered":"2024-01-15T18:05:59.516663+0000","last_clean":"2024-01-15T18:05:59.516663+0000","last_became_active":"2024-01-15T18:05:58.088797+0000","last_became_peered":"2024-01-15T18:05:58.088797+0000","last_unstale":"2024-01-15T18:05:59.516663+0000","last_undegraded":"2024-01-15T18:05:59.516663+0000","last_fullsized":"2024-01-15T18:05:59.516663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:07.557677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763466+0000","last_change":"2024-01-15T18:05:58.531213+0000","last_active":"2024-01-15T18:05:59.763466+0000","last_peered":"2024-01-15T18:05:59.763466+0000","last_clean":"2024-01-15T18:05:59.763466+0000","last_became_active":"2024-01-15T18:05:58.530956+0000","last_became_peered":"2024-01-15T18:05:58.530956+0000","last_unstale":"2024-01-15T18:05:59.763466+0000","last_undegraded":"2024-01-15T18:05:59.763466+0000","last_fullsized":"2024-01-15T18:05:59.763466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:21.429830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.247026+0000","last_change":"2024-01-15T18:05:58.099183+0000","last_active":"2024-01-15T18:06:00.247026+0000","last_peered":"2024-01-15T18:06:00.247026+0000","last_clean":"2024-01-15T18:06:00.247026+0000","last_became_active":"2024-01-15T18:05:58.098974+0000","last_became_peered":"2024-01-15T18:05:58.098974+0000","last_unstale":"2024-01-15T18:06:00.247026+0000","last_undegraded":"2024-01-15T18:06:00.247026+0000","last_fullsized":"2024-01-15T18:06:00.247026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:12.210058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246955+0000","last_change":"2024-01-15T18:05:58.525755+0000","last_active":"2024-01-15T18:06:00.246955+0000","last_peered":"2024-01-15T18:06:00.246955+0000","last_clean":"2024-01-15T18:06:00.246955+0000","last_became_active":"2024-01-15T18:05:58.525537+0000","last_became_peered":"2024-01-15T18:05:58.525537+0000","last_unstale":"2024-01-15T18:06:00.246955+0000","last_undegraded":"2024-01-15T18:06:00.246955+0000","last_fullsized":"2024-01-15T18:06:00.246955+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:05.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160424+0000","last_change":"2024-01-15T18:05:58.100616+0000","last_active":"2024-01-15T18:06:00.160424+0000","last_peered":"2024-01-15T18:06:00.160424+0000","last_clean":"2024-01-15T18:06:00.160424+0000","last_became_active":"2024-01-15T18:05:58.100152+0000","last_became_peered":"2024-01-15T18:05:58.100152+0000","last_unstale":"2024-01-15T18:06:00.160424+0000","last_undegraded":"2024-01-15T18:06:00.160424+0000","last_fullsized":"2024-01-15T18:06:00.160424+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:22.814848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246965+0000","last_change":"2024-01-15T18:05:58.533034+0000","last_active":"2024-01-15T18:06:00.246965+0000","last_peered":"2024-01-15T18:06:00.246965+0000","last_clean":"2024-01-15T18:06:00.246965+0000","last_became_active":"2024-01-15T18:05:58.532600+0000","last_became_peered":"2024-01-15T18:05:58.532600+0000","last_unstale":"2024-01-15T18:06:00.246965+0000","last_undegraded":"2024-01-15T18:06:00.246965+0000","last_fullsized":"2024-01-15T18:06:00.246965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.408307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763345+0000","last_change":"2024-01-15T18:05:58.093705+0000","last_active":"2024-01-15T18:05:59.763345+0000","last_peered":"2024-01-15T18:05:59.763345+0000","last_clean":"2024-01-15T18:05:59.763345+0000","last_became_active":"2024-01-15T18:05:58.093273+0000","last_became_peered":"2024-01-15T18:05:58.093273+0000","last_unstale":"2024-01-15T18:05:59.763345+0000","last_undegraded":"2024-01-15T18:05:59.763345+0000","last_fullsized":"2024-01-15T18:05:59.763345+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.833990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763259+0000","last_change":"2024-01-15T18:05:58.527097+0000","last_active":"2024-01-15T18:05:59.763259+0000","last_peered":"2024-01-15T18:05:59.763259+0000","last_clean":"2024-01-15T18:05:59.763259+0000","last_became_active":"2024-01-15T18:05:58.526332+0000","last_became_peered":"2024-01-15T18:05:58.526332+0000","last_unstale":"2024-01-15T18:05:59.763259+0000","last_undegraded":"2024-01-15T18:05:59.763259+0000","last_fullsized":"2024-01-15T18:05:59.763259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:32.228628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246917+0000","last_change":"2024-01-15T18:05:58.095806+0000","last_active":"2024-01-15T18:06:00.246917+0000","last_peered":"2024-01-15T18:06:00.246917+0000","last_clean":"2024-01-15T18:06:00.246917+0000","last_became_active":"2024-01-15T18:05:58.095542+0000","last_became_peered":"2024-01-15T18:05:58.095542+0000","last_unstale":"2024-01-15T18:06:00.246917+0000","last_undegraded":"2024-01-15T18:06:00.246917+0000","last_fullsized":"2024-01-15T18:06:00.246917+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:17.822159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153116+0000","last_change":"2024-01-15T18:05:58.532494+0000","last_active":"2024-01-15T18:06:00.153116+0000","last_peered":"2024-01-15T18:06:00.153116+0000","last_clean":"2024-01-15T18:06:00.153116+0000","last_became_active":"2024-01-15T18:05:58.530694+0000","last_became_peered":"2024-01-15T18:05:58.530694+0000","last_unstale":"2024-01-15T18:06:00.153116+0000","last_undegraded":"2024-01-15T18:06:00.153116+0000","last_fullsized":"2024-01-15T18:06:00.153116+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:44:07.926148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516869+0000","last_change":"2024-01-15T18:05:58.089048+0000","last_active":"2024-01-15T18:05:59.516869+0000","last_peered":"2024-01-15T18:05:59.516869+0000","last_clean":"2024-01-15T18:05:59.516869+0000","last_became_active":"2024-01-15T18:05:58.088713+0000","last_became_peered":"2024-01-15T18:05:58.088713+0000","last_unstale":"2024-01-15T18:05:59.516869+0000","last_undegraded":"2024-01-15T18:05:59.516869+0000","last_fullsized":"2024-01-15T18:05:59.516869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:54.029185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246893+0000","last_change":"2024-01-15T18:05:58.525053+0000","last_active":"2024-01-15T18:06:00.246893+0000","last_peered":"2024-01-15T18:06:00.246893+0000","last_clean":"2024-01-15T18:06:00.246893+0000","last_became_active":"2024-01-15T18:05:58.524703+0000","last_became_peered":"2024-01-15T18:05:58.524703+0000","last_unstale":"2024-01-15T18:06:00.246893+0000","last_undegraded":"2024-01-15T18:06:00.246893+0000","last_fullsized":"2024-01-15T18:06:00.246893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:32.228628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153136+0000","last_change":"2024-01-15T18:05:58.088554+0000","last_active":"2024-01-15T18:06:00.153136+0000","last_peered":"2024-01-15T18:06:00.153136+0000","last_clean":"2024-01-15T18:06:00.153136+0000","last_became_active":"2024-01-15T18:05:58.088258+0000","last_became_peered":"2024-01-15T18:05:58.088258+0000","last_unstale":"2024-01-15T18:06:00.153136+0000","last_undegraded":"2024-01-15T18:06:00.153136+0000","last_fullsized":"2024-01-15T18:06:00.153136+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849105+0000","last_change":"2024-01-15T18:05:58.535158+0000","last_active":"2024-01-15T18:05:59.849105+0000","last_peered":"2024-01-15T18:05:59.849105+0000","last_clean":"2024-01-15T18:05:59.849105+0000","last_became_active":"2024-01-15T18:05:58.534876+0000","last_became_peered":"2024-01-15T18:05:58.534876+0000","last_unstale":"2024-01-15T18:05:59.849105+0000","last_undegraded":"2024-01-15T18:05:59.849105+0000","last_fullsized":"2024-01-15T18:05:59.849105+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:16:20.186827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246456+0000","last_change":"2024-01-15T18:05:58.100011+0000","last_active":"2024-01-15T18:06:00.246456+0000","last_peered":"2024-01-15T18:06:00.246456+0000","last_clean":"2024-01-15T18:06:00.246456+0000","last_became_active":"2024-01-15T18:05:58.099711+0000","last_became_peered":"2024-01-15T18:05:58.099711+0000","last_unstale":"2024-01-15T18:06:00.246456+0000","last_undegraded":"2024-01-15T18:06:00.246456+0000","last_fullsized":"2024-01-15T18:06:00.246456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514426+0000","last_change":"2024-01-15T18:05:58.521804+0000","last_active":"2024-01-15T18:05:59.514426+0000","last_peered":"2024-01-15T18:05:59.514426+0000","last_clean":"2024-01-15T18:05:59.514426+0000","last_became_active":"2024-01-15T18:05:58.521514+0000","last_became_peered":"2024-01-15T18:05:58.521514+0000","last_unstale":"2024-01-15T18:05:59.514426+0000","last_undegraded":"2024-01-15T18:05:59.514426+0000","last_fullsized":"2024-01-15T18:05:59.514426+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:15.105803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246514+0000","last_change":"2024-01-15T18:05:58.096932+0000","last_active":"2024-01-15T18:06:00.246514+0000","last_peered":"2024-01-15T18:06:00.246514+0000","last_clean":"2024-01-15T18:06:00.246514+0000","last_became_active":"2024-01-15T18:05:58.096550+0000","last_became_peered":"2024-01-15T18:05:58.096550+0000","last_unstale":"2024-01-15T18:06:00.246514+0000","last_undegraded":"2024-01-15T18:06:00.246514+0000","last_fullsized":"2024-01-15T18:06:00.246514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:34.224174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517149+0000","last_change":"2024-01-15T18:05:58.528027+0000","last_active":"2024-01-15T18:05:59.517149+0000","last_peered":"2024-01-15T18:05:59.517149+0000","last_clean":"2024-01-15T18:05:59.517149+0000","last_became_active":"2024-01-15T18:05:58.527673+0000","last_became_peered":"2024-01-15T18:05:58.527673+0000","last_unstale":"2024-01-15T18:05:59.517149+0000","last_undegraded":"2024-01-15T18:05:59.517149+0000","last_fullsized":"2024-01-15T18:05:59.517149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:52:49.889606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153686+0000","last_change":"2024-01-15T18:05:58.098672+0000","last_active":"2024-01-15T18:06:00.153686+0000","last_peered":"2024-01-15T18:06:00.153686+0000","last_clean":"2024-01-15T18:06:00.153686+0000","last_became_active":"2024-01-15T18:05:58.098332+0000","last_became_peered":"2024-01-15T18:05:58.098332+0000","last_unstale":"2024-01-15T18:06:00.153686+0000","last_undegraded":"2024-01-15T18:06:00.153686+0000","last_fullsized":"2024-01-15T18:06:00.153686+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:17.646988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763391+0000","last_change":"2024-01-15T18:05:58.519891+0000","last_active":"2024-01-15T18:05:59.763391+0000","last_peered":"2024-01-15T18:05:59.763391+0000","last_clean":"2024-01-15T18:05:59.763391+0000","last_became_active":"2024-01-15T18:05:58.518714+0000","last_became_peered":"2024-01-15T18:05:58.518714+0000","last_unstale":"2024-01-15T18:05:59.763391+0000","last_undegraded":"2024-01-15T18:05:59.763391+0000","last_fullsized":"2024-01-15T18:05:59.763391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:05.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517214+0000","last_change":"2024-01-15T18:05:58.089201+0000","last_active":"2024-01-15T18:05:59.517214+0000","last_peered":"2024-01-15T18:05:59.517214+0000","last_clean":"2024-01-15T18:05:59.517214+0000","last_became_active":"2024-01-15T18:05:58.088786+0000","last_became_peered":"2024-01-15T18:05:58.088786+0000","last_unstale":"2024-01-15T18:05:59.517214+0000","last_undegraded":"2024-01-15T18:05:59.517214+0000","last_fullsized":"2024-01-15T18:05:59.517214+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153378+0000","last_change":"2024-01-15T18:05:58.533723+0000","last_active":"2024-01-15T18:06:00.153378+0000","last_peered":"2024-01-15T18:06:00.153378+0000","last_clean":"2024-01-15T18:06:00.153378+0000","last_became_active":"2024-01-15T18:05:58.533389+0000","last_became_peered":"2024-01-15T18:05:58.533389+0000","last_unstale":"2024-01-15T18:06:00.153378+0000","last_undegraded":"2024-01-15T18:06:00.153378+0000","last_fullsized":"2024-01-15T18:06:00.153378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:29.324616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517085+0000","last_change":"2024-01-15T18:05:58.091057+0000","last_active":"2024-01-15T18:05:59.517085+0000","last_peered":"2024-01-15T18:05:59.517085+0000","last_clean":"2024-01-15T18:05:59.517085+0000","last_became_active":"2024-01-15T18:05:58.090687+0000","last_became_peered":"2024-01-15T18:05:58.090687+0000","last_unstale":"2024-01-15T18:05:59.517085+0000","last_undegraded":"2024-01-15T18:05:59.517085+0000","last_fullsized":"2024-01-15T18:05:59.517085+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:31:02.330657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512852+0000","last_change":"2024-01-15T18:05:58.531553+0000","last_active":"2024-01-15T18:05:59.512852+0000","last_peered":"2024-01-15T18:05:59.512852+0000","last_clean":"2024-01-15T18:05:59.512852+0000","last_became_active":"2024-01-15T18:05:58.531297+0000","last_became_peered":"2024-01-15T18:05:58.531297+0000","last_unstale":"2024-01-15T18:05:59.512852+0000","last_undegraded":"2024-01-15T18:05:59.512852+0000","last_fullsized":"2024-01-15T18:05:59.512852+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:08.568321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.517135+0000","last_change":"2024-01-15T18:05:58.099072+0000","last_active":"2024-01-15T18:05:59.517135+0000","last_peered":"2024-01-15T18:05:59.517135+0000","last_clean":"2024-01-15T18:05:59.517135+0000","last_became_active":"2024-01-15T18:05:58.098747+0000","last_became_peered":"2024-01-15T18:05:58.098747+0000","last_unstale":"2024-01-15T18:05:59.517135+0000","last_undegraded":"2024-01-15T18:05:59.517135+0000","last_fullsized":"2024-01-15T18:05:59.517135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512620+0000","last_change":"2024-01-15T18:05:58.526920+0000","last_active":"2024-01-15T18:05:59.512620+0000","last_peered":"2024-01-15T18:05:59.512620+0000","last_clean":"2024-01-15T18:05:59.512620+0000","last_became_active":"2024-01-15T18:05:58.526672+0000","last_became_peered":"2024-01-15T18:05:58.526672+0000","last_unstale":"2024-01-15T18:05:59.512620+0000","last_undegraded":"2024-01-15T18:05:59.512620+0000","last_fullsized":"2024-01-15T18:05:59.512620+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:07.406787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160172+0000","last_change":"2024-01-15T18:05:58.089592+0000","last_active":"2024-01-15T18:06:00.160172+0000","last_peered":"2024-01-15T18:06:00.160172+0000","last_clean":"2024-01-15T18:06:00.160172+0000","last_became_active":"2024-01-15T18:05:58.089308+0000","last_became_peered":"2024-01-15T18:05:58.089308+0000","last_unstale":"2024-01-15T18:06:00.160172+0000","last_undegraded":"2024-01-15T18:06:00.160172+0000","last_fullsized":"2024-01-15T18:06:00.160172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512815+0000","last_change":"2024-01-15T18:05:58.524151+0000","last_active":"2024-01-15T18:05:59.512815+0000","last_peered":"2024-01-15T18:05:59.512815+0000","last_clean":"2024-01-15T18:05:59.512815+0000","last_became_active":"2024-01-15T18:05:58.523828+0000","last_became_peered":"2024-01-15T18:05:58.523828+0000","last_unstale":"2024-01-15T18:05:59.512815+0000","last_undegraded":"2024-01-15T18:05:59.512815+0000","last_fullsized":"2024-01-15T18:05:59.512815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:55.039829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514310+0000","last_change":"2024-01-15T18:05:58.090143+0000","last_active":"2024-01-15T18:05:59.514310+0000","last_peered":"2024-01-15T18:05:59.514310+0000","last_clean":"2024-01-15T18:05:59.514310+0000","last_became_active":"2024-01-15T18:05:58.089793+0000","last_became_peered":"2024-01-15T18:05:58.089793+0000","last_unstale":"2024-01-15T18:05:59.514310+0000","last_undegraded":"2024-01-15T18:05:59.514310+0000","last_fullsized":"2024-01-15T18:05:59.514310+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512492+0000","last_change":"2024-01-15T18:05:58.520203+0000","last_active":"2024-01-15T18:05:59.512492+0000","last_peered":"2024-01-15T18:05:59.512492+0000","last_clean":"2024-01-15T18:05:59.512492+0000","last_became_active":"2024-01-15T18:05:58.519935+0000","last_became_peered":"2024-01-15T18:05:58.519935+0000","last_unstale":"2024-01-15T18:05:59.512492+0000","last_undegraded":"2024-01-15T18:05:59.512492+0000","last_fullsized":"2024-01-15T18:05:59.512492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:01.713324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516640+0000","last_change":"2024-01-15T18:05:58.089185+0000","last_active":"2024-01-15T18:05:59.516640+0000","last_peered":"2024-01-15T18:05:59.516640+0000","last_clean":"2024-01-15T18:05:59.516640+0000","last_became_active":"2024-01-15T18:05:58.088876+0000","last_became_peered":"2024-01-15T18:05:58.088876+0000","last_unstale":"2024-01-15T18:05:59.516640+0000","last_undegraded":"2024-01-15T18:05:59.516640+0000","last_fullsized":"2024-01-15T18:05:59.516640+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:05.223309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849017+0000","last_change":"2024-01-15T18:05:58.532686+0000","last_active":"2024-01-15T18:05:59.849017+0000","last_peered":"2024-01-15T18:05:59.849017+0000","last_clean":"2024-01-15T18:05:59.849017+0000","last_became_active":"2024-01-15T18:05:58.531997+0000","last_became_peered":"2024-01-15T18:05:58.531997+0000","last_unstale":"2024-01-15T18:05:59.849017+0000","last_undegraded":"2024-01-15T18:05:59.849017+0000","last_fullsized":"2024-01-15T18:05:59.849017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:39.209214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.762811+0000","last_change":"2024-01-15T18:05:58.091112+0000","last_active":"2024-01-15T18:05:59.762811+0000","last_peered":"2024-01-15T18:05:59.762811+0000","last_clean":"2024-01-15T18:05:59.762811+0000","last_became_active":"2024-01-15T18:05:58.090824+0000","last_became_peered":"2024-01-15T18:05:58.090824+0000","last_unstale":"2024-01-15T18:05:59.762811+0000","last_undegraded":"2024-01-15T18:05:59.762811+0000","last_fullsized":"2024-01-15T18:05:59.762811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:41.987219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.512816+0000","last_change":"2024-01-15T18:05:58.523797+0000","last_active":"2024-01-15T18:05:59.512816+0000","last_peered":"2024-01-15T18:05:59.512816+0000","last_clean":"2024-01-15T18:05:59.512816+0000","last_became_active":"2024-01-15T18:05:58.523541+0000","last_became_peered":"2024-01-15T18:05:58.523541+0000","last_unstale":"2024-01-15T18:05:59.512816+0000","last_undegraded":"2024-01-15T18:05:59.512816+0000","last_fullsized":"2024-01-15T18:05:59.512816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:31:03.341301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153665+0000","last_change":"2024-01-15T18:05:58.090681+0000","last_active":"2024-01-15T18:06:00.153665+0000","last_peered":"2024-01-15T18:06:00.153665+0000","last_clean":"2024-01-15T18:06:00.153665+0000","last_became_active":"2024-01-15T18:05:58.090415+0000","last_became_peered":"2024-01-15T18:05:58.090415+0000","last_unstale":"2024-01-15T18:06:00.153665+0000","last_undegraded":"2024-01-15T18:06:00.153665+0000","last_fullsized":"2024-01-15T18:06:00.153665+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:01.634759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.763412+0000","last_change":"2024-01-15T18:05:58.527992+0000","last_active":"2024-01-15T18:05:59.763412+0000","last_peered":"2024-01-15T18:05:59.763412+0000","last_clean":"2024-01-15T18:05:59.763412+0000","last_became_active":"2024-01-15T18:05:58.527648+0000","last_became_peered":"2024-01-15T18:05:58.527648+0000","last_unstale":"2024-01-15T18:05:59.763412+0000","last_undegraded":"2024-01-15T18:05:59.763412+0000","last_fullsized":"2024-01-15T18:05:59.763412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.408307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:01.214936+0000","last_change":"2024-01-15T18:05:58.089651+0000","last_active":"2024-01-15T18:06:01.214936+0000","last_peered":"2024-01-15T18:06:01.214936+0000","last_clean":"2024-01-15T18:06:01.214936+0000","last_became_active":"2024-01-15T18:05:58.089376+0000","last_became_peered":"2024-01-15T18:05:58.089376+0000","last_unstale":"2024-01-15T18:06:01.214936+0000","last_undegraded":"2024-01-15T18:06:01.214936+0000","last_fullsized":"2024-01-15T18:06:01.214936+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160192+0000","last_change":"2024-01-15T18:05:58.522824+0000","last_active":"2024-01-15T18:06:00.160192+0000","last_peered":"2024-01-15T18:06:00.160192+0000","last_clean":"2024-01-15T18:06:00.160192+0000","last_became_active":"2024-01-15T18:05:58.522453+0000","last_became_peered":"2024-01-15T18:05:58.522453+0000","last_unstale":"2024-01-15T18:06:00.160192+0000","last_undegraded":"2024-01-15T18:06:00.160192+0000","last_fullsized":"2024-01-15T18:06:00.160192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:02.645403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153418+0000","last_change":"2024-01-15T18:05:58.086488+0000","last_active":"2024-01-15T18:06:00.153418+0000","last_peered":"2024-01-15T18:06:00.153418+0000","last_clean":"2024-01-15T18:06:00.153418+0000","last_became_active":"2024-01-15T18:05:58.086252+0000","last_became_peered":"2024-01-15T18:05:58.086252+0000","last_unstale":"2024-01-15T18:06:00.153418+0000","last_undegraded":"2024-01-15T18:06:00.153418+0000","last_fullsized":"2024-01-15T18:06:00.153418+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:43.938464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514265+0000","last_change":"2024-01-15T18:05:58.531200+0000","last_active":"2024-01-15T18:05:59.514265+0000","last_peered":"2024-01-15T18:05:59.514265+0000","last_clean":"2024-01-15T18:05:59.514265+0000","last_became_active":"2024-01-15T18:05:58.530654+0000","last_became_peered":"2024-01-15T18:05:58.530654+0000","last_unstale":"2024-01-15T18:05:59.514265+0000","last_undegraded":"2024-01-15T18:05:59.514265+0000","last_fullsized":"2024-01-15T18:05:59.514265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:32.949718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160304+0000","last_change":"2024-01-15T18:05:58.087272+0000","last_active":"2024-01-15T18:06:00.160304+0000","last_peered":"2024-01-15T18:06:00.160304+0000","last_clean":"2024-01-15T18:06:00.160304+0000","last_became_active":"2024-01-15T18:05:58.086833+0000","last_became_peered":"2024-01-15T18:05:58.086833+0000","last_unstale":"2024-01-15T18:06:00.160304+0000","last_undegraded":"2024-01-15T18:06:00.160304+0000","last_fullsized":"2024-01-15T18:06:00.160304+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:05.223309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.160209+0000","last_change":"2024-01-15T18:05:58.522790+0000","last_active":"2024-01-15T18:06:00.160209+0000","last_peered":"2024-01-15T18:06:00.160209+0000","last_clean":"2024-01-15T18:06:00.160209+0000","last_became_active":"2024-01-15T18:05:58.522188+0000","last_became_peered":"2024-01-15T18:05:58.522188+0000","last_unstale":"2024-01-15T18:06:00.160209+0000","last_undegraded":"2024-01-15T18:06:00.160209+0000","last_fullsized":"2024-01-15T18:06:00.160209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:35.234818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.153224+0000","last_change":"2024-01-15T18:05:58.093409+0000","last_active":"2024-01-15T18:06:00.153224+0000","last_peered":"2024-01-15T18:06:00.153224+0000","last_clean":"2024-01-15T18:06:00.153224+0000","last_became_active":"2024-01-15T18:05:58.092978+0000","last_became_peered":"2024-01-15T18:05:58.092978+0000","last_unstale":"2024-01-15T18:06:00.153224+0000","last_undegraded":"2024-01-15T18:06:00.153224+0000","last_fullsized":"2024-01-15T18:06:00.153224+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:31.939074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246415+0000","last_change":"2024-01-15T18:05:58.517491+0000","last_active":"2024-01-15T18:06:00.246415+0000","last_peered":"2024-01-15T18:06:00.246415+0000","last_clean":"2024-01-15T18:06:00.246415+0000","last_became_active":"2024-01-15T18:05:58.517181+0000","last_became_peered":"2024-01-15T18:05:58.517181+0000","last_unstale":"2024-01-15T18:06:00.246415+0000","last_undegraded":"2024-01-15T18:06:00.246415+0000","last_fullsized":"2024-01-15T18:06:00.246415+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:12:05.866938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516706+0000","last_change":"2024-01-15T18:05:58.092520+0000","last_active":"2024-01-15T18:05:59.516706+0000","last_peered":"2024-01-15T18:05:59.516706+0000","last_clean":"2024-01-15T18:05:59.516706+0000","last_became_active":"2024-01-15T18:05:58.092236+0000","last_became_peered":"2024-01-15T18:05:58.092236+0000","last_unstale":"2024-01-15T18:05:59.516706+0000","last_undegraded":"2024-01-15T18:05:59.516706+0000","last_fullsized":"2024-01-15T18:05:59.516706+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:36:44.129866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.516595+0000","last_change":"2024-01-15T18:05:58.523903+0000","last_active":"2024-01-15T18:05:59.516595+0000","last_peered":"2024-01-15T18:05:59.516595+0000","last_clean":"2024-01-15T18:05:59.516595+0000","last_became_active":"2024-01-15T18:05:58.523410+0000","last_became_peered":"2024-01-15T18:05:58.523410+0000","last_unstale":"2024-01-15T18:05:59.516595+0000","last_undegraded":"2024-01-15T18:05:59.516595+0000","last_fullsized":"2024-01-15T18:05:59.516595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:15.105803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:06:00.246442+0000","last_change":"2024-01-15T18:05:58.101671+0000","last_active":"2024-01-15T18:06:00.246442+0000","last_peered":"2024-01-15T18:06:00.246442+0000","last_clean":"2024-01-15T18:06:00.246442+0000","last_became_active":"2024-01-15T18:05:58.101363+0000","last_became_peered":"2024-01-15T18:05:58.101363+0000","last_unstale":"2024-01-15T18:06:00.246442+0000","last_undegraded":"2024-01-15T18:06:00.246442+0000","last_fullsized":"2024-01-15T18:06:00.246442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:27:01.634759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.849211+0000","last_change":"2024-01-15T18:05:58.531818+0000","last_active":"2024-01-15T18:05:59.849211+0000","last_peered":"2024-01-15T18:05:59.849211+0000","last_clean":"2024-01-15T18:05:59.849211+0000","last_became_active":"2024-01-15T18:05:58.531459+0000","last_became_peered":"2024-01-15T18:05:58.531459+0000","last_unstale":"2024-01-15T18:05:59.849211+0000","last_undegraded":"2024-01-15T18:05:59.849211+0000","last_fullsized":"2024-01-15T18:05:59.849211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:58.061661+0000","last_clean_scrub_stamp":"2024-01-15T18:05:58.061661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:23.838470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T18:05:59.514292+0000","last_change":"2024-01-15T18:05:58.090778+0000","last_active":"2024-01-15T18:05:59.514292+0000","last_peered":"2024-01-15T18:05:59.514292+0000","last_clean":"2024-01-15T18:05:59.514292+0000","last_became_active":"2024-01-15T18:05:58.087855+0000","last_became_peered":"2024-01-15T18:05:58.087855+0000","last_unstale":"2024-01-15T18:05:59.514292+0000","last_undegraded":"2024-01-15T18:05:59.514292+0000","last_fullsized":"2024-01-15T18:05:59.514292+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:05:57.051017+0000","last_clean_scrub_stamp":"2024-01-15T18:05:57.051017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:07.557677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-15T18:05:58.522325+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:05:58.522325+0000","last_peered":"2024-01-15T18:05:58.522325+0000","last_clean":"2024-01-15T18:05:58.522325+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:05:58.522325+0000","last_undegraded":"2024-01-15T18:05:58.522325+0000","last_fullsized":"2024-01-15T18:05:58.522325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":737050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.754}]},{"osd":1,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.82399999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.7759999999999998,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.78800000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.85599999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:05:50 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.89200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":147208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.623},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67100000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68799999999999994}]},{"osd":4,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.495}]},{"osd":5,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.72199999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":147264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.752}]},{"osd":1,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80600000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.82299999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.621}]},{"osd":6,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.878}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.436}]},{"osd":1,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.53900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.503},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.61399999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.52100000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.73299999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.63},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.438},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.32600000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.47099999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.379},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.22,"5min":1.22,"15min":1.22},"last":0.68899999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.65000000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:05:46 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.66200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":146662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.64400000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.67900000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.749}]},{"osd":6,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:05:48 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.86899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":148226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":4.532,"5min":4.532,"15min":4.532},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":4.7969999999999997,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":4.6840000000000002,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.5890000000000004,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.57999999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":4.8259999999999996,"5min":4.8259999999999996,"15min":4.8259999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.5599999999999996,"5min":4.5599999999999996,"15min":4.5599999999999996},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":4.7629999999999999,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":4.4480000000000004,"5min":4.4480000000000004,"15min":4.4480000000000004},"last":0.69699999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.4909999999999997,"5min":4.4909999999999997,"15min":4.4909999999999997},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":4.7309999999999999,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.754}]},{"osd":7,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":4.6120000000000001,"5min":4.6120000000000001,"15min":4.6120000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":4.3550000000000004,"5min":4.3550000000000004,"15min":4.3550000000000004},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":966656,"data_stored":736960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":8.1050000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.40600000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.436}]},{"osd":3,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":8.0579999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48499999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.9820000000000002,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.746},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":4.3159999999999998,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.622}]},{"osd":5,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":4.4539999999999997,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":4.056,"5min":4.056,"15min":4.056},"last":0.69399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":4.0190000000000001,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:05:49 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:06:03.534 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:06:03.534 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:06:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7367e00640 1 Processor -- start 2024-01-15T18:06:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7367e00640 1 -- start start 2024-01-15T18:06:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7367e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73600b3490 0x7f73600b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7367e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600721b0 con 0x7f73600b1da0 2024-01-15T18:06:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7367e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73600b3e80 con 0x7f73601436f0 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7367e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7360072640 con 0x7f73600b3490 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7364a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73600b3490 0x7f73600b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7365e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f73600b1da0 0x7f73600b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40920/0 (socket says 172.21.15.143:40920) 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.714+0000 7f7365e00640 1 -- 172.21.15.143:0/4017780021 learned_addr learned my addr 172.21.15.143:0/4017780021 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f735fe00640 1 -- 172.21.15.143:0/4017780021 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203564845 0 0) 0x7f73600721b0 con 0x7f73600b1da0 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f7364a00640 1 -- 172.21.15.143:0/4017780021 >> v1:172.21.15.204:6789/0 conn(0x7f73601436f0 legacy=0x7f73600ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f7364a00640 1 -- 172.21.15.143:0/4017780021 >> v1:172.21.15.143:6789/0 conn(0x7f73600b1da0 legacy=0x7f73600b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f7364a00640 1 -- 172.21.15.143:0/4017780021 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600715a0 con 0x7f73600b3490 2024-01-15T18:06:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f7364a00640 1 --2- 172.21.15.143:0/4017780021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f73600b3940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f73480097c0 tx=0x7f734801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2613bf1520879c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f735fe00640 1 -- 172.21.15.143:0/4017780021 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480ab650 con 0x7f73600b3490 2024-01-15T18:06:03.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f735fe00640 1 -- 172.21.15.143:0/4017780021 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73480ab7b0 con 0x7f73600b3490 2024-01-15T18:06:03.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.718+0000 7f735fe00640 1 -- 172.21.15.143:0/4017780021 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480abb40 con 0x7f73600b3490 2024-01-15T18:06:03.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- 172.21.15.143:0/4017780021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 msgr2=0x7f73600b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 --2- 172.21.15.143:0/4017780021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f73600b3940 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f73480097c0 tx=0x7f734801c450 comp rx=0 tx=0).stop 2024-01-15T18:06:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- 172.21.15.143:0/4017780021 shutdown_connections 2024-01-15T18:06:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 --2- 172.21.15.143:0/4017780021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f73600b3940 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- 172.21.15.143:0/4017780021 >> 172.21.15.143:0/4017780021 conn(0x7f736006a890 msgr2=0x7f736006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- 172.21.15.143:0/4017780021 shutdown_connections 2024-01-15T18:06:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- 172.21.15.143:0/4017780021 wait complete. 2024-01-15T18:06:03.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 Processor -- start 2024-01-15T18:06:03.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- start start 2024-01-15T18:06:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73600b1da0 0x7f73601f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f7360160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73601436f0 0x7f736015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73601f9fe0 con 0x7f73600b3490 2024-01-15T18:06:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7360161370 con 0x7f73600b1da0 2024-01-15T18:06:03.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7367e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600cbb50 con 0x7f73601436f0 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7364a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f7360160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7364a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f7360160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57420/0 (socket says 172.21.15.143:57420) 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7364a00640 1 -- 172.21.15.143:0/20736964 learned_addr learned my addr 172.21.15.143:0/20736964 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7365400640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73600b1da0 0x7f73601f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.722+0000 7f7365e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73601436f0 0x7f736015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7364a00640 1 -- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73601436f0 msgr2=0x7f736015bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7364a00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73601436f0 0x7f736015bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7364a00640 1 -- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73600b1da0 msgr2=0x7f73601f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7364a00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73600b1da0 0x7f73601f9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7364a00640 1 -- 172.21.15.143:0/20736964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7348009420 con 0x7f73600b3490 2024-01-15T18:06:03.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7364a00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f7360160e30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f73480ad040 tx=0x7f734801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480b5810 con 0x7f73600b3490 2024-01-15T18:06:03.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73480b5990 con 0x7f73600b3490 2024-01-15T18:06:03.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73480b5d40 con 0x7f73600b3490 2024-01-15T18:06:03.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73600cbe30 con 0x7f73600b3490 2024-01-15T18:06:03.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73600cc3f0 con 0x7f73600b3490 2024-01-15T18:06:03.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7360146c80 con 0x7f73600b3490 2024-01-15T18:06:03.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f73480bf410 con 0x7f73600b3490 2024-01-15T18:06:03.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f735d600640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7338066920 0x7f7338068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f734811ef20 con 0x7f73600b3490 2024-01-15T18:06:03.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.726+0000 7f7365400640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7338066920 0x7f7338068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.730+0000 7f7365400640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7338066920 0x7f7338068de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7350007fd0 tx=0x7f7350009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.730+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f73480ec290 con 0x7f73600b3490 2024-01-15T18:06:03.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.930+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7360067500 con 0x7f73600b3490 2024-01-15T18:06:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.934+0000 7f735d600640 1 -- 172.21.15.143:0/20736964 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f73480ef8b0 con 0x7f73600b3490 2024-01-15T18:06:03.941 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:06:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:06:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:06:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:06:03.941 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:06:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:06:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7338066920 msgr2=0x7f7338068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7338066920 0x7f7338068de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7350007fd0 tx=0x7f7350009040 comp rx=0 tx=0).stop 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 msgr2=0x7f7360160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f7360160e30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f73480ad040 tx=0x7f734801ac90 comp rx=0 tx=0).stop 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 shutdown_connections 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7338066920 0x7f7338068de0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73601436f0 0x7f736015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73600b3490 0x7f7360160e30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 --2- 172.21.15.143:0/20736964 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73600b1da0 0x7f73601f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 >> 172.21.15.143:0/20736964 conn(0x7f736006a890 msgr2=0x7f736006db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 shutdown_connections 2024-01-15T18:06:03.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:03.938+0000 7f7367e00640 1 -- 172.21.15.143:0/20736964 wait complete. 2024-01-15T18:06:03.962 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:06:04.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7587200640 1 Processor -- start 2024-01-15T18:06:04.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7587200640 1 -- start start 2024-01-15T18:06:04.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7587200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f758015b260 0x7f75800cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7587200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75800c4d90 con 0x7f75800ccaf0 2024-01-15T18:06:04.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7587200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758015b640 con 0x7f75800d19f0 2024-01-15T18:06:04.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7587200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75800cf800 con 0x7f758015b260 2024-01-15T18:06:04.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7585200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f75800ccaf0 0x7f75800c5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40922/0 (socket says 172.21.15.143:40922) 2024-01-15T18:06:04.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f7585200640 1 -- 172.21.15.143:0/2522758766 learned_addr learned my addr 172.21.15.143:0/2522758766 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:04.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757fe00640 1 --2- 172.21.15.143:0/2522758766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f758015b260 0x7f75800cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757ea00640 1 -- 172.21.15.143:0/2522758766 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753681730 0 0) 0x7f75800c4d90 con 0x7f75800ccaf0 2024-01-15T18:06:04.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757ea00640 1 -- 172.21.15.143:0/2522758766 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7560003530 con 0x7f75800ccaf0 2024-01-15T18:06:04.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757ea00640 1 -- 172.21.15.143:0/2522758766 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025552619 0 0) 0x7f758015b640 con 0x7f75800d19f0 2024-01-15T18:06:04.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757fe00640 1 -- 172.21.15.143:0/2522758766 >> v1:172.21.15.204:6789/0 conn(0x7f75800d19f0 legacy=0x7f75800c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757fe00640 1 -- 172.21.15.143:0/2522758766 >> v1:172.21.15.143:6789/0 conn(0x7f75800ccaf0 legacy=0x7f75800c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.094+0000 7f757fe00640 1 -- 172.21.15.143:0/2522758766 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800cfae0 con 0x7f758015b260 2024-01-15T18:06:04.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f757fe00640 1 --2- 172.21.15.143:0/2522758766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f758015b260 0x7f75800cd810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f75700097f0 tx=0x7f757001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9320cc92502c7c4f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f757ea00640 1 -- 172.21.15.143:0/2522758766 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700ab600 con 0x7f758015b260 2024-01-15T18:06:04.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f757ea00640 1 -- 172.21.15.143:0/2522758766 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700a7050 con 0x7f758015b260 2024-01-15T18:06:04.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f757ea00640 1 -- 172.21.15.143:0/2522758766 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700ab980 con 0x7f758015b260 2024-01-15T18:06:04.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 -- 172.21.15.143:0/2522758766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f758015b260 msgr2=0x7f75800cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 --2- 172.21.15.143:0/2522758766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f758015b260 0x7f75800cd810 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f75700097f0 tx=0x7f757001c4c0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 -- 172.21.15.143:0/2522758766 shutdown_connections 2024-01-15T18:06:04.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 --2- 172.21.15.143:0/2522758766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f758015b260 0x7f75800cd810 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 -- 172.21.15.143:0/2522758766 >> 172.21.15.143:0/2522758766 conn(0x7f75800d0ed0 msgr2=0x7f75800d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 -- 172.21.15.143:0/2522758766 shutdown_connections 2024-01-15T18:06:04.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.098+0000 7f7587200640 1 -- 172.21.15.143:0/2522758766 wait complete. 2024-01-15T18:06:04.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 Processor -- start 2024-01-15T18:06:04.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 -- start start 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 0x7f75801576a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75800d19f0 0x7f758014a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758015b260 0x7f75800c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758014c650 con 0x7f75800ccaf0 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758014c930 con 0x7f758015b260 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758014cc10 con 0x7f75800d19f0 2024-01-15T18:06:04.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 0x7f75801576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 0x7f75801576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57434/0 (socket says 172.21.15.143:57434) 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 -- 172.21.15.143:0/4111437294 learned_addr learned my addr 172.21.15.143:0/4111437294 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757f400640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75800d19f0 0x7f758014a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7585200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758015b260 0x7f75800c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 -- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75800d19f0 msgr2=0x7f758014a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75800d19f0 0x7f758014a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 -- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758015b260 msgr2=0x7f75800c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758015b260 0x7f75800c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 -- 172.21.15.143:0/4111437294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570009420 con 0x7f75800ccaf0 2024-01-15T18:06:04.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f757fe00640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 0x7f75801576a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f75700097f0 tx=0x7f7570019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570004020 con 0x7f75800ccaf0 2024-01-15T18:06:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7570004180 con 0x7f75800ccaf0 2024-01-15T18:06:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758014cef0 con 0x7f75800ccaf0 2024-01-15T18:06:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75700044f0 con 0x7f75800ccaf0 2024-01-15T18:06:04.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.102+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758014c340 con 0x7f75800ccaf0 2024-01-15T18:06:04.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.106+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7580070190 con 0x7f75800ccaf0 2024-01-15T18:06:04.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.106+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f75700b2040 con 0x7f75800ccaf0 2024-01-15T18:06:04.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.106+0000 7f7577e00640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75580668f0 0x7f7558068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.106+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f757011dd00 con 0x7f75800ccaf0 2024-01-15T18:06:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.106+0000 7f757f400640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75580668f0 0x7f7558068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.106+0000 7f757f400640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75580668f0 0x7f7558068db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f75800700d0 tx=0x7f7568009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.110+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f75700eaec0 con 0x7f75800ccaf0 2024-01-15T18:06:04.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.290+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7580067500 con 0x7f75800ccaf0 2024-01-15T18:06:04.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7577e00640 1 -- 172.21.15.143:0/4111437294 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f75700bd030 con 0x7f75800ccaf0 2024-01-15T18:06:04.298 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:04.298 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3272757327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3272757327},{"type":"v1","addr":"172.21.15.204:6834","nonce":3272757327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/518983608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":518983608},{"type":"v1","addr":"172.21.15.143:6836","nonce":518983608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/434311355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":434311355},{"type":"v1","addr":"172.21.15.143:6837","nonce":434311355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:05:58.451230+0000","modified":"2024-01-15T18:06:01.458502+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/4126176885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":4126176885},{"type":"v1","addr":"172.21.15.204:6835","nonce":4126176885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-15T18:06:04.299 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 17 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75580668f0 msgr2=0x7f7558068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75580668f0 0x7f7558068db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f75800700d0 tx=0x7f7568009040 comp rx=0 tx=0).stop 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 msgr2=0x7f75801576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 0x7f75801576a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f75700097f0 tx=0x7f7570019260 comp rx=0 tx=0).stop 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 shutdown_connections 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75580668f0 0x7f7558068db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f758015b260 0x7f75800c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75800d19f0 0x7f758014a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 --2- 172.21.15.143:0/4111437294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75800ccaf0 0x7f75801576a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 >> 172.21.15.143:0/4111437294 conn(0x7f75800d0ed0 msgr2=0x7f7580157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 shutdown_connections 2024-01-15T18:06:04.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.294+0000 7f7587200640 1 -- 172.21.15.143:0/4111437294 wait complete. 2024-01-15T18:06:04.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:05:58.451230+0000', 'modified': '2024-01-15T18:06:01.458502+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5232': {'gid': 5232, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/4126176885', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 4126176885}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 4126176885}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:06:04.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:06:04.725 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-15T18:06:04.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 Processor -- start 2024-01-15T18:06:04.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 -- start start 2024-01-15T18:06:04.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff20c143780 0x7ff20c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff20c0ce520 0x7ff20c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff20c0ceb50 con 0x7ff20c0720f0 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c15c110 con 0x7ff20c143780 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff214200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20c072530 con 0x7ff20c0ce520 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff211800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff20c143780 0x7ff20c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff211800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff20c143780 0x7ff20c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57442/0 (socket says 172.21.15.143:57442) 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff211800640 1 -- 172.21.15.143:0/2862023654 learned_addr learned my addr 172.21.15.143:0/2862023654 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.858+0000 7ff210e00640 1 --2- 172.21.15.143:0/2862023654 >> v2:172.21.15.204:3300/0 conn(0x7ff20c0ce520 0x7ff20c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff211800640 1 -- 172.21.15.143:0/2862023654 >> v2:172.21.15.204:3300/0 conn(0x7ff20c0ce520 msgr2=0x7ff20c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff211800640 1 --2- 172.21.15.143:0/2862023654 >> v2:172.21.15.204:3300/0 conn(0x7ff20c0ce520 0x7ff20c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff211800640 1 -- 172.21.15.143:0/2862023654 >> v1:172.21.15.204:6789/0 conn(0x7ff20c0720f0 legacy=0x7ff20c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff211800640 1 -- 172.21.15.143:0/2862023654 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c15b7e0 con 0x7ff20c143780 2024-01-15T18:06:04.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff210e00640 1 --2- 172.21.15.143:0/2862023654 >> v2:172.21.15.204:3300/0 conn(0x7ff20c0ce520 0x7ff20c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:04.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff211800640 1 --2- 172.21.15.143:0/2862023654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c0cdfe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff20c06beb0 tx=0x7ff1fc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad114bb960c28ed6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff20be00640 1 -- 172.21.15.143:0/2862023654 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0ab610 con 0x7ff20c143780 2024-01-15T18:06:04.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff20be00640 1 -- 172.21.15.143:0/2862023654 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc0ab770 con 0x7ff20c143780 2024-01-15T18:06:04.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.862+0000 7ff20be00640 1 -- 172.21.15.143:0/2862023654 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0abb00 con 0x7ff20c143780 2024-01-15T18:06:04.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- 172.21.15.143:0/2862023654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 msgr2=0x7ff20c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 --2- 172.21.15.143:0/2862023654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c0cdfe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff20c06beb0 tx=0x7ff1fc01c500 comp rx=0 tx=0).stop 2024-01-15T18:06:04.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- 172.21.15.143:0/2862023654 shutdown_connections 2024-01-15T18:06:04.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 --2- 172.21.15.143:0/2862023654 >> v2:172.21.15.204:3300/0 conn(0x7ff20c0ce520 0x7ff20c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 --2- 172.21.15.143:0/2862023654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c0cdfe0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- 172.21.15.143:0/2862023654 >> 172.21.15.143:0/2862023654 conn(0x7ff20c06c3f0 msgr2=0x7ff20c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- 172.21.15.143:0/2862023654 shutdown_connections 2024-01-15T18:06:04.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- 172.21.15.143:0/2862023654 wait complete. 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 Processor -- start 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- start start 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff20c0720f0 0x7ff20c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff20c0ce520 0x7ff20c0adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0ad490 con 0x7ff20c143780 2024-01-15T18:06:04.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c0ad770 con 0x7ff20c0720f0 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff214200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20c13be50 con 0x7ff20c0ce520 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c1426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57456/0 (socket says 172.21.15.143:57456) 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 -- 172.21.15.143:0/1930196441 learned_addr learned my addr 172.21.15.143:0/1930196441 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff211800640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff20c0720f0 0x7ff20c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff210e00640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff20c0ce520 0x7ff20c0adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 -- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff20c0ce520 msgr2=0x7ff20c0adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff20c0ce520 0x7ff20c0adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 -- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff20c0720f0 msgr2=0x7ff20c1594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.866+0000 7ff212200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff20c0720f0 0x7ff20c1594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff212200640 1 -- 172.21.15.143:0/1930196441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0094d0 con 0x7ff20c143780 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff211800640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff20c0720f0 0x7ff20c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff212200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c1426a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff20400c6c0 tx=0x7ff20400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff209600640 1 -- 172.21.15.143:0/1930196441 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204003b80 con 0x7ff20c143780 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff209600640 1 -- 172.21.15.143:0/1930196441 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204003ce0 con 0x7ff20c143780 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20c13c130 con 0x7ff20c143780 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff209600640 1 -- 172.21.15.143:0/1930196441 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2040052f0 con 0x7ff20c143780 2024-01-15T18:06:04.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20c142be0 con 0x7ff20c143780 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff209600640 1 -- 172.21.15.143:0/1930196441 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff204005450 con 0x7ff20c143780 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff209600640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1e4066920 0x7ff1e4068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff209600640 1 -- 172.21.15.143:0/1930196441 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff204083fe0 con 0x7ff20c143780 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.870+0000 7ff211800640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1e4066920 0x7ff1e4068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 Processor -- start 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff211800640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1e4066920 0x7ff1e4068de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0af000 tx=0x7ff1fc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- start start 2024-01-15T18:06:04.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e11c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 0x55917e0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0d2320 0x55917e0d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917df3c130 con 0x55917e11c5b0 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917df299a0 con 0x55917e0d2320 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55917df26ed0 con 0x55917e11bc90 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 0x55917e0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 0x55917e0d1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57460/0 (socket says 172.21.15.143:57460) 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 -- 172.21.15.143:0/819349269 learned_addr learned my addr 172.21.15.143:0/819349269 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff212200640 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0d2320 0x55917e0d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 -- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 msgr2=0x55917e11c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e11c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 -- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0d2320 msgr2=0x55917e0d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0d2320 0x55917e0d2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 -- 172.21.15.143:0/819349269 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55917dc1a890 con 0x55917e11c5b0 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff210e00640 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 0x55917e0d1de0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009a10 tx=0x7ff1f4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dd9697c220ca29 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff202000640 1 -- 172.21.15.143:0/819349269 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4010d00 con 0x55917e11c5b0 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff202000640 1 -- 172.21.15.143:0/819349269 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f400c050 con 0x55917e11c5b0 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff202000640 1 -- 172.21.15.143:0/819349269 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4015670 con 0x55917e11c5b0 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- 172.21.15.143:0/819349269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 msgr2=0x55917e0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 0x55917e0d1de0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009a10 tx=0x7ff1f4009ed0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- 172.21.15.143:0/819349269 shutdown_connections 2024-01-15T18:06:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0d2320 0x55917e0d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11c5b0 0x55917e0d1de0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 --2- 172.21.15.143:0/819349269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e11c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- 172.21.15.143:0/819349269 >> 172.21.15.143:0/819349269 conn(0x55917e0ce2d0 msgr2=0x55917e0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- 172.21.15.143:0/819349269 shutdown_connections 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.874+0000 7ff217580000 1 -- 172.21.15.143:0/819349269 wait complete. 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- start start 2024-01-15T18:06:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 Processor -- start 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e0d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 0x55917e0d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11c5b0 0x55917e0ea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917df227c0 con 0x55917e11bc90 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917dedabf0 con 0x55917e11c5b0 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55917dee0a90 con 0x55917e0d2320 2024-01-15T18:06:04.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 10 client.0 _ll_register_callbacks cb 0x55917e0bd3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 0x55917e0d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 0x55917e0d7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57474/0 (socket says 172.21.15.143:57474) 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 -- 172.21.15.143:0/3944834542 learned_addr learned my addr 172.21.15.143:0/3944834542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff212200640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11c5b0 0x55917e0ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff211800640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e0d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 -- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 msgr2=0x55917e0d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e0d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 -- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11c5b0 msgr2=0x55917e0ea910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11c5b0 0x55917e0ea910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 -- 172.21.15.143:0/3944834542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f4009640 con 0x55917e11bc90 2024-01-15T18:06:04.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff210e00640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 0x55917e0d7280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40105c0 tx=0x7ff1f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf2097238cebf21 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff1faa00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff1faa00640 1 -- 172.21.15.143:0/3944834542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4015c60 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff1faa00640 1 -- 172.21.15.143:0/3944834542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1f4015dc0 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55917dee4470 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff1faa00640 1 -- 172.21.15.143:0/3944834542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f4004020 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 10 client.5397 fetch_fsmap learned FSMap version 17 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55917dee3190 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff1faa00640 1 -- 172.21.15.143:0/3944834542 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55917dee4470 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff1faa00640 1 -- 172.21.15.143:0/3944834542 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1f4004180 con 0x55917e11bc90 2024-01-15T18:06:04.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 10 client.5397 fetch_fsmap finished waiting for FSMap version 17 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 10 client.5397 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 20 client.5397 populate_metadata read hostname 'smithi143' 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e0eb320 0x55917e12df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 4 client.5397 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.878+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 --> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55917dee7b50 con 0x55917e0eb320 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff211800640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e0eb320 0x55917e12df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff211800640 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e0eb320 0x55917e12df50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c856dd585d8529 server_cookie=5c88afad3169f62b in_seq=0 out_seq=0 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff1faa00640 10 client.5397 ms_handle_connect on v2:172.21.15.204:6833/4126176885 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff1faa00640 1 -- 172.21.15.143:0/3944834542 <== mds.0 v2:172.21.15.204:6833/4126176885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55917dee7b50 con 0x55917e0eb320 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff1faa00640 10 client.5397 handle_command_reply: tid=0 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 client.5397 shutdown 2024-01-15T18:06:04.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 msgr2=0x55917e0d7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 0x55917e0d7280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40105c0 tx=0x7ff1f4004740 comp rx=0 tx=0).stop 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 shutdown_connections 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e0eb320 0x55917e12df50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11c5b0 0x55917e0ea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e11bc90 0x55917e0d7280 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff1faa00640 0 client.5397 ms_handle_reset on v2:172.21.15.204:6833/4126176885 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 --2- 172.21.15.143:0/3944834542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e0d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 >> 172.21.15.143:0/3944834542 conn(0x55917e0ce2d0 msgr2=0x55917e0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 shutdown_connections 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 1 -- 172.21.15.143:0/3944834542 wait complete. 2024-01-15T18:06:04.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.882+0000 7ff217580000 20 client.5397 trim_cache size 0 max 16384 2024-01-15T18:06:04.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 Processor -- start 2024-01-15T18:06:04.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 -- start start 2024-01-15T18:06:04.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 0x55917e119b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e11a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11bc90 0x55917e0fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917dbdb940 con 0x55917e0ce2d0 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917ded7760 con 0x55917e11bc90 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55917dfa9fa0 con 0x55917e0d2320 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 0x55917e119b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 0x55917e119b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57486/0 (socket says 172.21.15.143:57486) 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 -- 172.21.15.143:0/148528053 learned_addr learned my addr 172.21.15.143:0/148528053 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff212200640 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11bc90 0x55917e0fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 -- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 msgr2=0x55917e11a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e11a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 -- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11bc90 msgr2=0x55917e0fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11bc90 0x55917e0fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 -- 172.21.15.143:0/148528053 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55917dec9490 con 0x55917e0ce2d0 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff212200640 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11bc90 0x55917e0fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff211800640 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 0x55917e119b30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0ad040 tx=0x7ff1fc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdf8ab1b29b01f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff202a00640 1 -- 172.21.15.143:0/148528053 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0c3040 con 0x55917e0ce2d0 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff202a00640 1 -- 172.21.15.143:0/148528053 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc0c5040 con 0x55917e0ce2d0 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.898+0000 7ff202a00640 1 -- 172.21.15.143:0/148528053 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0ca830 con 0x55917e0ce2d0 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/148528053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 msgr2=0x55917e119b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 0x55917e119b30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0ad040 tx=0x7ff1fc0046d0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/148528053 shutdown_connections 2024-01-15T18:06:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e11bc90 0x55917e0fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e0d2320 0x55917e11a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- 172.21.15.143:0/148528053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0ce2d0 0x55917e119b30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/148528053 >> 172.21.15.143:0/148528053 conn(0x55917e0d6f10 msgr2=0x55917e0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/148528053 shutdown_connections 2024-01-15T18:06:04.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/148528053 wait complete. 2024-01-15T18:06:04.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- start start 2024-01-15T18:06:04.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 Processor -- start 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0ce2d0 0x55917e0febd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 0x55917e12df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e12e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917ded9a10 con 0x55917e0d2320 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55917df21fd0 con 0x55917e0ce2d0 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55917defbe10 con 0x55917e11bc90 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 10 client.0 _ll_register_callbacks cb 0x55917df70c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 0x55917e12df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff211800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0ce2d0 0x55917e0febd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 0x55917e12df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57500/0 (socket says 172.21.15.143:57500) 2024-01-15T18:06:04.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 -- 172.21.15.143:0/3787283829 learned_addr learned my addr 172.21.15.143:0/3787283829 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff212200640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e12e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 -- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 msgr2=0x55917e12e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e12e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 -- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0ce2d0 msgr2=0x55917e0febd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0ce2d0 0x55917e0febd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 -- 172.21.15.143:0/3787283829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc0ba070 con 0x55917e0d2320 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff211800640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0ce2d0 0x55917e0febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff210e00640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 0x55917e12df50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009f10 tx=0x7ff1f40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff93fcad9b17ec98 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff1faa00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff1faa00640 1 -- 172.21.15.143:0/3787283829 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4010e10 con 0x55917e0d2320 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff1faa00640 1 -- 172.21.15.143:0/3787283829 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1f4032680 con 0x55917e0d2320 2024-01-15T18:06:04.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55917df21110 con 0x55917e0d2320 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff1faa00640 1 -- 172.21.15.143:0/3787283829 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f4032840 con 0x55917e0d2320 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 10 client.5403 fetch_fsmap learned FSMap version 17 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.902+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55917dc1acc0 con 0x55917e0d2320 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff1faa00640 1 -- 172.21.15.143:0/3787283829 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55917df21110 con 0x55917e0d2320 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff1faa00640 1 -- 172.21.15.143:0/3787283829 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1f40329a0 con 0x55917e0d2320 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 10 client.5403 fetch_fsmap finished waiting for FSMap version 17 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 10 client.5403 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 20 client.5403 populate_metadata read hostname 'smithi143' 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e11c5b0 0x55917e184e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 4 client.5403 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:06:04.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 --> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55917df509a0 con 0x55917e11c5b0 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff211800640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e11c5b0 0x55917e184e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff211800640 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e11c5b0 0x55917e184e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3805aff31b6588 server_cookie=748f81e7a0f6c339 in_seq=0 out_seq=0 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff1faa00640 10 client.5403 ms_handle_connect on v2:172.21.15.204:6833/4126176885 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff1faa00640 1 -- 172.21.15.143:0/3787283829 <== mds.0 v2:172.21.15.204:6833/4126176885 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55917df509a0 con 0x55917e11c5b0 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff1faa00640 10 client.5403 handle_command_reply: tid=0 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 client.5403 shutdown 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 msgr2=0x55917e12df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 0x55917e12df50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009f10 tx=0x7ff1f40046d0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 shutdown_connections 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:6833/4126176885,v1:172.21.15.204:6835/4126176885] conn(0x55917e11c5b0 0x55917e184e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55917e11bc90 0x55917e12e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55917e0d2320 0x55917e12df50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff1faa00640 0 client.5403 ms_handle_reset on v2:172.21.15.204:6833/4126176885 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 --2- 172.21.15.143:0/3787283829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55917e0ce2d0 0x55917e0febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 >> 172.21.15.143:0/3787283829 conn(0x55917e0d6f10 msgr2=0x55917e0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 shutdown_connections 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 1 -- 172.21.15.143:0/3787283829 wait complete. 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff217580000 20 client.5403 trim_cache size 0 max 16384 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:06:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5232, 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8365137159999998, 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 17, 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 33, 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 33, 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 13.699961837 2024-01-15T18:06:04.919 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1e4066920 msgr2=0x7ff1e4068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1e4066920 0x7ff1e4068de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc0af000 tx=0x7ff1fc0a7000 comp rx=0 tx=0).stop 2024-01-15T18:06:04.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 msgr2=0x7ff20c1426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c1426a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff20400c6c0 tx=0x7ff20400cb80 comp rx=0 tx=0).stop 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 shutdown_connections 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1e4066920 0x7ff1e4068de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff20c143780 0x7ff20c1426a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff20c0ce520 0x7ff20c0adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 --2- 172.21.15.143:0/1930196441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff20c0720f0 0x7ff20c1594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 >> 172.21.15.143:0/1930196441 conn(0x7ff20c06c3f0 msgr2=0x7ff20c0b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 shutdown_connections 2024-01-15T18:06:04.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:04.906+0000 7ff214200640 1 -- 172.21.15.143:0/1930196441 wait complete. 2024-01-15T18:06:04.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:06:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.062+0000 7f0f91c00640 1 Processor -- start 2024-01-15T18:06:05.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.062+0000 7f0f91c00640 1 -- start start 2024-01-15T18:06:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f91c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0f8c072030 0x7f0f8c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f91c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c0ceb50 con 0x7f0f8c143780 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f91c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f8c15c110 con 0x7f0f8c071bb0 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f91c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8c15b7e0 con 0x7f0f8c072030 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0f8c143780 0x7f0f8c0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40930/0 (socket says 172.21.15.143:40930) 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8aa00640 1 -- 172.21.15.143:0/2166229726 learned_addr learned my addr 172.21.15.143:0/2166229726 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 702136043 0 0) 0x7f0f7c001120 con 0x7f0f8c143780 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8b400640 1 --2- 172.21.15.143:0/2166229726 >> v2:172.21.15.204:3300/0 conn(0x7f0f8c072030 0x7f0f8c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f6c003530 con 0x7f0f8c143780 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 37435931 0 0) 0x7f0f6c003530 con 0x7f0f8c143780 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 >> v2:172.21.15.204:3300/0 conn(0x7f0f8c072030 msgr2=0x7f0f8c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 --2- 172.21.15.143:0/2166229726 >> v2:172.21.15.204:3300/0 conn(0x7f0f8c072030 0x7f0f8c15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 >> v1:172.21.15.204:6789/0 conn(0x7f0f8c071bb0 legacy=0x7f0f8c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c15bac0 con 0x7f0f8c143780 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f0f7c001810 con 0x7f0f8c143780 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 >> v1:172.21.15.143:6789/0 conn(0x7f0f8c143780 legacy=0x7f0f8c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f6c003980 0x7f0f6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f6c0047d0 0x7f0f6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c005190 0x7f0f6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c15b7e0 con 0x7f0f6c0047d0 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c00c030 con 0x7f0f6c003980 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c00c1f0 con 0x7f0f6c005190 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8a000640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f6c0047d0 0x7f0f6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8aa00640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f6c003980 0x7f0f6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8b400640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c005190 0x7f0f6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8a000640 1 -- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c005190 msgr2=0x7f0f6c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8a000640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c005190 0x7f0f6c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8a000640 1 -- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f6c003980 msgr2=0x7f0f6c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8a000640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f6c003980 0x7f0f6c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8a000640 1 -- 172.21.15.143:0/2166229726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c003530 con 0x7f0f6c0047d0 2024-01-15T18:06:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.066+0000 7f0f8b400640 1 -- 172.21.15.143:0/2166229726 reap_dead start 2024-01-15T18:06:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f8a000640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f6c0047d0 0x7f0f6c004c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f74001780 tx=0x7f0f7401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284a451c26d5fab8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f740052d0 con 0x7f0f6c0047d0 2024-01-15T18:06:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f740a7050 con 0x7f0f6c0047d0 2024-01-15T18:06:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f89600640 1 -- 172.21.15.143:0/2166229726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74003600 con 0x7f0f6c0047d0 2024-01-15T18:06:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 -- 172.21.15.143:0/2166229726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f6c0047d0 msgr2=0x7f0f6c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f6c0047d0 0x7f0f6c004c50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0f74001780 tx=0x7f0f7401c3e0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 -- 172.21.15.143:0/2166229726 shutdown_connections 2024-01-15T18:06:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 --2- 172.21.15.143:0/2166229726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f6c0047d0 0x7f0f6c004c50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 -- 172.21.15.143:0/2166229726 >> 172.21.15.143:0/2166229726 conn(0x7f0f8c06c3f0 msgr2=0x7f0f8c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 -- 172.21.15.143:0/2166229726 shutdown_connections 2024-01-15T18:06:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.070+0000 7f0f91c00640 1 -- 172.21.15.143:0/2166229726 wait complete. 2024-01-15T18:06:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 Processor -- start 2024-01-15T18:06:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 -- start start 2024-01-15T18:06:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c0047d0 0x7f0f8c1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8c159910 0x7f0f8c1e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 0x7f0f8c142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c1428d0 con 0x7f0f8c141f00 2024-01-15T18:06:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c13c7f0 con 0x7f0f8c159910 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c13ca00 con 0x7f0f6c0047d0 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 0x7f0f8c142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 0x7f0f8c142390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57520/0 (socket says 172.21.15.143:57520) 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 -- 172.21.15.143:0/3856607013 learned_addr learned my addr 172.21.15.143:0/3856607013 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8aa00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c0047d0 0x7f0f8c1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8a000640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8c159910 0x7f0f8c1e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 -- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c0047d0 msgr2=0x7f0f8c1590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c0047d0 0x7f0f8c1590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 -- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8c159910 msgr2=0x7f0f8c1e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8c159910 0x7f0f8c1e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 -- 172.21.15.143:0/3856607013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c13cce0 con 0x7f0f8c141f00 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f8b400640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 0x7f0f8c142390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0b1b40 tx=0x7f0f84011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400e040 con 0x7f0f8c141f00 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8400f040 con 0x7f0f8c141f00 2024-01-15T18:06:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400fdf0 con 0x7f0f8c141f00 2024-01-15T18:06:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8c13cfc0 con 0x7f0f8c141f00 2024-01-15T18:06:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.074+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8c1595f0 con 0x7f0f8c141f00 2024-01-15T18:06:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.078+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8c13cb40 con 0x7f0f8c141f00 2024-01-15T18:06:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.078+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f0f84015430 con 0x7f0f8c141f00 2024-01-15T18:06:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.078+0000 7f0f82a00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f64066920 0x7f0f64068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.078+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f84084cb0 con 0x7f0f8c141f00 2024-01-15T18:06:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.078+0000 7f0f8aa00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f64066920 0x7f0f64068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.082+0000 7f0f8aa00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f64066920 0x7f0f64068de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c001060 tx=0x7f0f7c001da0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:05.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.082+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0f8c13cb40 con 0x7f0f8c141f00 2024-01-15T18:06:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f8c13cb40 con 0x7f0f8c141f00 2024-01-15T18:06:05.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f82a00640 1 -- 172.21.15.143:0/3856607013 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f8c13cb40 con 0x7f0f8c141f00 2024-01-15T18:06:05.270 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f64066920 msgr2=0x7f0f64068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f64066920 0x7f0f64068de0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c001060 tx=0x7f0f7c001da0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 msgr2=0x7f0f8c142390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 0x7f0f8c142390 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0b1b40 tx=0x7f0f84011e60 comp rx=0 tx=0).stop 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 shutdown_connections 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f64066920 0x7f0f64068de0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f8c141f00 0x7f0f8c142390 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f8c159910 0x7f0f8c1e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 --2- 172.21.15.143:0/3856607013 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f6c0047d0 0x7f0f8c1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 >> 172.21.15.143:0/3856607013 conn(0x7f0f8c06c3f0 msgr2=0x7f0f8c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 shutdown_connections 2024-01-15T18:06:05.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:05.266+0000 7f0f91c00640 1 -- 172.21.15.143:0/3856607013 wait complete. 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:06:05.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:06:05.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.293 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:06:05.302 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:06:05.302 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:06:05.302 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:06:05.302 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:06:05.303 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.304 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:05.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.305 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:06:05.363 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:06:05.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.365 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:06:05.416 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:06:05.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.417 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:06:05.417 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:06:05.417 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:06:05.417 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:06:05.417 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:06:05.417 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:06:05.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.505 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:06:05.515 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:06:05.515 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:06:05.515 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:06:05.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:06:05.582 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:06:05.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:06:05.582 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:06:05.634 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:06:05.634 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:06:05.634 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:06:05.689 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:06:05.754 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:06:05.807 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:06:05.807 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:06:05.807 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:06:05.808 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:06:05.809 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:06:05.810 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:06:05.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.812 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:06:05.862 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:06:05.862 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:06:05.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:06:05.929 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:06:05.983 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:06:05.983 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:06:05.983 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:06:05.983 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:06:05.984 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:06:05.985 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:06:05.986 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:06:05.986 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:06:05.986 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:06:05.986 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:06:05.986 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:06:05.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:05.987 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:06:06.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:06:05.998+0000 7fdbab34a580 -1 init, newargv = 0x56470a4f8e10 newargc=15 2024-01-15T18:06:06.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[37699]: starting ceph client 2024-01-15T18:06:06.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[37699]: starting fuse 2024-01-15T18:06:06.012 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:06:06.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:06:06.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:06:06.012 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:06:06.012 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:06:06.012 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:06:06.013 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:06:06.014 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:06:06.139 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.37699.asok 2024-01-15T18:06:06.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:06:06.150 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37699.asok 2024-01-15T18:06:06.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37699.asok status 2024-01-15T18:06:06.321 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:06.321 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:06:06.321 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "37699", 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:06:06.322 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5415, 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 5415 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:06:06.323 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1077319388 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 1077319388 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.5415 192.168.0.1:0/1077319388", 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/1077319388", 2024-01-15T18:06:06.324 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:06:06.325 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 17, 2024-01-15T18:06:06.325 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 33, 2024-01-15T18:06:06.325 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:06:06.325 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:06:06.325 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:06:06.325 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:06.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.337 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:06:06.337 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:06:06.347 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:06:06.347 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:06:06.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:06:06.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:06:06.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1341e00640 1 Processor -- start 2024-01-15T18:06:06.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1341e00640 1 -- start start 2024-01-15T18:06:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1341e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f133c0b29a0 0x7f133c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1341e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c15f7c0 con 0x7f133c15aef0 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1341e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c06c310 con 0x7f133c06c560 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1341e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c15ebb0 con 0x7f133c0b29a0 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f133aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f133c15aef0 0x7f133c0b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40946/0 (socket says 172.21.15.143:40946) 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f133aa00640 1 -- 172.21.15.143:0/845063834 learned_addr learned my addr 172.21.15.143:0/845063834 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f133a000640 1 --2- 172.21.15.143:0/845063834 >> v2:172.21.15.204:3300/0 conn(0x7f133c0b29a0 0x7f133c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362814882 0 0) 0x7f133c15f7c0 con 0x7f133c15aef0 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.586+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c003530 con 0x7f133c15aef0 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148023454 0 0) 0x7f133c06c310 con 0x7f133c06c560 2024-01-15T18:06:06.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133c15f7c0 con 0x7f133c06c560 2024-01-15T18:06:06.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1052864007 0 0) 0x7f131c003530 con 0x7f133c15aef0 2024-01-15T18:06:06.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f133a000640 1 -- 172.21.15.143:0/845063834 >> v1:172.21.15.204:6789/0 conn(0x7f133c06c560 legacy=0x7f133c0ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f133a000640 1 -- 172.21.15.143:0/845063834 >> v1:172.21.15.143:6789/0 conn(0x7f133c15aef0 legacy=0x7f133c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f133a000640 1 -- 172.21.15.143:0/845063834 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c15ee90 con 0x7f133c0b29a0 2024-01-15T18:06:06.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f133a000640 1 --2- 172.21.15.143:0/845063834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0b29a0 0x7f133c06b320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13240097c0 tx=0x7f132401c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efce633dee75320a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c001680 con 0x7f133c0b29a0 2024-01-15T18:06:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13240a7050 con 0x7f133c0b29a0 2024-01-15T18:06:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.590+0000 7f1339600640 1 -- 172.21.15.143:0/845063834 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240ab620 con 0x7f133c0b29a0 2024-01-15T18:06:06.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- 172.21.15.143:0/845063834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0b29a0 msgr2=0x7f133c06b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 --2- 172.21.15.143:0/845063834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0b29a0 0x7f133c06b320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f13240097c0 tx=0x7f132401c370 comp rx=0 tx=0).stop 2024-01-15T18:06:06.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- 172.21.15.143:0/845063834 shutdown_connections 2024-01-15T18:06:06.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 --2- 172.21.15.143:0/845063834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0b29a0 0x7f133c06b320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- 172.21.15.143:0/845063834 >> 172.21.15.143:0/845063834 conn(0x7f133c066720 msgr2=0x7f133c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:06.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- 172.21.15.143:0/845063834 shutdown_connections 2024-01-15T18:06:06.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- 172.21.15.143:0/845063834 wait complete. 2024-01-15T18:06:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 Processor -- start 2024-01-15T18:06:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- start start 2024-01-15T18:06:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 0x7f133c0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0b29a0 0x7f133c141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c15aef0 0x7f133c15e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0ac8e0 con 0x7f133c06c560 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c15e550 con 0x7f133c15aef0 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.594+0000 7f1341e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0b4fc0 con 0x7f133c0b29a0 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 0x7f133c0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 0x7f133c0b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57526/0 (socket says 172.21.15.143:57526) 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 -- 172.21.15.143:0/2262866726 learned_addr learned my addr 172.21.15.143:0/2262866726 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133be00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c15aef0 0x7f133c15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133a000640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0b29a0 0x7f133c141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 -- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0b29a0 msgr2=0x7f133c141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0b29a0 0x7f133c141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 -- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c15aef0 msgr2=0x7f133c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c15aef0 0x7f133c15e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 -- 172.21.15.143:0/2262866726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009420 con 0x7f133c06c560 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f133aa00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 0x7f133c0b49b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c750 tx=0x7f132c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c005af0 con 0x7f133c06c560 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c005c50 con 0x7f133c06c560 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c0b52a0 con 0x7f133c06c560 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c007070 con 0x7f133c06c560 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c0ace10 con 0x7f133c06c560 2024-01-15T18:06:06.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c0ad000 con 0x7f133c06c560 2024-01-15T18:06:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.598+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f132c0071d0 con 0x7f133c06c560 2024-01-15T18:06:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.602+0000 7f1333400640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13140668f0 0x7f1314068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.602+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f132c0841d0 con 0x7f133c06c560 2024-01-15T18:06:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.602+0000 7f133a000640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13140668f0 0x7f1314068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:06.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.602+0000 7f133a000640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13140668f0 0x7f1314068db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f13240ad000 tx=0x7f13240a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:06.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.606+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f132c051400 con 0x7f133c06c560 2024-01-15T18:06:06.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.790+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f133c067680 con 0x7f133c06c560 2024-01-15T18:06:06.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1333400640 1 -- 172.21.15.143:0/2262866726 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f132c054a20 con 0x7f133c06c560 2024-01-15T18:06:06.799 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:06:06.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13140668f0 msgr2=0x7f1314068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13140668f0 0x7f1314068db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f13240ad000 tx=0x7f13240a6000 comp rx=0 tx=0).stop 2024-01-15T18:06:06.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 msgr2=0x7f133c0b49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:06.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 0x7f133c0b49b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f132c00c750 tx=0x7f132c00cc10 comp rx=0 tx=0).stop 2024-01-15T18:06:06.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 shutdown_connections 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13140668f0 0x7f1314068db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c15aef0 0x7f133c15e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0b29a0 0x7f133c141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 --2- 172.21.15.143:0/2262866726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c06c560 0x7f133c0b49b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 >> 172.21.15.143:0/2262866726 conn(0x7f133c066720 msgr2=0x7f133c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 shutdown_connections 2024-01-15T18:06:06.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:06.794+0000 7f1341e00640 1 -- 172.21.15.143:0/2262866726 wait complete. 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:06:06.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:06:06.822 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:06:06.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.822 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:06:06.833 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:06:06.833 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:06:06.833 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:06:06.833 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.833 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:06:06.834 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42371sec preferred_lft 42371sec 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.835 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.836 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:06:06.837 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:06:06.837 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:06:06.837 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:06:06.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.837 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:06:06.895 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:06:06.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.898 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:06:06.951 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:06:06.951 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:06:06.951 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:06:06.951 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:06:06.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:06.953 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:06:06.953 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:06:06.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:06:06.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:06:06.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:06:06.953 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:06:07.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.045 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:06:07.055 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:06:07.055 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:06:07.056 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:06:07.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.056 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:06:07.116 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:06:07.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:06:07.117 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:06:07.172 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:06:07.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:06:07.173 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:06:07.223 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:06:07.289 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:06:07.341 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:06:07.342 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:06:07.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:06:07.344 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:06:07.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:06:07.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:06:07.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:06:07.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:06:07.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.346 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:06:07.396 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:06:07.396 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:06:07.443 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:06:07.466 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:06:07.518 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:06:07.519 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:06:07.520 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:06:07.521 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:06:07.521 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:06:07.521 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:06:07.521 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:06:07.521 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:06:07.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.522 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:06:07.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:06:07.537+0000 7f3804bab580 -1 init, newargv = 0x55ff3975b3f0 newargc=15 2024-01-15T18:06:07.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33445]: starting ceph client 2024-01-15T18:06:07.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33445]: starting fuse 2024-01-15T18:06:07.550 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:06:07.551 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:06:07.551 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:06:07.552 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:06:07.553 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:06:07.671 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.33445.asok 2024-01-15T18:06:07.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:06:07.683 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33445.asok 2024-01-15T18:06:07.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33445.asok status 2024-01-15T18:06:07.850 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:06:07.850 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:06:07.850 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:06:07.850 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:06:07.850 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "33445", 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 5054, 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:06:07.851 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 5054 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 373159135 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:06:07.852 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 373159135 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.5054 192.168.0.1:0/373159135", 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/373159135", 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 17, 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 33, 2024-01-15T18:06:07.853 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:06:07.854 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:06:07.854 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:06:07.854 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:06:07.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.865 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:06:07.865 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:06:07.875 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:06:07.875 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:06:07.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:07.875 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:06:07.934 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:06:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.078+0000 7f9ab4800640 1 Processor -- start 2024-01-15T18:06:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.078+0000 7f9ab4800640 1 -- start start 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab4800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9aac15d570 0x7f9aac0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0adf10 0x7f9aac0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab4800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac0ad990 con 0x7f9aac0ac060 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab4800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aac0725b0 con 0x7f9aac15d570 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac072890 con 0x7f9aac0adf10 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9aac15d570 0x7f9aac0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9aac15d570 0x7f9aac0ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57398/0 (socket says 172.21.15.143:57398) 2024-01-15T18:06:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9aac0ac060 0x7f9aac0af1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:46250/0 (socket says 172.21.15.143:46250) 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 -- 172.21.15.143:0/2486737236 learned_addr learned my addr 172.21.15.143:0/2486737236 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab2800640 1 --2- 172.21.15.143:0/2486737236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0adf10 0x7f9aac0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 -- 172.21.15.143:0/2486737236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0adf10 msgr2=0x7f9aac0af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 --2- 172.21.15.143:0/2486737236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0adf10 0x7f9aac0af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 -- 172.21.15.143:0/2486737236 >> v1:172.21.15.204:6789/0 conn(0x7f9aac0ac060 legacy=0x7f9aac0af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab0a00640 1 -- 172.21.15.143:0/2486737236 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861051680 0 0) 0x7f9aac0ad990 con 0x7f9aac0ac060 2024-01-15T18:06:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.082+0000 7f9ab1400640 1 -- 172.21.15.143:0/2486737236 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0b3f30 con 0x7f9aac15d570 2024-01-15T18:06:08.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab1400640 1 --2- 172.21.15.143:0/2486737236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac15d570 0x7f9aac0ad450 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009870 tx=0x7f9aac0b0610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f786c7c80534e7d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:08.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab0a00640 1 -- 172.21.15.143:0/2486737236 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940a9620 con 0x7f9aac15d570 2024-01-15T18:06:08.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab0a00640 1 -- 172.21.15.143:0/2486737236 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a940a9780 con 0x7f9aac15d570 2024-01-15T18:06:08.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab0a00640 1 -- 172.21.15.143:0/2486737236 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940a9b10 con 0x7f9aac15d570 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 -- 172.21.15.143:0/2486737236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac15d570 msgr2=0x7f9aac0ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2486737236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac15d570 0x7f9aac0ad450 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9a94009870 tx=0x7f9aac0b0610 comp rx=0 tx=0).stop 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 -- 172.21.15.143:0/2486737236 shutdown_connections 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2486737236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0adf10 0x7f9aac0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2486737236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac15d570 0x7f9aac0ad450 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 -- 172.21.15.143:0/2486737236 >> 172.21.15.143:0/2486737236 conn(0x7f9aac071a90 msgr2=0x7f9aac071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 -- 172.21.15.143:0/2486737236 shutdown_connections 2024-01-15T18:06:08.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 -- 172.21.15.143:0/2486737236 wait complete. 2024-01-15T18:06:08.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.086+0000 7f9ab4800640 1 Processor -- start 2024-01-15T18:06:08.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- start start 2024-01-15T18:06:08.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0ac060 0x7f9aac099250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 0x7f9aac15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aac15f660 0x7f9aac15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac14b0f0 con 0x7f9aac0adf10 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac136b80 con 0x7f9aac15f660 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac160400 con 0x7f9aac0ac060 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 0x7f9aac15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 0x7f9aac15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57400/0 (socket says 172.21.15.143:57400) 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 -- 172.21.15.143:0/2336648074 learned_addr learned my addr 172.21.15.143:0/2336648074 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:08.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1e00640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0ac060 0x7f9aac099250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab2800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aac15f660 0x7f9aac15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 -- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0ac060 msgr2=0x7f9aac099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0ac060 0x7f9aac099250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 -- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aac15f660 msgr2=0x7f9aac15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aac15f660 0x7f9aac15fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 -- 172.21.15.143:0/2336648074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940094d0 con 0x7f9aac0adf10 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab2800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aac15f660 0x7f9aac15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab1400640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 0x7f9aac15f120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9a9401a170 tx=0x7f9a940192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940a9e80 con 0x7f9aac0adf10 2024-01-15T18:06:08.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a940b9430 con 0x7f9aac0adf10 2024-01-15T18:06:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac1606e0 con 0x7f9aac0adf10 2024-01-15T18:06:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a940b97a0 con 0x7f9aac0adf10 2024-01-15T18:06:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac137100 con 0x7f9aac0adf10 2024-01-15T18:06:08.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.090+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac1372f0 con 0x7f9aac0adf10 2024-01-15T18:06:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.094+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f9a940b9900 con 0x7f9aac0adf10 2024-01-15T18:06:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.094+0000 7f9aaa000640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9a840668f0 0x7f9a84068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.094+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a9411e0c0 con 0x7f9aac0adf10 2024-01-15T18:06:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.094+0000 7f9ab1e00640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9a840668f0 0x7f9a84068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.094+0000 7f9ab1e00640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9a840668f0 0x7f9a84068db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c007fd0 tx=0x7f9a9c007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:08.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.098+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a940eb430 con 0x7f9aac0adf10 2024-01-15T18:06:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.278+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9aac067500 con 0x7f9aac0adf10 2024-01-15T18:06:08.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9aaa000640 1 -- 172.21.15.143:0/2336648074 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9a940eea50 con 0x7f9aac0adf10 2024-01-15T18:06:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9a840668f0 msgr2=0x7f9a84068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9a840668f0 0x7f9a84068db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c007fd0 tx=0x7f9a9c007d60 comp rx=0 tx=0).stop 2024-01-15T18:06:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 msgr2=0x7f9aac15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 0x7f9aac15f120 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9a9401a170 tx=0x7f9a940192e0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 shutdown_connections 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9a840668f0 0x7f9a84068db0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aac15f660 0x7f9aac15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9aac0adf10 0x7f9aac15f120 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 --2- 172.21.15.143:0/2336648074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9aac0ac060 0x7f9aac099250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 >> 172.21.15.143:0/2336648074 conn(0x7f9aac071a90 msgr2=0x7f9aac15bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 shutdown_connections 2024-01-15T18:06:08.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.694+0000 7f9ab4800640 1 -- 172.21.15.143:0/2336648074 wait complete. 2024-01-15T18:06:08.722 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:06:08.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6728a00640 1 Processor -- start 2024-01-15T18:06:08.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6728a00640 1 -- start start 2024-01-15T18:06:08.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6728a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6728a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67200ceb50 con 0x7f672015c5a0 2024-01-15T18:06:08.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6728a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672015c110 con 0x7f6720143780 2024-01-15T18:06:08.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6728a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6720072570 con 0x7f6720071bb0 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6726a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f672015c5a0 0x7f672015ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55074/0 (socket says 172.21.15.143:55074) 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6726a00640 1 -- 172.21.15.143:0/2671656308 learned_addr learned my addr 172.21.15.143:0/2671656308 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6725600640 1 --2- 172.21.15.143:0/2671656308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803780810 0 0) 0x7f67180014a0 con 0x7f672015c5a0 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6700003530 con 0x7f672015c5a0 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717844601 0 0) 0x7f672015c110 con 0x7f6720143780 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67180014a0 con 0x7f6720143780 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6725600640 1 -- 172.21.15.143:0/2671656308 >> v1:172.21.15.204:6789/0 conn(0x7f6720143780 legacy=0x7f67200cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6725600640 1 -- 172.21.15.143:0/2671656308 >> v1:172.21.15.143:6789/0 conn(0x7f672015c5a0 legacy=0x7f672015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.854+0000 7f6725600640 1 -- 172.21.15.143:0/2671656308 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672015b7e0 con 0x7f6720071bb0 2024-01-15T18:06:08.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6725600640 1 --2- 172.21.15.143:0/2671656308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720072030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67080097f0 tx=0x7f670801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=373c660f5fb02af0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:08.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67080aa620 con 0x7f6720071bb0 2024-01-15T18:06:08.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67080aa780 con 0x7f6720071bb0 2024-01-15T18:06:08.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6724c00640 1 -- 172.21.15.143:0/2671656308 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67080aab10 con 0x7f6720071bb0 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 -- 172.21.15.143:0/2671656308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 msgr2=0x7f6720072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 --2- 172.21.15.143:0/2671656308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720072030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f67080097f0 tx=0x7f670801c4c0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 -- 172.21.15.143:0/2671656308 shutdown_connections 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 --2- 172.21.15.143:0/2671656308 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720072030 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 -- 172.21.15.143:0/2671656308 >> 172.21.15.143:0/2671656308 conn(0x7f672006c3f0 msgr2=0x7f672006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 -- 172.21.15.143:0/2671656308 shutdown_connections 2024-01-15T18:06:08.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.858+0000 7f6728a00640 1 -- 172.21.15.143:0/2671656308 wait complete. 2024-01-15T18:06:08.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 Processor -- start 2024-01-15T18:06:08.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 -- start start 2024-01-15T18:06:08.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6720143780 0x7f6720132070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f672015c5a0 0x7f6720137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201329c0 con 0x7f672015c5a0 2024-01-15T18:06:08.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6720132ca0 con 0x7f6720143780 2024-01-15T18:06:08.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6720132f80 con 0x7f6720071bb0 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f672015c5a0 0x7f6720137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6725600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6720143780 0x7f6720132070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41558/0 (socket says 172.21.15.143:41558) 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 -- 172.21.15.143:0/143136433 learned_addr learned my addr 172.21.15.143:0/143136433 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 -- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6720143780 msgr2=0x7f6720132070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6720143780 0x7f6720132070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 -- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f672015c5a0 msgr2=0x7f6720137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f672015c5a0 0x7f6720137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 -- 172.21.15.143:0/143136433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6708009420 con 0x7f6720071bb0 2024-01-15T18:06:08.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6725600640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6720143780 0x7f6720132070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:08.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6726000640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720158ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f671000c490 tx=0x7f671000c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:08.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67100052d0 con 0x7f6720071bb0 2024-01-15T18:06:08.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710005430 con 0x7f6720071bb0 2024-01-15T18:06:08.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6720133260 con 0x7f6720071bb0 2024-01-15T18:06:08.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710012870 con 0x7f6720071bb0 2024-01-15T18:06:08.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.862+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201593e0 con 0x7f6720071bb0 2024-01-15T18:06:08.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.866+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6720067500 con 0x7f6720071bb0 2024-01-15T18:06:08.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.866+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f67100129d0 con 0x7f6720071bb0 2024-01-15T18:06:08.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.866+0000 7f671e000640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f66f80668f0 0x7f66f8068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:08.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.866+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6710083f00 con 0x7f6720071bb0 2024-01-15T18:06:08.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.866+0000 7f6725600640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f66f80668f0 0x7f66f8068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:08.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.866+0000 7f6725600640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f66f80668f0 0x7f66f8068db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67080ae000 tx=0x7f67080a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:08.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:08.870+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6710051150 con 0x7f6720071bb0 2024-01-15T18:06:09.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.042+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f67200b4ad0 con 0x7f66f80668f0 2024-01-15T18:06:09.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.042+0000 7f671e000640 1 -- 172.21.15.143:0/143136433 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f67200b4ad0 con 0x7f66f80668f0 2024-01-15T18:06:09.048 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:06:09.048 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:09.048 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:06:09.048 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:09.048 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:06:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.042+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f66f80668f0 msgr2=0x7f66f8068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.042+0000 7f6728a00640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f66f80668f0 0x7f66f8068db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f67080ae000 tx=0x7f67080a7000 comp rx=0 tx=0).stop 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.042+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 msgr2=0x7f6720158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.042+0000 7f6728a00640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720158ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f671000c490 tx=0x7f671000c950 comp rx=0 tx=0).stop 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 shutdown_connections 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f66f80668f0 0x7f66f8068db0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f672015c5a0 0x7f6720137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6720143780 0x7f6720132070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 --2- 172.21.15.143:0/143136433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6720071bb0 0x7f6720158ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 >> 172.21.15.143:0/143136433 conn(0x7f672006c3f0 msgr2=0x7f6720142950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 shutdown_connections 2024-01-15T18:06:09.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.046+0000 7f6728a00640 1 -- 172.21.15.143:0/143136433 wait complete. 2024-01-15T18:06:09.069 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:06:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 Processor -- start 2024-01-15T18:06:09.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 -- start start 2024-01-15T18:06:09.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 0x7ff0f00aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0f015d510 0x7ff0f006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f0160eb0 con 0x7ff0f015d130 2024-01-15T18:06:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f0161190 con 0x7ff0f015d510 2024-01-15T18:06:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f7400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0161470 con 0x7ff0f006aba0 2024-01-15T18:06:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f4a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff0f015d130 0x7ff0f006a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55082/0 (socket says 172.21.15.143:55082) 2024-01-15T18:06:09.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f4a00640 1 -- 172.21.15.143:0/287009425 learned_addr learned my addr 172.21.15.143:0/287009425 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f5400640 1 --2- 172.21.15.143:0/287009425 >> v2:172.21.15.204:3300/0 conn(0x7ff0f015d510 0x7ff0f006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0efe00640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 0x7ff0f00aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554395883 0 0) 0x7ff0f0160eb0 con 0x7ff0f015d130 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d0003530 con 0x7ff0f015d130 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3438044294 0 0) 0x7ff0d0003530 con 0x7ff0f015d130 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 msgr2=0x7ff0f00aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 0x7ff0f00aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 >> v2:172.21.15.204:3300/0 conn(0x7ff0f015d510 msgr2=0x7ff0f006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 --2- 172.21.15.143:0/287009425 >> v2:172.21.15.204:3300/0 conn(0x7ff0f015d510 0x7ff0f006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f0161750 con 0x7ff0f015d130 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff0e0001680 con 0x7ff0f015d130 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0f5400640 1 --2- 172.21.15.143:0/287009425 >> v2:172.21.15.204:3300/0 conn(0x7ff0f015d510 0x7ff0f006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 >> v1:172.21.15.143:6789/0 conn(0x7ff0f015d130 legacy=0x7ff0f006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 0x7ff0d0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0d00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d0004d10 0x7ff0d0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0161190 con 0x7ff0d0003f00 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f0161470 con 0x7ff0d0004d10 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.202+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d000b990 con 0x7ff0f006aba0 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0d00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0f4a00640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 0x7ff0d0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0f5400640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d0004d10 0x7ff0d0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 -- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 msgr2=0x7ff0d0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f006aba0 0x7ff0d0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 -- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d0004d10 msgr2=0x7ff0d0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0d0004d10 0x7ff0d0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 -- 172.21.15.143:0/287009425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0003530 con 0x7ff0d0003f00 2024-01-15T18:06:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0efe00640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0d00047d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009ae0 tx=0x7ff0d801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb56cf207319446e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:09.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80a5640 con 0x7ff0d0003f00 2024-01-15T18:06:09.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d80a57a0 con 0x7ff0d0003f00 2024-01-15T18:06:09.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.206+0000 7ff0ef400640 1 -- 172.21.15.143:0/287009425 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d80a5b50 con 0x7ff0d0003f00 2024-01-15T18:06:09.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- 172.21.15.143:0/287009425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 msgr2=0x7ff0d00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 --2- 172.21.15.143:0/287009425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0d00047d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009ae0 tx=0x7ff0d801c9f0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f5400640 1 -- 172.21.15.143:0/287009425 reap_dead start 2024-01-15T18:06:09.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- 172.21.15.143:0/287009425 shutdown_connections 2024-01-15T18:06:09.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- 172.21.15.143:0/287009425 >> 172.21.15.143:0/287009425 conn(0x7ff0f00b1b90 msgr2=0x7ff0f00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:09.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- 172.21.15.143:0/287009425 shutdown_connections 2024-01-15T18:06:09.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- 172.21.15.143:0/287009425 wait complete. 2024-01-15T18:06:09.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 Processor -- start 2024-01-15T18:06:09.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- start start 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0f0148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f015dd30 0x7ff0f015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0f01481e0 0x7ff0f0145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f015e1b0 con 0x7ff0d0003f00 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f015b1e0 con 0x7ff0f01481e0 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.210+0000 7ff0f7400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f00c4fa0 con 0x7ff0f015dd30 2024-01-15T18:06:09.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0f0148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0f0148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57426/0 (socket says 172.21.15.143:57426) 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 -- 172.21.15.143:0/3160369466 learned_addr learned my addr 172.21.15.143:0/3160369466 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0efe00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f015dd30 0x7ff0f015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f5400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0f01481e0 0x7ff0f0145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 -- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f015dd30 msgr2=0x7ff0f015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f015dd30 0x7ff0f015b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 -- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0f01481e0 msgr2=0x7ff0f0145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0f01481e0 0x7ff0f0145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 -- 172.21.15.143:0/3160369466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f00c5280 con 0x7ff0d0003f00 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f4a00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0f0148b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0012540 tx=0x7ff0e0012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014810 con 0x7ff0d0003f00 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e0014970 con 0x7ff0d0003f00 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f015b480 con 0x7ff0d0003f00 2024-01-15T18:06:09.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0014d20 con 0x7ff0d0003f00 2024-01-15T18:06:09.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f015bee0 con 0x7ff0d0003f00 2024-01-15T18:06:09.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f00b5400 con 0x7ff0d0003f00 2024-01-15T18:06:09.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.214+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7ff0e0016480 con 0x7ff0d0003f00 2024-01-15T18:06:09.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.218+0000 7ff0ecc00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0c8066920 0x7ff0c8068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.218+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0e0087690 con 0x7ff0d0003f00 2024-01-15T18:06:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.218+0000 7ff0efe00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0c8066920 0x7ff0c8068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.218+0000 7ff0efe00640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0c8066920 0x7ff0c8068de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80af000 tx=0x7ff0d80046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:09.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.222+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0e0054a40 con 0x7ff0d0003f00 2024-01-15T18:06:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.370+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff0f015c100 con 0x7ff0d0003f00 2024-01-15T18:06:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.370+0000 7ff0ecc00640 1 -- 172.21.15.143:0/3160369466 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0e0058060 con 0x7ff0d0003f00 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0c8066920 msgr2=0x7ff0c8068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0c8066920 0x7ff0c8068de0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80af000 tx=0x7ff0d80046d0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 msgr2=0x7ff0f0148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0f0148b20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0012540 tx=0x7ff0e0012a00 comp rx=0 tx=0).stop 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 shutdown_connections 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0c8066920 0x7ff0c8068de0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0f01481e0 0x7ff0f0145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0f015dd30 0x7ff0f015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 --2- 172.21.15.143:0/3160369466 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d0003f00 0x7ff0f0148b20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 >> 172.21.15.143:0/3160369466 conn(0x7ff0f00b1b90 msgr2=0x7ff0f00ad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 shutdown_connections 2024-01-15T18:06:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.374+0000 7ff0f7400640 1 -- 172.21.15.143:0/3160369466 wait complete. 2024-01-15T18:06:09.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:06:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.534+0000 7f407c000640 1 Processor -- start 2024-01-15T18:06:09.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.534+0000 7f407c000640 1 -- start start 2024-01-15T18:06:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.534+0000 7f407c000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f40740b3610 0x7f40740cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.534+0000 7f407c000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40740b45d0 0x7f40740cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f407c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40740cefa0 con 0x7f40740b4080 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f407c000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40740c9fa0 con 0x7f40740b45d0 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f407c000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40740b3ef0 con 0x7f40740b3610 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40740b45d0 0x7f40740cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40740b45d0 0x7f40740cff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57428/0 (socket says 172.21.15.143:57428) 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 -- 172.21.15.143:0/1422530283 learned_addr learned my addr 172.21.15.143:0/1422530283 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4079600640 1 --2- 172.21.15.143:0/1422530283 >> v2:172.21.15.204:3300/0 conn(0x7f40740b3610 0x7f40740cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 -- 172.21.15.143:0/1422530283 >> v2:172.21.15.204:3300/0 conn(0x7f40740b3610 msgr2=0x7f40740cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 --2- 172.21.15.143:0/1422530283 >> v2:172.21.15.204:3300/0 conn(0x7f40740b3610 0x7f40740cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 -- 172.21.15.143:0/1422530283 >> v1:172.21.15.204:6789/0 conn(0x7f40740b4080 legacy=0x7f40740cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 -- 172.21.15.143:0/1422530283 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40740d0f90 con 0x7f40740b45d0 2024-01-15T18:06:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.538+0000 7f4078c00640 1 --2- 172.21.15.143:0/1422530283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b45d0 0x7f40740cff80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f407408d580 tx=0x7f405c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a375b8be7f0cd1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f4073e00640 1 -- 172.21.15.143:0/1422530283 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0a89f0 con 0x7f40740b45d0 2024-01-15T18:06:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f4073e00640 1 -- 172.21.15.143:0/1422530283 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c0a8b50 con 0x7f40740b45d0 2024-01-15T18:06:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f4073e00640 1 -- 172.21.15.143:0/1422530283 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0b0440 con 0x7f40740b45d0 2024-01-15T18:06:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 -- 172.21.15.143:0/1422530283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b45d0 msgr2=0x7f40740cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 --2- 172.21.15.143:0/1422530283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b45d0 0x7f40740cff80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f407408d580 tx=0x7f405c01ecf0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 -- 172.21.15.143:0/1422530283 shutdown_connections 2024-01-15T18:06:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 --2- 172.21.15.143:0/1422530283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b45d0 0x7f40740cff80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 --2- 172.21.15.143:0/1422530283 >> v2:172.21.15.204:3300/0 conn(0x7f40740b3610 0x7f40740cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 -- 172.21.15.143:0/1422530283 >> 172.21.15.143:0/1422530283 conn(0x7f40740d3030 msgr2=0x7f40740d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 -- 172.21.15.143:0/1422530283 shutdown_connections 2024-01-15T18:06:09.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 -- 172.21.15.143:0/1422530283 wait complete. 2024-01-15T18:06:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 Processor -- start 2024-01-15T18:06:09.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 -- start start 2024-01-15T18:06:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 0x7f40740cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40740b4080 0x7f40740b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.542+0000 7f407c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740b45d0 0x7f4074142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407c000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40740cb150 con 0x7f40740b3610 2024-01-15T18:06:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40740cb3a0 con 0x7f40740b45d0 2024-01-15T18:06:09.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40740cb680 con 0x7f40740b4080 2024-01-15T18:06:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 0x7f40740cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 0x7f40740cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57436/0 (socket says 172.21.15.143:57436) 2024-01-15T18:06:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 -- 172.21.15.143:0/563400016 learned_addr learned my addr 172.21.15.143:0/563400016 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407a000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740b45d0 0x7f4074142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4078c00640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40740b4080 0x7f40740b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 -- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40740b4080 msgr2=0x7f40740b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40740b4080 0x7f40740b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 -- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740b45d0 msgr2=0x7f4074142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740b45d0 0x7f4074142ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 -- 172.21.15.143:0/563400016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c0094d0 con 0x7f40740b3610 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4079600640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 0x7f40740cc440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f406400d800 tx=0x7f406400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:09.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406400b6c0 con 0x7f40740b3610 2024-01-15T18:06:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406400b820 con 0x7f40740b3610 2024-01-15T18:06:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406400bb90 con 0x7f40740b3610 2024-01-15T18:06:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40740cb960 con 0x7f40740b3610 2024-01-15T18:06:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40740cbf20 con 0x7f40740b3610 2024-01-15T18:06:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4074067500 con 0x7f40740b3610 2024-01-15T18:06:09.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.546+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f406400bcf0 con 0x7f40740b3610 2024-01-15T18:06:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.550+0000 7f4071600640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f404c0668f0 0x7f404c068db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.550+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40640853a0 con 0x7f40740b3610 2024-01-15T18:06:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.550+0000 7f4078c00640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f404c0668f0 0x7f404c068db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:09.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.550+0000 7f4078c00640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f404c0668f0 0x7f404c068db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f405c0b2000 tx=0x7f405c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:09.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.550+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4064052670 con 0x7f40740b3610 2024-01-15T18:06:09.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.726+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f40740747b0 con 0x7f404c0668f0 2024-01-15T18:06:09.736 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:09.730+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:09.736 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:09.730+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:09.736 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:09.730+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:09.736 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:09.730+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:09.736 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:09.730+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:09.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.890+0000 7f4071600640 1 -- 172.21.15.143:0/563400016 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f40740747b0 con 0x7f404c0668f0 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f404c0668f0 msgr2=0x7f404c068db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f404c0668f0 0x7f404c068db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f405c0b2000 tx=0x7f405c009040 comp rx=0 tx=0).stop 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 msgr2=0x7f40740cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 0x7f40740cc440 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f406400d800 tx=0x7f406400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 shutdown_connections 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f404c0668f0 0x7f404c068db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40740b45d0 0x7f4074142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40740b4080 0x7f40740b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 --2- 172.21.15.143:0/563400016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40740b3610 0x7f40740cc440 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 >> 172.21.15.143:0/563400016 conn(0x7f40740d3030 msgr2=0x7f40740d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 shutdown_connections 2024-01-15T18:06:09.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:09.894+0000 7f407c000640 1 -- 172.21.15.143:0/563400016 wait complete. 2024-01-15T18:06:09.920 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:06:10.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.054+0000 7f7771600640 1 Processor -- start 2024-01-15T18:06:10.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.054+0000 7f7771600640 1 -- start start 2024-01-15T18:06:10.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f7771600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f776c141660 0x7f776c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:10.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f7771600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f776c071d00 0x7f776c072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:10.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f7771600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f776c072650 con 0x7f776c15ec80 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f7771600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c15fbb0 con 0x7f776c141660 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f7771600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c15f130 con 0x7f776c071d00 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f776c141660 0x7f776c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f776c15ec80 0x7f776c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55088/0 (socket says 172.21.15.143:55088) 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776a000640 1 -- 172.21.15.143:0/2470888725 learned_addr learned my addr 172.21.15.143:0/2470888725 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776b400640 1 --2- 172.21.15.143:0/2470888725 >> v2:172.21.15.204:3300/0 conn(0x7f776c071d00 0x7f776c072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776aa00640 1 -- 172.21.15.143:0/2470888725 >> v2:172.21.15.204:3300/0 conn(0x7f776c071d00 msgr2=0x7f776c072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776aa00640 1 --2- 172.21.15.143:0/2470888725 >> v2:172.21.15.204:3300/0 conn(0x7f776c071d00 0x7f776c072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776aa00640 1 -- 172.21.15.143:0/2470888725 >> v1:172.21.15.143:6789/0 conn(0x7f776c15ec80 legacy=0x7f776c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776aa00640 1 -- 172.21.15.143:0/2470888725 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c15f310 con 0x7f776c141660 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f7769600640 1 -- 172.21.15.143:0/2470888725 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620717480 0 0) 0x7f7754001120 con 0x7f776c15ec80 2024-01-15T18:06:10.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.058+0000 7f776b400640 1 --2- 172.21.15.143:0/2470888725 >> v2:172.21.15.204:3300/0 conn(0x7f776c071d00 0x7f776c072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:10.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f776aa00640 1 --2- 172.21.15.143:0/2470888725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c141660 0x7f776c0ce560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f776c06e250 tx=0x7f775c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf00359d71827cd9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:10.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7769600640 1 -- 172.21.15.143:0/2470888725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c0a7070 con 0x7f776c141660 2024-01-15T18:06:10.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7769600640 1 -- 172.21.15.143:0/2470888725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c0ab430 con 0x7f776c141660 2024-01-15T18:06:10.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7769600640 1 -- 172.21.15.143:0/2470888725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c0ab7c0 con 0x7f776c141660 2024-01-15T18:06:10.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 -- 172.21.15.143:0/2470888725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c141660 msgr2=0x7f776c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:10.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 --2- 172.21.15.143:0/2470888725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c141660 0x7f776c0ce560 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f776c06e250 tx=0x7f775c01c430 comp rx=0 tx=0).stop 2024-01-15T18:06:10.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 -- 172.21.15.143:0/2470888725 shutdown_connections 2024-01-15T18:06:10.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 --2- 172.21.15.143:0/2470888725 >> v2:172.21.15.204:3300/0 conn(0x7f776c071d00 0x7f776c072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 --2- 172.21.15.143:0/2470888725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c141660 0x7f776c0ce560 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 -- 172.21.15.143:0/2470888725 >> 172.21.15.143:0/2470888725 conn(0x7f776c06c3f0 msgr2=0x7f776c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:10.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 -- 172.21.15.143:0/2470888725 shutdown_connections 2024-01-15T18:06:10.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 -- 172.21.15.143:0/2470888725 wait complete. 2024-01-15T18:06:10.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 Processor -- start 2024-01-15T18:06:10.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.062+0000 7f7771600640 1 -- start start 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c071d00 0x7f776c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c141660 0x7f776c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 0x7f776c0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c15cfa0 con 0x7f776c15ec80 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c15b190 con 0x7f776c071d00 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c15cc70 con 0x7f776c141660 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 0x7f776c0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:10.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 0x7f776c0b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57476/0 (socket says 172.21.15.143:57476) 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 -- 172.21.15.143:0/3448099884 learned_addr learned my addr 172.21.15.143:0/3448099884 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776aa00640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c071d00 0x7f776c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776a000640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c141660 0x7f776c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 -- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c141660 msgr2=0x7f776c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c141660 0x7f776c15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 -- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c071d00 msgr2=0x7f776c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c071d00 0x7f776c0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 -- 172.21.15.143:0/3448099884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c009420 con 0x7f776c15ec80 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f776b400640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 0x7f776c0b1ea0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7764007e40 tx=0x7f776400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764003ba0 con 0x7f776c15ec80 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776400f040 con 0x7f776c15ec80 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c0b23e0 con 0x7f776c15ec80 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776400de80 con 0x7f776c15ec80 2024-01-15T18:06:10.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c0b2eb0 con 0x7f776c15ec80 2024-01-15T18:06:10.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.066+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c15d0e0 con 0x7f776c15ec80 2024-01-15T18:06:10.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.070+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 86160+0+0 (secure 0 0 0) 0x7f7764004030 con 0x7f776c15ec80 2024-01-15T18:06:10.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.070+0000 7f7762a00640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7744066920 0x7f7744068de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:10.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.070+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77640846b0 con 0x7f776c15ec80 2024-01-15T18:06:10.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.070+0000 7f776aa00640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7744066920 0x7f7744068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:10.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.070+0000 7f776aa00640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7744066920 0x7f7744068de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f775c0af000 tx=0x7f775c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:10.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.074+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f776c15d0e0 con 0x7f776c15ec80 2024-01-15T18:06:10.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.246+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f776c142640 con 0x7f7744066920 2024-01-15T18:06:10.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.250+0000 7f7762a00640 1 -- 172.21.15.143:0/3448099884 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f775c004300 con 0x7f7744066920 2024-01-15T18:06:10.257 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7744066920 msgr2=0x7f7744068de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7744066920 0x7f7744068de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f775c0af000 tx=0x7f775c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 msgr2=0x7f776c0b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 0x7f776c0b1ea0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7764007e40 tx=0x7f776400d4a0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 shutdown_connections 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7744066920 0x7f7744068de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f776c15ec80 0x7f776c0b1ea0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f776c141660 0x7f776c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 --2- 172.21.15.143:0/3448099884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f776c071d00 0x7f776c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 >> 172.21.15.143:0/3448099884 conn(0x7f776c06c3f0 msgr2=0x7f776c0b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 shutdown_connections 2024-01-15T18:06:10.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:10.254+0000 7f7771600640 1 -- 172.21.15.143:0/3448099884 wait complete. 2024-01-15T18:06:10.278 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699 2024-01-15T18:06:10.278 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:06:10.278 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:06:10.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:10.287 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:06:17.436 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:06:17.437 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:06:17.437 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.0793 s, 152 MB/s 2024-01-15T18:06:17.438 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:06:17.438 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:06:17.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:17.448 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:06:24.357 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:06:24.357 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:06:24.357 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85013 s, 157 MB/s 2024-01-15T18:06:24.359 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:06:24.359 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:06:24.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:24.369 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:06:27.446 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:27.451 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:31.346 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:06:31.346 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:06:31.346 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91949 s, 155 MB/s 2024-01-15T18:06:31.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T18:06:31.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 Processor -- start 2024-01-15T18:06:31.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 -- start start 2024-01-15T18:06:31.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59e40b2790 0x7f59e4159930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f59e40745d0 0x7f59e40ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e40ac440 con 0x7f59e4159550 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e4159e70 con 0x7f59e40b2790 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.477+0000 7f59e9a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e40ac720 con 0x7f59e40745d0 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59e40b2790 0x7f59e4159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59e40b2790 0x7f59e4159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51518/0 (socket says 172.21.15.143:51518) 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 -- 172.21.15.143:0/2249790069 learned_addr learned my addr 172.21.15.143:0/2249790069 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e3400640 1 --2- 172.21.15.143:0/2249790069 >> v2:172.21.15.204:3300/0 conn(0x7f59e40745d0 0x7f59e40ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 -- 172.21.15.143:0/2249790069 >> v2:172.21.15.204:3300/0 conn(0x7f59e40745d0 msgr2=0x7f59e40ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 --2- 172.21.15.143:0/2249790069 >> v2:172.21.15.204:3300/0 conn(0x7f59e40745d0 0x7f59e40ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 -- 172.21.15.143:0/2249790069 >> v1:172.21.15.204:6789/0 conn(0x7f59e4159550 legacy=0x7f59e40b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 -- 172.21.15.143:0/2249790069 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40aca00 con 0x7f59e40b2790 2024-01-15T18:06:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.481+0000 7f59e2000640 1 --2- 172.21.15.143:0/2249790069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e4159930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f59e40d0230 tx=0x7f59cc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c097ac2248f07dde server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e1600640 1 -- 172.21.15.143:0/2249790069 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0aa430 con 0x7f59e40b2790 2024-01-15T18:06:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e1600640 1 -- 172.21.15.143:0/2249790069 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc0aa590 con 0x7f59e40b2790 2024-01-15T18:06:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e1600640 1 -- 172.21.15.143:0/2249790069 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0aa920 con 0x7f59e40b2790 2024-01-15T18:06:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- 172.21.15.143:0/2249790069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 msgr2=0x7f59e4159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 --2- 172.21.15.143:0/2249790069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e4159930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f59e40d0230 tx=0x7f59cc01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- 172.21.15.143:0/2249790069 shutdown_connections 2024-01-15T18:06:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 --2- 172.21.15.143:0/2249790069 >> v2:172.21.15.204:3300/0 conn(0x7f59e40745d0 0x7f59e40ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 --2- 172.21.15.143:0/2249790069 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e4159930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- 172.21.15.143:0/2249790069 >> 172.21.15.143:0/2249790069 conn(0x7f59e4071a90 msgr2=0x7f59e4071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- 172.21.15.143:0/2249790069 shutdown_connections 2024-01-15T18:06:31.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- 172.21.15.143:0/2249790069 wait complete. 2024-01-15T18:06:31.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 Processor -- start 2024-01-15T18:06:31.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- start start 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e40745d0 0x7f59e414b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e414ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e4159550 0x7f59e4097ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e414f210 con 0x7f59e40b2790 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4154e20 con 0x7f59e40745d0 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.485+0000 7f59e9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4155100 con 0x7f59e4159550 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e414ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e40745d0 0x7f59e414b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e414ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51532/0 (socket says 172.21.15.143:51532) 2024-01-15T18:06:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 -- 172.21.15.143:0/1484158320 learned_addr learned my addr 172.21.15.143:0/1484158320 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:31.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e3400640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e4159550 0x7f59e4097ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 -- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e4159550 msgr2=0x7f59e4097ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e4159550 0x7f59e4097ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 -- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e40745d0 msgr2=0x7f59e414b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e40745d0 0x7f59e414b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 -- 172.21.15.143:0/1484158320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0094d0 con 0x7f59e40b2790 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e40745d0 0x7f59e414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e2000640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e414ecd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0190e0 tx=0x7f59cc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0aa430 con 0x7f59e40b2790 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc0b3820 con 0x7f59e40b2790 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0b3bd0 con 0x7f59e40b2790 2024-01-15T18:06:31.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e41553e0 con 0x7f59e40b2790 2024-01-15T18:06:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e4098530 con 0x7f59e40b2790 2024-01-15T18:06:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e40cf540 con 0x7f59e40b2790 2024-01-15T18:06:31.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.489+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 86211+0+0 (secure 0 0 0) 0x7f59cc0b3d30 con 0x7f59e40b2790 2024-01-15T18:06:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.493+0000 7f59daa00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59bc066980 0x7f59bc068e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.493+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59cc11e6f0 con 0x7f59e40b2790 2024-01-15T18:06:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.493+0000 7f59e2a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59bc066980 0x7f59bc068e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.493+0000 7f59e2a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59bc066980 0x7f59bc068e40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59d4007fd0 tx=0x7f59d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:31.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.497+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f59cc0eb970 con 0x7f59e40b2790 2024-01-15T18:06:31.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.645+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f59e4067500 con 0x7f59e40b2790 2024-01-15T18:06:31.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.649+0000 7f59daa00640 1 -- 172.21.15.143:0/1484158320 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v11)=0 v11) v1 ==== 135+0+0 (secure 0 0 0) 0x7f59cc0eef90 con 0x7f59e40b2790 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59bc066980 msgr2=0x7f59bc068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59bc066980 0x7f59bc068e40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f59d4007fd0 tx=0x7f59d4009040 comp rx=0 tx=0).stop 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 msgr2=0x7f59e414ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e414ecd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0190e0 tx=0x7f59cc01ad10 comp rx=0 tx=0).stop 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 shutdown_connections 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59bc066980 0x7f59bc068e40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e4159550 0x7f59e4097ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e40b2790 0x7f59e414ecd0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 --2- 172.21.15.143:0/1484158320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e40745d0 0x7f59e414b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 >> 172.21.15.143:0/1484158320 conn(0x7f59e4071a90 msgr2=0x7f59e406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 shutdown_connections 2024-01-15T18:06:31.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.653+0000 7f59e9a00640 1 -- 172.21.15.143:0/1484158320 wait complete. 2024-01-15T18:06:31.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:06:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.813+0000 7f743f400640 1 Processor -- start 2024-01-15T18:06:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.813+0000 7f743f400640 1 -- start start 2024-01-15T18:06:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.813+0000 7f743f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7438143780 0x7f74380cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.813+0000 7f743f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74380ce520 0x7f7438071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.813+0000 7f743f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74380ceb50 con 0x7f74380720f0 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f743f400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743815c110 con 0x7f74380ce520 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f743f400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7438072530 con 0x7f7438143780 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74380ce520 0x7f7438071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74380ce520 0x7f7438071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51546/0 (socket says 172.21.15.143:51546) 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 -- 172.21.15.143:0/4075813854 learned_addr learned my addr 172.21.15.143:0/4075813854 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f743ca00640 1 --2- 172.21.15.143:0/4075813854 >> v2:172.21.15.204:3300/0 conn(0x7f7438143780 0x7f74380cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 -- 172.21.15.143:0/4075813854 >> v2:172.21.15.204:3300/0 conn(0x7f7438143780 msgr2=0x7f74380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 --2- 172.21.15.143:0/4075813854 >> v2:172.21.15.204:3300/0 conn(0x7f7438143780 0x7f74380cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 -- 172.21.15.143:0/4075813854 >> v1:172.21.15.204:6789/0 conn(0x7f74380720f0 legacy=0x7f743815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 -- 172.21.15.143:0/4075813854 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743815b7e0 con 0x7f74380ce520 2024-01-15T18:06:31.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f743ca00640 1 --2- 172.21.15.143:0/4075813854 >> v2:172.21.15.204:3300/0 conn(0x7f7438143780 0x7f74380cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:31.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.817+0000 7f7437e00640 1 --2- 172.21.15.143:0/4075813854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438071bb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f743806be30 tx=0x7f742001e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a574b5428c8023a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f7437400640 1 -- 172.21.15.143:0/4075813854 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200aa650 con 0x7f74380ce520 2024-01-15T18:06:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f7437400640 1 -- 172.21.15.143:0/4075813854 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74200aa7b0 con 0x7f74380ce520 2024-01-15T18:06:31.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f7437400640 1 -- 172.21.15.143:0/4075813854 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200aab40 con 0x7f74380ce520 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- 172.21.15.143:0/4075813854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 msgr2=0x7f7438071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 --2- 172.21.15.143:0/4075813854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438071bb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f743806be30 tx=0x7f742001e710 comp rx=0 tx=0).stop 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- 172.21.15.143:0/4075813854 shutdown_connections 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 --2- 172.21.15.143:0/4075813854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438071bb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 --2- 172.21.15.143:0/4075813854 >> v2:172.21.15.204:3300/0 conn(0x7f7438143780 0x7f74380cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- 172.21.15.143:0/4075813854 >> 172.21.15.143:0/4075813854 conn(0x7f743806c3f0 msgr2=0x7f743806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:31.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- 172.21.15.143:0/4075813854 shutdown_connections 2024-01-15T18:06:31.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- 172.21.15.143:0/4075813854 wait complete. 2024-01-15T18:06:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 Processor -- start 2024-01-15T18:06:31.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- start start 2024-01-15T18:06:31.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74380720f0 0x7f7438137330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438138700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438143780 0x7f7438141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438137960 con 0x7f74380ce520 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438137c40 con 0x7f74380720f0 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.821+0000 7f743f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7438137f20 con 0x7f7438143780 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438138700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438138700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51558/0 (socket says 172.21.15.143:51558) 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 -- 172.21.15.143:0/2200247946 learned_addr learned my addr 172.21.15.143:0/2200247946 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f743ca00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74380720f0 0x7f7438137330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f743d400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438143780 0x7f7438141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 -- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438143780 msgr2=0x7f7438141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438143780 0x7f7438141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 -- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74380720f0 msgr2=0x7f7438137330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:31.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74380720f0 0x7f7438137330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 -- 172.21.15.143:0/2200247946 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200094d0 con 0x7f74380ce520 2024-01-15T18:06:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7437e00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438138700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f742001c340 tx=0x7f742001b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:31.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200bb430 con 0x7f74380ce520 2024-01-15T18:06:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74200b6070 con 0x7f74380ce520 2024-01-15T18:06:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7438138200 con 0x7f74380ce520 2024-01-15T18:06:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200bb7b0 con 0x7f74380ce520 2024-01-15T18:06:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7438142500 con 0x7f74380ce520 2024-01-15T18:06:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7438067500 con 0x7f74380ce520 2024-01-15T18:06:31.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.825+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 86211+0+0 (secure 0 0 0) 0x7f74200b8030 con 0x7f74380ce520 2024-01-15T18:06:31.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.829+0000 7f7434c00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066980 0x7f7410068e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:31.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.829+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f742011e490 con 0x7f74380ce520 2024-01-15T18:06:31.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.829+0000 7f743ca00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066980 0x7f7410068e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:31.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.829+0000 7f743ca00640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066980 0x7f7410068e40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7438142880 tx=0x7f7428008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:31.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:31.833+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74200eb8b0 con 0x7f74380ce520 2024-01-15T18:06:32.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.009+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7438139800 con 0x7f7410066980 2024-01-15T18:06:32.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.021+0000 7f7434c00640 1 -- 172.21.15.143:0/2200247946 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7438139800 con 0x7f7410066980 2024-01-15T18:06:32.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066980 msgr2=0x7f7410068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066980 0x7f7410068e40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7438142880 tx=0x7f7428008040 comp rx=0 tx=0).stop 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 msgr2=0x7f7438138700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438138700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f742001c340 tx=0x7f742001b4b0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 shutdown_connections 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066980 0x7f7410068e40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438143780 0x7f7438141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74380ce520 0x7f7438138700 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 --2- 172.21.15.143:0/2200247946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74380720f0 0x7f7438137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 >> 172.21.15.143:0/2200247946 conn(0x7f743806c3f0 msgr2=0x7f743815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 shutdown_connections 2024-01-15T18:06:32.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.025+0000 7f743f400640 1 -- 172.21.15.143:0/2200247946 wait complete. 2024-01-15T18:06:32.052 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:06:32.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 Processor -- start 2024-01-15T18:06:32.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 -- start start 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6940ce690 0x7fa6940ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa6940b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa694070bf0 con 0x7fa6940b1da0 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69415dba0 con 0x7fa6940ce690 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940b4920 con 0x7fa6940b3bf0 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6940ce690 0x7fa6940ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6940ce690 0x7fa6940ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51562/0 (socket says 172.21.15.143:51562) 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 -- 172.21.15.143:0/1154694068 learned_addr learned my addr 172.21.15.143:0/1154694068 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69a000640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa6940b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 -- 172.21.15.143:0/1154694068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 msgr2=0x7fa6940b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa6940b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 -- 172.21.15.143:0/1154694068 >> v1:172.21.15.204:6789/0 conn(0x7fa6940b1da0 legacy=0x7fa6940b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:32.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa699600640 1 -- 172.21.15.143:0/1154694068 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940b4c00 con 0x7fa6940ce690 2024-01-15T18:06:32.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.185+0000 7fa69a000640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa6940b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:32.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa699600640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6940ac8e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa684009870 tx=0x7fa68401cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec7f9763a7934c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:32.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa693e00640 1 -- 172.21.15.143:0/1154694068 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6840a95f0 con 0x7fa6940ce690 2024-01-15T18:06:32.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa693e00640 1 -- 172.21.15.143:0/1154694068 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6840a9750 con 0x7fa6940ce690 2024-01-15T18:06:32.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa693e00640 1 -- 172.21.15.143:0/1154694068 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6840a9ae0 con 0x7fa6940ce690 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 -- 172.21.15.143:0/1154694068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 msgr2=0x7fa6940ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6940ac8e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa684009870 tx=0x7fa68401cf70 comp rx=0 tx=0).stop 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 -- 172.21.15.143:0/1154694068 shutdown_connections 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa6940b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 --2- 172.21.15.143:0/1154694068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6940ac8e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 -- 172.21.15.143:0/1154694068 >> 172.21.15.143:0/1154694068 conn(0x7fa69406c3f0 msgr2=0x7fa69406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:32.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 -- 172.21.15.143:0/1154694068 shutdown_connections 2024-01-15T18:06:32.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.189+0000 7fa69c000640 1 -- 172.21.15.143:0/1154694068 wait complete. 2024-01-15T18:06:32.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 Processor -- start 2024-01-15T18:06:32.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 -- start start 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940b1da0 0x7fa694159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa694144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6941454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa694159550 con 0x7fa6940ce690 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940b01e0 con 0x7fa6940b1da0 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940af5d0 con 0x7fa6940b3bf0 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6941454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:32.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940b1da0 0x7fa694159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6941454c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51572/0 (socket says 172.21.15.143:51572) 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69a000640 1 -- 172.21.15.143:0/1142951539 learned_addr learned my addr 172.21.15.143:0/1142951539 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa698c00640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa694144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 -- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 msgr2=0x7fa694144f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa694144f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 -- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 msgr2=0x7fa6941454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6941454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 -- 172.21.15.143:0/1142951539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6840094d0 con 0x7fa6940b1da0 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69a000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6941454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:32.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa699600640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940b1da0 0x7fa694159010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa6840aa000 tx=0x7fa6840192b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:32.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6840b1800 con 0x7fa6940b1da0 2024-01-15T18:06:32.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6840b1980 con 0x7fa6940b1da0 2024-01-15T18:06:32.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6940af8b0 con 0x7fa6940b1da0 2024-01-15T18:06:32.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6840b1d30 con 0x7fa6940b1da0 2024-01-15T18:06:32.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.193+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6940afe70 con 0x7fa6940b1da0 2024-01-15T18:06:32.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.197+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa694067500 con 0x7fa6940b1da0 2024-01-15T18:06:32.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.197+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 6) v1 ==== 86211+0+0 (secure 0 0 0) 0x7fa6840c23b0 con 0x7fa6940b1da0 2024-01-15T18:06:32.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.197+0000 7fa691600640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa66c0669b0 0x7fa66c068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:32.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.197+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa68411ea40 con 0x7fa6940b1da0 2024-01-15T18:06:32.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.197+0000 7fa698c00640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa66c0669b0 0x7fa66c068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:32.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.197+0000 7fa698c00640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa66c0669b0 0x7fa66c068e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa694143150 tx=0x7fa67c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:32.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.201+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6840ebd50 con 0x7fa6940b1da0 2024-01-15T18:06:32.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.373+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa694167ef0 con 0x7fa66c0669b0 2024-01-15T18:06:32.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.377+0000 7fa691600640 1 -- 172.21.15.143:0/1142951539 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fa694167ef0 con 0x7fa66c0669b0 2024-01-15T18:06:32.385 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699 2024-01-15T18:06:32.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa66c0669b0 msgr2=0x7fa66c068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa66c0669b0 0x7fa66c068e70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa694143150 tx=0x7fa67c009040 comp rx=0 tx=0).stop 2024-01-15T18:06:32.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940b1da0 msgr2=0x7fa694159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:32.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940b1da0 0x7fa694159010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa6840aa000 tx=0x7fa6840192b0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 shutdown_connections 2024-01-15T18:06:32.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa66c0669b0 0x7fa66c068e70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6940ce690 0x7fa6941454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6940b3bf0 0x7fa694144f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 --2- 172.21.15.143:0/1142951539 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa6940b1da0 0x7fa694159010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 >> 172.21.15.143:0/1142951539 conn(0x7fa69406c3f0 msgr2=0x7fa6940ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 shutdown_connections 2024-01-15T18:06:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:32.381+0000 7fa69c000640 1 -- 172.21.15.143:0/1142951539 wait complete. 2024-01-15T18:06:32.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:32.407 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699) 2024-01-15T18:06:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699 2024-01-15T18:06:32.426 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:06:32.426 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:32.427 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:06:34.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:34.428 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699) 2024-01-15T18:06:34.443 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699 2024-01-15T18:06:34.444 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:06:34.444 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:34.445 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:06:36.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:36.447 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699) 2024-01-15T18:06:36.461 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/29c49ed7-21bc-40b0-a171-db0251a1b699 2024-01-15T18:06:36.461 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:06:36.461 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:36.462 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:06:36.463 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-15T18:06:36.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.649+0000 7f71c5800640 1 Processor -- start 2024-01-15T18:06:36.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71c5800640 1 -- start start 2024-01-15T18:06:36.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71c5800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71c006bfb0 0x7f71c006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:36.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71c5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71c00d0210 0x7f71c00ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:36.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71c5800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c00cbff0 con 0x7f71c00b0310 2024-01-15T18:06:36.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71c5800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c00d0c50 con 0x7f71c00d0210 2024-01-15T18:06:36.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71c5800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c006c8d0 con 0x7f71c006bfb0 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71c00d0210 0x7f71c00ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71c00d0210 0x7f71c00ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51576/0 (socket says 172.21.15.143:51576) 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 -- 172.21.15.143:0/1264029656 learned_addr learned my addr 172.21.15.143:0/1264029656 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bea00640 1 --2- 172.21.15.143:0/1264029656 >> v2:172.21.15.204:3300/0 conn(0x7f71c006bfb0 0x7f71c006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 -- 172.21.15.143:0/1264029656 >> v2:172.21.15.204:3300/0 conn(0x7f71c006bfb0 msgr2=0x7f71c006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 --2- 172.21.15.143:0/1264029656 >> v2:172.21.15.204:3300/0 conn(0x7f71c006bfb0 0x7f71c006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 -- 172.21.15.143:0/1264029656 >> v1:172.21.15.204:6789/0 conn(0x7f71c00b0310 legacy=0x7f71c00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.653+0000 7f71bf400640 1 -- 172.21.15.143:0/1264029656 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c00cb6c0 con 0x7f71c00d0210 2024-01-15T18:06:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71bf400640 1 --2- 172.21.15.143:0/1264029656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c00ccb90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71c00716c0 tx=0x7f71b801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f738104f26913534 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71bd600640 1 -- 172.21.15.143:0/1264029656 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80a57d0 con 0x7f71c00d0210 2024-01-15T18:06:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71bd600640 1 -- 172.21.15.143:0/1264029656 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b80a5930 con 0x7f71c00d0210 2024-01-15T18:06:36.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71bd600640 1 -- 172.21.15.143:0/1264029656 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80a5cc0 con 0x7f71c00d0210 2024-01-15T18:06:36.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 -- 172.21.15.143:0/1264029656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 msgr2=0x7f71c00ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 --2- 172.21.15.143:0/1264029656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c00ccb90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71c00716c0 tx=0x7f71b801c910 comp rx=0 tx=0).stop 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 -- 172.21.15.143:0/1264029656 shutdown_connections 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 --2- 172.21.15.143:0/1264029656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c00ccb90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 --2- 172.21.15.143:0/1264029656 >> v2:172.21.15.204:3300/0 conn(0x7f71c006bfb0 0x7f71c006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 -- 172.21.15.143:0/1264029656 >> 172.21.15.143:0/1264029656 conn(0x7f71c00d0ed0 msgr2=0x7f71c00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 -- 172.21.15.143:0/1264029656 shutdown_connections 2024-01-15T18:06:36.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.657+0000 7f71c5800640 1 -- 172.21.15.143:0/1264029656 wait complete. 2024-01-15T18:06:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 Processor -- start 2024-01-15T18:06:36.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- start start 2024-01-15T18:06:36.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c006bfb0 0x7f71c00c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:36.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c00b0310 0x7f71c01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:36.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c0157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:36.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0157c50 con 0x7f71c00d0210 2024-01-15T18:06:36.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00c8b50 con 0x7f71c00b0310 2024-01-15T18:06:36.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c00c9e10 con 0x7f71c006bfb0 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c0157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c0157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51586/0 (socket says 172.21.15.143:51586) 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 -- 172.21.15.143:0/1948803845 learned_addr learned my addr 172.21.15.143:0/1948803845 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bea00640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c006bfb0 0x7f71c00c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71be000640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c00b0310 0x7f71c01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 -- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c006bfb0 msgr2=0x7f71c00c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c006bfb0 0x7f71c00c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 -- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c00b0310 msgr2=0x7f71c01548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c00b0310 0x7f71c01548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 -- 172.21.15.143:0/1948803845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b8009b30 con 0x7f71c00d0210 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71bf400640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c0157400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f71b8019520 tx=0x7f71b80196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80b8670 con 0x7f71c00d0210 2024-01-15T18:06:36.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b80b87d0 con 0x7f71c00d0210 2024-01-15T18:06:36.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c00c8df0 con 0x7f71c00d0210 2024-01-15T18:06:36.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80b8b80 con 0x7f71c00d0210 2024-01-15T18:06:36.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c00c93b0 con 0x7f71c00d0210 2024-01-15T18:06:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.661+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c00c95d0 con 0x7f71c00d0210 2024-01-15T18:06:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.665+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 86211+0+0 (secure 0 0 0) 0x7f71b80b8ce0 con 0x7f71c00d0210 2024-01-15T18:06:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.665+0000 7f71b6a00640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7198066980 0x7f7198068e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.665+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71b811fa20 con 0x7f71c00d0210 2024-01-15T18:06:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.665+0000 7f71bea00640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7198066980 0x7f7198068e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:36.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.665+0000 7f71bea00640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7198066980 0x7f7198068e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f71b00079c0 tx=0x7f71b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:36.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.669+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f71b80ecd40 con 0x7f71c00d0210 2024-01-15T18:06:36.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.849+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f71c00ce660 con 0x7f7198066980 2024-01-15T18:06:36.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.909+0000 7f71b6a00640 1 -- 172.21.15.143:0/1948803845 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f71c00ce660 con 0x7f7198066980 2024-01-15T18:06:36.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7198066980 msgr2=0x7f7198068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7198066980 0x7f7198068e40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f71b00079c0 tx=0x7f71b0008040 comp rx=0 tx=0).stop 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 msgr2=0x7f71c0157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c0157400 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f71b8019520 tx=0x7f71b80196b0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 shutdown_connections 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7198066980 0x7f7198068e40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c00d0210 0x7f71c0157400 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c00b0310 0x7f71c01548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 --2- 172.21.15.143:0/1948803845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c006bfb0 0x7f71c00c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 >> 172.21.15.143:0/1948803845 conn(0x7f71c00d0ed0 msgr2=0x7f71c00b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 shutdown_connections 2024-01-15T18:06:36.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:36.913+0000 7f71c5800640 1 -- 172.21.15.143:0/1948803845 wait complete. 2024-01-15T18:06:36.922 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.917+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.922 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.917+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.922 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.917+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.922 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.917+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.922 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.917+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.941 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.933+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.941 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.933+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.941 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.933+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.941 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.933+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.941 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:36.933+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:36.943 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:06:36.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 Processor -- start 2024-01-15T18:06:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 -- start start 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1f806bfb0 0x7fe1f806c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1f80b0310 0x7fe1f80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f80cbff0 con 0x7fe1f80cc9c0 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f80d0c50 con 0x7fe1f80b0310 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.081+0000 7fe1fe800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f806c8d0 con 0x7fe1f806bfb0 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1f80b0310 0x7fe1f80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1f80b0310 0x7fe1f80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51632/0 (socket says 172.21.15.143:51632) 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1f806bfb0 0x7fe1f806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 -- 172.21.15.143:0/3721477057 learned_addr learned my addr 172.21.15.143:0/3721477057 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f7e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe1f80cc9c0 0x7fe1f80cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40462/0 (socket says 172.21.15.143:40462) 2024-01-15T18:06:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 -- 172.21.15.143:0/3721477057 >> v2:172.21.15.204:3300/0 conn(0x7fe1f806bfb0 msgr2=0x7fe1f806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 --2- 172.21.15.143:0/3721477057 >> v2:172.21.15.204:3300/0 conn(0x7fe1f806bfb0 0x7fe1f806c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 -- 172.21.15.143:0/3721477057 >> v1:172.21.15.143:6789/0 conn(0x7fe1f80cc9c0 legacy=0x7fe1f80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 -- 172.21.15.143:0/3721477057 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f80cb6c0 con 0x7fe1f80b0310 2024-01-15T18:06:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6000640 1 -- 172.21.15.143:0/3721477057 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244947506 0 0) 0x7fe1f80cbff0 con 0x7fe1f80cc9c0 2024-01-15T18:06:37.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6a00640 1 --2- 172.21.15.143:0/3721477057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80cc480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80716c0 tx=0x7fe1e001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3c2c5bf41a5265 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:37.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6000640 1 -- 172.21.15.143:0/3721477057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00ab640 con 0x7fe1f80b0310 2024-01-15T18:06:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6000640 1 -- 172.21.15.143:0/3721477057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e00a7070 con 0x7fe1f80b0310 2024-01-15T18:06:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.085+0000 7fe1f6000640 1 -- 172.21.15.143:0/3721477057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00ab9c0 con 0x7fe1f80b0310 2024-01-15T18:06:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- 172.21.15.143:0/3721477057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 msgr2=0x7fe1f80cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 --2- 172.21.15.143:0/3721477057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80cc480 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80716c0 tx=0x7fe1e001c440 comp rx=0 tx=0).stop 2024-01-15T18:06:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- 172.21.15.143:0/3721477057 shutdown_connections 2024-01-15T18:06:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 --2- 172.21.15.143:0/3721477057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80cc480 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 --2- 172.21.15.143:0/3721477057 >> v2:172.21.15.204:3300/0 conn(0x7fe1f806bfb0 0x7fe1f806c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- 172.21.15.143:0/3721477057 >> 172.21.15.143:0/3721477057 conn(0x7fe1f80d0ed0 msgr2=0x7fe1f80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- 172.21.15.143:0/3721477057 shutdown_connections 2024-01-15T18:06:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- 172.21.15.143:0/3721477057 wait complete. 2024-01-15T18:06:37.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 Processor -- start 2024-01-15T18:06:37.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- start start 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f806bfb0 0x7fe1f80c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f80cc9c0 0x7fe1f814d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8150ba0 con 0x7fe1f80b0310 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f814e2e0 con 0x7fe1f80cc9c0 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.089+0000 7fe1fe800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81543e0 con 0x7fe1f806bfb0 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51642/0 (socket says 172.21.15.143:51642) 2024-01-15T18:06:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 -- 172.21.15.143:0/4059216974 learned_addr learned my addr 172.21.15.143:0/4059216974 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f7400640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f806bfb0 0x7fe1f80c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f7e00640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f80cc9c0 0x7fe1f814d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 -- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f806bfb0 msgr2=0x7fe1f80c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f806bfb0 0x7fe1f80c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 -- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f80cc9c0 msgr2=0x7fe1f814d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f80cc9c0 0x7fe1f814d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 -- 172.21.15.143:0/4059216974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e00094d0 con 0x7fe1f80b0310 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f7e00640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f80cc9c0 0x7fe1f814d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1f6a00640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80c8200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0019050 tx=0x7fe1e001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00abe80 con 0x7fe1f80b0310 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e00be3f0 con 0x7fe1f80b0310 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f81546c0 con 0x7fe1f80b0310 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e00be7a0 con 0x7fe1f80b0310 2024-01-15T18:06:37.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f80ca670 con 0x7fe1f80b0310 2024-01-15T18:06:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f8154850 con 0x7fe1f80b0310 2024-01-15T18:06:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.093+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 6) v1 ==== 86211+0+0 (secure 0 0 0) 0x7fe1e00b43f0 con 0x7fe1f80b0310 2024-01-15T18:06:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.097+0000 7fe1ef400640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1d0066980 0x7fe1d0068e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.097+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1e011e6d0 con 0x7fe1f80b0310 2024-01-15T18:06:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.097+0000 7fe1f7400640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1d0066980 0x7fe1d0068e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.097+0000 7fe1f7400640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1d0066980 0x7fe1d0068e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009ac0 tx=0x7fe1e8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:37.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.101+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1e00eb7c0 con 0x7fe1f80b0310 2024-01-15T18:06:37.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:37.277+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1f806b7b0 con 0x7fe1d0066980 2024-01-15T18:06:39.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:39.525+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7fe1e00ebb90 con 0x7fe1f80b0310 2024-01-15T18:06:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1ef400640 1 -- 172.21.15.143:0/4059216974 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fe1e80119e0 con 0x7fe1d0066980 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:41.231 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.232 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.232 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1d0066980 msgr2=0x7fe1d0068e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1d0066980 0x7fe1d0068e40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009ac0 tx=0x7fe1e8009190 comp rx=0 tx=0).stop 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 msgr2=0x7fe1f80c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80c8200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0019050 tx=0x7fe1e001ac80 comp rx=0 tx=0).stop 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 shutdown_connections 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1d0066980 0x7fe1d0068e40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f80cc9c0 0x7fe1f814d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f80b0310 0x7fe1f80c8200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 --2- 172.21.15.143:0/4059216974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f806bfb0 0x7fe1f80c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 >> 172.21.15.143:0/4059216974 conn(0x7fe1f80d0ed0 msgr2=0x7fe1f806ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 shutdown_connections 2024-01-15T18:06:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.225+0000 7fe1fe800640 1 -- 172.21.15.143:0/4059216974 wait complete. 2024-01-15T18:06:41.258 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:06:41.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa57000640 1 Processor -- start 2024-01-15T18:06:41.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa57000640 1 -- start start 2024-01-15T18:06:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa57000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa50072410 0x7faa5015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa57000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5015f310 con 0x7faa500ce560 2024-01-15T18:06:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa57000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5015f5f0 con 0x7faa50141660 2024-01-15T18:06:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa57000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5015fbb0 con 0x7faa50072410 2024-01-15T18:06:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa4f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faa500ce560 0x7faa50071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58630/0 (socket says 172.21.15.143:58630) 2024-01-15T18:06:41.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.389+0000 7faa4f400640 1 -- 172.21.15.143:0/1367040797 learned_addr learned my addr 172.21.15.143:0/1367040797 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa55000640 1 --2- 172.21.15.143:0/1367040797 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa50072410 0x7faa5015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408574606 0 0) 0x7faa5015f310 con 0x7faa500ce560 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa30003530 con 0x7faa500ce560 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800051037 0 0) 0x7faa5015f5f0 con 0x7faa50141660 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa5015f310 con 0x7faa50141660 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa55000640 1 -- 172.21.15.143:0/1367040797 >> v1:172.21.15.204:6789/0 conn(0x7faa50141660 legacy=0x7faa50141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa55000640 1 -- 172.21.15.143:0/1367040797 >> v1:172.21.15.143:6789/0 conn(0x7faa500ce560 legacy=0x7faa50071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa55000640 1 -- 172.21.15.143:0/1367040797 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5015f8d0 con 0x7faa50072410 2024-01-15T18:06:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa55000640 1 --2- 172.21.15.143:0/1367040797 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa50072410 0x7faa5015ec80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faa48009dc0 tx=0x7faa4801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e23f8d31ee4969 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480a5940 con 0x7faa50072410 2024-01-15T18:06:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480a5aa0 con 0x7faa50072410 2024-01-15T18:06:41.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.393+0000 7faa4ea00640 1 -- 172.21.15.143:0/1367040797 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480a5e30 con 0x7faa50072410 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- 172.21.15.143:0/1367040797 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa50072410 msgr2=0x7faa5015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 --2- 172.21.15.143:0/1367040797 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa50072410 0x7faa5015ec80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faa48009dc0 tx=0x7faa4801c970 comp rx=0 tx=0).stop 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- 172.21.15.143:0/1367040797 shutdown_connections 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 --2- 172.21.15.143:0/1367040797 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa50072410 0x7faa5015ec80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- 172.21.15.143:0/1367040797 >> 172.21.15.143:0/1367040797 conn(0x7faa5006c3f0 msgr2=0x7faa5006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- 172.21.15.143:0/1367040797 shutdown_connections 2024-01-15T18:06:41.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- 172.21.15.143:0/1367040797 wait complete. 2024-01-15T18:06:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 Processor -- start 2024-01-15T18:06:41.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- start start 2024-01-15T18:06:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa50072410 0x7faa50144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa500ce560 0x7faa5015a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 0x7faa5015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50143310 con 0x7faa50141660 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50143e90 con 0x7faa50072410 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.397+0000 7faa57000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa50143560 con 0x7faa500ce560 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 0x7faa5015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa50072410 0x7faa50144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 0x7faa5015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37956/0 (socket says 172.21.15.143:37956) 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 -- 172.21.15.143:0/1116149603 learned_addr learned my addr 172.21.15.143:0/1116149603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:41.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa4f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa500ce560 0x7faa5015a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 -- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa500ce560 msgr2=0x7faa5015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa500ce560 0x7faa5015a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 -- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa50072410 msgr2=0x7faa50144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa50072410 0x7faa50144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 -- 172.21.15.143:0/1116149603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48009a70 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa4fe00640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa50072410 0x7faa50144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa55000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 0x7faa5015a120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa4801ce50 tx=0x7faa48019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480adc80 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480ade00 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480b9690 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa500cd150 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa50143b20 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50067500 con 0x7faa50141660 2024-01-15T18:06:41.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7faa480b97f0 con 0x7faa50141660 2024-01-15T18:06:41.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa47e00640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa28066aa0 0x7faa28068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.401+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa4811fc80 con 0x7faa50141660 2024-01-15T18:06:41.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.405+0000 7faa4fe00640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa28066aa0 0x7faa28068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.405+0000 7faa4fe00640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa28066aa0 0x7faa28068f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa400011f0 tx=0x7faa40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:41.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.409+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa480ecd90 con 0x7faa50141660 2024-01-15T18:06:41.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.613+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faa50143fd0 con 0x7faa50141660 2024-01-15T18:06:41.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.613+0000 7faa47e00640 1 -- 172.21.15.143:0/1116149603 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2934 (secure 0 0 0) 0x7faa480bdb90 con 0x7faa50141660 2024-01-15T18:06:41.620 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:41.620 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:41.620 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:06:41.621 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:06:41.622 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 127, 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 33, 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:06:41.623 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:06:41.624 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 793, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3222015920, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 6685417472, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 766408695808, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 61506947, 2024-01-15T18:06:41.625 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 0, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 15 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 18, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 2, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:41.626 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 2, 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "d", 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 5232 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:06:41.627 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:06:41.628 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 3, 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:05:48.631499+0000", 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "a": { 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:06:41.629 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "b": { 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:06:41.630 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "d": { 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.631 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.632 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.632 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.632 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": {} 2024-01-15T18:06:41.632 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.613+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa28066aa0 msgr2=0x7faa28068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.613+0000 7faa57000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa28066aa0 0x7faa28068f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa400011f0 tx=0x7faa40009040 comp rx=0 tx=0).stop 2024-01-15T18:06:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.613+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 msgr2=0x7faa5015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.613+0000 7faa57000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 0x7faa5015a120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faa4801ce50 tx=0x7faa48019710 comp rx=0 tx=0).stop 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 shutdown_connections 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa28066aa0 0x7faa28068f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa50141660 0x7faa5015a120 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa500ce560 0x7faa5015a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 --2- 172.21.15.143:0/1116149603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa50072410 0x7faa50144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 >> 172.21.15.143:0/1116149603 conn(0x7faa5006c3f0 msgr2=0x7faa500aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 shutdown_connections 2024-01-15T18:06:41.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.617+0000 7faa57000640 1 -- 172.21.15.143:0/1116149603 wait complete. 2024-01-15T18:06:41.641 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:06:41.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:41.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.773+0000 7f7123400640 1 Processor -- start 2024-01-15T18:06:41.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.773+0000 7f7123400640 1 -- start start 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7123400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f711c072030 0x7f711c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7123400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c0ceb50 con 0x7f711c071bb0 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7123400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c15c110 con 0x7f711c143780 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7123400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c15b7e0 con 0x7f711c072030 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f711c071bb0 0x7f711c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58638/0 (socket says 172.21.15.143:58638) 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 -- 172.21.15.143:0/3851231513 learned_addr learned my addr 172.21.15.143:0/3851231513 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964895445 0 0) 0x7f7104001120 con 0x7f711c071bb0 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70fc003530 con 0x7f711c071bb0 2024-01-15T18:06:41.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7121400640 1 --2- 172.21.15.143:0/3851231513 >> v2:172.21.15.204:3300/0 conn(0x7f711c072030 0x7f711c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 661863142 0 0) 0x7f70fc003530 con 0x7f711c071bb0 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 >> v2:172.21.15.204:3300/0 conn(0x7f711c072030 msgr2=0x7f711c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 --2- 172.21.15.143:0/3851231513 >> v2:172.21.15.204:3300/0 conn(0x7f711c072030 0x7f711c15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 >> v1:172.21.15.204:6789/0 conn(0x7f711c143780 legacy=0x7f711c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c15bac0 con 0x7f711c071bb0 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f7104001810 con 0x7f711c071bb0 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 >> v1:172.21.15.143:6789/0 conn(0x7f711c071bb0 legacy=0x7f711c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70fc0039d0 0x7f70fc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70fc0047d0 0x7f70fc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc005190 0x7f70fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c15b7e0 con 0x7f70fc0047d0 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc00c090 con 0x7f70fc0039d0 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc00c250 con 0x7f70fc005190 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70fc0047d0 0x7f70fc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7120a00640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70fc0039d0 0x7f70fc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7121400640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc005190 0x7f70fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 -- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc005190 msgr2=0x7f70fc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc005190 0x7f70fc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 -- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70fc0039d0 msgr2=0x7f70fc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70fc0039d0 0x7f70fc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f711be00640 1 -- 172.21.15.143:0/3851231513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc003530 con 0x7f70fc0047d0 2024-01-15T18:06:41.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.777+0000 7f7121400640 1 -- 172.21.15.143:0/3851231513 reap_dead start 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f711be00640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70fc0047d0 0x7f70fc004c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7104007fd0 tx=0x7f710401f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6a53956982098a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040a8e50 con 0x7f70fc0047d0 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71040058e0 con 0x7f70fc0047d0 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f711b400640 1 -- 172.21.15.143:0/3851231513 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104005cd0 con 0x7f70fc0047d0 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 -- 172.21.15.143:0/3851231513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70fc0047d0 msgr2=0x7f70fc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70fc0047d0 0x7f70fc004c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7104007fd0 tx=0x7f710401f230 comp rx=0 tx=0).stop 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 -- 172.21.15.143:0/3851231513 shutdown_connections 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 --2- 172.21.15.143:0/3851231513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70fc0047d0 0x7f70fc004c50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 -- 172.21.15.143:0/3851231513 >> 172.21.15.143:0/3851231513 conn(0x7f711c06c3f0 msgr2=0x7f711c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:41.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 -- 172.21.15.143:0/3851231513 shutdown_connections 2024-01-15T18:06:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.781+0000 7f7123400640 1 -- 172.21.15.143:0/3851231513 wait complete. 2024-01-15T18:06:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 Processor -- start 2024-01-15T18:06:41.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 -- start start 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc0047d0 0x7f711c1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711c13ecc0 0x7f711c0cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 0x7f711c141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c13dc20 con 0x7f711c13f140 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c13df00 con 0x7f711c13ecc0 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c13e1e0 con 0x7f70fc0047d0 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 0x7f711c141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 0x7f711c141c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37968/0 (socket says 172.21.15.143:37968) 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 -- 172.21.15.143:0/1919744012 learned_addr learned my addr 172.21.15.143:0/1919744012 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:41.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7120a00640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc0047d0 0x7f711c1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f711be00640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711c13ecc0 0x7f711c0cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 -- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc0047d0 msgr2=0x7f711c1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc0047d0 0x7f711c1598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 -- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711c13ecc0 msgr2=0x7f711c0cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711c13ecc0 0x7f711c0cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 -- 172.21.15.143:0/1919744012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c13e4c0 con 0x7f711c13f140 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7121400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 0x7f711c141c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b1b40 tx=0x7f71140115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711400e070 con 0x7f711c13f140 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c13e7a0 con 0x7f711c13f140 2024-01-15T18:06:41.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7114011db0 con 0x7f711c13f140 2024-01-15T18:06:41.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71140039a0 con 0x7f711c13f140 2024-01-15T18:06:41.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.785+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c159e00 con 0x7f711c13f140 2024-01-15T18:06:41.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.789+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c13e9e0 con 0x7f711c13f140 2024-01-15T18:06:41.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.789+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f7114003b00 con 0x7f711c13f140 2024-01-15T18:06:41.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.789+0000 7f7118c00640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f70f4066aa0 0x7f70f4068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:41.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.789+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7114084f70 con 0x7f711c13f140 2024-01-15T18:06:41.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.789+0000 7f7120a00640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f70f4066aa0 0x7f70f4068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:41.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.789+0000 7f7120a00640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f70f4066aa0 0x7f70f4068f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f710c007fd0 tx=0x7f710c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:41.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.793+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f711c13e9e0 con 0x7f711c13f140 2024-01-15T18:06:41.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.973+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f711c072470 con 0x7f70f4066aa0 2024-01-15T18:06:41.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7118c00640 1 -- 172.21.15.143:0/1919744012 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f710c001c50 con 0x7f70f4066aa0 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:41.987 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:41.988 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f70f4066aa0 msgr2=0x7f70f4068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f70f4066aa0 0x7f70f4068f60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f710c007fd0 tx=0x7f710c009040 comp rx=0 tx=0).stop 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 msgr2=0x7f711c141c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 0x7f711c141c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f711c0b1b40 tx=0x7f71140115a0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 shutdown_connections 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f70f4066aa0 0x7f70f4068f60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f711c13f140 0x7f711c141c40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f711c13ecc0 0x7f711c0cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 --2- 172.21.15.143:0/1919744012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70fc0047d0 0x7f711c1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 >> 172.21.15.143:0/1919744012 conn(0x7f711c06c3f0 msgr2=0x7f711c0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 shutdown_connections 2024-01-15T18:06:41.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:41.981+0000 7f7123400640 1 -- 172.21.15.143:0/1919744012 wait complete. 2024-01-15T18:06:42.008 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:43.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1dae00640 1 Processor -- start 2024-01-15T18:06:43.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1dae00640 1 -- start start 2024-01-15T18:06:43.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1dae00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1d40c6230 0x7fc1d40cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1dae00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d40c4d90 con 0x7fc1d40cdf20 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1dae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d415b640 con 0x7fc1d415b260 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1dae00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d40cf800 con 0x7fc1d40c6230 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1d8e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1d40c6230 0x7fc1d40cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1d3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc1d40cdf20 0x7fc1d40cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58646/0 (socket says 172.21.15.143:58646) 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1d3400640 1 -- 172.21.15.143:0/815952195 learned_addr learned my addr 172.21.15.143:0/815952195 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1d8e00640 1 -- 172.21.15.143:0/815952195 >> v1:172.21.15.204:6789/0 conn(0x7fc1d415b260 legacy=0x7fc1d40cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1d8e00640 1 -- 172.21.15.143:0/815952195 >> v1:172.21.15.143:6789/0 conn(0x7fc1d40cdf20 legacy=0x7fc1d40cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.153+0000 7fc1d8e00640 1 -- 172.21.15.143:0/815952195 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40cfae0 con 0x7fc1d40c6230 2024-01-15T18:06:43.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1d8e00640 1 --2- 172.21.15.143:0/815952195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d40c6230 0x7fc1d40cc1f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009d90 tx=0x7fc1cc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290bcaf4e644b739 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:43.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1d2a00640 1 -- 172.21.15.143:0/815952195 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0a59e0 con 0x7fc1d40c6230 2024-01-15T18:06:43.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1d2a00640 1 -- 172.21.15.143:0/815952195 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d40ccf40 con 0x7fc1d40c6230 2024-01-15T18:06:43.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1d2a00640 1 -- 172.21.15.143:0/815952195 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0a5d60 con 0x7fc1d40c6230 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1dae00640 1 -- 172.21.15.143:0/815952195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d40c6230 msgr2=0x7fc1d40cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1dae00640 1 --2- 172.21.15.143:0/815952195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d40c6230 0x7fc1d40cc1f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009d90 tx=0x7fc1cc01c920 comp rx=0 tx=0).stop 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1dae00640 1 -- 172.21.15.143:0/815952195 shutdown_connections 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1dae00640 1 --2- 172.21.15.143:0/815952195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d40c6230 0x7fc1d40cc1f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1dae00640 1 -- 172.21.15.143:0/815952195 >> 172.21.15.143:0/815952195 conn(0x7fc1d40d0ed0 msgr2=0x7fc1d40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.157+0000 7fc1dae00640 1 -- 172.21.15.143:0/815952195 shutdown_connections 2024-01-15T18:06:43.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 -- 172.21.15.143:0/815952195 wait complete. 2024-01-15T18:06:43.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 Processor -- start 2024-01-15T18:06:43.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 -- start start 2024-01-15T18:06:43.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1d40c6230 0x7fc1d4150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1d40cdf20 0x7fc1d40c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 0x7fc1d40c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4150970 con 0x7fc1d415b260 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d40c84e0 con 0x7fc1d40c6230 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1dae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d406ceb0 con 0x7fc1d40cdf20 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 0x7fc1d40c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 0x7fc1d40c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37984/0 (socket says 172.21.15.143:37984) 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 -- 172.21.15.143:0/3422964753 learned_addr learned my addr 172.21.15.143:0/3422964753 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d3400640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1d40cdf20 0x7fc1d40c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:43.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d3e00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1d40c6230 0x7fc1d4150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:43.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 -- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1d40cdf20 msgr2=0x7fc1d40c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1d40cdf20 0x7fc1d40c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 -- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1d40c6230 msgr2=0x7fc1d4150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1d40c6230 0x7fc1d4150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 -- 172.21.15.143:0/3422964753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc009a70 con 0x7fc1d415b260 2024-01-15T18:06:43.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.161+0000 7fc1d8e00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 0x7fc1d40c9900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0b0040 tx=0x7fc1cc0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:43.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0b9860 con 0x7fc1d415b260 2024-01-15T18:06:43.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc0b99c0 con 0x7fc1d415b260 2024-01-15T18:06:43.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0b9d50 con 0x7fc1d415b260 2024-01-15T18:06:43.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d40c9e40 con 0x7fc1d415b260 2024-01-15T18:06:43.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d40b59e0 con 0x7fc1d415b260 2024-01-15T18:06:43.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d4067500 con 0x7fc1d415b260 2024-01-15T18:06:43.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7fc1cc0c4460 con 0x7fc1d415b260 2024-01-15T18:06:43.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1cbe00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1ac066aa0 0x7fc1ac068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:43.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1cc11fa40 con 0x7fc1d415b260 2024-01-15T18:06:43.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.165+0000 7fc1d3e00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1ac066aa0 0x7fc1ac068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:43.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.169+0000 7fc1d3e00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1ac066aa0 0x7fc1ac068f60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4007fd0 tx=0x7fc1c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:43.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.169+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1cc0ecad0 con 0x7fc1d415b260 2024-01-15T18:06:43.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.389+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1d406cff0 con 0x7fc1ac066aa0 2024-01-15T18:06:43.424 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:43.424 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:06:43.425 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:06:43.426 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:43.426 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:43.426 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:43.426 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:43.426 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.417+0000 7fc1cbe00640 1 -- 172.21.15.143:0/3422964753 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fc1d406cff0 con 0x7fc1ac066aa0 2024-01-15T18:06:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1ac066aa0 msgr2=0x7fc1ac068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1ac066aa0 0x7fc1ac068f60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4007fd0 tx=0x7fc1c4009040 comp rx=0 tx=0).stop 2024-01-15T18:06:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 msgr2=0x7fc1d40c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 0x7fc1d40c9900 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0b0040 tx=0x7fc1cc0196c0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 shutdown_connections 2024-01-15T18:06:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1ac066aa0 0x7fc1ac068f60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1d415b260 0x7fc1d40c9900 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1d40cdf20 0x7fc1d40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 --2- 172.21.15.143:0/3422964753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1d40c6230 0x7fc1d4150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 >> 172.21.15.143:0/3422964753 conn(0x7fc1d40d0ed0 msgr2=0x7fc1d406eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 shutdown_connections 2024-01-15T18:06:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:43.421+0000 7fc1dae00640 1 -- 172.21.15.143:0/3422964753 wait complete. 2024-01-15T18:06:43.475 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:44.476 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:44.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.769+0000 7f31de800640 1 Processor -- start 2024-01-15T18:06:44.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.769+0000 7f31de800640 1 -- start start 2024-01-15T18:06:44.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.773+0000 7f31de800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d815b2e0 0x7f31d80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.773+0000 7f31de800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d80b0cb0 con 0x7f31d80cf420 2024-01-15T18:06:44.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.773+0000 7f31de800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d815b6c0 con 0x7f31d80cca60 2024-01-15T18:06:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.773+0000 7f31de800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d80d1de0 con 0x7f31d815b2e0 2024-01-15T18:06:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.777+0000 7f31d7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d815b2e0 0x7f31d80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.777+0000 7f31d7e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f31d80cca60 0x7f31d80b1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:34048/0 (socket says 172.21.15.143:34048) 2024-01-15T18:06:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.777+0000 7f31d7e00640 1 -- 172.21.15.143:0/1646871596 learned_addr learned my addr 172.21.15.143:0/1646871596 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.777+0000 7f31d7400640 1 -- 172.21.15.143:0/1646871596 >> v1:172.21.15.204:6789/0 conn(0x7f31d80cca60 legacy=0x7f31d80b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.777+0000 7f31d7400640 1 -- 172.21.15.143:0/1646871596 >> v1:172.21.15.143:6789/0 conn(0x7f31d80cf420 legacy=0x7f31d80b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:44.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.777+0000 7f31d7400640 1 -- 172.21.15.143:0/1646871596 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80d1b00 con 0x7f31d815b2e0 2024-01-15T18:06:44.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.781+0000 7f31d7400640 1 --2- 172.21.15.143:0/1646871596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d815b2e0 0x7f31d80cde80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31d00097f0 tx=0x7f31d001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901c198589311074 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:44.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.781+0000 7f31d6000640 1 -- 172.21.15.143:0/1646871596 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00a8970 con 0x7f31d815b2e0 2024-01-15T18:06:44.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.781+0000 7f31d6000640 1 -- 172.21.15.143:0/1646871596 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d00a8ad0 con 0x7f31d815b2e0 2024-01-15T18:06:44.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.781+0000 7f31d6000640 1 -- 172.21.15.143:0/1646871596 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d00a8e60 con 0x7f31d815b2e0 2024-01-15T18:06:44.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 -- 172.21.15.143:0/1646871596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d815b2e0 msgr2=0x7f31d80cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 --2- 172.21.15.143:0/1646871596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d815b2e0 0x7f31d80cde80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31d00097f0 tx=0x7f31d001ec70 comp rx=0 tx=0).stop 2024-01-15T18:06:44.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 -- 172.21.15.143:0/1646871596 shutdown_connections 2024-01-15T18:06:44.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 --2- 172.21.15.143:0/1646871596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d815b2e0 0x7f31d80cde80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 -- 172.21.15.143:0/1646871596 >> 172.21.15.143:0/1646871596 conn(0x7f31d80d0e40 msgr2=0x7f31d80d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:44.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 -- 172.21.15.143:0/1646871596 shutdown_connections 2024-01-15T18:06:44.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.785+0000 7f31de800640 1 -- 172.21.15.143:0/1646871596 wait complete. 2024-01-15T18:06:44.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 Processor -- start 2024-01-15T18:06:44.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 -- start start 2024-01-15T18:06:44.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31d80cca60 0x7f31d81575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d80cf420 0x7f31d8151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 0x7f31d8155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8151dd0 con 0x7f31d80cca60 2024-01-15T18:06:44.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8152840 con 0x7f31d815b2e0 2024-01-15T18:06:44.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31de800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d8154bf0 con 0x7f31d80cf420 2024-01-15T18:06:44.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 0x7f31d8155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 0x7f31d8155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:37512/0 (socket says 172.21.15.143:37512) 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31d7e00640 1 -- 172.21.15.143:0/2906351482 learned_addr learned my addr 172.21.15.143:0/2906351482 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31d6a00640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d80cf420 0x7f31d8151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.789+0000 7f31d7400640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31d80cca60 0x7f31d81575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31d7e00640 1 -- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d80cf420 msgr2=0x7f31d8151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31d7e00640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d80cf420 0x7f31d8151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31d7e00640 1 -- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31d80cca60 msgr2=0x7f31d81575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31d7e00640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31d80cca60 0x7f31d81575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31d7e00640 1 -- 172.21.15.143:0/2906351482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0009420 con 0x7f31d815b2e0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31d7e00640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 0x7f31d8155c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31c000c400 tx=0x7f31c000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0005cf0 con 0x7f31d815b2e0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0005e50 con 0x7f31d815b2e0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c00072d0 con 0x7f31d815b2e0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d8154ed0 con 0x7f31d815b2e0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d8155490 con 0x7f31d815b2e0 2024-01-15T18:06:44.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.793+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d8067c10 con 0x7f31d815b2e0 2024-01-15T18:06:44.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.797+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f31c0007430 con 0x7f31d815b2e0 2024-01-15T18:06:44.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.797+0000 7f31cf400640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31b0066aa0 0x7f31b0068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.797+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31c00841e0 con 0x7f31d815b2e0 2024-01-15T18:06:44.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.801+0000 7f31d7400640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31b0066aa0 0x7f31b0068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.801+0000 7f31d7400640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31b0066aa0 0x7f31b0068f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f31d00b2000 tx=0x7f31d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:44.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:44.801+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31c00513f0 con 0x7f31d815b2e0 2024-01-15T18:06:45.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.173+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f31d80b1f60 con 0x7f31b0066aa0 2024-01-15T18:06:45.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.205+0000 7f31cf400640 1 -- 172.21.15.143:0/2906351482 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f31d00b9620 con 0x7f31b0066aa0 2024-01-15T18:06:45.211 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:06:45.212 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:06:45.213 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:06:45.213 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:45.213 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:45.213 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:45.213 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:45.213 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31b0066aa0 msgr2=0x7f31b0068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31b0066aa0 0x7f31b0068f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f31d00b2000 tx=0x7f31d0008040 comp rx=0 tx=0).stop 2024-01-15T18:06:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 msgr2=0x7f31d8155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:45.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 0x7f31d8155c90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f31c000c400 tx=0x7f31c000c8c0 comp rx=0 tx=0).stop 2024-01-15T18:06:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 shutdown_connections 2024-01-15T18:06:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31b0066aa0 0x7f31b0068f60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31d815b2e0 0x7f31d8155c90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31d80cf420 0x7f31d8151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 --2- 172.21.15.143:0/2906351482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31d80cca60 0x7f31d81575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:45.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.209+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 >> 172.21.15.143:0/2906351482 conn(0x7f31d80d0e40 msgr2=0x7f31d80ae780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.213+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 shutdown_connections 2024-01-15T18:06:45.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:45.213+0000 7f31de800640 1 -- 172.21.15.143:0/2906351482 wait complete. 2024-01-15T18:06:45.285 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:46.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:46.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.425+0000 7f75c7c00640 1 Processor -- start 2024-01-15T18:06:46.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.425+0000 7f75c7c00640 1 -- start start 2024-01-15T18:06:46.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75c7c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75c0071bb0 0x7f75c0072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75c7c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00ceb50 con 0x7f75c0143780 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75c7c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c015c110 con 0x7f75c015c5a0 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75c7c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c0072570 con 0x7f75c0071bb0 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75bfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75c0071bb0 0x7f75c0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75c5200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f75c0143780 0x7f75c00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58664/0 (socket says 172.21.15.143:58664) 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75c5200640 1 -- 172.21.15.143:0/995926878 learned_addr learned my addr 172.21.15.143:0/995926878 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75bfe00640 1 -- 172.21.15.143:0/995926878 >> v1:172.21.15.204:6789/0 conn(0x7f75c015c5a0 legacy=0x7f75c015c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75bfe00640 1 -- 172.21.15.143:0/995926878 >> v1:172.21.15.143:6789/0 conn(0x7f75c0143780 legacy=0x7f75c00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75bf400640 1 -- 172.21.15.143:0/995926878 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92743927 0 0) 0x7f75b0001120 con 0x7f75c0143780 2024-01-15T18:06:46.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.429+0000 7f75bfe00640 1 -- 172.21.15.143:0/995926878 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c015b7e0 con 0x7f75c0071bb0 2024-01-15T18:06:46.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75bfe00640 1 --2- 172.21.15.143:0/995926878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0072030 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f75a80097c0 tx=0x7f75a801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe08c1f3e9e18f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:46.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75bf400640 1 -- 172.21.15.143:0/995926878 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a80ab600 con 0x7f75c0071bb0 2024-01-15T18:06:46.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75bf400640 1 -- 172.21.15.143:0/995926878 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a80a7050 con 0x7f75c0071bb0 2024-01-15T18:06:46.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75bf400640 1 -- 172.21.15.143:0/995926878 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a80ab980 con 0x7f75c0071bb0 2024-01-15T18:06:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 -- 172.21.15.143:0/995926878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 msgr2=0x7f75c0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 --2- 172.21.15.143:0/995926878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0072030 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f75a80097c0 tx=0x7f75a801c440 comp rx=0 tx=0).stop 2024-01-15T18:06:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 -- 172.21.15.143:0/995926878 shutdown_connections 2024-01-15T18:06:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 --2- 172.21.15.143:0/995926878 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0072030 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 -- 172.21.15.143:0/995926878 >> 172.21.15.143:0/995926878 conn(0x7f75c006c3f0 msgr2=0x7f75c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 -- 172.21.15.143:0/995926878 shutdown_connections 2024-01-15T18:06:46.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 -- 172.21.15.143:0/995926878 wait complete. 2024-01-15T18:06:46.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 Processor -- start 2024-01-15T18:06:46.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.433+0000 7f75c7c00640 1 -- start start 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c0143780 0x7f75c01414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75c015c5a0 0x7f75c006d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0140350 con 0x7f75c0071bb0 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c013b100 con 0x7f75c015c5a0 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c013fca0 con 0x7f75c0143780 2024-01-15T18:06:46.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38002/0 (socket says 172.21.15.143:38002) 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 -- 172.21.15.143:0/2624708226 learned_addr learned my addr 172.21.15.143:0/2624708226 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75c015c5a0 0x7f75c006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75bfe00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c0143780 0x7f75c01414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 -- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c0143780 msgr2=0x7f75c01414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c0143780 0x7f75c01414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 -- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75c015c5a0 msgr2=0x7f75c006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75c015c5a0 0x7f75c006d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 -- 172.21.15.143:0/2624708226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8009420 con 0x7f75c0071bb0 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c5200640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0159b90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c780 tx=0x7f75b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:46.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00072d0 con 0x7f75c0071bb0 2024-01-15T18:06:46.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0007450 con 0x7f75c0071bb0 2024-01-15T18:06:46.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c013f400 con 0x7f75c0071bb0 2024-01-15T18:06:46.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00047f0 con 0x7f75c0071bb0 2024-01-15T18:06:46.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c013f9c0 con 0x7f75c0071bb0 2024-01-15T18:06:46.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c0067500 con 0x7f75c0071bb0 2024-01-15T18:06:46.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.437+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f75b0004950 con 0x7f75c0071bb0 2024-01-15T18:06:46.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.441+0000 7f75bcc00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7598066aa0 0x7f7598068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:46.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.441+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75b00837d0 con 0x7f75c0071bb0 2024-01-15T18:06:46.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.441+0000 7f75bfe00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7598066aa0 0x7f7598068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:46.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.441+0000 7f75bfe00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7598066aa0 0x7f7598068f60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75c006be30 tx=0x7f75a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:46.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.445+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f75b0050850 con 0x7f75c0071bb0 2024-01-15T18:06:46.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.629+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f75c0167ad0 con 0x7f7598066aa0 2024-01-15T18:06:46.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.633+0000 7f75bcc00640 1 -- 172.21.15.143:0/2624708226 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f75a80ab740 con 0x7f7598066aa0 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:06:46.641 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:06:46.642 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:06:46.642 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:46.642 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:46.642 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:46.642 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:46.642 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7598066aa0 msgr2=0x7f7598068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7598066aa0 0x7f7598068f60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75c006be30 tx=0x7f75a8008040 comp rx=0 tx=0).stop 2024-01-15T18:06:46.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 msgr2=0x7f75c0159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0159b90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c780 tx=0x7f75b000cc40 comp rx=0 tx=0).stop 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 shutdown_connections 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7598066aa0 0x7f7598068f60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75c015c5a0 0x7f75c006d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75c0143780 0x7f75c01414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 --2- 172.21.15.143:0/2624708226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75c0071bb0 0x7f75c0159b90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 >> 172.21.15.143:0/2624708226 conn(0x7f75c006c3f0 msgr2=0x7f75c00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 shutdown_connections 2024-01-15T18:06:46.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:46.637+0000 7f75c7c00640 1 -- 172.21.15.143:0/2624708226 wait complete. 2024-01-15T18:06:46.663 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:47.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:47.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e3a00640 1 Processor -- start 2024-01-15T18:06:47.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e3a00640 1 -- start start 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e3a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f94dc15b260 0x7f94dc0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e3a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc0c4d90 con 0x7f94dc0ccaf0 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc15b640 con 0x7f94dc0d19f0 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e3a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0cf800 con 0x7f94dc15b260 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e1a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94dc0ccaf0 0x7f94dc0c5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55106/0 (socket says 172.21.15.143:55106) 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e1a00640 1 -- 172.21.15.143:0/3723517873 learned_addr learned my addr 172.21.15.143:0/3723517873 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.929+0000 7f94e1000640 1 --2- 172.21.15.143:0/3723517873 >> v2:172.21.15.204:3300/0 conn(0x7f94dc15b260 0x7f94dc0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:47.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.933+0000 7f94db400640 1 -- 172.21.15.143:0/3723517873 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730460290 0 0) 0x7f94dc0c4d90 con 0x7f94dc0ccaf0 2024-01-15T18:06:47.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.933+0000 7f94db400640 1 -- 172.21.15.143:0/3723517873 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94bc003530 con 0x7f94dc0ccaf0 2024-01-15T18:06:47.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.933+0000 7f94e1000640 1 -- 172.21.15.143:0/3723517873 >> v1:172.21.15.204:6789/0 conn(0x7f94dc0d19f0 legacy=0x7f94dc0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.933+0000 7f94e1000640 1 -- 172.21.15.143:0/3723517873 >> v1:172.21.15.143:6789/0 conn(0x7f94dc0ccaf0 legacy=0x7f94dc0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.933+0000 7f94e1000640 1 -- 172.21.15.143:0/3723517873 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0cfae0 con 0x7f94dc15b260 2024-01-15T18:06:47.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.937+0000 7f94e1000640 1 --2- 172.21.15.143:0/3723517873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0cd810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0097c0 tx=0x7f94cc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5329b0cc5fb3fd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.937+0000 7f94db400640 1 -- 172.21.15.143:0/3723517873 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0ab600 con 0x7f94dc15b260 2024-01-15T18:06:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.937+0000 7f94db400640 1 -- 172.21.15.143:0/3723517873 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc0a7050 con 0x7f94dc15b260 2024-01-15T18:06:47.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.937+0000 7f94db400640 1 -- 172.21.15.143:0/3723517873 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0ab980 con 0x7f94dc15b260 2024-01-15T18:06:47.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 -- 172.21.15.143:0/3723517873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 msgr2=0x7f94dc0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 --2- 172.21.15.143:0/3723517873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0cd810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0097c0 tx=0x7f94cc01c480 comp rx=0 tx=0).stop 2024-01-15T18:06:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 -- 172.21.15.143:0/3723517873 shutdown_connections 2024-01-15T18:06:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 --2- 172.21.15.143:0/3723517873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0cd810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 -- 172.21.15.143:0/3723517873 >> 172.21.15.143:0/3723517873 conn(0x7f94dc0d0ed0 msgr2=0x7f94dc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 -- 172.21.15.143:0/3723517873 shutdown_connections 2024-01-15T18:06:47.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.941+0000 7f94e3a00640 1 -- 172.21.15.143:0/3723517873 wait complete. 2024-01-15T18:06:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 Processor -- start 2024-01-15T18:06:47.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 -- start start 2024-01-15T18:06:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0ccaf0 0x7f94dc14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 0x7f94dc0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc150580 con 0x7f94dc0ccaf0 2024-01-15T18:06:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0b33d0 con 0x7f94dc15b260 2024-01-15T18:06:47.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.945+0000 7f94e3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0b36b0 con 0x7f94dc0d19f0 2024-01-15T18:06:47.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 0x7f94dc0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:47.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94e1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:47.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 0x7f94dc0c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:45844/0 (socket says 172.21.15.143:45844) 2024-01-15T18:06:47.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 -- 172.21.15.143:0/1751211218 learned_addr learned my addr 172.21.15.143:0/1751211218 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:47.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 -- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 msgr2=0x7f94dc0c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 -- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0ccaf0 msgr2=0x7f94dc14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0ccaf0 0x7f94dc14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 -- 172.21.15.143:0/1751211218 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc009420 con 0x7f94dc0d19f0 2024-01-15T18:06:47.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94dbe00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 0x7f94dc0c5220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c390 tx=0x7f94c400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:47.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40052d0 con 0x7f94dc0d19f0 2024-01-15T18:06:47.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c4005430 con 0x7f94dc0d19f0 2024-01-15T18:06:47.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c4003820 con 0x7f94dc0d19f0 2024-01-15T18:06:47.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.949+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc14ff30 con 0x7f94dc0d19f0 2024-01-15T18:06:47.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.953+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc0b3990 con 0x7f94dc0d19f0 2024-01-15T18:06:47.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.953+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc067500 con 0x7f94dc0d19f0 2024-01-15T18:06:47.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.957+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f94c4003980 con 0x7f94dc0d19f0 2024-01-15T18:06:47.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.957+0000 7f94d8c00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94b4066aa0 0x7f94b4068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:47.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.957+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94c40838e0 con 0x7f94dc0d19f0 2024-01-15T18:06:47.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.957+0000 7f94e1000640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94b4066aa0 0x7f94b4068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:47.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.957+0000 7f94e1000640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94b4066aa0 0x7f94b4068f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ad000 tx=0x7f94cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:47.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:47.961+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94c40509e0 con 0x7f94dc0d19f0 2024-01-15T18:06:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.269+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f94dc1506f0 con 0x7f94b4066aa0 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.301+0000 7f94d8c00640 1 -- 172.21.15.143:0/1751211218 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f94dc1506f0 con 0x7f94b4066aa0 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:48.308 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:48.309 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94b4066aa0 msgr2=0x7f94b4068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94b4066aa0 0x7f94b4068f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ad000 tx=0x7f94cc008040 comp rx=0 tx=0).stop 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 msgr2=0x7f94dc0c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 0x7f94dc0c5220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f94c400c390 tx=0x7f94c400c850 comp rx=0 tx=0).stop 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 shutdown_connections 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94b4066aa0 0x7f94b4068f60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94dc15b260 0x7f94dc0c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94dc0d19f0 0x7f94dc0c5220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 --2- 172.21.15.143:0/1751211218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94dc0ccaf0 0x7f94dc14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 >> 172.21.15.143:0/1751211218 conn(0x7f94dc0d0ed0 msgr2=0x7f94dc0c80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 shutdown_connections 2024-01-15T18:06:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:48.305+0000 7f94e3a00640 1 -- 172.21.15.143:0/1751211218 wait complete. 2024-01-15T18:06:48.359 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:49.360 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:49.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.493+0000 7f9610600640 1 Processor -- start 2024-01-15T18:06:49.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9610600640 1 -- start start 2024-01-15T18:06:49.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9610600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f96080cdf20 0x7f96080c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9610600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96080cf400 0x7f96080c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9610600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080cf970 con 0x7f960815b120 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9610600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080cfc50 con 0x7f96080cdf20 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9610600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080c4b10 con 0x7f96080cf400 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960d200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f96080cdf20 0x7f96080c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f960815b120 0x7f96080cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55110/0 (socket says 172.21.15.143:55110) 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960dc00640 1 -- 172.21.15.143:0/602173596 learned_addr learned my addr 172.21.15.143:0/602173596 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:49.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960e600640 1 --2- 172.21.15.143:0/602173596 >> v2:172.21.15.204:3300/0 conn(0x7f96080cf400 0x7f96080c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:49.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960d200640 1 -- 172.21.15.143:0/602173596 >> v2:172.21.15.204:3300/0 conn(0x7f96080cf400 msgr2=0x7f96080c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:49.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960d200640 1 --2- 172.21.15.143:0/602173596 >> v2:172.21.15.204:3300/0 conn(0x7f96080cf400 0x7f96080c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960d200640 1 -- 172.21.15.143:0/602173596 >> v1:172.21.15.143:6789/0 conn(0x7f960815b120 legacy=0x7f96080cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:49.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f960d200640 1 -- 172.21.15.143:0/602173596 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080c4df0 con 0x7f96080cdf20 2024-01-15T18:06:49.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.497+0000 7f9607e00640 1 -- 172.21.15.143:0/602173596 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744088983 0 0) 0x7f96080cf970 con 0x7f960815b120 2024-01-15T18:06:49.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f960d200640 1 --2- 172.21.15.143:0/602173596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f96080c5310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f96080cb400 tx=0x7f95f001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaf8010a544255a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:49.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9607e00640 1 -- 172.21.15.143:0/602173596 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ab600 con 0x7f96080cdf20 2024-01-15T18:06:49.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9607e00640 1 -- 172.21.15.143:0/602173596 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00a7050 con 0x7f96080cdf20 2024-01-15T18:06:49.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9607e00640 1 -- 172.21.15.143:0/602173596 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00ab980 con 0x7f96080cdf20 2024-01-15T18:06:49.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 -- 172.21.15.143:0/602173596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 msgr2=0x7f96080c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:49.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 --2- 172.21.15.143:0/602173596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f96080c5310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f96080cb400 tx=0x7f95f001c3d0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 -- 172.21.15.143:0/602173596 shutdown_connections 2024-01-15T18:06:49.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 --2- 172.21.15.143:0/602173596 >> v2:172.21.15.204:3300/0 conn(0x7f96080cf400 0x7f96080c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 --2- 172.21.15.143:0/602173596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f96080c5310 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 -- 172.21.15.143:0/602173596 >> 172.21.15.143:0/602173596 conn(0x7f96080d0e20 msgr2=0x7f96080d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:49.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 -- 172.21.15.143:0/602173596 shutdown_connections 2024-01-15T18:06:49.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.501+0000 7f9610600640 1 -- 172.21.15.143:0/602173596 wait complete. 2024-01-15T18:06:49.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 Processor -- start 2024-01-15T18:06:49.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- start start 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f960806c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96080cf400 0x7f9608156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960815b120 0x7f9608157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960806c2c0 con 0x7f96080cdf20 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081f7800 con 0x7f960815b120 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960806b640 con 0x7f96080cf400 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f960806c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f960806c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36464/0 (socket says 172.21.15.143:36464) 2024-01-15T18:06:49.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 -- 172.21.15.143:0/3101926882 learned_addr learned my addr 172.21.15.143:0/3101926882 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960e600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960815b120 0x7f9608157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960d200640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96080cf400 0x7f9608156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 -- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96080cf400 msgr2=0x7f9608156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96080cf400 0x7f9608156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 -- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960815b120 msgr2=0x7f9608157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960815b120 0x7f9608157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 -- 172.21.15.143:0/3101926882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f0009420 con 0x7f96080cdf20 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960d200640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96080cf400 0x7f9608156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f960dc00640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f960806c660 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f960000c7c0 tx=0x7f960000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:49.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000a640 con 0x7f96080cdf20 2024-01-15T18:06:49.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960000a7c0 con 0x7f96080cdf20 2024-01-15T18:06:49.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960806b920 con 0x7f96080cdf20 2024-01-15T18:06:49.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000ab70 con 0x7f96080cdf20 2024-01-15T18:06:49.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960806bee0 con 0x7f96080cdf20 2024-01-15T18:06:49.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.505+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960806c400 con 0x7f96080cdf20 2024-01-15T18:06:49.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.509+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f960000acd0 con 0x7f96080cdf20 2024-01-15T18:06:49.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.509+0000 7f9605600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95e0066aa0 0x7f95e0068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:49.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.509+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9600083eb0 con 0x7f96080cdf20 2024-01-15T18:06:49.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.509+0000 7f960d200640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95e0066aa0 0x7f95e0068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:49.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.509+0000 7f960d200640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95e0066aa0 0x7f95e0068f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ad000 tx=0x7f95f00a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:49.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.513+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f960806c400 con 0x7f96080cdf20 2024-01-15T18:06:49.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.697+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f960806c400 con 0x7f95e0066aa0 2024-01-15T18:06:49.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.701+0000 7f9605600640 1 -- 172.21.15.143:0/3101926882 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f960806c400 con 0x7f95e0066aa0 2024-01-15T18:06:49.708 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:49.709 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:49.710 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95e0066aa0 msgr2=0x7f95e0068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95e0066aa0 0x7f95e0068f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ad000 tx=0x7f95f00a6000 comp rx=0 tx=0).stop 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 msgr2=0x7f960806c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f960806c660 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f960000c7c0 tx=0x7f960000cc80 comp rx=0 tx=0).stop 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 shutdown_connections 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f95e0066aa0 0x7f95e0068f60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f960815b120 0x7f9608157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96080cf400 0x7f9608156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 --2- 172.21.15.143:0/3101926882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96080cdf20 0x7f960806c660 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:49.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 >> 172.21.15.143:0/3101926882 conn(0x7f96080d0e20 msgr2=0x7f96080d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:49.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 shutdown_connections 2024-01-15T18:06:49.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:49.705+0000 7f9610600640 1 -- 172.21.15.143:0/3101926882 wait complete. 2024-01-15T18:06:49.730 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:50.732 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:50.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 Processor -- start 2024-01-15T18:06:50.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 -- start start 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f92fc06bfb0 0x7f92fc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92fc0b0310 0x7f92fc0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92fc0cbff0 con 0x7f92fc0cc9c0 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc0d0c50 con 0x7f92fc0b0310 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9304400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92fc06c8d0 con 0x7f92fc06bfb0 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92fc0b0310 0x7f92fc0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f92fc0b0310 0x7f92fc0cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36474/0 (socket says 172.21.15.143:36474) 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 -- 172.21.15.143:0/2463416239 learned_addr learned my addr 172.21.15.143:0/2463416239 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:50.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301a00640 1 --2- 172.21.15.143:0/2463416239 >> v2:172.21.15.204:3300/0 conn(0x7f92fc06bfb0 0x7f92fc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:50.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 -- 172.21.15.143:0/2463416239 >> v2:172.21.15.204:3300/0 conn(0x7f92fc06bfb0 msgr2=0x7f92fc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:50.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 --2- 172.21.15.143:0/2463416239 >> v2:172.21.15.204:3300/0 conn(0x7f92fc06bfb0 0x7f92fc06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:50.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 -- 172.21.15.143:0/2463416239 >> v1:172.21.15.204:6789/0 conn(0x7f92fc0cc9c0 legacy=0x7f92fc0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:50.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.865+0000 7f9301000640 1 -- 172.21.15.143:0/2463416239 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92fc0cb6c0 con 0x7f92fc0b0310 2024-01-15T18:06:50.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9301000640 1 --2- 172.21.15.143:0/2463416239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0cc480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0716c0 tx=0x7f92e401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c389a323f45dfb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:50.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f92fbe00640 1 -- 172.21.15.143:0/2463416239 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40ab600 con 0x7f92fc0b0310 2024-01-15T18:06:50.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f92fbe00640 1 -- 172.21.15.143:0/2463416239 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e40ab760 con 0x7f92fc0b0310 2024-01-15T18:06:50.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f92fbe00640 1 -- 172.21.15.143:0/2463416239 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40abaf0 con 0x7f92fc0b0310 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 -- 172.21.15.143:0/2463416239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 msgr2=0x7f92fc0cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 --2- 172.21.15.143:0/2463416239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0cc480 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f92fc0716c0 tx=0x7f92e401c400 comp rx=0 tx=0).stop 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 -- 172.21.15.143:0/2463416239 shutdown_connections 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 --2- 172.21.15.143:0/2463416239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0cc480 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 --2- 172.21.15.143:0/2463416239 >> v2:172.21.15.204:3300/0 conn(0x7f92fc06bfb0 0x7f92fc06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 -- 172.21.15.143:0/2463416239 >> 172.21.15.143:0/2463416239 conn(0x7f92fc0d0ed0 msgr2=0x7f92fc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:50.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 -- 172.21.15.143:0/2463416239 shutdown_connections 2024-01-15T18:06:50.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.869+0000 7f9304400640 1 -- 172.21.15.143:0/2463416239 wait complete. 2024-01-15T18:06:50.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 Processor -- start 2024-01-15T18:06:50.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 -- start start 2024-01-15T18:06:50.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92fc06bfb0 0x7f92fc0c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:50.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:50.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc0cc9c0 0x7f92fc15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:50.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc15a630 con 0x7f92fc0b0310 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c6f30 con 0x7f92fc0cc9c0 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92fc0c6320 con 0x7f92fc06bfb0 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36486/0 (socket says 172.21.15.143:36486) 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 -- 172.21.15.143:0/840949157 learned_addr learned my addr 172.21.15.143:0/840949157 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 -- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92fc06bfb0 msgr2=0x7f92fc0c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92fc06bfb0 0x7f92fc0c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 -- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc0cc9c0 msgr2=0x7f92fc15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:50.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9302400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc0cc9c0 0x7f92fc15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc0cc9c0 0x7f92fc15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 -- 172.21.15.143:0/840949157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40094d0 con 0x7f92fc0b0310 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9302400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc0cc9c0 0x7f92fc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9301000640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0c5c40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f92e40ad040 tx=0x7f92e401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40b4800 con 0x7f92fc0b0310 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e40b4980 con 0x7f92fc0b0310 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e40b4d30 con 0x7f92fc0b0310 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92fc0c6600 con 0x7f92fc0b0310 2024-01-15T18:06:50.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.873+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92fc0c6bc0 con 0x7f92fc0b0310 2024-01-15T18:06:50.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.877+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92fc067500 con 0x7f92fc0b0310 2024-01-15T18:06:50.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.877+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f92e40b3380 con 0x7f92fc0b0310 2024-01-15T18:06:50.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.877+0000 7f92f9600640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92d4066ad0 0x7f92d4068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:50.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.877+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92e411eac0 con 0x7f92fc0b0310 2024-01-15T18:06:50.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.877+0000 7f9301a00640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92d4066ad0 0x7f92d4068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:50.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.877+0000 7f9301a00640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92d4066ad0 0x7f92d4068f90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00b850 tx=0x7f92ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:50.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:50.881+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f92e40eb980 con 0x7f92fc0b0310 2024-01-15T18:06:51.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.061+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f92fc06b7b0 con 0x7f92d4066ad0 2024-01-15T18:06:51.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f92f9600640 1 -- 172.21.15.143:0/840949157 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f92ec00be30 con 0x7f92d4066ad0 2024-01-15T18:06:51.075 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:51.076 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:51.077 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:51.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92d4066ad0 msgr2=0x7f92d4068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92d4066ad0 0x7f92d4068f90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00b850 tx=0x7f92ec008040 comp rx=0 tx=0).stop 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 msgr2=0x7f92fc0c5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0c5c40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f92e40ad040 tx=0x7f92e401ac40 comp rx=0 tx=0).stop 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 shutdown_connections 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92d4066ad0 0x7f92d4068f90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92fc0cc9c0 0x7f92fc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92fc0b0310 0x7f92fc0c5c40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 --2- 172.21.15.143:0/840949157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f92fc06bfb0 0x7f92fc0c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 >> 172.21.15.143:0/840949157 conn(0x7f92fc0d0ed0 msgr2=0x7f92fc06fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 shutdown_connections 2024-01-15T18:06:51.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:51.069+0000 7f9304400640 1 -- 172.21.15.143:0/840949157 wait complete. 2024-01-15T18:06:51.097 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:52.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:52.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bf2400640 1 Processor -- start 2024-01-15T18:06:52.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bf2400640 1 -- start start 2024-01-15T18:06:52.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bf2400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2bec0721b0 0x7f2bec0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bf2400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0b3a10 con 0x7f2bec0af860 2024-01-15T18:06:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bf2400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec06a490 con 0x7f2bec159360 2024-01-15T18:06:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bf2400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec06a750 con 0x7f2bec0721b0 2024-01-15T18:06:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2beb400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2bec159360 0x7f2bec071aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43850/0 (socket says 172.21.15.143:43850) 2024-01-15T18:06:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2beb400640 1 -- 172.21.15.143:0/3333019754 learned_addr learned my addr 172.21.15.143:0/3333019754 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2beaa00640 1 --2- 172.21.15.143:0/3333019754 >> v2:172.21.15.204:3300/0 conn(0x7f2bec0721b0 0x7f2bec0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bea000640 1 -- 172.21.15.143:0/3333019754 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565233222 0 0) 0x7f2bec06a490 con 0x7f2bec159360 2024-01-15T18:06:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2bea000640 1 -- 172.21.15.143:0/3333019754 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bcc003530 con 0x7f2bec159360 2024-01-15T18:06:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2beaa00640 1 -- 172.21.15.143:0/3333019754 >> v1:172.21.15.204:6789/0 conn(0x7f2bec159360 legacy=0x7f2bec071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2beaa00640 1 -- 172.21.15.143:0/3333019754 >> v1:172.21.15.143:6789/0 conn(0x7f2bec0af860 legacy=0x7f2bec0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.361+0000 7f2beaa00640 1 -- 172.21.15.143:0/3333019754 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec06aa30 con 0x7f2bec0721b0 2024-01-15T18:06:52.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.369+0000 7f2beaa00640 1 --2- 172.21.15.143:0/3333019754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0721b0 0x7f2bec0b3ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097f0 tx=0x7f2bd401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff9a1707bc7aba8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:52.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.369+0000 7f2bea000640 1 -- 172.21.15.143:0/3333019754 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40ab670 con 0x7f2bec0721b0 2024-01-15T18:06:52.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.369+0000 7f2bea000640 1 -- 172.21.15.143:0/3333019754 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd40ab7d0 con 0x7f2bec0721b0 2024-01-15T18:06:52.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.369+0000 7f2bea000640 1 -- 172.21.15.143:0/3333019754 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40abb60 con 0x7f2bec0721b0 2024-01-15T18:06:52.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.369+0000 7f2bf2400640 1 -- 172.21.15.143:0/3333019754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0721b0 msgr2=0x7f2bec0b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.369+0000 7f2bf2400640 1 --2- 172.21.15.143:0/3333019754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0721b0 0x7f2bec0b3ea0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40097f0 tx=0x7f2bd401c470 comp rx=0 tx=0).stop 2024-01-15T18:06:52.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.373+0000 7f2bf2400640 1 -- 172.21.15.143:0/3333019754 shutdown_connections 2024-01-15T18:06:52.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.373+0000 7f2bf2400640 1 --2- 172.21.15.143:0/3333019754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0721b0 0x7f2bec0b3ea0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.373+0000 7f2bf2400640 1 -- 172.21.15.143:0/3333019754 >> 172.21.15.143:0/3333019754 conn(0x7f2bec066720 msgr2=0x7f2bec06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:52.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.373+0000 7f2bf2400640 1 -- 172.21.15.143:0/3333019754 shutdown_connections 2024-01-15T18:06:52.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.377+0000 7f2bf2400640 1 -- 172.21.15.143:0/3333019754 wait complete. 2024-01-15T18:06:52.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.377+0000 7f2bf2400640 1 Processor -- start 2024-01-15T18:06:52.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.377+0000 7f2bf2400640 1 -- start start 2024-01-15T18:06:52.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf2400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 0x7f2bec150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:52.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0af860 0x7f2bec15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:52.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf2400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bec159360 0x7f2bec15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:52.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf2400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec153c60 con 0x7f2bec159360 2024-01-15T18:06:52.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf2400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec153050 con 0x7f2bec0af860 2024-01-15T18:06:52.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf2400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec153330 con 0x7f2bec0721b0 2024-01-15T18:06:52.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 0x7f2bec150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 0x7f2bec150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:45870/0 (socket says 172.21.15.143:45870) 2024-01-15T18:06:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 -- 172.21.15.143:0/1796937910 learned_addr learned my addr 172.21.15.143:0/1796937910 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beaa00640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0af860 0x7f2bec15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2bf0e00640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bec159360 0x7f2bec15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:52.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 -- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0af860 msgr2=0x7f2bec15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0af860 0x7f2bec15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 -- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bec159360 msgr2=0x7f2bec15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bec159360 0x7f2bec15c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.381+0000 7f2beb400640 1 -- 172.21.15.143:0/1796937910 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4009420 con 0x7f2bec0721b0 2024-01-15T18:06:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.389+0000 7f2beb400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 0x7f2bec150570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c420 tx=0x7f2bdc00c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.393+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc005cf0 con 0x7f2bec0721b0 2024-01-15T18:06:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.393+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc005e50 con 0x7f2bec0721b0 2024-01-15T18:06:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.393+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0072d0 con 0x7f2bec0721b0 2024-01-15T18:06:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.393+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec153610 con 0x7f2bec0721b0 2024-01-15T18:06:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.393+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec150ab0 con 0x7f2bec0721b0 2024-01-15T18:06:52.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.397+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec067680 con 0x7f2bec0721b0 2024-01-15T18:06:52.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.397+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f2bdc007430 con 0x7f2bec0721b0 2024-01-15T18:06:52.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.397+0000 7f2be3400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc4066b70 0x7f2bc4069030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:52.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.397+0000 7f2beaa00640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc4066b70 0x7f2bc4069030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.401+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bdc083c10 con 0x7f2bec0721b0 2024-01-15T18:06:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.401+0000 7f2beaa00640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc4066b70 0x7f2bc4069030 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40af000 tx=0x7f2bd40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:52.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.405+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2bdc050d10 con 0x7f2bec0721b0 2024-01-15T18:06:52.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.733+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bec0b06d0 con 0x7f2bc4066b70 2024-01-15T18:06:52.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.745+0000 7f2be3400640 1 -- 172.21.15.143:0/1796937910 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f2bec0b06d0 con 0x7f2bc4066b70 2024-01-15T18:06:52.754 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:52.754 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:52.754 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:06:52.754 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:06:52.754 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:06:52.755 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:06:52.756 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:52.756 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:52.756 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:52.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc4066b70 msgr2=0x7f2bc4069030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc4066b70 0x7f2bc4069030 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40af000 tx=0x7f2bd40a7000 comp rx=0 tx=0).stop 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 msgr2=0x7f2bec150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 0x7f2bec150570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00c420 tx=0x7f2bdc00c8e0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 shutdown_connections 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2bc4066b70 0x7f2bc4069030 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2bec159360 0x7f2bec15c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2bec0af860 0x7f2bec15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 --2- 172.21.15.143:0/1796937910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2bec0721b0 0x7f2bec150570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:52.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 >> 172.21.15.143:0/1796937910 conn(0x7f2bec066720 msgr2=0x7f2bec06ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:52.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 shutdown_connections 2024-01-15T18:06:52.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:52.749+0000 7f2bf2400640 1 -- 172.21.15.143:0/1796937910 wait complete. 2024-01-15T18:06:52.800 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:06:53.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:06:53.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.933+0000 7fdb1f000640 1 Processor -- start 2024-01-15T18:06:53.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.933+0000 7fdb1f000640 1 -- start start 2024-01-15T18:06:53.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb1f000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb18071bb0 0x7fdb18072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:53.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb1f000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb180ceb50 con 0x7fdb1815c5a0 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb1f000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1815c110 con 0x7fdb18143780 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb1f000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb18072570 con 0x7fdb18071bb0 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb17400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb18071bb0 0x7fdb18072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb1d000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb1815c5a0 0x7fdb1815c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55136/0 (socket says 172.21.15.143:55136) 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb1d000640 1 -- 172.21.15.143:0/266719609 learned_addr learned my addr 172.21.15.143:0/266719609 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb17400640 1 -- 172.21.15.143:0/266719609 >> v1:172.21.15.204:6789/0 conn(0x7fdb18143780 legacy=0x7fdb180cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb17400640 1 -- 172.21.15.143:0/266719609 >> v1:172.21.15.143:6789/0 conn(0x7fdb1815c5a0 legacy=0x7fdb1815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:53.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb16a00640 1 -- 172.21.15.143:0/266719609 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 107392190 0 0) 0x7fdb100014a0 con 0x7fdb1815c5a0 2024-01-15T18:06:53.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.937+0000 7fdb17400640 1 -- 172.21.15.143:0/266719609 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1815b7e0 con 0x7fdb18071bb0 2024-01-15T18:06:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb17400640 1 --2- 172.21.15.143:0/266719609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb18072030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb000097c0 tx=0x7fdb0001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce24058e054e67a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb16a00640 1 -- 172.21.15.143:0/266719609 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000ab600 con 0x7fdb18071bb0 2024-01-15T18:06:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb16a00640 1 -- 172.21.15.143:0/266719609 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb000a7050 con 0x7fdb18071bb0 2024-01-15T18:06:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb16a00640 1 -- 172.21.15.143:0/266719609 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000ab980 con 0x7fdb18071bb0 2024-01-15T18:06:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 -- 172.21.15.143:0/266719609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 msgr2=0x7fdb18072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 --2- 172.21.15.143:0/266719609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb18072030 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb000097c0 tx=0x7fdb0001c440 comp rx=0 tx=0).stop 2024-01-15T18:06:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 -- 172.21.15.143:0/266719609 shutdown_connections 2024-01-15T18:06:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 --2- 172.21.15.143:0/266719609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb18072030 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:53.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 -- 172.21.15.143:0/266719609 >> 172.21.15.143:0/266719609 conn(0x7fdb1806c3f0 msgr2=0x7fdb1806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:53.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 -- 172.21.15.143:0/266719609 shutdown_connections 2024-01-15T18:06:53.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 -- 172.21.15.143:0/266719609 wait complete. 2024-01-15T18:06:53.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 Processor -- start 2024-01-15T18:06:53.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.941+0000 7fdb1f000640 1 -- start start 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb1813b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb18143780 0x7fdb18158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb1815c5a0 0x7fdb1813e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1813baf0 con 0x7fdb18071bb0 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18159250 con 0x7fdb18143780 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18139f80 con 0x7fdb1815c5a0 2024-01-15T18:06:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb1813b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb18143780 0x7fdb18158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb1813b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36514/0 (socket says 172.21.15.143:36514) 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 -- 172.21.15.143:0/1981937519 learned_addr learned my addr 172.21.15.143:0/1981937519 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1d000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb1815c5a0 0x7fdb1813e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 -- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb1815c5a0 msgr2=0x7fdb1813e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb1815c5a0 0x7fdb1813e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 -- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb18143780 msgr2=0x7fdb18158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb18143780 0x7fdb18158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 -- 172.21.15.143:0/1981937519 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb00009420 con 0x7fdb18071bb0 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb17e00640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb1813b5b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c450 tx=0x7fdb0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08012830 con 0x7fdb18071bb0 2024-01-15T18:06:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb080129b0 con 0x7fdb18071bb0 2024-01-15T18:06:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08012d60 con 0x7fdb18071bb0 2024-01-15T18:06:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1813a260 con 0x7fdb18071bb0 2024-01-15T18:06:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1813a820 con 0x7fdb18071bb0 2024-01-15T18:06:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.945+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb18067500 con 0x7fdb18071bb0 2024-01-15T18:06:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.949+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7fdb0801c460 con 0x7fdb18071bb0 2024-01-15T18:06:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.949+0000 7fdb0fe00640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdaf0066aa0 0x7fdaf0068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.949+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb080845a0 con 0x7fdb18071bb0 2024-01-15T18:06:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.949+0000 7fdb17400640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdaf0066aa0 0x7fdaf0068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.949+0000 7fdb17400640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdaf0066aa0 0x7fdaf0068f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb18070a20 tx=0x7fdb000a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:53.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:53.953+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb08051570 con 0x7fdb18071bb0 2024-01-15T18:06:54.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.137+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb18167ad0 con 0x7fdaf0066aa0 2024-01-15T18:06:54.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb0fe00640 1 -- 172.21.15.143:0/1981937519 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fdb000ab6c0 con 0x7fdaf0066aa0 2024-01-15T18:06:54.331 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:06:54.331 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:06:54.331 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:06:54.332 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:54.332 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdaf0066aa0 msgr2=0x7fdaf0068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdaf0066aa0 0x7fdaf0068f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb18070a20 tx=0x7fdb000a6000 comp rx=0 tx=0).stop 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 msgr2=0x7fdb1813b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb1813b5b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fdb0800c450 tx=0x7fdb0800c910 comp rx=0 tx=0).stop 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 shutdown_connections 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdaf0066aa0 0x7fdaf0068f60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb1815c5a0 0x7fdb1813e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb18143780 0x7fdb18158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 --2- 172.21.15.143:0/1981937519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb18071bb0 0x7fdb1813b5b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 >> 172.21.15.143:0/1981937519 conn(0x7fdb1806c3f0 msgr2=0x7fdb180ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 shutdown_connections 2024-01-15T18:06:54.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.325+0000 7fdb1f000640 1 -- 172.21.15.143:0/1981937519 wait complete. 2024-01-15T18:06:54.354 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:06:54.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13ce200640 1 Processor -- start 2024-01-15T18:06:54.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13ce200640 1 -- start start 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13ce200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 0x7f13c8164540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13ce200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80b2a20 con 0x7f13c8157300 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13ce200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80b2d00 con 0x7f13c8159b70 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13ce200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80b35a0 con 0x7f13c80b4ac0 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c7400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f13c8157300 0x7f13c80b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55142/0 (socket says 172.21.15.143:55142) 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c7400640 1 -- 172.21.15.143:0/873876375 learned_addr learned my addr 172.21.15.143:0/873876375 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c7e00640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 0x7f13c8164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814325221 0 0) 0x7f13c80b2a20 con 0x7f13c8157300 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13a8003530 con 0x7f13c8157300 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604301273 0 0) 0x7f13c80b2d00 con 0x7f13c8159b70 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c80b2a20 con 0x7f13c8159b70 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 923536826 0 0) 0x7f13a8003530 con 0x7f13c8157300 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 msgr2=0x7f13c8164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 0x7f13c8164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 >> v1:172.21.15.204:6789/0 conn(0x7f13c8159b70 legacy=0x7f13c80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80b2fe0 con 0x7f13c8157300 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f13b8001680 con 0x7f13c8157300 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 >> v1:172.21.15.143:6789/0 conn(0x7f13c8157300 legacy=0x7f13c80b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13a8003b40 0x7f13a8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13a80047d0 0x7f13a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 0x7f13a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c80b35a0 con 0x7f13a8003b40 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a800ba70 con 0x7f13a80047d0 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.485+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a800bc30 con 0x7f13c80b4ac0 2024-01-15T18:06:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13a8003b40 0x7f13a8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7e00640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 0x7f13a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c6a00640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13a80047d0 0x7f13a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 -- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 msgr2=0x7f13a8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13c80b4ac0 0x7f13a8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 -- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13a80047d0 msgr2=0x7f13a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13a80047d0 0x7f13a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 -- 172.21.15.143:0/873876375 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a8003530 con 0x7f13a8003b40 2024-01-15T18:06:54.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c6a00640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13a80047d0 0x7f13a8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c7400640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13a8003b40 0x7f13a8003f40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f13b8007fa0 tx=0x7f13b801cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ad1195fc073359 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8005ae0 con 0x7f13a8003b40 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8005c40 con 0x7f13a8003b40 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.489+0000 7f13c6000640 1 -- 172.21.15.143:0/873876375 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b80a57d0 con 0x7f13a8003b40 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- 172.21.15.143:0/873876375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13a8003b40 msgr2=0x7f13a8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 --2- 172.21.15.143:0/873876375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13a8003b40 0x7f13a8003f40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f13b8007fa0 tx=0x7f13b801cad0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13c7e00640 1 -- 172.21.15.143:0/873876375 reap_dead start 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- 172.21.15.143:0/873876375 shutdown_connections 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- 172.21.15.143:0/873876375 >> 172.21.15.143:0/873876375 conn(0x7f13c806c3f0 msgr2=0x7f13c806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- 172.21.15.143:0/873876375 shutdown_connections 2024-01-15T18:06:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- 172.21.15.143:0/873876375 wait complete. 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 Processor -- start 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- start start 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13a8003b40 0x7f13c8161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 0x7f13c80a9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13c81557b0 0x7f13c80aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c8159fd0 con 0x7f13c8155330 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81586b0 con 0x7f13c81557b0 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13ce200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c81583d0 con 0x7f13a8003b40 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13c6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 0x7f13c80a9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13c6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 0x7f13c80a9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36532/0 (socket says 172.21.15.143:36532) 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13c6a00640 1 -- 172.21.15.143:0/766374105 learned_addr learned my addr 172.21.15.143:0/766374105 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13c7400640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13a8003b40 0x7f13c8161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.493+0000 7f13c7e00640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13c81557b0 0x7f13c80aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c6a00640 1 -- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13a8003b40 msgr2=0x7f13c8161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c6a00640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13a8003b40 0x7f13c8161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c6a00640 1 -- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13c81557b0 msgr2=0x7f13c80aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c6a00640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13c81557b0 0x7f13c80aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c6a00640 1 -- 172.21.15.143:0/766374105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c8154850 con 0x7f13c8155330 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c6a00640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 0x7f13c80a9b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13b000e530 tx=0x7f13b000e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0017070 con 0x7f13c8155330 2024-01-15T18:06:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b00052e0 con 0x7f13c8155330 2024-01-15T18:06:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0005470 con 0x7f13c8155330 2024-01-15T18:06:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c8154b30 con 0x7f13c8155330 2024-01-15T18:06:54.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c8161b80 con 0x7f13c8155330 2024-01-15T18:06:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c81546b0 con 0x7f13c8155330 2024-01-15T18:06:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f13b0007020 con 0x7f13c8155330 2024-01-15T18:06:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13bf400640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13a0066aa0 0x7f13a0068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13b0083de0 con 0x7f13c8155330 2024-01-15T18:06:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.497+0000 7f13c7400640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13a0066aa0 0x7f13a0068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.501+0000 7f13c7400640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13a0066aa0 0x7f13a0068f60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13b80ab000 tx=0x7f13b80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:54.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.501+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13c81546b0 con 0x7f13c8155330 2024-01-15T18:06:54.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.681+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f13c81546b0 con 0x7f13a0066aa0 2024-01-15T18:06:54.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.681+0000 7f13bf400640 1 -- 172.21.15.143:0/766374105 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f13c81546b0 con 0x7f13a0066aa0 2024-01-15T18:06:54.690 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:06:54.690 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:54.690 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:06:54.690 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:54.690 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:06:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13a0066aa0 msgr2=0x7f13a0068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13a0066aa0 0x7f13a0068f60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13b80ab000 tx=0x7f13b80a7070 comp rx=0 tx=0).stop 2024-01-15T18:06:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 msgr2=0x7f13c80a9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 0x7f13c80a9b90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13b000e530 tx=0x7f13b000e9f0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 shutdown_connections 2024-01-15T18:06:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13a0066aa0 0x7f13a0068f60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13c81557b0 0x7f13c80aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13c8155330 0x7f13c80a9b90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 --2- 172.21.15.143:0/766374105 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13a8003b40 0x7f13c8161640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 >> 172.21.15.143:0/766374105 conn(0x7f13c806c3f0 msgr2=0x7f13c80afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:54.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 shutdown_connections 2024-01-15T18:06:54.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.685+0000 7f13ce200640 1 -- 172.21.15.143:0/766374105 wait complete. 2024-01-15T18:06:54.711 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:06:54.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.841+0000 7f3741600640 1 Processor -- start 2024-01-15T18:06:54.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.841+0000 7f3741600640 1 -- start start 2024-01-15T18:06:54.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f3741600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f373c14b110 0x7f373c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f3741600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c0b4150 0x7f373c071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f3741600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373c0b30a0 con 0x7f373c0cc8f0 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f3741600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373c070340 con 0x7f373c14b110 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f3741600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c070620 con 0x7f373c0b4150 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f373c14b110 0x7f373c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f373c14b110 0x7f373c0cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36534/0 (socket says 172.21.15.143:36534) 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 -- 172.21.15.143:0/3013130200 learned_addr learned my addr 172.21.15.143:0/3013130200 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373b400640 1 --2- 172.21.15.143:0/3013130200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c0b4150 0x7f373c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 -- 172.21.15.143:0/3013130200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c0b4150 msgr2=0x7f373c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:54.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 --2- 172.21.15.143:0/3013130200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c0b4150 0x7f373c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 -- 172.21.15.143:0/3013130200 >> v1:172.21.15.204:6789/0 conn(0x7f373c0cc8f0 legacy=0x7f373c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.845+0000 7f373aa00640 1 -- 172.21.15.143:0/3013130200 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373c070900 con 0x7f373c14b110 2024-01-15T18:06:54.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f373aa00640 1 --2- 172.21.15.143:0/3013130200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c14b110 0x7f373c0cc3b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f372c009870 tx=0x7f372c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba787a0b8d80957 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:54.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3739600640 1 -- 172.21.15.143:0/3013130200 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0a6040 con 0x7f373c14b110 2024-01-15T18:06:54.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3739600640 1 -- 172.21.15.143:0/3013130200 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c0b2e90 con 0x7f373c14b110 2024-01-15T18:06:54.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3739600640 1 -- 172.21.15.143:0/3013130200 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0aa820 con 0x7f373c14b110 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 -- 172.21.15.143:0/3013130200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c14b110 msgr2=0x7f373c0cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 --2- 172.21.15.143:0/3013130200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c14b110 0x7f373c0cc3b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f372c009870 tx=0x7f372c01c480 comp rx=0 tx=0).stop 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 -- 172.21.15.143:0/3013130200 shutdown_connections 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 --2- 172.21.15.143:0/3013130200 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c0b4150 0x7f373c071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 --2- 172.21.15.143:0/3013130200 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c14b110 0x7f373c0cc3b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 -- 172.21.15.143:0/3013130200 >> 172.21.15.143:0/3013130200 conn(0x7f373c06c3f0 msgr2=0x7f373c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 -- 172.21.15.143:0/3013130200 shutdown_connections 2024-01-15T18:06:54.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 -- 172.21.15.143:0/3013130200 wait complete. 2024-01-15T18:06:54.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.849+0000 7f3741600640 1 Processor -- start 2024-01-15T18:06:54.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- start start 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 0x7f373c15f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c0cc8f0 0x7f373c163f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c14b110 0x7f373c148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c15fec0 con 0x7f373c0b4150 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373c160140 con 0x7f373c0cc8f0 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373c160420 con 0x7f373c14b110 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 0x7f373c15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 0x7f373c15f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36536/0 (socket says 172.21.15.143:36536) 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 -- 172.21.15.143:0/2689734178 learned_addr learned my addr 172.21.15.143:0/2689734178 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373b400640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c14b110 0x7f373c148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373a000640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c0cc8f0 0x7f373c163f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 -- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c14b110 msgr2=0x7f373c148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c14b110 0x7f373c148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 -- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c0cc8f0 msgr2=0x7f373c163f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c0cc8f0 0x7f373c163f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 -- 172.21.15.143:0/2689734178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c0094d0 con 0x7f373c0b4150 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373a000640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c0cc8f0 0x7f373c163f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f373aa00640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 0x7f373c15f980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f372c01a0b0 tx=0x7f372c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0bd5f0 con 0x7f373c0b4150 2024-01-15T18:06:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c0bd750 con 0x7f373c0b4150 2024-01-15T18:06:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0bdae0 con 0x7f373c0b4150 2024-01-15T18:06:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373c160700 con 0x7f373c0b4150 2024-01-15T18:06:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f373c1644d0 con 0x7f373c0b4150 2024-01-15T18:06:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.853+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f373c0cb5f0 con 0x7f373c0b4150 2024-01-15T18:06:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.857+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f372c0c6430 con 0x7f373c0b4150 2024-01-15T18:06:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.857+0000 7f3732a00640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3714066aa0 0x7f3714068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.857+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f372c11f620 con 0x7f373c0b4150 2024-01-15T18:06:54.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.857+0000 7f373a000640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3714066aa0 0x7f3714068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.857+0000 7f373a000640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3714066aa0 0x7f3714068f60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f373c144690 tx=0x7f3724009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:54.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:54.861+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f372c0ec570 con 0x7f373c0b4150 2024-01-15T18:06:55.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.033+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f373c1646c0 con 0x7f3714066aa0 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3732a00640 1 -- 172.21.15.143:0/2689734178 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7f373c1646c0 con 0x7f3714066aa0 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1clone1" 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:55.044 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3714066aa0 msgr2=0x7f3714068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3714066aa0 0x7f3714068f60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f373c144690 tx=0x7f3724009040 comp rx=0 tx=0).stop 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 msgr2=0x7f373c15f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 0x7f373c15f980 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f372c01a0b0 tx=0x7f372c01acc0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 shutdown_connections 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3714066aa0 0x7f3714068f60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f373c14b110 0x7f373c148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f373c0cc8f0 0x7f373c163f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 --2- 172.21.15.143:0/2689734178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f373c0b4150 0x7f373c15f980 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 >> 172.21.15.143:0/2689734178 conn(0x7f373c06c3f0 msgr2=0x7f373c0aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:55.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 shutdown_connections 2024-01-15T18:06:55.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.037+0000 7f3741600640 1 -- 172.21.15.143:0/2689734178 wait complete. 2024-01-15T18:06:55.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:06:55.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.197+0000 7fc197c00640 1 Processor -- start 2024-01-15T18:06:55.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc197c00640 1 -- start start 2024-01-15T18:06:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc197c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc190143780 0x7fc1900cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc197c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1900ceb50 con 0x7fc1900ce520 2024-01-15T18:06:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc197c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19015c110 con 0x7fc1900722c0 2024-01-15T18:06:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc197c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19015b7e0 con 0x7fc190143780 2024-01-15T18:06:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc195200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc190143780 0x7fc1900cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc195200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc190143780 0x7fc1900cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36540/0 (socket says 172.21.15.143:36540) 2024-01-15T18:06:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc195200640 1 -- 172.21.15.143:0/1043119265 learned_addr learned my addr 172.21.15.143:0/1043119265 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc18fe00640 1 --1- 172.21.15.143:0/1043119265 >> v1:172.21.15.143:6789/0 conn(0x7fc1900ce520 0x7fc190071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55156/0 (socket says 172.21.15.143:55156) 2024-01-15T18:06:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc195200640 1 -- 172.21.15.143:0/1043119265 >> v1:172.21.15.204:6789/0 conn(0x7fc1900722c0 legacy=0x7fc19015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc195200640 1 -- 172.21.15.143:0/1043119265 >> v1:172.21.15.143:6789/0 conn(0x7fc1900ce520 legacy=0x7fc190071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.201+0000 7fc195200640 1 -- 172.21.15.143:0/1043119265 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19015bac0 con 0x7fc190143780 2024-01-15T18:06:55.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc195200640 1 --2- 172.21.15.143:0/1043119265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc190143780 0x7fc1900cdfe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc180009870 tx=0x7fc18001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11dcb4eb1a9572e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc18f400640 1 -- 172.21.15.143:0/1043119265 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800ab650 con 0x7fc190143780 2024-01-15T18:06:55.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc18f400640 1 -- 172.21.15.143:0/1043119265 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1800ab7b0 con 0x7fc190143780 2024-01-15T18:06:55.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc18f400640 1 -- 172.21.15.143:0/1043119265 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800abb40 con 0x7fc190143780 2024-01-15T18:06:55.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 -- 172.21.15.143:0/1043119265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc190143780 msgr2=0x7fc1900cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 --2- 172.21.15.143:0/1043119265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc190143780 0x7fc1900cdfe0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc180009870 tx=0x7fc18001c540 comp rx=0 tx=0).stop 2024-01-15T18:06:55.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 -- 172.21.15.143:0/1043119265 shutdown_connections 2024-01-15T18:06:55.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 --2- 172.21.15.143:0/1043119265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc190143780 0x7fc1900cdfe0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 -- 172.21.15.143:0/1043119265 >> 172.21.15.143:0/1043119265 conn(0x7fc19006c3f0 msgr2=0x7fc19006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:55.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 -- 172.21.15.143:0/1043119265 shutdown_connections 2024-01-15T18:06:55.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.205+0000 7fc197c00640 1 -- 172.21.15.143:0/1043119265 wait complete. 2024-01-15T18:06:55.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 Processor -- start 2024-01-15T18:06:55.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- start start 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1900722c0 0x7fc1900cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 0x7fc1900af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc190143780 0x7fc19013cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900cbba0 con 0x7fc1900ce520 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1901f57b0 con 0x7fc1900722c0 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1901f5a20 con 0x7fc190143780 2024-01-15T18:06:55.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 0x7fc1900af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 0x7fc1900af9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36548/0 (socket says 172.21.15.143:36548) 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 -- 172.21.15.143:0/1111802424 learned_addr learned my addr 172.21.15.143:0/1111802424 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc195200640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1900722c0 0x7fc1900cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc195c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc190143780 0x7fc19013cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 -- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc190143780 msgr2=0x7fc19013cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc190143780 0x7fc19013cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 -- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1900722c0 msgr2=0x7fc1900cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1900722c0 0x7fc1900cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 -- 172.21.15.143:0/1111802424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800094d0 con 0x7fc1900ce520 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18fe00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 0x7fc1900af9b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c490 tx=0x7fc17800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178007480 con 0x7fc1900ce520 2024-01-15T18:06:55.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1780075e0 con 0x7fc1900ce520 2024-01-15T18:06:55.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1901f5d00 con 0x7fc1900ce520 2024-01-15T18:06:55.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178004a10 con 0x7fc1900ce520 2024-01-15T18:06:55.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1901f62c0 con 0x7fc1900ce520 2024-01-15T18:06:55.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.209+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc190067500 con 0x7fc1900ce520 2024-01-15T18:06:55.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.213+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7fc178004b70 con 0x7fc1900ce520 2024-01-15T18:06:55.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.213+0000 7fc18cc00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc168066aa0 0x7fc168068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.213+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc178083aa0 con 0x7fc1900ce520 2024-01-15T18:06:55.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.213+0000 7fc195200640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc168066aa0 0x7fc168068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.213+0000 7fc195200640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc168066aa0 0x7fc168068f60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc1800af000 tx=0x7fc1800a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.217+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1780509c0 con 0x7fc1900ce520 2024-01-15T18:06:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.393+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc19015d750 con 0x7fc168066aa0 2024-01-15T18:06:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc18cc00640 1 -- 172.21.15.143:0/1111802424 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc1800ab920 con 0x7fc168066aa0 2024-01-15T18:06:55.416 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:55.416 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:06:55.416 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:06:55.416 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:06:55.416 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:06:55.416 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:06:55.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc168066aa0 msgr2=0x7fc168068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc168066aa0 0x7fc168068f60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc1800af000 tx=0x7fc1800a7000 comp rx=0 tx=0).stop 2024-01-15T18:06:55.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 msgr2=0x7fc1900af9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 0x7fc1900af9b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c490 tx=0x7fc17800c950 comp rx=0 tx=0).stop 2024-01-15T18:06:55.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 shutdown_connections 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc168066aa0 0x7fc168068f60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc190143780 0x7fc19013cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1900ce520 0x7fc1900af9b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 --2- 172.21.15.143:0/1111802424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1900722c0 0x7fc1900cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 >> 172.21.15.143:0/1111802424 conn(0x7fc19006c3f0 msgr2=0x7fc19015c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 shutdown_connections 2024-01-15T18:06:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.409+0000 7fc197c00640 1 -- 172.21.15.143:0/1111802424 wait complete. 2024-01-15T18:06:55.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:06:55.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f5392800640 1 Processor -- start 2024-01-15T18:06:55.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f5392800640 1 -- start start 2024-01-15T18:06:55.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f5392800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f538c06aba0 0x7f538c0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f5392800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538c140b90 con 0x7f538c15b610 2024-01-15T18:06:55.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f5392800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538c0cfba0 con 0x7f538c140700 2024-01-15T18:06:55.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f5392800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538c13e420 con 0x7f538c06aba0 2024-01-15T18:06:55.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f538c15b610 0x7f538c071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55168/0 (socket says 172.21.15.143:55168) 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538b400640 1 -- 172.21.15.143:0/1999812911 learned_addr learned my addr 172.21.15.143:0/1999812911 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538be00640 1 --2- 172.21.15.143:0/1999812911 >> v2:172.21.15.204:3300/0 conn(0x7f538c06aba0 0x7f538c0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268515439 0 0) 0x7f538c140b90 con 0x7f538c15b610 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f536c003530 con 0x7f538c15b610 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170657619 0 0) 0x7f536c003530 con 0x7f538c15b610 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 >> v2:172.21.15.204:3300/0 conn(0x7f538c06aba0 msgr2=0x7f538c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 --2- 172.21.15.143:0/1999812911 >> v2:172.21.15.204:3300/0 conn(0x7f538c06aba0 0x7f538c0cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 >> v1:172.21.15.204:6789/0 conn(0x7f538c140700 legacy=0x7f538c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c140e00 con 0x7f538c15b610 2024-01-15T18:06:55.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f537c001680 con 0x7f538c15b610 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 >> v1:172.21.15.143:6789/0 conn(0x7f538c15b610 legacy=0x7f538c071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f536c003980 0x7f536c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536c004380 0x7f536c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c005050 0x7f536c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c00bf80 con 0x7f536c004380 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c00c0f0 con 0x7f536c005050 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.573+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c00c2b0 con 0x7f536c003980 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536c004380 0x7f536c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538be00640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c005050 0x7f536c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538b400640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f536c003980 0x7f536c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 -- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f536c003980 msgr2=0x7f536c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:55.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f536c003980 0x7f536c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 -- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c005050 msgr2=0x7f536c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:55.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c005050 0x7f536c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 -- 172.21.15.143:0/1999812911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c003530 con 0x7f536c004380 2024-01-15T18:06:55.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538be00640 1 -- 172.21.15.143:0/1999812911 reap_dead start 2024-01-15T18:06:55.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538aa00640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536c004380 0x7f536c004850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5374001910 tx=0x7f537401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10756b64d7e60cbb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53740a88b0 con 0x7f536c004380 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53740a8a30 con 0x7f536c004380 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.577+0000 7f538a000640 1 -- 172.21.15.143:0/1999812911 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53740a8de0 con 0x7f536c004380 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- 172.21.15.143:0/1999812911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536c004380 msgr2=0x7f536c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536c004380 0x7f536c004850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5374001910 tx=0x7f537401ec90 comp rx=0 tx=0).stop 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- 172.21.15.143:0/1999812911 shutdown_connections 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 --2- 172.21.15.143:0/1999812911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f536c004380 0x7f536c004850 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- 172.21.15.143:0/1999812911 >> 172.21.15.143:0/1999812911 conn(0x7f538c0b1b90 msgr2=0x7f538c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- 172.21.15.143:0/1999812911 shutdown_connections 2024-01-15T18:06:55.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- 172.21.15.143:0/1999812911 wait complete. 2024-01-15T18:06:55.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 Processor -- start 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- start start 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c004380 0x7f538c13c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 0x7f538c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0b0ea0 0x7f538c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c0b1890 con 0x7f538c143510 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c13cd30 con 0x7f536c004380 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f5392800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c1f2520 con 0x7f538c0b0ea0 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f538aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 0x7f538c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f538aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 0x7f538c0b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36566/0 (socket says 172.21.15.143:36566) 2024-01-15T18:06:55.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f538aa00640 1 -- 172.21.15.143:0/885429794 learned_addr learned my addr 172.21.15.143:0/885429794 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f538b400640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c004380 0x7f538c13c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.581+0000 7f538be00640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0b0ea0 0x7f538c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538aa00640 1 -- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0b0ea0 msgr2=0x7f538c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538aa00640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0b0ea0 0x7f538c0b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538aa00640 1 -- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c004380 msgr2=0x7f538c13c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538aa00640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c004380 0x7f538c13c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538aa00640 1 -- 172.21.15.143:0/885429794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c1439a0 con 0x7f538c143510 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538aa00640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 0x7f538c0b49c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f537401eac0 tx=0x7f537401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53740069e0 con 0x7f538c143510 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5374006b40 con 0x7f538c143510 2024-01-15T18:06:55.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5374007030 con 0x7f538c143510 2024-01-15T18:06:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c143c80 con 0x7f538c143510 2024-01-15T18:06:55.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c144240 con 0x7f538c143510 2024-01-15T18:06:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c067500 con 0x7f538c143510 2024-01-15T18:06:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f53740b7070 con 0x7f538c143510 2024-01-15T18:06:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5383400640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5364066aa0 0x7f5364068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53741217e0 con 0x7f538c143510 2024-01-15T18:06:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.585+0000 7f538b400640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5364066aa0 0x7f5364068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.589+0000 7f538b400640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5364066aa0 0x7f5364068f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f537c001c30 tx=0x7f537c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.589+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f53740ee8f0 con 0x7f538c143510 2024-01-15T18:06:55.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.769+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f538c071400 con 0x7f5364066aa0 2024-01-15T18:06:55.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.797+0000 7f5383400640 1 -- 172.21.15.143:0/885429794 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f537c011920 con 0x7f5364066aa0 2024-01-15T18:06:55.806 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5364066aa0 msgr2=0x7f5364068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5364066aa0 0x7f5364068f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f537c001c30 tx=0x7f537c007040 comp rx=0 tx=0).stop 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 msgr2=0x7f538c0b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 0x7f538c0b49c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f537401eac0 tx=0x7f537401ba30 comp rx=0 tx=0).stop 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 shutdown_connections 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5364066aa0 0x7f5364068f60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f538c0b0ea0 0x7f538c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f538c143510 0x7f538c0b49c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 --2- 172.21.15.143:0/885429794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f536c004380 0x7f538c13c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 >> 172.21.15.143:0/885429794 conn(0x7f538c0b1b90 msgr2=0x7f538c0afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:55.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 shutdown_connections 2024-01-15T18:06:55.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.801+0000 7f5392800640 1 -- 172.21.15.143:0/885429794 wait complete. 2024-01-15T18:06:55.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:06:55.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 Processor -- start 2024-01-15T18:06:55.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 -- start start 2024-01-15T18:06:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f092c071f70 0x7f092c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f092c0b1da0 0x7f092c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c0b30a0 con 0x7f092c14b110 2024-01-15T18:06:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c070340 con 0x7f092c071f70 2024-01-15T18:06:55.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0934200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c070620 con 0x7f092c0b1da0 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.961+0000 7f0930e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f092c071f70 0x7f092c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0931800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f092c14b110 0x7f092c0cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55178/0 (socket says 172.21.15.143:55178) 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0931800640 1 -- 172.21.15.143:0/3476953023 learned_addr learned my addr 172.21.15.143:0/3476953023 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0932200640 1 --2- 172.21.15.143:0/3476953023 >> v2:172.21.15.204:3300/0 conn(0x7f092c0b1da0 0x7f092c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0930e00640 1 -- 172.21.15.143:0/3476953023 >> v2:172.21.15.204:3300/0 conn(0x7f092c0b1da0 msgr2=0x7f092c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0930e00640 1 --2- 172.21.15.143:0/3476953023 >> v2:172.21.15.204:3300/0 conn(0x7f092c0b1da0 0x7f092c0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0930e00640 1 -- 172.21.15.143:0/3476953023 >> v1:172.21.15.143:6789/0 conn(0x7f092c14b110 legacy=0x7f092c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0930e00640 1 -- 172.21.15.143:0/3476953023 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c070900 con 0x7f092c071f70 2024-01-15T18:06:55.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0932200640 1 --2- 172.21.15.143:0/3476953023 >> v2:172.21.15.204:3300/0 conn(0x7f092c0b1da0 0x7f092c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:55.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f0930e00640 1 --2- 172.21.15.143:0/3476953023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c071f70 0x7f092c0b3a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f092c06fe30 tx=0x7f091401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f10c94b156e69 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f092be00640 1 -- 172.21.15.143:0/3476953023 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140ab5f0 con 0x7f092c071f70 2024-01-15T18:06:55.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f092be00640 1 -- 172.21.15.143:0/3476953023 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140ab750 con 0x7f092c071f70 2024-01-15T18:06:55.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.965+0000 7f092be00640 1 -- 172.21.15.143:0/3476953023 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140abae0 con 0x7f092c071f70 2024-01-15T18:06:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- 172.21.15.143:0/3476953023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c071f70 msgr2=0x7f092c0b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 --2- 172.21.15.143:0/3476953023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c071f70 0x7f092c0b3a40 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f092c06fe30 tx=0x7f091401c3d0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- 172.21.15.143:0/3476953023 shutdown_connections 2024-01-15T18:06:55.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 --2- 172.21.15.143:0/3476953023 >> v2:172.21.15.204:3300/0 conn(0x7f092c0b1da0 0x7f092c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 --2- 172.21.15.143:0/3476953023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c071f70 0x7f092c0b3a40 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- 172.21.15.143:0/3476953023 >> 172.21.15.143:0/3476953023 conn(0x7f092c06c3f0 msgr2=0x7f092c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- 172.21.15.143:0/3476953023 shutdown_connections 2024-01-15T18:06:55.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- 172.21.15.143:0/3476953023 wait complete. 2024-01-15T18:06:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 Processor -- start 2024-01-15T18:06:55.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- start start 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f092c071f70 0x7f092c1430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 0x7f092c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f092c14b110 0x7f092c158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0a84d0 con 0x7f092c0b1da0 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c159690 con 0x7f092c14b110 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0934200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c15dca0 con 0x7f092c071f70 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0930e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 0x7f092c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0930e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 0x7f092c1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36584/0 (socket says 172.21.15.143:36584) 2024-01-15T18:06:55.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.969+0000 7f0930e00640 1 -- 172.21.15.143:0/1957801834 learned_addr learned my addr 172.21.15.143:0/1957801834 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0932200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f092c14b110 0x7f092c158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0931800640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f092c071f70 0x7f092c1430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0930e00640 1 -- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f092c071f70 msgr2=0x7f092c1430b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0930e00640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f092c071f70 0x7f092c1430b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0930e00640 1 -- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f092c14b110 msgr2=0x7f092c158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0930e00640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f092c14b110 0x7f092c158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0930e00640 1 -- 172.21.15.143:0/1957801834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009420 con 0x7f092c0b1da0 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0930e00640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 0x7f092c1603f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0914018fe0 tx=0x7f091401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140bf610 con 0x7f092c0b1da0 2024-01-15T18:06:55.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140bf770 con 0x7f092c0b1da0 2024-01-15T18:06:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140bfb20 con 0x7f092c0b1da0 2024-01-15T18:06:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c15df80 con 0x7f092c0b1da0 2024-01-15T18:06:55.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c15e540 con 0x7f092c0b1da0 2024-01-15T18:06:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c067500 con 0x7f092c0b1da0 2024-01-15T18:06:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.973+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7f09140bfc80 con 0x7f092c0b1da0 2024-01-15T18:06:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.977+0000 7f0929600640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0904066aa0 0x7f0904068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.977+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09140a8050 con 0x7f092c0b1da0 2024-01-15T18:06:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.977+0000 7f0931800640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0904066aa0 0x7f0904068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:55.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.977+0000 7f0931800640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0904066aa0 0x7f0904068f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f091c005890 tx=0x7f091c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:55.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:55.981+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f09140ebdf0 con 0x7f092c0b1da0 2024-01-15T18:06:56.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.153+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f092c16e540 con 0x7f0904066aa0 2024-01-15T18:06:56.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.165+0000 7f0929600640 1 -- 172.21.15.143:0/1957801834 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f091c001c80 con 0x7f0904066aa0 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0904066aa0 msgr2=0x7f0904068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0904066aa0 0x7f0904068f60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f091c005890 tx=0x7f091c009040 comp rx=0 tx=0).stop 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 msgr2=0x7f092c1603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 0x7f092c1603f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0914018fe0 tx=0x7f091401ac10 comp rx=0 tx=0).stop 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 shutdown_connections 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0904066aa0 0x7f0904068f60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f092c14b110 0x7f092c158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f092c0b1da0 0x7f092c1603f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 --2- 172.21.15.143:0/1957801834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f092c071f70 0x7f092c1430b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 >> 172.21.15.143:0/1957801834 conn(0x7f092c06c3f0 msgr2=0x7f092c163ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:56.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 shutdown_connections 2024-01-15T18:06:56.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.169+0000 7f0934200640 1 -- 172.21.15.143:0/1957801834 wait complete. 2024-01-15T18:06:56.178 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.173+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.179 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.173+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.179 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.173+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.179 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.173+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.179 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.173+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.196 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.189+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.196 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.189+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.196 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.189+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.196 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.189+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.196 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:06:56.189+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:56.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-15T18:06:56.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.337+0000 7fc19ce00640 1 Processor -- start 2024-01-15T18:06:56.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc19ce00640 1 -- start start 2024-01-15T18:06:56.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc19ce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1980b0310 0x7fc1980cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980cf740 0x7fc1980cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc19ce00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980cbff0 con 0x7fc19806bfb0 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc19ce00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1980d0c50 con 0x7fc1980b0310 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc19ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980cb6c0 con 0x7fc1980cf740 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc1980b0310 0x7fc1980cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc19806bfb0 0x7fc19806c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55188/0 (socket says 172.21.15.143:55188) 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195e00640 1 -- 172.21.15.143:0/4287607718 learned_addr learned my addr 172.21.15.143:0/4287607718 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc196800640 1 --2- 172.21.15.143:0/4287607718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980cf740 0x7fc1980cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195400640 1 -- 172.21.15.143:0/4287607718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980cf740 msgr2=0x7fc1980cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195400640 1 --2- 172.21.15.143:0/4287607718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980cf740 0x7fc1980cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195400640 1 -- 172.21.15.143:0/4287607718 >> v1:172.21.15.143:6789/0 conn(0x7fc19806bfb0 legacy=0x7fc19806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.341+0000 7fc195400640 1 -- 172.21.15.143:0/4287607718 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980cb9a0 con 0x7fc1980b0310 2024-01-15T18:06:56.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc195400640 1 --2- 172.21.15.143:0/4287607718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1980b0310 0x7fc1980cf200 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc1800097c0 tx=0x7fc18001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97f6d3d33bf8c47 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:56.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc194a00640 1 -- 172.21.15.143:0/4287607718 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800ac620 con 0x7fc1980b0310 2024-01-15T18:06:56.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc194a00640 1 -- 172.21.15.143:0/4287607718 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1800ac780 con 0x7fc1980b0310 2024-01-15T18:06:56.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc194a00640 1 -- 172.21.15.143:0/4287607718 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800acb10 con 0x7fc1980b0310 2024-01-15T18:06:56.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 -- 172.21.15.143:0/4287607718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1980b0310 msgr2=0x7fc1980cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 --2- 172.21.15.143:0/4287607718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1980b0310 0x7fc1980cf200 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc1800097c0 tx=0x7fc18001c3c0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 -- 172.21.15.143:0/4287607718 shutdown_connections 2024-01-15T18:06:56.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 --2- 172.21.15.143:0/4287607718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980cf740 0x7fc1980cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 --2- 172.21.15.143:0/4287607718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1980b0310 0x7fc1980cf200 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 -- 172.21.15.143:0/4287607718 >> 172.21.15.143:0/4287607718 conn(0x7fc1980d0ed0 msgr2=0x7fc1980d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:56.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 -- 172.21.15.143:0/4287607718 shutdown_connections 2024-01-15T18:06:56.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.345+0000 7fc19ce00640 1 -- 172.21.15.143:0/4287607718 wait complete. 2024-01-15T18:06:56.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 Processor -- start 2024-01-15T18:06:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 -- start start 2024-01-15T18:06:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 0x7fc1980c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980b0310 0x7fc198153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1980cf740 0x7fc1980c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c9400 con 0x7fc19806bfb0 2024-01-15T18:06:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c84e0 con 0x7fc1980cf740 2024-01-15T18:06:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980c8780 con 0x7fc1980b0310 2024-01-15T18:06:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980b0310 0x7fc198153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 0x7fc1980c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc196800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1980cf740 0x7fc1980c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc196800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1980cf740 0x7fc1980c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:53598/0 (socket says 172.21.15.143:53598) 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 0x7fc1980c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36626/0 (socket says 172.21.15.143:36626) 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc196800640 1 -- 172.21.15.143:0/392731560 learned_addr learned my addr 172.21.15.143:0/392731560 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 -- 172.21.15.143:0/392731560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980b0310 msgr2=0x7fc198153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980b0310 0x7fc198153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 -- 172.21.15.143:0/392731560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1980cf740 msgr2=0x7fc1980c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1980cf740 0x7fc1980c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 -- 172.21.15.143:0/392731560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180009420 con 0x7fc19806bfb0 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc195e00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 0x7fc1980c7fa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c340 tx=0x7fc18800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188005cf0 con 0x7fc19806bfb0 2024-01-15T18:06:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188005e50 con 0x7fc19806bfb0 2024-01-15T18:06:56.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1880072d0 con 0x7fc19806bfb0 2024-01-15T18:06:56.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1980c8a60 con 0x7fc19806bfb0 2024-01-15T18:06:56.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.349+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1980c9020 con 0x7fc19806bfb0 2024-01-15T18:06:56.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.353+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1980c9670 con 0x7fc19806bfb0 2024-01-15T18:06:56.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.353+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 7) v1 ==== 86313+0+0 (secure 0 0 0) 0x7fc188007430 con 0x7fc19806bfb0 2024-01-15T18:06:56.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.353+0000 7fc18e000640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc170066aa0 0x7fc170068f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.353+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc188083ad0 con 0x7fc19806bfb0 2024-01-15T18:06:56.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.353+0000 7fc195400640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc170066aa0 0x7fc170068f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.353+0000 7fc195400640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc170066aa0 0x7fc170068f60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1800ae000 tx=0x7fc180008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:56.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.357+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc188050be0 con 0x7fc19806bfb0 2024-01-15T18:06:56.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.533+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1980ce690 con 0x7fc170066aa0 2024-01-15T18:06:56.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc18e000640 1 -- 172.21.15.143:0/392731560 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc1800ac8f0 con 0x7fc170066aa0 2024-01-15T18:06:56.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc170066aa0 msgr2=0x7fc170068f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc170066aa0 0x7fc170068f60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc1800ae000 tx=0x7fc180008040 comp rx=0 tx=0).stop 2024-01-15T18:06:56.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 msgr2=0x7fc1980c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 0x7fc1980c7fa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c340 tx=0x7fc18800c800 comp rx=0 tx=0).stop 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 shutdown_connections 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc170066aa0 0x7fc170068f60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1980cf740 0x7fc1980c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1980b0310 0x7fc198153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 --2- 172.21.15.143:0/392731560 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc19806bfb0 0x7fc1980c7fa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 >> 172.21.15.143:0/392731560 conn(0x7fc1980d0ed0 msgr2=0x7fc1980d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 shutdown_connections 2024-01-15T18:06:56.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.549+0000 7fc19ce00640 1 -- 172.21.15.143:0/392731560 wait complete. 2024-01-15T18:06:56.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:06:56.580 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:06:56.599 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-15T18:06:56.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:06:56.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 Processor -- start 2024-01-15T18:06:56.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 -- start start 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780cdf20 0x7fa8780cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa8780c6230 0x7fa8780cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780c4d90 con 0x7fa87815b260 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87815b640 con 0x7fa8780c6230 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa87e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780cf800 con 0x7fa8780cdf20 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa8780c6230 0x7fa8780cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa87815b260 0x7fa8780cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43894/0 (socket says 172.21.15.143:43894) 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa876a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780cdf20 0x7fa8780cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877400640 1 -- 172.21.15.143:0/887531666 learned_addr learned my addr 172.21.15.143:0/887531666 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:06:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877e00640 1 -- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780cdf20 msgr2=0x7fa8780cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877e00640 1 --2- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780cdf20 0x7fa8780cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877e00640 1 -- 172.21.15.143:0/887531666 >> v1:172.21.15.204:6789/0 conn(0x7fa87815b260 legacy=0x7fa8780cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa876000640 1 -- 172.21.15.143:0/887531666 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119709016 0 0) 0x7fa8780c4d90 con 0x7fa87815b260 2024-01-15T18:06:56.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.793+0000 7fa877e00640 1 -- 172.21.15.143:0/887531666 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780cfae0 con 0x7fa8780c6230 2024-01-15T18:06:56.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa877e00640 1 --2- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8780c6230 0x7fa8780cc020 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa870009d90 tx=0x7fa87001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a083001b6f04987 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:56.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa876000640 1 -- 172.21.15.143:0/887531666 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8700a59e0 con 0x7fa8780c6230 2024-01-15T18:06:56.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa876000640 1 -- 172.21.15.143:0/887531666 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8700a5b40 con 0x7fa8780c6230 2024-01-15T18:06:56.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa876000640 1 -- 172.21.15.143:0/887531666 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8700ad440 con 0x7fa8780c6230 2024-01-15T18:06:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 -- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8780c6230 msgr2=0x7fa8780cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 --2- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8780c6230 0x7fa8780cc020 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa870009d90 tx=0x7fa87001c920 comp rx=0 tx=0).stop 2024-01-15T18:06:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 -- 172.21.15.143:0/887531666 shutdown_connections 2024-01-15T18:06:56.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 --2- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8780c6230 0x7fa8780cc020 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 --2- 172.21.15.143:0/887531666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780cdf20 0x7fa8780cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 -- 172.21.15.143:0/887531666 >> 172.21.15.143:0/887531666 conn(0x7fa8780d0ed0 msgr2=0x7fa8780d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 -- 172.21.15.143:0/887531666 shutdown_connections 2024-01-15T18:06:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.797+0000 7fa87e600640 1 -- 172.21.15.143:0/887531666 wait complete. 2024-01-15T18:06:56.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 Processor -- start 2024-01-15T18:06:56.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 -- start start 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780c6230 0x7fa878156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 0x7fa87814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 0x7fa87814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87814c020 con 0x7fa8780cdf20 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878157330 con 0x7fa87815b260 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa878157610 con 0x7fa8780c6230 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 0x7fa87814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa877e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 0x7fa87814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 0x7fa87814bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36638/0 (socket says 172.21.15.143:36638) 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa877e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 0x7fa87814f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:53618/0 (socket says 172.21.15.143:53618) 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa877400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780c6230 0x7fa878156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 -- 172.21.15.143:0/4010962818 learned_addr learned my addr 172.21.15.143:0/4010962818 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 -- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780c6230 msgr2=0x7fa878156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780c6230 0x7fa878156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 -- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 msgr2=0x7fa87814f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 0x7fa87814f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 -- 172.21.15.143:0/4010962818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009a70 con 0x7fa8780cdf20 2024-01-15T18:06:56.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa877e00640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 0x7fa87814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa876a00640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 0x7fa87814bae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e870 tx=0x7fa86000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860009510 con 0x7fa8780cdf20 2024-01-15T18:06:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000eec0 con 0x7fa8780cdf20 2024-01-15T18:06:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8781578f0 con 0x7fa8780cdf20 2024-01-15T18:06:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000cb40 con 0x7fa8780cdf20 2024-01-15T18:06:56.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.801+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8780c1b50 con 0x7fa8780cdf20 2024-01-15T18:06:56.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.805+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878067500 con 0x7fa8780cdf20 2024-01-15T18:06:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.805+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 8) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa86000cca0 con 0x7fa8780cdf20 2024-01-15T18:06:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.805+0000 7fa86f400640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa850066bc0 0x7fa850069080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.805+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa8600849d0 con 0x7fa8780cdf20 2024-01-15T18:06:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.805+0000 7fa877400640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa850066bc0 0x7fa850069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:56.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.805+0000 7fa877400640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa850066bc0 0x7fa850069080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa868007fd0 tx=0x7fa868007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:56.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.809+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8600518f0 con 0x7fa8780cdf20 2024-01-15T18:06:56.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:56.977+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fa878157780 con 0x7fa850066bc0 2024-01-15T18:06:57.029 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:06:57.021+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:06:57.112 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:06:57.511 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:57.514 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:59.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:06:59.540+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fa860051cc0 con 0x7fa8780cdf20 2024-01-15T18:07:00.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.048+0000 7fa86f400640 1 -- 172.21.15.143:0/4010962818 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fa878157780 con 0x7fa850066bc0 2024-01-15T18:07:00.057 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa850066bc0 msgr2=0x7fa850069080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa850066bc0 0x7fa850069080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa868007fd0 tx=0x7fa868007d60 comp rx=0 tx=0).stop 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 msgr2=0x7fa87814bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 0x7fa87814bae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e870 tx=0x7fa86000ec70 comp rx=0 tx=0).stop 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 shutdown_connections 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa850066bc0 0x7fa850069080 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa87815b260 0x7fa87814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8780cdf20 0x7fa87814bae0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 --2- 172.21.15.143:0/4010962818 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8780c6230 0x7fa878156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.052+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 >> 172.21.15.143:0/4010962818 conn(0x7fa8780d0ed0 msgr2=0x7fa878158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:00.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.056+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 shutdown_connections 2024-01-15T18:07:00.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.056+0000 7fa87e600640 1 -- 172.21.15.143:0/4010962818 wait complete. 2024-01-15T18:07:00.101 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:07:00.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:07:00.204 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:07:00.334 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:07:00.335 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:00.335 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:00.349 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:00.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.604+0000 7f5644a00640 1 Processor -- start 2024-01-15T18:07:00.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.604+0000 7f5644a00640 1 -- start start 2024-01-15T18:07:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5644a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f563c071140 0x7f563c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5644a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0ce810 con 0x7f563c15da90 2024-01-15T18:07:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5644a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0cb980 con 0x7f563c071520 2024-01-15T18:07:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5644a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0cc220 con 0x7f563c071140 2024-01-15T18:07:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5642000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f563c071140 0x7f563c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5641600640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f563c071520 0x7f563c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42310/0 (socket says 172.21.15.143:42310) 2024-01-15T18:07:00.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5641600640 1 -- 172.21.15.143:0/2904225197 learned_addr learned my addr 172.21.15.143:0/2904225197 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5642000640 1 -- 172.21.15.143:0/2904225197 >> v1:172.21.15.204:6789/0 conn(0x7f563c071520 legacy=0x7f563c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5640c00640 1 -- 172.21.15.143:0/2904225197 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699790308 0 0) 0x7f563c0cb980 con 0x7f563c071520 2024-01-15T18:07:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5642000640 1 -- 172.21.15.143:0/2904225197 >> v1:172.21.15.143:6789/0 conn(0x7f563c15da90 legacy=0x7f563c164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.608+0000 7f5642000640 1 -- 172.21.15.143:0/2904225197 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c0cbc60 con 0x7f563c071140 2024-01-15T18:07:00.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5642000640 1 --2- 172.21.15.143:0/2904225197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071140 0x7f563c0ce2d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f562c009d90 tx=0x7f562c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1229123a443388b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5640c00640 1 -- 172.21.15.143:0/2904225197 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0a5900 con 0x7f563c071140 2024-01-15T18:07:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5640c00640 1 -- 172.21.15.143:0/2904225197 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0a5a60 con 0x7f563c071140 2024-01-15T18:07:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5640c00640 1 -- 172.21.15.143:0/2904225197 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0a5df0 con 0x7f563c071140 2024-01-15T18:07:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 -- 172.21.15.143:0/2904225197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071140 msgr2=0x7f563c0ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 --2- 172.21.15.143:0/2904225197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071140 0x7f563c0ce2d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f562c009d90 tx=0x7f562c01c950 comp rx=0 tx=0).stop 2024-01-15T18:07:00.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 -- 172.21.15.143:0/2904225197 shutdown_connections 2024-01-15T18:07:00.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 --2- 172.21.15.143:0/2904225197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071140 0x7f563c0ce2d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 -- 172.21.15.143:0/2904225197 >> 172.21.15.143:0/2904225197 conn(0x7f563c0cfd20 msgr2=0x7f563c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:00.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 -- 172.21.15.143:0/2904225197 shutdown_connections 2024-01-15T18:07:00.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.612+0000 7f5644a00640 1 -- 172.21.15.143:0/2904225197 wait complete. 2024-01-15T18:07:00.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 Processor -- start 2024-01-15T18:07:00.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 -- start start 2024-01-15T18:07:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 0x7f563c1fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071520 0x7f563c1441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f563c15da90 0x7f563c0b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1faa50 con 0x7f563c071140 2024-01-15T18:07:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1fad30 con 0x7f563c071520 2024-01-15T18:07:00.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c1fb010 con 0x7f563c15da90 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 0x7f563c1fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 0x7f563c1fa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36084/0 (socket says 172.21.15.143:36084) 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 -- 172.21.15.143:0/2946431356 learned_addr learned my addr 172.21.15.143:0/2946431356 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f563c15da90 0x7f563c0b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5641600640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071520 0x7f563c1441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 -- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f563c15da90 msgr2=0x7f563c0b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f563c15da90 0x7f563c0b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 -- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071520 msgr2=0x7f563c1441f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071520 0x7f563c1441f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 -- 172.21.15.143:0/2946431356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c009a70 con 0x7f563c071140 2024-01-15T18:07:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5642000640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 0x7f563c1fa460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f562c019560 tx=0x7f562c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0ad850 con 0x7f563c071140 2024-01-15T18:07:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0ad9b0 con 0x7f563c071140 2024-01-15T18:07:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c1fb2f0 con 0x7f563c071140 2024-01-15T18:07:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0adda0 con 0x7f563c071140 2024-01-15T18:07:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.616+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c06e8d0 con 0x7f563c071140 2024-01-15T18:07:00.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.620+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563c071950 con 0x7f563c071140 2024-01-15T18:07:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.624+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f562c0b4050 con 0x7f563c071140 2024-01-15T18:07:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.624+0000 7f563a000640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5614066b30 0x7f5614068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.624+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f562c11fac0 con 0x7f563c071140 2024-01-15T18:07:00.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.624+0000 7f5641600640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5614066b30 0x7f5614068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.624+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f562c0ecda0 con 0x7f563c071140 2024-01-15T18:07:00.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.624+0000 7f5641600640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5614066b30 0x7f5614068ff0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f563c0b4e80 tx=0x7f5624007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:00.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f563c1fb150 con 0x7f563c071140 2024-01-15T18:07:00.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f563a000640 1 -- 172.21.15.143:0/2946431356 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f563c1fb150 con 0x7f563c071140 2024-01-15T18:07:00.788 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:00.788 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":37,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:00.044806+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.204:6835/4126176885":"2024-01-16T18:06:57.026013+0000","172.21.15.204:6833/4126176885":"2024-01-16T18:06:57.026013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5614066b30 msgr2=0x7f5614068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5614066b30 0x7f5614068ff0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f563c0b4e80 tx=0x7f5624007df0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 msgr2=0x7f563c1fa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 0x7f563c1fa460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f562c019560 tx=0x7f562c0196f0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 shutdown_connections 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5614066b30 0x7f5614068ff0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f563c15da90 0x7f563c0b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f563c071520 0x7f563c1441f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 --2- 172.21.15.143:0/2946431356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f563c071140 0x7f563c1fa460 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 >> 172.21.15.143:0/2946431356 conn(0x7f563c0cfd20 msgr2=0x7f563c0d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 shutdown_connections 2024-01-15T18:07:00.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.780+0000 7f5644a00640 1 -- 172.21.15.143:0/2946431356 wait complete. 2024-01-15T18:07:00.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:07:00.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:07:00.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 Processor -- start 2024-01-15T18:07:00.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 -- start start 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db41607c0 0x7f5db406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5db40cfb30 0x7f5db406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40cffd0 con 0x7f5db406c230 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db406c670 con 0x7f5db40cfb30 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.940+0000 7f5db9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40d1ce0 con 0x7f5db41607c0 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5db406c230 0x7f5db40cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44776/0 (socket says 172.21.15.143:44776) 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db3400640 1 -- 172.21.15.143:0/50749735 learned_addr learned my addr 172.21.15.143:0/50749735 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2000640 1 --2- 172.21.15.143:0/50749735 >> v2:172.21.15.204:3300/0 conn(0x7f5db40cfb30 0x7f5db406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2a00640 1 --2- 172.21.15.143:0/50749735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db41607c0 0x7f5db406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db1600640 1 -- 172.21.15.143:0/50749735 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939395148 0 0) 0x7f5db40cffd0 con 0x7f5db406c230 2024-01-15T18:07:00.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db1600640 1 -- 172.21.15.143:0/50749735 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d94003530 con 0x7f5db406c230 2024-01-15T18:07:00.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2a00640 1 -- 172.21.15.143:0/50749735 >> v2:172.21.15.204:3300/0 conn(0x7f5db40cfb30 msgr2=0x7f5db406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:00.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2a00640 1 --2- 172.21.15.143:0/50749735 >> v2:172.21.15.204:3300/0 conn(0x7f5db40cfb30 0x7f5db406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2a00640 1 -- 172.21.15.143:0/50749735 >> v1:172.21.15.143:6789/0 conn(0x7f5db406c230 legacy=0x7f5db40cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2a00640 1 -- 172.21.15.143:0/50749735 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40d02b0 con 0x7f5db41607c0 2024-01-15T18:07:00.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2000640 1 --2- 172.21.15.143:0/50749735 >> v2:172.21.15.204:3300/0 conn(0x7f5db40cfb30 0x7f5db406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:00.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.944+0000 7f5db2a00640 1 --2- 172.21.15.143:0/50749735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db41607c0 0x7f5db406b7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5da40097c0 tx=0x7f5da401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d76cd22af514987 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:00.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db1600640 1 -- 172.21.15.143:0/50749735 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da40a8970 con 0x7f5db41607c0 2024-01-15T18:07:00.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db1600640 1 -- 172.21.15.143:0/50749735 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40a8ad0 con 0x7f5db41607c0 2024-01-15T18:07:00.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db1600640 1 -- 172.21.15.143:0/50749735 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da40a8e60 con 0x7f5db41607c0 2024-01-15T18:07:00.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 -- 172.21.15.143:0/50749735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db41607c0 msgr2=0x7f5db406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 --2- 172.21.15.143:0/50749735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db41607c0 0x7f5db406b7b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5da40097c0 tx=0x7f5da401ec70 comp rx=0 tx=0).stop 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 -- 172.21.15.143:0/50749735 shutdown_connections 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 --2- 172.21.15.143:0/50749735 >> v2:172.21.15.204:3300/0 conn(0x7f5db40cfb30 0x7f5db406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 --2- 172.21.15.143:0/50749735 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db41607c0 0x7f5db406b7b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 -- 172.21.15.143:0/50749735 >> 172.21.15.143:0/50749735 conn(0x7f5db40d0e20 msgr2=0x7f5db40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 -- 172.21.15.143:0/50749735 shutdown_connections 2024-01-15T18:07:00.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 -- 172.21.15.143:0/50749735 wait complete. 2024-01-15T18:07:00.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 Processor -- start 2024-01-15T18:07:00.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.948+0000 7f5db9a00640 1 -- start start 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db406c230 0x7f5db4156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db40cfb30 0x7f5db40c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db41607c0 0x7f5db4152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41f2f40 con 0x7f5db41607c0 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c6120 con 0x7f5db406c230 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c6400 con 0x7f5db40cfb30 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db41607c0 0x7f5db4152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db40cfb30 0x7f5db40c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db41607c0 0x7f5db4152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36086/0 (socket says 172.21.15.143:36086) 2024-01-15T18:07:00.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db406c230 0x7f5db4156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db3400640 1 -- 172.21.15.143:0/140745338 learned_addr learned my addr 172.21.15.143:0/140745338 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 -- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db406c230 msgr2=0x7f5db4156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db406c230 0x7f5db4156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 -- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db41607c0 msgr2=0x7f5db4152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db41607c0 0x7f5db4152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 -- 172.21.15.143:0/140745338 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c009420 con 0x7f5db40cfb30 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db2000640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db40cfb30 0x7f5db40c58d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e9b0 tx=0x7f5d9c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00c9a0 con 0x7f5db40cfb30 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c00cb00 con 0x7f5db40cfb30 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da4009420 con 0x7f5db40cfb30 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c013470 con 0x7f5db40cfb30 2024-01-15T18:07:00.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db40c69c0 con 0x7f5db40cfb30 2024-01-15T18:07:00.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db406d4b0 con 0x7f5db40cfb30 2024-01-15T18:07:00.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.952+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f5d9c0135d0 con 0x7f5db40cfb30 2024-01-15T18:07:00.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.956+0000 7f5daaa00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d8c066b30 0x7f5d8c068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:00.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.956+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5d9c084790 con 0x7f5db40cfb30 2024-01-15T18:07:00.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.956+0000 7f5db2a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d8c066b30 0x7f5d8c068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:00.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.956+0000 7f5db2a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d8c066b30 0x7f5d8c068ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5da40b2000 tx=0x7f5da40aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:00.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:00.960+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5d9c051960 con 0x7f5db40cfb30 2024-01-15T18:07:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.112+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5db4067500 con 0x7f5db40cfb30 2024-01-15T18:07:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.112+0000 7f5daaa00640 1 -- 172.21.15.143:0/140745338 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d9c054f80 con 0x7f5db40cfb30 2024-01-15T18:07:01.121 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d8c066b30 msgr2=0x7f5d8c068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d8c066b30 0x7f5d8c068ff0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5da40b2000 tx=0x7f5da40aa070 comp rx=0 tx=0).stop 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db40cfb30 msgr2=0x7f5db40c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db40cfb30 0x7f5db40c58d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00e9b0 tx=0x7f5d9c00ee70 comp rx=0 tx=0).stop 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 shutdown_connections 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d8c066b30 0x7f5d8c068ff0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5db41607c0 0x7f5db4152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5db40cfb30 0x7f5db40c58d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 --2- 172.21.15.143:0/140745338 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5db406c230 0x7f5db4156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 >> 172.21.15.143:0/140745338 conn(0x7f5db40d0e20 msgr2=0x7f5db40cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 shutdown_connections 2024-01-15T18:07:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.116+0000 7f5db9a00640 1 -- 172.21.15.143:0/140745338 wait complete. 2024-01-15T18:07:01.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:07:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.272+0000 7fb8e2000640 1 Processor -- start 2024-01-15T18:07:01.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.272+0000 7fb8e2000640 1 -- start start 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8e2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8e2000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc0cda20 con 0x7fb8dc06a490 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8e2000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc0cdd00 con 0x7fb8dc1614e0 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8e2000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc0cdfe0 con 0x7fb8dc165ec0 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8dbe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb8dc06a490 0x7fb8dc06a8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44788/0 (socket says 172.21.15.143:44788) 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8dbe00640 1 -- 172.21.15.143:0/1062342748 learned_addr learned my addr 172.21.15.143:0/1062342748 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8daa00640 1 --2- 172.21.15.143:0/1062342748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681968484 0 0) 0x7fb8dc0cda20 con 0x7fb8dc06a490 2024-01-15T18:07:01.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8bc003530 con 0x7fb8dc06a490 2024-01-15T18:07:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922463191 0 0) 0x7fb8dc0cdd00 con 0x7fb8dc1614e0 2024-01-15T18:07:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8dc0cda20 con 0x7fb8dc1614e0 2024-01-15T18:07:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8daa00640 1 -- 172.21.15.143:0/1062342748 >> v1:172.21.15.204:6789/0 conn(0x7fb8dc1614e0 legacy=0x7fb8dc0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8daa00640 1 -- 172.21.15.143:0/1062342748 >> v1:172.21.15.143:6789/0 conn(0x7fb8dc06a490 legacy=0x7fb8dc06a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.276+0000 7fb8daa00640 1 -- 172.21.15.143:0/1062342748 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0ce2c0 con 0x7fb8dc165ec0 2024-01-15T18:07:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8daa00640 1 --2- 172.21.15.143:0/1062342748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc06d140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40097f0 tx=0x7fb8c401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d9fe35fe19c7bc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40ab620 con 0x7fb8dc165ec0 2024-01-15T18:07:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40ab780 con 0x7fb8dc165ec0 2024-01-15T18:07:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8da000640 1 -- 172.21.15.143:0/1062342748 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40abb10 con 0x7fb8dc165ec0 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 -- 172.21.15.143:0/1062342748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 msgr2=0x7fb8dc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 --2- 172.21.15.143:0/1062342748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc06d140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40097f0 tx=0x7fb8c401c400 comp rx=0 tx=0).stop 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 -- 172.21.15.143:0/1062342748 shutdown_connections 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 --2- 172.21.15.143:0/1062342748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc06d140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 -- 172.21.15.143:0/1062342748 >> 172.21.15.143:0/1062342748 conn(0x7fb8dc0cfd20 msgr2=0x7fb8dc0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 -- 172.21.15.143:0/1062342748 shutdown_connections 2024-01-15T18:07:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 -- 172.21.15.143:0/1062342748 wait complete. 2024-01-15T18:07:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 Processor -- start 2024-01-15T18:07:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.280+0000 7fb8e2000640 1 -- start start 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 0x7fb8dc0c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8dc1614e0 0x7fb8dc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc0c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc06dc00 con 0x7fb8dc06a490 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc06dee0 con 0x7fb8dc1614e0 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc06e1c0 con 0x7fb8dc165ec0 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 0x7fb8dc0c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 0x7fb8dc0c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36088/0 (socket says 172.21.15.143:36088) 2024-01-15T18:07:01.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 -- 172.21.15.143:0/2303044040 learned_addr learned my addr 172.21.15.143:0/2303044040 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8daa00640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8dc1614e0 0x7fb8dc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8dbe00640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc0c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 -- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 msgr2=0x7fb8dc0c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc0c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 -- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8dc1614e0 msgr2=0x7fb8dc0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8dc1614e0 0x7fb8dc0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 -- 172.21.15.143:0/2303044040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4009420 con 0x7fb8dc06a490 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8daa00640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8dc1614e0 0x7fb8dc0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8db400640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 0x7fb8dc0c3a60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c490 tx=0x7fb8cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0052d0 con 0x7fb8dc06a490 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc005430 con 0x7fb8dc06a490 2024-01-15T18:07:01.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc012870 con 0x7fb8dc06a490 2024-01-15T18:07:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc06e4a0 con 0x7fb8dc06a490 2024-01-15T18:07:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc06ea60 con 0x7fb8dc06a490 2024-01-15T18:07:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc067500 con 0x7fb8dc06a490 2024-01-15T18:07:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.284+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fb8cc0129d0 con 0x7fb8dc06a490 2024-01-15T18:07:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.288+0000 7fb8d3400640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb8b4066b30 0x7fb8b4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.288+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fb8cc083fc0 con 0x7fb8dc06a490 2024-01-15T18:07:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.288+0000 7fb8daa00640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb8b4066b30 0x7fb8b4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.288+0000 7fb8daa00640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb8b4066b30 0x7fb8b4068ff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40af000 tx=0x7fb8c40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.292+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb8cc0b5280 con 0x7fb8dc06a490 2024-01-15T18:07:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.448+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb8dc0d0bc0 con 0x7fb8dc06a490 2024-01-15T18:07:01.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8d3400640 1 -- 172.21.15.143:0/2303044040 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v37) v1 ==== 80+0+124 (secure 0 0 0) 0x7fb8cc051230 con 0x7fb8dc06a490 2024-01-15T18:07:01.459 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.204:6835/4126176885 2024-01-16T18:06:57.026013+0000 2024-01-15T18:07:01.459 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.204:6833/4126176885 2024-01-16T18:06:57.026013+0000 2024-01-15T18:07:01.459 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:07:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb8b4066b30 msgr2=0x7fb8b4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb8b4066b30 0x7fb8b4068ff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40af000 tx=0x7fb8c40a7000 comp rx=0 tx=0).stop 2024-01-15T18:07:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 msgr2=0x7fb8dc0c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 0x7fb8dc0c3a60 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00c490 tx=0x7fb8cc00c950 comp rx=0 tx=0).stop 2024-01-15T18:07:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 shutdown_connections 2024-01-15T18:07:01.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb8b4066b30 0x7fb8b4068ff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb8dc165ec0 0x7fb8dc0c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8dc1614e0 0x7fb8dc0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 --2- 172.21.15.143:0/2303044040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8dc06a490 0x7fb8dc0c3a60 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 >> 172.21.15.143:0/2303044040 conn(0x7fb8dc0cfd20 msgr2=0x7fb8dc0d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 shutdown_connections 2024-01-15T18:07:01.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.452+0000 7fb8e2000640 1 -- 172.21.15.143:0/2303044040 wait complete. 2024-01-15T18:07:01.481 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:07:01.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:01.481 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:01.503 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:07:01.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:01.503 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:07:01.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[37699]: fuse finished with error 0 and tester_r 0 2024-01-15T18:07:01.645 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:07:01.645 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:01.654 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:01.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.816+0000 7f709fe00640 1 Processor -- start 2024-01-15T18:07:01.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.816+0000 7f709fe00640 1 -- start start 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f70981607c0 0x7f709806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709fe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980cffd0 con 0x7f70980cbf50 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70980d02b0 con 0x7f70980cfb30 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70980d1ce0 con 0x7f70981607c0 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f70980cbf50 0x7f70980cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44800/0 (socket says 172.21.15.143:44800) 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709de00640 1 -- 172.21.15.143:0/2131985304 learned_addr learned my addr 172.21.15.143:0/2131985304 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:01.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709d400640 1 --2- 172.21.15.143:0/2131985304 >> v2:172.21.15.204:3300/0 conn(0x7f70981607c0 0x7f709806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f7097e00640 1 -- 172.21.15.143:0/2131985304 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993001795 0 0) 0x7f70980cffd0 con 0x7f70980cbf50 2024-01-15T18:07:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f7097e00640 1 -- 172.21.15.143:0/2131985304 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7078003530 con 0x7f70980cbf50 2024-01-15T18:07:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709d400640 1 -- 172.21.15.143:0/2131985304 >> v1:172.21.15.204:6789/0 conn(0x7f70980cfb30 legacy=0x7f709806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f7097e00640 1 -- 172.21.15.143:0/2131985304 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888137629 0 0) 0x7f70980d02b0 con 0x7f70980cfb30 2024-01-15T18:07:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709d400640 1 -- 172.21.15.143:0/2131985304 >> v1:172.21.15.143:6789/0 conn(0x7f70980cbf50 legacy=0x7f70980cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.820+0000 7f709d400640 1 -- 172.21.15.143:0/2131985304 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70980cb470 con 0x7f70981607c0 2024-01-15T18:07:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709d400640 1 --2- 172.21.15.143:0/2131985304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f709806b7b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70880097c0 tx=0x7f708801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01ed2158e6d72be server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f7097e00640 1 -- 172.21.15.143:0/2131985304 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880a8970 con 0x7f70981607c0 2024-01-15T18:07:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f7097e00640 1 -- 172.21.15.143:0/2131985304 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70880a8ad0 con 0x7f70981607c0 2024-01-15T18:07:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f7097e00640 1 -- 172.21.15.143:0/2131985304 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70880a8e40 con 0x7f70981607c0 2024-01-15T18:07:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 -- 172.21.15.143:0/2131985304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 msgr2=0x7f709806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 --2- 172.21.15.143:0/2131985304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f709806b7b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70880097c0 tx=0x7f708801ec70 comp rx=0 tx=0).stop 2024-01-15T18:07:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 -- 172.21.15.143:0/2131985304 shutdown_connections 2024-01-15T18:07:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 --2- 172.21.15.143:0/2131985304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f709806b7b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 -- 172.21.15.143:0/2131985304 >> 172.21.15.143:0/2131985304 conn(0x7f70980d0e20 msgr2=0x7f70980d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 -- 172.21.15.143:0/2131985304 shutdown_connections 2024-01-15T18:07:01.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 -- 172.21.15.143:0/2131985304 wait complete. 2024-01-15T18:07:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 Processor -- start 2024-01-15T18:07:01.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.824+0000 7f709fe00640 1 -- start start 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70980cbf50 0x7f70981f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 0x7f70980c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f70980c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980ca3c0 con 0x7f70980cfb30 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70980c8670 con 0x7f70981607c0 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70981f6f60 con 0x7f70980cbf50 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 0x7f70980c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 0x7f70980c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36100/0 (socket says 172.21.15.143:36100) 2024-01-15T18:07:01.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 -- 172.21.15.143:0/2217889373 learned_addr learned my addr 172.21.15.143:0/2217889373 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709de00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f70980c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709d400640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70980cbf50 0x7f70981f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 -- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70980cbf50 msgr2=0x7f70981f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70980cbf50 0x7f70981f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 -- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 msgr2=0x7f70980c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f70980c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 -- 172.21.15.143:0/2217889373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7088009420 con 0x7f70980cfb30 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709ca00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 0x7f70980c4880 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f708000c4d0 tx=0x7f708000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080005480 con 0x7f70980cfb30 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080003600 con 0x7f70980cfb30 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70981f6c80 con 0x7f70980cfb30 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800039b0 con 0x7f70980cfb30 2024-01-15T18:07:01.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70980c8300 con 0x7f70980cfb30 2024-01-15T18:07:01.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709806d4b0 con 0x7f70980cfb30 2024-01-15T18:07:01.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.828+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f7080003b10 con 0x7f70980cfb30 2024-01-15T18:07:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.832+0000 7f7095600640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7070066b30 0x7f7070068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.832+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f7080083ae0 con 0x7f70980cfb30 2024-01-15T18:07:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.832+0000 7f709d400640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7070066b30 0x7f7070068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.832+0000 7f709d400640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7070066b30 0x7f7070068ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70880b2000 tx=0x7f7088008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.836+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7080050e00 con 0x7f70980cfb30 2024-01-15T18:07:01.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.984+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7098067500 con 0x7f70980cfb30 2024-01-15T18:07:01.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f7095600640 1 -- 172.21.15.143:0/2217889373 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f7080017070 con 0x7f70980cfb30 2024-01-15T18:07:01.995 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:01.995 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":37,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:00.044806+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.204:6835/4126176885":"2024-01-16T18:06:57.026013+0000","172.21.15.204:6833/4126176885":"2024-01-16T18:06:57.026013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:01.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7070066b30 msgr2=0x7f7070068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7070066b30 0x7f7070068ff0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f70880b2000 tx=0x7f7088008040 comp rx=0 tx=0).stop 2024-01-15T18:07:01.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 msgr2=0x7f70980c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 0x7f70980c4880 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f708000c4d0 tx=0x7f708000c990 comp rx=0 tx=0).stop 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 shutdown_connections 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7070066b30 0x7f7070068ff0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f70981607c0 0x7f70980c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70980cfb30 0x7f70980c4880 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 --2- 172.21.15.143:0/2217889373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f70980cbf50 0x7f70981f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 >> 172.21.15.143:0/2217889373 conn(0x7f70980d0e20 msgr2=0x7f7098153320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 shutdown_connections 2024-01-15T18:07:01.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:01.988+0000 7f709fe00640 1 -- 172.21.15.143:0/2217889373 wait complete. 2024-01-15T18:07:02.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:07:02.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:07:02.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 Processor -- start 2024-01-15T18:07:02.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 -- start start 2024-01-15T18:07:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71441607c0 0x7f714406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71440cfb30 0x7f714406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f714406c230 0x7f71440cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714406c670 con 0x7f71441607c0 2024-01-15T18:07:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71440d1ce0 con 0x7f714406c230 2024-01-15T18:07:02.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7149a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71440cffd0 con 0x7f71440cfb30 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71441607c0 0x7f714406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71441607c0 0x7f714406b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36102/0 (socket says 172.21.15.143:36102) 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 -- 172.21.15.143:0/3476257292 learned_addr learned my addr 172.21.15.143:0/3476257292 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142000640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71440cfb30 0x7f714406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7143400640 1 --2- 172.21.15.143:0/3476257292 >> v2:172.21.15.204:3300/0 conn(0x7f714406c230 0x7f71440cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 -- 172.21.15.143:0/3476257292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71440cfb30 msgr2=0x7f714406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71440cfb30 0x7f714406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 -- 172.21.15.143:0/3476257292 >> v2:172.21.15.204:3300/0 conn(0x7f714406c230 msgr2=0x7f71440cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 --2- 172.21.15.143:0/3476257292 >> v2:172.21.15.204:3300/0 conn(0x7f714406c230 0x7f71440cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142a00640 1 -- 172.21.15.143:0/3476257292 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440d02b0 con 0x7f71441607c0 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7142000640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71440cfb30 0x7f714406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:02.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.152+0000 7f7143400640 1 --2- 172.21.15.143:0/3476257292 >> v2:172.21.15.204:3300/0 conn(0x7f714406c230 0x7f71440cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:02.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7142a00640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71441607c0 0x7f714406b7b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f71440d0590 tx=0x7f713401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895d39626952566b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:02.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7141600640 1 -- 172.21.15.143:0/3476257292 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340a89b0 con 0x7f71441607c0 2024-01-15T18:07:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7141600640 1 -- 172.21.15.143:0/3476257292 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340a8b10 con 0x7f71441607c0 2024-01-15T18:07:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7141600640 1 -- 172.21.15.143:0/3476257292 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340a8ea0 con 0x7f71441607c0 2024-01-15T18:07:02.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 -- 172.21.15.143:0/3476257292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71441607c0 msgr2=0x7f714406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71441607c0 0x7f714406b7b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f71440d0590 tx=0x7f713401ecf0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 -- 172.21.15.143:0/3476257292 shutdown_connections 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 --2- 172.21.15.143:0/3476257292 >> v2:172.21.15.204:3300/0 conn(0x7f714406c230 0x7f71440cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71440cfb30 0x7f714406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 --2- 172.21.15.143:0/3476257292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71441607c0 0x7f714406b7b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 -- 172.21.15.143:0/3476257292 >> 172.21.15.143:0/3476257292 conn(0x7f71440d0e20 msgr2=0x7f71440d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 -- 172.21.15.143:0/3476257292 shutdown_connections 2024-01-15T18:07:02.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.156+0000 7f7149a00640 1 -- 172.21.15.143:0/3476257292 wait complete. 2024-01-15T18:07:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 Processor -- start 2024-01-15T18:07:02.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- start start 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714406c230 0x7f71440c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 0x7f714415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71441607c0 0x7f71441f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7144155a00 con 0x7f71440cfb30 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71441550f0 con 0x7f71441607c0 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714415ad30 con 0x7f714406c230 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 0x7f714415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 0x7f714415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36118/0 (socket says 172.21.15.143:36118) 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 -- 172.21.15.143:0/2382957656 learned_addr learned my addr 172.21.15.143:0/2382957656 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714406c230 0x7f71440c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7143400640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71441607c0 0x7f71441f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 -- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714406c230 msgr2=0x7f71440c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714406c230 0x7f71440c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 -- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71441607c0 msgr2=0x7f71441f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71441607c0 0x7f71441f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 -- 172.21.15.143:0/2382957656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340094d0 con 0x7f71440cfb30 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7142000640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 0x7f714415a270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f712c00e810 tx=0x7f712c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00c8d0 con 0x7f71440cfb30 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c00ca30 con 0x7f71440cfb30 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71441553b0 con 0x7f71440cfb30 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00cdc0 con 0x7f71440cfb30 2024-01-15T18:07:02.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71440c7030 con 0x7f71440cfb30 2024-01-15T18:07:02.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.160+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714406d4b0 con 0x7f71440cfb30 2024-01-15T18:07:02.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.164+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f712c013410 con 0x7f71440cfb30 2024-01-15T18:07:02.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.164+0000 7f713aa00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f711c066b60 0x7f711c069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.164+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f712c084320 con 0x7f71440cfb30 2024-01-15T18:07:02.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.164+0000 7f7142a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f711c066b60 0x7f711c069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.164+0000 7f7142a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f711c066b60 0x7f711c069020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71340b2000 tx=0x7f7134009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:02.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.168+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f712c051510 con 0x7f71440cfb30 2024-01-15T18:07:02.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.316+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7144155bb0 con 0x7f71440cfb30 2024-01-15T18:07:02.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.316+0000 7f713aa00640 1 -- 172.21.15.143:0/2382957656 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f712c009480 con 0x7f71440cfb30 2024-01-15T18:07:02.326 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:07:02.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f711c066b60 msgr2=0x7f711c069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f711c066b60 0x7f711c069020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f71340b2000 tx=0x7f7134009040 comp rx=0 tx=0).stop 2024-01-15T18:07:02.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 msgr2=0x7f714415a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 0x7f714415a270 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f712c00e810 tx=0x7f712c00ecd0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 shutdown_connections 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f711c066b60 0x7f711c069020 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71441607c0 0x7f71441f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71440cfb30 0x7f714415a270 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 --2- 172.21.15.143:0/2382957656 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714406c230 0x7f71440c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 >> 172.21.15.143:0/2382957656 conn(0x7f71440d0e20 msgr2=0x7f71440cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 shutdown_connections 2024-01-15T18:07:02.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.320+0000 7f7149a00640 1 -- 172.21.15.143:0/2382957656 wait complete. 2024-01-15T18:07:02.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:07:02.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.476+0000 7f4cfaa00640 1 Processor -- start 2024-01-15T18:07:02.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.476+0000 7f4cfaa00640 1 -- start start 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cfaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4cf41607c0 0x7f4cf406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cfaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4cf40cbf50 0x7f4cf40cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cfaa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf40cffd0 con 0x7f4cf40cfb30 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cfaa00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf40d1ce0 con 0x7f4cf40cbf50 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cfaa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf40d02b0 con 0x7f4cf41607c0 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4cf40cbf50 0x7f4cf40cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4cf40cbf50 0x7f4cf40cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36122/0 (socket says 172.21.15.143:36122) 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 -- 172.21.15.143:0/1191296684 learned_addr learned my addr 172.21.15.143:0/1191296684 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:02.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf3e00640 1 --2- 172.21.15.143:0/1191296684 >> v2:172.21.15.204:3300/0 conn(0x7f4cf41607c0 0x7f4cf406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 -- 172.21.15.143:0/1191296684 >> v2:172.21.15.204:3300/0 conn(0x7f4cf41607c0 msgr2=0x7f4cf406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:02.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 --2- 172.21.15.143:0/1191296684 >> v2:172.21.15.204:3300/0 conn(0x7f4cf41607c0 0x7f4cf406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 -- 172.21.15.143:0/1191296684 >> v1:172.21.15.204:6789/0 conn(0x7f4cf40cfb30 legacy=0x7f4cf406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.480+0000 7f4cf8a00640 1 -- 172.21.15.143:0/1191296684 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40cb470 con 0x7f4cf40cbf50 2024-01-15T18:07:02.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cf8a00640 1 --2- 172.21.15.143:0/1191296684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cbf50 0x7f4cf40cc3c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4071a00 tx=0x7f4cec01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e9a0e720e10fc1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:02.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cf2a00640 1 -- 172.21.15.143:0/1191296684 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0af400 con 0x7f4cf40cbf50 2024-01-15T18:07:02.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cf2a00640 1 -- 172.21.15.143:0/1191296684 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec0ab050 con 0x7f4cf40cbf50 2024-01-15T18:07:02.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cf2a00640 1 -- 172.21.15.143:0/1191296684 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec0af780 con 0x7f4cf40cbf50 2024-01-15T18:07:02.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 -- 172.21.15.143:0/1191296684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cbf50 msgr2=0x7f4cf40cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/1191296684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cbf50 0x7f4cf40cc3c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4071a00 tx=0x7f4cec01f1c0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 -- 172.21.15.143:0/1191296684 shutdown_connections 2024-01-15T18:07:02.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/1191296684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cbf50 0x7f4cf40cc3c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/1191296684 >> v2:172.21.15.204:3300/0 conn(0x7f4cf41607c0 0x7f4cf406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 -- 172.21.15.143:0/1191296684 >> 172.21.15.143:0/1191296684 conn(0x7f4cf40d0e20 msgr2=0x7f4cf40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:02.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 -- 172.21.15.143:0/1191296684 shutdown_connections 2024-01-15T18:07:02.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 -- 172.21.15.143:0/1191296684 wait complete. 2024-01-15T18:07:02.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 Processor -- start 2024-01-15T18:07:02.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.484+0000 7f4cfaa00640 1 -- start start 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cf40cbf50 0x7f4cf4151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 0x7f4cf415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf41607c0 0x7f4cf40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4151980 con 0x7f4cf40cfb30 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40c5af0 con 0x7f4cf41607c0 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40c5dd0 con 0x7f4cf40cbf50 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 0x7f4cf415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 0x7f4cf415a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36136/0 (socket says 172.21.15.143:36136) 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 -- 172.21.15.143:0/2598589986 learned_addr learned my addr 172.21.15.143:0/2598589986 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf8a00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf41607c0 0x7f4cf40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3e00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cf40cbf50 0x7f4cf4151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 -- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cf40cbf50 msgr2=0x7f4cf4151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cf40cbf50 0x7f4cf4151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 -- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf41607c0 msgr2=0x7f4cf40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf41607c0 0x7f4cf40ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 -- 172.21.15.143:0/2598589986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec009b30 con 0x7f4cf40cfb30 2024-01-15T18:07:02.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cf3400640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 0x7f4cf415a810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c500 tx=0x7f4cdc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc005220 con 0x7f4cf40cfb30 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cdc005380 con 0x7f4cf40cfb30 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc007270 con 0x7f4cf40cfb30 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf40c60b0 con 0x7f4cf40cfb30 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf40c6670 con 0x7f4cf40cfb30 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.488+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf406d4b0 con 0x7f4cf40cfb30 2024-01-15T18:07:02.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.492+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f4cdc0073d0 con 0x7f4cf40cfb30 2024-01-15T18:07:02.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.492+0000 7f4cebe00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ccc066b30 0x7f4ccc068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:02.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.492+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f4cdc083dd0 con 0x7f4cf40cfb30 2024-01-15T18:07:02.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.492+0000 7f4cf3e00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ccc066b30 0x7f4ccc068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:02.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.492+0000 7f4cf3e00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ccc066b30 0x7f4ccc068ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009ac0 tx=0x7f4ce4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:02.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.496+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4cdc088070 con 0x7f4cf40cfb30 2024-01-15T18:07:02.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.660+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4cf4067500 con 0x7f4cf40cfb30 2024-01-15T18:07:02.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.660+0000 7f4cebe00640 1 -- 172.21.15.143:0/2598589986 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v37) v1 ==== 80+0+124 (secure 0 0 0) 0x7f4cdc050fb0 con 0x7f4cf40cfb30 2024-01-15T18:07:02.669 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.204:6835/4126176885 2024-01-16T18:06:57.026013+0000 2024-01-15T18:07:02.669 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.204:6833/4126176885 2024-01-16T18:06:57.026013+0000 2024-01-15T18:07:02.669 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:07:02.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ccc066b30 msgr2=0x7f4ccc068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ccc066b30 0x7f4ccc068ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009ac0 tx=0x7f4ce4009190 comp rx=0 tx=0).stop 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 msgr2=0x7f4cf415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 0x7f4cf415a810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c500 tx=0x7f4cdc00c9c0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 shutdown_connections 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ccc066b30 0x7f4ccc068ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4cf41607c0 0x7f4cf40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4cf40cfb30 0x7f4cf415a810 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 --2- 172.21.15.143:0/2598589986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4cf40cbf50 0x7f4cf4151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 >> 172.21.15.143:0/2598589986 conn(0x7f4cf40d0e20 msgr2=0x7f4cf415d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 shutdown_connections 2024-01-15T18:07:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:02.664+0000 7f4cfaa00640 1 -- 172.21.15.143:0/2598589986 wait complete. 2024-01-15T18:07:02.691 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:07:02.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:02.691 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:02.711 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33445]: fuse finished with error 0 and tester_r 0 2024-01-15T18:07:02.713 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:07:02.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:02.713 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:07:02.883 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:02.883 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:02.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:07:02.892 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:07:02.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:02.892 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:07:02.941 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:07:02.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:07:02.942 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:07:02.942 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:02.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:07:02.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:07:02.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:02.952 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:07:03.003 INFO:teuthology.orchestra.run.smithi204.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:07:03.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:07:03.005 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:07:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 Processor -- start 2024-01-15T18:07:03.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 -- start start 2024-01-15T18:07:03.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5ec0b24b0 0x7fd5ec0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5ec0b3bf0 0x7fd5ec0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ec070bf0 con 0x7fd5ec0ce690 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec15dba0 con 0x7fd5ec0b3bf0 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5f1e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5ec0b4920 con 0x7fd5ec0b24b0 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5ec0b3bf0 0x7fd5ec0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5ec0b3bf0 0x7fd5ec0b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36142/0 (socket says 172.21.15.143:36142) 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 -- 172.21.15.143:0/4289579481 learned_addr learned my addr 172.21.15.143:0/4289579481 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5ea000640 1 --2- 172.21.15.143:0/4289579481 >> v2:172.21.15.204:3300/0 conn(0x7fd5ec0b24b0 0x7fd5ec0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 -- 172.21.15.143:0/4289579481 >> v2:172.21.15.204:3300/0 conn(0x7fd5ec0b24b0 msgr2=0x7fd5ec0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 --2- 172.21.15.143:0/4289579481 >> v2:172.21.15.204:3300/0 conn(0x7fd5ec0b24b0 0x7fd5ec0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 -- 172.21.15.143:0/4289579481 >> v1:172.21.15.143:6789/0 conn(0x7fd5ec0ce690 legacy=0x7fd5ec0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.136+0000 7fd5eb400640 1 -- 172.21.15.143:0/4289579481 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0b4c00 con 0x7fd5ec0b3bf0 2024-01-15T18:07:03.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5eb400640 1 --2- 172.21.15.143:0/4289579481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0b3bf0 0x7fd5ec0b40a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec06f940 tx=0x7fd5e401d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ba1c7a92625180 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5e9600640 1 -- 172.21.15.143:0/4289579481 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40a5880 con 0x7fd5ec0b3bf0 2024-01-15T18:07:03.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5e9600640 1 -- 172.21.15.143:0/4289579481 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40a59e0 con 0x7fd5ec0b3bf0 2024-01-15T18:07:03.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5e9600640 1 -- 172.21.15.143:0/4289579481 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40a5d70 con 0x7fd5ec0b3bf0 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 -- 172.21.15.143:0/4289579481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0b3bf0 msgr2=0x7fd5ec0b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/4289579481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0b3bf0 0x7fd5ec0b40a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec06f940 tx=0x7fd5e401d400 comp rx=0 tx=0).stop 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 -- 172.21.15.143:0/4289579481 shutdown_connections 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/4289579481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0b3bf0 0x7fd5ec0b40a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/4289579481 >> v2:172.21.15.204:3300/0 conn(0x7fd5ec0b24b0 0x7fd5ec0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 -- 172.21.15.143:0/4289579481 >> 172.21.15.143:0/4289579481 conn(0x7fd5ec06c3f0 msgr2=0x7fd5ec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 -- 172.21.15.143:0/4289579481 shutdown_connections 2024-01-15T18:07:03.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.140+0000 7fd5f1e00640 1 -- 172.21.15.143:0/4289579481 wait complete. 2024-01-15T18:07:03.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 Processor -- start 2024-01-15T18:07:03.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- start start 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec0b24b0 0x7fd5ec1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ec0b3bf0 0x7fd5ec13f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 0x7fd5ec12fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec140cc0 con 0x7fd5ec0ce690 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec1632b0 con 0x7fd5ec0b24b0 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ec159de0 con 0x7fd5ec0b3bf0 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 0x7fd5ec12fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 0x7fd5ec12fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36150/0 (socket says 172.21.15.143:36150) 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 -- 172.21.15.143:0/1192696204 learned_addr learned my addr 172.21.15.143:0/1192696204 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:03.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5ea000640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ec0b3bf0 0x7fd5ec13f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eaa00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec0b24b0 0x7fd5ec1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 -- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ec0b3bf0 msgr2=0x7fd5ec13f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ec0b3bf0 0x7fd5ec13f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 -- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec0b24b0 msgr2=0x7fd5ec1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec0b24b0 0x7fd5ec1598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 -- 172.21.15.143:0/1192696204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4009b30 con 0x7fd5ec0ce690 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5ea000640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ec0b3bf0 0x7fd5ec13f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eaa00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec0b24b0 0x7fd5ec1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5eb400640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 0x7fd5ec12fe10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40af040 tx=0x7fd5e4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40b8670 con 0x7fd5ec0ce690 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e40b87d0 con 0x7fd5ec0ce690 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ec13fc20 con 0x7fd5ec0ce690 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e40b8b40 con 0x7fd5ec0ce690 2024-01-15T18:07:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ec1401e0 con 0x7fd5ec0ce690 2024-01-15T18:07:03.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.144+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ec067500 con 0x7fd5ec0ce690 2024-01-15T18:07:03.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.148+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fd5e40b8ca0 con 0x7fd5ec0ce690 2024-01-15T18:07:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.148+0000 7fd5e3400640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5c4066b80 0x7fd5c4069040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.148+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd5e411fca0 con 0x7fd5ec0ce690 2024-01-15T18:07:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.148+0000 7fd5eaa00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5c4066b80 0x7fd5c4069040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.148+0000 7fd5eaa00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5c4066b80 0x7fd5c4069040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc000f70 tx=0x7fd5dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.152+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5e40ecf60 con 0x7fd5ec0ce690 2024-01-15T18:07:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.332+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5ec06e260 con 0x7fd5ec0ce690 2024-01-15T18:07:03.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.332+0000 7fd5e3400640 1 -- 172.21.15.143:0/1192696204 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fd5e40f0580 con 0x7fd5ec0ce690 2024-01-15T18:07:03.340 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:03.341 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3272757327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3272757327},{"type":"v1","addr":"172.21.15.204:6834","nonce":3272757327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/518983608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":518983608},{"type":"v1","addr":"172.21.15.143:6836","nonce":518983608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/434311355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":434311355},{"type":"v1","addr":"172.21.15.143:6837","nonce":434311355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2336035276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2336035276},{"type":"v1","addr":"172.21.15.204:6835","nonce":2336035276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-01-15T18:07:03.341 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 20 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5c4066b80 msgr2=0x7fd5c4069040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5c4066b80 0x7fd5c4069040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc000f70 tx=0x7fd5dc009040 comp rx=0 tx=0).stop 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 msgr2=0x7fd5ec12fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 0x7fd5ec12fe10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40af040 tx=0x7fd5e4019740 comp rx=0 tx=0).stop 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 shutdown_connections 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5c4066b80 0x7fd5c4069040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5ec0ce690 0x7fd5ec12fe10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5ec0b3bf0 0x7fd5ec13f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 --2- 172.21.15.143:0/1192696204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5ec0b24b0 0x7fd5ec1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 >> 172.21.15.143:0/1192696204 conn(0x7fd5ec06c3f0 msgr2=0x7fd5ec15a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 shutdown_connections 2024-01-15T18:07:03.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.336+0000 7fd5f1e00640 1 -- 172.21.15.143:0/1192696204 wait complete. 2024-01-15T18:07:03.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:07:03.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 Processor -- start 2024-01-15T18:07:03.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 -- start start 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8bf40ce520 0x7f8bf4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8bf40720f0 0x7f8bf415a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf4072580 con 0x7f8bf40720f0 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf40ceb50 con 0x7f8bf40ce520 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.492+0000 7f8bf9000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf415c110 con 0x7f8bf4143780 2024-01-15T18:07:03.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8bf40720f0 0x7f8bf415a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8bf40720f0 0x7f8bf415a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36162/0 (socket says 172.21.15.143:36162) 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 -- 172.21.15.143:0/1675642543 learned_addr learned my addr 172.21.15.143:0/1675642543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf1400640 1 --2- 172.21.15.143:0/1675642543 >> v2:172.21.15.204:3300/0 conn(0x7f8bf40ce520 0x7f8bf4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf1e00640 1 --2- 172.21.15.143:0/1675642543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 -- 172.21.15.143:0/1675642543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 msgr2=0x7f8bf40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 --2- 172.21.15.143:0/1675642543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 -- 172.21.15.143:0/1675642543 >> v2:172.21.15.204:3300/0 conn(0x7f8bf40ce520 msgr2=0x7f8bf4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 --2- 172.21.15.143:0/1675642543 >> v2:172.21.15.204:3300/0 conn(0x7f8bf40ce520 0x7f8bf4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 -- 172.21.15.143:0/1675642543 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf415b7e0 con 0x7f8bf40720f0 2024-01-15T18:07:03.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf2800640 1 --2- 172.21.15.143:0/1675642543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf415a450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bf406bee0 tx=0x7f8bec01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84c1c65988be641 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf0a00640 1 -- 172.21.15.143:0/1675642543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00b2d0 con 0x7f8bf40720f0 2024-01-15T18:07:03.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf0a00640 1 -- 172.21.15.143:0/1675642543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec00b430 con 0x7f8bf40720f0 2024-01-15T18:07:03.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.496+0000 7f8bf0a00640 1 -- 172.21.15.143:0/1675642543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0a7b90 con 0x7f8bf40720f0 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- 172.21.15.143:0/1675642543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 msgr2=0x7f8bf415a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- 172.21.15.143:0/1675642543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf415a450 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bf406bee0 tx=0x7f8bec01ebd0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- 172.21.15.143:0/1675642543 shutdown_connections 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- 172.21.15.143:0/1675642543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf415a450 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- 172.21.15.143:0/1675642543 >> v2:172.21.15.204:3300/0 conn(0x7f8bf40ce520 0x7f8bf4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- 172.21.15.143:0/1675642543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf40cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- 172.21.15.143:0/1675642543 >> 172.21.15.143:0/1675642543 conn(0x7f8bf406c3f0 msgr2=0x7f8bf406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:03.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- 172.21.15.143:0/1675642543 shutdown_connections 2024-01-15T18:07:03.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- 172.21.15.143:0/1675642543 wait complete. 2024-01-15T18:07:03.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 Processor -- start 2024-01-15T18:07:03.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- start start 2024-01-15T18:07:03.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf41418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bf40ce520 0x7f8bf4141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf413c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4131a40 con 0x7f8bf40720f0 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41580a0 con 0x7f8bf40ce520 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.500+0000 7f8bf9000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41310a0 con 0x7f8bf4143780 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf41418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf41418c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36164/0 (socket says 172.21.15.143:36164) 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 -- 172.21.15.143:0/2835545683 learned_addr learned my addr 172.21.15.143:0/2835545683 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf2800640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf413c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1400640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bf40ce520 0x7f8bf4141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 -- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 msgr2=0x7f8bf413c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf413c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 -- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bf40ce520 msgr2=0x7f8bf4141e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bf40ce520 0x7f8bf4141e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 -- 172.21.15.143:0/2835545683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec00bd40 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf1e00640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf41418c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e8a0 tx=0x7f8be400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4009510 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be400c940 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be400ccb0 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf4131380 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf4131e60 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf4131cb0 con 0x7f8bf40720f0 2024-01-15T18:07:03.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.504+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f8be4013470 con 0x7f8bf40720f0 2024-01-15T18:07:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.508+0000 7f8bea000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8bcc066b60 0x7f8bcc069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.508+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8be4084580 con 0x7f8bf40720f0 2024-01-15T18:07:03.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.508+0000 7f8bf1400640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8bcc066b60 0x7f8bcc069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.508+0000 7f8bf1400640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8bcc066b60 0x7f8bcc069020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bf415a0f0 tx=0x7f8bdc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.512+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8be40517f0 con 0x7f8bf40720f0 2024-01-15T18:07:03.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.664+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f8bf4067500 con 0x7f8bf40720f0 2024-01-15T18:07:03.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.668+0000 7f8bea000640 1 -- 172.21.15.143:0/2835545683 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v12) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8be4054e10 con 0x7f8bf40720f0 2024-01-15T18:07:03.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.668+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8bcc066b60 msgr2=0x7f8bcc069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.668+0000 7f8bf9000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8bcc066b60 0x7f8bcc069020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bf415a0f0 tx=0x7f8bdc009190 comp rx=0 tx=0).stop 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.668+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 msgr2=0x7f8bf41418c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.668+0000 7f8bf9000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf41418c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8be400e8a0 tx=0x7f8be400eca0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 shutdown_connections 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8bcc066b60 0x7f8bcc069020 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bf4143780 0x7f8bf413c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bf40ce520 0x7f8bf4141e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 --2- 172.21.15.143:0/2835545683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bf40720f0 0x7f8bf41418c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 >> 172.21.15.143:0/2835545683 conn(0x7f8bf406c3f0 msgr2=0x7f8bf40734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 shutdown_connections 2024-01-15T18:07:03.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.672+0000 7f8bf9000640 1 -- 172.21.15.143:0/2835545683 wait complete. 2024-01-15T18:07:03.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:07:03.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.828+0000 7fdbe3800640 1 Processor -- start 2024-01-15T18:07:03.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.828+0000 7fdbe3800640 1 -- start start 2024-01-15T18:07:03.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe3800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdbdc0cf410 0x7fdbdc0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc0ce7c0 con 0x7fdbdc0c5600 2024-01-15T18:07:03.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe3800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc0d1d90 con 0x7fdbdc0d1950 2024-01-15T18:07:03.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe3800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc0cef50 con 0x7fdbdc0cf410 2024-01-15T18:07:03.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe1800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdbdc0c5600 0x7fdbdc0c5a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44818/0 (socket says 172.21.15.143:44818) 2024-01-15T18:07:03.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe1800640 1 -- 172.21.15.143:0/4193663044 learned_addr learned my addr 172.21.15.143:0/4193663044 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/4193663044 >> v2:172.21.15.204:3300/0 conn(0x7fdbdc0cf410 0x7fdbdc0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163472432 0 0) 0x7fdbd40014a0 con 0x7fdbdc0c5600 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbbc003530 con 0x7fdbdc0c5600 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042803771 0 0) 0x7fdbbc003530 con 0x7fdbdc0c5600 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 >> v2:172.21.15.204:3300/0 conn(0x7fdbdc0cf410 msgr2=0x7fdbdc0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 --2- 172.21.15.143:0/4193663044 >> v2:172.21.15.204:3300/0 conn(0x7fdbdc0cf410 0x7fdbdc0cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 >> v1:172.21.15.204:6789/0 conn(0x7fdbdc0d1950 legacy=0x7fdbdc0cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0ce4e0 con 0x7fdbdc0c5600 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fdbd4001b90 con 0x7fdbdc0c5600 2024-01-15T18:07:03.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 >> v1:172.21.15.143:6789/0 conn(0x7fdbdc0c5600 legacy=0x7fdbdc0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbbc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbbc0047d0 0x7fdbbc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbbc005190 0x7fdbbc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0cef50 con 0x7fdbbc0039d0 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc00c090 con 0x7fdbbc005190 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc00c250 con 0x7fdbbc0047d0 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbbc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe1800640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbbc005190 0x7fdbbc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbdbe00640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbbc0047d0 0x7fdbbc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 -- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbbc0047d0 msgr2=0x7fdbbc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbbc0047d0 0x7fdbbc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 -- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbbc005190 msgr2=0x7fdbbc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbbc005190 0x7fdbbc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe0e00640 1 -- 172.21.15.143:0/4193663044 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc003530 con 0x7fdbbc0039d0 2024-01-15T18:07:03.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.832+0000 7fdbe1800640 1 -- 172.21.15.143:0/4193663044 reap_dead start 2024-01-15T18:07:03.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbbc003df0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ae0 tx=0x7fdbcc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757fc24157576c34 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0a6430 con 0x7fdbbc0039d0 2024-01-15T18:07:03.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0a6590 con 0x7fdbbc0039d0 2024-01-15T18:07:03.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbdb400640 1 -- 172.21.15.143:0/4193663044 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0a6940 con 0x7fdbbc0039d0 2024-01-15T18:07:03.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 -- 172.21.15.143:0/4193663044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 msgr2=0x7fdbbc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbbc003df0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ae0 tx=0x7fdbcc01ca50 comp rx=0 tx=0).stop 2024-01-15T18:07:03.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 -- 172.21.15.143:0/4193663044 shutdown_connections 2024-01-15T18:07:03.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 --2- 172.21.15.143:0/4193663044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbbc003df0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 -- 172.21.15.143:0/4193663044 >> 172.21.15.143:0/4193663044 conn(0x7fdbdc0d0e30 msgr2=0x7fdbdc0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:03.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 -- 172.21.15.143:0/4193663044 shutdown_connections 2024-01-15T18:07:03.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.836+0000 7fdbe3800640 1 -- 172.21.15.143:0/4193663044 wait complete. 2024-01-15T18:07:03.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 Processor -- start 2024-01-15T18:07:03.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- start start 2024-01-15T18:07:03.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbdc0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc15c040 0x7fdbdc14af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc06ec00 0x7fdbdc06f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc149350 con 0x7fdbbc0039d0 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc06f5d0 con 0x7fdbdc06ec00 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc0b1e50 con 0x7fdbdc15c040 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbdc0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbdc0b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36190/0 (socket says 172.21.15.143:36190) 2024-01-15T18:07:03.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 -- 172.21.15.143:0/3256879110 learned_addr learned my addr 172.21.15.143:0/3256879110 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:03.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbdbe00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc15c040 0x7fdbdc14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe1800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc06ec00 0x7fdbdc06f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 -- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc15c040 msgr2=0x7fdbdc14af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc15c040 0x7fdbdc14af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 -- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc06ec00 msgr2=0x7fdbdc06f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc06ec00 0x7fdbdc06f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 -- 172.21.15.143:0/3256879110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc14b860 con 0x7fdbbc0039d0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe1800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc06ec00 0x7fdbdc06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbdbe00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc15c040 0x7fdbdc14af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe0e00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbdc0b1910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ae0 tx=0x7fdbcc0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0a6d80 con 0x7fdbbc0039d0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0b9440 con 0x7fdbbc0039d0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0b97d0 con 0x7fdbbc0039d0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbdc14bb40 con 0x7fdbbc0039d0 2024-01-15T18:07:03.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc148f70 con 0x7fdbbc0039d0 2024-01-15T18:07:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.840+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc067500 con 0x7fdbbc0039d0 2024-01-15T18:07:03.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.844+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fdbcc0c2440 con 0x7fdbbc0039d0 2024-01-15T18:07:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.844+0000 7fdbd8c00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdbb4066b30 0x7fdbb4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.844+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fdbcc11e940 con 0x7fdbbc0039d0 2024-01-15T18:07:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.844+0000 7fdbdbe00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdbb4066b30 0x7fdbb4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:03.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.844+0000 7fdbdbe00640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdbb4066b30 0x7fdbb4068ff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4000c10 tx=0x7fdbc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:03.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.848+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdbcc0eb9e0 con 0x7fdbbc0039d0 2024-01-15T18:07:04.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:03.996+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdbdc0c70d0 con 0x7fdbbc0039d0 2024-01-15T18:07:04.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.000+0000 7fdbd8c00640 1 -- 172.21.15.143:0/3256879110 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdbcc0ef000 con 0x7fdbbc0039d0 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdbb4066b30 msgr2=0x7fdbb4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdbb4066b30 0x7fdbb4068ff0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4000c10 tx=0x7fdbc4009040 comp rx=0 tx=0).stop 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 msgr2=0x7fdbdc0b1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbdc0b1910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009ae0 tx=0x7fdbcc0197f0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 shutdown_connections 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdbb4066b30 0x7fdbb4068ff0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdbdc06ec00 0x7fdbdc06f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdbdc15c040 0x7fdbdc14af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 --2- 172.21.15.143:0/3256879110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbbc0039d0 0x7fdbdc0b1910 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 >> 172.21.15.143:0/3256879110 conn(0x7fdbdc0d0e30 msgr2=0x7fdbdc06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 shutdown_connections 2024-01-15T18:07:04.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.004+0000 7fdbe3800640 1 -- 172.21.15.143:0/3256879110 wait complete. 2024-01-15T18:07:04.032 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone' 2024-01-15T18:07:04.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.160+0000 7fd04e000640 1 Processor -- start 2024-01-15T18:07:04.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.160+0000 7fd04e000640 1 -- start start 2024-01-15T18:07:04.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.160+0000 7fd04e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd048159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.160+0000 7fd04e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0480b2a20 con 0x7fd0480b4ac0 2024-01-15T18:07:04.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.160+0000 7fd04e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0480b2d00 con 0x7fd0480b21d0 2024-01-15T18:07:04.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.160+0000 7fd04e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480b35a0 con 0x7fd048157300 2024-01-15T18:07:04.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd0480b4ac0 0x7fd048070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44820/0 (socket says 172.21.15.143:44820) 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 -- 172.21.15.143:0/2759662608 learned_addr learned my addr 172.21.15.143:0/2759662608 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047400640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd048159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051798891 0 0) 0x7fd0480b2a20 con 0x7fd0480b4ac0 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd028005740 con 0x7fd0480b4ac0 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517791690 0 0) 0x7fd028005740 con 0x7fd0480b4ac0 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 msgr2=0x7fd048159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd048159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 >> v1:172.21.15.204:6789/0 conn(0x7fd0480b21d0 legacy=0x7fd0480b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480b2fe0 con 0x7fd0480b4ac0 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fd040001a00 con 0x7fd0480b4ac0 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 >> v1:172.21.15.143:6789/0 conn(0x7fd0480b4ac0 legacy=0x7fd048070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd028005bc0 0x7fd028005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd0280069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd0280073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480b35a0 con 0x7fd028005bc0 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02800dc80 con 0x7fd028006f20 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02800de40 con 0x7fd048157300 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd0280073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047400640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd028005bc0 0x7fd028005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046a00640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd0280069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 -- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 msgr2=0x7fd0280069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd0280069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 -- 172.21.15.143:0/2759662608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd028005bc0 msgr2=0x7fd028005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd028005bc0 0x7fd028005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047e00640 1 -- 172.21.15.143:0/2759662608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028005740 con 0x7fd028006f20 2024-01-15T18:07:04.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd047400640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd028005bc0 0x7fd028005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:04.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.164+0000 7fd046a00640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048157300 0x7fd0280069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd047e00640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd0280073a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd040002240 tx=0x7fd04001d1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67ba83796d32e044 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400ad630 con 0x7fd028006f20 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400a8070 con 0x7fd028006f20 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd046000640 1 -- 172.21.15.143:0/2759662608 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400ad9f0 con 0x7fd028006f20 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 -- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 msgr2=0x7fd0280073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 --2- 172.21.15.143:0/2759662608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd0280073a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd040002240 tx=0x7fd04001d1b0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd047e00640 1 -- 172.21.15.143:0/2759662608 reap_dead start 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 -- 172.21.15.143:0/2759662608 shutdown_connections 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 -- 172.21.15.143:0/2759662608 >> 172.21.15.143:0/2759662608 conn(0x7fd04806c3f0 msgr2=0x7fd04806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:04.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 -- 172.21.15.143:0/2759662608 shutdown_connections 2024-01-15T18:07:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 -- 172.21.15.143:0/2759662608 wait complete. 2024-01-15T18:07:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 Processor -- start 2024-01-15T18:07:04.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.168+0000 7fd04e000640 1 -- start start 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd04814b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048153e70 0x7fd048152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 0x7fd0481526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048153a50 con 0x7fd0481542f0 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048152dd0 con 0x7fd028006f20 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0481530b0 con 0x7fd048153e70 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 0x7fd0481526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 0x7fd0481526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36206/0 (socket says 172.21.15.143:36206) 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 -- 172.21.15.143:0/1098163994 learned_addr learned my addr 172.21.15.143:0/1098163994 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:04.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047400640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd04814b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd046a00640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048153e70 0x7fd048152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 -- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048153e70 msgr2=0x7fd048152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048153e70 0x7fd048152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 -- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 msgr2=0x7fd04814b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd04814b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 -- 172.21.15.143:0/1098163994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048153390 con 0x7fd0481542f0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd047e00640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 0x7fd0481526d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd04001cfe0 tx=0x7fd028007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400bb070 con 0x7fd0481542f0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040001210 con 0x7fd0481542f0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd048153670 con 0x7fd0481542f0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040001940 con 0x7fd0481542f0 2024-01-15T18:07:04.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04814be60 con 0x7fd0481542f0 2024-01-15T18:07:04.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.172+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd048153b90 con 0x7fd0481542f0 2024-01-15T18:07:04.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.176+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fd0400aa030 con 0x7fd0481542f0 2024-01-15T18:07:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.176+0000 7fd03f400640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd020066b30 0x7fd020068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.176+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd04011e330 con 0x7fd0481542f0 2024-01-15T18:07:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.176+0000 7fd047400640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd020066b30 0x7fd020068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:04.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.176+0000 7fd047400640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd020066b30 0x7fd020068ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd04815cff0 tx=0x7fd038009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:04.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.180+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd048153b90 con 0x7fd0481542f0 2024-01-15T18:07:04.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:04.384+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]} v 0) v1 -- 0x7fd048153b90 con 0x7fd0481542f0 2024-01-15T18:07:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.064+0000 7fd03f400640 1 -- 172.21.15.143:0/1098163994 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]}]=0 v130) v1 ==== 152+0+0 (secure 0 0 0) 0x7fd048153b90 con 0x7fd0481542f0 2024-01-15T18:07:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd020066b30 msgr2=0x7fd020068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd020066b30 0x7fd020068ff0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd04815cff0 tx=0x7fd038009190 comp rx=0 tx=0).stop 2024-01-15T18:07:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 msgr2=0x7fd0481526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 0x7fd0481526d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd04001cfe0 tx=0x7fd028007a10 comp rx=0 tx=0).stop 2024-01-15T18:07:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 shutdown_connections 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd020066b30 0x7fd020068ff0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0481542f0 0x7fd0481526d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd048153e70 0x7fd048152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 --2- 172.21.15.143:0/1098163994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd028006f20 0x7fd04814b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 >> 172.21.15.143:0/1098163994 conn(0x7fd04806c3f0 msgr2=0x7fd0480adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 shutdown_connections 2024-01-15T18:07:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.068+0000 7fd04e000640 1 -- 172.21.15.143:0/1098163994 wait complete. 2024-01-15T18:07:05.099 INFO:tasks.cephfs_test_runner:Test that when one clone is in progress, one progress bar is printed ... ok 2024-01-15T18:07:05.099 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgress) X = number of clones issued. 2024-01-15T18:07:05.099 INFO:tasks.cephfs_test_runner:test_progress_bar_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:07:05.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:05.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.228+0000 7efe3a000640 1 Processor -- start 2024-01-15T18:07:05.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.228+0000 7efe3a000640 1 -- start start 2024-01-15T18:07:05.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe3a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efe340ce690 0x7efe340ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe3a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe340b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe3a000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe34070bf0 con 0x7efe340b1da0 2024-01-15T18:07:05.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe3a000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3415dba0 con 0x7efe340ce690 2024-01-15T18:07:05.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe3a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe340b4920 con 0x7efe340b3bf0 2024-01-15T18:07:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe33e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe340b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe33400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efe340ce690 0x7efe340ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe32a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7efe340b1da0 0x7efe340b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42374/0 (socket says 172.21.15.143:42374) 2024-01-15T18:07:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe32a00640 1 -- 172.21.15.143:0/547743635 learned_addr learned my addr 172.21.15.143:0/547743635 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:05.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe33e00640 1 -- 172.21.15.143:0/547743635 >> v2:172.21.15.204:3300/0 conn(0x7efe340ce690 msgr2=0x7efe340ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:05.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe33e00640 1 --2- 172.21.15.143:0/547743635 >> v2:172.21.15.204:3300/0 conn(0x7efe340ce690 0x7efe340ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe33e00640 1 -- 172.21.15.143:0/547743635 >> v1:172.21.15.204:6789/0 conn(0x7efe340b1da0 legacy=0x7efe340b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.232+0000 7efe33e00640 1 -- 172.21.15.143:0/547743635 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe340b4c00 con 0x7efe340b3bf0 2024-01-15T18:07:05.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe33e00640 1 --2- 172.21.15.143:0/547743635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe340b40a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009d90 tx=0x7efe2c01d380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850c7d4859ba62d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe32000640 1 -- 172.21.15.143:0/547743635 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0a5a00 con 0x7efe340b3bf0 2024-01-15T18:07:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe32000640 1 -- 172.21.15.143:0/547743635 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0a5b60 con 0x7efe340b3bf0 2024-01-15T18:07:05.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe32000640 1 -- 172.21.15.143:0/547743635 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0ad440 con 0x7efe340b3bf0 2024-01-15T18:07:05.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 -- 172.21.15.143:0/547743635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 msgr2=0x7efe340b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 --2- 172.21.15.143:0/547743635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe340b40a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009d90 tx=0x7efe2c01d380 comp rx=0 tx=0).stop 2024-01-15T18:07:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 -- 172.21.15.143:0/547743635 shutdown_connections 2024-01-15T18:07:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 --2- 172.21.15.143:0/547743635 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe340b40a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 --2- 172.21.15.143:0/547743635 >> v2:172.21.15.204:3300/0 conn(0x7efe340ce690 0x7efe340ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 -- 172.21.15.143:0/547743635 >> 172.21.15.143:0/547743635 conn(0x7efe3406c3f0 msgr2=0x7efe3406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 -- 172.21.15.143:0/547743635 shutdown_connections 2024-01-15T18:07:05.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.236+0000 7efe3a000640 1 -- 172.21.15.143:0/547743635 wait complete. 2024-01-15T18:07:05.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 Processor -- start 2024-01-15T18:07:05.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 -- start start 2024-01-15T18:07:05.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 0x7efe34142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe341674d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe340ce690 0x7efe3415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe341402a0 con 0x7efe340b1da0 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3413f250 con 0x7efe340ce690 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3413f530 con 0x7efe340b3bf0 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 0x7efe34142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 0x7efe34142c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36218/0 (socket says 172.21.15.143:36218) 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe32a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe341674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 -- 172.21.15.143:0/2204235349 learned_addr learned my addr 172.21.15.143:0/2204235349 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33e00640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe340ce690 0x7efe3415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 -- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 msgr2=0x7efe341674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe341674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 -- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe340ce690 msgr2=0x7efe3415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe340ce690 0x7efe3415c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 -- 172.21.15.143:0/2204235349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c009a70 con 0x7efe340b1da0 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe33400640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 0x7efe34142c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efe2400e870 tx=0x7efe2400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24009510 con 0x7efe340b1da0 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400eec0 con 0x7efe340b1da0 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400cae0 con 0x7efe340b1da0 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe341404f0 con 0x7efe340b1da0 2024-01-15T18:07:05.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.240+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe34143180 con 0x7efe340b1da0 2024-01-15T18:07:05.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.244+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe34067500 con 0x7efe340b1da0 2024-01-15T18:07:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.244+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7efe2400cc40 con 0x7efe340b1da0 2024-01-15T18:07:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.244+0000 7efe2b400640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe0c066b60 0x7efe0c069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.244+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7efe24084540 con 0x7efe340b1da0 2024-01-15T18:07:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.244+0000 7efe32a00640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe0c066b60 0x7efe0c069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.244+0000 7efe32a00640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe0c066b60 0x7efe0c069020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe1c000f70 tx=0x7efe1c007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.248+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efe240517a0 con 0x7efe340b1da0 2024-01-15T18:07:05.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe340b3610 con 0x7efe340b1da0 2024-01-15T18:07:05.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe2b400640 1 -- 172.21.15.143:0/2204235349 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13067 (secure 0 0 0) 0x7efe24054dc0 con 0x7efe340b1da0 2024-01-15T18:07:05.408 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:05.408 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":37,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:00.044806+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.204:6835/4126176885":"2024-01-16T18:06:57.026013+0000","172.21.15.204:6833/4126176885":"2024-01-16T18:06:57.026013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:05.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe0c066b60 msgr2=0x7efe0c069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe0c066b60 0x7efe0c069020 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe1c000f70 tx=0x7efe1c007e90 comp rx=0 tx=0).stop 2024-01-15T18:07:05.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 msgr2=0x7efe34142c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 0x7efe34142c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efe2400e870 tx=0x7efe2400ec70 comp rx=0 tx=0).stop 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 shutdown_connections 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe0c066b60 0x7efe0c069020 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efe340ce690 0x7efe3415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efe340b3bf0 0x7efe341674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 --2- 172.21.15.143:0/2204235349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efe340b1da0 0x7efe34142c40 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 >> 172.21.15.143:0/2204235349 conn(0x7efe3406c3f0 msgr2=0x7efe3415ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 shutdown_connections 2024-01-15T18:07:05.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.400+0000 7efe3a000640 1 -- 172.21.15.143:0/2204235349 wait complete. 2024-01-15T18:07:05.430 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:07:05.430 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:07:05.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 Processor -- start 2024-01-15T18:07:05.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 -- start start 2024-01-15T18:07:05.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3840cbf50 0x7fd3840cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3840cffd0 con 0x7fd3840cfb30 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3840d1ce0 con 0x7fd3840cbf50 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd38ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840d02b0 con 0x7fd3841607c0 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd389a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3840cbf50 0x7fd3840cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd389a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3840cbf50 0x7fd3840cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36232/0 (socket says 172.21.15.143:36232) 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.560+0000 7fd389a00640 1 -- 172.21.15.143:0/1210026111 learned_addr learned my addr 172.21.15.143:0/1210026111 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd389000640 1 --2- 172.21.15.143:0/1210026111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd389a00640 1 -- 172.21.15.143:0/1210026111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 msgr2=0x7fd38406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:05.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd389a00640 1 --2- 172.21.15.143:0/1210026111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd389a00640 1 -- 172.21.15.143:0/1210026111 >> v1:172.21.15.204:6789/0 conn(0x7fd3840cfb30 legacy=0x7fd38406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd389a00640 1 -- 172.21.15.143:0/1210026111 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3840cb470 con 0x7fd3840cbf50 2024-01-15T18:07:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd389a00640 1 --2- 172.21.15.143:0/1210026111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cbf50 0x7fd3840cc390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009e50 tx=0x7fd37c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e90925dd0dd276 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd383400640 1 -- 172.21.15.143:0/1210026111 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0af400 con 0x7fd3840cbf50 2024-01-15T18:07:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd383400640 1 -- 172.21.15.143:0/1210026111 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c0ab050 con 0x7fd3840cbf50 2024-01-15T18:07:05.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.564+0000 7fd383400640 1 -- 172.21.15.143:0/1210026111 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0af780 con 0x7fd3840cbf50 2024-01-15T18:07:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- 172.21.15.143:0/1210026111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cbf50 msgr2=0x7fd3840cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 --2- 172.21.15.143:0/1210026111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cbf50 0x7fd3840cc390 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009e50 tx=0x7fd37c01f1f0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- 172.21.15.143:0/1210026111 shutdown_connections 2024-01-15T18:07:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 --2- 172.21.15.143:0/1210026111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cbf50 0x7fd3840cc390 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 --2- 172.21.15.143:0/1210026111 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- 172.21.15.143:0/1210026111 >> 172.21.15.143:0/1210026111 conn(0x7fd3840d0e20 msgr2=0x7fd3840d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- 172.21.15.143:0/1210026111 shutdown_connections 2024-01-15T18:07:05.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- 172.21.15.143:0/1210026111 wait complete. 2024-01-15T18:07:05.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 Processor -- start 2024-01-15T18:07:05.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- start start 2024-01-15T18:07:05.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3840cbf50 0x7fd3841583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 0x7fd3840c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3840c7840 con 0x7fd3840cfb30 2024-01-15T18:07:05.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38414e810 con 0x7fd3840cbf50 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd38ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd384158a70 con 0x7fd3841607c0 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 0x7fd3840c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 0x7fd3840c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36248/0 (socket says 172.21.15.143:36248) 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 -- 172.21.15.143:0/4102777861 learned_addr learned my addr 172.21.15.143:0/4102777861 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd389a00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd389000640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3840cbf50 0x7fd3841583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 -- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 msgr2=0x7fd38414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 -- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3840cbf50 msgr2=0x7fd3841583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3840cbf50 0x7fd3841583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd383e00640 1 -- 172.21.15.143:0/4102777861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c009b30 con 0x7fd3840cfb30 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.568+0000 7fd389000640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3840cbf50 0x7fd3841583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd383e00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 0x7fd3840c7300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c500 tx=0x7fd36c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c005220 con 0x7fd3840cfb30 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c005380 con 0x7fd3840cfb30 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c007270 con 0x7fd3840cfb30 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd384158d50 con 0x7fd3840cfb30 2024-01-15T18:07:05.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3840c4c00 con 0x7fd3840cfb30 2024-01-15T18:07:05.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38406d4b0 con 0x7fd3840cfb30 2024-01-15T18:07:05.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fd36c0073d0 con 0x7fd3840cfb30 2024-01-15T18:07:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd380c00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd35c066b60 0x7fd35c069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd36c0839f0 con 0x7fd3840cfb30 2024-01-15T18:07:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.572+0000 7fd389000640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd35c066b60 0x7fd35c069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.576+0000 7fd389000640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd35c066b60 0x7fd35c069020 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd38415b120 tx=0x7fd374008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.576+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd36c050bf0 con 0x7fd3840cfb30 2024-01-15T18:07:05.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.728+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3840c4df0 con 0x7fd3840cfb30 2024-01-15T18:07:05.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.728+0000 7fd380c00640 1 -- 172.21.15.143:0/4102777861 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd36c054210 con 0x7fd3840cfb30 2024-01-15T18:07:05.737 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:07:05.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd35c066b60 msgr2=0x7fd35c069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd35c066b60 0x7fd35c069020 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd38415b120 tx=0x7fd374008040 comp rx=0 tx=0).stop 2024-01-15T18:07:05.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 msgr2=0x7fd3840c7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 0x7fd3840c7300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00c500 tx=0x7fd36c00c9c0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 shutdown_connections 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd35c066b60 0x7fd35c069020 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3841607c0 0x7fd38414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3840cfb30 0x7fd3840c7300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 --2- 172.21.15.143:0/4102777861 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3840cbf50 0x7fd3841583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 >> 172.21.15.143:0/4102777861 conn(0x7fd3840d0e20 msgr2=0x7fd3840d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 shutdown_connections 2024-01-15T18:07:05.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.732+0000 7fd38ba00640 1 -- 172.21.15.143:0/4102777861 wait complete. 2024-01-15T18:07:05.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones' 2024-01-15T18:07:05.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 Processor -- start 2024-01-15T18:07:05.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 -- start start 2024-01-15T18:07:05.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c141660 0x7f644c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f644c071d00 0x7f644c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c15f310 con 0x7f644c15ec80 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c15fbb0 con 0x7f644c071d00 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6452a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c072690 con 0x7f644c141660 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6450a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f644c071d00 0x7f644c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6450a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f644c071d00 0x7f644c072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:44144/0 (socket says 172.21.15.143:44144) 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f6450a00640 1 -- 172.21.15.143:0/684052882 learned_addr learned my addr 172.21.15.143:0/684052882 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.888+0000 7f644be00640 1 --2- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c141660 0x7f644c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6450a00640 1 -- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c141660 msgr2=0x7f644c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:05.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6450a00640 1 --2- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c141660 0x7f644c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6450a00640 1 -- 172.21.15.143:0/684052882 >> v1:172.21.15.204:6789/0 conn(0x7f644c15ec80 legacy=0x7f644c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6450a00640 1 -- 172.21.15.143:0/684052882 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c15f5f0 con 0x7f644c071d00 2024-01-15T18:07:05.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f644aa00640 1 -- 172.21.15.143:0/684052882 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133149319 0 0) 0x7f644c15f310 con 0x7f644c15ec80 2024-01-15T18:07:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6450a00640 1 --2- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c071d00 0x7f644c072150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6444009e50 tx=0x7f644401ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a66776dd0ee908 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f644aa00640 1 -- 172.21.15.143:0/684052882 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440a57d0 con 0x7f644c071d00 2024-01-15T18:07:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f644aa00640 1 -- 172.21.15.143:0/684052882 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64440a5930 con 0x7f644c071d00 2024-01-15T18:07:05.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f644aa00640 1 -- 172.21.15.143:0/684052882 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64440a5cc0 con 0x7f644c071d00 2024-01-15T18:07:05.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6452a00640 1 -- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c071d00 msgr2=0x7f644c072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.892+0000 7f6452a00640 1 --2- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c071d00 0x7f644c072150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6444009e50 tx=0x7f644401ca10 comp rx=0 tx=0).stop 2024-01-15T18:07:05.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- 172.21.15.143:0/684052882 shutdown_connections 2024-01-15T18:07:05.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 --2- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c071d00 0x7f644c072150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 --2- 172.21.15.143:0/684052882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c141660 0x7f644c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- 172.21.15.143:0/684052882 >> 172.21.15.143:0/684052882 conn(0x7f644c06c3f0 msgr2=0x7f644c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- 172.21.15.143:0/684052882 shutdown_connections 2024-01-15T18:07:05.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- 172.21.15.143:0/684052882 wait complete. 2024-01-15T18:07:05.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 Processor -- start 2024-01-15T18:07:05.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- start start 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 0x7f644c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c141660 0x7f644c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c15ec80 0x7f644c0b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c15b140 con 0x7f644c071d00 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0b2920 con 0x7f644c141660 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6452a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c15bdb0 con 0x7f644c15ec80 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 0x7f644c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 0x7f644c15b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36252/0 (socket says 172.21.15.143:36252) 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c141660 0x7f644c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 -- 172.21.15.143:0/2848707854 learned_addr learned my addr 172.21.15.143:0/2848707854 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f6450a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c15ec80 0x7f644c0b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 -- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c15ec80 msgr2=0x7f644c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c15ec80 0x7f644c0b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 -- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c141660 msgr2=0x7f644c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c141660 0x7f644c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.896+0000 7f644be00640 1 -- 172.21.15.143:0/2848707854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009b30 con 0x7f644c071d00 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f644be00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 0x7f644c15b720 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f643c00e8a0 tx=0x7f643c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c009510 con 0x7f644c071d00 2024-01-15T18:07:05.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c00c940 con 0x7f644c071d00 2024-01-15T18:07:05.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c00ccb0 con 0x7f644c071d00 2024-01-15T18:07:05.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c0cd150 con 0x7f644c071d00 2024-01-15T18:07:05.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c0b22d0 con 0x7f644c071d00 2024-01-15T18:07:05.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c0ccfb0 con 0x7f644c071d00 2024-01-15T18:07:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f643c013470 con 0x7f644c071d00 2024-01-15T18:07:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6443e00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6424066b60 0x7f6424069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.900+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f643c084620 con 0x7f644c071d00 2024-01-15T18:07:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.904+0000 7f644b400640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6424066b60 0x7f6424069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.904+0000 7f644b400640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6424066b60 0x7f6424069020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f644c13f8d0 tx=0x7f6434009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:05.908+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f644c0ccfb0 con 0x7f644c071d00 2024-01-15T18:07:06.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:06.108+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]} v 0) v1 -- 0x7f644c0ccf80 con 0x7f644c071d00 2024-01-15T18:07:07.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.072+0000 7f6443e00640 1 -- 172.21.15.143:0/2848707854 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]}]=0 v132) v1 ==== 156+0+0 (secure 0 0 0) 0x7f643c0517f0 con 0x7f644c071d00 2024-01-15T18:07:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6424066b60 msgr2=0x7f6424069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6424066b60 0x7f6424069020 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f644c13f8d0 tx=0x7f6434009040 comp rx=0 tx=0).stop 2024-01-15T18:07:07.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 msgr2=0x7f644c15b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 0x7f644c15b720 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f643c00e8a0 tx=0x7f643c00eca0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 shutdown_connections 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6424066b60 0x7f6424069020 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f644c15ec80 0x7f644c0b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f644c141660 0x7f644c0b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 --2- 172.21.15.143:0/2848707854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f644c071d00 0x7f644c15b720 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 >> 172.21.15.143:0/2848707854 conn(0x7f644c06c3f0 msgr2=0x7f644c0afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 shutdown_connections 2024-01-15T18:07:07.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.076+0000 7f6452a00640 1 -- 172.21.15.143:0/2848707854 wait complete. 2024-01-15T18:07:07.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:07:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 Processor -- start 2024-01-15T18:07:07.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 -- start start 2024-01-15T18:07:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 0x7fdedc06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdedc0cbf50 0x7fdedc0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdedc0cffd0 con 0x7fdedc0cfb30 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdedc0d1ce0 con 0x7fdedc0cbf50 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0d02b0 con 0x7fdedc1607c0 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee1200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdedc0cfb30 0x7fdedc06bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49472/0 (socket says 172.21.15.143:49472) 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee1200640 1 -- 172.21.15.143:0/660264485 learned_addr learned my addr 172.21.15.143:0/660264485 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee1c00640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 0x7fdedc06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee2600640 1 --2- 172.21.15.143:0/660264485 >> v2:172.21.15.204:3300/0 conn(0x7fdedc0cbf50 0x7fdedc0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837318967 0 0) 0x7fdedc0cffd0 con 0x7fdedc0cfb30 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdebc003530 con 0x7fdedc0cfb30 2024-01-15T18:07:07.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1854047348 0 0) 0x7fdebc003530 con 0x7fdedc0cfb30 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 msgr2=0x7fdedc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 0x7fdedc06b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 >> v2:172.21.15.204:3300/0 conn(0x7fdedc0cbf50 msgr2=0x7fdedc0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 --2- 172.21.15.143:0/660264485 >> v2:172.21.15.204:3300/0 conn(0x7fdedc0cbf50 0x7fdedc0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc0cb470 con 0x7fdedc0cfb30 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fdec4001680 con 0x7fdedc0cfb30 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 >> v1:172.21.15.143:6789/0 conn(0x7fdedc0cfb30 legacy=0x7fdedc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdebc003980 0x7fdebc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 0x7fdebc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdebc004d10 0x7fdebc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0d1ce0 con 0x7fdebc004d10 2024-01-15T18:07:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0d02b0 con 0x7fdebc003980 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc00b9c0 con 0x7fdedc1607c0 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee2600640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdebc004d10 0x7fdebc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee1200640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 0x7fdebc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.244+0000 7fdee1c00640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdebc003980 0x7fdebc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 -- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 msgr2=0x7fdebc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdedc1607c0 0x7fdebc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 -- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdebc003980 msgr2=0x7fdebc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdebc003980 0x7fdebc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 -- 172.21.15.143:0/660264485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc003530 con 0x7fdebc004d10 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdebc004d10 0x7fdebc005190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fded4009fc0 tx=0x7fded401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17afc4b1caac68 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40a5cb0 con 0x7fdebc004d10 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded400f040 con 0x7fdebc004d10 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdedbe00640 1 -- 172.21.15.143:0/660264485 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40ad680 con 0x7fdebc004d10 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee4600640 1 -- 172.21.15.143:0/660264485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdebc004d10 msgr2=0x7fdebc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee4600640 1 --2- 172.21.15.143:0/660264485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdebc004d10 0x7fdebc005190 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fded4009fc0 tx=0x7fded401cca0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.248+0000 7fdee2600640 1 -- 172.21.15.143:0/660264485 reap_dead start 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- 172.21.15.143:0/660264485 shutdown_connections 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- 172.21.15.143:0/660264485 >> 172.21.15.143:0/660264485 conn(0x7fdedc0d0e20 msgr2=0x7fdedc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:07.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- 172.21.15.143:0/660264485 shutdown_connections 2024-01-15T18:07:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- 172.21.15.143:0/660264485 wait complete. 2024-01-15T18:07:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 Processor -- start 2024-01-15T18:07:07.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- start start 2024-01-15T18:07:07.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdebc004d10 0x7fdedc1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc1e5b90 0x7fdedc1e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 0x7fdedc0c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0c52e0 con 0x7fdedc1e6530 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc0c4a10 con 0x7fdedc1e5b90 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdedc1e69c0 con 0x7fdebc004d10 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 0x7fdedc0c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 0x7fdedc0c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58632/0 (socket says 172.21.15.143:58632) 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 -- 172.21.15.143:0/2366637387 learned_addr learned my addr 172.21.15.143:0/2366637387 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee1c00640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdebc004d10 0x7fdedc1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee1200640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc1e5b90 0x7fdedc1e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 -- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdebc004d10 msgr2=0x7fdedc1f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdebc004d10 0x7fdedc1f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 -- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc1e5b90 msgr2=0x7fdedc1e5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc1e5b90 0x7fdedc1e5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.252+0000 7fdee2600640 1 -- 172.21.15.143:0/2366637387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc0cadb0 con 0x7fdedc1e6530 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fdee2600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 0x7fdedc0c5700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fded4002240 tx=0x7fdebc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40039a0 con 0x7fdedc1e6530 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded4003b00 con 0x7fdedc1e6530 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdedc0c4c20 con 0x7fdedc1e6530 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4003e70 con 0x7fdedc1e6530 2024-01-15T18:07:07.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdedc1f2710 con 0x7fdedc1e6530 2024-01-15T18:07:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdedc06d4b0 con 0x7fdedc1e6530 2024-01-15T18:07:07.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fded40ae890 con 0x7fdedc1e6530 2024-01-15T18:07:07.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fded9600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeb4066b30 0x7fdeb4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fded411eef0 con 0x7fdedc1e6530 2024-01-15T18:07:07.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fdee1c00640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeb4066b30 0x7fdeb4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.256+0000 7fdee1c00640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeb4066b30 0x7fdeb4068ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009be0 tx=0x7fdecc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:07.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.260+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fded40ec1e0 con 0x7fdedc1e6530 2024-01-15T18:07:07.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.412+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdedc0c5420 con 0x7fdedc1e6530 2024-01-15T18:07:07.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fded9600640 1 -- 172.21.15.143:0/2366637387 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdedc0c5420 con 0x7fdedc1e6530 2024-01-15T18:07:07.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeb4066b30 msgr2=0x7fdeb4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeb4066b30 0x7fdeb4068ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009be0 tx=0x7fdecc009190 comp rx=0 tx=0).stop 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 msgr2=0x7fdedc0c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 0x7fdedc0c5700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fded4002240 tx=0x7fdebc005800 comp rx=0 tx=0).stop 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 shutdown_connections 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeb4066b30 0x7fdeb4068ff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdedc1e6530 0x7fdedc0c5700 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdedc1e5b90 0x7fdedc1e5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 --2- 172.21.15.143:0/2366637387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdebc004d10 0x7fdedc1f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 >> 172.21.15.143:0/2366637387 conn(0x7fdedc0d0e20 msgr2=0x7fdedc0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 shutdown_connections 2024-01-15T18:07:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.416+0000 7fdee4600640 1 -- 172.21.15.143:0/2366637387 wait complete. 2024-01-15T18:07:07.445 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:07:07.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:07:07.537 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:07:07.603 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:07:07.604 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:07.604 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:07.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:07:07.613 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:07:07.613 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:07.623 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:07:07.623 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:07.623 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:07.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:07:07.662 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:07:07.663 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:07.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:07:07.673 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:07:07.673 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:07:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.832+0000 7fb4c1a00640 1 Processor -- start 2024-01-15T18:07:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.832+0000 7fb4c1a00640 1 -- start start 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4c1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4bc0b24b0 0x7fb4bc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4c1a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc070bf0 con 0x7fb4bc0ce690 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4c1a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4bc0b4920 con 0x7fb4bc0b3bf0 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4c1a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4bc15dba0 con 0x7fb4bc0b24b0 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb4bc0ce690 0x7fb4bc0b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49484/0 (socket says 172.21.15.143:49484) 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 -- 172.21.15.143:0/2241501688 learned_addr learned my addr 172.21.15.143:0/2241501688 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4ba000640 1 --2- 172.21.15.143:0/2241501688 >> v2:172.21.15.204:3300/0 conn(0x7fb4bc0b24b0 0x7fb4bc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752575868 0 0) 0x7fb4bc070bf0 con 0x7fb4bc0ce690 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb49c003530 con 0x7fb4bc0ce690 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1001995876 0 0) 0x7fb49c003530 con 0x7fb4bc0ce690 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 >> v2:172.21.15.204:3300/0 conn(0x7fb4bc0b24b0 msgr2=0x7fb4bc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 --2- 172.21.15.143:0/2241501688 >> v2:172.21.15.204:3300/0 conn(0x7fb4bc0b24b0 0x7fb4bc0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 >> v1:172.21.15.204:6789/0 conn(0x7fb4bc0b3bf0 legacy=0x7fb4bc0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc0b4c00 con 0x7fb4bc0ce690 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fb4ac001680 con 0x7fb4bc0ce690 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 >> v1:172.21.15.143:6789/0 conn(0x7fb4bc0ce690 legacy=0x7fb4bc0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb49c003980 0x7fb49c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c0047d0 0x7fb49c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb49c005190 0x7fb49c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc15dba0 con 0x7fb49c003980 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c00c030 con 0x7fb49c0047d0 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c00c1f0 con 0x7fb49c005190 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb49c003980 0x7fb49c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4ba000640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c0047d0 0x7fb49c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4bb400640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb49c005190 0x7fb49c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 -- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb49c005190 msgr2=0x7fb49c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb49c005190 0x7fb49c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 -- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c0047d0 msgr2=0x7fb49c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c0047d0 0x7fb49c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4baa00640 1 -- 172.21.15.143:0/2241501688 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c003530 con 0x7fb49c003980 2024-01-15T18:07:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.836+0000 7fb4bb400640 1 -- 172.21.15.143:0/2241501688 reap_dead start 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4baa00640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb49c003980 0x7fb49c003da0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c1c0 tx=0x7fb4ac01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b4cdb3638706ac server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac005ae0 con 0x7fb49c003980 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac005c40 con 0x7fb49c003980 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4b9600640 1 -- 172.21.15.143:0/2241501688 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0a5a20 con 0x7fb49c003980 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 -- 172.21.15.143:0/2241501688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb49c003980 msgr2=0x7fb49c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb49c003980 0x7fb49c003da0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c1c0 tx=0x7fb4ac01cbf0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 -- 172.21.15.143:0/2241501688 shutdown_connections 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/2241501688 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb49c003980 0x7fb49c003da0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 -- 172.21.15.143:0/2241501688 >> 172.21.15.143:0/2241501688 conn(0x7fb4bc06c3f0 msgr2=0x7fb4bc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:07.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 -- 172.21.15.143:0/2241501688 shutdown_connections 2024-01-15T18:07:07.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.840+0000 7fb4c1a00640 1 -- 172.21.15.143:0/2241501688 wait complete. 2024-01-15T18:07:07.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 Processor -- start 2024-01-15T18:07:07.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 -- start start 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c003980 0x7fb4bc1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 0x7fb4bc13f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4bc155ff0 0x7fb4bc141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc140740 con 0x7fb4bc153600 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc13fb30 con 0x7fb49c003980 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4bc13fe10 con 0x7fb4bc155ff0 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 0x7fb4bc13f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 0x7fb4bc13f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58650/0 (socket says 172.21.15.143:58650) 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 -- 172.21.15.143:0/1777839465 learned_addr learned my addr 172.21.15.143:0/1777839465 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4baa00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c003980 0x7fb4bc1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4bb400640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4bc155ff0 0x7fb4bc141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 -- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4bc155ff0 msgr2=0x7fb4bc141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4bc155ff0 0x7fb4bc141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 -- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c003980 msgr2=0x7fb4bc1426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c003980 0x7fb4bc1426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 -- 172.21.15.143:0/1777839465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc1400f0 con 0x7fb4bc153600 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4ba000640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 0x7fb4bc13f180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40119a0 tx=0x7fb4a4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400faf0 con 0x7fb4bc153600 2024-01-15T18:07:07.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a400fc50 con 0x7fb4bc153600 2024-01-15T18:07:07.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4bc1403d0 con 0x7fb4bc153600 2024-01-15T18:07:07.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4014440 con 0x7fb4bc153600 2024-01-15T18:07:07.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.844+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4bc156470 con 0x7fb4bc153600 2024-01-15T18:07:07.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.848+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4bc067500 con 0x7fb4bc153600 2024-01-15T18:07:07.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.848+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fb4a40145a0 con 0x7fb4bc153600 2024-01-15T18:07:07.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.848+0000 7fb4b2a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb494066b30 0x7fb494068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:07.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.848+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fb4a4084dc0 con 0x7fb4bc153600 2024-01-15T18:07:07.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.848+0000 7fb4baa00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb494066b30 0x7fb494068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:07.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.848+0000 7fb4baa00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb494066b30 0x7fb494068ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0ab000 tx=0x7fb4ac0b1040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:07.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:07.852+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4a4051fa0 con 0x7fb4bc153600 2024-01-15T18:07:08.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.028+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4bc06e260 con 0x7fb4bc153600 2024-01-15T18:07:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4b2a00640 1 -- 172.21.15.143:0/1777839465 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fb4a40555c0 con 0x7fb4bc153600 2024-01-15T18:07:08.039 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:08.039 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4922,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3272757327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3272757327},{"type":"v1","addr":"172.21.15.204:6834","nonce":3272757327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5217,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/518983608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":518983608},{"type":"v1","addr":"172.21.15.143:6836","nonce":518983608}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/434311355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":434311355},{"type":"v1","addr":"172.21.15.143:6837","nonce":434311355}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2336035276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":2336035276},{"type":"v1","addr":"172.21.15.204:6835","nonce":2336035276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-01-15T18:07:08.039 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 20 2024-01-15T18:07:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb494066b30 msgr2=0x7fb494068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb494066b30 0x7fb494068ff0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0ab000 tx=0x7fb4ac0b1040 comp rx=0 tx=0).stop 2024-01-15T18:07:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 msgr2=0x7fb4bc13f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 0x7fb4bc13f180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40119a0 tx=0x7fb4a4011e60 comp rx=0 tx=0).stop 2024-01-15T18:07:08.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 shutdown_connections 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb494066b30 0x7fb494068ff0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4bc155ff0 0x7fb4bc141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4bc153600 0x7fb4bc13f180 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 --2- 172.21.15.143:0/1777839465 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb49c003980 0x7fb4bc1426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 >> 172.21.15.143:0/1777839465 conn(0x7fb4bc06c3f0 msgr2=0x7fb4bc15c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 shutdown_connections 2024-01-15T18:07:08.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.032+0000 7fb4c1a00640 1 -- 172.21.15.143:0/1777839465 wait complete. 2024-01-15T18:07:08.062 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:07:08.062 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:07:08.062 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:07:08.063 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:07:08.063 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:07:08.063 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:07:08.063 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:07:08.063 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:07:08.064 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:07:08.064 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:07:08.064 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:07:08.064 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:07:08.064 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:07:08.065 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:07:08.065 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:07:08.065 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:07:08.079 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:07:08.079 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:07:08.080 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:07:08.080 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:07:08.086 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:07:08.087 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:07:08.088 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:07:08.089 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:07:08.125 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:07:08.126 DEBUG:teuthology.parallel:result is None 2024-01-15T18:07:08.127 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:07:08.127 DEBUG:teuthology.parallel:result is None 2024-01-15T18:07:08.134 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:07:08.135 DEBUG:teuthology.parallel:result is None 2024-01-15T18:07:08.137 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:07:08.137 DEBUG:teuthology.parallel:result is None 2024-01-15T18:07:08.137 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:07:08.214 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:07:08.215 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:07:08.226 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:07:08.227 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:07:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 Processor -- start 2024-01-15T18:07:08.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 -- start start 2024-01-15T18:07:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c0b29a0 0x7f3a9c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a9c06c560 0x7f3a9c0ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c15f7c0 con 0x7f3a9c15aef0 2024-01-15T18:07:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c06c310 con 0x7f3a9c06c560 2024-01-15T18:07:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15ebb0 con 0x7f3a9c0b29a0 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa0a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a9c06c560 0x7f3a9c0ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3a9a800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3a9c15aef0 0x7f3a9c0b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49492/0 (socket says 172.21.15.143:49492) 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3a9a800640 1 -- 172.21.15.143:0/324162856 learned_addr learned my addr 172.21.15.143:0/324162856 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3a93e00640 1 --2- 172.21.15.143:0/324162856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c0b29a0 0x7f3a9c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa0a00640 1 -- 172.21.15.143:0/324162856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c0b29a0 msgr2=0x7f3a9c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa0a00640 1 --2- 172.21.15.143:0/324162856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c0b29a0 0x7f3a9c06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa0a00640 1 -- 172.21.15.143:0/324162856 >> v1:172.21.15.143:6789/0 conn(0x7f3a9c15aef0 legacy=0x7f3a9c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3aa0a00640 1 -- 172.21.15.143:0/324162856 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c15ee90 con 0x7f3a9c06c560 2024-01-15T18:07:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.276+0000 7f3a99e00640 1 -- 172.21.15.143:0/324162856 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625302042 0 0) 0x7f3a9c15f7c0 con 0x7f3a9c15aef0 2024-01-15T18:07:08.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa0a00640 1 --2- 172.21.15.143:0/324162856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c06c560 0x7f3a9c0ceca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009d90 tx=0x7f3a9401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d474a1871ac06a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:08.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3a99e00640 1 -- 172.21.15.143:0/324162856 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940a5960 con 0x7f3a9c06c560 2024-01-15T18:07:08.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3a99e00640 1 -- 172.21.15.143:0/324162856 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a940a5ac0 con 0x7f3a9c06c560 2024-01-15T18:07:08.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3a99e00640 1 -- 172.21.15.143:0/324162856 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940a5e50 con 0x7f3a9c06c560 2024-01-15T18:07:08.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa1400640 1 -- 172.21.15.143:0/324162856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c06c560 msgr2=0x7f3a9c0ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa1400640 1 --2- 172.21.15.143:0/324162856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c06c560 0x7f3a9c0ceca0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009d90 tx=0x7f3a9401c920 comp rx=0 tx=0).stop 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa1400640 1 -- 172.21.15.143:0/324162856 shutdown_connections 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa1400640 1 --2- 172.21.15.143:0/324162856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c06c560 0x7f3a9c0ceca0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa1400640 1 --2- 172.21.15.143:0/324162856 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c0b29a0 0x7f3a9c06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.280+0000 7f3aa1400640 1 -- 172.21.15.143:0/324162856 >> 172.21.15.143:0/324162856 conn(0x7f3a9c066720 msgr2=0x7f3a9c06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 -- 172.21.15.143:0/324162856 shutdown_connections 2024-01-15T18:07:08.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 -- 172.21.15.143:0/324162856 wait complete. 2024-01-15T18:07:08.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 Processor -- start 2024-01-15T18:07:08.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 -- start start 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c06c560 0x7f3a9c1418c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a9c0b29a0 0x7f3a9c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 0x7f3a9c15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c141e00 con 0x7f3a9c15aef0 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c152d60 con 0x7f3a9c0b29a0 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1f6f20 con 0x7f3a9c06c560 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 0x7f3a9c15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 0x7f3a9c15d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58698/0 (socket says 172.21.15.143:58698) 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 -- 172.21.15.143:0/1541700487 learned_addr learned my addr 172.21.15.143:0/1541700487 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:08.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3a9a800640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c06c560 0x7f3a9c1418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3a93e00640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a9c0b29a0 0x7f3a9c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 -- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c06c560 msgr2=0x7f3a9c1418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c06c560 0x7f3a9c1418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 -- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a9c0b29a0 msgr2=0x7f3a9c152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a9c0b29a0 0x7f3a9c152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 -- 172.21.15.143:0/1541700487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94009a70 con 0x7f3a9c15aef0 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3a9a800640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c06c560 0x7f3a9c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.284+0000 7f3aa0a00640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 0x7f3a9c15d4d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009d90 tx=0x7f3a940196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94009450 con 0x7f3a9c15aef0 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a940adb70 con 0x7f3a9c15aef0 2024-01-15T18:07:08.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940b8420 con 0x7f3a9c15aef0 2024-01-15T18:07:08.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c1f7200 con 0x7f3a9c15aef0 2024-01-15T18:07:08.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c1f77c0 con 0x7f3a9c15aef0 2024-01-15T18:07:08.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c067680 con 0x7f3a9c15aef0 2024-01-15T18:07:08.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f3a940b8580 con 0x7f3a9c15aef0 2024-01-15T18:07:08.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a92a00640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a74066b30 0x7f3a74068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:08.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3a9411fb40 con 0x7f3a9c15aef0 2024-01-15T18:07:08.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.288+0000 7f3a9a800640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a74066b30 0x7f3a74068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:08.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.292+0000 7f3a9a800640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a74066b30 0x7f3a74068ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000f70 tx=0x7f3a8c007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:08.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.292+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3a940ecd10 con 0x7f3a9c15aef0 2024-01-15T18:07:08.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:08.456+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3a9c06e790 con 0x7f3a9c15aef0 2024-01-15T18:07:09.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.552+0000 7f3a92a00640 1 -- 172.21.15.143:0/1541700487 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v42) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3a940c6080 con 0x7f3a9c15aef0 2024-01-15T18:07:09.561 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:07:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a74066b30 msgr2=0x7f3a74068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a74066b30 0x7f3a74068ff0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c000f70 tx=0x7f3a8c007df0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 msgr2=0x7f3a9c15d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 0x7f3a9c15d4d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3a94009d90 tx=0x7f3a940196c0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 shutdown_connections 2024-01-15T18:07:09.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a74066b30 0x7f3a74068ff0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a9c15aef0 0x7f3a9c15d4d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a9c0b29a0 0x7f3a9c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 --2- 172.21.15.143:0/1541700487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a9c06c560 0x7f3a9c1418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 >> 172.21.15.143:0/1541700487 conn(0x7f3a9c066720 msgr2=0x7f3a9c06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:09.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 shutdown_connections 2024-01-15T18:07:09.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.556+0000 7f3aa1400640 1 -- 172.21.15.143:0/1541700487 wait complete. 2024-01-15T18:07:09.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:07:09.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.720+0000 7ff82b200640 1 Processor -- start 2024-01-15T18:07:09.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.720+0000 7ff82b200640 1 -- start start 2024-01-15T18:07:09.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff82b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff824071bb0 0x7ff824072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff82b200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff82415c5a0 0x7ff82415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff82b200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240ceb50 con 0x7ff824143780 2024-01-15T18:07:09.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff82b200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff824072570 con 0x7ff82415c5a0 2024-01-15T18:07:09.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff82b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82415c110 con 0x7ff824071bb0 2024-01-15T18:07:09.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff823e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff824143780 0x7ff8240cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49500/0 (socket says 172.21.15.143:49500) 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff823e00640 1 -- 172.21.15.143:0/381685268 learned_addr learned my addr 172.21.15.143:0/381685268 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395009114 0 0) 0x7ff814001120 con 0x7ff824143780 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 --2- 172.21.15.143:0/381685268 >> v2:172.21.15.204:3300/0 conn(0x7ff82415c5a0 0x7ff82415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff804005740 con 0x7ff824143780 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff823400640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff824071bb0 0x7ff824072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3594672507 0 0) 0x7ff804005740 con 0x7ff824143780 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff824071bb0 msgr2=0x7ff824072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:09.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff824071bb0 0x7ff824072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 >> v2:172.21.15.204:3300/0 conn(0x7ff82415c5a0 msgr2=0x7ff82415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 --2- 172.21.15.143:0/381685268 >> v2:172.21.15.204:3300/0 conn(0x7ff82415c5a0 0x7ff82415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82415b7e0 con 0x7ff824143780 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff814001810 con 0x7ff824143780 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 >> v1:172.21.15.143:6789/0 conn(0x7ff824143780 legacy=0x7ff8240cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804005b90 0x7ff804005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff804006580 0x7ff804006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff804007180 0x7ff80400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824072570 con 0x7ff804007180 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82415c110 con 0x7ff804006580 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80400e110 con 0x7ff804005b90 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff823400640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff824071bb0 0x7ff824072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff804007180 0x7ff80400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff823e00640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804005b90 0x7ff804005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff823400640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff804006580 0x7ff804006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 -- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804005b90 msgr2=0x7ff804005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:09.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804005b90 0x7ff804005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 -- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff804006580 msgr2=0x7ff804006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:09.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff804006580 0x7ff804006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.724+0000 7ff829200640 1 -- 172.21.15.143:0/381685268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804005740 con 0x7ff804007180 2024-01-15T18:07:09.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff829200640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff804007180 0x7ff80400dba0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009fc0 tx=0x7ff81c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4162641e41daa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:09.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0a5cb0 con 0x7ff804007180 2024-01-15T18:07:09.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c00f040 con 0x7ff804007180 2024-01-15T18:07:09.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff822a00640 1 -- 172.21.15.143:0/381685268 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0ad680 con 0x7ff804007180 2024-01-15T18:07:09.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff82b200640 1 -- 172.21.15.143:0/381685268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff804007180 msgr2=0x7ff80400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff82b200640 1 --2- 172.21.15.143:0/381685268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff804007180 0x7ff80400dba0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff81c009fc0 tx=0x7ff81c01cca0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff829200640 1 -- 172.21.15.143:0/381685268 reap_dead start 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff82b200640 1 -- 172.21.15.143:0/381685268 shutdown_connections 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff82b200640 1 -- 172.21.15.143:0/381685268 >> 172.21.15.143:0/381685268 conn(0x7ff82406c3f0 msgr2=0x7ff82406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff82b200640 1 -- 172.21.15.143:0/381685268 shutdown_connections 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.728+0000 7ff82b200640 1 -- 172.21.15.143:0/381685268 wait complete. 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 Processor -- start 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 -- start start 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804007180 0x7ff8241e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8241e3330 0x7ff82406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 0x7ff824157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241e5730 con 0x7ff824157480 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824157e50 con 0x7ff8241e3330 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241e4b20 con 0x7ff804007180 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 0x7ff824157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 0x7ff824157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58722/0 (socket says 172.21.15.143:58722) 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 -- 172.21.15.143:0/747720272 learned_addr learned my addr 172.21.15.143:0/747720272 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff823e00640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804007180 0x7ff8241e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff823400640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8241e3330 0x7ff82406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 -- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804007180 msgr2=0x7ff8241e20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804007180 0x7ff8241e20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 -- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8241e3330 msgr2=0x7ff82406d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8241e3330 0x7ff82406d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 -- 172.21.15.143:0/747720272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8241e4e00 con 0x7ff824157480 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff829200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 0x7ff824157910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0b0040 tx=0x7ff81c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0039a0 con 0x7ff824157480 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c003b20 con 0x7ff824157480 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0ae950 con 0x7ff824157480 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8241e50e0 con 0x7ff824157480 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.732+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8241e25e0 con 0x7ff824157480 2024-01-15T18:07:09.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.736+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8240b4cc0 con 0x7ff824157480 2024-01-15T18:07:09.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.736+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7ff81c0b4040 con 0x7ff824157480 2024-01-15T18:07:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.736+0000 7ff81be00640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff7fc066b60 0x7ff7fc069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.736+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff81c11e3d0 con 0x7ff824157480 2024-01-15T18:07:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.736+0000 7ff823e00640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff7fc066b60 0x7ff7fc069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.736+0000 7ff823e00640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff7fc066b60 0x7ff7fc069020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff82406e3d0 tx=0x7ff814009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:09.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.740+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8240b4cc0 con 0x7ff824157480 2024-01-15T18:07:09.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff8240b4cc0 con 0x7ff824157480 2024-01-15T18:07:09.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff81be00640 1 -- 172.21.15.143:0/747720272 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff8240b4cc0 con 0x7ff824157480 2024-01-15T18:07:09.948 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:09.948 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:07:09.948 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:07:09.948 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:07:09.949 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.950 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.951 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.952 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.954 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.955 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.956 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.957 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:07:09.958 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.959 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:07:09.960 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.961 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:07:09.962 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:07:09.963 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:09.964 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:07:09.965 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:07:09.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff7fc066b60 msgr2=0x7ff7fc069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff7fc066b60 0x7ff7fc069020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff82406e3d0 tx=0x7ff814009040 comp rx=0 tx=0).stop 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 msgr2=0x7ff824157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 0x7ff824157910 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0b0040 tx=0x7ff81c009750 comp rx=0 tx=0).stop 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 shutdown_connections 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff7fc066b60 0x7ff7fc069020 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff824157480 0x7ff824157910 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8241e3330 0x7ff82406d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 --2- 172.21.15.143:0/747720272 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff804007180 0x7ff8241e20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.940+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 >> 172.21.15.143:0/747720272 conn(0x7ff82406c3f0 msgr2=0x7ff8240aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.944+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 shutdown_connections 2024-01-15T18:07:09.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:09.944+0000 7ff82b200640 1 -- 172.21.15.143:0/747720272 wait complete. 2024-01-15T18:07:09.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:10.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 Processor -- start 2024-01-15T18:07:10.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 -- start start 2024-01-15T18:07:10.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f997c141660 0x7f997c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c15ec80 0x7f997c141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c15f310 con 0x7f997c142470 2024-01-15T18:07:10.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c15fbb0 con 0x7f997c141660 2024-01-15T18:07:10.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f9983200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c15f100 con 0x7f997c15ec80 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f997c141660 0x7f997c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f997c141660 0x7f997c0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58734/0 (socket says 172.21.15.143:58734) 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 -- 172.21.15.143:0/4083108164 learned_addr learned my addr 172.21.15.143:0/4083108164 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 -- 172.21.15.143:0/4083108164 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c15ec80 msgr2=0x7f997c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 --2- 172.21.15.143:0/4083108164 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c15ec80 0x7f997c141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 -- 172.21.15.143:0/4083108164 >> v1:172.21.15.204:6789/0 conn(0x7f997c142470 legacy=0x7f997c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:10.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.100+0000 7f997be00640 1 -- 172.21.15.143:0/4083108164 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c15f5f0 con 0x7f997c141660 2024-01-15T18:07:10.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f997be00640 1 --2- 172.21.15.143:0/4083108164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c141660 0x7f997c0ce560 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f996c009870 tx=0x7f996c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524d6e77e9eab1b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f997aa00640 1 -- 172.21.15.143:0/4083108164 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0aa5f0 con 0x7f997c141660 2024-01-15T18:07:10.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f997aa00640 1 -- 172.21.15.143:0/4083108164 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0aa750 con 0x7f997c141660 2024-01-15T18:07:10.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f997aa00640 1 -- 172.21.15.143:0/4083108164 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0aaae0 con 0x7f997c141660 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 -- 172.21.15.143:0/4083108164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c141660 msgr2=0x7f997c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 --2- 172.21.15.143:0/4083108164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c141660 0x7f997c0ce560 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f996c009870 tx=0x7f996c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 -- 172.21.15.143:0/4083108164 shutdown_connections 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 --2- 172.21.15.143:0/4083108164 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c15ec80 0x7f997c141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 --2- 172.21.15.143:0/4083108164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c141660 0x7f997c0ce560 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 -- 172.21.15.143:0/4083108164 >> 172.21.15.143:0/4083108164 conn(0x7f997c06c3f0 msgr2=0x7f997c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:10.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 -- 172.21.15.143:0/4083108164 shutdown_connections 2024-01-15T18:07:10.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.104+0000 7f9983200640 1 -- 172.21.15.143:0/4083108164 wait complete. 2024-01-15T18:07:10.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 Processor -- start 2024-01-15T18:07:10.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 -- start start 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997c141660 0x7f997c139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c142470 0x7f997c13a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 0x7f997c14b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c15b2e0 con 0x7f997c15ec80 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c14be30 con 0x7f997c141660 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c159c70 con 0x7f997c142470 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 0x7f997c14b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 0x7f997c14b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58742/0 (socket says 172.21.15.143:58742) 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 -- 172.21.15.143:0/3804370359 learned_addr learned my addr 172.21.15.143:0/3804370359 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f997b400640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c142470 0x7f997c13a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f997be00640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997c141660 0x7f997c139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 -- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c142470 msgr2=0x7f997c13a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c142470 0x7f997c13a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 -- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997c141660 msgr2=0x7f997c139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997c141660 0x7f997c139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 -- 172.21.15.143:0/3804370359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0094d0 con 0x7f997c15ec80 2024-01-15T18:07:10.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9981200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 0x7f997c14b8f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f997400c780 tx=0x7f997400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974003d90 con 0x7f997c15ec80 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997400ce30 con 0x7f997c15ec80 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c0cd150 con 0x7f997c15ec80 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974005290 con 0x7f997c15ec80 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.108+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c15a230 con 0x7f997c15ec80 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.112+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c067500 con 0x7f997c15ec80 2024-01-15T18:07:10.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.112+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f99740053f0 con 0x7f997c15ec80 2024-01-15T18:07:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.112+0000 7f9973e00640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9954066b30 0x7f9954068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.112+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f9974083950 con 0x7f997c15ec80 2024-01-15T18:07:10.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.112+0000 7f997be00640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9954066b30 0x7f9954068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.112+0000 7f997be00640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9954066b30 0x7f9954068ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f996c0ae000 tx=0x7f996c0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.116+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9974050b80 con 0x7f997c15ec80 2024-01-15T18:07:10.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f997c06e2a0 con 0x7f997c15ec80 2024-01-15T18:07:10.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9973e00640 1 -- 172.21.15.143:0/3804370359 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12936 (secure 0 0 0) 0x7f9974017090 con 0x7f997c15ec80 2024-01-15T18:07:10.276 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:10.276 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":42,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:09.549337+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9954066b30 msgr2=0x7f9954068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9954066b30 0x7f9954068ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f996c0ae000 tx=0x7f996c0b5040 comp rx=0 tx=0).stop 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 msgr2=0x7f997c14b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 0x7f997c14b8f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f997400c780 tx=0x7f997400cc40 comp rx=0 tx=0).stop 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 shutdown_connections 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9954066b30 0x7f9954068ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997c15ec80 0x7f997c14b8f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f997c142470 0x7f997c13a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 --2- 172.21.15.143:0/3804370359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f997c141660 0x7f997c139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 >> 172.21.15.143:0/3804370359 conn(0x7f997c06c3f0 msgr2=0x7f997c0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:10.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 shutdown_connections 2024-01-15T18:07:10.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.268+0000 7f9983200640 1 -- 172.21.15.143:0/3804370359 wait complete. 2024-01-15T18:07:10.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:07:10.298 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:07:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 Processor -- start 2024-01-15T18:07:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 -- start start 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5110157300 0x7f5110159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51100b21d0 0x7f51100b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51100b4d20 con 0x7f51100b48f0 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51100b35a0 con 0x7f51100b21d0 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f5115a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51100b2a20 con 0x7f5110157300 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51100b21d0 0x7f51100b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f51100b21d0 0x7f51100b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58758/0 (socket says 172.21.15.143:58758) 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 -- 172.21.15.143:0/641996511 learned_addr learned my addr 172.21.15.143:0/641996511 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510f400640 1 --1- 172.21.15.143:0/641996511 >> v1:172.21.15.143:6789/0 conn(0x7f51100b48f0 0x7f511015d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49502/0 (socket says 172.21.15.143:49502) 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510ea00640 1 --2- 172.21.15.143:0/641996511 >> v2:172.21.15.204:3300/0 conn(0x7f5110157300 0x7f5110159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 -- 172.21.15.143:0/641996511 >> v2:172.21.15.204:3300/0 conn(0x7f5110157300 msgr2=0x7f5110159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 --2- 172.21.15.143:0/641996511 >> v2:172.21.15.204:3300/0 conn(0x7f5110157300 0x7f5110159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 -- 172.21.15.143:0/641996511 >> v1:172.21.15.143:6789/0 conn(0x7f51100b48f0 legacy=0x7f511015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510e000640 1 -- 172.21.15.143:0/641996511 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100b2d00 con 0x7f51100b21d0 2024-01-15T18:07:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.428+0000 7f510d600640 1 -- 172.21.15.143:0/641996511 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568056799 0 0) 0x7f51100b4d20 con 0x7f51100b48f0 2024-01-15T18:07:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f510e000640 1 --2- 172.21.15.143:0/641996511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51100b21d0 0x7f51100b43b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f51100b1aa0 tx=0x7f50f801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e9d315ca9c7f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f510d600640 1 -- 172.21.15.143:0/641996511 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80ac620 con 0x7f51100b21d0 2024-01-15T18:07:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f510d600640 1 -- 172.21.15.143:0/641996511 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f80ac780 con 0x7f51100b21d0 2024-01-15T18:07:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f510d600640 1 -- 172.21.15.143:0/641996511 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80acb10 con 0x7f51100b21d0 2024-01-15T18:07:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f5115a00640 1 -- 172.21.15.143:0/641996511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51100b21d0 msgr2=0x7f51100b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f5115a00640 1 --2- 172.21.15.143:0/641996511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51100b21d0 0x7f51100b43b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f51100b1aa0 tx=0x7f50f801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f5115a00640 1 -- 172.21.15.143:0/641996511 shutdown_connections 2024-01-15T18:07:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f5115a00640 1 --2- 172.21.15.143:0/641996511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51100b21d0 0x7f51100b43b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f5115a00640 1 --2- 172.21.15.143:0/641996511 >> v2:172.21.15.204:3300/0 conn(0x7f5110157300 0x7f5110159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.432+0000 7f5115a00640 1 -- 172.21.15.143:0/641996511 >> 172.21.15.143:0/641996511 conn(0x7f511006c3f0 msgr2=0x7f511006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 -- 172.21.15.143:0/641996511 shutdown_connections 2024-01-15T18:07:10.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 -- 172.21.15.143:0/641996511 wait complete. 2024-01-15T18:07:10.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 Processor -- start 2024-01-15T18:07:10.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 -- start start 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51100b21d0 0x7f511014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51100b48f0 0x7f51101590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 0x7f51100aea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110152540 con 0x7f5110157300 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110166660 con 0x7f51100b48f0 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f5115a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5110150610 con 0x7f51100b21d0 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 0x7f51100aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 0x7f51100aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58766/0 (socket says 172.21.15.143:58766) 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510f400640 1 -- 172.21.15.143:0/495174626 learned_addr learned my addr 172.21.15.143:0/495174626 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510e000640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51100b48f0 0x7f51101590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510ea00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51100b21d0 0x7f511014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510e000640 1 -- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51100b21d0 msgr2=0x7f511014b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510e000640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51100b21d0 0x7f511014b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510e000640 1 -- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 msgr2=0x7f51100aea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510e000640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 0x7f51100aea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510e000640 1 -- 172.21.15.143:0/495174626 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80094d0 con 0x7f51100b48f0 2024-01-15T18:07:10.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.436+0000 7f510f400640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 0x7f51100aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f510e000640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51100b48f0 0x7f51101590b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50f80ac000 tx=0x7f50f8019270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80ba070 con 0x7f51100b48f0 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f80ac5e0 con 0x7f51100b48f0 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80b5680 con 0x7f51100b48f0 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511014b550 con 0x7f51100b48f0 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5110166870 con 0x7f51100b48f0 2024-01-15T18:07:10.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5110067500 con 0x7f51100b48f0 2024-01-15T18:07:10.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f50f80b3070 con 0x7f51100b48f0 2024-01-15T18:07:10.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5106a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50e8066b60 0x7f50e8069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f50f811dbd0 con 0x7f51100b48f0 2024-01-15T18:07:10.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.440+0000 7f510ea00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50e8066b60 0x7f50e8069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.444+0000 7f510ea00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50e8066b60 0x7f50e8069020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f51000079c0 tx=0x7f5100008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.448+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50f80eace0 con 0x7f51100b48f0 2024-01-15T18:07:10.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f511006e4e0 con 0x7f51100b48f0 2024-01-15T18:07:10.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5106a00640 1 -- 172.21.15.143:0/495174626 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50f80b10a0 con 0x7f51100b48f0 2024-01-15T18:07:10.603 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:07:10.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50e8066b60 msgr2=0x7f50e8069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50e8066b60 0x7f50e8069020 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f51000079c0 tx=0x7f5100008040 comp rx=0 tx=0).stop 2024-01-15T18:07:10.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51100b48f0 msgr2=0x7f51101590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51100b48f0 0x7f51101590b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50f80ac000 tx=0x7f50f8019270 comp rx=0 tx=0).stop 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 shutdown_connections 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50e8066b60 0x7f50e8069020 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5110157300 0x7f51100aea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51100b48f0 0x7f51101590b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 --2- 172.21.15.143:0/495174626 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51100b21d0 0x7f511014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 >> 172.21.15.143:0/495174626 conn(0x7f511006c3f0 msgr2=0x7f51101527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 shutdown_connections 2024-01-15T18:07:10.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.596+0000 7f5115a00640 1 -- 172.21.15.143:0/495174626 wait complete. 2024-01-15T18:07:10.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:07:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a14200640 1 Processor -- start 2024-01-15T18:07:10.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a14200640 1 -- start start 2024-01-15T18:07:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a14200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6a0c0ccf60 0x7f6a0c167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a14200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0c06c5d0 con 0x7f6a0c1607c0 2024-01-15T18:07:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a14200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0c0cffd0 con 0x7f6a0c0cfb30 2024-01-15T18:07:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a14200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d1ce0 con 0x7f6a0c0ccf60 2024-01-15T18:07:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6a0c1607c0 0x7f6a0c06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49510/0 (socket says 172.21.15.143:49510) 2024-01-15T18:07:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 -- 172.21.15.143:0/1308403077 learned_addr learned my addr 172.21.15.143:0/1308403077 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a12200640 1 --2- 172.21.15.143:0/1308403077 >> v2:172.21.15.204:3300/0 conn(0x7f6a0c0ccf60 0x7f6a0c167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397776511 0 0) 0x7f6a0c06c5d0 con 0x7f6a0c1607c0 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ec003530 con 0x7f6a0c1607c0 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2224913013 0 0) 0x7f69ec003530 con 0x7f6a0c1607c0 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 >> v2:172.21.15.204:3300/0 conn(0x7f6a0c0ccf60 msgr2=0x7f6a0c167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 --2- 172.21.15.143:0/1308403077 >> v2:172.21.15.204:3300/0 conn(0x7f6a0c0ccf60 0x7f6a0c167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 >> v1:172.21.15.204:6789/0 conn(0x7f6a0c0cfb30 legacy=0x7f6a0c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c0d02b0 con 0x7f6a0c1607c0 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f69fc001680 con 0x7f6a0c1607c0 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 >> v1:172.21.15.143:6789/0 conn(0x7f6a0c1607c0 legacy=0x7f6a0c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f69ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69ec0047d0 0x7f69ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec005190 0x7f69ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0d1ce0 con 0x7f69ec003980 2024-01-15T18:07:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec00c040 con 0x7f69ec005190 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec00c200 con 0x7f69ec0047d0 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f69ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a10e00640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69ec0047d0 0x7f69ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a12200640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec005190 0x7f69ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 -- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69ec0047d0 msgr2=0x7f69ec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f69ec0047d0 0x7f69ec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 -- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec005190 msgr2=0x7f69ec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec005190 0x7f69ec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a11800640 1 -- 172.21.15.143:0/1308403077 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ec003530 con 0x7f69ec003980 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a12200640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69ec005190 0x7f69ec00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.756+0000 7f6a12200640 1 -- 172.21.15.143:0/1308403077 reap_dead start 2024-01-15T18:07:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a11800640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f69ec003da0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c240 tx=0x7f69fc01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c1a8c91959fc0b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc005ae0 con 0x7f69ec003980 2024-01-15T18:07:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc0ab070 con 0x7f69ec003980 2024-01-15T18:07:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a0be00640 1 -- 172.21.15.143:0/1308403077 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc005ea0 con 0x7f69ec003980 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a14200640 1 -- 172.21.15.143:0/1308403077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 msgr2=0x7f69ec003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a14200640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f69ec003da0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c240 tx=0x7f69fc01f370 comp rx=0 tx=0).stop 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a14200640 1 -- 172.21.15.143:0/1308403077 shutdown_connections 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a14200640 1 --2- 172.21.15.143:0/1308403077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f69ec003da0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.760+0000 7f6a14200640 1 -- 172.21.15.143:0/1308403077 >> 172.21.15.143:0/1308403077 conn(0x7f6a0c0d0e20 msgr2=0x7f6a0c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 -- 172.21.15.143:0/1308403077 shutdown_connections 2024-01-15T18:07:10.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 -- 172.21.15.143:0/1308403077 wait complete. 2024-01-15T18:07:10.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 Processor -- start 2024-01-15T18:07:10.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 -- start start 2024-01-15T18:07:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f6a0c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a0c0b1110 0x7f6a0c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c151e30 0x7f6a0c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0adc70 con 0x7f69ec003980 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c1527f0 con 0x7f6a0c151e30 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a14200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a0c0ca590 con 0x7f6a0c0b1110 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f6a0c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f6a0c06e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58770/0 (socket says 172.21.15.143:58770) 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 -- 172.21.15.143:0/2675548428 learned_addr learned my addr 172.21.15.143:0/2675548428 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a12200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c151e30 0x7f6a0c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a10e00640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a0c0b1110 0x7f6a0c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 -- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a0c0b1110 msgr2=0x7f6a0c0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a0c0b1110 0x7f6a0c0b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 -- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c151e30 msgr2=0x7f6a0c1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c151e30 0x7f6a0c1522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 -- 172.21.15.143:0/2675548428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0c0c9cf0 con 0x7f69ec003980 2024-01-15T18:07:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a12200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c151e30 0x7f6a0c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.764+0000 7f6a11800640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f6a0c06e710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69fc01f1a0 tx=0x7f69ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc001680 con 0x7f69ec003980 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69fc007070 con 0x7f69ec003980 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc007400 con 0x7f69ec003980 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0c0c9fd0 con 0x7f69ec003980 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c0c9950 con 0x7f69ec003980 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0c06d4b0 con 0x7f69ec003980 2024-01-15T18:07:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f69fc007560 con 0x7f69ec003980 2024-01-15T18:07:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a09600640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69e4066b60 0x7f69e4069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f69fc0b7070 con 0x7f69ec003980 2024-01-15T18:07:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a10e00640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69e4066b60 0x7f69e4069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.768+0000 7f6a10e00640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69e4066b60 0x7f69e4069020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0cd7a0 tx=0x7f69f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:10.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.772+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f69fc0ee130 con 0x7f69ec003980 2024-01-15T18:07:10.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6a0c067500 con 0x7f69ec003980 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a09600640 1 -- 172.21.15.143:0/2675548428 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f69fc0f1750 con 0x7f69ec003980 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:07:10.942 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:10.943 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:07:10.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69e4066b60 msgr2=0x7f69e4069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69e4066b60 0x7f69e4069020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0cd7a0 tx=0x7f69f4009040 comp rx=0 tx=0).stop 2024-01-15T18:07:10.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 msgr2=0x7f6a0c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:10.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f6a0c06e710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f69fc01f1a0 tx=0x7f69ec005800 comp rx=0 tx=0).stop 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 shutdown_connections 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69e4066b60 0x7f69e4069020 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a0c151e30 0x7f6a0c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a0c0b1110 0x7f6a0c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 --2- 172.21.15.143:0/2675548428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f69ec003980 0x7f6a0c06e710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.924+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 >> 172.21.15.143:0/2675548428 conn(0x7f6a0c0d0e20 msgr2=0x7f6a0c0c74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.928+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 shutdown_connections 2024-01-15T18:07:10.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:10.928+0000 7f6a14200640 1 -- 172.21.15.143:0/2675548428 wait complete. 2024-01-15T18:07:10.955 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:07:10.955 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:07:11.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.088+0000 7f0b13000640 1 Processor -- start 2024-01-15T18:07:11.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.088+0000 7f0b13000640 1 -- start start 2024-01-15T18:07:11.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.088+0000 7f0b13000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.088+0000 7f0b13000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c06ae30 con 0x7f0b0c146c00 2024-01-15T18:07:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.088+0000 7f0b13000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c06e8b0 con 0x7f0b0c0714a0 2024-01-15T18:07:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b13000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c146fe0 con 0x7f0b0c06a490 2024-01-15T18:07:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b0be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0b0c146c00 0x7f0b0c06f300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49518/0 (socket says 172.21.15.143:49518) 2024-01-15T18:07:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b0be00640 1 -- 172.21.15.143:0/4009917845 learned_addr learned my addr 172.21.15.143:0/4009917845 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b0b400640 1 --2- 172.21.15.143:0/4009917845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b0b400640 1 -- 172.21.15.143:0/4009917845 >> v1:172.21.15.204:6789/0 conn(0x7f0b0c0714a0 legacy=0x7f0b0c071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b0b400640 1 -- 172.21.15.143:0/4009917845 >> v1:172.21.15.143:6789/0 conn(0x7f0b0c146c00 legacy=0x7f0b0c06f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.092+0000 7f0b0b400640 1 -- 172.21.15.143:0/4009917845 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c06ee70 con 0x7f0b0c06a490 2024-01-15T18:07:11.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b0b400640 1 --2- 172.21.15.143:0/4009917845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c06a8f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0af40097f0 tx=0x7f0af401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=218eafb5b72a15b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:11.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b0aa00640 1 -- 172.21.15.143:0/4009917845 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40aa5f0 con 0x7f0b0c06a490 2024-01-15T18:07:11.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b0aa00640 1 -- 172.21.15.143:0/4009917845 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af40aa750 con 0x7f0b0c06a490 2024-01-15T18:07:11.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b0aa00640 1 -- 172.21.15.143:0/4009917845 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40aaae0 con 0x7f0b0c06a490 2024-01-15T18:07:11.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 -- 172.21.15.143:0/4009917845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 msgr2=0x7f0b0c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 --2- 172.21.15.143:0/4009917845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c06a8f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0af40097f0 tx=0x7f0af401c490 comp rx=0 tx=0).stop 2024-01-15T18:07:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 -- 172.21.15.143:0/4009917845 shutdown_connections 2024-01-15T18:07:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 --2- 172.21.15.143:0/4009917845 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c06a8f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 -- 172.21.15.143:0/4009917845 >> 172.21.15.143:0/4009917845 conn(0x7f0b0c0b1d50 msgr2=0x7f0b0c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 -- 172.21.15.143:0/4009917845 shutdown_connections 2024-01-15T18:07:11.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.096+0000 7f0b13000640 1 -- 172.21.15.143:0/4009917845 wait complete. 2024-01-15T18:07:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 Processor -- start 2024-01-15T18:07:11.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 -- start start 2024-01-15T18:07:11.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c13da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 0x7f0b0c15f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c146c00 0x7f0b0c15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c15ec80 con 0x7f0b0c0714a0 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0ad560 con 0x7f0b0c146c00 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0ad840 con 0x7f0b0c06a490 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 0x7f0b0c15f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 0x7f0b0c15f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58776/0 (socket says 172.21.15.143:58776) 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 -- 172.21.15.143:0/2104581955 learned_addr learned my addr 172.21.15.143:0/2104581955 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0be00640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b11000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c146c00 0x7f0b0c15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 -- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 msgr2=0x7f0b0c13da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c13da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 -- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c146c00 msgr2=0x7f0b0c15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c146c00 0x7f0b0c15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 -- 172.21.15.143:0/2104581955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af4009420 con 0x7f0b0c0714a0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b0b400640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 0x7f0b0c15f390 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0af40097f0 tx=0x7f0af401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40b4810 con 0x7f0b0c0714a0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af40b4990 con 0x7f0b0c0714a0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c06fb90 con 0x7f0b0c0714a0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40b4d40 con 0x7f0b0c0714a0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.100+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c0ade00 con 0x7f0b0c0714a0 2024-01-15T18:07:11.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.104+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c067c10 con 0x7f0b0c0714a0 2024-01-15T18:07:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.104+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f0af40be410 con 0x7f0b0c0714a0 2024-01-15T18:07:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.104+0000 7f0b03e00640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae4066b30 0x7f0ae4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.104+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f0af40a7070 con 0x7f0b0c0714a0 2024-01-15T18:07:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.104+0000 7f0b0be00640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae4066b30 0x7f0ae4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.104+0000 7f0b0be00640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae4066b30 0x7f0ae4068ff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005890 tx=0x7f0afc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.108+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0af40ec120 con 0x7f0b0c0714a0 2024-01-15T18:07:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:11.260+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0b0c0b29b0 con 0x7f0b0c0714a0 2024-01-15T18:07:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.104+0000 7f0b03e00640 1 -- 172.21.15.143:0/2104581955 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0af40ef740 con 0x7f0b0c0714a0 2024-01-15T18:07:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae4066b30 msgr2=0x7f0ae4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae4066b30 0x7f0ae4068ff0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005890 tx=0x7f0afc009040 comp rx=0 tx=0).stop 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 msgr2=0x7f0b0c15f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 0x7f0b0c15f390 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0af40097f0 tx=0x7f0af401acd0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 shutdown_connections 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae4066b30 0x7f0ae4068ff0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c146c00 0x7f0b0c15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0714a0 0x7f0b0c15f390 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 --2- 172.21.15.143:0/2104581955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c06a490 0x7f0b0c13da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 >> 172.21.15.143:0/2104581955 conn(0x7f0b0c0b1d50 msgr2=0x7f0b0c1414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 shutdown_connections 2024-01-15T18:07:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.108+0000 7f0b13000640 1 -- 172.21.15.143:0/2104581955 wait complete. 2024-01-15T18:07:12.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:07:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.280+0000 7f14f0e00640 1 Processor -- start 2024-01-15T18:07:12.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.280+0000 7f14f0e00640 1 -- start start 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.280+0000 7f14f0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13cf10 0x7f14ec13f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.280+0000 7f14f0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14ec13faf0 0x7f14ec0cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.280+0000 7f14f0e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec13ff50 con 0x7f14ec0cdc30 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14f0e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0ce470 con 0x7f14ec13faf0 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14f0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0ce750 con 0x7f14ec13cf10 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14ec13faf0 0x7f14ec0cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14ec13faf0 0x7f14ec0cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58786/0 (socket says 172.21.15.143:58786) 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 -- 172.21.15.143:0/3663095870 learned_addr learned my addr 172.21.15.143:0/3663095870 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 -- 172.21.15.143:0/3663095870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13cf10 msgr2=0x7f14ec13f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:12.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9e00640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13cf10 0x7f14ec13f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13cf10 0x7f14ec13f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 -- 172.21.15.143:0/3663095870 >> v1:172.21.15.204:6789/0 conn(0x7f14ec0cdc30 legacy=0x7f14ec15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 -- 172.21.15.143:0/3663095870 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0cea30 con 0x7f14ec13faf0 2024-01-15T18:07:12.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9e00640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13cf10 0x7f14ec13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:12.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e9400640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec13faf0 0x7f14ec0cd6f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009870 tx=0x7f14d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbd38f7a2dac56a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e8a00640 1 -- 172.21.15.143:0/3663095870 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40ab600 con 0x7f14ec13faf0 2024-01-15T18:07:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e8a00640 1 -- 172.21.15.143:0/3663095870 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d40a7050 con 0x7f14ec13faf0 2024-01-15T18:07:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.284+0000 7f14e8a00640 1 -- 172.21.15.143:0/3663095870 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40ab980 con 0x7f14ec13faf0 2024-01-15T18:07:12.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- 172.21.15.143:0/3663095870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec13faf0 msgr2=0x7f14ec0cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec13faf0 0x7f14ec0cd6f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009870 tx=0x7f14d401c470 comp rx=0 tx=0).stop 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- 172.21.15.143:0/3663095870 shutdown_connections 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec13faf0 0x7f14ec0cd6f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3663095870 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13cf10 0x7f14ec13f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- 172.21.15.143:0/3663095870 >> 172.21.15.143:0/3663095870 conn(0x7f14ec06a890 msgr2=0x7f14ec06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- 172.21.15.143:0/3663095870 shutdown_connections 2024-01-15T18:07:12.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- 172.21.15.143:0/3663095870 wait complete. 2024-01-15T18:07:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 Processor -- start 2024-01-15T18:07:12.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- start start 2024-01-15T18:07:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 0x7f14ec15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14ec13cf10 0x7f14ec15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13faf0 0x7f14ec13a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec13f3a0 con 0x7f14ec0cdc30 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1599f0 con 0x7f14ec13cf10 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.288+0000 7f14f0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec158de0 con 0x7f14ec13faf0 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 0x7f14ec15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 0x7f14ec15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58788/0 (socket says 172.21.15.143:58788) 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 -- 172.21.15.143:0/3851314994 learned_addr learned my addr 172.21.15.143:0/3851314994 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9400640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14ec13cf10 0x7f14ec15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14ea800640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13faf0 0x7f14ec13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 -- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13faf0 msgr2=0x7f14ec13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:12.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13faf0 0x7f14ec13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 -- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14ec13cf10 msgr2=0x7f14ec15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14ec13cf10 0x7f14ec15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 -- 172.21.15.143:0/3851314994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40094d0 con 0x7f14ec0cdc30 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e9e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 0x7f14ec15c5c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c7c0 tx=0x7f14dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc007a30 con 0x7f14ec0cdc30 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc007b90 con 0x7f14ec0cdc30 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec1590c0 con 0x7f14ec0cdc30 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc00ce70 con 0x7f14ec0cdc30 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec159680 con 0x7f14ec0cdc30 2024-01-15T18:07:12.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec067500 con 0x7f14ec0cdc30 2024-01-15T18:07:12.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.292+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f14dc0113f0 con 0x7f14ec0cdc30 2024-01-15T18:07:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.296+0000 7f14e2000640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14c4066b60 0x7f14c4069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.296+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f14dc084e50 con 0x7f14ec0cdc30 2024-01-15T18:07:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.296+0000 7f14e9400640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14c4066b60 0x7f14c4069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:12.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.296+0000 7f14e9400640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14c4066b60 0x7f14c4069020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f14d40ad000 tx=0x7f14d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:12.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.300+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14dc051de0 con 0x7f14ec0cdc30 2024-01-15T18:07:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:12.448+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f14ec0ae7f0 con 0x7f14ec0cdc30 2024-01-15T18:07:13.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.112+0000 7f14e2000640 1 -- 172.21.15.143:0/3851314994 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f14dc01e0d0 con 0x7f14ec0cdc30 2024-01-15T18:07:13.119 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:07:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.112+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14c4066b60 msgr2=0x7f14c4069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.112+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14c4066b60 0x7f14c4069020 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f14d40ad000 tx=0x7f14d4008040 comp rx=0 tx=0).stop 2024-01-15T18:07:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.112+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 msgr2=0x7f14ec15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.112+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 0x7f14ec15c5c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c7c0 tx=0x7f14dc00cc80 comp rx=0 tx=0).stop 2024-01-15T18:07:13.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 shutdown_connections 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14c4066b60 0x7f14c4069020 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14ec13faf0 0x7f14ec13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14ec13cf10 0x7f14ec15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 --2- 172.21.15.143:0/3851314994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14ec0cdc30 0x7f14ec15c5c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 >> 172.21.15.143:0/3851314994 conn(0x7f14ec06a890 msgr2=0x7f14ec0acd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 shutdown_connections 2024-01-15T18:07:13.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.116+0000 7f14f0e00640 1 -- 172.21.15.143:0/3851314994 wait complete. 2024-01-15T18:07:13.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:07:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 Processor -- start 2024-01-15T18:07:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 -- start start 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f48161880 0x7f9f480adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806b5a0 0x7f9f480c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f480cec40 con 0x7f9f4806ba30 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f48161c60 con 0x7f9f48161880 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480cef20 con 0x7f9f4806b5a0 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f48161880 0x7f9f480adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f48161880 0x7f9f480adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58796/0 (socket says 172.21.15.143:58796) 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 -- 172.21.15.143:0/3804253941 learned_addr learned my addr 172.21.15.143:0/3804253941 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:13.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4ca00640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806b5a0 0x7f9f480c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:13.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 -- 172.21.15.143:0/3804253941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806b5a0 msgr2=0x7f9f480c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:13.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806b5a0 0x7f9f480c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 -- 172.21.15.143:0/3804253941 >> v1:172.21.15.204:6789/0 conn(0x7f9f4806ba30 legacy=0x7f9f480cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:13.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4d400640 1 -- 172.21.15.143:0/3804253941 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480cf200 con 0x7f9f48161880 2024-01-15T18:07:13.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.296+0000 7f9f4ca00640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806b5a0 0x7f9f480c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:13.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4d400640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480adaf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009870 tx=0x7f9f3801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e0eca0ae1e42c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:13.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f47e00640 1 -- 172.21.15.143:0/3804253941 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380aa640 con 0x7f9f48161880 2024-01-15T18:07:13.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f47e00640 1 -- 172.21.15.143:0/3804253941 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f380aa7a0 con 0x7f9f48161880 2024-01-15T18:07:13.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f47e00640 1 -- 172.21.15.143:0/3804253941 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380aab30 con 0x7f9f48161880 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 -- 172.21.15.143:0/3804253941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 msgr2=0x7f9f480adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480adaf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009870 tx=0x7f9f3801c500 comp rx=0 tx=0).stop 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 -- 172.21.15.143:0/3804253941 shutdown_connections 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806b5a0 0x7f9f480c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/3804253941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480adaf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 -- 172.21.15.143:0/3804253941 >> 172.21.15.143:0/3804253941 conn(0x7f9f480d0ed0 msgr2=0x7f9f480d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:13.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 -- 172.21.15.143:0/3804253941 shutdown_connections 2024-01-15T18:07:13.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.300+0000 7f9f4fe00640 1 -- 172.21.15.143:0/3804253941 wait complete. 2024-01-15T18:07:13.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 Processor -- start 2024-01-15T18:07:13.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 -- start start 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4806b5a0 0x7f9f48157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806ba30 0x7f9f48155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815ab70 con 0x7f9f48161880 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815a1f0 con 0x7f9f4806b5a0 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480c78c0 con 0x7f9f4806ba30 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58812/0 (socket says 172.21.15.143:58812) 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 -- 172.21.15.143:0/923113999 learned_addr learned my addr 172.21.15.143:0/923113999 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4ca00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806ba30 0x7f9f48155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4d400640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4806b5a0 0x7f9f48157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 -- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806ba30 msgr2=0x7f9f48155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806ba30 0x7f9f48155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 -- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4806b5a0 msgr2=0x7f9f48157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:13.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4806b5a0 0x7f9f48157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:13.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 -- 172.21.15.143:0/923113999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f380094d0 con 0x7f9f48161880 2024-01-15T18:07:13.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f4de00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480c8de0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000c780 tx=0x7f9f4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.304+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004b70 con 0x7f9f48161880 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4000ce90 con 0x7f9f48161880 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f480d0980 con 0x7f9f48161880 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000a850 con 0x7f9f48161880 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4815a790 con 0x7f9f48161880 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4815acb0 con 0x7f9f48161880 2024-01-15T18:07:13.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f9f40006b60 con 0x7f9f48161880 2024-01-15T18:07:13.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f45600640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f20066b60 0x7f9f20069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:13.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f9f40084fe0 con 0x7f9f48161880 2024-01-15T18:07:13.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.308+0000 7f9f4d400640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f20066b60 0x7f9f20069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:13.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.312+0000 7f9f4d400640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f20066b60 0x7f9f20069020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9f380ae000 tx=0x7f9f380b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.312+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9f40052020 con 0x7f9f48161880 2024-01-15T18:07:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:13.492+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9f48067500 con 0x7f9f48161880 2024-01-15T18:07:14.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.112+0000 7f9f45600640 1 -- 172.21.15.143:0/923113999 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9f40055640 con 0x7f9f48161880 2024-01-15T18:07:14.121 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 6 and data pool 7 2024-01-15T18:07:14.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f20066b60 msgr2=0x7f9f20069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f20066b60 0x7f9f20069020 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9f380ae000 tx=0x7f9f380b5040 comp rx=0 tx=0).stop 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 msgr2=0x7f9f480c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480c8de0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000c780 tx=0x7f9f4000cc40 comp rx=0 tx=0).stop 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 shutdown_connections 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f20066b60 0x7f9f20069020 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f48161880 0x7f9f480c8de0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f4806ba30 0x7f9f48155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 --2- 172.21.15.143:0/923113999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f4806b5a0 0x7f9f48157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 >> 172.21.15.143:0/923113999 conn(0x7f9f480d0ed0 msgr2=0x7f9f480af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:14.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 shutdown_connections 2024-01-15T18:07:14.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.116+0000 7f9f4fe00640 1 -- 172.21.15.143:0/923113999 wait complete. 2024-01-15T18:07:14.153 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:14.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.280+0000 7f12bbc00640 1 Processor -- start 2024-01-15T18:07:14.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.280+0000 7f12bbc00640 1 -- start start 2024-01-15T18:07:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12bbc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12b4143780 0x7f12b40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12bbc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f12b40ce520 0x7f12b4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12bbc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40ceb50 con 0x7f12b40720f0 2024-01-15T18:07:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12bbc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b415c110 con 0x7f12b4143780 2024-01-15T18:07:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12bbc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b4072530 con 0x7f12b40ce520 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12b4143780 0x7f12b40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12b40720f0 0x7f12b415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49522/0 (socket says 172.21.15.143:49522) 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9c00640 1 -- 172.21.15.143:0/1348455316 learned_addr learned my addr 172.21.15.143:0/1348455316 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b3e00640 1 --2- 172.21.15.143:0/1348455316 >> v2:172.21.15.204:3300/0 conn(0x7f12b40ce520 0x7f12b4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9200640 1 -- 172.21.15.143:0/1348455316 >> v2:172.21.15.204:3300/0 conn(0x7f12b40ce520 msgr2=0x7f12b4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9200640 1 --2- 172.21.15.143:0/1348455316 >> v2:172.21.15.204:3300/0 conn(0x7f12b40ce520 0x7f12b4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9200640 1 -- 172.21.15.143:0/1348455316 >> v1:172.21.15.143:6789/0 conn(0x7f12b40720f0 legacy=0x7f12b415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9200640 1 -- 172.21.15.143:0/1348455316 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b415b7e0 con 0x7f12b4143780 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b3400640 1 -- 172.21.15.143:0/1348455316 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747358607 0 0) 0x7f12ac0014a0 con 0x7f12b40720f0 2024-01-15T18:07:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b3e00640 1 --2- 172.21.15.143:0/1348455316 >> v2:172.21.15.204:3300/0 conn(0x7f12b40ce520 0x7f12b4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.284+0000 7f12b9200640 1 --2- 172.21.15.143:0/1348455316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b4143780 0x7f12b40cdfe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f12b406beb0 tx=0x7f12a401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7668bdebd3af8e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12b3400640 1 -- 172.21.15.143:0/1348455316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40ab600 con 0x7f12b4143780 2024-01-15T18:07:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12b3400640 1 -- 172.21.15.143:0/1348455316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a40a7050 con 0x7f12b4143780 2024-01-15T18:07:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12b3400640 1 -- 172.21.15.143:0/1348455316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a40ab980 con 0x7f12b4143780 2024-01-15T18:07:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 -- 172.21.15.143:0/1348455316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b4143780 msgr2=0x7f12b40cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 --2- 172.21.15.143:0/1348455316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b4143780 0x7f12b40cdfe0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f12b406beb0 tx=0x7f12a401c460 comp rx=0 tx=0).stop 2024-01-15T18:07:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 -- 172.21.15.143:0/1348455316 shutdown_connections 2024-01-15T18:07:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 --2- 172.21.15.143:0/1348455316 >> v2:172.21.15.204:3300/0 conn(0x7f12b40ce520 0x7f12b4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 --2- 172.21.15.143:0/1348455316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b4143780 0x7f12b40cdfe0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 -- 172.21.15.143:0/1348455316 >> 172.21.15.143:0/1348455316 conn(0x7f12b406c3f0 msgr2=0x7f12b406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:14.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 -- 172.21.15.143:0/1348455316 shutdown_connections 2024-01-15T18:07:14.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 -- 172.21.15.143:0/1348455316 wait complete. 2024-01-15T18:07:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 Processor -- start 2024-01-15T18:07:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 -- start start 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.288+0000 7f12bbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12b40720f0 0x7f12b40b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 0x7f12b40b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4143780 0x7f12b4157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40b2140 con 0x7f12b40ce520 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40b2420 con 0x7f12b4143780 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40b2700 con 0x7f12b40720f0 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 0x7f12b40b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 0x7f12b40b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58828/0 (socket says 172.21.15.143:58828) 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 -- 172.21.15.143:0/4199902761 learned_addr learned my addr 172.21.15.143:0/4199902761 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b9200640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12b40720f0 0x7f12b40b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 -- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12b40720f0 msgr2=0x7f12b40b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b9c00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4143780 0x7f12b4157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12b40720f0 0x7f12b40b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 -- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4143780 msgr2=0x7f12b4157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4143780 0x7f12b4157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 -- 172.21.15.143:0/4199902761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a4009420 con 0x7f12b40ce520 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b9200640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12b40720f0 0x7f12b40b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b9c00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4143780 0x7f12b4157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b3e00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 0x7f12b40b3900 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f129c00c7c0 tx=0x7f129c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c007a30 con 0x7f12b40ce520 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c007b90 con 0x7f12b40ce520 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b40b29e0 con 0x7f12b40ce520 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c00ce70 con 0x7f12b40ce520 2024-01-15T18:07:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b40b3e40 con 0x7f12b40ce520 2024-01-15T18:07:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b40b2c20 con 0x7f12b40ce520 2024-01-15T18:07:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.292+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f129c0113f0 con 0x7f12b40ce520 2024-01-15T18:07:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.296+0000 7f12b0c00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f128c066b30 0x7f128c068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.296+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f129c084e00 con 0x7f12b40ce520 2024-01-15T18:07:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.296+0000 7f12b9200640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f128c066b30 0x7f128c068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.296+0000 7f12b9200640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f128c066b30 0x7f128c068ff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f12a40ad000 tx=0x7f12a40a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.300+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f12b40b2c20 con 0x7f12b40ce520 2024-01-15T18:07:14.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.448+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12b40b2280 con 0x7f12b40ce520 2024-01-15T18:07:14.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.448+0000 7f12b0c00640 1 -- 172.21.15.143:0/4199902761 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16605 (secure 0 0 0) 0x7f12b40b2280 con 0x7f12b40ce520 2024-01-15T18:07:14.458 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:14.458 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":45,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:13.499502+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:07:11.307827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-15T18:07:12.496377+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:14.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f128c066b30 msgr2=0x7f128c068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f128c066b30 0x7f128c068ff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f12a40ad000 tx=0x7f12a40a6000 comp rx=0 tx=0).stop 2024-01-15T18:07:14.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 msgr2=0x7f12b40b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 0x7f12b40b3900 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f129c00c7c0 tx=0x7f129c00cc80 comp rx=0 tx=0).stop 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 shutdown_connections 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f128c066b30 0x7f128c068ff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f12b4143780 0x7f12b4157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b40ce520 0x7f12b40b3900 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 --2- 172.21.15.143:0/4199902761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f12b40720f0 0x7f12b40b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 >> 172.21.15.143:0/4199902761 conn(0x7f12b406c3f0 msgr2=0x7f12b40ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 shutdown_connections 2024-01-15T18:07:14.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.452+0000 7f12bbc00640 1 -- 172.21.15.143:0/4199902761 wait complete. 2024-01-15T18:07:14.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:14.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.608+0000 7fb631c00640 1 Processor -- start 2024-01-15T18:07:14.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.608+0000 7fb631c00640 1 -- start start 2024-01-15T18:07:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb631c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb62c157300 0x7fb62c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb631c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb62c0b21d0 0x7fb62c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb631c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0b4d20 con 0x7fb62c0b48f0 2024-01-15T18:07:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb631c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c0b35a0 con 0x7fb62c0b21d0 2024-01-15T18:07:14.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb631c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c0b2a20 con 0x7fb62c157300 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb62c0b21d0 0x7fb62c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb62c0b21d0 0x7fb62c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58842/0 (socket says 172.21.15.143:58842) 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 -- 172.21.15.143:0/2811011079 learned_addr learned my addr 172.21.15.143:0/2811011079 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62aa00640 1 --2- 172.21.15.143:0/2811011079 >> v2:172.21.15.204:3300/0 conn(0x7fb62c157300 0x7fb62c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 -- 172.21.15.143:0/2811011079 >> v2:172.21.15.204:3300/0 conn(0x7fb62c157300 msgr2=0x7fb62c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 --2- 172.21.15.143:0/2811011079 >> v2:172.21.15.204:3300/0 conn(0x7fb62c157300 0x7fb62c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 -- 172.21.15.143:0/2811011079 >> v1:172.21.15.204:6789/0 conn(0x7fb62c0b48f0 legacy=0x7fb62c15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:14.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 -- 172.21.15.143:0/2811011079 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c0b2d00 con 0x7fb62c0b21d0 2024-01-15T18:07:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.612+0000 7fb62a000640 1 --2- 172.21.15.143:0/2811011079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c0b43b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0b4fe0 tx=0x7fb61401d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5c9695f3d39e49 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb629600640 1 -- 172.21.15.143:0/2811011079 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140a68f0 con 0x7fb62c0b21d0 2024-01-15T18:07:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb629600640 1 -- 172.21.15.143:0/2811011079 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6140a6a50 con 0x7fb62c0b21d0 2024-01-15T18:07:14.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb629600640 1 -- 172.21.15.143:0/2811011079 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140a6de0 con 0x7fb62c0b21d0 2024-01-15T18:07:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- 172.21.15.143:0/2811011079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 msgr2=0x7fb62c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 --2- 172.21.15.143:0/2811011079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c0b43b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb62c0b4fe0 tx=0x7fb61401d840 comp rx=0 tx=0).stop 2024-01-15T18:07:14.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- 172.21.15.143:0/2811011079 shutdown_connections 2024-01-15T18:07:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 --2- 172.21.15.143:0/2811011079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c0b43b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 --2- 172.21.15.143:0/2811011079 >> v2:172.21.15.204:3300/0 conn(0x7fb62c157300 0x7fb62c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- 172.21.15.143:0/2811011079 >> 172.21.15.143:0/2811011079 conn(0x7fb62c06c3f0 msgr2=0x7fb62c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- 172.21.15.143:0/2811011079 shutdown_connections 2024-01-15T18:07:14.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- 172.21.15.143:0/2811011079 wait complete. 2024-01-15T18:07:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 Processor -- start 2024-01-15T18:07:14.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- start start 2024-01-15T18:07:14.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c15e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb62c0b48f0 0x7fb62c152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb62c157300 0x7fb62c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c15f200 con 0x7fb62c0b21d0 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c06c7d0 con 0x7fb62c0b48f0 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.616+0000 7fb631c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c1525c0 con 0x7fb62c157300 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c15e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c15e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58854/0 (socket says 172.21.15.143:58854) 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 -- 172.21.15.143:0/3238343219 learned_addr learned my addr 172.21.15.143:0/3238343219 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62a000640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb62c0b48f0 0x7fb62c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62b400640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb62c157300 0x7fb62c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 -- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb62c157300 msgr2=0x7fb62c15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb62c157300 0x7fb62c15c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 -- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb62c0b48f0 msgr2=0x7fb62c152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb62c0b48f0 0x7fb62c152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 -- 172.21.15.143:0/3238343219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61400a670 con 0x7fb62c0b21d0 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62b400640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb62c157300 0x7fb62c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb62aa00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c15e940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00d5f0 tx=0x7fb61c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0125f0 con 0x7fb62c0b21d0 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c012750 con 0x7fb62c0b21d0 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c15ef20 con 0x7fb62c0b21d0 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012ae0 con 0x7fb62c0b21d0 2024-01-15T18:07:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c06cd00 con 0x7fb62c0b21d0 2024-01-15T18:07:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c067500 con 0x7fb62c0b21d0 2024-01-15T18:07:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.620+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fb61c012c40 con 0x7fb62c0b21d0 2024-01-15T18:07:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.624+0000 7fb622a00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb604066b60 0x7fb604069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.624+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb61c084d40 con 0x7fb62c0b21d0 2024-01-15T18:07:14.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.624+0000 7fb62a000640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb604066b60 0x7fb604069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.624+0000 7fb62a000640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb604066b60 0x7fb604069020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6140b0000 tx=0x7fb614008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.628+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb61c051b70 con 0x7fb62c0b21d0 2024-01-15T18:07:14.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.784+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb62c0b0880 con 0x7fb62c0b21d0 2024-01-15T18:07:14.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.784+0000 7fb622a00640 1 -- 172.21.15.143:0/3238343219 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16592 (secure 0 0 0) 0x7fb61c007380 con 0x7fb62c0b21d0 2024-01-15T18:07:14.793 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:14.793 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":46,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:14.533888+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:07:11.307827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-15T18:07:12.496377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb604066b60 msgr2=0x7fb604069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb604066b60 0x7fb604069020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6140b0000 tx=0x7fb614008040 comp rx=0 tx=0).stop 2024-01-15T18:07:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 msgr2=0x7fb62c15e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c15e940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00d5f0 tx=0x7fb61c00dab0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 shutdown_connections 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb604066b60 0x7fb604069020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb62c157300 0x7fb62c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb62c0b48f0 0x7fb62c152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 --2- 172.21.15.143:0/3238343219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb62c0b21d0 0x7fb62c15e940 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 >> 172.21.15.143:0/3238343219 conn(0x7fb62c06c3f0 msgr2=0x7fb62c0aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 shutdown_connections 2024-01-15T18:07:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.788+0000 7fb631c00640 1 -- 172.21.15.143:0/3238343219 wait complete. 2024-01-15T18:07:14.815 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:07:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 Processor -- start 2024-01-15T18:07:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 -- start start 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1900cde50 0x7fe1900d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1900c5600 0x7fe1900c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1900ce7c0 con 0x7fe1900cf410 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900cef50 con 0x7fe1900c5600 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe198400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1900ce290 con 0x7fe1900cde50 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1900c5600 0x7fe1900c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe1900c5600 0x7fe1900c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58862/0 (socket says 172.21.15.143:58862) 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 -- 172.21.15.143:0/3199493727 learned_addr learned my addr 172.21.15.143:0/3199493727 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe195000640 1 --2- 172.21.15.143:0/3199493727 >> v2:172.21.15.204:3300/0 conn(0x7fe1900cde50 0x7fe1900d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 -- 172.21.15.143:0/3199493727 >> v2:172.21.15.204:3300/0 conn(0x7fe1900cde50 msgr2=0x7fe1900d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 --2- 172.21.15.143:0/3199493727 >> v2:172.21.15.204:3300/0 conn(0x7fe1900cde50 0x7fe1900d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 -- 172.21.15.143:0/3199493727 >> v1:172.21.15.204:6789/0 conn(0x7fe1900cf410 legacy=0x7fe1900cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe196400640 1 -- 172.21.15.143:0/3199493727 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1900ce4e0 con 0x7fe1900c5600 2024-01-15T18:07:14.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.948+0000 7fe195000640 1 --2- 172.21.15.143:0/3199493727 >> v2:172.21.15.204:3300/0 conn(0x7fe1900cde50 0x7fe1900d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe196400640 1 --2- 172.21.15.143:0/3199493727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe1900c5a90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe190071070 tx=0x7fe18801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791a838b92fd33f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe18fe00640 1 -- 172.21.15.143:0/3199493727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880a5740 con 0x7fe1900c5600 2024-01-15T18:07:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe18fe00640 1 -- 172.21.15.143:0/3199493727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1880a58a0 con 0x7fe1900c5600 2024-01-15T18:07:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe18fe00640 1 -- 172.21.15.143:0/3199493727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1880a5c30 con 0x7fe1900c5600 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 -- 172.21.15.143:0/3199493727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 msgr2=0x7fe1900c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 --2- 172.21.15.143:0/3199493727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe1900c5a90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe190071070 tx=0x7fe18801c990 comp rx=0 tx=0).stop 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 -- 172.21.15.143:0/3199493727 shutdown_connections 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 --2- 172.21.15.143:0/3199493727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe1900c5a90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 --2- 172.21.15.143:0/3199493727 >> v2:172.21.15.204:3300/0 conn(0x7fe1900cde50 0x7fe1900d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 -- 172.21.15.143:0/3199493727 >> 172.21.15.143:0/3199493727 conn(0x7fe1900d0e30 msgr2=0x7fe1900d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 -- 172.21.15.143:0/3199493727 shutdown_connections 2024-01-15T18:07:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.952+0000 7fe198400640 1 -- 172.21.15.143:0/3199493727 wait complete. 2024-01-15T18:07:14.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 Processor -- start 2024-01-15T18:07:14.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- start start 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe190165800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1900cde50 0x7fe190163a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1900cf410 0x7fe190157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190164200 con 0x7fe1900c5600 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190163fd0 con 0x7fe1900cf410 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19006c150 con 0x7fe1900cde50 2024-01-15T18:07:14.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe190165800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe190165800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58878/0 (socket says 172.21.15.143:58878) 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 -- 172.21.15.143:0/347002073 learned_addr learned my addr 172.21.15.143:0/347002073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe196400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1900cf410 0x7fe190157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195000640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1900cde50 0x7fe190163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 -- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1900cde50 msgr2=0x7fe190163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1900cde50 0x7fe190163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 -- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1900cf410 msgr2=0x7fe190157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1900cf410 0x7fe190157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 -- 172.21.15.143:0/347002073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009b30 con 0x7fe1900c5600 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe195a00640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe190165800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b410 tx=0x7fe18000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180004020 con 0x7fe1900c5600 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe180004180 con 0x7fe1900c5600 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe190163850 con 0x7fe1900c5600 2024-01-15T18:07:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe180011630 con 0x7fe1900c5600 2024-01-15T18:07:14.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe190163570 con 0x7fe1900c5600 2024-01-15T18:07:14.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.956+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190067500 con 0x7fe1900c5600 2024-01-15T18:07:14.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.960+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fe180011790 con 0x7fe1900c5600 2024-01-15T18:07:14.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.960+0000 7fe18d600640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe168066b30 0x7fe168068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:14.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.960+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe180084550 con 0x7fe1900c5600 2024-01-15T18:07:14.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.960+0000 7fe195000640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe168066b30 0x7fe168068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:14.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.960+0000 7fe195000640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe168066b30 0x7fe168068ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe19015eb60 tx=0x7fe178008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:14.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:14.964+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe180051420 con 0x7fe1900c5600 2024-01-15T18:07:15.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:15.148+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe1900b6cb0 con 0x7fe1900c5600 2024-01-15T18:07:16.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.132+0000 7fe18d600640 1 -- 172.21.15.143:0/347002073 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe180054a40 con 0x7fe1900c5600 2024-01-15T18:07:16.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe168066b30 msgr2=0x7fe168068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe168066b30 0x7fe168068ff0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe19015eb60 tx=0x7fe178008040 comp rx=0 tx=0).stop 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 msgr2=0x7fe190165800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe190165800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b410 tx=0x7fe18000b8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 shutdown_connections 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe168066b30 0x7fe168068ff0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1900cf410 0x7fe190157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1900cde50 0x7fe190163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 --2- 172.21.15.143:0/347002073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1900c5600 0x7fe190165800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 >> 172.21.15.143:0/347002073 conn(0x7fe1900d0e30 msgr2=0x7fe19006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 shutdown_connections 2024-01-15T18:07:16.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.136+0000 7fe198400640 1 -- 172.21.15.143:0/347002073 wait complete. 2024-01-15T18:07:16.168 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:07:16.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:07:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.304+0000 7ff85d800640 1 Processor -- start 2024-01-15T18:07:16.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.304+0000 7ff85d800640 1 -- start start 2024-01-15T18:07:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff85d800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff858071bb0 0x7ff858072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff85d800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff85815c5a0 0x7ff85815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff85d800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8580ceb50 con 0x7ff858143780 2024-01-15T18:07:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff85d800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff858072570 con 0x7ff85815c5a0 2024-01-15T18:07:16.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff85d800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85815c110 con 0x7ff858071bb0 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff857400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff85815c5a0 0x7ff85815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff856a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff858143780 0x7ff8580cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49528/0 (socket says 172.21.15.143:49528) 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff856a00640 1 -- 172.21.15.143:0/3593403836 learned_addr learned my addr 172.21.15.143:0/3593403836 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff856000640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff858071bb0 0x7ff858072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff857400640 1 -- 172.21.15.143:0/3593403836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff858071bb0 msgr2=0x7ff858072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff857400640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff858071bb0 0x7ff858072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff857400640 1 -- 172.21.15.143:0/3593403836 >> v1:172.21.15.143:6789/0 conn(0x7ff858143780 legacy=0x7ff8580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff857400640 1 -- 172.21.15.143:0/3593403836 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85815b7e0 con 0x7ff85815c5a0 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff855600640 1 -- 172.21.15.143:0/3593403836 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359774599 0 0) 0x7ff848001120 con 0x7ff858143780 2024-01-15T18:07:16.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.308+0000 7ff856000640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff858071bb0 0x7ff858072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff857400640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff85815c5a0 0x7ff85815ca30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff850009d90 tx=0x7ff85001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc67ce1c1482193 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff855600640 1 -- 172.21.15.143:0/3593403836 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500a5a10 con 0x7ff85815c5a0 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff855600640 1 -- 172.21.15.143:0/3593403836 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8500a5b70 con 0x7ff85815c5a0 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff855600640 1 -- 172.21.15.143:0/3593403836 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500ad440 con 0x7ff85815c5a0 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 -- 172.21.15.143:0/3593403836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff85815c5a0 msgr2=0x7ff85815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff85815c5a0 0x7ff85815ca30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff850009d90 tx=0x7ff85001c950 comp rx=0 tx=0).stop 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 -- 172.21.15.143:0/3593403836 shutdown_connections 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff85815c5a0 0x7ff85815ca30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 --2- 172.21.15.143:0/3593403836 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff858071bb0 0x7ff858072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 -- 172.21.15.143:0/3593403836 >> 172.21.15.143:0/3593403836 conn(0x7ff85806c3f0 msgr2=0x7ff85806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 -- 172.21.15.143:0/3593403836 shutdown_connections 2024-01-15T18:07:16.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 -- 172.21.15.143:0/3593403836 wait complete. 2024-01-15T18:07:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 Processor -- start 2024-01-15T18:07:16.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.312+0000 7ff85d800640 1 -- start start 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 0x7ff85813fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff858143780 0x7ff85814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff85815c5a0 0x7ff8580b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858134550 con 0x7ff858071bb0 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858134830 con 0x7ff858143780 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff858134b10 con 0x7ff85815c5a0 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 0x7ff85813fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 0x7ff85813fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58900/0 (socket says 172.21.15.143:58900) 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 -- 172.21.15.143:0/3321034330 learned_addr learned my addr 172.21.15.143:0/3321034330 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856000640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff858143780 0x7ff85814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff857400640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff85815c5a0 0x7ff8580b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 -- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff85815c5a0 msgr2=0x7ff8580b2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff85815c5a0 0x7ff8580b2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 -- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff858143780 msgr2=0x7ff85814d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff858143780 0x7ff85814d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 -- 172.21.15.143:0/3321034330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850009a70 con 0x7ff858071bb0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856000640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff858143780 0x7ff85814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff857400640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff85815c5a0 0x7ff8580b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff856a00640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 0x7ff85813fd70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c780 tx=0x7ff84800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480072d0 con 0x7ff858071bb0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848007450 con 0x7ff858071bb0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff858134df0 con 0x7ff858071bb0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480047f0 con 0x7ff858071bb0 2024-01-15T18:07:16.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8581402b0 con 0x7ff858071bb0 2024-01-15T18:07:16.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff858135310 con 0x7ff858071bb0 2024-01-15T18:07:16.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.316+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7ff848004950 con 0x7ff858071bb0 2024-01-15T18:07:16.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.320+0000 7ff84ea00640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff830066b60 0x7ff830069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.320+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff848083980 con 0x7ff858071bb0 2024-01-15T18:07:16.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.320+0000 7ff856000640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff830066b60 0x7ff830069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.320+0000 7ff856000640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff830066b60 0x7ff830069020 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff85813fc50 tx=0x7ff840008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.324+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff858135310 con 0x7ff858071bb0 2024-01-15T18:07:16.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:16.504+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff8580ceb50 con 0x7ff858071bb0 2024-01-15T18:07:17.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.140+0000 7ff84ea00640 1 -- 172.21.15.143:0/3321034330 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8580ceb50 con 0x7ff858071bb0 2024-01-15T18:07:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff830066b60 msgr2=0x7ff830069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff830066b60 0x7ff830069020 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff85813fc50 tx=0x7ff840008040 comp rx=0 tx=0).stop 2024-01-15T18:07:17.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 msgr2=0x7ff85813fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 0x7ff85813fd70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c780 tx=0x7ff84800cc40 comp rx=0 tx=0).stop 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 shutdown_connections 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff830066b60 0x7ff830069020 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff85815c5a0 0x7ff8580b2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff858143780 0x7ff85814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 --2- 172.21.15.143:0/3321034330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff858071bb0 0x7ff85813fd70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 >> 172.21.15.143:0/3321034330 conn(0x7ff85806c3f0 msgr2=0x7ff8580acc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 shutdown_connections 2024-01-15T18:07:17.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.144+0000 7ff85d800640 1 -- 172.21.15.143:0/3321034330 wait complete. 2024-01-15T18:07:17.174 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:07:17.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:07:17.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 Processor -- start 2024-01-15T18:07:17.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 -- start start 2024-01-15T18:07:17.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9bd4141660 0x7f9bd40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9bd415ec80 0x7f9bd4141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd415fbb0 con 0x7f9bd415ec80 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd415f130 con 0x7f9bd4141660 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd8e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd415f310 con 0x7f9bd4142470 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9bd415ec80 0x7f9bd4141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9bd415ec80 0x7f9bd4141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47050/0 (socket says 172.21.15.143:47050) 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 -- 172.21.15.143:0/780309928 learned_addr learned my addr 172.21.15.143:0/780309928 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1e00640 1 --2- 172.21.15.143:0/780309928 >> v2:172.21.15.204:3300/0 conn(0x7f9bd4141660 0x7f9bd40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd2800640 1 --2- 172.21.15.143:0/780309928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 -- 172.21.15.143:0/780309928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 msgr2=0x7f9bd4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 --2- 172.21.15.143:0/780309928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 -- 172.21.15.143:0/780309928 >> v2:172.21.15.204:3300/0 conn(0x7f9bd4141660 msgr2=0x7f9bd40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 --2- 172.21.15.143:0/780309928 >> v2:172.21.15.204:3300/0 conn(0x7f9bd4141660 0x7f9bd40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.304+0000 7f9bd1400640 1 -- 172.21.15.143:0/780309928 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd415f5f0 con 0x7f9bd415ec80 2024-01-15T18:07:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd1400640 1 --2- 172.21.15.143:0/780309928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd415ec80 0x7f9bd4141f30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40b1aa0 tx=0x7f9bbc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc9b22d127ea01d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd0a00640 1 -- 172.21.15.143:0/780309928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0ac600 con 0x7f9bd415ec80 2024-01-15T18:07:17.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd0a00640 1 -- 172.21.15.143:0/780309928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bbc0a8050 con 0x7f9bd415ec80 2024-01-15T18:07:17.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd0a00640 1 -- 172.21.15.143:0/780309928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc0ac980 con 0x7f9bd415ec80 2024-01-15T18:07:17.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 -- 172.21.15.143:0/780309928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd415ec80 msgr2=0x7f9bd4141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/780309928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd415ec80 0x7f9bd4141f30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40b1aa0 tx=0x7f9bbc01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 -- 172.21.15.143:0/780309928 shutdown_connections 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/780309928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/780309928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd415ec80 0x7f9bd4141f30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/780309928 >> v2:172.21.15.204:3300/0 conn(0x7f9bd4141660 0x7f9bd40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 -- 172.21.15.143:0/780309928 >> 172.21.15.143:0/780309928 conn(0x7f9bd406c3f0 msgr2=0x7f9bd406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 -- 172.21.15.143:0/780309928 shutdown_connections 2024-01-15T18:07:17.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.308+0000 7f9bd8e00640 1 -- 172.21.15.143:0/780309928 wait complete. 2024-01-15T18:07:17.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 Processor -- start 2024-01-15T18:07:17.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 -- start start 2024-01-15T18:07:17.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 0x7f9bd4149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd415ec80 0x7f9bd413fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd4143560 con 0x7f9bd4141660 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd41583d0 con 0x7f9bd415ec80 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd41586b0 con 0x7f9bd4142470 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 0x7f9bd4149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 0x7f9bd4149730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47060/0 (socket says 172.21.15.143:47060) 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 -- 172.21.15.143:0/626221723 learned_addr learned my addr 172.21.15.143:0/626221723 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1400640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd2800640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd415ec80 0x7f9bd413fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 -- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 msgr2=0x7f9bd4157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 -- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd415ec80 msgr2=0x7f9bd413fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd415ec80 0x7f9bd413fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 -- 172.21.15.143:0/626221723 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0094d0 con 0x7f9bd4141660 2024-01-15T18:07:17.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd1e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 0x7f9bd4149730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400d800 tx=0x7f9bc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40096a0 con 0x7f9bd4141660 2024-01-15T18:07:17.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc4009800 con 0x7f9bd4141660 2024-01-15T18:07:17.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4009b70 con 0x7f9bd4141660 2024-01-15T18:07:17.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd40cd150 con 0x7f9bd4141660 2024-01-15T18:07:17.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.312+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd4158c70 con 0x7f9bd4141660 2024-01-15T18:07:17.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.316+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd4067500 con 0x7f9bd4141660 2024-01-15T18:07:17.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.316+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f9bc4009cd0 con 0x7f9bd4141660 2024-01-15T18:07:17.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.316+0000 7f9bca000640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bac066b60 0x7f9bac069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.316+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9bc4084ed0 con 0x7f9bd4141660 2024-01-15T18:07:17.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.316+0000 7f9bd1400640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bac066b60 0x7f9bac069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.316+0000 7f9bd1400640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bac066b60 0x7f9bac069020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0ae000 tx=0x7f9bbc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.320+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9bc4051da0 con 0x7f9bd4141660 2024-01-15T18:07:17.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bd406e2a0 con 0x7f9bd4141660 2024-01-15T18:07:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bca000640 1 -- 172.21.15.143:0/626221723 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f9bc40553c0 con 0x7f9bd4141660 2024-01-15T18:07:17.512 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:17.512 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2863517742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":2863517742},{"type":"v1","addr":"172.21.15.143:6837","nonce":2863517742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/4241131349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":4241131349},{"type":"v1","addr":"172.21.15.204:6835","nonce":4241131349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1888637866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1888637866},{"type":"v1","addr":"172.21.15.204:6834","nonce":1888637866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:07:13.499835+0000","modified":"2024-01-15T18:07:16.510525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/1835320705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1835320705},{"type":"v1","addr":"172.21.15.143:6836","nonce":1835320705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-15T18:07:17.512 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bac066b60 msgr2=0x7f9bac069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bac066b60 0x7f9bac069020 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc0ae000 tx=0x7f9bbc0a7000 comp rx=0 tx=0).stop 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 msgr2=0x7f9bd4149730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 0x7f9bd4149730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400d800 tx=0x7f9bc400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 shutdown_connections 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bac066b60 0x7f9bac069020 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd415ec80 0x7f9bd413fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd4142470 0x7f9bd4157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 --2- 172.21.15.143:0/626221723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd4141660 0x7f9bd4149730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 >> 172.21.15.143:0/626221723 conn(0x7f9bd406c3f0 msgr2=0x7f9bd40705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 shutdown_connections 2024-01-15T18:07:17.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.504+0000 7f9bd8e00640 1 -- 172.21.15.143:0/626221723 wait complete. 2024-01-15T18:07:17.534 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:17.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.660+0000 7fd415200640 1 Processor -- start 2024-01-15T18:07:17.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 -- start start 2024-01-15T18:07:17.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41015d130 0x7fd4100aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd41015d510 0x7fd41006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd41006a490 0x7fd41006a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41006ae40 con 0x7fd41015d510 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd410160eb0 con 0x7fd41006a490 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd415200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410161190 con 0x7fd41015d130 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd41015d510 0x7fd41006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd41015d510 0x7fd41006e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47068/0 (socket says 172.21.15.143:47068) 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 -- 172.21.15.143:0/148808052 learned_addr learned my addr 172.21.15.143:0/148808052 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40de00640 1 --2- 172.21.15.143:0/148808052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41015d130 0x7fd4100aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40e800640 1 --2- 172.21.15.143:0/148808052 >> v2:172.21.15.204:3300/0 conn(0x7fd41006a490 0x7fd41006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 -- 172.21.15.143:0/148808052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41015d130 msgr2=0x7fd4100aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 --2- 172.21.15.143:0/148808052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41015d130 0x7fd4100aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 -- 172.21.15.143:0/148808052 >> v2:172.21.15.204:3300/0 conn(0x7fd41006a490 msgr2=0x7fd41006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 --2- 172.21.15.143:0/148808052 >> v2:172.21.15.204:3300/0 conn(0x7fd41006a490 0x7fd41006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.664+0000 7fd40d400640 1 -- 172.21.15.143:0/148808052 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd410161470 con 0x7fd41015d510 2024-01-15T18:07:17.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd40d400640 1 --2- 172.21.15.143:0/148808052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd41006e060 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd4100b4f50 tx=0x7fd3f801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49fa22866225fac server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd40ca00640 1 -- 172.21.15.143:0/148808052 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80aa640 con 0x7fd41015d510 2024-01-15T18:07:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd40ca00640 1 -- 172.21.15.143:0/148808052 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f80aa7a0 con 0x7fd41015d510 2024-01-15T18:07:17.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd40ca00640 1 -- 172.21.15.143:0/148808052 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f80aab30 con 0x7fd41015d510 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 -- 172.21.15.143:0/148808052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 msgr2=0x7fd41006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 --2- 172.21.15.143:0/148808052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd41006e060 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd4100b4f50 tx=0x7fd3f801c500 comp rx=0 tx=0).stop 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 -- 172.21.15.143:0/148808052 shutdown_connections 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 --2- 172.21.15.143:0/148808052 >> v2:172.21.15.204:3300/0 conn(0x7fd41006a490 0x7fd41006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 --2- 172.21.15.143:0/148808052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd41006e060 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 --2- 172.21.15.143:0/148808052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41015d130 0x7fd4100aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 -- 172.21.15.143:0/148808052 >> 172.21.15.143:0/148808052 conn(0x7fd4100b1b90 msgr2=0x7fd4100b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 -- 172.21.15.143:0/148808052 shutdown_connections 2024-01-15T18:07:17.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.668+0000 7fd415200640 1 -- 172.21.15.143:0/148808052 wait complete. 2024-01-15T18:07:17.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 Processor -- start 2024-01-15T18:07:17.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- start start 2024-01-15T18:07:17.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41006a490 0x7fd410160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd41015d130 0x7fd4101400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd410154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410154560 con 0x7fd41015d510 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41014ff40 con 0x7fd41015d130 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41015de60 con 0x7fd41006a490 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd410154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd410154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47084/0 (socket says 172.21.15.143:47084) 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 -- 172.21.15.143:0/689492439 learned_addr learned my addr 172.21.15.143:0/689492439 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40de00640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41006a490 0x7fd410160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40d400640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd41015d130 0x7fd4101400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 -- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41006a490 msgr2=0x7fd410160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41006a490 0x7fd410160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 -- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd41015d130 msgr2=0x7fd4101400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd41015d130 0x7fd4101400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 -- 172.21.15.143:0/689492439 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80094d0 con 0x7fd41015d510 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd40e800640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd410154020 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd408007d20 tx=0x7fd40800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408014420 con 0x7fd41015d510 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080145a0 con 0x7fd41015d510 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41015f840 con 0x7fd41015d510 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408014950 con 0x7fd41015d510 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41015fe00 con 0x7fd41015d510 2024-01-15T18:07:17.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.672+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd410067500 con 0x7fd41015d510 2024-01-15T18:07:17.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.676+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fd408014ab0 con 0x7fd41015d510 2024-01-15T18:07:17.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.676+0000 7fd406000640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3e8066b30 0x7fd3e8068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.676+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd408084f40 con 0x7fd41015d510 2024-01-15T18:07:17.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.676+0000 7fd40de00640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3e8066b30 0x7fd3e8068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.676+0000 7fd40de00640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3e8066b30 0x7fd3e8068ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd4100b0700 tx=0x7fd400009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.680+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd408051ef0 con 0x7fd41015d510 2024-01-15T18:07:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.832+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4100b2180 con 0x7fd41015d510 2024-01-15T18:07:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.832+0000 7fd406000640 1 -- 172.21.15.143:0/689492439 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16592 (secure 0 0 0) 0x7fd408055510 con 0x7fd41015d510 2024-01-15T18:07:17.842 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:17.842 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":46,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:07:14.533888+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:07:11.307827+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-15T18:07:12.496377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3e8066b30 msgr2=0x7fd3e8068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3e8066b30 0x7fd3e8068ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd4100b0700 tx=0x7fd400009190 comp rx=0 tx=0).stop 2024-01-15T18:07:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 msgr2=0x7fd410154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd410154020 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd408007d20 tx=0x7fd40800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 shutdown_connections 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3e8066b30 0x7fd3e8068ff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd41015d510 0x7fd410154020 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd41015d130 0x7fd4101400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 --2- 172.21.15.143:0/689492439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd41006a490 0x7fd410160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 >> 172.21.15.143:0/689492439 conn(0x7fd4100b1b90 msgr2=0x7fd4100afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 shutdown_connections 2024-01-15T18:07:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.836+0000 7fd415200640 1 -- 172.21.15.143:0/689492439 wait complete. 2024-01-15T18:07:17.864 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:07:17.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:07:18.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 Processor -- start 2024-01-15T18:07:18.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 -- start start 2024-01-15T18:07:18.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f85740ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8574071d00 0x7f8574072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857415f310 con 0x7f857415ec80 2024-01-15T18:07:18.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857415fbb0 con 0x7f8574071d00 2024-01-15T18:07:18.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8574072690 con 0x7f8574141660 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8574071d00 0x7f8574072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8574071d00 0x7f8574072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47098/0 (socket says 172.21.15.143:47098) 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857aa00640 1 -- 172.21.15.143:0/2698957268 learned_addr learned my addr 172.21.15.143:0/2698957268 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.992+0000 7f857a000640 1 --2- 172.21.15.143:0/2698957268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f85740ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857aa00640 1 -- 172.21.15.143:0/2698957268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 msgr2=0x7f85740ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857aa00640 1 --2- 172.21.15.143:0/2698957268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f85740ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857aa00640 1 -- 172.21.15.143:0/2698957268 >> v1:172.21.15.143:6789/0 conn(0x7f857415ec80 legacy=0x7f8574141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857aa00640 1 -- 172.21.15.143:0/2698957268 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857415f5f0 con 0x7f8574071d00 2024-01-15T18:07:18.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857aa00640 1 --2- 172.21.15.143:0/2698957268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574072150 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f856c009e50 tx=0x7f856c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd54995a4522956 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f8578c00640 1 -- 172.21.15.143:0/2698957268 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c0a57d0 con 0x7f8574071d00 2024-01-15T18:07:18.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f8578c00640 1 -- 172.21.15.143:0/2698957268 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c0a5930 con 0x7f8574071d00 2024-01-15T18:07:18.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f8578c00640 1 -- 172.21.15.143:0/2698957268 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c0a5cc0 con 0x7f8574071d00 2024-01-15T18:07:18.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857ca00640 1 -- 172.21.15.143:0/2698957268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 msgr2=0x7f8574072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:17.996+0000 7f857ca00640 1 --2- 172.21.15.143:0/2698957268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574072150 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f856c009e50 tx=0x7f856c01ca10 comp rx=0 tx=0).stop 2024-01-15T18:07:18.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- 172.21.15.143:0/2698957268 shutdown_connections 2024-01-15T18:07:18.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 --2- 172.21.15.143:0/2698957268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574072150 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 --2- 172.21.15.143:0/2698957268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f85740ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- 172.21.15.143:0/2698957268 >> 172.21.15.143:0/2698957268 conn(0x7f857406c3f0 msgr2=0x7f857406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- 172.21.15.143:0/2698957268 shutdown_connections 2024-01-15T18:07:18.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- 172.21.15.143:0/2698957268 wait complete. 2024-01-15T18:07:18.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 Processor -- start 2024-01-15T18:07:18.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- start start 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f857415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f857415ec80 0x7f857413af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857415a180 con 0x7f8574071d00 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8574142980 con 0x7f857415ec80 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857415ab30 con 0x7f8574141660 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47112/0 (socket says 172.21.15.143:47112) 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 -- 172.21.15.143:0/1693353973 learned_addr learned my addr 172.21.15.143:0/1693353973 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 -- 172.21.15.143:0/1693353973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 msgr2=0x7f857415ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f857415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 -- 172.21.15.143:0/1693353973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f857415ec80 msgr2=0x7f857413af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f857415ec80 0x7f857413af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.000+0000 7f857a000640 1 -- 172.21.15.143:0/1693353973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c009b30 con 0x7f8574071d00 2024-01-15T18:07:18.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f857a000640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574159c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f856400e870 tx=0x7f856400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564009510 con 0x7f8574071d00 2024-01-15T18:07:18.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856400eec0 con 0x7f8574071d00 2024-01-15T18:07:18.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856400cb40 con 0x7f8574071d00 2024-01-15T18:07:18.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85740cd150 con 0x7f8574071d00 2024-01-15T18:07:18.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574143bf0 con 0x7f8574071d00 2024-01-15T18:07:18.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8574067500 con 0x7f8574071d00 2024-01-15T18:07:18.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f856400ccc0 con 0x7f8574071d00 2024-01-15T18:07:18.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8572000640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f854c066b30 0x7f854c068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8564084c60 con 0x7f8574071d00 2024-01-15T18:07:18.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.004+0000 7f8579600640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f854c066b30 0x7f854c068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.008+0000 7f8579600640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f854c066b30 0x7f854c068ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f855c000f70 tx=0x7f855c0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.008+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8564051bb0 con 0x7f8574071d00 2024-01-15T18:07:18.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.156+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85740ccfd0 con 0x7f854c066b30 2024-01-15T18:07:18.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.164+0000 7f8572000640 1 -- 172.21.15.143:0/1693353973 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273969 (secure 0 0 0) 0x7f855c005080 con 0x7f854c066b30 2024-01-15T18:07:18.176 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:18.180 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:07:18.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f854c066b30 msgr2=0x7f854c068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f854c066b30 0x7f854c068ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f855c000f70 tx=0x7f855c0079c0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 msgr2=0x7f8574159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574159c40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f856400e870 tx=0x7f856400ec70 comp rx=0 tx=0).stop 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 shutdown_connections 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f854c066b30 0x7f854c068ff0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f857415ec80 0x7f857413af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8574141660 0x7f857415ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 --2- 172.21.15.143:0/1693353973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8574071d00 0x7f8574159c40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 >> 172.21.15.143:0/1693353973 conn(0x7f857406c3f0 msgr2=0x7f857413f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 shutdown_connections 2024-01-15T18:07:18.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.168+0000 7f857ca00640 1 -- 172.21.15.143:0/1693353973 wait complete. 2024-01-15T18:07:18.197 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-01-15T18:07:16.650439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":254488,"kb_used_data":5000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720232,"statfs":{"total":773094113280,"available":772833517568,"internally_reserved":0,"allocated":5120000,"data_stored":2753120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.540655"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-01-15T18:07:08.556016+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:07:08.556016+0000","last_peered":"2024-01-15T18:07:08.556016+0000","last_clean":"2024-01-15T18:07:08.556016+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:07:08.556016+0000","last_undegraded":"2024-01-15T18:07:08.556016+0000","last_fullsized":"2024-01-15T18:07:08.556016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.71999999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77000000000000002,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.499},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1419999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.86099999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.747,"15min":0.76600000000000001},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91700000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.879},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72199999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92100000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.626},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.746},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.95399999999999996,"5min":1.107,"15min":1.107},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97799999999999998,"5min":1.401,"15min":1.401},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.745},{"interface":"front","average":{"1min":0.63,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.67000000000000004,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.76800000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.032,"5min":5.032,"15min":5.032},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69399999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.97799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.78300000000000003,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.752},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.82999999999999996,"15min":0.81399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.73299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46700000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90000000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94099999999999995,"5min":1.054,"15min":1.054},"last":0.60299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83999999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90400000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.62}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":1.218,"15min":1.218},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":1.179,"15min":1.179},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70599999999999996,"15min":0.72199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.52700000000000002,"15min":0.47099999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.47599999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.444,"5min":3.444,"15min":3.444},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.181,"5min":7.181,"15min":7.181},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.61299999999999999,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.248},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":1.087}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.81399999999999995,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":1.056},{"interface":"front","average":{"1min":0.995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":1.135}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.875,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":1.1990000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.47399999999999998,"15min":0.45200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.4880000000000004,"5min":5.4880000000000004,"15min":5.4880000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.65800000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.48399999999999999,"15min":0.46000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81200000000000006,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.874}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.72399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":3.2010000000000001,"5min":3.2010000000000001,"15min":3.2010000000000001},"last":0.80400000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.89500000000000002,"15min":0.876},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2570000000000001,"5min":4.532,"15min":4.532},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.0409999999999999,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.3610000000000002,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.92100000000000004,"15min":0.81299999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":21.416,"5min":21.416,"15min":21.416},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.101,"5min":0.90900000000000003,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.022},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.238,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.0209999999999999,"15min":0.98299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":12.997,"5min":12.997,"15min":12.997},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.123},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":13.054,"5min":13.054,"15min":13.054},"last":0.92600000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.2170000000000001,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.876,"5min":1.008,"15min":1.03},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.71399999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.7729999999999997,"5min":4.7729999999999997,"15min":4.7729999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.274,"5min":1.1060000000000001,"15min":1.0780000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":13.089,"5min":13.089,"15min":13.089},"last":0.88500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.79000000000000004,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.65300000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.81999999999999995,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.36499999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.85299999999999998,"15min":0.91400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98199999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.40500000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.88300000000000001,"15min":0.91800000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.92600000000000005,"15min":0.96999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1240000000000001,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91800000000000004,"15min":0.95399999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":4.056,"15min":4.056},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0529999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1399999999999999,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.026,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.95399999999999996,"15min":1},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93400000000000005,"5min":4.407,"15min":4.407},"last":0.56499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:07:18.202 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:07:18.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facb4c00640 1 Processor -- start 2024-01-15T18:07:18.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facb4c00640 1 -- start start 2024-01-15T18:07:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facb4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7facb0157300 0x7facb0159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facb4c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb00b2a20 con 0x7facb00b4ac0 2024-01-15T18:07:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facb4c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facb00b2d00 con 0x7facb00b21d0 2024-01-15T18:07:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facb4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facb00b35a0 con 0x7facb0157300 2024-01-15T18:07:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facae800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7facb00b4ac0 0x7facb0070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50046/0 (socket says 172.21.15.143:50046) 2024-01-15T18:07:18.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facae800640 1 -- 172.21.15.143:0/4218563178 learned_addr learned my addr 172.21.15.143:0/4218563178 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facade00640 1 --2- 172.21.15.143:0/4218563178 >> v2:172.21.15.204:3300/0 conn(0x7facb0157300 0x7facb0159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367872233 0 0) 0x7facb00b2a20 con 0x7facb00b4ac0 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac90003450 con 0x7facb00b4ac0 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885064450 0 0) 0x7facb00b2d00 con 0x7facb00b21d0 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb00b2a20 con 0x7facb00b21d0 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facade00640 1 -- 172.21.15.143:0/4218563178 >> v1:172.21.15.204:6789/0 conn(0x7facb00b21d0 legacy=0x7facb00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facade00640 1 -- 172.21.15.143:0/4218563178 >> v1:172.21.15.143:6789/0 conn(0x7facb00b4ac0 legacy=0x7facb0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.368+0000 7facade00640 1 -- 172.21.15.143:0/4218563178 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb00b2fe0 con 0x7facb0157300 2024-01-15T18:07:18.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facade00640 1 --2- 172.21.15.143:0/4218563178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb0157300 0x7facb0159b70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faca00097c0 tx=0x7faca001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1308b7bcf6c2ce9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00ab670 con 0x7facb0157300 2024-01-15T18:07:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00ab7d0 con 0x7facb0157300 2024-01-15T18:07:18.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facaca00640 1 -- 172.21.15.143:0/4218563178 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00abb40 con 0x7facb0157300 2024-01-15T18:07:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 -- 172.21.15.143:0/4218563178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb0157300 msgr2=0x7facb0159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 --2- 172.21.15.143:0/4218563178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb0157300 0x7facb0159b70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faca00097c0 tx=0x7faca001c450 comp rx=0 tx=0).stop 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 -- 172.21.15.143:0/4218563178 shutdown_connections 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 --2- 172.21.15.143:0/4218563178 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb0157300 0x7facb0159b70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 -- 172.21.15.143:0/4218563178 >> 172.21.15.143:0/4218563178 conn(0x7facb006c3f0 msgr2=0x7facb006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 -- 172.21.15.143:0/4218563178 shutdown_connections 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.372+0000 7facb4c00640 1 -- 172.21.15.143:0/4218563178 wait complete. 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 Processor -- start 2024-01-15T18:07:18.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 -- start start 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 0x7facb013a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb00b4ac0 0x7facb0152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb0157300 0x7facb0153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb01565c0 con 0x7facb00b21d0 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facb01568a0 con 0x7facb00b4ac0 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facb0156b80 con 0x7facb0157300 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 0x7facb013a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 0x7facb013a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47114/0 (socket says 172.21.15.143:47114) 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 -- 172.21.15.143:0/2101771191 learned_addr learned my addr 172.21.15.143:0/2101771191 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facad400640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb00b4ac0 0x7facb0152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facae800640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb0157300 0x7facb0153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 -- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb0157300 msgr2=0x7facb0153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb0157300 0x7facb0153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 -- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb00b4ac0 msgr2=0x7facb0152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb00b4ac0 0x7facb0152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 -- 172.21.15.143:0/2101771191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0009420 con 0x7facb00b21d0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facade00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 0x7facb013a430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faca00ad040 tx=0x7faca001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00bf610 con 0x7facb00b21d0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faca00bf770 con 0x7facb00b21d0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb0156e60 con 0x7facb00b21d0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faca00bfae0 con 0x7facb00b21d0 2024-01-15T18:07:18.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.376+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facb006da20 con 0x7facb00b21d0 2024-01-15T18:07:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.380+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facb0067500 con 0x7facb00b21d0 2024-01-15T18:07:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.380+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7faca00bfc40 con 0x7facb00b21d0 2024-01-15T18:07:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.380+0000 7faca6000640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fac88066b30 0x7fac88068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.380+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7facb00b27c0 con 0x7facb00b21d0 2024-01-15T18:07:18.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.380+0000 7facad400640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fac88066b30 0x7fac88068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.380+0000 7facad400640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fac88066b30 0x7fac88068ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac98007fd0 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.384+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faca00ec390 con 0x7facb00b21d0 2024-01-15T18:07:18.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.532+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facb006e4e0 con 0x7fac88066b30 2024-01-15T18:07:18.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.540+0000 7faca6000640 1 -- 172.21.15.143:0/2101771191 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273969 (secure 0 0 0) 0x7facb006e4e0 con 0x7fac88066b30 2024-01-15T18:07:18.550 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:18.552 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fac88066b30 msgr2=0x7fac88068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fac88066b30 0x7fac88068ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac98007fd0 tx=0x7fac98009040 comp rx=0 tx=0).stop 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 msgr2=0x7facb013a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 0x7facb013a430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faca00ad040 tx=0x7faca001ac90 comp rx=0 tx=0).stop 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 shutdown_connections 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fac88066b30 0x7fac88068ff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facb0157300 0x7facb0153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facb00b4ac0 0x7facb0152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 --2- 172.21.15.143:0/2101771191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facb00b21d0 0x7facb013a430 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 >> 172.21.15.143:0/2101771191 conn(0x7facb006c3f0 msgr2=0x7facb00b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 shutdown_connections 2024-01-15T18:07:18.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.544+0000 7facb4c00640 1 -- 172.21.15.143:0/2101771191 wait complete. 2024-01-15T18:07:18.571 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-01-15T18:07:16.650439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":254488,"kb_used_data":5000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720232,"statfs":{"total":773094113280,"available":772833517568,"internally_reserved":0,"allocated":5120000,"data_stored":2753120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.540655"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:13.109262+0000","last_change":"2024-01-15T18:07:13.109262+0000","last_active":"2024-01-15T18:07:13.109262+0000","last_peered":"2024-01-15T18:07:13.109262+0000","last_clean":"2024-01-15T18:07:13.109262+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:13.109262+0000","last_undegraded":"2024-01-15T18:07:13.109262+0000","last_fullsized":"2024-01-15T18:07:13.109262+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:07:12.100516+0000","last_change":"2024-01-15T18:07:12.100516+0000","last_active":"2024-01-15T18:07:12.100516+0000","last_peered":"2024-01-15T18:07:12.100516+0000","last_clean":"2024-01-15T18:07:12.100516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:07:12.100516+0000","last_undegraded":"2024-01-15T18:07:12.100516+0000","last_fullsized":"2024-01-15T18:07:12.100516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-01-15T18:07:08.556016+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:07:08.556016+0000","last_peered":"2024-01-15T18:07:08.556016+0000","last_clean":"2024-01-15T18:07:08.556016+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:07:08.556016+0000","last_undegraded":"2024-01-15T18:07:08.556016+0000","last_fullsized":"2024-01-15T18:07:08.556016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.71999999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77000000000000002,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.499},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1419999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.86099999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.56999999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.747,"15min":0.76600000000000001},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.91700000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.879},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72199999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69599999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92100000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.626},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.746},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.95399999999999996,"5min":1.107,"15min":1.107},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97799999999999998,"5min":1.401,"15min":1.401},"last":0.77400000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.745},{"interface":"front","average":{"1min":0.63,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.67000000000000004,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.76800000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.032,"5min":5.032,"15min":5.032},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69399999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.97799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.78300000000000003,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.752},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.82999999999999996,"15min":0.81399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.73299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31676,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46700000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.51200000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90000000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94099999999999995,"5min":1.054,"15min":1.054},"last":0.60299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83999999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90400000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.62}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":1.218,"15min":1.218},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":1.179,"15min":1.179},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70599999999999996,"15min":0.72199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.52700000000000002,"15min":0.47099999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.47599999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.444,"5min":3.444,"15min":3.444},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.181,"5min":7.181,"15min":7.181},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.61299999999999999,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.65800000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.248},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":1.087}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.81399999999999995,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":1.056},{"interface":"front","average":{"1min":0.995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":1.135}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.875,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":1.1719999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":1.1990000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.47399999999999998,"15min":0.45200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.4880000000000004,"5min":5.4880000000000004,"15min":5.4880000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.65800000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.48399999999999999,"15min":0.46000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81200000000000006,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.874}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.72399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":3.2010000000000001,"5min":3.2010000000000001,"15min":3.2010000000000001},"last":0.80400000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.89500000000000002,"15min":0.876},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31660,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":491520,"data_stored":196548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2570000000000001,"5min":4.532,"15min":4.532},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.0409999999999999,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.3610000000000002,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.92100000000000004,"15min":0.81299999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":21.416,"5min":21.416,"15min":21.416},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.101,"5min":0.90900000000000003,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.022},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.238,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.0209999999999999,"15min":0.98299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":12.997,"5min":12.997,"15min":12.997},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.123},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":13.054,"5min":13.054,"15min":13.054},"last":0.92600000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.2170000000000001,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.876,"5min":1.008,"15min":1.03},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.71399999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.7729999999999997,"5min":4.7729999999999997,"15min":4.7729999999999997},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.274,"5min":1.1060000000000001,"15min":1.0780000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":13.089,"5min":13.089,"15min":13.089},"last":0.88500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1085440,"data_stored":786916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.79000000000000004,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.32600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.65300000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.81999999999999995,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.36499999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.85299999999999998,"15min":0.91400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98199999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.40500000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.88300000000000001,"15min":0.91800000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.92600000000000005,"15min":0.96999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1240000000000001,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91800000000000004,"15min":0.95399999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":4.056,"15min":4.056},"last":0.55600000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0529999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1399999999999999,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.026,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.95399999999999996,"15min":1},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93400000000000005,"5min":4.407,"15min":4.407},"last":0.56499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:07:18.577 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:07:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 Processor -- start 2024-01-15T18:07:18.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 -- start start 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54fc0d0150 0x7f54fc0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f54fc0b45d0 0x7f54fc0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0cefa0 con 0x7f54fc0b3610 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0c9fa0 con 0x7f54fc0d0150 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5503400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0b3ef0 con 0x7f54fc0b45d0 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54fc0d0150 0x7f54fc0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54fc0d0150 0x7f54fc0c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47130/0 (socket says 172.21.15.143:47130) 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 -- 172.21.15.143:0/1920937162 learned_addr learned my addr 172.21.15.143:0/1920937162 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f5501400640 1 --2- 172.21.15.143:0/1920937162 >> v2:172.21.15.204:3300/0 conn(0x7f54fc0b45d0 0x7f54fc0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 -- 172.21.15.143:0/1920937162 >> v2:172.21.15.204:3300/0 conn(0x7f54fc0b45d0 msgr2=0x7f54fc0c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 --2- 172.21.15.143:0/1920937162 >> v2:172.21.15.204:3300/0 conn(0x7f54fc0b45d0 0x7f54fc0c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 -- 172.21.15.143:0/1920937162 >> v1:172.21.15.204:6789/0 conn(0x7f54fc0b3610 legacy=0x7f54fc0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.744+0000 7f54fbe00640 1 -- 172.21.15.143:0/1920937162 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0d0f90 con 0x7f54fc0d0150 2024-01-15T18:07:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f54fbe00640 1 --2- 172.21.15.143:0/1920937162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0d0150 0x7f54fc0c6b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f54fc08b300 tx=0x7f54e401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac7b7acd0bab742 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f54fb400640 1 -- 172.21.15.143:0/1920937162 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40a89f0 con 0x7f54fc0d0150 2024-01-15T18:07:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f54fb400640 1 -- 172.21.15.143:0/1920937162 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e40a8b50 con 0x7f54fc0d0150 2024-01-15T18:07:18.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f54fb400640 1 -- 172.21.15.143:0/1920937162 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40b0440 con 0x7f54fc0d0150 2024-01-15T18:07:18.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 -- 172.21.15.143:0/1920937162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0d0150 msgr2=0x7f54fc0c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 --2- 172.21.15.143:0/1920937162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0d0150 0x7f54fc0c6b80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f54fc08b300 tx=0x7f54e401ecf0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 -- 172.21.15.143:0/1920937162 shutdown_connections 2024-01-15T18:07:18.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 --2- 172.21.15.143:0/1920937162 >> v2:172.21.15.204:3300/0 conn(0x7f54fc0b45d0 0x7f54fc0c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 --2- 172.21.15.143:0/1920937162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0d0150 0x7f54fc0c6b80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 -- 172.21.15.143:0/1920937162 >> 172.21.15.143:0/1920937162 conn(0x7f54fc0d3030 msgr2=0x7f54fc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 -- 172.21.15.143:0/1920937162 shutdown_connections 2024-01-15T18:07:18.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.748+0000 7f5503400640 1 -- 172.21.15.143:0/1920937162 wait complete. 2024-01-15T18:07:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 Processor -- start 2024-01-15T18:07:18.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 -- start start 2024-01-15T18:07:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 0x7f54fc1f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54fc0b45d0 0x7f54fc156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc0d0150 0x7f54fc0cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc15ba90 con 0x7f54fc0b3610 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1f7530 con 0x7f54fc0d0150 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1f6c90 con 0x7f54fc0b45d0 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 0x7f54fc1f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 0x7f54fc1f66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47132/0 (socket says 172.21.15.143:47132) 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5501400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc0d0150 0x7f54fc0cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 -- 172.21.15.143:0/1976083286 learned_addr learned my addr 172.21.15.143:0/1976083286 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:18.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f54fbe00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54fc0b45d0 0x7f54fc156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 -- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54fc0b45d0 msgr2=0x7f54fc156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54fc0b45d0 0x7f54fc156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 -- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc0d0150 msgr2=0x7f54fc0cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc0d0150 0x7f54fc0cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 -- 172.21.15.143:0/1976083286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40094d0 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5501400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc0d0150 0x7f54fc0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5500a00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 0x7f54fc1f66b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00c500 tx=0x7f54ec00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec005220 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec005380 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc1f6f70 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec007270 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.752+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc156e50 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.756+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc067500 con 0x7f54fc0b3610 2024-01-15T18:07:18.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.756+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f54ec0073d0 con 0x7f54fc0b3610 2024-01-15T18:07:18.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.756+0000 7f54f8c00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54d4066b60 0x7f54d4069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.756+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f54ec084080 con 0x7f54fc0b3610 2024-01-15T18:07:18.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.756+0000 7f54fbe00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54d4066b60 0x7f54d4069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.756+0000 7f54fbe00640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54d4066b60 0x7f54d4069020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f54e40b2000 tx=0x7f54e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.760+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54ec0b5280 con 0x7f54fc0b3610 2024-01-15T18:07:18.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.908+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54fc08d110 con 0x7f54d4066b60 2024-01-15T18:07:18.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.916+0000 7f54f8c00640 1 -- 172.21.15.143:0/1976083286 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288700 (secure 0 0 0) 0x7f54fc08d110 con 0x7f54d4066b60 2024-01-15T18:07:18.924 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:07:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.916+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54d4066b60 msgr2=0x7f54d4069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.916+0000 7f5503400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54d4066b60 0x7f54d4069020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f54e40b2000 tx=0x7f54e4009040 comp rx=0 tx=0).stop 2024-01-15T18:07:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 msgr2=0x7f54fc1f66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 0x7f54fc1f66b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00c500 tx=0x7f54ec00c9c0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 shutdown_connections 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54d4066b60 0x7f54d4069020 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54fc0d0150 0x7f54fc0cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54fc0b45d0 0x7f54fc156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 --2- 172.21.15.143:0/1976083286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54fc0b3610 0x7f54fc1f66b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 >> 172.21.15.143:0/1976083286 conn(0x7f54fc0d3030 msgr2=0x7f54fc0c6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 shutdown_connections 2024-01-15T18:07:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:18.920+0000 7f5503400640 1 -- 172.21.15.143:0/1976083286 wait complete. 2024-01-15T18:07:18.947 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-01-15T18:07:18.650776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254920,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719800,"statfs":{"total":773094113280,"available":772833075200,"internally_reserved":0,"allocated":5578752,"data_stored":2960108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000992"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775090+0000","last_change":"2024-01-15T18:07:13.565379+0000","last_active":"2024-01-15T18:07:14.775090+0000","last_peered":"2024-01-15T18:07:14.775090+0000","last_clean":"2024-01-15T18:07:14.775090+0000","last_became_active":"2024-01-15T18:07:13.565044+0000","last_became_peered":"2024-01-15T18:07:13.565044+0000","last_unstale":"2024-01-15T18:07:14.775090+0000","last_undegraded":"2024-01-15T18:07:14.775090+0000","last_fullsized":"2024-01-15T18:07:14.775090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:42.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543575+0000","last_change":"2024-01-15T18:07:13.566360+0000","last_active":"2024-01-15T18:07:14.543575+0000","last_peered":"2024-01-15T18:07:14.543575+0000","last_clean":"2024-01-15T18:07:14.543575+0000","last_became_active":"2024-01-15T18:07:13.566039+0000","last_became_peered":"2024-01-15T18:07:13.566039+0000","last_unstale":"2024-01-15T18:07:14.543575+0000","last_undegraded":"2024-01-15T18:07:14.543575+0000","last_fullsized":"2024-01-15T18:07:14.543575+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:20.336277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542050+0000","last_change":"2024-01-15T18:07:13.552196+0000","last_active":"2024-01-15T18:07:14.542050+0000","last_peered":"2024-01-15T18:07:14.542050+0000","last_clean":"2024-01-15T18:07:14.542050+0000","last_became_active":"2024-01-15T18:07:13.551879+0000","last_became_peered":"2024-01-15T18:07:13.551879+0000","last_unstale":"2024-01-15T18:07:14.542050+0000","last_undegraded":"2024-01-15T18:07:14.542050+0000","last_fullsized":"2024-01-15T18:07:14.542050+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:53.079369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774299+0000","last_change":"2024-01-15T18:07:13.547096+0000","last_active":"2024-01-15T18:07:14.774299+0000","last_peered":"2024-01-15T18:07:14.774299+0000","last_clean":"2024-01-15T18:07:14.774299+0000","last_became_active":"2024-01-15T18:07:13.546783+0000","last_became_peered":"2024-01-15T18:07:13.546783+0000","last_unstale":"2024-01-15T18:07:14.774299+0000","last_undegraded":"2024-01-15T18:07:14.774299+0000","last_fullsized":"2024-01-15T18:07:14.774299+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:08.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543306+0000","last_change":"2024-01-15T18:07:13.559790+0000","last_active":"2024-01-15T18:07:14.543306+0000","last_peered":"2024-01-15T18:07:14.543306+0000","last_clean":"2024-01-15T18:07:14.543306+0000","last_became_active":"2024-01-15T18:07:13.559024+0000","last_became_peered":"2024-01-15T18:07:13.559024+0000","last_unstale":"2024-01-15T18:07:14.543306+0000","last_undegraded":"2024-01-15T18:07:14.543306+0000","last_fullsized":"2024-01-15T18:07:14.543306+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:19.613876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774435+0000","last_change":"2024-01-15T18:07:13.557998+0000","last_active":"2024-01-15T18:07:14.774435+0000","last_peered":"2024-01-15T18:07:14.774435+0000","last_clean":"2024-01-15T18:07:14.774435+0000","last_became_active":"2024-01-15T18:07:13.557712+0000","last_became_peered":"2024-01-15T18:07:13.557712+0000","last_unstale":"2024-01-15T18:07:14.774435+0000","last_undegraded":"2024-01-15T18:07:14.774435+0000","last_fullsized":"2024-01-15T18:07:14.774435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:46.069477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774823+0000","last_change":"2024-01-15T18:07:13.567253+0000","last_active":"2024-01-15T18:07:14.774823+0000","last_peered":"2024-01-15T18:07:14.774823+0000","last_clean":"2024-01-15T18:07:14.774823+0000","last_became_active":"2024-01-15T18:07:13.566953+0000","last_became_peered":"2024-01-15T18:07:13.566953+0000","last_unstale":"2024-01-15T18:07:14.774823+0000","last_undegraded":"2024-01-15T18:07:14.774823+0000","last_fullsized":"2024-01-15T18:07:14.774823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:44.412896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774145+0000","last_change":"2024-01-15T18:07:13.545892+0000","last_active":"2024-01-15T18:07:14.774145+0000","last_peered":"2024-01-15T18:07:14.774145+0000","last_clean":"2024-01-15T18:07:14.774145+0000","last_became_active":"2024-01-15T18:07:13.545624+0000","last_became_peered":"2024-01-15T18:07:13.545624+0000","last_unstale":"2024-01-15T18:07:14.774145+0000","last_undegraded":"2024-01-15T18:07:14.774145+0000","last_fullsized":"2024-01-15T18:07:14.774145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:53.079369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774766+0000","last_change":"2024-01-15T18:07:13.565934+0000","last_active":"2024-01-15T18:07:14.774766+0000","last_peered":"2024-01-15T18:07:14.774766+0000","last_clean":"2024-01-15T18:07:14.774766+0000","last_became_active":"2024-01-15T18:07:13.565348+0000","last_became_peered":"2024-01-15T18:07:13.565348+0000","last_unstale":"2024-01-15T18:07:14.774766+0000","last_undegraded":"2024-01-15T18:07:14.774766+0000","last_fullsized":"2024-01-15T18:07:14.774766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:19.613876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774608+0000","last_change":"2024-01-15T18:07:13.545220+0000","last_active":"2024-01-15T18:07:14.774608+0000","last_peered":"2024-01-15T18:07:14.774608+0000","last_clean":"2024-01-15T18:07:14.774608+0000","last_became_active":"2024-01-15T18:07:13.544957+0000","last_became_peered":"2024-01-15T18:07:13.544957+0000","last_unstale":"2024-01-15T18:07:14.774608+0000","last_undegraded":"2024-01-15T18:07:14.774608+0000","last_fullsized":"2024-01-15T18:07:14.774608+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:58.216097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774680+0000","last_change":"2024-01-15T18:07:13.565635+0000","last_active":"2024-01-15T18:07:14.774680+0000","last_peered":"2024-01-15T18:07:14.774680+0000","last_clean":"2024-01-15T18:07:14.774680+0000","last_became_active":"2024-01-15T18:07:13.565356+0000","last_became_peered":"2024-01-15T18:07:13.565356+0000","last_unstale":"2024-01-15T18:07:14.774680+0000","last_undegraded":"2024-01-15T18:07:14.774680+0000","last_fullsized":"2024-01-15T18:07:14.774680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:15.249248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543524+0000","last_change":"2024-01-15T18:07:13.562099+0000","last_active":"2024-01-15T18:07:14.543524+0000","last_peered":"2024-01-15T18:07:14.543524+0000","last_clean":"2024-01-15T18:07:14.543524+0000","last_became_active":"2024-01-15T18:07:13.561790+0000","last_became_peered":"2024-01-15T18:07:13.561790+0000","last_unstale":"2024-01-15T18:07:14.543524+0000","last_undegraded":"2024-01-15T18:07:14.543524+0000","last_fullsized":"2024-01-15T18:07:14.543524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:43.794608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773525+0000","last_change":"2024-01-15T18:07:13.560152+0000","last_active":"2024-01-15T18:07:14.773525+0000","last_peered":"2024-01-15T18:07:14.773525+0000","last_clean":"2024-01-15T18:07:14.773525+0000","last_became_active":"2024-01-15T18:07:13.559682+0000","last_became_peered":"2024-01-15T18:07:13.559682+0000","last_unstale":"2024-01-15T18:07:14.773525+0000","last_undegraded":"2024-01-15T18:07:14.773525+0000","last_fullsized":"2024-01-15T18:07:14.773525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:14:35.267226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773436+0000","last_change":"2024-01-15T18:07:13.562323+0000","last_active":"2024-01-15T18:07:14.773436+0000","last_peered":"2024-01-15T18:07:14.773436+0000","last_clean":"2024-01-15T18:07:14.773436+0000","last_became_active":"2024-01-15T18:07:13.562052+0000","last_became_peered":"2024-01-15T18:07:13.562052+0000","last_unstale":"2024-01-15T18:07:14.773436+0000","last_undegraded":"2024-01-15T18:07:14.773436+0000","last_fullsized":"2024-01-15T18:07:14.773436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:15.081589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774888+0000","last_change":"2024-01-15T18:07:13.555476+0000","last_active":"2024-01-15T18:07:14.774888+0000","last_peered":"2024-01-15T18:07:14.774888+0000","last_clean":"2024-01-15T18:07:14.774888+0000","last_became_active":"2024-01-15T18:07:13.555192+0000","last_became_peered":"2024-01-15T18:07:13.555192+0000","last_unstale":"2024-01-15T18:07:14.774888+0000","last_undegraded":"2024-01-15T18:07:14.774888+0000","last_fullsized":"2024-01-15T18:07:14.774888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:50:27.459870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541451+0000","last_change":"2024-01-15T18:07:13.545439+0000","last_active":"2024-01-15T18:07:14.541451+0000","last_peered":"2024-01-15T18:07:14.541451+0000","last_clean":"2024-01-15T18:07:14.541451+0000","last_became_active":"2024-01-15T18:07:13.545109+0000","last_became_peered":"2024-01-15T18:07:13.545109+0000","last_unstale":"2024-01-15T18:07:14.541451+0000","last_undegraded":"2024-01-15T18:07:14.541451+0000","last_fullsized":"2024-01-15T18:07:14.541451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:12:37.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542612+0000","last_change":"2024-01-15T18:07:13.564080+0000","last_active":"2024-01-15T18:07:14.542612+0000","last_peered":"2024-01-15T18:07:14.542612+0000","last_clean":"2024-01-15T18:07:14.542612+0000","last_became_active":"2024-01-15T18:07:13.563671+0000","last_became_peered":"2024-01-15T18:07:13.563671+0000","last_unstale":"2024-01-15T18:07:14.542612+0000","last_undegraded":"2024-01-15T18:07:14.542612+0000","last_fullsized":"2024-01-15T18:07:14.542612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:58.216097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542189+0000","last_change":"2024-01-15T18:07:13.552275+0000","last_active":"2024-01-15T18:07:14.542189+0000","last_peered":"2024-01-15T18:07:14.542189+0000","last_clean":"2024-01-15T18:07:14.542189+0000","last_became_active":"2024-01-15T18:07:13.551396+0000","last_became_peered":"2024-01-15T18:07:13.551396+0000","last_unstale":"2024-01-15T18:07:14.542189+0000","last_undegraded":"2024-01-15T18:07:14.542189+0000","last_fullsized":"2024-01-15T18:07:14.542189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:08.081272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542176+0000","last_change":"2024-01-15T18:07:13.557559+0000","last_active":"2024-01-15T18:07:14.542176+0000","last_peered":"2024-01-15T18:07:14.542176+0000","last_clean":"2024-01-15T18:07:14.542176+0000","last_became_active":"2024-01-15T18:07:13.557217+0000","last_became_peered":"2024-01-15T18:07:13.557217+0000","last_unstale":"2024-01-15T18:07:14.542176+0000","last_undegraded":"2024-01-15T18:07:14.542176+0000","last_fullsized":"2024-01-15T18:07:14.542176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:51.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773926+0000","last_change":"2024-01-15T18:07:13.554980+0000","last_active":"2024-01-15T18:07:14.773926+0000","last_peered":"2024-01-15T18:07:14.773926+0000","last_clean":"2024-01-15T18:07:14.773926+0000","last_became_active":"2024-01-15T18:07:13.554690+0000","last_became_peered":"2024-01-15T18:07:13.554690+0000","last_unstale":"2024-01-15T18:07:14.773926+0000","last_undegraded":"2024-01-15T18:07:14.773926+0000","last_fullsized":"2024-01-15T18:07:14.773926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:37.290014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774535+0000","last_change":"2024-01-15T18:07:13.560084+0000","last_active":"2024-01-15T18:07:14.774535+0000","last_peered":"2024-01-15T18:07:14.774535+0000","last_clean":"2024-01-15T18:07:14.774535+0000","last_became_active":"2024-01-15T18:07:13.559818+0000","last_became_peered":"2024-01-15T18:07:13.559818+0000","last_unstale":"2024-01-15T18:07:14.774535+0000","last_undegraded":"2024-01-15T18:07:14.774535+0000","last_fullsized":"2024-01-15T18:07:14.774535+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:20.316030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773559+0000","last_change":"2024-01-15T18:07:13.558125+0000","last_active":"2024-01-15T18:07:14.773559+0000","last_peered":"2024-01-15T18:07:14.773559+0000","last_clean":"2024-01-15T18:07:14.773559+0000","last_became_active":"2024-01-15T18:07:13.557757+0000","last_became_peered":"2024-01-15T18:07:13.557757+0000","last_unstale":"2024-01-15T18:07:14.773559+0000","last_undegraded":"2024-01-15T18:07:14.773559+0000","last_fullsized":"2024-01-15T18:07:14.773559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:04.463119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541350+0000","last_change":"2024-01-15T18:07:13.560763+0000","last_active":"2024-01-15T18:07:14.541350+0000","last_peered":"2024-01-15T18:07:14.541350+0000","last_clean":"2024-01-15T18:07:14.541350+0000","last_became_active":"2024-01-15T18:07:13.560247+0000","last_became_peered":"2024-01-15T18:07:13.560247+0000","last_unstale":"2024-01-15T18:07:14.541350+0000","last_undegraded":"2024-01-15T18:07:14.541350+0000","last_fullsized":"2024-01-15T18:07:14.541350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:36:45.095291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541338+0000","last_change":"2024-01-15T18:07:13.554018+0000","last_active":"2024-01-15T18:07:14.541338+0000","last_peered":"2024-01-15T18:07:14.541338+0000","last_clean":"2024-01-15T18:07:14.541338+0000","last_became_active":"2024-01-15T18:07:13.553693+0000","last_became_peered":"2024-01-15T18:07:13.553693+0000","last_unstale":"2024-01-15T18:07:14.541338+0000","last_undegraded":"2024-01-15T18:07:14.541338+0000","last_fullsized":"2024-01-15T18:07:14.541338+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:08.982957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542676+0000","last_change":"2024-01-15T18:07:13.568220+0000","last_active":"2024-01-15T18:07:14.542676+0000","last_peered":"2024-01-15T18:07:14.542676+0000","last_clean":"2024-01-15T18:07:14.542676+0000","last_became_active":"2024-01-15T18:07:13.567961+0000","last_became_peered":"2024-01-15T18:07:13.567961+0000","last_unstale":"2024-01-15T18:07:14.542676+0000","last_undegraded":"2024-01-15T18:07:14.542676+0000","last_fullsized":"2024-01-15T18:07:14.542676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:19:58.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773932+0000","last_change":"2024-01-15T18:07:13.556487+0000","last_active":"2024-01-15T18:07:14.773932+0000","last_peered":"2024-01-15T18:07:14.773932+0000","last_clean":"2024-01-15T18:07:14.773932+0000","last_became_active":"2024-01-15T18:07:13.556178+0000","last_became_peered":"2024-01-15T18:07:13.556178+0000","last_unstale":"2024-01-15T18:07:14.773932+0000","last_undegraded":"2024-01-15T18:07:14.773932+0000","last_fullsized":"2024-01-15T18:07:14.773932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:56.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542225+0000","last_change":"2024-01-15T18:07:13.558444+0000","last_active":"2024-01-15T18:07:14.542225+0000","last_peered":"2024-01-15T18:07:14.542225+0000","last_clean":"2024-01-15T18:07:14.542225+0000","last_became_active":"2024-01-15T18:07:13.558068+0000","last_became_peered":"2024-01-15T18:07:13.558068+0000","last_unstale":"2024-01-15T18:07:14.542225+0000","last_undegraded":"2024-01-15T18:07:14.542225+0000","last_fullsized":"2024-01-15T18:07:14.542225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:06.472843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543524+0000","last_change":"2024-01-15T18:07:13.561182+0000","last_active":"2024-01-15T18:07:14.543524+0000","last_peered":"2024-01-15T18:07:14.543524+0000","last_clean":"2024-01-15T18:07:14.543524+0000","last_became_active":"2024-01-15T18:07:13.559200+0000","last_became_peered":"2024-01-15T18:07:13.559200+0000","last_unstale":"2024-01-15T18:07:14.543524+0000","last_undegraded":"2024-01-15T18:07:14.543524+0000","last_fullsized":"2024-01-15T18:07:14.543524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:22.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541598+0000","last_change":"2024-01-15T18:07:13.566721+0000","last_active":"2024-01-15T18:07:14.541598+0000","last_peered":"2024-01-15T18:07:14.541598+0000","last_clean":"2024-01-15T18:07:14.541598+0000","last_became_active":"2024-01-15T18:07:13.566446+0000","last_became_peered":"2024-01-15T18:07:13.566446+0000","last_unstale":"2024-01-15T18:07:14.541598+0000","last_undegraded":"2024-01-15T18:07:14.541598+0000","last_fullsized":"2024-01-15T18:07:14.541598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.108103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541736+0000","last_change":"2024-01-15T18:07:13.559670+0000","last_active":"2024-01-15T18:07:14.541736+0000","last_peered":"2024-01-15T18:07:14.541736+0000","last_clean":"2024-01-15T18:07:14.541736+0000","last_became_active":"2024-01-15T18:07:13.558831+0000","last_became_peered":"2024-01-15T18:07:13.558831+0000","last_unstale":"2024-01-15T18:07:14.541736+0000","last_undegraded":"2024-01-15T18:07:14.541736+0000","last_fullsized":"2024-01-15T18:07:14.541736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:27.810935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774974+0000","last_change":"2024-01-15T18:07:13.563324+0000","last_active":"2024-01-15T18:07:14.774974+0000","last_peered":"2024-01-15T18:07:14.774974+0000","last_clean":"2024-01-15T18:07:14.774974+0000","last_became_active":"2024-01-15T18:07:13.563088+0000","last_became_peered":"2024-01-15T18:07:13.563088+0000","last_unstale":"2024-01-15T18:07:14.774974+0000","last_undegraded":"2024-01-15T18:07:14.774974+0000","last_fullsized":"2024-01-15T18:07:14.774974+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:06.463252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773794+0000","last_change":"2024-01-15T18:07:13.558815+0000","last_active":"2024-01-15T18:07:14.773794+0000","last_peered":"2024-01-15T18:07:14.773794+0000","last_clean":"2024-01-15T18:07:14.773794+0000","last_became_active":"2024-01-15T18:07:13.558545+0000","last_became_peered":"2024-01-15T18:07:13.558545+0000","last_unstale":"2024-01-15T18:07:14.773794+0000","last_undegraded":"2024-01-15T18:07:14.773794+0000","last_fullsized":"2024-01-15T18:07:14.773794+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:55:13.674874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773433+0000","last_change":"2024-01-15T18:07:13.560097+0000","last_active":"2024-01-15T18:07:14.773433+0000","last_peered":"2024-01-15T18:07:14.773433+0000","last_clean":"2024-01-15T18:07:14.773433+0000","last_became_active":"2024-01-15T18:07:13.559817+0000","last_became_peered":"2024-01-15T18:07:13.559817+0000","last_unstale":"2024-01-15T18:07:14.773433+0000","last_undegraded":"2024-01-15T18:07:14.773433+0000","last_fullsized":"2024-01-15T18:07:14.773433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:19:58.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775400+0000","last_change":"2024-01-15T18:07:13.143155+0000","last_active":"2024-01-15T18:07:14.775400+0000","last_peered":"2024-01-15T18:07:14.775400+0000","last_clean":"2024-01-15T18:07:14.775400+0000","last_became_active":"2024-01-15T18:07:13.142887+0000","last_became_peered":"2024-01-15T18:07:13.142887+0000","last_unstale":"2024-01-15T18:07:14.775400+0000","last_undegraded":"2024-01-15T18:07:14.775400+0000","last_fullsized":"2024-01-15T18:07:14.775400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:48:17.532793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542751+0000","last_change":"2024-01-15T18:07:13.557581+0000","last_active":"2024-01-15T18:07:14.542751+0000","last_peered":"2024-01-15T18:07:14.542751+0000","last_clean":"2024-01-15T18:07:14.542751+0000","last_became_active":"2024-01-15T18:07:13.557104+0000","last_became_peered":"2024-01-15T18:07:13.557104+0000","last_unstale":"2024-01-15T18:07:14.542751+0000","last_undegraded":"2024-01-15T18:07:14.542751+0000","last_fullsized":"2024-01-15T18:07:14.542751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:20.253082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542255+0000","last_change":"2024-01-15T18:07:13.134532+0000","last_active":"2024-01-15T18:07:14.542255+0000","last_peered":"2024-01-15T18:07:14.542255+0000","last_clean":"2024-01-15T18:07:14.542255+0000","last_became_active":"2024-01-15T18:07:13.133873+0000","last_became_peered":"2024-01-15T18:07:13.133873+0000","last_unstale":"2024-01-15T18:07:14.542255+0000","last_undegraded":"2024-01-15T18:07:14.542255+0000","last_fullsized":"2024-01-15T18:07:14.542255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:24:53.119165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543674+0000","last_change":"2024-01-15T18:07:13.561484+0000","last_active":"2024-01-15T18:07:14.543674+0000","last_peered":"2024-01-15T18:07:14.543674+0000","last_clean":"2024-01-15T18:07:14.543674+0000","last_became_active":"2024-01-15T18:07:13.561171+0000","last_became_peered":"2024-01-15T18:07:13.561171+0000","last_unstale":"2024-01-15T18:07:14.543674+0000","last_undegraded":"2024-01-15T18:07:14.543674+0000","last_fullsized":"2024-01-15T18:07:14.543674+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:48:18.541539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774323+0000","last_change":"2024-01-15T18:07:13.141848+0000","last_active":"2024-01-15T18:07:14.774323+0000","last_peered":"2024-01-15T18:07:14.774323+0000","last_clean":"2024-01-15T18:07:14.774323+0000","last_became_active":"2024-01-15T18:07:13.141503+0000","last_became_peered":"2024-01-15T18:07:13.141503+0000","last_unstale":"2024-01-15T18:07:14.774323+0000","last_undegraded":"2024-01-15T18:07:14.774323+0000","last_fullsized":"2024-01-15T18:07:14.774323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:50.037313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774114+0000","last_change":"2024-01-15T18:07:13.562984+0000","last_active":"2024-01-15T18:07:14.774114+0000","last_peered":"2024-01-15T18:07:14.774114+0000","last_clean":"2024-01-15T18:07:14.774114+0000","last_became_active":"2024-01-15T18:07:13.562555+0000","last_became_peered":"2024-01-15T18:07:13.562555+0000","last_unstale":"2024-01-15T18:07:14.774114+0000","last_undegraded":"2024-01-15T18:07:14.774114+0000","last_fullsized":"2024-01-15T18:07:14.774114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:55.672164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774796+0000","last_change":"2024-01-15T18:07:13.151750+0000","last_active":"2024-01-15T18:07:14.774796+0000","last_peered":"2024-01-15T18:07:14.774796+0000","last_clean":"2024-01-15T18:07:14.774796+0000","last_became_active":"2024-01-15T18:07:13.151403+0000","last_became_peered":"2024-01-15T18:07:13.151403+0000","last_unstale":"2024-01-15T18:07:14.774796+0000","last_undegraded":"2024-01-15T18:07:14.774796+0000","last_fullsized":"2024-01-15T18:07:14.774796+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:41.327706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542105+0000","last_change":"2024-01-15T18:07:13.561187+0000","last_active":"2024-01-15T18:07:14.542105+0000","last_peered":"2024-01-15T18:07:14.542105+0000","last_clean":"2024-01-15T18:07:14.542105+0000","last_became_active":"2024-01-15T18:07:13.559238+0000","last_became_peered":"2024-01-15T18:07:13.559238+0000","last_unstale":"2024-01-15T18:07:14.542105+0000","last_undegraded":"2024-01-15T18:07:14.542105+0000","last_fullsized":"2024-01-15T18:07:14.542105+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:42.336452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774648+0000","last_change":"2024-01-15T18:07:13.147475+0000","last_active":"2024-01-15T18:07:14.774648+0000","last_peered":"2024-01-15T18:07:14.774648+0000","last_clean":"2024-01-15T18:07:14.774648+0000","last_became_active":"2024-01-15T18:07:13.147200+0000","last_became_peered":"2024-01-15T18:07:13.147200+0000","last_unstale":"2024-01-15T18:07:14.774648+0000","last_undegraded":"2024-01-15T18:07:14.774648+0000","last_fullsized":"2024-01-15T18:07:14.774648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:19.327531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774252+0000","last_change":"2024-01-15T18:07:13.553134+0000","last_active":"2024-01-15T18:07:14.774252+0000","last_peered":"2024-01-15T18:07:14.774252+0000","last_clean":"2024-01-15T18:07:14.774252+0000","last_became_active":"2024-01-15T18:07:13.552899+0000","last_became_peered":"2024-01-15T18:07:13.552899+0000","last_unstale":"2024-01-15T18:07:14.774252+0000","last_undegraded":"2024-01-15T18:07:14.774252+0000","last_fullsized":"2024-01-15T18:07:14.774252+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:42.336452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541945+0000","last_change":"2024-01-15T18:07:13.141705+0000","last_active":"2024-01-15T18:07:14.541945+0000","last_peered":"2024-01-15T18:07:14.541945+0000","last_clean":"2024-01-15T18:07:14.541945+0000","last_became_active":"2024-01-15T18:07:13.141399+0000","last_became_peered":"2024-01-15T18:07:13.141399+0000","last_unstale":"2024-01-15T18:07:14.541945+0000","last_undegraded":"2024-01-15T18:07:14.541945+0000","last_fullsized":"2024-01-15T18:07:14.541945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:54.663418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773391+0000","last_change":"2024-01-15T18:07:13.545374+0000","last_active":"2024-01-15T18:07:14.773391+0000","last_peered":"2024-01-15T18:07:14.773391+0000","last_clean":"2024-01-15T18:07:14.773391+0000","last_became_active":"2024-01-15T18:07:13.545106+0000","last_became_peered":"2024-01-15T18:07:13.545106+0000","last_unstale":"2024-01-15T18:07:14.773391+0000","last_undegraded":"2024-01-15T18:07:14.773391+0000","last_fullsized":"2024-01-15T18:07:14.773391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:07:52.124299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543180+0000","last_change":"2024-01-15T18:07:13.143744+0000","last_active":"2024-01-15T18:07:14.543180+0000","last_peered":"2024-01-15T18:07:14.543180+0000","last_clean":"2024-01-15T18:07:14.543180+0000","last_became_active":"2024-01-15T18:07:13.143405+0000","last_became_peered":"2024-01-15T18:07:13.143405+0000","last_unstale":"2024-01-15T18:07:14.543180+0000","last_undegraded":"2024-01-15T18:07:14.543180+0000","last_fullsized":"2024-01-15T18:07:14.543180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:57.207351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541293+0000","last_change":"2024-01-15T18:07:13.560545+0000","last_active":"2024-01-15T18:07:14.541293+0000","last_peered":"2024-01-15T18:07:14.541293+0000","last_clean":"2024-01-15T18:07:14.541293+0000","last_became_active":"2024-01-15T18:07:13.559806+0000","last_became_peered":"2024-01-15T18:07:13.559806+0000","last_unstale":"2024-01-15T18:07:14.541293+0000","last_undegraded":"2024-01-15T18:07:14.541293+0000","last_fullsized":"2024-01-15T18:07:14.541293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:56.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773476+0000","last_change":"2024-01-15T18:07:13.141203+0000","last_active":"2024-01-15T18:07:14.773476+0000","last_peered":"2024-01-15T18:07:14.773476+0000","last_clean":"2024-01-15T18:07:14.773476+0000","last_became_active":"2024-01-15T18:07:13.140933+0000","last_became_peered":"2024-01-15T18:07:13.140933+0000","last_unstale":"2024-01-15T18:07:14.773476+0000","last_undegraded":"2024-01-15T18:07:14.773476+0000","last_fullsized":"2024-01-15T18:07:14.773476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:50.037313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542212+0000","last_change":"2024-01-15T18:07:13.552946+0000","last_active":"2024-01-15T18:07:14.542212+0000","last_peered":"2024-01-15T18:07:14.542212+0000","last_clean":"2024-01-15T18:07:14.542212+0000","last_became_active":"2024-01-15T18:07:13.550975+0000","last_became_peered":"2024-01-15T18:07:13.550975+0000","last_unstale":"2024-01-15T18:07:14.542212+0000","last_undegraded":"2024-01-15T18:07:14.542212+0000","last_fullsized":"2024-01-15T18:07:14.542212+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:08.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542648+0000","last_change":"2024-01-15T18:07:13.136136+0000","last_active":"2024-01-15T18:07:14.542648+0000","last_peered":"2024-01-15T18:07:14.542648+0000","last_clean":"2024-01-15T18:07:14.542648+0000","last_became_active":"2024-01-15T18:07:13.135415+0000","last_became_peered":"2024-01-15T18:07:13.135415+0000","last_unstale":"2024-01-15T18:07:14.542648+0000","last_undegraded":"2024-01-15T18:07:14.542648+0000","last_fullsized":"2024-01-15T18:07:14.542648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:07:51.115553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775144+0000","last_change":"2024-01-15T18:07:13.570342+0000","last_active":"2024-01-15T18:07:14.775144+0000","last_peered":"2024-01-15T18:07:14.775144+0000","last_clean":"2024-01-15T18:07:14.775144+0000","last_became_active":"2024-01-15T18:07:13.570076+0000","last_became_peered":"2024-01-15T18:07:13.570076+0000","last_unstale":"2024-01-15T18:07:14.775144+0000","last_undegraded":"2024-01-15T18:07:14.775144+0000","last_fullsized":"2024-01-15T18:07:14.775144+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:20.316030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774778+0000","last_change":"2024-01-15T18:07:13.144905+0000","last_active":"2024-01-15T18:07:14.774778+0000","last_peered":"2024-01-15T18:07:14.774778+0000","last_clean":"2024-01-15T18:07:14.774778+0000","last_became_active":"2024-01-15T18:07:13.144602+0000","last_became_peered":"2024-01-15T18:07:13.144602+0000","last_unstale":"2024-01-15T18:07:14.774778+0000","last_undegraded":"2024-01-15T18:07:14.774778+0000","last_fullsized":"2024-01-15T18:07:14.774778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:55.876186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775180+0000","last_change":"2024-01-15T18:07:13.560963+0000","last_active":"2024-01-15T18:07:14.775180+0000","last_peered":"2024-01-15T18:07:14.775180+0000","last_clean":"2024-01-15T18:07:14.775180+0000","last_became_active":"2024-01-15T18:07:13.560690+0000","last_became_peered":"2024-01-15T18:07:13.560690+0000","last_unstale":"2024-01-15T18:07:14.775180+0000","last_undegraded":"2024-01-15T18:07:14.775180+0000","last_fullsized":"2024-01-15T18:07:14.775180+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:08:08.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774970+0000","last_change":"2024-01-15T18:07:13.153076+0000","last_active":"2024-01-15T18:07:14.774970+0000","last_peered":"2024-01-15T18:07:14.774970+0000","last_clean":"2024-01-15T18:07:14.774970+0000","last_became_active":"2024-01-15T18:07:13.152768+0000","last_became_peered":"2024-01-15T18:07:13.152768+0000","last_unstale":"2024-01-15T18:07:14.774970+0000","last_undegraded":"2024-01-15T18:07:14.774970+0000","last_fullsized":"2024-01-15T18:07:14.774970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:21.659078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774770+0000","last_change":"2024-01-15T18:07:13.569527+0000","last_active":"2024-01-15T18:07:14.774770+0000","last_peered":"2024-01-15T18:07:14.774770+0000","last_clean":"2024-01-15T18:07:14.774770+0000","last_became_active":"2024-01-15T18:07:13.569076+0000","last_became_peered":"2024-01-15T18:07:13.569076+0000","last_unstale":"2024-01-15T18:07:14.774770+0000","last_undegraded":"2024-01-15T18:07:14.774770+0000","last_fullsized":"2024-01-15T18:07:14.774770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:06:15.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774015+0000","last_change":"2024-01-15T18:07:13.135598+0000","last_active":"2024-01-15T18:07:14.774015+0000","last_peered":"2024-01-15T18:07:14.774015+0000","last_clean":"2024-01-15T18:07:14.774015+0000","last_became_active":"2024-01-15T18:07:13.135334+0000","last_became_peered":"2024-01-15T18:07:13.135334+0000","last_unstale":"2024-01-15T18:07:14.774015+0000","last_undegraded":"2024-01-15T18:07:14.774015+0000","last_fullsized":"2024-01-15T18:07:14.774015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:19.099418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541680+0000","last_change":"2024-01-15T18:07:13.558361+0000","last_active":"2024-01-15T18:07:14.541680+0000","last_peered":"2024-01-15T18:07:14.541680+0000","last_clean":"2024-01-15T18:07:14.541680+0000","last_became_active":"2024-01-15T18:07:13.557736+0000","last_became_peered":"2024-01-15T18:07:13.557736+0000","last_unstale":"2024-01-15T18:07:14.541680+0000","last_undegraded":"2024-01-15T18:07:14.541680+0000","last_fullsized":"2024-01-15T18:07:14.541680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:11.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774691+0000","last_change":"2024-01-15T18:07:13.138892+0000","last_active":"2024-01-15T18:07:14.774691+0000","last_peered":"2024-01-15T18:07:14.774691+0000","last_clean":"2024-01-15T18:07:14.774691+0000","last_became_active":"2024-01-15T18:07:13.137384+0000","last_became_peered":"2024-01-15T18:07:13.137384+0000","last_unstale":"2024-01-15T18:07:14.774691+0000","last_undegraded":"2024-01-15T18:07:14.774691+0000","last_fullsized":"2024-01-15T18:07:14.774691+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:21.659078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774267+0000","last_change":"2024-01-15T18:07:13.555879+0000","last_active":"2024-01-15T18:07:14.774267+0000","last_peered":"2024-01-15T18:07:14.774267+0000","last_clean":"2024-01-15T18:07:14.774267+0000","last_became_active":"2024-01-15T18:07:13.555634+0000","last_became_peered":"2024-01-15T18:07:13.555634+0000","last_unstale":"2024-01-15T18:07:14.774267+0000","last_undegraded":"2024-01-15T18:07:14.774267+0000","last_fullsized":"2024-01-15T18:07:14.774267+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:08.081272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774706+0000","last_change":"2024-01-15T18:07:13.148916+0000","last_active":"2024-01-15T18:07:14.774706+0000","last_peered":"2024-01-15T18:07:14.774706+0000","last_clean":"2024-01-15T18:07:14.774706+0000","last_became_active":"2024-01-15T18:07:13.148524+0000","last_became_peered":"2024-01-15T18:07:13.148524+0000","last_unstale":"2024-01-15T18:07:14.774706+0000","last_undegraded":"2024-01-15T18:07:14.774706+0000","last_fullsized":"2024-01-15T18:07:14.774706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:18.605130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774437+0000","last_change":"2024-01-15T18:07:13.554144+0000","last_active":"2024-01-15T18:07:14.774437+0000","last_peered":"2024-01-15T18:07:14.774437+0000","last_clean":"2024-01-15T18:07:14.774437+0000","last_became_active":"2024-01-15T18:07:13.553831+0000","last_became_peered":"2024-01-15T18:07:13.553831+0000","last_unstale":"2024-01-15T18:07:14.774437+0000","last_undegraded":"2024-01-15T18:07:14.774437+0000","last_fullsized":"2024-01-15T18:07:14.774437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:42.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775054+0000","last_change":"2024-01-15T18:07:13.139960+0000","last_active":"2024-01-15T18:07:14.775054+0000","last_peered":"2024-01-15T18:07:14.775054+0000","last_clean":"2024-01-15T18:07:14.775054+0000","last_became_active":"2024-01-15T18:07:13.136640+0000","last_became_peered":"2024-01-15T18:07:13.136640+0000","last_unstale":"2024-01-15T18:07:14.775054+0000","last_undegraded":"2024-01-15T18:07:14.775054+0000","last_fullsized":"2024-01-15T18:07:14.775054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:21.093486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775013+0000","last_change":"2024-01-15T18:07:13.565438+0000","last_active":"2024-01-15T18:07:14.775013+0000","last_peered":"2024-01-15T18:07:14.775013+0000","last_clean":"2024-01-15T18:07:14.775013+0000","last_became_active":"2024-01-15T18:07:13.565147+0000","last_became_peered":"2024-01-15T18:07:13.565147+0000","last_unstale":"2024-01-15T18:07:14.775013+0000","last_undegraded":"2024-01-15T18:07:14.775013+0000","last_fullsized":"2024-01-15T18:07:14.775013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:46.069477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773565+0000","last_change":"2024-01-15T18:07:13.137547+0000","last_active":"2024-01-15T18:07:14.773565+0000","last_peered":"2024-01-15T18:07:14.773565+0000","last_clean":"2024-01-15T18:07:14.773565+0000","last_became_active":"2024-01-15T18:07:13.137239+0000","last_became_peered":"2024-01-15T18:07:13.137239+0000","last_unstale":"2024-01-15T18:07:14.773565+0000","last_undegraded":"2024-01-15T18:07:14.773565+0000","last_fullsized":"2024-01-15T18:07:14.773565+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:54.663418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774058+0000","last_change":"2024-01-15T18:07:13.562544+0000","last_active":"2024-01-15T18:07:14.774058+0000","last_peered":"2024-01-15T18:07:14.774058+0000","last_clean":"2024-01-15T18:07:14.774058+0000","last_became_active":"2024-01-15T18:07:13.562286+0000","last_became_peered":"2024-01-15T18:07:13.562286+0000","last_unstale":"2024-01-15T18:07:14.774058+0000","last_undegraded":"2024-01-15T18:07:14.774058+0000","last_fullsized":"2024-01-15T18:07:14.774058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:08.982957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773481+0000","last_change":"2024-01-15T18:07:13.139177+0000","last_active":"2024-01-15T18:07:14.773481+0000","last_peered":"2024-01-15T18:07:14.773481+0000","last_clean":"2024-01-15T18:07:14.773481+0000","last_became_active":"2024-01-15T18:07:13.138899+0000","last_became_peered":"2024-01-15T18:07:13.138899+0000","last_unstale":"2024-01-15T18:07:14.773481+0000","last_undegraded":"2024-01-15T18:07:14.773481+0000","last_fullsized":"2024-01-15T18:07:14.773481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774888+0000","last_change":"2024-01-15T18:07:13.561087+0000","last_active":"2024-01-15T18:07:14.774888+0000","last_peered":"2024-01-15T18:07:14.774888+0000","last_clean":"2024-01-15T18:07:14.774888+0000","last_became_active":"2024-01-15T18:07:13.560854+0000","last_became_peered":"2024-01-15T18:07:13.560854+0000","last_unstale":"2024-01-15T18:07:14.774888+0000","last_undegraded":"2024-01-15T18:07:14.774888+0000","last_fullsized":"2024-01-15T18:07:14.774888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:37.407192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542278+0000","last_change":"2024-01-15T18:07:13.144653+0000","last_active":"2024-01-15T18:07:14.542278+0000","last_peered":"2024-01-15T18:07:14.542278+0000","last_clean":"2024-01-15T18:07:14.542278+0000","last_became_active":"2024-01-15T18:07:13.144221+0000","last_became_peered":"2024-01-15T18:07:13.144221+0000","last_unstale":"2024-01-15T18:07:14.542278+0000","last_undegraded":"2024-01-15T18:07:14.542278+0000","last_fullsized":"2024-01-15T18:07:14.542278+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:38:46.176990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541667+0000","last_change":"2024-01-15T18:07:13.560705+0000","last_active":"2024-01-15T18:07:14.541667+0000","last_peered":"2024-01-15T18:07:14.541667+0000","last_clean":"2024-01-15T18:07:14.541667+0000","last_became_active":"2024-01-15T18:07:13.560263+0000","last_became_peered":"2024-01-15T18:07:13.560263+0000","last_unstale":"2024-01-15T18:07:14.541667+0000","last_undegraded":"2024-01-15T18:07:14.541667+0000","last_fullsized":"2024-01-15T18:07:14.541667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:43.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543562+0000","last_change":"2024-01-15T18:07:13.142213+0000","last_active":"2024-01-15T18:07:14.543562+0000","last_peered":"2024-01-15T18:07:14.543562+0000","last_clean":"2024-01-15T18:07:14.543562+0000","last_became_active":"2024-01-15T18:07:13.141943+0000","last_became_peered":"2024-01-15T18:07:13.141943+0000","last_unstale":"2024-01-15T18:07:14.543562+0000","last_undegraded":"2024-01-15T18:07:14.543562+0000","last_fullsized":"2024-01-15T18:07:14.543562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:14.240502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541986+0000","last_change":"2024-01-15T18:07:13.567698+0000","last_active":"2024-01-15T18:07:14.541986+0000","last_peered":"2024-01-15T18:07:14.541986+0000","last_clean":"2024-01-15T18:07:14.541986+0000","last_became_active":"2024-01-15T18:07:13.567421+0000","last_became_peered":"2024-01-15T18:07:13.567421+0000","last_unstale":"2024-01-15T18:07:14.541986+0000","last_undegraded":"2024-01-15T18:07:14.541986+0000","last_fullsized":"2024-01-15T18:07:14.541986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:56.884932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775254+0000","last_change":"2024-01-15T18:07:13.143810+0000","last_active":"2024-01-15T18:07:14.775254+0000","last_peered":"2024-01-15T18:07:14.775254+0000","last_clean":"2024-01-15T18:07:14.775254+0000","last_became_active":"2024-01-15T18:07:13.143480+0000","last_became_peered":"2024-01-15T18:07:13.143480+0000","last_unstale":"2024-01-15T18:07:14.775254+0000","last_undegraded":"2024-01-15T18:07:14.775254+0000","last_fullsized":"2024-01-15T18:07:14.775254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:43.404150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774844+0000","last_change":"2024-01-15T18:07:13.557697+0000","last_active":"2024-01-15T18:07:14.774844+0000","last_peered":"2024-01-15T18:07:14.774844+0000","last_clean":"2024-01-15T18:07:14.774844+0000","last_became_active":"2024-01-15T18:07:13.557436+0000","last_became_peered":"2024-01-15T18:07:13.557436+0000","last_unstale":"2024-01-15T18:07:14.774844+0000","last_undegraded":"2024-01-15T18:07:14.774844+0000","last_fullsized":"2024-01-15T18:07:14.774844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:28.246734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774771+0000","last_change":"2024-01-15T18:07:13.145389+0000","last_active":"2024-01-15T18:07:14.774771+0000","last_peered":"2024-01-15T18:07:14.774771+0000","last_clean":"2024-01-15T18:07:14.774771+0000","last_became_active":"2024-01-15T18:07:13.145110+0000","last_became_peered":"2024-01-15T18:07:13.145110+0000","last_unstale":"2024-01-15T18:07:14.774771+0000","last_undegraded":"2024-01-15T18:07:14.774771+0000","last_fullsized":"2024-01-15T18:07:14.774771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:42.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773907+0000","last_change":"2024-01-15T18:07:13.563791+0000","last_active":"2024-01-15T18:07:14.773907+0000","last_peered":"2024-01-15T18:07:14.773907+0000","last_clean":"2024-01-15T18:07:14.773907+0000","last_became_active":"2024-01-15T18:07:13.563499+0000","last_became_peered":"2024-01-15T18:07:13.563499+0000","last_unstale":"2024-01-15T18:07:14.773907+0000","last_undegraded":"2024-01-15T18:07:14.773907+0000","last_fullsized":"2024-01-15T18:07:14.773907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:20.253082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543216+0000","last_change":"2024-01-15T18:07:13.140448+0000","last_active":"2024-01-15T18:07:14.543216+0000","last_peered":"2024-01-15T18:07:14.543216+0000","last_clean":"2024-01-15T18:07:14.543216+0000","last_became_active":"2024-01-15T18:07:13.139194+0000","last_became_peered":"2024-01-15T18:07:13.139194+0000","last_unstale":"2024-01-15T18:07:14.543216+0000","last_undegraded":"2024-01-15T18:07:14.543216+0000","last_fullsized":"2024-01-15T18:07:14.543216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:05.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773323+0000","last_change":"2024-01-15T18:07:13.553150+0000","last_active":"2024-01-15T18:07:14.773323+0000","last_peered":"2024-01-15T18:07:14.773323+0000","last_clean":"2024-01-15T18:07:14.773323+0000","last_became_active":"2024-01-15T18:07:13.552870+0000","last_became_peered":"2024-01-15T18:07:13.552870+0000","last_unstale":"2024-01-15T18:07:14.773323+0000","last_undegraded":"2024-01-15T18:07:14.773323+0000","last_fullsized":"2024-01-15T18:07:14.773323+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:29:34.507041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773835+0000","last_change":"2024-01-15T18:07:13.138368+0000","last_active":"2024-01-15T18:07:14.773835+0000","last_peered":"2024-01-15T18:07:14.773835+0000","last_clean":"2024-01-15T18:07:14.773835+0000","last_became_active":"2024-01-15T18:07:13.138032+0000","last_became_peered":"2024-01-15T18:07:13.138032+0000","last_unstale":"2024-01-15T18:07:14.773835+0000","last_undegraded":"2024-01-15T18:07:14.773835+0000","last_fullsized":"2024-01-15T18:07:14.773835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:29:33.498295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773758+0000","last_change":"2024-01-15T18:07:13.558500+0000","last_active":"2024-01-15T18:07:14.773758+0000","last_peered":"2024-01-15T18:07:14.773758+0000","last_clean":"2024-01-15T18:07:14.773758+0000","last_became_active":"2024-01-15T18:07:13.557620+0000","last_became_peered":"2024-01-15T18:07:13.557620+0000","last_unstale":"2024-01-15T18:07:14.773758+0000","last_undegraded":"2024-01-15T18:07:14.773758+0000","last_fullsized":"2024-01-15T18:07:14.773758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:27.810935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774532+0000","last_change":"2024-01-15T18:07:13.148670+0000","last_active":"2024-01-15T18:07:14.774532+0000","last_peered":"2024-01-15T18:07:14.774532+0000","last_clean":"2024-01-15T18:07:14.774532+0000","last_became_active":"2024-01-15T18:07:13.148342+0000","last_became_peered":"2024-01-15T18:07:13.148342+0000","last_unstale":"2024-01-15T18:07:14.774532+0000","last_undegraded":"2024-01-15T18:07:14.774532+0000","last_fullsized":"2024-01-15T18:07:14.774532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:19.327531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542821+0000","last_change":"2024-01-15T18:07:13.550052+0000","last_active":"2024-01-15T18:07:14.542821+0000","last_peered":"2024-01-15T18:07:14.542821+0000","last_clean":"2024-01-15T18:07:14.542821+0000","last_became_active":"2024-01-15T18:07:13.549747+0000","last_became_peered":"2024-01-15T18:07:13.549747+0000","last_unstale":"2024-01-15T18:07:14.542821+0000","last_undegraded":"2024-01-15T18:07:14.542821+0000","last_fullsized":"2024-01-15T18:07:14.542821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:55.672164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.776330+0000","last_change":"2024-01-15T18:07:13.145472+0000","last_active":"2024-01-15T18:07:14.776330+0000","last_peered":"2024-01-15T18:07:14.776330+0000","last_clean":"2024-01-15T18:07:14.776330+0000","last_became_active":"2024-01-15T18:07:13.145151+0000","last_became_peered":"2024-01-15T18:07:13.145151+0000","last_unstale":"2024-01-15T18:07:14.776330+0000","last_undegraded":"2024-01-15T18:07:14.776330+0000","last_fullsized":"2024-01-15T18:07:14.776330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:52.070623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774409+0000","last_change":"2024-01-15T18:07:13.547632+0000","last_active":"2024-01-15T18:07:14.774409+0000","last_peered":"2024-01-15T18:07:14.774409+0000","last_clean":"2024-01-15T18:07:14.774409+0000","last_became_active":"2024-01-15T18:07:13.547354+0000","last_became_peered":"2024-01-15T18:07:13.547354+0000","last_unstale":"2024-01-15T18:07:14.774409+0000","last_undegraded":"2024-01-15T18:07:14.774409+0000","last_fullsized":"2024-01-15T18:07:14.774409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:06.472843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775040+0000","last_change":"2024-01-15T18:07:13.152981+0000","last_active":"2024-01-15T18:07:14.775040+0000","last_peered":"2024-01-15T18:07:14.775040+0000","last_clean":"2024-01-15T18:07:14.775040+0000","last_became_active":"2024-01-15T18:07:13.152735+0000","last_became_peered":"2024-01-15T18:07:13.152735+0000","last_unstale":"2024-01-15T18:07:14.775040+0000","last_undegraded":"2024-01-15T18:07:14.775040+0000","last_fullsized":"2024-01-15T18:07:14.775040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:42.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774975+0000","last_change":"2024-01-15T18:07:13.568915+0000","last_active":"2024-01-15T18:07:14.774975+0000","last_peered":"2024-01-15T18:07:14.774975+0000","last_clean":"2024-01-15T18:07:14.774975+0000","last_became_active":"2024-01-15T18:07:13.567703+0000","last_became_peered":"2024-01-15T18:07:13.567703+0000","last_unstale":"2024-01-15T18:07:14.774975+0000","last_undegraded":"2024-01-15T18:07:14.774975+0000","last_fullsized":"2024-01-15T18:07:14.774975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:36.540360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773425+0000","last_change":"2024-01-15T18:07:13.143026+0000","last_active":"2024-01-15T18:07:14.773425+0000","last_peered":"2024-01-15T18:07:14.773425+0000","last_clean":"2024-01-15T18:07:14.773425+0000","last_became_active":"2024-01-15T18:07:13.142687+0000","last_became_peered":"2024-01-15T18:07:13.142687+0000","last_unstale":"2024-01-15T18:07:14.773425+0000","last_undegraded":"2024-01-15T18:07:14.773425+0000","last_fullsized":"2024-01-15T18:07:14.773425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:55.876186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774904+0000","last_change":"2024-01-15T18:07:13.558506+0000","last_active":"2024-01-15T18:07:14.774904+0000","last_peered":"2024-01-15T18:07:14.774904+0000","last_clean":"2024-01-15T18:07:14.774904+0000","last_became_active":"2024-01-15T18:07:13.558261+0000","last_became_peered":"2024-01-15T18:07:13.558261+0000","last_unstale":"2024-01-15T18:07:14.774904+0000","last_undegraded":"2024-01-15T18:07:14.774904+0000","last_fullsized":"2024-01-15T18:07:14.774904+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:46:06.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774816+0000","last_change":"2024-01-15T18:07:13.153254+0000","last_active":"2024-01-15T18:07:14.774816+0000","last_peered":"2024-01-15T18:07:14.774816+0000","last_clean":"2024-01-15T18:07:14.774816+0000","last_became_active":"2024-01-15T18:07:13.152937+0000","last_became_peered":"2024-01-15T18:07:13.152937+0000","last_unstale":"2024-01-15T18:07:14.774816+0000","last_undegraded":"2024-01-15T18:07:14.774816+0000","last_fullsized":"2024-01-15T18:07:14.774816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:14.240502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774602+0000","last_change":"2024-01-15T18:07:13.554436+0000","last_active":"2024-01-15T18:07:14.774602+0000","last_peered":"2024-01-15T18:07:14.774602+0000","last_clean":"2024-01-15T18:07:14.774602+0000","last_became_active":"2024-01-15T18:07:13.554156+0000","last_became_peered":"2024-01-15T18:07:13.554156+0000","last_unstale":"2024-01-15T18:07:14.774602+0000","last_undegraded":"2024-01-15T18:07:14.774602+0000","last_fullsized":"2024-01-15T18:07:14.774602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:52:24.006274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542033+0000","last_change":"2024-01-15T18:07:13.132084+0000","last_active":"2024-01-15T18:07:14.542033+0000","last_peered":"2024-01-15T18:07:14.542033+0000","last_clean":"2024-01-15T18:07:14.542033+0000","last_became_active":"2024-01-15T18:07:13.131783+0000","last_became_peered":"2024-01-15T18:07:13.131783+0000","last_unstale":"2024-01-15T18:07:14.542033+0000","last_undegraded":"2024-01-15T18:07:14.542033+0000","last_fullsized":"2024-01-15T18:07:14.542033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:57:50.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543321+0000","last_change":"2024-01-15T18:07:13.562769+0000","last_active":"2024-01-15T18:07:14.543321+0000","last_peered":"2024-01-15T18:07:14.543321+0000","last_clean":"2024-01-15T18:07:14.543321+0000","last_became_active":"2024-01-15T18:07:13.562497+0000","last_became_peered":"2024-01-15T18:07:13.562497+0000","last_unstale":"2024-01-15T18:07:14.543321+0000","last_undegraded":"2024-01-15T18:07:14.543321+0000","last_fullsized":"2024-01-15T18:07:14.543321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:37.407192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773745+0000","last_change":"2024-01-15T18:07:13.141244+0000","last_active":"2024-01-15T18:07:14.773745+0000","last_peered":"2024-01-15T18:07:14.773745+0000","last_clean":"2024-01-15T18:07:14.773745+0000","last_became_active":"2024-01-15T18:07:13.140989+0000","last_became_peered":"2024-01-15T18:07:13.140989+0000","last_unstale":"2024-01-15T18:07:14.773745+0000","last_undegraded":"2024-01-15T18:07:14.773745+0000","last_fullsized":"2024-01-15T18:07:14.773745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:10.344122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541640+0000","last_change":"2024-01-15T18:07:13.556894+0000","last_active":"2024-01-15T18:07:14.541640+0000","last_peered":"2024-01-15T18:07:14.541640+0000","last_clean":"2024-01-15T18:07:14.541640+0000","last_became_active":"2024-01-15T18:07:13.556530+0000","last_became_peered":"2024-01-15T18:07:13.556530+0000","last_unstale":"2024-01-15T18:07:14.541640+0000","last_undegraded":"2024-01-15T18:07:14.541640+0000","last_fullsized":"2024-01-15T18:07:14.541640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:04.463119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773646+0000","last_change":"2024-01-15T18:07:13.134762+0000","last_active":"2024-01-15T18:07:14.773646+0000","last_peered":"2024-01-15T18:07:14.773646+0000","last_clean":"2024-01-15T18:07:14.773646+0000","last_became_active":"2024-01-15T18:07:13.134496+0000","last_became_peered":"2024-01-15T18:07:13.134496+0000","last_unstale":"2024-01-15T18:07:14.773646+0000","last_undegraded":"2024-01-15T18:07:14.773646+0000","last_fullsized":"2024-01-15T18:07:14.773646+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:57:50.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774724+0000","last_change":"2024-01-15T18:07:13.562552+0000","last_active":"2024-01-15T18:07:14.774724+0000","last_peered":"2024-01-15T18:07:14.774724+0000","last_clean":"2024-01-15T18:07:14.774724+0000","last_became_active":"2024-01-15T18:07:13.562292+0000","last_became_peered":"2024-01-15T18:07:13.562292+0000","last_unstale":"2024-01-15T18:07:14.774724+0000","last_undegraded":"2024-01-15T18:07:14.774724+0000","last_fullsized":"2024-01-15T18:07:14.774724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:22.102232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543658+0000","last_change":"2024-01-15T18:07:13.145974+0000","last_active":"2024-01-15T18:07:14.543658+0000","last_peered":"2024-01-15T18:07:14.543658+0000","last_clean":"2024-01-15T18:07:14.543658+0000","last_became_active":"2024-01-15T18:07:13.145622+0000","last_became_peered":"2024-01-15T18:07:13.145622+0000","last_unstale":"2024-01-15T18:07:14.543658+0000","last_undegraded":"2024-01-15T18:07:14.543658+0000","last_fullsized":"2024-01-15T18:07:14.543658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:21.093486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-01-15T18:07:13.555978+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:07:13.555978+0000","last_peered":"2024-01-15T18:07:13.555978+0000","last_clean":"2024-01-15T18:07:13.555978+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:07:13.555978+0000","last_undegraded":"2024-01-15T18:07:13.555978+0000","last_fullsized":"2024-01-15T18:07:13.555978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":812277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77000000000000002,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1419999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.63600000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.51200000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.747,"15min":0.76600000000000001},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":222491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.877}]},{"osd":1,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85699999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92100000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.89500000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.746},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.95399999999999996,"5min":1.107,"15min":1.107},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97799999999999998,"5min":1.401,"15min":1.401},"last":0.77200000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.67000000000000004,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.79600000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.032,"5min":5.032,"15min":5.032},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69399999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.97799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.78300000000000003,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.82999999999999996,"15min":0.81399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.95599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":223431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502}]},{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90000000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94099999999999995,"5min":1.054,"15min":1.054},"last":0.62}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83999999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90400000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":1.218,"15min":1.218},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":1.179,"15min":1.179},"last":0.748}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70599999999999996,"15min":0.72199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.78000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":222457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.52700000000000002,"15min":0.47099999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.47599999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.444,"5min":3.444,"15min":3.444},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.181,"5min":7.181,"15min":7.181},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.61299999999999999,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.57099999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.83199999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.81399999999999995,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.94399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.875,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.92800000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":223453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.47399999999999998,"15min":0.45200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.4880000000000004,"5min":5.4880000000000004,"15min":5.4880000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.622}]},{"osd":3,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.48399999999999999,"15min":0.46000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.497},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81200000000000006,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.88900000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.72399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.056},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":3.2010000000000001,"5min":3.2010000000000001,"15min":3.2010000000000001},"last":1.077}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.0960000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.89500000000000002,"15min":0.876},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":1.1319999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2570000000000001,"5min":4.532,"15min":4.532},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.0409999999999999,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.53400000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.438},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.3610000000000002,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.59799999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.92100000000000004,"15min":0.81299999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":21.416,"5min":21.416,"15min":21.416},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.101,"5min":0.90900000000000003,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.022},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.238,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.0209999999999999,"15min":0.98299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":12.997,"5min":12.997,"15min":12.997},"last":0.86899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.123},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":13.054,"5min":13.054,"15min":13.054},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.2170000000000001,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.876,"5min":1.008,"15min":1.03},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.94099999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.7729999999999997,"5min":4.7729999999999997,"15min":4.7729999999999997},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.274,"5min":1.1060000000000001,"15min":1.0780000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":13.089,"5min":13.089,"15min":13.089},"last":0.78400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":812221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.79000000000000004,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.629},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.65300000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.81999999999999995,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.85299999999999998,"15min":0.91400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98199999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.88300000000000001,"15min":0.91800000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.753},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.876}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.92600000000000005,"15min":0.96999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1240000000000001,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91800000000000004,"15min":0.95399999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":4.056,"15min":4.056},"last":0.91800000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0529999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1399999999999999,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.79800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.026,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.95399999999999996,"15min":1},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93400000000000005,"5min":4.407,"15min":4.407},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:07:18.951 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T18:07:19.099 INFO:tasks.ceph.mds.a.smithi143.stderr:2024-01-15T18:07:19.092+0000 7f79c2600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:07:19.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff004400640 1 Processor -- start 2024-01-15T18:07:19.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff004400640 1 -- start start 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff004400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feffc06bfb0 0x7feffc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff004400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feffc0cbff0 con 0x7feffc0b0310 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff004400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feffc0d0c50 con 0x7feffc0d0210 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff004400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feffc06c8d0 con 0x7feffc06bfb0 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff001a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feffc06bfb0 0x7feffc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff001000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7feffc0b0310 0x7feffc0cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50054/0 (socket says 172.21.15.143:50054) 2024-01-15T18:07:19.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff001000640 1 -- 172.21.15.143:0/683574740 learned_addr learned my addr 172.21.15.143:0/683574740 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:19.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff001a00640 1 -- 172.21.15.143:0/683574740 >> v1:172.21.15.204:6789/0 conn(0x7feffc0d0210 legacy=0x7feffc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:19.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff001a00640 1 -- 172.21.15.143:0/683574740 >> v1:172.21.15.143:6789/0 conn(0x7feffc0b0310 legacy=0x7feffc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7ff001a00640 1 -- 172.21.15.143:0/683574740 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc0cb6c0 con 0x7feffc06bfb0 2024-01-15T18:07:19.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.120+0000 7feffbe00640 1 -- 172.21.15.143:0/683574740 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125637877 0 0) 0x7feffc0cbff0 con 0x7feffc0b0310 2024-01-15T18:07:19.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff001a00640 1 --2- 172.21.15.143:0/683574740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc06bfb0 0x7feffc06c390 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fefec0097c0 tx=0x7fefec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbef56264c6c560 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7feffbe00640 1 -- 172.21.15.143:0/683574740 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0ad680 con 0x7feffc06bfb0 2024-01-15T18:07:19.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7feffbe00640 1 -- 172.21.15.143:0/683574740 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec0ad7e0 con 0x7feffc06bfb0 2024-01-15T18:07:19.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7feffbe00640 1 -- 172.21.15.143:0/683574740 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec0adb70 con 0x7feffc06bfb0 2024-01-15T18:07:19.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 -- 172.21.15.143:0/683574740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc06bfb0 msgr2=0x7feffc06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 --2- 172.21.15.143:0/683574740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc06bfb0 0x7feffc06c390 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fefec0097c0 tx=0x7fefec01c380 comp rx=0 tx=0).stop 2024-01-15T18:07:19.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 -- 172.21.15.143:0/683574740 shutdown_connections 2024-01-15T18:07:19.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 --2- 172.21.15.143:0/683574740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc06bfb0 0x7feffc06c390 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 -- 172.21.15.143:0/683574740 >> 172.21.15.143:0/683574740 conn(0x7feffc0d0ed0 msgr2=0x7feffc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 -- 172.21.15.143:0/683574740 shutdown_connections 2024-01-15T18:07:19.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.124+0000 7ff004400640 1 -- 172.21.15.143:0/683574740 wait complete. 2024-01-15T18:07:19.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 Processor -- start 2024-01-15T18:07:19.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- start start 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc06bfb0 0x7feffc0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 0x7feffc15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feffc0d0210 0x7feffc159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0b0cc0 con 0x7feffc0b0310 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feffc0b1430 con 0x7feffc06bfb0 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feffc15a320 con 0x7feffc0d0210 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 0x7feffc15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 0x7feffc15bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47148/0 (socket says 172.21.15.143:47148) 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 -- 172.21.15.143:0/2471486165 learned_addr learned my addr 172.21.15.143:0/2471486165 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001a00640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc06bfb0 0x7feffc0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff002400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feffc0d0210 0x7feffc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 -- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feffc0d0210 msgr2=0x7feffc159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feffc0d0210 0x7feffc159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 -- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc06bfb0 msgr2=0x7feffc0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc06bfb0 0x7feffc0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 -- 172.21.15.143:0/2471486165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec009420 con 0x7feffc0b0310 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001a00640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc06bfb0 0x7feffc0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff001000640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 0x7feffc15bcc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c450 tx=0x7fefe400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40074e0 con 0x7feffc0b0310 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4007640 con 0x7feffc0b0310 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40049f0 con 0x7feffc0b0310 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feffc15a600 con 0x7feffc0b0310 2024-01-15T18:07:19.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc0c84e0 con 0x7feffc0b0310 2024-01-15T18:07:19.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.128+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feffc0b1150 con 0x7feffc0b0310 2024-01-15T18:07:19.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.132+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fefe4004b50 con 0x7feffc0b0310 2024-01-15T18:07:19.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.132+0000 7feff9600640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fefd4066b30 0x7fefd4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.132+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fefe4083ab0 con 0x7feffc0b0310 2024-01-15T18:07:19.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.132+0000 7ff001a00640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fefd4066b30 0x7fefd4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.132+0000 7ff001a00640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fefd4066b30 0x7fefd4068ff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefec0af000 tx=0x7fefec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.136+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feffc0b1150 con 0x7feffc0b0310 2024-01-15T18:07:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.340+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7feffc0b1150 con 0x7feffc0b0310 2024-01-15T18:07:19.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.340+0000 7feff9600640 1 -- 172.21.15.143:0/2471486165 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1183 (secure 0 0 0) 0x7feffc0b1150 con 0x7feffc0b0310 2024-01-15T18:07:19.349 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:19.349 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341886,"num_in_osds":8,"osd_in_since":1705341876,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":260595712,"bytes_avail":772833517568,"bytes_total":773094113280,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5736}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-01-15T18:06:54.646619+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fefd4066b30 msgr2=0x7fefd4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fefd4066b30 0x7fefd4068ff0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefec0af000 tx=0x7fefec008040 comp rx=0 tx=0).stop 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 msgr2=0x7feffc15bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 0x7feffc15bcc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fefe400c450 tx=0x7fefe400c910 comp rx=0 tx=0).stop 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 shutdown_connections 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fefd4066b30 0x7fefd4068ff0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feffc0d0210 0x7feffc159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feffc0b0310 0x7feffc15bcc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 --2- 172.21.15.143:0/2471486165 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feffc06bfb0 0x7feffc0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 >> 172.21.15.143:0/2471486165 conn(0x7feffc0d0ed0 msgr2=0x7feffc06ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 shutdown_connections 2024-01-15T18:07:19.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.344+0000 7ff004400640 1 -- 172.21.15.143:0/2471486165 wait complete. 2024-01-15T18:07:19.372 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T18:07:19.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:07:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 Processor -- start 2024-01-15T18:07:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 -- start start 2024-01-15T18:07:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0d814b110 0x7fc0d80cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d80b30a0 con 0x7fc0d8071f70 2024-01-15T18:07:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d8070340 con 0x7fc0d814b110 2024-01-15T18:07:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0de600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8070620 con 0x7fc0d80cc8f0 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0d6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0d7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc0d814b110 0x7fc0d80cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0d7e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc0d8071f70 0x7fc0d8071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:38886/0 (socket says 172.21.15.143:38886) 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.500+0000 7fc0d7e00640 1 -- 172.21.15.143:0/1174900235 learned_addr learned my addr 172.21.15.143:0/1174900235 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d6000640 1 -- 172.21.15.143:0/1174900235 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574331723 0 0) 0x7fc0d80b30a0 con 0x7fc0d8071f70 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d6000640 1 -- 172.21.15.143:0/1174900235 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b8003530 con 0x7fc0d8071f70 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d7400640 1 -- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 msgr2=0x7fc0d80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d7400640 1 --2- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d80b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d7400640 1 -- 172.21.15.143:0/1174900235 >> v1:172.21.15.204:6789/0 conn(0x7fc0d8071f70 legacy=0x7fc0d8071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d7400640 1 -- 172.21.15.143:0/1174900235 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8070900 con 0x7fc0d814b110 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d6a00640 1 --2- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d7400640 1 --2- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80cc3b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80097c0 tx=0x7fc0c801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4fc26385875c343 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d6000640 1 -- 172.21.15.143:0/1174900235 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80ac640 con 0x7fc0d814b110 2024-01-15T18:07:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d6000640 1 -- 172.21.15.143:0/1174900235 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d80b2e90 con 0x7fc0d814b110 2024-01-15T18:07:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.504+0000 7fc0d6000640 1 -- 172.21.15.143:0/1174900235 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80ac9c0 con 0x7fc0d814b110 2024-01-15T18:07:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 msgr2=0x7fc0d80cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 --2- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80cc3b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80097c0 tx=0x7fc0c801c3c0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- 172.21.15.143:0/1174900235 shutdown_connections 2024-01-15T18:07:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 --2- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 --2- 172.21.15.143:0/1174900235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80cc3b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- 172.21.15.143:0/1174900235 >> 172.21.15.143:0/1174900235 conn(0x7fc0d806c3f0 msgr2=0x7fc0d806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- 172.21.15.143:0/1174900235 shutdown_connections 2024-01-15T18:07:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- 172.21.15.143:0/1174900235 wait complete. 2024-01-15T18:07:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 Processor -- start 2024-01-15T18:07:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- start start 2024-01-15T18:07:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 0x7fc0d8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d8144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d81584d0 con 0x7fc0d8071f70 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d81587b0 con 0x7fc0d814b110 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0de600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8161d10 con 0x7fc0d80cc8f0 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0d7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 0x7fc0d8157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.508+0000 7fc0d7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 0x7fc0d8157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47160/0 (socket says 172.21.15.143:47160) 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 -- 172.21.15.143:0/513796181 learned_addr learned my addr 172.21.15.143:0/513796181 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d6a00640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d8144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7e00640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 -- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 msgr2=0x7fc0d8144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d8144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 -- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 msgr2=0x7fc0d80ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 -- 172.21.15.143:0/513796181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c0009420 con 0x7fc0d8071f70 2024-01-15T18:07:19.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0d7400640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 0x7fc0d8157ef0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8019ff0 tx=0x7fc0c801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80b7450 con 0x7fc0d8071f70 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c80b75d0 con 0x7fc0d8071f70 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80b7980 con 0x7fc0d8071f70 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c8009420 con 0x7fc0d8071f70 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d8158a90 con 0x7fc0d8071f70 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d8067500 con 0x7fc0d8071f70 2024-01-15T18:07:19.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fc0c80b3070 con 0x7fc0d8071f70 2024-01-15T18:07:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0cf400640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0b0066b60 0x7fc0b0069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.512+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc0c811ef30 con 0x7fc0d8071f70 2024-01-15T18:07:19.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.516+0000 7fc0d6a00640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0b0066b60 0x7fc0b0069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.516+0000 7fc0d6a00640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0b0066b60 0x7fc0b0069020 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0d815baa0 tx=0x7fc0c0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.516+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc0c80ebe90 con 0x7fc0d8071f70 2024-01-15T18:07:19.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.668+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0d816ed50 con 0x7fc0b0066b60 2024-01-15T18:07:19.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.676+0000 7fc0cf400640 1 -- 172.21.15.143:0/513796181 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288700 (secure 0 0 0) 0x7fc0c0009940 con 0x7fc0b0066b60 2024-01-15T18:07:19.685 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:19.688 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:07:19.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0b0066b60 msgr2=0x7fc0b0069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0b0066b60 0x7fc0b0069020 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc0d815baa0 tx=0x7fc0c0009190 comp rx=0 tx=0).stop 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 msgr2=0x7fc0d8157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 0x7fc0d8157ef0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8019ff0 tx=0x7fc0c801ac00 comp rx=0 tx=0).stop 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 shutdown_connections 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc0b0066b60 0x7fc0b0069020 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc0d814b110 0x7fc0d80ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc0d80cc8f0 0x7fc0d8144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 --2- 172.21.15.143:0/513796181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0d8071f70 0x7fc0d8157ef0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 >> 172.21.15.143:0/513796181 conn(0x7fc0d806c3f0 msgr2=0x7fc0d80b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 shutdown_connections 2024-01-15T18:07:19.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:19.680+0000 7fc0de600640 1 -- 172.21.15.143:0/513796181 wait complete. 2024-01-15T18:07:19.709 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-01-15T18:07:18.650776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254920,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719800,"statfs":{"total":773094113280,"available":772833075200,"internally_reserved":0,"allocated":5578752,"data_stored":2960108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000992"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775090+0000","last_change":"2024-01-15T18:07:13.565379+0000","last_active":"2024-01-15T18:07:14.775090+0000","last_peered":"2024-01-15T18:07:14.775090+0000","last_clean":"2024-01-15T18:07:14.775090+0000","last_became_active":"2024-01-15T18:07:13.565044+0000","last_became_peered":"2024-01-15T18:07:13.565044+0000","last_unstale":"2024-01-15T18:07:14.775090+0000","last_undegraded":"2024-01-15T18:07:14.775090+0000","last_fullsized":"2024-01-15T18:07:14.775090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:42.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543575+0000","last_change":"2024-01-15T18:07:13.566360+0000","last_active":"2024-01-15T18:07:14.543575+0000","last_peered":"2024-01-15T18:07:14.543575+0000","last_clean":"2024-01-15T18:07:14.543575+0000","last_became_active":"2024-01-15T18:07:13.566039+0000","last_became_peered":"2024-01-15T18:07:13.566039+0000","last_unstale":"2024-01-15T18:07:14.543575+0000","last_undegraded":"2024-01-15T18:07:14.543575+0000","last_fullsized":"2024-01-15T18:07:14.543575+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:20.336277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542050+0000","last_change":"2024-01-15T18:07:13.552196+0000","last_active":"2024-01-15T18:07:14.542050+0000","last_peered":"2024-01-15T18:07:14.542050+0000","last_clean":"2024-01-15T18:07:14.542050+0000","last_became_active":"2024-01-15T18:07:13.551879+0000","last_became_peered":"2024-01-15T18:07:13.551879+0000","last_unstale":"2024-01-15T18:07:14.542050+0000","last_undegraded":"2024-01-15T18:07:14.542050+0000","last_fullsized":"2024-01-15T18:07:14.542050+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:53.079369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774299+0000","last_change":"2024-01-15T18:07:13.547096+0000","last_active":"2024-01-15T18:07:14.774299+0000","last_peered":"2024-01-15T18:07:14.774299+0000","last_clean":"2024-01-15T18:07:14.774299+0000","last_became_active":"2024-01-15T18:07:13.546783+0000","last_became_peered":"2024-01-15T18:07:13.546783+0000","last_unstale":"2024-01-15T18:07:14.774299+0000","last_undegraded":"2024-01-15T18:07:14.774299+0000","last_fullsized":"2024-01-15T18:07:14.774299+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:08.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543306+0000","last_change":"2024-01-15T18:07:13.559790+0000","last_active":"2024-01-15T18:07:14.543306+0000","last_peered":"2024-01-15T18:07:14.543306+0000","last_clean":"2024-01-15T18:07:14.543306+0000","last_became_active":"2024-01-15T18:07:13.559024+0000","last_became_peered":"2024-01-15T18:07:13.559024+0000","last_unstale":"2024-01-15T18:07:14.543306+0000","last_undegraded":"2024-01-15T18:07:14.543306+0000","last_fullsized":"2024-01-15T18:07:14.543306+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:19.613876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774435+0000","last_change":"2024-01-15T18:07:13.557998+0000","last_active":"2024-01-15T18:07:14.774435+0000","last_peered":"2024-01-15T18:07:14.774435+0000","last_clean":"2024-01-15T18:07:14.774435+0000","last_became_active":"2024-01-15T18:07:13.557712+0000","last_became_peered":"2024-01-15T18:07:13.557712+0000","last_unstale":"2024-01-15T18:07:14.774435+0000","last_undegraded":"2024-01-15T18:07:14.774435+0000","last_fullsized":"2024-01-15T18:07:14.774435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:46.069477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774823+0000","last_change":"2024-01-15T18:07:13.567253+0000","last_active":"2024-01-15T18:07:14.774823+0000","last_peered":"2024-01-15T18:07:14.774823+0000","last_clean":"2024-01-15T18:07:14.774823+0000","last_became_active":"2024-01-15T18:07:13.566953+0000","last_became_peered":"2024-01-15T18:07:13.566953+0000","last_unstale":"2024-01-15T18:07:14.774823+0000","last_undegraded":"2024-01-15T18:07:14.774823+0000","last_fullsized":"2024-01-15T18:07:14.774823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:44.412896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774145+0000","last_change":"2024-01-15T18:07:13.545892+0000","last_active":"2024-01-15T18:07:14.774145+0000","last_peered":"2024-01-15T18:07:14.774145+0000","last_clean":"2024-01-15T18:07:14.774145+0000","last_became_active":"2024-01-15T18:07:13.545624+0000","last_became_peered":"2024-01-15T18:07:13.545624+0000","last_unstale":"2024-01-15T18:07:14.774145+0000","last_undegraded":"2024-01-15T18:07:14.774145+0000","last_fullsized":"2024-01-15T18:07:14.774145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:53.079369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774766+0000","last_change":"2024-01-15T18:07:13.565934+0000","last_active":"2024-01-15T18:07:14.774766+0000","last_peered":"2024-01-15T18:07:14.774766+0000","last_clean":"2024-01-15T18:07:14.774766+0000","last_became_active":"2024-01-15T18:07:13.565348+0000","last_became_peered":"2024-01-15T18:07:13.565348+0000","last_unstale":"2024-01-15T18:07:14.774766+0000","last_undegraded":"2024-01-15T18:07:14.774766+0000","last_fullsized":"2024-01-15T18:07:14.774766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:19.613876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774608+0000","last_change":"2024-01-15T18:07:13.545220+0000","last_active":"2024-01-15T18:07:14.774608+0000","last_peered":"2024-01-15T18:07:14.774608+0000","last_clean":"2024-01-15T18:07:14.774608+0000","last_became_active":"2024-01-15T18:07:13.544957+0000","last_became_peered":"2024-01-15T18:07:13.544957+0000","last_unstale":"2024-01-15T18:07:14.774608+0000","last_undegraded":"2024-01-15T18:07:14.774608+0000","last_fullsized":"2024-01-15T18:07:14.774608+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:58.216097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774680+0000","last_change":"2024-01-15T18:07:13.565635+0000","last_active":"2024-01-15T18:07:14.774680+0000","last_peered":"2024-01-15T18:07:14.774680+0000","last_clean":"2024-01-15T18:07:14.774680+0000","last_became_active":"2024-01-15T18:07:13.565356+0000","last_became_peered":"2024-01-15T18:07:13.565356+0000","last_unstale":"2024-01-15T18:07:14.774680+0000","last_undegraded":"2024-01-15T18:07:14.774680+0000","last_fullsized":"2024-01-15T18:07:14.774680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:15.249248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543524+0000","last_change":"2024-01-15T18:07:13.562099+0000","last_active":"2024-01-15T18:07:14.543524+0000","last_peered":"2024-01-15T18:07:14.543524+0000","last_clean":"2024-01-15T18:07:14.543524+0000","last_became_active":"2024-01-15T18:07:13.561790+0000","last_became_peered":"2024-01-15T18:07:13.561790+0000","last_unstale":"2024-01-15T18:07:14.543524+0000","last_undegraded":"2024-01-15T18:07:14.543524+0000","last_fullsized":"2024-01-15T18:07:14.543524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:43.794608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773525+0000","last_change":"2024-01-15T18:07:13.560152+0000","last_active":"2024-01-15T18:07:14.773525+0000","last_peered":"2024-01-15T18:07:14.773525+0000","last_clean":"2024-01-15T18:07:14.773525+0000","last_became_active":"2024-01-15T18:07:13.559682+0000","last_became_peered":"2024-01-15T18:07:13.559682+0000","last_unstale":"2024-01-15T18:07:14.773525+0000","last_undegraded":"2024-01-15T18:07:14.773525+0000","last_fullsized":"2024-01-15T18:07:14.773525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:14:35.267226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773436+0000","last_change":"2024-01-15T18:07:13.562323+0000","last_active":"2024-01-15T18:07:14.773436+0000","last_peered":"2024-01-15T18:07:14.773436+0000","last_clean":"2024-01-15T18:07:14.773436+0000","last_became_active":"2024-01-15T18:07:13.562052+0000","last_became_peered":"2024-01-15T18:07:13.562052+0000","last_unstale":"2024-01-15T18:07:14.773436+0000","last_undegraded":"2024-01-15T18:07:14.773436+0000","last_fullsized":"2024-01-15T18:07:14.773436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:15.081589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774888+0000","last_change":"2024-01-15T18:07:13.555476+0000","last_active":"2024-01-15T18:07:14.774888+0000","last_peered":"2024-01-15T18:07:14.774888+0000","last_clean":"2024-01-15T18:07:14.774888+0000","last_became_active":"2024-01-15T18:07:13.555192+0000","last_became_peered":"2024-01-15T18:07:13.555192+0000","last_unstale":"2024-01-15T18:07:14.774888+0000","last_undegraded":"2024-01-15T18:07:14.774888+0000","last_fullsized":"2024-01-15T18:07:14.774888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:50:27.459870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541451+0000","last_change":"2024-01-15T18:07:13.545439+0000","last_active":"2024-01-15T18:07:14.541451+0000","last_peered":"2024-01-15T18:07:14.541451+0000","last_clean":"2024-01-15T18:07:14.541451+0000","last_became_active":"2024-01-15T18:07:13.545109+0000","last_became_peered":"2024-01-15T18:07:13.545109+0000","last_unstale":"2024-01-15T18:07:14.541451+0000","last_undegraded":"2024-01-15T18:07:14.541451+0000","last_fullsized":"2024-01-15T18:07:14.541451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:12:37.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542612+0000","last_change":"2024-01-15T18:07:13.564080+0000","last_active":"2024-01-15T18:07:14.542612+0000","last_peered":"2024-01-15T18:07:14.542612+0000","last_clean":"2024-01-15T18:07:14.542612+0000","last_became_active":"2024-01-15T18:07:13.563671+0000","last_became_peered":"2024-01-15T18:07:13.563671+0000","last_unstale":"2024-01-15T18:07:14.542612+0000","last_undegraded":"2024-01-15T18:07:14.542612+0000","last_fullsized":"2024-01-15T18:07:14.542612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:58.216097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542189+0000","last_change":"2024-01-15T18:07:13.552275+0000","last_active":"2024-01-15T18:07:14.542189+0000","last_peered":"2024-01-15T18:07:14.542189+0000","last_clean":"2024-01-15T18:07:14.542189+0000","last_became_active":"2024-01-15T18:07:13.551396+0000","last_became_peered":"2024-01-15T18:07:13.551396+0000","last_unstale":"2024-01-15T18:07:14.542189+0000","last_undegraded":"2024-01-15T18:07:14.542189+0000","last_fullsized":"2024-01-15T18:07:14.542189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:08.081272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542176+0000","last_change":"2024-01-15T18:07:13.557559+0000","last_active":"2024-01-15T18:07:14.542176+0000","last_peered":"2024-01-15T18:07:14.542176+0000","last_clean":"2024-01-15T18:07:14.542176+0000","last_became_active":"2024-01-15T18:07:13.557217+0000","last_became_peered":"2024-01-15T18:07:13.557217+0000","last_unstale":"2024-01-15T18:07:14.542176+0000","last_undegraded":"2024-01-15T18:07:14.542176+0000","last_fullsized":"2024-01-15T18:07:14.542176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:51.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773926+0000","last_change":"2024-01-15T18:07:13.554980+0000","last_active":"2024-01-15T18:07:14.773926+0000","last_peered":"2024-01-15T18:07:14.773926+0000","last_clean":"2024-01-15T18:07:14.773926+0000","last_became_active":"2024-01-15T18:07:13.554690+0000","last_became_peered":"2024-01-15T18:07:13.554690+0000","last_unstale":"2024-01-15T18:07:14.773926+0000","last_undegraded":"2024-01-15T18:07:14.773926+0000","last_fullsized":"2024-01-15T18:07:14.773926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:37.290014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774535+0000","last_change":"2024-01-15T18:07:13.560084+0000","last_active":"2024-01-15T18:07:14.774535+0000","last_peered":"2024-01-15T18:07:14.774535+0000","last_clean":"2024-01-15T18:07:14.774535+0000","last_became_active":"2024-01-15T18:07:13.559818+0000","last_became_peered":"2024-01-15T18:07:13.559818+0000","last_unstale":"2024-01-15T18:07:14.774535+0000","last_undegraded":"2024-01-15T18:07:14.774535+0000","last_fullsized":"2024-01-15T18:07:14.774535+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:20.316030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773559+0000","last_change":"2024-01-15T18:07:13.558125+0000","last_active":"2024-01-15T18:07:14.773559+0000","last_peered":"2024-01-15T18:07:14.773559+0000","last_clean":"2024-01-15T18:07:14.773559+0000","last_became_active":"2024-01-15T18:07:13.557757+0000","last_became_peered":"2024-01-15T18:07:13.557757+0000","last_unstale":"2024-01-15T18:07:14.773559+0000","last_undegraded":"2024-01-15T18:07:14.773559+0000","last_fullsized":"2024-01-15T18:07:14.773559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:04.463119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541350+0000","last_change":"2024-01-15T18:07:13.560763+0000","last_active":"2024-01-15T18:07:14.541350+0000","last_peered":"2024-01-15T18:07:14.541350+0000","last_clean":"2024-01-15T18:07:14.541350+0000","last_became_active":"2024-01-15T18:07:13.560247+0000","last_became_peered":"2024-01-15T18:07:13.560247+0000","last_unstale":"2024-01-15T18:07:14.541350+0000","last_undegraded":"2024-01-15T18:07:14.541350+0000","last_fullsized":"2024-01-15T18:07:14.541350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:36:45.095291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541338+0000","last_change":"2024-01-15T18:07:13.554018+0000","last_active":"2024-01-15T18:07:14.541338+0000","last_peered":"2024-01-15T18:07:14.541338+0000","last_clean":"2024-01-15T18:07:14.541338+0000","last_became_active":"2024-01-15T18:07:13.553693+0000","last_became_peered":"2024-01-15T18:07:13.553693+0000","last_unstale":"2024-01-15T18:07:14.541338+0000","last_undegraded":"2024-01-15T18:07:14.541338+0000","last_fullsized":"2024-01-15T18:07:14.541338+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:08.982957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542676+0000","last_change":"2024-01-15T18:07:13.568220+0000","last_active":"2024-01-15T18:07:14.542676+0000","last_peered":"2024-01-15T18:07:14.542676+0000","last_clean":"2024-01-15T18:07:14.542676+0000","last_became_active":"2024-01-15T18:07:13.567961+0000","last_became_peered":"2024-01-15T18:07:13.567961+0000","last_unstale":"2024-01-15T18:07:14.542676+0000","last_undegraded":"2024-01-15T18:07:14.542676+0000","last_fullsized":"2024-01-15T18:07:14.542676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:19:58.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773932+0000","last_change":"2024-01-15T18:07:13.556487+0000","last_active":"2024-01-15T18:07:14.773932+0000","last_peered":"2024-01-15T18:07:14.773932+0000","last_clean":"2024-01-15T18:07:14.773932+0000","last_became_active":"2024-01-15T18:07:13.556178+0000","last_became_peered":"2024-01-15T18:07:13.556178+0000","last_unstale":"2024-01-15T18:07:14.773932+0000","last_undegraded":"2024-01-15T18:07:14.773932+0000","last_fullsized":"2024-01-15T18:07:14.773932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:56.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542225+0000","last_change":"2024-01-15T18:07:13.558444+0000","last_active":"2024-01-15T18:07:14.542225+0000","last_peered":"2024-01-15T18:07:14.542225+0000","last_clean":"2024-01-15T18:07:14.542225+0000","last_became_active":"2024-01-15T18:07:13.558068+0000","last_became_peered":"2024-01-15T18:07:13.558068+0000","last_unstale":"2024-01-15T18:07:14.542225+0000","last_undegraded":"2024-01-15T18:07:14.542225+0000","last_fullsized":"2024-01-15T18:07:14.542225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:06.472843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543524+0000","last_change":"2024-01-15T18:07:13.561182+0000","last_active":"2024-01-15T18:07:14.543524+0000","last_peered":"2024-01-15T18:07:14.543524+0000","last_clean":"2024-01-15T18:07:14.543524+0000","last_became_active":"2024-01-15T18:07:13.559200+0000","last_became_peered":"2024-01-15T18:07:13.559200+0000","last_unstale":"2024-01-15T18:07:14.543524+0000","last_undegraded":"2024-01-15T18:07:14.543524+0000","last_fullsized":"2024-01-15T18:07:14.543524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:22.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541598+0000","last_change":"2024-01-15T18:07:13.566721+0000","last_active":"2024-01-15T18:07:14.541598+0000","last_peered":"2024-01-15T18:07:14.541598+0000","last_clean":"2024-01-15T18:07:14.541598+0000","last_became_active":"2024-01-15T18:07:13.566446+0000","last_became_peered":"2024-01-15T18:07:13.566446+0000","last_unstale":"2024-01-15T18:07:14.541598+0000","last_undegraded":"2024-01-15T18:07:14.541598+0000","last_fullsized":"2024-01-15T18:07:14.541598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.108103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541736+0000","last_change":"2024-01-15T18:07:13.559670+0000","last_active":"2024-01-15T18:07:14.541736+0000","last_peered":"2024-01-15T18:07:14.541736+0000","last_clean":"2024-01-15T18:07:14.541736+0000","last_became_active":"2024-01-15T18:07:13.558831+0000","last_became_peered":"2024-01-15T18:07:13.558831+0000","last_unstale":"2024-01-15T18:07:14.541736+0000","last_undegraded":"2024-01-15T18:07:14.541736+0000","last_fullsized":"2024-01-15T18:07:14.541736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:27.810935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774974+0000","last_change":"2024-01-15T18:07:13.563324+0000","last_active":"2024-01-15T18:07:14.774974+0000","last_peered":"2024-01-15T18:07:14.774974+0000","last_clean":"2024-01-15T18:07:14.774974+0000","last_became_active":"2024-01-15T18:07:13.563088+0000","last_became_peered":"2024-01-15T18:07:13.563088+0000","last_unstale":"2024-01-15T18:07:14.774974+0000","last_undegraded":"2024-01-15T18:07:14.774974+0000","last_fullsized":"2024-01-15T18:07:14.774974+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:06.463252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773794+0000","last_change":"2024-01-15T18:07:13.558815+0000","last_active":"2024-01-15T18:07:14.773794+0000","last_peered":"2024-01-15T18:07:14.773794+0000","last_clean":"2024-01-15T18:07:14.773794+0000","last_became_active":"2024-01-15T18:07:13.558545+0000","last_became_peered":"2024-01-15T18:07:13.558545+0000","last_unstale":"2024-01-15T18:07:14.773794+0000","last_undegraded":"2024-01-15T18:07:14.773794+0000","last_fullsized":"2024-01-15T18:07:14.773794+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:55:13.674874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773433+0000","last_change":"2024-01-15T18:07:13.560097+0000","last_active":"2024-01-15T18:07:14.773433+0000","last_peered":"2024-01-15T18:07:14.773433+0000","last_clean":"2024-01-15T18:07:14.773433+0000","last_became_active":"2024-01-15T18:07:13.559817+0000","last_became_peered":"2024-01-15T18:07:13.559817+0000","last_unstale":"2024-01-15T18:07:14.773433+0000","last_undegraded":"2024-01-15T18:07:14.773433+0000","last_fullsized":"2024-01-15T18:07:14.773433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:19:58.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775400+0000","last_change":"2024-01-15T18:07:13.143155+0000","last_active":"2024-01-15T18:07:14.775400+0000","last_peered":"2024-01-15T18:07:14.775400+0000","last_clean":"2024-01-15T18:07:14.775400+0000","last_became_active":"2024-01-15T18:07:13.142887+0000","last_became_peered":"2024-01-15T18:07:13.142887+0000","last_unstale":"2024-01-15T18:07:14.775400+0000","last_undegraded":"2024-01-15T18:07:14.775400+0000","last_fullsized":"2024-01-15T18:07:14.775400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:48:17.532793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542751+0000","last_change":"2024-01-15T18:07:13.557581+0000","last_active":"2024-01-15T18:07:14.542751+0000","last_peered":"2024-01-15T18:07:14.542751+0000","last_clean":"2024-01-15T18:07:14.542751+0000","last_became_active":"2024-01-15T18:07:13.557104+0000","last_became_peered":"2024-01-15T18:07:13.557104+0000","last_unstale":"2024-01-15T18:07:14.542751+0000","last_undegraded":"2024-01-15T18:07:14.542751+0000","last_fullsized":"2024-01-15T18:07:14.542751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:20.253082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542255+0000","last_change":"2024-01-15T18:07:13.134532+0000","last_active":"2024-01-15T18:07:14.542255+0000","last_peered":"2024-01-15T18:07:14.542255+0000","last_clean":"2024-01-15T18:07:14.542255+0000","last_became_active":"2024-01-15T18:07:13.133873+0000","last_became_peered":"2024-01-15T18:07:13.133873+0000","last_unstale":"2024-01-15T18:07:14.542255+0000","last_undegraded":"2024-01-15T18:07:14.542255+0000","last_fullsized":"2024-01-15T18:07:14.542255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:24:53.119165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543674+0000","last_change":"2024-01-15T18:07:13.561484+0000","last_active":"2024-01-15T18:07:14.543674+0000","last_peered":"2024-01-15T18:07:14.543674+0000","last_clean":"2024-01-15T18:07:14.543674+0000","last_became_active":"2024-01-15T18:07:13.561171+0000","last_became_peered":"2024-01-15T18:07:13.561171+0000","last_unstale":"2024-01-15T18:07:14.543674+0000","last_undegraded":"2024-01-15T18:07:14.543674+0000","last_fullsized":"2024-01-15T18:07:14.543674+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:48:18.541539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774323+0000","last_change":"2024-01-15T18:07:13.141848+0000","last_active":"2024-01-15T18:07:14.774323+0000","last_peered":"2024-01-15T18:07:14.774323+0000","last_clean":"2024-01-15T18:07:14.774323+0000","last_became_active":"2024-01-15T18:07:13.141503+0000","last_became_peered":"2024-01-15T18:07:13.141503+0000","last_unstale":"2024-01-15T18:07:14.774323+0000","last_undegraded":"2024-01-15T18:07:14.774323+0000","last_fullsized":"2024-01-15T18:07:14.774323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:50.037313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774114+0000","last_change":"2024-01-15T18:07:13.562984+0000","last_active":"2024-01-15T18:07:14.774114+0000","last_peered":"2024-01-15T18:07:14.774114+0000","last_clean":"2024-01-15T18:07:14.774114+0000","last_became_active":"2024-01-15T18:07:13.562555+0000","last_became_peered":"2024-01-15T18:07:13.562555+0000","last_unstale":"2024-01-15T18:07:14.774114+0000","last_undegraded":"2024-01-15T18:07:14.774114+0000","last_fullsized":"2024-01-15T18:07:14.774114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:55.672164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774796+0000","last_change":"2024-01-15T18:07:13.151750+0000","last_active":"2024-01-15T18:07:14.774796+0000","last_peered":"2024-01-15T18:07:14.774796+0000","last_clean":"2024-01-15T18:07:14.774796+0000","last_became_active":"2024-01-15T18:07:13.151403+0000","last_became_peered":"2024-01-15T18:07:13.151403+0000","last_unstale":"2024-01-15T18:07:14.774796+0000","last_undegraded":"2024-01-15T18:07:14.774796+0000","last_fullsized":"2024-01-15T18:07:14.774796+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:41.327706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542105+0000","last_change":"2024-01-15T18:07:13.561187+0000","last_active":"2024-01-15T18:07:14.542105+0000","last_peered":"2024-01-15T18:07:14.542105+0000","last_clean":"2024-01-15T18:07:14.542105+0000","last_became_active":"2024-01-15T18:07:13.559238+0000","last_became_peered":"2024-01-15T18:07:13.559238+0000","last_unstale":"2024-01-15T18:07:14.542105+0000","last_undegraded":"2024-01-15T18:07:14.542105+0000","last_fullsized":"2024-01-15T18:07:14.542105+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:42.336452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774648+0000","last_change":"2024-01-15T18:07:13.147475+0000","last_active":"2024-01-15T18:07:14.774648+0000","last_peered":"2024-01-15T18:07:14.774648+0000","last_clean":"2024-01-15T18:07:14.774648+0000","last_became_active":"2024-01-15T18:07:13.147200+0000","last_became_peered":"2024-01-15T18:07:13.147200+0000","last_unstale":"2024-01-15T18:07:14.774648+0000","last_undegraded":"2024-01-15T18:07:14.774648+0000","last_fullsized":"2024-01-15T18:07:14.774648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:19.327531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774252+0000","last_change":"2024-01-15T18:07:13.553134+0000","last_active":"2024-01-15T18:07:14.774252+0000","last_peered":"2024-01-15T18:07:14.774252+0000","last_clean":"2024-01-15T18:07:14.774252+0000","last_became_active":"2024-01-15T18:07:13.552899+0000","last_became_peered":"2024-01-15T18:07:13.552899+0000","last_unstale":"2024-01-15T18:07:14.774252+0000","last_undegraded":"2024-01-15T18:07:14.774252+0000","last_fullsized":"2024-01-15T18:07:14.774252+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:42.336452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541945+0000","last_change":"2024-01-15T18:07:13.141705+0000","last_active":"2024-01-15T18:07:14.541945+0000","last_peered":"2024-01-15T18:07:14.541945+0000","last_clean":"2024-01-15T18:07:14.541945+0000","last_became_active":"2024-01-15T18:07:13.141399+0000","last_became_peered":"2024-01-15T18:07:13.141399+0000","last_unstale":"2024-01-15T18:07:14.541945+0000","last_undegraded":"2024-01-15T18:07:14.541945+0000","last_fullsized":"2024-01-15T18:07:14.541945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:54.663418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773391+0000","last_change":"2024-01-15T18:07:13.545374+0000","last_active":"2024-01-15T18:07:14.773391+0000","last_peered":"2024-01-15T18:07:14.773391+0000","last_clean":"2024-01-15T18:07:14.773391+0000","last_became_active":"2024-01-15T18:07:13.545106+0000","last_became_peered":"2024-01-15T18:07:13.545106+0000","last_unstale":"2024-01-15T18:07:14.773391+0000","last_undegraded":"2024-01-15T18:07:14.773391+0000","last_fullsized":"2024-01-15T18:07:14.773391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:07:52.124299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543180+0000","last_change":"2024-01-15T18:07:13.143744+0000","last_active":"2024-01-15T18:07:14.543180+0000","last_peered":"2024-01-15T18:07:14.543180+0000","last_clean":"2024-01-15T18:07:14.543180+0000","last_became_active":"2024-01-15T18:07:13.143405+0000","last_became_peered":"2024-01-15T18:07:13.143405+0000","last_unstale":"2024-01-15T18:07:14.543180+0000","last_undegraded":"2024-01-15T18:07:14.543180+0000","last_fullsized":"2024-01-15T18:07:14.543180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:57.207351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541293+0000","last_change":"2024-01-15T18:07:13.560545+0000","last_active":"2024-01-15T18:07:14.541293+0000","last_peered":"2024-01-15T18:07:14.541293+0000","last_clean":"2024-01-15T18:07:14.541293+0000","last_became_active":"2024-01-15T18:07:13.559806+0000","last_became_peered":"2024-01-15T18:07:13.559806+0000","last_unstale":"2024-01-15T18:07:14.541293+0000","last_undegraded":"2024-01-15T18:07:14.541293+0000","last_fullsized":"2024-01-15T18:07:14.541293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:56.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773476+0000","last_change":"2024-01-15T18:07:13.141203+0000","last_active":"2024-01-15T18:07:14.773476+0000","last_peered":"2024-01-15T18:07:14.773476+0000","last_clean":"2024-01-15T18:07:14.773476+0000","last_became_active":"2024-01-15T18:07:13.140933+0000","last_became_peered":"2024-01-15T18:07:13.140933+0000","last_unstale":"2024-01-15T18:07:14.773476+0000","last_undegraded":"2024-01-15T18:07:14.773476+0000","last_fullsized":"2024-01-15T18:07:14.773476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:50.037313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542212+0000","last_change":"2024-01-15T18:07:13.552946+0000","last_active":"2024-01-15T18:07:14.542212+0000","last_peered":"2024-01-15T18:07:14.542212+0000","last_clean":"2024-01-15T18:07:14.542212+0000","last_became_active":"2024-01-15T18:07:13.550975+0000","last_became_peered":"2024-01-15T18:07:13.550975+0000","last_unstale":"2024-01-15T18:07:14.542212+0000","last_undegraded":"2024-01-15T18:07:14.542212+0000","last_fullsized":"2024-01-15T18:07:14.542212+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:08.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542648+0000","last_change":"2024-01-15T18:07:13.136136+0000","last_active":"2024-01-15T18:07:14.542648+0000","last_peered":"2024-01-15T18:07:14.542648+0000","last_clean":"2024-01-15T18:07:14.542648+0000","last_became_active":"2024-01-15T18:07:13.135415+0000","last_became_peered":"2024-01-15T18:07:13.135415+0000","last_unstale":"2024-01-15T18:07:14.542648+0000","last_undegraded":"2024-01-15T18:07:14.542648+0000","last_fullsized":"2024-01-15T18:07:14.542648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:07:51.115553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775144+0000","last_change":"2024-01-15T18:07:13.570342+0000","last_active":"2024-01-15T18:07:14.775144+0000","last_peered":"2024-01-15T18:07:14.775144+0000","last_clean":"2024-01-15T18:07:14.775144+0000","last_became_active":"2024-01-15T18:07:13.570076+0000","last_became_peered":"2024-01-15T18:07:13.570076+0000","last_unstale":"2024-01-15T18:07:14.775144+0000","last_undegraded":"2024-01-15T18:07:14.775144+0000","last_fullsized":"2024-01-15T18:07:14.775144+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:20.316030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774778+0000","last_change":"2024-01-15T18:07:13.144905+0000","last_active":"2024-01-15T18:07:14.774778+0000","last_peered":"2024-01-15T18:07:14.774778+0000","last_clean":"2024-01-15T18:07:14.774778+0000","last_became_active":"2024-01-15T18:07:13.144602+0000","last_became_peered":"2024-01-15T18:07:13.144602+0000","last_unstale":"2024-01-15T18:07:14.774778+0000","last_undegraded":"2024-01-15T18:07:14.774778+0000","last_fullsized":"2024-01-15T18:07:14.774778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:55.876186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775180+0000","last_change":"2024-01-15T18:07:13.560963+0000","last_active":"2024-01-15T18:07:14.775180+0000","last_peered":"2024-01-15T18:07:14.775180+0000","last_clean":"2024-01-15T18:07:14.775180+0000","last_became_active":"2024-01-15T18:07:13.560690+0000","last_became_peered":"2024-01-15T18:07:13.560690+0000","last_unstale":"2024-01-15T18:07:14.775180+0000","last_undegraded":"2024-01-15T18:07:14.775180+0000","last_fullsized":"2024-01-15T18:07:14.775180+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:08:08.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774970+0000","last_change":"2024-01-15T18:07:13.153076+0000","last_active":"2024-01-15T18:07:14.774970+0000","last_peered":"2024-01-15T18:07:14.774970+0000","last_clean":"2024-01-15T18:07:14.774970+0000","last_became_active":"2024-01-15T18:07:13.152768+0000","last_became_peered":"2024-01-15T18:07:13.152768+0000","last_unstale":"2024-01-15T18:07:14.774970+0000","last_undegraded":"2024-01-15T18:07:14.774970+0000","last_fullsized":"2024-01-15T18:07:14.774970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:21.659078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774770+0000","last_change":"2024-01-15T18:07:13.569527+0000","last_active":"2024-01-15T18:07:14.774770+0000","last_peered":"2024-01-15T18:07:14.774770+0000","last_clean":"2024-01-15T18:07:14.774770+0000","last_became_active":"2024-01-15T18:07:13.569076+0000","last_became_peered":"2024-01-15T18:07:13.569076+0000","last_unstale":"2024-01-15T18:07:14.774770+0000","last_undegraded":"2024-01-15T18:07:14.774770+0000","last_fullsized":"2024-01-15T18:07:14.774770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:06:15.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774015+0000","last_change":"2024-01-15T18:07:13.135598+0000","last_active":"2024-01-15T18:07:14.774015+0000","last_peered":"2024-01-15T18:07:14.774015+0000","last_clean":"2024-01-15T18:07:14.774015+0000","last_became_active":"2024-01-15T18:07:13.135334+0000","last_became_peered":"2024-01-15T18:07:13.135334+0000","last_unstale":"2024-01-15T18:07:14.774015+0000","last_undegraded":"2024-01-15T18:07:14.774015+0000","last_fullsized":"2024-01-15T18:07:14.774015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:19.099418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541680+0000","last_change":"2024-01-15T18:07:13.558361+0000","last_active":"2024-01-15T18:07:14.541680+0000","last_peered":"2024-01-15T18:07:14.541680+0000","last_clean":"2024-01-15T18:07:14.541680+0000","last_became_active":"2024-01-15T18:07:13.557736+0000","last_became_peered":"2024-01-15T18:07:13.557736+0000","last_unstale":"2024-01-15T18:07:14.541680+0000","last_undegraded":"2024-01-15T18:07:14.541680+0000","last_fullsized":"2024-01-15T18:07:14.541680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:11.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774691+0000","last_change":"2024-01-15T18:07:13.138892+0000","last_active":"2024-01-15T18:07:14.774691+0000","last_peered":"2024-01-15T18:07:14.774691+0000","last_clean":"2024-01-15T18:07:14.774691+0000","last_became_active":"2024-01-15T18:07:13.137384+0000","last_became_peered":"2024-01-15T18:07:13.137384+0000","last_unstale":"2024-01-15T18:07:14.774691+0000","last_undegraded":"2024-01-15T18:07:14.774691+0000","last_fullsized":"2024-01-15T18:07:14.774691+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:21.659078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774267+0000","last_change":"2024-01-15T18:07:13.555879+0000","last_active":"2024-01-15T18:07:14.774267+0000","last_peered":"2024-01-15T18:07:14.774267+0000","last_clean":"2024-01-15T18:07:14.774267+0000","last_became_active":"2024-01-15T18:07:13.555634+0000","last_became_peered":"2024-01-15T18:07:13.555634+0000","last_unstale":"2024-01-15T18:07:14.774267+0000","last_undegraded":"2024-01-15T18:07:14.774267+0000","last_fullsized":"2024-01-15T18:07:14.774267+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:08.081272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774706+0000","last_change":"2024-01-15T18:07:13.148916+0000","last_active":"2024-01-15T18:07:14.774706+0000","last_peered":"2024-01-15T18:07:14.774706+0000","last_clean":"2024-01-15T18:07:14.774706+0000","last_became_active":"2024-01-15T18:07:13.148524+0000","last_became_peered":"2024-01-15T18:07:13.148524+0000","last_unstale":"2024-01-15T18:07:14.774706+0000","last_undegraded":"2024-01-15T18:07:14.774706+0000","last_fullsized":"2024-01-15T18:07:14.774706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:18.605130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774437+0000","last_change":"2024-01-15T18:07:13.554144+0000","last_active":"2024-01-15T18:07:14.774437+0000","last_peered":"2024-01-15T18:07:14.774437+0000","last_clean":"2024-01-15T18:07:14.774437+0000","last_became_active":"2024-01-15T18:07:13.553831+0000","last_became_peered":"2024-01-15T18:07:13.553831+0000","last_unstale":"2024-01-15T18:07:14.774437+0000","last_undegraded":"2024-01-15T18:07:14.774437+0000","last_fullsized":"2024-01-15T18:07:14.774437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:42.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775054+0000","last_change":"2024-01-15T18:07:13.139960+0000","last_active":"2024-01-15T18:07:14.775054+0000","last_peered":"2024-01-15T18:07:14.775054+0000","last_clean":"2024-01-15T18:07:14.775054+0000","last_became_active":"2024-01-15T18:07:13.136640+0000","last_became_peered":"2024-01-15T18:07:13.136640+0000","last_unstale":"2024-01-15T18:07:14.775054+0000","last_undegraded":"2024-01-15T18:07:14.775054+0000","last_fullsized":"2024-01-15T18:07:14.775054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:21.093486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775013+0000","last_change":"2024-01-15T18:07:13.565438+0000","last_active":"2024-01-15T18:07:14.775013+0000","last_peered":"2024-01-15T18:07:14.775013+0000","last_clean":"2024-01-15T18:07:14.775013+0000","last_became_active":"2024-01-15T18:07:13.565147+0000","last_became_peered":"2024-01-15T18:07:13.565147+0000","last_unstale":"2024-01-15T18:07:14.775013+0000","last_undegraded":"2024-01-15T18:07:14.775013+0000","last_fullsized":"2024-01-15T18:07:14.775013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:46.069477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773565+0000","last_change":"2024-01-15T18:07:13.137547+0000","last_active":"2024-01-15T18:07:14.773565+0000","last_peered":"2024-01-15T18:07:14.773565+0000","last_clean":"2024-01-15T18:07:14.773565+0000","last_became_active":"2024-01-15T18:07:13.137239+0000","last_became_peered":"2024-01-15T18:07:13.137239+0000","last_unstale":"2024-01-15T18:07:14.773565+0000","last_undegraded":"2024-01-15T18:07:14.773565+0000","last_fullsized":"2024-01-15T18:07:14.773565+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:54.663418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774058+0000","last_change":"2024-01-15T18:07:13.562544+0000","last_active":"2024-01-15T18:07:14.774058+0000","last_peered":"2024-01-15T18:07:14.774058+0000","last_clean":"2024-01-15T18:07:14.774058+0000","last_became_active":"2024-01-15T18:07:13.562286+0000","last_became_peered":"2024-01-15T18:07:13.562286+0000","last_unstale":"2024-01-15T18:07:14.774058+0000","last_undegraded":"2024-01-15T18:07:14.774058+0000","last_fullsized":"2024-01-15T18:07:14.774058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:08.982957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773481+0000","last_change":"2024-01-15T18:07:13.139177+0000","last_active":"2024-01-15T18:07:14.773481+0000","last_peered":"2024-01-15T18:07:14.773481+0000","last_clean":"2024-01-15T18:07:14.773481+0000","last_became_active":"2024-01-15T18:07:13.138899+0000","last_became_peered":"2024-01-15T18:07:13.138899+0000","last_unstale":"2024-01-15T18:07:14.773481+0000","last_undegraded":"2024-01-15T18:07:14.773481+0000","last_fullsized":"2024-01-15T18:07:14.773481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774888+0000","last_change":"2024-01-15T18:07:13.561087+0000","last_active":"2024-01-15T18:07:14.774888+0000","last_peered":"2024-01-15T18:07:14.774888+0000","last_clean":"2024-01-15T18:07:14.774888+0000","last_became_active":"2024-01-15T18:07:13.560854+0000","last_became_peered":"2024-01-15T18:07:13.560854+0000","last_unstale":"2024-01-15T18:07:14.774888+0000","last_undegraded":"2024-01-15T18:07:14.774888+0000","last_fullsized":"2024-01-15T18:07:14.774888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:37.407192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542278+0000","last_change":"2024-01-15T18:07:13.144653+0000","last_active":"2024-01-15T18:07:14.542278+0000","last_peered":"2024-01-15T18:07:14.542278+0000","last_clean":"2024-01-15T18:07:14.542278+0000","last_became_active":"2024-01-15T18:07:13.144221+0000","last_became_peered":"2024-01-15T18:07:13.144221+0000","last_unstale":"2024-01-15T18:07:14.542278+0000","last_undegraded":"2024-01-15T18:07:14.542278+0000","last_fullsized":"2024-01-15T18:07:14.542278+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:38:46.176990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541667+0000","last_change":"2024-01-15T18:07:13.560705+0000","last_active":"2024-01-15T18:07:14.541667+0000","last_peered":"2024-01-15T18:07:14.541667+0000","last_clean":"2024-01-15T18:07:14.541667+0000","last_became_active":"2024-01-15T18:07:13.560263+0000","last_became_peered":"2024-01-15T18:07:13.560263+0000","last_unstale":"2024-01-15T18:07:14.541667+0000","last_undegraded":"2024-01-15T18:07:14.541667+0000","last_fullsized":"2024-01-15T18:07:14.541667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:43.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543562+0000","last_change":"2024-01-15T18:07:13.142213+0000","last_active":"2024-01-15T18:07:14.543562+0000","last_peered":"2024-01-15T18:07:14.543562+0000","last_clean":"2024-01-15T18:07:14.543562+0000","last_became_active":"2024-01-15T18:07:13.141943+0000","last_became_peered":"2024-01-15T18:07:13.141943+0000","last_unstale":"2024-01-15T18:07:14.543562+0000","last_undegraded":"2024-01-15T18:07:14.543562+0000","last_fullsized":"2024-01-15T18:07:14.543562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:14.240502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541986+0000","last_change":"2024-01-15T18:07:13.567698+0000","last_active":"2024-01-15T18:07:14.541986+0000","last_peered":"2024-01-15T18:07:14.541986+0000","last_clean":"2024-01-15T18:07:14.541986+0000","last_became_active":"2024-01-15T18:07:13.567421+0000","last_became_peered":"2024-01-15T18:07:13.567421+0000","last_unstale":"2024-01-15T18:07:14.541986+0000","last_undegraded":"2024-01-15T18:07:14.541986+0000","last_fullsized":"2024-01-15T18:07:14.541986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:56.884932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775254+0000","last_change":"2024-01-15T18:07:13.143810+0000","last_active":"2024-01-15T18:07:14.775254+0000","last_peered":"2024-01-15T18:07:14.775254+0000","last_clean":"2024-01-15T18:07:14.775254+0000","last_became_active":"2024-01-15T18:07:13.143480+0000","last_became_peered":"2024-01-15T18:07:13.143480+0000","last_unstale":"2024-01-15T18:07:14.775254+0000","last_undegraded":"2024-01-15T18:07:14.775254+0000","last_fullsized":"2024-01-15T18:07:14.775254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:43.404150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774844+0000","last_change":"2024-01-15T18:07:13.557697+0000","last_active":"2024-01-15T18:07:14.774844+0000","last_peered":"2024-01-15T18:07:14.774844+0000","last_clean":"2024-01-15T18:07:14.774844+0000","last_became_active":"2024-01-15T18:07:13.557436+0000","last_became_peered":"2024-01-15T18:07:13.557436+0000","last_unstale":"2024-01-15T18:07:14.774844+0000","last_undegraded":"2024-01-15T18:07:14.774844+0000","last_fullsized":"2024-01-15T18:07:14.774844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:28.246734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774771+0000","last_change":"2024-01-15T18:07:13.145389+0000","last_active":"2024-01-15T18:07:14.774771+0000","last_peered":"2024-01-15T18:07:14.774771+0000","last_clean":"2024-01-15T18:07:14.774771+0000","last_became_active":"2024-01-15T18:07:13.145110+0000","last_became_peered":"2024-01-15T18:07:13.145110+0000","last_unstale":"2024-01-15T18:07:14.774771+0000","last_undegraded":"2024-01-15T18:07:14.774771+0000","last_fullsized":"2024-01-15T18:07:14.774771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:42.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773907+0000","last_change":"2024-01-15T18:07:13.563791+0000","last_active":"2024-01-15T18:07:14.773907+0000","last_peered":"2024-01-15T18:07:14.773907+0000","last_clean":"2024-01-15T18:07:14.773907+0000","last_became_active":"2024-01-15T18:07:13.563499+0000","last_became_peered":"2024-01-15T18:07:13.563499+0000","last_unstale":"2024-01-15T18:07:14.773907+0000","last_undegraded":"2024-01-15T18:07:14.773907+0000","last_fullsized":"2024-01-15T18:07:14.773907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:20.253082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543216+0000","last_change":"2024-01-15T18:07:13.140448+0000","last_active":"2024-01-15T18:07:14.543216+0000","last_peered":"2024-01-15T18:07:14.543216+0000","last_clean":"2024-01-15T18:07:14.543216+0000","last_became_active":"2024-01-15T18:07:13.139194+0000","last_became_peered":"2024-01-15T18:07:13.139194+0000","last_unstale":"2024-01-15T18:07:14.543216+0000","last_undegraded":"2024-01-15T18:07:14.543216+0000","last_fullsized":"2024-01-15T18:07:14.543216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:05.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773323+0000","last_change":"2024-01-15T18:07:13.553150+0000","last_active":"2024-01-15T18:07:14.773323+0000","last_peered":"2024-01-15T18:07:14.773323+0000","last_clean":"2024-01-15T18:07:14.773323+0000","last_became_active":"2024-01-15T18:07:13.552870+0000","last_became_peered":"2024-01-15T18:07:13.552870+0000","last_unstale":"2024-01-15T18:07:14.773323+0000","last_undegraded":"2024-01-15T18:07:14.773323+0000","last_fullsized":"2024-01-15T18:07:14.773323+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:29:34.507041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773835+0000","last_change":"2024-01-15T18:07:13.138368+0000","last_active":"2024-01-15T18:07:14.773835+0000","last_peered":"2024-01-15T18:07:14.773835+0000","last_clean":"2024-01-15T18:07:14.773835+0000","last_became_active":"2024-01-15T18:07:13.138032+0000","last_became_peered":"2024-01-15T18:07:13.138032+0000","last_unstale":"2024-01-15T18:07:14.773835+0000","last_undegraded":"2024-01-15T18:07:14.773835+0000","last_fullsized":"2024-01-15T18:07:14.773835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:29:33.498295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773758+0000","last_change":"2024-01-15T18:07:13.558500+0000","last_active":"2024-01-15T18:07:14.773758+0000","last_peered":"2024-01-15T18:07:14.773758+0000","last_clean":"2024-01-15T18:07:14.773758+0000","last_became_active":"2024-01-15T18:07:13.557620+0000","last_became_peered":"2024-01-15T18:07:13.557620+0000","last_unstale":"2024-01-15T18:07:14.773758+0000","last_undegraded":"2024-01-15T18:07:14.773758+0000","last_fullsized":"2024-01-15T18:07:14.773758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:27.810935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774532+0000","last_change":"2024-01-15T18:07:13.148670+0000","last_active":"2024-01-15T18:07:14.774532+0000","last_peered":"2024-01-15T18:07:14.774532+0000","last_clean":"2024-01-15T18:07:14.774532+0000","last_became_active":"2024-01-15T18:07:13.148342+0000","last_became_peered":"2024-01-15T18:07:13.148342+0000","last_unstale":"2024-01-15T18:07:14.774532+0000","last_undegraded":"2024-01-15T18:07:14.774532+0000","last_fullsized":"2024-01-15T18:07:14.774532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:19.327531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542821+0000","last_change":"2024-01-15T18:07:13.550052+0000","last_active":"2024-01-15T18:07:14.542821+0000","last_peered":"2024-01-15T18:07:14.542821+0000","last_clean":"2024-01-15T18:07:14.542821+0000","last_became_active":"2024-01-15T18:07:13.549747+0000","last_became_peered":"2024-01-15T18:07:13.549747+0000","last_unstale":"2024-01-15T18:07:14.542821+0000","last_undegraded":"2024-01-15T18:07:14.542821+0000","last_fullsized":"2024-01-15T18:07:14.542821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:55.672164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.776330+0000","last_change":"2024-01-15T18:07:13.145472+0000","last_active":"2024-01-15T18:07:14.776330+0000","last_peered":"2024-01-15T18:07:14.776330+0000","last_clean":"2024-01-15T18:07:14.776330+0000","last_became_active":"2024-01-15T18:07:13.145151+0000","last_became_peered":"2024-01-15T18:07:13.145151+0000","last_unstale":"2024-01-15T18:07:14.776330+0000","last_undegraded":"2024-01-15T18:07:14.776330+0000","last_fullsized":"2024-01-15T18:07:14.776330+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:52.070623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774409+0000","last_change":"2024-01-15T18:07:13.547632+0000","last_active":"2024-01-15T18:07:14.774409+0000","last_peered":"2024-01-15T18:07:14.774409+0000","last_clean":"2024-01-15T18:07:14.774409+0000","last_became_active":"2024-01-15T18:07:13.547354+0000","last_became_peered":"2024-01-15T18:07:13.547354+0000","last_unstale":"2024-01-15T18:07:14.774409+0000","last_undegraded":"2024-01-15T18:07:14.774409+0000","last_fullsized":"2024-01-15T18:07:14.774409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:06.472843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775040+0000","last_change":"2024-01-15T18:07:13.152981+0000","last_active":"2024-01-15T18:07:14.775040+0000","last_peered":"2024-01-15T18:07:14.775040+0000","last_clean":"2024-01-15T18:07:14.775040+0000","last_became_active":"2024-01-15T18:07:13.152735+0000","last_became_peered":"2024-01-15T18:07:13.152735+0000","last_unstale":"2024-01-15T18:07:14.775040+0000","last_undegraded":"2024-01-15T18:07:14.775040+0000","last_fullsized":"2024-01-15T18:07:14.775040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:42.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774975+0000","last_change":"2024-01-15T18:07:13.568915+0000","last_active":"2024-01-15T18:07:14.774975+0000","last_peered":"2024-01-15T18:07:14.774975+0000","last_clean":"2024-01-15T18:07:14.774975+0000","last_became_active":"2024-01-15T18:07:13.567703+0000","last_became_peered":"2024-01-15T18:07:13.567703+0000","last_unstale":"2024-01-15T18:07:14.774975+0000","last_undegraded":"2024-01-15T18:07:14.774975+0000","last_fullsized":"2024-01-15T18:07:14.774975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:36.540360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773425+0000","last_change":"2024-01-15T18:07:13.143026+0000","last_active":"2024-01-15T18:07:14.773425+0000","last_peered":"2024-01-15T18:07:14.773425+0000","last_clean":"2024-01-15T18:07:14.773425+0000","last_became_active":"2024-01-15T18:07:13.142687+0000","last_became_peered":"2024-01-15T18:07:13.142687+0000","last_unstale":"2024-01-15T18:07:14.773425+0000","last_undegraded":"2024-01-15T18:07:14.773425+0000","last_fullsized":"2024-01-15T18:07:14.773425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:55.876186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774904+0000","last_change":"2024-01-15T18:07:13.558506+0000","last_active":"2024-01-15T18:07:14.774904+0000","last_peered":"2024-01-15T18:07:14.774904+0000","last_clean":"2024-01-15T18:07:14.774904+0000","last_became_active":"2024-01-15T18:07:13.558261+0000","last_became_peered":"2024-01-15T18:07:13.558261+0000","last_unstale":"2024-01-15T18:07:14.774904+0000","last_undegraded":"2024-01-15T18:07:14.774904+0000","last_fullsized":"2024-01-15T18:07:14.774904+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:46:06.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774816+0000","last_change":"2024-01-15T18:07:13.153254+0000","last_active":"2024-01-15T18:07:14.774816+0000","last_peered":"2024-01-15T18:07:14.774816+0000","last_clean":"2024-01-15T18:07:14.774816+0000","last_became_active":"2024-01-15T18:07:13.152937+0000","last_became_peered":"2024-01-15T18:07:13.152937+0000","last_unstale":"2024-01-15T18:07:14.774816+0000","last_undegraded":"2024-01-15T18:07:14.774816+0000","last_fullsized":"2024-01-15T18:07:14.774816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:14.240502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774602+0000","last_change":"2024-01-15T18:07:13.554436+0000","last_active":"2024-01-15T18:07:14.774602+0000","last_peered":"2024-01-15T18:07:14.774602+0000","last_clean":"2024-01-15T18:07:14.774602+0000","last_became_active":"2024-01-15T18:07:13.554156+0000","last_became_peered":"2024-01-15T18:07:13.554156+0000","last_unstale":"2024-01-15T18:07:14.774602+0000","last_undegraded":"2024-01-15T18:07:14.774602+0000","last_fullsized":"2024-01-15T18:07:14.774602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:52:24.006274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542033+0000","last_change":"2024-01-15T18:07:13.132084+0000","last_active":"2024-01-15T18:07:14.542033+0000","last_peered":"2024-01-15T18:07:14.542033+0000","last_clean":"2024-01-15T18:07:14.542033+0000","last_became_active":"2024-01-15T18:07:13.131783+0000","last_became_peered":"2024-01-15T18:07:13.131783+0000","last_unstale":"2024-01-15T18:07:14.542033+0000","last_undegraded":"2024-01-15T18:07:14.542033+0000","last_fullsized":"2024-01-15T18:07:14.542033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:57:50.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543321+0000","last_change":"2024-01-15T18:07:13.562769+0000","last_active":"2024-01-15T18:07:14.543321+0000","last_peered":"2024-01-15T18:07:14.543321+0000","last_clean":"2024-01-15T18:07:14.543321+0000","last_became_active":"2024-01-15T18:07:13.562497+0000","last_became_peered":"2024-01-15T18:07:13.562497+0000","last_unstale":"2024-01-15T18:07:14.543321+0000","last_undegraded":"2024-01-15T18:07:14.543321+0000","last_fullsized":"2024-01-15T18:07:14.543321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:37.407192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773745+0000","last_change":"2024-01-15T18:07:13.141244+0000","last_active":"2024-01-15T18:07:14.773745+0000","last_peered":"2024-01-15T18:07:14.773745+0000","last_clean":"2024-01-15T18:07:14.773745+0000","last_became_active":"2024-01-15T18:07:13.140989+0000","last_became_peered":"2024-01-15T18:07:13.140989+0000","last_unstale":"2024-01-15T18:07:14.773745+0000","last_undegraded":"2024-01-15T18:07:14.773745+0000","last_fullsized":"2024-01-15T18:07:14.773745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:10.344122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541640+0000","last_change":"2024-01-15T18:07:13.556894+0000","last_active":"2024-01-15T18:07:14.541640+0000","last_peered":"2024-01-15T18:07:14.541640+0000","last_clean":"2024-01-15T18:07:14.541640+0000","last_became_active":"2024-01-15T18:07:13.556530+0000","last_became_peered":"2024-01-15T18:07:13.556530+0000","last_unstale":"2024-01-15T18:07:14.541640+0000","last_undegraded":"2024-01-15T18:07:14.541640+0000","last_fullsized":"2024-01-15T18:07:14.541640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:04.463119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773646+0000","last_change":"2024-01-15T18:07:13.134762+0000","last_active":"2024-01-15T18:07:14.773646+0000","last_peered":"2024-01-15T18:07:14.773646+0000","last_clean":"2024-01-15T18:07:14.773646+0000","last_became_active":"2024-01-15T18:07:13.134496+0000","last_became_peered":"2024-01-15T18:07:13.134496+0000","last_unstale":"2024-01-15T18:07:14.773646+0000","last_undegraded":"2024-01-15T18:07:14.773646+0000","last_fullsized":"2024-01-15T18:07:14.773646+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:57:50.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774724+0000","last_change":"2024-01-15T18:07:13.562552+0000","last_active":"2024-01-15T18:07:14.774724+0000","last_peered":"2024-01-15T18:07:14.774724+0000","last_clean":"2024-01-15T18:07:14.774724+0000","last_became_active":"2024-01-15T18:07:13.562292+0000","last_became_peered":"2024-01-15T18:07:13.562292+0000","last_unstale":"2024-01-15T18:07:14.774724+0000","last_undegraded":"2024-01-15T18:07:14.774724+0000","last_fullsized":"2024-01-15T18:07:14.774724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:22.102232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543658+0000","last_change":"2024-01-15T18:07:13.145974+0000","last_active":"2024-01-15T18:07:14.543658+0000","last_peered":"2024-01-15T18:07:14.543658+0000","last_clean":"2024-01-15T18:07:14.543658+0000","last_became_active":"2024-01-15T18:07:13.145622+0000","last_became_peered":"2024-01-15T18:07:13.145622+0000","last_unstale":"2024-01-15T18:07:14.543658+0000","last_undegraded":"2024-01-15T18:07:14.543658+0000","last_fullsized":"2024-01-15T18:07:14.543658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:21.093486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-01-15T18:07:13.555978+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:07:13.555978+0000","last_peered":"2024-01-15T18:07:13.555978+0000","last_clean":"2024-01-15T18:07:13.555978+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:07:13.555978+0000","last_undegraded":"2024-01-15T18:07:13.555978+0000","last_fullsized":"2024-01-15T18:07:13.555978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":812277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.74299999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77000000000000002,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.59899999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1419999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.63600000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.51200000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.747,"15min":0.76600000000000001},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.55700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":222491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.877}]},{"osd":1,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.85699999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92100000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83699999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.89500000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.746},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.95399999999999996,"5min":1.107,"15min":1.107},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97799999999999998,"5min":1.401,"15min":1.401},"last":0.77200000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.74199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.67000000000000004,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.79600000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.032,"5min":5.032,"15min":5.032},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69399999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.97799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.78300000000000003,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.82999999999999996,"15min":0.81399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.95599999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":223431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502}]},{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90000000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94099999999999995,"5min":1.054,"15min":1.054},"last":0.62}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.58499999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83999999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90400000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":1.218,"15min":1.218},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":1.179,"15min":1.179},"last":0.748}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70599999999999996,"15min":0.72199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.78000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":222457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.52700000000000002,"15min":0.47099999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.47599999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.444,"5min":3.444,"15min":3.444},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.181,"5min":7.181,"15min":7.181},"last":0.47899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.61299999999999999,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.57099999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.83199999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.81399999999999995,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.94399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.875,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.92800000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.91400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":223453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.47399999999999998,"15min":0.45200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.4880000000000004,"5min":5.4880000000000004,"15min":5.4880000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.622}]},{"osd":3,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.48399999999999999,"15min":0.46000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.497},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81200000000000006,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.88900000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.72399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":1.056},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":3.2010000000000001,"5min":3.2010000000000001,"15min":3.2010000000000001},"last":1.077}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":1.0960000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.89500000000000002,"15min":0.876},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":1.1319999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2570000000000001,"5min":4.532,"15min":4.532},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.0409999999999999,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.53400000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.438},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.3610000000000002,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.59799999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.92100000000000004,"15min":0.81299999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":21.416,"5min":21.416,"15min":21.416},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.101,"5min":0.90900000000000003,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.022},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.238,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.0209999999999999,"15min":0.98299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":12.997,"5min":12.997,"15min":12.997},"last":0.86899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.123},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":13.054,"5min":13.054,"15min":13.054},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.2170000000000001,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.876,"5min":1.008,"15min":1.03},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.94099999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.7729999999999997,"5min":4.7729999999999997,"15min":4.7729999999999997},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.274,"5min":1.1060000000000001,"15min":1.0780000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":13.089,"5min":13.089,"15min":13.089},"last":0.78400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":812221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.79000000000000004,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.629},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.65300000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.81999999999999995,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.85299999999999998,"15min":0.91400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98199999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.88300000000000001,"15min":0.91800000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.753},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.876}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.92600000000000005,"15min":0.96999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1240000000000001,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91800000000000004,"15min":0.95399999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":4.056,"15min":4.056},"last":0.91800000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0529999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1399999999999999,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.79800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.026,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.95399999999999996,"15min":1},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93400000000000005,"5min":4.407,"15min":4.407},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:07:22.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:07:22.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.852+0000 7f41d5600640 1 Processor -- start 2024-01-15T18:07:22.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.852+0000 7f41d5600640 1 -- start start 2024-01-15T18:07:22.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41d5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 0x7f41d00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41d5600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d00b2a20 con 0x7f41d00b48f0 2024-01-15T18:07:22.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41d5600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d00b2d00 con 0x7f41d0157300 2024-01-15T18:07:22.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41d5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b35a0 con 0x7f41d0159b70 2024-01-15T18:07:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cf400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f41d00b48f0 0x7f41d0070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50060/0 (socket says 172.21.15.143:50060) 2024-01-15T18:07:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cf400640 1 -- 172.21.15.143:0/3463753701 learned_addr learned my addr 172.21.15.143:0/3463753701 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41ce000640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 0x7f41d00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425773585 0 0) 0x7f41d00b2a20 con 0x7f41d00b48f0 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b0003530 con 0x7f41d00b48f0 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843154984 0 0) 0x7f41d00b2d00 con 0x7f41d0157300 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41d00b2a20 con 0x7f41d0157300 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 708058899 0 0) 0x7f41b0003530 con 0x7f41d00b48f0 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 msgr2=0x7f41d00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 0x7f41d00b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 >> v1:172.21.15.204:6789/0 conn(0x7f41d0157300 legacy=0x7f41d00b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00b2fe0 con 0x7f41d00b48f0 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f41c8001a00 con 0x7f41d00b48f0 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 >> v1:172.21.15.143:6789/0 conn(0x7f41d00b48f0 legacy=0x7f41d0070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41b0003b40 0x7f41b0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 0x7f41b00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b0004d10 0x7f41b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b35a0 con 0x7f41b0003b40 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b000ba40 con 0x7f41b0004d10 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b000bc00 con 0x7f41d0159b70 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cea00640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41b0003b40 0x7f41b0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41ce000640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 0x7f41b00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cf400640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b0004d10 0x7f41b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cea00640 1 -- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 msgr2=0x7f41b00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cea00640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41d0159b70 0x7f41b00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cea00640 1 -- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b0004d10 msgr2=0x7f41b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cea00640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41b0004d10 0x7f41b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.856+0000 7f41cea00640 1 -- 172.21.15.143:0/3463753701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b0003530 con 0x7f41b0003b40 2024-01-15T18:07:22.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41cea00640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41b0003b40 0x7f41b0003f40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f41c00018d0 tx=0x7f41c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d9c5e60b383f93 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c00052d0 con 0x7f41b0003b40 2024-01-15T18:07:22.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c00a7050 con 0x7f41b0003b40 2024-01-15T18:07:22.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41cd600640 1 -- 172.21.15.143:0/3463753701 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c0003600 con 0x7f41b0003b40 2024-01-15T18:07:22.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41d5600640 1 -- 172.21.15.143:0/3463753701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41b0003b40 msgr2=0x7f41b0003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41d5600640 1 --2- 172.21.15.143:0/3463753701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41b0003b40 0x7f41b0003f40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f41c00018d0 tx=0x7f41c001c560 comp rx=0 tx=0).stop 2024-01-15T18:07:22.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41cf400640 1 -- 172.21.15.143:0/3463753701 reap_dead start 2024-01-15T18:07:22.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41d5600640 1 -- 172.21.15.143:0/3463753701 shutdown_connections 2024-01-15T18:07:22.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41d5600640 1 -- 172.21.15.143:0/3463753701 >> 172.21.15.143:0/3463753701 conn(0x7f41d006c3f0 msgr2=0x7f41d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:22.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41d5600640 1 -- 172.21.15.143:0/3463753701 shutdown_connections 2024-01-15T18:07:22.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.860+0000 7f41d5600640 1 -- 172.21.15.143:0/3463753701 wait complete. 2024-01-15T18:07:22.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 Processor -- start 2024-01-15T18:07:22.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 -- start start 2024-01-15T18:07:22.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b0003b40 0x7f41d00c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41d01f9e90 0x7f41d00c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 0x7f41d00c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d0159fd0 con 0x7f41d01fa310 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d01f7180 con 0x7f41d01f9e90 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00c6b60 con 0x7f41b0003b40 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 0x7f41d00c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b0003b40 0x7f41d00c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 0x7f41d00c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47176/0 (socket says 172.21.15.143:47176) 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 -- 172.21.15.143:0/2634700136 learned_addr learned my addr 172.21.15.143:0/2634700136 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:22.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41ce000640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41d01f9e90 0x7f41d00c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 -- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b0003b40 msgr2=0x7f41d00c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b0003b40 0x7f41d00c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 -- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41d01f9e90 msgr2=0x7f41d00c41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41d01f9e90 0x7f41d00c41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 -- 172.21.15.143:0/2634700136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0160130 con 0x7f41d01fa310 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41cf400640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 0x7f41d00c46e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f41c8007740 tx=0x7f41c8007b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c800aa40 con 0x7f41d01fa310 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c800abc0 con 0x7f41d01fa310 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80133f0 con 0x7f41d01fa310 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d01f68e0 con 0x7f41d01fa310 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.864+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d01f6ea0 con 0x7f41d01fa310 2024-01-15T18:07:22.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.868+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d0067500 con 0x7f41d01fa310 2024-01-15T18:07:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.868+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f41c8013550 con 0x7f41d01fa310 2024-01-15T18:07:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.868+0000 7f41c6a00640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41a8066b60 0x7f41a8069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.868+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f41c8084d80 con 0x7f41d01fa310 2024-01-15T18:07:22.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.868+0000 7f41cea00640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41a8066b60 0x7f41a8069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.868+0000 7f41cea00640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41a8066b60 0x7f41a8069020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f41c00ac000 tx=0x7f41c00a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:22.872+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f41c8051ba0 con 0x7f41d01fa310 2024-01-15T18:07:23.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.020+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41d006e4e0 con 0x7f41a8066b60 2024-01-15T18:07:23.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.024+0000 7f41c6a00640 1 -- 172.21.15.143:0/2634700136 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288618 (secure 0 0 0) 0x7f41d006e4e0 con 0x7f41a8066b60 2024-01-15T18:07:23.037 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41a8066b60 msgr2=0x7f41a8069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41a8066b60 0x7f41a8069020 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f41c00ac000 tx=0x7f41c00a6000 comp rx=0 tx=0).stop 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 msgr2=0x7f41d00c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 0x7f41d00c46e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f41c8007740 tx=0x7f41c8007b40 comp rx=0 tx=0).stop 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 shutdown_connections 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f41a8066b60 0x7f41a8069020 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f41d01fa310 0x7f41d00c46e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f41d01f9e90 0x7f41d00c41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 --2- 172.21.15.143:0/2634700136 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f41b0003b40 0x7f41d00c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 >> 172.21.15.143:0/2634700136 conn(0x7f41d006c3f0 msgr2=0x7f41d00cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 shutdown_connections 2024-01-15T18:07:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.028+0000 7f41d5600640 1 -- 172.21.15.143:0/2634700136 wait complete. 2024-01-15T18:07:23.058 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-01-15T18:07:22.651828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254920,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719800,"statfs":{"total":773094113280,"available":772833075200,"internally_reserved":0,"allocated":5578752,"data_stored":2960108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.114641"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775090+0000","last_change":"2024-01-15T18:07:13.565379+0000","last_active":"2024-01-15T18:07:14.775090+0000","last_peered":"2024-01-15T18:07:14.775090+0000","last_clean":"2024-01-15T18:07:14.775090+0000","last_became_active":"2024-01-15T18:07:13.565044+0000","last_became_peered":"2024-01-15T18:07:13.565044+0000","last_unstale":"2024-01-15T18:07:14.775090+0000","last_undegraded":"2024-01-15T18:07:14.775090+0000","last_fullsized":"2024-01-15T18:07:14.775090+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:42.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543575+0000","last_change":"2024-01-15T18:07:13.566360+0000","last_active":"2024-01-15T18:07:14.543575+0000","last_peered":"2024-01-15T18:07:14.543575+0000","last_clean":"2024-01-15T18:07:14.543575+0000","last_became_active":"2024-01-15T18:07:13.566039+0000","last_became_peered":"2024-01-15T18:07:13.566039+0000","last_unstale":"2024-01-15T18:07:14.543575+0000","last_undegraded":"2024-01-15T18:07:14.543575+0000","last_fullsized":"2024-01-15T18:07:14.543575+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:20.336277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542050+0000","last_change":"2024-01-15T18:07:13.552196+0000","last_active":"2024-01-15T18:07:14.542050+0000","last_peered":"2024-01-15T18:07:14.542050+0000","last_clean":"2024-01-15T18:07:14.542050+0000","last_became_active":"2024-01-15T18:07:13.551879+0000","last_became_peered":"2024-01-15T18:07:13.551879+0000","last_unstale":"2024-01-15T18:07:14.542050+0000","last_undegraded":"2024-01-15T18:07:14.542050+0000","last_fullsized":"2024-01-15T18:07:14.542050+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:53.079369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774299+0000","last_change":"2024-01-15T18:07:13.547096+0000","last_active":"2024-01-15T18:07:14.774299+0000","last_peered":"2024-01-15T18:07:14.774299+0000","last_clean":"2024-01-15T18:07:14.774299+0000","last_became_active":"2024-01-15T18:07:13.546783+0000","last_became_peered":"2024-01-15T18:07:13.546783+0000","last_unstale":"2024-01-15T18:07:14.774299+0000","last_undegraded":"2024-01-15T18:07:14.774299+0000","last_fullsized":"2024-01-15T18:07:14.774299+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:08.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543306+0000","last_change":"2024-01-15T18:07:13.559790+0000","last_active":"2024-01-15T18:07:14.543306+0000","last_peered":"2024-01-15T18:07:14.543306+0000","last_clean":"2024-01-15T18:07:14.543306+0000","last_became_active":"2024-01-15T18:07:13.559024+0000","last_became_peered":"2024-01-15T18:07:13.559024+0000","last_unstale":"2024-01-15T18:07:14.543306+0000","last_undegraded":"2024-01-15T18:07:14.543306+0000","last_fullsized":"2024-01-15T18:07:14.543306+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:19.613876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774435+0000","last_change":"2024-01-15T18:07:13.557998+0000","last_active":"2024-01-15T18:07:14.774435+0000","last_peered":"2024-01-15T18:07:14.774435+0000","last_clean":"2024-01-15T18:07:14.774435+0000","last_became_active":"2024-01-15T18:07:13.557712+0000","last_became_peered":"2024-01-15T18:07:13.557712+0000","last_unstale":"2024-01-15T18:07:14.774435+0000","last_undegraded":"2024-01-15T18:07:14.774435+0000","last_fullsized":"2024-01-15T18:07:14.774435+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:46.069477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774823+0000","last_change":"2024-01-15T18:07:13.567253+0000","last_active":"2024-01-15T18:07:14.774823+0000","last_peered":"2024-01-15T18:07:14.774823+0000","last_clean":"2024-01-15T18:07:14.774823+0000","last_became_active":"2024-01-15T18:07:13.566953+0000","last_became_peered":"2024-01-15T18:07:13.566953+0000","last_unstale":"2024-01-15T18:07:14.774823+0000","last_undegraded":"2024-01-15T18:07:14.774823+0000","last_fullsized":"2024-01-15T18:07:14.774823+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:44.412896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774145+0000","last_change":"2024-01-15T18:07:13.545892+0000","last_active":"2024-01-15T18:07:14.774145+0000","last_peered":"2024-01-15T18:07:14.774145+0000","last_clean":"2024-01-15T18:07:14.774145+0000","last_became_active":"2024-01-15T18:07:13.545624+0000","last_became_peered":"2024-01-15T18:07:13.545624+0000","last_unstale":"2024-01-15T18:07:14.774145+0000","last_undegraded":"2024-01-15T18:07:14.774145+0000","last_fullsized":"2024-01-15T18:07:14.774145+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:53.079369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774766+0000","last_change":"2024-01-15T18:07:13.565934+0000","last_active":"2024-01-15T18:07:14.774766+0000","last_peered":"2024-01-15T18:07:14.774766+0000","last_clean":"2024-01-15T18:07:14.774766+0000","last_became_active":"2024-01-15T18:07:13.565348+0000","last_became_peered":"2024-01-15T18:07:13.565348+0000","last_unstale":"2024-01-15T18:07:14.774766+0000","last_undegraded":"2024-01-15T18:07:14.774766+0000","last_fullsized":"2024-01-15T18:07:14.774766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:19.613876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774608+0000","last_change":"2024-01-15T18:07:13.545220+0000","last_active":"2024-01-15T18:07:14.774608+0000","last_peered":"2024-01-15T18:07:14.774608+0000","last_clean":"2024-01-15T18:07:14.774608+0000","last_became_active":"2024-01-15T18:07:13.544957+0000","last_became_peered":"2024-01-15T18:07:13.544957+0000","last_unstale":"2024-01-15T18:07:14.774608+0000","last_undegraded":"2024-01-15T18:07:14.774608+0000","last_fullsized":"2024-01-15T18:07:14.774608+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:58.216097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774680+0000","last_change":"2024-01-15T18:07:13.565635+0000","last_active":"2024-01-15T18:07:14.774680+0000","last_peered":"2024-01-15T18:07:14.774680+0000","last_clean":"2024-01-15T18:07:14.774680+0000","last_became_active":"2024-01-15T18:07:13.565356+0000","last_became_peered":"2024-01-15T18:07:13.565356+0000","last_unstale":"2024-01-15T18:07:14.774680+0000","last_undegraded":"2024-01-15T18:07:14.774680+0000","last_fullsized":"2024-01-15T18:07:14.774680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:15.249248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543524+0000","last_change":"2024-01-15T18:07:13.562099+0000","last_active":"2024-01-15T18:07:14.543524+0000","last_peered":"2024-01-15T18:07:14.543524+0000","last_clean":"2024-01-15T18:07:14.543524+0000","last_became_active":"2024-01-15T18:07:13.561790+0000","last_became_peered":"2024-01-15T18:07:13.561790+0000","last_unstale":"2024-01-15T18:07:14.543524+0000","last_undegraded":"2024-01-15T18:07:14.543524+0000","last_fullsized":"2024-01-15T18:07:14.543524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:43.794608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773525+0000","last_change":"2024-01-15T18:07:13.560152+0000","last_active":"2024-01-15T18:07:14.773525+0000","last_peered":"2024-01-15T18:07:14.773525+0000","last_clean":"2024-01-15T18:07:14.773525+0000","last_became_active":"2024-01-15T18:07:13.559682+0000","last_became_peered":"2024-01-15T18:07:13.559682+0000","last_unstale":"2024-01-15T18:07:14.773525+0000","last_undegraded":"2024-01-15T18:07:14.773525+0000","last_fullsized":"2024-01-15T18:07:14.773525+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:14:35.267226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773436+0000","last_change":"2024-01-15T18:07:13.562323+0000","last_active":"2024-01-15T18:07:14.773436+0000","last_peered":"2024-01-15T18:07:14.773436+0000","last_clean":"2024-01-15T18:07:14.773436+0000","last_became_active":"2024-01-15T18:07:13.562052+0000","last_became_peered":"2024-01-15T18:07:13.562052+0000","last_unstale":"2024-01-15T18:07:14.773436+0000","last_undegraded":"2024-01-15T18:07:14.773436+0000","last_fullsized":"2024-01-15T18:07:14.773436+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:15.081589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774888+0000","last_change":"2024-01-15T18:07:13.555476+0000","last_active":"2024-01-15T18:07:14.774888+0000","last_peered":"2024-01-15T18:07:14.774888+0000","last_clean":"2024-01-15T18:07:14.774888+0000","last_became_active":"2024-01-15T18:07:13.555192+0000","last_became_peered":"2024-01-15T18:07:13.555192+0000","last_unstale":"2024-01-15T18:07:14.774888+0000","last_undegraded":"2024-01-15T18:07:14.774888+0000","last_fullsized":"2024-01-15T18:07:14.774888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:50:27.459870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541451+0000","last_change":"2024-01-15T18:07:13.545439+0000","last_active":"2024-01-15T18:07:14.541451+0000","last_peered":"2024-01-15T18:07:14.541451+0000","last_clean":"2024-01-15T18:07:14.541451+0000","last_became_active":"2024-01-15T18:07:13.545109+0000","last_became_peered":"2024-01-15T18:07:13.545109+0000","last_unstale":"2024-01-15T18:07:14.541451+0000","last_undegraded":"2024-01-15T18:07:14.541451+0000","last_fullsized":"2024-01-15T18:07:14.541451+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:12:37.185901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542612+0000","last_change":"2024-01-15T18:07:13.564080+0000","last_active":"2024-01-15T18:07:14.542612+0000","last_peered":"2024-01-15T18:07:14.542612+0000","last_clean":"2024-01-15T18:07:14.542612+0000","last_became_active":"2024-01-15T18:07:13.563671+0000","last_became_peered":"2024-01-15T18:07:13.563671+0000","last_unstale":"2024-01-15T18:07:14.542612+0000","last_undegraded":"2024-01-15T18:07:14.542612+0000","last_fullsized":"2024-01-15T18:07:14.542612+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:58.216097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542189+0000","last_change":"2024-01-15T18:07:13.552275+0000","last_active":"2024-01-15T18:07:14.542189+0000","last_peered":"2024-01-15T18:07:14.542189+0000","last_clean":"2024-01-15T18:07:14.542189+0000","last_became_active":"2024-01-15T18:07:13.551396+0000","last_became_peered":"2024-01-15T18:07:13.551396+0000","last_unstale":"2024-01-15T18:07:14.542189+0000","last_undegraded":"2024-01-15T18:07:14.542189+0000","last_fullsized":"2024-01-15T18:07:14.542189+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:08.081272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542176+0000","last_change":"2024-01-15T18:07:13.557559+0000","last_active":"2024-01-15T18:07:14.542176+0000","last_peered":"2024-01-15T18:07:14.542176+0000","last_clean":"2024-01-15T18:07:14.542176+0000","last_became_active":"2024-01-15T18:07:13.557217+0000","last_became_peered":"2024-01-15T18:07:13.557217+0000","last_unstale":"2024-01-15T18:07:14.542176+0000","last_undegraded":"2024-01-15T18:07:14.542176+0000","last_fullsized":"2024-01-15T18:07:14.542176+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:51.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773926+0000","last_change":"2024-01-15T18:07:13.554980+0000","last_active":"2024-01-15T18:07:14.773926+0000","last_peered":"2024-01-15T18:07:14.773926+0000","last_clean":"2024-01-15T18:07:14.773926+0000","last_became_active":"2024-01-15T18:07:13.554690+0000","last_became_peered":"2024-01-15T18:07:13.554690+0000","last_unstale":"2024-01-15T18:07:14.773926+0000","last_undegraded":"2024-01-15T18:07:14.773926+0000","last_fullsized":"2024-01-15T18:07:14.773926+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:37.290014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774535+0000","last_change":"2024-01-15T18:07:13.560084+0000","last_active":"2024-01-15T18:07:14.774535+0000","last_peered":"2024-01-15T18:07:14.774535+0000","last_clean":"2024-01-15T18:07:14.774535+0000","last_became_active":"2024-01-15T18:07:13.559818+0000","last_became_peered":"2024-01-15T18:07:13.559818+0000","last_unstale":"2024-01-15T18:07:14.774535+0000","last_undegraded":"2024-01-15T18:07:14.774535+0000","last_fullsized":"2024-01-15T18:07:14.774535+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:20.316030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773559+0000","last_change":"2024-01-15T18:07:13.558125+0000","last_active":"2024-01-15T18:07:14.773559+0000","last_peered":"2024-01-15T18:07:14.773559+0000","last_clean":"2024-01-15T18:07:14.773559+0000","last_became_active":"2024-01-15T18:07:13.557757+0000","last_became_peered":"2024-01-15T18:07:13.557757+0000","last_unstale":"2024-01-15T18:07:14.773559+0000","last_undegraded":"2024-01-15T18:07:14.773559+0000","last_fullsized":"2024-01-15T18:07:14.773559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:04.463119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541350+0000","last_change":"2024-01-15T18:07:13.560763+0000","last_active":"2024-01-15T18:07:14.541350+0000","last_peered":"2024-01-15T18:07:14.541350+0000","last_clean":"2024-01-15T18:07:14.541350+0000","last_became_active":"2024-01-15T18:07:13.560247+0000","last_became_peered":"2024-01-15T18:07:13.560247+0000","last_unstale":"2024-01-15T18:07:14.541350+0000","last_undegraded":"2024-01-15T18:07:14.541350+0000","last_fullsized":"2024-01-15T18:07:14.541350+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:36:45.095291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541338+0000","last_change":"2024-01-15T18:07:13.554018+0000","last_active":"2024-01-15T18:07:14.541338+0000","last_peered":"2024-01-15T18:07:14.541338+0000","last_clean":"2024-01-15T18:07:14.541338+0000","last_became_active":"2024-01-15T18:07:13.553693+0000","last_became_peered":"2024-01-15T18:07:13.553693+0000","last_unstale":"2024-01-15T18:07:14.541338+0000","last_undegraded":"2024-01-15T18:07:14.541338+0000","last_fullsized":"2024-01-15T18:07:14.541338+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:08.982957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542676+0000","last_change":"2024-01-15T18:07:13.568220+0000","last_active":"2024-01-15T18:07:14.542676+0000","last_peered":"2024-01-15T18:07:14.542676+0000","last_clean":"2024-01-15T18:07:14.542676+0000","last_became_active":"2024-01-15T18:07:13.567961+0000","last_became_peered":"2024-01-15T18:07:13.567961+0000","last_unstale":"2024-01-15T18:07:14.542676+0000","last_undegraded":"2024-01-15T18:07:14.542676+0000","last_fullsized":"2024-01-15T18:07:14.542676+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:19:58.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773932+0000","last_change":"2024-01-15T18:07:13.556487+0000","last_active":"2024-01-15T18:07:14.773932+0000","last_peered":"2024-01-15T18:07:14.773932+0000","last_clean":"2024-01-15T18:07:14.773932+0000","last_became_active":"2024-01-15T18:07:13.556178+0000","last_became_peered":"2024-01-15T18:07:13.556178+0000","last_unstale":"2024-01-15T18:07:14.773932+0000","last_undegraded":"2024-01-15T18:07:14.773932+0000","last_fullsized":"2024-01-15T18:07:14.773932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:56.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542225+0000","last_change":"2024-01-15T18:07:13.558444+0000","last_active":"2024-01-15T18:07:14.542225+0000","last_peered":"2024-01-15T18:07:14.542225+0000","last_clean":"2024-01-15T18:07:14.542225+0000","last_became_active":"2024-01-15T18:07:13.558068+0000","last_became_peered":"2024-01-15T18:07:13.558068+0000","last_unstale":"2024-01-15T18:07:14.542225+0000","last_undegraded":"2024-01-15T18:07:14.542225+0000","last_fullsized":"2024-01-15T18:07:14.542225+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:06.472843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543524+0000","last_change":"2024-01-15T18:07:13.561182+0000","last_active":"2024-01-15T18:07:14.543524+0000","last_peered":"2024-01-15T18:07:14.543524+0000","last_clean":"2024-01-15T18:07:14.543524+0000","last_became_active":"2024-01-15T18:07:13.559200+0000","last_became_peered":"2024-01-15T18:07:13.559200+0000","last_unstale":"2024-01-15T18:07:14.543524+0000","last_undegraded":"2024-01-15T18:07:14.543524+0000","last_fullsized":"2024-01-15T18:07:14.543524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:22.667824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541598+0000","last_change":"2024-01-15T18:07:13.566721+0000","last_active":"2024-01-15T18:07:14.541598+0000","last_peered":"2024-01-15T18:07:14.541598+0000","last_clean":"2024-01-15T18:07:14.541598+0000","last_became_active":"2024-01-15T18:07:13.566446+0000","last_became_peered":"2024-01-15T18:07:13.566446+0000","last_unstale":"2024-01-15T18:07:14.541598+0000","last_undegraded":"2024-01-15T18:07:14.541598+0000","last_fullsized":"2024-01-15T18:07:14.541598+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.108103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541736+0000","last_change":"2024-01-15T18:07:13.559670+0000","last_active":"2024-01-15T18:07:14.541736+0000","last_peered":"2024-01-15T18:07:14.541736+0000","last_clean":"2024-01-15T18:07:14.541736+0000","last_became_active":"2024-01-15T18:07:13.558831+0000","last_became_peered":"2024-01-15T18:07:13.558831+0000","last_unstale":"2024-01-15T18:07:14.541736+0000","last_undegraded":"2024-01-15T18:07:14.541736+0000","last_fullsized":"2024-01-15T18:07:14.541736+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:27.810935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774974+0000","last_change":"2024-01-15T18:07:13.563324+0000","last_active":"2024-01-15T18:07:14.774974+0000","last_peered":"2024-01-15T18:07:14.774974+0000","last_clean":"2024-01-15T18:07:14.774974+0000","last_became_active":"2024-01-15T18:07:13.563088+0000","last_became_peered":"2024-01-15T18:07:13.563088+0000","last_unstale":"2024-01-15T18:07:14.774974+0000","last_undegraded":"2024-01-15T18:07:14.774974+0000","last_fullsized":"2024-01-15T18:07:14.774974+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:06.463252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773794+0000","last_change":"2024-01-15T18:07:13.558815+0000","last_active":"2024-01-15T18:07:14.773794+0000","last_peered":"2024-01-15T18:07:14.773794+0000","last_clean":"2024-01-15T18:07:14.773794+0000","last_became_active":"2024-01-15T18:07:13.558545+0000","last_became_peered":"2024-01-15T18:07:13.558545+0000","last_unstale":"2024-01-15T18:07:14.773794+0000","last_undegraded":"2024-01-15T18:07:14.773794+0000","last_fullsized":"2024-01-15T18:07:14.773794+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:55:13.674874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773433+0000","last_change":"2024-01-15T18:07:13.560097+0000","last_active":"2024-01-15T18:07:14.773433+0000","last_peered":"2024-01-15T18:07:14.773433+0000","last_clean":"2024-01-15T18:07:14.773433+0000","last_became_active":"2024-01-15T18:07:13.559817+0000","last_became_peered":"2024-01-15T18:07:13.559817+0000","last_unstale":"2024-01-15T18:07:14.773433+0000","last_undegraded":"2024-01-15T18:07:14.773433+0000","last_fullsized":"2024-01-15T18:07:14.773433+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:19:58.699494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775400+0000","last_change":"2024-01-15T18:07:13.143155+0000","last_active":"2024-01-15T18:07:14.775400+0000","last_peered":"2024-01-15T18:07:14.775400+0000","last_clean":"2024-01-15T18:07:14.775400+0000","last_became_active":"2024-01-15T18:07:13.142887+0000","last_became_peered":"2024-01-15T18:07:13.142887+0000","last_unstale":"2024-01-15T18:07:14.775400+0000","last_undegraded":"2024-01-15T18:07:14.775400+0000","last_fullsized":"2024-01-15T18:07:14.775400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:48:17.532793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542751+0000","last_change":"2024-01-15T18:07:13.557581+0000","last_active":"2024-01-15T18:07:14.542751+0000","last_peered":"2024-01-15T18:07:14.542751+0000","last_clean":"2024-01-15T18:07:14.542751+0000","last_became_active":"2024-01-15T18:07:13.557104+0000","last_became_peered":"2024-01-15T18:07:13.557104+0000","last_unstale":"2024-01-15T18:07:14.542751+0000","last_undegraded":"2024-01-15T18:07:14.542751+0000","last_fullsized":"2024-01-15T18:07:14.542751+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:20.253082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542255+0000","last_change":"2024-01-15T18:07:13.134532+0000","last_active":"2024-01-15T18:07:14.542255+0000","last_peered":"2024-01-15T18:07:14.542255+0000","last_clean":"2024-01-15T18:07:14.542255+0000","last_became_active":"2024-01-15T18:07:13.133873+0000","last_became_peered":"2024-01-15T18:07:13.133873+0000","last_unstale":"2024-01-15T18:07:14.542255+0000","last_undegraded":"2024-01-15T18:07:14.542255+0000","last_fullsized":"2024-01-15T18:07:14.542255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:24:53.119165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543674+0000","last_change":"2024-01-15T18:07:13.561484+0000","last_active":"2024-01-15T18:07:14.543674+0000","last_peered":"2024-01-15T18:07:14.543674+0000","last_clean":"2024-01-15T18:07:14.543674+0000","last_became_active":"2024-01-15T18:07:13.561171+0000","last_became_peered":"2024-01-15T18:07:13.561171+0000","last_unstale":"2024-01-15T18:07:14.543674+0000","last_undegraded":"2024-01-15T18:07:14.543674+0000","last_fullsized":"2024-01-15T18:07:14.543674+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:48:18.541539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774323+0000","last_change":"2024-01-15T18:07:13.141848+0000","last_active":"2024-01-15T18:07:14.774323+0000","last_peered":"2024-01-15T18:07:14.774323+0000","last_clean":"2024-01-15T18:07:14.774323+0000","last_became_active":"2024-01-15T18:07:13.141503+0000","last_became_peered":"2024-01-15T18:07:13.141503+0000","last_unstale":"2024-01-15T18:07:14.774323+0000","last_undegraded":"2024-01-15T18:07:14.774323+0000","last_fullsized":"2024-01-15T18:07:14.774323+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:50.037313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774114+0000","last_change":"2024-01-15T18:07:13.562984+0000","last_active":"2024-01-15T18:07:14.774114+0000","last_peered":"2024-01-15T18:07:14.774114+0000","last_clean":"2024-01-15T18:07:14.774114+0000","last_became_active":"2024-01-15T18:07:13.562555+0000","last_became_peered":"2024-01-15T18:07:13.562555+0000","last_unstale":"2024-01-15T18:07:14.774114+0000","last_undegraded":"2024-01-15T18:07:14.774114+0000","last_fullsized":"2024-01-15T18:07:14.774114+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:55.672164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774796+0000","last_change":"2024-01-15T18:07:13.151750+0000","last_active":"2024-01-15T18:07:14.774796+0000","last_peered":"2024-01-15T18:07:14.774796+0000","last_clean":"2024-01-15T18:07:14.774796+0000","last_became_active":"2024-01-15T18:07:13.151403+0000","last_became_peered":"2024-01-15T18:07:13.151403+0000","last_unstale":"2024-01-15T18:07:14.774796+0000","last_undegraded":"2024-01-15T18:07:14.774796+0000","last_fullsized":"2024-01-15T18:07:14.774796+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:41.327706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542105+0000","last_change":"2024-01-15T18:07:13.561187+0000","last_active":"2024-01-15T18:07:14.542105+0000","last_peered":"2024-01-15T18:07:14.542105+0000","last_clean":"2024-01-15T18:07:14.542105+0000","last_became_active":"2024-01-15T18:07:13.559238+0000","last_became_peered":"2024-01-15T18:07:13.559238+0000","last_unstale":"2024-01-15T18:07:14.542105+0000","last_undegraded":"2024-01-15T18:07:14.542105+0000","last_fullsized":"2024-01-15T18:07:14.542105+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:42.336452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774648+0000","last_change":"2024-01-15T18:07:13.147475+0000","last_active":"2024-01-15T18:07:14.774648+0000","last_peered":"2024-01-15T18:07:14.774648+0000","last_clean":"2024-01-15T18:07:14.774648+0000","last_became_active":"2024-01-15T18:07:13.147200+0000","last_became_peered":"2024-01-15T18:07:13.147200+0000","last_unstale":"2024-01-15T18:07:14.774648+0000","last_undegraded":"2024-01-15T18:07:14.774648+0000","last_fullsized":"2024-01-15T18:07:14.774648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:19.327531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774252+0000","last_change":"2024-01-15T18:07:13.553134+0000","last_active":"2024-01-15T18:07:14.774252+0000","last_peered":"2024-01-15T18:07:14.774252+0000","last_clean":"2024-01-15T18:07:14.774252+0000","last_became_active":"2024-01-15T18:07:13.552899+0000","last_became_peered":"2024-01-15T18:07:13.552899+0000","last_unstale":"2024-01-15T18:07:14.774252+0000","last_undegraded":"2024-01-15T18:07:14.774252+0000","last_fullsized":"2024-01-15T18:07:14.774252+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:42.336452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541945+0000","last_change":"2024-01-15T18:07:13.141705+0000","last_active":"2024-01-15T18:07:14.541945+0000","last_peered":"2024-01-15T18:07:14.541945+0000","last_clean":"2024-01-15T18:07:14.541945+0000","last_became_active":"2024-01-15T18:07:13.141399+0000","last_became_peered":"2024-01-15T18:07:13.141399+0000","last_unstale":"2024-01-15T18:07:14.541945+0000","last_undegraded":"2024-01-15T18:07:14.541945+0000","last_fullsized":"2024-01-15T18:07:14.541945+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:54.663418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773391+0000","last_change":"2024-01-15T18:07:13.545374+0000","last_active":"2024-01-15T18:07:14.773391+0000","last_peered":"2024-01-15T18:07:14.773391+0000","last_clean":"2024-01-15T18:07:14.773391+0000","last_became_active":"2024-01-15T18:07:13.545106+0000","last_became_peered":"2024-01-15T18:07:13.545106+0000","last_unstale":"2024-01-15T18:07:14.773391+0000","last_undegraded":"2024-01-15T18:07:14.773391+0000","last_fullsized":"2024-01-15T18:07:14.773391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:07:52.124299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543180+0000","last_change":"2024-01-15T18:07:13.143744+0000","last_active":"2024-01-15T18:07:14.543180+0000","last_peered":"2024-01-15T18:07:14.543180+0000","last_clean":"2024-01-15T18:07:14.543180+0000","last_became_active":"2024-01-15T18:07:13.143405+0000","last_became_peered":"2024-01-15T18:07:13.143405+0000","last_unstale":"2024-01-15T18:07:14.543180+0000","last_undegraded":"2024-01-15T18:07:14.543180+0000","last_fullsized":"2024-01-15T18:07:14.543180+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:57.207351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541293+0000","last_change":"2024-01-15T18:07:13.560545+0000","last_active":"2024-01-15T18:07:14.541293+0000","last_peered":"2024-01-15T18:07:14.541293+0000","last_clean":"2024-01-15T18:07:14.541293+0000","last_became_active":"2024-01-15T18:07:13.559806+0000","last_became_peered":"2024-01-15T18:07:13.559806+0000","last_unstale":"2024-01-15T18:07:14.541293+0000","last_undegraded":"2024-01-15T18:07:14.541293+0000","last_fullsized":"2024-01-15T18:07:14.541293+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:56.038348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773476+0000","last_change":"2024-01-15T18:07:13.141203+0000","last_active":"2024-01-15T18:07:14.773476+0000","last_peered":"2024-01-15T18:07:14.773476+0000","last_clean":"2024-01-15T18:07:14.773476+0000","last_became_active":"2024-01-15T18:07:13.140933+0000","last_became_peered":"2024-01-15T18:07:13.140933+0000","last_unstale":"2024-01-15T18:07:14.773476+0000","last_undegraded":"2024-01-15T18:07:14.773476+0000","last_fullsized":"2024-01-15T18:07:14.773476+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:08:50.037313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542212+0000","last_change":"2024-01-15T18:07:13.552946+0000","last_active":"2024-01-15T18:07:14.542212+0000","last_peered":"2024-01-15T18:07:14.542212+0000","last_clean":"2024-01-15T18:07:14.542212+0000","last_became_active":"2024-01-15T18:07:13.550975+0000","last_became_peered":"2024-01-15T18:07:13.550975+0000","last_unstale":"2024-01-15T18:07:14.542212+0000","last_undegraded":"2024-01-15T18:07:14.542212+0000","last_fullsized":"2024-01-15T18:07:14.542212+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:08.250326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542648+0000","last_change":"2024-01-15T18:07:13.136136+0000","last_active":"2024-01-15T18:07:14.542648+0000","last_peered":"2024-01-15T18:07:14.542648+0000","last_clean":"2024-01-15T18:07:14.542648+0000","last_became_active":"2024-01-15T18:07:13.135415+0000","last_became_peered":"2024-01-15T18:07:13.135415+0000","last_unstale":"2024-01-15T18:07:14.542648+0000","last_undegraded":"2024-01-15T18:07:14.542648+0000","last_fullsized":"2024-01-15T18:07:14.542648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:07:51.115553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775144+0000","last_change":"2024-01-15T18:07:13.570342+0000","last_active":"2024-01-15T18:07:14.775144+0000","last_peered":"2024-01-15T18:07:14.775144+0000","last_clean":"2024-01-15T18:07:14.775144+0000","last_became_active":"2024-01-15T18:07:13.570076+0000","last_became_peered":"2024-01-15T18:07:13.570076+0000","last_unstale":"2024-01-15T18:07:14.775144+0000","last_undegraded":"2024-01-15T18:07:14.775144+0000","last_fullsized":"2024-01-15T18:07:14.775144+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:20.316030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774778+0000","last_change":"2024-01-15T18:07:13.144905+0000","last_active":"2024-01-15T18:07:14.774778+0000","last_peered":"2024-01-15T18:07:14.774778+0000","last_clean":"2024-01-15T18:07:14.774778+0000","last_became_active":"2024-01-15T18:07:13.144602+0000","last_became_peered":"2024-01-15T18:07:13.144602+0000","last_unstale":"2024-01-15T18:07:14.774778+0000","last_undegraded":"2024-01-15T18:07:14.774778+0000","last_fullsized":"2024-01-15T18:07:14.774778+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:55.876186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775180+0000","last_change":"2024-01-15T18:07:13.560963+0000","last_active":"2024-01-15T18:07:14.775180+0000","last_peered":"2024-01-15T18:07:14.775180+0000","last_clean":"2024-01-15T18:07:14.775180+0000","last_became_active":"2024-01-15T18:07:13.560690+0000","last_became_peered":"2024-01-15T18:07:13.560690+0000","last_unstale":"2024-01-15T18:07:14.775180+0000","last_undegraded":"2024-01-15T18:07:14.775180+0000","last_fullsized":"2024-01-15T18:07:14.775180+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:08:08.478498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774970+0000","last_change":"2024-01-15T18:07:13.153076+0000","last_active":"2024-01-15T18:07:14.774970+0000","last_peered":"2024-01-15T18:07:14.774970+0000","last_clean":"2024-01-15T18:07:14.774970+0000","last_became_active":"2024-01-15T18:07:13.152768+0000","last_became_peered":"2024-01-15T18:07:13.152768+0000","last_unstale":"2024-01-15T18:07:14.774970+0000","last_undegraded":"2024-01-15T18:07:14.774970+0000","last_fullsized":"2024-01-15T18:07:14.774970+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:21.659078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774770+0000","last_change":"2024-01-15T18:07:13.569527+0000","last_active":"2024-01-15T18:07:14.774770+0000","last_peered":"2024-01-15T18:07:14.774770+0000","last_clean":"2024-01-15T18:07:14.774770+0000","last_became_active":"2024-01-15T18:07:13.569076+0000","last_became_peered":"2024-01-15T18:07:13.569076+0000","last_unstale":"2024-01-15T18:07:14.774770+0000","last_undegraded":"2024-01-15T18:07:14.774770+0000","last_fullsized":"2024-01-15T18:07:14.774770+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:06:15.780637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774015+0000","last_change":"2024-01-15T18:07:13.135598+0000","last_active":"2024-01-15T18:07:14.774015+0000","last_peered":"2024-01-15T18:07:14.774015+0000","last_clean":"2024-01-15T18:07:14.774015+0000","last_became_active":"2024-01-15T18:07:13.135334+0000","last_became_peered":"2024-01-15T18:07:13.135334+0000","last_unstale":"2024-01-15T18:07:14.774015+0000","last_undegraded":"2024-01-15T18:07:14.774015+0000","last_fullsized":"2024-01-15T18:07:14.774015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:19.099418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541680+0000","last_change":"2024-01-15T18:07:13.558361+0000","last_active":"2024-01-15T18:07:14.541680+0000","last_peered":"2024-01-15T18:07:14.541680+0000","last_clean":"2024-01-15T18:07:14.541680+0000","last_became_active":"2024-01-15T18:07:13.557736+0000","last_became_peered":"2024-01-15T18:07:13.557736+0000","last_unstale":"2024-01-15T18:07:14.541680+0000","last_undegraded":"2024-01-15T18:07:14.541680+0000","last_fullsized":"2024-01-15T18:07:14.541680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:11.352868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774691+0000","last_change":"2024-01-15T18:07:13.138892+0000","last_active":"2024-01-15T18:07:14.774691+0000","last_peered":"2024-01-15T18:07:14.774691+0000","last_clean":"2024-01-15T18:07:14.774691+0000","last_became_active":"2024-01-15T18:07:13.137384+0000","last_became_peered":"2024-01-15T18:07:13.137384+0000","last_unstale":"2024-01-15T18:07:14.774691+0000","last_undegraded":"2024-01-15T18:07:14.774691+0000","last_fullsized":"2024-01-15T18:07:14.774691+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:21.659078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774267+0000","last_change":"2024-01-15T18:07:13.555879+0000","last_active":"2024-01-15T18:07:14.774267+0000","last_peered":"2024-01-15T18:07:14.774267+0000","last_clean":"2024-01-15T18:07:14.774267+0000","last_became_active":"2024-01-15T18:07:13.555634+0000","last_became_peered":"2024-01-15T18:07:13.555634+0000","last_unstale":"2024-01-15T18:07:14.774267+0000","last_undegraded":"2024-01-15T18:07:14.774267+0000","last_fullsized":"2024-01-15T18:07:14.774267+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:08.081272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774706+0000","last_change":"2024-01-15T18:07:13.148916+0000","last_active":"2024-01-15T18:07:14.774706+0000","last_peered":"2024-01-15T18:07:14.774706+0000","last_clean":"2024-01-15T18:07:14.774706+0000","last_became_active":"2024-01-15T18:07:13.148524+0000","last_became_peered":"2024-01-15T18:07:13.148524+0000","last_unstale":"2024-01-15T18:07:14.774706+0000","last_undegraded":"2024-01-15T18:07:14.774706+0000","last_fullsized":"2024-01-15T18:07:14.774706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:18.605130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774437+0000","last_change":"2024-01-15T18:07:13.554144+0000","last_active":"2024-01-15T18:07:14.774437+0000","last_peered":"2024-01-15T18:07:14.774437+0000","last_clean":"2024-01-15T18:07:14.774437+0000","last_became_active":"2024-01-15T18:07:13.553831+0000","last_became_peered":"2024-01-15T18:07:13.553831+0000","last_unstale":"2024-01-15T18:07:14.774437+0000","last_undegraded":"2024-01-15T18:07:14.774437+0000","last_fullsized":"2024-01-15T18:07:14.774437+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:42.656502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775054+0000","last_change":"2024-01-15T18:07:13.139960+0000","last_active":"2024-01-15T18:07:14.775054+0000","last_peered":"2024-01-15T18:07:14.775054+0000","last_clean":"2024-01-15T18:07:14.775054+0000","last_became_active":"2024-01-15T18:07:13.136640+0000","last_became_peered":"2024-01-15T18:07:13.136640+0000","last_unstale":"2024-01-15T18:07:14.775054+0000","last_undegraded":"2024-01-15T18:07:14.775054+0000","last_fullsized":"2024-01-15T18:07:14.775054+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:21.093486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775013+0000","last_change":"2024-01-15T18:07:13.565438+0000","last_active":"2024-01-15T18:07:14.775013+0000","last_peered":"2024-01-15T18:07:14.775013+0000","last_clean":"2024-01-15T18:07:14.775013+0000","last_became_active":"2024-01-15T18:07:13.565147+0000","last_became_peered":"2024-01-15T18:07:13.565147+0000","last_unstale":"2024-01-15T18:07:14.775013+0000","last_undegraded":"2024-01-15T18:07:14.775013+0000","last_fullsized":"2024-01-15T18:07:14.775013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:46.069477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773565+0000","last_change":"2024-01-15T18:07:13.137547+0000","last_active":"2024-01-15T18:07:14.773565+0000","last_peered":"2024-01-15T18:07:14.773565+0000","last_clean":"2024-01-15T18:07:14.773565+0000","last_became_active":"2024-01-15T18:07:13.137239+0000","last_became_peered":"2024-01-15T18:07:13.137239+0000","last_unstale":"2024-01-15T18:07:14.773565+0000","last_undegraded":"2024-01-15T18:07:14.773565+0000","last_fullsized":"2024-01-15T18:07:14.773565+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:54.663418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774058+0000","last_change":"2024-01-15T18:07:13.562544+0000","last_active":"2024-01-15T18:07:14.774058+0000","last_peered":"2024-01-15T18:07:14.774058+0000","last_clean":"2024-01-15T18:07:14.774058+0000","last_became_active":"2024-01-15T18:07:13.562286+0000","last_became_peered":"2024-01-15T18:07:13.562286+0000","last_unstale":"2024-01-15T18:07:14.774058+0000","last_undegraded":"2024-01-15T18:07:14.774058+0000","last_fullsized":"2024-01-15T18:07:14.774058+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:08.982957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773481+0000","last_change":"2024-01-15T18:07:13.139177+0000","last_active":"2024-01-15T18:07:14.773481+0000","last_peered":"2024-01-15T18:07:14.773481+0000","last_clean":"2024-01-15T18:07:14.773481+0000","last_became_active":"2024-01-15T18:07:13.138899+0000","last_became_peered":"2024-01-15T18:07:13.138899+0000","last_unstale":"2024-01-15T18:07:14.773481+0000","last_undegraded":"2024-01-15T18:07:14.773481+0000","last_fullsized":"2024-01-15T18:07:14.773481+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774888+0000","last_change":"2024-01-15T18:07:13.561087+0000","last_active":"2024-01-15T18:07:14.774888+0000","last_peered":"2024-01-15T18:07:14.774888+0000","last_clean":"2024-01-15T18:07:14.774888+0000","last_became_active":"2024-01-15T18:07:13.560854+0000","last_became_peered":"2024-01-15T18:07:13.560854+0000","last_unstale":"2024-01-15T18:07:14.774888+0000","last_undegraded":"2024-01-15T18:07:14.774888+0000","last_fullsized":"2024-01-15T18:07:14.774888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:37.407192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542278+0000","last_change":"2024-01-15T18:07:13.144653+0000","last_active":"2024-01-15T18:07:14.542278+0000","last_peered":"2024-01-15T18:07:14.542278+0000","last_clean":"2024-01-15T18:07:14.542278+0000","last_became_active":"2024-01-15T18:07:13.144221+0000","last_became_peered":"2024-01-15T18:07:13.144221+0000","last_unstale":"2024-01-15T18:07:14.542278+0000","last_undegraded":"2024-01-15T18:07:14.542278+0000","last_fullsized":"2024-01-15T18:07:14.542278+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:38:46.176990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541667+0000","last_change":"2024-01-15T18:07:13.560705+0000","last_active":"2024-01-15T18:07:14.541667+0000","last_peered":"2024-01-15T18:07:14.541667+0000","last_clean":"2024-01-15T18:07:14.541667+0000","last_became_active":"2024-01-15T18:07:13.560263+0000","last_became_peered":"2024-01-15T18:07:13.560263+0000","last_unstale":"2024-01-15T18:07:14.541667+0000","last_undegraded":"2024-01-15T18:07:14.541667+0000","last_fullsized":"2024-01-15T18:07:14.541667+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:43.398122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543562+0000","last_change":"2024-01-15T18:07:13.142213+0000","last_active":"2024-01-15T18:07:14.543562+0000","last_peered":"2024-01-15T18:07:14.543562+0000","last_clean":"2024-01-15T18:07:14.543562+0000","last_became_active":"2024-01-15T18:07:13.141943+0000","last_became_peered":"2024-01-15T18:07:13.141943+0000","last_unstale":"2024-01-15T18:07:14.543562+0000","last_undegraded":"2024-01-15T18:07:14.543562+0000","last_fullsized":"2024-01-15T18:07:14.543562+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:14.240502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541986+0000","last_change":"2024-01-15T18:07:13.567698+0000","last_active":"2024-01-15T18:07:14.541986+0000","last_peered":"2024-01-15T18:07:14.541986+0000","last_clean":"2024-01-15T18:07:14.541986+0000","last_became_active":"2024-01-15T18:07:13.567421+0000","last_became_peered":"2024-01-15T18:07:13.567421+0000","last_unstale":"2024-01-15T18:07:14.541986+0000","last_undegraded":"2024-01-15T18:07:14.541986+0000","last_fullsized":"2024-01-15T18:07:14.541986+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:56.884932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775254+0000","last_change":"2024-01-15T18:07:13.143810+0000","last_active":"2024-01-15T18:07:14.775254+0000","last_peered":"2024-01-15T18:07:14.775254+0000","last_clean":"2024-01-15T18:07:14.775254+0000","last_became_active":"2024-01-15T18:07:13.143480+0000","last_became_peered":"2024-01-15T18:07:13.143480+0000","last_unstale":"2024-01-15T18:07:14.775254+0000","last_undegraded":"2024-01-15T18:07:14.775254+0000","last_fullsized":"2024-01-15T18:07:14.775254+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:43.404150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774844+0000","last_change":"2024-01-15T18:07:13.557697+0000","last_active":"2024-01-15T18:07:14.774844+0000","last_peered":"2024-01-15T18:07:14.774844+0000","last_clean":"2024-01-15T18:07:14.774844+0000","last_became_active":"2024-01-15T18:07:13.557436+0000","last_became_peered":"2024-01-15T18:07:13.557436+0000","last_unstale":"2024-01-15T18:07:14.774844+0000","last_undegraded":"2024-01-15T18:07:14.774844+0000","last_fullsized":"2024-01-15T18:07:14.774844+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:28.246734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774771+0000","last_change":"2024-01-15T18:07:13.145389+0000","last_active":"2024-01-15T18:07:14.774771+0000","last_peered":"2024-01-15T18:07:14.774771+0000","last_clean":"2024-01-15T18:07:14.774771+0000","last_became_active":"2024-01-15T18:07:13.145110+0000","last_became_peered":"2024-01-15T18:07:13.145110+0000","last_unstale":"2024-01-15T18:07:14.774771+0000","last_undegraded":"2024-01-15T18:07:14.774771+0000","last_fullsized":"2024-01-15T18:07:14.774771+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:42.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773907+0000","last_change":"2024-01-15T18:07:13.563791+0000","last_active":"2024-01-15T18:07:14.773907+0000","last_peered":"2024-01-15T18:07:14.773907+0000","last_clean":"2024-01-15T18:07:14.773907+0000","last_became_active":"2024-01-15T18:07:13.563499+0000","last_became_peered":"2024-01-15T18:07:13.563499+0000","last_unstale":"2024-01-15T18:07:14.773907+0000","last_undegraded":"2024-01-15T18:07:14.773907+0000","last_fullsized":"2024-01-15T18:07:14.773907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:20.253082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543216+0000","last_change":"2024-01-15T18:07:13.140448+0000","last_active":"2024-01-15T18:07:14.543216+0000","last_peered":"2024-01-15T18:07:14.543216+0000","last_clean":"2024-01-15T18:07:14.543216+0000","last_became_active":"2024-01-15T18:07:13.139194+0000","last_became_peered":"2024-01-15T18:07:13.139194+0000","last_unstale":"2024-01-15T18:07:14.543216+0000","last_undegraded":"2024-01-15T18:07:14.543216+0000","last_fullsized":"2024-01-15T18:07:14.543216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:05.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773323+0000","last_change":"2024-01-15T18:07:13.553150+0000","last_active":"2024-01-15T18:07:14.773323+0000","last_peered":"2024-01-15T18:07:14.773323+0000","last_clean":"2024-01-15T18:07:14.773323+0000","last_became_active":"2024-01-15T18:07:13.552870+0000","last_became_peered":"2024-01-15T18:07:13.552870+0000","last_unstale":"2024-01-15T18:07:14.773323+0000","last_undegraded":"2024-01-15T18:07:14.773323+0000","last_fullsized":"2024-01-15T18:07:14.773323+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:29:34.507041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773835+0000","last_change":"2024-01-15T18:07:13.138368+0000","last_active":"2024-01-15T18:07:14.773835+0000","last_peered":"2024-01-15T18:07:14.773835+0000","last_clean":"2024-01-15T18:07:14.773835+0000","last_became_active":"2024-01-15T18:07:13.138032+0000","last_became_peered":"2024-01-15T18:07:13.138032+0000","last_unstale":"2024-01-15T18:07:14.773835+0000","last_undegraded":"2024-01-15T18:07:14.773835+0000","last_fullsized":"2024-01-15T18:07:14.773835+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:29:33.498295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773758+0000","last_change":"2024-01-15T18:07:13.558500+0000","last_active":"2024-01-15T18:07:14.773758+0000","last_peered":"2024-01-15T18:07:14.773758+0000","last_clean":"2024-01-15T18:07:14.773758+0000","last_became_active":"2024-01-15T18:07:13.557620+0000","last_became_peered":"2024-01-15T18:07:13.557620+0000","last_unstale":"2024-01-15T18:07:14.773758+0000","last_undegraded":"2024-01-15T18:07:14.773758+0000","last_fullsized":"2024-01-15T18:07:14.773758+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:27.810935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774532+0000","last_change":"2024-01-15T18:07:13.148670+0000","last_active":"2024-01-15T18:07:14.774532+0000","last_peered":"2024-01-15T18:07:14.774532+0000","last_clean":"2024-01-15T18:07:14.774532+0000","last_became_active":"2024-01-15T18:07:13.148342+0000","last_became_peered":"2024-01-15T18:07:13.148342+0000","last_unstale":"2024-01-15T18:07:14.774532+0000","last_undegraded":"2024-01-15T18:07:14.774532+0000","last_fullsized":"2024-01-15T18:07:14.774532+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:19.327531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542821+0000","last_change":"2024-01-15T18:07:13.550052+0000","last_active":"2024-01-15T18:07:14.542821+0000","last_peered":"2024-01-15T18:07:14.542821+0000","last_clean":"2024-01-15T18:07:14.542821+0000","last_became_active":"2024-01-15T18:07:13.549747+0000","last_became_peered":"2024-01-15T18:07:13.549747+0000","last_unstale":"2024-01-15T18:07:14.542821+0000","last_undegraded":"2024-01-15T18:07:14.542821+0000","last_fullsized":"2024-01-15T18:07:14.542821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:55.672164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:16.670226+0000","last_change":"2024-01-15T18:07:13.145472+0000","last_active":"2024-01-15T18:07:16.670226+0000","last_peered":"2024-01-15T18:07:16.670226+0000","last_clean":"2024-01-15T18:07:16.670226+0000","last_became_active":"2024-01-15T18:07:13.145151+0000","last_became_peered":"2024-01-15T18:07:13.145151+0000","last_unstale":"2024-01-15T18:07:16.670226+0000","last_undegraded":"2024-01-15T18:07:16.670226+0000","last_fullsized":"2024-01-15T18:07:16.670226+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:52.070623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774409+0000","last_change":"2024-01-15T18:07:13.547632+0000","last_active":"2024-01-15T18:07:14.774409+0000","last_peered":"2024-01-15T18:07:14.774409+0000","last_clean":"2024-01-15T18:07:14.774409+0000","last_became_active":"2024-01-15T18:07:13.547354+0000","last_became_peered":"2024-01-15T18:07:13.547354+0000","last_unstale":"2024-01-15T18:07:14.774409+0000","last_undegraded":"2024-01-15T18:07:14.774409+0000","last_fullsized":"2024-01-15T18:07:14.774409+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:06.472843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.775040+0000","last_change":"2024-01-15T18:07:13.152981+0000","last_active":"2024-01-15T18:07:14.775040+0000","last_peered":"2024-01-15T18:07:14.775040+0000","last_clean":"2024-01-15T18:07:14.775040+0000","last_became_active":"2024-01-15T18:07:13.152735+0000","last_became_peered":"2024-01-15T18:07:13.152735+0000","last_unstale":"2024-01-15T18:07:14.775040+0000","last_undegraded":"2024-01-15T18:07:14.775040+0000","last_fullsized":"2024-01-15T18:07:14.775040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:42.785862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774975+0000","last_change":"2024-01-15T18:07:13.568915+0000","last_active":"2024-01-15T18:07:14.774975+0000","last_peered":"2024-01-15T18:07:14.774975+0000","last_clean":"2024-01-15T18:07:14.774975+0000","last_became_active":"2024-01-15T18:07:13.567703+0000","last_became_peered":"2024-01-15T18:07:13.567703+0000","last_unstale":"2024-01-15T18:07:14.774975+0000","last_undegraded":"2024-01-15T18:07:14.774975+0000","last_fullsized":"2024-01-15T18:07:14.774975+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:36.540360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773425+0000","last_change":"2024-01-15T18:07:13.143026+0000","last_active":"2024-01-15T18:07:14.773425+0000","last_peered":"2024-01-15T18:07:14.773425+0000","last_clean":"2024-01-15T18:07:14.773425+0000","last_became_active":"2024-01-15T18:07:13.142687+0000","last_became_peered":"2024-01-15T18:07:13.142687+0000","last_unstale":"2024-01-15T18:07:14.773425+0000","last_undegraded":"2024-01-15T18:07:14.773425+0000","last_fullsized":"2024-01-15T18:07:14.773425+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:55.876186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774904+0000","last_change":"2024-01-15T18:07:13.558506+0000","last_active":"2024-01-15T18:07:14.774904+0000","last_peered":"2024-01-15T18:07:14.774904+0000","last_clean":"2024-01-15T18:07:14.774904+0000","last_became_active":"2024-01-15T18:07:13.558261+0000","last_became_peered":"2024-01-15T18:07:13.558261+0000","last_unstale":"2024-01-15T18:07:14.774904+0000","last_undegraded":"2024-01-15T18:07:14.774904+0000","last_fullsized":"2024-01-15T18:07:14.774904+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:46:06.569155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774816+0000","last_change":"2024-01-15T18:07:13.153254+0000","last_active":"2024-01-15T18:07:14.774816+0000","last_peered":"2024-01-15T18:07:14.774816+0000","last_clean":"2024-01-15T18:07:14.774816+0000","last_became_active":"2024-01-15T18:07:13.152937+0000","last_became_peered":"2024-01-15T18:07:13.152937+0000","last_unstale":"2024-01-15T18:07:14.774816+0000","last_undegraded":"2024-01-15T18:07:14.774816+0000","last_fullsized":"2024-01-15T18:07:14.774816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:14.240502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774602+0000","last_change":"2024-01-15T18:07:13.554436+0000","last_active":"2024-01-15T18:07:14.774602+0000","last_peered":"2024-01-15T18:07:14.774602+0000","last_clean":"2024-01-15T18:07:14.774602+0000","last_became_active":"2024-01-15T18:07:13.554156+0000","last_became_peered":"2024-01-15T18:07:13.554156+0000","last_unstale":"2024-01-15T18:07:14.774602+0000","last_undegraded":"2024-01-15T18:07:14.774602+0000","last_fullsized":"2024-01-15T18:07:14.774602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:52:24.006274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.542033+0000","last_change":"2024-01-15T18:07:13.132084+0000","last_active":"2024-01-15T18:07:14.542033+0000","last_peered":"2024-01-15T18:07:14.542033+0000","last_clean":"2024-01-15T18:07:14.542033+0000","last_became_active":"2024-01-15T18:07:13.131783+0000","last_became_peered":"2024-01-15T18:07:13.131783+0000","last_unstale":"2024-01-15T18:07:14.542033+0000","last_undegraded":"2024-01-15T18:07:14.542033+0000","last_fullsized":"2024-01-15T18:07:14.542033+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:57:50.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543321+0000","last_change":"2024-01-15T18:07:13.562769+0000","last_active":"2024-01-15T18:07:14.543321+0000","last_peered":"2024-01-15T18:07:14.543321+0000","last_clean":"2024-01-15T18:07:14.543321+0000","last_became_active":"2024-01-15T18:07:13.562497+0000","last_became_peered":"2024-01-15T18:07:13.562497+0000","last_unstale":"2024-01-15T18:07:14.543321+0000","last_undegraded":"2024-01-15T18:07:14.543321+0000","last_fullsized":"2024-01-15T18:07:14.543321+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:37.407192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773745+0000","last_change":"2024-01-15T18:07:13.141244+0000","last_active":"2024-01-15T18:07:14.773745+0000","last_peered":"2024-01-15T18:07:14.773745+0000","last_clean":"2024-01-15T18:07:14.773745+0000","last_became_active":"2024-01-15T18:07:13.140989+0000","last_became_peered":"2024-01-15T18:07:13.140989+0000","last_unstale":"2024-01-15T18:07:14.773745+0000","last_undegraded":"2024-01-15T18:07:14.773745+0000","last_fullsized":"2024-01-15T18:07:14.773745+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:10.344122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.541640+0000","last_change":"2024-01-15T18:07:13.556894+0000","last_active":"2024-01-15T18:07:14.541640+0000","last_peered":"2024-01-15T18:07:14.541640+0000","last_clean":"2024-01-15T18:07:14.541640+0000","last_became_active":"2024-01-15T18:07:13.556530+0000","last_became_peered":"2024-01-15T18:07:13.556530+0000","last_unstale":"2024-01-15T18:07:14.541640+0000","last_undegraded":"2024-01-15T18:07:14.541640+0000","last_fullsized":"2024-01-15T18:07:14.541640+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:04.463119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.773646+0000","last_change":"2024-01-15T18:07:13.134762+0000","last_active":"2024-01-15T18:07:14.773646+0000","last_peered":"2024-01-15T18:07:14.773646+0000","last_clean":"2024-01-15T18:07:14.773646+0000","last_became_active":"2024-01-15T18:07:13.134496+0000","last_became_peered":"2024-01-15T18:07:13.134496+0000","last_unstale":"2024-01-15T18:07:14.773646+0000","last_undegraded":"2024-01-15T18:07:14.773646+0000","last_fullsized":"2024-01-15T18:07:14.773646+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:57:50.317014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.774724+0000","last_change":"2024-01-15T18:07:13.562552+0000","last_active":"2024-01-15T18:07:14.774724+0000","last_peered":"2024-01-15T18:07:14.774724+0000","last_clean":"2024-01-15T18:07:14.774724+0000","last_became_active":"2024-01-15T18:07:13.562292+0000","last_became_peered":"2024-01-15T18:07:13.562292+0000","last_unstale":"2024-01-15T18:07:14.774724+0000","last_undegraded":"2024-01-15T18:07:14.774724+0000","last_fullsized":"2024-01-15T18:07:14.774724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:13.109262+0000","last_clean_scrub_stamp":"2024-01-15T18:07:13.109262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:22.102232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T18:07:14.543658+0000","last_change":"2024-01-15T18:07:13.145974+0000","last_active":"2024-01-15T18:07:14.543658+0000","last_peered":"2024-01-15T18:07:14.543658+0000","last_clean":"2024-01-15T18:07:14.543658+0000","last_became_active":"2024-01-15T18:07:13.145622+0000","last_became_peered":"2024-01-15T18:07:13.145622+0000","last_unstale":"2024-01-15T18:07:14.543658+0000","last_undegraded":"2024-01-15T18:07:14.543658+0000","last_fullsized":"2024-01-15T18:07:14.543658+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:07:12.100516+0000","last_clean_scrub_stamp":"2024-01-15T18:07:12.100516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:35:21.093486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-01-15T18:07:13.555978+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:07:13.555978+0000","last_peered":"2024-01-15T18:07:13.555978+0000","last_clean":"2024-01-15T18:07:13.555978+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:07:13.555978+0000","last_undegraded":"2024-01-15T18:07:13.555978+0000","last_fullsized":"2024-01-15T18:07:13.555978+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":812277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77500000000000002,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.50800000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.71799999999999997,"15min":0.73799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77000000000000002,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.75700000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":3.403,"5min":3.403,"15min":3.403},"last":0.60099999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71199999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.82299999999999995},"min":{"1min":0.54100000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.1419999999999999,"5min":3.7759999999999998,"15min":3.7759999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80500000000000005,"15min":0.81499999999999995},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.57699999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.64500000000000002,"15min":0.66600000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.872,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":4.3710000000000004,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.622}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.73499999999999999,"15min":0.75800000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.747,"15min":0.76600000000000001},"min":{"1min":0.46500000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.97999999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.79700000000000004,"15min":0.81799999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91200000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.754,"15min":0.77400000000000002},"min":{"1min":0.45900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.67500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":548864,"data_stored":222491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.623,"15min":0.623},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.72199999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.69999999999999996,"15min":0.70599999999999996},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78300000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92100000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.748},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.76300000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93400000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.54600000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92300000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.1319999999999997,"5min":4.1319999999999997,"15min":4.1319999999999997},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.72099999999999997,"15min":0.746},"min":{"1min":0.38800000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.95399999999999996,"5min":1.107,"15min":1.107},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.51900000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97799999999999998,"5min":1.401,"15min":1.401},"last":0.79800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.081,"15min":1.081},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81699999999999995,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.81699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.67000000000000004,"15min":0.63700000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.47499999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.032,"5min":5.032,"15min":5.032},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.68700000000000006,"15min":0.68999999999999995},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.69399999999999995,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70399999999999996,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.97799999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76700000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.78300000000000003,"15min":0.76000000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":4.1369999999999996,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69199999999999995,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.504}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.70099999999999996,"15min":0.69299999999999995},"min":{"1min":0.38700000000000001,"5min":0.377,"15min":0.377},"max":{"1min":4.1589999999999998,"5min":4.1589999999999998,"15min":4.1589999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.443}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.76100000000000001,"15min":0.77300000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72899999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.82999999999999996,"15min":0.81399999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.1799999999999997,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.2010000000000001,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.59099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":223431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61499999999999999,"15min":0.61099999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.84499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.871}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90000000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.94099999999999995,"5min":1.054,"15min":1.054},"last":0.81799999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.70099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63900000000000001,"15min":0.64500000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83999999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.501},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90400000000000003,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.84299999999999997,"5min":1.218,"15min":1.218},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.73699999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88300000000000001,"5min":1.179,"15min":1.179},"last":0.61699999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.70599999999999996,"15min":0.72199999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88100000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.55400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":552960,"data_stored":222457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.52700000000000002,"15min":0.47099999999999997},"min":{"1min":0.30599999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":7.2199999999999998,"5min":7.2199999999999998,"15min":7.2199999999999998},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.496,"15min":0.45600000000000002},"min":{"1min":0.32600000000000001,"5min":0.316,"15min":0.316},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.55600000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.47599999999999998,"15min":0.443},"min":{"1min":0.34699999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":3.444,"5min":3.444,"15min":3.444},"last":0.499},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.623,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":7.181,"5min":7.181,"15min":7.181},"last":0.52400000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.53100000000000003,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.61299999999999999,"15min":0.55600000000000005},"min":{"1min":0.42299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73199999999999998,"15min":0.72599999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.76500000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.274,"5min":3.274,"15min":3.274},"last":0.61199999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":0.81399999999999995,"15min":0.77400000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":3.734,"5min":3.734,"15min":3.734},"last":0.63200000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.77300000000000002,"15min":0.76100000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.0600000000000001,"5min":0.875,"15min":0.84399999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":7.1399999999999997,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.73299999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77200000000000002,"15min":0.75600000000000001},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":3.4129999999999998,"5min":3.4129999999999998,"15min":3.4129999999999998},"last":0.69699999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":223453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.47399999999999998,"15min":0.45200000000000001},"min":{"1min":0.28499999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":2.8149999999999999,"5min":2.8149999999999999,"15min":2.8149999999999999},"last":0.31},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.45400000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.56899999999999995,"15min":0.51300000000000001},"min":{"1min":0.34300000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":5.4880000000000004,"5min":5.4880000000000004,"15min":5.4880000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.497,"15min":0.49099999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.48399999999999999,"15min":0.46000000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":3.23,"5min":3.23,"15min":3.23},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.55600000000000005,"15min":0.53400000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.9100000000000001,"5min":2.9100000000000001,"15min":2.9100000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.81200000000000006,"15min":0.79800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":3.1549999999999998,"5min":3.1549999999999998,"15min":3.1549999999999998},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.73099999999999998,"15min":0.72499999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.80300000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.74399999999999999,"15min":0.72399999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1419999999999999,"5min":3.1419999999999999,"15min":3.1419999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.82799999999999996,"15min":0.81699999999999995},"min":{"1min":0.52800000000000002,"5min":0.496,"15min":0.496},"max":{"1min":3.2010000000000001,"5min":3.2010000000000001,"15min":3.2010000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.2610000000000001,"5min":3.2610000000000001,"15min":3.2610000000000001},"last":0.83199999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.83999999999999997,"15min":0.83499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.1070000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.89500000000000002,"15min":0.876},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":3.194,"5min":3.194,"15min":3.194},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":544768,"data_stored":221889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.66100000000000003,"15min":0.68000000000000005},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.2570000000000001,"5min":4.532,"15min":4.532},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73599999999999999,"15min":0.747},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":3.0409999999999999,"5min":4.7969999999999997,"15min":4.7969999999999997},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.66100000000000003,"15min":0.69499999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":4.6840000000000002,"15min":4.6840000000000002},"last":0.436},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":3.3610000000000002,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.503}]},{"osd":3,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":0.92100000000000004,"15min":0.81299999999999994},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":21.416,"5min":21.416,"15min":21.416},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.101,"5min":0.90900000000000003,"15min":0.877},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":13.119999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.56999999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.022},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.238,"5min":4.7629999999999999,"15min":4.7629999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.0209999999999999,"15min":0.98299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":12.997,"5min":12.997,"15min":12.997},"last":0.61799999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.90500000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2639999999999998,"5min":3.2639999999999998,"15min":3.2639999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.145,"15min":1.123},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":13.054,"5min":13.054,"15min":13.054},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.97399999999999998,"15min":0.98599999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.2170000000000001,"5min":4.7309999999999999,"15min":4.7309999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.876,"5min":1.008,"15min":1.03},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":4.6909999999999998,"5min":4.6909999999999998,"15min":4.6909999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":1.0309999999999999,"15min":1.0549999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.7729999999999997,"5min":4.7729999999999997,"15min":4.7729999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.274,"5min":1.1060000000000001,"15min":1.0780000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":13.089,"5min":13.089,"15min":13.089},"last":0.73099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":812221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.79000000000000004,"15min":0.85099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.79800000000000004,"5min":8.1050000000000004,"15min":8.1050000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47599999999999998,"15min":0.46999999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.505},"min":{"1min":0.34699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.623,"15min":0.65300000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.81999999999999995,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.39400000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.85299999999999998,"15min":0.91400000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.98199999999999998,"5min":8.0579999999999998,"15min":8.0579999999999998},"last":0.499},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55900000000000005,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.88300000000000001,"15min":0.91800000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.079,"5min":3.9820000000000002,"15min":3.9820000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.90900000000000003,"15min":0.94599999999999995},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.0429999999999999,"5min":4.3159999999999998,"15min":4.3159999999999998},"last":0.72499999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.92600000000000005,"15min":0.96999999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1240000000000001,"5min":4.4539999999999997,"15min":4.4539999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.91800000000000004,"15min":0.95399999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.98899999999999999,"5min":4.056,"15min":4.056},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.80300000000000005,"15min":0.81699999999999995},"min":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0529999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.97099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.1399999999999999,"5min":4.0190000000000001,"15min":4.0190000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:06:49 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77400000000000002,"15min":0.78600000000000003},"min":{"1min":0.55900000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.026,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.622},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.95399999999999996,"15min":1},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93400000000000005,"5min":4.407,"15min":4.407},"last":0.69899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:07:23.062 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:07:23.063 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:07:23.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad73800640 1 Processor -- start 2024-01-15T18:07:23.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad73800640 1 -- start start 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad73800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad6c1436f0 0x7fad6c0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad73800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c0721b0 con 0x7fad6c0ae5c0 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad73800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c072640 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad73800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6c0715a0 con 0x7fad6c1436f0 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad70e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad6c1436f0 0x7fad6c0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad6be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fad6c0ae5c0 0x7fad6c0b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50064/0 (socket says 172.21.15.143:50064) 2024-01-15T18:07:23.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad6be00640 1 -- 172.21.15.143:0/2299677096 learned_addr learned my addr 172.21.15.143:0/2299677096 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:23.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad70e00640 1 -- 172.21.15.143:0/2299677096 >> v1:172.21.15.204:6789/0 conn(0x7fad6c0b3ba0 legacy=0x7fad6c0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad70e00640 1 -- 172.21.15.143:0/2299677096 >> v1:172.21.15.143:6789/0 conn(0x7fad6c0ae5c0 legacy=0x7fad6c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad70e00640 1 -- 172.21.15.143:0/2299677096 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c071880 con 0x7fad6c1436f0 2024-01-15T18:07:23.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad6b400640 1 -- 172.21.15.143:0/2299677096 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503829083 0 0) 0x7fad6c0721b0 con 0x7fad6c0ae5c0 2024-01-15T18:07:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad70e00640 1 --2- 172.21.15.143:0/2299677096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c1436f0 0x7fad6c0ae080 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0097c0 tx=0x7fad5c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60555ffe88cea620 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad6b400640 1 -- 172.21.15.143:0/2299677096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0ab650 con 0x7fad6c1436f0 2024-01-15T18:07:23.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad6b400640 1 -- 172.21.15.143:0/2299677096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0ab7b0 con 0x7fad6c1436f0 2024-01-15T18:07:23.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.240+0000 7fad6b400640 1 -- 172.21.15.143:0/2299677096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c0abb40 con 0x7fad6c1436f0 2024-01-15T18:07:23.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- 172.21.15.143:0/2299677096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c1436f0 msgr2=0x7fad6c0ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 --2- 172.21.15.143:0/2299677096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c1436f0 0x7fad6c0ae080 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0097c0 tx=0x7fad5c01c450 comp rx=0 tx=0).stop 2024-01-15T18:07:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- 172.21.15.143:0/2299677096 shutdown_connections 2024-01-15T18:07:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 --2- 172.21.15.143:0/2299677096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c1436f0 0x7fad6c0ae080 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- 172.21.15.143:0/2299677096 >> 172.21.15.143:0/2299677096 conn(0x7fad6c06a890 msgr2=0x7fad6c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- 172.21.15.143:0/2299677096 shutdown_connections 2024-01-15T18:07:23.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- 172.21.15.143:0/2299677096 wait complete. 2024-01-15T18:07:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 Processor -- start 2024-01-15T18:07:23.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- start start 2024-01-15T18:07:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0ae5c0 0x7fad6c145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 0x7fad6c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c1436f0 0x7fad6c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c15d160 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0cc070 con 0x7fad6c0ae5c0 2024-01-15T18:07:23.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad73800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0cc350 con 0x7fad6c1436f0 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad6be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 0x7fad6c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad6be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 0x7fad6c0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47194/0 (socket says 172.21.15.143:47194) 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad70e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0ae5c0 0x7fad6c145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad6be00640 1 -- 172.21.15.143:0/1671560699 learned_addr learned my addr 172.21.15.143:0/1671560699 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.244+0000 7fad71800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c1436f0 0x7fad6c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad6be00640 1 -- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c1436f0 msgr2=0x7fad6c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad6be00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c1436f0 0x7fad6c0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad6be00640 1 -- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0ae5c0 msgr2=0x7fad6c145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad6be00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0ae5c0 0x7fad6c145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad6be00640 1 -- 172.21.15.143:0/1671560699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c009420 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad71800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c1436f0 0x7fad6c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad70e00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0ae5c0 0x7fad6c145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad6be00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 0x7fad6c0cb4f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fad5400c4a0 tx=0x7fad5400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540074e0 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad54007640 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54004a30 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c0cc630 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c0ccbf0 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c067500 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fad54004b90 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad68c00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44066b30 0x7fad44068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fad54083a70 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.248+0000 7fad70e00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44066b30 0x7fad44068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.252+0000 7fad70e00640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44066b30 0x7fad44068ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0af000 tx=0x7fad5c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.252+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad540509d0 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.456+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fad6c0af700 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.456+0000 7fad68c00640 1 -- 172.21.15.143:0/1671560699 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fad54018070 con 0x7fad6c0b3ba0 2024-01-15T18:07:23.466 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:07:23.466 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:07:23.466 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:07:23.466 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:07:23.466 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:07:23.466 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:07:23.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44066b30 msgr2=0x7fad44068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44066b30 0x7fad44068ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0af000 tx=0x7fad5c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:07:23.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 msgr2=0x7fad6c0cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 0x7fad6c0cb4f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fad5400c4a0 tx=0x7fad5400c960 comp rx=0 tx=0).stop 2024-01-15T18:07:23.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 shutdown_connections 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44066b30 0x7fad44068ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c1436f0 0x7fad6c0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c0b3ba0 0x7fad6c0cb4f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 --2- 172.21.15.143:0/1671560699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0ae5c0 0x7fad6c145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 >> 172.21.15.143:0/1671560699 conn(0x7fad6c06a890 msgr2=0x7fad6c06d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 shutdown_connections 2024-01-15T18:07:23.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.460+0000 7fad73800640 1 -- 172.21.15.143:0/1671560699 wait complete. 2024-01-15T18:07:23.488 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:07:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 Processor -- start 2024-01-15T18:07:23.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 -- start start 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c88141660 0x7f6c880ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c88071d00 0x7f6c88072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c88072650 con 0x7f6c8815ec80 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8815fbb0 con 0x7f6c88071d00 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c8d600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8815f130 con 0x7f6c88141660 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c88071d00 0x7f6c88072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c88071d00 0x7f6c88072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47208/0 (socket says 172.21.15.143:47208) 2024-01-15T18:07:23.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 -- 172.21.15.143:0/929829192 learned_addr learned my addr 172.21.15.143:0/929829192 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:23.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c86a00640 1 --2- 172.21.15.143:0/929829192 >> v2:172.21.15.204:3300/0 conn(0x7f6c88141660 0x7f6c880ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 -- 172.21.15.143:0/929829192 >> v2:172.21.15.204:3300/0 conn(0x7f6c88141660 msgr2=0x7f6c880ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:23.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 --2- 172.21.15.143:0/929829192 >> v2:172.21.15.204:3300/0 conn(0x7f6c88141660 0x7f6c880ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 -- 172.21.15.143:0/929829192 >> v1:172.21.15.204:6789/0 conn(0x7f6c8815ec80 legacy=0x7f6c88141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:23.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 -- 172.21.15.143:0/929829192 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8815f310 con 0x7f6c88071d00 2024-01-15T18:07:23.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c86a00640 1 --2- 172.21.15.143:0/929829192 >> v2:172.21.15.204:3300/0 conn(0x7f6c88141660 0x7f6c880ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:23.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c87400640 1 --2- 172.21.15.143:0/929829192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88071d00 0x7f6c88072110 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6c880b1aa0 tx=0x7f6c8001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ae4a6442756f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c85600640 1 -- 172.21.15.143:0/929829192 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800a5820 con 0x7f6c88071d00 2024-01-15T18:07:23.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c85600640 1 -- 172.21.15.143:0/929829192 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c800a5980 con 0x7f6c88071d00 2024-01-15T18:07:23.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.620+0000 7f6c85600640 1 -- 172.21.15.143:0/929829192 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800a5d10 con 0x7f6c88071d00 2024-01-15T18:07:23.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- 172.21.15.143:0/929829192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88071d00 msgr2=0x7f6c88072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 --2- 172.21.15.143:0/929829192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88071d00 0x7f6c88072110 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6c880b1aa0 tx=0x7f6c8001c960 comp rx=0 tx=0).stop 2024-01-15T18:07:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- 172.21.15.143:0/929829192 shutdown_connections 2024-01-15T18:07:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 --2- 172.21.15.143:0/929829192 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88071d00 0x7f6c88072110 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 --2- 172.21.15.143:0/929829192 >> v2:172.21.15.204:3300/0 conn(0x7f6c88141660 0x7f6c880ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- 172.21.15.143:0/929829192 >> 172.21.15.143:0/929829192 conn(0x7f6c8806c3f0 msgr2=0x7f6c8806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- 172.21.15.143:0/929829192 shutdown_connections 2024-01-15T18:07:23.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- 172.21.15.143:0/929829192 wait complete. 2024-01-15T18:07:23.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 Processor -- start 2024-01-15T18:07:23.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- start start 2024-01-15T18:07:23.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c88071d00 0x7f6c88158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 0x7f6c8813f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8815ec80 0x7f6c8813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815af20 con 0x7f6c88141660 2024-01-15T18:07:23.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815de60 con 0x7f6c8815ec80 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c8d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8815e140 con 0x7f6c88071d00 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 0x7f6c8813f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c88071d00 0x7f6c88158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 0x7f6c8813f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47222/0 (socket says 172.21.15.143:47222) 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 -- 172.21.15.143:0/1495285932 learned_addr learned my addr 172.21.15.143:0/1495285932 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c87400640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8815ec80 0x7f6c8813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 -- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c88071d00 msgr2=0x7f6c88158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c88071d00 0x7f6c88158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 -- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8815ec80 msgr2=0x7f6c8813fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8815ec80 0x7f6c8813fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c86000640 1 -- 172.21.15.143:0/1495285932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c80009b30 con 0x7f6c88141660 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.624+0000 7f6c87400640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8815ec80 0x7f6c8813fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:23.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c86000640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 0x7f6c8813f7d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c450 tx=0x7f6c7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70012830 con 0x7f6c88141660 2024-01-15T18:07:23.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700129b0 con 0x7f6c88141660 2024-01-15T18:07:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8815e420 con 0x7f6c88141660 2024-01-15T18:07:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70012d60 con 0x7f6c88141660 2024-01-15T18:07:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c88159420 con 0x7f6c88141660 2024-01-15T18:07:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8815e660 con 0x7f6c88141660 2024-01-15T18:07:23.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f6c7001c460 con 0x7f6c88141660 2024-01-15T18:07:23.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.628+0000 7f6c7ea00640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c60066b60 0x7f6c60069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.632+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6c70084430 con 0x7f6c88141660 2024-01-15T18:07:23.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.632+0000 7f6c86a00640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c60066b60 0x7f6c60069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.632+0000 7f6c86a00640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c60066b60 0x7f6c60069020 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6c880b0650 tx=0x7f6c78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.636+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c8815e660 con 0x7f6c88141660 2024-01-15T18:07:23.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c8815b060 con 0x7f6c88141660 2024-01-15T18:07:23.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c7ea00640 1 -- 172.21.15.143:0/1495285932 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f6c8815b060 con 0x7f6c88141660 2024-01-15T18:07:23.824 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:23.824 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2863517742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":2863517742},{"type":"v1","addr":"172.21.15.143:6837","nonce":2863517742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/4241131349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":4241131349},{"type":"v1","addr":"172.21.15.204:6835","nonce":4241131349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1888637866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1888637866},{"type":"v1","addr":"172.21.15.204:6834","nonce":1888637866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:07:13.499835+0000","modified":"2024-01-15T18:07:16.510525+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5736},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5736":{"gid":5736,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6836/1835320705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1835320705},{"type":"v1","addr":"172.21.15.143:6836","nonce":1835320705}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-15T18:07:23.825 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 26 2024-01-15T18:07:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c60066b60 msgr2=0x7f6c60069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c60066b60 0x7f6c60069020 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6c880b0650 tx=0x7f6c78008040 comp rx=0 tx=0).stop 2024-01-15T18:07:23.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 msgr2=0x7f6c8813f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 0x7f6c8813f7d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000c450 tx=0x7f6c7000c910 comp rx=0 tx=0).stop 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 shutdown_connections 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c60066b60 0x7f6c60069020 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c8815ec80 0x7f6c8813fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c88141660 0x7f6c8813f7d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 --2- 172.21.15.143:0/1495285932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c88071d00 0x7f6c88158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.816+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 >> 172.21.15.143:0/1495285932 conn(0x7f6c8806c3f0 msgr2=0x7f6c881427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.820+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 shutdown_connections 2024-01-15T18:07:23.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:23.820+0000 7f6c8d600640 1 -- 172.21.15.143:0/1495285932 wait complete. 2024-01-15T18:07:24.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:07:13.499835+0000', 'modified': '2024-01-15T18:07:16.510525+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5736}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5736': {'gid': 5736, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6836/1835320705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 1835320705}, {'type': 'v1', 'addr': '172.21.15.143:6836', 'nonce': 1835320705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:07:24.249 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:07:24.249 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-15T18:07:24.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.380+0000 7fc55e000640 1 Processor -- start 2024-01-15T18:07:24.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.380+0000 7fc55e000640 1 -- start start 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc55e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc558159b70 0x7fc5580b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc55e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5580b48f0 0x7fc558070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc55e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5580b2a20 con 0x7fc558157300 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc55e000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5580b35a0 con 0x7fc558159b70 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc55e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc558159fd0 con 0x7fc5580b48f0 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc556a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc558159b70 0x7fc5580b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc557400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc558157300 0x7fc5580b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50074/0 (socket says 172.21.15.143:50074) 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc557400640 1 -- 172.21.15.143:0/3270774163 learned_addr learned my addr 172.21.15.143:0/3270774163 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc557e00640 1 --2- 172.21.15.143:0/3270774163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5580b48f0 0x7fc558070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc556a00640 1 -- 172.21.15.143:0/3270774163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5580b48f0 msgr2=0x7fc558070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc556a00640 1 --2- 172.21.15.143:0/3270774163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5580b48f0 0x7fc558070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc556a00640 1 -- 172.21.15.143:0/3270774163 >> v1:172.21.15.143:6789/0 conn(0x7fc558157300 legacy=0x7fc5580b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc556000640 1 -- 172.21.15.143:0/3270774163 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111310907 0 0) 0x7fc5580b2a20 con 0x7fc558157300 2024-01-15T18:07:24.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.384+0000 7fc556a00640 1 -- 172.21.15.143:0/3270774163 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580b2d00 con 0x7fc558159b70 2024-01-15T18:07:24.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc556a00640 1 --2- 172.21.15.143:0/3270774163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc558159b70 0x7fc5580b43b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097c0 tx=0x7fc54001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c968d650e026af server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc556000640 1 -- 172.21.15.143:0/3270774163 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400ab600 con 0x7fc558159b70 2024-01-15T18:07:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc556000640 1 -- 172.21.15.143:0/3270774163 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5400a7050 con 0x7fc558159b70 2024-01-15T18:07:24.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc556000640 1 -- 172.21.15.143:0/3270774163 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400ab980 con 0x7fc558159b70 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 -- 172.21.15.143:0/3270774163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc558159b70 msgr2=0x7fc5580b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 --2- 172.21.15.143:0/3270774163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc558159b70 0x7fc5580b43b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097c0 tx=0x7fc54001c480 comp rx=0 tx=0).stop 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 -- 172.21.15.143:0/3270774163 shutdown_connections 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 --2- 172.21.15.143:0/3270774163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5580b48f0 0x7fc558070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 --2- 172.21.15.143:0/3270774163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc558159b70 0x7fc5580b43b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 -- 172.21.15.143:0/3270774163 >> 172.21.15.143:0/3270774163 conn(0x7fc55806c3f0 msgr2=0x7fc55806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 -- 172.21.15.143:0/3270774163 shutdown_connections 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 -- 172.21.15.143:0/3270774163 wait complete. 2024-01-15T18:07:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 Processor -- start 2024-01-15T18:07:24.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.388+0000 7fc55e000640 1 -- start start 2024-01-15T18:07:24.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 0x7fc558149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc558157300 0x7fc55815af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc558159b70 0x7fc55815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5581579f0 con 0x7fc5580b48f0 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55815bb00 con 0x7fc558157300 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55815bde0 con 0x7fc558159b70 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 0x7fc558149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 0x7fc558149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47242/0 (socket says 172.21.15.143:47242) 2024-01-15T18:07:24.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 -- 172.21.15.143:0/2038276770 learned_addr learned my addr 172.21.15.143:0/2038276770 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc556a00640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc558157300 0x7fc55815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557e00640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc558159b70 0x7fc55815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 -- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc558159b70 msgr2=0x7fc55815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc558159b70 0x7fc55815b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 -- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc558157300 msgr2=0x7fc55815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc558157300 0x7fc55815af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 -- 172.21.15.143:0/2038276770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc540009420 con 0x7fc5580b48f0 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557e00640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc558159b70 0x7fc55815b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc557400640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 0x7fc558149920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc5480011f0 tx=0x7fc54800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc54f400640 1 -- 172.21.15.143:0/2038276770 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480074e0 con 0x7fc5580b48f0 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc54f400640 1 -- 172.21.15.143:0/2038276770 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc548007640 con 0x7fc5580b48f0 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc54f400640 1 -- 172.21.15.143:0/2038276770 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc548004a10 con 0x7fc5580b48f0 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55815c0c0 con 0x7fc5580b48f0 2024-01-15T18:07:24.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc558150c40 con 0x7fc5580b48f0 2024-01-15T18:07:24.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.392+0000 7fc54f400640 1 -- 172.21.15.143:0/2038276770 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fc548004b70 con 0x7fc5580b48f0 2024-01-15T18:07:24.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc54f400640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc530066b30 0x7fc530068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc54f400640 1 -- 172.21.15.143:0/2038276770 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc548083b70 con 0x7fc5580b48f0 2024-01-15T18:07:24.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc556a00640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc530066b30 0x7fc530068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 Processor -- start 2024-01-15T18:07:24.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 -- start start 2024-01-15T18:07:24.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc556a00640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc530066b30 0x7fc530068ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5400ad000 tx=0x7fc5400a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0209070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01bede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 0x561ef01bf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef0029130 con 0x561ef01bf320 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef00169a0 con 0x561ef0208c90 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ef0013ed0 con 0x561ef02095b0 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 0x561ef01bf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0209070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 0x561ef01bf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47256/0 (socket says 172.21.15.143:47256) 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 -- 172.21.15.143:0/321435893 learned_addr learned my addr 172.21.15.143:0/321435893 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc556a00640 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01bede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 -- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 msgr2=0x561ef01bede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01bede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 -- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 msgr2=0x561ef0209070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0209070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 -- 172.21.15.143:0/321435893 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eefd07890 con 0x561ef01bf320 2024-01-15T18:07:24.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc557e00640 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 0x561ef01bf700 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc5500073e0 tx=0x7fc5500078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3241eb4242ab728 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc547e00640 1 -- 172.21.15.143:0/321435893 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550003db0 con 0x561ef01bf320 2024-01-15T18:07:24.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc547e00640 1 -- 172.21.15.143:0/321435893 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55000f050 con 0x561ef01bf320 2024-01-15T18:07:24.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.396+0000 7fc547e00640 1 -- 172.21.15.143:0/321435893 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550004230 con 0x561ef01bf320 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/321435893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 msgr2=0x561ef01bf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 0x561ef01bf700 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc5500073e0 tx=0x7fc5500078a0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/321435893 shutdown_connections 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef01bf320 0x561ef01bf700 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01bede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- 172.21.15.143:0/321435893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0209070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/321435893 >> 172.21.15.143:0/321435893 conn(0x561ef01bb2d0 msgr2=0x561ef01bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/321435893 shutdown_connections 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/321435893 wait complete. 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- start start 2024-01-15T18:07:24.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 Processor -- start 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01c4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01d7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef000f7c0 con 0x561ef0208c90 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeffc7bf0 con 0x561ef01bf320 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x561eeffcda90 con 0x561ef02095b0 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 10 client.0 _ll_register_callbacks cb 0x561ef01aa3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:07:24.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47262/0 (socket says 172.21.15.143:47262) 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 -- 172.21.15.143:0/4212678500 learned_addr learned my addr 172.21.15.143:0/4212678500 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc557400640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc557e00640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01d7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 -- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 msgr2=0x561ef01d7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01d7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 -- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 msgr2=0x561ef01c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 -- 172.21.15.143:0/4212678500 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc550009cc0 con 0x561ef0208c90 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc556a00640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01c4280 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c10 tx=0x7fc54001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a82c5da551a990 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc544c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:07:24.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc544c00640 1 -- 172.21.15.143:0/4212678500 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540003620 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc544c00640 1 -- 172.21.15.143:0/4212678500 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc540003780 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561eeffd1470 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc544c00640 1 -- 172.21.15.143:0/4212678500 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc540003940 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 10 client.5913 fetch_fsmap learned FSMap version 26 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.400+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x561eeffd0190 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc544c00640 1 -- 172.21.15.143:0/4212678500 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561eeffd1470 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc544c00640 1 -- 172.21.15.143:0/4212678500 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc540003aa0 con 0x561ef0208c90 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 10 client.5913 fetch_fsmap finished waiting for FSMap version 26 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 10 client.5913 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 20 client.5913 populate_metadata read hostname 'smithi143' 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef01d8320 0x561ef021af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 4 client.5913 mds_command: new command op to 5736 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 --> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561eeffd4b50 con 0x561ef01d8320 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc557400640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef01d8320 0x561ef021af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc544c00640 10 client.5913 ms_handle_connect on v2:172.21.15.143:6834/1835320705 2024-01-15T18:07:24.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc557400640 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef01d8320 0x561ef021af50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b86e3e3bba338d server_cookie=bdaa1104d9e14a2f in_seq=0 out_seq=0 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc544c00640 1 -- 172.21.15.143:0/4212678500 <== mds.0 v2:172.21.15.143:6834/1835320705 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561eeffd4b50 con 0x561ef01d8320 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc544c00640 10 client.5913 handle_command_reply: tid=0 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 client.5913 shutdown 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 msgr2=0x561ef01c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01c4280 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c10 tx=0x7fc54001acc0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 shutdown_connections 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef01d8320 0x561ef021af50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef01d7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01c4280 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc544c00640 0 client.5913 ms_handle_reset on v2:172.21.15.143:6834/1835320705 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 --2- 172.21.15.143:0/4212678500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 >> 172.21.15.143:0/4212678500 conn(0x561ef01bb2d0 msgr2=0x561ef01bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 shutdown_connections 2024-01-15T18:07:24.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 1 -- 172.21.15.143:0/4212678500 wait complete. 2024-01-15T18:07:24.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.404+0000 7fc561508000 20 client.5913 trim_cache size 0 max 16384 2024-01-15T18:07:24.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 Processor -- start 2024-01-15T18:07:24.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 -- start start 2024-01-15T18:07:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef01bf320 0x561ef0207a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0206b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 0x561ef0207070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eefcc8940 con 0x561ef02095b0 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef000cbc0 con 0x561ef0208c90 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ef000cfd0 con 0x561ef01bf320 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 0x561ef0207070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 0x561ef0207070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47272/0 (socket says 172.21.15.143:47272) 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557e00640 1 -- 172.21.15.143:0/2736254934 learned_addr learned my addr 172.21.15.143:0/2736254934 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc556a00640 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0206b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557400640 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef01bf320 0x561ef0207a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557e00640 1 -- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef01bf320 msgr2=0x561ef0207a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557e00640 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef01bf320 0x561ef0207a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.420+0000 7fc557e00640 1 -- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 msgr2=0x561ef0206b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc557e00640 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0206b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc557e00640 1 -- 172.21.15.143:0/2736254934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eeffb6490 con 0x561ef02095b0 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc556a00640 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0206b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc557e00640 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 0x561ef0207070 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x561ef00c5e70 tx=0x7fc55000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297868219a17ec05 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc54cc00640 1 -- 172.21.15.143:0/2736254934 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550015670 con 0x561ef02095b0 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc54cc00640 1 -- 172.21.15.143:0/2736254934 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5500157d0 con 0x561ef02095b0 2024-01-15T18:07:24.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc54cc00640 1 -- 172.21.15.143:0/2736254934 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550015b60 con 0x561ef02095b0 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- 172.21.15.143:0/2736254934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 msgr2=0x561ef0207070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 0x561ef0207070 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x561ef00c5e70 tx=0x7fc55000c000 comp rx=0 tx=0).stop 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- 172.21.15.143:0/2736254934 shutdown_connections 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef02095b0 0x561ef0207070 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef0208c90 0x561ef0206b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- 172.21.15.143:0/2736254934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef01bf320 0x561ef0207a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- 172.21.15.143:0/2736254934 >> 172.21.15.143:0/2736254934 conn(0x561ef01bb2d0 msgr2=0x561ef01bd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- 172.21.15.143:0/2736254934 shutdown_connections 2024-01-15T18:07:24.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- 172.21.15.143:0/2736254934 wait complete. 2024-01-15T18:07:24.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- start start 2024-01-15T18:07:24.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 Processor -- start 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01eb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef021af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eefd081b0 con 0x561ef0208c90 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eeffd1470 con 0x561ef01bf320 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x561eeffe8e10 con 0x561ef02095b0 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc561508000 10 client.0 _ll_register_callbacks cb 0x561ef005dc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc556a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc556a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01ebd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47286/0 (socket says 172.21.15.143:47286) 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc556a00640 1 -- 172.21.15.143:0/3101601424 learned_addr learned my addr 172.21.15.143:0/3101601424 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc557400640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01eb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.424+0000 7fc557e00640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef021af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc556a00640 1 -- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 msgr2=0x561ef021af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc556a00640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef021af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc556a00640 1 -- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 msgr2=0x561ef01eb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc556a00640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01eb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc556a00640 1 -- 172.21.15.143:0/3101601424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561eefd07890 con 0x561ef0208c90 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc557400640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01eb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc556a00640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01ebd20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc54001c740 tx=0x7fc540009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9554ed03a18a1b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 1 -- 172.21.15.143:0/3101601424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400aba70 con 0x561ef0208c90 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 1 -- 172.21.15.143:0/3101601424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc5400044c0 con 0x561ef0208c90 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ef000e110 con 0x561ef0208c90 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 1 -- 172.21.15.143:0/3101601424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc540003ec0 con 0x561ef0208c90 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 10 client.5919 fetch_fsmap learned FSMap version 26 2024-01-15T18:07:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x561eefd07cc0 con 0x561ef0208c90 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 1 -- 172.21.15.143:0/3101601424 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561ef000e110 con 0x561ef0208c90 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 1 -- 172.21.15.143:0/3101601424 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc5400e1470 con 0x561ef0208c90 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 10 client.5919 fetch_fsmap finished waiting for FSMap version 26 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 10 client.5919 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 20 client.5919 populate_metadata read hostname 'smithi143' 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef021b490 0x561ef021b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 4 client.5919 mds_command: new command op to 5736 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 --> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561ef003d9a0 con 0x561ef021b490 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc557400640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef021b490 0x561ef021b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc557400640 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef021b490 0x561ef021b870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd52f55ced8cf765 server_cookie=2e4f19a25cfc4346 in_seq=0 out_seq=0 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 10 client.5919 ms_handle_connect on v2:172.21.15.143:6834/1835320705 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 1 -- 172.21.15.143:0/3101601424 <== mds.0 v2:172.21.15.143:6834/1835320705 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561ef003d9a0 con 0x561ef021b490 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 10 client.5919 handle_command_reply: tid=0 2024-01-15T18:07:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 client.5919 shutdown 2024-01-15T18:07:24.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 msgr2=0x561ef01ebd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01ebd20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc54001c740 tx=0x7fc540009190 comp rx=0 tx=0).stop 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 shutdown_connections 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:6834/1835320705,v1:172.21.15.143:6836/1835320705] conn(0x561ef021b490 0x561ef021b870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x561ef02095b0 0x561ef021af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561ef0208c90 0x561ef01ebd20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 --2- 172.21.15.143:0/3101601424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561ef01bf320 0x561ef01eb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc544c00640 0 client.5919 ms_handle_reset on v2:172.21.15.143:6834/1835320705 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 >> 172.21.15.143:0/3101601424 conn(0x561ef01bb2d0 msgr2=0x561ef02160f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 shutdown_connections 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 1 -- 172.21.15.143:0/3101601424 wait complete. 2024-01-15T18:07:24.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.428+0000 7fc561508000 20 client.5919 trim_cache size 0 max 16384 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5736, 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.310903776, 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 26, 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 46, 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 46, 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 16.214967434999998 2024-01-15T18:07:24.438 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:07:24.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc530066b30 msgr2=0x7fc530068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc530066b30 0x7fc530068ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5400ad000 tx=0x7fc5400a6000 comp rx=0 tx=0).stop 2024-01-15T18:07:24.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 msgr2=0x7fc558149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 0x7fc558149920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fc5480011f0 tx=0x7fc54800c9b0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 shutdown_connections 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc530066b30 0x7fc530068ff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc558159b70 0x7fc55815b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc558157300 0x7fc55815af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 --2- 172.21.15.143:0/2038276770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5580b48f0 0x7fc558149920 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 >> 172.21.15.143:0/2038276770 conn(0x7fc55806c3f0 msgr2=0x7fc5580aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 shutdown_connections 2024-01-15T18:07:24.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.432+0000 7fc55e000640 1 -- 172.21.15.143:0/2038276770 wait complete. 2024-01-15T18:07:24.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:07:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.584+0000 7fcc24200640 1 Processor -- start 2024-01-15T18:07:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.584+0000 7fcc24200640 1 -- start start 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc24200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc1c0b24b0 0x7fcc1c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc24200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc24200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1c070bf0 con 0x7fcc1c0ce690 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc24200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1c15dba0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc24200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0b4920 con 0x7fcc1c0b3bf0 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc1c0b24b0 0x7fcc1c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc1c0b24b0 0x7fcc1c0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47288/0 (socket says 172.21.15.143:47288) 2024-01-15T18:07:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 -- 172.21.15.143:0/1266635755 learned_addr learned my addr 172.21.15.143:0/1266635755 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc22200640 1 --2- 172.21.15.143:0/1266635755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 -- 172.21.15.143:0/1266635755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 msgr2=0x7fcc1c0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 --2- 172.21.15.143:0/1266635755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 -- 172.21.15.143:0/1266635755 >> v1:172.21.15.204:6789/0 conn(0x7fcc1c0ce690 legacy=0x7fcc1c0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.588+0000 7fcc20e00640 1 -- 172.21.15.143:0/1266635755 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0b4c00 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc20e00640 1 --2- 172.21.15.143:0/1266635755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c0ac8e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009870 tx=0x7fcc0401cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703400ce0b7360e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc1be00640 1 -- 172.21.15.143:0/1266635755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040a95f0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc1be00640 1 -- 172.21.15.143:0/1266635755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc040a9750 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc1be00640 1 -- 172.21.15.143:0/1266635755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040a9ae0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 -- 172.21.15.143:0/1266635755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 msgr2=0x7fcc1c0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 --2- 172.21.15.143:0/1266635755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c0ac8e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009870 tx=0x7fcc0401cf70 comp rx=0 tx=0).stop 2024-01-15T18:07:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 -- 172.21.15.143:0/1266635755 shutdown_connections 2024-01-15T18:07:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 --2- 172.21.15.143:0/1266635755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 --2- 172.21.15.143:0/1266635755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c0ac8e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 -- 172.21.15.143:0/1266635755 >> 172.21.15.143:0/1266635755 conn(0x7fcc1c06c3f0 msgr2=0x7fcc1c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 -- 172.21.15.143:0/1266635755 shutdown_connections 2024-01-15T18:07:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 -- 172.21.15.143:0/1266635755 wait complete. 2024-01-15T18:07:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 Processor -- start 2024-01-15T18:07:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.592+0000 7fcc24200640 1 -- start start 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c13f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c13da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c15aa50 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c136f90 con 0x7fcc1c0ce690 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c15a4f0 con 0x7fcc1c0b3bf0 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c13f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc20e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c13f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47296/0 (socket says 172.21.15.143:47296) 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 -- 172.21.15.143:0/963931595 learned_addr learned my addr 172.21.15.143:0/963931595 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc22200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c13da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 -- 172.21.15.143:0/963931595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 msgr2=0x7fcc1c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 -- 172.21.15.143:0/963931595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc1c0ce690 msgr2=0x7fcc1c13da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c13da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 -- 172.21.15.143:0/963931595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040094d0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc21800640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c13f460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c3c0 tx=0x7fcc0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c0052d0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c005430 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c1422c0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c003840 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c13fad0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.596+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1c067500 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.600+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7fcc0c0039a0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.600+0000 7fcc19600640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbf4066b30 0x7fcbf4068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.600+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcc0c083ce0 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.600+0000 7fcc20e00640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbf4066b30 0x7fcbf4068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.600+0000 7fcc20e00640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbf4066b30 0x7fcbf4068ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcc040ae000 tx=0x7fcc040b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.604+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc0c050c80 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcc1c06e260 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc19600640 1 -- 172.21.15.143:0/963931595 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcc0c017070 con 0x7fcc1c0b24b0 2024-01-15T18:07:24.795 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:07:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbf4066b30 msgr2=0x7fcbf4068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbf4066b30 0x7fcbf4068ff0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcc040ae000 tx=0x7fcc040b3040 comp rx=0 tx=0).stop 2024-01-15T18:07:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 msgr2=0x7fcc1c13f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c13f460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00c3c0 tx=0x7fcc0c00c880 comp rx=0 tx=0).stop 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 shutdown_connections 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbf4066b30 0x7fcbf4068ff0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc1c0ce690 0x7fcc1c13da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc1c0b3bf0 0x7fcc1c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 --2- 172.21.15.143:0/963931595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc1c0b24b0 0x7fcc1c13f460 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 >> 172.21.15.143:0/963931595 conn(0x7fcc1c06c3f0 msgr2=0x7fcc1c15f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 shutdown_connections 2024-01-15T18:07:24.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:24.792+0000 7fcc24200640 1 -- 172.21.15.143:0/963931595 wait complete. 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:07:24.818 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:07:24.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:07:24.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:24.819 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:07:24.828 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:07:24.829 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42298sec preferred_lft 42298sec 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.830 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:24.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:24.831 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:07:24.887 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:07:24.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:24.890 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:07:24.941 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:07:24.941 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:07:24.941 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:07:24.941 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:07:24.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:24.941 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:07:24.941 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:07:24.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:07:24.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:07:24.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:07:24.941 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:07:25.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.030 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:07:25.039 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:07:25.039 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:07:25.039 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:07:25.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:07:25.101 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:07:25.101 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:07:25.101 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:25.150 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:07:25.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:07:25.150 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:25.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:07:25.262 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:07:25.315 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:07:25.315 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:07:25.315 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:07:25.315 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:07:25.316 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:07:25.317 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:07:25.318 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:07:25.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.318 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:07:25.369 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:07:25.369 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:07:25.413 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:07:25.433 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:07:25.483 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:07:25.483 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:07:25.483 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:07:25.483 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:07:25.484 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:07:25.485 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:07:25.486 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:07:25.486 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:07:25.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.487 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:07:25.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:07:25.504+0000 7fce288ba580 -1 init, newargv = 0x5591bcce9400 newargc=15 2024-01-15T18:07:25.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[39710]: starting ceph client 2024-01-15T18:07:25.514 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[39710]: starting fuse 2024-01-15T18:07:25.516 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:07:25.516 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:07:25.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:07:25.516 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:07:25.516 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:07:25.516 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:07:25.516 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:07:25.517 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:07:25.518 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:07:25.518 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:07:25.518 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:07:25.518 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:07:25.518 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:07:25.641 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.39710.asok 2024-01-15T18:07:25.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:07:25.653 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39710.asok 2024-01-15T18:07:25.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39710.asok status 2024-01-15T18:07:25.826 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:07:25.826 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:07:25.826 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:07:25.826 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:07:25.826 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:07:25.826 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "39710", 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5928, 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 5928 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:07:25.827 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2537565126 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2537565126 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.5928 192.168.0.1:0/2537565126", 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/2537565126", 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 26, 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 46, 2024-01-15T18:07:25.828 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:07:25.829 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:07:25.829 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:07:25.829 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:07:25.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.841 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:07:25.841 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:25.850 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:07:25.850 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:25.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:25.851 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:07:25.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:07:26.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.088+0000 7f387ca00640 1 Processor -- start 2024-01-15T18:07:26.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.088+0000 7f387ca00640 1 -- start start 2024-01-15T18:07:26.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.088+0000 7f387ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 0x7f387406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.088+0000 7f387ca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38740cffd0 con 0x7f38740cbf50 2024-01-15T18:07:26.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.088+0000 7f387ca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38740d02b0 con 0x7f38740cfb30 2024-01-15T18:07:26.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.088+0000 7f387ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740d1ce0 con 0x7f38741607c0 2024-01-15T18:07:26.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f387aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f38740cbf50 0x7f38740cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50084/0 (socket says 172.21.15.143:50084) 2024-01-15T18:07:26.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f387aa00640 1 -- 172.21.15.143:0/3773378096 learned_addr learned my addr 172.21.15.143:0/3773378096 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:26.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f387a000640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 0x7f387406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746718728 0 0) 0x7f38740cffd0 con 0x7f38740cbf50 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3854003530 con 0x7f38740cbf50 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249531026 0 0) 0x7f38740d02b0 con 0x7f38740cfb30 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38740cffd0 con 0x7f38740cfb30 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1171218585 0 0) 0x7f3854003530 con 0x7f38740cbf50 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 msgr2=0x7f387406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 0x7f387406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 >> v1:172.21.15.204:6789/0 conn(0x7f38740cfb30 legacy=0x7f387406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740cb470 con 0x7f38740cbf50 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f386c001a00 con 0x7f38740cbf50 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 >> v1:172.21.15.143:6789/0 conn(0x7f38740cbf50 legacy=0x7f38740cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3854003b40 0x7f3854003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38540047d0 0x7f3854004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 0x7f3854005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740d1ce0 con 0x7f38540047d0 2024-01-15T18:07:26.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385400ba40 con 0x7f3854003b40 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385400bbd0 con 0x7f38741607c0 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3879600640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38540047d0 0x7f3854004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f387aa00640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 0x7f3854005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f387a000640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3854003b40 0x7f3854003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3879600640 1 -- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 msgr2=0x7f3854005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3879600640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38741607c0 0x7f3854005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3879600640 1 -- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3854003b40 msgr2=0x7f3854003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3879600640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3854003b40 0x7f3854003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.092+0000 7f3879600640 1 -- 172.21.15.143:0/3773378096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854003530 con 0x7f38540047d0 2024-01-15T18:07:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f3879600640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38540047d0 0x7f3854004c50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f385c0018d0 tx=0x7f385c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1904ba18a892e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0052d0 con 0x7f38540047d0 2024-01-15T18:07:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c0a7070 con 0x7f38540047d0 2024-01-15T18:07:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f3878c00640 1 -- 172.21.15.143:0/3773378096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c003600 con 0x7f38540047d0 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 -- 172.21.15.143:0/3773378096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38540047d0 msgr2=0x7f3854004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 --2- 172.21.15.143:0/3773378096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38540047d0 0x7f3854004c50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f385c0018d0 tx=0x7f385c01c500 comp rx=0 tx=0).stop 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387aa00640 1 -- 172.21.15.143:0/3773378096 reap_dead start 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 -- 172.21.15.143:0/3773378096 shutdown_connections 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 -- 172.21.15.143:0/3773378096 >> 172.21.15.143:0/3773378096 conn(0x7f38740d0e20 msgr2=0x7f38740d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 -- 172.21.15.143:0/3773378096 shutdown_connections 2024-01-15T18:07:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 -- 172.21.15.143:0/3773378096 wait complete. 2024-01-15T18:07:26.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 Processor -- start 2024-01-15T18:07:26.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.096+0000 7f387ca00640 1 -- start start 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38540047d0 0x7f38741481b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 0x7f38740c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38741f4370 0x7f38740c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c4880 con 0x7f38741f3f10 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c5400 con 0x7f38741f4370 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c4ad0 con 0x7f38540047d0 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 0x7f38740c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 0x7f38740c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47318/0 (socket says 172.21.15.143:47318) 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 -- 172.21.15.143:0/3607992606 learned_addr learned my addr 172.21.15.143:0/3607992606 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387a000640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38540047d0 0x7f38741481b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387aa00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38741f4370 0x7f38740c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 -- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38540047d0 msgr2=0x7f38741481b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38540047d0 0x7f38741481b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 -- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38741f4370 msgr2=0x7f38740c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38741f4370 0x7f38740c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 -- 172.21.15.143:0/3607992606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740c4db0 con 0x7f38741f3f10 2024-01-15T18:07:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387aa00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38741f4370 0x7f38740c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3879600640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 0x7f38740c5890 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ae040 tx=0x7f3854005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0069e0 con 0x7f38741f3f10 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c006b40 con 0x7f38741f3f10 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38740c5090 con 0x7f38741f3f10 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c007030 con 0x7f38741f3f10 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38740c9d20 con 0x7f38741f3f10 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.100+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387406d4b0 con 0x7f38741f3f10 2024-01-15T18:07:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.104+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f385c0b2040 con 0x7f38741f3f10 2024-01-15T18:07:26.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.104+0000 7f3872000640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f384c066b60 0x7f384c069020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.104+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f385c11e3a0 con 0x7f38741f3f10 2024-01-15T18:07:26.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.104+0000 7f387a000640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f384c066b60 0x7f384c069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.104+0000 7f387a000640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f384c066b60 0x7f384c069020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3864009a00 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.108+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f385c0eb280 con 0x7f38741f3f10 2024-01-15T18:07:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.288+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38740c4c10 con 0x7f38741f3f10 2024-01-15T18:07:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.288+0000 7f3872000640 1 -- 172.21.15.143:0/3607992606 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f38740c4c10 con 0x7f38741f3f10 2024-01-15T18:07:26.294 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:07:26.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f384c066b60 msgr2=0x7f384c069020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f384c066b60 0x7f384c069020 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3864009a00 tx=0x7f3864008040 comp rx=0 tx=0).stop 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 msgr2=0x7f38740c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 0x7f38740c5890 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f385c0ae040 tx=0x7f3854005800 comp rx=0 tx=0).stop 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 shutdown_connections 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f384c066b60 0x7f384c069020 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38741f4370 0x7f38740c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38741f3f10 0x7f38740c5890 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 --2- 172.21.15.143:0/3607992606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38540047d0 0x7f38741481b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 >> 172.21.15.143:0/3607992606 conn(0x7f38740d0e20 msgr2=0x7f387415f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 shutdown_connections 2024-01-15T18:07:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:26.292+0000 7f387ca00640 1 -- 172.21.15.143:0/3607992606 wait complete. 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:07:26.316 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:07:26.317 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:07:26.317 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:07:26.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.317 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:26.327 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42291sec preferred_lft 42291sec 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:07:26.328 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:07:26.329 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:07:26.330 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:07:26.330 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:07:26.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.330 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:07:26.391 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:07:26.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.393 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:07:26.447 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:07:26.447 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:07:26.447 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:07:26.447 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:07:26.447 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:07:26.447 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:07:26.448 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:07:26.448 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:07:26.448 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:07:26.448 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:07:26.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.448 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:07:26.448 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:07:26.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:07:26.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:07:26.448 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:07:26.448 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:07:26.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.540 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:07:26.549 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:07:26.549 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:07:26.549 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:07:26.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.549 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:07:26.612 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:07:26.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:07:26.612 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:26.667 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:07:26.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:07:26.668 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:26.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:07:26.780 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:07:26.833 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:07:26.834 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:07:26.835 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:07:26.835 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:07:26.835 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:07:26.835 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:07:26.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:26.836 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:07:26.887 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:07:26.887 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:07:26.930 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:07:26.950 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:07:27.001 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:07:27.002 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:07:27.003 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:07:27.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:27.004 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:07:27.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:07:27.020+0000 7fc621f38580 -1 init, newargv = 0x5584d5dfd3b0 newargc=15 2024-01-15T18:07:27.025 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33616]: starting ceph client 2024-01-15T18:07:27.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33616]: starting fuse 2024-01-15T18:07:27.033 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:07:27.033 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:07:27.034 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:07:27.035 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:07:27.159 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.33616.asok 2024-01-15T18:07:27.162 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:07:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:07:27.170 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33616.asok 2024-01-15T18:07:27.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:27.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33616.asok status 2024-01-15T18:07:27.341 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "33616", 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 5393, 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:07:27.342 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 5393 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 1464666032 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 1464666032 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:07:27.343 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.5393 192.168.0.1:0/1464666032", 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/1464666032", 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 26, 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 46, 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:07:27.344 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:07:27.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:27.356 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:07:27.356 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:27.366 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:07:27.366 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:27.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:27.366 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:07:27.426 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:07:27.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 Processor -- start 2024-01-15T18:07:27.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 -- start start 2024-01-15T18:07:27.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f509c0ccaf0 0x7f509c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c0c4d90 con 0x7f509c0d19f0 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c15b640 con 0x7f509c0ccaf0 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0cf800 con 0x7f509c15b260 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a1600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f509c0d19f0 0x7f509c0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52450/0 (socket says 172.21.15.143:52450) 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a1600640 1 -- 172.21.15.143:0/2090573234 learned_addr learned my addr 172.21.15.143:0/2090573234 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a2a00640 1 --2- 172.21.15.143:0/2090573234 >> v2:172.21.15.204:3300/0 conn(0x7f509c0ccaf0 0x7f509c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a2000640 1 --2- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a0c00640 1 -- 172.21.15.143:0/2090573234 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889812824 0 0) 0x7f509c0c4d90 con 0x7f509c0d19f0 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a0c00640 1 -- 172.21.15.143:0/2090573234 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f507c003530 con 0x7f509c0d19f0 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a2a00640 1 -- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 msgr2=0x7f509c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a2a00640 1 --2- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.572+0000 7f50a2a00640 1 -- 172.21.15.143:0/2090573234 >> v1:172.21.15.143:6789/0 conn(0x7f509c0d19f0 legacy=0x7f509c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a2a00640 1 -- 172.21.15.143:0/2090573234 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0cfae0 con 0x7f509c0ccaf0 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a2a00640 1 --2- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0cdd50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f508c009d90 tx=0x7f508c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d521e59af44e1aea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a0c00640 1 -- 172.21.15.143:0/2090573234 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c0a5900 con 0x7f509c0ccaf0 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a0c00640 1 -- 172.21.15.143:0/2090573234 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c0a5a60 con 0x7f509c0ccaf0 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a0c00640 1 -- 172.21.15.143:0/2090573234 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c0a5df0 con 0x7f509c0ccaf0 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a4a00640 1 -- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 msgr2=0x7f509c0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.576+0000 7f50a4a00640 1 --2- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0cdd50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f508c009d90 tx=0x7f508c01c950 comp rx=0 tx=0).stop 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- 172.21.15.143:0/2090573234 shutdown_connections 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 --2- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0cdd50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 --2- 172.21.15.143:0/2090573234 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- 172.21.15.143:0/2090573234 >> 172.21.15.143:0/2090573234 conn(0x7f509c0d0ed0 msgr2=0x7f509c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- 172.21.15.143:0/2090573234 shutdown_connections 2024-01-15T18:07:27.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- 172.21.15.143:0/2090573234 wait complete. 2024-01-15T18:07:27.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 Processor -- start 2024-01-15T18:07:27.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- start start 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 0x7f509c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0ae030 con 0x7f509c0d19f0 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1f6290 con 0x7f509c0ccaf0 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a4a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1f6570 con 0x7f509c15b260 2024-01-15T18:07:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 0x7f509c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 0x7f509c155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41900/0 (socket says 172.21.15.143:41900) 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a1600640 1 -- 172.21.15.143:0/3302125377 learned_addr learned my addr 172.21.15.143:0/3302125377 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a2a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.580+0000 7f50a2000640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a1600640 1 -- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 msgr2=0x7f509c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a1600640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a1600640 1 -- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 msgr2=0x7f509c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a1600640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a1600640 1 -- 172.21.15.143:0/3302125377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009a70 con 0x7f509c0d19f0 2024-01-15T18:07:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a2a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a2000640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a1600640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 0x7f509c155380 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f508400c420 tx=0x7f508400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084005cf0 con 0x7f509c0d19f0 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084005e50 con 0x7f509c0d19f0 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c0c9170 con 0x7f509c0d19f0 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840072d0 con 0x7f509c0d19f0 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c06fbf0 con 0x7f509c0d19f0 2024-01-15T18:07:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c1f6790 con 0x7f509c0d19f0 2024-01-15T18:07:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f5084007430 con 0x7f509c0d19f0 2024-01-15T18:07:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f509a000640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5074066b30 0x7f5074068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.584+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5084083da0 con 0x7f509c0d19f0 2024-01-15T18:07:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.588+0000 7f50a2000640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5074066b30 0x7f5074068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.588+0000 7f50a2000640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5074066b30 0x7f5074068ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5094009eb0 tx=0x7f5094009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.592+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5084050d00 con 0x7f509c0d19f0 2024-01-15T18:07:27.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:27.644 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:27.776+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f509c067500 con 0x7f509c0d19f0 2024-01-15T18:07:28.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.732+0000 7f509a000640 1 -- 172.21.15.143:0/3302125377 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7f50840170c0 con 0x7f509c0d19f0 2024-01-15T18:07:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5074066b30 msgr2=0x7f5074068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5074066b30 0x7f5074068ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5094009eb0 tx=0x7f5094009190 comp rx=0 tx=0).stop 2024-01-15T18:07:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 msgr2=0x7f509c155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 0x7f509c155380 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f508400c420 tx=0x7f508400c8e0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 shutdown_connections 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5074066b30 0x7f5074068ff0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f509c15b260 0x7f509c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f509c0d19f0 0x7f509c155380 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 --2- 172.21.15.143:0/3302125377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f509c0ccaf0 0x7f509c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 >> 172.21.15.143:0/3302125377 conn(0x7f509c0d0ed0 msgr2=0x7f509c0c6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 shutdown_connections 2024-01-15T18:07:28.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.736+0000 7f50a4a00640 1 -- 172.21.15.143:0/3302125377 wait complete. 2024-01-15T18:07:28.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:07:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 Processor -- start 2024-01-15T18:07:28.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 -- start start 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2d90143780 0x7f2d900cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d900ce520 0x7f2d90071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d900ceb50 con 0x7f2d900720f0 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9015c110 con 0x7f2d900ce520 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d97200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d90072530 con 0x7f2d90143780 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d900ce520 0x7f2d90071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d900ce520 0x7f2d90071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41914/0 (socket says 172.21.15.143:41914) 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 -- 172.21.15.143:0/638367316 learned_addr learned my addr 172.21.15.143:0/638367316 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d95200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2d900720f0 0x7f2d9015a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52460/0 (socket says 172.21.15.143:52460) 2024-01-15T18:07:28.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8fe00640 1 --2- 172.21.15.143:0/638367316 >> v2:172.21.15.204:3300/0 conn(0x7f2d90143780 0x7f2d900cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 -- 172.21.15.143:0/638367316 >> v2:172.21.15.204:3300/0 conn(0x7f2d90143780 msgr2=0x7f2d900cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 --2- 172.21.15.143:0/638367316 >> v2:172.21.15.204:3300/0 conn(0x7f2d90143780 0x7f2d900cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 -- 172.21.15.143:0/638367316 >> v1:172.21.15.143:6789/0 conn(0x7f2d900720f0 legacy=0x7f2d9015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8ea00640 1 -- 172.21.15.143:0/638367316 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322080439 0 0) 0x7f2d880014a0 con 0x7f2d900720f0 2024-01-15T18:07:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.904+0000 7f2d8f400640 1 -- 172.21.15.143:0/638367316 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9015b7e0 con 0x7f2d900ce520 2024-01-15T18:07:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d8f400640 1 --2- 172.21.15.143:0/638367316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d900ce520 0x7f2d90071bb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006be30 tx=0x7f2d7801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a096ba328cdc5c03 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d8ea00640 1 -- 172.21.15.143:0/638367316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780a7bb0 con 0x7f2d900ce520 2024-01-15T18:07:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d8ea00640 1 -- 172.21.15.143:0/638367316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d780a7d10 con 0x7f2d900ce520 2024-01-15T18:07:28.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d8ea00640 1 -- 172.21.15.143:0/638367316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780af660 con 0x7f2d900ce520 2024-01-15T18:07:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 -- 172.21.15.143:0/638367316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d900ce520 msgr2=0x7f2d90071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 --2- 172.21.15.143:0/638367316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d900ce520 0x7f2d90071bb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006be30 tx=0x7f2d7801ebd0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 -- 172.21.15.143:0/638367316 shutdown_connections 2024-01-15T18:07:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 --2- 172.21.15.143:0/638367316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d900ce520 0x7f2d90071bb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 --2- 172.21.15.143:0/638367316 >> v2:172.21.15.204:3300/0 conn(0x7f2d90143780 0x7f2d900cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 -- 172.21.15.143:0/638367316 >> 172.21.15.143:0/638367316 conn(0x7f2d9006c3f0 msgr2=0x7f2d9006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 -- 172.21.15.143:0/638367316 shutdown_connections 2024-01-15T18:07:28.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.908+0000 7f2d97200640 1 -- 172.21.15.143:0/638367316 wait complete. 2024-01-15T18:07:28.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 Processor -- start 2024-01-15T18:07:28.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 -- start start 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d900720f0 0x7f2d90159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d900ce520 0x7f2d90135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 0x7f2d900cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90136180 con 0x7f2d90143780 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90135ea0 con 0x7f2d900720f0 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90130140 con 0x7f2d900ce520 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 0x7f2d900cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 0x7f2d900cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41916/0 (socket says 172.21.15.143:41916) 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 -- 172.21.15.143:0/2471493316 learned_addr learned my addr 172.21.15.143:0/2471493316 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d8f400640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d900ce520 0x7f2d90135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d8fe00640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d900720f0 0x7f2d90159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 -- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d900ce520 msgr2=0x7f2d90135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d900ce520 0x7f2d90135450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 -- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d900720f0 msgr2=0x7f2d90159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d900720f0 0x7f2d90159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 -- 172.21.15.143:0/2471493316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7800b770 con 0x7f2d90143780 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d95200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 0x7f2d900cd490 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c730 tx=0x7f2d8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800ce40 con 0x7f2d90143780 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8800a640 con 0x7f2d90143780 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800a9f0 con 0x7f2d90143780 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d90130420 con 0x7f2d90143780 2024-01-15T18:07:28.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.912+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d90159e90 con 0x7f2d90143780 2024-01-15T18:07:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.916+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9012ffa0 con 0x7f2d90143780 2024-01-15T18:07:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.916+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f2d8800ab50 con 0x7f2d90143780 2024-01-15T18:07:28.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.916+0000 7f2d87e00640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d68066b30 0x7f2d68068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.916+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2d88083c70 con 0x7f2d90143780 2024-01-15T18:07:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.916+0000 7f2d8fe00640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d68066b30 0x7f2d68068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.916+0000 7f2d8fe00640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d68066b30 0x7f2d68068ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009ac0 tx=0x7f2d80009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:28.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:28.920+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d9012ffa0 con 0x7f2d90143780 2024-01-15T18:07:29.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.092+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d90130940 con 0x7f2d68066b30 2024-01-15T18:07:29.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.092+0000 7f2d87e00640 1 -- 172.21.15.143:0/2471493316 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2d90130940 con 0x7f2d68066b30 2024-01-15T18:07:29.097 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:07:29.098 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:07:29.098 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:07:29.098 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:07:29.098 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:07:29.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d68066b30 msgr2=0x7f2d68068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d68066b30 0x7f2d68068ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009ac0 tx=0x7f2d80009190 comp rx=0 tx=0).stop 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 msgr2=0x7f2d900cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 0x7f2d900cd490 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c730 tx=0x7f2d8800cbf0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 shutdown_connections 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d68066b30 0x7f2d68068ff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d90143780 0x7f2d900cd490 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d900ce520 0x7f2d90135450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 --2- 172.21.15.143:0/2471493316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d900720f0 0x7f2d90159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 >> 172.21.15.143:0/2471493316 conn(0x7f2d9006c3f0 msgr2=0x7f2d900afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 shutdown_connections 2024-01-15T18:07:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.096+0000 7f2d97200640 1 -- 172.21.15.143:0/2471493316 wait complete. 2024-01-15T18:07:29.119 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:07:29.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 Processor -- start 2024-01-15T18:07:29.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 -- start start 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0b0310 0x7f476c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f476c0cf740 0x7f476c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f476c0cbff0 con 0x7f476c06bfb0 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476c0d0c50 con 0x7f476c0cf740 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4774600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0cb6c0 con 0x7f476c0b0310 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4771c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f476c06bfb0 0x7f476c06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52474/0 (socket says 172.21.15.143:52474) 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4771c00640 1 -- 172.21.15.143:0/1837625502 learned_addr learned my addr 172.21.15.143:0/1837625502 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:29.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4772600640 1 --2- 172.21.15.143:0/1837625502 >> v2:172.21.15.143:3300/0 conn(0x7f476c0cf740 0x7f476c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4771200640 1 --2- 172.21.15.143:0/1837625502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0b0310 0x7f476c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4772600640 1 -- 172.21.15.143:0/1837625502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0b0310 msgr2=0x7f476c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4772600640 1 --2- 172.21.15.143:0/1837625502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0b0310 0x7f476c0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4772600640 1 -- 172.21.15.143:0/1837625502 >> v1:172.21.15.143:6789/0 conn(0x7f476c06bfb0 legacy=0x7f476c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f4772600640 1 -- 172.21.15.143:0/1837625502 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0cb9a0 con 0x7f476c0cf740 2024-01-15T18:07:29.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.256+0000 7f476be00640 1 -- 172.21.15.143:0/1837625502 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653269234 0 0) 0x7f476c0cbff0 con 0x7f476c06bfb0 2024-01-15T18:07:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4772600640 1 --2- 172.21.15.143:0/1837625502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0cf740 0x7f476c0cfbf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4764009d90 tx=0x7f476401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54f7833b37e7444 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f476be00640 1 -- 172.21.15.143:0/1837625502 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640a5950 con 0x7f476c0cf740 2024-01-15T18:07:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f476be00640 1 -- 172.21.15.143:0/1837625502 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47640a5ab0 con 0x7f476c0cf740 2024-01-15T18:07:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f476be00640 1 -- 172.21.15.143:0/1837625502 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640a5e40 con 0x7f476c0cf740 2024-01-15T18:07:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 -- 172.21.15.143:0/1837625502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0cf740 msgr2=0x7f476c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 --2- 172.21.15.143:0/1837625502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0cf740 0x7f476c0cfbf0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4764009d90 tx=0x7f476401c890 comp rx=0 tx=0).stop 2024-01-15T18:07:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 -- 172.21.15.143:0/1837625502 shutdown_connections 2024-01-15T18:07:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 --2- 172.21.15.143:0/1837625502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0cf740 0x7f476c0cfbf0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 --2- 172.21.15.143:0/1837625502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0b0310 0x7f476c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 -- 172.21.15.143:0/1837625502 >> 172.21.15.143:0/1837625502 conn(0x7f476c0d0ed0 msgr2=0x7f476c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 -- 172.21.15.143:0/1837625502 shutdown_connections 2024-01-15T18:07:29.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.260+0000 7f4774600640 1 -- 172.21.15.143:0/1837625502 wait complete. 2024-01-15T18:07:29.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 Processor -- start 2024-01-15T18:07:29.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 -- start start 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f476c06bfb0 0x7f476c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 0x7f476c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0cf740 0x7f476c154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0b0060 con 0x7f476c0b0310 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0aea70 con 0x7f476c06bfb0 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476c0ae1d0 con 0x7f476c0cf740 2024-01-15T18:07:29.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 0x7f476c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 0x7f476c0af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41942/0 (socket says 172.21.15.143:41942) 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 -- 172.21.15.143:0/479495360 learned_addr learned my addr 172.21.15.143:0/479495360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4772600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0cf740 0x7f476c154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771c00640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f476c06bfb0 0x7f476c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 -- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0cf740 msgr2=0x7f476c154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0cf740 0x7f476c154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 -- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f476c06bfb0 msgr2=0x7f476c0adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f476c06bfb0 0x7f476c0adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 -- 172.21.15.143:0/479495360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4764009a70 con 0x7f476c0b0310 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4771200640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 0x7f476c0af0d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f475400e930 tx=0x7f475400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754009690 con 0x7f476c0b0310 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475400ca00 con 0x7f476c0b0310 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.264+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476c0ae4b0 con 0x7f476c0b0310 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400cdb0 con 0x7f476c0b0310 2024-01-15T18:07:29.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f476c14dc80 con 0x7f476c0b0310 2024-01-15T18:07:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c14de10 con 0x7f476c0b0310 2024-01-15T18:07:29.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f4754013470 con 0x7f476c0b0310 2024-01-15T18:07:29.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4769600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4744066b30 0x7f4744068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4754084a90 con 0x7f476c0b0310 2024-01-15T18:07:29.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4771c00640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4744066b30 0x7f4744068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.268+0000 7f4771c00640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4744066b30 0x7f4744068ff0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f475c000f70 tx=0x7f475c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.272+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4754051a00 con 0x7f476c0b0310 2024-01-15T18:07:29.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f476c067500 con 0x7f476c0b0310 2024-01-15T18:07:29.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4769600640 1 -- 172.21.15.143:0/479495360 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f47540170c0 con 0x7f476c0b0310 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4744066b30 msgr2=0x7f4744068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4744066b30 0x7f4744068ff0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f475c000f70 tx=0x7f475c007c10 comp rx=0 tx=0).stop 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 msgr2=0x7f476c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 0x7f476c0af0d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f475400e930 tx=0x7f475400edf0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 shutdown_connections 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4744066b30 0x7f4744068ff0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f476c0cf740 0x7f476c154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f476c0b0310 0x7f476c0af0d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 --2- 172.21.15.143:0/479495360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f476c06bfb0 0x7f476c0adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 >> 172.21.15.143:0/479495360 conn(0x7f476c0d0ed0 msgr2=0x7f476c0cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:29.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.424+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 shutdown_connections 2024-01-15T18:07:29.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.428+0000 7f4774600640 1 -- 172.21.15.143:0/479495360 wait complete. 2024-01-15T18:07:29.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:07:29.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 Processor -- start 2024-01-15T18:07:29.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 -- start start 2024-01-15T18:07:29.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f312015b260 0x7f31200cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f31200cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31200c4d90 con 0x7f31200c5220 2024-01-15T18:07:29.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312015b640 con 0x7f312015b260 2024-01-15T18:07:29.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f3125a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31200cf800 con 0x7f31200d19f0 2024-01-15T18:07:29.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f311f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f31200c5220 0x7f31200c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52484/0 (socket says 172.21.15.143:52484) 2024-01-15T18:07:29.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f311f400640 1 -- 172.21.15.143:0/3545899612 learned_addr learned my addr 172.21.15.143:0/3545899612 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f311ea00640 1 --2- 172.21.15.143:0/3545899612 >> v2:172.21.15.204:3300/0 conn(0x7f312015b260 0x7f31200cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f311e000640 1 --2- 172.21.15.143:0/3545899612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f31200cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f311d600640 1 -- 172.21.15.143:0/3545899612 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988144337 0 0) 0x7f31200c4d90 con 0x7f31200c5220 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.592+0000 7f311d600640 1 -- 172.21.15.143:0/3545899612 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3100003530 con 0x7f31200c5220 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311e000640 1 -- 172.21.15.143:0/3545899612 >> v2:172.21.15.204:3300/0 conn(0x7f312015b260 msgr2=0x7f31200cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311e000640 1 --2- 172.21.15.143:0/3545899612 >> v2:172.21.15.204:3300/0 conn(0x7f312015b260 0x7f31200cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311d600640 1 -- 172.21.15.143:0/3545899612 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1051152358 0 0) 0x7f3100003530 con 0x7f31200c5220 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311e000640 1 -- 172.21.15.143:0/3545899612 >> v1:172.21.15.143:6789/0 conn(0x7f31200c5220 legacy=0x7f31200c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311e000640 1 -- 172.21.15.143:0/3545899612 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31200cfae0 con 0x7f31200d19f0 2024-01-15T18:07:29.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311ea00640 1 --2- 172.21.15.143:0/3545899612 >> v2:172.21.15.204:3300/0 conn(0x7f312015b260 0x7f31200cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:29.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311e000640 1 --2- 172.21.15.143:0/3545899612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f31200cdd50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31080097c0 tx=0x7f310801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd1a7179b1d0e3c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311d600640 1 -- 172.21.15.143:0/3545899612 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118001a00 con 0x7f31200d19f0 2024-01-15T18:07:29.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311d600640 1 -- 172.21.15.143:0/3545899612 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080a7050 con 0x7f31200d19f0 2024-01-15T18:07:29.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.596+0000 7f311d600640 1 -- 172.21.15.143:0/3545899612 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31080ab600 con 0x7f31200d19f0 2024-01-15T18:07:29.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- 172.21.15.143:0/3545899612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 msgr2=0x7f31200cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 --2- 172.21.15.143:0/3545899612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f31200cdd50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31080097c0 tx=0x7f310801c370 comp rx=0 tx=0).stop 2024-01-15T18:07:29.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- 172.21.15.143:0/3545899612 shutdown_connections 2024-01-15T18:07:29.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 --2- 172.21.15.143:0/3545899612 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f31200cdd50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 --2- 172.21.15.143:0/3545899612 >> v2:172.21.15.204:3300/0 conn(0x7f312015b260 0x7f31200cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- 172.21.15.143:0/3545899612 >> 172.21.15.143:0/3545899612 conn(0x7f31200d0ed0 msgr2=0x7f31200d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:29.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- 172.21.15.143:0/3545899612 shutdown_connections 2024-01-15T18:07:29.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- 172.21.15.143:0/3545899612 wait complete. 2024-01-15T18:07:29.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 Processor -- start 2024-01-15T18:07:29.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- start start 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31200c5220 0x7f3120157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f312009d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 0x7f3120151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312009ac70 con 0x7f312015b260 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312009af50 con 0x7f31200c5220 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f3125a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312009b230 con 0x7f31200d19f0 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f311f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 0x7f3120151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f311f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 0x7f3120151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41948/0 (socket says 172.21.15.143:41948) 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f311f400640 1 -- 172.21.15.143:0/3188679853 learned_addr learned my addr 172.21.15.143:0/3188679853 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f311ea00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31200c5220 0x7f3120157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.600+0000 7f311e000640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f312009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311f400640 1 -- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 msgr2=0x7f312009d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311f400640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f312009d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311f400640 1 -- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31200c5220 msgr2=0x7f3120157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311f400640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31200c5220 0x7f3120157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311f400640 1 -- 172.21.15.143:0/3188679853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110009420 con 0x7f312015b260 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311f400640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 0x7f3120151680 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f311800cc00 tx=0x7f311800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311800ce10 con 0x7f312015b260 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118012400 con 0x7f312015b260 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3108009420 con 0x7f312015b260 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180127b0 con 0x7f312015b260 2024-01-15T18:07:29.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312009b7f0 con 0x7f312015b260 2024-01-15T18:07:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312009b9e0 con 0x7f312015b260 2024-01-15T18:07:29.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f3118012910 con 0x7f312015b260 2024-01-15T18:07:29.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3116a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30f8066b30 0x7f30f8068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3118084e30 con 0x7f312015b260 2024-01-15T18:07:29.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.604+0000 7f311ea00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30f8066b30 0x7f30f8068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.608+0000 7f311ea00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30f8066b30 0x7f30f8068ff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3110009c50 tx=0x7f3110009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.608+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3118051d90 con 0x7f312015b260 2024-01-15T18:07:29.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.784+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f3120067500 con 0x7f30f8066b30 2024-01-15T18:07:29.792 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:29.788+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:29.792 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:29.788+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:29.792 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:29.788+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:29.792 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:29.788+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:29.792 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:29.788+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:29.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.824+0000 7f3116a00640 1 -- 172.21.15.143:0/3188679853 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3120067500 con 0x7f30f8066b30 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30f8066b30 msgr2=0x7f30f8068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30f8066b30 0x7f30f8068ff0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3110009c50 tx=0x7f3110009190 comp rx=0 tx=0).stop 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 msgr2=0x7f3120151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 0x7f3120151680 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f311800cc00 tx=0x7f311800a430 comp rx=0 tx=0).stop 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 shutdown_connections 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30f8066b30 0x7f30f8068ff0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f312015b260 0x7f3120151680 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31200d19f0 0x7f312009d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 --2- 172.21.15.143:0/3188679853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31200c5220 0x7f3120157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 >> 172.21.15.143:0/3188679853 conn(0x7f31200d0ed0 msgr2=0x7f31200c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 shutdown_connections 2024-01-15T18:07:29.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.828+0000 7f3125a00640 1 -- 172.21.15.143:0/3188679853 wait complete. 2024-01-15T18:07:29.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:07:29.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.984+0000 7f84d7c00640 1 Processor -- start 2024-01-15T18:07:29.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.984+0000 7f84d7c00640 1 -- start start 2024-01-15T18:07:29.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d7c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84d00cfb30 0x7f84d006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d7c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d00cbf50 0x7f84d00cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:29.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d7c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d00cffd0 con 0x7f84d01607c0 2024-01-15T18:07:29.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d7c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d00d1ce0 con 0x7f84d00cfb30 2024-01-15T18:07:29.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d7c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00d02b0 con 0x7f84d00cbf50 2024-01-15T18:07:29.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d00cbf50 0x7f84d00cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84d00cfb30 0x7f84d006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:29.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f84d00cfb30 0x7f84d006bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:47434/0 (socket says 172.21.15.143:47434) 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d00cbf50 0x7f84d00cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:56758/0 (socket says 172.21.15.143:56758) 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 -- 172.21.15.143:0/2607265031 learned_addr learned my addr 172.21.15.143:0/2607265031 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84d5200640 1 --1- 172.21.15.143:0/2607265031 >> v1:172.21.15.204:6789/0 conn(0x7f84d01607c0 0x7f84d006b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:56246/0 (socket says 172.21.15.143:56246) 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 -- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d00cbf50 msgr2=0x7f84d00cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 --2- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d00cbf50 0x7f84d00cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 -- 172.21.15.143:0/2607265031 >> v1:172.21.15.204:6789/0 conn(0x7f84d01607c0 legacy=0x7f84d006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.988+0000 7f84cfe00640 1 -- 172.21.15.143:0/2607265031 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00cb470 con 0x7f84d00cfb30 2024-01-15T18:07:29.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84cfe00640 1 --2- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cfb30 0x7f84d006bec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009870 tx=0x7f84b801ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad177046073d04f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:29.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84cf400640 1 -- 172.21.15.143:0/2607265031 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80a8a30 con 0x7f84d00cfb30 2024-01-15T18:07:29.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84cf400640 1 -- 172.21.15.143:0/2607265031 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b80a8b90 con 0x7f84d00cfb30 2024-01-15T18:07:29.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84cf400640 1 -- 172.21.15.143:0/2607265031 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80b0440 con 0x7f84d00cfb30 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 -- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cfb30 msgr2=0x7f84d006bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cfb30 0x7f84d006bec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009870 tx=0x7f84b801ed30 comp rx=0 tx=0).stop 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 -- 172.21.15.143:0/2607265031 shutdown_connections 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d00cbf50 0x7f84d00cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2607265031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cfb30 0x7f84d006bec0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 -- 172.21.15.143:0/2607265031 >> 172.21.15.143:0/2607265031 conn(0x7f84d00d0e20 msgr2=0x7f84d00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 -- 172.21.15.143:0/2607265031 shutdown_connections 2024-01-15T18:07:29.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.992+0000 7f84d7c00640 1 -- 172.21.15.143:0/2607265031 wait complete. 2024-01-15T18:07:29.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 Processor -- start 2024-01-15T18:07:29.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 -- start start 2024-01-15T18:07:30.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cbf50 0x7f84d0157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 0x7f84d00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d01607c0 0x7f84d00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00adb40 con 0x7f84d00cfb30 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d01f3280 con 0x7f84d00cbf50 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00ca130 con 0x7f84d01607c0 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 0x7f84d00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 0x7f84d00b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41986/0 (socket says 172.21.15.143:41986) 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 -- 172.21.15.143:0/2712199615 learned_addr learned my addr 172.21.15.143:0/2712199615 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d5200640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cbf50 0x7f84d0157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d5c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d01607c0 0x7f84d00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 -- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d01607c0 msgr2=0x7f84d00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d01607c0 0x7f84d00cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 -- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cbf50 msgr2=0x7f84d0157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cbf50 0x7f84d0157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 -- 172.21.15.143:0/2712199615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80094d0 con 0x7f84d00cfb30 2024-01-15T18:07:30.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84cfe00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 0x7f84d00b6520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f84b80b3040 tx=0x7f84b801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80b0dd0 con 0x7f84d00cfb30 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b80bb460 con 0x7f84d00cfb30 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80bb810 con 0x7f84d00cfb30 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d00ca410 con 0x7f84d00cfb30 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:29.996+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d00ca9d0 con 0x7f84d00cfb30 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.000+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d006d4b0 con 0x7f84d00cfb30 2024-01-15T18:07:30.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.000+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 9) v1 ==== 86364+0+0 (secure 0 0 0) 0x7f84b80b83f0 con 0x7f84d00cfb30 2024-01-15T18:07:30.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.000+0000 7f84ccc00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f84a8066b30 0x7f84a8068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:30.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.000+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f84b81221d0 con 0x7f84d00cfb30 2024-01-15T18:07:30.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.000+0000 7f84d5200640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f84a8066b30 0x7f84a8068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:30.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.000+0000 7f84d5200640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f84a8066b30 0x7f84a8068ff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f84c0000f70 tx=0x7f84c0007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:30.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.004+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f84b80ef0a0 con 0x7f84d00cfb30 2024-01-15T18:07:30.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.176+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f84d00ce6b0 con 0x7f84a8066b30 2024-01-15T18:07:30.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.180+0000 7f84ccc00640 1 -- 172.21.15.143:0/2712199615 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f84c00047f0 con 0x7f84a8066b30 2024-01-15T18:07:30.186 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7 2024-01-15T18:07:30.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f84a8066b30 msgr2=0x7f84a8068ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:30.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f84a8066b30 0x7f84a8068ff0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f84c0000f70 tx=0x7f84c0007e50 comp rx=0 tx=0).stop 2024-01-15T18:07:30.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 msgr2=0x7f84d00b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:30.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 0x7f84d00b6520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f84b80b3040 tx=0x7f84b801bad0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 shutdown_connections 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f84a8066b30 0x7f84a8068ff0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f84d01607c0 0x7f84d00cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84d00cfb30 0x7f84d00b6520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 --2- 172.21.15.143:0/2712199615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84d00cbf50 0x7f84d0157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 >> 172.21.15.143:0/2712199615 conn(0x7f84d00d0e20 msgr2=0x7f84d015dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 shutdown_connections 2024-01-15T18:07:30.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:30.184+0000 7f84d7c00640 1 -- 172.21.15.143:0/2712199615 wait complete. 2024-01-15T18:07:30.208 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7 2024-01-15T18:07:30.208 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:30.208 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:30.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:30.217 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:07:37.419 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:07:37.419 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:07:37.419 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.13539 s, 150 MB/s 2024-01-15T18:07:37.420 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:37.420 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:37.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:37.429 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:07:44.460 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:07:44.460 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:07:44.460 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.97823 s, 154 MB/s 2024-01-15T18:07:44.461 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:07:44.461 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:44.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:44.470 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:07:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:07:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:07:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.91145 s, 155 MB/s 2024-01-15T18:07:51.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:07:51.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 Processor -- start 2024-01-15T18:07:51.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 -- start start 2024-01-15T18:07:51.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 0x7f820c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f820c071d00 0x7f820c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c15f310 con 0x7f820c141660 2024-01-15T18:07:51.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c15fbb0 con 0x7f820c071d00 2024-01-15T18:07:51.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f8212800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0ce9e0 con 0x7f820c0ce560 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 0x7f820c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f820c071d00 0x7f820c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 0x7f820c15ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:37560/0 (socket says 172.21.15.143:37560) 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 -- 172.21.15.143:0/1407368319 learned_addr learned my addr 172.21.15.143:0/1407368319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820b400640 1 --1- 172.21.15.143:0/1407368319 >> v1:172.21.15.204:6789/0 conn(0x7f820c141660 0x7f820c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:40690/0 (socket says 172.21.15.143:40690) 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 -- 172.21.15.143:0/1407368319 >> v2:172.21.15.204:3300/0 conn(0x7f820c071d00 msgr2=0x7f820c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 --2- 172.21.15.143:0/1407368319 >> v2:172.21.15.204:3300/0 conn(0x7f820c071d00 0x7f820c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 -- 172.21.15.143:0/1407368319 >> v1:172.21.15.204:6789/0 conn(0x7f820c141660 legacy=0x7f820c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820aa00640 1 -- 172.21.15.143:0/1407368319 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c15f5f0 con 0x7f820c0ce560 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820be00640 1 --2- 172.21.15.143:0/1407368319 >> v2:172.21.15.204:3300/0 conn(0x7f820c071d00 0x7f820c072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:51.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.579+0000 7f820a000640 1 -- 172.21.15.143:0/1407368319 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666523869 0 0) 0x7f820c15f310 con 0x7f820c141660 2024-01-15T18:07:51.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f820aa00640 1 --2- 172.21.15.143:0/1407368319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 0x7f820c15ec80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009870 tx=0x7f81f401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9d8719dfae1cd8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f820a000640 1 -- 172.21.15.143:0/1407368319 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40a7070 con 0x7f820c0ce560 2024-01-15T18:07:51.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f820a000640 1 -- 172.21.15.143:0/1407368319 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f40ab430 con 0x7f820c0ce560 2024-01-15T18:07:51.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f820a000640 1 -- 172.21.15.143:0/1407368319 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40ab7c0 con 0x7f820c0ce560 2024-01-15T18:07:51.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 -- 172.21.15.143:0/1407368319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 msgr2=0x7f820c15ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 --2- 172.21.15.143:0/1407368319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 0x7f820c15ec80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009870 tx=0x7f81f401c540 comp rx=0 tx=0).stop 2024-01-15T18:07:51.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 -- 172.21.15.143:0/1407368319 shutdown_connections 2024-01-15T18:07:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 --2- 172.21.15.143:0/1407368319 >> v2:172.21.15.204:3300/0 conn(0x7f820c071d00 0x7f820c072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 --2- 172.21.15.143:0/1407368319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c0ce560 0x7f820c15ec80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 -- 172.21.15.143:0/1407368319 >> 172.21.15.143:0/1407368319 conn(0x7f820c06c3f0 msgr2=0x7f820c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 -- 172.21.15.143:0/1407368319 shutdown_connections 2024-01-15T18:07:51.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.583+0000 7f8212800640 1 -- 172.21.15.143:0/1407368319 wait complete. 2024-01-15T18:07:51.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 Processor -- start 2024-01-15T18:07:51.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 -- start start 2024-01-15T18:07:51.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f820c071d00 0x7f820c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 0x7f820c0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c141660 0x7f820c143ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c15c500 con 0x7f820c0ce560 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b2040 con 0x7f820c071d00 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0b2320 con 0x7f820c141660 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 0x7f820c0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 0x7f820c0b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50898/0 (socket says 172.21.15.143:50898) 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 -- 172.21.15.143:0/3894287138 learned_addr learned my addr 172.21.15.143:0/3894287138 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820b400640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f820c071d00 0x7f820c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820be00640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c141660 0x7f820c143ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 -- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c141660 msgr2=0x7f820c143ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c141660 0x7f820c143ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 -- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f820c071d00 msgr2=0x7f820c15ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f820c071d00 0x7f820c15ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 -- 172.21.15.143:0/3894287138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009a70 con 0x7f820c0ce560 2024-01-15T18:07:51.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f820aa00640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 0x7f820c0b2e00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f81f40ad040 tx=0x7f81f40192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40bc070 con 0x7f820c0ce560 2024-01-15T18:07:51.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f40b4420 con 0x7f820c0ce560 2024-01-15T18:07:51.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40b47d0 con 0x7f820c0ce560 2024-01-15T18:07:51.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f40094d0 con 0x7f820c0ce560 2024-01-15T18:07:51.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.587+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c0cd150 con 0x7f820c0ce560 2024-01-15T18:07:51.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.591+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c15b220 con 0x7f820c0ce560 2024-01-15T18:07:51.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.591+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81f40b4930 con 0x7f820c0ce560 2024-01-15T18:07:51.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.591+0000 7f8203400640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e4066bf0 0x7f81e40690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.591+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f81f411e210 con 0x7f820c0ce560 2024-01-15T18:07:51.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.591+0000 7f820b400640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e4066bf0 0x7f81e40690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.591+0000 7f820b400640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e4066bf0 0x7f81e40690b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f81fc000f70 tx=0x7f81fc007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.595+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81f40eb010 con 0x7f820c0ce560 2024-01-15T18:07:51.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.751+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7f820c067500 con 0x7f820c0ce560 2024-01-15T18:07:51.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.755+0000 7f8203400640 1 -- 172.21.15.143:0/3894287138 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v15)=0 v15) v1 ==== 131+0+0 (secure 0 0 0) 0x7f81f40ee630 con 0x7f820c0ce560 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e4066bf0 msgr2=0x7f81e40690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e4066bf0 0x7f81e40690b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f81fc000f70 tx=0x7f81fc007df0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 msgr2=0x7f820c0b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 0x7f820c0b2e00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f81f40ad040 tx=0x7f81f40192e0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 shutdown_connections 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e4066bf0 0x7f81e40690b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f820c141660 0x7f820c143ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820c0ce560 0x7f820c0b2e00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 --2- 172.21.15.143:0/3894287138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f820c071d00 0x7f820c15ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 >> 172.21.15.143:0/3894287138 conn(0x7f820c06c3f0 msgr2=0x7f820c0afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 shutdown_connections 2024-01-15T18:07:51.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.759+0000 7f8212800640 1 -- 172.21.15.143:0/3894287138 wait complete. 2024-01-15T18:07:51.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T18:07:51.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f4676000640 1 Processor -- start 2024-01-15T18:07:51.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f4676000640 1 -- start start 2024-01-15T18:07:51.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f4676000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 0x7f4670164540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f4676000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46700b2a20 con 0x7f4670159b70 2024-01-15T18:07:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f4676000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46700b2d00 con 0x7f4670157300 2024-01-15T18:07:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f4676000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46700b35a0 con 0x7f46700b4ac0 2024-01-15T18:07:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466ea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4670159b70 0x7f46700b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45148/0 (socket says 172.21.15.143:45148) 2024-01-15T18:07:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466ea00640 1 -- 172.21.15.143:0/2359799385 learned_addr learned my addr 172.21.15.143:0/2359799385 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466fe00640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 0x7f4670164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235120977 0 0) 0x7f46700b2a20 con 0x7f4670159b70 2024-01-15T18:07:51.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4650003530 con 0x7f4670159b70 2024-01-15T18:07:51.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826958324 0 0) 0x7f46700b2d00 con 0x7f4670157300 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46700b2a20 con 0x7f4670157300 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4099352238 0 0) 0x7f4650003530 con 0x7f4670159b70 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 msgr2=0x7f4670164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 0x7f4670164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 >> v1:172.21.15.204:6789/0 conn(0x7f4670157300 legacy=0x7f46700b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46700b2fe0 con 0x7f4670159b70 2024-01-15T18:07:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4658001680 con 0x7f4670159b70 2024-01-15T18:07:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 >> v1:172.21.15.143:6789/0 conn(0x7f4670159b70 legacy=0x7f46700b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 0x7f4650003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46500040a0 0x7f46500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4650004d10 0x7f4650005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46700b35a0 con 0x7f46500040a0 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465000ba70 con 0x7f4650004d10 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465000bc30 con 0x7f46700b4ac0 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466ea00640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46500040a0 0x7f46500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466f400640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 0x7f4650003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.919+0000 7f466fe00640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4650004d10 0x7f4650005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466ea00640 1 -- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 msgr2=0x7f4650003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466ea00640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46700b4ac0 0x7f4650003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466ea00640 1 -- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4650004d10 msgr2=0x7f4650005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:51.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466ea00640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4650004d10 0x7f4650005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466ea00640 1 -- 172.21.15.143:0/2359799385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650003530 con 0x7f46500040a0 2024-01-15T18:07:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466ea00640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46500040a0 0x7f46500047d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f465800c160 tx=0x7f465801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c070cfad5f4e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580058e0 con 0x7f46500040a0 2024-01-15T18:07:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4658005a60 con 0x7f46500040a0 2024-01-15T18:07:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466e000640 1 -- 172.21.15.143:0/2359799385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658005e50 con 0x7f46500040a0 2024-01-15T18:07:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f4676000640 1 -- 172.21.15.143:0/2359799385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46500040a0 msgr2=0x7f46500047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f4676000640 1 --2- 172.21.15.143:0/2359799385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46500040a0 0x7f46500047d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f465800c160 tx=0x7f465801cba0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.923+0000 7f466fe00640 1 -- 172.21.15.143:0/2359799385 reap_dead start 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- 172.21.15.143:0/2359799385 shutdown_connections 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- 172.21.15.143:0/2359799385 >> 172.21.15.143:0/2359799385 conn(0x7f467006c3f0 msgr2=0x7f467006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- 172.21.15.143:0/2359799385 shutdown_connections 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- 172.21.15.143:0/2359799385 wait complete. 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 Processor -- start 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- start start 2024-01-15T18:07:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500040a0 0x7f4670149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 0x7f46700cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46701e2a30 0x7f46700cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4670159fd0 con 0x7f46701e25b0 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467014d640 con 0x7f46701e2a30 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f4676000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4670159140 con 0x7f46500040a0 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 0x7f46700cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500040a0 0x7f4670149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 0x7f46700cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50920/0 (socket says 172.21.15.143:50920) 2024-01-15T18:07:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 -- 172.21.15.143:0/2854712312 learned_addr learned my addr 172.21.15.143:0/2854712312 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466fe00640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46701e2a30 0x7f46700cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 -- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500040a0 msgr2=0x7f4670149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500040a0 0x7f4670149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 -- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46701e2a30 msgr2=0x7f46700cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46701e2a30 0x7f46700cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.927+0000 7f466ea00640 1 -- 172.21.15.143:0/2854712312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670158530 con 0x7f46701e25b0 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f466ea00640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 0x7f46700cd4d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f465801c9d0 tx=0x7f4650005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658004c00 con 0x7f46701e25b0 2024-01-15T18:07:51.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4658004d60 con 0x7f46701e25b0 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658007310 con 0x7f46701e25b0 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4670158810 con 0x7f46701e25b0 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4670158dd0 con 0x7f46701e25b0 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4670151a90 con 0x7f46701e25b0 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4658007470 con 0x7f46701e25b0 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4667400640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4648066bf0 0x7f46480690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f465811e1c0 con 0x7f46701e25b0 2024-01-15T18:07:51.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.931+0000 7f466f400640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4648066bf0 0x7f46480690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.935+0000 7f466f400640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4648066bf0 0x7f46480690b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f466000dfd0 tx=0x7f4660000bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:51.935+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46580eb010 con 0x7f46701e25b0 2024-01-15T18:07:52.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.087+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f4670067500 con 0x7f46701e25b0 2024-01-15T18:07:52.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.091+0000 7f4667400640 1 -- 172.21.15.143:0/2854712312 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v16)=0 v16) v1 ==== 135+0+0 (secure 0 0 0) 0x7f46580ee630 con 0x7f46701e25b0 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4648066bf0 msgr2=0x7f46480690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4648066bf0 0x7f46480690b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f466000dfd0 tx=0x7f4660000bd0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 msgr2=0x7f46700cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 0x7f46700cd4d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f465801c9d0 tx=0x7f4650005800 comp rx=0 tx=0).stop 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 shutdown_connections 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4648066bf0 0x7f46480690b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46701e2a30 0x7f46700cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46701e25b0 0x7f46700cd4d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 --2- 172.21.15.143:0/2854712312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46500040a0 0x7f4670149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 >> 172.21.15.143:0/2854712312 conn(0x7f467006c3f0 msgr2=0x7f46701556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 shutdown_connections 2024-01-15T18:07:52.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.095+0000 7f4676000640 1 -- 172.21.15.143:0/2854712312 wait complete. 2024-01-15T18:07:52.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:07:52.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.255+0000 7f97a6e00640 1 Processor -- start 2024-01-15T18:07:52.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.255+0000 7f97a6e00640 1 -- start start 2024-01-15T18:07:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.255+0000 7f97a6e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f97a0071bb0 0x7f97a0072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.255+0000 7f97a6e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a00ceb50 con 0x7f97a015c5a0 2024-01-15T18:07:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f97a6e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a015c110 con 0x7f97a0143780 2024-01-15T18:07:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f97a6e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a0072570 con 0x7f97a0071bb0 2024-01-15T18:07:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f97a4e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f97a015c5a0 0x7f97a015c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45162/0 (socket says 172.21.15.143:45162) 2024-01-15T18:07:52.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f97a4e00640 1 -- 172.21.15.143:0/2001765018 learned_addr learned my addr 172.21.15.143:0/2001765018 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:52.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979f400640 1 --2- 172.21.15.143:0/2001765018 >> v2:172.21.15.204:3300/0 conn(0x7f97a0071bb0 0x7f97a0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979ea00640 1 -- 172.21.15.143:0/2001765018 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913429100 0 0) 0x7f97980014a0 con 0x7f97a015c5a0 2024-01-15T18:07:52.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979ea00640 1 -- 172.21.15.143:0/2001765018 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9780003530 con 0x7f97a015c5a0 2024-01-15T18:07:52.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979f400640 1 -- 172.21.15.143:0/2001765018 >> v1:172.21.15.204:6789/0 conn(0x7f97a0143780 legacy=0x7f97a00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979f400640 1 -- 172.21.15.143:0/2001765018 >> v1:172.21.15.143:6789/0 conn(0x7f97a015c5a0 legacy=0x7f97a015c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979f400640 1 -- 172.21.15.143:0/2001765018 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a015b7e0 con 0x7f97a0071bb0 2024-01-15T18:07:52.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.259+0000 7f979f400640 1 --2- 172.21.15.143:0/2001765018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a0071bb0 0x7f97a0072030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97880097c0 tx=0x7f978801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188829b244b2bfea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f979ea00640 1 -- 172.21.15.143:0/2001765018 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880ad680 con 0x7f97a0071bb0 2024-01-15T18:07:52.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f979ea00640 1 -- 172.21.15.143:0/2001765018 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97880ad7e0 con 0x7f97a0071bb0 2024-01-15T18:07:52.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f979ea00640 1 -- 172.21.15.143:0/2001765018 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97880adb70 con 0x7f97a0071bb0 2024-01-15T18:07:52.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 -- 172.21.15.143:0/2001765018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a0071bb0 msgr2=0x7f97a0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 --2- 172.21.15.143:0/2001765018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a0071bb0 0x7f97a0072030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f97880097c0 tx=0x7f978801c480 comp rx=0 tx=0).stop 2024-01-15T18:07:52.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 -- 172.21.15.143:0/2001765018 shutdown_connections 2024-01-15T18:07:52.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 --2- 172.21.15.143:0/2001765018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a0071bb0 0x7f97a0072030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 -- 172.21.15.143:0/2001765018 >> 172.21.15.143:0/2001765018 conn(0x7f97a006c3f0 msgr2=0x7f97a006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 -- 172.21.15.143:0/2001765018 shutdown_connections 2024-01-15T18:07:52.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 -- 172.21.15.143:0/2001765018 wait complete. 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 Processor -- start 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 -- start start 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.263+0000 7f97a6e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 0x7f97a013d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97a0143780 0x7f97a00cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a015c5a0 0x7f97a00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a013e8d0 con 0x7f97a0071bb0 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a013dcc0 con 0x7f97a015c5a0 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a013dfa0 con 0x7f97a0143780 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 0x7f97a013d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 0x7f97a013d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50928/0 (socket says 172.21.15.143:50928) 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 -- 172.21.15.143:0/1875667730 learned_addr learned my addr 172.21.15.143:0/1875667730 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a4e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a015c5a0 0x7f97a00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979f400640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97a0143780 0x7f97a00cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 -- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97a0143780 msgr2=0x7f97a00cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97a0143780 0x7f97a00cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 -- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a015c5a0 msgr2=0x7f97a00cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a015c5a0 0x7f97a00cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 -- 172.21.15.143:0/1875667730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9788009420 con 0x7f97a0071bb0 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f979fe00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 0x7f97a013d200 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f979000c500 tx=0x7f979000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790005220 con 0x7f97a0071bb0 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9790005380 con 0x7f97a0071bb0 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790007270 con 0x7f97a0071bb0 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a013e280 con 0x7f97a0071bb0 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a01599b0 con 0x7f97a0071bb0 2024-01-15T18:07:52.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a0067500 con 0x7f97a0071bb0 2024-01-15T18:07:52.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.267+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f97900073d0 con 0x7f97a0071bb0 2024-01-15T18:07:52.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.271+0000 7f9797e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9778066bc0 0x7f9778069080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.271+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9790084120 con 0x7f97a0071bb0 2024-01-15T18:07:52.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.271+0000 7f979f400640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9778066bc0 0x7f9778069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.271+0000 7f979f400640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9778066bc0 0x7f9778069080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97880af000 tx=0x7f97880a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.275+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9790051070 con 0x7f97a0071bb0 2024-01-15T18:07:52.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.447+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f97a0167ad0 con 0x7f9778066bc0 2024-01-15T18:07:52.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f9797e00640 1 -- 172.21.15.143:0/1875667730 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9788006e20 con 0x7f9778066bc0 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9778066bc0 msgr2=0x7f9778069080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9778066bc0 0x7f9778069080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97880af000 tx=0x7f97880a7000 comp rx=0 tx=0).stop 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 msgr2=0x7f97a013d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 0x7f97a013d200 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f979000c500 tx=0x7f979000c9c0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 shutdown_connections 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9778066bc0 0x7f9778069080 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97a015c5a0 0x7f97a00cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97a0143780 0x7f97a00cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 --2- 172.21.15.143:0/1875667730 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97a0071bb0 0x7f97a013d200 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 >> 172.21.15.143:0/1875667730 conn(0x7f97a006c3f0 msgr2=0x7f97a00ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 shutdown_connections 2024-01-15T18:07:52.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.463+0000 7f97a6e00640 1 -- 172.21.15.143:0/1875667730 wait complete. 2024-01-15T18:07:52.489 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:07:52.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f7465000640 1 Processor -- start 2024-01-15T18:07:52.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f7465000640 1 -- start start 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f7465000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f74600b4300 0x7f74600ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f7465000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7460070bf0 con 0x7f74600ce690 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f7465000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74600b4920 con 0x7f74600b24b0 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f7465000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746015dba0 con 0x7f74600b4300 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f74600ce690 0x7f74600b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45168/0 (socket says 172.21.15.143:45168) 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745de00640 1 -- 172.21.15.143:0/3226222273 learned_addr learned my addr 172.21.15.143:0/3226222273 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745e800640 1 --2- 172.21.15.143:0/3226222273 >> v2:172.21.15.204:3300/0 conn(0x7f74600b4300 0x7f74600ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558841636 0 0) 0x7f7460070bf0 con 0x7f74600ce690 2024-01-15T18:07:52.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7440003530 con 0x7f74600ce690 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3593580564 0 0) 0x7f7440003530 con 0x7f74600ce690 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 >> v2:172.21.15.204:3300/0 conn(0x7f74600b4300 msgr2=0x7f74600ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 --2- 172.21.15.143:0/3226222273 >> v2:172.21.15.204:3300/0 conn(0x7f74600b4300 0x7f74600ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 >> v1:172.21.15.204:6789/0 conn(0x7f74600b24b0 legacy=0x7f74600b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74600b4c00 con 0x7f74600ce690 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f7450001680 con 0x7f74600ce690 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745e800640 1 --2- 172.21.15.143:0/3226222273 >> v2:172.21.15.204:3300/0 conn(0x7f74600b4300 0x7f74600ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:52.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 >> v1:172.21.15.143:6789/0 conn(0x7f74600ce690 legacy=0x7f74600b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7440003980 0x7f7440003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f7440004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7440005190 0x7f744000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746015dba0 con 0x7f74400047d0 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744000c030 con 0x7f7440005190 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744000c1f0 con 0x7f7440003980 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745d400640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f7440004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745e800640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7440005190 0x7f744000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.623+0000 7f745de00640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7440003980 0x7f7440003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745d400640 1 -- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7440003980 msgr2=0x7f7440003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745d400640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7440003980 0x7f7440003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745d400640 1 -- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7440005190 msgr2=0x7f744000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745d400640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7440005190 0x7f744000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745d400640 1 -- 172.21.15.143:0/3226222273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7440003530 con 0x7f74400047d0 2024-01-15T18:07:52.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745e800640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7440005190 0x7f744000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:52.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745e800640 1 -- 172.21.15.143:0/3226222273 reap_dead start 2024-01-15T18:07:52.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745d400640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f7440004c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7448001750 tx=0x7f744801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3c327ea33f5b57 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74480052d0 con 0x7f74400047d0 2024-01-15T18:07:52.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74480a7050 con 0x7f74400047d0 2024-01-15T18:07:52.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.627+0000 7f745ca00640 1 -- 172.21.15.143:0/3226222273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448003600 con 0x7f74400047d0 2024-01-15T18:07:52.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- 172.21.15.143:0/3226222273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 msgr2=0x7f7440004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f7440004c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7448001750 tx=0x7f744801c3a0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- 172.21.15.143:0/3226222273 shutdown_connections 2024-01-15T18:07:52.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 --2- 172.21.15.143:0/3226222273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f7440004c50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- 172.21.15.143:0/3226222273 >> 172.21.15.143:0/3226222273 conn(0x7f746006c3f0 msgr2=0x7f746006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- 172.21.15.143:0/3226222273 shutdown_connections 2024-01-15T18:07:52.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- 172.21.15.143:0/3226222273 wait complete. 2024-01-15T18:07:52.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 Processor -- start 2024-01-15T18:07:52.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- start start 2024-01-15T18:07:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f746013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74600b02a0 0x7f746013b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746015c4a0 0x7f746015c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746015ce60 con 0x7f74400047d0 2024-01-15T18:07:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746013ef50 con 0x7f74600b02a0 2024-01-15T18:07:52.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f7465000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746013def0 con 0x7f746015c4a0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f745de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f746013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f745de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f746013cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50946/0 (socket says 172.21.15.143:50946) 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f745de00640 1 -- 172.21.15.143:0/862604076 learned_addr learned my addr 172.21.15.143:0/862604076 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f745d400640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74600b02a0 0x7f746013b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.631+0000 7f745e800640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746015c4a0 0x7f746015c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745de00640 1 -- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746015c4a0 msgr2=0x7f746015c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745de00640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746015c4a0 0x7f746015c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745de00640 1 -- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74600b02a0 msgr2=0x7f746013b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745de00640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74600b02a0 0x7f746013b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745de00640 1 -- 172.21.15.143:0/862604076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746013e190 con 0x7f74400047d0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745d400640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74600b02a0 0x7f746013b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f745de00640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f746013cdb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74500124a0 tx=0x7f7450012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450014810 con 0x7f74400047d0 2024-01-15T18:07:52.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450014970 con 0x7f74400047d0 2024-01-15T18:07:52.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746013e470 con 0x7f74400047d0 2024-01-15T18:07:52.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450014d20 con 0x7f74400047d0 2024-01-15T18:07:52.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746013ea30 con 0x7f74400047d0 2024-01-15T18:07:52.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7460067500 con 0x7f74400047d0 2024-01-15T18:07:52.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.635+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f74500153e0 con 0x7f74400047d0 2024-01-15T18:07:52.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.639+0000 7f7456000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7438066bf0 0x7f74380690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.639+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f74500870a0 con 0x7f74400047d0 2024-01-15T18:07:52.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.639+0000 7f745d400640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7438066bf0 0x7f74380690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.639+0000 7f745d400640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7438066bf0 0x7f74380690b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74600b2940 tx=0x7f74480011a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.643+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7450053fa0 con 0x7f74400047d0 2024-01-15T18:07:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.819+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f74600cea70 con 0x7f7438066bf0 2024-01-15T18:07:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7456000640 1 -- 172.21.15.143:0/862604076 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f7448005480 con 0x7f7438066bf0 2024-01-15T18:07:52.826 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7438066bf0 msgr2=0x7f74380690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7438066bf0 0x7f74380690b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f74600b2940 tx=0x7f74480011a0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 msgr2=0x7f746013cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f746013cdb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f74500124a0 tx=0x7f7450012960 comp rx=0 tx=0).stop 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 shutdown_connections 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7438066bf0 0x7f74380690b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f746015c4a0 0x7f746015c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74600b02a0 0x7f746013b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 --2- 172.21.15.143:0/862604076 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74400047d0 0x7f746013cdb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 >> 172.21.15.143:0/862604076 conn(0x7f746006c3f0 msgr2=0x7f7460071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 shutdown_connections 2024-01-15T18:07:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:52.823+0000 7f7465000640 1 -- 172.21.15.143:0/862604076 wait complete. 2024-01-15T18:07:52.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:52.848 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7) 2024-01-15T18:07:52.862 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7 2024-01-15T18:07:52.862 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:07:52.862 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:52.863 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:07:54.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:54.863 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7) 2024-01-15T18:07:54.878 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/8e360574-7965-4be1-b063-031c059f4ac7 2024-01-15T18:07:54.878 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:07:54.878 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:07:54.879 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:07:54.880 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87831 2024-01-15T18:07:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.067+0000 7f8c4da00640 1 Processor -- start 2024-01-15T18:07:55.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.067+0000 7f8c4da00640 1 -- start start 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.067+0000 7f8c4da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c48141660 0x7f8c480ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c4da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8c4815ec80 0x7f8c48141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c4da00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4815f310 con 0x7f8c48142470 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c4da00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4815fbb0 con 0x7f8c48141660 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c4da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4815f130 con 0x7f8c4815ec80 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c48141660 0x7f8c480ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c47400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8c48142470 0x7f8c48071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45182/0 (socket says 172.21.15.143:45182) 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c47400640 1 -- 172.21.15.143:0/3254902271 learned_addr learned my addr 172.21.15.143:0/3254902271 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46000640 1 --2- 172.21.15.143:0/3254902271 >> v2:172.21.15.204:3300/0 conn(0x7f8c4815ec80 0x7f8c48141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46a00640 1 -- 172.21.15.143:0/3254902271 >> v2:172.21.15.204:3300/0 conn(0x7f8c4815ec80 msgr2=0x7f8c48141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46a00640 1 --2- 172.21.15.143:0/3254902271 >> v2:172.21.15.204:3300/0 conn(0x7f8c4815ec80 0x7f8c48141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46a00640 1 -- 172.21.15.143:0/3254902271 >> v1:172.21.15.143:6789/0 conn(0x7f8c48142470 legacy=0x7f8c48071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46a00640 1 -- 172.21.15.143:0/3254902271 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4815f5f0 con 0x7f8c48141660 2024-01-15T18:07:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c46a00640 1 --2- 172.21.15.143:0/3254902271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48141660 0x7f8c480ce560 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8c4806f940 tx=0x7f8c3801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6f47129f8a04c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.071+0000 7f8c45600640 1 -- 172.21.15.143:0/3254902271 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380a7070 con 0x7f8c48141660 2024-01-15T18:07:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c45600640 1 -- 172.21.15.143:0/3254902271 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c380ab430 con 0x7f8c48141660 2024-01-15T18:07:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c45600640 1 -- 172.21.15.143:0/3254902271 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380ab7c0 con 0x7f8c48141660 2024-01-15T18:07:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 -- 172.21.15.143:0/3254902271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48141660 msgr2=0x7f8c480ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 --2- 172.21.15.143:0/3254902271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48141660 0x7f8c480ce560 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8c4806f940 tx=0x7f8c3801c430 comp rx=0 tx=0).stop 2024-01-15T18:07:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 -- 172.21.15.143:0/3254902271 shutdown_connections 2024-01-15T18:07:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 --2- 172.21.15.143:0/3254902271 >> v2:172.21.15.204:3300/0 conn(0x7f8c4815ec80 0x7f8c48141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 --2- 172.21.15.143:0/3254902271 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48141660 0x7f8c480ce560 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 -- 172.21.15.143:0/3254902271 >> 172.21.15.143:0/3254902271 conn(0x7f8c4806c3f0 msgr2=0x7f8c4806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 -- 172.21.15.143:0/3254902271 shutdown_connections 2024-01-15T18:07:55.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 -- 172.21.15.143:0/3254902271 wait complete. 2024-01-15T18:07:55.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 Processor -- start 2024-01-15T18:07:55.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 -- start start 2024-01-15T18:07:55.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.075+0000 7f8c4da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c48141660 0x7f8c48158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 0x7f8c48159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c4815ec80 0x7f8c48159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48140e50 con 0x7f8c48142470 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4813fe20 con 0x7f8c48141660 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48140100 con 0x7f8c4815ec80 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 0x7f8c48159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 0x7f8c48159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:50972/0 (socket says 172.21.15.143:50972) 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 -- 172.21.15.143:0/1891163892 learned_addr learned my addr 172.21.15.143:0/1891163892 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46a00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c48141660 0x7f8c48158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c47400640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c4815ec80 0x7f8c48159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 -- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c4815ec80 msgr2=0x7f8c48159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c4815ec80 0x7f8c48159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 -- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c48141660 msgr2=0x7f8c48158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c48141660 0x7f8c48158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 -- 172.21.15.143:0/1891163892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38009420 con 0x7f8c48142470 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c46000640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 0x7f8c48159320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000e870 tx=0x7f8c3000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30009510 con 0x7f8c48142470 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3000eec0 con 0x7f8c48142470 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000cb40 con 0x7f8c48142470 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c480cd150 con 0x7f8c48142470 2024-01-15T18:07:55.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4815ba40 con 0x7f8c48142470 2024-01-15T18:07:55.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c48067500 con 0x7f8c48142470 2024-01-15T18:07:55.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.079+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c3000cca0 con 0x7f8c48142470 2024-01-15T18:07:55.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.083+0000 7f8c3ea00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c20066bc0 0x7f8c20069080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.083+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8c30084a10 con 0x7f8c48142470 2024-01-15T18:07:55.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.083+0000 7f8c46a00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c20066bc0 0x7f8c20069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.083+0000 7f8c46a00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c20066bc0 0x7f8c20069080 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c380af000 tx=0x7f8c380a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.087+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8c300518a0 con 0x7f8c48142470 2024-01-15T18:07:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.263+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c480ccfd0 con 0x7f8c20066bc0 2024-01-15T18:07:55.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.323+0000 7f8c3ea00640 1 -- 172.21.15.143:0/1891163892 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8c380ab8e0 con 0x7f8c20066bc0 2024-01-15T18:07:55.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c20066bc0 msgr2=0x7f8c20069080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c20066bc0 0x7f8c20069080 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8c380af000 tx=0x7f8c380a7000 comp rx=0 tx=0).stop 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 msgr2=0x7f8c48159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 0x7f8c48159320 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000e870 tx=0x7f8c3000ec70 comp rx=0 tx=0).stop 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 shutdown_connections 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c20066bc0 0x7f8c20069080 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c4815ec80 0x7f8c48159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c48142470 0x7f8c48159320 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 --2- 172.21.15.143:0/1891163892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c48141660 0x7f8c48158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 >> 172.21.15.143:0/1891163892 conn(0x7f8c4806c3f0 msgr2=0x7f8c4813dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 shutdown_connections 2024-01-15T18:07:55.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f8c4da00640 1 -- 172.21.15.143:0/1891163892 wait complete. 2024-01-15T18:07:55.333 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.333 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.333 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.334 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.334 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.327+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.353 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.347+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.353 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.347+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.353 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.347+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.353 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.347+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.353 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:07:55.347+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:55.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87832 2024-01-15T18:07:55.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2499000640 1 Processor -- start 2024-01-15T18:07:55.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2499000640 1 -- start start 2024-01-15T18:07:55.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2499000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24940ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2499000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249415f310 con 0x7f249415ec80 2024-01-15T18:07:55.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2499000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249415f5f0 con 0x7f2494071d00 2024-01-15T18:07:55.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2499000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249415fbb0 con 0x7f2494141660 2024-01-15T18:07:55.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2491400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f249415ec80 0x7f2494141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45186/0 (socket says 172.21.15.143:45186) 2024-01-15T18:07:55.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2491400640 1 -- 172.21.15.143:0/2419738359 learned_addr learned my addr 172.21.15.143:0/2419738359 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:55.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953086101 0 0) 0x7f249415f310 con 0x7f249415ec80 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2474003530 con 0x7f249415ec80 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2491e00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24940ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621680072 0 0) 0x7f2474003530 con 0x7f249415ec80 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 msgr2=0x7f24940ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24940ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 >> v1:172.21.15.204:6789/0 conn(0x7f2494071d00 legacy=0x7f24940b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249415f8d0 con 0x7f249415ec80 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f247c001680 con 0x7f249415ec80 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 >> v1:172.21.15.143:6789/0 conn(0x7f249415ec80 legacy=0x7f2494141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24740039b0 0x7f2474003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2474004d10 0x7f2474005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249415fbb0 con 0x7f24740039b0 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247400bb20 con 0x7f2474004d10 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247400bcb0 con 0x7f2494141660 2024-01-15T18:07:55.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2491e00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24740039b0 0x7f2474003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2491400640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.495+0000 7f2492800640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2474004d10 0x7f2474005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491e00640 1 -- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 msgr2=0x7f24740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491e00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491e00640 1 -- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2474004d10 msgr2=0x7f2474005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491e00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2474004d10 0x7f2474005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491e00640 1 -- 172.21.15.143:0/2419738359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474003530 con 0x7f24740039b0 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491400640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2494141660 0x7f24740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:55.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2492800640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2474004d10 0x7f2474005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2491e00640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24740039b0 0x7f2474003dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f248400ba60 tx=0x7f248401c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e29b908d1c0e142 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840ac600 con 0x7f24740039b0 2024-01-15T18:07:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24840ac760 con 0x7f24740039b0 2024-01-15T18:07:55.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2490a00640 1 -- 172.21.15.143:0/2419738359 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24840acb10 con 0x7f24740039b0 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2499000640 1 -- 172.21.15.143:0/2419738359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24740039b0 msgr2=0x7f2474003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2499000640 1 --2- 172.21.15.143:0/2419738359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24740039b0 0x7f2474003dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f248400ba60 tx=0x7f248401c660 comp rx=0 tx=0).stop 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2492800640 1 -- 172.21.15.143:0/2419738359 reap_dead start 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2499000640 1 -- 172.21.15.143:0/2419738359 shutdown_connections 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.499+0000 7f2499000640 1 -- 172.21.15.143:0/2419738359 >> 172.21.15.143:0/2419738359 conn(0x7f249406c3f0 msgr2=0x7f249406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 -- 172.21.15.143:0/2419738359 shutdown_connections 2024-01-15T18:07:55.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 -- 172.21.15.143:0/2419738359 wait complete. 2024-01-15T18:07:55.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 Processor -- start 2024-01-15T18:07:55.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 -- start start 2024-01-15T18:07:55.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24740039b0 0x7f24940a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 0x7f249413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f249415bda0 0x7f249415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24940a9100 con 0x7f2494140690 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249413d5f0 con 0x7f249415bda0 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2499000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24940a8b40 con 0x7f24740039b0 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 0x7f249413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 0x7f249413d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51028/0 (socket says 172.21.15.143:51028) 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 -- 172.21.15.143:0/3620467576 learned_addr learned my addr 172.21.15.143:0/3620467576 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2492800640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f249415bda0 0x7f249415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491e00640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24740039b0 0x7f24940a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 -- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24740039b0 msgr2=0x7f24940a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24740039b0 0x7f24940a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 -- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f249415bda0 msgr2=0x7f249415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f249415bda0 0x7f249415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 -- 172.21.15.143:0/3620467576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24940cd150 con 0x7f2494140690 2024-01-15T18:07:55.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.503+0000 7f2491400640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 0x7f249413d0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e8c0 tx=0x7f247c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00c8d0 con 0x7f2494140690 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c00ca30 con 0x7f2494140690 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24940a8e20 con 0x7f2494140690 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c00cda0 con 0x7f2494140690 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249415b880 con 0x7f2494140690 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2494067500 con 0x7f2494140690 2024-01-15T18:07:55.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f247c01d460 con 0x7f2494140690 2024-01-15T18:07:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f248a000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f246c066bf0 0x7f246c0690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f247c084730 con 0x7f2494140690 2024-01-15T18:07:55.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.507+0000 7f2491e00640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f246c066bf0 0x7f246c0690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.511+0000 7f2491e00640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f246c066bf0 0x7f246c0690b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24840b0000 tx=0x7f24840b7040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.511+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f247c051700 con 0x7f2494140690 2024-01-15T18:07:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.691+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87832", "target": ["mon-mgr", ""]}) v1 -- 0x7f24940ccfd0 con 0x7f246c066bf0 2024-01-15T18:07:55.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.747+0000 7f248a000640 1 -- 172.21.15.143:0/3620467576 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f24840ac5c0 con 0x7f246c066bf0 2024-01-15T18:07:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.747+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f246c066bf0 msgr2=0x7f246c0690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.747+0000 7f2499000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f246c066bf0 0x7f246c0690b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f24840b0000 tx=0x7f24840b7040 comp rx=0 tx=0).stop 2024-01-15T18:07:55.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.747+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 msgr2=0x7f249413d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.747+0000 7f2499000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 0x7f249413d0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e8c0 tx=0x7f247c00ed80 comp rx=0 tx=0).stop 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 shutdown_connections 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f246c066bf0 0x7f246c0690b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f249415bda0 0x7f249415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2494140690 0x7f249413d0b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 --2- 172.21.15.143:0/3620467576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24740039b0 0x7f24940a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 >> 172.21.15.143:0/3620467576 conn(0x7f249406c3f0 msgr2=0x7f24940cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 shutdown_connections 2024-01-15T18:07:55.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.751+0000 7f2499000640 1 -- 172.21.15.143:0/3620467576 wait complete. 2024-01-15T18:07:55.777 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87833 2024-01-15T18:07:55.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.911+0000 7ff9a7800640 1 Processor -- start 2024-01-15T18:07:55.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.911+0000 7ff9a7800640 1 -- start start 2024-01-15T18:07:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a7800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff9a00711f0 0x7ff9a00ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a7800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a00b3370 con 0x7ff9a0070d60 2024-01-15T18:07:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a7800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a015d5d0 con 0x7ff9a015d1f0 2024-01-15T18:07:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a7800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9a006a800 con 0x7ff9a00711f0 2024-01-15T18:07:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff9a0070d60 0x7ff9a00acbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45190/0 (socket says 172.21.15.143:45190) 2024-01-15T18:07:55.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99fe00640 1 -- 172.21.15.143:0/2944723064 learned_addr learned my addr 172.21.15.143:0/2944723064 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:55.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 --2- 172.21.15.143:0/2944723064 >> v2:172.21.15.204:3300/0 conn(0x7ff9a00711f0 0x7ff9a00ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145747262 0 0) 0x7ff9a00b3370 con 0x7ff9a0070d60 2024-01-15T18:07:55.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff980003530 con 0x7ff9a0070d60 2024-01-15T18:07:55.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 45786818 0 0) 0x7ff980003530 con 0x7ff9a0070d60 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 >> v2:172.21.15.204:3300/0 conn(0x7ff9a00711f0 msgr2=0x7ff9a00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 --2- 172.21.15.143:0/2944723064 >> v2:172.21.15.204:3300/0 conn(0x7ff9a00711f0 0x7ff9a00ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 >> v1:172.21.15.204:6789/0 conn(0x7ff9a015d1f0 legacy=0x7ff9a006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a006aae0 con 0x7ff9a0070d60 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff988001680 con 0x7ff9a0070d60 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 >> v1:172.21.15.143:6789/0 conn(0x7ff9a0070d60 legacy=0x7ff9a00acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980003980 0x7ff980003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9800047d0 0x7ff980004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff980005190 0x7ff98000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a006a800 con 0x7ff980005190 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98000c030 con 0x7ff9800047d0 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98000c1f0 con 0x7ff980003980 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff980005190 0x7ff98000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff99fe00640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9800047d0 0x7ff980004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a4e00640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980003980 0x7ff980003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 -- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980003980 msgr2=0x7ff980003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980003980 0x7ff980003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 -- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9800047d0 msgr2=0x7ff980004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9800047d0 0x7ff980004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.915+0000 7ff9a5800640 1 -- 172.21.15.143:0/2944723064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980003530 con 0x7ff980005190 2024-01-15T18:07:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a5800640 1 -- 172.21.15.143:0/2944723064 reap_dead start 2024-01-15T18:07:55.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a5800640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff980005190 0x7ff98000b990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff998009fc0 tx=0x7ff99801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8370c5cbb3d2a1b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980a5cb0 con 0x7ff980005190 2024-01-15T18:07:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99800f040 con 0x7ff980005190 2024-01-15T18:07:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff99f400640 1 -- 172.21.15.143:0/2944723064 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998003be0 con 0x7ff980005190 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 -- 172.21.15.143:0/2944723064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff980005190 msgr2=0x7ff98000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff980005190 0x7ff98000b990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ff998009fc0 tx=0x7ff99801cca0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 -- 172.21.15.143:0/2944723064 shutdown_connections 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 --2- 172.21.15.143:0/2944723064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff980005190 0x7ff98000b990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 -- 172.21.15.143:0/2944723064 >> 172.21.15.143:0/2944723064 conn(0x7ff9a00b1610 msgr2=0x7ff9a00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 -- 172.21.15.143:0/2944723064 shutdown_connections 2024-01-15T18:07:55.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.919+0000 7ff9a7800640 1 -- 172.21.15.143:0/2944723064 wait complete. 2024-01-15T18:07:55.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 Processor -- start 2024-01-15T18:07:55.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 -- start start 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980005190 0x7ff9a0147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a00ad400 0x7ff9a00ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 0x7ff9a0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a0154590 con 0x7ff9a0153bd0 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015e210 con 0x7ff9a00ad400 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a015f1d0 con 0x7ff980005190 2024-01-15T18:07:55.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 0x7ff9a0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 0x7ff9a0154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51048/0 (socket says 172.21.15.143:51048) 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 -- 172.21.15.143:0/80863086 learned_addr learned my addr 172.21.15.143:0/80863086 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff99fe00640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a00ad400 0x7ff9a00ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a4e00640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980005190 0x7ff9a0147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 -- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980005190 msgr2=0x7ff9a0147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980005190 0x7ff9a0147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 -- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a00ad400 msgr2=0x7ff9a00ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a00ad400 0x7ff9a00ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 -- 172.21.15.143:0/80863086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a00adf50 con 0x7ff9a0153bd0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a5800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 0x7ff9a0154050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9980987d0 tx=0x7ff998009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9980aed40 con 0x7ff9a0153bd0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9980aeec0 con 0x7ff9a0153bd0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a00ae230 con 0x7ff9a0153bd0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff998003be0 con 0x7ff9a0153bd0 2024-01-15T18:07:55.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.923+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a00ae7f0 con 0x7ff9a0153bd0 2024-01-15T18:07:55.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.927+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a0074b00 con 0x7ff9a0153bd0 2024-01-15T18:07:55.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.927+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 10) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff998003d40 con 0x7ff9a0153bd0 2024-01-15T18:07:55.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.927+0000 7ff99cc00640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff978066bf0 0x7ff9780690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.927+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff9980a6e40 con 0x7ff9a0153bd0 2024-01-15T18:07:55.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.927+0000 7ff9a4e00640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff978066bf0 0x7ff9780690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.927+0000 7ff9a4e00640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff978066bf0 0x7ff9780690b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c10 tx=0x7ff990009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:55.931+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9980ecb20 con 0x7ff9a0153bd0 2024-01-15T18:07:56.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:56.111+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87833", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9a006b380 con 0x7ff978066bf0 2024-01-15T18:07:57.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:57.567+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 11) v1 ==== 86517+0+0 (secure 0 0 0) 0x7ff9980ecef0 con 0x7ff9a0153bd0 2024-01-15T18:07:57.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:57.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:58.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.299+0000 7ff99cc00640 1 -- 172.21.15.143:0/80863086 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff990001d40 con 0x7ff978066bf0 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff978066bf0 msgr2=0x7ff9780690b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff978066bf0 0x7ff9780690b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff990000c10 tx=0x7ff990009040 comp rx=0 tx=0).stop 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 msgr2=0x7ff9a0154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 0x7ff9a0154050 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9980987d0 tx=0x7ff998009750 comp rx=0 tx=0).stop 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 shutdown_connections 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff978066bf0 0x7ff9780690b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9a0153bd0 0x7ff9a0154050 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9a00ad400 0x7ff9a00ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 --2- 172.21.15.143:0/80863086 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff980005190 0x7ff9a0147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 >> 172.21.15.143:0/80863086 conn(0x7ff9a00b1610 msgr2=0x7ff9a01e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 shutdown_connections 2024-01-15T18:07:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.303+0000 7ff9a7800640 1 -- 172.21.15.143:0/80863086 wait complete. 2024-01-15T18:07:58.331 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone87834 2024-01-15T18:07:58.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.467+0000 7fbf68a00640 1 Processor -- start 2024-01-15T18:07:58.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.467+0000 7fbf68a00640 1 -- start start 2024-01-15T18:07:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf68a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 0x7fbf600b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf68a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600b2a20 con 0x7fbf60157300 2024-01-15T18:07:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf68a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf600b2d00 con 0x7fbf600b48f0 2024-01-15T18:07:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf68a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600b35a0 con 0x7fbf60159b70 2024-01-15T18:07:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbf60157300 0x7fbf600b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54588/0 (socket says 172.21.15.143:54588) 2024-01-15T18:07:58.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66000640 1 -- 172.21.15.143:0/4194540124 learned_addr learned my addr 172.21.15.143:0/4194540124 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:07:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf65600640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 0x7fbf600b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190942855 0 0) 0x7fbf600b2a20 con 0x7fbf60157300 2024-01-15T18:07:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf40003450 con 0x7fbf60157300 2024-01-15T18:07:58.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628914005 0 0) 0x7fbf600b2d00 con 0x7fbf600b48f0 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf600b2a20 con 0x7fbf600b48f0 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3132405816 0 0) 0x7fbf40003450 con 0x7fbf60157300 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 msgr2=0x7fbf600b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 0x7fbf600b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 >> v1:172.21.15.204:6789/0 conn(0x7fbf600b48f0 legacy=0x7fbf60070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600b2fe0 con 0x7fbf60157300 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fbf50001680 con 0x7fbf60157300 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 >> v1:172.21.15.143:6789/0 conn(0x7fbf60157300 legacy=0x7fbf600b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 0x7fbf40003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf40008860 0x7fbf4000ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf4000b190 0x7fbf4000d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600b35a0 con 0x7fbf4000b190 2024-01-15T18:07:58.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000dcb0 con 0x7fbf40008860 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4000de70 con 0x7fbf60159b70 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66a00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf4000b190 0x7fbf4000d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66000640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 0x7fbf40003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf65600640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf40008860 0x7fbf4000ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66a00640 1 -- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 msgr2=0x7fbf40003c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66a00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf60159b70 0x7fbf40003c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66a00640 1 -- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf40008860 msgr2=0x7fbf4000ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66a00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf40008860 0x7fbf4000ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.471+0000 7fbf66a00640 1 -- 172.21.15.143:0/4194540124 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40003450 con 0x7fbf4000b190 2024-01-15T18:07:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf66a00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf4000b190 0x7fbf4000d580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbf58001fd0 tx=0x7fbf5801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac28b71899822e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58003b80 con 0x7fbf4000b190 2024-01-15T18:07:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58003d00 con 0x7fbf4000b190 2024-01-15T18:07:58.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf64c00640 1 -- 172.21.15.143:0/4194540124 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58005310 con 0x7fbf4000b190 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf68a00640 1 -- 172.21.15.143:0/4194540124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf4000b190 msgr2=0x7fbf4000d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf68a00640 1 --2- 172.21.15.143:0/4194540124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf4000b190 0x7fbf4000d580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fbf58001fd0 tx=0x7fbf5801c8e0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf66a00640 1 -- 172.21.15.143:0/4194540124 reap_dead start 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf68a00640 1 -- 172.21.15.143:0/4194540124 shutdown_connections 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf68a00640 1 -- 172.21.15.143:0/4194540124 >> 172.21.15.143:0/4194540124 conn(0x7fbf6006c3f0 msgr2=0x7fbf6006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf68a00640 1 -- 172.21.15.143:0/4194540124 shutdown_connections 2024-01-15T18:07:58.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.475+0000 7fbf68a00640 1 -- 172.21.15.143:0/4194540124 wait complete. 2024-01-15T18:07:58.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 Processor -- start 2024-01-15T18:07:58.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 -- start start 2024-01-15T18:07:58.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf4000b190 0x7fbf601f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 0x7fbf6014b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf6015ae00 0x7fbf6014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf60159fd0 con 0x7fbf6015a980 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf60158220 con 0x7fbf6015ae00 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf60157880 con 0x7fbf4000b190 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 0x7fbf6014b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf66a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf6015ae00 0x7fbf6014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf66000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf4000b190 0x7fbf601f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 0x7fbf6014b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42198/0 (socket says 172.21.15.143:42198) 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 -- 172.21.15.143:0/3410630661 learned_addr learned my addr 172.21.15.143:0/3410630661 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 -- 172.21.15.143:0/3410630661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf4000b190 msgr2=0x7fbf601f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf4000b190 0x7fbf601f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 -- 172.21.15.143:0/3410630661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf6015ae00 msgr2=0x7fbf6014ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf6015ae00 0x7fbf6014ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 -- 172.21.15.143:0/3410630661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60157b60 con 0x7fbf6015a980 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf65600640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 0x7fbf6014b510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbf480119d0 tx=0x7fbf48011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4800fad0 con 0x7fbf6015a980 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4800fc30 con 0x7fbf6015a980 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48014440 con 0x7fbf6015a980 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf60157e40 con 0x7fbf6015a980 2024-01-15T18:07:58.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.479+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf601f2aa0 con 0x7fbf6015a980 2024-01-15T18:07:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.483+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf60067500 con 0x7fbf6015a980 2024-01-15T18:07:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.483+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86517+0+0 (secure 0 0 0) 0x7fbf480145a0 con 0x7fbf6015a980 2024-01-15T18:07:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.483+0000 7fbf5e000640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbf38066d10 0x7fbf380691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.483+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbf48085230 con 0x7fbf6015a980 2024-01-15T18:07:58.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.483+0000 7fbf66000640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbf38066d10 0x7fbf380691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:58.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.483+0000 7fbf66000640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbf38066d10 0x7fbf380691d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbf500011f0 tx=0x7fbf50000ee0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:58.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.487+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbf48052030 con 0x7fbf6015a980 2024-01-15T18:07:58.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:07:58.667+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone87834", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf600725c0 con 0x7fbf38066d10 2024-01-15T18:08:03.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.295+0000 7fbf5e000640 1 -- 172.21.15.143:0/3410630661 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf600725c0 con 0x7fbf38066d10 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbf38066d10 msgr2=0x7fbf380691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbf38066d10 0x7fbf380691d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbf500011f0 tx=0x7fbf50000ee0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 msgr2=0x7fbf6014b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 0x7fbf6014b510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fbf480119d0 tx=0x7fbf48011e90 comp rx=0 tx=0).stop 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 shutdown_connections 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbf38066d10 0x7fbf380691d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbf6015ae00 0x7fbf6014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbf6015a980 0x7fbf6014b510 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 --2- 172.21.15.143:0/3410630661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbf4000b190 0x7fbf601f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 >> 172.21.15.143:0/3410630661 conn(0x7fbf6006c3f0 msgr2=0x7fbf6014fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:03.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 shutdown_connections 2024-01-15T18:08:03.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.299+0000 7fbf68a00640 1 -- 172.21.15.143:0/3410630661 wait complete. 2024-01-15T18:08:03.326 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:08:03.327 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:03.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 Processor -- start 2024-01-15T18:08:03.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 -- start start 2024-01-15T18:08:03.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f02140ce690 0x7f02140ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f02140b1da0 0x7f02140b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0214070bf0 con 0x7f02140b3bf0 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021415dba0 con 0x7f02140ce690 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0218e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02140b2770 con 0x7f02140b1da0 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f02140ce690 0x7f02140ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f02140ce690 0x7f02140ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42206/0 (socket says 172.21.15.143:42206) 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 -- 172.21.15.143:0/2968256725 learned_addr learned my addr 172.21.15.143:0/2968256725 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211400640 1 --2- 172.21.15.143:0/2968256725 >> v2:172.21.15.204:3300/0 conn(0x7f02140b1da0 0x7f02140b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 -- 172.21.15.143:0/2968256725 >> v2:172.21.15.204:3300/0 conn(0x7f02140b1da0 msgr2=0x7f02140b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 --2- 172.21.15.143:0/2968256725 >> v2:172.21.15.204:3300/0 conn(0x7f02140b1da0 0x7f02140b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 -- 172.21.15.143:0/2968256725 >> v1:172.21.15.204:6789/0 conn(0x7f02140b3bf0 legacy=0x7f02140b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.463+0000 7f0211e00640 1 -- 172.21.15.143:0/2968256725 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140b4920 con 0x7f02140ce690 2024-01-15T18:08:03.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0211e00640 1 --2- 172.21.15.143:0/2968256725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140ce690 0x7f02140ac8e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f021406f940 tx=0x7f020401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2074ddae2f38c2e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0210a00640 1 -- 172.21.15.143:0/2968256725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040ab5f0 con 0x7f02140ce690 2024-01-15T18:08:03.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0210a00640 1 -- 172.21.15.143:0/2968256725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02040ab750 con 0x7f02140ce690 2024-01-15T18:08:03.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0210a00640 1 -- 172.21.15.143:0/2968256725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040abae0 con 0x7f02140ce690 2024-01-15T18:08:03.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 -- 172.21.15.143:0/2968256725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140ce690 msgr2=0x7f02140ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 --2- 172.21.15.143:0/2968256725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140ce690 0x7f02140ac8e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f021406f940 tx=0x7f020401c500 comp rx=0 tx=0).stop 2024-01-15T18:08:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 -- 172.21.15.143:0/2968256725 shutdown_connections 2024-01-15T18:08:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 --2- 172.21.15.143:0/2968256725 >> v2:172.21.15.204:3300/0 conn(0x7f02140b1da0 0x7f02140b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 --2- 172.21.15.143:0/2968256725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140ce690 0x7f02140ac8e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 -- 172.21.15.143:0/2968256725 >> 172.21.15.143:0/2968256725 conn(0x7f021406c3f0 msgr2=0x7f021406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 -- 172.21.15.143:0/2968256725 shutdown_connections 2024-01-15T18:08:03.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.467+0000 7f0218e00640 1 -- 172.21.15.143:0/2968256725 wait complete. 2024-01-15T18:08:03.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 Processor -- start 2024-01-15T18:08:03.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 -- start start 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02140b1da0 0x7f02140ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 0x7f0214159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02140ce690 0x7f021414afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021415b830 con 0x7f02140b3bf0 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214153a80 con 0x7f02140ce690 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02141534c0 con 0x7f02140b1da0 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 0x7f0214159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 0x7f0214159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42212/0 (socket says 172.21.15.143:42212) 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 -- 172.21.15.143:0/2064835562 learned_addr learned my addr 172.21.15.143:0/2064835562 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:03.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0212800640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02140ce690 0x7f021414afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02140b1da0 0x7f02140ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 -- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02140b1da0 msgr2=0x7f02140ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02140b1da0 0x7f02140ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 -- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02140ce690 msgr2=0x7f021414afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02140ce690 0x7f021414afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 -- 172.21.15.143:0/2064835562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02040094d0 con 0x7f02140b3bf0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0211400640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 0x7f0214159860 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d800 tx=0x7f01fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00b6c0 con 0x7f02140b3bf0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00b820 con 0x7f02140b3bf0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00bb90 con 0x7f02140b3bf0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02141537a0 con 0x7f02140b3bf0 2024-01-15T18:08:03.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.471+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02140aedd0 con 0x7f02140b3bf0 2024-01-15T18:08:03.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.475+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0214067500 con 0x7f02140b3bf0 2024-01-15T18:08:03.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.475+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86517+0+0 (secure 0 0 0) 0x7f01fc00bcf0 con 0x7f02140b3bf0 2024-01-15T18:08:03.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.475+0000 7f020a000640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f01ec066ce0 0x7f01ec0691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.475+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f01fc084f60 con 0x7f02140b3bf0 2024-01-15T18:08:03.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.475+0000 7f0211e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f01ec066ce0 0x7f01ec0691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.475+0000 7f0211e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f01ec066ce0 0x7f01ec0691a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f02040af000 tx=0x7f02040b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.479+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f01fc051c60 con 0x7f02140b3bf0 2024-01-15T18:08:03.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:03.659+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f021406e260 con 0x7f01ec066ce0 2024-01-15T18:08:08.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.239+0000 7f020a000640 1 -- 172.21.15.143:0/2064835562 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f021406e260 con 0x7f01ec066ce0 2024-01-15T18:08:08.244 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:08.244 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:08.244 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:08:08.244 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:08.245 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:08.245 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:08.245 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:08.245 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:08.245 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:08.245 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f01ec066ce0 msgr2=0x7f01ec0691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f01ec066ce0 0x7f01ec0691a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f02040af000 tx=0x7f02040b6040 comp rx=0 tx=0).stop 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 msgr2=0x7f0214159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 0x7f0214159860 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f01fc00d800 tx=0x7f01fc00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 shutdown_connections 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f01ec066ce0 0x7f01ec0691a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f02140ce690 0x7f021414afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02140b3bf0 0x7f0214159860 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 --2- 172.21.15.143:0/2064835562 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f02140b1da0 0x7f02140ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 >> 172.21.15.143:0/2064835562 conn(0x7f021406c3f0 msgr2=0x7f0214157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:08.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 shutdown_connections 2024-01-15T18:08:08.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:08.243+0000 7f0218e00640 1 -- 172.21.15.143:0/2064835562 wait complete. 2024-01-15T18:08:08.275 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:08:09.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:09.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 Processor -- start 2024-01-15T18:08:09.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 -- start start 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc980ce560 0x7fbc9815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbc98071d00 0x7fbc98072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc98072650 con 0x7fbc98141660 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9815fbb0 con 0x7fbc980ce560 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbca0800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9815f310 con 0x7fbc98071d00 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc980ce560 0x7fbc9815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc980ce560 0x7fbc9815ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43452/0 (socket says 172.21.15.143:43452) 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 -- 172.21.15.143:0/3318034925 learned_addr learned my addr 172.21.15.143:0/3318034925 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9e800640 1 --2- 172.21.15.143:0/3318034925 >> v2:172.21.15.204:3300/0 conn(0x7fbc98071d00 0x7fbc98072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 -- 172.21.15.143:0/3318034925 >> v2:172.21.15.204:3300/0 conn(0x7fbc98071d00 msgr2=0x7fbc98072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 --2- 172.21.15.143:0/3318034925 >> v2:172.21.15.204:3300/0 conn(0x7fbc98071d00 0x7fbc98072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 -- 172.21.15.143:0/3318034925 >> v1:172.21.15.204:6789/0 conn(0x7fbc98141660 legacy=0x7fbc98141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9d400640 1 -- 172.21.15.143:0/3318034925 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9815f5f0 con 0x7fbc980ce560 2024-01-15T18:08:09.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.423+0000 7fbc9e800640 1 --2- 172.21.15.143:0/3318034925 >> v2:172.21.15.204:3300/0 conn(0x7fbc98071d00 0x7fbc98072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:09.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbc9d400640 1 --2- 172.21.15.143:0/3318034925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc980ce560 0x7fbc9815ec80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbc9806e250 tx=0x7fbc8001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e91e77edfa5457f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbc9ca00640 1 -- 172.21.15.143:0/3318034925 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800ab5f0 con 0x7fbc980ce560 2024-01-15T18:08:09.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbc9ca00640 1 -- 172.21.15.143:0/3318034925 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc800ab750 con 0x7fbc980ce560 2024-01-15T18:08:09.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbc9ca00640 1 -- 172.21.15.143:0/3318034925 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc800abae0 con 0x7fbc980ce560 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 -- 172.21.15.143:0/3318034925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc980ce560 msgr2=0x7fbc9815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 --2- 172.21.15.143:0/3318034925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc980ce560 0x7fbc9815ec80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbc9806e250 tx=0x7fbc8001c500 comp rx=0 tx=0).stop 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 -- 172.21.15.143:0/3318034925 shutdown_connections 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 --2- 172.21.15.143:0/3318034925 >> v2:172.21.15.204:3300/0 conn(0x7fbc98071d00 0x7fbc98072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 --2- 172.21.15.143:0/3318034925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc980ce560 0x7fbc9815ec80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 -- 172.21.15.143:0/3318034925 >> 172.21.15.143:0/3318034925 conn(0x7fbc9806c3f0 msgr2=0x7fbc9806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:09.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 -- 172.21.15.143:0/3318034925 shutdown_connections 2024-01-15T18:08:09.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.427+0000 7fbca0800640 1 -- 172.21.15.143:0/3318034925 wait complete. 2024-01-15T18:08:09.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 Processor -- start 2024-01-15T18:08:09.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 -- start start 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc98071d00 0x7fbc98137530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc980ce560 0x7fbc980ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 0x7fbc980ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980b2260 con 0x7fbc98141660 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc98140410 con 0x7fbc980ce560 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc981406f0 con 0x7fbc98071d00 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 0x7fbc980ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 0x7fbc980ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43454/0 (socket says 172.21.15.143:43454) 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 -- 172.21.15.143:0/1007841471 learned_addr learned my addr 172.21.15.143:0/1007841471 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9d400640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc980ce560 0x7fbc980ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9de00640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc98071d00 0x7fbc98137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 -- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc98071d00 msgr2=0x7fbc98137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc98071d00 0x7fbc98137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 -- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc980ce560 msgr2=0x7fbc980ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc980ce560 0x7fbc980ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 -- 172.21.15.143:0/1007841471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc800094d0 con 0x7fbc98141660 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9d400640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc980ce560 0x7fbc980ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:09.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc9e800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 0x7fbc980ace20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbc90007de0 tx=0x7fbc9000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900039a0 con 0x7fbc98141660 2024-01-15T18:08:09.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9000f040 con 0x7fbc98141660 2024-01-15T18:08:09.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90003d40 con 0x7fbc98141660 2024-01-15T18:08:09.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc980cd150 con 0x7fbc98141660 2024-01-15T18:08:09.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.431+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc981385a0 con 0x7fbc98141660 2024-01-15T18:08:09.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.435+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc981387c0 con 0x7fbc98141660 2024-01-15T18:08:09.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.435+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86517+0+0 (secure 0 0 0) 0x7fbc90004030 con 0x7fbc98141660 2024-01-15T18:08:09.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.435+0000 7fbc96000640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc70066ce0 0x7fbc700691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.435+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbc90085180 con 0x7fbc98141660 2024-01-15T18:08:09.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.435+0000 7fbc9de00640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc70066ce0 0x7fbc700691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.435+0000 7fbc9de00640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc70066ce0 0x7fbc700691a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc88000f70 tx=0x7fbc88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.439+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc90051ba0 con 0x7fbc98141660 2024-01-15T18:08:09.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:09.627+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc98140860 con 0x7fbc70066ce0 2024-01-15T18:08:13.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.235+0000 7fbc96000640 1 -- 172.21.15.143:0/1007841471 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fbc98140860 con 0x7fbc70066ce0 2024-01-15T18:08:13.244 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.245 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc70066ce0 msgr2=0x7fbc700691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc70066ce0 0x7fbc700691a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc88000f70 tx=0x7fbc88009040 comp rx=0 tx=0).stop 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 msgr2=0x7fbc980ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 0x7fbc980ace20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbc90007de0 tx=0x7fbc9000d4a0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 shutdown_connections 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc70066ce0 0x7fbc700691a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc98141660 0x7fbc980ace20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc980ce560 0x7fbc980ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 --2- 172.21.15.143:0/1007841471 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc98071d00 0x7fbc98137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 >> 172.21.15.143:0/1007841471 conn(0x7fbc9806c3f0 msgr2=0x7fbc9806d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:13.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 shutdown_connections 2024-01-15T18:08:13.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.239+0000 7fbca0800640 1 -- 172.21.15.143:0/1007841471 wait complete. 2024-01-15T18:08:13.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:13.247+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:13.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:13.247+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:13.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:13.247+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:13.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:13.247+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:13.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:13.247+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:13.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:08:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f4245800640 1 Processor -- start 2024-01-15T18:08:13.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f4245800640 1 -- start start 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f4245800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f42400cf910 0x7f42400cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f4245800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42400cbff0 con 0x7f424006bfb0 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f4245800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42400d0c50 con 0x7f42400b0310 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f4245800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42400cb6c0 con 0x7f42400cf910 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423ea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f424006bfb0 0x7f424006c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49794/0 (socket says 172.21.15.143:49794) 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423ea00640 1 -- 172.21.15.143:0/3591003437 learned_addr learned my addr 172.21.15.143:0/3591003437 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423f400640 1 --2- 172.21.15.143:0/3591003437 >> v2:172.21.15.204:3300/0 conn(0x7f42400cf910 0x7f42400cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987016581 0 0) 0x7f42400cbff0 con 0x7f424006bfb0 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4220003530 con 0x7f424006bfb0 2024-01-15T18:08:13.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839012582 0 0) 0x7f42400d0c50 con 0x7f42400b0310 2024-01-15T18:08:13.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42400cbff0 con 0x7f42400b0310 2024-01-15T18:08:13.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423f400640 1 -- 172.21.15.143:0/3591003437 >> v1:172.21.15.204:6789/0 conn(0x7f42400b0310 legacy=0x7f42400cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423f400640 1 -- 172.21.15.143:0/3591003437 >> v1:172.21.15.143:6789/0 conn(0x7f424006bfb0 legacy=0x7f424006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.419+0000 7f423f400640 1 -- 172.21.15.143:0/3591003437 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400cb9a0 con 0x7f42400cf910 2024-01-15T18:08:13.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f423f400640 1 --2- 172.21.15.143:0/3591003437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400cf910 0x7f42400cfdc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4238009d90 tx=0x7f423801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2c8af4b7fb9a69d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380a5880 con 0x7f42400cf910 2024-01-15T18:08:13.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380a59e0 con 0x7f42400cf910 2024-01-15T18:08:13.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f423d600640 1 -- 172.21.15.143:0/3591003437 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380a5d70 con 0x7f42400cf910 2024-01-15T18:08:13.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 -- 172.21.15.143:0/3591003437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400cf910 msgr2=0x7f42400cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 --2- 172.21.15.143:0/3591003437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400cf910 0x7f42400cfdc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4238009d90 tx=0x7f423801c890 comp rx=0 tx=0).stop 2024-01-15T18:08:13.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 -- 172.21.15.143:0/3591003437 shutdown_connections 2024-01-15T18:08:13.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 --2- 172.21.15.143:0/3591003437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400cf910 0x7f42400cfdc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 -- 172.21.15.143:0/3591003437 >> 172.21.15.143:0/3591003437 conn(0x7f42400d0ed0 msgr2=0x7f42400d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:13.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 -- 172.21.15.143:0/3591003437 shutdown_connections 2024-01-15T18:08:13.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.423+0000 7f4245800640 1 -- 172.21.15.143:0/3591003437 wait complete. 2024-01-15T18:08:13.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 Processor -- start 2024-01-15T18:08:13.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 -- start start 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f424006bfb0 0x7f424014e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400b0310 0x7f42400aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 0x7f42400c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400af5a0 con 0x7f42400cf910 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400af880 con 0x7f42400b0310 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42400afb60 con 0x7f424006bfb0 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 0x7f42400c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 0x7f42400c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43482/0 (socket says 172.21.15.143:43482) 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 -- 172.21.15.143:0/1993233223 learned_addr learned my addr 172.21.15.143:0/1993233223 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423ea00640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f424006bfb0 0x7f424014e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423e000640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400b0310 0x7f42400aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 -- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f424006bfb0 msgr2=0x7f424014e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f424006bfb0 0x7f424014e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 -- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400b0310 msgr2=0x7f42400aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400b0310 0x7f42400aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 -- 172.21.15.143:0/1993233223 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238009a70 con 0x7f42400cf910 2024-01-15T18:08:13.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f423f400640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 0x7f42400c72e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f423801a4c0 tx=0x7f4238019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380ad640 con 0x7f42400cf910 2024-01-15T18:08:13.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380ad7a0 con 0x7f42400cf910 2024-01-15T18:08:13.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42400c6ae0 con 0x7f42400cf910 2024-01-15T18:08:13.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380adb10 con 0x7f42400cf910 2024-01-15T18:08:13.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.427+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42400af2b0 con 0x7f42400cf910 2024-01-15T18:08:13.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.431+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42400c6940 con 0x7f42400cf910 2024-01-15T18:08:13.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.431+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86517+0+0 (secure 0 0 0) 0x7f42380b4050 con 0x7f42400cf910 2024-01-15T18:08:13.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.431+0000 7f4236a00640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4218066ce0 0x7f42180691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.431+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f423811fc70 con 0x7f42400cf910 2024-01-15T18:08:13.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.431+0000 7f423ea00640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4218066ce0 0x7f42180691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.431+0000 7f423ea00640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4218066ce0 0x7f42180691a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4230000f70 tx=0x7f4230009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.435+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f42380ec9d0 con 0x7f42400cf910 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.635+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f42400c6940 con 0x7f42400cf910 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.635+0000 7f4236a00640 1 -- 172.21.15.143:0/1993233223 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2305 (secure 0 0 0) 0x7f42400c6940 con 0x7f42400cf910 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:08:13.642 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 219, 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:08:13.643 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 48, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:13.644 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 800, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3222224108, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 6704480256, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 766389633024, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 767, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 15185, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 0, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 5 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.645 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 27, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 3, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 3, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "a", 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 5736 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:08:13.646 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 6, 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:08:10.663288+0000", 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:08:13.647 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "a": { 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:13.648 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": {} 2024-01-15T18:08:13.649 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4218066ce0 msgr2=0x7f42180691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4218066ce0 0x7f42180691a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4230000f70 tx=0x7f4230009040 comp rx=0 tx=0).stop 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 msgr2=0x7f42400c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 0x7f42400c72e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f423801a4c0 tx=0x7f4238019630 comp rx=0 tx=0).stop 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 shutdown_connections 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4218066ce0 0x7f42180691a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f42400cf910 0x7f42400c72e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f42400b0310 0x7f42400aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 --2- 172.21.15.143:0/1993233223 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f424006bfb0 0x7f424014e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 >> 172.21.15.143:0/1993233223 conn(0x7f42400d0ed0 msgr2=0x7f424006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 shutdown_connections 2024-01-15T18:08:13.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.639+0000 7f4245800640 1 -- 172.21.15.143:0/1993233223 wait complete. 2024-01-15T18:08:13.664 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:08:13.664 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:13.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 Processor -- start 2024-01-15T18:08:13.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 -- start start 2024-01-15T18:08:13.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9df00b3f60 0x7f9df00b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df015b5d0 0x7f9df015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df006c2c0 con 0x7f9df006bee0 2024-01-15T18:08:13.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df00b3780 con 0x7f9df00b3f60 2024-01-15T18:08:13.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df015aed0 con 0x7f9df015b5d0 2024-01-15T18:08:13.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df5800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9df006bee0 0x7f9df015a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49808/0 (socket says 172.21.15.143:49808) 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df5800640 1 -- 172.21.15.143:0/174821911 learned_addr learned my addr 172.21.15.143:0/174821911 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df4e00640 1 --2- 172.21.15.143:0/174821911 >> v2:172.21.15.204:3300/0 conn(0x7f9df00b3f60 0x7f9df00b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.799+0000 7f9df6c00640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df015b5d0 0x7f9df015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9defe00640 1 -- 172.21.15.143:0/174821911 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756548368 0 0) 0x7f9de0001120 con 0x7f9df006bee0 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9defe00640 1 -- 172.21.15.143:0/174821911 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd0003530 con 0x7f9df006bee0 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9df4e00640 1 -- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df015b5d0 msgr2=0x7f9df015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9df4e00640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df015b5d0 0x7f9df015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9df4e00640 1 -- 172.21.15.143:0/174821911 >> v1:172.21.15.143:6789/0 conn(0x7f9df006bee0 legacy=0x7f9df015a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9df4e00640 1 -- 172.21.15.143:0/174821911 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df015b1b0 con 0x7f9df00b3f60 2024-01-15T18:08:13.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9df6c00640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df015b5d0 0x7f9df015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:13.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9df4e00640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df00b3f60 0x7f9df00b4410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80097c0 tx=0x7f9dd801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dff28611d6ca5a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9defe00640 1 -- 172.21.15.143:0/174821911 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80a6040 con 0x7f9df00b3f60 2024-01-15T18:08:13.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9defe00640 1 -- 172.21.15.143:0/174821911 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd80aa600 con 0x7f9df00b3f60 2024-01-15T18:08:13.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.803+0000 7f9defe00640 1 -- 172.21.15.143:0/174821911 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd80aa990 con 0x7f9df00b3f60 2024-01-15T18:08:13.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df00b3f60 msgr2=0x7f9df00b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df00b3f60 0x7f9df00b4410 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80097c0 tx=0x7f9dd801c480 comp rx=0 tx=0).stop 2024-01-15T18:08:13.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- 172.21.15.143:0/174821911 shutdown_connections 2024-01-15T18:08:13.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df015b5d0 0x7f9df015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 --2- 172.21.15.143:0/174821911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df00b3f60 0x7f9df00b4410 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- 172.21.15.143:0/174821911 >> 172.21.15.143:0/174821911 conn(0x7f9df0066720 msgr2=0x7f9df006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:13.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- 172.21.15.143:0/174821911 shutdown_connections 2024-01-15T18:08:13.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- 172.21.15.143:0/174821911 wait complete. 2024-01-15T18:08:13.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 Processor -- start 2024-01-15T18:08:13.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- start start 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 0x7f9df01411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df00b3f60 0x7f9df0071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df015b5d0 0x7f9df0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0141720 con 0x7f9df006bee0 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df0163640 con 0x7f9df015b5d0 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df01625a0 con 0x7f9df00b3f60 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 0x7f9df01411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 0x7f9df01411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43488/0 (socket says 172.21.15.143:43488) 2024-01-15T18:08:13.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df5800640 1 -- 172.21.15.143:0/3409041228 learned_addr learned my addr 172.21.15.143:0/3409041228 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df6c00640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df015b5d0 0x7f9df0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.807+0000 7f9df4e00640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df00b3f60 0x7f9df0071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df5800640 1 -- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df00b3f60 msgr2=0x7f9df0071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df5800640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df00b3f60 0x7f9df0071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df5800640 1 -- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df015b5d0 msgr2=0x7f9df0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df5800640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df015b5d0 0x7f9df0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df5800640 1 -- 172.21.15.143:0/3409041228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8009a70 con 0x7f9df006bee0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df5800640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 0x7f9df01411e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9de000c730 tx=0x7f9de000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0005ac0 con 0x7f9df006bee0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0005c20 con 0x7f9df006bee0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000ce70 con 0x7f9df006bee0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd8009420 con 0x7f9df006bee0 2024-01-15T18:08:13.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df0162b60 con 0x7f9df006bee0 2024-01-15T18:08:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df0067680 con 0x7f9df006bee0 2024-01-15T18:08:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 11) v1 ==== 86517+0+0 (secure 0 0 0) 0x7f9de0007070 con 0x7f9df006bee0 2024-01-15T18:08:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9ded600640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dc8066cd0 0x7f9dc8069190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9de0083f00 con 0x7f9df006bee0 2024-01-15T18:08:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.811+0000 7f9df4e00640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dc8066cd0 0x7f9dc8069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.815+0000 7f9df4e00640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dc8066cd0 0x7f9dc8069190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80ae000 tx=0x7f9dd80b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.815+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9de0050b70 con 0x7f9df006bee0 2024-01-15T18:08:14.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:13.999+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f9df006c2c0 con 0x7f9dc8066cd0 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9ded600640 1 -- 172.21.15.143:0/3409041228 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f9df006c2c0 con 0x7f9dc8066cd0 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:14.011 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:14.012 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:14.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dc8066cd0 msgr2=0x7f9dc8069190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dc8066cd0 0x7f9dc8069190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80ae000 tx=0x7f9dd80b5040 comp rx=0 tx=0).stop 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 msgr2=0x7f9df01411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 0x7f9df01411e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9de000c730 tx=0x7f9de000cb30 comp rx=0 tx=0).stop 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 shutdown_connections 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dc8066cd0 0x7f9dc8069190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9df015b5d0 0x7f9df0071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9df00b3f60 0x7f9df0071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 --2- 172.21.15.143:0/3409041228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9df006bee0 0x7f9df01411e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 >> 172.21.15.143:0/3409041228 conn(0x7f9df0066720 msgr2=0x7f9df0151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 shutdown_connections 2024-01-15T18:08:14.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:14.007+0000 7f9df8200640 1 -- 172.21.15.143:0/3409041228 wait complete. 2024-01-15T18:08:14.033 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:15.034 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:15.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 Processor -- start 2024-01-15T18:08:15.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 -- start start 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71c81607c0 0x7f71c806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f71c80cfb30 0x7f71c806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c806c230 0x7f71c80cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c806c670 con 0x7f71c81607c0 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80d1ce0 con 0x7f71c80cfb30 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71ce400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80cffd0 con 0x7f71c806c230 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71c81607c0 0x7f71c806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f71c81607c0 0x7f71c806b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43502/0 (socket says 172.21.15.143:43502) 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 -- 172.21.15.143:0/350838047 learned_addr learned my addr 172.21.15.143:0/350838047 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7e00640 1 --2- 172.21.15.143:0/350838047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c806c230 0x7f71c80cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c6a00640 1 --2- 172.21.15.143:0/350838047 >> v2:172.21.15.204:3300/0 conn(0x7f71c80cfb30 0x7f71c806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 -- 172.21.15.143:0/350838047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c806c230 msgr2=0x7f71c80cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 --2- 172.21.15.143:0/350838047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c806c230 0x7f71c80cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 -- 172.21.15.143:0/350838047 >> v2:172.21.15.204:3300/0 conn(0x7f71c80cfb30 msgr2=0x7f71c806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 --2- 172.21.15.143:0/350838047 >> v2:172.21.15.204:3300/0 conn(0x7f71c80cfb30 0x7f71c806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.179+0000 7f71c7400640 1 -- 172.21.15.143:0/350838047 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80d02b0 con 0x7f71c81607c0 2024-01-15T18:08:15.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71c7400640 1 --2- 172.21.15.143:0/350838047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81607c0 0x7f71c806b7b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f71c809a980 tx=0x7f71b801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07334f335690d8c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:15.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71c6000640 1 -- 172.21.15.143:0/350838047 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80a89b0 con 0x7f71c81607c0 2024-01-15T18:08:15.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71c6000640 1 -- 172.21.15.143:0/350838047 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b80a8b10 con 0x7f71c81607c0 2024-01-15T18:08:15.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71c6000640 1 -- 172.21.15.143:0/350838047 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b80a8ea0 con 0x7f71c81607c0 2024-01-15T18:08:15.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 -- 172.21.15.143:0/350838047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81607c0 msgr2=0x7f71c806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 --2- 172.21.15.143:0/350838047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81607c0 0x7f71c806b7b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f71c809a980 tx=0x7f71b801ecf0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 -- 172.21.15.143:0/350838047 shutdown_connections 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 --2- 172.21.15.143:0/350838047 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c806c230 0x7f71c80cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 --2- 172.21.15.143:0/350838047 >> v2:172.21.15.204:3300/0 conn(0x7f71c80cfb30 0x7f71c806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 --2- 172.21.15.143:0/350838047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c81607c0 0x7f71c806b7b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 -- 172.21.15.143:0/350838047 >> 172.21.15.143:0/350838047 conn(0x7f71c80d0e20 msgr2=0x7f71c80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 -- 172.21.15.143:0/350838047 shutdown_connections 2024-01-15T18:08:15.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.183+0000 7f71ce400640 1 -- 172.21.15.143:0/350838047 wait complete. 2024-01-15T18:08:15.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 Processor -- start 2024-01-15T18:08:15.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 -- start start 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c806c230 0x7f71c80c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 0x7f71c80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c81607c0 0x7f71c8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c80b4900 con 0x7f71c80cfb30 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81f3e10 con 0x7f71c806c230 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c81f3b30 con 0x7f71c81607c0 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 0x7f71c80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 0x7f71c80b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43516/0 (socket says 172.21.15.143:43516) 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 -- 172.21.15.143:0/65499481 learned_addr learned my addr 172.21.15.143:0/65499481 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c7e00640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c81607c0 0x7f71c8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c7400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c806c230 0x7f71c80c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 -- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c81607c0 msgr2=0x7f71c8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c81607c0 0x7f71c8150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 -- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c806c230 msgr2=0x7f71c80c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c806c230 0x7f71c80c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 -- 172.21.15.143:0/65499481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b80094d0 con 0x7f71c80cfb30 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c7e00640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c81607c0 0x7f71c8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c7400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c806c230 0x7f71c80c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71c6a00640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 0x7f71c80b2c60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f71b000d800 tx=0x7f71b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000b6c0 con 0x7f71c80cfb30 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b000b820 con 0x7f71c80cfb30 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c80b4b30 con 0x7f71c80cfb30 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b000bb90 con 0x7f71c80cfb30 2024-01-15T18:08:15.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.187+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c80c9b10 con 0x7f71c80cfb30 2024-01-15T18:08:15.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.191+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c806d4b0 con 0x7f71c80cfb30 2024-01-15T18:08:15.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.191+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 12) v1 ==== 86568+0+0 (secure 0 0 0) 0x7f71b000bcf0 con 0x7f71c80cfb30 2024-01-15T18:08:15.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.191+0000 7f71bf400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a0066d70 0x7f71a0069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.191+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f71b0085190 con 0x7f71c80cfb30 2024-01-15T18:08:15.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.191+0000 7f71c7400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a0066d70 0x7f71a0069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.191+0000 7f71c7400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a0066d70 0x7f71a0069230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71b80b2000 tx=0x7f71b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:15.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.195+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f71b0051da0 con 0x7f71c80cfb30 2024-01-15T18:08:15.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.567+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f71c80cba30 con 0x7f71a0066d70 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.655+0000 7f71bf400640 1 -- 172.21.15.143:0/65499481 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f71b80b0440 con 0x7f71a0066d70 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:15.661 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:15.662 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:15.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a0066d70 msgr2=0x7f71a0069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a0066d70 0x7f71a0069230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f71b80b2000 tx=0x7f71b8008040 comp rx=0 tx=0).stop 2024-01-15T18:08:15.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 msgr2=0x7f71c80b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 0x7f71c80b2c60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f71b000d800 tx=0x7f71b000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 shutdown_connections 2024-01-15T18:08:15.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a0066d70 0x7f71a0069230 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71c81607c0 0x7f71c8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71c80cfb30 0x7f71c80b2c60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 --2- 172.21.15.143:0/65499481 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71c806c230 0x7f71c80c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.663+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 >> 172.21.15.143:0/65499481 conn(0x7f71c80d0e20 msgr2=0x7f71c806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:15.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.667+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 shutdown_connections 2024-01-15T18:08:15.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:15.671+0000 7f71ce400640 1 -- 172.21.15.143:0/65499481 wait complete. 2024-01-15T18:08:15.745 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:16.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:17.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.239+0000 7fbe7f400640 1 Processor -- start 2024-01-15T18:08:17.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.239+0000 7fbe7f400640 1 -- start start 2024-01-15T18:08:17.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe780b35c0 0x7fbe78165fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7e000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fbe78144740 0x7fbe780ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:50578/0 (socket says 172.21.15.143:50578) 2024-01-15T18:08:17.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7e000640 1 -- 172.21.15.143:0/1843277636 learned_addr learned my addr 172.21.15.143:0/1843277636 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:17.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7815bc40 con 0x7fbe7815b5e0 2024-01-15T18:08:17.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7815bf20 con 0x7fbe78144740 2024-01-15T18:08:17.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7815c8f0 con 0x7fbe780b35c0 2024-01-15T18:08:17.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7ea00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe780b35c0 0x7fbe78165fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338799730 0 0) 0x7fbe7815bf20 con 0x7fbe78144740 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe58003530 con 0x7fbe78144740 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446958415 0 0) 0x7fbe58003530 con 0x7fbe78144740 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe780b35c0 msgr2=0x7fbe78165fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe780b35c0 0x7fbe78165fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 >> v1:172.21.15.143:6789/0 conn(0x7fbe7815b5e0 legacy=0x7fbe780b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7815c200 con 0x7fbe78144740 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.243+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fbe60001680 con 0x7fbe78144740 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 >> v1:172.21.15.204:6789/0 conn(0x7fbe78144740 legacy=0x7fbe780ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe58003980 0x7fbe58003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe580047d0 0x7fbe58004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe58009980 0x7fbe58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7815c8f0 con 0x7fbe580047d0 2024-01-15T18:08:17.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe580147d0 con 0x7fbe58009980 2024-01-15T18:08:17.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe58014990 con 0x7fbe58003980 2024-01-15T18:08:17.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7d600640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe580047d0 0x7fbe58004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.247+0000 7fbe7ea00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe58009980 0x7fbe58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.251+0000 7fbe7ea00640 1 -- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe58003980 msgr2=0x7fbe58003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:17.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.251+0000 7fbe7ea00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe58003980 0x7fbe58003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.251+0000 7fbe7ea00640 1 -- 172.21.15.143:0/1843277636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe580047d0 msgr2=0x7fbe58004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:17.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.251+0000 7fbe7ea00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe580047d0 0x7fbe58004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.251+0000 7fbe7ea00640 1 -- 172.21.15.143:0/1843277636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58003530 con 0x7fbe58009980 2024-01-15T18:08:17.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.259+0000 7fbe7ea00640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe58009980 0x7fbe58005190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009f00 tx=0x7fbe7001cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741a87c4a3268ba3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700a5bf0 con 0x7fbe58009980 2024-01-15T18:08:17.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7000f040 con 0x7fbe58009980 2024-01-15T18:08:17.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7cc00640 1 -- 172.21.15.143:0/1843277636 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700ad440 con 0x7fbe58009980 2024-01-15T18:08:17.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe58009980 msgr2=0x7fbe58005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7f400640 1 --2- 172.21.15.143:0/1843277636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe58009980 0x7fbe58005190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009f00 tx=0x7fbe7001cba0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7ea00640 1 -- 172.21.15.143:0/1843277636 reap_dead start 2024-01-15T18:08:17.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 shutdown_connections 2024-01-15T18:08:17.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.263+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 >> 172.21.15.143:0/1843277636 conn(0x7fbe780b0aa0 msgr2=0x7fbe780b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:17.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.267+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 shutdown_connections 2024-01-15T18:08:17.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.267+0000 7fbe7f400640 1 -- 172.21.15.143:0/1843277636 wait complete. 2024-01-15T18:08:17.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 Processor -- start 2024-01-15T18:08:17.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 -- start start 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7815ac50 0x7fbe781ee590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 0x7fbe7806de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7806c2f0 0x7fbe781406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7815ba60 con 0x7fbe7815ac50 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781f1be0 con 0x7fbe7806c2f0 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.271+0000 7fbe7f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe781eeba0 con 0x7fbe7815b030 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 0x7fbe7806de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 0x7fbe7806de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55894/0 (socket says 172.21.15.143:55894) 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7d600640 1 -- 172.21.15.143:0/3817932185 learned_addr learned my addr 172.21.15.143:0/3817932185 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7e000640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7815ac50 0x7fbe781ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7ea00640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7806c2f0 0x7fbe781406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7d600640 1 -- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7806c2f0 msgr2=0x7fbe781406b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7d600640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7806c2f0 0x7fbe781406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.275+0000 7fbe7d600640 1 -- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7815ac50 msgr2=0x7fbe781ee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.279+0000 7fbe7d600640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7815ac50 0x7fbe781ee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.279+0000 7fbe7d600640 1 -- 172.21.15.143:0/3817932185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78144ca0 con 0x7fbe7815b030 2024-01-15T18:08:17.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.279+0000 7fbe7e000640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7815ac50 0x7fbe781ee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:17.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.283+0000 7fbe7d600640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 0x7fbe7806de30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800f5e0 tx=0x7fbe6800faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.287+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe680052e0 con 0x7fbe7815b030 2024-01-15T18:08:17.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.287+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe68005460 con 0x7fbe7815b030 2024-01-15T18:08:17.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.287+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe68003840 con 0x7fbe7815b030 2024-01-15T18:08:17.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.287+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe781eee80 con 0x7fbe7815b030 2024-01-15T18:08:17.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.291+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7806e370 con 0x7fbe7815b030 2024-01-15T18:08:17.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.299+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe781f1db0 con 0x7fbe7815b030 2024-01-15T18:08:17.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.299+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 12) v1 ==== 86568+0+0 (secure 0 0 0) 0x7fbe680039a0 con 0x7fbe7815b030 2024-01-15T18:08:17.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.303+0000 7fbe76000640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe50066e40 0x7fbe50069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.303+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbe68084aa0 con 0x7fbe7815b030 2024-01-15T18:08:17.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.311+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbe781f1db0 con 0x7fbe7815b030 2024-01-15T18:08:17.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.319+0000 7fbe7e000640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe50066e40 0x7fbe50069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.319+0000 7fbe7e000640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe50066e40 0x7fbe50069300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbe60001830 tx=0x7fbe6000c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.767+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe78144b20 con 0x7fbe50066e40 2024-01-15T18:08:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.835+0000 7fbe76000640 1 -- 172.21.15.143:0/3817932185 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fbe600056f0 con 0x7fbe50066e40 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:17.843 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:17.844 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe50066e40 msgr2=0x7fbe50069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe50066e40 0x7fbe50069300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbe60001830 tx=0x7fbe6000c040 comp rx=0 tx=0).stop 2024-01-15T18:08:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 msgr2=0x7fbe7806de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 0x7fbe7806de30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800f5e0 tx=0x7fbe6800faa0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 shutdown_connections 2024-01-15T18:08:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe50066e40 0x7fbe50069300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe7806c2f0 0x7fbe781406b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe7815b030 0x7fbe7806de30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 --2- 172.21.15.143:0/3817932185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7815ac50 0x7fbe781ee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 >> 172.21.15.143:0/3817932185 conn(0x7fbe780b0aa0 msgr2=0x7fbe78070210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:17.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 shutdown_connections 2024-01-15T18:08:17.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:17.843+0000 7fbe7f400640 1 -- 172.21.15.143:0/3817932185 wait complete. 2024-01-15T18:08:17.945 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:18.394 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:18.387+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:18.394 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:18.387+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:18.394 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:18.387+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:18.394 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:18.387+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:18.394 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:18.387+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:18.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:19.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.351+0000 7f4f51e00640 1 Processor -- start 2024-01-15T18:08:19.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.351+0000 7f4f51e00640 1 -- start start 2024-01-15T18:08:19.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.351+0000 7f4f51e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f4c14afa0 0x7f4f4c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f4aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f4c14afa0 0x7f4f4c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f4aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4f4c14afa0 0x7f4f4c06ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36510/0 (socket says 172.21.15.143:36510) 2024-01-15T18:08:19.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f4a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4f4c06d340 0x7f4f4c0b3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48606/0 (socket says 172.21.15.143:48606) 2024-01-15T18:08:19.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f4aa00640 1 -- 172.21.15.143:0/2120093771 learned_addr learned my addr 172.21.15.143:0/2120093771 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:19.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f51e00640 1 --2- 172.21.15.143:0/2120093771 >> v2:172.21.15.204:3300/0 conn(0x7f4f4c0b3f80 0x7f4f4c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c0cc800 con 0x7f4f4c06d340 2024-01-15T18:08:19.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c14b380 con 0x7f4f4c14afa0 2024-01-15T18:08:19.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.355+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f4c06d7c0 con 0x7f4f4c0b3f80 2024-01-15T18:08:19.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f4b400640 1 --2- 172.21.15.143:0/2120093771 >> v2:172.21.15.204:3300/0 conn(0x7f4f4c0b3f80 0x7f4f4c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f49600640 1 -- 172.21.15.143:0/2120093771 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732762867 0 0) 0x7f4f4c0cc800 con 0x7f4f4c06d340 2024-01-15T18:08:19.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f49600640 1 -- 172.21.15.143:0/2120093771 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f2c003530 con 0x7f4f4c06d340 2024-01-15T18:08:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f4b400640 1 -- 172.21.15.143:0/2120093771 >> v2:172.21.15.143:3300/0 conn(0x7f4f4c14afa0 msgr2=0x7f4f4c06ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f4b400640 1 --2- 172.21.15.143:0/2120093771 >> v2:172.21.15.143:3300/0 conn(0x7f4f4c14afa0 0x7f4f4c06ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f49600640 1 -- 172.21.15.143:0/2120093771 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444217017 0 0) 0x7f4f2c003530 con 0x7f4f4c06d340 2024-01-15T18:08:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f4b400640 1 -- 172.21.15.143:0/2120093771 >> v1:172.21.15.143:6789/0 conn(0x7f4f4c06d340 legacy=0x7f4f4c0b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.359+0000 7f4f4b400640 1 -- 172.21.15.143:0/2120093771 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c06daa0 con 0x7f4f4c0b3f80 2024-01-15T18:08:19.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.367+0000 7f4f4b400640 1 --2- 172.21.15.143:0/2120093771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c0b3f80 0x7f4f4c0ccc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0747f0 tx=0x7f4f4401c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2c79d6a5344cb74 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:19.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.367+0000 7f4f49600640 1 -- 172.21.15.143:0/2120093771 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f34001660 con 0x7f4f4c0b3f80 2024-01-15T18:08:19.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.367+0000 7f4f49600640 1 -- 172.21.15.143:0/2120093771 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440a6480 con 0x7f4f4c0b3f80 2024-01-15T18:08:19.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.367+0000 7f4f49600640 1 -- 172.21.15.143:0/2120093771 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440a6810 con 0x7f4f4c0b3f80 2024-01-15T18:08:19.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.367+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c0b3f80 msgr2=0x7f4f4c0ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.367+0000 7f4f51e00640 1 --2- 172.21.15.143:0/2120093771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c0b3f80 0x7f4f4c0ccc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0747f0 tx=0x7f4f4401c830 comp rx=0 tx=0).stop 2024-01-15T18:08:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.371+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 shutdown_connections 2024-01-15T18:08:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.371+0000 7f4f51e00640 1 --2- 172.21.15.143:0/2120093771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c0b3f80 0x7f4f4c0ccc90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.371+0000 7f4f51e00640 1 --2- 172.21.15.143:0/2120093771 >> v2:172.21.15.143:3300/0 conn(0x7f4f4c14afa0 0x7f4f4c06ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.371+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 >> 172.21.15.143:0/2120093771 conn(0x7f4f4c0aea10 msgr2=0x7f4f4c0aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.371+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 shutdown_connections 2024-01-15T18:08:19.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.371+0000 7f4f51e00640 1 -- 172.21.15.143:0/2120093771 wait complete. 2024-01-15T18:08:19.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.379+0000 7f4f51e00640 1 Processor -- start 2024-01-15T18:08:19.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.379+0000 7f4f51e00640 1 -- start start 2024-01-15T18:08:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.383+0000 7f4f51e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c06d340 0x7f4f4c1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.383+0000 7f4f51e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f4c0b3f80 0x7f4f4c142a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.383+0000 7f4f51e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 0x7f4f4c0a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.383+0000 7f4f51e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c148c50 con 0x7f4f4c0b3f80 2024-01-15T18:08:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.383+0000 7f4f51e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c148360 con 0x7f4f4c06d340 2024-01-15T18:08:19.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.383+0000 7f4f51e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c148600 con 0x7f4f4c14afa0 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 0x7f4f4c0a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 0x7f4f4c0a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55920/0 (socket says 172.21.15.143:55920) 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 -- 172.21.15.143:0/2977499104 learned_addr learned my addr 172.21.15.143:0/2977499104 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 -- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c06d340 msgr2=0x7f4f4c1490e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c06d340 0x7f4f4c1490e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 -- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f4c0b3f80 msgr2=0x7f4f4c142a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f4c0b3f80 0x7f4f4c142a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 -- 172.21.15.143:0/2977499104 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009a70 con 0x7f4f4c14afa0 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f4b400640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 0x7f4f4c0a9e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4f4401aee0 tx=0x7f4f440195d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440b7070 con 0x7f4f4c14afa0 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440a6e90 con 0x7f4f4c14afa0 2024-01-15T18:08:19.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.387+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440a7680 con 0x7f4f4c14afa0 2024-01-15T18:08:19.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.391+0000 7f4f51e00640 1 -- 172.21.15.143:0/2977499104 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c1488e0 con 0x7f4f4c14afa0 2024-01-15T18:08:19.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.391+0000 7f4f51e00640 1 -- 172.21.15.143:0/2977499104 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c0aa8e0 con 0x7f4f4c14afa0 2024-01-15T18:08:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.395+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 12) v1 ==== 86568+0+0 (secure 0 0 0) 0x7f4f440a77e0 con 0x7f4f4c14afa0 2024-01-15T18:08:19.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.399+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c067500 con 0x7f4f4c14afa0 2024-01-15T18:08:19.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.399+0000 7f4f43400640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f24066e40 0x7f4f24069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.403+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4f4411f510 con 0x7f4f4c14afa0 2024-01-15T18:08:19.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.415+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f440ef9f0 con 0x7f4f4c14afa0 2024-01-15T18:08:19.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.451+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f24066e40 0x7f4f24069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.475+0000 7f4f4aa00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f24066e40 0x7f4f24069300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009940 tx=0x7f4f3c009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:19.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:19.671+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f4f440ec330 con 0x7f4f4c14afa0 2024-01-15T18:08:20.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.051+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f4c147f10 con 0x7f4f24066e40 2024-01-15T18:08:20.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.123+0000 7f4f43400640 1 -- 172.21.15.143:0/2977499104 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f4f4c147f10 con 0x7f4f24066e40 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:08:20.129 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:20.130 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:20.130 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:20.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.131+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f24066e40 msgr2=0x7f4f24069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.131+0000 7f4f40c00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f24066e40 0x7f4f24069300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009940 tx=0x7f4f3c009240 comp rx=0 tx=0).stop 2024-01-15T18:08:20.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.131+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 msgr2=0x7f4f4c0a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.131+0000 7f4f40c00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 0x7f4f4c0a9e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4f4401aee0 tx=0x7f4f440195d0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.135+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 shutdown_connections 2024-01-15T18:08:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.135+0000 7f4f40c00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f24066e40 0x7f4f24069300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.135+0000 7f4f40c00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f4c14afa0 0x7f4f4c0a9e40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.135+0000 7f4f40c00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f4c0b3f80 0x7f4f4c142a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.135+0000 7f4f40c00640 1 --2- 172.21.15.143:0/2977499104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f4c06d340 0x7f4f4c1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.135+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 >> 172.21.15.143:0/2977499104 conn(0x7f4f4c0aea10 msgr2=0x7f4f4c071d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:20.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.139+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 shutdown_connections 2024-01-15T18:08:20.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:20.143+0000 7f4f40c00640 1 -- 172.21.15.143:0/2977499104 wait complete. 2024-01-15T18:08:20.187 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:21.187 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:21.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.439+0000 7fbde3800640 1 Processor -- start 2024-01-15T18:08:21.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.439+0000 7fbde3800640 1 -- start start 2024-01-15T18:08:21.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde3800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbddc15b260 0x7fbddc0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc0c4d90 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde3800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc15b640 con 0x7fbddc0d19f0 2024-01-15T18:08:21.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde3800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbddc0cf800 con 0x7fbddc15b260 2024-01-15T18:08:21.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde0e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbddc15b260 0x7fbddc0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbddbe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fbddc0d19f0 0x7fbddc0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:50594/0 (socket says 172.21.15.143:50594) 2024-01-15T18:08:21.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbddbe00640 1 -- 172.21.15.143:0/2064041903 learned_addr learned my addr 172.21.15.143:0/2064041903 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbddb400640 1 -- 172.21.15.143:0/2064041903 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862880594 0 0) 0x7fbddc0c4d90 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbddb400640 1 -- 172.21.15.143:0/2064041903 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdbc003530 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde0e00640 1 -- 172.21.15.143:0/2064041903 >> v1:172.21.15.204:6789/0 conn(0x7fbddc0d19f0 legacy=0x7fbddc0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbddb400640 1 -- 172.21.15.143:0/2064041903 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242555830 0 0) 0x7fbddc15b640 con 0x7fbddc0d19f0 2024-01-15T18:08:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde0e00640 1 -- 172.21.15.143:0/2064041903 >> v1:172.21.15.143:6789/0 conn(0x7fbddc0ccaf0 legacy=0x7fbddc0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.443+0000 7fbde0e00640 1 -- 172.21.15.143:0/2064041903 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0cfae0 con 0x7fbddc15b260 2024-01-15T18:08:21.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde0e00640 1 --2- 172.21.15.143:0/2064041903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc15b260 0x7fbddc0cd810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009d90 tx=0x7fbdcc01c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8298e1f4fbd40153 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbddb400640 1 -- 172.21.15.143:0/2064041903 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4001530 con 0x7fbddc15b260 2024-01-15T18:08:21.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbddb400640 1 -- 172.21.15.143:0/2064041903 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc0a6420 con 0x7fbddc15b260 2024-01-15T18:08:21.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbddb400640 1 -- 172.21.15.143:0/2064041903 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0a6790 con 0x7fbddc15b260 2024-01-15T18:08:21.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 -- 172.21.15.143:0/2064041903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc15b260 msgr2=0x7fbddc0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 --2- 172.21.15.143:0/2064041903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc15b260 0x7fbddc0cd810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009d90 tx=0x7fbdcc01c7b0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 -- 172.21.15.143:0/2064041903 shutdown_connections 2024-01-15T18:08:21.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 --2- 172.21.15.143:0/2064041903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc15b260 0x7fbddc0cd810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 -- 172.21.15.143:0/2064041903 >> 172.21.15.143:0/2064041903 conn(0x7fbddc0d0ed0 msgr2=0x7fbddc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 -- 172.21.15.143:0/2064041903 shutdown_connections 2024-01-15T18:08:21.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.451+0000 7fbde3800640 1 -- 172.21.15.143:0/2064041903 wait complete. 2024-01-15T18:08:21.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 Processor -- start 2024-01-15T18:08:21.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 -- start start 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc0ccaf0 0x7fbddc146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbddc0d19f0 0x7fbddc0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbddc15b260 0x7fbddc0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc09cc20 con 0x7fbddc0d19f0 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc09cec0 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc09d1a0 con 0x7fbddc15b260 2024-01-15T18:08:21.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbddbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbddc0d19f0 0x7fbddc0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbddc15b260 0x7fbddc0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc0ccaf0 0x7fbddc146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbddc15b260 0x7fbddc0b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55924/0 (socket says 172.21.15.143:55924) 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde1800640 1 -- 172.21.15.143:0/3975265544 learned_addr learned my addr 172.21.15.143:0/3975265544 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde0e00640 1 -- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbddc15b260 msgr2=0x7fbddc0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde0e00640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbddc15b260 0x7fbddc0b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde0e00640 1 -- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbddc0d19f0 msgr2=0x7fbddc0b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde0e00640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbddc0d19f0 0x7fbddc0b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.455+0000 7fbde0e00640 1 -- 172.21.15.143:0/3975265544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc009a70 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbde0e00640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc0ccaf0 0x7fbddc146180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0ac000 tx=0x7fbdcc019550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0b4040 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc0ba840 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0babb0 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc09d480 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc09da40 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc067500 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.459+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fbdcc0bad10 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.463+0000 7fbdd8c00640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdb4066e00 0x7fbdb40692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.463+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbdcc11fe50 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.467+0000 7fbddbe00640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdb4066e00 0x7fbdb40692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.467+0000 7fbddbe00640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdb4066e00 0x7fbdb40692c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0cac70 tx=0x7fbdd4007220 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.467+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbdcc123030 con 0x7fbddc0ccaf0 2024-01-15T18:08:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.791+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7fbddc0cf4b0 con 0x7fbdb4066e00 2024-01-15T18:08:21.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.811+0000 7fbdd8c00640 1 -- 172.21.15.143:0/3975265544 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fbdd40073b0 con 0x7fbdb4066e00 2024-01-15T18:08:21.817 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:21.818 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:21.819 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:21.819 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:21.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.823+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdb4066e00 msgr2=0x7fbdb40692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.823+0000 7fbde3800640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdb4066e00 0x7fbdb40692c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0cac70 tx=0x7fbdd4007220 comp rx=0 tx=0).stop 2024-01-15T18:08:21.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.823+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc0ccaf0 msgr2=0x7fbddc146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.823+0000 7fbde3800640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc0ccaf0 0x7fbddc146180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0ac000 tx=0x7fbdcc019550 comp rx=0 tx=0).stop 2024-01-15T18:08:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 shutdown_connections 2024-01-15T18:08:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdb4066e00 0x7fbdb40692c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbddc15b260 0x7fbddc0b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbddc0d19f0 0x7fbddc0b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 --2- 172.21.15.143:0/3975265544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbddc0ccaf0 0x7fbddc146180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 >> 172.21.15.143:0/3975265544 conn(0x7fbddc0d0ed0 msgr2=0x7fbddc06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.827+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 shutdown_connections 2024-01-15T18:08:21.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:21.831+0000 7fbde3800640 1 -- 172.21.15.143:0/3975265544 wait complete. 2024-01-15T18:08:21.898 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:22.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:23.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.279+0000 7f5e8de00640 1 Processor -- start 2024-01-15T18:08:23.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.279+0000 7f5e8de00640 1 -- start start 2024-01-15T18:08:23.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.283+0000 7f5e8de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5e880ce310 0x7f5e88070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:23.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.283+0000 7f5e8de00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8806cf30 con 0x7f5e880708b0 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.283+0000 7f5e87400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5e880708b0 0x7f5e8815cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48620/0 (socket says 172.21.15.143:48620) 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e87400640 1 -- 172.21.15.143:0/3706215743 learned_addr learned my addr 172.21.15.143:0/3706215743 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e880cb2f0 con 0x7f5e88159170 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e880cb0e0 con 0x7f5e880ce310 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.283+0000 7f5e86a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f5e88159170 0x7f5e8806d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:50604/0 (socket says 172.21.15.143:50604) 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.287+0000 7f5e86000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5e880ce310 0x7f5e88070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e85600640 1 -- 172.21.15.143:0/3706215743 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836392068 0 0) 0x7f5e880cb2f0 con 0x7f5e88159170 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e85600640 1 -- 172.21.15.143:0/3706215743 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e68003530 con 0x7f5e88159170 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e86000640 1 -- 172.21.15.143:0/3706215743 >> v1:172.21.15.204:6789/0 conn(0x7f5e88159170 legacy=0x7f5e8806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:23.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e85600640 1 -- 172.21.15.143:0/3706215743 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4214483295 0 0) 0x7f5e68003530 con 0x7f5e88159170 2024-01-15T18:08:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e86000640 1 -- 172.21.15.143:0/3706215743 >> v1:172.21.15.143:6789/0 conn(0x7f5e880708b0 legacy=0x7f5e8815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:23.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.291+0000 7f5e86000640 1 -- 172.21.15.143:0/3706215743 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880cb5d0 con 0x7f5e880ce310 2024-01-15T18:08:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.303+0000 7f5e86000640 1 --2- 172.21.15.143:0/3706215743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e880ce310 0x7f5e88070370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5e70006de0 tx=0x7f5e70020620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177934d1cf11146f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.307+0000 7f5e85600640 1 -- 172.21.15.143:0/3706215743 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e880cebe0 con 0x7f5e880ce310 2024-01-15T18:08:23.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.307+0000 7f5e85600640 1 -- 172.21.15.143:0/3706215743 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e700aa410 con 0x7f5e880ce310 2024-01-15T18:08:23.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.307+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e880ce310 msgr2=0x7f5e88070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:23.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.307+0000 7f5e8de00640 1 --2- 172.21.15.143:0/3706215743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e880ce310 0x7f5e88070370 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5e70006de0 tx=0x7f5e70020620 comp rx=0 tx=0).stop 2024-01-15T18:08:23.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.307+0000 7f5e85600640 1 -- 172.21.15.143:0/3706215743 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78001680 con 0x7f5e880ce310 2024-01-15T18:08:23.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.319+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 shutdown_connections 2024-01-15T18:08:23.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.319+0000 7f5e8de00640 1 --2- 172.21.15.143:0/3706215743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e880ce310 0x7f5e88070370 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:23.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.319+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 >> 172.21.15.143:0/3706215743 conn(0x7f5e880b0bf0 msgr2=0x7f5e880b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:23.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.331+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 shutdown_connections 2024-01-15T18:08:23.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.335+0000 7f5e8de00640 1 -- 172.21.15.143:0/3706215743 wait complete. 2024-01-15T18:08:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.359+0000 7f5e8de00640 1 Processor -- start 2024-01-15T18:08:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.359+0000 7f5e8de00640 1 -- start start 2024-01-15T18:08:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e8de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e880708b0 0x7f5e88147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e8de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e880ce310 0x7f5e88150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e8de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 0x7f5e8814c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e8de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8814cc60 con 0x7f5e880708b0 2024-01-15T18:08:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e8de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8814cf40 con 0x7f5e88159170 2024-01-15T18:08:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e8de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e881473e0 con 0x7f5e880ce310 2024-01-15T18:08:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e86000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e880ce310 0x7f5e88150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 0x7f5e8814c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 0x7f5e8814c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:52696/0 (socket says 172.21.15.143:52696) 2024-01-15T18:08:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 -- 172.21.15.143:0/695629592 learned_addr learned my addr 172.21.15.143:0/695629592 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:23.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 -- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e880ce310 msgr2=0x7f5e88150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e880ce310 0x7f5e88150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 -- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e880708b0 msgr2=0x7f5e88147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e880708b0 0x7f5e88147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.371+0000 7f5e87400640 1 -- 172.21.15.143:0/695629592 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70009420 con 0x7f5e88159170 2024-01-15T18:08:23.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.375+0000 7f5e87400640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 0x7f5e8814c1c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000cbc0 tx=0x7f5e8000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.375+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000ad40 con 0x7f5e88159170 2024-01-15T18:08:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.375+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e80012400 con 0x7f5e88159170 2024-01-15T18:08:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.375+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8000aea0 con 0x7f5e88159170 2024-01-15T18:08:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.375+0000 7f5e8de00640 1 -- 172.21.15.143:0/695629592 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e881476c0 con 0x7f5e88159170 2024-01-15T18:08:23.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.375+0000 7f5e8de00640 1 -- 172.21.15.143:0/695629592 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e88148400 con 0x7f5e88159170 2024-01-15T18:08:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.383+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e88067680 con 0x7f5e88159170 2024-01-15T18:08:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.387+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f5e80007070 con 0x7f5e88159170 2024-01-15T18:08:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.391+0000 7f5e7f400640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e60066e00 0x7f5e600692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:23.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.391+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5e80084b40 con 0x7f5e88159170 2024-01-15T18:08:23.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.395+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e80051740 con 0x7f5e88159170 2024-01-15T18:08:23.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.411+0000 7f5e86a00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e60066e00 0x7f5e600692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:23.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.423+0000 7f5e86a00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e60066e00 0x7f5e600692c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e780011f0 tx=0x7f5e78001160 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:23.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:23.947+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e88159760 con 0x7f5e60066e00 2024-01-15T18:08:24.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.195+0000 7f5e7f400640 1 -- 172.21.15.143:0/695629592 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f5e78001b60 con 0x7f5e60066e00 2024-01-15T18:08:24.203 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:24.203 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:24.203 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:24.203 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:24.203 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:24.203 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:24.204 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:24.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e60066e00 msgr2=0x7f5e600692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e60066e00 0x7f5e600692c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e780011f0 tx=0x7f5e78001160 comp rx=0 tx=0).stop 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 msgr2=0x7f5e8814c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 0x7f5e8814c1c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000cbc0 tx=0x7f5e8000a430 comp rx=0 tx=0).stop 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 shutdown_connections 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e60066e00 0x7f5e600692c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e88159170 0x7f5e8814c1c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e880ce310 0x7f5e88150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 --2- 172.21.15.143:0/695629592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e880708b0 0x7f5e88147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.211+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 >> 172.21.15.143:0/695629592 conn(0x7f5e880b0bf0 msgr2=0x7f5e880b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.219+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 shutdown_connections 2024-01-15T18:08:24.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:24.219+0000 7f5e7cc00640 1 -- 172.21.15.143:0/695629592 wait complete. 2024-01-15T18:08:24.333 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:25.334 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:25.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.915+0000 7f3e2be00640 1 Processor -- start 2024-01-15T18:08:25.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.915+0000 7f3e2be00640 1 -- start start 2024-01-15T18:08:25.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e2be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3e24159380 0x7f3e24072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e24070a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e2be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e240cf3b0 0x7f3e240cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e2be00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240cfd80 con 0x7f3e24159380 2024-01-15T18:08:25.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e2be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e24159760 con 0x7f3e240cf3b0 2024-01-15T18:08:25.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e2be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24071cf0 con 0x7f3e240ce950 2024-01-15T18:08:25.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.919+0000 7f3e29e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e240cf3b0 0x7f3e240cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:25.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e24070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e24070a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55944/0 (socket says 172.21.15.143:55944) 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 -- 172.21.15.143:0/511901150 learned_addr learned my addr 172.21.15.143:0/511901150 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 -- 172.21.15.143:0/511901150 >> v2:172.21.15.204:3300/0 conn(0x7f3e240cf3b0 msgr2=0x7f3e240cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 --2- 172.21.15.143:0/511901150 >> v2:172.21.15.204:3300/0 conn(0x7f3e240cf3b0 0x7f3e240cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 -- 172.21.15.143:0/511901150 >> v2:172.21.15.143:3300/0 conn(0x7f3e24159380 msgr2=0x7f3e24072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 --2- 172.21.15.143:0/511901150 >> v2:172.21.15.143:3300/0 conn(0x7f3e24159380 0x7f3e24072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.923+0000 7f3e28a00640 1 -- 172.21.15.143:0/511901150 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240ce520 con 0x7f3e240ce950 2024-01-15T18:08:25.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.931+0000 7f3e28a00640 1 --2- 172.21.15.143:0/511901150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e24070a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3e240cb940 tx=0x7f3e0c01ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd18822573e757e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.931+0000 7f3e23e00640 1 -- 172.21.15.143:0/511901150 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0a89b0 con 0x7f3e240ce950 2024-01-15T18:08:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.931+0000 7f3e23e00640 1 -- 172.21.15.143:0/511901150 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0a8b10 con 0x7f3e240ce950 2024-01-15T18:08:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.931+0000 7f3e23e00640 1 -- 172.21.15.143:0/511901150 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0a8ea0 con 0x7f3e240ce950 2024-01-15T18:08:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 -- 172.21.15.143:0/511901150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 msgr2=0x7f3e24070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:25.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 --2- 172.21.15.143:0/511901150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e24070a00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3e240cb940 tx=0x7f3e0c01ecf0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 -- 172.21.15.143:0/511901150 shutdown_connections 2024-01-15T18:08:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 --2- 172.21.15.143:0/511901150 >> v2:172.21.15.204:3300/0 conn(0x7f3e240cf3b0 0x7f3e240cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 --2- 172.21.15.143:0/511901150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e24070a00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 --2- 172.21.15.143:0/511901150 >> v2:172.21.15.143:3300/0 conn(0x7f3e24159380 0x7f3e24072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 -- 172.21.15.143:0/511901150 >> 172.21.15.143:0/511901150 conn(0x7f3e2406a890 msgr2=0x7f3e2406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:25.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 -- 172.21.15.143:0/511901150 shutdown_connections 2024-01-15T18:08:25.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.935+0000 7f3e2be00640 1 -- 172.21.15.143:0/511901150 wait complete. 2024-01-15T18:08:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 Processor -- start 2024-01-15T18:08:25.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 -- start start 2024-01-15T18:08:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e2414a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e240cf3b0 0x7f3e2415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 0x7f3e240b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2414ae70 con 0x7f3e240cf3b0 2024-01-15T18:08:25.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2415bb40 con 0x7f3e24159380 2024-01-15T18:08:25.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e2be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2415af30 con 0x7f3e240ce950 2024-01-15T18:08:25.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e2414a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:25.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 0x7f3e240b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 0x7f3e240b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:52704/0 (socket says 172.21.15.143:52704) 2024-01-15T18:08:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 -- 172.21.15.143:0/2289431230 learned_addr learned my addr 172.21.15.143:0/2289431230 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e28a00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e240cf3b0 0x7f3e2415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 -- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 msgr2=0x7f3e2414a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e2414a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 -- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e240cf3b0 msgr2=0x7f3e2415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e240cf3b0 0x7f3e2415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e29e00640 1 -- 172.21.15.143:0/2289431230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0094d0 con 0x7f3e24159380 2024-01-15T18:08:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.939+0000 7f3e28a00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e240cf3b0 0x7f3e2415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e29e00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 0x7f3e240b0bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c007e40 tx=0x7f3e1c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c014620 con 0x7f3e24159380 2024-01-15T18:08:25.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c00ee90 con 0x7f3e24159380 2024-01-15T18:08:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c014620 con 0x7f3e24159380 2024-01-15T18:08:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2415b210 con 0x7f3e24159380 2024-01-15T18:08:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2415b7d0 con 0x7f3e24159380 2024-01-15T18:08:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.943+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f3e1c014780 con 0x7f3e24159380 2024-01-15T18:08:25.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.947+0000 7f3e21600640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3dfc066e90 0x7f3dfc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.947+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e24067680 con 0x7f3e24159380 2024-01-15T18:08:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.947+0000 7f3e29400640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3dfc066e90 0x7f3dfc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.947+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3e1c0852c0 con 0x7f3e24159380 2024-01-15T18:08:25.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.947+0000 7f3e29400640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3dfc066e90 0x7f3dfc069350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400a790 tx=0x7f3e14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:25.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:25.951+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e1c0558e0 con 0x7f3e24159380 2024-01-15T18:08:26.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.411+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e240716a0 con 0x7f3dfc066e90 2024-01-15T18:08:26.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.471+0000 7f3e21600640 1 -- 172.21.15.143:0/2289431230 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f3e1400a380 con 0x7f3dfc066e90 2024-01-15T18:08:26.490 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:26.491 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:26.491 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:26.491 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:26.491 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:26.491 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:26.491 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:26.492 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:26.492 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:26.492 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:08:26.492 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:08:26.492 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:08:26.493 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:26.493 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:26.493 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:26.493 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:26.493 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:26.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.495+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3dfc066e90 msgr2=0x7f3dfc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.495+0000 7f3e2be00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3dfc066e90 0x7f3dfc069350 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400a790 tx=0x7f3e14008040 comp rx=0 tx=0).stop 2024-01-15T18:08:26.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.495+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 msgr2=0x7f3e240b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.495+0000 7f3e2be00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 0x7f3e240b0bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c007e40 tx=0x7f3e1c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 shutdown_connections 2024-01-15T18:08:26.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3dfc066e90 0x7f3dfc069350 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e24159380 0x7f3e240b0bd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e240cf3b0 0x7f3e2415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 --2- 172.21.15.143:0/2289431230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e240ce950 0x7f3e2414a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 >> 172.21.15.143:0/2289431230 conn(0x7f3e2406a890 msgr2=0x7f3e2406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 shutdown_connections 2024-01-15T18:08:26.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:26.499+0000 7f3e2be00640 1 -- 172.21.15.143:0/2289431230 wait complete. 2024-01-15T18:08:26.629 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:27.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:27.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:27.755 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:27.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.939+0000 7f7a40600640 1 Processor -- start 2024-01-15T18:08:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.943+0000 7f7a40600640 1 -- start start 2024-01-15T18:08:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.943+0000 7f7a40600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 0x7f7a38070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.943+0000 7f7a3dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7a38159170 0x7f7a3806d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33066/0 (socket says 172.21.15.143:33066) 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 0x7f7a38070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 0x7f7a38070370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55076/0 (socket says 172.21.15.143:55076) 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a40600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3806cf30 con 0x7f7a38159170 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a40600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a380cb2f0 con 0x7f7a380708b0 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a40600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380cb0e0 con 0x7f7a380ce310 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a3dc00640 1 -- 172.21.15.143:0/2367036488 learned_addr learned my addr 172.21.15.143:0/2367036488 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a3d200640 1 -- 172.21.15.143:0/2367036488 >> v1:172.21.15.204:6789/0 conn(0x7f7a380708b0 legacy=0x7f7a3815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:27.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a3d200640 1 -- 172.21.15.143:0/2367036488 >> v1:172.21.15.143:6789/0 conn(0x7f7a38159170 legacy=0x7f7a3806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.955+0000 7f7a3d200640 1 -- 172.21.15.143:0/2367036488 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380cb5d0 con 0x7f7a380ce310 2024-01-15T18:08:27.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.963+0000 7f7a3d200640 1 --2- 172.21.15.143:0/2367036488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 0x7f7a38070370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a200097f0 tx=0x7f7a2001e540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b4701c3988c1cc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:27.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.963+0000 7f7a37e00640 1 -- 172.21.15.143:0/2367036488 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a380cebe0 con 0x7f7a380ce310 2024-01-15T18:08:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.963+0000 7f7a37e00640 1 -- 172.21.15.143:0/2367036488 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a200ab630 con 0x7f7a380ce310 2024-01-15T18:08:27.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.963+0000 7f7a37e00640 1 -- 172.21.15.143:0/2367036488 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200ab9c0 con 0x7f7a380ce310 2024-01-15T18:08:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 -- 172.21.15.143:0/2367036488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 msgr2=0x7f7a38070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 --2- 172.21.15.143:0/2367036488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 0x7f7a38070370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a200097f0 tx=0x7f7a2001e540 comp rx=0 tx=0).stop 2024-01-15T18:08:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 -- 172.21.15.143:0/2367036488 shutdown_connections 2024-01-15T18:08:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 --2- 172.21.15.143:0/2367036488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a380ce310 0x7f7a38070370 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 -- 172.21.15.143:0/2367036488 >> 172.21.15.143:0/2367036488 conn(0x7f7a380b0bf0 msgr2=0x7f7a380b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 -- 172.21.15.143:0/2367036488 shutdown_connections 2024-01-15T18:08:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.971+0000 7f7a40600640 1 -- 172.21.15.143:0/2367036488 wait complete. 2024-01-15T18:08:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 Processor -- start 2024-01-15T18:08:27.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 -- start start 2024-01-15T18:08:27.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a380708b0 0x7f7a38159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a380ce310 0x7f7a38153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a38159170 0x7f7a3814b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a381562f0 con 0x7f7a380ce310 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3815a6e0 con 0x7f7a380708b0 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a40600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3815a9c0 con 0x7f7a38159170 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a380ce310 0x7f7a38153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a380ce310 0x7f7a38153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46928/0 (socket says 172.21.15.143:46928) 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.979+0000 7f7a3d200640 1 -- 172.21.15.143:0/1350784335 learned_addr learned my addr 172.21.15.143:0/1350784335 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a380708b0 0x7f7a38159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3e600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a38159170 0x7f7a3814b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 -- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a38159170 msgr2=0x7f7a3814b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a38159170 0x7f7a3814b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 -- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a380ce310 msgr2=0x7f7a38153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a380ce310 0x7f7a38153960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 -- 172.21.15.143:0/1350784335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a20009420 con 0x7f7a380708b0 2024-01-15T18:08:27.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3e600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a38159170 0x7f7a3814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:27.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a3dc00640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a380708b0 0x7f7a38159840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c430 tx=0x7f7a2800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:27.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28005cf0 con 0x7f7a380708b0 2024-01-15T18:08:27.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a28005e50 con 0x7f7a380708b0 2024-01-15T18:08:27.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280072d0 con 0x7f7a380708b0 2024-01-15T18:08:27.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a38155f80 con 0x7f7a380708b0 2024-01-15T18:08:27.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.983+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a38153ea0 con 0x7f7a380708b0 2024-01-15T18:08:27.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.987+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a38067680 con 0x7f7a380708b0 2024-01-15T18:08:27.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.991+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f7a28007430 con 0x7f7a380708b0 2024-01-15T18:08:27.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.991+0000 7f7a35600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a10066e00 0x7f7a100692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.991+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7a28084170 con 0x7f7a380708b0 2024-01-15T18:08:28.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.995+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7a28050d70 con 0x7f7a380708b0 2024-01-15T18:08:28.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:27.999+0000 7f7a3d200640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a10066e00 0x7f7a100692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.003+0000 7f7a3d200640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a10066e00 0x7f7a100692c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7a2001c170 tx=0x7f7a200a9000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.443+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a38155e00 con 0x7f7a10066e00 2024-01-15T18:08:28.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.459+0000 7f7a35600640 1 -- 172.21.15.143:0/1350784335 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f7a200ab8a0 con 0x7f7a10066e00 2024-01-15T18:08:28.463 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:28.464 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:28.465 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:28.465 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:28.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a10066e00 msgr2=0x7f7a100692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a10066e00 0x7f7a100692c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7a2001c170 tx=0x7f7a200a9000 comp rx=0 tx=0).stop 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a380708b0 msgr2=0x7f7a38159840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a380708b0 0x7f7a38159840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c430 tx=0x7f7a2800c8f0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 shutdown_connections 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7a10066e00 0x7f7a100692c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a38159170 0x7f7a3814b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a380ce310 0x7f7a38153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 --2- 172.21.15.143:0/1350784335 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a380708b0 0x7f7a38159840 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 >> 172.21.15.143:0/1350784335 conn(0x7f7a380b0bf0 msgr2=0x7f7a380b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 shutdown_connections 2024-01-15T18:08:28.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:28.463+0000 7f7a40600640 1 -- 172.21.15.143:0/1350784335 wait complete. 2024-01-15T18:08:28.513 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:29.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:29.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b39c00640 1 Processor -- start 2024-01-15T18:08:29.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b39c00640 1 -- start start 2024-01-15T18:08:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b39c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3b340b0310 0x7f3b340cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b39c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340cbff0 con 0x7f3b3406bfb0 2024-01-15T18:08:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b39c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340d0c50 con 0x7f3b340cf740 2024-01-15T18:08:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b39c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b340cb6c0 con 0x7f3b340b0310 2024-01-15T18:08:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b32a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3b3406bfb0 0x7f3b3406c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33080/0 (socket says 172.21.15.143:33080) 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b32a00640 1 -- 172.21.15.143:0/931881728 learned_addr learned my addr 172.21.15.143:0/931881728 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b32000640 1 --2- 172.21.15.143:0/931881728 >> v2:172.21.15.204:3300/0 conn(0x7f3b340b0310 0x7f3b340cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901241415 0 0) 0x7f3b340cbff0 con 0x7f3b3406bfb0 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b14003530 con 0x7f3b3406bfb0 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932122325 0 0) 0x7f3b340d0c50 con 0x7f3b340cf740 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b340cbff0 con 0x7f3b340cf740 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b32000640 1 -- 172.21.15.143:0/931881728 >> v1:172.21.15.204:6789/0 conn(0x7f3b340cf740 legacy=0x7f3b340cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787538290 0 0) 0x7f3b14003530 con 0x7f3b3406bfb0 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b32000640 1 -- 172.21.15.143:0/931881728 >> v1:172.21.15.143:6789/0 conn(0x7f3b3406bfb0 legacy=0x7f3b3406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.759+0000 7f3b32000640 1 -- 172.21.15.143:0/931881728 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340cb9a0 con 0x7f3b340b0310 2024-01-15T18:08:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b32000640 1 --2- 172.21.15.143:0/931881728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340cf200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097c0 tx=0x7f3b1c01c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06ee3a51e8dee73 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24001680 con 0x7f3b340b0310 2024-01-15T18:08:29.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c0a7050 con 0x7f3b340b0310 2024-01-15T18:08:29.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b31600640 1 -- 172.21.15.143:0/931881728 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0ab620 con 0x7f3b340b0310 2024-01-15T18:08:29.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b39c00640 1 -- 172.21.15.143:0/931881728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 msgr2=0x7f3b340cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b39c00640 1 --2- 172.21.15.143:0/931881728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340cf200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097c0 tx=0x7f3b1c01c2e0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b39c00640 1 -- 172.21.15.143:0/931881728 shutdown_connections 2024-01-15T18:08:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b39c00640 1 --2- 172.21.15.143:0/931881728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340cf200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.763+0000 7f3b39c00640 1 -- 172.21.15.143:0/931881728 >> 172.21.15.143:0/931881728 conn(0x7f3b340d0ed0 msgr2=0x7f3b340d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 -- 172.21.15.143:0/931881728 shutdown_connections 2024-01-15T18:08:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 -- 172.21.15.143:0/931881728 wait complete. 2024-01-15T18:08:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 Processor -- start 2024-01-15T18:08:29.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 -- start start 2024-01-15T18:08:29.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3406bfb0 0x7f3b3414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 0x7f3b340c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340aea50 con 0x7f3b340cf740 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340aed30 con 0x7f3b340b0310 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b39c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340af010 con 0x7f3b3406bfb0 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 0x7f3b340c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b32a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3406bfb0 0x7f3b3414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 0x7f3b340c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46934/0 (socket says 172.21.15.143:46934) 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 -- 172.21.15.143:0/3243205245 learned_addr learned my addr 172.21.15.143:0/3243205245 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b32000640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 -- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3406bfb0 msgr2=0x7f3b3414b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3406bfb0 0x7f3b3414b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 -- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 msgr2=0x7f3b340afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b33400640 1 -- 172.21.15.143:0/3243205245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c009420 con 0x7f3b340cf740 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.767+0000 7f3b32a00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3406bfb0 0x7f3b3414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b33400640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 0x7f3b340c5b00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00c730 tx=0x7f3b2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c003b80 con 0x7f3b340cf740 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c003ce0 con 0x7f3b340cf740 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b340af2f0 con 0x7f3b340cf740 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c00aa50 con 0x7f3b340cf740 2024-01-15T18:08:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3414b710 con 0x7f3b340cf740 2024-01-15T18:08:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34067500 con 0x7f3b340cf740 2024-01-15T18:08:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.771+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f3b2c00abb0 con 0x7f3b340cf740 2024-01-15T18:08:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.775+0000 7f3b2aa00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b0c066e00 0x7f3b0c0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.775+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3b2c085540 con 0x7f3b340cf740 2024-01-15T18:08:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.775+0000 7f3b32a00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b0c066e00 0x7f3b0c0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.775+0000 7f3b32a00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b0c066e00 0x7f3b0c0692c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b240010f0 tx=0x7f3b24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:29.779+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3b2c052110 con 0x7f3b340cf740 2024-01-15T18:08:30.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.151+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b3416b6b0 con 0x7f3b0c066e00 2024-01-15T18:08:30.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.211+0000 7f3b2aa00640 1 -- 172.21.15.143:0/3243205245 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f3b24001c10 con 0x7f3b0c066e00 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:30.219 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:30.220 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:30.220 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:30.220 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:30.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b0c066e00 msgr2=0x7f3b0c0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:30.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b0c066e00 0x7f3b0c0692c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3b240010f0 tx=0x7f3b24009040 comp rx=0 tx=0).stop 2024-01-15T18:08:30.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 msgr2=0x7f3b340c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:30.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 0x7f3b340c5b00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c00c730 tx=0x7f3b2c00cbf0 comp rx=0 tx=0).stop 2024-01-15T18:08:30.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 shutdown_connections 2024-01-15T18:08:30.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b0c066e00 0x7f3b0c0692c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:30.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b340cf740 0x7f3b340c5b00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:30.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b340b0310 0x7f3b340afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:30.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 --2- 172.21.15.143:0/3243205245 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b3406bfb0 0x7f3b3414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:30.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 >> 172.21.15.143:0/3243205245 conn(0x7f3b340d0ed0 msgr2=0x7f3b34154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:30.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 shutdown_connections 2024-01-15T18:08:30.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:30.231+0000 7f3b39c00640 1 -- 172.21.15.143:0/3243205245 wait complete. 2024-01-15T18:08:30.322 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:31.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:31.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.715+0000 7ff0a0c00640 1 Processor -- start 2024-01-15T18:08:31.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.715+0000 7ff0a0c00640 1 -- start start 2024-01-15T18:08:31.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.715+0000 7ff0a0c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff09c14b1e0 0x7ff09c0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:31.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.715+0000 7ff0a0c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c0cfc20 con 0x7ff09c070fd0 2024-01-15T18:08:31.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.715+0000 7ff0a0c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09c162080 con 0x7ff09c0cf7c0 2024-01-15T18:08:31.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.715+0000 7ff0a0c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c161d70 con 0x7ff09c14b1e0 2024-01-15T18:08:31.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.719+0000 7ff099400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7ff09c0cf7c0 0x7ff09c0708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43642/0 (socket says 172.21.15.143:43642) 2024-01-15T18:08:31.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.719+0000 7ff099400640 1 -- 172.21.15.143:0/2431020752 learned_addr learned my addr 172.21.15.143:0/2431020752 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:31.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.719+0000 7ff09a800640 1 --1- 172.21.15.143:0/2431020752 >> v1:172.21.15.143:6789/0 conn(0x7ff09c070fd0 0x7ff09c162510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33082/0 (socket says 172.21.15.143:33082) 2024-01-15T18:08:31.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.719+0000 7ff099e00640 1 --2- 172.21.15.143:0/2431020752 >> v2:172.21.15.204:3300/0 conn(0x7ff09c14b1e0 0x7ff09c0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:31.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.723+0000 7ff098a00640 1 -- 172.21.15.143:0/2431020752 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915576128 0 0) 0x7ff09c162080 con 0x7ff09c0cf7c0 2024-01-15T18:08:31.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.723+0000 7ff098a00640 1 -- 172.21.15.143:0/2431020752 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff07c003530 con 0x7ff09c0cf7c0 2024-01-15T18:08:31.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.723+0000 7ff099e00640 1 -- 172.21.15.143:0/2431020752 >> v1:172.21.15.204:6789/0 conn(0x7ff09c0cf7c0 legacy=0x7ff09c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:31.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.723+0000 7ff099e00640 1 -- 172.21.15.143:0/2431020752 >> v1:172.21.15.143:6789/0 conn(0x7ff09c070fd0 legacy=0x7ff09c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:31.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.723+0000 7ff099e00640 1 -- 172.21.15.143:0/2431020752 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c14b5c0 con 0x7ff09c14b1e0 2024-01-15T18:08:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.731+0000 7ff099e00640 1 --2- 172.21.15.143:0/2431020752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c0cf280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff094009d90 tx=0x7ff09401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a24366cbe5188cf4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.731+0000 7ff098a00640 1 -- 172.21.15.143:0/2431020752 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940a59a0 con 0x7ff09c14b1e0 2024-01-15T18:08:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.731+0000 7ff098a00640 1 -- 172.21.15.143:0/2431020752 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0940a5b00 con 0x7ff09c14b1e0 2024-01-15T18:08:31.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.731+0000 7ff098a00640 1 -- 172.21.15.143:0/2431020752 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940a5e90 con 0x7ff09c14b1e0 2024-01-15T18:08:31.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.731+0000 7ff0a0c00640 1 -- 172.21.15.143:0/2431020752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 msgr2=0x7ff09c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:31.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.731+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/2431020752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c0cf280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff094009d90 tx=0x7ff09401c8e0 comp rx=0 tx=0).stop 2024-01-15T18:08:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.735+0000 7ff0a0c00640 1 -- 172.21.15.143:0/2431020752 shutdown_connections 2024-01-15T18:08:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.735+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/2431020752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c0cf280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:31.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.735+0000 7ff0a0c00640 1 -- 172.21.15.143:0/2431020752 >> 172.21.15.143:0/2431020752 conn(0x7ff09c06a890 msgr2=0x7ff09c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:31.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.735+0000 7ff0a0c00640 1 -- 172.21.15.143:0/2431020752 shutdown_connections 2024-01-15T18:08:31.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.739+0000 7ff0a0c00640 1 -- 172.21.15.143:0/2431020752 wait complete. 2024-01-15T18:08:31.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.755+0000 7ff0a0c00640 1 Processor -- start 2024-01-15T18:08:31.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.755+0000 7ff0a0c00640 1 -- start start 2024-01-15T18:08:31.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff0a0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 0x7ff09c0aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff0a0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0cf7c0 0x7ff09c141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff0a0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff0a0c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0aa160 con 0x7ff09c070fd0 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff0a0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0aa440 con 0x7ff09c14b1e0 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff0a0c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0aa720 con 0x7ff09c0cf7c0 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 0x7ff09c0aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 0x7ff09c0aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46936/0 (socket says 172.21.15.143:46936) 2024-01-15T18:08:31.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 -- 172.21.15.143:0/523324882 learned_addr learned my addr 172.21.15.143:0/523324882 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:31.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff09a800640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:31.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 -- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0cf7c0 msgr2=0x7ff09c141c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:08:31.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0cf7c0 0x7ff09c141c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:31.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 -- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 msgr2=0x7ff09c160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:31.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:31.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.759+0000 7ff099e00640 1 -- 172.21.15.143:0/523324882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff094009a70 con 0x7ff09c070fd0 2024-01-15T18:08:31.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.763+0000 7ff099e00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 0x7ff09c0aaf20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff094019510 tx=0x7ff0940196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:31.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.771+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940adb70 con 0x7ff09c070fd0 2024-01-15T18:08:31.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.771+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0940adcf0 con 0x7ff09c070fd0 2024-01-15T18:08:31.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.771+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0940b8660 con 0x7ff09c070fd0 2024-01-15T18:08:31.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.771+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c0aaa00 con 0x7ff09c070fd0 2024-01-15T18:08:31.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.771+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c142310 con 0x7ff09c070fd0 2024-01-15T18:08:31.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.775+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c0b2c90 con 0x7ff09c070fd0 2024-01-15T18:08:31.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.779+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ff0940b87c0 con 0x7ff09c070fd0 2024-01-15T18:08:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.783+0000 7ff092000640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff074066ed0 0x7ff074069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.783+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff094120850 con 0x7ff09c070fd0 2024-01-15T18:08:31.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.795+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0940ed390 con 0x7ff09c070fd0 2024-01-15T18:08:31.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.795+0000 7ff099400640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff074066ed0 0x7ff074069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:31.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:31.795+0000 7ff099400640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff074066ed0 0x7ff074069390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c10 tx=0x7ff084007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:32.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.375+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7ff09c0aa5b0 con 0x7ff074066ed0 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:32.451 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:32.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.435+0000 7ff092000640 1 -- 172.21.15.143:0/523324882 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7ff09c0aa5b0 con 0x7ff074066ed0 2024-01-15T18:08:32.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.451+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff074066ed0 msgr2=0x7ff074069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:32.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.451+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff074066ed0 0x7ff074069390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff084000c10 tx=0x7ff084007950 comp rx=0 tx=0).stop 2024-01-15T18:08:32.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.451+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 msgr2=0x7ff09c0aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:32.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.451+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 0x7ff09c0aaf20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff094019510 tx=0x7ff0940196a0 comp rx=0 tx=0).stop 2024-01-15T18:08:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.459+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 shutdown_connections 2024-01-15T18:08:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.459+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff074066ed0 0x7ff074069390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:32.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.459+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c14b1e0 0x7ff09c160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.459+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c0cf7c0 0x7ff09c141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.459+0000 7ff0a0c00640 1 --2- 172.21.15.143:0/523324882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c070fd0 0x7ff09c0aaf20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:32.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.459+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 >> 172.21.15.143:0/523324882 conn(0x7ff09c06a890 msgr2=0x7ff09c0b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:32.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.463+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 shutdown_connections 2024-01-15T18:08:32.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:32.463+0000 7ff0a0c00640 1 -- 172.21.15.143:0/523324882 wait complete. 2024-01-15T18:08:32.727 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:33.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:34.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.167+0000 7f2af5400640 1 Processor -- start 2024-01-15T18:08:34.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.167+0000 7f2af5400640 1 -- start start 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.167+0000 7f2af5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2af0071970 0x7f2af0164e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.167+0000 7f2af5400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00ce110 con 0x7f2af00cdc10 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.167+0000 7f2af5400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00ce3f0 con 0x7f2af0159dd0 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.167+0000 7f2af5400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00ced90 con 0x7f2af0071970 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2aefe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2af0159dd0 0x7f2af006ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43658/0 (socket says 172.21.15.143:43658) 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2aefe00640 1 -- 172.21.15.143:0/1925227030 learned_addr learned my addr 172.21.15.143:0/1925227030 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2af4a00640 1 --2- 172.21.15.143:0/1925227030 >> v2:172.21.15.143:3300/0 conn(0x7f2af0071970 0x7f2af0164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2aeea00640 1 -- 172.21.15.143:0/1925227030 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567560942 0 0) 0x7f2af00ce3f0 con 0x7f2af0159dd0 2024-01-15T18:08:34.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2aeea00640 1 -- 172.21.15.143:0/1925227030 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad0003530 con 0x7f2af0159dd0 2024-01-15T18:08:34.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2af4a00640 1 -- 172.21.15.143:0/1925227030 >> v1:172.21.15.204:6789/0 conn(0x7f2af0159dd0 legacy=0x7f2af006ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:34.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2af4a00640 1 -- 172.21.15.143:0/1925227030 >> v1:172.21.15.143:6789/0 conn(0x7f2af00cdc10 legacy=0x7f2af0071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:34.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.171+0000 7f2af4a00640 1 -- 172.21.15.143:0/1925227030 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00ce6d0 con 0x7f2af0071970 2024-01-15T18:08:34.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2af4a00640 1 --2- 172.21.15.143:0/1925227030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0164e80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009d90 tx=0x7f2ae801ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3811f7a3677950 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:34.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2aeea00640 1 -- 172.21.15.143:0/1925227030 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80a7970 con 0x7f2af0071970 2024-01-15T18:08:34.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2aeea00640 1 -- 172.21.15.143:0/1925227030 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae80a7ad0 con 0x7f2af0071970 2024-01-15T18:08:34.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2aeea00640 1 -- 172.21.15.143:0/1925227030 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80a7e60 con 0x7f2af0071970 2024-01-15T18:08:34.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2af5400640 1 -- 172.21.15.143:0/1925227030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 msgr2=0x7f2af0164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:34.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2af5400640 1 --2- 172.21.15.143:0/1925227030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0164e80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009d90 tx=0x7f2ae801ea90 comp rx=0 tx=0).stop 2024-01-15T18:08:34.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2af5400640 1 -- 172.21.15.143:0/1925227030 shutdown_connections 2024-01-15T18:08:34.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2af5400640 1 --2- 172.21.15.143:0/1925227030 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0164e80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:34.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.179+0000 7f2af5400640 1 -- 172.21.15.143:0/1925227030 >> 172.21.15.143:0/1925227030 conn(0x7f2af00cc790 msgr2=0x7f2af00ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:34.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.183+0000 7f2af5400640 1 -- 172.21.15.143:0/1925227030 shutdown_connections 2024-01-15T18:08:34.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.183+0000 7f2af5400640 1 -- 172.21.15.143:0/1925227030 wait complete. 2024-01-15T18:08:34.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.183+0000 7f2af5400640 1 Processor -- start 2024-01-15T18:08:34.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.183+0000 7f2af5400640 1 -- start start 2024-01-15T18:08:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af00cdc10 0x7f2af00b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af0159dd0 0x7f2af01591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0151990 con 0x7f2af0071970 2024-01-15T18:08:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00b5c40 con 0x7f2af00cdc10 2024-01-15T18:08:34.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00b4660 con 0x7f2af0159dd0 2024-01-15T18:08:34.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:34.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46956/0 (socket says 172.21.15.143:46956) 2024-01-15T18:08:34.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 -- 172.21.15.143:0/2222475785 learned_addr learned my addr 172.21.15.143:0/2222475785 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:34.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af4a00640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af0159dd0 0x7f2af01591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:34.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 -- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af0159dd0 msgr2=0x7f2af01591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af0159dd0 0x7f2af01591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 -- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af00cdc10 msgr2=0x7f2af00b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af00cdc10 0x7f2af00b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 -- 172.21.15.143:0/2222475785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8009a70 con 0x7f2af0071970 2024-01-15T18:08:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2aefe00640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0151450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c4d0 tx=0x7f2ad800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:34.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8005cf0 con 0x7f2af0071970 2024-01-15T18:08:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8005e50 con 0x7f2af0071970 2024-01-15T18:08:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8007270 con 0x7f2af0071970 2024-01-15T18:08:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.187+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af00b4940 con 0x7f2af0071970 2024-01-15T18:08:34.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.191+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af00b4f00 con 0x7f2af0071970 2024-01-15T18:08:34.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.191+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af0067500 con 0x7f2af0071970 2024-01-15T18:08:34.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.199+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f2ad80073d0 con 0x7f2af0071970 2024-01-15T18:08:34.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.199+0000 7f2ae7e00640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ac8066ed0 0x7f2ac8069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:34.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.199+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2ad8084670 con 0x7f2af0071970 2024-01-15T18:08:34.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.203+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2ad80512f0 con 0x7f2af0071970 2024-01-15T18:08:34.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.207+0000 7f2aef400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ac8066ed0 0x7f2ac8069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:34.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.207+0000 7f2aef400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ac8066ed0 0x7f2ac8069390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2af00ce040 tx=0x7f2ae00079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:34.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:34.827+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f2af0072140 con 0x7f2ac8066ed0 2024-01-15T18:08:35.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.035+0000 7f2ae7e00640 1 -- 172.21.15.143:0/2222475785 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f2af0072140 con 0x7f2ac8066ed0 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:08:35.042 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:08:35.043 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:08:35.043 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:35.043 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:35.043 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:35.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.043+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ac8066ed0 msgr2=0x7f2ac8069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:35.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.043+0000 7f2af5400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ac8066ed0 0x7f2ac8069390 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2af00ce040 tx=0x7f2ae00079c0 comp rx=0 tx=0).stop 2024-01-15T18:08:35.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.043+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 msgr2=0x7f2af0151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:35.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.043+0000 7f2af5400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0151450 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800c4d0 tx=0x7f2ad800c990 comp rx=0 tx=0).stop 2024-01-15T18:08:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 shutdown_connections 2024-01-15T18:08:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ac8066ed0 0x7f2ac8069390 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af0159dd0 0x7f2af01591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af00cdc10 0x7f2af00b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 --2- 172.21.15.143:0/2222475785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af0071970 0x7f2af0151450 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:35.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 >> 172.21.15.143:0/2222475785 conn(0x7f2af00cc790 msgr2=0x7f2af006b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:35.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 shutdown_connections 2024-01-15T18:08:35.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:35.047+0000 7f2af5400640 1 -- 172.21.15.143:0/2222475785 wait complete. 2024-01-15T18:08:35.127 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:08:36.128 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87831 2024-01-15T18:08:36.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbdece00640 1 Processor -- start 2024-01-15T18:08:36.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbdece00640 1 -- start start 2024-01-15T18:08:36.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbdece00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:36.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbdece00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde806e9e0 con 0x7fbde8161d90 2024-01-15T18:08:36.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbdece00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde80ce8c0 con 0x7fbde8143b90 2024-01-15T18:08:36.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbdece00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde806fe80 con 0x7fbde80cdf60 2024-01-15T18:08:36.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde5e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fbde8143b90 0x7fbde80cc720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43672/0 (socket says 172.21.15.143:43672) 2024-01-15T18:08:36.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde5e00640 1 -- 172.21.15.143:0/1247241529 learned_addr learned my addr 172.21.15.143:0/1247241529 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde5400640 1 --2- 172.21.15.143:0/1247241529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228717371 0 0) 0x7fbde80ce8c0 con 0x7fbde8143b90 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc8005740 con 0x7fbde8143b90 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721406293 0 0) 0x7fbde806e9e0 con 0x7fbde8161d90 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde80ce8c0 con 0x7fbde8161d90 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde5400640 1 -- 172.21.15.143:0/1247241529 >> v1:172.21.15.204:6789/0 conn(0x7fbde8143b90 legacy=0x7fbde80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde5400640 1 -- 172.21.15.143:0/1247241529 >> v1:172.21.15.143:6789/0 conn(0x7fbde8161d90 legacy=0x7fbde8164180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.378+0000 7fbde5400640 1 -- 172.21.15.143:0/1247241529 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80cd1e0 con 0x7fbde80cdf60 2024-01-15T18:08:36.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.382+0000 7fbde5400640 1 --2- 172.21.15.143:0/1247241529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde80ce380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009770 tx=0x7fbdd801eab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80718e59bcad41aa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:36.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0001680 con 0x7fbde80cdf60 2024-01-15T18:08:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80a85f0 con 0x7fbde80cdf60 2024-01-15T18:08:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbde4a00640 1 -- 172.21.15.143:0/1247241529 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80a8980 con 0x7fbde80cdf60 2024-01-15T18:08:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 -- 172.21.15.143:0/1247241529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 msgr2=0x7fbde80ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 --2- 172.21.15.143:0/1247241529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde80ce380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009770 tx=0x7fbdd801eab0 comp rx=0 tx=0).stop 2024-01-15T18:08:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 -- 172.21.15.143:0/1247241529 shutdown_connections 2024-01-15T18:08:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 --2- 172.21.15.143:0/1247241529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde80ce380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 -- 172.21.15.143:0/1247241529 >> 172.21.15.143:0/1247241529 conn(0x7fbde80b1d50 msgr2=0x7fbde80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 -- 172.21.15.143:0/1247241529 shutdown_connections 2024-01-15T18:08:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.386+0000 7fbdece00640 1 -- 172.21.15.143:0/1247241529 wait complete. 2024-01-15T18:08:36.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 Processor -- start 2024-01-15T18:08:36.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 -- start start 2024-01-15T18:08:36.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 0x7fbde80b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbde8161d90 0x7fbde813a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde815c020 con 0x7fbde8161d90 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde815ec00 con 0x7fbde8143b90 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde80afca0 con 0x7fbde80cdf60 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 0x7fbde80b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 0x7fbde80b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:39290/0 (socket says 172.21.15.143:39290) 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 -- 172.21.15.143:0/2379959303 learned_addr learned my addr 172.21.15.143:0/2379959303 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde6800640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbde8161d90 0x7fbde813a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 -- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 msgr2=0x7fbde815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 -- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbde8161d90 msgr2=0x7fbde813a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:36.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbde8161d90 0x7fbde813a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 -- 172.21.15.143:0/2379959303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8009420 con 0x7fbde8143b90 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde6800640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbde8161d90 0x7fbde813a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbde5400640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 0x7fbde80b1450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b3040 tx=0x7fbdd801b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80b0850 con 0x7fbde8143b90 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80b09d0 con 0x7fbde8143b90 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde80aff80 con 0x7fbde8143b90 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80b0d80 con 0x7fbde8143b90 2024-01-15T18:08:36.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.390+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde80b0540 con 0x7fbde8143b90 2024-01-15T18:08:36.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.394+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde8067c10 con 0x7fbde8143b90 2024-01-15T18:08:36.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.394+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fbdd80b7040 con 0x7fbde8143b90 2024-01-15T18:08:36.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.394+0000 7fbdde000640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc0066e00 0x7fbdc00692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:36.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.394+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbdd81221b0 con 0x7fbde8143b90 2024-01-15T18:08:36.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.394+0000 7fbde5e00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc0066e00 0x7fbdc00692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:36.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.394+0000 7fbde5e00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc0066e00 0x7fbdc00692c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00011f0 tx=0x7fbdd0000fa0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:36.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.398+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbdd80eed40 con 0x7fbde8143b90 2024-01-15T18:08:36.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:36.586+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7fbde80cda80 con 0x7fbdc0066e00 2024-01-15T18:08:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdde000640 1 -- 172.21.15.143:0/2379959303 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fbdd0004910 con 0x7fbdc0066e00 2024-01-15T18:08:38.239 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:38.240 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:38.240 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:08:38.240 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:38.240 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc0066e00 msgr2=0x7fbdc00692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc0066e00 0x7fbdc00692c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00011f0 tx=0x7fbdd0000fa0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 msgr2=0x7fbde80b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 0x7fbde80b1450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b3040 tx=0x7fbdd801b850 comp rx=0 tx=0).stop 2024-01-15T18:08:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 shutdown_connections 2024-01-15T18:08:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc0066e00 0x7fbdc00692c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbde8161d90 0x7fbde813a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbde8143b90 0x7fbde80b1450 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 --2- 172.21.15.143:0/2379959303 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbde80cdf60 0x7fbde815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 >> 172.21.15.143:0/2379959303 conn(0x7fbde80b1d50 msgr2=0x7fbde8159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 shutdown_connections 2024-01-15T18:08:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.234+0000 7fbdece00640 1 -- 172.21.15.143:0/2379959303 wait complete. 2024-01-15T18:08:38.268 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:08:38.268 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87832 2024-01-15T18:08:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 Processor -- start 2024-01-15T18:08:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 -- start start 2024-01-15T18:08:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c15b260 0x7fd89c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd89c0d19f0 0x7fd89c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c0c4d90 con 0x7fd89c0c5220 2024-01-15T18:08:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c15b640 con 0x7fd89c0d19f0 2024-01-15T18:08:38.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0cf800 con 0x7fd89c15b260 2024-01-15T18:08:38.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd89c0d19f0 0x7fd89c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c15b260 0x7fd89c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd89c0d19f0 0x7fd89c0cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:37694/0 (socket says 172.21.15.143:37694) 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 -- 172.21.15.143:0/2058553766 learned_addr learned my addr 172.21.15.143:0/2058553766 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 -- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c15b260 msgr2=0x7fd89c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 --2- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c15b260 0x7fd89c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 -- 172.21.15.143:0/2058553766 >> v1:172.21.15.204:6789/0 conn(0x7fd89c0c5220 legacy=0x7fd89c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a0a00640 1 -- 172.21.15.143:0/2058553766 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0cfae0 con 0x7fd89c0d19f0 2024-01-15T18:08:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.406+0000 7fd8a1400640 1 --2- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c15b260 0x7fd89c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:38.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a0a00640 1 --2- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0d19f0 0x7fd89c0cdd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd884009870 tx=0x7fd88401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31af263afd34019 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:38.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd89be00640 1 -- 172.21.15.143:0/2058553766 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840ab620 con 0x7fd89c0d19f0 2024-01-15T18:08:38.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd89be00640 1 -- 172.21.15.143:0/2058553766 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8840ab780 con 0x7fd89c0d19f0 2024-01-15T18:08:38.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd89be00640 1 -- 172.21.15.143:0/2058553766 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8840abb10 con 0x7fd89c0d19f0 2024-01-15T18:08:38.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 -- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0d19f0 msgr2=0x7fd89c0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:38.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0d19f0 0x7fd89c0cdd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd884009870 tx=0x7fd88401c510 comp rx=0 tx=0).stop 2024-01-15T18:08:38.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 -- 172.21.15.143:0/2058553766 shutdown_connections 2024-01-15T18:08:38.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0d19f0 0x7fd89c0cdd50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/2058553766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c15b260 0x7fd89c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 -- 172.21.15.143:0/2058553766 >> 172.21.15.143:0/2058553766 conn(0x7fd89c0d0ed0 msgr2=0x7fd89c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:38.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 -- 172.21.15.143:0/2058553766 shutdown_connections 2024-01-15T18:08:38.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.410+0000 7fd8a3e00640 1 -- 172.21.15.143:0/2058553766 wait complete. 2024-01-15T18:08:38.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 Processor -- start 2024-01-15T18:08:38.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 -- start start 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0c5220 0x7fd89c1f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c0d19f0 0x7fd89c0c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 0x7fd89c158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0adb90 con 0x7fd89c15b260 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0ade70 con 0x7fd89c0c5220 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0ae150 con 0x7fd89c0d19f0 2024-01-15T18:08:38.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 0x7fd89c158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c0d19f0 0x7fd89c0c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 0x7fd89c158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49770/0 (socket says 172.21.15.143:49770) 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 -- 172.21.15.143:0/3258797244 learned_addr learned my addr 172.21.15.143:0/3258797244 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1400640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0c5220 0x7fd89c1f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 -- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c0d19f0 msgr2=0x7fd89c0c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c0d19f0 0x7fd89c0c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 -- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0c5220 msgr2=0x7fd89c1f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0c5220 0x7fd89c1f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 -- 172.21.15.143:0/3258797244 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c009420 con 0x7fd89c15b260 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1400640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0c5220 0x7fd89c1f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a1e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 0x7fd89c158420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd89400c730 tx=0x7fd89400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894003b80 con 0x7fd89c15b260 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd894003ce0 con 0x7fd89c15b260 2024-01-15T18:08:38.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8840094d0 con 0x7fd89c15b260 2024-01-15T18:08:38.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400aa30 con 0x7fd89c15b260 2024-01-15T18:08:38.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.414+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c0ae710 con 0x7fd89c15b260 2024-01-15T18:08:38.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.418+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c067500 con 0x7fd89c15b260 2024-01-15T18:08:38.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.418+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fd89400ab90 con 0x7fd89c15b260 2024-01-15T18:08:38.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.418+0000 7fd899600640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd874066e00 0x7fd8740692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:38.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.418+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd8940849f0 con 0x7fd89c15b260 2024-01-15T18:08:38.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.418+0000 7fd8a1400640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd874066e00 0x7fd8740692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:38.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.418+0000 7fd8a1400640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd874066e00 0x7fd8740692c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd88c011310 tx=0x7fd88c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:38.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.422+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd894051560 con 0x7fd89c15b260 2024-01-15T18:08:38.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:38.606+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87832", "target": ["mon-mgr", ""]}) v1 -- 0x7fd89c15a2c0 con 0x7fd874066e00 2024-01-15T18:08:43.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.234+0000 7fd899600640 1 -- 172.21.15.143:0/3258797244 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fd89c15a2c0 con 0x7fd874066e00 2024-01-15T18:08:43.240 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:43.240 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:43.240 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:08:43.240 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:43.240 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:43.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd874066e00 msgr2=0x7fd8740692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:43.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd874066e00 0x7fd8740692c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd88c011310 tx=0x7fd88c009190 comp rx=0 tx=0).stop 2024-01-15T18:08:43.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 msgr2=0x7fd89c158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:43.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 0x7fd89c158420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd89400c730 tx=0x7fd89400cbf0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 shutdown_connections 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd874066e00 0x7fd8740692c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd89c15b260 0x7fd89c158420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd89c0d19f0 0x7fd89c0c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 --2- 172.21.15.143:0/3258797244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd89c0c5220 0x7fd89c1f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 >> 172.21.15.143:0/3258797244 conn(0x7fd89c0d0ed0 msgr2=0x7fd89c06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 shutdown_connections 2024-01-15T18:08:43.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.238+0000 7fd8a3e00640 1 -- 172.21.15.143:0/3258797244 wait complete. 2024-01-15T18:08:43.266 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:08:43.266 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87833 2024-01-15T18:08:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.402+0000 7f4995200640 1 Processor -- start 2024-01-15T18:08:43.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.402+0000 7f4995200640 1 -- start start 2024-01-15T18:08:43.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f4995200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4990158bc0 0x7f4990071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f4995200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 0x7f499006a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f4995200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499015c4f0 con 0x7f499006f880 2024-01-15T18:08:43.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f4995200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4990158fa0 con 0x7f4990158bc0 2024-01-15T18:08:43.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f4995200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499015b970 con 0x7f499006f3f0 2024-01-15T18:08:43.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498e800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f499006f880 0x7f49900ca940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35116/0 (socket says 172.21.15.143:35116) 2024-01-15T18:08:43.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498e800640 1 -- 172.21.15.143:0/1322329988 learned_addr learned my addr 172.21.15.143:0/1322329988 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:43.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 --2- 172.21.15.143:0/1322329988 >> v2:172.21.15.204:3300/0 conn(0x7f4990158bc0 0x7f4990071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498d400640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 0x7f499006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452723102 0 0) 0x7f499015c4f0 con 0x7f499006f880 2024-01-15T18:08:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4970003530 con 0x7f499006f880 2024-01-15T18:08:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110778420 0 0) 0x7f4970003530 con 0x7f499006f880 2024-01-15T18:08:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 msgr2=0x7f499006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 0x7f499006a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 >> v2:172.21.15.204:3300/0 conn(0x7f4990158bc0 msgr2=0x7f4990071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:43.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 --2- 172.21.15.143:0/1322329988 >> v2:172.21.15.204:3300/0 conn(0x7f4990158bc0 0x7f4990071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499015bc50 con 0x7f499006f880 2024-01-15T18:08:43.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4988001a00 con 0x7f499006f880 2024-01-15T18:08:43.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 >> v1:172.21.15.143:6789/0 conn(0x7f499006f880 legacy=0x7f49900ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:43.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4970003980 0x7f4970003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 0x7f49700047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970004d10 0x7f4970005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990158fa0 con 0x7f4970003980 2024-01-15T18:08:43.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499015b970 con 0x7f4970004d10 2024-01-15T18:08:43.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497000ba40 con 0x7f499006f3f0 2024-01-15T18:08:43.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4970003980 0x7f4970003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498e800640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970004d10 0x7f4970005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498d400640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 0x7f49700047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 -- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 msgr2=0x7f49700047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:43.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f499006f3f0 0x7f49700047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 -- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970004d10 msgr2=0x7f4970005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:43.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970004d10 0x7f4970005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.406+0000 7f498de00640 1 -- 172.21.15.143:0/1322329988 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970003530 con 0x7f4970003980 2024-01-15T18:08:43.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f498de00640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4970003980 0x7f4970003da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4980009ae0 tx=0x7f498001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8eab730cfc58c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:43.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800a5540 con 0x7f4970003980 2024-01-15T18:08:43.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800a56a0 con 0x7f4970003980 2024-01-15T18:08:43.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f498ca00640 1 -- 172.21.15.143:0/1322329988 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800a5a50 con 0x7f4970003980 2024-01-15T18:08:43.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f4995200640 1 -- 172.21.15.143:0/1322329988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4970003980 msgr2=0x7f4970003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:43.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f4995200640 1 --2- 172.21.15.143:0/1322329988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4970003980 0x7f4970003da0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4980009ae0 tx=0x7f498001ca60 comp rx=0 tx=0).stop 2024-01-15T18:08:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f498e800640 1 -- 172.21.15.143:0/1322329988 reap_dead start 2024-01-15T18:08:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f4995200640 1 -- 172.21.15.143:0/1322329988 shutdown_connections 2024-01-15T18:08:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f4995200640 1 -- 172.21.15.143:0/1322329988 >> 172.21.15.143:0/1322329988 conn(0x7f49900cfd20 msgr2=0x7f49900d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f4995200640 1 -- 172.21.15.143:0/1322329988 shutdown_connections 2024-01-15T18:08:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.410+0000 7f4995200640 1 -- 172.21.15.143:0/1322329988 wait complete. 2024-01-15T18:08:43.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 Processor -- start 2024-01-15T18:08:43.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- start start 2024-01-15T18:08:43.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970003980 0x7f4990156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 0x7f49900c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49900b0ad0 0x7f499015fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990156000 con 0x7f4990155780 2024-01-15T18:08:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49901535a0 con 0x7f4970003980 2024-01-15T18:08:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990156dc0 con 0x7f49900b0ad0 2024-01-15T18:08:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 0x7f49900c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 0x7f49900c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49780/0 (socket says 172.21.15.143:49780) 2024-01-15T18:08:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 -- 172.21.15.143:0/160720253 learned_addr learned my addr 172.21.15.143:0/160720253 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:43.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498e800640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49900b0ad0 0x7f499015fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498de00640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970003980 0x7f4990156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 -- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49900b0ad0 msgr2=0x7f499015fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49900b0ad0 0x7f499015fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 -- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970003980 msgr2=0x7f4990156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970003980 0x7f4990156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 -- 172.21.15.143:0/160720253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990155d20 con 0x7f4990155780 2024-01-15T18:08:43.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f498d400640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 0x7f49900c7a70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f49780119d0 tx=0x7f4978011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800faf0 con 0x7f4990155780 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497800fc50 con 0x7f4990155780 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900c6f90 con 0x7f4990155780 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4978014440 con 0x7f4990155780 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900c7550 con 0x7f4990155780 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.414+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49900c6dd0 con 0x7f4990155780 2024-01-15T18:08:43.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.418+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f49780145a0 con 0x7f4990155780 2024-01-15T18:08:43.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.418+0000 7f4986000640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4968066e00 0x7f49680692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:43.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.418+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4978085360 con 0x7f4990155780 2024-01-15T18:08:43.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.418+0000 7f498de00640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4968066e00 0x7f49680692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:43.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.418+0000 7f498de00640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4968066e00 0x7f49680692c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f49800af000 tx=0x7f49800046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:43.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.422+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49780520b0 con 0x7f4990155780 2024-01-15T18:08:43.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:43.602+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87833", "target": ["mon-mgr", ""]}) v1 -- 0x7f4990165f00 con 0x7f4968066e00 2024-01-15T18:08:48.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.234+0000 7f4986000640 1 -- 172.21.15.143:0/160720253 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f4980004040 con 0x7f4968066e00 2024-01-15T18:08:48.240 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:48.240 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:48.240 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:08:48.240 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:48.241 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4968066e00 msgr2=0x7f49680692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4968066e00 0x7f49680692c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f49800af000 tx=0x7f49800046d0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 msgr2=0x7f49900c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 0x7f49900c7a70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f49780119d0 tx=0x7f4978011e90 comp rx=0 tx=0).stop 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 shutdown_connections 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4968066e00 0x7f49680692c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49900b0ad0 0x7f499015fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4990155780 0x7f49900c7a70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 --2- 172.21.15.143:0/160720253 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4970003980 0x7f4990156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 >> 172.21.15.143:0/160720253 conn(0x7f49900cfd20 msgr2=0x7f49901570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 shutdown_connections 2024-01-15T18:08:48.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.238+0000 7f4995200640 1 -- 172.21.15.143:0/160720253 wait complete. 2024-01-15T18:08:48.266 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:08:48.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone87834 2024-01-15T18:08:48.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 Processor -- start 2024-01-15T18:08:48.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 -- start start 2024-01-15T18:08:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f11540cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f11540d19f0 0x7f11540cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f11540c5220 0x7f11540c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11540c4d90 con 0x7f11540c5220 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115415b640 con 0x7f11540d19f0 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f115a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11540cf800 con 0x7f115415b260 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f11540c5220 0x7f11540c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f11540c5220 0x7f11540c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56156/0 (socket says 172.21.15.143:56156) 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 -- 172.21.15.143:0/2666782159 learned_addr learned my addr 172.21.15.143:0/2666782159 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 -- 172.21.15.143:0/2666782159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 msgr2=0x7f11540cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153400640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f11540cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1152a00640 1 --2- 172.21.15.143:0/2666782159 >> v2:172.21.15.204:3300/0 conn(0x7f11540d19f0 0x7f11540cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f11540cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 -- 172.21.15.143:0/2666782159 >> v2:172.21.15.204:3300/0 conn(0x7f11540d19f0 msgr2=0x7f11540cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 --2- 172.21.15.143:0/2666782159 >> v2:172.21.15.204:3300/0 conn(0x7f11540d19f0 0x7f11540cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153e00640 1 -- 172.21.15.143:0/2666782159 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11540cfae0 con 0x7f11540c5220 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1153400640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f11540cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:48.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.406+0000 7f1152a00640 1 --2- 172.21.15.143:0/2666782159 >> v2:172.21.15.204:3300/0 conn(0x7f11540d19f0 0x7f11540cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f1153e00640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11540c56b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1154071670 tx=0x7f114c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d894b74134c6ec15 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f1152000640 1 -- 172.21.15.143:0/2666782159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0a5820 con 0x7f11540c5220 2024-01-15T18:08:48.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f1152000640 1 -- 172.21.15.143:0/2666782159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c0a5980 con 0x7f11540c5220 2024-01-15T18:08:48.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f1152000640 1 -- 172.21.15.143:0/2666782159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0a5d10 con 0x7f11540c5220 2024-01-15T18:08:48.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 -- 172.21.15.143:0/2666782159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 msgr2=0x7f11540c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:48.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11540c56b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1154071670 tx=0x7f114c01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 -- 172.21.15.143:0/2666782159 shutdown_connections 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11540c56b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 --2- 172.21.15.143:0/2666782159 >> v2:172.21.15.204:3300/0 conn(0x7f11540d19f0 0x7f11540cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 --2- 172.21.15.143:0/2666782159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f11540cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 -- 172.21.15.143:0/2666782159 >> 172.21.15.143:0/2666782159 conn(0x7f11540d0ed0 msgr2=0x7f11540d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 -- 172.21.15.143:0/2666782159 shutdown_connections 2024-01-15T18:08:48.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.410+0000 7f115a800640 1 -- 172.21.15.143:0/2666782159 wait complete. 2024-01-15T18:08:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 Processor -- start 2024-01-15T18:08:48.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 -- start start 2024-01-15T18:08:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11541528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11540d19f0 0x7f115409ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f115409b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115406d030 con 0x7f11540c5220 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11541534a0 con 0x7f11540d19f0 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f115a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115406c4b0 con 0x7f115415b260 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11541528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11541528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56160/0 (socket says 172.21.15.143:56160) 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 -- 172.21.15.143:0/4073103940 learned_addr learned my addr 172.21.15.143:0/4073103940 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1152a00640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11540d19f0 0x7f115409ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153e00640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f115409b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 -- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 msgr2=0x7f115409b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f115409b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 -- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11540d19f0 msgr2=0x7f115409ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11540d19f0 0x7f115409ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 -- 172.21.15.143:0/4073103940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c009b30 con 0x7f11540c5220 2024-01-15T18:08:48.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1152a00640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11540d19f0 0x7f115409ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153e00640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f115409b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f1153400640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11541528c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f114400c7f0 tx=0x7f114400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.414+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144007820 con 0x7f11540c5220 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1144007980 con 0x7f11540c5220 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144007d30 con 0x7f11540c5220 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115406c790 con 0x7f11540c5220 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115406cd50 con 0x7f11540c5220 2024-01-15T18:08:48.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1154067500 con 0x7f11540c5220 2024-01-15T18:08:48.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f11440113f0 con 0x7f11540c5220 2024-01-15T18:08:48.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f114b400640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f112c066e00 0x7f112c0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:48.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1144085100 con 0x7f11540c5220 2024-01-15T18:08:48.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f1152a00640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f112c066e00 0x7f112c0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:48.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.418+0000 7f1152a00640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f112c066e00 0x7f112c0692c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f113c006fd0 tx=0x7f113c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:48.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.422+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f11440b6280 con 0x7f11540c5220 2024-01-15T18:08:48.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:48.602+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone87834", "target": ["mon-mgr", ""]}) v1 -- 0x7f11540c4140 con 0x7f112c066e00 2024-01-15T18:08:53.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.230+0000 7f114b400640 1 -- 172.21.15.143:0/4073103940 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f11540c4140 con 0x7f112c066e00 2024-01-15T18:08:53.236 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:08:53.236 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:08:53.236 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:08:53.237 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:53.237 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f112c066e00 msgr2=0x7f112c0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f112c066e00 0x7f112c0692c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f113c006fd0 tx=0x7f113c008040 comp rx=0 tx=0).stop 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 msgr2=0x7f11541528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11541528c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f114400c7f0 tx=0x7f114400ccb0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 shutdown_connections 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f112c066e00 0x7f112c0692c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f115415b260 0x7f115409b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11540d19f0 0x7f115409ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 --2- 172.21.15.143:0/4073103940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11540c5220 0x7f11541528c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 >> 172.21.15.143:0/4073103940 conn(0x7f11540d0ed0 msgr2=0x7f11540c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:53.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 shutdown_connections 2024-01-15T18:08:53.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.234+0000 7f115a800640 1 -- 172.21.15.143:0/4073103940 wait complete. 2024-01-15T18:08:53.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:08:53.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.398+0000 7fa059c00640 1 Processor -- start 2024-01-15T18:08:53.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.398+0000 7fa059c00640 1 -- start start 2024-01-15T18:08:53.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa059c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa059c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0540b4ac0 0x7fa054070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa059c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540b2a20 con 0x7fa0540b21d0 2024-01-15T18:08:53.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa059c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540b35a0 con 0x7fa0540b4ac0 2024-01-15T18:08:53.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa059c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540b2d00 con 0x7fa054157300 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0540b4ac0 0x7fa054070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa052a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0540b4ac0 0x7fa054070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:57930/0 (socket says 172.21.15.143:57930) 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 -- 172.21.15.143:0/1219010816 learned_addr learned my addr 172.21.15.143:0/1219010816 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 -- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 msgr2=0x7fa054159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 --2- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 -- 172.21.15.143:0/1219010816 >> v1:172.21.15.204:6789/0 conn(0x7fa0540b21d0 legacy=0x7fa0540b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa053400640 1 -- 172.21.15.143:0/1219010816 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540b2fe0 con 0x7fa0540b4ac0 2024-01-15T18:08:53.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.402+0000 7fa052a00640 1 --2- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:53.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa053400640 1 --2- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b4ac0 0x7fa054070e80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00af60 tx=0x7fa04c01dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc3aa9002d738e09 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa051600640 1 -- 172.21.15.143:0/1219010816 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00a2d0 con 0x7fa0540b4ac0 2024-01-15T18:08:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa051600640 1 -- 172.21.15.143:0/1219010816 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c0a69e0 con 0x7fa0540b4ac0 2024-01-15T18:08:53.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa051600640 1 -- 172.21.15.143:0/1219010816 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0a6d70 con 0x7fa0540b4ac0 2024-01-15T18:08:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 -- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b4ac0 msgr2=0x7fa054070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 --2- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b4ac0 0x7fa054070e80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00af60 tx=0x7fa04c01dae0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 -- 172.21.15.143:0/1219010816 shutdown_connections 2024-01-15T18:08:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 --2- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b4ac0 0x7fa054070e80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 --2- 172.21.15.143:0/1219010816 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 -- 172.21.15.143:0/1219010816 >> 172.21.15.143:0/1219010816 conn(0x7fa05406c3f0 msgr2=0x7fa05406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 -- 172.21.15.143:0/1219010816 shutdown_connections 2024-01-15T18:08:53.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 -- 172.21.15.143:0/1219010816 wait complete. 2024-01-15T18:08:53.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 Processor -- start 2024-01-15T18:08:53.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.406+0000 7fa059c00640 1 -- start start 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b21d0 0x7fa054158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 0x7fa054154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054158970 con 0x7fa0540b4ac0 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05414d220 con 0x7fa0540b21d0 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa054158320 con 0x7fa054157300 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 0x7fa054154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 0x7fa054154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56162/0 (socket says 172.21.15.143:56162) 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 -- 172.21.15.143:0/3195888944 learned_addr learned my addr 172.21.15.143:0/3195888944 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:53.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052a00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b21d0 0x7fa054158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa053400640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 -- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 msgr2=0x7fa054157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 -- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b21d0 msgr2=0x7fa054158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b21d0 0x7fa054158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 -- 172.21.15.143:0/3195888944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044009420 con 0x7fa0540b4ac0 2024-01-15T18:08:53.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa052000640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 0x7fa054154bc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00c420 tx=0x7fa03c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0052d0 con 0x7fa0540b4ac0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c005430 con 0x7fa0540b4ac0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c00ac40 con 0x7fa0540b4ac0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c007260 con 0x7fa0540b4ac0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054155100 con 0x7fa0540b4ac0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.410+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0540cdf10 con 0x7fa0540b4ac0 2024-01-15T18:08:53.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.414+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa03c0073c0 con 0x7fa0540b4ac0 2024-01-15T18:08:53.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.414+0000 7fa04aa00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c066e00 0x7fa02c0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.414+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa03c084050 con 0x7fa0540b4ac0 2024-01-15T18:08:53.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.414+0000 7fa052a00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c066e00 0x7fa02c0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.414+0000 7fa052a00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c066e00 0x7fa02c0692c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa044009c50 tx=0x7fa044009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:53.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.418+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa03c050c50 con 0x7fa0540b4ac0 2024-01-15T18:08:53.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.586+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0541587d0 con 0x7fa02c066e00 2024-01-15T18:08:53.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.586+0000 7fa04aa00640 1 -- 172.21.15.143:0/3195888944 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa0440098a0 con 0x7fa02c066e00 2024-01-15T18:08:53.593 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:08:53.593 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:53.593 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:08:53.593 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:53.593 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:08:53.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c066e00 msgr2=0x7fa02c0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c066e00 0x7fa02c0692c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa044009c50 tx=0x7fa044009190 comp rx=0 tx=0).stop 2024-01-15T18:08:53.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 msgr2=0x7fa054154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 0x7fa054154bc0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00c420 tx=0x7fa03c00c8e0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 shutdown_connections 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c066e00 0x7fa02c0692c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054157300 0x7fa054157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0540b4ac0 0x7fa054154bc0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 --2- 172.21.15.143:0/3195888944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0540b21d0 0x7fa054158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 >> 172.21.15.143:0/3195888944 conn(0x7fa05406c3f0 msgr2=0x7fa054156990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 shutdown_connections 2024-01-15T18:08:53.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.590+0000 7fa059c00640 1 -- 172.21.15.143:0/3195888944 wait complete. 2024-01-15T18:08:53.616 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:08:53.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 Processor -- start 2024-01-15T18:08:53.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 -- start start 2024-01-15T18:08:53.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd806aba0 0x7f9bd815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9bd815de90 0x7f9bd80ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd80ae830 con 0x7f9bd80ae450 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80ad900 con 0x7f9bd815de90 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdf000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd815e6c0 con 0x7f9bd806aba0 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f9bd80ae450 0x7f9bd806a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:50862/0 (socket says 172.21.15.143:50862) 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7e00640 1 -- 172.21.15.143:0/1051776605 learned_addr learned my addr 172.21.15.143:0/1051776605 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7400640 1 --2- 172.21.15.143:0/1051776605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd806aba0 0x7f9bd815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdd000640 1 --2- 172.21.15.143:0/1051776605 >> v2:172.21.15.204:3300/0 conn(0x7f9bd815de90 0x7f9bd80ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd6a00640 1 -- 172.21.15.143:0/1051776605 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763229857 0 0) 0x7f9bd80ae830 con 0x7f9bd80ae450 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd6a00640 1 -- 172.21.15.143:0/1051776605 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb8003530 con 0x7f9bd80ae450 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7400640 1 -- 172.21.15.143:0/1051776605 >> v2:172.21.15.204:3300/0 conn(0x7f9bd815de90 msgr2=0x7f9bd80ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7400640 1 --2- 172.21.15.143:0/1051776605 >> v2:172.21.15.204:3300/0 conn(0x7f9bd815de90 0x7f9bd80ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7400640 1 -- 172.21.15.143:0/1051776605 >> v1:172.21.15.204:6789/0 conn(0x7f9bd80ae450 legacy=0x7f9bd806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bd7400640 1 -- 172.21.15.143:0/1051776605 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8160e20 con 0x7f9bd806aba0 2024-01-15T18:08:53.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.750+0000 7f9bdd000640 1 --2- 172.21.15.143:0/1051776605 >> v2:172.21.15.204:3300/0 conn(0x7f9bd815de90 0x7f9bd80ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:53.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bd7400640 1 --2- 172.21.15.143:0/1051776605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd806aba0 0x7f9bd815d950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00097c0 tx=0x7f9bc001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea7de0223217f27e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:53.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bd6a00640 1 -- 172.21.15.143:0/1051776605 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00ab650 con 0x7f9bd806aba0 2024-01-15T18:08:53.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bd6a00640 1 -- 172.21.15.143:0/1051776605 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc00ab7b0 con 0x7f9bd806aba0 2024-01-15T18:08:53.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bd6a00640 1 -- 172.21.15.143:0/1051776605 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc00abb40 con 0x7f9bd806aba0 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 -- 172.21.15.143:0/1051776605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd806aba0 msgr2=0x7f9bd815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 --2- 172.21.15.143:0/1051776605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd806aba0 0x7f9bd815d950 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00097c0 tx=0x7f9bc001c450 comp rx=0 tx=0).stop 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 -- 172.21.15.143:0/1051776605 shutdown_connections 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 --2- 172.21.15.143:0/1051776605 >> v2:172.21.15.204:3300/0 conn(0x7f9bd815de90 0x7f9bd80ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 --2- 172.21.15.143:0/1051776605 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd806aba0 0x7f9bd815d950 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 -- 172.21.15.143:0/1051776605 >> 172.21.15.143:0/1051776605 conn(0x7f9bd80cfd20 msgr2=0x7f9bd80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 -- 172.21.15.143:0/1051776605 shutdown_connections 2024-01-15T18:08:53.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.754+0000 7f9bdf000640 1 -- 172.21.15.143:0/1051776605 wait complete. 2024-01-15T18:08:53.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 Processor -- start 2024-01-15T18:08:53.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 -- start start 2024-01-15T18:08:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 0x7f9bd81524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd80ae450 0x7f9bd8152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd815de90 0x7f9bd8154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81538b0 con 0x7f9bd806aba0 2024-01-15T18:08:53.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd8153010 con 0x7f9bd815de90 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81532f0 con 0x7f9bd80ae450 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 0x7f9bd81524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 0x7f9bd81524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56170/0 (socket says 172.21.15.143:56170) 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 -- 172.21.15.143:0/3645944687 learned_addr learned my addr 172.21.15.143:0/3645944687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7400640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd80ae450 0x7f9bd8152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdd000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd815de90 0x7f9bd8154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 -- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd80ae450 msgr2=0x7f9bd8152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd80ae450 0x7f9bd8152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 -- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd815de90 msgr2=0x7f9bd8154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd815de90 0x7f9bd8154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 -- 172.21.15.143:0/3645944687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd0009a70 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bd7e00640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 0x7f9bd81524c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800c4d0 tx=0x7f9bc800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005cf0 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8005e50 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80072d0 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.758+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc0009420 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd8155920 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd8067500 con 0x7f9bd806aba0 2024-01-15T18:08:53.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f9bc8007430 con 0x7f9bd806aba0 2024-01-15T18:08:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bcfe00640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bb0066e00 0x7f9bb00692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9bc80842b0 con 0x7f9bd806aba0 2024-01-15T18:08:53.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bd7400640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bb0066e00 0x7f9bb00692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:53.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.762+0000 7f9bd7400640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bb0066e00 0x7f9bb00692c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00af000 tx=0x7f9bc00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:53.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.766+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9bc8050e20 con 0x7f9bd806aba0 2024-01-15T18:08:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.942+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bd80ada40 con 0x7f9bb0066e00 2024-01-15T18:08:53.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.946+0000 7f9bcfe00640 1 -- 172.21.15.143:0/3645944687 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+234 (secure 0 0 0) 0x7f9bd80ada40 con 0x7f9bb0066e00 2024-01-15T18:08:53.951 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone87834" 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone87832" 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:53.952 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone87833" 2024-01-15T18:08:53.953 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:08:53.953 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:53.953 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone87831" 2024-01-15T18:08:53.953 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:53.953 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.946+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bb0066e00 msgr2=0x7f9bb00692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.946+0000 7f9bdf000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bb0066e00 0x7f9bb00692c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00af000 tx=0x7f9bc00a7000 comp rx=0 tx=0).stop 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 msgr2=0x7f9bd81524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 0x7f9bd81524c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800c4d0 tx=0x7f9bc800c8d0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 shutdown_connections 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bb0066e00 0x7f9bb00692c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bd815de90 0x7f9bd8154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bd80ae450 0x7f9bd8152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 --2- 172.21.15.143:0/3645944687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bd806aba0 0x7f9bd81524c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 >> 172.21.15.143:0/3645944687 conn(0x7f9bd80cfd20 msgr2=0x7f9bd80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:53.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 shutdown_connections 2024-01-15T18:08:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:53.950+0000 7f9bdf000640 1 -- 172.21.15.143:0/3645944687 wait complete. 2024-01-15T18:08:53.974 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:08:54.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 Processor -- start 2024-01-15T18:08:54.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 -- start start 2024-01-15T18:08:54.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f91a01607c0 0x7f91a006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91a00cbf50 0x7f91a00cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a00cffd0 con 0x7f91a00cfb30 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00d1ce0 con 0x7f91a00cbf50 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f91a5600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a00d02b0 con 0x7f91a01607c0 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91a00cbf50 0x7f91a00cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91a00cbf50 0x7f91a00cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56174/0 (socket says 172.21.15.143:56174) 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 -- 172.21.15.143:0/1165199897 learned_addr learned my addr 172.21.15.143:0/1165199897 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919e000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f91a00cfb30 0x7f91a006bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42296/0 (socket says 172.21.15.143:42296) 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919ea00640 1 --2- 172.21.15.143:0/1165199897 >> v2:172.21.15.204:3300/0 conn(0x7f91a01607c0 0x7f91a006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 -- 172.21.15.143:0/1165199897 >> v2:172.21.15.204:3300/0 conn(0x7f91a01607c0 msgr2=0x7f91a006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 --2- 172.21.15.143:0/1165199897 >> v2:172.21.15.204:3300/0 conn(0x7f91a01607c0 0x7f91a006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:54.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 -- 172.21.15.143:0/1165199897 >> v1:172.21.15.143:6789/0 conn(0x7f91a00cfb30 legacy=0x7f91a006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:54.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.110+0000 7f919f400640 1 -- 172.21.15.143:0/1165199897 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00cb470 con 0x7f91a00cbf50 2024-01-15T18:08:54.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f919f400640 1 --2- 172.21.15.143:0/1165199897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a00cc3c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f91a0071a00 tx=0x7f919801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8318ddaf9ea3353f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:54.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f919d600640 1 -- 172.21.15.143:0/1165199897 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980a5750 con 0x7f91a00cbf50 2024-01-15T18:08:54.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f919d600640 1 -- 172.21.15.143:0/1165199897 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91980a58b0 con 0x7f91a00cbf50 2024-01-15T18:08:54.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f919d600640 1 -- 172.21.15.143:0/1165199897 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980a5c40 con 0x7f91a00cbf50 2024-01-15T18:08:54.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 -- 172.21.15.143:0/1165199897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 msgr2=0x7f91a00cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 --2- 172.21.15.143:0/1165199897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a00cc3c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f91a0071a00 tx=0x7f919801c8f0 comp rx=0 tx=0).stop 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 -- 172.21.15.143:0/1165199897 shutdown_connections 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 --2- 172.21.15.143:0/1165199897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a00cc3c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 --2- 172.21.15.143:0/1165199897 >> v2:172.21.15.204:3300/0 conn(0x7f91a01607c0 0x7f91a006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 -- 172.21.15.143:0/1165199897 >> 172.21.15.143:0/1165199897 conn(0x7f91a00d0e20 msgr2=0x7f91a00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 -- 172.21.15.143:0/1165199897 shutdown_connections 2024-01-15T18:08:54.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.114+0000 7f91a5600640 1 -- 172.21.15.143:0/1165199897 wait complete. 2024-01-15T18:08:54.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 Processor -- start 2024-01-15T18:08:54.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- start start 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a01587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a00cfb30 0x7f91a00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91a01607c0 0x7f91a01f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00c9cf0 con 0x7f91a00cbf50 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00c90e0 con 0x7f91a00cfb30 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00c93c0 con 0x7f91a01607c0 2024-01-15T18:08:54.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a01587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a01587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56182/0 (socket says 172.21.15.143:56182) 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 -- 172.21.15.143:0/178442771 learned_addr learned my addr 172.21.15.143:0/178442771 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919e000640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a00cfb30 0x7f91a00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919f400640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91a01607c0 0x7f91a01f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 -- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91a01607c0 msgr2=0x7f91a01f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91a01607c0 0x7f91a01f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 -- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a00cfb30 msgr2=0x7f91a00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a00cfb30 0x7f91a00c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 -- 172.21.15.143:0/178442771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9198009b30 con 0x7f91a00cbf50 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f919ea00640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a01587e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f919000e870 tx=0x7f919000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190009510 con 0x7f91a00cbf50 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919000eec0 con 0x7f91a00cbf50 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000cb40 con 0x7f91a00cbf50 2024-01-15T18:08:54.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a00c96a0 con 0x7f91a00cbf50 2024-01-15T18:08:54.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a00c6b80 con 0x7f91a00cbf50 2024-01-15T18:08:54.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.118+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a006d4b0 con 0x7f91a00cbf50 2024-01-15T18:08:54.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.122+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f919000ccc0 con 0x7f91a00cbf50 2024-01-15T18:08:54.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.122+0000 7f9196a00640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9178066e00 0x7f91780692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:54.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.122+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f91900850b0 con 0x7f91a00cbf50 2024-01-15T18:08:54.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.122+0000 7f919e000640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9178066e00 0x7f91780692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:54.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.122+0000 7f919e000640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9178066e00 0x7f91780692c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f91a0158d20 tx=0x7f918800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:54.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.126+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91900b6280 con 0x7f91a00cbf50 2024-01-15T18:08:54.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:54.302+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91a00ce6b0 con 0x7f9178066e00 2024-01-15T18:08:57.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:57.883 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.234+0000 7f9196a00640 1 -- 172.21.15.143:0/178442771 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9188016450 con 0x7f9178066e00 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:08:58.239 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.234+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9178066e00 msgr2=0x7f91780692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.234+0000 7f91a5600640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9178066e00 0x7f91780692c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f91a0158d20 tx=0x7f918800b040 comp rx=0 tx=0).stop 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.234+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 msgr2=0x7f91a01587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.234+0000 7f91a5600640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a01587e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f919000e870 tx=0x7f919000ec70 comp rx=0 tx=0).stop 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 shutdown_connections 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9178066e00 0x7f91780692c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91a01607c0 0x7f91a01f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91a00cfb30 0x7f91a00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 --2- 172.21.15.143:0/178442771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91a00cbf50 0x7f91a01587e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 >> 172.21.15.143:0/178442771 conn(0x7f91a00d0e20 msgr2=0x7f91a006c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:58.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 shutdown_connections 2024-01-15T18:08:58.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.238+0000 7f91a5600640 1 -- 172.21.15.143:0/178442771 wait complete. 2024-01-15T18:08:58.266 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:08:58.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 Processor -- start 2024-01-15T18:08:58.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 -- start start 2024-01-15T18:08:58.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa8dc06b930 0x7fa8dc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8dc0b09a0 0x7fa8dc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc15ea90 con 0x7fa8dc06bd10 2024-01-15T18:08:58.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0cce10 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8e1400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0aee50 con 0x7fa8dc06b930 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8db200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8dc0b09a0 0x7fa8dc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8db200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa8dc0b09a0 0x7fa8dc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58002/0 (socket says 172.21.15.143:58002) 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8db200640 1 -- 172.21.15.143:0/613024181 learned_addr learned my addr 172.21.15.143:0/613024181 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.406+0000 7fa8da800640 1 --2- 172.21.15.143:0/613024181 >> v2:172.21.15.204:3300/0 conn(0x7fa8dc06b930 0x7fa8dc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8db200640 1 -- 172.21.15.143:0/613024181 >> v2:172.21.15.204:3300/0 conn(0x7fa8dc06b930 msgr2=0x7fa8dc0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8db200640 1 --2- 172.21.15.143:0/613024181 >> v2:172.21.15.204:3300/0 conn(0x7fa8dc06b930 0x7fa8dc0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8db200640 1 -- 172.21.15.143:0/613024181 >> v1:172.21.15.204:6789/0 conn(0x7fa8dc06bd10 legacy=0x7fa8dc0b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8db200640 1 -- 172.21.15.143:0/613024181 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc15e160 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8db200640 1 --2- 172.21.15.143:0/613024181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0cd3f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0b0fd0 tx=0x7fa8d401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512c63172d8ac382 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:58.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8d9400640 1 -- 172.21.15.143:0/613024181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a5770 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8d9400640 1 -- 172.21.15.143:0/613024181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40a58d0 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8d9400640 1 -- 172.21.15.143:0/613024181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40a5c60 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8e1400640 1 -- 172.21.15.143:0/613024181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 msgr2=0x7fa8dc0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.410+0000 7fa8e1400640 1 --2- 172.21.15.143:0/613024181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0cd3f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0b0fd0 tx=0x7fa8d401c9a0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- 172.21.15.143:0/613024181 shutdown_connections 2024-01-15T18:08:58.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 --2- 172.21.15.143:0/613024181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0cd3f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 --2- 172.21.15.143:0/613024181 >> v2:172.21.15.204:3300/0 conn(0x7fa8dc06b930 0x7fa8dc0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- 172.21.15.143:0/613024181 >> 172.21.15.143:0/613024181 conn(0x7fa8dc071cb0 msgr2=0x7fa8dc072090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:58.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- 172.21.15.143:0/613024181 shutdown_connections 2024-01-15T18:08:58.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- 172.21.15.143:0/613024181 wait complete. 2024-01-15T18:08:58.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 Processor -- start 2024-01-15T18:08:58.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- start start 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8dc06b930 0x7fa8dc0acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc06bd10 0x7fa8dc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0c3d40 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0c4300 con 0x7fa8dc06bd10 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8e1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc155580 con 0x7fa8dc06b930 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58010/0 (socket says 172.21.15.143:58010) 2024-01-15T18:08:58.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 -- 172.21.15.143:0/2714890573 learned_addr learned my addr 172.21.15.143:0/2714890573 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8da800640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8dc06b930 0x7fa8dc0acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8d9e00640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc06bd10 0x7fa8dc06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 -- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8dc06b930 msgr2=0x7fa8dc0acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8dc06b930 0x7fa8dc0acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 -- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc06bd10 msgr2=0x7fa8dc06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc06bd10 0x7fa8dc06cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.414+0000 7fa8db200640 1 -- 172.21.15.143:0/2714890573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d4009b30 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8db200640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0c4790 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40af040 tx=0x7fa8d4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40b8670 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d40b87d0 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc164d60 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40b8b80 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc0c3f90 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc067500 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fa8d40b8ce0 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8d2a00640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8b4066e00 0x7fa8b40692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa8d4120260 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.418+0000 7fa8da800640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8b4066e00 0x7fa8b40692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.422+0000 7fa8da800640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8b4066e00 0x7fa8b40692c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009ac0 tx=0x7fa8cc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:58.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.422+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa8d40ecef0 con 0x7fa8dc0b09a0 2024-01-15T18:08:58.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.602+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8dc0668e0 con 0x7fa8b4066e00 2024-01-15T18:08:58.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.626+0000 7fa8d2a00640 1 -- 172.21.15.143:0/2714890573 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa8cc0119e0 con 0x7fa8b4066e00 2024-01-15T18:08:58.633 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8b4066e00 msgr2=0x7fa8b40692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8b4066e00 0x7fa8b40692c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009ac0 tx=0x7fa8cc009190 comp rx=0 tx=0).stop 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 msgr2=0x7fa8dc0c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0c4790 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8d40af040 tx=0x7fa8d4019740 comp rx=0 tx=0).stop 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 shutdown_connections 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa8b4066e00 0x7fa8b40692c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa8dc0b09a0 0x7fa8dc0c4790 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8dc06bd10 0x7fa8dc06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 --2- 172.21.15.143:0/2714890573 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8dc06b930 0x7fa8dc0acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 >> 172.21.15.143:0/2714890573 conn(0x7fa8dc071cb0 msgr2=0x7fa8dc06e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 shutdown_connections 2024-01-15T18:08:58.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.630+0000 7fa8e1400640 1 -- 172.21.15.143:0/2714890573 wait complete. 2024-01-15T18:08:58.655 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:08:58.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 Processor -- start 2024-01-15T18:08:58.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 -- start start 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d141607c0 0x7f6d1406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d140cfb30 0x7f6d1406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d1406c230 0x7f6d140cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1406c670 con 0x7f6d140cfb30 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140d1ce0 con 0x7f6d1406c230 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d140cffd0 con 0x7f6d141607c0 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d140cfb30 0x7f6d1406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d140cfb30 0x7f6d1406bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58024/0 (socket says 172.21.15.143:58024) 2024-01-15T18:08:58.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 -- 172.21.15.143:0/2174324381 learned_addr learned my addr 172.21.15.143:0/2174324381 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d1a600640 1 --2- 172.21.15.143:0/2174324381 >> v2:172.21.15.204:3300/0 conn(0x7f6d1406c230 0x7f6d140cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19c00640 1 --2- 172.21.15.143:0/2174324381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d141607c0 0x7f6d1406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 -- 172.21.15.143:0/2174324381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d141607c0 msgr2=0x7f6d1406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 --2- 172.21.15.143:0/2174324381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d141607c0 0x7f6d1406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 -- 172.21.15.143:0/2174324381 >> v2:172.21.15.204:3300/0 conn(0x7f6d1406c230 msgr2=0x7f6d140cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 --2- 172.21.15.143:0/2174324381 >> v2:172.21.15.204:3300/0 conn(0x7f6d1406c230 0x7f6d140cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.790+0000 7f6d19200640 1 -- 172.21.15.143:0/2174324381 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d140d02b0 con 0x7f6d140cfb30 2024-01-15T18:08:58.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d19200640 1 --2- 172.21.15.143:0/2174324381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d140cfb30 0x7f6d1406bcf0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6d140d0590 tx=0x7f6cfc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff358527a89fcd7b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d13e00640 1 -- 172.21.15.143:0/2174324381 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a89b0 con 0x7f6d140cfb30 2024-01-15T18:08:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d13e00640 1 -- 172.21.15.143:0/2174324381 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc0a8b10 con 0x7f6d140cfb30 2024-01-15T18:08:58.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d13e00640 1 -- 172.21.15.143:0/2174324381 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a8ea0 con 0x7f6d140cfb30 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 -- 172.21.15.143:0/2174324381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d140cfb30 msgr2=0x7f6d1406bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 --2- 172.21.15.143:0/2174324381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d140cfb30 0x7f6d1406bcf0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6d140d0590 tx=0x7f6cfc01ecf0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 -- 172.21.15.143:0/2174324381 shutdown_connections 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 --2- 172.21.15.143:0/2174324381 >> v2:172.21.15.204:3300/0 conn(0x7f6d1406c230 0x7f6d140cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 --2- 172.21.15.143:0/2174324381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d140cfb30 0x7f6d1406bcf0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 --2- 172.21.15.143:0/2174324381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d141607c0 0x7f6d1406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 -- 172.21.15.143:0/2174324381 >> 172.21.15.143:0/2174324381 conn(0x7f6d140d0e20 msgr2=0x7f6d140d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 -- 172.21.15.143:0/2174324381 shutdown_connections 2024-01-15T18:08:58.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.794+0000 7f6d1c600640 1 -- 172.21.15.143:0/2174324381 wait complete. 2024-01-15T18:08:58.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 Processor -- start 2024-01-15T18:08:58.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 -- start start 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1406c230 0x7f6d140ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d140cfb30 0x7f6d14153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 0x7f6d14154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d140c8f40 con 0x7f6d141607c0 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14153740 con 0x7f6d1406c230 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d140c9d60 con 0x7f6d140cfb30 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 0x7f6d14154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d140cfb30 0x7f6d14153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1406c230 0x7f6d140ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 0x7f6d14154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58034/0 (socket says 172.21.15.143:58034) 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1a600640 1 -- 172.21.15.143:0/419200017 learned_addr learned my addr 172.21.15.143:0/419200017 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 -- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1406c230 msgr2=0x7f6d140ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1406c230 0x7f6d140ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 -- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 msgr2=0x7f6d14154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 0x7f6d14154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 -- 172.21.15.143:0/419200017 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc0094d0 con 0x7f6d140cfb30 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19c00640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1406c230 0x7f6d140ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1a600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 0x7f6d14154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d19200640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d140cfb30 0x7f6d14153ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0af000 tx=0x7f6cfc01ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0bf070 con 0x7f6d140cfb30 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc0b1bb0 con 0x7f6d140cfb30 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d140c9150 con 0x7f6d140cfb30 2024-01-15T18:08:58.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0b9410 con 0x7f6d140cfb30 2024-01-15T18:08:58.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.798+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d140c9710 con 0x7f6d140cfb30 2024-01-15T18:08:58.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.802+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1406d4b0 con 0x7f6d140cfb30 2024-01-15T18:08:58.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.802+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6cfc0b7040 con 0x7f6d140cfb30 2024-01-15T18:08:58.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.802+0000 7f6d11600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cec066e00 0x7f6cec0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:58.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.802+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6cfc121cc0 con 0x7f6d140cfb30 2024-01-15T18:08:58.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.802+0000 7f6d19c00640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cec066e00 0x7f6cec0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:58.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.802+0000 7f6d19c00640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cec066e00 0x7f6cec0692c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009eb0 tx=0x7f6d04009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:58.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.806+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6cfc0ee990 con 0x7f6d140cfb30 2024-01-15T18:08:58.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.978+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d140cba30 con 0x7f6cec066e00 2024-01-15T18:08:59.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.994+0000 7f6d11600640 1 -- 172.21.15.143:0/419200017 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6d04011dd0 con 0x7f6cec066e00 2024-01-15T18:08:59.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cec066e00 msgr2=0x7f6cec0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cec066e00 0x7f6cec0692c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009eb0 tx=0x7f6d04009190 comp rx=0 tx=0).stop 2024-01-15T18:08:59.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d140cfb30 msgr2=0x7f6d14153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d140cfb30 0x7f6d14153ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0af000 tx=0x7f6cfc01ba90 comp rx=0 tx=0).stop 2024-01-15T18:08:59.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 shutdown_connections 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cec066e00 0x7f6cec0692c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d141607c0 0x7f6d14154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d140cfb30 0x7f6d14153ee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 --2- 172.21.15.143:0/419200017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1406c230 0x7f6d140ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 >> 172.21.15.143:0/419200017 conn(0x7f6d140d0e20 msgr2=0x7f6d140cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 shutdown_connections 2024-01-15T18:08:59.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:58.998+0000 7f6d1c600640 1 -- 172.21.15.143:0/419200017 wait complete. 2024-01-15T18:08:59.005 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.002+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.005 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.002+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.005 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.002+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.005 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.002+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.006 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.002+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.022 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.018+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.022 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.018+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.023 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.018+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.023 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.018+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.023 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:08:59.018+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:59.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87834 2024-01-15T18:08:59.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.170+0000 7f6c9c800640 1 Processor -- start 2024-01-15T18:08:59.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.170+0000 7f6c9c800640 1 -- start start 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9c800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c9415b260 0x7f6c940cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9c800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6c940ccaf0 0x7f6c940cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9c800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c940c4d90 con 0x7f6c940d19f0 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9c800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9415b640 con 0x7f6c9415b260 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9c800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c940cf800 con 0x7f6c940ccaf0 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c9415b260 0x7f6c940cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6c940d19f0 0x7f6c940c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37736/0 (socket says 172.21.15.143:37736) 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99400640 1 -- 172.21.15.143:0/2382626034 learned_addr learned my addr 172.21.15.143:0/2382626034 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9a800640 1 --2- 172.21.15.143:0/2382626034 >> v2:172.21.15.204:3300/0 conn(0x7f6c940ccaf0 0x7f6c940cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99e00640 1 -- 172.21.15.143:0/2382626034 >> v2:172.21.15.204:3300/0 conn(0x7f6c940ccaf0 msgr2=0x7f6c940cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99e00640 1 --2- 172.21.15.143:0/2382626034 >> v2:172.21.15.204:3300/0 conn(0x7f6c940ccaf0 0x7f6c940cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99e00640 1 -- 172.21.15.143:0/2382626034 >> v1:172.21.15.143:6789/0 conn(0x7f6c940d19f0 legacy=0x7f6c940c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c99e00640 1 -- 172.21.15.143:0/2382626034 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940cfae0 con 0x7f6c9415b260 2024-01-15T18:08:59.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.174+0000 7f6c9a800640 1 --2- 172.21.15.143:0/2382626034 >> v2:172.21.15.204:3300/0 conn(0x7f6c940ccaf0 0x7f6c940cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:59.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c99e00640 1 --2- 172.21.15.143:0/2382626034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c9415b260 0x7f6c940cd810 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6c94071260 tx=0x7f6c8401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814a56fdea032df0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c98a00640 1 -- 172.21.15.143:0/2382626034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84004230 con 0x7f6c9415b260 2024-01-15T18:08:59.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c98a00640 1 -- 172.21.15.143:0/2382626034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c840a7050 con 0x7f6c9415b260 2024-01-15T18:08:59.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c98a00640 1 -- 172.21.15.143:0/2382626034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84003620 con 0x7f6c9415b260 2024-01-15T18:08:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 -- 172.21.15.143:0/2382626034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c9415b260 msgr2=0x7f6c940cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 --2- 172.21.15.143:0/2382626034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c9415b260 0x7f6c940cd810 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6c94071260 tx=0x7f6c8401c460 comp rx=0 tx=0).stop 2024-01-15T18:08:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 -- 172.21.15.143:0/2382626034 shutdown_connections 2024-01-15T18:08:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 --2- 172.21.15.143:0/2382626034 >> v2:172.21.15.204:3300/0 conn(0x7f6c940ccaf0 0x7f6c940cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 --2- 172.21.15.143:0/2382626034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c9415b260 0x7f6c940cd810 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 -- 172.21.15.143:0/2382626034 >> 172.21.15.143:0/2382626034 conn(0x7f6c940d0ed0 msgr2=0x7f6c940d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:59.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 -- 172.21.15.143:0/2382626034 shutdown_connections 2024-01-15T18:08:59.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 -- 172.21.15.143:0/2382626034 wait complete. 2024-01-15T18:08:59.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 Processor -- start 2024-01-15T18:08:59.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.178+0000 7f6c9c800640 1 -- start start 2024-01-15T18:08:59.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 0x7f6c941526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c940d19f0 0x7f6c9414bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9415b260 0x7f6c94151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9414d220 con 0x7f6c940ccaf0 2024-01-15T18:08:59.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94159320 con 0x7f6c940d19f0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94159600 con 0x7f6c9415b260 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 0x7f6c941526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 0x7f6c941526e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58092/0 (socket says 172.21.15.143:58092) 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 -- 172.21.15.143:0/1721437804 learned_addr learned my addr 172.21.15.143:0/1721437804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c940d19f0 0x7f6c9414bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9a800640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9415b260 0x7f6c94151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 -- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9415b260 msgr2=0x7f6c94151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9415b260 0x7f6c94151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 -- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c940d19f0 msgr2=0x7f6c9414bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c940d19f0 0x7f6c9414bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 -- 172.21.15.143:0/1721437804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009420 con 0x7f6c940ccaf0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c99e00640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 0x7f6c941526e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6c840ae040 tx=0x7f6c84019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84003bf0 con 0x7f6c940ccaf0 2024-01-15T18:08:59.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c84003d70 con 0x7f6c940ccaf0 2024-01-15T18:08:59.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 -- 172.21.15.143:0/1721437804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c941598e0 con 0x7f6c940ccaf0 2024-01-15T18:08:59.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c840ab680 con 0x7f6c940ccaf0 2024-01-15T18:08:59.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c9c800640 1 -- 172.21.15.143:0/1721437804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c94152c20 con 0x7f6c940ccaf0 2024-01-15T18:08:59.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.182+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94067500 con 0x7f6c940ccaf0 2024-01-15T18:08:59.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.186+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6c840b2040 con 0x7f6c940ccaf0 2024-01-15T18:08:59.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.186+0000 7f6c92000640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c6c066e00 0x7f6c6c0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.186+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6c8411eba0 con 0x7f6c940ccaf0 2024-01-15T18:08:59.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.186+0000 7f6c99400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c6c066e00 0x7f6c6c0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.186+0000 7f6c99400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c6c066e00 0x7f6c6c0692c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6c940b2310 tx=0x7f6c7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.190+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c840eb6d0 con 0x7f6c940ccaf0 2024-01-15T18:08:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.362+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87834", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c94150490 con 0x7f6c6c066e00 2024-01-15T18:08:59.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.382+0000 7f6c92000640 1 -- 172.21.15.143:0/1721437804 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6c94150490 con 0x7f6c6c066e00 2024-01-15T18:08:59.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c6c066e00 msgr2=0x7f6c6c0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c6c066e00 0x7f6c6c0692c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6c940b2310 tx=0x7f6c7c009040 comp rx=0 tx=0).stop 2024-01-15T18:08:59.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 msgr2=0x7f6c941526e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 0x7f6c941526e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6c840ae040 tx=0x7f6c84019200 comp rx=0 tx=0).stop 2024-01-15T18:08:59.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 shutdown_connections 2024-01-15T18:08:59.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c6c066e00 0x7f6c6c0692c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c9415b260 0x7f6c94151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c940d19f0 0x7f6c9414bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 --2- 172.21.15.143:0/1721437804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c940ccaf0 0x7f6c941526e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 >> 172.21.15.143:0/1721437804 conn(0x7f6c940d0ed0 msgr2=0x7f6c940c5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:59.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 shutdown_connections 2024-01-15T18:08:59.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.386+0000 7f6c8b400640 1 -- 172.21.15.143:0/1721437804 wait complete. 2024-01-15T18:08:59.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87832 2024-01-15T18:08:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 Processor -- start 2024-01-15T18:08:59.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 -- start start 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0e00d0210 0x7fb0e00ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00cbff0 con 0x7fb0e00b0310 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00d0c50 con 0x7fb0e00d0210 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0e5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e006c8d0 con 0x7fb0e006bfb0 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0e00d0210 0x7fb0e00ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0e00d0210 0x7fb0e00ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58100/0 (socket says 172.21.15.143:58100) 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 -- 172.21.15.143:0/2432171078 learned_addr learned my addr 172.21.15.143:0/2432171078 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0dde00640 1 --2- 172.21.15.143:0/2432171078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 -- 172.21.15.143:0/2432171078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 msgr2=0x7fb0e006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 --2- 172.21.15.143:0/2432171078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 -- 172.21.15.143:0/2432171078 >> v1:172.21.15.204:6789/0 conn(0x7fb0e00b0310 legacy=0x7fb0e00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.558+0000 7fb0de800640 1 -- 172.21.15.143:0/2432171078 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00cb6c0 con 0x7fb0e00d0210 2024-01-15T18:08:59.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0de800640 1 --2- 172.21.15.143:0/2432171078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00ccb90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009e50 tx=0x7fb0d801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dae19a90ba217 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0dca00640 1 -- 172.21.15.143:0/2432171078 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a5810 con 0x7fb0e00d0210 2024-01-15T18:08:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0dca00640 1 -- 172.21.15.143:0/2432171078 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80a5970 con 0x7fb0e00d0210 2024-01-15T18:08:59.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0dca00640 1 -- 172.21.15.143:0/2432171078 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a5d00 con 0x7fb0e00d0210 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 -- 172.21.15.143:0/2432171078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 msgr2=0x7fb0e00ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 --2- 172.21.15.143:0/2432171078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00ccb90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009e50 tx=0x7fb0d801c950 comp rx=0 tx=0).stop 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 -- 172.21.15.143:0/2432171078 shutdown_connections 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 --2- 172.21.15.143:0/2432171078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00ccb90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 --2- 172.21.15.143:0/2432171078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e006c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 -- 172.21.15.143:0/2432171078 >> 172.21.15.143:0/2432171078 conn(0x7fb0e00d0ed0 msgr2=0x7fb0e00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 -- 172.21.15.143:0/2432171078 shutdown_connections 2024-01-15T18:08:59.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 -- 172.21.15.143:0/2432171078 wait complete. 2024-01-15T18:08:59.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 Processor -- start 2024-01-15T18:08:59.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.566+0000 7fb0e5000640 1 -- start start 2024-01-15T18:08:59.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e00b0310 0x7fb0e014c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e00aec50 con 0x7fb0e00d0210 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0154a70 con 0x7fb0e00b0310 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015e2d0 con 0x7fb0e006bfb0 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58106/0 (socket says 172.21.15.143:58106) 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 -- 172.21.15.143:0/1056560588 learned_addr learned my addr 172.21.15.143:0/1056560588 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0dde00640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0dd400640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e00b0310 0x7fb0e014c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 -- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 msgr2=0x7fb0e015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 -- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e00b0310 msgr2=0x7fb0e014c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e00b0310 0x7fb0e014c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 -- 172.21.15.143:0/1056560588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8009b30 con 0x7fb0e00d0210 2024-01-15T18:08:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0de800640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00aef80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb0d801b000 tx=0x7fb0d80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b8640 con 0x7fb0e00d0210 2024-01-15T18:08:59.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80b87c0 con 0x7fb0e00d0210 2024-01-15T18:08:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b8b70 con 0x7fb0e00d0210 2024-01-15T18:08:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e00ae0d0 con 0x7fb0e00d0210 2024-01-15T18:08:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00ae690 con 0x7fb0e00d0210 2024-01-15T18:08:59.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.570+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e0067500 con 0x7fb0e00d0210 2024-01-15T18:08:59.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.574+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7fb0d80b8cd0 con 0x7fb0e00d0210 2024-01-15T18:08:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.574+0000 7fb0d6000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0b8066e00 0x7fb0b80692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.574+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb0d811fc20 con 0x7fb0e00d0210 2024-01-15T18:08:59.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.574+0000 7fb0dde00640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0b8066e00 0x7fb0b80692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.574+0000 7fb0dde00640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0b8066e00 0x7fb0b80692c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00079c0 tx=0x7fb0d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.578+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0d80ec860 con 0x7fb0e00d0210 2024-01-15T18:08:59.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.750+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87832", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e006ff20 con 0x7fb0b8066e00 2024-01-15T18:08:59.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.766+0000 7fb0d6000640 1 -- 172.21.15.143:0/1056560588 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb0e006ff20 con 0x7fb0b8066e00 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0b8066e00 msgr2=0x7fb0b80692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0b8066e00 0x7fb0b80692c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00079c0 tx=0x7fb0d0008040 comp rx=0 tx=0).stop 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 msgr2=0x7fb0e00aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00aef80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fb0d801b000 tx=0x7fb0d80196f0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 shutdown_connections 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0b8066e00 0x7fb0b80692c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e00d0210 0x7fb0e00aef80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e00b0310 0x7fb0e014c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 --2- 172.21.15.143:0/1056560588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e006bfb0 0x7fb0e015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 >> 172.21.15.143:0/1056560588 conn(0x7fb0e00d0ed0 msgr2=0x7fb0e006f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:59.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 shutdown_connections 2024-01-15T18:08:59.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.770+0000 7fb0e5000640 1 -- 172.21.15.143:0/1056560588 wait complete. 2024-01-15T18:08:59.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87833 2024-01-15T18:08:59.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.934+0000 7ff0c1800640 1 Processor -- start 2024-01-15T18:08:59.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0c1800640 1 -- start start 2024-01-15T18:08:59.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0c1800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff0bc0cfb30 0x7ff0bc06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0c1800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0cffd0 con 0x7ff0bc0cbf50 2024-01-15T18:08:59.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0c1800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0d02b0 con 0x7ff0bc1607c0 2024-01-15T18:08:59.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0c1800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc0d1ce0 con 0x7ff0bc0cfb30 2024-01-15T18:08:59.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff0bc0cbf50 0x7ff0bc0cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37752/0 (socket says 172.21.15.143:37752) 2024-01-15T18:08:59.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 -- 172.21.15.143:0/1960641407 learned_addr learned my addr 172.21.15.143:0/1960641407 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0ba000640 1 --2- 172.21.15.143:0/1960641407 >> v2:172.21.15.204:3300/0 conn(0x7ff0bc0cfb30 0x7ff0bc06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545274215 0 0) 0x7ff0bc0cffd0 con 0x7ff0bc0cbf50 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff09c003530 con 0x7ff0bc0cbf50 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280516470 0 0) 0x7ff09c003530 con 0x7ff0bc0cbf50 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 >> v2:172.21.15.204:3300/0 conn(0x7ff0bc0cfb30 msgr2=0x7ff0bc06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 --2- 172.21.15.143:0/1960641407 >> v2:172.21.15.204:3300/0 conn(0x7ff0bc0cfb30 0x7ff0bc06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 >> v1:172.21.15.204:6789/0 conn(0x7ff0bc1607c0 legacy=0x7ff0bc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0cb470 con 0x7ff0bc0cbf50 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff0b4001a00 con 0x7ff0bc0cbf50 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 >> v1:172.21.15.143:6789/0 conn(0x7ff0bc0cbf50 legacy=0x7ff0bc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c003980 0x7ff09c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c0047d0 0x7ff09c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c005190 0x7ff09c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0d1ce0 con 0x7ff09c005190 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c00c040 con 0x7ff09c0047d0 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c00c200 con 0x7ff09c003980 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c005190 0x7ff09c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0baa00640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c003980 0x7ff09c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0ba000640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c0047d0 0x7ff09c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 -- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c003980 msgr2=0x7ff09c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c003980 0x7ff09c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 -- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c0047d0 msgr2=0x7ff09c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff09c0047d0 0x7ff09c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.938+0000 7ff0bb400640 1 -- 172.21.15.143:0/1960641407 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c003530 con 0x7ff09c005190 2024-01-15T18:08:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0bb400640 1 -- 172.21.15.143:0/1960641407 reap_dead start 2024-01-15T18:08:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0bb400640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c005190 0x7ff09c00b990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400a830 tx=0x7ff0b401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500331c8c36a285 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40072d0 con 0x7ff09c005190 2024-01-15T18:08:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40a8070 con 0x7ff09c005190 2024-01-15T18:08:59.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0b9600640 1 -- 172.21.15.143:0/1960641407 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40076b0 con 0x7ff09c005190 2024-01-15T18:08:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 -- 172.21.15.143:0/1960641407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c005190 msgr2=0x7ff09c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c005190 0x7ff09c00b990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400a830 tx=0x7ff0b401d340 comp rx=0 tx=0).stop 2024-01-15T18:08:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 -- 172.21.15.143:0/1960641407 shutdown_connections 2024-01-15T18:08:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1960641407 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff09c005190 0x7ff09c00b990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 -- 172.21.15.143:0/1960641407 >> 172.21.15.143:0/1960641407 conn(0x7ff0bc0d0e20 msgr2=0x7ff0bc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:59.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 -- 172.21.15.143:0/1960641407 shutdown_connections 2024-01-15T18:08:59.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.942+0000 7ff0c1800640 1 -- 172.21.15.143:0/1960641407 wait complete. 2024-01-15T18:08:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 Processor -- start 2024-01-15T18:08:59.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 -- start start 2024-01-15T18:08:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c005190 0x7ff0bc155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 0x7ff0bc152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0bc0c7130 0x7ff0bc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc155e60 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc154880 con 0x7ff0bc0c7130 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc154b60 con 0x7ff09c005190 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 0x7ff0bc152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 0x7ff0bc152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58124/0 (socket says 172.21.15.143:58124) 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 -- 172.21.15.143:0/1623328039 learned_addr learned my addr 172.21.15.143:0/1623328039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0baa00640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c005190 0x7ff0bc155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0bb400640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0bc0c7130 0x7ff0bc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 -- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c005190 msgr2=0x7ff0bc155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c005190 0x7ff0bc155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 -- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0bc0c7130 msgr2=0x7ff0bc153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0bc0c7130 0x7ff0bc153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 -- 172.21.15.143:0/1623328039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc154e40 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0ba000640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 0x7ff0bc152ed0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4011960 tx=0x7ff0a4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400faf0 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a400fc50 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4014440 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc155120 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.946+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc153b90 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.950+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc06d4b0 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.950+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7ff0a40145a0 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.950+0000 7ff0b2a00640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff094066e00 0x7ff0940692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.950+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff0a4085370 con 0x7ff0bc0c6cb0 2024-01-15T18:08:59.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.950+0000 7ff0baa00640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff094066e00 0x7ff0940692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:59.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.950+0000 7ff0baa00640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff094066e00 0x7ff0940692c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac000c10 tx=0x7ff0ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:59.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:08:59.954+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff0a4052000 con 0x7ff0bc0c6cb0 2024-01-15T18:09:00.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.134+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87833", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0bc0cba30 con 0x7ff094066e00 2024-01-15T18:09:00.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0b2a00640 1 -- 172.21.15.143:0/1623328039 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff0ac001530 con 0x7ff094066e00 2024-01-15T18:09:00.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff094066e00 msgr2=0x7ff0940692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff094066e00 0x7ff0940692c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac000c10 tx=0x7ff0ac009040 comp rx=0 tx=0).stop 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 msgr2=0x7ff0bc152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 0x7ff0bc152ed0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4011960 tx=0x7ff0a4011e20 comp rx=0 tx=0).stop 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 shutdown_connections 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff094066e00 0x7ff0940692c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0bc0c7130 0x7ff0bc153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0bc0c6cb0 0x7ff0bc152ed0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 --2- 172.21.15.143:0/1623328039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff09c005190 0x7ff0bc155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 >> 172.21.15.143:0/1623328039 conn(0x7ff0bc0d0e20 msgr2=0x7ff0bc0cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 shutdown_connections 2024-01-15T18:09:00.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.146+0000 7ff0c1800640 1 -- 172.21.15.143:0/1623328039 wait complete. 2024-01-15T18:09:00.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone87831 2024-01-15T18:09:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 Processor -- start 2024-01-15T18:09:00.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 -- start start 2024-01-15T18:09:00.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6bfc143780 0x7f6bfc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc0ce520 0x7f6bfc071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6bfc0720f0 0x7f6bfc15a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc072580 con 0x7f6bfc143780 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bfc0ceb50 con 0x7f6bfc0720f0 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c02a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc15c110 con 0x7f6bfc0ce520 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6bfc143780 0x7f6bfc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6bfc143780 0x7f6bfc0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58130/0 (socket says 172.21.15.143:58130) 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 -- 172.21.15.143:0/665123107 learned_addr learned my addr 172.21.15.143:0/665123107 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfb400640 1 --2- 172.21.15.143:0/665123107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc0ce520 0x7f6bfc071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6c00a00640 1 --2- 172.21.15.143:0/665123107 >> v2:172.21.15.204:3300/0 conn(0x7f6bfc0720f0 0x7f6bfc15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 -- 172.21.15.143:0/665123107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc0ce520 msgr2=0x7f6bfc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/665123107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc0ce520 0x7f6bfc071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 -- 172.21.15.143:0/665123107 >> v2:172.21.15.204:3300/0 conn(0x7f6bfc0720f0 msgr2=0x7f6bfc15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/665123107 >> v2:172.21.15.204:3300/0 conn(0x7f6bfc0720f0 0x7f6bfc15a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.314+0000 7f6bfbe00640 1 -- 172.21.15.143:0/665123107 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bfc15b7e0 con 0x7f6bfc143780 2024-01-15T18:09:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/665123107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc143780 0x7f6bfc0cdfe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0b1da0 tx=0x7f6bec01e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d21e57a045de8d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6bfaa00640 1 -- 172.21.15.143:0/665123107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0aa610 con 0x7f6bfc143780 2024-01-15T18:09:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6bfaa00640 1 -- 172.21.15.143:0/665123107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec0aa770 con 0x7f6bfc143780 2024-01-15T18:09:00.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6bfaa00640 1 -- 172.21.15.143:0/665123107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0aab00 con 0x7f6bfc143780 2024-01-15T18:09:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 -- 172.21.15.143:0/665123107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc143780 msgr2=0x7f6bfc0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 --2- 172.21.15.143:0/665123107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc143780 0x7f6bfc0cdfe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0b1da0 tx=0x7f6bec01e710 comp rx=0 tx=0).stop 2024-01-15T18:09:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 -- 172.21.15.143:0/665123107 shutdown_connections 2024-01-15T18:09:00.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 --2- 172.21.15.143:0/665123107 >> v2:172.21.15.204:3300/0 conn(0x7f6bfc0720f0 0x7f6bfc15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 --2- 172.21.15.143:0/665123107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc0ce520 0x7f6bfc071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 --2- 172.21.15.143:0/665123107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc143780 0x7f6bfc0cdfe0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 -- 172.21.15.143:0/665123107 >> 172.21.15.143:0/665123107 conn(0x7f6bfc06c3f0 msgr2=0x7f6bfc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 -- 172.21.15.143:0/665123107 shutdown_connections 2024-01-15T18:09:00.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.318+0000 7f6c02a00640 1 -- 172.21.15.143:0/665123107 wait complete. 2024-01-15T18:09:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 Processor -- start 2024-01-15T18:09:00.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 -- start start 2024-01-15T18:09:00.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bfc0720f0 0x7f6bfc134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 0x7f6bfc13d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc143780 0x7f6bfc136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc134760 con 0x7f6bfc0ce520 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc159c10 con 0x7f6bfc0720f0 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc1427f0 con 0x7f6bfc143780 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 0x7f6bfc13d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 0x7f6bfc13d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58138/0 (socket says 172.21.15.143:58138) 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfb400640 1 -- 172.21.15.143:0/1736221714 learned_addr learned my addr 172.21.15.143:0/1736221714 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bfc0720f0 0x7f6bfc134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c00a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc143780 0x7f6bfc136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 -- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc143780 msgr2=0x7f6bfc136ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc143780 0x7f6bfc136ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 -- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 msgr2=0x7f6bfc13d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 0x7f6bfc13d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 -- 172.21.15.143:0/1736221714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0094d0 con 0x7f6bfc0720f0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfb400640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 0x7f6bfc13d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bfbe00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bfc0720f0 0x7f6bfc134220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0ae000 tx=0x7f6bec01b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0aa610 con 0x7f6bfc0720f0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec0b2850 con 0x7f6bfc0720f0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec0b2c00 con 0x7f6bfc0720f0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc135d20 con 0x7f6bfc0720f0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.322+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc1362e0 con 0x7f6bfc0720f0 2024-01-15T18:09:00.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.326+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc067500 con 0x7f6bfc0720f0 2024-01-15T18:09:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.326+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f6bec0a8040 con 0x7f6bfc0720f0 2024-01-15T18:09:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.326+0000 7f6bf3e00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bd4066e00 0x7f6bd40692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.326+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6bec11ed90 con 0x7f6bfc0720f0 2024-01-15T18:09:00.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.326+0000 7f6bfb400640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bd4066e00 0x7f6bd40692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.326+0000 7f6bfb400640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bd4066e00 0x7f6bd40692c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc06de50 tx=0x7f6be4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:00.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.330+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6bec0eb9d0 con 0x7f6bfc0720f0 2024-01-15T18:09:00.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.502+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone87831", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bfc13daa0 con 0x7f6bd4066e00 2024-01-15T18:09:00.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6bf3e00640 1 -- 172.21.15.143:0/1736221714 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6bfc13daa0 con 0x7f6bd4066e00 2024-01-15T18:09:00.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bd4066e00 msgr2=0x7f6bd40692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bd4066e00 0x7f6bd40692c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc06de50 tx=0x7f6be4008040 comp rx=0 tx=0).stop 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bfc0720f0 msgr2=0x7f6bfc134220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bfc0720f0 0x7f6bfc134220 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0ae000 tx=0x7f6bec01b4b0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 shutdown_connections 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6bd4066e00 0x7f6bd40692c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6bfc143780 0x7f6bfc136ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6bfc0ce520 0x7f6bfc13d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 --2- 172.21.15.143:0/1736221714 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6bfc0720f0 0x7f6bfc134220 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 >> 172.21.15.143:0/1736221714 conn(0x7f6bfc06c3f0 msgr2=0x7f6bfc0ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 shutdown_connections 2024-01-15T18:09:00.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.518+0000 7f6c02a00640 1 -- 172.21.15.143:0/1736221714 wait complete. 2024-01-15T18:09:00.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:00.550 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:09:00.568 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-15T18:09:00.569 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:09:00.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.746+0000 7f1579000640 1 Processor -- start 2024-01-15T18:09:00.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.746+0000 7f1579000640 1 -- start start 2024-01-15T18:09:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1579000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f157415d510 0x7f15740aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1579000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1574160eb0 con 0x7f157415d130 2024-01-15T18:09:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1579000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1574161190 con 0x7f157406aba0 2024-01-15T18:09:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1579000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1574161470 con 0x7f157415d510 2024-01-15T18:09:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f157415d130 0x7f157406a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37754/0 (socket says 172.21.15.143:37754) 2024-01-15T18:09:00.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 -- 172.21.15.143:0/3309345243 learned_addr learned my addr 172.21.15.143:0/3309345243 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:00.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1572800640 1 --2- 172.21.15.143:0/3309345243 >> v2:172.21.15.204:3300/0 conn(0x7f157415d510 0x7f15740aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289057181 0 0) 0x7f1574160eb0 con 0x7f157415d130 2024-01-15T18:09:00.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003530 con 0x7f157415d130 2024-01-15T18:09:00.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615863601 0 0) 0x7f1554003530 con 0x7f157415d130 2024-01-15T18:09:00.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 >> v2:172.21.15.204:3300/0 conn(0x7f157415d510 msgr2=0x7f15740aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 --2- 172.21.15.143:0/3309345243 >> v2:172.21.15.204:3300/0 conn(0x7f157415d510 0x7f15740aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 >> v1:172.21.15.204:6789/0 conn(0x7f157406aba0 legacy=0x7f15740b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574161750 con 0x7f157415d130 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f1564001680 con 0x7f157415d130 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 >> v1:172.21.15.143:6789/0 conn(0x7f157415d130 legacy=0x7f157406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554003980 0x7f1554003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554004380 0x7f1554004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1554005050 0x7f155400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1574161470 con 0x7f1554003980 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155400bfb0 con 0x7f1554004380 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155400c170 con 0x7f1554005050 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554003980 0x7f1554003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571400640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554004380 0x7f1554004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1572800640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1554005050 0x7f155400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 -- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1554005050 msgr2=0x7f155400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1554005050 0x7f155400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 -- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554004380 msgr2=0x7f1554004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554004380 0x7f1554004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1571e00640 1 -- 172.21.15.143:0/3309345243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1554003530 con 0x7f1554003980 2024-01-15T18:09:00.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.750+0000 7f1572800640 1 -- 172.21.15.143:0/3309345243 reap_dead start 2024-01-15T18:09:00.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1571e00640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554003980 0x7f1554003d60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1564004ff0 tx=0x7f156401d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9048e537043a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:00.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564005ae0 con 0x7f1554003980 2024-01-15T18:09:00.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15640a8070 con 0x7f1554003980 2024-01-15T18:09:00.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1570a00640 1 -- 172.21.15.143:0/3309345243 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1564005ea0 con 0x7f1554003980 2024-01-15T18:09:00.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 -- 172.21.15.143:0/3309345243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554003980 msgr2=0x7f1554003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554003980 0x7f1554003d60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1564004ff0 tx=0x7f156401d1e0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 -- 172.21.15.143:0/3309345243 shutdown_connections 2024-01-15T18:09:00.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 --2- 172.21.15.143:0/3309345243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1554003980 0x7f1554003d60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 -- 172.21.15.143:0/3309345243 >> 172.21.15.143:0/3309345243 conn(0x7f15740b1b90 msgr2=0x7f15740b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:00.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 -- 172.21.15.143:0/3309345243 shutdown_connections 2024-01-15T18:09:00.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.754+0000 7f1579000640 1 -- 172.21.15.143:0/3309345243 wait complete. 2024-01-15T18:09:00.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 Processor -- start 2024-01-15T18:09:00.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- start start 2024-01-15T18:09:00.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554003980 0x7f15740b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415e900 0x7f157414a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 0x7f157415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157415e4e0 con 0x7f1574149f70 2024-01-15T18:09:00.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157415d8f0 con 0x7f1554003980 2024-01-15T18:09:00.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157415db40 con 0x7f157415e900 2024-01-15T18:09:00.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 0x7f157415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 0x7f157415b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58150/0 (socket says 172.21.15.143:58150) 2024-01-15T18:09:00.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 -- 172.21.15.143:0/2844985402 learned_addr learned my addr 172.21.15.143:0/2844985402 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:00.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1571400640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415e900 0x7f157414a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1571e00640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554003980 0x7f15740b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 -- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415e900 msgr2=0x7f157414a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415e900 0x7f157414a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 -- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554003980 msgr2=0x7f15740b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:00.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554003980 0x7f15740b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:00.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 -- 172.21.15.143:0/2844985402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157415de20 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1572800640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 0x7f157415b780 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f157406b4f0 tx=0x7f156c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00e070 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c011e40 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157415e100 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c0039a0 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157414a3f0 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.758+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157415e620 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.762+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) v1 ==== 86619+0+0 (secure 0 0 0) 0x7f156c003b20 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.762+0000 7f156a000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f154c066e00 0x7f154c0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.762+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f156c085410 con 0x7f1574149f70 2024-01-15T18:09:00.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.762+0000 7f1571e00640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f154c066e00 0x7f154c0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:00.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.762+0000 7f1571e00640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f154c066e00 0x7f154c0692c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1564001ec0 tx=0x7f1564008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:00.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.766+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f157415e620 con 0x7f1574149f70 2024-01-15T18:09:00.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:00.934+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f157415e620 con 0x7f154c066e00 2024-01-15T18:09:00.969 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:09:00.962+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:09:01.069 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:09:01.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:01.610+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 14) v1 ==== 86721+0+0 (secure 0 0 0) 0x7f156c052360 con 0x7f1574149f70 2024-01-15T18:09:03.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.622+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f156c082f20 con 0x7f1574149f70 2024-01-15T18:09:03.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.982+0000 7f156a000640 1 -- 172.21.15.143:0/2844985402 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f157415e620 con 0x7f154c066e00 2024-01-15T18:09:03.988 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:09:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f154c066e00 msgr2=0x7f154c0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f154c066e00 0x7f154c0692c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1564001ec0 tx=0x7f1564008040 comp rx=0 tx=0).stop 2024-01-15T18:09:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 msgr2=0x7f157415b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 0x7f157415b780 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f157406b4f0 tx=0x7f156c0115a0 comp rx=0 tx=0).stop 2024-01-15T18:09:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 shutdown_connections 2024-01-15T18:09:03.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f154c066e00 0x7f154c0692c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1574149f70 0x7f157415b780 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f157415e900 0x7f157414a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 --2- 172.21.15.143:0/2844985402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1554003980 0x7f15740b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 >> 172.21.15.143:0/2844985402 conn(0x7f15740b1b90 msgr2=0x7f157406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 shutdown_connections 2024-01-15T18:09:03.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:03.986+0000 7f1579000640 1 -- 172.21.15.143:0/2844985402 wait complete. 2024-01-15T18:09:04.015 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:09:04.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:09:04.124 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:09:04.209 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:09:04.209 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:04.210 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:04.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 Processor -- start 2024-01-15T18:09:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 -- start start 2024-01-15T18:09:04.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f554015d130 0x7f55400aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 0x7f554006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5540160eb0 con 0x7f554006a490 2024-01-15T18:09:04.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5540161190 con 0x7f554015d130 2024-01-15T18:09:04.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.394+0000 7f5547000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5540161470 con 0x7f554015d510 2024-01-15T18:09:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 0x7f554006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 0x7f554006e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41324/0 (socket says 172.21.15.143:41324) 2024-01-15T18:09:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 -- 172.21.15.143:0/259382268 learned_addr learned my addr 172.21.15.143:0/259382268 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 -- 172.21.15.143:0/259382268 >> v2:172.21.15.204:3300/0 conn(0x7f554015d130 msgr2=0x7f55400aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 --2- 172.21.15.143:0/259382268 >> v2:172.21.15.204:3300/0 conn(0x7f554015d130 0x7f55400aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 -- 172.21.15.143:0/259382268 >> v1:172.21.15.204:6789/0 conn(0x7f554006a490 legacy=0x7f5540071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.398+0000 7f553f400640 1 -- 172.21.15.143:0/259382268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5540161750 con 0x7f554015d510 2024-01-15T18:09:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f553f400640 1 --2- 172.21.15.143:0/259382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 0x7f554006e060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5528009870 tx=0x7f552801c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79edab43dd5912f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f553ea00640 1 -- 172.21.15.143:0/259382268 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55280aa640 con 0x7f554015d510 2024-01-15T18:09:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f553ea00640 1 -- 172.21.15.143:0/259382268 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55280aa7a0 con 0x7f554015d510 2024-01-15T18:09:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f553ea00640 1 -- 172.21.15.143:0/259382268 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55280aab30 con 0x7f554015d510 2024-01-15T18:09:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 -- 172.21.15.143:0/259382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 msgr2=0x7f554006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 --2- 172.21.15.143:0/259382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 0x7f554006e060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5528009870 tx=0x7f552801c500 comp rx=0 tx=0).stop 2024-01-15T18:09:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 -- 172.21.15.143:0/259382268 shutdown_connections 2024-01-15T18:09:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 --2- 172.21.15.143:0/259382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d510 0x7f554006e060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 --2- 172.21.15.143:0/259382268 >> v2:172.21.15.204:3300/0 conn(0x7f554015d130 0x7f55400aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 -- 172.21.15.143:0/259382268 >> 172.21.15.143:0/259382268 conn(0x7f55400b1b90 msgr2=0x7f55400b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:04.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 -- 172.21.15.143:0/259382268 shutdown_connections 2024-01-15T18:09:04.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.402+0000 7f5547000640 1 -- 172.21.15.143:0/259382268 wait complete. 2024-01-15T18:09:04.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 Processor -- start 2024-01-15T18:09:04.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 -- start start 2024-01-15T18:09:04.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554006a490 0x7f55401571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d130 0x7f554013d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 0x7f5540160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554014f340 con 0x7f554015d510 2024-01-15T18:09:04.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554013d750 con 0x7f554006a490 2024-01-15T18:09:04.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554013da30 con 0x7f554015d130 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 0x7f5540160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 0x7f5540160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58182/0 (socket says 172.21.15.143:58182) 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 -- 172.21.15.143:0/823282005 learned_addr learned my addr 172.21.15.143:0/823282005 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 -- 172.21.15.143:0/823282005 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d130 msgr2=0x7f554013d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d130 0x7f554013d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 -- 172.21.15.143:0/823282005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554006a490 msgr2=0x7f55401571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554006a490 0x7f55401571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 -- 172.21.15.143:0/823282005 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280094d0 con 0x7f554015d510 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5545000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 0x7f5540160110 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f553800c780 tx=0x7f553800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5538003d90 con 0x7f554015d510 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553800ce90 con 0x7f554015d510 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554013dd10 con 0x7f554015d510 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55380052b0 con 0x7f554015d510 2024-01-15T18:09:04.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.406+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5540160650 con 0x7f554015d510 2024-01-15T18:09:04.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.410+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5540067500 con 0x7f554015d510 2024-01-15T18:09:04.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.410+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f5538005410 con 0x7f554015d510 2024-01-15T18:09:04.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.410+0000 7f5537e00640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5518066e90 0x7f5518069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.410+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5538084370 con 0x7f554015d510 2024-01-15T18:09:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.410+0000 7f553fe00640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5518066e90 0x7f5518069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.410+0000 7f553fe00640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5518066e90 0x7f5518069350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5530009730 tx=0x7f5530006ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:04.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.414+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5538051120 con 0x7f554015d510 2024-01-15T18:09:04.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55400b4d30 con 0x7f554015d510 2024-01-15T18:09:04.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5537e00640 1 -- 172.21.15.143:0/823282005 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f5538054740 con 0x7f554015d510 2024-01-15T18:09:04.571 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:04.571 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":51,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:03.981057+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6836/1835320705":"2024-01-16T18:09:00.967606+0000","172.21.15.143:6834/1835320705":"2024-01-16T18:09:00.967606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:04.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5518066e90 msgr2=0x7f5518069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5518066e90 0x7f5518069350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5530009730 tx=0x7f5530006ca0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 msgr2=0x7f5540160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 0x7f5540160110 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f553800c780 tx=0x7f553800cc40 comp rx=0 tx=0).stop 2024-01-15T18:09:04.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 shutdown_connections 2024-01-15T18:09:04.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5518066e90 0x7f5518069350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554015d510 0x7f5540160110 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554015d130 0x7f554013d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 --2- 172.21.15.143:0/823282005 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554006a490 0x7f55401571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 >> 172.21.15.143:0/823282005 conn(0x7f55400b1b90 msgr2=0x7f5540071260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:04.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 shutdown_connections 2024-01-15T18:09:04.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.566+0000 7f5547000640 1 -- 172.21.15.143:0/823282005 wait complete. 2024-01-15T18:09:04.593 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:09:04.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:09:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 Processor -- start 2024-01-15T18:09:04.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 -- start start 2024-01-15T18:09:04.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61e4071bb0 0x7f61e4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f61e415c5a0 0x7f61e415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e40ceb50 con 0x7f61e4143780 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e4072570 con 0x7f61e4071bb0 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61eb400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e415c110 con 0x7f61e415c5a0 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61e4071bb0 0x7f61e4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f61e4071bb0 0x7f61e4072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58184/0 (socket says 172.21.15.143:58184) 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 -- 172.21.15.143:0/4283582994 learned_addr learned my addr 172.21.15.143:0/4283582994 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e9400640 1 --2- 172.21.15.143:0/4283582994 >> v2:172.21.15.204:3300/0 conn(0x7f61e415c5a0 0x7f61e415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 -- 172.21.15.143:0/4283582994 >> v2:172.21.15.204:3300/0 conn(0x7f61e415c5a0 msgr2=0x7f61e415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 --2- 172.21.15.143:0/4283582994 >> v2:172.21.15.204:3300/0 conn(0x7f61e415c5a0 0x7f61e415c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 -- 172.21.15.143:0/4283582994 >> v1:172.21.15.143:6789/0 conn(0x7f61e4143780 legacy=0x7f61e40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e3e00640 1 -- 172.21.15.143:0/4283582994 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e415b7e0 con 0x7f61e4071bb0 2024-01-15T18:09:04.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.730+0000 7f61e9400640 1 --2- 172.21.15.143:0/4283582994 >> v2:172.21.15.204:3300/0 conn(0x7f61e415c5a0 0x7f61e415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:04.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61e3e00640 1 --2- 172.21.15.143:0/4283582994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4072030 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f61e4070a20 tx=0x7f61cc01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8aaa07e11b1171 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61e3400640 1 -- 172.21.15.143:0/4283582994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0a7ca0 con 0x7f61e4071bb0 2024-01-15T18:09:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61e3400640 1 -- 172.21.15.143:0/4283582994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc0a7e00 con 0x7f61e4071bb0 2024-01-15T18:09:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61e3400640 1 -- 172.21.15.143:0/4283582994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0af660 con 0x7f61e4071bb0 2024-01-15T18:09:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 -- 172.21.15.143:0/4283582994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 msgr2=0x7f61e4072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 --2- 172.21.15.143:0/4283582994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4072030 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f61e4070a20 tx=0x7f61cc01ebd0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 -- 172.21.15.143:0/4283582994 shutdown_connections 2024-01-15T18:09:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 --2- 172.21.15.143:0/4283582994 >> v2:172.21.15.204:3300/0 conn(0x7f61e415c5a0 0x7f61e415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 --2- 172.21.15.143:0/4283582994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4072030 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 -- 172.21.15.143:0/4283582994 >> 172.21.15.143:0/4283582994 conn(0x7f61e406c3f0 msgr2=0x7f61e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 -- 172.21.15.143:0/4283582994 shutdown_connections 2024-01-15T18:09:04.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.734+0000 7f61eb400640 1 -- 172.21.15.143:0/4283582994 wait complete. 2024-01-15T18:09:04.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 Processor -- start 2024-01-15T18:09:04.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 -- start start 2024-01-15T18:09:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61e4143780 0x7f61e40b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e415c5a0 0x7f61e40b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41579b0 con 0x7f61e4071bb0 2024-01-15T18:09:04.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4158fa0 con 0x7f61e415c5a0 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4137780 con 0x7f61e4143780 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58190/0 (socket says 172.21.15.143:58190) 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 -- 172.21.15.143:0/2139871365 learned_addr learned my addr 172.21.15.143:0/2139871365 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e415c5a0 0x7f61e40b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e3e00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61e4143780 0x7f61e40b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 -- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61e4143780 msgr2=0x7f61e40b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61e4143780 0x7f61e40b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 -- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e415c5a0 msgr2=0x7f61e40b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e415c5a0 0x7f61e40b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 -- 172.21.15.143:0/2139871365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc00b770 con 0x7f61e4071bb0 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e8a00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4156e70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c500 tx=0x7f61d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:04.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4005220 con 0x7f61e4071bb0 2024-01-15T18:09:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d4005380 con 0x7f61e4071bb0 2024-01-15T18:09:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d4007270 con 0x7f61e4071bb0 2024-01-15T18:09:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e41374a0 con 0x7f61e4071bb0 2024-01-15T18:09:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.738+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e4158c30 con 0x7f61e4071bb0 2024-01-15T18:09:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.742+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e41587b0 con 0x7f61e4071bb0 2024-01-15T18:09:04.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.742+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f61d40073d0 con 0x7f61e4071bb0 2024-01-15T18:09:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.742+0000 7f61e0c00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61bc066ec0 0x7f61bc069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.742+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f61d4084220 con 0x7f61e4071bb0 2024-01-15T18:09:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.742+0000 7f61e3e00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61bc066ec0 0x7f61bc069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:04.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.742+0000 7f61e3e00640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61bc066ec0 0x7f61bc069380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b1000 tx=0x7f61cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:04.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:04.746+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f61e41587b0 con 0x7f61e4071bb0 2024-01-15T18:09:05.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.022+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61e40ceb50 con 0x7f61e4071bb0 2024-01-15T18:09:05.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.022+0000 7f61e0c00640 1 -- 172.21.15.143:0/2139871365 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61e40ceb50 con 0x7f61e4071bb0 2024-01-15T18:09:05.028 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:09:05.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61bc066ec0 msgr2=0x7f61bc069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:05.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61bc066ec0 0x7f61bc069380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0b1000 tx=0x7f61cc008040 comp rx=0 tx=0).stop 2024-01-15T18:09:05.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 msgr2=0x7f61e4156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:05.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4156e70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f61d400c500 tx=0x7f61d400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 shutdown_connections 2024-01-15T18:09:05.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61bc066ec0 0x7f61bc069380 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f61e415c5a0 0x7f61e40b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f61e4143780 0x7f61e40b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 --2- 172.21.15.143:0/2139871365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f61e4071bb0 0x7f61e4156e70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.026+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 >> 172.21.15.143:0/2139871365 conn(0x7f61e406c3f0 msgr2=0x7f61e4138ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:05.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.030+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 shutdown_connections 2024-01-15T18:09:05.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.030+0000 7f61eb400640 1 -- 172.21.15.143:0/2139871365 wait complete. 2024-01-15T18:09:05.094 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:09:05.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.414+0000 7f2129800640 1 Processor -- start 2024-01-15T18:09:05.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.414+0000 7f2129800640 1 -- start start 2024-01-15T18:09:05.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2129800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21240d0150 0x7f21240c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:05.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2129800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21240b45d0 0x7f21240c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:05.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2129800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240cefa0 con 0x7f21240b3610 2024-01-15T18:09:05.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2129800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240c9fa0 con 0x7f21240d0150 2024-01-15T18:09:05.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2129800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240b3ef0 con 0x7f21240b45d0 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2123400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21240b45d0 0x7f21240c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2122a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f21240b3610 0x7f21240cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:51980/0 (socket says 172.21.15.143:51980) 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2122a00640 1 -- 172.21.15.143:0/603717252 learned_addr learned my addr 172.21.15.143:0/603717252 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2122000640 1 --2- 172.21.15.143:0/603717252 >> v2:172.21.15.143:3300/0 conn(0x7f21240d0150 0x7f21240c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2123400640 1 -- 172.21.15.143:0/603717252 >> v2:172.21.15.143:3300/0 conn(0x7f21240d0150 msgr2=0x7f21240c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2123400640 1 --2- 172.21.15.143:0/603717252 >> v2:172.21.15.143:3300/0 conn(0x7f21240d0150 0x7f21240c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2123400640 1 -- 172.21.15.143:0/603717252 >> v1:172.21.15.204:6789/0 conn(0x7f21240b3610 legacy=0x7f21240cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2123400640 1 -- 172.21.15.143:0/603717252 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240d0f90 con 0x7f21240b45d0 2024-01-15T18:09:05.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.418+0000 7f2122000640 1 --2- 172.21.15.143:0/603717252 >> v2:172.21.15.143:3300/0 conn(0x7f21240d0150 0x7f21240c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:05.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.422+0000 7f2123400640 1 --2- 172.21.15.143:0/603717252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 0x7f21240c70c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f212408b300 tx=0x7f211c01f0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b66066880c580393 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:05.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.426+0000 7f2121600640 1 -- 172.21.15.143:0/603717252 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0b0440 con 0x7f21240b45d0 2024-01-15T18:09:05.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.426+0000 7f2121600640 1 -- 172.21.15.143:0/603717252 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c0b05a0 con 0x7f21240b45d0 2024-01-15T18:09:05.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.426+0000 7f2121600640 1 -- 172.21.15.143:0/603717252 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0b0930 con 0x7f21240b45d0 2024-01-15T18:09:05.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.426+0000 7f2129800640 1 -- 172.21.15.143:0/603717252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 msgr2=0x7f21240c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:05.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.426+0000 7f2129800640 1 --2- 172.21.15.143:0/603717252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 0x7f21240c70c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f212408b300 tx=0x7f211c01f0c0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.434+0000 7f2129800640 1 -- 172.21.15.143:0/603717252 shutdown_connections 2024-01-15T18:09:05.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.434+0000 7f2129800640 1 --2- 172.21.15.143:0/603717252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 0x7f21240c70c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.434+0000 7f2129800640 1 --2- 172.21.15.143:0/603717252 >> v2:172.21.15.143:3300/0 conn(0x7f21240d0150 0x7f21240c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.434+0000 7f2129800640 1 -- 172.21.15.143:0/603717252 >> 172.21.15.143:0/603717252 conn(0x7f21240d3030 msgr2=0x7f21240d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.438+0000 7f2129800640 1 -- 172.21.15.143:0/603717252 shutdown_connections 2024-01-15T18:09:05.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.442+0000 7f2129800640 1 -- 172.21.15.143:0/603717252 wait complete. 2024-01-15T18:09:05.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.442+0000 7f2129800640 1 Processor -- start 2024-01-15T18:09:05.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.442+0000 7f2129800640 1 -- start start 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2129800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240b3610 0x7f212415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2129800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 0x7f21240cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2129800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 0x7f21241593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2129800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240b6970 con 0x7f21240b3610 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2129800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240b6ef0 con 0x7f21240b45d0 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2129800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21240cc7f0 con 0x7f21240d0150 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 0x7f21241593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 0x7f21241593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41344/0 (socket says 172.21.15.143:41344) 2024-01-15T18:09:05.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 -- 172.21.15.143:0/2054185685 learned_addr learned my addr 172.21.15.143:0/2054185685 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 -- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 msgr2=0x7f21240cbd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 0x7f21240cbd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 -- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240b3610 msgr2=0x7f212415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240b3610 0x7f212415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 -- 172.21.15.143:0/2054185685 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c009a70 con 0x7f21240d0150 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f2123400640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 0x7f21241593e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f211c01ccf0 tx=0x7f211c01be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0bc860 con 0x7f21240d0150 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c0bc9c0 con 0x7f21240d0150 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.446+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0bcdb0 con 0x7f21240d0150 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.450+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21240ccad0 con 0x7f21240d0150 2024-01-15T18:09:05.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.450+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212415a1d0 con 0x7f21240d0150 2024-01-15T18:09:05.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.454+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f211c0c6400 con 0x7f21240d0150 2024-01-15T18:09:05.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.454+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2124067500 con 0x7f21240d0150 2024-01-15T18:09:05.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.454+0000 7f211aa00640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20fc066f10 0x7f20fc0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:05.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.454+0000 7f2122a00640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20fc066f10 0x7f20fc0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:05.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.454+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f211c0c4310 con 0x7f21240d0150 2024-01-15T18:09:05.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.458+0000 7f2122a00640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20fc066f10 0x7f20fc0693d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2114004fa0 tx=0x7f2114007ab0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:05.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.462+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f211c0f0a00 con 0x7f21240d0150 2024-01-15T18:09:05.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.818+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f21240b7060 con 0x7f21240d0150 2024-01-15T18:09:05.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.818+0000 7f211aa00640 1 -- 172.21.15.143:0/2054185685 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v51) v1 ==== 80+0+124 (secure 0 0 0) 0x7f211c0f0ba0 con 0x7f21240d0150 2024-01-15T18:09:05.829 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/1835320705 2024-01-16T18:09:00.967606+0000 2024-01-15T18:09:05.829 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6834/1835320705 2024-01-16T18:09:00.967606+0000 2024-01-15T18:09:05.830 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:09:05.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20fc066f10 msgr2=0x7f20fc0693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:05.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20fc066f10 0x7f20fc0693d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2114004fa0 tx=0x7f2114007ab0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 msgr2=0x7f21241593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:05.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 0x7f21241593e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f211c01ccf0 tx=0x7f211c01be60 comp rx=0 tx=0).stop 2024-01-15T18:09:05.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 shutdown_connections 2024-01-15T18:09:05.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20fc066f10 0x7f20fc0693d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21240d0150 0x7f21241593e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21240b45d0 0x7f21240cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 --2- 172.21.15.143:0/2054185685 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21240b3610 0x7f212415cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:05.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 >> 172.21.15.143:0/2054185685 conn(0x7f21240d3030 msgr2=0x7f21240cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:05.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 shutdown_connections 2024-01-15T18:09:05.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:05.826+0000 7f2129800640 1 -- 172.21.15.143:0/2054185685 wait complete. 2024-01-15T18:09:05.870 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:09:05.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:05.870 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:05.910 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:09:05.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:05.910 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:09:05.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[39710]: fuse finished with error 0 and tester_r 0 2024-01-15T18:09:06.079 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:09:06.079 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:06.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:06.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.254+0000 7f71d4600640 1 Processor -- start 2024-01-15T18:09:06.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.254+0000 7f71d4600640 1 -- start start 2024-01-15T18:09:06.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc0b0310 0x7f71cc0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d4600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0cbff0 con 0x7f71cc0cf740 2024-01-15T18:09:06.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d4600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0d0c50 con 0x7f71cc06bfb0 2024-01-15T18:09:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0cb6c0 con 0x7f71cc0b0310 2024-01-15T18:09:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d2600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f71cc0cf740 0x7f71cc0cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37768/0 (socket says 172.21.15.143:37768) 2024-01-15T18:09:06.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d2600640 1 -- 172.21.15.143:0/2231403766 learned_addr learned my addr 172.21.15.143:0/2231403766 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d1200640 1 --2- 172.21.15.143:0/2231403766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc0b0310 0x7f71cc0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660064367 0 0) 0x7f71cc0cbff0 con 0x7f71cc0cf740 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71ac003530 con 0x7f71cc0cf740 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165847053 0 0) 0x7f71cc0d0c50 con 0x7f71cc06bfb0 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71cc0cbff0 con 0x7f71cc06bfb0 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4195397348 0 0) 0x7f71ac003530 con 0x7f71cc0cf740 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d1200640 1 -- 172.21.15.143:0/2231403766 >> v1:172.21.15.204:6789/0 conn(0x7f71cc06bfb0 legacy=0x7f71cc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d1200640 1 -- 172.21.15.143:0/2231403766 >> v1:172.21.15.143:6789/0 conn(0x7f71cc0cf740 legacy=0x7f71cc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.258+0000 7f71d1200640 1 -- 172.21.15.143:0/2231403766 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0cb9a0 con 0x7f71cc0b0310 2024-01-15T18:09:06.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d1200640 1 --2- 172.21.15.143:0/2231403766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc0b0310 0x7f71cc0cf200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71b40097f0 tx=0x7f71b401c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1f46a32c32e50 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4001a00 con 0x7f71cc0b0310 2024-01-15T18:09:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b40a7050 con 0x7f71cc0b0310 2024-01-15T18:09:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71cbe00640 1 -- 172.21.15.143:0/2231403766 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40ab620 con 0x7f71cc0b0310 2024-01-15T18:09:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 -- 172.21.15.143:0/2231403766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc0b0310 msgr2=0x7f71cc0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 --2- 172.21.15.143:0/2231403766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc0b0310 0x7f71cc0cf200 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71b40097f0 tx=0x7f71b401c320 comp rx=0 tx=0).stop 2024-01-15T18:09:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 -- 172.21.15.143:0/2231403766 shutdown_connections 2024-01-15T18:09:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 --2- 172.21.15.143:0/2231403766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc0b0310 0x7f71cc0cf200 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 -- 172.21.15.143:0/2231403766 >> 172.21.15.143:0/2231403766 conn(0x7f71cc0d0ed0 msgr2=0x7f71cc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 -- 172.21.15.143:0/2231403766 shutdown_connections 2024-01-15T18:09:06.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.262+0000 7f71d4600640 1 -- 172.21.15.143:0/2231403766 wait complete. 2024-01-15T18:09:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 Processor -- start 2024-01-15T18:09:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 -- start start 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc06bfb0 0x7f71cc0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 0x7f71cc0ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 0x7f71cc15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0c50e0 con 0x7f71cc0cf740 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0c53c0 con 0x7f71cc0b0310 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0c56a0 con 0x7f71cc06bfb0 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 0x7f71cc15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc06bfb0 0x7f71cc0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 0x7f71cc0ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 0x7f71cc0ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:44006/0 (socket says 172.21.15.143:44006) 2024-01-15T18:09:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 0x7f71cc15c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58206/0 (socket says 172.21.15.143:58206) 2024-01-15T18:09:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 -- 172.21.15.143:0/95412462 learned_addr learned my addr 172.21.15.143:0/95412462 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:06.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc06bfb0 0x7f71cc0c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41376/0 (socket says 172.21.15.143:41376) 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 -- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc06bfb0 msgr2=0x7f71cc0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc06bfb0 0x7f71cc0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 -- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 msgr2=0x7f71cc0ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 0x7f71cc0ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 -- 172.21.15.143:0/95412462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4009420 con 0x7f71cc0cf740 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d1200640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 0x7f71cc0ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d2600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 0x7f71cc15c810 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f71c400cc80 tx=0x7f71c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c400ce90 con 0x7f71cc0cf740 2024-01-15T18:09:06.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c4012400 con 0x7f71cc0cf740 2024-01-15T18:09:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc0c5980 con 0x7f71cc0cf740 2024-01-15T18:09:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40127f0 con 0x7f71cc0cf740 2024-01-15T18:09:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.266+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc15cd50 con 0x7f71cc0cf740 2024-01-15T18:09:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.270+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc067500 con 0x7f71cc0cf740 2024-01-15T18:09:06.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.270+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f71c4012950 con 0x7f71cc0cf740 2024-01-15T18:09:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.270+0000 7f71c9600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a4066e90 0x7f71a4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.270+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f71c4085240 con 0x7f71cc0cf740 2024-01-15T18:09:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.270+0000 7f71d1c00640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a4066e90 0x7f71a4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.270+0000 7f71d1c00640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a4066e90 0x7f71a4069350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0011f0 tx=0x7f71bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.274+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f71c4051ee0 con 0x7f71cc0cf740 2024-01-15T18:09:06.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71cc0b6d10 con 0x7f71cc0cf740 2024-01-15T18:09:06.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71c9600640 1 -- 172.21.15.143:0/95412462 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f71c4055500 con 0x7f71cc0cf740 2024-01-15T18:09:06.431 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:06.431 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":51,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:03.981057+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6836/1835320705":"2024-01-16T18:09:00.967606+0000","172.21.15.143:6834/1835320705":"2024-01-16T18:09:00.967606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a4066e90 msgr2=0x7f71a4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a4066e90 0x7f71a4069350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0011f0 tx=0x7f71bc009040 comp rx=0 tx=0).stop 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 msgr2=0x7f71cc15c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 0x7f71cc15c810 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f71c400cc80 tx=0x7f71c400a430 comp rx=0 tx=0).stop 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 shutdown_connections 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71a4066e90 0x7f71a4069350 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f71cc0cf740 0x7f71cc15c810 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f71cc0b0310 0x7f71cc0ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 --2- 172.21.15.143:0/95412462 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f71cc06bfb0 0x7f71cc0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 >> 172.21.15.143:0/95412462 conn(0x7f71cc0d0ed0 msgr2=0x7f71cc06f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:06.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 shutdown_connections 2024-01-15T18:09:06.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.426+0000 7f71d4600640 1 -- 172.21.15.143:0/95412462 wait complete. 2024-01-15T18:09:06.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:09:06.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:09:06.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.586+0000 7fe5e3800640 1 Processor -- start 2024-01-15T18:09:06.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.586+0000 7fe5e3800640 1 -- start start 2024-01-15T18:09:06.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.586+0000 7fe5e3800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5dc071bb0 0x7fe5dc072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.586+0000 7fe5e3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5e3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0ceb50 con 0x7fe5dc143780 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5e3800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc072570 con 0x7fe5dc071bb0 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5e3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc15c110 con 0x7fe5dc15c5a0 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5dc071bb0 0x7fe5dc072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5e0e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe5dc143780 0x7fe5dc0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37774/0 (socket says 172.21.15.143:37774) 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe5dc071bb0 0x7fe5dc072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58220/0 (socket says 172.21.15.143:58220) 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5e0e00640 1 -- 172.21.15.143:0/614816158 learned_addr learned my addr 172.21.15.143:0/614816158 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5e1800640 1 --2- 172.21.15.143:0/614816158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 -- 172.21.15.143:0/614816158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 msgr2=0x7fe5dc15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 --2- 172.21.15.143:0/614816158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc15ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 -- 172.21.15.143:0/614816158 >> v1:172.21.15.143:6789/0 conn(0x7fe5dc143780 legacy=0x7fe5dc0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 -- 172.21.15.143:0/614816158 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc15b7e0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5dbe00640 1 --2- 172.21.15.143:0/614816158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc072030 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40097c0 tx=0x7fe5c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688ac6af3bcaaf10 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5db400640 1 -- 172.21.15.143:0/614816158 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40ab680 con 0x7fe5dc071bb0 2024-01-15T18:09:06.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5db400640 1 -- 172.21.15.143:0/614816158 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40ab7e0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.590+0000 7fe5db400640 1 -- 172.21.15.143:0/614816158 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40abb70 con 0x7fe5dc071bb0 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- 172.21.15.143:0/614816158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 msgr2=0x7fe5dc072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 --2- 172.21.15.143:0/614816158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc072030 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40097c0 tx=0x7fe5c401c480 comp rx=0 tx=0).stop 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- 172.21.15.143:0/614816158 shutdown_connections 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 --2- 172.21.15.143:0/614816158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 --2- 172.21.15.143:0/614816158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc072030 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- 172.21.15.143:0/614816158 >> 172.21.15.143:0/614816158 conn(0x7fe5dc06c3f0 msgr2=0x7fe5dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- 172.21.15.143:0/614816158 shutdown_connections 2024-01-15T18:09:06.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- 172.21.15.143:0/614816158 wait complete. 2024-01-15T18:09:06.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 Processor -- start 2024-01-15T18:09:06.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- start start 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc13a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc143780 0x7fe5dc1398e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc13c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc13c5b0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc138b90 con 0x7fe5dc143780 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.594+0000 7fe5e3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc138e00 con 0x7fe5dc15c5a0 2024-01-15T18:09:06.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc13a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc13a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58236/0 (socket says 172.21.15.143:58236) 2024-01-15T18:09:06.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 -- 172.21.15.143:0/2863526142 learned_addr learned my addr 172.21.15.143:0/2863526142 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:06.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5dbe00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc143780 0x7fe5dc1398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e1800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc13c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 -- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 msgr2=0x7fe5dc13c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc13c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 -- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc143780 msgr2=0x7fe5dc1398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc143780 0x7fe5dc1398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 -- 172.21.15.143:0/2863526142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4009420 con 0x7fe5dc071bb0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e0e00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc13a4b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c4d0 tx=0x7fe5cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc005cf0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc005e50 con 0x7fe5dc071bb0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc1390e0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0072b0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc13df80 con 0x7fe5dc071bb0 2024-01-15T18:09:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc067500 con 0x7fe5dc071bb0 2024-01-15T18:09:06.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fe5cc007410 con 0x7fe5dc071bb0 2024-01-15T18:09:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5d8c00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe5b4066e90 0x7fe5b4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.598+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fe5cc083eb0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.602+0000 7fe5dbe00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe5b4066e90 0x7fe5b4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.602+0000 7fe5dbe00640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe5b4066e90 0x7fe5b4069350 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40af000 tx=0x7fe5c40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.602+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe5cc050bd0 con 0x7fe5dc071bb0 2024-01-15T18:09:06.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5dc0ceb50 con 0x7fe5dc071bb0 2024-01-15T18:09:06.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5d8c00640 1 -- 172.21.15.143:0/2863526142 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5dc0ceb50 con 0x7fe5dc071bb0 2024-01-15T18:09:06.763 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe5b4066e90 msgr2=0x7fe5b4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe5b4066e90 0x7fe5b4069350 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40af000 tx=0x7fe5c40a7000 comp rx=0 tx=0).stop 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 msgr2=0x7fe5dc13a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc13a4b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c4d0 tx=0x7fe5cc00c990 comp rx=0 tx=0).stop 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 shutdown_connections 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe5b4066e90 0x7fe5b4069350 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe5dc15c5a0 0x7fe5dc13c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe5dc143780 0x7fe5dc1398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 --2- 172.21.15.143:0/2863526142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe5dc071bb0 0x7fe5dc13a4b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 >> 172.21.15.143:0/2863526142 conn(0x7fe5dc06c3f0 msgr2=0x7fe5dc06d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:06.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 shutdown_connections 2024-01-15T18:09:06.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.758+0000 7fe5e3800640 1 -- 172.21.15.143:0/2863526142 wait complete. 2024-01-15T18:09:06.785 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:09:06.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 Processor -- start 2024-01-15T18:09:06.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 -- start start 2024-01-15T18:09:06.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f89f00ac060 0x7f89f00af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f89f00adf10 0x7f89f00ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f00ad990 con 0x7f89f015d570 2024-01-15T18:09:06.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00725b0 con 0x7f89f00adf10 2024-01-15T18:09:06.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.918+0000 7f89f8000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f0072890 con 0x7f89f00ac060 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f89f00adf10 0x7f89f00ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f89f00adf10 0x7f89f00ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58248/0 (socket says 172.21.15.143:58248) 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f4c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f89f015d570 0x7f89f00af6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37788/0 (socket says 172.21.15.143:37788) 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 -- 172.21.15.143:0/501628014 learned_addr learned my addr 172.21.15.143:0/501628014 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f5600640 1 --2- 172.21.15.143:0/501628014 >> v2:172.21.15.204:3300/0 conn(0x7f89f00ac060 0x7f89f00af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 -- 172.21.15.143:0/501628014 >> v2:172.21.15.204:3300/0 conn(0x7f89f00ac060 msgr2=0x7f89f00af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 --2- 172.21.15.143:0/501628014 >> v2:172.21.15.204:3300/0 conn(0x7f89f00ac060 0x7f89f00af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 -- 172.21.15.143:0/501628014 >> v1:172.21.15.143:6789/0 conn(0x7f89f015d570 legacy=0x7f89f00af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 -- 172.21.15.143:0/501628014 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00b3f30 con 0x7f89f00adf10 2024-01-15T18:09:06.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f5600640 1 --2- 172.21.15.143:0/501628014 >> v2:172.21.15.204:3300/0 conn(0x7f89f00ac060 0x7f89f00af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:06.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89f6000640 1 --2- 172.21.15.143:0/501628014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f00adf10 0x7f89f00ad450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f89f00714a0 tx=0x7f89f00b0610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447e41d0e82e049f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89efe00640 1 -- 172.21.15.143:0/501628014 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80a58b0 con 0x7f89f00adf10 2024-01-15T18:09:06.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89efe00640 1 -- 172.21.15.143:0/501628014 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e80a5a10 con 0x7f89f00adf10 2024-01-15T18:09:06.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.922+0000 7f89efe00640 1 -- 172.21.15.143:0/501628014 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80a5da0 con 0x7f89f00adf10 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- 172.21.15.143:0/501628014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f00adf10 msgr2=0x7f89f00ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 --2- 172.21.15.143:0/501628014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f00adf10 0x7f89f00ad450 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f89f00714a0 tx=0x7f89f00b0610 comp rx=0 tx=0).stop 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- 172.21.15.143:0/501628014 shutdown_connections 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 --2- 172.21.15.143:0/501628014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f00adf10 0x7f89f00ad450 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 --2- 172.21.15.143:0/501628014 >> v2:172.21.15.204:3300/0 conn(0x7f89f00ac060 0x7f89f00af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- 172.21.15.143:0/501628014 >> 172.21.15.143:0/501628014 conn(0x7f89f0071a90 msgr2=0x7f89f0071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- 172.21.15.143:0/501628014 shutdown_connections 2024-01-15T18:09:06.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- 172.21.15.143:0/501628014 wait complete. 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 Processor -- start 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- start start 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89f00ac060 0x7f89f00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89f00adf10 0x7f89f014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 0x7f89f0148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f014a920 con 0x7f89f014da00 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f014adb0 con 0x7f89f00adf10 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f8000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0153f00 con 0x7f89f00ac060 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 0x7f89f0148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 0x7f89f0148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58250/0 (socket says 172.21.15.143:58250) 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.926+0000 7f89f6000640 1 -- 172.21.15.143:0/2794646873 learned_addr learned my addr 172.21.15.143:0/2794646873 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f4c00640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89f00adf10 0x7f89f014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f5600640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89f00ac060 0x7f89f00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f6000640 1 -- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89f00ac060 msgr2=0x7f89f00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f6000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89f00ac060 0x7f89f00cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f6000640 1 -- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89f00adf10 msgr2=0x7f89f014d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f6000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89f00adf10 0x7f89f014d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f6000640 1 -- 172.21.15.143:0/2794646873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8009b30 con 0x7f89f014da00 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f6000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 0x7f89f0148f40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f89e80af040 tx=0x7f89e8019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80bd070 con 0x7f89f014da00 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8009250 con 0x7f89f014da00 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80b88a0 con 0x7f89f014da00 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f01541e0 con 0x7f89f014da00 2024-01-15T18:09:06.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f014a5b0 con 0x7f89f014da00 2024-01-15T18:09:06.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f014aa90 con 0x7f89f014da00 2024-01-15T18:09:06.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f89e80b8ac0 con 0x7f89f014da00 2024-01-15T18:09:06.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89ed600640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f89c8066e90 0x7f89c8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:06.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f89e811f870 con 0x7f89f014da00 2024-01-15T18:09:06.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.930+0000 7f89f5600640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f89c8066e90 0x7f89c8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:06.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.934+0000 7f89f5600640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f89c8066e90 0x7f89c8069350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f89e0006fd0 tx=0x7f89e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:06.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:06.934+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f89e80ec600 con 0x7f89f014da00 2024-01-15T18:09:07.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.098+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f89f0067500 con 0x7f89f014da00 2024-01-15T18:09:07.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89ed600640 1 -- 172.21.15.143:0/2794646873 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v51) v1 ==== 80+0+124 (secure 0 0 0) 0x7f89e80efc20 con 0x7f89f014da00 2024-01-15T18:09:07.105 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/1835320705 2024-01-16T18:09:00.967606+0000 2024-01-15T18:09:07.106 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6834/1835320705 2024-01-16T18:09:00.967606+0000 2024-01-15T18:09:07.106 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f89c8066e90 msgr2=0x7f89c8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f89c8066e90 0x7f89c8069350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f89e0006fd0 tx=0x7f89e0008040 comp rx=0 tx=0).stop 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 msgr2=0x7f89f0148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 0x7f89f0148f40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f89e80af040 tx=0x7f89e8019770 comp rx=0 tx=0).stop 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 shutdown_connections 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f89c8066e90 0x7f89c8069350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89f014da00 0x7f89f0148f40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89f00adf10 0x7f89f014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 --2- 172.21.15.143:0/2794646873 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f89f00ac060 0x7f89f00cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 >> 172.21.15.143:0/2794646873 conn(0x7f89f0071a90 msgr2=0x7f89f00aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 shutdown_connections 2024-01-15T18:09:07.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.102+0000 7f89f8000640 1 -- 172.21.15.143:0/2794646873 wait complete. 2024-01-15T18:09:07.128 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:09:07.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:07.128 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:07.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33616]: fuse finished with error 0 and tester_r 0 2024-01-15T18:09:07.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:09:07.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:07.150 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:09:07.320 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:07.320 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:07.332 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:09:07.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:09:07.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:07.332 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:09:07.380 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:09:07.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:09:07.381 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:09:07.381 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:07.391 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:09:07.391 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:09:07.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:07.391 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:09:07.439 INFO:teuthology.orchestra.run.smithi204.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:09:07.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:09:07.440 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:09:07.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f522ba00640 1 Processor -- start 2024-01-15T18:09:07.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f522ba00640 1 -- start start 2024-01-15T18:09:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f522ba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f52240b0310 0x7f52240cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f522ba00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240cbff0 con 0x7f52240cf740 2024-01-15T18:09:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f522ba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240d0c50 con 0x7f522406bfb0 2024-01-15T18:09:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f522ba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52240cb6c0 con 0x7f52240b0310 2024-01-15T18:09:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5229a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f52240cf740 0x7f52240cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57586/0 (socket says 172.21.15.143:57586) 2024-01-15T18:09:07.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5229a00640 1 -- 172.21.15.143:0/2165163195 learned_addr learned my addr 172.21.15.143:0/2165163195 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5223e00640 1 --2- 172.21.15.143:0/2165163195 >> v2:172.21.15.204:3300/0 conn(0x7f52240b0310 0x7f52240cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5223400640 1 -- 172.21.15.143:0/2165163195 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2349835859 0 0) 0x7f52240cbff0 con 0x7f52240cf740 2024-01-15T18:09:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5223400640 1 -- 172.21.15.143:0/2165163195 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5204003530 con 0x7f52240cf740 2024-01-15T18:09:07.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5223e00640 1 -- 172.21.15.143:0/2165163195 >> v1:172.21.15.204:6789/0 conn(0x7f522406bfb0 legacy=0x7f522406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5223e00640 1 -- 172.21.15.143:0/2165163195 >> v1:172.21.15.143:6789/0 conn(0x7f52240cf740 legacy=0x7f52240cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.574+0000 7f5223e00640 1 -- 172.21.15.143:0/2165163195 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240cb9a0 con 0x7f52240b0310 2024-01-15T18:09:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f5223e00640 1 --2- 172.21.15.143:0/2165163195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240b0310 0x7f52240cf200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f520c0097c0 tx=0x7f520c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2147515f1d41c7e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f5223400640 1 -- 172.21.15.143:0/2165163195 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c0ab600 con 0x7f52240b0310 2024-01-15T18:09:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f5223400640 1 -- 172.21.15.143:0/2165163195 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c0a7050 con 0x7f52240b0310 2024-01-15T18:09:07.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f5223400640 1 -- 172.21.15.143:0/2165163195 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c0ab980 con 0x7f52240b0310 2024-01-15T18:09:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 -- 172.21.15.143:0/2165163195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240b0310 msgr2=0x7f52240cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 --2- 172.21.15.143:0/2165163195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240b0310 0x7f52240cf200 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f520c0097c0 tx=0x7f520c01c3c0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 -- 172.21.15.143:0/2165163195 shutdown_connections 2024-01-15T18:09:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 --2- 172.21.15.143:0/2165163195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240b0310 0x7f52240cf200 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 -- 172.21.15.143:0/2165163195 >> 172.21.15.143:0/2165163195 conn(0x7f52240d0ed0 msgr2=0x7f52240d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 -- 172.21.15.143:0/2165163195 shutdown_connections 2024-01-15T18:09:07.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.578+0000 7f522ba00640 1 -- 172.21.15.143:0/2165163195 wait complete. 2024-01-15T18:09:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 Processor -- start 2024-01-15T18:09:07.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 -- start start 2024-01-15T18:09:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 0x7f52241596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52240b0310 0x7f5224154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240cf740 0x7f52240c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224159bf0 con 0x7f522406bfb0 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224154b60 con 0x7f52240cf740 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240c7750 con 0x7f52240b0310 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 0x7f52241596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 0x7f52241596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44394/0 (socket says 172.21.15.143:44394) 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 -- 172.21.15.143:0/14494097 learned_addr learned my addr 172.21.15.143:0/14494097 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5223e00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52240b0310 0x7f5224154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229a00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240cf740 0x7f52240c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 -- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52240b0310 msgr2=0x7f5224154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52240b0310 0x7f5224154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 -- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240cf740 msgr2=0x7f52240c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240cf740 0x7f52240c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 -- 172.21.15.143:0/14494097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c009420 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5229000640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 0x7f52241596b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f521400c370 tx=0x7f521400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52140052d0 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214005430 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52240c7a30 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214007260 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.582+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240c5ab0 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.586+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224067500 con 0x7f522406bfb0 2024-01-15T18:09:07.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.586+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f52140073c0 con 0x7f522406bfb0 2024-01-15T18:09:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.586+0000 7f5220c00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51fc066e90 0x7f51fc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.586+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5214083eb0 con 0x7f522406bfb0 2024-01-15T18:09:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.586+0000 7f5223e00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51fc066e90 0x7f51fc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.586+0000 7f5223e00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51fc066e90 0x7f51fc069350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f520c0ad000 tx=0x7f520c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:07.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.590+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5214050c20 con 0x7f522406bfb0 2024-01-15T18:09:07.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.774+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52240d0a30 con 0x7f522406bfb0 2024-01-15T18:09:07.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.774+0000 7f5220c00640 1 -- 172.21.15.143:0/14494097 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f5214054240 con 0x7f522406bfb0 2024-01-15T18:09:07.780 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:07.780 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2863517742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":2863517742},{"type":"v1","addr":"172.21.15.143:6837","nonce":2863517742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/4241131349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":4241131349},{"type":"v1","addr":"172.21.15.204:6835","nonce":4241131349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1888637866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1888637866},{"type":"v1","addr":"172.21.15.204:6834","nonce":1888637866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1953730588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1953730588},{"type":"v1","addr":"172.21.15.143:6836","nonce":1953730588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-01-15T18:09:07.780 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51fc066e90 msgr2=0x7f51fc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51fc066e90 0x7f51fc069350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f520c0ad000 tx=0x7f520c008040 comp rx=0 tx=0).stop 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 msgr2=0x7f52241596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 0x7f52241596b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f521400c370 tx=0x7f521400c830 comp rx=0 tx=0).stop 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 shutdown_connections 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51fc066e90 0x7f51fc069350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52240cf740 0x7f52240c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52240b0310 0x7f5224154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 --2- 172.21.15.143:0/14494097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f522406bfb0 0x7f52241596b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 >> 172.21.15.143:0/14494097 conn(0x7f52240d0ed0 msgr2=0x7f522406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 shutdown_connections 2024-01-15T18:09:07.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.778+0000 7f522ba00640 1 -- 172.21.15.143:0/14494097 wait complete. 2024-01-15T18:09:07.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:09:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.934+0000 7f0b2e000640 1 Processor -- start 2024-01-15T18:09:07.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.934+0000 7f0b2e000640 1 -- start start 2024-01-15T18:09:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.934+0000 7f0b2e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.934+0000 7f0b2e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b280ceb50 con 0x7f0b2815c5a0 2024-01-15T18:09:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.934+0000 7f0b2e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2815c110 con 0x7f0b28143780 2024-01-15T18:09:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.934+0000 7f0b2e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28072570 con 0x7f0b28071bb0 2024-01-15T18:09:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b27e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0b2815c5a0 0x7f0b2815ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57590/0 (socket says 172.21.15.143:57590) 2024-01-15T18:09:07.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b27e00640 1 -- 172.21.15.143:0/1407952874 learned_addr learned my addr 172.21.15.143:0/1407952874 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26a00640 1 --2- 172.21.15.143:0/1407952874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589054383 0 0) 0x7f0b200014a0 con 0x7f0b2815c5a0 2024-01-15T18:09:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b08005740 con 0x7f0b2815c5a0 2024-01-15T18:09:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724270432 0 0) 0x7f0b2815c110 con 0x7f0b28143780 2024-01-15T18:09:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b200014a0 con 0x7f0b28143780 2024-01-15T18:09:07.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26a00640 1 -- 172.21.15.143:0/1407952874 >> v1:172.21.15.204:6789/0 conn(0x7f0b28143780 legacy=0x7f0b280cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26a00640 1 -- 172.21.15.143:0/1407952874 >> v1:172.21.15.143:6789/0 conn(0x7f0b2815c5a0 legacy=0x7f0b2815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26a00640 1 -- 172.21.15.143:0/1407952874 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2815b7e0 con 0x7f0b28071bb0 2024-01-15T18:09:07.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.938+0000 7f0b26a00640 1 --2- 172.21.15.143:0/1407952874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28072030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097f0 tx=0x7f0b1001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86eb07e5c4fd0ec6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100aa620 con 0x7f0b28071bb0 2024-01-15T18:09:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100aa780 con 0x7f0b28071bb0 2024-01-15T18:09:07.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b26000640 1 -- 172.21.15.143:0/1407952874 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100aab10 con 0x7f0b28071bb0 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- 172.21.15.143:0/1407952874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 msgr2=0x7f0b28072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1407952874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28072030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097f0 tx=0x7f0b1001c4c0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- 172.21.15.143:0/1407952874 shutdown_connections 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1407952874 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28072030 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- 172.21.15.143:0/1407952874 >> 172.21.15.143:0/1407952874 conn(0x7f0b2806c3f0 msgr2=0x7f0b2806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- 172.21.15.143:0/1407952874 shutdown_connections 2024-01-15T18:09:07.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- 172.21.15.143:0/1407952874 wait complete. 2024-01-15T18:09:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 Processor -- start 2024-01-15T18:09:07.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- start start 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 0x7f0b28158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2815c5a0 0x7f0b28132ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28159130 con 0x7f0b28143780 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28139af0 con 0x7f0b2815c5a0 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.942+0000 7f0b2e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28142e70 con 0x7f0b28071bb0 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 0x7f0b28158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 0x7f0b28158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44404/0 (socket says 172.21.15.143:44404) 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 -- 172.21.15.143:0/1417582278 learned_addr learned my addr 172.21.15.143:0/1417582278 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b27e00640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2815c5a0 0x7f0b28132ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b27400640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 -- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 msgr2=0x7f0b28141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 -- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2815c5a0 msgr2=0x7f0b28132ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2815c5a0 0x7f0b28132ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 -- 172.21.15.143:0/1417582278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b10009420 con 0x7f0b28143780 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b27400640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:07.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b26a00640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 0x7f0b28158bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097f0 tx=0x7f0b1001ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100aadc0 con 0x7f0b28143780 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b100be410 con 0x7f0b28143780 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28139d40 con 0x7f0b28143780 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100be7c0 con 0x7f0b28143780 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2813a300 con 0x7f0b28143780 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2813a4f0 con 0x7f0b28143780 2024-01-15T18:09:07.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.946+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f0b100be920 con 0x7f0b28143780 2024-01-15T18:09:07.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.950+0000 7f0b1f400640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0b00066e90 0x7f0b00069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:07.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.950+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f0b1011f330 con 0x7f0b28143780 2024-01-15T18:09:07.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.950+0000 7f0b27400640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0b00066e90 0x7f0b00069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:07.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.950+0000 7f0b27400640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0b00066e90 0x7f0b00069350 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b180011f0 tx=0x7f0b18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:07.954+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0b100ebf40 con 0x7f0b28143780 2024-01-15T18:09:08.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.102+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f0b28067500 con 0x7f0b28143780 2024-01-15T18:09:08.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.106+0000 7f0b1f400640 1 -- 172.21.15.143:0/1417582278 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v17) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0b100ef560 con 0x7f0b28143780 2024-01-15T18:09:08.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0b00066e90 msgr2=0x7f0b00069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0b00066e90 0x7f0b00069350 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0b180011f0 tx=0x7f0b18009040 comp rx=0 tx=0).stop 2024-01-15T18:09:08.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 msgr2=0x7f0b28158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 0x7f0b28158bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0b100097f0 tx=0x7f0b1001ad00 comp rx=0 tx=0).stop 2024-01-15T18:09:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 shutdown_connections 2024-01-15T18:09:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0b00066e90 0x7f0b00069350 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b2815c5a0 0x7f0b28132ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b28143780 0x7f0b28158bf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 --2- 172.21.15.143:0/1417582278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b28071bb0 0x7f0b28141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 >> 172.21.15.143:0/1417582278 conn(0x7f0b2806c3f0 msgr2=0x7f0b2815a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:08.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 shutdown_connections 2024-01-15T18:09:08.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.110+0000 7f0b2e000640 1 -- 172.21.15.143:0/1417582278 wait complete. 2024-01-15T18:09:08.136 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:09:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 Processor -- start 2024-01-15T18:09:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 -- start start 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82b015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82b0070dd0 0x7f82b0071260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b006d170 con 0x7f82b0150690 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b00717a0 con 0x7f82b0070dd0 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b7c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0150a70 con 0x7f82b006a490 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82b0150690 0x7f82b00b2510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57598/0 (socket says 172.21.15.143:57598) 2024-01-15T18:09:08.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5200640 1 -- 172.21.15.143:0/468083176 learned_addr learned my addr 172.21.15.143:0/468083176 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 --2- 172.21.15.143:0/468083176 >> v2:172.21.15.204:3300/0 conn(0x7f82b0070dd0 0x7f82b0071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82afe00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82b015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398843494 0 0) 0x7f82b006d170 con 0x7f82b0150690 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8290003530 con 0x7f82b0150690 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2924148633 0 0) 0x7f8290003530 con 0x7f82b0150690 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 msgr2=0x7f82b015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82b015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 >> v2:172.21.15.204:3300/0 conn(0x7f82b0070dd0 msgr2=0x7f82b0071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 --2- 172.21.15.143:0/468083176 >> v2:172.21.15.204:3300/0 conn(0x7f82b0070dd0 0x7f82b0071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b006a8d0 con 0x7f82b0150690 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f82a0001680 con 0x7f82b0150690 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 >> v1:172.21.15.143:6789/0 conn(0x7f82b0150690 legacy=0x7f82b00b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290003980 0x7f8290003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8290004d10 0x7f8290005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00717a0 con 0x7f8290004d10 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0150a70 con 0x7f8290003980 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829000b990 con 0x7f82b006a490 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8290004d10 0x7f8290005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82afe00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5200640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290003980 0x7f8290003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 -- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 msgr2=0x7f82900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 -- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290003980 msgr2=0x7f8290003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290003980 0x7f8290003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82b5c00640 1 -- 172.21.15.143:0/468083176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290003530 con 0x7f8290004d10 2024-01-15T18:09:08.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.270+0000 7f82afe00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b006a490 0x7f82900047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82b5c00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8290004d10 0x7f8290005190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009fc0 tx=0x7f82a801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e501bc7d15f8302c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a80a5cb0 con 0x7f8290004d10 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800f040 con 0x7f8290004d10 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82af400640 1 -- 172.21.15.143:0/468083176 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a80ad680 con 0x7f8290004d10 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82b7c00640 1 -- 172.21.15.143:0/468083176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8290004d10 msgr2=0x7f8290005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82b7c00640 1 --2- 172.21.15.143:0/468083176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8290004d10 0x7f8290005190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009fc0 tx=0x7f82a801cca0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82b5c00640 1 -- 172.21.15.143:0/468083176 reap_dead start 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82b7c00640 1 -- 172.21.15.143:0/468083176 shutdown_connections 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.274+0000 7f82b7c00640 1 -- 172.21.15.143:0/468083176 >> 172.21.15.143:0/468083176 conn(0x7f82b00b1610 msgr2=0x7f82b00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:08.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- 172.21.15.143:0/468083176 shutdown_connections 2024-01-15T18:09:08.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- 172.21.15.143:0/468083176 wait complete. 2024-01-15T18:09:08.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 Processor -- start 2024-01-15T18:09:08.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- start start 2024-01-15T18:09:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290004d10 0x7f82b01365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 0x7f82b0140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b00cd800 0x7f82b01411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0141e00 con 0x7f82b00cd380 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0141840 con 0x7f8290004d10 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0141b20 con 0x7f82b00cd800 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 0x7f82b0140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 0x7f82b0140ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44414/0 (socket says 172.21.15.143:44414) 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 -- 172.21.15.143:0/1034920713 learned_addr learned my addr 172.21.15.143:0/1034920713 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b5c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b00cd800 0x7f82b01411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b5200640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290004d10 0x7f82b01365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 -- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b00cd800 msgr2=0x7f82b01411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b00cd800 0x7f82b01411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 -- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290004d10 msgr2=0x7f82b01365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290004d10 0x7f82b01365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 -- 172.21.15.143:0/1034920713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b013a640 con 0x7f82b00cd380 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b5c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b00cd800 0x7f82b01411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82afe00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 0x7f82b0140ca0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f82980115f0 tx=0x7f8298011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298004220 con 0x7f82b00cd380 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298004380 con 0x7f82b00cd380 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829800f8f0 con 0x7f82b00cd380 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.278+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b013a920 con 0x7f82b00cd380 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.282+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b0136b30 con 0x7f82b00cd380 2024-01-15T18:09:08.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.282+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b0074b00 con 0x7f82b00cd380 2024-01-15T18:09:08.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.282+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f829800fa50 con 0x7f82b00cd380 2024-01-15T18:09:08.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.282+0000 7f82acc00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8288066ec0 0x7f8288069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.282+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8298084e30 con 0x7f82b00cd380 2024-01-15T18:09:08.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.282+0000 7f82b5200640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8288066ec0 0x7f8288069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.286+0000 7f82b5200640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8288066ec0 0x7f8288069380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c10 tx=0x7f82a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.286+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8298051aa0 con 0x7f82b00cd380 2024-01-15T18:09:08.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.438+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f82b013a4a0 con 0x7f82b00cd380 2024-01-15T18:09:08.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82acc00640 1 -- 172.21.15.143:0/1034920713 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f82b013a4a0 con 0x7f82b00cd380 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8288066ec0 msgr2=0x7f8288069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8288066ec0 0x7f8288069380 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f82a0000c10 tx=0x7f82a0009040 comp rx=0 tx=0).stop 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 msgr2=0x7f82b0140ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 0x7f82b0140ca0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f82980115f0 tx=0x7f8298011ab0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 shutdown_connections 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8288066ec0 0x7f8288069380 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82b00cd800 0x7f82b01411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00cd380 0x7f82b0140ca0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 --2- 172.21.15.143:0/1034920713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8290004d10 0x7f82b01365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.442+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 >> 172.21.15.143:0/1034920713 conn(0x7f82b00b1610 msgr2=0x7f82b006d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:08.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.446+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 shutdown_connections 2024-01-15T18:09:08.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.446+0000 7f82b7c00640 1 -- 172.21.15.143:0/1034920713 wait complete. 2024-01-15T18:09:08.469 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-01-15T18:09:08.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 Processor -- start 2024-01-15T18:09:08.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 -- start start 2024-01-15T18:09:08.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3ed415b260 0x7f3ed40cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ed40d19f0 0x7f3ed40cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed40c4d90 con 0x7f3ed40c5220 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed415b640 con 0x7f3ed40d19f0 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3edc800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40cf800 con 0x7f3ed415b260 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ed40d19f0 0x7f3ed40cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ed40d19f0 0x7f3ed40cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44428/0 (socket says 172.21.15.143:44428) 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 -- 172.21.15.143:0/1205717097 learned_addr learned my addr 172.21.15.143:0/1205717097 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9e00640 1 --2- 172.21.15.143:0/1205717097 >> v2:172.21.15.204:3300/0 conn(0x7f3ed415b260 0x7f3ed40cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 -- 172.21.15.143:0/1205717097 >> v2:172.21.15.204:3300/0 conn(0x7f3ed415b260 msgr2=0x7f3ed40cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 --2- 172.21.15.143:0/1205717097 >> v2:172.21.15.204:3300/0 conn(0x7f3ed415b260 0x7f3ed40cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 -- 172.21.15.143:0/1205717097 >> v1:172.21.15.204:6789/0 conn(0x7f3ed40c5220 legacy=0x7f3ed40c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.602+0000 7f3ed9400640 1 -- 172.21.15.143:0/1205717097 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40cfae0 con 0x7f3ed40d19f0 2024-01-15T18:09:08.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3ed9400640 1 --2- 172.21.15.143:0/1205717097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40d19f0 0x7f3ed40cdd50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40681e0 tx=0x7f3ebc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fe122132bb2c1b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3ed8a00640 1 -- 172.21.15.143:0/1205717097 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0ab600 con 0x7f3ed40d19f0 2024-01-15T18:09:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3ed8a00640 1 -- 172.21.15.143:0/1205717097 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ebc0a7050 con 0x7f3ed40d19f0 2024-01-15T18:09:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3ed8a00640 1 -- 172.21.15.143:0/1205717097 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0ab980 con 0x7f3ed40d19f0 2024-01-15T18:09:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 -- 172.21.15.143:0/1205717097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40d19f0 msgr2=0x7f3ed40cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 --2- 172.21.15.143:0/1205717097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40d19f0 0x7f3ed40cdd50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40681e0 tx=0x7f3ebc01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 -- 172.21.15.143:0/1205717097 shutdown_connections 2024-01-15T18:09:08.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 --2- 172.21.15.143:0/1205717097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40d19f0 0x7f3ed40cdd50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 --2- 172.21.15.143:0/1205717097 >> v2:172.21.15.204:3300/0 conn(0x7f3ed415b260 0x7f3ed40cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 -- 172.21.15.143:0/1205717097 >> 172.21.15.143:0/1205717097 conn(0x7f3ed40d0ed0 msgr2=0x7f3ed40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:08.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 -- 172.21.15.143:0/1205717097 shutdown_connections 2024-01-15T18:09:08.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.606+0000 7f3edc800640 1 -- 172.21.15.143:0/1205717097 wait complete. 2024-01-15T18:09:08.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 Processor -- start 2024-01-15T18:09:08.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 -- start start 2024-01-15T18:09:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 0x7f3ed409b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed40d19f0 0x7f3ed409cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ed415b260 0x7f3ed409d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4150d70 con 0x7f3ed40c5220 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed409d8d0 con 0x7f3ed40d19f0 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3edc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed409dbb0 con 0x7f3ed415b260 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 0x7f3ed409b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 0x7f3ed409b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44438/0 (socket says 172.21.15.143:44438) 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 -- 172.21.15.143:0/473875127 learned_addr learned my addr 172.21.15.143:0/473875127 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3eda800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ed415b260 0x7f3ed409d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9400640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed40d19f0 0x7f3ed409cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 -- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ed415b260 msgr2=0x7f3ed409d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ed415b260 0x7f3ed409d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 -- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed40d19f0 msgr2=0x7f3ed409cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed40d19f0 0x7f3ed409cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 -- 172.21.15.143:0/473875127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0094d0 con 0x7f3ed40c5220 2024-01-15T18:09:08.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed9e00640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 0x7f3ed409b290 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e870 tx=0x7f3ec400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4009510 con 0x7f3ed40c5220 2024-01-15T18:09:08.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec400eec0 con 0x7f3ed40c5220 2024-01-15T18:09:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.610+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400cb40 con 0x7f3ed40c5220 2024-01-15T18:09:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed409de90 con 0x7f3ed40c5220 2024-01-15T18:09:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed4153500 con 0x7f3ed40c5220 2024-01-15T18:09:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed4067500 con 0x7f3ed40c5220 2024-01-15T18:09:08.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f3ec400cca0 con 0x7f3ed40c5220 2024-01-15T18:09:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3ed2000640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eac066ec0 0x7f3eac069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3ec4084c60 con 0x7f3ed40c5220 2024-01-15T18:09:08.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3ed9400640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eac066ec0 0x7f3eac069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.614+0000 7f3ed9400640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eac066ec0 0x7f3eac069380 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ad000 tx=0x7f3ebc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.618+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ec40519c0 con 0x7f3ed40c5220 2024-01-15T18:09:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.770+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) v1 -- 0x7f3ed40b6d50 con 0x7f3ed40c5220 2024-01-15T18:09:08.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.774+0000 7f3ed2000640 1 -- 172.21.15.143:0/473875127 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v19) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3ec40170b0 con 0x7f3ed40c5220 2024-01-15T18:09:08.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eac066ec0 msgr2=0x7f3eac069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eac066ec0 0x7f3eac069380 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ad000 tx=0x7f3ebc0a6000 comp rx=0 tx=0).stop 2024-01-15T18:09:08.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 msgr2=0x7f3ed409b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 0x7f3ed409b290 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400e870 tx=0x7f3ec400ec70 comp rx=0 tx=0).stop 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 shutdown_connections 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eac066ec0 0x7f3eac069380 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ed415b260 0x7f3ed409d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ed40d19f0 0x7f3ed409cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 --2- 172.21.15.143:0/473875127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ed40c5220 0x7f3ed409b290 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 >> 172.21.15.143:0/473875127 conn(0x7f3ed40d0ed0 msgr2=0x7f3ed40c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 shutdown_connections 2024-01-15T18:09:08.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.778+0000 7f3edc800640 1 -- 172.21.15.143:0/473875127 wait complete. 2024-01-15T18:09:08.804 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones' 2024-01-15T18:09:08.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.934+0000 7f2117e00640 1 Processor -- start 2024-01-15T18:09:08.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.934+0000 7f2117e00640 1 -- start start 2024-01-15T18:09:08.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2117e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2110143780 0x7f21100cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2117e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21100722c0 0x7f211015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2117e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21100ceb50 con 0x7f21100ce520 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2117e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211015c110 con 0x7f21100722c0 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2117e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211015b7e0 con 0x7f2110143780 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21100722c0 0x7f211015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21100722c0 0x7f211015c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44452/0 (socket says 172.21.15.143:44452) 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 -- 172.21.15.143:0/632231789 learned_addr learned my addr 172.21.15.143:0/632231789 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115400640 1 --2- 172.21.15.143:0/632231789 >> v2:172.21.15.204:3300/0 conn(0x7f2110143780 0x7f21100cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 -- 172.21.15.143:0/632231789 >> v2:172.21.15.204:3300/0 conn(0x7f2110143780 msgr2=0x7f21100cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 --2- 172.21.15.143:0/632231789 >> v2:172.21.15.204:3300/0 conn(0x7f2110143780 0x7f21100cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 -- 172.21.15.143:0/632231789 >> v1:172.21.15.204:6789/0 conn(0x7f21100ce520 legacy=0x7f2110071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.938+0000 7f2115e00640 1 -- 172.21.15.143:0/632231789 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211015bac0 con 0x7f21100722c0 2024-01-15T18:09:08.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2115e00640 1 --2- 172.21.15.143:0/632231789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100722c0 0x7f211015c5a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f211006be30 tx=0x7f210801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bc3f5c1ad89ba server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f210fe00640 1 -- 172.21.15.143:0/632231789 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080a57a0 con 0x7f21100722c0 2024-01-15T18:09:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f210fe00640 1 -- 172.21.15.143:0/632231789 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21080a5900 con 0x7f21100722c0 2024-01-15T18:09:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f210fe00640 1 -- 172.21.15.143:0/632231789 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21080a5c90 con 0x7f21100722c0 2024-01-15T18:09:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 -- 172.21.15.143:0/632231789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100722c0 msgr2=0x7f211015c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 --2- 172.21.15.143:0/632231789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100722c0 0x7f211015c5a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f211006be30 tx=0x7f210801c9d0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 -- 172.21.15.143:0/632231789 shutdown_connections 2024-01-15T18:09:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 --2- 172.21.15.143:0/632231789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100722c0 0x7f211015c5a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 --2- 172.21.15.143:0/632231789 >> v2:172.21.15.204:3300/0 conn(0x7f2110143780 0x7f21100cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 -- 172.21.15.143:0/632231789 >> 172.21.15.143:0/632231789 conn(0x7f211006c3f0 msgr2=0x7f211006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 -- 172.21.15.143:0/632231789 shutdown_connections 2024-01-15T18:09:08.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 -- 172.21.15.143:0/632231789 wait complete. 2024-01-15T18:09:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 Processor -- start 2024-01-15T18:09:08.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.942+0000 7f2117e00640 1 -- start start 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21100722c0 0x7f2110158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 0x7f21100ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2110143780 0x7f21100b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2110141290 con 0x7f21100ce520 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2110159520 con 0x7f2110143780 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21101326c0 con 0x7f21100722c0 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 0x7f21100ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 0x7f21100ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44454/0 (socket says 172.21.15.143:44454) 2024-01-15T18:09:08.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2115e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2110143780 0x7f21100b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 -- 172.21.15.143:0/3879253293 learned_addr learned my addr 172.21.15.143:0/3879253293 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2115400640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21100722c0 0x7f2110158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 -- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21100722c0 msgr2=0x7f2110158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21100722c0 0x7f2110158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 -- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2110143780 msgr2=0x7f21100b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2110143780 0x7f21100b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 -- 172.21.15.143:0/3879253293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2108009b30 con 0x7f21100ce520 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2115e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2110143780 0x7f21100b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2114a00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 0x7f21100ad2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f20f800c500 tx=0x7f20f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8005220 con 0x7f21100ce520 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f8005380 con 0x7f21100ce520 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f8007270 con 0x7f21100ce520 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2110140c60 con 0x7f21100ce520 2024-01-15T18:09:08.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21100ad830 con 0x7f21100ce520 2024-01-15T18:09:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.946+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21101413d0 con 0x7f21100ce520 2024-01-15T18:09:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.950+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f20f80073d0 con 0x7f21100ce520 2024-01-15T18:09:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.950+0000 7f210d600640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20e8066e90 0x7f20e8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.950+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f20f8083d50 con 0x7f21100ce520 2024-01-15T18:09:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.950+0000 7f2115400640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20e8066e90 0x7f20e8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.950+0000 7f2115400640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20e8066e90 0x7f20e8069350 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2100009ac0 tx=0x7f2100009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:08.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:08.954+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21101413d0 con 0x7f21100ce520 2024-01-15T18:09:09.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.154+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]} v 0) v1 -- 0x7f21100ceb50 con 0x7f21100ce520 2024-01-15T18:09:09.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f210d600640 1 -- 172.21.15.143:0/3879253293 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]}]=0 v219) v1 ==== 153+0+0 (secure 0 0 0) 0x7f21100ceb50 con 0x7f21100ce520 2024-01-15T18:09:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20e8066e90 msgr2=0x7f20e8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:09.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20e8066e90 0x7f20e8069350 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2100009ac0 tx=0x7f2100009190 comp rx=0 tx=0).stop 2024-01-15T18:09:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 msgr2=0x7f21100ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 0x7f21100ad2f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f20f800c500 tx=0x7f20f800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 shutdown_connections 2024-01-15T18:09:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20e8066e90 0x7f20e8069350 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2110143780 0x7f21100b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21100ce520 0x7f21100ad2f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 --2- 172.21.15.143:0/3879253293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21100722c0 0x7f2110158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.662+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 >> 172.21.15.143:0/3879253293 conn(0x7f211006c3f0 msgr2=0x7f21100af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:09.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.666+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 shutdown_connections 2024-01-15T18:09:09.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.666+0000 7f2117e00640 1 -- 172.21.15.143:0/3879253293 wait complete. 2024-01-15T18:09:09.692 INFO:tasks.cephfs_test_runner:X = number of clones issued. ... ok 2024-01-15T18:09:09.692 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgress) X = number of clones issued. 2024-01-15T18:09:09.692 INFO:tasks.cephfs_test_runner:test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:09:09.693 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:09.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 Processor -- start 2024-01-15T18:09:09.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 -- start start 2024-01-15T18:09:09.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 0x7f0c600aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:09.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0c6015d510 0x7f0c6006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:09.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c60160eb0 con 0x7f0c6006a490 2024-01-15T18:09:09.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c60161190 con 0x7f0c6015d510 2024-01-15T18:09:09.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c66600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60161470 con 0x7f0c6015d130 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c5f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 0x7f0c600aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c5ea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0c6015d510 0x7f0c6006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.826+0000 7f0c5f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 0x7f0c600aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:43448/0 (socket says 172.21.15.143:43448) 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5f400640 1 -- 172.21.15.143:0/1878184697 learned_addr learned my addr 172.21.15.143:0/1878184697 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5f400640 1 -- 172.21.15.143:0/1878184697 >> v2:172.21.15.204:3300/0 conn(0x7f0c6015d510 msgr2=0x7f0c6006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5f400640 1 --2- 172.21.15.143:0/1878184697 >> v2:172.21.15.204:3300/0 conn(0x7f0c6015d510 0x7f0c6006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5f400640 1 -- 172.21.15.143:0/1878184697 >> v1:172.21.15.204:6789/0 conn(0x7f0c6006a490 legacy=0x7f0c60071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5f400640 1 -- 172.21.15.143:0/1878184697 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60161750 con 0x7f0c6015d130 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5e000640 1 -- 172.21.15.143:0/1878184697 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710767128 0 0) 0x7f0c60160eb0 con 0x7f0c6006a490 2024-01-15T18:09:09.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5ea00640 1 --2- 172.21.15.143:0/1878184697 >> v2:172.21.15.204:3300/0 conn(0x7f0c6015d510 0x7f0c6006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:09.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5f400640 1 --2- 172.21.15.143:0/1878184697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 0x7f0c600aeed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009870 tx=0x7f0c5001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520ca299aedce7a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:09.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5e000640 1 -- 172.21.15.143:0/1878184697 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500a9620 con 0x7f0c6015d130 2024-01-15T18:09:09.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5e000640 1 -- 172.21.15.143:0/1878184697 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c500a9780 con 0x7f0c6015d130 2024-01-15T18:09:09.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.830+0000 7f0c5e000640 1 -- 172.21.15.143:0/1878184697 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c500a9b10 con 0x7f0c6015d130 2024-01-15T18:09:09.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- 172.21.15.143:0/1878184697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 msgr2=0x7f0c600aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:09.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 --2- 172.21.15.143:0/1878184697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 0x7f0c600aeed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009870 tx=0x7f0c5001c540 comp rx=0 tx=0).stop 2024-01-15T18:09:09.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- 172.21.15.143:0/1878184697 shutdown_connections 2024-01-15T18:09:09.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 --2- 172.21.15.143:0/1878184697 >> v2:172.21.15.204:3300/0 conn(0x7f0c6015d510 0x7f0c6006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 --2- 172.21.15.143:0/1878184697 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d130 0x7f0c600aeed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- 172.21.15.143:0/1878184697 >> 172.21.15.143:0/1878184697 conn(0x7f0c600b1b90 msgr2=0x7f0c600b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:09.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- 172.21.15.143:0/1878184697 shutdown_connections 2024-01-15T18:09:09.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- 172.21.15.143:0/1878184697 wait complete. 2024-01-15T18:09:09.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 Processor -- start 2024-01-15T18:09:09.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- start start 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6006a490 0x7f0c60153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 0x7f0c601603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d510 0x7f0c60149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60154ad0 con 0x7f0c6015d130 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60160920 con 0x7f0c6006a490 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.834+0000 7f0c66600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015eba0 con 0x7f0c6015d510 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 0x7f0c601603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 0x7f0c601603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44470/0 (socket says 172.21.15.143:44470) 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 -- 172.21.15.143:0/2961366107 learned_addr learned my addr 172.21.15.143:0/2961366107 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:09.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5f400640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6006a490 0x7f0c60153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5fe00640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d510 0x7f0c60149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 -- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d510 msgr2=0x7f0c60149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d510 0x7f0c60149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 -- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6006a490 msgr2=0x7f0c60153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6006a490 0x7f0c60153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 -- 172.21.15.143:0/2961366107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48009420 con 0x7f0c6015d130 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5fe00640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d510 0x7f0c60149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c5ea00640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 0x7f0c601603e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e980 tx=0x7f0c4800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800ca00 con 0x7f0c6015d130 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4800cb60 con 0x7f0c6015d130 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c500094d0 con 0x7f0c6015d130 2024-01-15T18:09:09.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48013470 con 0x7f0c6015d130 2024-01-15T18:09:09.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6015e520 con 0x7f0c6015d130 2024-01-15T18:09:09.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6015ee10 con 0x7f0c6015d130 2024-01-15T18:09:09.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.838+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f0c480135d0 con 0x7f0c6015d130 2024-01-15T18:09:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.842+0000 7f0c57400640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0c38066e90 0x7f0c38069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.842+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f0c48084db0 con 0x7f0c6015d130 2024-01-15T18:09:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.842+0000 7f0c5f400640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0c38066e90 0x7f0c38069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:09.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.842+0000 7f0c5f400640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0c38066e90 0x7f0c38069350 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0c500ae000 tx=0x7f0c500b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:09.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.846+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0c48051ab0 con 0x7f0c6015d130 2024-01-15T18:09:09.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.994+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c60067500 con 0x7f0c6015d130 2024-01-15T18:09:10.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.994+0000 7f0c57400640 1 -- 172.21.15.143:0/2961366107 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f0c480550d0 con 0x7f0c6015d130 2024-01-15T18:09:10.000 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:10.000 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":51,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:03.981057+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6836/1835320705":"2024-01-16T18:09:00.967606+0000","172.21.15.143:6834/1835320705":"2024-01-16T18:09:00.967606+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:10.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0c38066e90 msgr2=0x7f0c38069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0c38066e90 0x7f0c38069350 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0c500ae000 tx=0x7f0c500b3040 comp rx=0 tx=0).stop 2024-01-15T18:09:10.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 msgr2=0x7f0c601603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 0x7f0c601603e0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800e980 tx=0x7f0c4800ee40 comp rx=0 tx=0).stop 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 shutdown_connections 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0c38066e90 0x7f0c38069350 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0c6015d510 0x7f0c60149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0c6015d130 0x7f0c601603e0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 --2- 172.21.15.143:0/2961366107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0c6006a490 0x7f0c60153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 >> 172.21.15.143:0/2961366107 conn(0x7f0c600b1b90 msgr2=0x7f0c600cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 shutdown_connections 2024-01-15T18:09:10.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:09.998+0000 7f0c66600640 1 -- 172.21.15.143:0/2961366107 wait complete. 2024-01-15T18:09:10.022 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:09:10.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:09:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 Processor -- start 2024-01-15T18:09:10.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 -- start start 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29d00b24b0 0x7f29d00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29d00b3bf0 0x7f29d00b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d0070bf0 con 0x7f29d00ce690 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d015dba0 con 0x7f29d00b24b0 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d8600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d00b4920 con 0x7f29d00b3bf0 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29d00b24b0 0x7f29d00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29d00b24b0 0x7f29d00ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44478/0 (socket says 172.21.15.143:44478) 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 -- 172.21.15.143:0/4170348052 learned_addr learned my addr 172.21.15.143:0/4170348052 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:10.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d6600640 1 --2- 172.21.15.143:0/4170348052 >> v2:172.21.15.204:3300/0 conn(0x7f29d00b3bf0 0x7f29d00b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 -- 172.21.15.143:0/4170348052 >> v2:172.21.15.204:3300/0 conn(0x7f29d00b3bf0 msgr2=0x7f29d00b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 --2- 172.21.15.143:0/4170348052 >> v2:172.21.15.204:3300/0 conn(0x7f29d00b3bf0 0x7f29d00b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 -- 172.21.15.143:0/4170348052 >> v1:172.21.15.204:6789/0 conn(0x7f29d00ce690 legacy=0x7f29d00b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.158+0000 7f29d5200640 1 -- 172.21.15.143:0/4170348052 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d00b4c00 con 0x7f29d00b24b0 2024-01-15T18:09:10.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d5200640 1 --2- 172.21.15.143:0/4170348052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b24b0 0x7f29d00ac8e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f29d015e870 tx=0x7f29b801cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3a587b1a26dee9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:10.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29cfe00640 1 -- 172.21.15.143:0/4170348052 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80aa650 con 0x7f29d00b24b0 2024-01-15T18:09:10.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29cfe00640 1 -- 172.21.15.143:0/4170348052 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b80aa7b0 con 0x7f29d00b24b0 2024-01-15T18:09:10.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29cfe00640 1 -- 172.21.15.143:0/4170348052 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80aab40 con 0x7f29d00b24b0 2024-01-15T18:09:10.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 -- 172.21.15.143:0/4170348052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b24b0 msgr2=0x7f29d00ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 --2- 172.21.15.143:0/4170348052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b24b0 0x7f29d00ac8e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f29d015e870 tx=0x7f29b801cf30 comp rx=0 tx=0).stop 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 -- 172.21.15.143:0/4170348052 shutdown_connections 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 --2- 172.21.15.143:0/4170348052 >> v2:172.21.15.204:3300/0 conn(0x7f29d00b3bf0 0x7f29d00b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 --2- 172.21.15.143:0/4170348052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b24b0 0x7f29d00ac8e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 -- 172.21.15.143:0/4170348052 >> 172.21.15.143:0/4170348052 conn(0x7f29d006c3f0 msgr2=0x7f29d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 -- 172.21.15.143:0/4170348052 shutdown_connections 2024-01-15T18:09:10.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.162+0000 7f29d8600640 1 -- 172.21.15.143:0/4170348052 wait complete. 2024-01-15T18:09:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 Processor -- start 2024-01-15T18:09:10.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 -- start start 2024-01-15T18:09:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d00b24b0 0x7f29d0142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 0x7f29d0155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00ce690 0x7f29d013d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0159610 con 0x7f29d00b3bf0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d015a950 con 0x7f29d00b24b0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d0142d50 con 0x7f29d00ce690 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 0x7f29d0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 0x7f29d0155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44486/0 (socket says 172.21.15.143:44486) 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 -- 172.21.15.143:0/874463996 learned_addr learned my addr 172.21.15.143:0/874463996 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5c00640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d00b24b0 0x7f29d0142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d6600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00ce690 0x7f29d013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 -- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00ce690 msgr2=0x7f29d013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00ce690 0x7f29d013d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 -- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d00b24b0 msgr2=0x7f29d0142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d00b24b0 0x7f29d0142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 -- 172.21.15.143:0/874463996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b80094d0 con 0x7f29d00b3bf0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d5200640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 0x7f29d0155860 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f29b80ac040 tx=0x7f29b801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:10.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80b3800 con 0x7f29d00b3bf0 2024-01-15T18:09:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b80b3980 con 0x7f29d00b3bf0 2024-01-15T18:09:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80b3d30 con 0x7f29d00b3bf0 2024-01-15T18:09:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.166+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d01598b0 con 0x7f29d00b3bf0 2024-01-15T18:09:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.170+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d0159e70 con 0x7f29d00b3bf0 2024-01-15T18:09:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.170+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d0067500 con 0x7f29d00b3bf0 2024-01-15T18:09:10.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.170+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f29b80b2380 con 0x7f29d00b3bf0 2024-01-15T18:09:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.170+0000 7f29cd600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29a8066e90 0x7f29a8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.170+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f29b811f3a0 con 0x7f29d00b3bf0 2024-01-15T18:09:10.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.170+0000 7f29d5c00640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29a8066e90 0x7f29a8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.174+0000 7f29d5c00640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29a8066e90 0x7f29a8069350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29c0007fd0 tx=0x7f29c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:10.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.174+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29b80ec0f0 con 0x7f29d00b3bf0 2024-01-15T18:09:10.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.326+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29d0167ef0 con 0x7f29d00b3bf0 2024-01-15T18:09:10.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29cd600640 1 -- 172.21.15.143:0/874463996 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29b80ef710 con 0x7f29d00b3bf0 2024-01-15T18:09:10.334 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:09:10.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29a8066e90 msgr2=0x7f29a8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29a8066e90 0x7f29a8069350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f29c0007fd0 tx=0x7f29c0009040 comp rx=0 tx=0).stop 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 msgr2=0x7f29d0155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 0x7f29d0155860 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f29b80ac040 tx=0x7f29b801ad10 comp rx=0 tx=0).stop 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 shutdown_connections 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29a8066e90 0x7f29a8069350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29d00ce690 0x7f29d013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29d00b3bf0 0x7f29d0155860 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 --2- 172.21.15.143:0/874463996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29d00b24b0 0x7f29d0142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 >> 172.21.15.143:0/874463996 conn(0x7f29d006c3f0 msgr2=0x7f29d0146970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 shutdown_connections 2024-01-15T18:09:10.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.330+0000 7f29d8600640 1 -- 172.21.15.143:0/874463996 wait complete. 2024-01-15T18:09:10.356 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones' 2024-01-15T18:09:10.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 Processor -- start 2024-01-15T18:09:10.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 -- start start 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46bc1607c0 0x7f46bc06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f46bc0cbf50 0x7f46bc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46bc0cffd0 con 0x7f46bc0cfb30 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46bc0d1ce0 con 0x7f46bc1607c0 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c4400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46bc0d02b0 con 0x7f46bc0cbf50 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.486+0000 7f46c1a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46bc1607c0 0x7f46bc06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f46bc0cfb30 0x7f46bc06bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57608/0 (socket says 172.21.15.143:57608) 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1000640 1 -- 172.21.15.143:0/1173568987 learned_addr learned my addr 172.21.15.143:0/1173568987 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:10.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c2400640 1 --2- 172.21.15.143:0/1173568987 >> v2:172.21.15.204:3300/0 conn(0x7f46bc0cbf50 0x7f46bc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1a00640 1 -- 172.21.15.143:0/1173568987 >> v2:172.21.15.204:3300/0 conn(0x7f46bc0cbf50 msgr2=0x7f46bc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:10.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1a00640 1 --2- 172.21.15.143:0/1173568987 >> v2:172.21.15.204:3300/0 conn(0x7f46bc0cbf50 0x7f46bc0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1a00640 1 -- 172.21.15.143:0/1173568987 >> v1:172.21.15.143:6789/0 conn(0x7f46bc0cfb30 legacy=0x7f46bc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46bbe00640 1 -- 172.21.15.143:0/1173568987 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900035556 0 0) 0x7f46bc0cffd0 con 0x7f46bc0cfb30 2024-01-15T18:09:10.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1a00640 1 -- 172.21.15.143:0/1173568987 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc0cb470 con 0x7f46bc1607c0 2024-01-15T18:09:10.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46c1a00640 1 --2- 172.21.15.143:0/1173568987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc06b7b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cef70 tx=0x7f46ac01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef71aa8be124b837 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:10.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46bbe00640 1 -- 172.21.15.143:0/1173568987 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0a8950 con 0x7f46bc1607c0 2024-01-15T18:09:10.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46bbe00640 1 -- 172.21.15.143:0/1173568987 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac0a8ab0 con 0x7f46bc1607c0 2024-01-15T18:09:10.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.490+0000 7f46bbe00640 1 -- 172.21.15.143:0/1173568987 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0a8e40 con 0x7f46bc1607c0 2024-01-15T18:09:10.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- 172.21.15.143:0/1173568987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 msgr2=0x7f46bc06b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 --2- 172.21.15.143:0/1173568987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc06b7b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cef70 tx=0x7f46ac01ec50 comp rx=0 tx=0).stop 2024-01-15T18:09:10.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- 172.21.15.143:0/1173568987 shutdown_connections 2024-01-15T18:09:10.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 --2- 172.21.15.143:0/1173568987 >> v2:172.21.15.204:3300/0 conn(0x7f46bc0cbf50 0x7f46bc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 --2- 172.21.15.143:0/1173568987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc06b7b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- 172.21.15.143:0/1173568987 >> 172.21.15.143:0/1173568987 conn(0x7f46bc0d0e20 msgr2=0x7f46bc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:10.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- 172.21.15.143:0/1173568987 shutdown_connections 2024-01-15T18:09:10.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- 172.21.15.143:0/1173568987 wait complete. 2024-01-15T18:09:10.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 Processor -- start 2024-01-15T18:09:10.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- start start 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46bc0cbf50 0x7f46bc153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46bc0cfb30 0x7f46bc1f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc0c5ee0 con 0x7f46bc1607c0 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc156a70 con 0x7f46bc0cfb30 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc0c5540 con 0x7f46bc0cbf50 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c2400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c2400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44512/0 (socket says 172.21.15.143:44512) 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c2400640 1 -- 172.21.15.143:0/539010901 learned_addr learned my addr 172.21.15.143:0/539010901 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c1000640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46bc0cfb30 0x7f46bc1f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.494+0000 7f46c1a00640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46bc0cbf50 0x7f46bc153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c2400640 1 -- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46bc0cbf50 msgr2=0x7f46bc153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c2400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46bc0cbf50 0x7f46bc153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c2400640 1 -- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46bc0cfb30 msgr2=0x7f46bc1f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c2400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46bc0cfb30 0x7f46bc1f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c2400640 1 -- 172.21.15.143:0/539010901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009420 con 0x7f46bc1607c0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c2400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc155ff0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f46b400efc0 tx=0x7f46b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4014480 con 0x7f46bc1607c0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b400f040 con 0x7f46bc1607c0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc0c5820 con 0x7f46bc1607c0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b4014820 con 0x7f46bc1607c0 2024-01-15T18:09:10.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc153780 con 0x7f46bc1607c0 2024-01-15T18:09:10.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46bc06d4b0 con 0x7f46bc1607c0 2024-01-15T18:09:10.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f46b4014980 con 0x7f46bc1607c0 2024-01-15T18:09:10.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46b9600640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4694066ec0 0x7f4694069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:10.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.498+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f46b4085b20 con 0x7f46bc1607c0 2024-01-15T18:09:10.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.502+0000 7f46c1a00640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4694066ec0 0x7f4694069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:10.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.502+0000 7f46c1a00640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4694066ec0 0x7f4694069380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0b2000 tx=0x7f46ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:10.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.502+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46b4052620 con 0x7f46bc1607c0 2024-01-15T18:09:10.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:10.706+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]} v 0) v1 -- 0x7f46bc0c6020 con 0x7f46bc1607c0 2024-01-15T18:09:11.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.670+0000 7f46b9600640 1 -- 172.21.15.143:0/539010901 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]}]=0 v221) v1 ==== 157+0+0 (secure 0 0 0) 0x7f46bc0c6020 con 0x7f46bc1607c0 2024-01-15T18:09:11.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4694066ec0 msgr2=0x7f4694069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4694066ec0 0x7f4694069380 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0b2000 tx=0x7f46ac008040 comp rx=0 tx=0).stop 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 msgr2=0x7f46bc155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc155ff0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f46b400efc0 tx=0x7f46b400c490 comp rx=0 tx=0).stop 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 shutdown_connections 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4694066ec0 0x7f4694069380 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46bc1607c0 0x7f46bc155ff0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46bc0cfb30 0x7f46bc1f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 --2- 172.21.15.143:0/539010901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46bc0cbf50 0x7f46bc153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 >> 172.21.15.143:0/539010901 conn(0x7f46bc0d0e20 msgr2=0x7f46bc15dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:11.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 shutdown_connections 2024-01-15T18:09:11.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.674+0000 7f46c4400640 1 -- 172.21.15.143:0/539010901 wait complete. 2024-01-15T18:09:11.702 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:09:11.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 Processor -- start 2024-01-15T18:09:11.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 -- start start 2024-01-15T18:09:11.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fec4006bfb0 0x7fec4006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:11.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fec400b0310 0x7fec400cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:11.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cbff0 con 0x7fec400cc9c0 2024-01-15T18:09:11.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec400d0c50 con 0x7fec4006bfb0 2024-01-15T18:09:11.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec46800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4006c8d0 con 0x7fec400b0310 2024-01-15T18:09:11.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fec4006bfb0 0x7fec4006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:11.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fec400cc9c0 0x7fec400cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57620/0 (socket says 172.21.15.143:57620) 2024-01-15T18:09:11.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3fe00640 1 -- 172.21.15.143:0/498992998 learned_addr learned my addr 172.21.15.143:0/498992998 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:11.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3f400640 1 -- 172.21.15.143:0/498992998 >> v2:172.21.15.204:3300/0 conn(0x7fec400b0310 msgr2=0x7fec400cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:11.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3f400640 1 --2- 172.21.15.143:0/498992998 >> v2:172.21.15.204:3300/0 conn(0x7fec400b0310 0x7fec400cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3f400640 1 -- 172.21.15.143:0/498992998 >> v1:172.21.15.143:6789/0 conn(0x7fec400cc9c0 legacy=0x7fec400cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:11.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3f400640 1 -- 172.21.15.143:0/498992998 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400cb6c0 con 0x7fec4006bfb0 2024-01-15T18:09:11.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.842+0000 7fec3e000640 1 -- 172.21.15.143:0/498992998 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773387117 0 0) 0x7fec400cbff0 con 0x7fec400cc9c0 2024-01-15T18:09:11.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec3f400640 1 --2- 172.21.15.143:0/498992998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec4006c390 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fec4006bf00 tx=0x7fec3001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6060e5d7f1a161e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:11.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec3e000640 1 -- 172.21.15.143:0/498992998 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300ac600 con 0x7fec4006bfb0 2024-01-15T18:09:11.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec3e000640 1 -- 172.21.15.143:0/498992998 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300a8050 con 0x7fec4006bfb0 2024-01-15T18:09:11.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec3e000640 1 -- 172.21.15.143:0/498992998 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300ac980 con 0x7fec4006bfb0 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 -- 172.21.15.143:0/498992998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 msgr2=0x7fec4006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 --2- 172.21.15.143:0/498992998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec4006c390 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fec4006bf00 tx=0x7fec3001c3a0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 -- 172.21.15.143:0/498992998 shutdown_connections 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 --2- 172.21.15.143:0/498992998 >> v2:172.21.15.204:3300/0 conn(0x7fec400b0310 0x7fec400cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 --2- 172.21.15.143:0/498992998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec4006c390 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 -- 172.21.15.143:0/498992998 >> 172.21.15.143:0/498992998 conn(0x7fec400d0ed0 msgr2=0x7fec400d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 -- 172.21.15.143:0/498992998 shutdown_connections 2024-01-15T18:09:11.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.846+0000 7fec46800640 1 -- 172.21.15.143:0/498992998 wait complete. 2024-01-15T18:09:11.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 Processor -- start 2024-01-15T18:09:11.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 -- start start 2024-01-15T18:09:11.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec400c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:11.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec400b0310 0x7fec4014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:11.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec400cc9c0 0x7fec40159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:11.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401544e0 con 0x7fec4006bfb0 2024-01-15T18:09:11.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400ca3a0 con 0x7fec400cc9c0 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400ca680 con 0x7fec400b0310 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec400c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec400c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44520/0 (socket says 172.21.15.143:44520) 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 -- 172.21.15.143:0/1107339058 learned_addr learned my addr 172.21.15.143:0/1107339058 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3ea00640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec400b0310 0x7fec4014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3fe00640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec400cc9c0 0x7fec40159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 -- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec400b0310 msgr2=0x7fec4014d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec400b0310 0x7fec4014d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 -- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec400cc9c0 msgr2=0x7fec40159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec400cc9c0 0x7fec40159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 -- 172.21.15.143:0/1107339058 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec30009420 con 0x7fec4006bfb0 2024-01-15T18:09:11.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec3f400640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec400c9d30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fec30018fb0 tx=0x7fec3001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:11.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300b7620 con 0x7fec4006bfb0 2024-01-15T18:09:11.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300b7780 con 0x7fec4006bfb0 2024-01-15T18:09:11.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300b7af0 con 0x7fec4006bfb0 2024-01-15T18:09:11.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec400ca960 con 0x7fec4006bfb0 2024-01-15T18:09:11.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.850+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4014de20 con 0x7fec4006bfb0 2024-01-15T18:09:11.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.854+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec40154650 con 0x7fec4006bfb0 2024-01-15T18:09:11.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.854+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fec300b3040 con 0x7fec4006bfb0 2024-01-15T18:09:11.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.854+0000 7fec37400640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec18066e90 0x7fec18069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:11.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.854+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fec3011f290 con 0x7fec4006bfb0 2024-01-15T18:09:11.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.854+0000 7fec3ea00640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec18066e90 0x7fec18069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:11.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.854+0000 7fec3ea00640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec18066e90 0x7fec18069350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:11.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:11.858+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec40154650 con 0x7fec4006bfb0 2024-01-15T18:09:12.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.010+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fec400cae10 con 0x7fec4006bfb0 2024-01-15T18:09:12.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec37400640 1 -- 172.21.15.143:0/1107339058 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fec300ebe80 con 0x7fec4006bfb0 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec18066e90 msgr2=0x7fec18069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec18066e90 0x7fec18069350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28008040 comp rx=0 tx=0).stop 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 msgr2=0x7fec400c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec400c9d30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fec30018fb0 tx=0x7fec3001abe0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 shutdown_connections 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec18066e90 0x7fec18069350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec400cc9c0 0x7fec40159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec400b0310 0x7fec4014d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 --2- 172.21.15.143:0/1107339058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec4006bfb0 0x7fec400c9d30 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 >> 172.21.15.143:0/1107339058 conn(0x7fec400d0ed0 msgr2=0x7fec4006fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:12.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 shutdown_connections 2024-01-15T18:09:12.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.014+0000 7fec46800640 1 -- 172.21.15.143:0/1107339058 wait complete. 2024-01-15T18:09:12.040 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:09:12.040 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:09:12.135 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:09:12.214 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:09:12.214 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:12.214 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:12.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:09:12.223 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:09:12.223 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:12.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:09:12.232 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:12.232 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:12.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:09:12.273 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:09:12.273 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:12.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:09:12.283 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:09:12.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:09:12.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.446+0000 7f52e4600640 1 Processor -- start 2024-01-15T18:09:12.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.446+0000 7f52e4600640 1 -- start start 2024-01-15T18:09:12.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e4600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52dc0b3bf0 0x7f52dc0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e4600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52dc070bf0 con 0x7f52dc0b1da0 2024-01-15T18:09:12.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e4600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52dc15dba0 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc0b4920 con 0x7f52dc0ce690 2024-01-15T18:09:12.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52dc0b3bf0 0x7f52dc0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52dc0b3bf0 0x7f52dc0b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44526/0 (socket says 172.21.15.143:44526) 2024-01-15T18:09:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 -- 172.21.15.143:0/3799191188 learned_addr learned my addr 172.21.15.143:0/3799191188 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e1c00640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 -- 172.21.15.143:0/3799191188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 msgr2=0x7f52dc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 -- 172.21.15.143:0/3799191188 >> v1:172.21.15.204:6789/0 conn(0x7f52dc0b1da0 legacy=0x7f52dc0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:12.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e2600640 1 -- 172.21.15.143:0/3799191188 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc0b4c00 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.450+0000 7f52e1c00640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:12.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e2600640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc0b4080 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d401d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b85ddab55222a24 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:12.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52dbe00640 1 -- 172.21.15.143:0/3799191188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40a5840 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52dbe00640 1 -- 172.21.15.143:0/3799191188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d40a59a0 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52dbe00640 1 -- 172.21.15.143:0/3799191188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40a5d30 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 -- 172.21.15.143:0/3799191188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 msgr2=0x7f52dc0b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc0b4080 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009e50 tx=0x7f52d401d400 comp rx=0 tx=0).stop 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 -- 172.21.15.143:0/3799191188 shutdown_connections 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc0b4080 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 --2- 172.21.15.143:0/3799191188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 -- 172.21.15.143:0/3799191188 >> 172.21.15.143:0/3799191188 conn(0x7f52dc06c3f0 msgr2=0x7f52dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 -- 172.21.15.143:0/3799191188 shutdown_connections 2024-01-15T18:09:12.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 -- 172.21.15.143:0/3799191188 wait complete. 2024-01-15T18:09:12.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 Processor -- start 2024-01-15T18:09:12.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.454+0000 7f52e4600640 1 -- start start 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc0b1da0 0x7f52dc13d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1591d0 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc13f570 con 0x7f52dc0b1da0 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc13ddd0 con 0x7f52dc0ce690 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44530/0 (socket says 172.21.15.143:44530) 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 -- 172.21.15.143:0/71115035 learned_addr learned my addr 172.21.15.143:0/71115035 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e2600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1c00640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc0b1da0 0x7f52dc13d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 -- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 msgr2=0x7f52dc06c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc06c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 -- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc0b1da0 msgr2=0x7f52dc13d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc0b1da0 0x7f52dc13d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 -- 172.21.15.143:0/71115035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009b30 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e1200640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc158970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c450 tx=0x7f52c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012830 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40129b0 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc13f220 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012d60 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc0afd20 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.458+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc067500 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.462+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f52c401c460 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.462+0000 7f52d9600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b4066e90 0x7f52b4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.462+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f52c4084870 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.462+0000 7f52e1c00640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b4066e90 0x7f52b4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.462+0000 7f52e1c00640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b4066e90 0x7f52b4069350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006fd0 tx=0x7f52cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:12.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.466+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52c40515d0 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52dc06e260 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52d9600640 1 -- 172.21.15.143:0/71115035 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f52c4054bf0 con 0x7f52dc0b3bf0 2024-01-15T18:09:12.650 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:12.650 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2863517742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":2863517742},{"type":"v1","addr":"172.21.15.143:6837","nonce":2863517742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5273,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/4241131349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":4241131349},{"type":"v1","addr":"172.21.15.204:6835","nonce":4241131349}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5695,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1888637866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1888637866},{"type":"v1","addr":"172.21.15.204:6834","nonce":1888637866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6213,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/1953730588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1953730588},{"type":"v1","addr":"172.21.15.143:6836","nonce":1953730588}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-01-15T18:09:12.651 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 29 2024-01-15T18:09:12.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b4066e90 msgr2=0x7f52b4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b4066e90 0x7f52b4069350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f52cc006fd0 tx=0x7f52cc008040 comp rx=0 tx=0).stop 2024-01-15T18:09:12.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 msgr2=0x7f52dc158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc158970 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c450 tx=0x7f52c400c910 comp rx=0 tx=0).stop 2024-01-15T18:09:12.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 shutdown_connections 2024-01-15T18:09:12.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f52b4066e90 0x7f52b4069350 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52dc0ce690 0x7f52dc06c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52dc0b3bf0 0x7f52dc158970 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 --2- 172.21.15.143:0/71115035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52dc0b1da0 0x7f52dc13d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 >> 172.21.15.143:0/71115035 conn(0x7f52dc06c3f0 msgr2=0x7f52dc1404a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:12.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 shutdown_connections 2024-01-15T18:09:12.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.646+0000 7f52e4600640 1 -- 172.21.15.143:0/71115035 wait complete. 2024-01-15T18:09:12.673 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:09:12.673 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:09:12.673 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:09:12.673 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:09:12.673 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:09:12.673 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:09:12.674 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:09:12.674 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:09:12.674 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:09:12.674 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:09:12.674 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:09:12.674 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:09:12.674 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:09:12.675 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:09:12.675 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:09:12.675 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:09:12.688 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:09:12.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:09:12.689 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:09:12.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:09:12.691 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:09:12.691 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:09:12.692 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:09:12.692 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:09:12.735 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:09:12.735 DEBUG:teuthology.parallel:result is None 2024-01-15T18:09:12.737 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:09:12.738 DEBUG:teuthology.parallel:result is None 2024-01-15T18:09:12.738 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:09:12.738 DEBUG:teuthology.parallel:result is None 2024-01-15T18:09:12.740 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:09:12.740 DEBUG:teuthology.parallel:result is None 2024-01-15T18:09:12.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:09:12.823 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:09:12.824 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:09:12.826 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:09:12.827 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:09:12.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 Processor -- start 2024-01-15T18:09:12.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 -- start start 2024-01-15T18:09:12.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f883006bfb0 0x7f883006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f88300b0310 0x7f88300cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88300cbff0 con 0x7f88300cc9c0 2024-01-15T18:09:12.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88300d0c50 con 0x7f88300b0310 2024-01-15T18:09:12.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8837e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883006c8d0 con 0x7f883006bfb0 2024-01-15T18:09:12.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f88300cc9c0 0x7f88300cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:56464/0 (socket says 172.21.15.143:56464) 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835e00640 1 -- 172.21.15.143:0/3915862825 learned_addr learned my addr 172.21.15.143:0/3915862825 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835400640 1 --2- 172.21.15.143:0/3915862825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f883006bfb0 0x7f883006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8834a00640 1 --2- 172.21.15.143:0/3915862825 >> v2:172.21.15.204:3300/0 conn(0x7f88300b0310 0x7f88300cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835400640 1 -- 172.21.15.143:0/3915862825 >> v2:172.21.15.204:3300/0 conn(0x7f88300b0310 msgr2=0x7f88300cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835400640 1 --2- 172.21.15.143:0/3915862825 >> v2:172.21.15.204:3300/0 conn(0x7f88300b0310 0x7f88300cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835400640 1 -- 172.21.15.143:0/3915862825 >> v1:172.21.15.204:6789/0 conn(0x7f88300cc9c0 legacy=0x7f88300cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8835400640 1 -- 172.21.15.143:0/3915862825 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88300cb6c0 con 0x7f883006bfb0 2024-01-15T18:09:12.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.882+0000 7f8834a00640 1 --2- 172.21.15.143:0/3915862825 >> v2:172.21.15.204:3300/0 conn(0x7f88300b0310 0x7f88300cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:12.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8835400640 1 --2- 172.21.15.143:0/3915862825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f883006bfb0 0x7f883006c390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88200097c0 tx=0x7f882001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14db2edd30573b93 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:12.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f882fe00640 1 -- 172.21.15.143:0/3915862825 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200ac620 con 0x7f883006bfb0 2024-01-15T18:09:12.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f882fe00640 1 -- 172.21.15.143:0/3915862825 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88200ac780 con 0x7f883006bfb0 2024-01-15T18:09:12.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f882fe00640 1 -- 172.21.15.143:0/3915862825 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200acb10 con 0x7f883006bfb0 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 -- 172.21.15.143:0/3915862825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f883006bfb0 msgr2=0x7f883006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 --2- 172.21.15.143:0/3915862825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f883006bfb0 0x7f883006c390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88200097c0 tx=0x7f882001c3c0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 -- 172.21.15.143:0/3915862825 shutdown_connections 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 --2- 172.21.15.143:0/3915862825 >> v2:172.21.15.204:3300/0 conn(0x7f88300b0310 0x7f88300cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 --2- 172.21.15.143:0/3915862825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f883006bfb0 0x7f883006c390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 -- 172.21.15.143:0/3915862825 >> 172.21.15.143:0/3915862825 conn(0x7f88300d0ed0 msgr2=0x7f88300d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 -- 172.21.15.143:0/3915862825 shutdown_connections 2024-01-15T18:09:12.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.886+0000 7f8837e00640 1 -- 172.21.15.143:0/3915862825 wait complete. 2024-01-15T18:09:12.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 Processor -- start 2024-01-15T18:09:12.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 -- start start 2024-01-15T18:09:12.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f883006bfb0 0x7f8830156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88300b0310 0x7f883015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 0x7f8830158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8830157270 con 0x7f88300cc9c0 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8830157550 con 0x7f883006bfb0 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8830157830 con 0x7f88300b0310 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 0x7f8830158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 0x7f8830158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44556/0 (socket says 172.21.15.143:44556) 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 -- 172.21.15.143:0/3244494397 learned_addr learned my addr 172.21.15.143:0/3244494397 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835400640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f883006bfb0 0x7f8830156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8834a00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88300b0310 0x7f883015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 -- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88300b0310 msgr2=0x7f883015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88300b0310 0x7f883015bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 -- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f883006bfb0 msgr2=0x7f8830156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f883006bfb0 0x7f8830156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 -- 172.21.15.143:0/3244494397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8818009420 con 0x7f88300cc9c0 2024-01-15T18:09:12.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8834a00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88300b0310 0x7f883015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:12.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8835e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 0x7f8830158310 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f882800cbc0 tx=0x7f882800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:12.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800ad40 con 0x7f88300cc9c0 2024-01-15T18:09:12.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828012400 con 0x7f88300cc9c0 2024-01-15T18:09:12.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8820009420 con 0x7f88300cc9c0 2024-01-15T18:09:12.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800aea0 con 0x7f88300cc9c0 2024-01-15T18:09:12.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.890+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8830157df0 con 0x7f88300cc9c0 2024-01-15T18:09:12.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.894+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8830157fe0 con 0x7f88300cc9c0 2024-01-15T18:09:12.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.894+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f8828007070 con 0x7f88300cc9c0 2024-01-15T18:09:12.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.894+0000 7f882d600640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8808066e90 0x7f8808069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:12.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.894+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8828084ad0 con 0x7f88300cc9c0 2024-01-15T18:09:12.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.894+0000 7f8835400640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8808066e90 0x7f8808069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:12.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.894+0000 7f8835400640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8808066e90 0x7f8808069350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88200ae000 tx=0x7f8820008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:12.898+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f88280517f0 con 0x7f88300cc9c0 2024-01-15T18:09:13.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:13.062+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8830067500 con 0x7f88300cc9c0 2024-01-15T18:09:14.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.622+0000 7f882d600640 1 -- 172.21.15.143:0/3244494397 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8828017020 con 0x7f88300cc9c0 2024-01-15T18:09:14.626 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.622+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8808066e90 msgr2=0x7f8808069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.622+0000 7f8837e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8808066e90 0x7f8808069350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88200ae000 tx=0x7f8820008040 comp rx=0 tx=0).stop 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.622+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 msgr2=0x7f8830158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.622+0000 7f8837e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 0x7f8830158310 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f882800cbc0 tx=0x7f882800a430 comp rx=0 tx=0).stop 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 shutdown_connections 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8808066e90 0x7f8808069350 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88300cc9c0 0x7f8830158310 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88300b0310 0x7f883015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 --2- 172.21.15.143:0/3244494397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f883006bfb0 0x7f8830156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 >> 172.21.15.143:0/3244494397 conn(0x7f88300d0ed0 msgr2=0x7f883006f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 shutdown_connections 2024-01-15T18:09:14.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.626+0000 7f8837e00640 1 -- 172.21.15.143:0/3244494397 wait complete. 2024-01-15T18:09:14.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:09:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 Processor -- start 2024-01-15T18:09:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 -- start start 2024-01-15T18:09:14.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa14006bfb0 0x7fa14006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa1400ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1400cbff0 con 0x7fa1400b0310 2024-01-15T18:09:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400d0c50 con 0x7fa14006bfb0 2024-01-15T18:09:14.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa144c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14006c8d0 con 0x7fa1400d0210 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13d400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa1400b0310 0x7fa1400cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57650/0 (socket says 172.21.15.143:57650) 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13d400640 1 -- 172.21.15.143:0/2831541669 learned_addr learned my addr 172.21.15.143:0/2831541669 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13e800640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa1400ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13de00640 1 --2- 172.21.15.143:0/2831541669 >> v2:172.21.15.204:3300/0 conn(0x7fa14006bfb0 0x7fa14006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181213339 0 0) 0x7fa1400cbff0 con 0x7fa1400b0310 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa120003530 con 0x7fa1400b0310 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3748680245 0 0) 0x7fa120003530 con 0x7fa1400b0310 2024-01-15T18:09:14.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 msgr2=0x7fa1400ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa1400ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 >> v2:172.21.15.204:3300/0 conn(0x7fa14006bfb0 msgr2=0x7fa14006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 --2- 172.21.15.143:0/2831541669 >> v2:172.21.15.204:3300/0 conn(0x7fa14006bfb0 0x7fa14006c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400cb6c0 con 0x7fa1400b0310 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.790+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fa128001680 con 0x7fa1400b0310 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 >> v1:172.21.15.143:6789/0 conn(0x7fa1400b0310 legacy=0x7fa1400cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa120003980 0x7fa120003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1200047d0 0x7fa120004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa120005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400d0c50 con 0x7fa120003980 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14006c8d0 con 0x7fa1200047d0 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12000b9c0 con 0x7fa1400d0210 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa120003980 0x7fa120003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13d400640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1200047d0 0x7fa120004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13e800640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa120005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 -- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 msgr2=0x7fa120005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa120005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 -- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1200047d0 msgr2=0x7fa120004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1200047d0 0x7fa120004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 -- 172.21.15.143:0/2831541669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120003530 con 0x7fa120003980 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13d400640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1200047d0 0x7fa120004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:14.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13e800640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1400d0210 0x7fa120005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:14.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13de00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa120003980 0x7fa120003da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa130009ae0 tx=0x7fa13001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b14c68195975c04 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:14.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1300a6470 con 0x7fa120003980 2024-01-15T18:09:14.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1300a65d0 con 0x7fa120003980 2024-01-15T18:09:14.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.794+0000 7fa13ca00640 1 -- 172.21.15.143:0/2831541669 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1300a6980 con 0x7fa120003980 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- 172.21.15.143:0/2831541669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa120003980 msgr2=0x7fa120003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 --2- 172.21.15.143:0/2831541669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa120003980 0x7fa120003da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa130009ae0 tx=0x7fa13001ca00 comp rx=0 tx=0).stop 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa13e800640 1 -- 172.21.15.143:0/2831541669 reap_dead start 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- 172.21.15.143:0/2831541669 shutdown_connections 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- 172.21.15.143:0/2831541669 >> 172.21.15.143:0/2831541669 conn(0x7fa1400d0ed0 msgr2=0x7fa1400d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- 172.21.15.143:0/2831541669 shutdown_connections 2024-01-15T18:09:14.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- 172.21.15.143:0/2831541669 wait complete. 2024-01-15T18:09:14.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 Processor -- start 2024-01-15T18:09:14.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- start start 2024-01-15T18:09:14.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa120003980 0x7fa140154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 0x7fa14013ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa14013f2a0 0x7fa140156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140156820 con 0x7fa140154b40 2024-01-15T18:09:14.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401561c0 con 0x7fa14013f2a0 2024-01-15T18:09:14.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa144c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14013f7e0 con 0x7fa120003980 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.798+0000 7fa13d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 0x7fa14013ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 0x7fa14013ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44584/0 (socket says 172.21.15.143:44584) 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 -- 172.21.15.143:0/2294724899 learned_addr learned my addr 172.21.15.143:0/2294724899 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13de00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa120003980 0x7fa140154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13e800640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa14013f2a0 0x7fa140156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 -- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa120003980 msgr2=0x7fa140154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa120003980 0x7fa140154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 -- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa14013f2a0 msgr2=0x7fa140156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa14013f2a0 0x7fa140156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:14.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 -- 172.21.15.143:0/2294724899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14013fac0 con 0x7fa140154b40 2024-01-15T18:09:14.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa13d400640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 0x7fa14013ed60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa128001ab0 tx=0x7fa12800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:14.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12800ee60 con 0x7fa140154b40 2024-01-15T18:09:14.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12800c8d0 con 0x7fa140154b40 2024-01-15T18:09:14.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa140156440 con 0x7fa140154b40 2024-01-15T18:09:14.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12800cc80 con 0x7fa140154b40 2024-01-15T18:09:14.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa140157180 con 0x7fa140154b40 2024-01-15T18:09:14.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa140156960 con 0x7fa140154b40 2024-01-15T18:09:14.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fa12801d460 con 0x7fa140154b40 2024-01-15T18:09:14.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.802+0000 7fa136000640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa118066e90 0x7fa118069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:14.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.806+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fa128084980 con 0x7fa140154b40 2024-01-15T18:09:14.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.806+0000 7fa13de00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa118066e90 0x7fa118069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:14.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.806+0000 7fa13de00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa118066e90 0x7fa118069350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa1300ad000 tx=0x7fa1300046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:14.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:14.806+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa140156960 con 0x7fa140154b40 2024-01-15T18:09:15.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.010+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa140156960 con 0x7fa140154b40 2024-01-15T18:09:15.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.010+0000 7fa136000640 1 -- 172.21.15.143:0/2294724899 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa140156960 con 0x7fa140154b40 2024-01-15T18:09:15.016 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:15.016 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:09:15.017 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:09:15.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:09:15.019 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.020 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:09:15.022 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.023 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2024-01-15T18:09:15.024 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:09:15.025 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:09:15.026 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:09:15.027 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.028 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:15.029 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:09:15.029 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa118066e90 msgr2=0x7fa118069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa118066e90 0x7fa118069350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa1300ad000 tx=0x7fa1300046d0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 msgr2=0x7fa14013ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 0x7fa14013ed60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa128001ab0 tx=0x7fa12800ec10 comp rx=0 tx=0).stop 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 shutdown_connections 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa118066e90 0x7fa118069350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa14013f2a0 0x7fa140156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa140154b40 0x7fa14013ed60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 --2- 172.21.15.143:0/2294724899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa120003980 0x7fa140154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 >> 172.21.15.143:0/2294724899 conn(0x7fa1400d0ed0 msgr2=0x7fa1400d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 shutdown_connections 2024-01-15T18:09:15.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.014+0000 7fa144c00640 1 -- 172.21.15.143:0/2294724899 wait complete. 2024-01-15T18:09:15.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:15.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 Processor -- start 2024-01-15T18:09:15.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 -- start start 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8d0c0cdf20 0x7f8d0c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d0c0c6230 0x7f8d0c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c0c4d90 con 0x7f8d0c15b260 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c15b640 con 0x7f8d0c0c6230 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d12800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c0cf800 con 0x7f8d0c0cdf20 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d0c0c6230 0x7f8d0c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8d0c0c6230 0x7f8d0c0cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44592/0 (socket says 172.21.15.143:44592) 2024-01-15T18:09:15.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 -- 172.21.15.143:0/1739231256 learned_addr learned my addr 172.21.15.143:0/1739231256 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:15.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0aa00640 1 --2- 172.21.15.143:0/1739231256 >> v2:172.21.15.204:3300/0 conn(0x7f8d0c0cdf20 0x7f8d0c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 -- 172.21.15.143:0/1739231256 >> v2:172.21.15.204:3300/0 conn(0x7f8d0c0cdf20 msgr2=0x7f8d0c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:15.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 --2- 172.21.15.143:0/1739231256 >> v2:172.21.15.204:3300/0 conn(0x7f8d0c0cdf20 0x7f8d0c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 -- 172.21.15.143:0/1739231256 >> v1:172.21.15.204:6789/0 conn(0x7f8d0c15b260 legacy=0x7f8d0c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:15.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0be00640 1 -- 172.21.15.143:0/1739231256 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0cfae0 con 0x7f8d0c0c6230 2024-01-15T18:09:15.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.170+0000 7f8d0aa00640 1 --2- 172.21.15.143:0/1739231256 >> v2:172.21.15.204:3300/0 conn(0x7f8d0c0cdf20 0x7f8d0c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:15.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d0be00640 1 --2- 172.21.15.143:0/1739231256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0cc020 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c09ee40 tx=0x7f8d0401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616ba3383da4825c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d0a000640 1 -- 172.21.15.143:0/1739231256 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040a5820 con 0x7f8d0c0c6230 2024-01-15T18:09:15.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d0a000640 1 -- 172.21.15.143:0/1739231256 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d040a5980 con 0x7f8d0c0c6230 2024-01-15T18:09:15.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d0a000640 1 -- 172.21.15.143:0/1739231256 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d040a5d10 con 0x7f8d0c0c6230 2024-01-15T18:09:15.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 -- 172.21.15.143:0/1739231256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 msgr2=0x7f8d0c0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 --2- 172.21.15.143:0/1739231256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0cc020 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c09ee40 tx=0x7f8d0401c960 comp rx=0 tx=0).stop 2024-01-15T18:09:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 -- 172.21.15.143:0/1739231256 shutdown_connections 2024-01-15T18:09:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 --2- 172.21.15.143:0/1739231256 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0cc020 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 --2- 172.21.15.143:0/1739231256 >> v2:172.21.15.204:3300/0 conn(0x7f8d0c0cdf20 0x7f8d0c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 -- 172.21.15.143:0/1739231256 >> 172.21.15.143:0/1739231256 conn(0x7f8d0c0d0ed0 msgr2=0x7f8d0c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 -- 172.21.15.143:0/1739231256 shutdown_connections 2024-01-15T18:09:15.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.174+0000 7f8d12800640 1 -- 172.21.15.143:0/1739231256 wait complete. 2024-01-15T18:09:15.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 Processor -- start 2024-01-15T18:09:15.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 -- start start 2024-01-15T18:09:15.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d0c0cdf20 0x7f8d0c147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c15b260 0x7f8d0c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0cb400 con 0x7f8d0c0c6230 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0b3e00 con 0x7f8d0c15b260 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0b40e0 con 0x7f8d0c0cdf20 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44596/0 (socket says 172.21.15.143:44596) 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 -- 172.21.15.143:0/2635977102 learned_addr learned my addr 172.21.15.143:0/2635977102 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0be00640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c15b260 0x7f8d0c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0aa00640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d0c0cdf20 0x7f8d0c147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 -- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d0c0cdf20 msgr2=0x7f8d0c147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d0c0cdf20 0x7f8d0c147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 -- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c15b260 msgr2=0x7f8d0c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c15b260 0x7f8d0c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 -- 172.21.15.143:0/2635977102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04009b30 con 0x7f8d0c0c6230 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0aa00640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d0c0cdf20 0x7f8d0c147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d0b400640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0caec0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00b3e0 tx=0x7f8cfc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc011830 con 0x7f8d0c0c6230 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc0119b0 con 0x7f8d0c0c6230 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c0b43c0 con 0x7f8d0c0c6230 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc011d60 con 0x7f8d0c0c6230 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.178+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c0b4980 con 0x7f8d0c0c6230 2024-01-15T18:09:15.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.182+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c0b4ba0 con 0x7f8d0c0c6230 2024-01-15T18:09:15.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.182+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f8cfc01a400 con 0x7f8d0c0c6230 2024-01-15T18:09:15.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.182+0000 7f8d03400640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8ce4066e90 0x7f8ce4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.182+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f8cfc084cb0 con 0x7f8d0c0c6230 2024-01-15T18:09:15.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.182+0000 7f8d0aa00640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8ce4066e90 0x7f8ce4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.182+0000 7f8d0aa00640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8ce4066e90 0x7f8ce4069350 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c06e6e0 tx=0x7f8cf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.186+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8cfc051970 con 0x7f8d0c0c6230 2024-01-15T18:09:15.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.338+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d0c067500 con 0x7f8d0c0c6230 2024-01-15T18:09:15.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d03400640 1 -- 172.21.15.143:0/2635977102 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12936 (secure 0 0 0) 0x7f8cfc015020 con 0x7f8d0c0c6230 2024-01-15T18:09:15.345 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:15.345 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":56,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:14.622395+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8ce4066e90 msgr2=0x7f8ce4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8ce4066e90 0x7f8ce4069350 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c06e6e0 tx=0x7f8cf4008040 comp rx=0 tx=0).stop 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 msgr2=0x7f8d0c0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0caec0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00b3e0 tx=0x7f8cfc00b8a0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 shutdown_connections 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8ce4066e90 0x7f8ce4069350 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d0c15b260 0x7f8d0c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d0c0cdf20 0x7f8d0c147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 --2- 172.21.15.143:0/2635977102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d0c0c6230 0x7f8d0c0caec0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 >> 172.21.15.143:0/2635977102 conn(0x7f8d0c0d0ed0 msgr2=0x7f8d0c06f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:15.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 shutdown_connections 2024-01-15T18:09:15.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.342+0000 7f8d12800640 1 -- 172.21.15.143:0/2635977102 wait complete. 2024-01-15T18:09:15.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:09:15.368 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:09:15.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3909600640 1 Processor -- start 2024-01-15T18:09:15.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3909600640 1 -- start start 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3909600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f39040cda70 0x7f39040cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3909600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390415b5b0 con 0x7f39040cf800 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3909600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39040ccd30 con 0x7f39040cf400 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3909600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39040cba70 con 0x7f39040cda70 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3903400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f39040cf800 0x7f39040b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57662/0 (socket says 172.21.15.143:57662) 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3903400640 1 -- 172.21.15.143:0/15106920 learned_addr learned my addr 172.21.15.143:0/15106920 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3902a00640 1 --2- 172.21.15.143:0/15106920 >> v2:172.21.15.204:3300/0 conn(0x7f39040cda70 0x7f39040cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085992911 0 0) 0x7f390415b5b0 con 0x7f39040cf800 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38e4003530 con 0x7f39040cf800 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1375304478 0 0) 0x7f38e4003530 con 0x7f39040cf800 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 >> v2:172.21.15.204:3300/0 conn(0x7f39040cda70 msgr2=0x7f39040cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 --2- 172.21.15.143:0/15106920 >> v2:172.21.15.204:3300/0 conn(0x7f39040cda70 0x7f39040cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 >> v1:172.21.15.204:6789/0 conn(0x7f39040cf400 legacy=0x7f39040cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39040cbd10 con 0x7f39040cf800 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f38fc001a00 con 0x7f39040cf800 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.502+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 >> v1:172.21.15.143:6789/0 conn(0x7f39040cf800 legacy=0x7f39040b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e40047d0 0x7f38e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38e4005190 0x7f38e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39040cba70 con 0x7f38e4003980 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400c030 con 0x7f38e4005190 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e400c1f0 con 0x7f38e40047d0 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902000640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e40047d0 0x7f38e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3903400640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38e4005190 0x7f38e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 -- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e40047d0 msgr2=0x7f38e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e40047d0 0x7f38e4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 -- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38e4005190 msgr2=0x7f38e400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38e4005190 0x7f38e400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3903400640 1 -- 172.21.15.143:0/15106920 reap_dead start 2024-01-15T18:09:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 -- 172.21.15.143:0/15106920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4003530 con 0x7f38e4003980 2024-01-15T18:09:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3902a00640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009ae0 tx=0x7f38f401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acedd70431c9f1ba server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40a5590 con 0x7f38e4003980 2024-01-15T18:09:15.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f40a56f0 con 0x7f38e4003980 2024-01-15T18:09:15.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.506+0000 7f3901600640 1 -- 172.21.15.143:0/15106920 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f40a5aa0 con 0x7f38e4003980 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- 172.21.15.143:0/15106920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4003980 msgr2=0x7f38e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f38f4009ae0 tx=0x7f38f401ca20 comp rx=0 tx=0).stop 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- 172.21.15.143:0/15106920 shutdown_connections 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 --2- 172.21.15.143:0/15106920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38e4003980 0x7f38e4003da0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- 172.21.15.143:0/15106920 >> 172.21.15.143:0/15106920 conn(0x7f39040d0e20 msgr2=0x7f39040d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- 172.21.15.143:0/15106920 shutdown_connections 2024-01-15T18:09:15.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- 172.21.15.143:0/15106920 wait complete. 2024-01-15T18:09:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 Processor -- start 2024-01-15T18:09:15.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- start start 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e4003980 0x7f3904156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39040c7ef0 0x7f39040ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 0x7f39040ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39040c99e0 con 0x7f39040c8370 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39041572e0 con 0x7f39040c7ef0 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3909600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39040c8940 con 0x7f38e4003980 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3903400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 0x7f39040ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3903400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 0x7f39040ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44606/0 (socket says 172.21.15.143:44606) 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.510+0000 7f3903400640 1 -- 172.21.15.143:0/1430862289 learned_addr learned my addr 172.21.15.143:0/1430862289 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3902a00640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e4003980 0x7f3904156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3902000640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39040c7ef0 0x7f39040ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3903400640 1 -- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e4003980 msgr2=0x7f3904156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3903400640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e4003980 0x7f3904156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3903400640 1 -- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39040c7ef0 msgr2=0x7f39040ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3903400640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39040c7ef0 0x7f39040ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3903400640 1 -- 172.21.15.143:0/1430862289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39040c8c20 con 0x7f39040c8370 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3902000640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39040c7ef0 0x7f39040ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3903400640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 0x7f39040ca7d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3904072310 tx=0x7f38fc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00aa90 con 0x7f39040c8370 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc00abf0 con 0x7f39040c8370 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39040c8f00 con 0x7f39040c8370 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0133f0 con 0x7f39040c8370 2024-01-15T18:09:15.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39040c94c0 con 0x7f39040c8370 2024-01-15T18:09:15.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39040c44d0 con 0x7f39040c8370 2024-01-15T18:09:15.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.514+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f38fc013550 con 0x7f39040c8370 2024-01-15T18:09:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.518+0000 7f38faa00640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38dc066e90 0x7f38dc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.518+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f38fc085320 con 0x7f39040c8370 2024-01-15T18:09:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.518+0000 7f3902a00640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38dc066e90 0x7f38dc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.518+0000 7f3902a00640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38dc066e90 0x7f38dc069350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f38f40af000 tx=0x7f38f40046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.518+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38fc052070 con 0x7f39040c8370 2024-01-15T18:09:15.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.670+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3904067500 con 0x7f39040c8370 2024-01-15T18:09:15.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.670+0000 7f38faa00640 1 -- 172.21.15.143:0/1430862289 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38fc055690 con 0x7f39040c8370 2024-01-15T18:09:15.675 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:09:15.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38dc066e90 msgr2=0x7f38dc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38dc066e90 0x7f38dc069350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f38f40af000 tx=0x7f38f40046d0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 msgr2=0x7f39040ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 0x7f39040ca7d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3904072310 tx=0x7f38fc007a90 comp rx=0 tx=0).stop 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 shutdown_connections 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f38dc066e90 0x7f38dc069350 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39040c8370 0x7f39040ca7d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39040c7ef0 0x7f39040ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 --2- 172.21.15.143:0/1430862289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38e4003980 0x7f3904156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 >> 172.21.15.143:0/1430862289 conn(0x7f39040d0e20 msgr2=0x7f390406ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 shutdown_connections 2024-01-15T18:09:15.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.674+0000 7f3909600640 1 -- 172.21.15.143:0/1430862289 wait complete. 2024-01-15T18:09:15.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:09:15.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.830+0000 7f2d90e00640 1 Processor -- start 2024-01-15T18:09:15.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.830+0000 7f2d90e00640 1 -- start start 2024-01-15T18:09:15.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.830+0000 7f2d90e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d8c0ccf60 0x7f2d8c167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.830+0000 7f2d90e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c06c5d0 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.830+0000 7f2d90e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8c0cffd0 con 0x7f2d8c1607c0 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.830+0000 7f2d90e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8c0d1ce0 con 0x7f2d8c0ccf60 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d8c0ccf60 0x7f2d8c167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d8c0ccf60 0x7f2d8c167fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44612/0 (socket says 172.21.15.143:44612) 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 -- 172.21.15.143:0/3334084016 learned_addr learned my addr 172.21.15.143:0/3334084016 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d89400640 1 --1- 172.21.15.143:0/3334084016 >> v1:172.21.15.143:6789/0 conn(0x7f2d8c0cfb30 0x7f2d8c06bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57668/0 (socket says 172.21.15.143:57668) 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 -- 172.21.15.143:0/3334084016 >> v1:172.21.15.204:6789/0 conn(0x7f2d8c1607c0 legacy=0x7f2d8c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 -- 172.21.15.143:0/3334084016 >> v1:172.21.15.143:6789/0 conn(0x7f2d8c0cfb30 legacy=0x7f2d8c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 -- 172.21.15.143:0/3334084016 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c0d02b0 con 0x7f2d8c0ccf60 2024-01-15T18:09:15.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d88a00640 1 -- 172.21.15.143:0/3334084016 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197318481 0 0) 0x7f2d8c06c5d0 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d8a800640 1 --2- 172.21.15.143:0/3334084016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c167fb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009e50 tx=0x7f2d8401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551473308edc7b63 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d88a00640 1 -- 172.21.15.143:0/3334084016 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840af400 con 0x7f2d8c0ccf60 2024-01-15T18:09:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d88a00640 1 -- 172.21.15.143:0/3334084016 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d840ab050 con 0x7f2d8c0ccf60 2024-01-15T18:09:15.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.834+0000 7f2d88a00640 1 -- 172.21.15.143:0/3334084016 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840af780 con 0x7f2d8c0ccf60 2024-01-15T18:09:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- 172.21.15.143:0/3334084016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0ccf60 msgr2=0x7f2d8c167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 --2- 172.21.15.143:0/3334084016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c167fb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009e50 tx=0x7f2d8401f1f0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- 172.21.15.143:0/3334084016 shutdown_connections 2024-01-15T18:09:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 --2- 172.21.15.143:0/3334084016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c167fb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- 172.21.15.143:0/3334084016 >> 172.21.15.143:0/3334084016 conn(0x7f2d8c0d0e20 msgr2=0x7f2d8c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- 172.21.15.143:0/3334084016 shutdown_connections 2024-01-15T18:09:15.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- 172.21.15.143:0/3334084016 wait complete. 2024-01-15T18:09:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 Processor -- start 2024-01-15T18:09:15.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- start start 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 0x7f2d8c0b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d8c1607c0 0x7f2d8c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b5d30 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b5120 con 0x7f2d8c0ccf60 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.838+0000 7f2d90e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c0b5400 con 0x7f2d8c1607c0 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 0x7f2d8c0b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 0x7f2d8c0b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44614/0 (socket says 172.21.15.143:44614) 2024-01-15T18:09:15.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 -- 172.21.15.143:0/1195108601 learned_addr learned my addr 172.21.15.143:0/1195108601 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d8a800640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d8c1607c0 0x7f2d8c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 -- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d8c1607c0 msgr2=0x7f2d8c06e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d8c1607c0 0x7f2d8c06e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 -- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d8c0ccf60 msgr2=0x7f2d8c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 -- 172.21.15.143:0/1195108601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009b30 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d89400640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 0x7f2d8c0b46f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c4a0 tx=0x7f2d7400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d740074e0 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74007640 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004a30 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c0b56e0 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c15ac80 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c06d4b0 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f2d74004b90 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.842+0000 7f2d82000640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d64066ec0 0x7f2d64069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.846+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f2d74083c80 con 0x7f2d8c0cfb30 2024-01-15T18:09:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.846+0000 7f2d89e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d64066ec0 0x7f2d64069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:15.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.846+0000 7f2d89e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d64066ec0 0x7f2d64069380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007fd0 tx=0x7f2d7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:15.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.850+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d74050a30 con 0x7f2d8c0cfb30 2024-01-15T18:09:16.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.998+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2d8c067500 con 0x7f2d8c0cfb30 2024-01-15T18:09:16.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:15.998+0000 7f2d82000640 1 -- 172.21.15.143:0/1195108601 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2d74018090 con 0x7f2d8c0cfb30 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:16.005 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d64066ec0 msgr2=0x7f2d64069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d64066ec0 0x7f2d64069380 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c007fd0 tx=0x7f2d7c009040 comp rx=0 tx=0).stop 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 msgr2=0x7f2d8c0b46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 0x7f2d8c0b46f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400c4a0 tx=0x7f2d7400c960 comp rx=0 tx=0).stop 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 shutdown_connections 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d64066ec0 0x7f2d64069380 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d8c1607c0 0x7f2d8c06e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d8c0cfb30 0x7f2d8c0b46f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 --2- 172.21.15.143:0/1195108601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d8c0ccf60 0x7f2d8c15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 >> 172.21.15.143:0/1195108601 conn(0x7f2d8c0d0e20 msgr2=0x7f2d8c06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:16.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 shutdown_connections 2024-01-15T18:09:16.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.002+0000 7f2d90e00640 1 -- 172.21.15.143:0/1195108601 wait complete. 2024-01-15T18:09:16.026 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:09:16.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:09:16.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.158+0000 7f8d58200640 1 Processor -- start 2024-01-15T18:09:16.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d58200640 1 -- start start 2024-01-15T18:09:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d58200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d50072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d58200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d500ceb50 con 0x7f8d5015c5a0 2024-01-15T18:09:16.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d58200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5015c110 con 0x7f8d50143780 2024-01-15T18:09:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d58200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50072570 con 0x7f8d50071bb0 2024-01-15T18:09:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d56200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8d5015c5a0 0x7f8d5015ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57684/0 (socket says 172.21.15.143:57684) 2024-01-15T18:09:16.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d56200640 1 -- 172.21.15.143:0/3496279444 learned_addr learned my addr 172.21.15.143:0/3496279444 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d50072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060213249 0 0) 0x7f8d480014a0 con 0x7f8d5015c5a0 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d30003530 con 0x7f8d5015c5a0 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805271076 0 0) 0x7f8d5015c110 con 0x7f8d50143780 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d480014a0 con 0x7f8d50143780 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633288829 0 0) 0x7f8d30003530 con 0x7f8d5015c5a0 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 msgr2=0x7f8d50072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d50072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 >> v1:172.21.15.204:6789/0 conn(0x7f8d50143780 legacy=0x7f8d500cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5015b7e0 con 0x7f8d5015c5a0 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8d48001b90 con 0x7f8d5015c5a0 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 >> v1:172.21.15.143:6789/0 conn(0x7f8d5015c5a0 legacy=0x7f8d5015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d30003b40 0x7f8d30003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d30005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50072570 con 0x7f8d300047d0 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3000ba40 con 0x7f8d30003b40 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3000bc00 con 0x7f8d50071bb0 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d55800640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d30003b40 0x7f8d30003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d56200640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d30005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 -- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 msgr2=0x7f8d30005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d30005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 -- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d30003b40 msgr2=0x7f8d30003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d30003b40 0x7f8d30003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d54e00640 1 -- 172.21.15.143:0/3496279444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30003530 con 0x7f8d300047d0 2024-01-15T18:09:16.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.162+0000 7f8d56200640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d50071bb0 0x7f8d30005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:16.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d54e00640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009ae0 tx=0x7f8d3801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacb3bc37e571dce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380a5640 con 0x7f8d300047d0 2024-01-15T18:09:16.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d380a57a0 con 0x7f8d300047d0 2024-01-15T18:09:16.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d4fe00640 1 -- 172.21.15.143:0/3496279444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d380a5b50 con 0x7f8d300047d0 2024-01-15T18:09:16.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d58200640 1 -- 172.21.15.143:0/3496279444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d300047d0 msgr2=0x7f8d30004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d58200640 1 --2- 172.21.15.143:0/3496279444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009ae0 tx=0x7f8d3801ca50 comp rx=0 tx=0).stop 2024-01-15T18:09:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d56200640 1 -- 172.21.15.143:0/3496279444 reap_dead start 2024-01-15T18:09:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d58200640 1 -- 172.21.15.143:0/3496279444 shutdown_connections 2024-01-15T18:09:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.166+0000 7f8d58200640 1 -- 172.21.15.143:0/3496279444 >> 172.21.15.143:0/3496279444 conn(0x7f8d5006c3f0 msgr2=0x7f8d5006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 -- 172.21.15.143:0/3496279444 shutdown_connections 2024-01-15T18:09:16.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 -- 172.21.15.143:0/3496279444 wait complete. 2024-01-15T18:09:16.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 Processor -- start 2024-01-15T18:09:16.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 -- start start 2024-01-15T18:09:16.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d300047d0 0x7f8d50142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5013d370 0x7f8d5013ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 0x7f8d50141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5013e210 con 0x7f8d50159860 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5013df30 con 0x7f8d5013d370 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d58200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500c3e40 con 0x7f8d300047d0 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 0x7f8d50141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 0x7f8d50141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44626/0 (socket says 172.21.15.143:44626) 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 -- 172.21.15.143:0/3078228402 learned_addr learned my addr 172.21.15.143:0/3078228402 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d54e00640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5013d370 0x7f8d5013ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d55800640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d300047d0 0x7f8d50142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 -- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d300047d0 msgr2=0x7f8d50142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d300047d0 0x7f8d50142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 -- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5013d370 msgr2=0x7f8d5013ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5013d370 0x7f8d5013ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 -- 172.21.15.143:0/3078228402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5013ce50 con 0x7f8d50159860 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d56200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 0x7f8d50141960 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8d50070a50 tx=0x7f8d4800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800eb90 con 0x7f8d50159860 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.170+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4800ecf0 con 0x7f8d50159860 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5013b0a0 con 0x7f8d50159860 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48013660 con 0x7f8d50159860 2024-01-15T18:09:16.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d50141ea0 con 0x7f8d50159860 2024-01-15T18:09:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d50067500 con 0x7f8d50159860 2024-01-15T18:09:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f8d480137c0 con 0x7f8d50159860 2024-01-15T18:09:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d4d600640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8d28066ec0 0x7f8d28069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f8d48084ce0 con 0x7f8d50159860 2024-01-15T18:09:16.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d55800640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8d28066ec0 0x7f8d28069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.174+0000 7f8d55800640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8d28066ec0 0x7f8d28069380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d40000c10 tx=0x7f8d40006ff0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.178+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8d480519a0 con 0x7f8d50159860 2024-01-15T18:09:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.330+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8d500ceb50 con 0x7f8d50159860 2024-01-15T18:09:16.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d4d600640 1 -- 172.21.15.143:0/3078228402 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8d500ceb50 con 0x7f8d50159860 2024-01-15T18:09:16.706 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:09:16.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8d28066ec0 msgr2=0x7f8d28069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8d28066ec0 0x7f8d28069380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8d40000c10 tx=0x7f8d40006ff0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 msgr2=0x7f8d50141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 0x7f8d50141960 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8d50070a50 tx=0x7f8d4800e530 comp rx=0 tx=0).stop 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 shutdown_connections 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8d28066ec0 0x7f8d28069380 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d50159860 0x7f8d50141960 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d5013d370 0x7f8d5013ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 --2- 172.21.15.143:0/3078228402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8d300047d0 0x7f8d50142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 >> 172.21.15.143:0/3078228402 conn(0x7f8d5006c3f0 msgr2=0x7f8d500cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 shutdown_connections 2024-01-15T18:09:16.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.702+0000 7f8d58200640 1 -- 172.21.15.143:0/3078228402 wait complete. 2024-01-15T18:09:16.734 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:09:16.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 Processor -- start 2024-01-15T18:09:16.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 -- start start 2024-01-15T18:09:16.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad40b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ad406d080 0x7f3ad406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad40cd5b0 con 0x7f3ad41594b0 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad406da30 con 0x7f3ad406d080 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3adc400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40b2ec0 con 0x7f3ad40b3d80 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ada400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ad406d080 0x7f3ad406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ad9a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3ad41594b0 0x7f3ad40cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57690/0 (socket says 172.21.15.143:57690) 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ada400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ad406d080 0x7f3ad406d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44640/0 (socket says 172.21.15.143:44640) 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ad9a00640 1 -- 172.21.15.143:0/3813941461 learned_addr learned my addr 172.21.15.143:0/3813941461 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ad9000640 1 --2- 172.21.15.143:0/3813941461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad40b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ada400640 1 -- 172.21.15.143:0/3813941461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 msgr2=0x7f3ad40b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ada400640 1 --2- 172.21.15.143:0/3813941461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad40b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ada400640 1 -- 172.21.15.143:0/3813941461 >> v1:172.21.15.143:6789/0 conn(0x7f3ad41594b0 legacy=0x7f3ad40cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3ada400640 1 -- 172.21.15.143:0/3813941461 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40ccf60 con 0x7f3ad406d080 2024-01-15T18:09:16.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.874+0000 7f3ad3e00640 1 -- 172.21.15.143:0/3813941461 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836177264 0 0) 0x7f3ad40cd5b0 con 0x7f3ad41594b0 2024-01-15T18:09:16.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3ada400640 1 --2- 172.21.15.143:0/3813941461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad406d080 0x7f3ad406d4f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00aea0 tx=0x7f3acc01da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43d36baccb6fda9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3ad3e00640 1 -- 172.21.15.143:0/3813941461 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00a2d0 con 0x7f3ad406d080 2024-01-15T18:09:16.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3ad3e00640 1 -- 172.21.15.143:0/3813941461 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc0a67c0 con 0x7f3ad406d080 2024-01-15T18:09:16.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3ad3e00640 1 -- 172.21.15.143:0/3813941461 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0a6b50 con 0x7f3ad406d080 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3adc400640 1 -- 172.21.15.143:0/3813941461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad406d080 msgr2=0x7f3ad406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.878+0000 7f3adc400640 1 --2- 172.21.15.143:0/3813941461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad406d080 0x7f3ad406d4f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00aea0 tx=0x7f3acc01da50 comp rx=0 tx=0).stop 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- 172.21.15.143:0/3813941461 shutdown_connections 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 --2- 172.21.15.143:0/3813941461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad406d080 0x7f3ad406d4f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 --2- 172.21.15.143:0/3813941461 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad40b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- 172.21.15.143:0/3813941461 >> 172.21.15.143:0/3813941461 conn(0x7f3ad406a890 msgr2=0x7f3ad406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- 172.21.15.143:0/3813941461 shutdown_connections 2024-01-15T18:09:16.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- 172.21.15.143:0/3813941461 wait complete. 2024-01-15T18:09:16.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 Processor -- start 2024-01-15T18:09:16.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- start start 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ad406d080 0x7f3ad415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad4147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 0x7f3ad4148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad4153e50 con 0x7f3ad41594b0 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad415e810 con 0x7f3ad406d080 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3adc400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad41540f0 con 0x7f3ad40b3d80 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 0x7f3ad4148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 0x7f3ad4148470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44644/0 (socket says 172.21.15.143:44644) 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 -- 172.21.15.143:0/2642265663 learned_addr learned my addr 172.21.15.143:0/2642265663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ad9a00640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ad406d080 0x7f3ad415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ad9000640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad4147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 -- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 msgr2=0x7f3ad4147f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad4147f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 -- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ad406d080 msgr2=0x7f3ad415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ad406d080 0x7f3ad415e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ada400640 1 -- 172.21.15.143:0/2642265663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc00ab80 con 0x7f3ad41594b0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.882+0000 7f3ad9a00640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ad406d080 0x7f3ad415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ada400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 0x7f3ad4148470 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3acc01b680 tx=0x7f3acc01a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0b7680 con 0x7f3ad41594b0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad41543d0 con 0x7f3ad41594b0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc0b77e0 con 0x7f3ad41594b0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0b7b90 con 0x7f3ad41594b0 2024-01-15T18:09:16.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad415eae0 con 0x7f3ad41594b0 2024-01-15T18:09:16.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad41548f0 con 0x7f3ad41594b0 2024-01-15T18:09:16.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f3acc0b7cf0 con 0x7f3ad41594b0 2024-01-15T18:09:16.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad1600640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3aac066e90 0x7f3aac069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f3acc1208e0 con 0x7f3ad41594b0 2024-01-15T18:09:16.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.886+0000 7f3ad9a00640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3aac066e90 0x7f3aac069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.890+0000 7f3ad9a00640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3aac066e90 0x7f3aac069350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000f70 tx=0x7f3ac4007cb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:16.890+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3acc0ed480 con 0x7f3ad41594b0 2024-01-15T18:09:17.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.042+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3ad41548f0 con 0x7f3ad41594b0 2024-01-15T18:09:17.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3ad1600640 1 -- 172.21.15.143:0/2642265663 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3ad41548f0 con 0x7f3ad41594b0 2024-01-15T18:09:17.710 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:09:17.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3aac066e90 msgr2=0x7f3aac069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3aac066e90 0x7f3aac069350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000f70 tx=0x7f3ac4007cb0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 msgr2=0x7f3ad4148470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 0x7f3ad4148470 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3acc01b680 tx=0x7f3acc01a7f0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 shutdown_connections 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3aac066e90 0x7f3aac069350 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ad41594b0 0x7f3ad4148470 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ad40b3d80 0x7f3ad4147f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 --2- 172.21.15.143:0/2642265663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ad406d080 0x7f3ad415e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.706+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 >> 172.21.15.143:0/2642265663 conn(0x7f3ad406a890 msgr2=0x7f3ad41626e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:17.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.710+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 shutdown_connections 2024-01-15T18:09:17.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.710+0000 7f3adc400640 1 -- 172.21.15.143:0/2642265663 wait complete. 2024-01-15T18:09:17.761 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:09:17.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 Processor -- start 2024-01-15T18:09:17.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 -- start start 2024-01-15T18:09:17.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3e54159b70 0x7f3e540b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 0x7f3e54070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e540b2a20 con 0x7f3e54157300 2024-01-15T18:09:17.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540b35a0 con 0x7f3e54159b70 2024-01-15T18:09:17.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e5bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54159fd0 con 0x7f3e540b48f0 2024-01-15T18:09:17.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e59200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3e54157300 0x7f3e540b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44856/0 (socket says 172.21.15.143:44856) 2024-01-15T18:09:17.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.898+0000 7f3e59200640 1 -- 172.21.15.143:0/1933195724 learned_addr learned my addr 172.21.15.143:0/1933195724 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e59c00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 0x7f3e54070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 --2- 172.21.15.143:0/1933195724 >> v2:172.21.15.204:3300/0 conn(0x7f3e54159b70 0x7f3e540b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096315192 0 0) 0x7f3e540b2a20 con 0x7f3e54157300 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e34003530 con 0x7f3e54157300 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3560485428 0 0) 0x7f3e34003530 con 0x7f3e54157300 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 msgr2=0x7f3e54070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 0x7f3e54070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 >> v2:172.21.15.204:3300/0 conn(0x7f3e54159b70 msgr2=0x7f3e540b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 --2- 172.21.15.143:0/1933195724 >> v2:172.21.15.204:3300/0 conn(0x7f3e54159b70 0x7f3e540b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540b2d00 con 0x7f3e54157300 2024-01-15T18:09:17.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3e44001680 con 0x7f3e54157300 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 >> v1:172.21.15.143:6789/0 conn(0x7f3e54157300 legacy=0x7f3e540b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 0x7f3e34003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e34003f00 0x7f3e340047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34004d10 0x7f3e34005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540b35a0 con 0x7f3e34003f00 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e54159fd0 con 0x7f3e34004d10 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3400b990 con 0x7f3e540b48f0 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e34003f00 0x7f3e340047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e59200640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 0x7f3e34003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e59c00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34004d10 0x7f3e34005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 -- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 msgr2=0x7f3e34003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540b48f0 0x7f3e34003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 -- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34004d10 msgr2=0x7f3e34005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34004d10 0x7f3e34005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.902+0000 7f3e53e00640 1 -- 172.21.15.143:0/1933195724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e34003530 con 0x7f3e34003f00 2024-01-15T18:09:17.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e53e00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e34003f00 0x7f3e340047d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ae0 tx=0x7f3e3c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a70a630899b9ac3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0a5680 con 0x7f3e34003f00 2024-01-15T18:09:17.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c0a57e0 con 0x7f3e34003f00 2024-01-15T18:09:17.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e53400640 1 -- 172.21.15.143:0/1933195724 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0a5b90 con 0x7f3e34003f00 2024-01-15T18:09:17.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 -- 172.21.15.143:0/1933195724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e34003f00 msgr2=0x7f3e340047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/1933195724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e34003f00 0x7f3e340047d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ae0 tx=0x7f3e3c01ca30 comp rx=0 tx=0).stop 2024-01-15T18:09:17.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e59c00640 1 -- 172.21.15.143:0/1933195724 reap_dead start 2024-01-15T18:09:17.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 -- 172.21.15.143:0/1933195724 shutdown_connections 2024-01-15T18:09:17.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 -- 172.21.15.143:0/1933195724 >> 172.21.15.143:0/1933195724 conn(0x7f3e5406c3f0 msgr2=0x7f3e5406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:17.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 -- 172.21.15.143:0/1933195724 shutdown_connections 2024-01-15T18:09:17.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 -- 172.21.15.143:0/1933195724 wait complete. 2024-01-15T18:09:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 Processor -- start 2024-01-15T18:09:17.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.906+0000 7f3e5bc00640 1 -- start start 2024-01-15T18:09:17.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34003f00 0x7f3e5415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 0x7f3e5414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540af280 0x7f3e540af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5414ff10 con 0x7f3e5406db00 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540afc40 con 0x7f3e34003f00 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5415c360 con 0x7f3e540af280 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 0x7f3e5414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 0x7f3e5414dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42230/0 (socket says 172.21.15.143:42230) 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 -- 172.21.15.143:0/259961322 learned_addr learned my addr 172.21.15.143:0/259961322 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e59200640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34003f00 0x7f3e5415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e59c00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540af280 0x7f3e540af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 -- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540af280 msgr2=0x7f3e540af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540af280 0x7f3e540af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 -- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34003f00 msgr2=0x7f3e5415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34003f00 0x7f3e5415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 -- 172.21.15.143:0/259961322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54150120 con 0x7f3e5406db00 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e53e00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 0x7f3e5414dad0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ae0 tx=0x7f3e34005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0be070 con 0x7f3e5406db00 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3c004020 con 0x7f3e5406db00 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e54150400 con 0x7f3e5406db00 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0a5680 con 0x7f3e5406db00 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e54151840 con 0x7f3e5406db00 2024-01-15T18:09:17.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.910+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e54067500 con 0x7f3e5406db00 2024-01-15T18:09:17.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.914+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f3e3c0b4040 con 0x7f3e5406db00 2024-01-15T18:09:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.914+0000 7f3e50c00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e2c066ec0 0x7f3e2c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.914+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f3e3c11e650 con 0x7f3e5406db00 2024-01-15T18:09:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.914+0000 7f3e59200640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e2c066ec0 0x7f3e2c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.914+0000 7f3e59200640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e2c066ec0 0x7f3e2c069380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005180 tx=0x7f3e44007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:17.918+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3e3c0eb220 con 0x7f3e5406db00 2024-01-15T18:09:18.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.098+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3e5406e4e0 con 0x7f3e5406db00 2024-01-15T18:09:18.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.734+0000 7f3e50c00640 1 -- 172.21.15.143:0/259961322 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3e3c0ee840 con 0x7f3e5406db00 2024-01-15T18:09:18.740 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 8 and data pool 9 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e2c066ec0 msgr2=0x7f3e2c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e2c066ec0 0x7f3e2c069380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005180 tx=0x7f3e44007900 comp rx=0 tx=0).stop 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 msgr2=0x7f3e5414dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 0x7f3e5414dad0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009ae0 tx=0x7f3e34005800 comp rx=0 tx=0).stop 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 shutdown_connections 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3e2c066ec0 0x7f3e2c069380 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3e540af280 0x7f3e540af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3e5406db00 0x7f3e5414dad0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 --2- 172.21.15.143:0/259961322 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3e34003f00 0x7f3e5415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 >> 172.21.15.143:0/259961322 conn(0x7f3e5406c3f0 msgr2=0x7f3e5406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:18.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 shutdown_connections 2024-01-15T18:09:18.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.738+0000 7f3e5bc00640 1 -- 172.21.15.143:0/259961322 wait complete. 2024-01-15T18:09:18.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:18.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.902+0000 7f8c8fc00640 1 Processor -- start 2024-01-15T18:09:18.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.902+0000 7f8c8fc00640 1 -- start start 2024-01-15T18:09:18.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.902+0000 7f8c8fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 0x7f8c880aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.902+0000 7f8c8fc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c88160eb0 con 0x7f8c8806aba0 2024-01-15T18:09:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.902+0000 7f8c8fc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c88161190 con 0x7f8c8815d510 2024-01-15T18:09:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.902+0000 7f8c8fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88161470 con 0x7f8c8815d130 2024-01-15T18:09:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8c8806aba0 0x7f8c880b3440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44866/0 (socket says 172.21.15.143:44866) 2024-01-15T18:09:18.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8dc00640 1 -- 172.21.15.143:0/1755851638 learned_addr learned my addr 172.21.15.143:0/1755851638 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 0x7f8c880aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901918562 0 0) 0x7f8c88160eb0 con 0x7f8c8806aba0 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c68003530 con 0x7f8c8806aba0 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904545313 0 0) 0x7f8c88161190 con 0x7f8c8815d510 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c88160eb0 con 0x7f8c8815d510 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2313192797 0 0) 0x7f8c68003530 con 0x7f8c8806aba0 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 msgr2=0x7f8c880aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 0x7f8c880aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 >> v1:172.21.15.204:6789/0 conn(0x7f8c8815d510 legacy=0x7f8c8806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88161750 con 0x7f8c8806aba0 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8c80001a00 con 0x7f8c8806aba0 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 >> v1:172.21.15.143:6789/0 conn(0x7f8c8806aba0 legacy=0x7f8c880b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c68003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 0x7f8c680047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c68004d10 0x7f8c68005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88161470 con 0x7f8c68003b40 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6800ba40 con 0x7f8c68004d10 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6800bbb0 con 0x7f8c8815d130 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c68003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8dc00640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c68004d10 0x7f8c68005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c87e00640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 0x7f8c680047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 -- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 msgr2=0x7f8c680047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c8815d130 0x7f8c680047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 -- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c68004d10 msgr2=0x7f8c68005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c68004d10 0x7f8c68005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8d200640 1 -- 172.21.15.143:0/1755851638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68003530 con 0x7f8c68003b40 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.906+0000 7f8c8dc00640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c68004d10 0x7f8c68005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:18.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8d200640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c68003f40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009ae0 tx=0x7f8c7801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daa6a8467c549f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780a5530 con 0x7f8c68003b40 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c780a5690 con 0x7f8c68003b40 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c87400640 1 -- 172.21.15.143:0/1755851638 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780a5a40 con 0x7f8c68003b40 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 -- 172.21.15.143:0/1755851638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 msgr2=0x7f8c68003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/1755851638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c68003f40 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009ae0 tx=0x7f8c7801ca50 comp rx=0 tx=0).stop 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8dc00640 1 -- 172.21.15.143:0/1755851638 reap_dead start 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 -- 172.21.15.143:0/1755851638 shutdown_connections 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 -- 172.21.15.143:0/1755851638 >> 172.21.15.143:0/1755851638 conn(0x7f8c880b1b90 msgr2=0x7f8c880b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 -- 172.21.15.143:0/1755851638 shutdown_connections 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 -- 172.21.15.143:0/1755851638 wait complete. 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 Processor -- start 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.910+0000 7f8c8fc00640 1 -- start start 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c88153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c881583c0 0x7f8c88158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c880b0580 0x7f8c880b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c88158d60 con 0x7f8c68003b40 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8813e120 con 0x7f8c881583c0 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b12d0 con 0x7f8c880b0580 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c88153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c88153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42246/0 (socket says 172.21.15.143:42246) 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 -- 172.21.15.143:0/2199347281 learned_addr learned my addr 172.21.15.143:0/2199347281 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8dc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c880b0580 0x7f8c880b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c87e00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c881583c0 0x7f8c88158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 -- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c880b0580 msgr2=0x7f8c880b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c880b0580 0x7f8c880b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 -- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c881583c0 msgr2=0x7f8c88158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c881583c0 0x7f8c88158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 -- 172.21.15.143:0/2199347281 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8815a7b0 con 0x7f8c68003b40 2024-01-15T18:09:18.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8d200640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c88153200 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009ae0 tx=0x7f8c68005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780be070 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c780a5e80 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8815aa90 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780a5530 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8815a460 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c88067500 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.914+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f8c780b4040 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.918+0000 7f8c84c00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c60066e90 0x7f8c60069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.918+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8c7811e6c0 con 0x7f8c68003b40 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.918+0000 7f8c87e00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c60066e90 0x7f8c60069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.918+0000 7f8c87e00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c60066e90 0x7f8c60069350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c70007fd0 tx=0x7f8c70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:18.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:18.922+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8c780eb170 con 0x7f8c68003b40 2024-01-15T18:09:19.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.070+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c880b1970 con 0x7f8c68003b40 2024-01-15T18:09:19.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c84c00640 1 -- 172.21.15.143:0/2199347281 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f8c780ee790 con 0x7f8c68003b40 2024-01-15T18:09:19.078 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:19.078 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:18.102657+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:09:16.375775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-15T18:09:17.090306+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c60066e90 msgr2=0x7f8c60069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c60066e90 0x7f8c60069350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c70007fd0 tx=0x7f8c70009040 comp rx=0 tx=0).stop 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 msgr2=0x7f8c88153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c88153200 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009ae0 tx=0x7f8c68005800 comp rx=0 tx=0).stop 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 shutdown_connections 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c60066e90 0x7f8c60069350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c880b0580 0x7f8c880b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c881583c0 0x7f8c88158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 --2- 172.21.15.143:0/2199347281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c68003b40 0x7f8c88153200 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 >> 172.21.15.143:0/2199347281 conn(0x7f8c880b1b90 msgr2=0x7f8c880ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:19.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 shutdown_connections 2024-01-15T18:09:19.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.074+0000 7f8c8fc00640 1 -- 172.21.15.143:0/2199347281 wait complete. 2024-01-15T18:09:19.099 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:19.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.230+0000 7fcfc3a00640 1 Processor -- start 2024-01-15T18:09:19.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.230+0000 7fcfc3a00640 1 -- start start 2024-01-15T18:09:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc3a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcfbc15d510 0x7fcfbc06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc160eb0 con 0x7fcfbc15d130 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc3a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc161190 con 0x7fcfbc15d510 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc161470 con 0x7fcfbc06aba0 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc1a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcfbc15d510 0x7fcfbc06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:47756/0 (socket says 172.21.15.143:47756) 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfbbe00640 1 -- 172.21.15.143:0/1975545529 learned_addr learned my addr 172.21.15.143:0/1975545529 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc1a00640 1 -- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 msgr2=0x7fcfbc0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:19.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc1a00640 1 --2- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc1a00640 1 -- 172.21.15.143:0/1975545529 >> v1:172.21.15.204:6789/0 conn(0x7fcfbc15d130 legacy=0x7fcfbc06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.234+0000 7fcfc1a00640 1 -- 172.21.15.143:0/1975545529 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc161750 con 0x7fcfbc15d510 2024-01-15T18:09:19.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc1a00640 1 --2- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc06e060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009d90 tx=0x7fcfb401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aaa0d1a99df77f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:19.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfbb400640 1 -- 172.21.15.143:0/1975545529 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40a5a30 con 0x7fcfbc15d510 2024-01-15T18:09:19.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfbb400640 1 -- 172.21.15.143:0/1975545529 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb40a5b90 con 0x7fcfbc15d510 2024-01-15T18:09:19.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfbb400640 1 -- 172.21.15.143:0/1975545529 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40ad440 con 0x7fcfbc15d510 2024-01-15T18:09:19.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 -- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 msgr2=0x7fcfbc06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc06e060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009d90 tx=0x7fcfb401c950 comp rx=0 tx=0).stop 2024-01-15T18:09:19.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 -- 172.21.15.143:0/1975545529 shutdown_connections 2024-01-15T18:09:19.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc06e060 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/1975545529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 -- 172.21.15.143:0/1975545529 >> 172.21.15.143:0/1975545529 conn(0x7fcfbc0b1b90 msgr2=0x7fcfbc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:19.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 -- 172.21.15.143:0/1975545529 shutdown_connections 2024-01-15T18:09:19.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 -- 172.21.15.143:0/1975545529 wait complete. 2024-01-15T18:09:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 Processor -- start 2024-01-15T18:09:19.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.238+0000 7fcfc3a00640 1 -- start start 2024-01-15T18:09:19.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 0x7fcfbc14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc14f620 con 0x7fcfbc15d130 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc15e620 con 0x7fcfbc15d510 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc14e9a0 con 0x7fcfbc06aba0 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 0x7fcfbc14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 0x7fcfbc14dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42260/0 (socket says 172.21.15.143:42260) 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 -- 172.21.15.143:0/3568613394 learned_addr learned my addr 172.21.15.143:0/3568613394 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc1a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc1000640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 -- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 msgr2=0x7fcfbc15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc15d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 -- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 msgr2=0x7fcfbc15fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc15fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 -- 172.21.15.143:0/3568613394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4009a70 con 0x7fcfbc15d130 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc1a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc15fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc1000640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfbbe00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 0x7fcfbc14dc60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400e890 tx=0x7fcfa400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4009620 con 0x7fcfbc15d130 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa400c940 con 0x7fcfbc15d130 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc14ec80 con 0x7fcfbc15d130 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400ccf0 con 0x7fcfbc15d130 2024-01-15T18:09:19.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc14f240 con 0x7fcfbc15d130 2024-01-15T18:09:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc067500 con 0x7fcfbc15d130 2024-01-15T18:09:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.242+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fcfa4013470 con 0x7fcfbc15d130 2024-01-15T18:09:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.246+0000 7fcfb8c00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcf94066e90 0x7fcf94069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.246+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcfa4085050 con 0x7fcfbc15d130 2024-01-15T18:09:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.246+0000 7fcfc1000640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcf94066e90 0x7fcf94069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.246+0000 7fcfc1000640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcf94066e90 0x7fcf94069350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcfac000f70 tx=0x7fcfac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.250+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcfa4051a50 con 0x7fcfbc15d130 2024-01-15T18:09:19.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.398+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfbc0b2320 con 0x7fcfbc15d130 2024-01-15T18:09:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.398+0000 7fcfb8c00640 1 -- 172.21.15.143:0/3568613394 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16657 (secure 0 0 0) 0x7fcfa4017090 con 0x7fcfbc15d130 2024-01-15T18:09:19.405 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:19.405 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":59,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:18.102657+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:09:16.375775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-15T18:09:17.090306+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcf94066e90 msgr2=0x7fcf94069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcf94066e90 0x7fcf94069350 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcfac000f70 tx=0x7fcfac009040 comp rx=0 tx=0).stop 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 msgr2=0x7fcfbc14dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 0x7fcfbc14dc60 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fcfa400e890 tx=0x7fcfa400ed50 comp rx=0 tx=0).stop 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 shutdown_connections 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcf94066e90 0x7fcf94069350 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcfbc15d510 0x7fcfbc15fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcfbc15d130 0x7fcfbc14dc60 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 --2- 172.21.15.143:0/3568613394 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcfbc06aba0 0x7fcfbc15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 >> 172.21.15.143:0/3568613394 conn(0x7fcfbc0b1b90 msgr2=0x7fcfbc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 shutdown_connections 2024-01-15T18:09:19.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.402+0000 7fcfc3a00640 1 -- 172.21.15.143:0/3568613394 wait complete. 2024-01-15T18:09:19.427 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:09:19.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.558+0000 7ff592000640 1 Processor -- start 2024-01-15T18:09:19.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.558+0000 7ff592000640 1 -- start start 2024-01-15T18:09:19.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff592000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff592000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c0ceb50 con 0x7ff58c0722c0 2024-01-15T18:09:19.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff592000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c15c110 con 0x7ff58c0ce520 2024-01-15T18:09:19.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff592000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c15b7e0 con 0x7ff58c143780 2024-01-15T18:09:19.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff58c0722c0 0x7ff58c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44870/0 (socket says 172.21.15.143:44870) 2024-01-15T18:09:19.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58be00640 1 -- 172.21.15.143:0/1046109663 learned_addr learned my addr 172.21.15.143:0/1046109663 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:19.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58b400640 1 --2- 172.21.15.143:0/1046109663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411716109 0 0) 0x7ff5840014a0 con 0x7ff58c0722c0 2024-01-15T18:09:19.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff56c003530 con 0x7ff58c0722c0 2024-01-15T18:09:19.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604102368 0 0) 0x7ff58c15c110 con 0x7ff58c0ce520 2024-01-15T18:09:19.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5840014a0 con 0x7ff58c0ce520 2024-01-15T18:09:19.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58b400640 1 -- 172.21.15.143:0/1046109663 >> v1:172.21.15.204:6789/0 conn(0x7ff58c0ce520 legacy=0x7ff58c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58b400640 1 -- 172.21.15.143:0/1046109663 >> v1:172.21.15.143:6789/0 conn(0x7ff58c0722c0 legacy=0x7ff58c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.562+0000 7ff58b400640 1 -- 172.21.15.143:0/1046109663 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c15bac0 con 0x7ff58c143780 2024-01-15T18:09:19.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff58b400640 1 --2- 172.21.15.143:0/1046109663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0cdfe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c683f30ee78a58b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:19.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0ac630 con 0x7ff58c143780 2024-01-15T18:09:19.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c0ac790 con 0x7ff58c143780 2024-01-15T18:09:19.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff58a000640 1 -- 172.21.15.143:0/1046109663 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0acb20 con 0x7ff58c143780 2024-01-15T18:09:19.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 -- 172.21.15.143:0/1046109663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 msgr2=0x7ff58c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 --2- 172.21.15.143:0/1046109663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0cdfe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c01c4c0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 -- 172.21.15.143:0/1046109663 shutdown_connections 2024-01-15T18:09:19.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 --2- 172.21.15.143:0/1046109663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0cdfe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 -- 172.21.15.143:0/1046109663 >> 172.21.15.143:0/1046109663 conn(0x7ff58c06c3f0 msgr2=0x7ff58c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:19.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 -- 172.21.15.143:0/1046109663 shutdown_connections 2024-01-15T18:09:19.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.566+0000 7ff592000640 1 -- 172.21.15.143:0/1046109663 wait complete. 2024-01-15T18:09:19.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 Processor -- start 2024-01-15T18:09:19.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- start start 2024-01-15T18:09:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 0x7ff58c140460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c0ce520 0x7ff58c1f9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1409a0 con 0x7ff58c0722c0 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1f9e70 con 0x7ff58c0ce520 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c142220 con 0x7ff58c143780 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 0x7ff58c140460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 0x7ff58c140460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42268/0 (socket says 172.21.15.143:42268) 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 -- 172.21.15.143:0/3116518446 learned_addr learned my addr 172.21.15.143:0/3116518446 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58be00640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58aa00640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c0ce520 0x7ff58c1f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 -- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 msgr2=0x7ff58c0afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 -- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c0ce520 msgr2=0x7ff58c1f9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c0ce520 0x7ff58c1f9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 -- 172.21.15.143:0/3116518446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c009420 con 0x7ff58c0722c0 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58be00640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:19.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff58b400640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 0x7ff58c140460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b6670 con 0x7ff58c0722c0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c0b67f0 con 0x7ff58c0722c0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c0b6ba0 con 0x7ff58c0722c0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c142500 con 0x7ff58c0722c0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c142ac0 con 0x7ff58c0722c0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.570+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c1f59a0 con 0x7ff58c0722c0 2024-01-15T18:09:19.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.574+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7ff57c0b3040 con 0x7ff58c0722c0 2024-01-15T18:09:19.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.574+0000 7ff583400640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff564066e90 0x7ff564069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:19.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.574+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff57c11f2a0 con 0x7ff58c0722c0 2024-01-15T18:09:19.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.574+0000 7ff58aa00640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff564066e90 0x7ff564069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:19.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.574+0000 7ff58aa00640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff564066e90 0x7ff564069350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0b3530 tx=0x7ff574009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:19.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.578+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff57c0ebb00 con 0x7ff58c0722c0 2024-01-15T18:09:19.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:19.766+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff58c067500 con 0x7ff58c0722c0 2024-01-15T18:09:20.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.746+0000 7ff583400640 1 -- 172.21.15.143:0/3116518446 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff57c0b10b0 con 0x7ff58c0722c0 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.750+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff564066e90 msgr2=0x7ff564069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.750+0000 7ff592000640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff564066e90 0x7ff564069350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0b3530 tx=0x7ff574009040 comp rx=0 tx=0).stop 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.750+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 msgr2=0x7ff58c140460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.750+0000 7ff592000640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 0x7ff58c140460 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0097f0 tx=0x7ff57c019260 comp rx=0 tx=0).stop 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 shutdown_connections 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff564066e90 0x7ff564069350 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff58c143780 0x7ff58c0afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff58c0ce520 0x7ff58c1f9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 --2- 172.21.15.143:0/3116518446 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff58c0722c0 0x7ff58c140460 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 >> 172.21.15.143:0/3116518446 conn(0x7ff58c06c3f0 msgr2=0x7ff58c0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:20.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 shutdown_connections 2024-01-15T18:09:20.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.754+0000 7ff592000640 1 -- 172.21.15.143:0/3116518446 wait complete. 2024-01-15T18:09:20.780 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:09:20.781 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:09:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 Processor -- start 2024-01-15T18:09:20.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 -- start start 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1ab4159b70 0x7f1ab40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 0x7f1ab4070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40b2a20 con 0x7f1ab4157300 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab40b35a0 con 0x7f1ab4159b70 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1abb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4159fd0 con 0x7f1ab40b48f0 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 0x7f1ab4070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1ab4159b70 0x7f1ab40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 0x7f1ab4070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:47798/0 (socket says 172.21.15.143:47798) 2024-01-15T18:09:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 -- 172.21.15.143:0/1537672970 learned_addr learned my addr 172.21.15.143:0/1537672970 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 -- 172.21.15.143:0/1537672970 >> v2:172.21.15.204:3300/0 conn(0x7f1ab4159b70 msgr2=0x7f1ab40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 --2- 172.21.15.143:0/1537672970 >> v2:172.21.15.204:3300/0 conn(0x7f1ab4159b70 0x7f1ab40b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 -- 172.21.15.143:0/1537672970 >> v1:172.21.15.204:6789/0 conn(0x7f1ab4157300 legacy=0x7f1ab40b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab9200640 1 -- 172.21.15.143:0/1537672970 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40b2d00 con 0x7f1ab40b48f0 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.922+0000 7f1ab3400640 1 --2- 172.21.15.143:0/1537672970 >> v2:172.21.15.204:3300/0 conn(0x7f1ab4159b70 0x7f1ab40b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1ab9200640 1 --2- 172.21.15.143:0/1537672970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 0x7f1ab4070e80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009e50 tx=0x7f1aac01ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9178a070c724612 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1ab2a00640 1 -- 172.21.15.143:0/1537672970 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0a57d0 con 0x7f1ab40b48f0 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1ab2a00640 1 -- 172.21.15.143:0/1537672970 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac0a5930 con 0x7f1ab40b48f0 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1ab2a00640 1 -- 172.21.15.143:0/1537672970 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0a5cc0 con 0x7f1ab40b48f0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 -- 172.21.15.143:0/1537672970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 msgr2=0x7f1ab4070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 --2- 172.21.15.143:0/1537672970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 0x7f1ab4070e80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009e50 tx=0x7f1aac01ca10 comp rx=0 tx=0).stop 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 -- 172.21.15.143:0/1537672970 shutdown_connections 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 --2- 172.21.15.143:0/1537672970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab40b48f0 0x7f1ab4070e80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 --2- 172.21.15.143:0/1537672970 >> v2:172.21.15.204:3300/0 conn(0x7f1ab4159b70 0x7f1ab40b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 -- 172.21.15.143:0/1537672970 >> 172.21.15.143:0/1537672970 conn(0x7f1ab406c3f0 msgr2=0x7f1ab406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 -- 172.21.15.143:0/1537672970 shutdown_connections 2024-01-15T18:09:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.926+0000 7f1abb200640 1 -- 172.21.15.143:0/1537672970 wait complete. 2024-01-15T18:09:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 Processor -- start 2024-01-15T18:09:20.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 -- start start 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab40b48f0 0x7f1ab414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 0x7f1ab4148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4159b70 0x7f1ab414fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab414f7b0 con 0x7f1ab4157300 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab414eb30 con 0x7f1ab40b48f0 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab414ee10 con 0x7f1ab4159b70 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 0x7f1ab4148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 0x7f1ab4148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42270/0 (socket says 172.21.15.143:42270) 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 -- 172.21.15.143:0/3421722781 learned_addr learned my addr 172.21.15.143:0/3421722781 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3e00640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab40b48f0 0x7f1ab414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab9200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4159b70 0x7f1ab414fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 -- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4159b70 msgr2=0x7f1ab414fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4159b70 0x7f1ab414fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 -- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab40b48f0 msgr2=0x7f1ab414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab40b48f0 0x7f1ab414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 -- 172.21.15.143:0/3421722781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a9c009420 con 0x7f1ab4157300 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1ab3400640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 0x7f1ab4148be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e980 tx=0x7f1a9c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00ca00 con 0x7f1ab4157300 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c00cb60 con 0x7f1ab4157300 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac009b30 con 0x7f1ab4157300 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c013470 con 0x7f1ab4157300 2024-01-15T18:09:20.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.930+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab414f3d0 con 0x7f1ab4157300 2024-01-15T18:09:20.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.934+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab414f8f0 con 0x7f1ab4157300 2024-01-15T18:09:20.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.934+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f1a9c0135d0 con 0x7f1ab4157300 2024-01-15T18:09:20.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.934+0000 7f1aabe00640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a8c066e90 0x7f1a8c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.934+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1a9c085020 con 0x7f1ab4157300 2024-01-15T18:09:20.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.934+0000 7f1ab3e00640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a8c066e90 0x7f1a8c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.934+0000 7f1ab3e00640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a8c066e90 0x7f1a8c069350 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4007fd0 tx=0x7f1aa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:20.938+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1ab414f8f0 con 0x7f1ab4157300 2024-01-15T18:09:21.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.118+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1ab414f8f0 con 0x7f1ab4157300 2024-01-15T18:09:21.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.754+0000 7f1aabe00640 1 -- 172.21.15.143:0/3421722781 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1ab414f8f0 con 0x7f1ab4157300 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a8c066e90 msgr2=0x7f1a8c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a8c066e90 0x7f1a8c069350 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4007fd0 tx=0x7f1aa4009040 comp rx=0 tx=0).stop 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 msgr2=0x7f1ab4148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 0x7f1ab4148be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00e980 tx=0x7f1a9c00ee40 comp rx=0 tx=0).stop 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 shutdown_connections 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1a8c066e90 0x7f1a8c069350 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1ab4159b70 0x7f1ab414fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1ab4157300 0x7f1ab4148be0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 --2- 172.21.15.143:0/3421722781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1ab40b48f0 0x7f1ab414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 >> 172.21.15.143:0/3421722781 conn(0x7f1ab406c3f0 msgr2=0x7f1ab41512d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 shutdown_connections 2024-01-15T18:09:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.758+0000 7f1abb200640 1 -- 172.21.15.143:0/3421722781 wait complete. 2024-01-15T18:09:21.787 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:09:21.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:09:21.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 Processor -- start 2024-01-15T18:09:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 -- start start 2024-01-15T18:09:21.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0cf400 0x7fcefc0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcefc0ce390 0x7fcefc0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc15b5b0 con 0x7fcefc0cda70 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc0cba70 con 0x7fcefc0ce390 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf04800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0cbd10 con 0x7fcefc0cf400 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf02800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcefc0ce390 0x7fcefc0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf01e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcefc0cda70 0x7fcefc0cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44874/0 (socket says 172.21.15.143:44874) 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf01e00640 1 -- 172.21.15.143:0/396079572 learned_addr learned my addr 172.21.15.143:0/396079572 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:21.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf01400640 1 --2- 172.21.15.143:0/396079572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0cf400 0x7fcefc0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:21.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf02800640 1 -- 172.21.15.143:0/396079572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0cf400 msgr2=0x7fcefc0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:21.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf02800640 1 --2- 172.21.15.143:0/396079572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0cf400 0x7fcefc0cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf02800640 1 -- 172.21.15.143:0/396079572 >> v1:172.21.15.143:6789/0 conn(0x7fcefc0cda70 legacy=0x7fcefc0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf02800640 1 -- 172.21.15.143:0/396079572 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc0cbff0 con 0x7fcefc0ce390 2024-01-15T18:09:21.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.922+0000 7fcf00a00640 1 -- 172.21.15.143:0/396079572 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143026312 0 0) 0x7fcefc15b5b0 con 0x7fcefc0cda70 2024-01-15T18:09:21.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf02800640 1 --2- 172.21.15.143:0/396079572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0ce390 0x7fcefc0b0740 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009d90 tx=0x7fcef401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62d3d3dcf6e3c93 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:21.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf00a00640 1 -- 172.21.15.143:0/396079572 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40a5a00 con 0x7fcefc0ce390 2024-01-15T18:09:21.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf00a00640 1 -- 172.21.15.143:0/396079572 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef40a5b60 con 0x7fcefc0ce390 2024-01-15T18:09:21.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf00a00640 1 -- 172.21.15.143:0/396079572 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40ad440 con 0x7fcefc0ce390 2024-01-15T18:09:21.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 -- 172.21.15.143:0/396079572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0ce390 msgr2=0x7fcefc0b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 --2- 172.21.15.143:0/396079572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0ce390 0x7fcefc0b0740 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009d90 tx=0x7fcef401c920 comp rx=0 tx=0).stop 2024-01-15T18:09:21.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 -- 172.21.15.143:0/396079572 shutdown_connections 2024-01-15T18:09:21.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 --2- 172.21.15.143:0/396079572 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0ce390 0x7fcefc0b0740 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 --2- 172.21.15.143:0/396079572 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0cf400 0x7fcefc0cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 -- 172.21.15.143:0/396079572 >> 172.21.15.143:0/396079572 conn(0x7fcefc0d0e20 msgr2=0x7fcefc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:21.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 -- 172.21.15.143:0/396079572 shutdown_connections 2024-01-15T18:09:21.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.926+0000 7fcf04800640 1 -- 172.21.15.143:0/396079572 wait complete. 2024-01-15T18:09:21.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 Processor -- start 2024-01-15T18:09:21.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- start start 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcefc0cda70 0x7fcefc14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0ce390 0x7fcefc0ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 0x7fcefc0b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0c8580 con 0x7fcefc0cf400 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0c8b40 con 0x7fcefc0cda70 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc14fc70 con 0x7fcefc0ce390 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 0x7fcefc0b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 0x7fcefc0b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42280/0 (socket says 172.21.15.143:42280) 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 -- 172.21.15.143:0/80436901 learned_addr learned my addr 172.21.15.143:0/80436901 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:21.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf01400640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0ce390 0x7fcefc0ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf01e00640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcefc0cda70 0x7fcefc14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 -- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0ce390 msgr2=0x7fcefc0ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0ce390 0x7fcefc0ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 -- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcefc0cda70 msgr2=0x7fcefc14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcefc0cda70 0x7fcefc14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 -- 172.21.15.143:0/80436901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4009a70 con 0x7fcefc0cf400 2024-01-15T18:09:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf02800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 0x7fcefc0b3a20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcef40b0040 tx=0x7fcef40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:21.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40adce0 con 0x7fcefc0cf400 2024-01-15T18:09:21.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef40ade40 con 0x7fcefc0cf400 2024-01-15T18:09:21.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc0aef60 con 0x7fcefc0cf400 2024-01-15T18:09:21.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef40b9690 con 0x7fcefc0cf400 2024-01-15T18:09:21.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc0c87d0 con 0x7fcefc0cf400 2024-01-15T18:09:21.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.930+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc0c8cb0 con 0x7fcefc0cf400 2024-01-15T18:09:21.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.934+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fcef40b97f0 con 0x7fcefc0cf400 2024-01-15T18:09:21.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.934+0000 7fcefa000640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fced4066ec0 0x7fced4069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:21.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.934+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcef4120500 con 0x7fcefc0cf400 2024-01-15T18:09:21.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.934+0000 7fcf01e00640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fced4066ec0 0x7fced4069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:21.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.934+0000 7fcf01e00640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fced4066ec0 0x7fced4069380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fceec000f70 tx=0x7fceec007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:21.938+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcef40ed0d0 con 0x7fcefc0cf400 2024-01-15T18:09:22.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.118+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcefc067500 con 0x7fcefc0cf400 2024-01-15T18:09:22.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.118+0000 7fcefa000640 1 -- 172.21.15.143:0/80436901 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fcef4009450 con 0x7fcefc0cf400 2024-01-15T18:09:22.123 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:22.123 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3162517776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3162517776},{"type":"v1","addr":"172.21.15.204:6834","nonce":3162517776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/348084006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":348084006},{"type":"v1","addr":"172.21.15.204:6835","nonce":348084006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2759637098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2759637098},{"type":"v1","addr":"172.21.15.143:6835","nonce":2759637098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:09:18.103084+0000","modified":"2024-01-15T18:09:21.123943+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6336":{"gid":6336,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3767686465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":3767686465},{"type":"v1","addr":"172.21.15.143:6837","nonce":3767686465}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-01-15T18:09:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2024-01-15T18:09:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fced4066ec0 msgr2=0x7fced4069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fced4066ec0 0x7fced4069380 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fceec000f70 tx=0x7fceec007df0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 msgr2=0x7fcefc0b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 0x7fcefc0b3a20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcef40b0040 tx=0x7fcef40196c0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 shutdown_connections 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fced4066ec0 0x7fced4069380 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcefc0cf400 0x7fcefc0b3a20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcefc0ce390 0x7fcefc0ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 --2- 172.21.15.143:0/80436901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcefc0cda70 0x7fcefc14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 >> 172.21.15.143:0/80436901 conn(0x7fcefc0d0e20 msgr2=0x7fcefc0b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 shutdown_connections 2024-01-15T18:09:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.122+0000 7fcf04800640 1 -- 172.21.15.143:0/80436901 wait complete. 2024-01-15T18:09:22.146 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:09:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 Processor -- start 2024-01-15T18:09:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 -- start start 2024-01-15T18:09:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc15b260 0x7fd6dc0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6dc0ccaf0 0x7fd6dc0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc0c4d90 con 0x7fd6dc0d19f0 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6dc15b640 con 0x7fd6dc0ccaf0 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0cf800 con 0x7fd6dc15b260 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6dc0ccaf0 0x7fd6dc0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd6dc0ccaf0 0x7fd6dc0cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42288/0 (socket says 172.21.15.143:42288) 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e1000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd6dc0d19f0 0x7fd6dc0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44882/0 (socket says 172.21.15.143:44882) 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 -- 172.21.15.143:0/840467123 learned_addr learned my addr 172.21.15.143:0/840467123 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e1a00640 1 --2- 172.21.15.143:0/840467123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc15b260 0x7fd6dc0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 -- 172.21.15.143:0/840467123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc15b260 msgr2=0x7fd6dc0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 --2- 172.21.15.143:0/840467123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc15b260 0x7fd6dc0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 -- 172.21.15.143:0/840467123 >> v1:172.21.15.143:6789/0 conn(0x7fd6dc0d19f0 legacy=0x7fd6dc0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6e2400640 1 -- 172.21.15.143:0/840467123 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0cfae0 con 0x7fd6dc0ccaf0 2024-01-15T18:09:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.278+0000 7fd6dbe00640 1 -- 172.21.15.143:0/840467123 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148902304 0 0) 0x7fd6dc0c4d90 con 0x7fd6dc0d19f0 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e2400640 1 --2- 172.21.15.143:0/840467123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0cdd50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009e50 tx=0x7fd6d401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d34227c1eb913b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6dbe00640 1 -- 172.21.15.143:0/840467123 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a57d0 con 0x7fd6dc0ccaf0 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6dbe00640 1 -- 172.21.15.143:0/840467123 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40a5930 con 0x7fd6dc0ccaf0 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6dbe00640 1 -- 172.21.15.143:0/840467123 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a5cc0 con 0x7fd6dc0ccaf0 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 -- 172.21.15.143:0/840467123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc0ccaf0 msgr2=0x7fd6dc0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 --2- 172.21.15.143:0/840467123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0cdd50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009e50 tx=0x7fd6d401ca10 comp rx=0 tx=0).stop 2024-01-15T18:09:22.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 -- 172.21.15.143:0/840467123 shutdown_connections 2024-01-15T18:09:22.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 --2- 172.21.15.143:0/840467123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0cdd50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 --2- 172.21.15.143:0/840467123 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc15b260 0x7fd6dc0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 -- 172.21.15.143:0/840467123 >> 172.21.15.143:0/840467123 conn(0x7fd6dc0d0ed0 msgr2=0x7fd6dc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 -- 172.21.15.143:0/840467123 shutdown_connections 2024-01-15T18:09:22.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.282+0000 7fd6e4400640 1 -- 172.21.15.143:0/840467123 wait complete. 2024-01-15T18:09:22.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 Processor -- start 2024-01-15T18:09:22.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 -- start start 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6dc0d19f0 0x7fd6dc0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 0x7fd6dc0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0b0480 con 0x7fd6dc15b260 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0b0670 con 0x7fd6dc0d19f0 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc0b0950 con 0x7fd6dc0ccaf0 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 0x7fd6dc0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 0x7fd6dc0aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42294/0 (socket says 172.21.15.143:42294) 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 -- 172.21.15.143:0/3153149644 learned_addr learned my addr 172.21.15.143:0/3153149644 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e1000640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6dc0d19f0 0x7fd6dc0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e1a00640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 -- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc0ccaf0 msgr2=0x7fd6dc0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 -- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6dc0d19f0 msgr2=0x7fd6dc0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6dc0d19f0 0x7fd6dc0afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 -- 172.21.15.143:0/3153149644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4009b30 con 0x7fd6dc15b260 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e2400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 0x7fd6dc0aff40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40af040 tx=0x7fd6d40197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40b8670 con 0x7fd6dc15b260 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40b87d0 con 0x7fd6dc15b260 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40b8b80 con 0x7fd6dc15b260 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc0b0c30 con 0x7fd6dc15b260 2024-01-15T18:09:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.286+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc1599c0 con 0x7fd6dc15b260 2024-01-15T18:09:22.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.290+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc0b10e0 con 0x7fd6dc15b260 2024-01-15T18:09:22.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.290+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fd6d40b8ce0 con 0x7fd6dc15b260 2024-01-15T18:09:22.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.290+0000 7fd6d9600640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6b4066e90 0x7fd6b4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.290+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd6d4120540 con 0x7fd6dc15b260 2024-01-15T18:09:22.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.290+0000 7fd6e1a00640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6b4066e90 0x7fd6b4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.290+0000 7fd6e1a00640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6b4066e90 0x7fd6b4069350 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0c9710 tx=0x7fd6cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.294+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd6d40ed110 con 0x7fd6dc15b260 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.442+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6dc0b0a90 con 0x7fd6dc15b260 2024-01-15T18:09:22.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6d9600640 1 -- 172.21.15.143:0/3153149644 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16644 (secure 0 0 0) 0x7fd6dc0b0a90 con 0x7fd6dc15b260 2024-01-15T18:09:22.450 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:22.450 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":60,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:09:19.596044+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:09:16.375775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-15T18:09:17.090306+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6b4066e90 msgr2=0x7fd6b4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6b4066e90 0x7fd6b4069350 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc0c9710 tx=0x7fd6cc008040 comp rx=0 tx=0).stop 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 msgr2=0x7fd6dc0aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 0x7fd6dc0aff40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40af040 tx=0x7fd6d40197b0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 shutdown_connections 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6b4066e90 0x7fd6b4069350 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6dc15b260 0x7fd6dc0aff40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6dc0d19f0 0x7fd6dc0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 --2- 172.21.15.143:0/3153149644 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6dc0ccaf0 0x7fd6dc0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 >> 172.21.15.143:0/3153149644 conn(0x7fd6dc0d0ed0 msgr2=0x7fd6dc0c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 shutdown_connections 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.446+0000 7fd6e4400640 1 -- 172.21.15.143:0/3153149644 wait complete. 2024-01-15T18:09:22.472 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:09:22.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:09:22.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.602+0000 7fb821e00640 1 Processor -- start 2024-01-15T18:09:22.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.602+0000 7fb821e00640 1 -- start start 2024-01-15T18:09:22.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.602+0000 7fb821e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb81c143780 0x7fb81c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb821e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb81c0ce520 0x7fb81c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb821e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81c0ceb50 con 0x7fb81c0720f0 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb821e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c15c110 con 0x7fb81c143780 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb821e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb81c072530 con 0x7fb81c0ce520 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb81c143780 0x7fb81c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb81c143780 0x7fb81c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42304/0 (socket says 172.21.15.143:42304) 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 -- 172.21.15.143:0/1107294078 learned_addr learned my addr 172.21.15.143:0/1107294078 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81a000640 1 --2- 172.21.15.143:0/1107294078 >> v2:172.21.15.204:3300/0 conn(0x7fb81c0ce520 0x7fb81c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 -- 172.21.15.143:0/1107294078 >> v2:172.21.15.204:3300/0 conn(0x7fb81c0ce520 msgr2=0x7fb81c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 --2- 172.21.15.143:0/1107294078 >> v2:172.21.15.204:3300/0 conn(0x7fb81c0ce520 0x7fb81c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 -- 172.21.15.143:0/1107294078 >> v1:172.21.15.143:6789/0 conn(0x7fb81c0720f0 legacy=0x7fb81c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 -- 172.21.15.143:0/1107294078 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c15b7e0 con 0x7fb81c143780 2024-01-15T18:09:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb819600640 1 -- 172.21.15.143:0/1107294078 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064586267 0 0) 0x7fb8140014a0 con 0x7fb81c0720f0 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb81aa00640 1 --2- 172.21.15.143:0/1107294078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c143780 0x7fb81c0cdfe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb81c06beb0 tx=0x7fb80c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283cd0eb407d5180 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.606+0000 7fb819600640 1 -- 172.21.15.143:0/1107294078 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0ab610 con 0x7fb81c143780 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb819600640 1 -- 172.21.15.143:0/1107294078 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c0ab770 con 0x7fb81c143780 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb819600640 1 -- 172.21.15.143:0/1107294078 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0abb00 con 0x7fb81c143780 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- 172.21.15.143:0/1107294078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c143780 msgr2=0x7fb81c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 --2- 172.21.15.143:0/1107294078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c143780 0x7fb81c0cdfe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb81c06beb0 tx=0x7fb80c01c500 comp rx=0 tx=0).stop 2024-01-15T18:09:22.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- 172.21.15.143:0/1107294078 shutdown_connections 2024-01-15T18:09:22.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 --2- 172.21.15.143:0/1107294078 >> v2:172.21.15.204:3300/0 conn(0x7fb81c0ce520 0x7fb81c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 --2- 172.21.15.143:0/1107294078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c143780 0x7fb81c0cdfe0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- 172.21.15.143:0/1107294078 >> 172.21.15.143:0/1107294078 conn(0x7fb81c06c3f0 msgr2=0x7fb81c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- 172.21.15.143:0/1107294078 shutdown_connections 2024-01-15T18:09:22.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- 172.21.15.143:0/1107294078 wait complete. 2024-01-15T18:09:22.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 Processor -- start 2024-01-15T18:09:22.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- start start 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 0x7fb81c1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb81c0ce520 0x7fb81c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb81c143780 0x7fb81c134740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c159cf0 con 0x7fb81c0720f0 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c140c60 con 0x7fb81c0ce520 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.610+0000 7fb821e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c1411e0 con 0x7fb81c143780 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 0x7fb81c1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 0x7fb81c1597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42314/0 (socket says 172.21.15.143:42314) 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 -- 172.21.15.143:0/33906689 learned_addr learned my addr 172.21.15.143:0/33906689 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81a000640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb81c0ce520 0x7fb81c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81b400640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb81c143780 0x7fb81c134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 -- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb81c143780 msgr2=0x7fb81c134740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb81c143780 0x7fb81c134740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 -- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb81c0ce520 msgr2=0x7fb81c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb81c0ce520 0x7fb81c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 -- 172.21.15.143:0/33906689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c0094d0 con 0x7fb81c0720f0 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81a000640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb81c0ce520 0x7fb81c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb81aa00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 0x7fb81c1597b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0ad040 tx=0x7fb80c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c0abcf0 con 0x7fb81c0720f0 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c0b8070 con 0x7fb81c0720f0 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c004220 con 0x7fb81c0720f0 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c0b37e0 con 0x7fb81c0720f0 2024-01-15T18:09:22.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c0b3d80 con 0x7fb81c0720f0 2024-01-15T18:09:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c0b3f70 con 0x7fb81c0720f0 2024-01-15T18:09:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fb80c004380 con 0x7fb81c0720f0 2024-01-15T18:09:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.614+0000 7fb813400640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7f4066e90 0x7fb7f4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.618+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb80c11ecd0 con 0x7fb81c0720f0 2024-01-15T18:09:22.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.618+0000 7fb81a000640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7f4066e90 0x7fb7f4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.618+0000 7fb81a000640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7f4066e90 0x7fb7f4069350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb81c134500 tx=0x7fb804008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.618+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb80c0eb760 con 0x7fb81c0720f0 2024-01-15T18:09:22.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.770+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb81c067500 con 0x7fb7f4066e90 2024-01-15T18:09:22.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.778+0000 7fb813400640 1 -- 172.21.15.143:0/33906689 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287990 (secure 0 0 0) 0x7fb81c067500 con 0x7fb7f4066e90 2024-01-15T18:09:22.785 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:22.787 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7f4066e90 msgr2=0x7fb7f4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7f4066e90 0x7fb7f4069350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb81c134500 tx=0x7fb804008040 comp rx=0 tx=0).stop 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 msgr2=0x7fb81c1597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 0x7fb81c1597b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0ad040 tx=0x7fb80c01ad40 comp rx=0 tx=0).stop 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 shutdown_connections 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb7f4066e90 0x7fb7f4069350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb81c143780 0x7fb81c134740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb81c0ce520 0x7fb81c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 --2- 172.21.15.143:0/33906689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb81c0720f0 0x7fb81c1597b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 >> 172.21.15.143:0/33906689 conn(0x7fb81c06c3f0 msgr2=0x7fb81c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 shutdown_connections 2024-01-15T18:09:22.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.782+0000 7fb821e00640 1 -- 172.21.15.143:0/33906689 wait complete. 2024-01-15T18:09:22.807 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-01-15T18:09:22.696589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289596,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685124,"statfs":{"total":773094113280,"available":772797566976,"internally_reserved":0,"allocated":7020544,"data_stored":3608564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991199"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683525+0000","last_change":"2024-01-15T18:09:18.622882+0000","last_active":"2024-01-15T18:09:19.683525+0000","last_peered":"2024-01-15T18:09:19.683525+0000","last_clean":"2024-01-15T18:09:19.683525+0000","last_became_active":"2024-01-15T18:09:18.622573+0000","last_became_peered":"2024-01-15T18:09:18.622573+0000","last_unstale":"2024-01-15T18:09:19.683525+0000","last_undegraded":"2024-01-15T18:09:19.683525+0000","last_fullsized":"2024-01-15T18:09:19.683525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:50.157284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685013+0000","last_change":"2024-01-15T18:09:18.618943+0000","last_active":"2024-01-15T18:09:19.685013+0000","last_peered":"2024-01-15T18:09:19.685013+0000","last_clean":"2024-01-15T18:09:19.685013+0000","last_became_active":"2024-01-15T18:09:18.618637+0000","last_became_peered":"2024-01-15T18:09:18.618637+0000","last_unstale":"2024-01-15T18:09:19.685013+0000","last_undegraded":"2024-01-15T18:09:19.685013+0000","last_fullsized":"2024-01-15T18:09:19.685013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:45.803655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683517+0000","last_change":"2024-01-15T18:09:18.624968+0000","last_active":"2024-01-15T18:09:19.683517+0000","last_peered":"2024-01-15T18:09:19.683517+0000","last_clean":"2024-01-15T18:09:19.683517+0000","last_became_active":"2024-01-15T18:09:18.624432+0000","last_became_peered":"2024-01-15T18:09:18.624432+0000","last_unstale":"2024-01-15T18:09:19.683517+0000","last_undegraded":"2024-01-15T18:09:19.683517+0000","last_fullsized":"2024-01-15T18:09:19.683517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:13.831851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605823+0000","last_change":"2024-01-15T18:09:18.628270+0000","last_active":"2024-01-15T18:09:19.605823+0000","last_peered":"2024-01-15T18:09:19.605823+0000","last_clean":"2024-01-15T18:09:19.605823+0000","last_became_active":"2024-01-15T18:09:18.627968+0000","last_became_peered":"2024-01-15T18:09:18.627968+0000","last_unstale":"2024-01-15T18:09:19.605823+0000","last_undegraded":"2024-01-15T18:09:19.605823+0000","last_fullsized":"2024-01-15T18:09:19.605823+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:09.480229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683476+0000","last_change":"2024-01-15T18:09:18.623667+0000","last_active":"2024-01-15T18:09:19.683476+0000","last_peered":"2024-01-15T18:09:19.683476+0000","last_clean":"2024-01-15T18:09:19.683476+0000","last_became_active":"2024-01-15T18:09:18.623377+0000","last_became_peered":"2024-01-15T18:09:18.623377+0000","last_unstale":"2024-01-15T18:09:19.683476+0000","last_undegraded":"2024-01-15T18:09:19.683476+0000","last_fullsized":"2024-01-15T18:09:19.683476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:58:59.747220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684671+0000","last_change":"2024-01-15T18:09:18.620077+0000","last_active":"2024-01-15T18:09:19.684671+0000","last_peered":"2024-01-15T18:09:19.684671+0000","last_clean":"2024-01-15T18:09:19.684671+0000","last_became_active":"2024-01-15T18:09:18.619368+0000","last_became_peered":"2024-01-15T18:09:18.619368+0000","last_unstale":"2024-01-15T18:09:19.684671+0000","last_undegraded":"2024-01-15T18:09:19.684671+0000","last_fullsized":"2024-01-15T18:09:19.684671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:12.189491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.608844+0000","last_change":"2024-01-15T18:09:18.618524+0000","last_active":"2024-01-15T18:09:19.608844+0000","last_peered":"2024-01-15T18:09:19.608844+0000","last_clean":"2024-01-15T18:09:19.608844+0000","last_became_active":"2024-01-15T18:09:18.618234+0000","last_became_peered":"2024-01-15T18:09:18.618234+0000","last_unstale":"2024-01-15T18:09:19.608844+0000","last_undegraded":"2024-01-15T18:09:19.608844+0000","last_fullsized":"2024-01-15T18:09:19.608844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:08.563799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683604+0000","last_change":"2024-01-15T18:09:18.615739+0000","last_active":"2024-01-15T18:09:19.683604+0000","last_peered":"2024-01-15T18:09:19.683604+0000","last_clean":"2024-01-15T18:09:19.683604+0000","last_became_active":"2024-01-15T18:09:18.615483+0000","last_became_peered":"2024-01-15T18:09:18.615483+0000","last_unstale":"2024-01-15T18:09:19.683604+0000","last_undegraded":"2024-01-15T18:09:19.683604+0000","last_fullsized":"2024-01-15T18:09:19.683604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:48:51.860359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.609446+0000","last_change":"2024-01-15T18:09:18.622350+0000","last_active":"2024-01-15T18:09:19.609446+0000","last_peered":"2024-01-15T18:09:19.609446+0000","last_clean":"2024-01-15T18:09:19.609446+0000","last_became_active":"2024-01-15T18:09:18.620475+0000","last_became_peered":"2024-01-15T18:09:18.620475+0000","last_unstale":"2024-01-15T18:09:19.609446+0000","last_undegraded":"2024-01-15T18:09:19.609446+0000","last_fullsized":"2024-01-15T18:09:19.609446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:46.800650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685817+0000","last_change":"2024-01-15T18:09:18.616969+0000","last_active":"2024-01-15T18:09:19.685817+0000","last_peered":"2024-01-15T18:09:19.685817+0000","last_clean":"2024-01-15T18:09:19.685817+0000","last_became_active":"2024-01-15T18:09:18.616088+0000","last_became_peered":"2024-01-15T18:09:18.616088+0000","last_unstale":"2024-01-15T18:09:19.685817+0000","last_undegraded":"2024-01-15T18:09:19.685817+0000","last_fullsized":"2024-01-15T18:09:19.685817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:29.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685635+0000","last_change":"2024-01-15T18:09:18.625131+0000","last_active":"2024-01-15T18:09:19.685635+0000","last_peered":"2024-01-15T18:09:19.685635+0000","last_clean":"2024-01-15T18:09:19.685635+0000","last_became_active":"2024-01-15T18:09:18.624838+0000","last_became_peered":"2024-01-15T18:09:18.624838+0000","last_unstale":"2024-01-15T18:09:19.685635+0000","last_undegraded":"2024-01-15T18:09:19.685635+0000","last_fullsized":"2024-01-15T18:09:19.685635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:11:03.527003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605868+0000","last_change":"2024-01-15T18:09:18.629407+0000","last_active":"2024-01-15T18:09:19.605868+0000","last_peered":"2024-01-15T18:09:19.605868+0000","last_clean":"2024-01-15T18:09:19.605868+0000","last_became_active":"2024-01-15T18:09:18.628281+0000","last_became_peered":"2024-01-15T18:09:18.628281+0000","last_unstale":"2024-01-15T18:09:19.605868+0000","last_undegraded":"2024-01-15T18:09:19.605868+0000","last_fullsized":"2024-01-15T18:09:19.605868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:00:01.967586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610308+0000","last_change":"2024-01-15T18:09:18.620730+0000","last_active":"2024-01-15T18:09:19.610308+0000","last_peered":"2024-01-15T18:09:19.610308+0000","last_clean":"2024-01-15T18:09:19.610308+0000","last_became_active":"2024-01-15T18:09:18.620225+0000","last_became_peered":"2024-01-15T18:09:18.620225+0000","last_unstale":"2024-01-15T18:09:19.610308+0000","last_undegraded":"2024-01-15T18:09:19.610308+0000","last_fullsized":"2024-01-15T18:09:19.610308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:53.807179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.609092+0000","last_change":"2024-01-15T18:09:18.614511+0000","last_active":"2024-01-15T18:09:19.609092+0000","last_peered":"2024-01-15T18:09:19.609092+0000","last_clean":"2024-01-15T18:09:19.609092+0000","last_became_active":"2024-01-15T18:09:18.614217+0000","last_became_peered":"2024-01-15T18:09:18.614217+0000","last_unstale":"2024-01-15T18:09:19.609092+0000","last_undegraded":"2024-01-15T18:09:19.609092+0000","last_fullsized":"2024-01-15T18:09:19.609092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:50.233905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.608934+0000","last_change":"2024-01-15T18:09:18.620903+0000","last_active":"2024-01-15T18:09:19.608934+0000","last_peered":"2024-01-15T18:09:19.608934+0000","last_clean":"2024-01-15T18:09:19.608934+0000","last_became_active":"2024-01-15T18:09:18.616572+0000","last_became_peered":"2024-01-15T18:09:18.616572+0000","last_unstale":"2024-01-15T18:09:19.608934+0000","last_undegraded":"2024-01-15T18:09:19.608934+0000","last_fullsized":"2024-01-15T18:09:19.608934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:39.789871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683900+0000","last_change":"2024-01-15T18:09:18.619610+0000","last_active":"2024-01-15T18:09:19.683900+0000","last_peered":"2024-01-15T18:09:19.683900+0000","last_clean":"2024-01-15T18:09:19.683900+0000","last_became_active":"2024-01-15T18:09:18.619274+0000","last_became_peered":"2024-01-15T18:09:18.619274+0000","last_unstale":"2024-01-15T18:09:19.683900+0000","last_undegraded":"2024-01-15T18:09:19.683900+0000","last_fullsized":"2024-01-15T18:09:19.683900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:17.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685032+0000","last_change":"2024-01-15T18:09:18.619973+0000","last_active":"2024-01-15T18:09:19.685032+0000","last_peered":"2024-01-15T18:09:19.685032+0000","last_clean":"2024-01-15T18:09:19.685032+0000","last_became_active":"2024-01-15T18:09:18.618815+0000","last_became_peered":"2024-01-15T18:09:18.618815+0000","last_unstale":"2024-01-15T18:09:19.685032+0000","last_undegraded":"2024-01-15T18:09:19.685032+0000","last_fullsized":"2024-01-15T18:09:19.685032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:34:45.496931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683546+0000","last_change":"2024-01-15T18:09:18.623820+0000","last_active":"2024-01-15T18:09:19.683546+0000","last_peered":"2024-01-15T18:09:19.683546+0000","last_clean":"2024-01-15T18:09:19.683546+0000","last_became_active":"2024-01-15T18:09:18.623525+0000","last_became_peered":"2024-01-15T18:09:18.623525+0000","last_unstale":"2024-01-15T18:09:19.683546+0000","last_undegraded":"2024-01-15T18:09:19.683546+0000","last_fullsized":"2024-01-15T18:09:19.683546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:32:44.586300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683464+0000","last_change":"2024-01-15T18:09:18.624355+0000","last_active":"2024-01-15T18:09:19.683464+0000","last_peered":"2024-01-15T18:09:19.683464+0000","last_clean":"2024-01-15T18:09:19.683464+0000","last_became_active":"2024-01-15T18:09:18.624036+0000","last_became_peered":"2024-01-15T18:09:18.624036+0000","last_unstale":"2024-01-15T18:09:19.683464+0000","last_undegraded":"2024-01-15T18:09:19.683464+0000","last_fullsized":"2024-01-15T18:09:19.683464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:05.821944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684818+0000","last_change":"2024-01-15T18:09:18.621278+0000","last_active":"2024-01-15T18:09:19.684818+0000","last_peered":"2024-01-15T18:09:19.684818+0000","last_clean":"2024-01-15T18:09:19.684818+0000","last_became_active":"2024-01-15T18:09:18.620995+0000","last_became_peered":"2024-01-15T18:09:18.620995+0000","last_unstale":"2024-01-15T18:09:19.684818+0000","last_undegraded":"2024-01-15T18:09:19.684818+0000","last_fullsized":"2024-01-15T18:09:19.684818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:54.637682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683347+0000","last_change":"2024-01-15T18:09:18.619885+0000","last_active":"2024-01-15T18:09:19.683347+0000","last_peered":"2024-01-15T18:09:19.683347+0000","last_clean":"2024-01-15T18:09:19.683347+0000","last_became_active":"2024-01-15T18:09:18.619592+0000","last_became_peered":"2024-01-15T18:09:18.619592+0000","last_unstale":"2024-01-15T18:09:19.683347+0000","last_undegraded":"2024-01-15T18:09:19.683347+0000","last_fullsized":"2024-01-15T18:09:19.683347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:55.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684740+0000","last_change":"2024-01-15T18:09:18.617385+0000","last_active":"2024-01-15T18:09:19.684740+0000","last_peered":"2024-01-15T18:09:19.684740+0000","last_clean":"2024-01-15T18:09:19.684740+0000","last_became_active":"2024-01-15T18:09:18.617115+0000","last_became_peered":"2024-01-15T18:09:18.617115+0000","last_unstale":"2024-01-15T18:09:19.684740+0000","last_undegraded":"2024-01-15T18:09:19.684740+0000","last_fullsized":"2024-01-15T18:09:19.684740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:48:19.391925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683258+0000","last_change":"2024-01-15T18:09:18.626964+0000","last_active":"2024-01-15T18:09:19.683258+0000","last_peered":"2024-01-15T18:09:19.683258+0000","last_clean":"2024-01-15T18:09:19.683258+0000","last_became_active":"2024-01-15T18:09:18.626671+0000","last_became_peered":"2024-01-15T18:09:18.626671+0000","last_unstale":"2024-01-15T18:09:19.683258+0000","last_undegraded":"2024-01-15T18:09:19.683258+0000","last_fullsized":"2024-01-15T18:09:19.683258+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:00:01.967586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610173+0000","last_change":"2024-01-15T18:09:18.628704+0000","last_active":"2024-01-15T18:09:19.610173+0000","last_peered":"2024-01-15T18:09:19.610173+0000","last_clean":"2024-01-15T18:09:19.610173+0000","last_became_active":"2024-01-15T18:09:18.628270+0000","last_became_peered":"2024-01-15T18:09:18.628270+0000","last_unstale":"2024-01-15T18:09:19.610173+0000","last_undegraded":"2024-01-15T18:09:19.610173+0000","last_fullsized":"2024-01-15T18:09:19.610173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:24.673049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683661+0000","last_change":"2024-01-15T18:09:18.617173+0000","last_active":"2024-01-15T18:09:19.683661+0000","last_peered":"2024-01-15T18:09:19.683661+0000","last_clean":"2024-01-15T18:09:19.683661+0000","last_became_active":"2024-01-15T18:09:18.616892+0000","last_became_peered":"2024-01-15T18:09:18.616892+0000","last_unstale":"2024-01-15T18:09:19.683661+0000","last_undegraded":"2024-01-15T18:09:19.683661+0000","last_fullsized":"2024-01-15T18:09:19.683661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:59.185525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605766+0000","last_change":"2024-01-15T18:09:18.619788+0000","last_active":"2024-01-15T18:09:19.605766+0000","last_peered":"2024-01-15T18:09:19.605766+0000","last_clean":"2024-01-15T18:09:19.605766+0000","last_became_active":"2024-01-15T18:09:18.619484+0000","last_became_peered":"2024-01-15T18:09:18.619484+0000","last_unstale":"2024-01-15T18:09:19.605766+0000","last_undegraded":"2024-01-15T18:09:19.605766+0000","last_fullsized":"2024-01-15T18:09:19.605766+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:56.729572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685303+0000","last_change":"2024-01-15T18:09:18.618042+0000","last_active":"2024-01-15T18:09:19.685303+0000","last_peered":"2024-01-15T18:09:19.685303+0000","last_clean":"2024-01-15T18:09:19.685303+0000","last_became_active":"2024-01-15T18:09:18.617708+0000","last_became_peered":"2024-01-15T18:09:18.617708+0000","last_unstale":"2024-01-15T18:09:19.685303+0000","last_undegraded":"2024-01-15T18:09:19.685303+0000","last_fullsized":"2024-01-15T18:09:19.685303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:26:00.018458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683811+0000","last_change":"2024-01-15T18:09:18.622766+0000","last_active":"2024-01-15T18:09:19.683811+0000","last_peered":"2024-01-15T18:09:19.683811+0000","last_clean":"2024-01-15T18:09:19.683811+0000","last_became_active":"2024-01-15T18:09:18.621303+0000","last_became_peered":"2024-01-15T18:09:18.621303+0000","last_unstale":"2024-01-15T18:09:19.683811+0000","last_undegraded":"2024-01-15T18:09:19.683811+0000","last_fullsized":"2024-01-15T18:09:19.683811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:44.854034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685373+0000","last_change":"2024-01-15T18:09:18.625819+0000","last_active":"2024-01-15T18:09:19.685373+0000","last_peered":"2024-01-15T18:09:19.685373+0000","last_clean":"2024-01-15T18:09:19.685373+0000","last_became_active":"2024-01-15T18:09:18.625540+0000","last_became_peered":"2024-01-15T18:09:18.625540+0000","last_unstale":"2024-01-15T18:09:19.685373+0000","last_undegraded":"2024-01-15T18:09:19.685373+0000","last_fullsized":"2024-01-15T18:09:19.685373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:52:44.879436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683165+0000","last_change":"2024-01-15T18:09:18.620524+0000","last_active":"2024-01-15T18:09:19.683165+0000","last_peered":"2024-01-15T18:09:19.683165+0000","last_clean":"2024-01-15T18:09:19.683165+0000","last_became_active":"2024-01-15T18:09:18.620252+0000","last_became_peered":"2024-01-15T18:09:18.620252+0000","last_unstale":"2024-01-15T18:09:19.683165+0000","last_undegraded":"2024-01-15T18:09:19.683165+0000","last_fullsized":"2024-01-15T18:09:19.683165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:09.480229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683160+0000","last_change":"2024-01-15T18:09:18.616783+0000","last_active":"2024-01-15T18:09:19.683160+0000","last_peered":"2024-01-15T18:09:19.683160+0000","last_clean":"2024-01-15T18:09:19.683160+0000","last_became_active":"2024-01-15T18:09:18.616346+0000","last_became_peered":"2024-01-15T18:09:18.616346+0000","last_unstale":"2024-01-15T18:09:19.683160+0000","last_undegraded":"2024-01-15T18:09:19.683160+0000","last_fullsized":"2024-01-15T18:09:19.683160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:55.016220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684351+0000","last_change":"2024-01-15T18:09:18.628746+0000","last_active":"2024-01-15T18:09:19.684351+0000","last_peered":"2024-01-15T18:09:19.684351+0000","last_clean":"2024-01-15T18:09:19.684351+0000","last_became_active":"2024-01-15T18:09:18.628474+0000","last_became_peered":"2024-01-15T18:09:18.628474+0000","last_unstale":"2024-01-15T18:09:19.684351+0000","last_undegraded":"2024-01-15T18:09:19.684351+0000","last_fullsized":"2024-01-15T18:09:19.684351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:56.729572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684792+0000","last_change":"2024-01-15T18:09:18.625975+0000","last_active":"2024-01-15T18:09:19.684792+0000","last_peered":"2024-01-15T18:09:19.684792+0000","last_clean":"2024-01-15T18:09:19.684792+0000","last_became_active":"2024-01-15T18:09:18.625673+0000","last_became_peered":"2024-01-15T18:09:18.625673+0000","last_unstale":"2024-01-15T18:09:19.684792+0000","last_undegraded":"2024-01-15T18:09:19.684792+0000","last_fullsized":"2024-01-15T18:09:19.684792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:55.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.754293+0000","last_change":"2024-01-15T18:09:17.732480+0000","last_active":"2024-01-15T18:09:18.754293+0000","last_peered":"2024-01-15T18:09:18.754293+0000","last_clean":"2024-01-15T18:09:18.754293+0000","last_became_active":"2024-01-15T18:09:17.732184+0000","last_became_peered":"2024-01-15T18:09:17.732184+0000","last_unstale":"2024-01-15T18:09:18.754293+0000","last_undegraded":"2024-01-15T18:09:18.754293+0000","last_fullsized":"2024-01-15T18:09:18.754293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:18.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685857+0000","last_change":"2024-01-15T18:09:18.626690+0000","last_active":"2024-01-15T18:09:19.685857+0000","last_peered":"2024-01-15T18:09:19.685857+0000","last_clean":"2024-01-15T18:09:19.685857+0000","last_became_active":"2024-01-15T18:09:18.626388+0000","last_became_peered":"2024-01-15T18:09:18.626388+0000","last_unstale":"2024-01-15T18:09:19.685857+0000","last_undegraded":"2024-01-15T18:09:19.685857+0000","last_fullsized":"2024-01-15T18:09:19.685857+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:05.533517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685004+0000","last_change":"2024-01-15T18:09:17.737390+0000","last_active":"2024-01-15T18:09:19.685004+0000","last_peered":"2024-01-15T18:09:19.685004+0000","last_clean":"2024-01-15T18:09:19.685004+0000","last_became_active":"2024-01-15T18:09:17.737051+0000","last_became_peered":"2024-01-15T18:09:17.737051+0000","last_unstale":"2024-01-15T18:09:19.685004+0000","last_undegraded":"2024-01-15T18:09:19.685004+0000","last_fullsized":"2024-01-15T18:09:19.685004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:54.296498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685080+0000","last_change":"2024-01-15T18:09:18.617990+0000","last_active":"2024-01-15T18:09:19.685080+0000","last_peered":"2024-01-15T18:09:19.685080+0000","last_clean":"2024-01-15T18:09:19.685080+0000","last_became_active":"2024-01-15T18:09:18.617626+0000","last_became_peered":"2024-01-15T18:09:18.617626+0000","last_unstale":"2024-01-15T18:09:19.685080+0000","last_undegraded":"2024-01-15T18:09:19.685080+0000","last_fullsized":"2024-01-15T18:09:19.685080+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:13.831851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.750931+0000","last_change":"2024-01-15T18:09:17.733411+0000","last_active":"2024-01-15T18:09:18.750931+0000","last_peered":"2024-01-15T18:09:18.750931+0000","last_clean":"2024-01-15T18:09:18.750931+0000","last_became_active":"2024-01-15T18:09:17.733112+0000","last_became_peered":"2024-01-15T18:09:17.733112+0000","last_unstale":"2024-01-15T18:09:18.750931+0000","last_undegraded":"2024-01-15T18:09:18.750931+0000","last_fullsized":"2024-01-15T18:09:18.750931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:41.918180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685436+0000","last_change":"2024-01-15T18:09:18.626341+0000","last_active":"2024-01-15T18:09:19.685436+0000","last_peered":"2024-01-15T18:09:19.685436+0000","last_clean":"2024-01-15T18:09:19.685436+0000","last_became_active":"2024-01-15T18:09:18.625990+0000","last_became_peered":"2024-01-15T18:09:18.625990+0000","last_unstale":"2024-01-15T18:09:19.685436+0000","last_undegraded":"2024-01-15T18:09:19.685436+0000","last_fullsized":"2024-01-15T18:09:19.685436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:38.466718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605959+0000","last_change":"2024-01-15T18:09:17.732662+0000","last_active":"2024-01-15T18:09:19.605959+0000","last_peered":"2024-01-15T18:09:19.605959+0000","last_clean":"2024-01-15T18:09:19.605959+0000","last_became_active":"2024-01-15T18:09:17.732350+0000","last_became_peered":"2024-01-15T18:09:17.732350+0000","last_unstale":"2024-01-15T18:09:19.605959+0000","last_undegraded":"2024-01-15T18:09:19.605959+0000","last_fullsized":"2024-01-15T18:09:19.605959+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:33.617418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684989+0000","last_change":"2024-01-15T18:09:18.615025+0000","last_active":"2024-01-15T18:09:19.684989+0000","last_peered":"2024-01-15T18:09:19.684989+0000","last_clean":"2024-01-15T18:09:19.684989+0000","last_became_active":"2024-01-15T18:09:18.614428+0000","last_became_peered":"2024-01-15T18:09:18.614428+0000","last_unstale":"2024-01-15T18:09:19.684989+0000","last_undegraded":"2024-01-15T18:09:19.684989+0000","last_fullsized":"2024-01-15T18:09:19.684989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:53.698036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.750748+0000","last_change":"2024-01-15T18:09:17.731122+0000","last_active":"2024-01-15T18:09:18.750748+0000","last_peered":"2024-01-15T18:09:18.750748+0000","last_clean":"2024-01-15T18:09:18.750748+0000","last_became_active":"2024-01-15T18:09:17.730734+0000","last_became_peered":"2024-01-15T18:09:17.730734+0000","last_unstale":"2024-01-15T18:09:18.750748+0000","last_undegraded":"2024-01-15T18:09:18.750748+0000","last_fullsized":"2024-01-15T18:09:18.750748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:09.626767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685275+0000","last_change":"2024-01-15T18:09:18.621477+0000","last_active":"2024-01-15T18:09:19.685275+0000","last_peered":"2024-01-15T18:09:19.685275+0000","last_clean":"2024-01-15T18:09:19.685275+0000","last_became_active":"2024-01-15T18:09:18.621143+0000","last_became_peered":"2024-01-15T18:09:18.621143+0000","last_unstale":"2024-01-15T18:09:19.685275+0000","last_undegraded":"2024-01-15T18:09:19.685275+0000","last_fullsized":"2024-01-15T18:09:19.685275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:41.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.748980+0000","last_change":"2024-01-15T18:09:17.740583+0000","last_active":"2024-01-15T18:09:18.748980+0000","last_peered":"2024-01-15T18:09:18.748980+0000","last_clean":"2024-01-15T18:09:18.748980+0000","last_became_active":"2024-01-15T18:09:17.740116+0000","last_became_peered":"2024-01-15T18:09:17.740116+0000","last_unstale":"2024-01-15T18:09:18.748980+0000","last_undegraded":"2024-01-15T18:09:18.748980+0000","last_fullsized":"2024-01-15T18:09:18.748980+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:45.794446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610113+0000","last_change":"2024-01-15T18:09:18.615675+0000","last_active":"2024-01-15T18:09:19.610113+0000","last_peered":"2024-01-15T18:09:19.610113+0000","last_clean":"2024-01-15T18:09:19.610113+0000","last_became_active":"2024-01-15T18:09:18.615168+0000","last_became_peered":"2024-01-15T18:09:18.615168+0000","last_unstale":"2024-01-15T18:09:19.610113+0000","last_undegraded":"2024-01-15T18:09:19.610113+0000","last_fullsized":"2024-01-15T18:09:19.610113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:57.675563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605537+0000","last_change":"2024-01-15T18:09:17.729345+0000","last_active":"2024-01-15T18:09:19.605537+0000","last_peered":"2024-01-15T18:09:19.605537+0000","last_clean":"2024-01-15T18:09:19.605537+0000","last_became_active":"2024-01-15T18:09:17.728928+0000","last_became_peered":"2024-01-15T18:09:17.728928+0000","last_unstale":"2024-01-15T18:09:19.605537+0000","last_undegraded":"2024-01-15T18:09:19.605537+0000","last_fullsized":"2024-01-15T18:09:19.605537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:38.783668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684224+0000","last_change":"2024-01-15T18:09:18.623008+0000","last_active":"2024-01-15T18:09:19.684224+0000","last_peered":"2024-01-15T18:09:19.684224+0000","last_clean":"2024-01-15T18:09:19.684224+0000","last_became_active":"2024-01-15T18:09:18.622716+0000","last_became_peered":"2024-01-15T18:09:18.622716+0000","last_unstale":"2024-01-15T18:09:19.684224+0000","last_undegraded":"2024-01-15T18:09:19.684224+0000","last_fullsized":"2024-01-15T18:09:19.684224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:03.810960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.604149+0000","last_change":"2024-01-15T18:09:17.737345+0000","last_active":"2024-01-15T18:09:18.604149+0000","last_peered":"2024-01-15T18:09:18.604149+0000","last_clean":"2024-01-15T18:09:18.604149+0000","last_became_active":"2024-01-15T18:09:17.737072+0000","last_became_peered":"2024-01-15T18:09:17.737072+0000","last_unstale":"2024-01-15T18:09:18.604149+0000","last_undegraded":"2024-01-15T18:09:18.604149+0000","last_fullsized":"2024-01-15T18:09:18.604149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:17.051164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.609410+0000","last_change":"2024-01-15T18:09:18.618214+0000","last_active":"2024-01-15T18:09:19.609410+0000","last_peered":"2024-01-15T18:09:19.609410+0000","last_clean":"2024-01-15T18:09:19.609410+0000","last_became_active":"2024-01-15T18:09:18.617903+0000","last_became_peered":"2024-01-15T18:09:18.617903+0000","last_unstale":"2024-01-15T18:09:19.609410+0000","last_undegraded":"2024-01-15T18:09:19.609410+0000","last_fullsized":"2024-01-15T18:09:19.609410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:14.028543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683694+0000","last_change":"2024-01-15T18:09:17.744952+0000","last_active":"2024-01-15T18:09:19.683694+0000","last_peered":"2024-01-15T18:09:19.683694+0000","last_clean":"2024-01-15T18:09:19.683694+0000","last_became_active":"2024-01-15T18:09:17.744693+0000","last_became_peered":"2024-01-15T18:09:17.744693+0000","last_unstale":"2024-01-15T18:09:19.683694+0000","last_undegraded":"2024-01-15T18:09:19.683694+0000","last_fullsized":"2024-01-15T18:09:19.683694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:41.918180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683557+0000","last_change":"2024-01-15T18:09:18.617363+0000","last_active":"2024-01-15T18:09:19.683557+0000","last_peered":"2024-01-15T18:09:19.683557+0000","last_clean":"2024-01-15T18:09:19.683557+0000","last_became_active":"2024-01-15T18:09:18.617048+0000","last_became_peered":"2024-01-15T18:09:18.617048+0000","last_unstale":"2024-01-15T18:09:19.683557+0000","last_undegraded":"2024-01-15T18:09:19.683557+0000","last_fullsized":"2024-01-15T18:09:19.683557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:34.918064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685084+0000","last_change":"2024-01-15T18:09:17.734771+0000","last_active":"2024-01-15T18:09:19.685084+0000","last_peered":"2024-01-15T18:09:19.685084+0000","last_clean":"2024-01-15T18:09:19.685084+0000","last_became_active":"2024-01-15T18:09:17.734386+0000","last_became_peered":"2024-01-15T18:09:17.734386+0000","last_unstale":"2024-01-15T18:09:19.685084+0000","last_undegraded":"2024-01-15T18:09:19.685084+0000","last_fullsized":"2024-01-15T18:09:19.685084+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:12.825647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684529+0000","last_change":"2024-01-15T18:09:18.629983+0000","last_active":"2024-01-15T18:09:19.684529+0000","last_peered":"2024-01-15T18:09:19.684529+0000","last_clean":"2024-01-15T18:09:19.684529+0000","last_became_active":"2024-01-15T18:09:18.629688+0000","last_became_peered":"2024-01-15T18:09:18.629688+0000","last_unstale":"2024-01-15T18:09:19.684529+0000","last_undegraded":"2024-01-15T18:09:19.684529+0000","last_fullsized":"2024-01-15T18:09:19.684529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:25:02.032448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683707+0000","last_change":"2024-01-15T18:09:17.740535+0000","last_active":"2024-01-15T18:09:19.683707+0000","last_peered":"2024-01-15T18:09:19.683707+0000","last_clean":"2024-01-15T18:09:19.683707+0000","last_became_active":"2024-01-15T18:09:17.740224+0000","last_became_peered":"2024-01-15T18:09:17.740224+0000","last_unstale":"2024-01-15T18:09:19.683707+0000","last_undegraded":"2024-01-15T18:09:19.683707+0000","last_fullsized":"2024-01-15T18:09:19.683707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:16.633464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683604+0000","last_change":"2024-01-15T18:09:18.617388+0000","last_active":"2024-01-15T18:09:19.683604+0000","last_peered":"2024-01-15T18:09:19.683604+0000","last_clean":"2024-01-15T18:09:19.683604+0000","last_became_active":"2024-01-15T18:09:18.616216+0000","last_became_peered":"2024-01-15T18:09:18.616216+0000","last_unstale":"2024-01-15T18:09:19.683604+0000","last_undegraded":"2024-01-15T18:09:19.683604+0000","last_fullsized":"2024-01-15T18:09:19.683604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:05:58.942405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.755660+0000","last_change":"2024-01-15T18:09:17.736247+0000","last_active":"2024-01-15T18:09:18.755660+0000","last_peered":"2024-01-15T18:09:18.755660+0000","last_clean":"2024-01-15T18:09:18.755660+0000","last_became_active":"2024-01-15T18:09:17.735931+0000","last_became_peered":"2024-01-15T18:09:17.735931+0000","last_unstale":"2024-01-15T18:09:18.755660+0000","last_undegraded":"2024-01-15T18:09:18.755660+0000","last_fullsized":"2024-01-15T18:09:18.755660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:37:37.174033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605619+0000","last_change":"2024-01-15T18:09:18.626316+0000","last_active":"2024-01-15T18:09:19.605619+0000","last_peered":"2024-01-15T18:09:19.605619+0000","last_clean":"2024-01-15T18:09:19.605619+0000","last_became_active":"2024-01-15T18:09:18.625246+0000","last_became_peered":"2024-01-15T18:09:18.625246+0000","last_unstale":"2024-01-15T18:09:19.605619+0000","last_undegraded":"2024-01-15T18:09:19.605619+0000","last_fullsized":"2024-01-15T18:09:19.605619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:17.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685242+0000","last_change":"2024-01-15T18:09:17.741082+0000","last_active":"2024-01-15T18:09:19.685242+0000","last_peered":"2024-01-15T18:09:19.685242+0000","last_clean":"2024-01-15T18:09:19.685242+0000","last_became_active":"2024-01-15T18:09:17.740396+0000","last_became_peered":"2024-01-15T18:09:17.740396+0000","last_unstale":"2024-01-15T18:09:19.685242+0000","last_undegraded":"2024-01-15T18:09:19.685242+0000","last_fullsized":"2024-01-15T18:09:19.685242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:32:43.580096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683534+0000","last_change":"2024-01-15T18:09:18.625338+0000","last_active":"2024-01-15T18:09:19.683534+0000","last_peered":"2024-01-15T18:09:19.683534+0000","last_clean":"2024-01-15T18:09:19.683534+0000","last_became_active":"2024-01-15T18:09:18.625054+0000","last_became_peered":"2024-01-15T18:09:18.625054+0000","last_unstale":"2024-01-15T18:09:19.683534+0000","last_undegraded":"2024-01-15T18:09:19.683534+0000","last_fullsized":"2024-01-15T18:09:19.683534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:38.466718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610100+0000","last_change":"2024-01-15T18:09:17.727366+0000","last_active":"2024-01-15T18:09:19.610100+0000","last_peered":"2024-01-15T18:09:19.610100+0000","last_clean":"2024-01-15T18:09:19.610100+0000","last_became_active":"2024-01-15T18:09:17.727056+0000","last_became_peered":"2024-01-15T18:09:17.727056+0000","last_unstale":"2024-01-15T18:09:19.610100+0000","last_undegraded":"2024-01-15T18:09:19.610100+0000","last_fullsized":"2024-01-15T18:09:19.610100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:54.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.608804+0000","last_change":"2024-01-15T18:09:18.615790+0000","last_active":"2024-01-15T18:09:19.608804+0000","last_peered":"2024-01-15T18:09:19.608804+0000","last_clean":"2024-01-15T18:09:19.608804+0000","last_became_active":"2024-01-15T18:09:18.615300+0000","last_became_peered":"2024-01-15T18:09:18.615300+0000","last_unstale":"2024-01-15T18:09:19.608804+0000","last_undegraded":"2024-01-15T18:09:19.608804+0000","last_fullsized":"2024-01-15T18:09:19.608804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:22:44.116280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605665+0000","last_change":"2024-01-15T18:09:17.734227+0000","last_active":"2024-01-15T18:09:19.605665+0000","last_peered":"2024-01-15T18:09:19.605665+0000","last_clean":"2024-01-15T18:09:19.605665+0000","last_became_active":"2024-01-15T18:09:17.733877+0000","last_became_peered":"2024-01-15T18:09:17.733877+0000","last_unstale":"2024-01-15T18:09:19.605665+0000","last_undegraded":"2024-01-15T18:09:19.605665+0000","last_fullsized":"2024-01-15T18:09:19.605665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:49.227701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683267+0000","last_change":"2024-01-15T18:09:18.626838+0000","last_active":"2024-01-15T18:09:19.683267+0000","last_peered":"2024-01-15T18:09:19.683267+0000","last_clean":"2024-01-15T18:09:19.683267+0000","last_became_active":"2024-01-15T18:09:18.626561+0000","last_became_peered":"2024-01-15T18:09:18.626561+0000","last_unstale":"2024-01-15T18:09:19.683267+0000","last_undegraded":"2024-01-15T18:09:19.683267+0000","last_fullsized":"2024-01-15T18:09:19.683267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:12.189491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683205+0000","last_change":"2024-01-15T18:09:17.737381+0000","last_active":"2024-01-15T18:09:19.683205+0000","last_peered":"2024-01-15T18:09:19.683205+0000","last_clean":"2024-01-15T18:09:19.683205+0000","last_became_active":"2024-01-15T18:09:17.737041+0000","last_became_peered":"2024-01-15T18:09:17.737041+0000","last_unstale":"2024-01-15T18:09:19.683205+0000","last_undegraded":"2024-01-15T18:09:19.683205+0000","last_fullsized":"2024-01-15T18:09:19.683205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:44.219467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683672+0000","last_change":"2024-01-15T18:09:18.620086+0000","last_active":"2024-01-15T18:09:19.683672+0000","last_peered":"2024-01-15T18:09:19.683672+0000","last_clean":"2024-01-15T18:09:19.683672+0000","last_became_active":"2024-01-15T18:09:18.619728+0000","last_became_peered":"2024-01-15T18:09:18.619728+0000","last_unstale":"2024-01-15T18:09:19.683672+0000","last_undegraded":"2024-01-15T18:09:19.683672+0000","last_fullsized":"2024-01-15T18:09:19.683672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:48:51.860359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685121+0000","last_change":"2024-01-15T18:09:17.741156+0000","last_active":"2024-01-15T18:09:19.685121+0000","last_peered":"2024-01-15T18:09:19.685121+0000","last_clean":"2024-01-15T18:09:19.685121+0000","last_became_active":"2024-01-15T18:09:17.740335+0000","last_became_peered":"2024-01-15T18:09:17.740335+0000","last_unstale":"2024-01-15T18:09:19.685121+0000","last_undegraded":"2024-01-15T18:09:19.685121+0000","last_fullsized":"2024-01-15T18:09:19.685121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:02.451959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606289+0000","last_change":"2024-01-15T18:09:18.623472+0000","last_active":"2024-01-15T18:09:19.606289+0000","last_peered":"2024-01-15T18:09:19.606289+0000","last_clean":"2024-01-15T18:09:19.606289+0000","last_became_active":"2024-01-15T18:09:18.623079+0000","last_became_peered":"2024-01-15T18:09:18.623079+0000","last_unstale":"2024-01-15T18:09:19.606289+0000","last_undegraded":"2024-01-15T18:09:19.606289+0000","last_fullsized":"2024-01-15T18:09:19.606289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:13.831851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.751162+0000","last_change":"2024-01-15T18:09:17.733526+0000","last_active":"2024-01-15T18:09:18.751162+0000","last_peered":"2024-01-15T18:09:18.751162+0000","last_clean":"2024-01-15T18:09:18.751162+0000","last_became_active":"2024-01-15T18:09:17.733181+0000","last_became_peered":"2024-01-15T18:09:17.733181+0000","last_unstale":"2024-01-15T18:09:18.751162+0000","last_undegraded":"2024-01-15T18:09:18.751162+0000","last_fullsized":"2024-01-15T18:09:18.751162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:25.765459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685170+0000","last_change":"2024-01-15T18:09:18.623762+0000","last_active":"2024-01-15T18:09:19.685170+0000","last_peered":"2024-01-15T18:09:19.685170+0000","last_clean":"2024-01-15T18:09:19.685170+0000","last_became_active":"2024-01-15T18:09:18.623464+0000","last_became_peered":"2024-01-15T18:09:18.623464+0000","last_unstale":"2024-01-15T18:09:19.685170+0000","last_undegraded":"2024-01-15T18:09:19.685170+0000","last_fullsized":"2024-01-15T18:09:19.685170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:46:18.332990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606053+0000","last_change":"2024-01-15T18:09:17.735004+0000","last_active":"2024-01-15T18:09:19.606053+0000","last_peered":"2024-01-15T18:09:19.606053+0000","last_clean":"2024-01-15T18:09:19.606053+0000","last_became_active":"2024-01-15T18:09:17.732323+0000","last_became_peered":"2024-01-15T18:09:17.732323+0000","last_unstale":"2024-01-15T18:09:19.606053+0000","last_undegraded":"2024-01-15T18:09:19.606053+0000","last_fullsized":"2024-01-15T18:09:19.606053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:07.557596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683658+0000","last_change":"2024-01-15T18:09:18.624732+0000","last_active":"2024-01-15T18:09:19.683658+0000","last_peered":"2024-01-15T18:09:19.683658+0000","last_clean":"2024-01-15T18:09:19.683658+0000","last_became_active":"2024-01-15T18:09:18.624030+0000","last_became_peered":"2024-01-15T18:09:18.624030+0000","last_unstale":"2024-01-15T18:09:19.683658+0000","last_undegraded":"2024-01-15T18:09:19.683658+0000","last_fullsized":"2024-01-15T18:09:19.683658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:03.458163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.610555+0000","last_change":"2024-01-15T18:09:17.741147+0000","last_active":"2024-01-15T18:09:18.610555+0000","last_peered":"2024-01-15T18:09:18.610555+0000","last_clean":"2024-01-15T18:09:18.610555+0000","last_became_active":"2024-01-15T18:09:17.740822+0000","last_became_peered":"2024-01-15T18:09:17.740822+0000","last_unstale":"2024-01-15T18:09:18.610555+0000","last_undegraded":"2024-01-15T18:09:18.610555+0000","last_fullsized":"2024-01-15T18:09:18.610555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:26.648940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606119+0000","last_change":"2024-01-15T18:09:18.619903+0000","last_active":"2024-01-15T18:09:19.606119+0000","last_peered":"2024-01-15T18:09:19.606119+0000","last_clean":"2024-01-15T18:09:19.606119+0000","last_became_active":"2024-01-15T18:09:18.619568+0000","last_became_peered":"2024-01-15T18:09:18.619568+0000","last_unstale":"2024-01-15T18:09:19.606119+0000","last_undegraded":"2024-01-15T18:09:19.606119+0000","last_fullsized":"2024-01-15T18:09:19.606119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:12.189491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685366+0000","last_change":"2024-01-15T18:09:17.738248+0000","last_active":"2024-01-15T18:09:19.685366+0000","last_peered":"2024-01-15T18:09:19.685366+0000","last_clean":"2024-01-15T18:09:19.685366+0000","last_became_active":"2024-01-15T18:09:17.737979+0000","last_became_peered":"2024-01-15T18:09:17.737979+0000","last_unstale":"2024-01-15T18:09:19.685366+0000","last_undegraded":"2024-01-15T18:09:19.685366+0000","last_fullsized":"2024-01-15T18:09:19.685366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.911860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684423+0000","last_change":"2024-01-15T18:09:18.617551+0000","last_active":"2024-01-15T18:09:19.684423+0000","last_peered":"2024-01-15T18:09:19.684423+0000","last_clean":"2024-01-15T18:09:19.684423+0000","last_became_active":"2024-01-15T18:09:18.617224+0000","last_became_peered":"2024-01-15T18:09:18.617224+0000","last_unstale":"2024-01-15T18:09:19.684423+0000","last_undegraded":"2024-01-15T18:09:19.684423+0000","last_fullsized":"2024-01-15T18:09:19.684423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:00:01.967586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684306+0000","last_change":"2024-01-15T18:09:17.726150+0000","last_active":"2024-01-15T18:09:19.684306+0000","last_peered":"2024-01-15T18:09:19.684306+0000","last_clean":"2024-01-15T18:09:19.684306+0000","last_became_active":"2024-01-15T18:09:17.725851+0000","last_became_peered":"2024-01-15T18:09:17.725851+0000","last_unstale":"2024-01-15T18:09:19.684306+0000","last_undegraded":"2024-01-15T18:09:19.684306+0000","last_fullsized":"2024-01-15T18:09:19.684306+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:52.800975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684372+0000","last_change":"2024-01-15T18:09:18.620897+0000","last_active":"2024-01-15T18:09:19.684372+0000","last_peered":"2024-01-15T18:09:19.684372+0000","last_clean":"2024-01-15T18:09:19.684372+0000","last_became_active":"2024-01-15T18:09:18.620599+0000","last_became_peered":"2024-01-15T18:09:18.620599+0000","last_unstale":"2024-01-15T18:09:19.684372+0000","last_undegraded":"2024-01-15T18:09:19.684372+0000","last_fullsized":"2024-01-15T18:09:19.684372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:09.480229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605599+0000","last_change":"2024-01-15T18:09:17.735217+0000","last_active":"2024-01-15T18:09:19.605599+0000","last_peered":"2024-01-15T18:09:19.605599+0000","last_clean":"2024-01-15T18:09:19.605599+0000","last_became_active":"2024-01-15T18:09:17.734896+0000","last_became_peered":"2024-01-15T18:09:17.734896+0000","last_unstale":"2024-01-15T18:09:19.605599+0000","last_undegraded":"2024-01-15T18:09:19.605599+0000","last_fullsized":"2024-01-15T18:09:19.605599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:13.022339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606243+0000","last_change":"2024-01-15T18:09:18.617996+0000","last_active":"2024-01-15T18:09:19.606243+0000","last_peered":"2024-01-15T18:09:19.606243+0000","last_clean":"2024-01-15T18:09:19.606243+0000","last_became_active":"2024-01-15T18:09:18.614900+0000","last_became_peered":"2024-01-15T18:09:18.614900+0000","last_unstale":"2024-01-15T18:09:19.606243+0000","last_undegraded":"2024-01-15T18:09:19.606243+0000","last_fullsized":"2024-01-15T18:09:19.606243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:55.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685792+0000","last_change":"2024-01-15T18:09:17.737201+0000","last_active":"2024-01-15T18:09:19.685792+0000","last_peered":"2024-01-15T18:09:19.685792+0000","last_clean":"2024-01-15T18:09:19.685792+0000","last_became_active":"2024-01-15T18:09:17.736865+0000","last_became_peered":"2024-01-15T18:09:17.736865+0000","last_unstale":"2024-01-15T18:09:19.685792+0000","last_undegraded":"2024-01-15T18:09:19.685792+0000","last_fullsized":"2024-01-15T18:09:19.685792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:38.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606336+0000","last_change":"2024-01-15T18:09:18.620490+0000","last_active":"2024-01-15T18:09:19.606336+0000","last_peered":"2024-01-15T18:09:19.606336+0000","last_clean":"2024-01-15T18:09:19.606336+0000","last_became_active":"2024-01-15T18:09:18.620195+0000","last_became_peered":"2024-01-15T18:09:18.620195+0000","last_unstale":"2024-01-15T18:09:19.606336+0000","last_undegraded":"2024-01-15T18:09:19.606336+0000","last_fullsized":"2024-01-15T18:09:19.606336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:26:00.018458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.610826+0000","last_change":"2024-01-15T18:09:17.731403+0000","last_active":"2024-01-15T18:09:18.610826+0000","last_peered":"2024-01-15T18:09:18.610826+0000","last_clean":"2024-01-15T18:09:18.610826+0000","last_became_active":"2024-01-15T18:09:17.730919+0000","last_became_peered":"2024-01-15T18:09:17.730919+0000","last_unstale":"2024-01-15T18:09:18.610826+0000","last_undegraded":"2024-01-15T18:09:18.610826+0000","last_fullsized":"2024-01-15T18:09:18.610826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:33.617418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684638+0000","last_change":"2024-01-15T18:09:18.627372+0000","last_active":"2024-01-15T18:09:19.684638+0000","last_peered":"2024-01-15T18:09:19.684638+0000","last_clean":"2024-01-15T18:09:19.684638+0000","last_became_active":"2024-01-15T18:09:18.627115+0000","last_became_peered":"2024-01-15T18:09:18.627115+0000","last_unstale":"2024-01-15T18:09:19.684638+0000","last_undegraded":"2024-01-15T18:09:19.684638+0000","last_fullsized":"2024-01-15T18:09:19.684638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:33.164470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610381+0000","last_change":"2024-01-15T18:09:17.722873+0000","last_active":"2024-01-15T18:09:19.610381+0000","last_peered":"2024-01-15T18:09:19.610381+0000","last_clean":"2024-01-15T18:09:19.610381+0000","last_became_active":"2024-01-15T18:09:17.722594+0000","last_became_peered":"2024-01-15T18:09:17.722594+0000","last_unstale":"2024-01-15T18:09:19.610381+0000","last_undegraded":"2024-01-15T18:09:19.610381+0000","last_fullsized":"2024-01-15T18:09:19.610381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:17:23.709145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685548+0000","last_change":"2024-01-15T18:09:18.617938+0000","last_active":"2024-01-15T18:09:19.685548+0000","last_peered":"2024-01-15T18:09:19.685548+0000","last_clean":"2024-01-15T18:09:19.685548+0000","last_became_active":"2024-01-15T18:09:18.617350+0000","last_became_peered":"2024-01-15T18:09:18.617350+0000","last_unstale":"2024-01-15T18:09:19.685548+0000","last_undegraded":"2024-01-15T18:09:19.685548+0000","last_fullsized":"2024-01-15T18:09:19.685548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:32:44.586300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606022+0000","last_change":"2024-01-15T18:09:17.728443+0000","last_active":"2024-01-15T18:09:19.606022+0000","last_peered":"2024-01-15T18:09:19.606022+0000","last_clean":"2024-01-15T18:09:19.606022+0000","last_became_active":"2024-01-15T18:09:17.728142+0000","last_became_peered":"2024-01-15T18:09:17.728142+0000","last_unstale":"2024-01-15T18:09:19.606022+0000","last_undegraded":"2024-01-15T18:09:19.606022+0000","last_fullsized":"2024-01-15T18:09:19.606022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:18.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685124+0000","last_change":"2024-01-15T18:09:18.615608+0000","last_active":"2024-01-15T18:09:19.685124+0000","last_peered":"2024-01-15T18:09:19.685124+0000","last_clean":"2024-01-15T18:09:19.685124+0000","last_became_active":"2024-01-15T18:09:18.615351+0000","last_became_peered":"2024-01-15T18:09:18.615351+0000","last_unstale":"2024-01-15T18:09:19.685124+0000","last_undegraded":"2024-01-15T18:09:19.685124+0000","last_fullsized":"2024-01-15T18:09:19.685124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:46:15.783337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684359+0000","last_change":"2024-01-15T18:09:17.726328+0000","last_active":"2024-01-15T18:09:19.684359+0000","last_peered":"2024-01-15T18:09:19.684359+0000","last_clean":"2024-01-15T18:09:19.684359+0000","last_became_active":"2024-01-15T18:09:17.725649+0000","last_became_peered":"2024-01-15T18:09:17.725649+0000","last_unstale":"2024-01-15T18:09:19.684359+0000","last_undegraded":"2024-01-15T18:09:19.684359+0000","last_fullsized":"2024-01-15T18:09:19.684359+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:46:07.566371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683534+0000","last_change":"2024-01-15T18:09:18.624775+0000","last_active":"2024-01-15T18:09:19.683534+0000","last_peered":"2024-01-15T18:09:19.683534+0000","last_clean":"2024-01-15T18:09:19.683534+0000","last_became_active":"2024-01-15T18:09:18.622596+0000","last_became_peered":"2024-01-15T18:09:18.622596+0000","last_unstale":"2024-01-15T18:09:19.683534+0000","last_undegraded":"2024-01-15T18:09:19.683534+0000","last_fullsized":"2024-01-15T18:09:19.683534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:26:00.018458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.604585+0000","last_change":"2024-01-15T18:09:17.741174+0000","last_active":"2024-01-15T18:09:18.604585+0000","last_peered":"2024-01-15T18:09:18.604585+0000","last_clean":"2024-01-15T18:09:18.604585+0000","last_became_active":"2024-01-15T18:09:17.740803+0000","last_became_peered":"2024-01-15T18:09:17.740803+0000","last_unstale":"2024-01-15T18:09:18.604585+0000","last_undegraded":"2024-01-15T18:09:18.604585+0000","last_fullsized":"2024-01-15T18:09:18.604585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:07.525089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683599+0000","last_change":"2024-01-15T18:09:18.617741+0000","last_active":"2024-01-15T18:09:19.683599+0000","last_peered":"2024-01-15T18:09:19.683599+0000","last_clean":"2024-01-15T18:09:19.683599+0000","last_became_active":"2024-01-15T18:09:18.616101+0000","last_became_peered":"2024-01-15T18:09:18.616101+0000","last_unstale":"2024-01-15T18:09:19.683599+0000","last_undegraded":"2024-01-15T18:09:19.683599+0000","last_fullsized":"2024-01-15T18:09:19.683599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:10:08.157767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684115+0000","last_change":"2024-01-15T18:09:17.741772+0000","last_active":"2024-01-15T18:09:19.684115+0000","last_peered":"2024-01-15T18:09:19.684115+0000","last_clean":"2024-01-15T18:09:19.684115+0000","last_became_active":"2024-01-15T18:09:17.741480+0000","last_became_peered":"2024-01-15T18:09:17.741480+0000","last_unstale":"2024-01-15T18:09:19.684115+0000","last_undegraded":"2024-01-15T18:09:19.684115+0000","last_fullsized":"2024-01-15T18:09:19.684115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:23.666845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684732+0000","last_change":"2024-01-15T18:09:18.618337+0000","last_active":"2024-01-15T18:09:19.684732+0000","last_peered":"2024-01-15T18:09:19.684732+0000","last_clean":"2024-01-15T18:09:19.684732+0000","last_became_active":"2024-01-15T18:09:18.618060+0000","last_became_peered":"2024-01-15T18:09:18.618060+0000","last_unstale":"2024-01-15T18:09:19.684732+0000","last_undegraded":"2024-01-15T18:09:19.684732+0000","last_fullsized":"2024-01-15T18:09:19.684732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:58:35.952158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683856+0000","last_change":"2024-01-15T18:09:17.739100+0000","last_active":"2024-01-15T18:09:19.683856+0000","last_peered":"2024-01-15T18:09:19.683856+0000","last_clean":"2024-01-15T18:09:19.683856+0000","last_became_active":"2024-01-15T18:09:17.736485+0000","last_became_peered":"2024-01-15T18:09:17.736485+0000","last_unstale":"2024-01-15T18:09:19.683856+0000","last_undegraded":"2024-01-15T18:09:19.683856+0000","last_fullsized":"2024-01-15T18:09:19.683856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:33.617418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683726+0000","last_change":"2024-01-15T18:09:18.621681+0000","last_active":"2024-01-15T18:09:19.683726+0000","last_peered":"2024-01-15T18:09:19.683726+0000","last_clean":"2024-01-15T18:09:19.683726+0000","last_became_active":"2024-01-15T18:09:18.621282+0000","last_became_peered":"2024-01-15T18:09:18.621282+0000","last_unstale":"2024-01-15T18:09:19.683726+0000","last_undegraded":"2024-01-15T18:09:19.683726+0000","last_fullsized":"2024-01-15T18:09:19.683726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:51.870805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610113+0000","last_change":"2024-01-15T18:09:17.727124+0000","last_active":"2024-01-15T18:09:19.610113+0000","last_peered":"2024-01-15T18:09:19.610113+0000","last_clean":"2024-01-15T18:09:19.610113+0000","last_became_active":"2024-01-15T18:09:17.726619+0000","last_became_peered":"2024-01-15T18:09:17.726619+0000","last_unstale":"2024-01-15T18:09:19.610113+0000","last_undegraded":"2024-01-15T18:09:19.610113+0000","last_fullsized":"2024-01-15T18:09:19.610113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:18:21.037790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":198,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.607140+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:09:18.607140+0000","last_peered":"2024-01-15T18:09:18.607140+0000","last_clean":"2024-01-15T18:09:18.607140+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:09:18.607140+0000","last_undegraded":"2024-01-15T18:09:18.607140+0000","last_fullsized":"2024-01-15T18:09:18.607140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1323008,"data_stored":893404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":0.73799999999999999,"15min":0.68000000000000005},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.53,"5min":0.85499999999999998,"15min":0.77600000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":13.744,"5min":13.744,"15min":13.744},"last":0.623}]},{"osd":1,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":0.83399999999999996,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.625,"5min":6.625,"15min":6.625},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.24,"5min":0.84099999999999997,"15min":0.78300000000000003},"min":{"1min":0.53900000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.5090000000000003,"5min":4.5090000000000003,"15min":4.5090000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":2.8580000000000001,"5min":1.1859999999999999,"15min":0.94499999999999995},"min":{"1min":0.48699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":39.695,"5min":39.695,"15min":39.695},"last":0.78200000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":2.2959999999999998,"5min":1.0549999999999999,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.437,"15min":0.437},"max":{"1min":26.463999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.4039999999999999,"5min":0.89500000000000002,"15min":0.84499999999999997},"min":{"1min":0.55600000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":10.09,"5min":10.09,"15min":10.09},"last":0.64100000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.62,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.71099999999999997,"15min":0.73699999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.7410000000000001,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.51400000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67100000000000004,"15min":0.73699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.69399999999999995,"15min":0.748},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.76100000000000001,"15min":0.80600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93300000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71099999999999997,"15min":0.75900000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":720896,"data_stored":302876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0150000000000001,"5min":1.0209999999999999,"15min":0.75800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":22.754000000000001,"5min":22.754000000000001,"15min":22.754000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.708,"5min":1.0509999999999999,"15min":0.76500000000000001},"min":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":9.1470000000000002,"5min":12.449,"15min":12.449},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.99399999999999999,"15min":0.752},"min":{"1min":0.45800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":14.246,"5min":14.246,"15min":14.246},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.609,"5min":0.94799999999999995,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":8.7590000000000003,"5min":8.7590000000000003,"15min":8.7590000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.883,"5min":1.4299999999999999,"15min":0.95699999999999996},"min":{"1min":0.46600000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":42.825000000000003,"5min":42.825000000000003,"15min":42.825000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":3.9569999999999999,"5min":1.544,"15min":0.95399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":42.851999999999997,"5min":42.851999999999997,"15min":42.851999999999997},"last":0.80800000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2959999999999998,"5min":1.179,"15min":0.85899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":24.199000000000002,"5min":24.199000000000002,"15min":24.199000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.5149999999999999,"5min":0.95799999999999996,"15min":0.79500000000000004},"min":{"1min":0.49399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":12.225,"5min":12.225,"15min":12.225},"last":0.82399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":7.1609999999999996,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.442},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.92300000000000004,"15min":0.77200000000000002},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.3059999999999992,"5min":12.491,"15min":12.491},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.81100000000000005,"15min":0.77600000000000002},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":9.4819999999999993,"15min":9.4819999999999993},"last":0.621},{"interface":"front","average":{"1min":1.401,"5min":1.1559999999999999,"15min":0.92900000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.945,"5min":12.945,"15min":12.945},"last":0.60099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.29,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.79200000000000004,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":303528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.66400000000000003,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.754},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.71799999999999997,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.2349999999999999,"5min":4.093,"15min":4.093},"last":0.85699999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.78700000000000003,"15min":0.71199999999999997},"min":{"1min":0.56599999999999995,"5min":0.443,"15min":0.443},"max":{"1min":5.2089999999999996,"5min":5.2089999999999996,"15min":5.2089999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":0.82999999999999996,"15min":0.73799999999999999},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.91700000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.76400000000000001,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.052,"5min":0.76600000000000001,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.434,"15min":0.434},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.873}]},{"osd":3,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.80600000000000005,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.8029999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.76500000000000001,"15min":0.70499999999999996},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.2190000000000003,"5min":4.2190000000000003,"15min":4.2190000000000003},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73599999999999999,"15min":0.70399999999999996},"min":{"1min":0.36699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":11.093,"5min":11.093,"15min":11.093},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.8080000000000001,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.65800000000000003,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76300000000000001,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.51100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.696,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.71599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":733184,"data_stored":304488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":2.79,"5min":1.0760000000000001,"15min":0.76400000000000001},"min":{"1min":0.38300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":34.802,"5min":34.802,"15min":34.802},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.8720000000000001,"5min":0.93100000000000005,"15min":0.72199999999999998},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":22.335999999999999,"5min":22.335999999999999,"15min":22.335999999999999},"last":0.996}]},{"osd":1,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":3.375,"5min":1.2649999999999999,"15min":0.90200000000000002},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":38.198,"5min":38.198,"15min":38.198},"last":0.89200000000000002},{"interface":"front","average":{"1min":3.3839999999999999,"5min":1.2569999999999999,"15min":0.877},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":38.220999999999997,"5min":38.220999999999997,"15min":38.220999999999997},"last":0.876}]},{"osd":2,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":0.81999999999999995,"15min":0.74399999999999999},"min":{"1min":0.63800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.1600000000000001,"5min":6.1600000000000001,"15min":6.1600000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.258,"5min":0.82899999999999996,"15min":0.76700000000000002},"min":{"1min":0.53200000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":0.89800000000000002,"15min":0.748},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.0549999999999997,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":0.86499999999999999,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":8.5399999999999991,"5min":8.5399999999999991,"15min":8.5399999999999991},"last":0.64500000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.76100000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.355,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":0.95599999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":10.284000000000001,"5min":10.284000000000001,"15min":10.284000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.92800000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81000000000000005,"15min":0.75600000000000001},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.4929999999999999,"5min":11.457000000000001,"15min":11.457000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":729088,"data_stored":302932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.958,"5min":0.82099999999999995,"15min":0.56899999999999995},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":25.771999999999998,"5min":25.771999999999998,"15min":25.771999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":2.6150000000000002,"5min":0.93899999999999995,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.246,"15min":0.246},"max":{"1min":28.332000000000001,"5min":28.332000000000001,"15min":28.332000000000001},"last":0.29499999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":2.9009999999999998,"5min":0.98699999999999999,"15min":0.61299999999999999},"min":{"1min":0.32100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":40.457999999999998,"5min":40.457999999999998,"15min":40.457999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":4.327,"5min":1.3740000000000001,"15min":0.81599999999999995},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":27.427,"5min":27.427,"15min":27.427},"last":0.36299999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.972,"5min":0.84699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.6319999999999997,"5min":8.6319999999999997,"15min":8.6319999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.7849999999999999,"5min":0.875,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.538,"5min":12.538,"15min":12.538},"last":0.42699999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":0.93999999999999995,"15min":0.79500000000000004},"min":{"1min":0.48899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":8.4339999999999993,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.353,"5min":0.88300000000000001,"15min":0.79300000000000004},"min":{"1min":0.503,"5min":0.377,"15min":0.377},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.627}]},{"osd":5,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":0.92000000000000004,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.5030000000000001,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.966,"5min":1.0580000000000001,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":8.5410000000000004,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":0.95799999999999996,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.3120000000000001,"5min":0.96699999999999997,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":4.1150000000000002,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":1.016,"15min":0.83799999999999997},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":8.4990000000000006,"5min":8.4990000000000006,"15min":8.4990000000000006},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.7150000000000001,"5min":0.997,"15min":0.86099999999999999},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":7.0410000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":302932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":5.0800000000000001,"5min":1.4490000000000001,"15min":0.77700000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":29.527999999999999,"5min":29.527999999999999,"15min":29.527999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":6.4829999999999997,"5min":1.762,"15min":0.92800000000000005},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":34.122999999999998,"5min":34.122999999999998,"15min":34.122999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":11.613,"5min":2.8079999999999998,"15min":1.2589999999999999},"min":{"1min":0.36199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":113.149,"5min":113.149,"15min":113.149},"last":0.51900000000000002},{"interface":"front","average":{"1min":4.931,"5min":1.421,"15min":0.79900000000000004},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":41.302,"5min":41.302,"15min":41.302},"last":0.495}]},{"osd":3,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":5.3769999999999998,"5min":1.502,"15min":0.79900000000000004},"min":{"1min":0.44600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":46.960000000000001,"5min":46.960000000000001,"15min":46.960000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":9.0719999999999992,"5min":2.3010000000000002,"15min":1.1160000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":115.693,"5min":115.693,"15min":115.693},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":4.9199999999999999,"5min":1.6919999999999999,"15min":1.0920000000000001},"min":{"1min":0.59499999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":24.366,"5min":24.366,"15min":24.366},"last":0.74199999999999999},{"interface":"front","average":{"1min":4.3840000000000003,"5min":1.506,"15min":0.98399999999999999},"min":{"1min":0.61099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":20.489000000000001,"5min":20.489000000000001,"15min":20.489000000000001},"last":0.625}]},{"osd":5,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":4.3129999999999997,"5min":1.486,"15min":0.97199999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":29.645,"5min":29.645,"15min":29.645},"last":0.64500000000000002},{"interface":"front","average":{"1min":4.577,"5min":1.6020000000000001,"15min":1.075},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":24.277999999999999,"5min":24.277999999999999,"15min":24.277999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":4.5819999999999999,"5min":1.569,"15min":1.05},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":20.370999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":4.3540000000000001,"5min":1.538,"15min":1.04},"min":{"1min":0.64500000000000002,"5min":0.505,"15min":0.505},"max":{"1min":29.690000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":3.6520000000000001,"5min":1.421,"15min":1.0289999999999999},"min":{"1min":0.53100000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":20.446999999999999,"5min":20.446999999999999,"15min":20.446999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":4.556,"5min":1.637,"15min":1.123},"min":{"1min":0.57699999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":29.785,"5min":29.785,"15min":29.785},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":724992,"data_stored":302966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.5489999999999999,"5min":1.1699999999999999,"15min":0.84999999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":39.948,"5min":39.948,"15min":39.948},"last":0.30399999999999999},{"interface":"front","average":{"1min":2.8610000000000002,"5min":1.1000000000000001,"15min":0.86799999999999999},"min":{"1min":0.36799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":24.824000000000002,"5min":24.824000000000002,"15min":24.824000000000002},"last":0.32400000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.79,"5min":1.224,"15min":0.88200000000000001},"min":{"1min":0.30099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":49.229999999999997,"5min":49.229999999999997,"15min":49.229999999999997},"last":0.375},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":10.349,"5min":10.349,"15min":10.349},"last":0.40699999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.9140000000000001,"5min":1.298,"15min":0.93799999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":25.427,"5min":25.427,"15min":25.427},"last":0.36699999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.879,"15min":0.86699999999999999},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.7949999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2240000000000002,"5min":1.1759999999999999,"15min":1.0780000000000001},"min":{"1min":0.46200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":18.814,"5min":18.814,"15min":18.814},"last":0.495},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94699999999999995,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9259999999999999,"5min":12.997,"15min":12.997},"last":0.54800000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.1360000000000001,"5min":1.1160000000000001,"15min":0.97399999999999998},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":18.736999999999998,"5min":18.736999999999998,"15min":18.736999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0309999999999999,"15min":1.085},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":2.3180000000000001,"5min":13.054,"15min":13.054},"last":0.626}]},{"osd":6,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2,"5min":1.105,"15min":1.0289999999999999},"min":{"1min":0.51300000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":18.777000000000001,"5min":18.777000000000001,"15min":18.777000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.0509999999999999,"15min":1.044},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":11.712999999999999,"5min":11.712999999999999,"15min":11.712999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":11.800000000000001,"5min":11.800000000000001,"15min":11.800000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.60799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.4159999999999999,"5min":13.089,"15min":13.089},"last":0.65100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36640,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335200,"statfs":{"total":96636764160,"available":96599244800,"internally_reserved":0,"allocated":1331200,"data_stored":895438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.7349999999999999,"5min":1.2869999999999999,"15min":1.0169999999999999},"min":{"1min":0.33600000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":31.977,"5min":31.977,"15min":31.977},"last":0.374},{"interface":"front","average":{"1min":3.694,"5min":1.1220000000000001,"15min":0.68600000000000005},"min":{"1min":0.39000000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":36.145000000000003,"5min":36.145000000000003,"15min":36.145000000000003},"last":0.44900000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.234,"5min":1.0549999999999999,"15min":0.68799999999999994},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":41.387999999999998,"5min":41.387999999999998,"15min":41.387999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":3.855,"5min":1.232,"15min":0.85599999999999998},"min":{"1min":0.33400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":33.219999999999999,"5min":33.219999999999999,"15min":33.219999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.9350000000000001,"5min":1.1639999999999999,"15min":1.018},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":31.943999999999999,"5min":31.943999999999999,"15min":31.943999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.984,"5min":0.84199999999999997,"15min":0.65600000000000003},"min":{"1min":0.495,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":9.5640000000000001,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.6309999999999998,"5min":1.1679999999999999,"15min":1.0129999999999999},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":32.932000000000002,"5min":32.932000000000002,"15min":32.932000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":3.0470000000000002,"5min":1.2689999999999999,"15min":1.0660000000000001},"min":{"1min":0.56699999999999995,"5min":0.44,"15min":0.44},"max":{"1min":31.782,"5min":31.782,"15min":31.782},"last":0.93899999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.1259999999999999,"5min":1.3160000000000001,"15min":1.1000000000000001},"min":{"1min":0.61699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":31.858000000000001,"5min":31.858000000000001,"15min":31.858000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":2.2240000000000002,"5min":1.111,"15min":1.0189999999999999},"min":{"1min":0.51400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":33.151000000000003,"5min":33.151000000000003,"15min":33.151000000000003},"last":0.97499999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8799999999999999,"5min":0.999,"15min":0.88300000000000001},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.6440000000000001,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":2.242,"5min":1.1259999999999999,"15min":1.036},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":33.098999999999997,"5min":33.098999999999997,"15min":33.098999999999997},"last":0.92500000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.837,"5min":0.97499999999999998,"15min":0.85299999999999998},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.501},"max":{"1min":9.6910000000000007,"5min":9.6910000000000007,"15min":9.6910000000000007},"last":0.95799999999999996},{"interface":"front","average":{"1min":3.044,"5min":1.3009999999999999,"15min":1.115},"min":{"1min":0.60699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":31.905999999999999,"5min":31.905999999999999,"15min":31.905999999999999},"last":0.83999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:09:22.813 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:09:22.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.978+0000 7f452cc00640 1 Processor -- start 2024-01-15T18:09:22.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.978+0000 7f452cc00640 1 -- start start 2024-01-15T18:09:22.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f452cc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4528071bb0 0x7f4528072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f452cc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45280ceb50 con 0x7f4528143780 2024-01-15T18:09:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f452cc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452815c110 con 0x7f452815c5a0 2024-01-15T18:09:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f452cc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4528072570 con 0x7f4528071bb0 2024-01-15T18:09:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4525400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4528071bb0 0x7f4528072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4525e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4528143780 0x7f45280cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44900/0 (socket says 172.21.15.143:44900) 2024-01-15T18:09:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4525e00640 1 -- 172.21.15.143:0/4244358919 learned_addr learned my addr 172.21.15.143:0/4244358919 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4525400640 1 -- 172.21.15.143:0/4244358919 >> v1:172.21.15.204:6789/0 conn(0x7f452815c5a0 legacy=0x7f452815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4525400640 1 -- 172.21.15.143:0/4244358919 >> v1:172.21.15.143:6789/0 conn(0x7f4528143780 legacy=0x7f45280cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4524a00640 1 -- 172.21.15.143:0/4244358919 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697383303 0 0) 0x7f4518001120 con 0x7f4528143780 2024-01-15T18:09:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.982+0000 7f4525400640 1 -- 172.21.15.143:0/4244358919 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452815b7e0 con 0x7f4528071bb0 2024-01-15T18:09:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f4525400640 1 --2- 172.21.15.143:0/4244358919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528071bb0 0x7f4528072030 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f45100097c0 tx=0x7f451001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4723eae9f1d36d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f4524a00640 1 -- 172.21.15.143:0/4244358919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100ad680 con 0x7f4528071bb0 2024-01-15T18:09:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f4524a00640 1 -- 172.21.15.143:0/4244358919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45100ad7e0 con 0x7f4528071bb0 2024-01-15T18:09:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f4524a00640 1 -- 172.21.15.143:0/4244358919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100adb70 con 0x7f4528071bb0 2024-01-15T18:09:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 -- 172.21.15.143:0/4244358919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528071bb0 msgr2=0x7f4528072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 --2- 172.21.15.143:0/4244358919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528071bb0 0x7f4528072030 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f45100097c0 tx=0x7f451001c480 comp rx=0 tx=0).stop 2024-01-15T18:09:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 -- 172.21.15.143:0/4244358919 shutdown_connections 2024-01-15T18:09:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 --2- 172.21.15.143:0/4244358919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528071bb0 0x7f4528072030 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 -- 172.21.15.143:0/4244358919 >> 172.21.15.143:0/4244358919 conn(0x7f452806c3f0 msgr2=0x7f452806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 -- 172.21.15.143:0/4244358919 shutdown_connections 2024-01-15T18:09:22.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 -- 172.21.15.143:0/4244358919 wait complete. 2024-01-15T18:09:22.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 Processor -- start 2024-01-15T18:09:22.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.986+0000 7f452cc00640 1 -- start start 2024-01-15T18:09:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4528071bb0 0x7f4528165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 0x7f452815da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f452815c5a0 0x7f452815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528159b70 con 0x7f4528143780 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4528146db0 con 0x7f452815c5a0 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4528147090 con 0x7f4528071bb0 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 0x7f452815da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 0x7f452815da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42344/0 (socket says 172.21.15.143:42344) 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 -- 172.21.15.143:0/2508858131 learned_addr learned my addr 172.21.15.143:0/2508858131 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4526800640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f452815c5a0 0x7f452815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525e00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4528071bb0 0x7f4528165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 -- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4528071bb0 msgr2=0x7f4528165300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4528071bb0 0x7f4528165300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 -- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f452815c5a0 msgr2=0x7f452815df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f452815c5a0 0x7f452815df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 -- 172.21.15.143:0/2508858131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4510009420 con 0x7f4528143780 2024-01-15T18:09:22.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525e00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4528071bb0 0x7f4528165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f4525400640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 0x7f452815da00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4510019090 tx=0x7f4510019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100add60 con 0x7f4528143780 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45100b6410 con 0x7f4528143780 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45100b67a0 con 0x7f4528143780 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4528147370 con 0x7f4528143780 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4528165840 con 0x7f4528143780 2024-01-15T18:09:22.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4528147b70 con 0x7f4528143780 2024-01-15T18:09:22.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.990+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f45100b4040 con 0x7f4528143780 2024-01-15T18:09:22.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.994+0000 7f451e000640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4500066e90 0x7f4500069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.994+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f451011f1d0 con 0x7f4528143780 2024-01-15T18:09:22.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.994+0000 7f4525e00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4500066e90 0x7f4500069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.994+0000 7f4525e00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4500066e90 0x7f4500069350 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4518007910 tx=0x7f4518007ae0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:22.998+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4528147b70 con 0x7f4528143780 2024-01-15T18:09:23.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.142+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4528167ad0 con 0x7f4500066e90 2024-01-15T18:09:23.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.150+0000 7f451e000640 1 -- 172.21.15.143:0/2508858131 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287990 (secure 0 0 0) 0x7f45180075e0 con 0x7f4500066e90 2024-01-15T18:09:23.158 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:23.159 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:09:23.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4500066e90 msgr2=0x7f4500069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4500066e90 0x7f4500069350 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4518007910 tx=0x7f4518007ae0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 msgr2=0x7f452815da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 0x7f452815da00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4510019090 tx=0x7f4510019220 comp rx=0 tx=0).stop 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 shutdown_connections 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4500066e90 0x7f4500069350 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f452815c5a0 0x7f452815df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4528143780 0x7f452815da00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 --2- 172.21.15.143:0/2508858131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4528071bb0 0x7f4528165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 >> 172.21.15.143:0/2508858131 conn(0x7f452806c3f0 msgr2=0x7f45280b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 shutdown_connections 2024-01-15T18:09:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.154+0000 7f452cc00640 1 -- 172.21.15.143:0/2508858131 wait complete. 2024-01-15T18:09:23.178 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-01-15T18:09:22.696589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289596,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":282611,"kb_avail":754685124,"statfs":{"total":773094113280,"available":772797566976,"internally_reserved":0,"allocated":7020544,"data_stored":3608564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":289394224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991199"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683525+0000","last_change":"2024-01-15T18:09:18.622882+0000","last_active":"2024-01-15T18:09:19.683525+0000","last_peered":"2024-01-15T18:09:19.683525+0000","last_clean":"2024-01-15T18:09:19.683525+0000","last_became_active":"2024-01-15T18:09:18.622573+0000","last_became_peered":"2024-01-15T18:09:18.622573+0000","last_unstale":"2024-01-15T18:09:19.683525+0000","last_undegraded":"2024-01-15T18:09:19.683525+0000","last_fullsized":"2024-01-15T18:09:19.683525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:50.157284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685013+0000","last_change":"2024-01-15T18:09:18.618943+0000","last_active":"2024-01-15T18:09:19.685013+0000","last_peered":"2024-01-15T18:09:19.685013+0000","last_clean":"2024-01-15T18:09:19.685013+0000","last_became_active":"2024-01-15T18:09:18.618637+0000","last_became_peered":"2024-01-15T18:09:18.618637+0000","last_unstale":"2024-01-15T18:09:19.685013+0000","last_undegraded":"2024-01-15T18:09:19.685013+0000","last_fullsized":"2024-01-15T18:09:19.685013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:45.803655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683517+0000","last_change":"2024-01-15T18:09:18.624968+0000","last_active":"2024-01-15T18:09:19.683517+0000","last_peered":"2024-01-15T18:09:19.683517+0000","last_clean":"2024-01-15T18:09:19.683517+0000","last_became_active":"2024-01-15T18:09:18.624432+0000","last_became_peered":"2024-01-15T18:09:18.624432+0000","last_unstale":"2024-01-15T18:09:19.683517+0000","last_undegraded":"2024-01-15T18:09:19.683517+0000","last_fullsized":"2024-01-15T18:09:19.683517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:13.831851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605823+0000","last_change":"2024-01-15T18:09:18.628270+0000","last_active":"2024-01-15T18:09:19.605823+0000","last_peered":"2024-01-15T18:09:19.605823+0000","last_clean":"2024-01-15T18:09:19.605823+0000","last_became_active":"2024-01-15T18:09:18.627968+0000","last_became_peered":"2024-01-15T18:09:18.627968+0000","last_unstale":"2024-01-15T18:09:19.605823+0000","last_undegraded":"2024-01-15T18:09:19.605823+0000","last_fullsized":"2024-01-15T18:09:19.605823+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:09.480229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683476+0000","last_change":"2024-01-15T18:09:18.623667+0000","last_active":"2024-01-15T18:09:19.683476+0000","last_peered":"2024-01-15T18:09:19.683476+0000","last_clean":"2024-01-15T18:09:19.683476+0000","last_became_active":"2024-01-15T18:09:18.623377+0000","last_became_peered":"2024-01-15T18:09:18.623377+0000","last_unstale":"2024-01-15T18:09:19.683476+0000","last_undegraded":"2024-01-15T18:09:19.683476+0000","last_fullsized":"2024-01-15T18:09:19.683476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:58:59.747220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684671+0000","last_change":"2024-01-15T18:09:18.620077+0000","last_active":"2024-01-15T18:09:19.684671+0000","last_peered":"2024-01-15T18:09:19.684671+0000","last_clean":"2024-01-15T18:09:19.684671+0000","last_became_active":"2024-01-15T18:09:18.619368+0000","last_became_peered":"2024-01-15T18:09:18.619368+0000","last_unstale":"2024-01-15T18:09:19.684671+0000","last_undegraded":"2024-01-15T18:09:19.684671+0000","last_fullsized":"2024-01-15T18:09:19.684671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:12.189491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.608844+0000","last_change":"2024-01-15T18:09:18.618524+0000","last_active":"2024-01-15T18:09:19.608844+0000","last_peered":"2024-01-15T18:09:19.608844+0000","last_clean":"2024-01-15T18:09:19.608844+0000","last_became_active":"2024-01-15T18:09:18.618234+0000","last_became_peered":"2024-01-15T18:09:18.618234+0000","last_unstale":"2024-01-15T18:09:19.608844+0000","last_undegraded":"2024-01-15T18:09:19.608844+0000","last_fullsized":"2024-01-15T18:09:19.608844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:08.563799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683604+0000","last_change":"2024-01-15T18:09:18.615739+0000","last_active":"2024-01-15T18:09:19.683604+0000","last_peered":"2024-01-15T18:09:19.683604+0000","last_clean":"2024-01-15T18:09:19.683604+0000","last_became_active":"2024-01-15T18:09:18.615483+0000","last_became_peered":"2024-01-15T18:09:18.615483+0000","last_unstale":"2024-01-15T18:09:19.683604+0000","last_undegraded":"2024-01-15T18:09:19.683604+0000","last_fullsized":"2024-01-15T18:09:19.683604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:48:51.860359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.609446+0000","last_change":"2024-01-15T18:09:18.622350+0000","last_active":"2024-01-15T18:09:19.609446+0000","last_peered":"2024-01-15T18:09:19.609446+0000","last_clean":"2024-01-15T18:09:19.609446+0000","last_became_active":"2024-01-15T18:09:18.620475+0000","last_became_peered":"2024-01-15T18:09:18.620475+0000","last_unstale":"2024-01-15T18:09:19.609446+0000","last_undegraded":"2024-01-15T18:09:19.609446+0000","last_fullsized":"2024-01-15T18:09:19.609446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:46.800650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685817+0000","last_change":"2024-01-15T18:09:18.616969+0000","last_active":"2024-01-15T18:09:19.685817+0000","last_peered":"2024-01-15T18:09:19.685817+0000","last_clean":"2024-01-15T18:09:19.685817+0000","last_became_active":"2024-01-15T18:09:18.616088+0000","last_became_peered":"2024-01-15T18:09:18.616088+0000","last_unstale":"2024-01-15T18:09:19.685817+0000","last_undegraded":"2024-01-15T18:09:19.685817+0000","last_fullsized":"2024-01-15T18:09:19.685817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:29.568816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685635+0000","last_change":"2024-01-15T18:09:18.625131+0000","last_active":"2024-01-15T18:09:19.685635+0000","last_peered":"2024-01-15T18:09:19.685635+0000","last_clean":"2024-01-15T18:09:19.685635+0000","last_became_active":"2024-01-15T18:09:18.624838+0000","last_became_peered":"2024-01-15T18:09:18.624838+0000","last_unstale":"2024-01-15T18:09:19.685635+0000","last_undegraded":"2024-01-15T18:09:19.685635+0000","last_fullsized":"2024-01-15T18:09:19.685635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:11:03.527003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605868+0000","last_change":"2024-01-15T18:09:18.629407+0000","last_active":"2024-01-15T18:09:19.605868+0000","last_peered":"2024-01-15T18:09:19.605868+0000","last_clean":"2024-01-15T18:09:19.605868+0000","last_became_active":"2024-01-15T18:09:18.628281+0000","last_became_peered":"2024-01-15T18:09:18.628281+0000","last_unstale":"2024-01-15T18:09:19.605868+0000","last_undegraded":"2024-01-15T18:09:19.605868+0000","last_fullsized":"2024-01-15T18:09:19.605868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:00:01.967586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610308+0000","last_change":"2024-01-15T18:09:18.620730+0000","last_active":"2024-01-15T18:09:19.610308+0000","last_peered":"2024-01-15T18:09:19.610308+0000","last_clean":"2024-01-15T18:09:19.610308+0000","last_became_active":"2024-01-15T18:09:18.620225+0000","last_became_peered":"2024-01-15T18:09:18.620225+0000","last_unstale":"2024-01-15T18:09:19.610308+0000","last_undegraded":"2024-01-15T18:09:19.610308+0000","last_fullsized":"2024-01-15T18:09:19.610308+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:53.807179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.609092+0000","last_change":"2024-01-15T18:09:18.614511+0000","last_active":"2024-01-15T18:09:19.609092+0000","last_peered":"2024-01-15T18:09:19.609092+0000","last_clean":"2024-01-15T18:09:19.609092+0000","last_became_active":"2024-01-15T18:09:18.614217+0000","last_became_peered":"2024-01-15T18:09:18.614217+0000","last_unstale":"2024-01-15T18:09:19.609092+0000","last_undegraded":"2024-01-15T18:09:19.609092+0000","last_fullsized":"2024-01-15T18:09:19.609092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:50.233905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.608934+0000","last_change":"2024-01-15T18:09:18.620903+0000","last_active":"2024-01-15T18:09:19.608934+0000","last_peered":"2024-01-15T18:09:19.608934+0000","last_clean":"2024-01-15T18:09:19.608934+0000","last_became_active":"2024-01-15T18:09:18.616572+0000","last_became_peered":"2024-01-15T18:09:18.616572+0000","last_unstale":"2024-01-15T18:09:19.608934+0000","last_undegraded":"2024-01-15T18:09:19.608934+0000","last_fullsized":"2024-01-15T18:09:19.608934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:39.789871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683900+0000","last_change":"2024-01-15T18:09:18.619610+0000","last_active":"2024-01-15T18:09:19.683900+0000","last_peered":"2024-01-15T18:09:19.683900+0000","last_clean":"2024-01-15T18:09:19.683900+0000","last_became_active":"2024-01-15T18:09:18.619274+0000","last_became_peered":"2024-01-15T18:09:18.619274+0000","last_unstale":"2024-01-15T18:09:19.683900+0000","last_undegraded":"2024-01-15T18:09:19.683900+0000","last_fullsized":"2024-01-15T18:09:19.683900+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:17.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685032+0000","last_change":"2024-01-15T18:09:18.619973+0000","last_active":"2024-01-15T18:09:19.685032+0000","last_peered":"2024-01-15T18:09:19.685032+0000","last_clean":"2024-01-15T18:09:19.685032+0000","last_became_active":"2024-01-15T18:09:18.618815+0000","last_became_peered":"2024-01-15T18:09:18.618815+0000","last_unstale":"2024-01-15T18:09:19.685032+0000","last_undegraded":"2024-01-15T18:09:19.685032+0000","last_fullsized":"2024-01-15T18:09:19.685032+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:34:45.496931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683546+0000","last_change":"2024-01-15T18:09:18.623820+0000","last_active":"2024-01-15T18:09:19.683546+0000","last_peered":"2024-01-15T18:09:19.683546+0000","last_clean":"2024-01-15T18:09:19.683546+0000","last_became_active":"2024-01-15T18:09:18.623525+0000","last_became_peered":"2024-01-15T18:09:18.623525+0000","last_unstale":"2024-01-15T18:09:19.683546+0000","last_undegraded":"2024-01-15T18:09:19.683546+0000","last_fullsized":"2024-01-15T18:09:19.683546+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:32:44.586300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683464+0000","last_change":"2024-01-15T18:09:18.624355+0000","last_active":"2024-01-15T18:09:19.683464+0000","last_peered":"2024-01-15T18:09:19.683464+0000","last_clean":"2024-01-15T18:09:19.683464+0000","last_became_active":"2024-01-15T18:09:18.624036+0000","last_became_peered":"2024-01-15T18:09:18.624036+0000","last_unstale":"2024-01-15T18:09:19.683464+0000","last_undegraded":"2024-01-15T18:09:19.683464+0000","last_fullsized":"2024-01-15T18:09:19.683464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:05.821944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684818+0000","last_change":"2024-01-15T18:09:18.621278+0000","last_active":"2024-01-15T18:09:19.684818+0000","last_peered":"2024-01-15T18:09:19.684818+0000","last_clean":"2024-01-15T18:09:19.684818+0000","last_became_active":"2024-01-15T18:09:18.620995+0000","last_became_peered":"2024-01-15T18:09:18.620995+0000","last_unstale":"2024-01-15T18:09:19.684818+0000","last_undegraded":"2024-01-15T18:09:19.684818+0000","last_fullsized":"2024-01-15T18:09:19.684818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:54.637682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683347+0000","last_change":"2024-01-15T18:09:18.619885+0000","last_active":"2024-01-15T18:09:19.683347+0000","last_peered":"2024-01-15T18:09:19.683347+0000","last_clean":"2024-01-15T18:09:19.683347+0000","last_became_active":"2024-01-15T18:09:18.619592+0000","last_became_peered":"2024-01-15T18:09:18.619592+0000","last_unstale":"2024-01-15T18:09:19.683347+0000","last_undegraded":"2024-01-15T18:09:19.683347+0000","last_fullsized":"2024-01-15T18:09:19.683347+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:55.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684740+0000","last_change":"2024-01-15T18:09:18.617385+0000","last_active":"2024-01-15T18:09:19.684740+0000","last_peered":"2024-01-15T18:09:19.684740+0000","last_clean":"2024-01-15T18:09:19.684740+0000","last_became_active":"2024-01-15T18:09:18.617115+0000","last_became_peered":"2024-01-15T18:09:18.617115+0000","last_unstale":"2024-01-15T18:09:19.684740+0000","last_undegraded":"2024-01-15T18:09:19.684740+0000","last_fullsized":"2024-01-15T18:09:19.684740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:48:19.391925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683258+0000","last_change":"2024-01-15T18:09:18.626964+0000","last_active":"2024-01-15T18:09:19.683258+0000","last_peered":"2024-01-15T18:09:19.683258+0000","last_clean":"2024-01-15T18:09:19.683258+0000","last_became_active":"2024-01-15T18:09:18.626671+0000","last_became_peered":"2024-01-15T18:09:18.626671+0000","last_unstale":"2024-01-15T18:09:19.683258+0000","last_undegraded":"2024-01-15T18:09:19.683258+0000","last_fullsized":"2024-01-15T18:09:19.683258+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:00:01.967586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610173+0000","last_change":"2024-01-15T18:09:18.628704+0000","last_active":"2024-01-15T18:09:19.610173+0000","last_peered":"2024-01-15T18:09:19.610173+0000","last_clean":"2024-01-15T18:09:19.610173+0000","last_became_active":"2024-01-15T18:09:18.628270+0000","last_became_peered":"2024-01-15T18:09:18.628270+0000","last_unstale":"2024-01-15T18:09:19.610173+0000","last_undegraded":"2024-01-15T18:09:19.610173+0000","last_fullsized":"2024-01-15T18:09:19.610173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:24.673049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683661+0000","last_change":"2024-01-15T18:09:18.617173+0000","last_active":"2024-01-15T18:09:19.683661+0000","last_peered":"2024-01-15T18:09:19.683661+0000","last_clean":"2024-01-15T18:09:19.683661+0000","last_became_active":"2024-01-15T18:09:18.616892+0000","last_became_peered":"2024-01-15T18:09:18.616892+0000","last_unstale":"2024-01-15T18:09:19.683661+0000","last_undegraded":"2024-01-15T18:09:19.683661+0000","last_fullsized":"2024-01-15T18:09:19.683661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:59.185525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605766+0000","last_change":"2024-01-15T18:09:18.619788+0000","last_active":"2024-01-15T18:09:19.605766+0000","last_peered":"2024-01-15T18:09:19.605766+0000","last_clean":"2024-01-15T18:09:19.605766+0000","last_became_active":"2024-01-15T18:09:18.619484+0000","last_became_peered":"2024-01-15T18:09:18.619484+0000","last_unstale":"2024-01-15T18:09:19.605766+0000","last_undegraded":"2024-01-15T18:09:19.605766+0000","last_fullsized":"2024-01-15T18:09:19.605766+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:56.729572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685303+0000","last_change":"2024-01-15T18:09:18.618042+0000","last_active":"2024-01-15T18:09:19.685303+0000","last_peered":"2024-01-15T18:09:19.685303+0000","last_clean":"2024-01-15T18:09:19.685303+0000","last_became_active":"2024-01-15T18:09:18.617708+0000","last_became_peered":"2024-01-15T18:09:18.617708+0000","last_unstale":"2024-01-15T18:09:19.685303+0000","last_undegraded":"2024-01-15T18:09:19.685303+0000","last_fullsized":"2024-01-15T18:09:19.685303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:26:00.018458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683811+0000","last_change":"2024-01-15T18:09:18.622766+0000","last_active":"2024-01-15T18:09:19.683811+0000","last_peered":"2024-01-15T18:09:19.683811+0000","last_clean":"2024-01-15T18:09:19.683811+0000","last_became_active":"2024-01-15T18:09:18.621303+0000","last_became_peered":"2024-01-15T18:09:18.621303+0000","last_unstale":"2024-01-15T18:09:19.683811+0000","last_undegraded":"2024-01-15T18:09:19.683811+0000","last_fullsized":"2024-01-15T18:09:19.683811+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:44.854034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685373+0000","last_change":"2024-01-15T18:09:18.625819+0000","last_active":"2024-01-15T18:09:19.685373+0000","last_peered":"2024-01-15T18:09:19.685373+0000","last_clean":"2024-01-15T18:09:19.685373+0000","last_became_active":"2024-01-15T18:09:18.625540+0000","last_became_peered":"2024-01-15T18:09:18.625540+0000","last_unstale":"2024-01-15T18:09:19.685373+0000","last_undegraded":"2024-01-15T18:09:19.685373+0000","last_fullsized":"2024-01-15T18:09:19.685373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:52:44.879436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683165+0000","last_change":"2024-01-15T18:09:18.620524+0000","last_active":"2024-01-15T18:09:19.683165+0000","last_peered":"2024-01-15T18:09:19.683165+0000","last_clean":"2024-01-15T18:09:19.683165+0000","last_became_active":"2024-01-15T18:09:18.620252+0000","last_became_peered":"2024-01-15T18:09:18.620252+0000","last_unstale":"2024-01-15T18:09:19.683165+0000","last_undegraded":"2024-01-15T18:09:19.683165+0000","last_fullsized":"2024-01-15T18:09:19.683165+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:09.480229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683160+0000","last_change":"2024-01-15T18:09:18.616783+0000","last_active":"2024-01-15T18:09:19.683160+0000","last_peered":"2024-01-15T18:09:19.683160+0000","last_clean":"2024-01-15T18:09:19.683160+0000","last_became_active":"2024-01-15T18:09:18.616346+0000","last_became_peered":"2024-01-15T18:09:18.616346+0000","last_unstale":"2024-01-15T18:09:19.683160+0000","last_undegraded":"2024-01-15T18:09:19.683160+0000","last_fullsized":"2024-01-15T18:09:19.683160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:55.016220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684351+0000","last_change":"2024-01-15T18:09:18.628746+0000","last_active":"2024-01-15T18:09:19.684351+0000","last_peered":"2024-01-15T18:09:19.684351+0000","last_clean":"2024-01-15T18:09:19.684351+0000","last_became_active":"2024-01-15T18:09:18.628474+0000","last_became_peered":"2024-01-15T18:09:18.628474+0000","last_unstale":"2024-01-15T18:09:19.684351+0000","last_undegraded":"2024-01-15T18:09:19.684351+0000","last_fullsized":"2024-01-15T18:09:19.684351+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:56.729572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684792+0000","last_change":"2024-01-15T18:09:18.625975+0000","last_active":"2024-01-15T18:09:19.684792+0000","last_peered":"2024-01-15T18:09:19.684792+0000","last_clean":"2024-01-15T18:09:19.684792+0000","last_became_active":"2024-01-15T18:09:18.625673+0000","last_became_peered":"2024-01-15T18:09:18.625673+0000","last_unstale":"2024-01-15T18:09:19.684792+0000","last_undegraded":"2024-01-15T18:09:19.684792+0000","last_fullsized":"2024-01-15T18:09:19.684792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:55.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.754293+0000","last_change":"2024-01-15T18:09:17.732480+0000","last_active":"2024-01-15T18:09:18.754293+0000","last_peered":"2024-01-15T18:09:18.754293+0000","last_clean":"2024-01-15T18:09:18.754293+0000","last_became_active":"2024-01-15T18:09:17.732184+0000","last_became_peered":"2024-01-15T18:09:17.732184+0000","last_unstale":"2024-01-15T18:09:18.754293+0000","last_undegraded":"2024-01-15T18:09:18.754293+0000","last_fullsized":"2024-01-15T18:09:18.754293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:18.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685857+0000","last_change":"2024-01-15T18:09:18.626690+0000","last_active":"2024-01-15T18:09:19.685857+0000","last_peered":"2024-01-15T18:09:19.685857+0000","last_clean":"2024-01-15T18:09:19.685857+0000","last_became_active":"2024-01-15T18:09:18.626388+0000","last_became_peered":"2024-01-15T18:09:18.626388+0000","last_unstale":"2024-01-15T18:09:19.685857+0000","last_undegraded":"2024-01-15T18:09:19.685857+0000","last_fullsized":"2024-01-15T18:09:19.685857+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:03:05.533517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685004+0000","last_change":"2024-01-15T18:09:17.737390+0000","last_active":"2024-01-15T18:09:19.685004+0000","last_peered":"2024-01-15T18:09:19.685004+0000","last_clean":"2024-01-15T18:09:19.685004+0000","last_became_active":"2024-01-15T18:09:17.737051+0000","last_became_peered":"2024-01-15T18:09:17.737051+0000","last_unstale":"2024-01-15T18:09:19.685004+0000","last_undegraded":"2024-01-15T18:09:19.685004+0000","last_fullsized":"2024-01-15T18:09:19.685004+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:54.296498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685080+0000","last_change":"2024-01-15T18:09:18.617990+0000","last_active":"2024-01-15T18:09:19.685080+0000","last_peered":"2024-01-15T18:09:19.685080+0000","last_clean":"2024-01-15T18:09:19.685080+0000","last_became_active":"2024-01-15T18:09:18.617626+0000","last_became_peered":"2024-01-15T18:09:18.617626+0000","last_unstale":"2024-01-15T18:09:19.685080+0000","last_undegraded":"2024-01-15T18:09:19.685080+0000","last_fullsized":"2024-01-15T18:09:19.685080+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:13.831851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.750931+0000","last_change":"2024-01-15T18:09:17.733411+0000","last_active":"2024-01-15T18:09:18.750931+0000","last_peered":"2024-01-15T18:09:18.750931+0000","last_clean":"2024-01-15T18:09:18.750931+0000","last_became_active":"2024-01-15T18:09:17.733112+0000","last_became_peered":"2024-01-15T18:09:17.733112+0000","last_unstale":"2024-01-15T18:09:18.750931+0000","last_undegraded":"2024-01-15T18:09:18.750931+0000","last_fullsized":"2024-01-15T18:09:18.750931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:41.918180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685436+0000","last_change":"2024-01-15T18:09:18.626341+0000","last_active":"2024-01-15T18:09:19.685436+0000","last_peered":"2024-01-15T18:09:19.685436+0000","last_clean":"2024-01-15T18:09:19.685436+0000","last_became_active":"2024-01-15T18:09:18.625990+0000","last_became_peered":"2024-01-15T18:09:18.625990+0000","last_unstale":"2024-01-15T18:09:19.685436+0000","last_undegraded":"2024-01-15T18:09:19.685436+0000","last_fullsized":"2024-01-15T18:09:19.685436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:38.466718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605959+0000","last_change":"2024-01-15T18:09:17.732662+0000","last_active":"2024-01-15T18:09:19.605959+0000","last_peered":"2024-01-15T18:09:19.605959+0000","last_clean":"2024-01-15T18:09:19.605959+0000","last_became_active":"2024-01-15T18:09:17.732350+0000","last_became_peered":"2024-01-15T18:09:17.732350+0000","last_unstale":"2024-01-15T18:09:19.605959+0000","last_undegraded":"2024-01-15T18:09:19.605959+0000","last_fullsized":"2024-01-15T18:09:19.605959+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:33.617418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684989+0000","last_change":"2024-01-15T18:09:18.615025+0000","last_active":"2024-01-15T18:09:19.684989+0000","last_peered":"2024-01-15T18:09:19.684989+0000","last_clean":"2024-01-15T18:09:19.684989+0000","last_became_active":"2024-01-15T18:09:18.614428+0000","last_became_peered":"2024-01-15T18:09:18.614428+0000","last_unstale":"2024-01-15T18:09:19.684989+0000","last_undegraded":"2024-01-15T18:09:19.684989+0000","last_fullsized":"2024-01-15T18:09:19.684989+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:53.698036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.750748+0000","last_change":"2024-01-15T18:09:17.731122+0000","last_active":"2024-01-15T18:09:18.750748+0000","last_peered":"2024-01-15T18:09:18.750748+0000","last_clean":"2024-01-15T18:09:18.750748+0000","last_became_active":"2024-01-15T18:09:17.730734+0000","last_became_peered":"2024-01-15T18:09:17.730734+0000","last_unstale":"2024-01-15T18:09:18.750748+0000","last_undegraded":"2024-01-15T18:09:18.750748+0000","last_fullsized":"2024-01-15T18:09:18.750748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:09.626767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685275+0000","last_change":"2024-01-15T18:09:18.621477+0000","last_active":"2024-01-15T18:09:19.685275+0000","last_peered":"2024-01-15T18:09:19.685275+0000","last_clean":"2024-01-15T18:09:19.685275+0000","last_became_active":"2024-01-15T18:09:18.621143+0000","last_became_peered":"2024-01-15T18:09:18.621143+0000","last_unstale":"2024-01-15T18:09:19.685275+0000","last_undegraded":"2024-01-15T18:09:19.685275+0000","last_fullsized":"2024-01-15T18:09:19.685275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:41.820904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.748980+0000","last_change":"2024-01-15T18:09:17.740583+0000","last_active":"2024-01-15T18:09:18.748980+0000","last_peered":"2024-01-15T18:09:18.748980+0000","last_clean":"2024-01-15T18:09:18.748980+0000","last_became_active":"2024-01-15T18:09:17.740116+0000","last_became_peered":"2024-01-15T18:09:17.740116+0000","last_unstale":"2024-01-15T18:09:18.748980+0000","last_undegraded":"2024-01-15T18:09:18.748980+0000","last_fullsized":"2024-01-15T18:09:18.748980+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:45.794446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610113+0000","last_change":"2024-01-15T18:09:18.615675+0000","last_active":"2024-01-15T18:09:19.610113+0000","last_peered":"2024-01-15T18:09:19.610113+0000","last_clean":"2024-01-15T18:09:19.610113+0000","last_became_active":"2024-01-15T18:09:18.615168+0000","last_became_peered":"2024-01-15T18:09:18.615168+0000","last_unstale":"2024-01-15T18:09:19.610113+0000","last_undegraded":"2024-01-15T18:09:19.610113+0000","last_fullsized":"2024-01-15T18:09:19.610113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:57.675563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605537+0000","last_change":"2024-01-15T18:09:17.729345+0000","last_active":"2024-01-15T18:09:19.605537+0000","last_peered":"2024-01-15T18:09:19.605537+0000","last_clean":"2024-01-15T18:09:19.605537+0000","last_became_active":"2024-01-15T18:09:17.728928+0000","last_became_peered":"2024-01-15T18:09:17.728928+0000","last_unstale":"2024-01-15T18:09:19.605537+0000","last_undegraded":"2024-01-15T18:09:19.605537+0000","last_fullsized":"2024-01-15T18:09:19.605537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:38.783668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684224+0000","last_change":"2024-01-15T18:09:18.623008+0000","last_active":"2024-01-15T18:09:19.684224+0000","last_peered":"2024-01-15T18:09:19.684224+0000","last_clean":"2024-01-15T18:09:19.684224+0000","last_became_active":"2024-01-15T18:09:18.622716+0000","last_became_peered":"2024-01-15T18:09:18.622716+0000","last_unstale":"2024-01-15T18:09:19.684224+0000","last_undegraded":"2024-01-15T18:09:19.684224+0000","last_fullsized":"2024-01-15T18:09:19.684224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:03.810960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.604149+0000","last_change":"2024-01-15T18:09:17.737345+0000","last_active":"2024-01-15T18:09:18.604149+0000","last_peered":"2024-01-15T18:09:18.604149+0000","last_clean":"2024-01-15T18:09:18.604149+0000","last_became_active":"2024-01-15T18:09:17.737072+0000","last_became_peered":"2024-01-15T18:09:17.737072+0000","last_unstale":"2024-01-15T18:09:18.604149+0000","last_undegraded":"2024-01-15T18:09:18.604149+0000","last_fullsized":"2024-01-15T18:09:18.604149+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:17.051164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.609410+0000","last_change":"2024-01-15T18:09:18.618214+0000","last_active":"2024-01-15T18:09:19.609410+0000","last_peered":"2024-01-15T18:09:19.609410+0000","last_clean":"2024-01-15T18:09:19.609410+0000","last_became_active":"2024-01-15T18:09:18.617903+0000","last_became_peered":"2024-01-15T18:09:18.617903+0000","last_unstale":"2024-01-15T18:09:19.609410+0000","last_undegraded":"2024-01-15T18:09:19.609410+0000","last_fullsized":"2024-01-15T18:09:19.609410+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:14.028543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683694+0000","last_change":"2024-01-15T18:09:17.744952+0000","last_active":"2024-01-15T18:09:19.683694+0000","last_peered":"2024-01-15T18:09:19.683694+0000","last_clean":"2024-01-15T18:09:19.683694+0000","last_became_active":"2024-01-15T18:09:17.744693+0000","last_became_peered":"2024-01-15T18:09:17.744693+0000","last_unstale":"2024-01-15T18:09:19.683694+0000","last_undegraded":"2024-01-15T18:09:19.683694+0000","last_fullsized":"2024-01-15T18:09:19.683694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:41.918180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683557+0000","last_change":"2024-01-15T18:09:18.617363+0000","last_active":"2024-01-15T18:09:19.683557+0000","last_peered":"2024-01-15T18:09:19.683557+0000","last_clean":"2024-01-15T18:09:19.683557+0000","last_became_active":"2024-01-15T18:09:18.617048+0000","last_became_peered":"2024-01-15T18:09:18.617048+0000","last_unstale":"2024-01-15T18:09:19.683557+0000","last_undegraded":"2024-01-15T18:09:19.683557+0000","last_fullsized":"2024-01-15T18:09:19.683557+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:34.918064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685084+0000","last_change":"2024-01-15T18:09:17.734771+0000","last_active":"2024-01-15T18:09:19.685084+0000","last_peered":"2024-01-15T18:09:19.685084+0000","last_clean":"2024-01-15T18:09:19.685084+0000","last_became_active":"2024-01-15T18:09:17.734386+0000","last_became_peered":"2024-01-15T18:09:17.734386+0000","last_unstale":"2024-01-15T18:09:19.685084+0000","last_undegraded":"2024-01-15T18:09:19.685084+0000","last_fullsized":"2024-01-15T18:09:19.685084+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:12.825647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684529+0000","last_change":"2024-01-15T18:09:18.629983+0000","last_active":"2024-01-15T18:09:19.684529+0000","last_peered":"2024-01-15T18:09:19.684529+0000","last_clean":"2024-01-15T18:09:19.684529+0000","last_became_active":"2024-01-15T18:09:18.629688+0000","last_became_peered":"2024-01-15T18:09:18.629688+0000","last_unstale":"2024-01-15T18:09:19.684529+0000","last_undegraded":"2024-01-15T18:09:19.684529+0000","last_fullsized":"2024-01-15T18:09:19.684529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:25:02.032448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683707+0000","last_change":"2024-01-15T18:09:17.740535+0000","last_active":"2024-01-15T18:09:19.683707+0000","last_peered":"2024-01-15T18:09:19.683707+0000","last_clean":"2024-01-15T18:09:19.683707+0000","last_became_active":"2024-01-15T18:09:17.740224+0000","last_became_peered":"2024-01-15T18:09:17.740224+0000","last_unstale":"2024-01-15T18:09:19.683707+0000","last_undegraded":"2024-01-15T18:09:19.683707+0000","last_fullsized":"2024-01-15T18:09:19.683707+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:16.633464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683604+0000","last_change":"2024-01-15T18:09:18.617388+0000","last_active":"2024-01-15T18:09:19.683604+0000","last_peered":"2024-01-15T18:09:19.683604+0000","last_clean":"2024-01-15T18:09:19.683604+0000","last_became_active":"2024-01-15T18:09:18.616216+0000","last_became_peered":"2024-01-15T18:09:18.616216+0000","last_unstale":"2024-01-15T18:09:19.683604+0000","last_undegraded":"2024-01-15T18:09:19.683604+0000","last_fullsized":"2024-01-15T18:09:19.683604+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:05:58.942405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.755660+0000","last_change":"2024-01-15T18:09:17.736247+0000","last_active":"2024-01-15T18:09:18.755660+0000","last_peered":"2024-01-15T18:09:18.755660+0000","last_clean":"2024-01-15T18:09:18.755660+0000","last_became_active":"2024-01-15T18:09:17.735931+0000","last_became_peered":"2024-01-15T18:09:17.735931+0000","last_unstale":"2024-01-15T18:09:18.755660+0000","last_undegraded":"2024-01-15T18:09:18.755660+0000","last_fullsized":"2024-01-15T18:09:18.755660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:37:37.174033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605619+0000","last_change":"2024-01-15T18:09:18.626316+0000","last_active":"2024-01-15T18:09:19.605619+0000","last_peered":"2024-01-15T18:09:19.605619+0000","last_clean":"2024-01-15T18:09:19.605619+0000","last_became_active":"2024-01-15T18:09:18.625246+0000","last_became_peered":"2024-01-15T18:09:18.625246+0000","last_unstale":"2024-01-15T18:09:19.605619+0000","last_undegraded":"2024-01-15T18:09:19.605619+0000","last_fullsized":"2024-01-15T18:09:19.605619+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:17.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685242+0000","last_change":"2024-01-15T18:09:17.741082+0000","last_active":"2024-01-15T18:09:19.685242+0000","last_peered":"2024-01-15T18:09:19.685242+0000","last_clean":"2024-01-15T18:09:19.685242+0000","last_became_active":"2024-01-15T18:09:17.740396+0000","last_became_peered":"2024-01-15T18:09:17.740396+0000","last_unstale":"2024-01-15T18:09:19.685242+0000","last_undegraded":"2024-01-15T18:09:19.685242+0000","last_fullsized":"2024-01-15T18:09:19.685242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:32:43.580096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683534+0000","last_change":"2024-01-15T18:09:18.625338+0000","last_active":"2024-01-15T18:09:19.683534+0000","last_peered":"2024-01-15T18:09:19.683534+0000","last_clean":"2024-01-15T18:09:19.683534+0000","last_became_active":"2024-01-15T18:09:18.625054+0000","last_became_peered":"2024-01-15T18:09:18.625054+0000","last_unstale":"2024-01-15T18:09:19.683534+0000","last_undegraded":"2024-01-15T18:09:19.683534+0000","last_fullsized":"2024-01-15T18:09:19.683534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:38.466718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610100+0000","last_change":"2024-01-15T18:09:17.727366+0000","last_active":"2024-01-15T18:09:19.610100+0000","last_peered":"2024-01-15T18:09:19.610100+0000","last_clean":"2024-01-15T18:09:19.610100+0000","last_became_active":"2024-01-15T18:09:17.727056+0000","last_became_peered":"2024-01-15T18:09:17.727056+0000","last_unstale":"2024-01-15T18:09:19.610100+0000","last_undegraded":"2024-01-15T18:09:19.610100+0000","last_fullsized":"2024-01-15T18:09:19.610100+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:54.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.608804+0000","last_change":"2024-01-15T18:09:18.615790+0000","last_active":"2024-01-15T18:09:19.608804+0000","last_peered":"2024-01-15T18:09:19.608804+0000","last_clean":"2024-01-15T18:09:19.608804+0000","last_became_active":"2024-01-15T18:09:18.615300+0000","last_became_peered":"2024-01-15T18:09:18.615300+0000","last_unstale":"2024-01-15T18:09:19.608804+0000","last_undegraded":"2024-01-15T18:09:19.608804+0000","last_fullsized":"2024-01-15T18:09:19.608804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:22:44.116280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605665+0000","last_change":"2024-01-15T18:09:17.734227+0000","last_active":"2024-01-15T18:09:19.605665+0000","last_peered":"2024-01-15T18:09:19.605665+0000","last_clean":"2024-01-15T18:09:19.605665+0000","last_became_active":"2024-01-15T18:09:17.733877+0000","last_became_peered":"2024-01-15T18:09:17.733877+0000","last_unstale":"2024-01-15T18:09:19.605665+0000","last_undegraded":"2024-01-15T18:09:19.605665+0000","last_fullsized":"2024-01-15T18:09:19.605665+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:49.227701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683267+0000","last_change":"2024-01-15T18:09:18.626838+0000","last_active":"2024-01-15T18:09:19.683267+0000","last_peered":"2024-01-15T18:09:19.683267+0000","last_clean":"2024-01-15T18:09:19.683267+0000","last_became_active":"2024-01-15T18:09:18.626561+0000","last_became_peered":"2024-01-15T18:09:18.626561+0000","last_unstale":"2024-01-15T18:09:19.683267+0000","last_undegraded":"2024-01-15T18:09:19.683267+0000","last_fullsized":"2024-01-15T18:09:19.683267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:12.189491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683205+0000","last_change":"2024-01-15T18:09:17.737381+0000","last_active":"2024-01-15T18:09:19.683205+0000","last_peered":"2024-01-15T18:09:19.683205+0000","last_clean":"2024-01-15T18:09:19.683205+0000","last_became_active":"2024-01-15T18:09:17.737041+0000","last_became_peered":"2024-01-15T18:09:17.737041+0000","last_unstale":"2024-01-15T18:09:19.683205+0000","last_undegraded":"2024-01-15T18:09:19.683205+0000","last_fullsized":"2024-01-15T18:09:19.683205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:44.219467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683672+0000","last_change":"2024-01-15T18:09:18.620086+0000","last_active":"2024-01-15T18:09:19.683672+0000","last_peered":"2024-01-15T18:09:19.683672+0000","last_clean":"2024-01-15T18:09:19.683672+0000","last_became_active":"2024-01-15T18:09:18.619728+0000","last_became_peered":"2024-01-15T18:09:18.619728+0000","last_unstale":"2024-01-15T18:09:19.683672+0000","last_undegraded":"2024-01-15T18:09:19.683672+0000","last_fullsized":"2024-01-15T18:09:19.683672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:48:51.860359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685121+0000","last_change":"2024-01-15T18:09:17.741156+0000","last_active":"2024-01-15T18:09:19.685121+0000","last_peered":"2024-01-15T18:09:19.685121+0000","last_clean":"2024-01-15T18:09:19.685121+0000","last_became_active":"2024-01-15T18:09:17.740335+0000","last_became_peered":"2024-01-15T18:09:17.740335+0000","last_unstale":"2024-01-15T18:09:19.685121+0000","last_undegraded":"2024-01-15T18:09:19.685121+0000","last_fullsized":"2024-01-15T18:09:19.685121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:02.451959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606289+0000","last_change":"2024-01-15T18:09:18.623472+0000","last_active":"2024-01-15T18:09:19.606289+0000","last_peered":"2024-01-15T18:09:19.606289+0000","last_clean":"2024-01-15T18:09:19.606289+0000","last_became_active":"2024-01-15T18:09:18.623079+0000","last_became_peered":"2024-01-15T18:09:18.623079+0000","last_unstale":"2024-01-15T18:09:19.606289+0000","last_undegraded":"2024-01-15T18:09:19.606289+0000","last_fullsized":"2024-01-15T18:09:19.606289+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:13.831851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.751162+0000","last_change":"2024-01-15T18:09:17.733526+0000","last_active":"2024-01-15T18:09:18.751162+0000","last_peered":"2024-01-15T18:09:18.751162+0000","last_clean":"2024-01-15T18:09:18.751162+0000","last_became_active":"2024-01-15T18:09:17.733181+0000","last_became_peered":"2024-01-15T18:09:17.733181+0000","last_unstale":"2024-01-15T18:09:18.751162+0000","last_undegraded":"2024-01-15T18:09:18.751162+0000","last_fullsized":"2024-01-15T18:09:18.751162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:25.765459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685170+0000","last_change":"2024-01-15T18:09:18.623762+0000","last_active":"2024-01-15T18:09:19.685170+0000","last_peered":"2024-01-15T18:09:19.685170+0000","last_clean":"2024-01-15T18:09:19.685170+0000","last_became_active":"2024-01-15T18:09:18.623464+0000","last_became_peered":"2024-01-15T18:09:18.623464+0000","last_unstale":"2024-01-15T18:09:19.685170+0000","last_undegraded":"2024-01-15T18:09:19.685170+0000","last_fullsized":"2024-01-15T18:09:19.685170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:46:18.332990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606053+0000","last_change":"2024-01-15T18:09:17.735004+0000","last_active":"2024-01-15T18:09:19.606053+0000","last_peered":"2024-01-15T18:09:19.606053+0000","last_clean":"2024-01-15T18:09:19.606053+0000","last_became_active":"2024-01-15T18:09:17.732323+0000","last_became_peered":"2024-01-15T18:09:17.732323+0000","last_unstale":"2024-01-15T18:09:19.606053+0000","last_undegraded":"2024-01-15T18:09:19.606053+0000","last_fullsized":"2024-01-15T18:09:19.606053+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:07.557596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683658+0000","last_change":"2024-01-15T18:09:18.624732+0000","last_active":"2024-01-15T18:09:19.683658+0000","last_peered":"2024-01-15T18:09:19.683658+0000","last_clean":"2024-01-15T18:09:19.683658+0000","last_became_active":"2024-01-15T18:09:18.624030+0000","last_became_peered":"2024-01-15T18:09:18.624030+0000","last_unstale":"2024-01-15T18:09:19.683658+0000","last_undegraded":"2024-01-15T18:09:19.683658+0000","last_fullsized":"2024-01-15T18:09:19.683658+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:03.458163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.610555+0000","last_change":"2024-01-15T18:09:17.741147+0000","last_active":"2024-01-15T18:09:18.610555+0000","last_peered":"2024-01-15T18:09:18.610555+0000","last_clean":"2024-01-15T18:09:18.610555+0000","last_became_active":"2024-01-15T18:09:17.740822+0000","last_became_peered":"2024-01-15T18:09:17.740822+0000","last_unstale":"2024-01-15T18:09:18.610555+0000","last_undegraded":"2024-01-15T18:09:18.610555+0000","last_fullsized":"2024-01-15T18:09:18.610555+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:26.648940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606119+0000","last_change":"2024-01-15T18:09:18.619903+0000","last_active":"2024-01-15T18:09:19.606119+0000","last_peered":"2024-01-15T18:09:19.606119+0000","last_clean":"2024-01-15T18:09:19.606119+0000","last_became_active":"2024-01-15T18:09:18.619568+0000","last_became_peered":"2024-01-15T18:09:18.619568+0000","last_unstale":"2024-01-15T18:09:19.606119+0000","last_undegraded":"2024-01-15T18:09:19.606119+0000","last_fullsized":"2024-01-15T18:09:19.606119+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:12.189491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685366+0000","last_change":"2024-01-15T18:09:17.738248+0000","last_active":"2024-01-15T18:09:19.685366+0000","last_peered":"2024-01-15T18:09:19.685366+0000","last_clean":"2024-01-15T18:09:19.685366+0000","last_became_active":"2024-01-15T18:09:17.737979+0000","last_became_peered":"2024-01-15T18:09:17.737979+0000","last_unstale":"2024-01-15T18:09:19.685366+0000","last_undegraded":"2024-01-15T18:09:19.685366+0000","last_fullsized":"2024-01-15T18:09:19.685366+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.911860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684423+0000","last_change":"2024-01-15T18:09:18.617551+0000","last_active":"2024-01-15T18:09:19.684423+0000","last_peered":"2024-01-15T18:09:19.684423+0000","last_clean":"2024-01-15T18:09:19.684423+0000","last_became_active":"2024-01-15T18:09:18.617224+0000","last_became_peered":"2024-01-15T18:09:18.617224+0000","last_unstale":"2024-01-15T18:09:19.684423+0000","last_undegraded":"2024-01-15T18:09:19.684423+0000","last_fullsized":"2024-01-15T18:09:19.684423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:00:01.967586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684306+0000","last_change":"2024-01-15T18:09:17.726150+0000","last_active":"2024-01-15T18:09:19.684306+0000","last_peered":"2024-01-15T18:09:19.684306+0000","last_clean":"2024-01-15T18:09:19.684306+0000","last_became_active":"2024-01-15T18:09:17.725851+0000","last_became_peered":"2024-01-15T18:09:17.725851+0000","last_unstale":"2024-01-15T18:09:19.684306+0000","last_undegraded":"2024-01-15T18:09:19.684306+0000","last_fullsized":"2024-01-15T18:09:19.684306+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:26:52.800975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684372+0000","last_change":"2024-01-15T18:09:18.620897+0000","last_active":"2024-01-15T18:09:19.684372+0000","last_peered":"2024-01-15T18:09:19.684372+0000","last_clean":"2024-01-15T18:09:19.684372+0000","last_became_active":"2024-01-15T18:09:18.620599+0000","last_became_peered":"2024-01-15T18:09:18.620599+0000","last_unstale":"2024-01-15T18:09:19.684372+0000","last_undegraded":"2024-01-15T18:09:19.684372+0000","last_fullsized":"2024-01-15T18:09:19.684372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:09.480229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.605599+0000","last_change":"2024-01-15T18:09:17.735217+0000","last_active":"2024-01-15T18:09:19.605599+0000","last_peered":"2024-01-15T18:09:19.605599+0000","last_clean":"2024-01-15T18:09:19.605599+0000","last_became_active":"2024-01-15T18:09:17.734896+0000","last_became_peered":"2024-01-15T18:09:17.734896+0000","last_unstale":"2024-01-15T18:09:19.605599+0000","last_undegraded":"2024-01-15T18:09:19.605599+0000","last_fullsized":"2024-01-15T18:09:19.605599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:13.022339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606243+0000","last_change":"2024-01-15T18:09:18.617996+0000","last_active":"2024-01-15T18:09:19.606243+0000","last_peered":"2024-01-15T18:09:19.606243+0000","last_clean":"2024-01-15T18:09:19.606243+0000","last_became_active":"2024-01-15T18:09:18.614900+0000","last_became_peered":"2024-01-15T18:09:18.614900+0000","last_unstale":"2024-01-15T18:09:19.606243+0000","last_undegraded":"2024-01-15T18:09:19.606243+0000","last_fullsized":"2024-01-15T18:09:19.606243+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:16:55.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685792+0000","last_change":"2024-01-15T18:09:17.737201+0000","last_active":"2024-01-15T18:09:19.685792+0000","last_peered":"2024-01-15T18:09:19.685792+0000","last_clean":"2024-01-15T18:09:19.685792+0000","last_became_active":"2024-01-15T18:09:17.736865+0000","last_became_peered":"2024-01-15T18:09:17.736865+0000","last_unstale":"2024-01-15T18:09:19.685792+0000","last_undegraded":"2024-01-15T18:09:19.685792+0000","last_fullsized":"2024-01-15T18:09:19.685792+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:38.804296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606336+0000","last_change":"2024-01-15T18:09:18.620490+0000","last_active":"2024-01-15T18:09:19.606336+0000","last_peered":"2024-01-15T18:09:19.606336+0000","last_clean":"2024-01-15T18:09:19.606336+0000","last_became_active":"2024-01-15T18:09:18.620195+0000","last_became_peered":"2024-01-15T18:09:18.620195+0000","last_unstale":"2024-01-15T18:09:19.606336+0000","last_undegraded":"2024-01-15T18:09:19.606336+0000","last_fullsized":"2024-01-15T18:09:19.606336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:26:00.018458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.610826+0000","last_change":"2024-01-15T18:09:17.731403+0000","last_active":"2024-01-15T18:09:18.610826+0000","last_peered":"2024-01-15T18:09:18.610826+0000","last_clean":"2024-01-15T18:09:18.610826+0000","last_became_active":"2024-01-15T18:09:17.730919+0000","last_became_peered":"2024-01-15T18:09:17.730919+0000","last_unstale":"2024-01-15T18:09:18.610826+0000","last_undegraded":"2024-01-15T18:09:18.610826+0000","last_fullsized":"2024-01-15T18:09:18.610826+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:33.617418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684638+0000","last_change":"2024-01-15T18:09:18.627372+0000","last_active":"2024-01-15T18:09:19.684638+0000","last_peered":"2024-01-15T18:09:19.684638+0000","last_clean":"2024-01-15T18:09:19.684638+0000","last_became_active":"2024-01-15T18:09:18.627115+0000","last_became_peered":"2024-01-15T18:09:18.627115+0000","last_unstale":"2024-01-15T18:09:19.684638+0000","last_undegraded":"2024-01-15T18:09:19.684638+0000","last_fullsized":"2024-01-15T18:09:19.684638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:33.164470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610381+0000","last_change":"2024-01-15T18:09:17.722873+0000","last_active":"2024-01-15T18:09:19.610381+0000","last_peered":"2024-01-15T18:09:19.610381+0000","last_clean":"2024-01-15T18:09:19.610381+0000","last_became_active":"2024-01-15T18:09:17.722594+0000","last_became_peered":"2024-01-15T18:09:17.722594+0000","last_unstale":"2024-01-15T18:09:19.610381+0000","last_undegraded":"2024-01-15T18:09:19.610381+0000","last_fullsized":"2024-01-15T18:09:19.610381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:17:23.709145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685548+0000","last_change":"2024-01-15T18:09:18.617938+0000","last_active":"2024-01-15T18:09:19.685548+0000","last_peered":"2024-01-15T18:09:19.685548+0000","last_clean":"2024-01-15T18:09:19.685548+0000","last_became_active":"2024-01-15T18:09:18.617350+0000","last_became_peered":"2024-01-15T18:09:18.617350+0000","last_unstale":"2024-01-15T18:09:19.685548+0000","last_undegraded":"2024-01-15T18:09:19.685548+0000","last_fullsized":"2024-01-15T18:09:19.685548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:32:44.586300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.606022+0000","last_change":"2024-01-15T18:09:17.728443+0000","last_active":"2024-01-15T18:09:19.606022+0000","last_peered":"2024-01-15T18:09:19.606022+0000","last_clean":"2024-01-15T18:09:19.606022+0000","last_became_active":"2024-01-15T18:09:17.728142+0000","last_became_peered":"2024-01-15T18:09:17.728142+0000","last_unstale":"2024-01-15T18:09:19.606022+0000","last_undegraded":"2024-01-15T18:09:19.606022+0000","last_fullsized":"2024-01-15T18:09:19.606022+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:18.020809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.685124+0000","last_change":"2024-01-15T18:09:18.615608+0000","last_active":"2024-01-15T18:09:19.685124+0000","last_peered":"2024-01-15T18:09:19.685124+0000","last_clean":"2024-01-15T18:09:19.685124+0000","last_became_active":"2024-01-15T18:09:18.615351+0000","last_became_peered":"2024-01-15T18:09:18.615351+0000","last_unstale":"2024-01-15T18:09:19.685124+0000","last_undegraded":"2024-01-15T18:09:19.685124+0000","last_fullsized":"2024-01-15T18:09:19.685124+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:46:15.783337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684359+0000","last_change":"2024-01-15T18:09:17.726328+0000","last_active":"2024-01-15T18:09:19.684359+0000","last_peered":"2024-01-15T18:09:19.684359+0000","last_clean":"2024-01-15T18:09:19.684359+0000","last_became_active":"2024-01-15T18:09:17.725649+0000","last_became_peered":"2024-01-15T18:09:17.725649+0000","last_unstale":"2024-01-15T18:09:19.684359+0000","last_undegraded":"2024-01-15T18:09:19.684359+0000","last_fullsized":"2024-01-15T18:09:19.684359+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:46:07.566371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683534+0000","last_change":"2024-01-15T18:09:18.624775+0000","last_active":"2024-01-15T18:09:19.683534+0000","last_peered":"2024-01-15T18:09:19.683534+0000","last_clean":"2024-01-15T18:09:19.683534+0000","last_became_active":"2024-01-15T18:09:18.622596+0000","last_became_peered":"2024-01-15T18:09:18.622596+0000","last_unstale":"2024-01-15T18:09:19.683534+0000","last_undegraded":"2024-01-15T18:09:19.683534+0000","last_fullsized":"2024-01-15T18:09:19.683534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:26:00.018458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.604585+0000","last_change":"2024-01-15T18:09:17.741174+0000","last_active":"2024-01-15T18:09:18.604585+0000","last_peered":"2024-01-15T18:09:18.604585+0000","last_clean":"2024-01-15T18:09:18.604585+0000","last_became_active":"2024-01-15T18:09:17.740803+0000","last_became_peered":"2024-01-15T18:09:17.740803+0000","last_unstale":"2024-01-15T18:09:18.604585+0000","last_undegraded":"2024-01-15T18:09:18.604585+0000","last_fullsized":"2024-01-15T18:09:18.604585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:07.525089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683599+0000","last_change":"2024-01-15T18:09:18.617741+0000","last_active":"2024-01-15T18:09:19.683599+0000","last_peered":"2024-01-15T18:09:19.683599+0000","last_clean":"2024-01-15T18:09:19.683599+0000","last_became_active":"2024-01-15T18:09:18.616101+0000","last_became_peered":"2024-01-15T18:09:18.616101+0000","last_unstale":"2024-01-15T18:09:19.683599+0000","last_undegraded":"2024-01-15T18:09:19.683599+0000","last_fullsized":"2024-01-15T18:09:19.683599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:10:08.157767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684115+0000","last_change":"2024-01-15T18:09:17.741772+0000","last_active":"2024-01-15T18:09:19.684115+0000","last_peered":"2024-01-15T18:09:19.684115+0000","last_clean":"2024-01-15T18:09:19.684115+0000","last_became_active":"2024-01-15T18:09:17.741480+0000","last_became_peered":"2024-01-15T18:09:17.741480+0000","last_unstale":"2024-01-15T18:09:19.684115+0000","last_undegraded":"2024-01-15T18:09:19.684115+0000","last_fullsized":"2024-01-15T18:09:19.684115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:23.666845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.684732+0000","last_change":"2024-01-15T18:09:18.618337+0000","last_active":"2024-01-15T18:09:19.684732+0000","last_peered":"2024-01-15T18:09:19.684732+0000","last_clean":"2024-01-15T18:09:19.684732+0000","last_became_active":"2024-01-15T18:09:18.618060+0000","last_became_peered":"2024-01-15T18:09:18.618060+0000","last_unstale":"2024-01-15T18:09:19.684732+0000","last_undegraded":"2024-01-15T18:09:19.684732+0000","last_fullsized":"2024-01-15T18:09:19.684732+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:58:35.952158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683856+0000","last_change":"2024-01-15T18:09:17.739100+0000","last_active":"2024-01-15T18:09:19.683856+0000","last_peered":"2024-01-15T18:09:19.683856+0000","last_clean":"2024-01-15T18:09:19.683856+0000","last_became_active":"2024-01-15T18:09:17.736485+0000","last_became_peered":"2024-01-15T18:09:17.736485+0000","last_unstale":"2024-01-15T18:09:19.683856+0000","last_undegraded":"2024-01-15T18:09:19.683856+0000","last_fullsized":"2024-01-15T18:09:19.683856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:33.617418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.683726+0000","last_change":"2024-01-15T18:09:18.621681+0000","last_active":"2024-01-15T18:09:19.683726+0000","last_peered":"2024-01-15T18:09:19.683726+0000","last_clean":"2024-01-15T18:09:19.683726+0000","last_became_active":"2024-01-15T18:09:18.621282+0000","last_became_peered":"2024-01-15T18:09:18.621282+0000","last_unstale":"2024-01-15T18:09:19.683726+0000","last_undegraded":"2024-01-15T18:09:19.683726+0000","last_fullsized":"2024-01-15T18:09:19.683726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:17.704550+0000","last_clean_scrub_stamp":"2024-01-15T18:09:17.704550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:51.870805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:09:19.610113+0000","last_change":"2024-01-15T18:09:17.727124+0000","last_active":"2024-01-15T18:09:19.610113+0000","last_peered":"2024-01-15T18:09:19.610113+0000","last_clean":"2024-01-15T18:09:19.610113+0000","last_became_active":"2024-01-15T18:09:17.726619+0000","last_became_peered":"2024-01-15T18:09:17.726619+0000","last_unstale":"2024-01-15T18:09:19.610113+0000","last_undegraded":"2024-01-15T18:09:19.610113+0000","last_fullsized":"2024-01-15T18:09:19.610113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:09:16.698346+0000","last_clean_scrub_stamp":"2024-01-15T18:09:16.698346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:18:21.037790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":198,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:09:18.607140+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:09:18.607140+0000","last_peered":"2024-01-15T18:09:18.607140+0000","last_clean":"2024-01-15T18:09:18.607140+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:09:18.607140+0000","last_undegraded":"2024-01-15T18:09:18.607140+0000","last_fullsized":"2024-01-15T18:09:18.607140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1323008,"data_stored":893404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":0.73799999999999999,"15min":0.68000000000000005},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.53,"5min":0.85499999999999998,"15min":0.77600000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":13.744,"5min":13.744,"15min":13.744},"last":0.623}]},{"osd":1,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":0.83399999999999996,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":6.625,"5min":6.625,"15min":6.625},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.24,"5min":0.84099999999999997,"15min":0.78300000000000003},"min":{"1min":0.53900000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":6.0629999999999997,"5min":6.0629999999999997,"15min":6.0629999999999997},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":0.77400000000000002,"15min":0.73599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":4.5090000000000003,"5min":4.5090000000000003,"15min":4.5090000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":2.8580000000000001,"5min":1.1859999999999999,"15min":0.94499999999999995},"min":{"1min":0.48699999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":39.695,"5min":39.695,"15min":39.695},"last":0.78200000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":2.2959999999999998,"5min":1.0549999999999999,"15min":0.91700000000000004},"min":{"1min":0.49099999999999999,"5min":0.437,"15min":0.437},"max":{"1min":26.463999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.4039999999999999,"5min":0.89500000000000002,"15min":0.84499999999999997},"min":{"1min":0.55600000000000005,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":10.09,"5min":10.09,"15min":10.09},"last":0.64100000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.62,"15min":0.65800000000000003},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.71099999999999997,"15min":0.73699999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.7410000000000001,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.51400000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67100000000000004,"15min":0.73699999999999999},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.69399999999999995,"15min":0.748},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.76100000000000001,"15min":0.80600000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.93300000000000005,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71099999999999997,"15min":0.75900000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.745}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":720896,"data_stored":302876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.0150000000000001,"5min":1.0209999999999999,"15min":0.75800000000000001},"min":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":22.754000000000001,"5min":22.754000000000001,"15min":22.754000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.708,"5min":1.0509999999999999,"15min":0.76500000000000001},"min":{"1min":0.41599999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":9.1470000000000002,"5min":12.449,"15min":12.449},"last":0.70999999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.627,"5min":0.99399999999999999,"15min":0.752},"min":{"1min":0.45800000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":14.246,"5min":14.246,"15min":14.246},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.609,"5min":0.94799999999999995,"15min":0.78900000000000003},"min":{"1min":0.44900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":8.7590000000000003,"5min":8.7590000000000003,"15min":8.7590000000000003},"last":0.64500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.883,"5min":1.4299999999999999,"15min":0.95699999999999996},"min":{"1min":0.46600000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":42.825000000000003,"5min":42.825000000000003,"15min":42.825000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":3.9569999999999999,"5min":1.544,"15min":0.95399999999999996},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":42.851999999999997,"5min":42.851999999999997,"15min":42.851999999999997},"last":0.80800000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2959999999999998,"5min":1.179,"15min":0.85899999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":24.199000000000002,"5min":24.199000000000002,"15min":24.199000000000002},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.5149999999999999,"5min":0.95799999999999996,"15min":0.79500000000000004},"min":{"1min":0.49399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":12.225,"5min":12.225,"15min":12.225},"last":0.82399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.69499999999999995,"15min":0.65900000000000003},"min":{"1min":0.37,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":7.1609999999999996,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.442},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.92300000000000004,"15min":0.77200000000000002},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":8.3059999999999992,"5min":12.491,"15min":12.491},"last":0.46999999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.81100000000000005,"15min":0.77600000000000002},"min":{"1min":0.439,"5min":0.372,"15min":0.372},"max":{"1min":0.87,"5min":9.4819999999999993,"15min":9.4819999999999993},"last":0.621},{"interface":"front","average":{"1min":1.401,"5min":1.1559999999999999,"15min":0.92900000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":12.945,"5min":12.945,"15min":12.945},"last":0.60099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.48399999999999999,"5min":0.44,"15min":0.44},"max":{"1min":1.29,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.79200000000000004,"15min":0.78400000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":7.2050000000000001,"5min":7.2050000000000001,"15min":7.2050000000000001},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":303528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.66400000000000003,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.754},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.71799999999999997,"15min":0.65300000000000002},"min":{"1min":0.55000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.2349999999999999,"5min":4.093,"15min":4.093},"last":0.85699999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.78700000000000003,"15min":0.71199999999999997},"min":{"1min":0.56599999999999995,"5min":0.443,"15min":0.443},"max":{"1min":5.2089999999999996,"5min":5.2089999999999996,"15min":5.2089999999999996},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":0.82999999999999996,"15min":0.73799999999999999},"min":{"1min":0.48499999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":7.7270000000000003,"5min":7.7270000000000003,"15min":7.7270000000000003},"last":0.91700000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.76400000000000001,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.8079999999999998,"5min":3.8079999999999998,"15min":3.8079999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.052,"5min":0.76600000000000001,"15min":0.71799999999999997},"min":{"1min":0.501,"5min":0.434,"15min":0.434},"max":{"1min":3.327,"5min":3.327,"15min":3.327},"last":0.873}]},{"osd":3,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.80600000000000005,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.8029999999999999,"5min":4.1369999999999996,"15min":4.1369999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.0589999999999999,"5min":0.76500000000000001,"15min":0.70499999999999996},"min":{"1min":0.46200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":4.2190000000000003,"5min":4.2190000000000003,"15min":4.2190000000000003},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.73599999999999999,"15min":0.70399999999999996},"min":{"1min":0.36699999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":11.093,"5min":11.093,"15min":11.093},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68000000000000005,"15min":0.68600000000000005},"min":{"1min":0.33900000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.8080000000000001,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.65800000000000003,"15min":0.72099999999999997},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:08:56 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.76300000000000001,"15min":0.79200000000000004},"min":{"1min":0.41999999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.95499999999999996,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77700000000000002,"15min":0.81200000000000006},"min":{"1min":0.51100000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.696,"5min":2.2010000000000001,"15min":2.2010000000000001},"last":0.71599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36060,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335780,"statfs":{"total":96636764160,"available":96599838720,"internally_reserved":0,"allocated":733184,"data_stored":304488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":2.79,"5min":1.0760000000000001,"15min":0.76400000000000001},"min":{"1min":0.38300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":34.802,"5min":34.802,"15min":34.802},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.8720000000000001,"5min":0.93100000000000005,"15min":0.72199999999999998},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":22.335999999999999,"5min":22.335999999999999,"15min":22.335999999999999},"last":0.996}]},{"osd":1,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":3.375,"5min":1.2649999999999999,"15min":0.90200000000000002},"min":{"1min":0.52700000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":38.198,"5min":38.198,"15min":38.198},"last":0.89200000000000002},{"interface":"front","average":{"1min":3.3839999999999999,"5min":1.2569999999999999,"15min":0.877},"min":{"1min":0.50800000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":38.220999999999997,"5min":38.220999999999997,"15min":38.220999999999997},"last":0.876}]},{"osd":2,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":0.81999999999999995,"15min":0.74399999999999999},"min":{"1min":0.63800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":6.1600000000000001,"5min":6.1600000000000001,"15min":6.1600000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.258,"5min":0.82899999999999996,"15min":0.76700000000000002},"min":{"1min":0.53200000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":5.9279999999999999,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.81200000000000006}]},{"osd":3,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":0.89800000000000002,"15min":0.748},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":6.0549999999999997,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.5860000000000001,"5min":0.86499999999999999,"15min":0.749},"min":{"1min":0.48299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":8.5399999999999991,"5min":8.5399999999999991,"15min":8.5399999999999991},"last":0.64500000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.65500000000000003,"15min":0.65000000000000002},"min":{"1min":0.372,"5min":0.371,"15min":0.371},"max":{"1min":2.5070000000000001,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.76100000000000001},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":3.355,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":0.95599999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":0.73799999999999999,"15min":0.71599999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":10.284000000000001,"5min":10.284000000000001,"15min":10.284000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.44500000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.92800000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:08:55 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81000000000000005,"15min":0.75600000000000001},"min":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":3.4929999999999999,"5min":11.457000000000001,"15min":11.457000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":729088,"data_stored":302932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.958,"5min":0.82099999999999995,"15min":0.56899999999999995},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":25.771999999999998,"5min":25.771999999999998,"15min":25.771999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":2.6150000000000002,"5min":0.93899999999999995,"15min":0.60299999999999998},"min":{"1min":0.34300000000000003,"5min":0.246,"15min":0.246},"max":{"1min":28.332000000000001,"5min":28.332000000000001,"15min":28.332000000000001},"last":0.29499999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":2.9009999999999998,"5min":0.98699999999999999,"15min":0.61299999999999999},"min":{"1min":0.32100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":40.457999999999998,"5min":40.457999999999998,"15min":40.457999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":4.327,"5min":1.3740000000000001,"15min":0.81599999999999995},"min":{"1min":0.34399999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":27.427,"5min":27.427,"15min":27.427},"last":0.36299999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.972,"5min":0.84699999999999998,"15min":0.59499999999999997},"min":{"1min":0.41499999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":8.6319999999999997,"5min":8.6319999999999997,"15min":8.6319999999999997},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.7849999999999999,"5min":0.875,"15min":0.64300000000000002},"min":{"1min":0.42499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":12.538,"5min":12.538,"15min":12.538},"last":0.42699999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":0.93999999999999995,"15min":0.79500000000000004},"min":{"1min":0.48899999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":8.4339999999999993,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.353,"5min":0.88300000000000001,"15min":0.79300000000000004},"min":{"1min":0.503,"5min":0.377,"15min":0.377},"max":{"1min":6.9870000000000001,"5min":6.9870000000000001,"15min":6.9870000000000001},"last":0.627}]},{"osd":5,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":0.92000000000000004,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":6.5030000000000001,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.966,"5min":1.0580000000000001,"15min":0.86599999999999999},"min":{"1min":0.59599999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":8.5410000000000004,"5min":8.5410000000000004,"15min":8.5410000000000004},"last":0.69699999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":0.95799999999999996,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":7.0810000000000004,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.3120000000000001,"5min":0.96699999999999997,"15min":0.875},"min":{"1min":0.60399999999999998,"5min":0.495,"15min":0.495},"max":{"1min":4.1150000000000002,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.80600000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":1.016,"15min":0.83799999999999997},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":8.4990000000000006,"5min":8.4990000000000006,"15min":8.4990000000000006},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.7150000000000001,"5min":0.997,"15min":0.86099999999999999},"min":{"1min":0.58599999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":7.0410000000000004,"5min":7.0410000000000004,"15min":7.0410000000000004},"last":0.75700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":302932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":5.0800000000000001,"5min":1.4490000000000001,"15min":0.77700000000000002},"min":{"1min":0.371,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":29.527999999999999,"5min":29.527999999999999,"15min":29.527999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":6.4829999999999997,"5min":1.762,"15min":0.92800000000000005},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":34.122999999999998,"5min":34.122999999999998,"15min":34.122999999999998},"last":0.54600000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":11.613,"5min":2.8079999999999998,"15min":1.2589999999999999},"min":{"1min":0.36199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":113.149,"5min":113.149,"15min":113.149},"last":0.51900000000000002},{"interface":"front","average":{"1min":4.931,"5min":1.421,"15min":0.79900000000000004},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":41.302,"5min":41.302,"15min":41.302},"last":0.495}]},{"osd":3,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":5.3769999999999998,"5min":1.502,"15min":0.79900000000000004},"min":{"1min":0.44600000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":46.960000000000001,"5min":46.960000000000001,"15min":46.960000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":9.0719999999999992,"5min":2.3010000000000002,"15min":1.1160000000000001},"min":{"1min":0.47699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":115.693,"5min":115.693,"15min":115.693},"last":0.47499999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":4.9199999999999999,"5min":1.6919999999999999,"15min":1.0920000000000001},"min":{"1min":0.59499999999999997,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":24.366,"5min":24.366,"15min":24.366},"last":0.74199999999999999},{"interface":"front","average":{"1min":4.3840000000000003,"5min":1.506,"15min":0.98399999999999999},"min":{"1min":0.61099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":20.489000000000001,"5min":20.489000000000001,"15min":20.489000000000001},"last":0.625}]},{"osd":5,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":4.3129999999999997,"5min":1.486,"15min":0.97199999999999998},"min":{"1min":0.503,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":29.645,"5min":29.645,"15min":29.645},"last":0.64500000000000002},{"interface":"front","average":{"1min":4.577,"5min":1.6020000000000001,"15min":1.075},"min":{"1min":0.59299999999999997,"5min":0.496,"15min":0.496},"max":{"1min":24.277999999999999,"5min":24.277999999999999,"15min":24.277999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":4.5819999999999999,"5min":1.569,"15min":1.05},"min":{"1min":0.60699999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":20.370999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":4.3540000000000001,"5min":1.538,"15min":1.04},"min":{"1min":0.64500000000000002,"5min":0.505,"15min":0.505},"max":{"1min":29.690000000000001,"5min":29.690000000000001,"15min":29.690000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:08:50 2024","interfaces":[{"interface":"back","average":{"1min":3.6520000000000001,"5min":1.421,"15min":1.0289999999999999},"min":{"1min":0.53100000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":20.446999999999999,"5min":20.446999999999999,"15min":20.446999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":4.556,"5min":1.637,"15min":1.123},"min":{"1min":0.57699999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":29.785,"5min":29.785,"15min":29.785},"last":0.80000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":724992,"data_stored":302966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.5489999999999999,"5min":1.1699999999999999,"15min":0.84999999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":39.948,"5min":39.948,"15min":39.948},"last":0.30399999999999999},{"interface":"front","average":{"1min":2.8610000000000002,"5min":1.1000000000000001,"15min":0.86799999999999999},"min":{"1min":0.36799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":24.824000000000002,"5min":24.824000000000002,"15min":24.824000000000002},"last":0.32400000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.79,"5min":1.224,"15min":0.88200000000000001},"min":{"1min":0.30099999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":49.229999999999997,"5min":49.229999999999997,"15min":49.229999999999997},"last":0.375},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.71799999999999997,"15min":0.73599999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":10.349,"5min":10.349,"15min":10.349},"last":0.40699999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.9140000000000001,"5min":1.298,"15min":0.93799999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":25.427,"5min":25.427,"15min":25.427},"last":0.36699999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.879,"15min":0.86699999999999999},"min":{"1min":0.42699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.7949999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.52000000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.2240000000000002,"5min":1.1759999999999999,"15min":1.0780000000000001},"min":{"1min":0.46200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":18.814,"5min":18.814,"15min":18.814},"last":0.495},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.94699999999999995,"15min":0.95799999999999996},"min":{"1min":0.54700000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.9259999999999999,"5min":12.997,"15min":12.997},"last":0.54800000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.1360000000000001,"5min":1.1160000000000001,"15min":0.97399999999999998},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":18.736999999999998,"5min":18.736999999999998,"15min":18.736999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":1.0309999999999999,"15min":1.085},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":2.3180000000000001,"5min":13.054,"15min":13.054},"last":0.626}]},{"osd":6,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2,"5min":1.105,"15min":1.0289999999999999},"min":{"1min":0.51300000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":18.777000000000001,"5min":18.777000000000001,"15min":18.777000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.0509999999999999,"15min":1.044},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":11.712999999999999,"5min":11.712999999999999,"15min":11.712999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.47099999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":11.800000000000001,"5min":11.800000000000001,"15min":11.800000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.99299999999999999,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.60799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":2.4159999999999999,"5min":13.089,"15min":13.089},"last":0.65100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36640,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335200,"statfs":{"total":96636764160,"available":96599244800,"internally_reserved":0,"allocated":1331200,"data_stored":895438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.7349999999999999,"5min":1.2869999999999999,"15min":1.0169999999999999},"min":{"1min":0.33600000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":31.977,"5min":31.977,"15min":31.977},"last":0.374},{"interface":"front","average":{"1min":3.694,"5min":1.1220000000000001,"15min":0.68600000000000005},"min":{"1min":0.39000000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":36.145000000000003,"5min":36.145000000000003,"15min":36.145000000000003},"last":0.44900000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.234,"5min":1.0549999999999999,"15min":0.68799999999999994},"min":{"1min":0.33500000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":41.387999999999998,"5min":41.387999999999998,"15min":41.387999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":3.855,"5min":1.232,"15min":0.85599999999999998},"min":{"1min":0.33400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":33.219999999999999,"5min":33.219999999999999,"15min":33.219999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.9350000000000001,"5min":1.1639999999999999,"15min":1.018},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":31.943999999999999,"5min":31.943999999999999,"15min":31.943999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.984,"5min":0.84199999999999997,"15min":0.65600000000000003},"min":{"1min":0.495,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":9.5640000000000001,"5min":9.5640000000000001,"15min":9.5640000000000001},"last":0.55200000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":2.6309999999999998,"5min":1.1679999999999999,"15min":1.0129999999999999},"min":{"1min":0.38300000000000001,"5min":0.372,"15min":0.372},"max":{"1min":32.932000000000002,"5min":32.932000000000002,"15min":32.932000000000002},"last":0.78500000000000003},{"interface":"front","average":{"1min":3.0470000000000002,"5min":1.2689999999999999,"15min":1.0660000000000001},"min":{"1min":0.56699999999999995,"5min":0.44,"15min":0.44},"max":{"1min":31.782,"5min":31.782,"15min":31.782},"last":0.93899999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":3.1259999999999999,"5min":1.3160000000000001,"15min":1.1000000000000001},"min":{"1min":0.61699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":31.858000000000001,"5min":31.858000000000001,"15min":31.858000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":2.2240000000000002,"5min":1.111,"15min":1.0189999999999999},"min":{"1min":0.51400000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":33.151000000000003,"5min":33.151000000000003,"15min":33.151000000000003},"last":0.97499999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.8799999999999999,"5min":0.999,"15min":0.88300000000000001},"min":{"1min":0.58599999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":9.6440000000000001,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":2.242,"5min":1.1259999999999999,"15min":1.036},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":33.098999999999997,"5min":33.098999999999997,"15min":33.098999999999997},"last":0.92500000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:08:51 2024","interfaces":[{"interface":"back","average":{"1min":1.837,"5min":0.97499999999999998,"15min":0.85299999999999998},"min":{"1min":0.60999999999999999,"5min":0.501,"15min":0.501},"max":{"1min":9.6910000000000007,"5min":9.6910000000000007,"15min":9.6910000000000007},"last":0.95799999999999996},{"interface":"front","average":{"1min":3.044,"5min":1.3009999999999999,"15min":1.115},"min":{"1min":0.60699999999999998,"5min":0.504,"15min":0.504},"max":{"1min":31.905999999999999,"5min":31.905999999999999,"15min":31.905999999999999},"last":0.83999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:09:23.183 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:09:23.183 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:09:23.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 Processor -- start 2024-01-15T18:09:23.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 -- start start 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7effc81594b0 0x7effc80b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7effc80b3d80 0x7effc80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc806d080 0x7effc806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80cd5b0 con 0x7effc81594b0 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc806da00 con 0x7effc80b3d80 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effcce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc80cdf80 con 0x7effc806d080 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7effc81594b0 0x7effc80b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7effc81594b0 0x7effc80b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42350/0 (socket says 172.21.15.143:42350) 2024-01-15T18:09:23.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 -- 172.21.15.143:0/1335226334 learned_addr learned my addr 172.21.15.143:0/1335226334 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5400640 1 --2- 172.21.15.143:0/1335226334 >> v2:172.21.15.204:3300/0 conn(0x7effc80b3d80 0x7effc80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc6800640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc806d080 0x7effc806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 -- 172.21.15.143:0/1335226334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc806d080 msgr2=0x7effc806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc806d080 0x7effc806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 -- 172.21.15.143:0/1335226334 >> v2:172.21.15.204:3300/0 conn(0x7effc80b3d80 msgr2=0x7effc80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 --2- 172.21.15.143:0/1335226334 >> v2:172.21.15.204:3300/0 conn(0x7effc80b3d80 0x7effc80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc5e00640 1 -- 172.21.15.143:0/1335226334 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80b2ec0 con 0x7effc81594b0 2024-01-15T18:09:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.358+0000 7effc6800640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc806d080 0x7effc806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effc5e00640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc81594b0 0x7effc80b27b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7effc80b43f0 tx=0x7effb801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ae90d3ce93b6be server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effc4a00640 1 -- 172.21.15.143:0/1335226334 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8004230 con 0x7effc81594b0 2024-01-15T18:09:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effc4a00640 1 -- 172.21.15.143:0/1335226334 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb80a7050 con 0x7effc81594b0 2024-01-15T18:09:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effc4a00640 1 -- 172.21.15.143:0/1335226334 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8003620 con 0x7effc81594b0 2024-01-15T18:09:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 -- 172.21.15.143:0/1335226334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc81594b0 msgr2=0x7effc80b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc81594b0 0x7effc80b27b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7effc80b43f0 tx=0x7effb801c500 comp rx=0 tx=0).stop 2024-01-15T18:09:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 -- 172.21.15.143:0/1335226334 shutdown_connections 2024-01-15T18:09:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc806d080 0x7effc806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 --2- 172.21.15.143:0/1335226334 >> v2:172.21.15.204:3300/0 conn(0x7effc80b3d80 0x7effc80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 --2- 172.21.15.143:0/1335226334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc81594b0 0x7effc80b27b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 -- 172.21.15.143:0/1335226334 >> 172.21.15.143:0/1335226334 conn(0x7effc806a890 msgr2=0x7effc806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 -- 172.21.15.143:0/1335226334 shutdown_connections 2024-01-15T18:09:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.362+0000 7effcce00640 1 -- 172.21.15.143:0/1335226334 wait complete. 2024-01-15T18:09:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 Processor -- start 2024-01-15T18:09:23.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 -- start start 2024-01-15T18:09:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 0x7effc814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effc80b3d80 0x7effc81626e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc81594b0 0x7effc8162c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc81669d0 con 0x7effc806d080 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc81517b0 con 0x7effc80b3d80 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc8163250 con 0x7effc81594b0 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 0x7effc814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 0x7effc814c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42362/0 (socket says 172.21.15.143:42362) 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 -- 172.21.15.143:0/3852912957 learned_addr learned my addr 172.21.15.143:0/3852912957 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5400640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effc80b3d80 0x7effc81626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc6800640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc81594b0 0x7effc8162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 -- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc81594b0 msgr2=0x7effc8162c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc81594b0 0x7effc8162c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 -- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effc80b3d80 msgr2=0x7effc81626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effc80b3d80 0x7effc81626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 -- 172.21.15.143:0/3852912957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb80094d0 con 0x7effc806d080 2024-01-15T18:09:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc6800640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc81594b0 0x7effc8162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effc5e00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 0x7effc814c7a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7effb80ae040 tx=0x7effb80192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8003bf0 con 0x7effc806d080 2024-01-15T18:09:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8003d70 con 0x7effc806d080 2024-01-15T18:09:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb80ab680 con 0x7effc806d080 2024-01-15T18:09:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc8163530 con 0x7effc806d080 2024-01-15T18:09:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.366+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc814a750 con 0x7effc806d080 2024-01-15T18:09:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.370+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc8163720 con 0x7effc806d080 2024-01-15T18:09:23.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.370+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7effb80b2040 con 0x7effc806d080 2024-01-15T18:09:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.370+0000 7effbe000640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7effa0066e90 0x7effa0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.370+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7effb811e1d0 con 0x7effc806d080 2024-01-15T18:09:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.370+0000 7effc5400640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7effa0066e90 0x7effa0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.370+0000 7effc5400640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7effa0066e90 0x7effa0069350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7effb0009b60 tx=0x7effb0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.374+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7effb80ead30 con 0x7effc806d080 2024-01-15T18:09:23.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.578+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7effc8067500 con 0x7effc806d080 2024-01-15T18:09:23.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.582+0000 7effbe000640 1 -- 172.21.15.143:0/3852912957 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7effb80ee350 con 0x7effc806d080 2024-01-15T18:09:23.587 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:09:23.587 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:09:23.587 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:09:23.587 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:09:23.587 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:09:23.588 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:09:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7effa0066e90 msgr2=0x7effa0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7effa0066e90 0x7effa0069350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7effb0009b60 tx=0x7effb0009190 comp rx=0 tx=0).stop 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 msgr2=0x7effc814c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 0x7effc814c7a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7effb80ae040 tx=0x7effb80192a0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 shutdown_connections 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7effa0066e90 0x7effa0069350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7effc81594b0 0x7effc8162c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7effc80b3d80 0x7effc81626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 --2- 172.21.15.143:0/3852912957 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7effc806d080 0x7effc814c7a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 >> 172.21.15.143:0/3852912957 conn(0x7effc806a890 msgr2=0x7effc815a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 shutdown_connections 2024-01-15T18:09:23.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.586+0000 7effcce00640 1 -- 172.21.15.143:0/3852912957 wait complete. 2024-01-15T18:09:23.610 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:09:23.698 INFO:tasks.ceph.mds.c.smithi143.stderr:2024-01-15T18:09:23.694+0000 7f5a77800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:09:23.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 Processor -- start 2024-01-15T18:09:23.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 -- start start 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9790071bb0 0x7f9790072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f979015ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97900ceb50 con 0x7f9790143780 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9790072570 con 0x7f9790071bb0 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f9794e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979015c110 con 0x7f979015c5a0 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9790071bb0 0x7f9790072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9790071bb0 0x7f9790072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42378/0 (socket says 172.21.15.143:42378) 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 -- 172.21.15.143:0/589327009 learned_addr learned my addr 172.21.15.143:0/589327009 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:23.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978e800640 1 --2- 172.21.15.143:0/589327009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f979015ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 -- 172.21.15.143:0/589327009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 msgr2=0x7f979015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:23.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 --2- 172.21.15.143:0/589327009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f979015ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 -- 172.21.15.143:0/589327009 >> v1:172.21.15.204:6789/0 conn(0x7f9790143780 legacy=0x7f97900cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.742+0000 7f978d400640 1 -- 172.21.15.143:0/589327009 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979015b7e0 con 0x7f9790071bb0 2024-01-15T18:09:23.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f978d400640 1 --2- 172.21.15.143:0/589327009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790071bb0 0x7f9790072030 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9778009870 tx=0x7f977801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f28d566e702ccfb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f978ca00640 1 -- 172.21.15.143:0/589327009 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780a9400 con 0x7f9790071bb0 2024-01-15T18:09:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f978ca00640 1 -- 172.21.15.143:0/589327009 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97780a9560 con 0x7f9790071bb0 2024-01-15T18:09:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f978ca00640 1 -- 172.21.15.143:0/589327009 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780a98f0 con 0x7f9790071bb0 2024-01-15T18:09:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 -- 172.21.15.143:0/589327009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790071bb0 msgr2=0x7f9790072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 --2- 172.21.15.143:0/589327009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790071bb0 0x7f9790072030 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9778009870 tx=0x7f977801c540 comp rx=0 tx=0).stop 2024-01-15T18:09:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 -- 172.21.15.143:0/589327009 shutdown_connections 2024-01-15T18:09:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 --2- 172.21.15.143:0/589327009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f979015ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 --2- 172.21.15.143:0/589327009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790071bb0 0x7f9790072030 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 -- 172.21.15.143:0/589327009 >> 172.21.15.143:0/589327009 conn(0x7f979006c3f0 msgr2=0x7f979006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 -- 172.21.15.143:0/589327009 shutdown_connections 2024-01-15T18:09:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.746+0000 7f9794e00640 1 -- 172.21.15.143:0/589327009 wait complete. 2024-01-15T18:09:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 Processor -- start 2024-01-15T18:09:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 -- start start 2024-01-15T18:09:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9790071bb0 0x7f979013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 0x7f97901590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f9790133430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790132fa0 con 0x7f9790143780 2024-01-15T18:09:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790132390 con 0x7f9790071bb0 2024-01-15T18:09:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9790132670 con 0x7f979015c5a0 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 0x7f97901590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 0x7f97901590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42394/0 (socket says 172.21.15.143:42394) 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 -- 172.21.15.143:0/4261765811 learned_addr learned my addr 172.21.15.143:0/4261765811 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978de00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9790071bb0 0x7f979013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978e800640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f9790133430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 -- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 msgr2=0x7f9790133430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f9790133430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 -- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9790071bb0 msgr2=0x7f979013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9790071bb0 0x7f979013fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 -- 172.21.15.143:0/4261765811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780094d0 con 0x7f9790143780 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f978d400640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 0x7f97901590b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f977801a170 tx=0x7f977801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780a9400 con 0x7f9790143780 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97780b1800 con 0x7f9790143780 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9790132950 con 0x7f9790143780 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780b1b70 con 0x7f9790143780 2024-01-15T18:09:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.750+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97901401f0 con 0x7f9790143780 2024-01-15T18:09:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.754+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9790133150 con 0x7f9790143780 2024-01-15T18:09:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.754+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f97780ba3a0 con 0x7f9790143780 2024-01-15T18:09:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.754+0000 7f9786000640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9768066e90 0x7f9768069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.754+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f977811f230 con 0x7f9790143780 2024-01-15T18:09:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.754+0000 7f978de00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9768066e90 0x7f9768069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.754+0000 7f978de00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9768066e90 0x7f9768069350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9780000f70 tx=0x7f9780009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.758+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9790133150 con 0x7f9790143780 2024-01-15T18:09:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.942+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f979013ea00 con 0x7f9790143780 2024-01-15T18:09:23.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.942+0000 7f9786000640 1 -- 172.21.15.143:0/4261765811 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f97780a6030 con 0x7f9790143780 2024-01-15T18:09:23.949 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:23.949 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3162517776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3162517776},{"type":"v1","addr":"172.21.15.204:6834","nonce":3162517776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/348084006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":348084006},{"type":"v1","addr":"172.21.15.204:6835","nonce":348084006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2759637098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2759637098},{"type":"v1","addr":"172.21.15.143:6835","nonce":2759637098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:09:18.103084+0000","modified":"2024-01-15T18:09:21.123943+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6336":{"gid":6336,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/3767686465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":3767686465},{"type":"v1","addr":"172.21.15.143:6837","nonce":3767686465}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-01-15T18:09:23.949 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 35 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9768066e90 msgr2=0x7f9768069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9768066e90 0x7f9768069350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9780000f70 tx=0x7f9780009040 comp rx=0 tx=0).stop 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 msgr2=0x7f97901590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 0x7f97901590b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f977801a170 tx=0x7f977801ad80 comp rx=0 tx=0).stop 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 shutdown_connections 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9768066e90 0x7f9768069350 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f979015c5a0 0x7f9790133430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9790143780 0x7f97901590b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 --2- 172.21.15.143:0/4261765811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9790071bb0 0x7f979013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 >> 172.21.15.143:0/4261765811 conn(0x7f979006c3f0 msgr2=0x7f979013e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 shutdown_connections 2024-01-15T18:09:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:23.946+0000 7f9794e00640 1 -- 172.21.15.143:0/4261765811 wait complete. 2024-01-15T18:09:24.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:09:18.103084+0000', 'modified': '2024-01-15T18:09:21.123943+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6336}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6336': {'gid': 6336, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6837/3767686465', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 3767686465}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 3767686465}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:09:24.373 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:09:24.374 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-15T18:09:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.506+0000 7f0698800640 1 Processor -- start 2024-01-15T18:09:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.506+0000 7f0698800640 1 -- start start 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0698800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f069015d3f0 0x7f06900cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0698800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f069006bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0698800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f06900b0490 0x7f069006c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0698800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06900cb480 con 0x7f06900b0490 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0698800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06900cb6d0 con 0x7f069015d3f0 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0698800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06900cb9b0 con 0x7f06900d0050 2024-01-15T18:09:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f06900b0490 0x7f069006c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f06900b0490 0x7f069006c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42400/0 (socket says 172.21.15.143:42400) 2024-01-15T18:09:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 -- 172.21.15.143:0/3619282974 learned_addr learned my addr 172.21.15.143:0/3619282974 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0695400640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f069006bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0695e00640 1 --2- 172.21.15.143:0/3619282974 >> v2:172.21.15.204:3300/0 conn(0x7f069015d3f0 0x7f06900cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 -- 172.21.15.143:0/3619282974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 msgr2=0x7f069006bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f069006bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 -- 172.21.15.143:0/3619282974 >> v2:172.21.15.204:3300/0 conn(0x7f069015d3f0 msgr2=0x7f06900cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 --2- 172.21.15.143:0/3619282974 >> v2:172.21.15.204:3300/0 conn(0x7f069015d3f0 0x7f06900cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 -- 172.21.15.143:0/3619282974 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900cbc90 con 0x7f06900b0490 2024-01-15T18:09:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0695400640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f069006bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:24.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.510+0000 7f0696800640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06900b0490 0x7f069006c460 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0690071420 tx=0x7f068801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747b6628d0f44294 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0694a00640 1 -- 172.21.15.143:0/3619282974 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880a5840 con 0x7f06900b0490 2024-01-15T18:09:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0694a00640 1 -- 172.21.15.143:0/3619282974 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06880a59a0 con 0x7f06900b0490 2024-01-15T18:09:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0694a00640 1 -- 172.21.15.143:0/3619282974 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880a5d30 con 0x7f06900b0490 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 -- 172.21.15.143:0/3619282974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06900b0490 msgr2=0x7f069006c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06900b0490 0x7f069006c460 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f0690071420 tx=0x7f068801c9a0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 -- 172.21.15.143:0/3619282974 shutdown_connections 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f06900b0490 0x7f069006c460 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 --2- 172.21.15.143:0/3619282974 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f069006bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 --2- 172.21.15.143:0/3619282974 >> v2:172.21.15.204:3300/0 conn(0x7f069015d3f0 0x7f06900cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 -- 172.21.15.143:0/3619282974 >> 172.21.15.143:0/3619282974 conn(0x7f06900d0e30 msgr2=0x7f06900d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 -- 172.21.15.143:0/3619282974 shutdown_connections 2024-01-15T18:09:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 -- 172.21.15.143:0/3619282974 wait complete. 2024-01-15T18:09:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 Processor -- start 2024-01-15T18:09:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.514+0000 7f0698800640 1 -- start start 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06900b0490 0x7f06900c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f06901505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 0x7f069015c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0690150b10 con 0x7f069015d3f0 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06900c62f0 con 0x7f06900b0490 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06900c65d0 con 0x7f06900d0050 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 0x7f069015c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 0x7f069015c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42404/0 (socket says 172.21.15.143:42404) 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 -- 172.21.15.143:0/4180300444 learned_addr learned my addr 172.21.15.143:0/4180300444 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0695e00640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06900b0490 0x7f06900c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0695400640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f06901505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 -- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 msgr2=0x7f06901505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f06901505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 -- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06900b0490 msgr2=0x7f06900c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06900b0490 0x7f06900c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 -- 172.21.15.143:0/4180300444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0688009b30 con 0x7f069015d3f0 2024-01-15T18:09:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0696800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 0x7f069015c560 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f06880af040 tx=0x7f0688019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f068e000640 1 -- 172.21.15.143:0/4180300444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880b8670 con 0x7f069015d3f0 2024-01-15T18:09:24.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f068e000640 1 -- 172.21.15.143:0/4180300444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06880b87f0 con 0x7f069015d3f0 2024-01-15T18:09:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f068e000640 1 -- 172.21.15.143:0/4180300444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880b8ba0 con 0x7f069015d3f0 2024-01-15T18:09:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06900c68b0 con 0x7f069015d3f0 2024-01-15T18:09:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06900c6e70 con 0x7f069015d3f0 2024-01-15T18:09:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.518+0000 7f068e000640 1 -- 172.21.15.143:0/4180300444 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f06880b8d00 con 0x7f069015d3f0 2024-01-15T18:09:24.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f068e000640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0668066e90 0x7f0668069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f068e000640 1 -- 172.21.15.143:0/4180300444 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f06881205f0 con 0x7f069015d3f0 2024-01-15T18:09:24.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695e00640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0668066e90 0x7f0668069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 Processor -- start 2024-01-15T18:09:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 -- start start 2024-01-15T18:09:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695e00640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0668066e90 0x7f0668069350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0680007940 tx=0x7f0680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80935070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc808eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc808eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc80755130 con 0x55fc809355b0 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc807429a0 con 0x55fc80934c90 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc8073fed0 con 0x55fc808eb320 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc808eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc808eade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42420/0 (socket says 172.21.15.143:42420) 2024-01-15T18:09:24.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 -- 172.21.15.143:0/3231271802 learned_addr learned my addr 172.21.15.143:0/3231271802 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695e00640 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80935070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0696800640 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc808eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 -- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 msgr2=0x55fc808eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc808eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 -- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 msgr2=0x55fc80935070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80935070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 -- 172.21.15.143:0/3231271802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc80433890 con 0x55fc809355b0 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0695400640 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc808eade0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f067800d8c0 tx=0x7f067800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9400deac3a3506b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0686a00640 1 -- 172.21.15.143:0/3231271802 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800f9c0 con 0x55fc809355b0 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0686a00640 1 -- 172.21.15.143:0/3231271802 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067800c030 con 0x55fc809355b0 2024-01-15T18:09:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.522+0000 7f0686a00640 1 -- 172.21.15.143:0/3231271802 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800fd20 con 0x55fc809355b0 2024-01-15T18:09:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- 172.21.15.143:0/3231271802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 msgr2=0x55fc808eade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc808eade0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f067800d8c0 tx=0x7f067800dd80 comp rx=0 tx=0).stop 2024-01-15T18:09:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- 172.21.15.143:0/3231271802 shutdown_connections 2024-01-15T18:09:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc808eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc808eade0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- 172.21.15.143:0/3231271802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80935070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- 172.21.15.143:0/3231271802 >> 172.21.15.143:0/3231271802 conn(0x55fc808e72d0 msgr2=0x55fc808e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- 172.21.15.143:0/3231271802 shutdown_connections 2024-01-15T18:09:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- 172.21.15.143:0/3231271802 wait complete. 2024-01-15T18:09:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- start start 2024-01-15T18:09:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 Processor -- start 2024-01-15T18:09:24.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc80919ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc80934c90 0x55fc80919ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc809355b0 0x55fc8091a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc8073b7c0 con 0x55fc808eb320 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc806f3bf0 con 0x55fc809355b0 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc806f9a90 con 0x55fc80934c90 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f069bc42000 10 client.0 _ll_register_callbacks cb 0x55fc808d63c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc80919ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc80919ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42424/0 (socket says 172.21.15.143:42424) 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 -- 172.21.15.143:0/459233764 learned_addr learned my addr 172.21.15.143:0/459233764 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695400640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc80934c90 0x55fc80919ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0696800640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc809355b0 0x55fc8091a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 -- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc80934c90 msgr2=0x55fc80919ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc80934c90 0x55fc80919ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 -- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc809355b0 msgr2=0x55fc8091a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc809355b0 0x55fc8091a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 -- 172.21.15.143:0/459233764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067800e040 con 0x55fc808eb320 2024-01-15T18:09:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f0695e00640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc80919ab0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f06800167d0 tx=0x7f06800046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcec9ce830b42a48 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f067f400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.526+0000 7f067f400640 1 -- 172.21.15.143:0/459233764 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680014880 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 1 -- 172.21.15.143:0/459233764 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f06800149e0 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc806fd470 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 1 -- 172.21.15.143:0/459233764 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0680014be0 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 10 client.6471 fetch_fsmap learned FSMap version 35 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55fc806fc190 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 1 -- 172.21.15.143:0/459233764 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc806fd470 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 1 -- 172.21.15.143:0/459233764 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f068002b440 con 0x55fc808eb320 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 10 client.6471 fetch_fsmap finished waiting for FSMap version 35 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 10 client.6471 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 20 client.6471 populate_metadata read hostname 'smithi143' 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc8091af40 0x55fc80946f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 4 client.6471 mds_command: new command op to 6336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 --> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc80700b50 con 0x55fc8091af40 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f0695400640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc8091af40 0x55fc80946f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f0695400640 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc8091af40 0x55fc80946f50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989a8d5633028695 server_cookie=a0a938c208f86011 in_seq=0 out_seq=0 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 10 client.6471 ms_handle_connect on v2:172.21.15.143:6836/3767686465 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 1 -- 172.21.15.143:0/459233764 <== mds.0 v2:172.21.15.143:6836/3767686465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc80700b50 con 0x55fc8091af40 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 10 client.6471 handle_command_reply: tid=0 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 client.6471 shutdown 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 msgr2=0x55fc80919ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc80919ab0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f06800167d0 tx=0x7f06800046d0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 shutdown_connections 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc8091af40 0x55fc80946f50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc809355b0 0x55fc8091a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc80934c90 0x55fc80919ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 --2- 172.21.15.143:0/459233764 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc80919ab0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f067f400640 0 client.6471 ms_handle_reset on v2:172.21.15.143:6836/3767686465 2024-01-15T18:09:24.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 >> 172.21.15.143:0/459233764 conn(0x55fc808e72d0 msgr2=0x55fc808e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 shutdown_connections 2024-01-15T18:09:24.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 1 -- 172.21.15.143:0/459233764 wait complete. 2024-01-15T18:09:24.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.530+0000 7f069bc42000 20 client.6471 trim_cache size 0 max 16384 2024-01-15T18:09:24.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 Processor -- start 2024-01-15T18:09:24.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 -- start start 2024-01-15T18:09:24.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc809333e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80933920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc809355b0 0x55fc808eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc803f4940 con 0x55fc808eb320 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc806f76f0 con 0x55fc80934c90 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc808f0fb0 con 0x55fc809355b0 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f0695e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc809333e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f0695e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc809333e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42440/0 (socket says 172.21.15.143:42440) 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.546+0000 7f0695e00640 1 -- 172.21.15.143:0/1356314908 learned_addr learned my addr 172.21.15.143:0/1356314908 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695400640 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80933920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0696800640 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc809355b0 0x55fc808eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 -- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc809355b0 msgr2=0x55fc808eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc809355b0 0x55fc808eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 -- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 msgr2=0x55fc80933920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80933920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 -- 172.21.15.143:0/1356314908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc808ee9b0 con 0x55fc808eb320 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0696800640 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc809355b0 0x55fc808eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:24.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc809333e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0680000c10 tx=0x7f0680023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837293a5a0aeb0b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0687400640 1 -- 172.21.15.143:0/1356314908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680014430 con 0x55fc808eb320 2024-01-15T18:09:24.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0687400640 1 -- 172.21.15.143:0/1356314908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680006a30 con 0x55fc808eb320 2024-01-15T18:09:24.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0687400640 1 -- 172.21.15.143:0/1356314908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680007250 con 0x55fc808eb320 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- 172.21.15.143:0/1356314908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 msgr2=0x55fc809333e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc809333e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0680000c10 tx=0x7f0680023000 comp rx=0 tx=0).stop 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- 172.21.15.143:0/1356314908 shutdown_connections 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc809355b0 0x55fc808eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc80933920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- 172.21.15.143:0/1356314908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc808eb320 0x55fc809333e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- 172.21.15.143:0/1356314908 >> 172.21.15.143:0/1356314908 conn(0x55fc808e72d0 msgr2=0x55fc808e96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- 172.21.15.143:0/1356314908 shutdown_connections 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- 172.21.15.143:0/1356314908 wait complete. 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- start start 2024-01-15T18:09:24.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 Processor -- start 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc80947fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc809484e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc809529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc80737130 con 0x55fc809355b0 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc80734d40 con 0x55fc80934c90 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc806f8e10 con 0x55fc808eb320 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f069bc42000 10 client.0 _ll_register_callbacks cb 0x55fc80789c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0696800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc809529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0696800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc809529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42442/0 (socket says 172.21.15.143:42442) 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0695e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc80947fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.550+0000 7f0696800640 1 -- 172.21.15.143:0/3586569070 learned_addr learned my addr 172.21.15.143:0/3586569070 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0695400640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc809484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0696800640 1 -- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 msgr2=0x55fc80947fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0696800640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc80947fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0696800640 1 -- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 msgr2=0x55fc809484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0696800640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc809484e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0696800640 1 -- 172.21.15.143:0/3586569070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc806fc190 con 0x55fc809355b0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0695e00640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc80947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0696800640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc809529e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f068801ce80 tx=0x7f0688120040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef22dea9f8399b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 1 -- 172.21.15.143:0/3586569070 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068811e120 con 0x55fc809355b0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 1 -- 172.21.15.143:0/3586569070 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f06880f9040 con 0x55fc809355b0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc808ef890 con 0x55fc809355b0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 1 -- 172.21.15.143:0/3586569070 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06880f0230 con 0x55fc809355b0 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 10 client.6477 fetch_fsmap learned FSMap version 35 2024-01-15T18:09:24.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55fc806e2490 con 0x55fc809355b0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 1 -- 172.21.15.143:0/3586569070 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc808ef890 con 0x55fc809355b0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 1 -- 172.21.15.143:0/3586569070 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0688005d20 con 0x55fc809355b0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 10 client.6477 fetch_fsmap finished waiting for FSMap version 35 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 10 client.6477 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 20 client.6477 populate_metadata read hostname 'smithi143' 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc809535b0 0x55fc8099de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 4 client.6477 mds_command: new command op to 6336 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 --> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fc8073a110 con 0x55fc809535b0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0695e00640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc809535b0 0x55fc8099de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 10 client.6477 ms_handle_connect on v2:172.21.15.143:6836/3767686465 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f0695e00640 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc809535b0 0x55fc8099de50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ff605de2234fe1 server_cookie=30859d4ce61e9454 in_seq=0 out_seq=0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 1 -- 172.21.15.143:0/3586569070 <== mds.0 v2:172.21.15.143:6836/3767686465 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fc8073a110 con 0x55fc809535b0 2024-01-15T18:09:24.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 10 client.6477 handle_command_reply: tid=0 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 client.6477 shutdown 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 msgr2=0x55fc809529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc809529e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f068801ce80 tx=0x7f0688120040 comp rx=0 tx=0).stop 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 shutdown_connections 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:6836/3767686465,v1:172.21.15.143:6837/3767686465] conn(0x55fc809535b0 0x55fc8099de50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc809355b0 0x55fc809529e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fc80934c90 0x55fc809484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f067f400640 0 client.6477 ms_handle_reset on v2:172.21.15.143:6836/3767686465 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 --2- 172.21.15.143:0/3586569070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fc808eb320 0x55fc80947fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 >> 172.21.15.143:0/3586569070 conn(0x55fc808e72d0 msgr2=0x55fc808e9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 shutdown_connections 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 1 -- 172.21.15.143:0/3586569070 wait complete. 2024-01-15T18:09:24.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.554+0000 7f069bc42000 20 client.6477 trim_cache size 0 max 16384 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6336, 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8193768730000004, 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 35, 2024-01-15T18:09:24.565 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 60, 2024-01-15T18:09:24.566 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 60, 2024-01-15T18:09:24.566 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.731136856999999 2024-01-15T18:09:24.566 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:09:24.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0668066e90 msgr2=0x7f0668069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0668066e90 0x7f0668069350 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0680007940 tx=0x7f0680008040 comp rx=0 tx=0).stop 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 msgr2=0x7f069015c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 0x7f069015c560 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f06880af040 tx=0x7f0688019740 comp rx=0 tx=0).stop 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 shutdown_connections 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0668066e90 0x7f0668069350 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f069015d3f0 0x7f069015c560 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06900d0050 0x7f06901505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 --2- 172.21.15.143:0/4180300444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06900b0490 0x7f06900c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 >> 172.21.15.143:0/4180300444 conn(0x7f06900d0e30 msgr2=0x7f06900b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 shutdown_connections 2024-01-15T18:09:24.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.558+0000 7f0698800640 1 -- 172.21.15.143:0/4180300444 wait complete. 2024-01-15T18:09:24.581 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:09:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 Processor -- start 2024-01-15T18:09:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 -- start start 2024-01-15T18:09:24.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd90406a490 0x7fd9040ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd904071540 0x7fd9040719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd904161250 con 0x7fd9040cf410 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd904161530 con 0x7fd904071540 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90c400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd904161810 con 0x7fd90406a490 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd904071540 0x7fd9040719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd909000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd9040cf410 0x7fd90406a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44908/0 (socket says 172.21.15.143:44908) 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd909000640 1 -- 172.21.15.143:0/1877658890 learned_addr learned my addr 172.21.15.143:0/1877658890 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd909a00640 1 --2- 172.21.15.143:0/1877658890 >> v2:172.21.15.204:3300/0 conn(0x7fd90406a490 0x7fd9040ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90a400640 1 -- 172.21.15.143:0/1877658890 >> v2:172.21.15.204:3300/0 conn(0x7fd90406a490 msgr2=0x7fd9040ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90a400640 1 --2- 172.21.15.143:0/1877658890 >> v2:172.21.15.204:3300/0 conn(0x7fd90406a490 0x7fd9040ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90a400640 1 -- 172.21.15.143:0/1877658890 >> v1:172.21.15.143:6789/0 conn(0x7fd9040cf410 legacy=0x7fd90406a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd90a400640 1 -- 172.21.15.143:0/1877658890 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd904161af0 con 0x7fd904071540 2024-01-15T18:09:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.714+0000 7fd903e00640 1 -- 172.21.15.143:0/1877658890 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220347887 0 0) 0x7fd904161250 con 0x7fd9040cf410 2024-01-15T18:09:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90a400640 1 --2- 172.21.15.143:0/1877658890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd904071540 0x7fd9040719d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd904070e00 tx=0x7fd8fc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bc855a9ebf8c1d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd903e00640 1 -- 172.21.15.143:0/1877658890 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0a5960 con 0x7fd904071540 2024-01-15T18:09:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd903e00640 1 -- 172.21.15.143:0/1877658890 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc0a5ac0 con 0x7fd904071540 2024-01-15T18:09:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd903e00640 1 -- 172.21.15.143:0/1877658890 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0a5e50 con 0x7fd904071540 2024-01-15T18:09:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 -- 172.21.15.143:0/1877658890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd904071540 msgr2=0x7fd9040719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 --2- 172.21.15.143:0/1877658890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd904071540 0x7fd9040719d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd904070e00 tx=0x7fd8fc01c910 comp rx=0 tx=0).stop 2024-01-15T18:09:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 -- 172.21.15.143:0/1877658890 shutdown_connections 2024-01-15T18:09:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 --2- 172.21.15.143:0/1877658890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd904071540 0x7fd9040719d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 --2- 172.21.15.143:0/1877658890 >> v2:172.21.15.204:3300/0 conn(0x7fd90406a490 0x7fd9040ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 -- 172.21.15.143:0/1877658890 >> 172.21.15.143:0/1877658890 conn(0x7fd9040cc7d0 msgr2=0x7fd9040ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 -- 172.21.15.143:0/1877658890 shutdown_connections 2024-01-15T18:09:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.718+0000 7fd90c400640 1 -- 172.21.15.143:0/1877658890 wait complete. 2024-01-15T18:09:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 Processor -- start 2024-01-15T18:09:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 -- start start 2024-01-15T18:09:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 0x7fd9041556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd904071540 0x7fd90415d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9040cf410 0x7fd90406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040b43a0 con 0x7fd90406a490 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd904155c30 con 0x7fd9040cf410 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9040b3a40 con 0x7fd904071540 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 0x7fd9041556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 0x7fd9041556f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42466/0 (socket says 172.21.15.143:42466) 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 -- 172.21.15.143:0/3354725212 learned_addr learned my addr 172.21.15.143:0/3354725212 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90a400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9040cf410 0x7fd90406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909000640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd904071540 0x7fd90415d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 -- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd904071540 msgr2=0x7fd90415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd904071540 0x7fd90415d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 -- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9040cf410 msgr2=0x7fd90406e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9040cf410 0x7fd90406e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 -- 172.21.15.143:0/3354725212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc009a70 con 0x7fd90406a490 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd909a00640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 0x7fd9041556f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400e930 tx=0x7fd8f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009690 con 0x7fd90406a490 2024-01-15T18:09:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f400ca00 con 0x7fd90406a490 2024-01-15T18:09:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9040b3ce0 con 0x7fd90406a490 2024-01-15T18:09:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.722+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400cd70 con 0x7fd90406a490 2024-01-15T18:09:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9040c7d60 con 0x7fd90406a490 2024-01-15T18:09:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd904067500 con 0x7fd90406a490 2024-01-15T18:09:24.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fd8f4013470 con 0x7fd90406a490 2024-01-15T18:09:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd901600640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8dc066e90 0x7fd8dc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd8f4085180 con 0x7fd90406a490 2024-01-15T18:09:24.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd909000640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8dc066e90 0x7fd8dc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:24.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.726+0000 7fd909000640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8dc066e90 0x7fd8dc069350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd9040b10a0 tx=0x7fd8ec007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:24.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.730+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd8f4051bb0 con 0x7fd90406a490 2024-01-15T18:09:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.914+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9040cd3c0 con 0x7fd90406a490 2024-01-15T18:09:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.914+0000 7fd901600640 1 -- 172.21.15.143:0/3354725212 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd8f40551d0 con 0x7fd90406a490 2024-01-15T18:09:24.920 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8dc066e90 msgr2=0x7fd8dc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8dc066e90 0x7fd8dc069350 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd9040b10a0 tx=0x7fd8ec007c10 comp rx=0 tx=0).stop 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 msgr2=0x7fd9041556f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 0x7fd9041556f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400e930 tx=0x7fd8f400edf0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 shutdown_connections 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8dc066e90 0x7fd8dc069350 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9040cf410 0x7fd90406e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd904071540 0x7fd90415d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 --2- 172.21.15.143:0/3354725212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd90406a490 0x7fd9041556f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 >> 172.21.15.143:0/3354725212 conn(0x7fd9040cc7d0 msgr2=0x7fd9040adb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 shutdown_connections 2024-01-15T18:09:24.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:24.918+0000 7fd90c400640 1 -- 172.21.15.143:0/3354725212 wait complete. 2024-01-15T18:09:24.942 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:09:24.942 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:09:24.942 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:09:24.942 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:09:24.943 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:24.943 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:09:24.943 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:09:24.943 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:09:24.943 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:09:24.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:24.943 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:09:24.952 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:09:24.953 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42178sec preferred_lft 42178sec 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.954 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:24.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:24.955 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:09:25.018 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:09:25.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.021 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:09:25.080 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:09:25.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.081 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:09:25.081 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:09:25.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:09:25.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:09:25.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:09:25.081 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:09:25.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.169 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:09:25.220 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:09:25.220 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:09:25.221 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:09:25.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.221 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:09:25.284 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:09:25.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:09:25.284 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:25.332 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:09:25.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:09:25.333 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:25.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:09:25.449 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:09:25.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.505 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:09:25.561 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:09:25.561 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:09:25.607 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:09:25.628 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:09:25.678 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:09:25.678 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:09:25.678 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:09:25.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:09:25.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:09:25.681 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:09:25.681 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:09:25.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.682 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:09:25.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:09:25.698+0000 7f824dfce580 -1 init, newargv = 0x562662542230 newargc=15 2024-01-15T18:09:25.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[42049]: starting ceph client 2024-01-15T18:09:25.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[42049]: starting fuse 2024-01-15T18:09:25.712 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:09:25.712 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:09:25.712 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:09:25.712 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:09:25.712 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:09:25.713 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:09:25.714 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:09:25.715 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:09:25.715 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:09:25.715 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:09:25.715 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:09:25.715 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:09:25.715 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:09:25.837 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.42049.asok 2024-01-15T18:09:25.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:09:25.848 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42049.asok 2024-01-15T18:09:25.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:25.848 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42049.asok status 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "42049", 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:09:26.019 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6492, 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 6492 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:09:26.020 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 531697337 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 531697337 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.6492 192.168.0.1:0/531697337", 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/531697337", 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 35, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 60, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:09:26.021 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:09:26.022 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:09:26.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.034 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:09:26.034 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:26.044 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:09:26.044 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:26.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.044 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:09:26.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:09:26.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 Processor -- start 2024-01-15T18:09:26.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 -- start start 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efeb4143780 0x7efeb40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb40ceb50 con 0x7efeb40ce520 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb415c110 con 0x7efeb4143780 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeba400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb415b7e0 con 0x7efeb40722c0 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeb3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efeb4143780 0x7efeb40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeb3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efeb4143780 0x7efeb40cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42476/0 (socket says 172.21.15.143:42476) 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.274+0000 7efeb3400640 1 -- 172.21.15.143:0/1744084885 learned_addr learned my addr 172.21.15.143:0/1744084885 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb3e00640 1 --2- 172.21.15.143:0/1744084885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:26.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb3400640 1 -- 172.21.15.143:0/1744084885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 msgr2=0x7efeb415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:26.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb3400640 1 --2- 172.21.15.143:0/1744084885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb3400640 1 -- 172.21.15.143:0/1744084885 >> v1:172.21.15.204:6789/0 conn(0x7efeb40ce520 legacy=0x7efeb4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:26.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb3400640 1 -- 172.21.15.143:0/1744084885 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb415bac0 con 0x7efeb4143780 2024-01-15T18:09:26.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb3400640 1 --2- 172.21.15.143:0/1744084885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb40cdfe0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efea400bb10 tx=0x7efea401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafaaac031d95eaa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:26.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb2000640 1 -- 172.21.15.143:0/1744084885 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40a7bf0 con 0x7efeb4143780 2024-01-15T18:09:26.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb2000640 1 -- 172.21.15.143:0/1744084885 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea40a7d50 con 0x7efeb4143780 2024-01-15T18:09:26.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.278+0000 7efeb2000640 1 -- 172.21.15.143:0/1744084885 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40af660 con 0x7efeb4143780 2024-01-15T18:09:26.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- 172.21.15.143:0/1744084885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 msgr2=0x7efeb40cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:26.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 --2- 172.21.15.143:0/1744084885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb40cdfe0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efea400bb10 tx=0x7efea401ec10 comp rx=0 tx=0).stop 2024-01-15T18:09:26.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- 172.21.15.143:0/1744084885 shutdown_connections 2024-01-15T18:09:26.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 --2- 172.21.15.143:0/1744084885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 --2- 172.21.15.143:0/1744084885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb40cdfe0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- 172.21.15.143:0/1744084885 >> 172.21.15.143:0/1744084885 conn(0x7efeb406c3f0 msgr2=0x7efeb406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:26.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- 172.21.15.143:0/1744084885 shutdown_connections 2024-01-15T18:09:26.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- 172.21.15.143:0/1744084885 wait complete. 2024-01-15T18:09:26.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 Processor -- start 2024-01-15T18:09:26.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- start start 2024-01-15T18:09:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb4158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb40ce520 0x7efeb413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb413c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb4140ab0 con 0x7efeb4143780 2024-01-15T18:09:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb413fcb0 con 0x7efeb40ce520 2024-01-15T18:09:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeba400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb4159230 con 0x7efeb40722c0 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb413c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb413c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42490/0 (socket says 172.21.15.143:42490) 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb3e00640 1 -- 172.21.15.143:0/2575040142 learned_addr learned my addr 172.21.15.143:0/2575040142 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb3400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb4158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb2a00640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb40ce520 0x7efeb413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb3e00640 1 -- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 msgr2=0x7efeb4158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.282+0000 7efeb3e00640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb4158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeb3e00640 1 -- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb40ce520 msgr2=0x7efeb413d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeb3e00640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb40ce520 0x7efeb413d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeb3e00640 1 -- 172.21.15.143:0/2575040142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea400b770 con 0x7efeb4143780 2024-01-15T18:09:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeb3e00640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb413c3a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efeac007db0 tx=0x7efeac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac014420 con 0x7efeb4143780 2024-01-15T18:09:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac014580 con 0x7efeb4143780 2024-01-15T18:09:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb413ff30 con 0x7efeb4143780 2024-01-15T18:09:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac014930 con 0x7efeb4143780 2024-01-15T18:09:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb41404f0 con 0x7efeb4143780 2024-01-15T18:09:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb40b2110 con 0x7efeb4143780 2024-01-15T18:09:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.286+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7efeac014a90 con 0x7efeb4143780 2024-01-15T18:09:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.290+0000 7efeab400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe8c066e90 0x7efe8c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.290+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efeac085650 con 0x7efeb4143780 2024-01-15T18:09:26.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.290+0000 7efeb3400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe8c066e90 0x7efe8c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.290+0000 7efeb3400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe8c066e90 0x7efe8c069350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efea40b1000 tx=0x7efea40a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.294+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efeb40b2110 con 0x7efeb4143780 2024-01-15T18:09:26.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.474+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efeb40b2110 con 0x7efeb4143780 2024-01-15T18:09:26.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.474+0000 7efeab400640 1 -- 172.21.15.143:0/2575040142 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7efeb40b2110 con 0x7efeb4143780 2024-01-15T18:09:26.480 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe8c066e90 msgr2=0x7efe8c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe8c066e90 0x7efe8c069350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efea40b1000 tx=0x7efea40a9070 comp rx=0 tx=0).stop 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 msgr2=0x7efeb413c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb413c3a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efeac007db0 tx=0x7efeac00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 shutdown_connections 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe8c066e90 0x7efe8c069350 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeb4143780 0x7efeb413c3a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efeb40ce520 0x7efeb413d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 --2- 172.21.15.143:0/2575040142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efeb40722c0 0x7efeb4158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 >> 172.21.15.143:0/2575040142 conn(0x7efeb406c3f0 msgr2=0x7efeb40cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 shutdown_connections 2024-01-15T18:09:26.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:26.478+0000 7efeba400640 1 -- 172.21.15.143:0/2575040142 wait complete. 2024-01-15T18:09:26.502 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:09:26.502 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:09:26.502 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:09:26.502 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:09:26.502 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:26.502 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:09:26.503 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:09:26.503 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:09:26.503 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:09:26.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.503 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:26.513 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42171sec preferred_lft 42171sec 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:26.514 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:09:26.515 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:09:26.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.516 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:09:26.578 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:09:26.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.581 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:09:26.637 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:09:26.637 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:09:26.637 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:09:26.637 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:09:26.637 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:09:26.637 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:09:26.638 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:09:26.638 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:09:26.638 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:09:26.638 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:09:26.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.638 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:09:26.638 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:09:26.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:09:26.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:09:26.638 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:09:26.638 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:09:26.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.732 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:09:26.742 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:09:26.743 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:09:26.743 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:09:26.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:26.743 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:09:26.804 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:09:26.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:09:26.804 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:26.855 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:09:26.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:09:26.856 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:26.907 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:09:26.972 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:09:27.025 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:09:27.026 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:27.027 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:27.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:27.028 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:09:27.083 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:09:27.083 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:09:27.130 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:09:27.152 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:09:27.205 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:09:27.206 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:27.207 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:27.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:27.208 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:09:27.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:09:27.223+0000 7f43f7b48580 -1 init, newargv = 0x55b4e14af190 newargc=15 2024-01-15T18:09:27.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33770]: starting ceph client 2024-01-15T18:09:27.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33770]: starting fuse 2024-01-15T18:09:27.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:27.940 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:28.337 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:09:28.359 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:09:28.417 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:09:28.417 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:09:28.418 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:09:28.419 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:09:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:09:28.420 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:28.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:28.421 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:09:28.471 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:09:28.472 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:09:28.472 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:09:28.473 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:09:28.474 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:09:28.474 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:09:28.474 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:09:28.602 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.33770.asok 2024-01-15T18:09:28.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:09:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:09:28.614 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33770.asok 2024-01-15T18:09:28.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:28.615 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33770.asok status 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "33770", 2024-01-15T18:09:28.784 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 6418, 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 6418 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 803084154 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:09:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 803084154 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.6418 192.168.0.1:0/803084154", 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/803084154", 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 35, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 60, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:09:28.786 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:09:28.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:28.800 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:09:28.800 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:28.809 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:09:28.809 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:28.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:28.809 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:09:28.869 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:09:29.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.014+0000 7fd381600640 1 Processor -- start 2024-01-15T18:09:29.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd381600640 1 -- start start 2024-01-15T18:09:29.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd381600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd37c1607c0 0x7fd37c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd381600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c0cfb30 0x7fd37c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd381600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c0cffd0 con 0x7fd37c06c230 2024-01-15T18:09:29.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd381600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37c06c670 con 0x7fd37c1607c0 2024-01-15T18:09:29.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd381600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0d1ce0 con 0x7fd37c0cfb30 2024-01-15T18:09:29.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd37c06c230 0x7fd37c0cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50382/0 (socket says 172.21.15.143:50382) 2024-01-15T18:09:29.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37b400640 1 -- 172.21.15.143:0/3019097959 learned_addr learned my addr 172.21.15.143:0/3019097959 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:29.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37aa00640 1 --2- 172.21.15.143:0/3019097959 >> v2:172.21.15.204:3300/0 conn(0x7fd37c1607c0 0x7fd37c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37a000640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c0cfb30 0x7fd37c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd379600640 1 -- 172.21.15.143:0/3019097959 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473760285 0 0) 0x7fd37c0cffd0 con 0x7fd37c06c230 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd379600640 1 -- 172.21.15.143:0/3019097959 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003530 con 0x7fd37c06c230 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37aa00640 1 -- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c0cfb30 msgr2=0x7fd37c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37aa00640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c0cfb30 0x7fd37c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37aa00640 1 -- 172.21.15.143:0/3019097959 >> v1:172.21.15.143:6789/0 conn(0x7fd37c06c230 legacy=0x7fd37c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37aa00640 1 -- 172.21.15.143:0/3019097959 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0d02b0 con 0x7fd37c1607c0 2024-01-15T18:09:29.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.018+0000 7fd37a000640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c0cfb30 0x7fd37c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:29.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd37aa00640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c1607c0 0x7fd37c06b7b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0097c0 tx=0x7fd36c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34438d5cacddeb25 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:29.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd379600640 1 -- 172.21.15.143:0/3019097959 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0a8970 con 0x7fd37c1607c0 2024-01-15T18:09:29.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd379600640 1 -- 172.21.15.143:0/3019097959 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c0a8ad0 con 0x7fd37c1607c0 2024-01-15T18:09:29.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd379600640 1 -- 172.21.15.143:0/3019097959 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0a8e60 con 0x7fd37c1607c0 2024-01-15T18:09:29.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 -- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c1607c0 msgr2=0x7fd37c06b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c1607c0 0x7fd37c06b7b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0097c0 tx=0x7fd36c01ec70 comp rx=0 tx=0).stop 2024-01-15T18:09:29.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 -- 172.21.15.143:0/3019097959 shutdown_connections 2024-01-15T18:09:29.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c0cfb30 0x7fd37c06bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 --2- 172.21.15.143:0/3019097959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c1607c0 0x7fd37c06b7b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 -- 172.21.15.143:0/3019097959 >> 172.21.15.143:0/3019097959 conn(0x7fd37c0d0e20 msgr2=0x7fd37c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:29.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 -- 172.21.15.143:0/3019097959 shutdown_connections 2024-01-15T18:09:29.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.022+0000 7fd381600640 1 -- 172.21.15.143:0/3019097959 wait complete. 2024-01-15T18:09:29.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 Processor -- start 2024-01-15T18:09:29.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 -- start start 2024-01-15T18:09:29.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c06c230 0x7fd37c1f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c0cfb30 0x7fd37c157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd37c1607c0 0x7fd37c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0c70c0 con 0x7fd37c1607c0 2024-01-15T18:09:29.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c159760 con 0x7fd37c0cfb30 2024-01-15T18:09:29.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0c64b0 con 0x7fd37c06c230 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd37c1607c0 0x7fd37c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c0cfb30 0x7fd37c157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd37c1607c0 0x7fd37c157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36606/0 (socket says 172.21.15.143:36606) 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c06c230 0x7fd37c1f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37b400640 1 -- 172.21.15.143:0/1009505671 learned_addr learned my addr 172.21.15.143:0/1009505671 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 -- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c0cfb30 msgr2=0x7fd37c157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c0cfb30 0x7fd37c157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 -- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd37c1607c0 msgr2=0x7fd37c157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd37c1607c0 0x7fd37c157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 -- 172.21.15.143:0/1009505671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364009420 con 0x7fd37c06c230 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37a000640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c0cfb30 0x7fd37c157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:29.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd37aa00640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c06c230 0x7fd37c1f25f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0af000 tx=0x7fd36c01ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:29.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0bd070 con 0x7fd37c06c230 2024-01-15T18:09:29.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c0b0b20 con 0x7fd37c06c230 2024-01-15T18:09:29.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c0b9430 con 0x7fd37c06c230 2024-01-15T18:09:29.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36c009420 con 0x7fd37c06c230 2024-01-15T18:09:29.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.026+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c0c6a70 con 0x7fd37c06c230 2024-01-15T18:09:29.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.030+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c0b6d70 con 0x7fd37c06c230 2024-01-15T18:09:29.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.030+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fd36c0b7040 con 0x7fd37c06c230 2024-01-15T18:09:29.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.030+0000 7fd372a00640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd354066ec0 0x7fd354069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.030+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd36c121ed0 con 0x7fd37c06c230 2024-01-15T18:09:29.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.030+0000 7fd37a000640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd354066ec0 0x7fd354069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.030+0000 7fd37a000640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd354066ec0 0x7fd354069380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd37c1f2b30 tx=0x7fd364009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:29.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.034+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd36c0ee870 con 0x7fd37c06c230 2024-01-15T18:09:29.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.226+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd37c06d4b0 con 0x7fd37c06c230 2024-01-15T18:09:29.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.598+0000 7fd372a00640 1 -- 172.21.15.143:0/1009505671 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd36c0f1e90 con 0x7fd37c06c230 2024-01-15T18:09:29.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd354066ec0 msgr2=0x7fd354069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd354066ec0 0x7fd354069380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd37c1f2b30 tx=0x7fd364009190 comp rx=0 tx=0).stop 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c06c230 msgr2=0x7fd37c1f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c06c230 0x7fd37c1f25f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd36c0af000 tx=0x7fd36c01ba10 comp rx=0 tx=0).stop 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 shutdown_connections 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd354066ec0 0x7fd354069380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd37c1607c0 0x7fd37c157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd37c0cfb30 0x7fd37c157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 --2- 172.21.15.143:0/1009505671 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd37c06c230 0x7fd37c1f25f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 >> 172.21.15.143:0/1009505671 conn(0x7fd37c0d0e20 msgr2=0x7fd37c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 shutdown_connections 2024-01-15T18:09:29.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.602+0000 7fd381600640 1 -- 172.21.15.143:0/1009505671 wait complete. 2024-01-15T18:09:29.627 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:09:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 Processor -- start 2024-01-15T18:09:29.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 -- start start 2024-01-15T18:09:29.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd908157300 0x7fd908159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd908070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9080b2a20 con 0x7fd9080b21d0 2024-01-15T18:09:29.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080b35a0 con 0x7fd908157300 2024-01-15T18:09:29.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd90dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080b2d00 con 0x7fd9080b4ac0 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd908157300 0x7fd908159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd908157300 0x7fd908159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36612/0 (socket says 172.21.15.143:36612) 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 -- 172.21.15.143:0/3321232648 learned_addr learned my addr 172.21.15.143:0/3321232648 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd907400640 1 --2- 172.21.15.143:0/3321232648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd908070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 -- 172.21.15.143:0/3321232648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 msgr2=0x7fd908070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 --2- 172.21.15.143:0/3321232648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd908070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 -- 172.21.15.143:0/3321232648 >> v1:172.21.15.204:6789/0 conn(0x7fd9080b21d0 legacy=0x7fd9080b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.762+0000 7fd906a00640 1 -- 172.21.15.143:0/3321232648 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080b2fe0 con 0x7fd908157300 2024-01-15T18:09:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd906a00640 1 --2- 172.21.15.143:0/3321232648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd908157300 0x7fd908159b70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800bb10 tx=0x7fd8f801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c1b0634743010 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd905600640 1 -- 172.21.15.143:0/3321232648 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80a7cd0 con 0x7fd908157300 2024-01-15T18:09:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd905600640 1 -- 172.21.15.143:0/3321232648 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f80a7e30 con 0x7fd908157300 2024-01-15T18:09:29.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd905600640 1 -- 172.21.15.143:0/3321232648 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80af660 con 0x7fd908157300 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 -- 172.21.15.143:0/3321232648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd908157300 msgr2=0x7fd908159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3321232648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd908157300 0x7fd908159b70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800bb10 tx=0x7fd8f801ebe0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 -- 172.21.15.143:0/3321232648 shutdown_connections 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3321232648 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd908070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3321232648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd908157300 0x7fd908159b70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 -- 172.21.15.143:0/3321232648 >> 172.21.15.143:0/3321232648 conn(0x7fd90806c3f0 msgr2=0x7fd90806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 -- 172.21.15.143:0/3321232648 shutdown_connections 2024-01-15T18:09:29.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.766+0000 7fd90dc00640 1 -- 172.21.15.143:0/3321232648 wait complete. 2024-01-15T18:09:29.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 Processor -- start 2024-01-15T18:09:29.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- start start 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 0x7fd90814ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd90814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd908157300 0x7fd90814e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908153740 con 0x7fd9080b21d0 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908152b30 con 0x7fd908157300 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd908152e10 con 0x7fd9080b4ac0 2024-01-15T18:09:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 0x7fd90814ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 0x7fd90814ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36628/0 (socket says 172.21.15.143:36628) 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 -- 172.21.15.143:0/3361870055 learned_addr learned my addr 172.21.15.143:0/3361870055 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd907400640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd908157300 0x7fd90814e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906000640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd90814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 -- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 msgr2=0x7fd90814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd90814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 -- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd908157300 msgr2=0x7fd90814e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd908157300 0x7fd90814e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 -- 172.21.15.143:0/3361870055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f800b770 con 0x7fd9080b21d0 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd906a00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 0x7fd90814ba40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800b2c0 tx=0x7fd8f801d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80a7cd0 con 0x7fd9080b21d0 2024-01-15T18:09:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f80ba450 con 0x7fd9080b21d0 2024-01-15T18:09:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9081530f0 con 0x7fd9080b21d0 2024-01-15T18:09:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f80ba820 con 0x7fd9080b21d0 2024-01-15T18:09:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd908152830 con 0x7fd9080b21d0 2024-01-15T18:09:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.770+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd908067500 con 0x7fd9080b21d0 2024-01-15T18:09:29.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.774+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7fd8f80b6070 con 0x7fd9080b21d0 2024-01-15T18:09:29.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.774+0000 7fd8fea00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8e0066e90 0x7fd8e0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.774+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd9080b27c0 con 0x7fd9080b21d0 2024-01-15T18:09:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.774+0000 7fd906000640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8e0066e90 0x7fd8e0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.774+0000 7fd906000640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8e0066e90 0x7fd8e0069350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd9081f99f0 tx=0x7fd8f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.778+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd8f80ecc20 con 0x7fd9080b21d0 2024-01-15T18:09:29.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.946+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9080b4180 con 0x7fd8e0066e90 2024-01-15T18:09:29.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd8fea00640 1 -- 172.21.15.143:0/3361870055 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd9080b4180 con 0x7fd8e0066e90 2024-01-15T18:09:29.954 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:09:29.954 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:09:29.954 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:09:29.954 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:09:29.954 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8e0066e90 msgr2=0x7fd8e0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8e0066e90 0x7fd8e0069350 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd9081f99f0 tx=0x7fd8f0009040 comp rx=0 tx=0).stop 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 msgr2=0x7fd90814ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 0x7fd90814ba40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd8f800b2c0 tx=0x7fd8f801d420 comp rx=0 tx=0).stop 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 shutdown_connections 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8e0066e90 0x7fd8e0069350 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd908157300 0x7fd90814e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9080b4ac0 0x7fd90814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 --2- 172.21.15.143:0/3361870055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9080b21d0 0x7fd90814ba40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 >> 172.21.15.143:0/3361870055 conn(0x7fd90806c3f0 msgr2=0x7fd9080b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 shutdown_connections 2024-01-15T18:09:29.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:29.950+0000 7fd90dc00640 1 -- 172.21.15.143:0/3361870055 wait complete. 2024-01-15T18:09:29.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:09:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 Processor -- start 2024-01-15T18:09:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 -- start start 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f900b24b0 0x7f1f900ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1f900b3bf0 0x7f1f900b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f90070bf0 con 0x7f1f900ce690 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9015dba0 con 0x7f1f900b24b0 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f97200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b4920 con 0x7f1f900b3bf0 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f8f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f900b24b0 0x7f1f900ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.110+0000 7f1f8fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1f900ce690 0x7f1f900b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50398/0 (socket says 172.21.15.143:50398) 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8fe00640 1 -- 172.21.15.143:0/1084530 learned_addr learned my addr 172.21.15.143:0/1084530 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:09:30.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f95200640 1 --2- 172.21.15.143:0/1084530 >> v2:172.21.15.204:3300/0 conn(0x7f1f900b3bf0 0x7f1f900b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8f400640 1 -- 172.21.15.143:0/1084530 >> v2:172.21.15.204:3300/0 conn(0x7f1f900b3bf0 msgr2=0x7f1f900b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8f400640 1 --2- 172.21.15.143:0/1084530 >> v2:172.21.15.204:3300/0 conn(0x7f1f900b3bf0 0x7f1f900b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8ea00640 1 -- 172.21.15.143:0/1084530 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880270174 0 0) 0x7f1f90070bf0 con 0x7f1f900ce690 2024-01-15T18:09:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8f400640 1 -- 172.21.15.143:0/1084530 >> v1:172.21.15.143:6789/0 conn(0x7f1f900ce690 legacy=0x7f1f900b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8f400640 1 -- 172.21.15.143:0/1084530 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900b4c00 con 0x7f1f900b24b0 2024-01-15T18:09:30.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8f400640 1 --2- 172.21.15.143:0/1084530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900b24b0 0x7f1f900ac8e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1f9015e870 tx=0x7f1f7801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0aad504fc65e4da server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8ea00640 1 -- 172.21.15.143:0/1084530 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780ab650 con 0x7f1f900b24b0 2024-01-15T18:09:30.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8ea00640 1 -- 172.21.15.143:0/1084530 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f780ab7b0 con 0x7f1f900b24b0 2024-01-15T18:09:30.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.114+0000 7f1f8ea00640 1 -- 172.21.15.143:0/1084530 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f780abb40 con 0x7f1f900b24b0 2024-01-15T18:09:30.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- 172.21.15.143:0/1084530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900b24b0 msgr2=0x7f1f900ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 --2- 172.21.15.143:0/1084530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900b24b0 0x7f1f900ac8e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1f9015e870 tx=0x7f1f7801ce90 comp rx=0 tx=0).stop 2024-01-15T18:09:30.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- 172.21.15.143:0/1084530 shutdown_connections 2024-01-15T18:09:30.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 --2- 172.21.15.143:0/1084530 >> v2:172.21.15.204:3300/0 conn(0x7f1f900b3bf0 0x7f1f900b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 --2- 172.21.15.143:0/1084530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900b24b0 0x7f1f900ac8e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- 172.21.15.143:0/1084530 >> 172.21.15.143:0/1084530 conn(0x7f1f9006c3f0 msgr2=0x7f1f9006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- 172.21.15.143:0/1084530 shutdown_connections 2024-01-15T18:09:30.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- 172.21.15.143:0/1084530 wait complete. 2024-01-15T18:09:30.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 Processor -- start 2024-01-15T18:09:30.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- start start 2024-01-15T18:09:30.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f900b24b0 0x7f1f90143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f900b3bf0 0x7f1f90145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 0x7f1f901653d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90167fa0 con 0x7f1f900ce690 2024-01-15T18:09:30.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015a420 con 0x7f1f900b3bf0 2024-01-15T18:09:30.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.118+0000 7f1f97200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90146820 con 0x7f1f900b24b0 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 0x7f1f901653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 0x7f1f901653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36642/0 (socket says 172.21.15.143:36642) 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 -- 172.21.15.143:0/1851245952 learned_addr learned my addr 172.21.15.143:0/1851245952 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f8f400640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f900b3bf0 0x7f1f90145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f8fe00640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f900b24b0 0x7f1f90143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 -- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f900b24b0 msgr2=0x7f1f90143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f900b24b0 0x7f1f90143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 -- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f900b3bf0 msgr2=0x7f1f90145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f900b3bf0 0x7f1f90145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 -- 172.21.15.143:0/1851245952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009420 con 0x7f1f900ce690 2024-01-15T18:09:30.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f95200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 0x7f1f901653d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009d20 tx=0x7f1f8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88009b10 con 0x7f1f900ce690 2024-01-15T18:09:30.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800f040 con 0x7f1f900ce690 2024-01-15T18:09:30.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f90145970 con 0x7f1f900ce690 2024-01-15T18:09:30.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880146c0 con 0x7f1f900ce690 2024-01-15T18:09:30.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f90145ef0 con 0x7f1f900ce690 2024-01-15T18:09:30.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90067500 con 0x7f1f900ce690 2024-01-15T18:09:30.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.122+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f1f88014820 con 0x7f1f900ce690 2024-01-15T18:09:30.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.126+0000 7f1f87e00640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f68066ec0 0x7f1f68069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.126+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1f88085ae0 con 0x7f1f900ce690 2024-01-15T18:09:30.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.126+0000 7f1f8fe00640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f68066ec0 0x7f1f68069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.126+0000 7f1f8fe00640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f68066ec0 0x7f1f68069380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000f70 tx=0x7f1f80007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.126+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1f880523d0 con 0x7f1f900ce690 2024-01-15T18:09:30.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.278+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1f90146960 con 0x7f1f900ce690 2024-01-15T18:09:30.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.278+0000 7f1f87e00640 1 -- 172.21.15.143:0/1851245952 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f880559f0 con 0x7f1f900ce690 2024-01-15T18:09:30.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f68066ec0 msgr2=0x7f1f68069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f68066ec0 0x7f1f68069380 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000f70 tx=0x7f1f80007c10 comp rx=0 tx=0).stop 2024-01-15T18:09:30.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 msgr2=0x7f1f901653d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 0x7f1f901653d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1f88009d20 tx=0x7f1f8800c490 comp rx=0 tx=0).stop 2024-01-15T18:09:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 shutdown_connections 2024-01-15T18:09:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f68066ec0 0x7f1f68069380 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f900ce690 0x7f1f901653d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f900b3bf0 0x7f1f90145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 --2- 172.21.15.143:0/1851245952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f900b24b0 0x7f1f90143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 >> 172.21.15.143:0/1851245952 conn(0x7f1f9006c3f0 msgr2=0x7f1f9015e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 shutdown_connections 2024-01-15T18:09:30.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.282+0000 7f1f97200640 1 -- 172.21.15.143:0/1851245952 wait complete. 2024-01-15T18:09:30.308 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:09:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.442+0000 7f3397600640 1 Processor -- start 2024-01-15T18:09:30.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3397600640 1 -- start start 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3397600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f33901607c0 0x7f339006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3397600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f33900cbf50 0x7f33900cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3397600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33900cffd0 con 0x7f33900cfb30 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3397600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33900d1ce0 con 0x7f33901607c0 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3397600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33900d02b0 con 0x7f33900cbf50 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f33901607c0 0x7f339006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f33901607c0 0x7f339006b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36656/0 (socket says 172.21.15.143:36656) 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 -- 172.21.15.143:0/320122222 learned_addr learned my addr 172.21.15.143:0/320122222 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3395600640 1 --2- 172.21.15.143:0/320122222 >> v2:172.21.15.204:3300/0 conn(0x7f33900cbf50 0x7f33900cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 -- 172.21.15.143:0/320122222 >> v2:172.21.15.204:3300/0 conn(0x7f33900cbf50 msgr2=0x7f33900cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 --2- 172.21.15.143:0/320122222 >> v2:172.21.15.204:3300/0 conn(0x7f33900cbf50 0x7f33900cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 -- 172.21.15.143:0/320122222 >> v1:172.21.15.204:6789/0 conn(0x7f33900cfb30 legacy=0x7f339006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3394c00640 1 -- 172.21.15.143:0/320122222 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900cb470 con 0x7f33901607c0 2024-01-15T18:09:30.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.446+0000 7f3395600640 1 --2- 172.21.15.143:0/320122222 >> v2:172.21.15.204:3300/0 conn(0x7f33900cbf50 0x7f33900cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3394c00640 1 --2- 172.21.15.143:0/320122222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33901607c0 0x7f339006b7b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f33900cef70 tx=0x7f338001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf9daad712370c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f338f400640 1 -- 172.21.15.143:0/320122222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800a89f0 con 0x7f33901607c0 2024-01-15T18:09:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f338f400640 1 -- 172.21.15.143:0/320122222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800a8b50 con 0x7f33901607c0 2024-01-15T18:09:30.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f338f400640 1 -- 172.21.15.143:0/320122222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800b0440 con 0x7f33901607c0 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 -- 172.21.15.143:0/320122222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33901607c0 msgr2=0x7f339006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 --2- 172.21.15.143:0/320122222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33901607c0 0x7f339006b7b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f33900cef70 tx=0x7f338001ecf0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 -- 172.21.15.143:0/320122222 shutdown_connections 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 --2- 172.21.15.143:0/320122222 >> v2:172.21.15.204:3300/0 conn(0x7f33900cbf50 0x7f33900cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 --2- 172.21.15.143:0/320122222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33901607c0 0x7f339006b7b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 -- 172.21.15.143:0/320122222 >> 172.21.15.143:0/320122222 conn(0x7f33900d0e20 msgr2=0x7f33900d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 -- 172.21.15.143:0/320122222 shutdown_connections 2024-01-15T18:09:30.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.450+0000 7f3397600640 1 -- 172.21.15.143:0/320122222 wait complete. 2024-01-15T18:09:30.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 Processor -- start 2024-01-15T18:09:30.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- start start 2024-01-15T18:09:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 0x7f339009b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33900cfb30 0x7f339015c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33901607c0 0x7f33900c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901f4050 con 0x7f33900cbf50 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901f4330 con 0x7f33901607c0 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33901f4610 con 0x7f33900cfb30 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 0x7f339009b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 0x7f339009b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36660/0 (socket says 172.21.15.143:36660) 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 -- 172.21.15.143:0/397046774 learned_addr learned my addr 172.21.15.143:0/397046774 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3395600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33901607c0 0x7f33900c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f338fe00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33900cfb30 0x7f339015c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 -- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33900cfb30 msgr2=0x7f339015c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33900cfb30 0x7f339015c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 -- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33901607c0 msgr2=0x7f33900c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33901607c0 0x7f33900c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 -- 172.21.15.143:0/397046774 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800094d0 con 0x7f33900cbf50 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3395600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33901607c0 0x7f33900c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3394c00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 0x7f339009b8b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f338001b900 tx=0x7f338001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800ba600 con 0x7f33900cbf50 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33800ba760 con 0x7f33900cbf50 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33800bab10 con 0x7f33900cbf50 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33901f48f0 con 0x7f33900cbf50 2024-01-15T18:09:30.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33901f3ce0 con 0x7f33900cbf50 2024-01-15T18:09:30.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.454+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339006d4b0 con 0x7f33900cbf50 2024-01-15T18:09:30.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.458+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f33800bac70 con 0x7f33900cbf50 2024-01-15T18:09:30.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.458+0000 7f338cc00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3368066e90 0x7f3368069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.458+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3380123400 con 0x7f33900cbf50 2024-01-15T18:09:30.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.458+0000 7f338fe00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3368066e90 0x7f3368069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.458+0000 7f338fe00640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3368066e90 0x7f3368069350 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3378000f70 tx=0x7f3378009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.462+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33800efd70 con 0x7f33900cbf50 2024-01-15T18:09:30.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.634+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f33900ce6b0 con 0x7f3368066e90 2024-01-15T18:09:30.643 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:09:30.638+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:30.643 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:09:30.638+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:30.644 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:09:30.638+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:30.644 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:09:30.638+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:30.644 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:09:30.638+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:30.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f338cc00640 1 -- 172.21.15.143:0/397046774 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3378001ce0 con 0x7f3368066e90 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3368066e90 msgr2=0x7f3368069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3368066e90 0x7f3368069350 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3378000f70 tx=0x7f3378009040 comp rx=0 tx=0).stop 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 msgr2=0x7f339009b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 0x7f339009b8b0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f338001b900 tx=0x7f338001ba90 comp rx=0 tx=0).stop 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 shutdown_connections 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3368066e90 0x7f3368069350 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33901607c0 0x7f33900c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33900cfb30 0x7f339015c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 --2- 172.21.15.143:0/397046774 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33900cbf50 0x7f339009b8b0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 >> 172.21.15.143:0/397046774 conn(0x7f33900d0e20 msgr2=0x7f339006c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 shutdown_connections 2024-01-15T18:09:30.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.674+0000 7f3397600640 1 -- 172.21.15.143:0/397046774 wait complete. 2024-01-15T18:09:30.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:09:30.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.834+0000 7f6863a00640 1 Processor -- start 2024-01-15T18:09:30.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6863a00640 1 -- start start 2024-01-15T18:09:30.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6863a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c0ce560 0x7f685c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6863a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f685c071d00 0x7f685c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6863a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685c15f310 con 0x7f685c141660 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6863a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685c15fbb0 con 0x7f685c071d00 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6863a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0ce9e0 con 0x7f685c0ce560 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f685c071d00 0x7f685c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f685c071d00 0x7f685c072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36676/0 (socket says 172.21.15.143:36676) 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f685c141660 0x7f685c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50400/0 (socket says 172.21.15.143:50400) 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 -- 172.21.15.143:0/388620376 learned_addr learned my addr 172.21.15.143:0/388620376 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 -- 172.21.15.143:0/388620376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c0ce560 msgr2=0x7f685c15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 --2- 172.21.15.143:0/388620376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c0ce560 0x7f685c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 -- 172.21.15.143:0/388620376 >> v1:172.21.15.143:6789/0 conn(0x7f685c141660 legacy=0x7f685c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.838+0000 7f6861a00640 1 -- 172.21.15.143:0/388620376 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c15f5f0 con 0x7f685c071d00 2024-01-15T18:09:30.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6861a00640 1 --2- 172.21.15.143:0/388620376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c071d00 0x7f685c072150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6854009e50 tx=0x7f685401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2fba21880cb8c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f685b400640 1 -- 172.21.15.143:0/388620376 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540a5590 con 0x7f685c071d00 2024-01-15T18:09:30.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f685b400640 1 -- 172.21.15.143:0/388620376 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68540a56f0 con 0x7f685c071d00 2024-01-15T18:09:30.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f685b400640 1 -- 172.21.15.143:0/388620376 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540a5a80 con 0x7f685c071d00 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 -- 172.21.15.143:0/388620376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c071d00 msgr2=0x7f685c072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 --2- 172.21.15.143:0/388620376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c071d00 0x7f685c072150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6854009e50 tx=0x7f685401c8a0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 -- 172.21.15.143:0/388620376 shutdown_connections 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 --2- 172.21.15.143:0/388620376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c071d00 0x7f685c072150 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 --2- 172.21.15.143:0/388620376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c0ce560 0x7f685c15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 -- 172.21.15.143:0/388620376 >> 172.21.15.143:0/388620376 conn(0x7f685c06c3f0 msgr2=0x7f685c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 -- 172.21.15.143:0/388620376 shutdown_connections 2024-01-15T18:09:30.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.842+0000 7f6863a00640 1 -- 172.21.15.143:0/388620376 wait complete. 2024-01-15T18:09:30.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 Processor -- start 2024-01-15T18:09:30.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- start start 2024-01-15T18:09:30.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f685c071d00 0x7f685c158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 0x7f685c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c141660 0x7f685c0ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c1404c0 con 0x7f685c0ce560 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c159420 con 0x7f685c071d00 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685c140710 con 0x7f685c141660 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 0x7f685c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 0x7f685c0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36688/0 (socket says 172.21.15.143:36688) 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 -- 172.21.15.143:0/2378072755 learned_addr learned my addr 172.21.15.143:0/2378072755 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6861000640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f685c071d00 0x7f685c158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6861a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c141660 0x7f685c0ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 -- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c141660 msgr2=0x7f685c0ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c141660 0x7f685c0ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 -- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f685c071d00 msgr2=0x7f685c158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f685c071d00 0x7f685c158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 -- 172.21.15.143:0/2378072755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854009b30 con 0x7f685c0ce560 2024-01-15T18:09:30.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6861000640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f685c071d00 0x7f685c158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:30.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f685be00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 0x7f685c0ac8e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f684400b580 tx=0x7f684400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844010870 con 0x7f685c0ce560 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68440109f0 con 0x7f685c0ce560 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c0cd150 con 0x7f685c0ce560 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844010da0 con 0x7f685c0ce560 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c140cd0 con 0x7f685c0ce560 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.846+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c067500 con 0x7f685c0ce560 2024-01-15T18:09:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.850+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) v1 ==== 86670+0+0 (secure 0 0 0) 0x7f684400f3c0 con 0x7f685c0ce560 2024-01-15T18:09:30.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.850+0000 7f6858c00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6834066e90 0x7f6834069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.850+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6844084f20 con 0x7f685c0ce560 2024-01-15T18:09:30.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.850+0000 7f6861000640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6834066e90 0x7f6834069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.850+0000 7f6861000640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6834066e90 0x7f6834069350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f684c000f70 tx=0x7f684c007e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:30.854+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6844051950 con 0x7f685c0ce560 2024-01-15T18:09:31.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.034+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f685c0ccfd0 con 0x7f6834066e90 2024-01-15T18:09:31.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.038+0000 7f6858c00640 1 -- 172.21.15.143:0/2378072755 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f684c001a40 con 0x7f6834066e90 2024-01-15T18:09:31.044 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:31.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6834066e90 msgr2=0x7f6834069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:31.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6834066e90 0x7f6834069350 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f684c000f70 tx=0x7f684c007e10 comp rx=0 tx=0).stop 2024-01-15T18:09:31.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 msgr2=0x7f685c0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:31.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 0x7f685c0ac8e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f684400b580 tx=0x7f684400ba40 comp rx=0 tx=0).stop 2024-01-15T18:09:31.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 shutdown_connections 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6834066e90 0x7f6834069350 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f685c141660 0x7f685c0ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f685c0ce560 0x7f685c0ac8e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 --2- 172.21.15.143:0/2378072755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f685c071d00 0x7f685c158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 >> 172.21.15.143:0/2378072755 conn(0x7f685c06c3f0 msgr2=0x7f685c0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 shutdown_connections 2024-01-15T18:09:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:31.042+0000 7f6863a00640 1 -- 172.21.15.143:0/2378072755 wait complete. 2024-01-15T18:09:31.066 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:31.066 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:31.066 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:31.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:31.075 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:09:38.267 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:09:38.267 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:09:38.267 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.12785 s, 151 MB/s 2024-01-15T18:09:38.271 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:38.271 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:38.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:38.280 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:09:45.339 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:09:45.339 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:09:45.339 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.00202 s, 153 MB/s 2024-01-15T18:09:45.340 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:09:45.340 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:45.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:45.350 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:09:52.338 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:09:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:09:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.93481 s, 155 MB/s 2024-01-15T18:09:52.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:09:52.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743fe00640 1 Processor -- start 2024-01-15T18:09:52.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743fe00640 1 -- start start 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f743806a490 0x7f743815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743fe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743806d170 con 0x7f7438070dd0 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743806a8d0 con 0x7f7438150690 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743fe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7438150a70 con 0x7f743806a490 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f743806a490 0x7f743815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f743806a490 0x7f743815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34254/0 (socket says 172.21.15.143:34254) 2024-01-15T18:09:52.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.474+0000 7f743ca00640 1 -- 172.21.15.143:0/1579633973 learned_addr learned my addr 172.21.15.143:0/1579633973 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:52.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f743ca00640 1 -- 172.21.15.143:0/1579633973 >> v1:172.21.15.204:6789/0 conn(0x7f7438150690 legacy=0x7f74380b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f743ca00640 1 -- 172.21.15.143:0/1579633973 >> v1:172.21.15.143:6789/0 conn(0x7f7438070dd0 legacy=0x7f743806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f743ca00640 1 -- 172.21.15.143:0/1579633973 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743806ab20 con 0x7f743806a490 2024-01-15T18:09:52.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f7437e00640 1 -- 172.21.15.143:0/1579633973 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146011815 0 0) 0x7f743806d170 con 0x7f7438070dd0 2024-01-15T18:09:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f743ca00640 1 --2- 172.21.15.143:0/1579633973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f743806a490 0x7f743815a5e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7420009870 tx=0x7f742001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7af58b21316272e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f7437e00640 1 -- 172.21.15.143:0/1579633973 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200a9400 con 0x7f743806a490 2024-01-15T18:09:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f7437e00640 1 -- 172.21.15.143:0/1579633973 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74200a9560 con 0x7f743806a490 2024-01-15T18:09:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.478+0000 7f7437e00640 1 -- 172.21.15.143:0/1579633973 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200a98f0 con 0x7f743806a490 2024-01-15T18:09:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- 172.21.15.143:0/1579633973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f743806a490 msgr2=0x7f743815a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 --2- 172.21.15.143:0/1579633973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f743806a490 0x7f743815a5e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7420009870 tx=0x7f742001c540 comp rx=0 tx=0).stop 2024-01-15T18:09:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- 172.21.15.143:0/1579633973 shutdown_connections 2024-01-15T18:09:52.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 --2- 172.21.15.143:0/1579633973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f743806a490 0x7f743815a5e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- 172.21.15.143:0/1579633973 >> 172.21.15.143:0/1579633973 conn(0x7f74380b1610 msgr2=0x7f74380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:52.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- 172.21.15.143:0/1579633973 shutdown_connections 2024-01-15T18:09:52.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- 172.21.15.143:0/1579633973 wait complete. 2024-01-15T18:09:52.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 Processor -- start 2024-01-15T18:09:52.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- start start 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 0x7f7438152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7438070dd0 0x7f7438158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438150690 0x7f743815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438152270 con 0x7f7438070dd0 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438152c40 con 0x7f743806a490 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74381516a0 con 0x7f7438150690 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7438070dd0 0x7f7438158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 0x7f7438152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 0x7f7438152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:37938/0 (socket says 172.21.15.143:37938) 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743d400640 1 -- 172.21.15.143:0/3356156594 learned_addr learned my addr 172.21.15.143:0/3356156594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:52.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743de00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438150690 0x7f743815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743ca00640 1 -- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438150690 msgr2=0x7f743815ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743ca00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438150690 0x7f743815ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743ca00640 1 -- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 msgr2=0x7f7438152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.482+0000 7f743ca00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 0x7f7438152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743ca00640 1 -- 172.21.15.143:0/3356156594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200094d0 con 0x7f7438070dd0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743d400640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 0x7f7438152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743ca00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7438070dd0 0x7f7438158800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f74200aa000 tx=0x7f742001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200b18c0 con 0x7f7438070dd0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74200b1a60 con 0x7f7438070dd0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200b1e10 con 0x7f7438070dd0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7438071770 con 0x7f7438070dd0 2024-01-15T18:09:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7438151c20 con 0x7f7438070dd0 2024-01-15T18:09:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7438074b00 con 0x7f7438070dd0 2024-01-15T18:09:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7f74200bc450 con 0x7f7438070dd0 2024-01-15T18:09:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f7435600640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066f50 0x7f7410069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f742011f690 con 0x7f7438070dd0 2024-01-15T18:09:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.486+0000 7f743d400640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066f50 0x7f7410069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.490+0000 7f743d400640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066f50 0x7f7410069410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7428001c00 tx=0x7f7428009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.494+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74200ec020 con 0x7f7438070dd0 2024-01-15T18:09:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.642+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7f7438067500 con 0x7f7438070dd0 2024-01-15T18:09:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.646+0000 7f7435600640 1 -- 172.21.15.143:0/3356156594 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v21)=0 v21) v1 ==== 131+0+0 (secure 0 0 0) 0x7f74200a6020 con 0x7f7438070dd0 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066f50 msgr2=0x7f7410069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066f50 0x7f7410069410 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7428001c00 tx=0x7f7428009040 comp rx=0 tx=0).stop 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7438070dd0 msgr2=0x7f7438158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7438070dd0 0x7f7438158800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f74200aa000 tx=0x7f742001ad80 comp rx=0 tx=0).stop 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 shutdown_connections 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7410066f50 0x7f7410069410 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7438150690 0x7f743815ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7438070dd0 0x7f7438158800 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 --2- 172.21.15.143:0/3356156594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f743806a490 0x7f7438152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 >> 172.21.15.143:0/3356156594 conn(0x7f74380b1610 msgr2=0x7f7438152e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 shutdown_connections 2024-01-15T18:09:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.650+0000 7f743fe00640 1 -- 172.21.15.143:0/3356156594 wait complete. 2024-01-15T18:09:52.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T18:09:52.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.810+0000 7fe821a00640 1 Processor -- start 2024-01-15T18:09:52.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.810+0000 7fe821a00640 1 -- start start 2024-01-15T18:09:52.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe821a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe81c070f20 0x7fe81c06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe821a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe81c071300 0x7fe81c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe821a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c15e6c0 con 0x7fe81c157700 2024-01-15T18:09:52.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe821a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c157b90 con 0x7fe81c071300 2024-01-15T18:09:52.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe821a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c06a490 con 0x7fe81c070f20 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe81c071300 0x7fe81c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe81c071300 0x7fe81c1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34286/0 (socket says 172.21.15.143:34286) 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 -- 172.21.15.143:0/3829654216 learned_addr learned my addr 172.21.15.143:0/3829654216 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81aa00640 1 --2- 172.21.15.143:0/3829654216 >> v2:172.21.15.204:3300/0 conn(0x7fe81c070f20 0x7fe81c06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 -- 172.21.15.143:0/3829654216 >> v2:172.21.15.204:3300/0 conn(0x7fe81c070f20 msgr2=0x7fe81c06b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 --2- 172.21.15.143:0/3829654216 >> v2:172.21.15.204:3300/0 conn(0x7fe81c070f20 0x7fe81c06b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 -- 172.21.15.143:0/3829654216 >> v1:172.21.15.204:6789/0 conn(0x7fe81c157700 legacy=0x7fe81c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81a000640 1 -- 172.21.15.143:0/3829654216 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c06a770 con 0x7fe81c071300 2024-01-15T18:09:52.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.814+0000 7fe81aa00640 1 --2- 172.21.15.143:0/3829654216 >> v2:172.21.15.204:3300/0 conn(0x7fe81c070f20 0x7fe81c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:52.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe81a000640 1 --2- 172.21.15.143:0/3829654216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c071300 0x7fe81c1571c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0d0640 tx=0x7fe80401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9c1f01f3027062 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:52.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe819600640 1 -- 172.21.15.143:0/3829654216 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8040ac410 con 0x7fe81c071300 2024-01-15T18:09:52.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe819600640 1 -- 172.21.15.143:0/3829654216 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8040ac570 con 0x7fe81c071300 2024-01-15T18:09:52.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe819600640 1 -- 172.21.15.143:0/3829654216 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8040ac900 con 0x7fe81c071300 2024-01-15T18:09:52.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 -- 172.21.15.143:0/3829654216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c071300 msgr2=0x7fe81c1571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 --2- 172.21.15.143:0/3829654216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c071300 0x7fe81c1571c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0d0640 tx=0x7fe80401c490 comp rx=0 tx=0).stop 2024-01-15T18:09:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 -- 172.21.15.143:0/3829654216 shutdown_connections 2024-01-15T18:09:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 --2- 172.21.15.143:0/3829654216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c071300 0x7fe81c1571c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 --2- 172.21.15.143:0/3829654216 >> v2:172.21.15.204:3300/0 conn(0x7fe81c070f20 0x7fe81c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 -- 172.21.15.143:0/3829654216 >> 172.21.15.143:0/3829654216 conn(0x7fe81c0b1610 msgr2=0x7fe81c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 -- 172.21.15.143:0/3829654216 shutdown_connections 2024-01-15T18:09:52.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 -- 172.21.15.143:0/3829654216 wait complete. 2024-01-15T18:09:52.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 Processor -- start 2024-01-15T18:09:52.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.818+0000 7fe821a00640 1 -- start start 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 0x7fe81c14af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe81c071300 0x7fe81c137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c157700 0x7fe81c151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1472d0 con 0x7fe81c070f20 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c147a10 con 0x7fe81c157700 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c14b470 con 0x7fe81c071300 2024-01-15T18:09:52.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 0x7fe81c14af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 0x7fe81c14af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34298/0 (socket says 172.21.15.143:34298) 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 -- 172.21.15.143:0/1045846274 learned_addr learned my addr 172.21.15.143:0/1045846274 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81a000640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe81c071300 0x7fe81c137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81b400640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c157700 0x7fe81c151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 -- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe81c071300 msgr2=0x7fe81c137fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe81c071300 0x7fe81c137fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 -- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c157700 msgr2=0x7fe81c151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c157700 0x7fe81c151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 -- 172.21.15.143:0/1045846274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8040094d0 con 0x7fe81c070f20 2024-01-15T18:09:52.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe81aa00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 0x7fe81c14af30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00c7f0 tx=0x7fe80c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00cea0 con 0x7fe81c070f20 2024-01-15T18:09:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c007a30 con 0x7fe81c070f20 2024-01-15T18:09:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c15e9f0 con 0x7fe81c070f20 2024-01-15T18:09:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c007de0 con 0x7fe81c070f20 2024-01-15T18:09:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c146650 con 0x7fe81c070f20 2024-01-15T18:09:52.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c074b00 con 0x7fe81c070f20 2024-01-15T18:09:52.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.822+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7fe80c0113f0 con 0x7fe81c070f20 2024-01-15T18:09:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.826+0000 7fe812a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f4066f20 0x7fe7f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.826+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe80c085130 con 0x7fe81c070f20 2024-01-15T18:09:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.826+0000 7fe81a000640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f4066f20 0x7fe7f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:52.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.826+0000 7fe81a000640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f4066f20 0x7fe7f40693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe8040ae000 tx=0x7fe804008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:52.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.830+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe80c051970 con 0x7fe81c070f20 2024-01-15T18:09:52.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.982+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7fe81c147410 con 0x7fe81c070f20 2024-01-15T18:09:52.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.986+0000 7fe812a00640 1 -- 172.21.15.143:0/1045846274 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v22)=0 v22) v1 ==== 135+0+0 (secure 0 0 0) 0x7fe80c054f90 con 0x7fe81c070f20 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f4066f20 msgr2=0x7fe7f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f4066f20 0x7fe7f40693e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe8040ae000 tx=0x7fe804008040 comp rx=0 tx=0).stop 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 msgr2=0x7fe81c14af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 0x7fe81c14af30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe80c00c7f0 tx=0x7fe80c00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 shutdown_connections 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f4066f20 0x7fe7f40693e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe81c157700 0x7fe81c151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe81c071300 0x7fe81c137fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 --2- 172.21.15.143:0/1045846274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe81c070f20 0x7fe81c14af30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 >> 172.21.15.143:0/1045846274 conn(0x7fe81c0b1610 msgr2=0x7fe81c14da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:52.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 shutdown_connections 2024-01-15T18:09:52.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:52.990+0000 7fe821a00640 1 -- 172.21.15.143:0/1045846274 wait complete. 2024-01-15T18:09:53.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:09:53.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 Processor -- start 2024-01-15T18:09:53.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 -- start start 2024-01-15T18:09:53.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 0x7f5c940cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5c940c6230 0x7f5c940cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c940c4d90 con 0x7f5c9415b260 2024-01-15T18:09:53.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9415b640 con 0x7f5c940c6230 2024-01-15T18:09:53.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940cf800 con 0x7f5c940cdf20 2024-01-15T18:09:53.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 0x7f5c940cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c9a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5c940c6230 0x7f5c940cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 0x7f5c940cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:53946/0 (socket says 172.21.15.143:53946) 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c99600640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f5c9415b260 0x7f5c940cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:54816/0 (socket says 172.21.15.143:54816) 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.150+0000 7f5c98c00640 1 -- 172.21.15.143:0/2605896480 learned_addr learned my addr 172.21.15.143:0/2605896480 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c98c00640 1 -- 172.21.15.143:0/2605896480 >> v2:172.21.15.204:3300/0 conn(0x7f5c940c6230 msgr2=0x7f5c940cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c98c00640 1 --2- 172.21.15.143:0/2605896480 >> v2:172.21.15.204:3300/0 conn(0x7f5c940c6230 0x7f5c940cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c98c00640 1 -- 172.21.15.143:0/2605896480 >> v1:172.21.15.204:6789/0 conn(0x7f5c9415b260 legacy=0x7f5c940cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c98c00640 1 -- 172.21.15.143:0/2605896480 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940cfae0 con 0x7f5c940cdf20 2024-01-15T18:09:53.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c9a000640 1 --2- 172.21.15.143:0/2605896480 >> v2:172.21.15.204:3300/0 conn(0x7f5c940c6230 0x7f5c940cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:53.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c98c00640 1 --2- 172.21.15.143:0/2605896480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 0x7f5c940cbae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009870 tx=0x7f5c7c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16475b94f47f58e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:53.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c93e00640 1 -- 172.21.15.143:0/2605896480 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0aa670 con 0x7f5c940cdf20 2024-01-15T18:09:53.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c93e00640 1 -- 172.21.15.143:0/2605896480 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0aa7d0 con 0x7f5c940cdf20 2024-01-15T18:09:53.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.154+0000 7f5c93e00640 1 -- 172.21.15.143:0/2605896480 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0aab60 con 0x7f5c940cdf20 2024-01-15T18:09:53.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- 172.21.15.143:0/2605896480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 msgr2=0x7f5c940cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 --2- 172.21.15.143:0/2605896480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 0x7f5c940cbae0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009870 tx=0x7f5c7c01c510 comp rx=0 tx=0).stop 2024-01-15T18:09:53.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- 172.21.15.143:0/2605896480 shutdown_connections 2024-01-15T18:09:53.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 --2- 172.21.15.143:0/2605896480 >> v2:172.21.15.204:3300/0 conn(0x7f5c940c6230 0x7f5c940cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 --2- 172.21.15.143:0/2605896480 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c940cdf20 0x7f5c940cbae0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- 172.21.15.143:0/2605896480 >> 172.21.15.143:0/2605896480 conn(0x7f5c940d0ed0 msgr2=0x7f5c940d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:53.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- 172.21.15.143:0/2605896480 shutdown_connections 2024-01-15T18:09:53.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- 172.21.15.143:0/2605896480 wait complete. 2024-01-15T18:09:53.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 Processor -- start 2024-01-15T18:09:53.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- start start 2024-01-15T18:09:53.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c940c6230 0x7f5c940adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 0x7f5c940b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c9415b260 0x7f5c941f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9409e980 con 0x7f5c940cdf20 2024-01-15T18:09:53.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9409bdd0 con 0x7f5c940c6230 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9409c0b0 con 0x7f5c9415b260 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 0x7f5c940b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c98c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 0x7f5c940b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34306/0 (socket says 172.21.15.143:34306) 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c98c00640 1 -- 172.21.15.143:0/1763620389 learned_addr learned my addr 172.21.15.143:0/1763620389 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c9a000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c9415b260 0x7f5c941f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.158+0000 7f5c99600640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c940c6230 0x7f5c940adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c98c00640 1 -- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c9415b260 msgr2=0x7f5c941f99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c98c00640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c9415b260 0x7f5c941f99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c98c00640 1 -- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c940c6230 msgr2=0x7f5c940adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c98c00640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c940c6230 0x7f5c940adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c98c00640 1 -- 172.21.15.143:0/1763620389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c009a70 con 0x7f5c940cdf20 2024-01-15T18:09:53.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c98c00640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 0x7f5c940b1c20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0ac040 tx=0x7f5c7c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:53.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0bd5f0 con 0x7f5c940cdf20 2024-01-15T18:09:53.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0bd770 con 0x7f5c940cdf20 2024-01-15T18:09:53.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c0094d0 con 0x7f5c940cdf20 2024-01-15T18:09:53.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0bdb20 con 0x7f5c940cdf20 2024-01-15T18:09:53.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c940b2160 con 0x7f5c940cdf20 2024-01-15T18:09:53.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9409bc30 con 0x7f5c940cdf20 2024-01-15T18:09:53.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7f5c7c0b33f0 con 0x7f5c940cdf20 2024-01-15T18:09:53.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c91600640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c6c066f20 0x7f5c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5c7c120280 con 0x7f5c940cdf20 2024-01-15T18:09:53.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.162+0000 7f5c99600640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c6c066f20 0x7f5c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.166+0000 7f5c99600640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c6c066f20 0x7f5c6c0693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000f70 tx=0x7f5c84007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:53.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.170+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c9409bc30 con 0x7f5c940cdf20 2024-01-15T18:09:53.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.342+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c9409bf40 con 0x7f5c6c066f20 2024-01-15T18:09:53.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c91600640 1 -- 172.21.15.143:0/1763620389 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5c9409bf40 con 0x7f5c6c066f20 2024-01-15T18:09:53.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c6c066f20 msgr2=0x7f5c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c6c066f20 0x7f5c6c0693e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c84000f70 tx=0x7f5c84007e50 comp rx=0 tx=0).stop 2024-01-15T18:09:53.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 msgr2=0x7f5c940b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 0x7f5c940b1c20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0ac040 tx=0x7f5c7c01ad50 comp rx=0 tx=0).stop 2024-01-15T18:09:53.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 shutdown_connections 2024-01-15T18:09:53.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c6c066f20 0x7f5c6c0693e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c9415b260 0x7f5c941f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c940cdf20 0x7f5c940b1c20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 --2- 172.21.15.143:0/1763620389 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c940c6230 0x7f5c940adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 >> 172.21.15.143:0/1763620389 conn(0x7f5c940d0ed0 msgr2=0x7f5c940cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:53.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 shutdown_connections 2024-01-15T18:09:53.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.358+0000 7f5c9c000640 1 -- 172.21.15.143:0/1763620389 wait complete. 2024-01-15T18:09:53.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:09:53.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 Processor -- start 2024-01-15T18:09:53.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 -- start start 2024-01-15T18:09:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe32415b260 0x7fe3240cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe3240d19f0 0x7fe3240cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3240c5220 0x7fe3240c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3240c4d90 con 0x7fe32415b260 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32415b640 con 0x7fe3240d19f0 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3240cf800 con 0x7fe3240c5220 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe32415b260 0x7fe3240cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe32415b260 0x7fe3240cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34308/0 (socket says 172.21.15.143:34308) 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 -- 172.21.15.143:0/3474871289 learned_addr learned my addr 172.21.15.143:0/3474871289 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe328e00640 1 --2- 172.21.15.143:0/3474871289 >> v2:172.21.15.204:3300/0 conn(0x7fe3240d19f0 0x7fe3240cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe32a200640 1 --2- 172.21.15.143:0/3474871289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3240c5220 0x7fe3240c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 -- 172.21.15.143:0/3474871289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3240c5220 msgr2=0x7fe3240c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 --2- 172.21.15.143:0/3474871289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3240c5220 0x7fe3240c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 -- 172.21.15.143:0/3474871289 >> v2:172.21.15.204:3300/0 conn(0x7fe3240d19f0 msgr2=0x7fe3240cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:53.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 --2- 172.21.15.143:0/3474871289 >> v2:172.21.15.204:3300/0 conn(0x7fe3240d19f0 0x7fe3240cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe329800640 1 -- 172.21.15.143:0/3474871289 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3240cfae0 con 0x7fe32415b260 2024-01-15T18:09:53.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.518+0000 7fe328e00640 1 --2- 172.21.15.143:0/3474871289 >> v2:172.21.15.204:3300/0 conn(0x7fe3240d19f0 0x7fe3240cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:53.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe329800640 1 --2- 172.21.15.143:0/3474871289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32415b260 0x7fe3240cd810 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe32409ee40 tx=0x7fe31401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e646a1c6b5371538 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:53.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe323e00640 1 -- 172.21.15.143:0/3474871289 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140aa5f0 con 0x7fe32415b260 2024-01-15T18:09:53.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe323e00640 1 -- 172.21.15.143:0/3474871289 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3140aa750 con 0x7fe32415b260 2024-01-15T18:09:53.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe323e00640 1 -- 172.21.15.143:0/3474871289 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140aaae0 con 0x7fe32415b260 2024-01-15T18:09:53.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 -- 172.21.15.143:0/3474871289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32415b260 msgr2=0x7fe3240cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 --2- 172.21.15.143:0/3474871289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32415b260 0x7fe3240cd810 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe32409ee40 tx=0x7fe31401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 -- 172.21.15.143:0/3474871289 shutdown_connections 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 --2- 172.21.15.143:0/3474871289 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3240c5220 0x7fe3240c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 --2- 172.21.15.143:0/3474871289 >> v2:172.21.15.204:3300/0 conn(0x7fe3240d19f0 0x7fe3240cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 --2- 172.21.15.143:0/3474871289 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe32415b260 0x7fe3240cd810 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.522+0000 7fe32c200640 1 -- 172.21.15.143:0/3474871289 >> 172.21.15.143:0/3474871289 conn(0x7fe3240d0ed0 msgr2=0x7fe3240d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 -- 172.21.15.143:0/3474871289 shutdown_connections 2024-01-15T18:09:53.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 -- 172.21.15.143:0/3474871289 wait complete. 2024-01-15T18:09:53.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 Processor -- start 2024-01-15T18:09:53.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 -- start start 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240c5220 0x7fe32409cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 0x7fe32415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe32415b260 0x7fe3241f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32409ac70 con 0x7fe3240d19f0 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32415aa90 con 0x7fe3240c5220 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32409d3e0 con 0x7fe32415b260 2024-01-15T18:09:53.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 0x7fe32415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 0x7fe32415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34316/0 (socket says 172.21.15.143:34316) 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 -- 172.21.15.143:0/3386658149 learned_addr learned my addr 172.21.15.143:0/3386658149 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe329800640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240c5220 0x7fe32409cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe32a200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe32415b260 0x7fe3241f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 -- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe32415b260 msgr2=0x7fe3241f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe32415b260 0x7fe3241f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 -- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240c5220 msgr2=0x7fe32409cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240c5220 0x7fe32409cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 -- 172.21.15.143:0/3386658149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3140094d0 con 0x7fe3240d19f0 2024-01-15T18:09:53.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.526+0000 7fe328e00640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 0x7fe32415a550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00e7e0 tx=0x7fe30c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:53.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00c6d0 con 0x7fe3240d19f0 2024-01-15T18:09:53.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30c00c830 con 0x7fe3240d19f0 2024-01-15T18:09:53.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00cbe0 con 0x7fe3240d19f0 2024-01-15T18:09:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32409d6c0 con 0x7fe3240d19f0 2024-01-15T18:09:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32409b1a0 con 0x7fe3240d19f0 2024-01-15T18:09:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe324067500 con 0x7fe3240d19f0 2024-01-15T18:09:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7fe30c00cd40 con 0x7fe3240d19f0 2024-01-15T18:09:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe321600640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe2fc066f20 0x7fe2fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:53.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe30c084eb0 con 0x7fe3240d19f0 2024-01-15T18:09:53.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.530+0000 7fe329800640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe2fc066f20 0x7fe2fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:53.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.534+0000 7fe329800640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe2fc066f20 0x7fe2fc0693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe3140ae000 tx=0x7fe3140b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:53.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.534+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe30c051780 con 0x7fe3240d19f0 2024-01-15T18:09:53.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.714+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3240b6d50 con 0x7fe2fc066f20 2024-01-15T18:09:53.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe321600640 1 -- 172.21.15.143:0/3386658149 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fe3240b6d50 con 0x7fe2fc066f20 2024-01-15T18:09:53.722 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe2fc066f20 msgr2=0x7fe2fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe2fc066f20 0x7fe2fc0693e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe3140ae000 tx=0x7fe3140b5040 comp rx=0 tx=0).stop 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 msgr2=0x7fe32415a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 0x7fe32415a550 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00e7e0 tx=0x7fe30c00eca0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 shutdown_connections 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe2fc066f20 0x7fe2fc0693e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe32415b260 0x7fe3241f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3240d19f0 0x7fe32415a550 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 --2- 172.21.15.143:0/3386658149 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3240c5220 0x7fe32409cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.718+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 >> 172.21.15.143:0/3386658149 conn(0x7fe3240d0ed0 msgr2=0x7fe3240c8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.722+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 shutdown_connections 2024-01-15T18:09:53.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:53.722+0000 7fe32c200640 1 -- 172.21.15.143:0/3386658149 wait complete. 2024-01-15T18:09:53.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:53.743 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3) 2024-01-15T18:09:53.756 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:53.757 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:09:53.757 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:53.757 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:09:55.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:55.759 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3) 2024-01-15T18:09:55.773 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:55.774 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:09:55.774 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:55.775 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:09:57.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:57.775 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3) 2024-01-15T18:09:57.789 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:57.789 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:09:57.789 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:57.790 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:09:58.036 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:58.040 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:59.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:59.791 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3) 2024-01-15T18:09:59.808 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/ac585799-2db8-459f-80f8-005a56067df3 2024-01-15T18:09:59.808 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:09:59.808 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:09:59.809 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:09:59.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42251 2024-01-15T18:10:00.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 Processor -- start 2024-01-15T18:10:00.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 -- start start 2024-01-15T18:10:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00ce690 0x7f3ee00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3ee00b1da0 0x7f3ee00b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee0070bf0 con 0x7f3ee00b3bf0 2024-01-15T18:10:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee015dba0 con 0x7f3ee00b1da0 2024-01-15T18:10:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00b2740 con 0x7f3ee00ce690 2024-01-15T18:10:00.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00ce690 0x7f3ee00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3ee00b1da0 0x7f3ee00b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3ee00b1da0 0x7f3ee00b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:34280/0 (socket says 172.21.15.143:34280) 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3ee5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00ce690 0x7f3ee00ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:47586/0 (socket says 172.21.15.143:47586) 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 -- 172.21.15.143:0/3263710152 learned_addr learned my addr 172.21.15.143:0/3263710152 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 -- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00ce690 msgr2=0x7f3ee00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 --2- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00ce690 0x7f3ee00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 -- 172.21.15.143:0/3263710152 >> v1:172.21.15.204:6789/0 conn(0x7f3ee00b3bf0 legacy=0x7f3ee00b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:09:59.998+0000 7f3edfe00640 1 -- 172.21.15.143:0/3263710152 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee00b4920 con 0x7f3ee00b1da0 2024-01-15T18:10:00.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.002+0000 7f3edfe00640 1 --2- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee00b2200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009870 tx=0x7f3ec801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dff8eb2f1e5392a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.002+0000 7f3edf400640 1 -- 172.21.15.143:0/3263710152 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80aa670 con 0x7f3ee00b1da0 2024-01-15T18:10:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.002+0000 7f3edf400640 1 -- 172.21.15.143:0/3263710152 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec80aa7d0 con 0x7f3ee00b1da0 2024-01-15T18:10:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.002+0000 7f3edf400640 1 -- 172.21.15.143:0/3263710152 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80aab60 con 0x7f3ee00b1da0 2024-01-15T18:10:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.002+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 msgr2=0x7f3ee00b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.002+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee00b2200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8009870 tx=0x7f3ec801c510 comp rx=0 tx=0).stop 2024-01-15T18:10:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3263710152 shutdown_connections 2024-01-15T18:10:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee00b2200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3263710152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00ce690 0x7f3ee00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3263710152 >> 172.21.15.143:0/3263710152 conn(0x7f3ee006c3f0 msgr2=0x7f3ee006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3263710152 shutdown_connections 2024-01-15T18:10:00.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3263710152 wait complete. 2024-01-15T18:10:00.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 Processor -- start 2024-01-15T18:10:00.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- start start 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee0145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00b3bf0 0x7f3ee00b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 0x7f3ee015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0139a50 con 0x7f3ee00ce690 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0143750 con 0x7f3ee00b1da0 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0143be0 con 0x7f3ee00b3bf0 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 0x7f3ee015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 0x7f3ee015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56070/0 (socket says 172.21.15.143:56070) 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 -- 172.21.15.143:0/3525099411 learned_addr learned my addr 172.21.15.143:0/3525099411 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5000640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee0145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3edfe00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00b3bf0 0x7f3ee00b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 -- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00b3bf0 msgr2=0x7f3ee00b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00b3bf0 0x7f3ee00b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 -- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 msgr2=0x7f3ee0145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee0145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.006+0000 7f3ee5a00640 1 -- 172.21.15.143:0/3525099411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec80094d0 con 0x7f3ee00ce690 2024-01-15T18:10:00.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3ee5a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 0x7f3ee015a1e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c6f0 tx=0x7f3ed800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8003980 con 0x7f3ee00ce690 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed8003ae0 con 0x7f3ee00ce690 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8003e90 con 0x7f3ee00ce690 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee0139ca0 con 0x7f3ee00ce690 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee01433e0 con 0x7f3ee00ce690 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee013a1c0 con 0x7f3ee00ce690 2024-01-15T18:10:00.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7f3ed800a850 con 0x7f3ee00ce690 2024-01-15T18:10:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3edcc00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eb8066f20 0x7f3eb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3ed8084c70 con 0x7f3ee00ce690 2024-01-15T18:10:00.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.010+0000 7f3ee5000640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eb8066f20 0x7f3eb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.014+0000 7f3ee5000640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eb8066f20 0x7f3eb80693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ac0 tx=0x7f3ed0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.014+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3ee013a1c0 con 0x7f3ee00ce690 2024-01-15T18:10:00.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.194+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42251", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ee0143240 con 0x7f3eb8066f20 2024-01-15T18:10:00.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3edcc00640 1 -- 172.21.15.143:0/3525099411 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3ee0143240 con 0x7f3eb8066f20 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eb8066f20 msgr2=0x7f3eb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eb8066f20 0x7f3eb80693e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009ac0 tx=0x7f3ed0009190 comp rx=0 tx=0).stop 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 msgr2=0x7f3ee015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 0x7f3ee015a1e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800c6f0 tx=0x7f3ed800cbb0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 shutdown_connections 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3eb8066f20 0x7f3eb80693e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ee00ce690 0x7f3ee015a1e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ee00b3bf0 0x7f3ee00b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 --2- 172.21.15.143:0/3525099411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ee00b1da0 0x7f3ee0145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 >> 172.21.15.143:0/3525099411 conn(0x7f3ee006c3f0 msgr2=0x7f3ee0146740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 shutdown_connections 2024-01-15T18:10:00.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.246+0000 7f3ee7a00640 1 -- 172.21.15.143:0/3525099411 wait complete. 2024-01-15T18:10:00.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.250+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.250+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.250+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.250+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.253 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.250+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.270 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.266+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.270 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.266+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.270 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.266+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.270 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.266+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.270 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:00.266+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:00.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42252 2024-01-15T18:10:00.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.414+0000 7fc33de00640 1 Processor -- start 2024-01-15T18:10:00.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.414+0000 7fc33de00640 1 -- start start 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc33de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc338143780 0x7fc3380cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc33de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3380ce520 0x7fc338071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc33de00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380ceb50 con 0x7fc3380720f0 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc33de00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc338072580 con 0x7fc3380ce520 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc33de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33815c110 con 0x7fc338143780 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc337400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc3380720f0 0x7fc33815a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:53028/0 (socket says 172.21.15.143:53028) 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc337400640 1 -- 172.21.15.143:0/3017797639 learned_addr learned my addr 172.21.15.143:0/3017797639 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc336000640 1 --2- 172.21.15.143:0/3017797639 >> v2:172.21.15.204:3300/0 conn(0x7fc3380ce520 0x7fc338071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc336a00640 1 --2- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc338143780 0x7fc3380cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc335600640 1 -- 172.21.15.143:0/3017797639 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807425726 0 0) 0x7fc3300014a0 con 0x7fc3380720f0 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc335600640 1 -- 172.21.15.143:0/3017797639 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc318003530 con 0x7fc3380720f0 2024-01-15T18:10:00.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc336000640 1 -- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc338143780 msgr2=0x7fc3380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:00.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc336000640 1 --2- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc338143780 0x7fc3380cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc336000640 1 -- 172.21.15.143:0/3017797639 >> v1:172.21.15.143:6789/0 conn(0x7fc3380720f0 legacy=0x7fc33815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.418+0000 7fc336000640 1 -- 172.21.15.143:0/3017797639 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33815b7e0 con 0x7fc3380ce520 2024-01-15T18:10:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc336000640 1 --2- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3380ce520 0x7fc338071bb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097c0 tx=0x7fc32001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c990619425ae5006 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc335600640 1 -- 172.21.15.143:0/3017797639 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200ad640 con 0x7fc3380ce520 2024-01-15T18:10:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc335600640 1 -- 172.21.15.143:0/3017797639 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3200ad7a0 con 0x7fc3380ce520 2024-01-15T18:10:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc335600640 1 -- 172.21.15.143:0/3017797639 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200adb30 con 0x7fc3380ce520 2024-01-15T18:10:00.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 -- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3380ce520 msgr2=0x7fc338071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 --2- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3380ce520 0x7fc338071bb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097c0 tx=0x7fc32001c480 comp rx=0 tx=0).stop 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 -- 172.21.15.143:0/3017797639 shutdown_connections 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 --2- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3380ce520 0x7fc338071bb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 --2- 172.21.15.143:0/3017797639 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc338143780 0x7fc3380cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 -- 172.21.15.143:0/3017797639 >> 172.21.15.143:0/3017797639 conn(0x7fc33806c3f0 msgr2=0x7fc33806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 -- 172.21.15.143:0/3017797639 shutdown_connections 2024-01-15T18:10:00.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.422+0000 7fc33de00640 1 -- 172.21.15.143:0/3017797639 wait complete. 2024-01-15T18:10:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 Processor -- start 2024-01-15T18:10:00.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- start start 2024-01-15T18:10:00.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 0x7fc3381597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3380ce520 0x7fc338133a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc338143780 0x7fc3380aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338134b20 con 0x7fc3380720f0 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338132ea0 con 0x7fc338143780 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338132320 con 0x7fc3380ce520 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 0x7fc3381597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 0x7fc3381597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56128/0 (socket says 172.21.15.143:56128) 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 -- 172.21.15.143:0/3918717273 learned_addr learned my addr 172.21.15.143:0/3918717273 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336000640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3380ce520 0x7fc338133a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc337400640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc338143780 0x7fc3380aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 -- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3380ce520 msgr2=0x7fc338133a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3380ce520 0x7fc338133a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 -- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc338143780 msgr2=0x7fc3380aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc338143780 0x7fc3380aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 -- 172.21.15.143:0/3918717273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc320009420 con 0x7fc3380720f0 2024-01-15T18:10:00.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc337400640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc338143780 0x7fc3380aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc336a00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 0x7fc3381597c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e870 tx=0x7fc32800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328009510 con 0x7fc3380720f0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32800eec0 con 0x7fc3380720f0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800cb40 con 0x7fc3380720f0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc338132600 con 0x7fc3380720f0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc338132bc0 con 0x7fc3380720f0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.426+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc338067500 con 0x7fc3380720f0 2024-01-15T18:10:00.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.430+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7fc32800cca0 con 0x7fc3380720f0 2024-01-15T18:10:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.430+0000 7fc32f400640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc310066f20 0x7fc3100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.430+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc328085160 con 0x7fc3380720f0 2024-01-15T18:10:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.430+0000 7fc336000640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc310066f20 0x7fc3100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.430+0000 7fc336000640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc310066f20 0x7fc3100693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc3200af000 tx=0x7fc320008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.434+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc3280b6280 con 0x7fc3380720f0 2024-01-15T18:10:00.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.614+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42252", "target": ["mon-mgr", ""]}) v1 -- 0x7fc338070f20 con 0x7fc310066f20 2024-01-15T18:10:00.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc32f400640 1 -- 172.21.15.143:0/3918717273 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc338070f20 con 0x7fc310066f20 2024-01-15T18:10:00.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc310066f20 msgr2=0x7fc3100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc310066f20 0x7fc3100693e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc3200af000 tx=0x7fc320008040 comp rx=0 tx=0).stop 2024-01-15T18:10:00.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 msgr2=0x7fc3381597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 0x7fc3381597c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc32800e870 tx=0x7fc32800ec70 comp rx=0 tx=0).stop 2024-01-15T18:10:00.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 shutdown_connections 2024-01-15T18:10:00.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc310066f20 0x7fc3100693e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc338143780 0x7fc3380aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3380ce520 0x7fc338133a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 --2- 172.21.15.143:0/3918717273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3380720f0 0x7fc3381597c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 >> 172.21.15.143:0/3918717273 conn(0x7fc33806c3f0 msgr2=0x7fc33815ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 shutdown_connections 2024-01-15T18:10:00.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.666+0000 7fc33de00640 1 -- 172.21.15.143:0/3918717273 wait complete. 2024-01-15T18:10:00.693 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42253 2024-01-15T18:10:00.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 Processor -- start 2024-01-15T18:10:00.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 -- start start 2024-01-15T18:10:00.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd806c540 0x7f5fd80cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fd806b7b0 0x7f5fd80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80d02d0 con 0x7f5fd80cf150 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd80cf6c0 con 0x7f5fd806b7b0 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80cf9a0 con 0x7f5fd806c540 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fd806b7b0 0x7f5fd80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5fd806b7b0 0x7f5fd80cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56136/0 (socket says 172.21.15.143:56136) 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 -- 172.21.15.143:0/3986493522 learned_addr learned my addr 172.21.15.143:0/3986493522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:00.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fd7e00640 1 --2- 172.21.15.143:0/3986493522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd806c540 0x7f5fd80cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 -- 172.21.15.143:0/3986493522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd806c540 msgr2=0x7f5fd80cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:00.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 --2- 172.21.15.143:0/3986493522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd806c540 0x7f5fd80cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 -- 172.21.15.143:0/3986493522 >> v1:172.21.15.204:6789/0 conn(0x7f5fd80cf150 legacy=0x7f5fd815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.834+0000 7f5fdd200640 1 -- 172.21.15.143:0/3986493522 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80cfc80 con 0x7f5fd806b7b0 2024-01-15T18:10:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdd200640 1 --2- 172.21.15.143:0/3986493522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806b7b0 0x7f5fd80cdd20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009e50 tx=0x7f5fd001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cb07b16fa265ab server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fd6a00640 1 -- 172.21.15.143:0/3986493522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00a5830 con 0x7f5fd806b7b0 2024-01-15T18:10:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fd6a00640 1 -- 172.21.15.143:0/3986493522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd00a5990 con 0x7f5fd806b7b0 2024-01-15T18:10:00.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fd6a00640 1 -- 172.21.15.143:0/3986493522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd00a5d20 con 0x7f5fd806b7b0 2024-01-15T18:10:00.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 -- 172.21.15.143:0/3986493522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806b7b0 msgr2=0x7f5fd80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3986493522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806b7b0 0x7f5fd80cdd20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009e50 tx=0x7f5fd001c950 comp rx=0 tx=0).stop 2024-01-15T18:10:00.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 -- 172.21.15.143:0/3986493522 shutdown_connections 2024-01-15T18:10:00.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3986493522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806b7b0 0x7f5fd80cdd20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3986493522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd806c540 0x7f5fd80cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 -- 172.21.15.143:0/3986493522 >> 172.21.15.143:0/3986493522 conn(0x7f5fd80d0e20 msgr2=0x7f5fd80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 -- 172.21.15.143:0/3986493522 shutdown_connections 2024-01-15T18:10:00.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.838+0000 7f5fdf200640 1 -- 172.21.15.143:0/3986493522 wait complete. 2024-01-15T18:10:00.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 Processor -- start 2024-01-15T18:10:00.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 -- start start 2024-01-15T18:10:00.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd806b7b0 0x7f5fd8158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 0x7f5fd815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd80cf150 0x7f5fd806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8159270 con 0x7f5fd806c540 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8160a60 con 0x7f5fd806b7b0 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd815fca0 con 0x7f5fd80cf150 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 0x7f5fd815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 0x7f5fd815dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56146/0 (socket says 172.21.15.143:56146) 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 -- 172.21.15.143:0/3039356677 learned_addr learned my addr 172.21.15.143:0/3039356677 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7e00640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd806b7b0 0x7f5fd8158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdd200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd80cf150 0x7f5fd806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 -- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd80cf150 msgr2=0x7f5fd806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd80cf150 0x7f5fd806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 -- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd806b7b0 msgr2=0x7f5fd8158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd806b7b0 0x7f5fd8158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 -- 172.21.15.143:0/3039356677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0009b30 con 0x7f5fd806c540 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fd7400640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 0x7f5fd815dc20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c3c0 tx=0x7f5fc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc00052d0 con 0x7f5fd806c540 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc0005430 con 0x7f5fd806c540 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd815ff80 con 0x7f5fd806c540 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0003840 con 0x7f5fd806c540 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.842+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd8160540 con 0x7f5fd806c540 2024-01-15T18:10:00.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.846+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd8067500 con 0x7f5fd806c540 2024-01-15T18:10:00.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.846+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) v1 ==== 86721+0+0 (secure 0 0 0) 0x7f5fc00039a0 con 0x7f5fd806c540 2024-01-15T18:10:00.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.846+0000 7f5fcfe00640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fb0066f20 0x7f5fb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.846+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5fc0084400 con 0x7f5fd806c540 2024-01-15T18:10:00.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.846+0000 7f5fd7e00640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fb0066f20 0x7f5fb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.846+0000 7f5fd7e00640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fb0066f20 0x7f5fb00693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8159a50 tx=0x7f5fc8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:00.850+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5fc0050de0 con 0x7f5fd806c540 2024-01-15T18:10:01.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:01.030+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42253", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fd80cbcf0 con 0x7f5fb0066f20 2024-01-15T18:10:01.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:01.626+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7f5fc00511b0 con 0x7f5fd806c540 2024-01-15T18:10:02.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.902+0000 7f5fcfe00640 1 -- 172.21.15.143:0/3039356677 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5fc8011a60 con 0x7f5fb0066f20 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fb0066f20 msgr2=0x7f5fb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fb0066f20 0x7f5fb00693e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8159a50 tx=0x7f5fc8009190 comp rx=0 tx=0).stop 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 msgr2=0x7f5fd815dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 0x7f5fd815dc20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000c3c0 tx=0x7f5fc000c880 comp rx=0 tx=0).stop 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 shutdown_connections 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fb0066f20 0x7f5fb00693e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5fd80cf150 0x7f5fd806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5fd806c540 0x7f5fd815dc20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 --2- 172.21.15.143:0/3039356677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5fd806b7b0 0x7f5fd8158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 >> 172.21.15.143:0/3039356677 conn(0x7f5fd80d0e20 msgr2=0x7f5fd80af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 shutdown_connections 2024-01-15T18:10:02.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:02.906+0000 7f5fdf200640 1 -- 172.21.15.143:0/3039356677 wait complete. 2024-01-15T18:10:02.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42254 2024-01-15T18:10:03.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.070+0000 7f52c5a00640 1 Processor -- start 2024-01-15T18:10:03.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.070+0000 7f52c5a00640 1 -- start start 2024-01-15T18:10:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52c5a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f52c0157300 0x7f52c0159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52c5a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52c00b21d0 0x7f52c00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52c5a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00b4d20 con 0x7f52c00b48f0 2024-01-15T18:10:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52c5a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00b35a0 con 0x7f52c00b21d0 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52c5a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00b2a20 con 0x7f52c0157300 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52c00b21d0 0x7f52c00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52c00b21d0 0x7f52c00b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56152/0 (socket says 172.21.15.143:56152) 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 -- 172.21.15.143:0/3365531790 learned_addr learned my addr 172.21.15.143:0/3365531790 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52bea00640 1 --2- 172.21.15.143:0/3365531790 >> v2:172.21.15.204:3300/0 conn(0x7f52c0157300 0x7f52c0159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 -- 172.21.15.143:0/3365531790 >> v2:172.21.15.204:3300/0 conn(0x7f52c0157300 msgr2=0x7f52c0159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 --2- 172.21.15.143:0/3365531790 >> v2:172.21.15.204:3300/0 conn(0x7f52c0157300 0x7f52c0159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 -- 172.21.15.143:0/3365531790 >> v1:172.21.15.204:6789/0 conn(0x7f52c00b48f0 legacy=0x7f52c015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.074+0000 7f52be000640 1 -- 172.21.15.143:0/3365531790 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00b2d00 con 0x7f52c00b21d0 2024-01-15T18:10:03.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52be000640 1 --2- 172.21.15.143:0/3365531790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c00b21d0 0x7f52c00b43b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f52c00b1aa0 tx=0x7f52a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7599737f847a761 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52bd600640 1 -- 172.21.15.143:0/3365531790 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80aa430 con 0x7f52c00b21d0 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52bd600640 1 -- 172.21.15.143:0/3365531790 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a80aa590 con 0x7f52c00b21d0 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52bd600640 1 -- 172.21.15.143:0/3365531790 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80aa920 con 0x7f52c00b21d0 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 -- 172.21.15.143:0/3365531790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c00b21d0 msgr2=0x7f52c00b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 --2- 172.21.15.143:0/3365531790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c00b21d0 0x7f52c00b43b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f52c00b1aa0 tx=0x7f52a801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 -- 172.21.15.143:0/3365531790 shutdown_connections 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 --2- 172.21.15.143:0/3365531790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c00b21d0 0x7f52c00b43b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 --2- 172.21.15.143:0/3365531790 >> v2:172.21.15.204:3300/0 conn(0x7f52c0157300 0x7f52c0159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 -- 172.21.15.143:0/3365531790 >> 172.21.15.143:0/3365531790 conn(0x7f52c006c3f0 msgr2=0x7f52c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:03.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 -- 172.21.15.143:0/3365531790 shutdown_connections 2024-01-15T18:10:03.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 -- 172.21.15.143:0/3365531790 wait complete. 2024-01-15T18:10:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 Processor -- start 2024-01-15T18:10:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.078+0000 7f52c5a00640 1 -- start start 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52c00b21d0 0x7f52c015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52c00b48f0 0x7f52c01f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 0x7f52c01405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c0157c20 con 0x7f52c0157300 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015c5e0 con 0x7f52c00b48f0 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015a430 con 0x7f52c00b21d0 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 0x7f52c01405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 0x7f52c01405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56162/0 (socket says 172.21.15.143:56162) 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 -- 172.21.15.143:0/587897150 learned_addr learned my addr 172.21.15.143:0/587897150 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:03.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52be000640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52c00b48f0 0x7f52c01f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bea00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52c00b21d0 0x7f52c015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 -- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52c00b21d0 msgr2=0x7f52c015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52c00b21d0 0x7f52c015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 -- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52c00b48f0 msgr2=0x7f52c01f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52c00b48f0 0x7f52c01f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 -- 172.21.15.143:0/587897150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a80094d0 con 0x7f52c0157300 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bea00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52c00b21d0 0x7f52c015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52bf400640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 0x7f52c01405e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c730 tx=0x7f52b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8003ba0 con 0x7f52c0157300 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8003d00 con 0x7f52c0157300 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c01f6b10 con 0x7f52c0157300 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8005310 con 0x7f52c0157300 2024-01-15T18:10:03.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c015ac00 con 0x7f52c0157300 2024-01-15T18:10:03.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.082+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c0157d60 con 0x7f52c0157300 2024-01-15T18:10:03.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.086+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7f52b8005680 con 0x7f52c0157300 2024-01-15T18:10:03.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.086+0000 7f52b6a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5298067040 0x7f5298069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.086+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f52b80854f0 con 0x7f52c0157300 2024-01-15T18:10:03.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.086+0000 7f52bea00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5298067040 0x7f5298069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.086+0000 7f52bea00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5298067040 0x7f5298069500 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b9f0 tx=0x7f52b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.090+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f52c0157d60 con 0x7f52c0157300 2024-01-15T18:10:03.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:03.266+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42254", "target": ["mon-mgr", ""]}) v1 -- 0x7f52c00ce670 con 0x7f5298067040 2024-01-15T18:10:07.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52b6a00640 1 -- 172.21.15.143:0/587897150 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f52c00ce670 con 0x7f5298067040 2024-01-15T18:10:07.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5298067040 msgr2=0x7f5298069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5298067040 0x7f5298069500 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b9f0 tx=0x7f52b0008040 comp rx=0 tx=0).stop 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 msgr2=0x7f52c01405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 0x7f52c01405e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f52b800c730 tx=0x7f52b800cbf0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 shutdown_connections 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5298067040 0x7f5298069500 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52c0157300 0x7f52c01405e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52c00b48f0 0x7f52c01f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 --2- 172.21.15.143:0/587897150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52c00b21d0 0x7f52c015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 >> 172.21.15.143:0/587897150 conn(0x7f52c006c3f0 msgr2=0x7f52c00cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 shutdown_connections 2024-01-15T18:10:07.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:07.902+0000 7f52c5a00640 1 -- 172.21.15.143:0/587897150 wait complete. 2024-01-15T18:10:07.931 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42255 2024-01-15T18:10:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 Processor -- start 2024-01-15T18:10:08.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 -- start start 2024-01-15T18:10:08.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3a8141660 0x7fd3a80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a815ec80 0x7fd3a8141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a815f310 con 0x7fd3a8142470 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a815fbb0 con 0x7fd3a8141660 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3ad600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a815f100 con 0x7fd3a815ec80 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3a8141660 0x7fd3a80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a7400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd3a8142470 0x7fd3a8071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45358/0 (socket says 172.21.15.143:45358) 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a7400640 1 -- 172.21.15.143:0/1977021906 learned_addr learned my addr 172.21.15.143:0/1977021906 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:08.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3a8141660 0x7fd3a80ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49156/0 (socket says 172.21.15.143:49156) 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6000640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a815ec80 0x7fd3a8141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1977021906 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a815ec80 msgr2=0x7fd3a8141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6a00640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a815ec80 0x7fd3a8141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1977021906 >> v1:172.21.15.143:6789/0 conn(0x7fd3a8142470 legacy=0x7fd3a8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1977021906 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a815f5f0 con 0x7fd3a8141660 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a5600640 1 -- 172.21.15.143:0/1977021906 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039332959 0 0) 0x7fd3a815f310 con 0x7fd3a8142470 2024-01-15T18:10:08.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.074+0000 7fd3a6000640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a815ec80 0x7fd3a8141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3a6a00640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a80ce560 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd3980097c0 tx=0x7fd39801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52a153fe70cab85 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3a5600640 1 -- 172.21.15.143:0/1977021906 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980ab650 con 0x7fd3a8141660 2024-01-15T18:10:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3a5600640 1 -- 172.21.15.143:0/1977021906 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3980ab7b0 con 0x7fd3a8141660 2024-01-15T18:10:08.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3a5600640 1 -- 172.21.15.143:0/1977021906 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980abb40 con 0x7fd3a8141660 2024-01-15T18:10:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 -- 172.21.15.143:0/1977021906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 msgr2=0x7fd3a80ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a80ce560 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd3980097c0 tx=0x7fd39801c450 comp rx=0 tx=0).stop 2024-01-15T18:10:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 -- 172.21.15.143:0/1977021906 shutdown_connections 2024-01-15T18:10:08.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a815ec80 0x7fd3a8141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1977021906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a80ce560 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 -- 172.21.15.143:0/1977021906 >> 172.21.15.143:0/1977021906 conn(0x7fd3a806c3f0 msgr2=0x7fd3a806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 -- 172.21.15.143:0/1977021906 shutdown_connections 2024-01-15T18:10:08.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.078+0000 7fd3ad600640 1 -- 172.21.15.143:0/1977021906 wait complete. 2024-01-15T18:10:08.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 Processor -- start 2024-01-15T18:10:08.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 -- start start 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a8143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a8142470 0x7fd3a80b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3a815ec80 0x7fd3a80b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a815ada0 con 0x7fd3a8141660 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a815a8b0 con 0x7fd3a815ec80 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80b27e0 con 0x7fd3a8142470 2024-01-15T18:10:08.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a8143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a8143e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49166/0 (socket says 172.21.15.143:49166) 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1793769319 learned_addr learned my addr 172.21.15.143:0/1793769319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6000640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a8142470 0x7fd3a80b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a7400640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3a815ec80 0x7fd3a80b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a8142470 msgr2=0x7fd3a80b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a8142470 0x7fd3a80b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3a815ec80 msgr2=0x7fd3a80b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3a815ec80 0x7fd3a80b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 -- 172.21.15.143:0/1793769319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009420 con 0x7fd3a8141660 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a7400640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3a815ec80 0x7fd3a80b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6000640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a8142470 0x7fd3a80b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:08.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3a6a00640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a8143e30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd3980ad040 tx=0x7fd39801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980b5810 con 0x7fd3a8141660 2024-01-15T18:10:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3980b59b0 con 0x7fd3a8141660 2024-01-15T18:10:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3980b5d60 con 0x7fd3a8141660 2024-01-15T18:10:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a80cd150 con 0x7fd3a8141660 2024-01-15T18:10:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.082+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a813aa80 con 0x7fd3a8141660 2024-01-15T18:10:08.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.086+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a8067500 con 0x7fd3a8141660 2024-01-15T18:10:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.086+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7fd3980b4390 con 0x7fd3a8141660 2024-01-15T18:10:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.086+0000 7fd39ea00640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd380067040 0x7fd380069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.086+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd39811fe50 con 0x7fd3a8141660 2024-01-15T18:10:08.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.086+0000 7fd3a6000640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd380067040 0x7fd380069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.086+0000 7fd3a6000640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd380067040 0x7fd380069500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80b1ae0 tx=0x7fd390008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:08.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.090+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd39811d700 con 0x7fd3a8141660 2024-01-15T18:10:08.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:08.270+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3a8072410 con 0x7fd380067040 2024-01-15T18:10:12.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.894+0000 7fd39ea00640 1 -- 172.21.15.143:0/1793769319 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd39000a1d0 con 0x7fd380067040 2024-01-15T18:10:12.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd380067040 msgr2=0x7fd380069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd380067040 0x7fd380069500 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80b1ae0 tx=0x7fd390008040 comp rx=0 tx=0).stop 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 msgr2=0x7fd3a8143e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a8143e30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd3980ad040 tx=0x7fd39801ac90 comp rx=0 tx=0).stop 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 shutdown_connections 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd380067040 0x7fd380069500 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3a815ec80 0x7fd3a80b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3a8142470 0x7fd3a80b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 --2- 172.21.15.143:0/1793769319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3a8141660 0x7fd3a8143e30 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 >> 172.21.15.143:0/1793769319 conn(0x7fd3a806c3f0 msgr2=0x7fd3a8140980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 shutdown_connections 2024-01-15T18:10:12.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:12.898+0000 7fd3ad600640 1 -- 172.21.15.143:0/1793769319 wait complete. 2024-01-15T18:10:12.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42256 2024-01-15T18:10:13.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.078+0000 7f9b08e00640 1 Processor -- start 2024-01-15T18:10:13.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.078+0000 7f9b08e00640 1 -- start start 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b08e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9b04141660 0x7f9b040ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b08e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0415f310 con 0x7f9b04071d00 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b08e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b0415f5f0 con 0x7f9b0415ec80 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b08e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0415fbb0 con 0x7f9b04141660 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b02800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9b04071d00 0x7f9b040b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45368/0 (socket says 172.21.15.143:45368) 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b02800640 1 -- 172.21.15.143:0/32152428 learned_addr learned my addr 172.21.15.143:0/32152428 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:13.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b01e00640 1 --2- 172.21.15.143:0/32152428 >> v2:172.21.15.143:3300/0 conn(0x7f9b04141660 0x7f9b040ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b01e00640 1 -- 172.21.15.143:0/32152428 >> v1:172.21.15.204:6789/0 conn(0x7f9b0415ec80 legacy=0x7f9b04141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b01e00640 1 -- 172.21.15.143:0/32152428 >> v1:172.21.15.143:6789/0 conn(0x7f9b04071d00 legacy=0x7f9b040b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b01e00640 1 -- 172.21.15.143:0/32152428 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0415f8d0 con 0x7f9b04141660 2024-01-15T18:10:13.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.082+0000 7f9b00a00640 1 -- 172.21.15.143:0/32152428 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720104615 0 0) 0x7f9b0415f310 con 0x7f9b04071d00 2024-01-15T18:10:13.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b01e00640 1 --2- 172.21.15.143:0/32152428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b04141660 0x7f9b040ce560 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9af40097c0 tx=0x7f9af401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa98999a06ba07b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b00a00640 1 -- 172.21.15.143:0/32152428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40ab670 con 0x7f9b04141660 2024-01-15T18:10:13.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b00a00640 1 -- 172.21.15.143:0/32152428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af40ab7d0 con 0x7f9b04141660 2024-01-15T18:10:13.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b00a00640 1 -- 172.21.15.143:0/32152428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40abb60 con 0x7f9b04141660 2024-01-15T18:10:13.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 -- 172.21.15.143:0/32152428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b04141660 msgr2=0x7f9b040ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 --2- 172.21.15.143:0/32152428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b04141660 0x7f9b040ce560 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9af40097c0 tx=0x7f9af401c450 comp rx=0 tx=0).stop 2024-01-15T18:10:13.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 -- 172.21.15.143:0/32152428 shutdown_connections 2024-01-15T18:10:13.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 --2- 172.21.15.143:0/32152428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b04141660 0x7f9b040ce560 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 -- 172.21.15.143:0/32152428 >> 172.21.15.143:0/32152428 conn(0x7f9b0406c3f0 msgr2=0x7f9b0406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:13.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 -- 172.21.15.143:0/32152428 shutdown_connections 2024-01-15T18:10:13.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 -- 172.21.15.143:0/32152428 wait complete. 2024-01-15T18:10:13.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 Processor -- start 2024-01-15T18:10:13.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.086+0000 7f9b08e00640 1 -- start start 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b04071d00 0x7f9b0413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b04141660 0x7f9b04137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 0x7f9b0415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415f130 con 0x7f9b0415ec80 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415e5c0 con 0x7f9b04071d00 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0413bfb0 con 0x7f9b04141660 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 0x7f9b0415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 0x7f9b0415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49176/0 (socket says 172.21.15.143:49176) 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 -- 172.21.15.143:0/348406437 learned_addr learned my addr 172.21.15.143:0/348406437 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:13.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b01e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b04071d00 0x7f9b0413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b01400640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b04141660 0x7f9b04137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 -- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b04141660 msgr2=0x7f9b04137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b04141660 0x7f9b04137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 -- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b04071d00 msgr2=0x7f9b0413ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b04071d00 0x7f9b0413ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 -- 172.21.15.143:0/348406437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009420 con 0x7f9b0415ec80 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b02800640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 0x7f9b0415d630 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9afc001570 tx=0x7f9afc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00cea0 con 0x7f9b0415ec80 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9afc00a5e0 con 0x7f9b0415ec80 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00a990 con 0x7f9b0415ec80 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b040cd150 con 0x7f9b0415ec80 2024-01-15T18:10:13.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b04143270 con 0x7f9b0415ec80 2024-01-15T18:10:13.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04067500 con 0x7f9b0415ec80 2024-01-15T18:10:13.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.090+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7f9afc00aaf0 con 0x7f9b0415ec80 2024-01-15T18:10:13.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.094+0000 7f9afa000640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9adc067040 0x7f9adc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.094+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9afc084970 con 0x7f9b0415ec80 2024-01-15T18:10:13.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.094+0000 7f9b01e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9adc067040 0x7f9adc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.094+0000 7f9b01e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9adc067040 0x7f9adc069500 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9af40af000 tx=0x7f9af40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.098+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9afc081e10 con 0x7f9b0415ec80 2024-01-15T18:10:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.274+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42256", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b0406e2a0 con 0x7f9adc067040 2024-01-15T18:10:13.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9afa000640 1 -- 172.21.15.143:0/348406437 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9b0406e2a0 con 0x7f9adc067040 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9adc067040 msgr2=0x7f9adc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9adc067040 0x7f9adc069500 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9af40af000 tx=0x7f9af40a7000 comp rx=0 tx=0).stop 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 msgr2=0x7f9b0415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 0x7f9b0415d630 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9afc001570 tx=0x7f9afc00cc50 comp rx=0 tx=0).stop 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 shutdown_connections 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9adc067040 0x7f9adc069500 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b0415ec80 0x7f9b0415d630 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b04141660 0x7f9b04137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 --2- 172.21.15.143:0/348406437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b04071d00 0x7f9b0413ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 >> 172.21.15.143:0/348406437 conn(0x7f9b0406c3f0 msgr2=0x7f9b040720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:13.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 shutdown_connections 2024-01-15T18:10:13.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.334+0000 7f9b08e00640 1 -- 172.21.15.143:0/348406437 wait complete. 2024-01-15T18:10:13.358 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone42257 2024-01-15T18:10:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.494+0000 7fd51da00640 1 Processor -- start 2024-01-15T18:10:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.494+0000 7fd51da00640 1 -- start start 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd51da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5180ce560 0x7fd51815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd51da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd518071d00 0x7fd518072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd51da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd518072650 con 0x7fd518141660 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd51da00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51815fbb0 con 0x7fd518071d00 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd51da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51815f310 con 0x7fd5180ce560 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd518071d00 0x7fd518072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd518071d00 0x7fd518072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49188/0 (socket says 172.21.15.143:49188) 2024-01-15T18:10:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 -- 172.21.15.143:0/2547758734 learned_addr learned my addr 172.21.15.143:0/2547758734 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd516000640 1 --2- 172.21.15.143:0/2547758734 >> v2:172.21.15.204:3300/0 conn(0x7fd5180ce560 0x7fd51815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 -- 172.21.15.143:0/2547758734 >> v2:172.21.15.204:3300/0 conn(0x7fd5180ce560 msgr2=0x7fd51815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 --2- 172.21.15.143:0/2547758734 >> v2:172.21.15.204:3300/0 conn(0x7fd5180ce560 0x7fd51815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 -- 172.21.15.143:0/2547758734 >> v1:172.21.15.204:6789/0 conn(0x7fd518141660 legacy=0x7fd518141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd517400640 1 -- 172.21.15.143:0/2547758734 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51815f5f0 con 0x7fd518071d00 2024-01-15T18:10:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.498+0000 7fd516000640 1 --2- 172.21.15.143:0/2547758734 >> v2:172.21.15.204:3300/0 conn(0x7fd5180ce560 0x7fd51815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd517400640 1 --2- 172.21.15.143:0/2547758734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518072110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd51806f940 tx=0x7fd51001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b20a10789f1723 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd515600640 1 -- 172.21.15.143:0/2547758734 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100a5740 con 0x7fd518071d00 2024-01-15T18:10:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd515600640 1 -- 172.21.15.143:0/2547758734 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100a58a0 con 0x7fd518071d00 2024-01-15T18:10:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd515600640 1 -- 172.21.15.143:0/2547758734 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100a5c30 con 0x7fd518071d00 2024-01-15T18:10:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 -- 172.21.15.143:0/2547758734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 msgr2=0x7fd518072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 --2- 172.21.15.143:0/2547758734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518072110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd51806f940 tx=0x7fd51001c990 comp rx=0 tx=0).stop 2024-01-15T18:10:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 -- 172.21.15.143:0/2547758734 shutdown_connections 2024-01-15T18:10:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 --2- 172.21.15.143:0/2547758734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518072110 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 --2- 172.21.15.143:0/2547758734 >> v2:172.21.15.204:3300/0 conn(0x7fd5180ce560 0x7fd51815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 -- 172.21.15.143:0/2547758734 >> 172.21.15.143:0/2547758734 conn(0x7fd51806c3f0 msgr2=0x7fd51806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 -- 172.21.15.143:0/2547758734 shutdown_connections 2024-01-15T18:10:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 -- 172.21.15.143:0/2547758734 wait complete. 2024-01-15T18:10:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 Processor -- start 2024-01-15T18:10:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.502+0000 7fd51da00640 1 -- start start 2024-01-15T18:10:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5180ce560 0x7fd518159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd518141660 0x7fd5180b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51815ada0 con 0x7fd518071d00 2024-01-15T18:10:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180b3cd0 con 0x7fd5180ce560 2024-01-15T18:10:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51815b800 con 0x7fd518141660 2024-01-15T18:10:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49202/0 (socket says 172.21.15.143:49202) 2024-01-15T18:10:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 -- 172.21.15.143:0/3892236781 learned_addr learned my addr 172.21.15.143:0/3892236781 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:13.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516000640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5180ce560 0x7fd518159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd517400640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd518141660 0x7fd5180b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 -- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd518141660 msgr2=0x7fd5180b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd518141660 0x7fd5180b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 -- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5180ce560 msgr2=0x7fd518159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5180ce560 0x7fd518159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 -- 172.21.15.143:0/3892236781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510009b30 con 0x7fd518071d00 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516000640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5180ce560 0x7fd518159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd516a00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518159950 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd50800b3e0 tx=0x7fd50800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508011830 con 0x7fd518071d00 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5080119b0 con 0x7fd518071d00 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180cd150 con 0x7fd518071d00 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508011d60 con 0x7fd518071d00 2024-01-15T18:10:13.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51815bdc0 con 0x7fd518071d00 2024-01-15T18:10:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.506+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd518067500 con 0x7fd518071d00 2024-01-15T18:10:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.510+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7fd50801a400 con 0x7fd518071d00 2024-01-15T18:10:13.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.510+0000 7fd50ea00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4f0067040 0x7fd4f0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.510+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd508085660 con 0x7fd518071d00 2024-01-15T18:10:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.510+0000 7fd516000640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4f0067040 0x7fd4f0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.510+0000 7fd516000640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4f0067040 0x7fd4f0069500 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd500006fd0 tx=0x7fd500008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.514+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd508082c50 con 0x7fd518071d00 2024-01-15T18:10:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.694+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone42257", "target": ["mon-mgr", ""]}) v1 -- 0x7fd518140ac0 con 0x7fd4f0067040 2024-01-15T18:10:13.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.754+0000 7fd50ea00640 1 -- 172.21.15.143:0/3892236781 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd518140ac0 con 0x7fd4f0067040 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4f0067040 msgr2=0x7fd4f0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4f0067040 0x7fd4f0069500 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd500006fd0 tx=0x7fd500008040 comp rx=0 tx=0).stop 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 msgr2=0x7fd518159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518159950 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd50800b3e0 tx=0x7fd50800b8a0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 shutdown_connections 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4f0067040 0x7fd4f0069500 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd518141660 0x7fd5180b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5180ce560 0x7fd518159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 --2- 172.21.15.143:0/3892236781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd518071d00 0x7fd518159950 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 >> 172.21.15.143:0/3892236781 conn(0x7fd51806c3f0 msgr2=0x7fd5180ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 shutdown_connections 2024-01-15T18:10:13.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.758+0000 7fd51da00640 1 -- 172.21.15.143:0/3892236781 wait complete. 2024-01-15T18:10:13.782 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:10:13.782 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42251 2024-01-15T18:10:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf7e00640 1 Processor -- start 2024-01-15T18:10:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf7e00640 1 -- start start 2024-01-15T18:10:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf006b860 0x7fbcf01649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf7e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00b3510 con 0x7fbcf006e260 2024-01-15T18:10:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf7e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00b37c0 con 0x7fbcf015a670 2024-01-15T18:10:13.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00b3aa0 con 0x7fbcf006b860 2024-01-15T18:10:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf4a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbcf006e260 0x7fbcf015aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45376/0 (socket says 172.21.15.143:45376) 2024-01-15T18:10:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf4a00640 1 -- 172.21.15.143:0/3354796018 learned_addr learned my addr 172.21.15.143:0/3354796018 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:13.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf5e00640 1 --2- 172.21.15.143:0/3354796018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf006b860 0x7fbcf01649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751821684 0 0) 0x7fbcf00b3510 con 0x7fbcf006e260 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd0003530 con 0x7fbcf006e260 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839787314 0 0) 0x7fbcf00b37c0 con 0x7fbcf015a670 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcf00b3510 con 0x7fbcf015a670 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf5e00640 1 -- 172.21.15.143:0/3354796018 >> v1:172.21.15.204:6789/0 conn(0x7fbcf015a670 legacy=0x7fbcf00b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf5e00640 1 -- 172.21.15.143:0/3354796018 >> v1:172.21.15.143:6789/0 conn(0x7fbcf006e260 legacy=0x7fbcf015aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3772147314 0 0) 0x7fbcd0003530 con 0x7fbcf006e260 2024-01-15T18:10:13.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.918+0000 7fbcf5e00640 1 -- 172.21.15.143:0/3354796018 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00b3d80 con 0x7fbcf006b860 2024-01-15T18:10:13.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcf5e00640 1 --2- 172.21.15.143:0/3354796018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf006b860 0x7fbcf01649d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009dc0 tx=0x7fbce801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9271fc572915d7e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8001680 con 0x7fbcf006b860 2024-01-15T18:10:13.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce80a55e0 con 0x7fbcf006b860 2024-01-15T18:10:13.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcefe00640 1 -- 172.21.15.143:0/3354796018 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80a5970 con 0x7fbcf006b860 2024-01-15T18:10:13.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcf7e00640 1 -- 172.21.15.143:0/3354796018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf006b860 msgr2=0x7fbcf01649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/3354796018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf006b860 0x7fbcf01649d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009dc0 tx=0x7fbce801c890 comp rx=0 tx=0).stop 2024-01-15T18:10:13.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcf7e00640 1 -- 172.21.15.143:0/3354796018 shutdown_connections 2024-01-15T18:10:13.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/3354796018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf006b860 0x7fbcf01649d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.922+0000 7fbcf7e00640 1 -- 172.21.15.143:0/3354796018 >> 172.21.15.143:0/3354796018 conn(0x7fbcf00b1610 msgr2=0x7fbcf00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:13.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 -- 172.21.15.143:0/3354796018 shutdown_connections 2024-01-15T18:10:13.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 -- 172.21.15.143:0/3354796018 wait complete. 2024-01-15T18:10:13.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 Processor -- start 2024-01-15T18:10:13.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 -- start start 2024-01-15T18:10:13.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 0x7fbcf0155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcf006e260 0x7fbcf0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf015a670 0x7fbcf0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf0152a70 con 0x7fbcf006b860 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf0152d50 con 0x7fbcf006e260 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf7e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf0153030 con 0x7fbcf015a670 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 0x7fbcf0155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 0x7fbcf0155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49206/0 (socket says 172.21.15.143:49206) 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 -- 172.21.15.143:0/1172252396 learned_addr learned my addr 172.21.15.143:0/1172252396 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf015a670 0x7fbcf0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf4a00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcf006e260 0x7fbcf0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 -- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf015a670 msgr2=0x7fbcf0157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf015a670 0x7fbcf0157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 -- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcf006e260 msgr2=0x7fbcf0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcf006e260 0x7fbcf0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 -- 172.21.15.143:0/1172252396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009a70 con 0x7fbcf006b860 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbcf5400640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 0x7fbcf0155780 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c500 tx=0x7fbce000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.926+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00052d0 con 0x7fbcf006b860 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce0005450 con 0x7fbcf006b860 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce0012870 con 0x7fbcf006b860 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf0153310 con 0x7fbcf006b860 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf01509a0 con 0x7fbcf006b860 2024-01-15T18:10:13.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf0074b00 con 0x7fbcf006b860 2024-01-15T18:10:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7fbce00129d0 con 0x7fbcf006b860 2024-01-15T18:10:13.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbced600640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcc8067040 0x7fbcc8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbce0085130 con 0x7fbcf006b860 2024-01-15T18:10:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbcf4a00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcc8067040 0x7fbcc8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:13.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.930+0000 7fbcf4a00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcc8067040 0x7fbcc8069500 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0159fd0 tx=0x7fbcd8001a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:13.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:13.934+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbce0082660 con 0x7fbcf006b860 2024-01-15T18:10:14.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.118+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42251", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcf0067500 con 0x7fbcc8067040 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.126+0000 7fbced600640 1 -- 172.21.15.143:0/1172252396 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fbcf0067500 con 0x7fbcc8067040 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:14.130 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.131 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:14.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcc8067040 msgr2=0x7fbcc8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcc8067040 0x7fbcc8069500 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0159fd0 tx=0x7fbcd8001a60 comp rx=0 tx=0).stop 2024-01-15T18:10:14.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 msgr2=0x7fbcf0155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 0x7fbcf0155780 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c500 tx=0x7fbce000c9c0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 shutdown_connections 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcc8067040 0x7fbcc8069500 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcf015a670 0x7fbcf0157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcf006e260 0x7fbcf0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 --2- 172.21.15.143:0/1172252396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcf006b860 0x7fbcf0155780 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 >> 172.21.15.143:0/1172252396 conn(0x7fbcf00b1610 msgr2=0x7fbcf015aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 shutdown_connections 2024-01-15T18:10:14.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.130+0000 7fbcf7e00640 1 -- 172.21.15.143:0/1172252396 wait complete. 2024-01-15T18:10:14.152 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:10:14.152 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42257 2024-01-15T18:10:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 Processor -- start 2024-01-15T18:10:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 -- start start 2024-01-15T18:10:14.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9b7006bfb0 0x7f9b7006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b700ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b700cbff0 con 0x7f9b700b0310 2024-01-15T18:10:14.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b700d0c50 con 0x7f9b7006bfb0 2024-01-15T18:10:14.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b77600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7006c8d0 con 0x7f9b700d0210 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9b7006bfb0 0x7f9b7006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9b7006bfb0 0x7f9b7006c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49222/0 (socket says 172.21.15.143:49222) 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 -- 172.21.15.143:0/1068118132 learned_addr learned my addr 172.21.15.143:0/1068118132 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 -- 172.21.15.143:0/1068118132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 msgr2=0x7f9b700ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 --2- 172.21.15.143:0/1068118132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b700ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 -- 172.21.15.143:0/1068118132 >> v1:172.21.15.204:6789/0 conn(0x7f9b700b0310 legacy=0x7f9b700cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.290+0000 7f9b74c00640 1 -- 172.21.15.143:0/1068118132 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b700cb6c0 con 0x7f9b7006bfb0 2024-01-15T18:10:14.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b74c00640 1 --2- 172.21.15.143:0/1068118132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b7006bfb0 0x7f9b7006c390 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009870 tx=0x7f9b6001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfc13699ecd9c7f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b6f400640 1 -- 172.21.15.143:0/1068118132 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600ab600 con 0x7f9b7006bfb0 2024-01-15T18:10:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b6f400640 1 -- 172.21.15.143:0/1068118132 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b600ab760 con 0x7f9b7006bfb0 2024-01-15T18:10:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b6f400640 1 -- 172.21.15.143:0/1068118132 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b600abaf0 con 0x7f9b7006bfb0 2024-01-15T18:10:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 -- 172.21.15.143:0/1068118132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b7006bfb0 msgr2=0x7f9b7006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 --2- 172.21.15.143:0/1068118132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b7006bfb0 0x7f9b7006c390 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9b60009870 tx=0x7f9b6001c440 comp rx=0 tx=0).stop 2024-01-15T18:10:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 -- 172.21.15.143:0/1068118132 shutdown_connections 2024-01-15T18:10:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 --2- 172.21.15.143:0/1068118132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b700ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 --2- 172.21.15.143:0/1068118132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b7006bfb0 0x7f9b7006c390 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 -- 172.21.15.143:0/1068118132 >> 172.21.15.143:0/1068118132 conn(0x7f9b700d0ed0 msgr2=0x7f9b700d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 -- 172.21.15.143:0/1068118132 shutdown_connections 2024-01-15T18:10:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.294+0000 7f9b77600640 1 -- 172.21.15.143:0/1068118132 wait complete. 2024-01-15T18:10:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 Processor -- start 2024-01-15T18:10:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 -- start start 2024-01-15T18:10:14.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b7006bfb0 0x7f9b700c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 0x7f9b700cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b7014b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c9950 con 0x7f9b700b0310 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c9c30 con 0x7f9b7006bfb0 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b700c9f10 con 0x7f9b700d0210 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 0x7f9b700cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 0x7f9b700cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49234/0 (socket says 172.21.15.143:49234) 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 -- 172.21.15.143:0/964967152 learned_addr learned my addr 172.21.15.143:0/964967152 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b75600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b7014b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b74c00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b7006bfb0 0x7f9b700c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 -- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 msgr2=0x7f9b7014b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b7014b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 -- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b7006bfb0 msgr2=0x7f9b700c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b7006bfb0 0x7f9b700c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 -- 172.21.15.143:0/964967152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600094d0 con 0x7f9b700b0310 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6fe00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 0x7f9b700cab30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b3e0 tx=0x7f9b5800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58011830 con 0x7f9b700b0310 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58011990 con 0x7f9b700b0310 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58011d60 con 0x7f9b700b0310 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b700ca1f0 con 0x7f9b700b0310 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.298+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7014c2d0 con 0x7f9b700b0310 2024-01-15T18:10:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.302+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b70067500 con 0x7f9b700b0310 2024-01-15T18:10:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.302+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7f9b5801a400 con 0x7f9b700b0310 2024-01-15T18:10:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.302+0000 7f9b6cc00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b48067040 0x7f9b48069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.302+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9b58082de0 con 0x7f9b700b0310 2024-01-15T18:10:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.302+0000 7f9b74c00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b48067040 0x7f9b48069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.302+0000 7f9b74c00640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b48067040 0x7f9b48069500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9b600af000 tx=0x7f9b600b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:14.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.306+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b58082af0 con 0x7f9b700b0310 2024-01-15T18:10:14.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.486+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42257", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b700ce690 con 0x7f9b48067040 2024-01-15T18:10:14.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.490+0000 7f9b6cc00640 1 -- 172.21.15.143:0/964967152 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f9b600ab8d0 con 0x7f9b48067040 2024-01-15T18:10:14.493 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:14.493 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.494 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.490+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b48067040 msgr2=0x7f9b48069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.490+0000 7f9b77600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b48067040 0x7f9b48069500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9b600af000 tx=0x7f9b600b6040 comp rx=0 tx=0).stop 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.490+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 msgr2=0x7f9b700cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.490+0000 7f9b77600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 0x7f9b700cab30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800b3e0 tx=0x7f9b5800b8a0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 shutdown_connections 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b48067040 0x7f9b48069500 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b700d0210 0x7f9b7014b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b700b0310 0x7f9b700cab30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 --2- 172.21.15.143:0/964967152 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b7006bfb0 0x7f9b700c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 >> 172.21.15.143:0/964967152 conn(0x7f9b700d0ed0 msgr2=0x7f9b70155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:14.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 shutdown_connections 2024-01-15T18:10:14.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.494+0000 7f9b77600640 1 -- 172.21.15.143:0/964967152 wait complete. 2024-01-15T18:10:14.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:14.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 Processor -- start 2024-01-15T18:10:14.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 -- start start 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5660143780 0x7f56600cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f56600ce520 0x7f5660071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56600ceb50 con 0x7f56600720f0 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566015c110 con 0x7f5660143780 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.650+0000 7f5667c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5660072530 con 0x7f56600ce520 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5660143780 0x7f56600cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5660143780 0x7f56600cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49246/0 (socket says 172.21.15.143:49246) 2024-01-15T18:10:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 -- 172.21.15.143:0/2409793140 learned_addr learned my addr 172.21.15.143:0/2409793140 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f565fe00640 1 --2- 172.21.15.143:0/2409793140 >> v2:172.21.15.204:3300/0 conn(0x7f56600ce520 0x7f5660071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 -- 172.21.15.143:0/2409793140 >> v2:172.21.15.204:3300/0 conn(0x7f56600ce520 msgr2=0x7f5660071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 --2- 172.21.15.143:0/2409793140 >> v2:172.21.15.204:3300/0 conn(0x7f56600ce520 0x7f5660071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 -- 172.21.15.143:0/2409793140 >> v1:172.21.15.204:6789/0 conn(0x7f56600720f0 legacy=0x7f566015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 -- 172.21.15.143:0/2409793140 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566015b7e0 con 0x7f5660143780 2024-01-15T18:10:14.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f5665200640 1 --2- 172.21.15.143:0/2409793140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5660143780 0x7f56600cdfe0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f566006beb0 tx=0x7f565001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f2c3ed7a70f07c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:14.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f565f400640 1 -- 172.21.15.143:0/2409793140 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500ac630 con 0x7f5660143780 2024-01-15T18:10:14.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f565f400640 1 -- 172.21.15.143:0/2409793140 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56500ac790 con 0x7f5660143780 2024-01-15T18:10:14.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.654+0000 7f565f400640 1 -- 172.21.15.143:0/2409793140 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56500acb20 con 0x7f5660143780 2024-01-15T18:10:14.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- 172.21.15.143:0/2409793140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5660143780 msgr2=0x7f56600cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 --2- 172.21.15.143:0/2409793140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5660143780 0x7f56600cdfe0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f566006beb0 tx=0x7f565001c4c0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- 172.21.15.143:0/2409793140 shutdown_connections 2024-01-15T18:10:14.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 --2- 172.21.15.143:0/2409793140 >> v2:172.21.15.204:3300/0 conn(0x7f56600ce520 0x7f5660071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 --2- 172.21.15.143:0/2409793140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5660143780 0x7f56600cdfe0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- 172.21.15.143:0/2409793140 >> 172.21.15.143:0/2409793140 conn(0x7f566006c3f0 msgr2=0x7f566006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:14.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- 172.21.15.143:0/2409793140 shutdown_connections 2024-01-15T18:10:14.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- 172.21.15.143:0/2409793140 wait complete. 2024-01-15T18:10:14.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 Processor -- start 2024-01-15T18:10:14.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- start start 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56600720f0 0x7f5660165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 0x7f5660139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5660143780 0x7f566014a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660139d80 con 0x7f56600ce520 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5660166440 con 0x7f56600720f0 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.658+0000 7f5667c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566014aa00 con 0x7f5660143780 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 0x7f5660139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 0x7f5660139840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49258/0 (socket says 172.21.15.143:49258) 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 -- 172.21.15.143:0/2020971489 learned_addr learned my addr 172.21.15.143:0/2020971489 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f5665c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5660143780 0x7f566014a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f5665200640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56600720f0 0x7f5660165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 -- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5660143780 msgr2=0x7f566014a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5660143780 0x7f566014a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 -- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56600720f0 msgr2=0x7f5660165130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56600720f0 0x7f5660165130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 -- 172.21.15.143:0/2020971489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500094d0 con 0x7f56600ce520 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f5665200640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56600720f0 0x7f5660165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565fe00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 0x7f5660139840 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f564800d800 tx=0x7f564800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800b6c0 con 0x7f56600ce520 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564800b820 con 0x7f56600ce520 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564800bb90 con 0x7f56600ce520 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5660165830 con 0x7f56600ce520 2024-01-15T18:10:14.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5660165df0 con 0x7f56600ce520 2024-01-15T18:10:14.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56601665f0 con 0x7f56600ce520 2024-01-15T18:10:14.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7f564800bcf0 con 0x7f56600ce520 2024-01-15T18:10:14.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.662+0000 7f565cc00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5638067040 0x7f5638069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:14.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.666+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5648085b30 con 0x7f56600ce520 2024-01-15T18:10:14.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.666+0000 7f5665200640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5638067040 0x7f5638069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:14.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.666+0000 7f5665200640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5638067040 0x7f5638069500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56500ae000 tx=0x7f56500a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:14.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.666+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56601665f0 con 0x7f56600ce520 2024-01-15T18:10:14.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f56601665f0 con 0x7f56600ce520 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f565cc00640 1 -- 172.21.15.143:0/2020971489 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2789 (secure 0 0 0) 0x7f56601665f0 con 0x7f56600ce520 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.885 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 341, 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:14.886 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:14.887 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 800, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3222199986, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 6739763200, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 766354350080, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 511, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 10151, 2024-01-15T18:10:14.888 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 0, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 3 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:14.889 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:14.890 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 8, 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:14.709192+0000", 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:10:14.891 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "5 ongoing+pending cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "2 ongoing cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:10:14.892 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:10:14.893 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:14.893 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.893 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:14.893 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5638067040 msgr2=0x7f5638069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5638067040 0x7f5638069500 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f56500ae000 tx=0x7f56500a7000 comp rx=0 tx=0).stop 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 msgr2=0x7f5660139840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 0x7f5660139840 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f564800d800 tx=0x7f564800dcc0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 shutdown_connections 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5638067040 0x7f5638069500 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5660143780 0x7f566014a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56600ce520 0x7f5660139840 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 --2- 172.21.15.143:0/2020971489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56600720f0 0x7f5660165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 >> 172.21.15.143:0/2020971489 conn(0x7f566006c3f0 msgr2=0x7f566006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 shutdown_connections 2024-01-15T18:10:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:14.882+0000 7f5667c00640 1 -- 172.21.15.143:0/2020971489 wait complete. 2024-01-15T18:10:15.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:16.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 Processor -- start 2024-01-15T18:10:16.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 -- start start 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec406e460 0x7efec4159380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efec40ce950 0x7efec4072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec4071cf0 con 0x7efec4070a00 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec4070e90 con 0x7efec40ce950 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.322+0000 7efecba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec40ce520 con 0x7efec406e460 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efec4070a00 0x7efec40cb9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45392/0 (socket says 172.21.15.143:45392) 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9a00640 1 -- 172.21.15.143:0/1237552466 learned_addr learned my addr 172.21.15.143:0/1237552466 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9000640 1 --2- 172.21.15.143:0/1237552466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec406e460 0x7efec4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9000640 1 -- 172.21.15.143:0/1237552466 >> v2:172.21.15.204:3300/0 conn(0x7efec40ce950 msgr2=0x7efec4072180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9000640 1 --2- 172.21.15.143:0/1237552466 >> v2:172.21.15.204:3300/0 conn(0x7efec40ce950 0x7efec4072180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9000640 1 -- 172.21.15.143:0/1237552466 >> v1:172.21.15.143:6789/0 conn(0x7efec4070a00 legacy=0x7efec40cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.326+0000 7efec9000640 1 -- 172.21.15.143:0/1237552466 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40710e0 con 0x7efec406e460 2024-01-15T18:10:16.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.338+0000 7efec9000640 1 --2- 172.21.15.143:0/1237552466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec406e460 0x7efec4159380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efebc009dc0 tx=0x7efebc01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b3edf096c62a31 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:16.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.338+0000 7efec3400640 1 -- 172.21.15.143:0/1237552466 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0af600 con 0x7efec406e460 2024-01-15T18:10:16.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.338+0000 7efec3400640 1 -- 172.21.15.143:0/1237552466 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc0ab050 con 0x7efec406e460 2024-01-15T18:10:16.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.338+0000 7efec3400640 1 -- 172.21.15.143:0/1237552466 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc0af980 con 0x7efec406e460 2024-01-15T18:10:16.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.342+0000 7efecba00640 1 -- 172.21.15.143:0/1237552466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec406e460 msgr2=0x7efec4159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.342+0000 7efecba00640 1 --2- 172.21.15.143:0/1237552466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec406e460 0x7efec4159380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efebc009dc0 tx=0x7efebc01f150 comp rx=0 tx=0).stop 2024-01-15T18:10:16.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.342+0000 7efecba00640 1 -- 172.21.15.143:0/1237552466 shutdown_connections 2024-01-15T18:10:16.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.342+0000 7efecba00640 1 --2- 172.21.15.143:0/1237552466 >> v2:172.21.15.204:3300/0 conn(0x7efec40ce950 0x7efec4072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.342+0000 7efecba00640 1 --2- 172.21.15.143:0/1237552466 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec406e460 0x7efec4159380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.342+0000 7efecba00640 1 -- 172.21.15.143:0/1237552466 >> 172.21.15.143:0/1237552466 conn(0x7efec406a890 msgr2=0x7efec406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:16.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.346+0000 7efecba00640 1 -- 172.21.15.143:0/1237552466 shutdown_connections 2024-01-15T18:10:16.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.346+0000 7efecba00640 1 -- 172.21.15.143:0/1237552466 wait complete. 2024-01-15T18:10:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.350+0000 7efecba00640 1 Processor -- start 2024-01-15T18:10:16.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.350+0000 7efecba00640 1 -- start start 2024-01-15T18:10:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.354+0000 7efecba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 0x7efec41537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.354+0000 7efecba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 0x7efec41e2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.354+0000 7efecba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 0x7efec415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.354+0000 7efec9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 0x7efec41537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.354+0000 7efec9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 0x7efec41537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49274/0 (socket says 172.21.15.143:49274) 2024-01-15T18:10:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 0x7efec41e2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 0x7efec41e2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46908/0 (socket says 172.21.15.143:46908) 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efecba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41f2bb0 con 0x7efec406e460 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efecba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41f1fc0 con 0x7efec40ce950 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efecba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec41f2210 con 0x7efec4070a00 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 0x7efec415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 0x7efec415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:42594/0 (socket says 172.21.15.143:42594) 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 -- 172.21.15.143:0/3207329881 learned_addr learned my addr 172.21.15.143:0/3207329881 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 -- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 msgr2=0x7efec41e2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 0x7efec41e2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 -- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 msgr2=0x7efec41537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 0x7efec41537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec9a00640 1 -- 172.21.15.143:0/3207329881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc009a70 con 0x7efec40ce950 2024-01-15T18:10:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.358+0000 7efec3e00640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 0x7efec41e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-01-15T18:10:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.362+0000 7efec9a00640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 0x7efec415bc10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c430 tx=0x7efeac00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:16.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.362+0000 7efec9000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 0x7efec41537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-01-15T18:10:16.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.362+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac005cf0 con 0x7efec40ce950 2024-01-15T18:10:16.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.362+0000 7efecba00640 1 -- 172.21.15.143:0/3207329881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec41f24f0 con 0x7efec40ce950 2024-01-15T18:10:16.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.362+0000 7efecba00640 1 -- 172.21.15.143:0/3207329881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec4153cf0 con 0x7efec40ce950 2024-01-15T18:10:16.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.366+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe94005260 con 0x7efec40ce950 2024-01-15T18:10:16.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.370+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeac005e50 con 0x7efec40ce950 2024-01-15T18:10:16.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.370+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac005cf0 con 0x7efec40ce950 2024-01-15T18:10:16.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.374+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7efeac007070 con 0x7efec40ce950 2024-01-15T18:10:16.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.378+0000 7efec0c00640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe9c0670a0 0x7efe9c069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.378+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efeac084cb0 con 0x7efec40ce950 2024-01-15T18:10:16.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.378+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efeac0821d0 con 0x7efec40ce950 2024-01-15T18:10:16.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.386+0000 7efec9000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe9c0670a0 0x7efe9c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.390+0000 7efec9000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe9c0670a0 0x7efe9c069560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efebc0b1000 tx=0x7efebc0aa000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:16.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.890+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efe940057e0 con 0x7efec40ce950 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.894+0000 7efec0c00640 1 -- 172.21.15.143:0/3207329881 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2799 (secure 0 0 0) 0x7efeac017020 con 0x7efec40ce950 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:16.908 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 343, 2024-01-15T18:10:16.909 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:16.910 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 916, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3691963651, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 7402483712, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 765691629568, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 47492342, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 39121441, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 12, 2024-01-15T18:10:16.911 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 13 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:16.912 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:16.913 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 8, 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:14.709192+0000", 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:10:16.914 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "5 ongoing+pending cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:10:16.915 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "2 ongoing cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:16.916 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:16.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.910+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe9c0670a0 msgr2=0x7efe9c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.910+0000 7efeba000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe9c0670a0 0x7efe9c069560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efebc0b1000 tx=0x7efebc0aa000 comp rx=0 tx=0).stop 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.910+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 msgr2=0x7efec415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.910+0000 7efeba000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 0x7efec415bc10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c430 tx=0x7efeac00c8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.910+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 shutdown_connections 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efe9c0670a0 0x7efe9c069560 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efec40ce950 0x7efec415bc10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efec4070a00 0x7efec41e2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 --2- 172.21.15.143:0/3207329881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efec406e460 0x7efec41537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 >> 172.21.15.143:0/3207329881 conn(0x7efec406a890 msgr2=0x7efec4154b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 shutdown_connections 2024-01-15T18:10:16.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:16.914+0000 7efeba000640 1 -- 172.21.15.143:0/3207329881 wait complete. 2024-01-15T18:10:17.917 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:17.910+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:17.917 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:17.910+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:17.917 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:17.910+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:17.917 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:17.910+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:17.917 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:17.910+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:19.013 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 Processor -- start 2024-01-15T18:10:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 -- start start 2024-01-15T18:10:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee540d0210 0x7fee540ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee540cbff0 con 0x7fee540b0310 2024-01-15T18:10:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee540d0c50 con 0x7fee540d0210 2024-01-15T18:10:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5406c8d0 con 0x7fee5406bfb0 2024-01-15T18:10:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee540d0210 0x7fee540ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee540d0210 0x7fee540ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53040/0 (socket says 172.21.15.143:53040) 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 -- 172.21.15.143:0/4095888133 learned_addr learned my addr 172.21.15.143:0/4095888133 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee59a00640 1 --2- 172.21.15.143:0/4095888133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 -- 172.21.15.143:0/4095888133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 msgr2=0x7fee5406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 --2- 172.21.15.143:0/4095888133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 -- 172.21.15.143:0/4095888133 >> v1:172.21.15.204:6789/0 conn(0x7fee540b0310 legacy=0x7fee540cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.158+0000 7fee5a400640 1 -- 172.21.15.143:0/4095888133 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee540cb6c0 con 0x7fee540d0210 2024-01-15T18:10:19.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5a400640 1 --2- 172.21.15.143:0/4095888133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee540ccb90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009e50 tx=0x7fee4c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ee811e6f07689b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee53e00640 1 -- 172.21.15.143:0/4095888133 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0a5810 con 0x7fee540d0210 2024-01-15T18:10:19.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee53e00640 1 -- 172.21.15.143:0/4095888133 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c0a5970 con 0x7fee540d0210 2024-01-15T18:10:19.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee53e00640 1 -- 172.21.15.143:0/4095888133 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0a5d00 con 0x7fee540d0210 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 -- 172.21.15.143:0/4095888133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 msgr2=0x7fee540ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 --2- 172.21.15.143:0/4095888133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee540ccb90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009e50 tx=0x7fee4c01c950 comp rx=0 tx=0).stop 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 -- 172.21.15.143:0/4095888133 shutdown_connections 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 --2- 172.21.15.143:0/4095888133 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee540ccb90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 --2- 172.21.15.143:0/4095888133 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 -- 172.21.15.143:0/4095888133 >> 172.21.15.143:0/4095888133 conn(0x7fee540d0ed0 msgr2=0x7fee540d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:19.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 -- 172.21.15.143:0/4095888133 shutdown_connections 2024-01-15T18:10:19.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 -- 172.21.15.143:0/4095888133 wait complete. 2024-01-15T18:10:19.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.162+0000 7fee5c400640 1 Processor -- start 2024-01-15T18:10:19.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- start start 2024-01-15T18:10:19.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee540b0310 0x7fee5415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee54154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5415c2c0 con 0x7fee540d0210 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5414a930 con 0x7fee540b0310 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5415b6b0 con 0x7fee5406bfb0 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee54154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee54154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53056/0 (socket says 172.21.15.143:53056) 2024-01-15T18:10:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 -- 172.21.15.143:0/2311118197 learned_addr learned my addr 172.21.15.143:0/2311118197 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee59000640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee540b0310 0x7fee5415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee59a00640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 -- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 msgr2=0x7fee5414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 -- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee540b0310 msgr2=0x7fee5415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee540b0310 0x7fee5415abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 -- 172.21.15.143:0/2311118197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c009b30 con 0x7fee540d0210 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee59a00640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5a400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee54154720 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0af040 tx=0x7fee4c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0b8640 con 0x7fee540d0210 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c0b87c0 con 0x7fee540d0210 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0b8b70 con 0x7fee540d0210 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5415b990 con 0x7fee540d0210 2024-01-15T18:10:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5415bf50 con 0x7fee540d0210 2024-01-15T18:10:19.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.166+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee54067500 con 0x7fee540d0210 2024-01-15T18:10:19.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.170+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) v1 ==== 86823+0+0 (secure 0 0 0) 0x7fee4c0b8cd0 con 0x7fee540d0210 2024-01-15T18:10:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.170+0000 7fee51600640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee2c067070 0x7fee2c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.170+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fee4c11fd30 con 0x7fee540d0210 2024-01-15T18:10:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.170+0000 7fee59a00640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee2c067070 0x7fee2c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.170+0000 7fee59a00640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee2c067070 0x7fee2c069530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee44009b10 tx=0x7fee44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.174+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee4c11d5e0 con 0x7fee540d0210 2024-01-15T18:10:19.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fee540ce660 con 0x7fee540d0210 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee51600640 1 -- 172.21.15.143:0/2311118197 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2799 (secure 0 0 0) 0x7fee4c0eff60 con 0x7fee540d0210 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:19.381 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 345, 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:19.382 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:19.383 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 916, 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3691963651, 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 7402483712, 2024-01-15T18:10:19.384 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 765691629568, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 47492342, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 39121441, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 12, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 13 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:19.385 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:19.386 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 8, 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:14.709192+0000", 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:10:19.387 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.388 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "5 ongoing+pending cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "2 ongoing cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.389 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:19.390 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee2c067070 msgr2=0x7fee2c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee2c067070 0x7fee2c069530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee44009b10 tx=0x7fee44008040 comp rx=0 tx=0).stop 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 msgr2=0x7fee54154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee54154720 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0af040 tx=0x7fee4c0196f0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 shutdown_connections 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee2c067070 0x7fee2c069530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee540d0210 0x7fee54154720 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee540b0310 0x7fee5415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 --2- 172.21.15.143:0/2311118197 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee5406bfb0 0x7fee5414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 >> 172.21.15.143:0/2311118197 conn(0x7fee540d0ed0 msgr2=0x7fee54155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 shutdown_connections 2024-01-15T18:10:19.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:19.378+0000 7fee5c400640 1 -- 172.21.15.143:0/2311118197 wait complete. 2024-01-15T18:10:21.403 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 Processor -- start 2024-01-15T18:10:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 -- start start 2024-01-15T18:10:21.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b6013df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5b6013e850 0x7f5b60140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b600ce900 con 0x7f5b600cdb90 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b60070410 con 0x7f5b6013e850 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.750+0000 7f5b68400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b600706f0 con 0x7f5b601407e0 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b6013df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b6013df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:54770/0 (socket says 172.21.15.143:54770) 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 -- 172.21.15.143:0/2481530352 learned_addr learned my addr 172.21.15.143:0/2481530352 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 -- 172.21.15.143:0/2481530352 >> v2:172.21.15.204:3300/0 conn(0x7f5b6013e850 msgr2=0x7f5b60140bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 --2- 172.21.15.143:0/2481530352 >> v2:172.21.15.204:3300/0 conn(0x7f5b6013e850 0x7f5b60140bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 -- 172.21.15.143:0/2481530352 >> v1:172.21.15.143:6789/0 conn(0x7f5b600cdb90 legacy=0x7f5b600ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:21.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.754+0000 7f5b65a00640 1 -- 172.21.15.143:0/2481530352 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b600709d0 con 0x7f5b601407e0 2024-01-15T18:10:21.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.758+0000 7f5b65a00640 1 --2- 172.21.15.143:0/2481530352 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b6013df90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b58014a00 tx=0x7f5b58025510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=576bbe42e782a8a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:21.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.758+0000 7f5b5fe00640 1 -- 172.21.15.143:0/2481530352 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580110d0 con 0x7f5b601407e0 2024-01-15T18:10:21.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.758+0000 7f5b5fe00640 1 -- 172.21.15.143:0/2481530352 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b58011230 con 0x7f5b601407e0 2024-01-15T18:10:21.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.758+0000 7f5b5fe00640 1 -- 172.21.15.143:0/2481530352 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580115c0 con 0x7f5b601407e0 2024-01-15T18:10:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 -- 172.21.15.143:0/2481530352 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 msgr2=0x7f5b6013df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 --2- 172.21.15.143:0/2481530352 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b6013df90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b58014a00 tx=0x7f5b58025510 comp rx=0 tx=0).stop 2024-01-15T18:10:21.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 -- 172.21.15.143:0/2481530352 shutdown_connections 2024-01-15T18:10:21.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 --2- 172.21.15.143:0/2481530352 >> v2:172.21.15.204:3300/0 conn(0x7f5b6013e850 0x7f5b60140bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 --2- 172.21.15.143:0/2481530352 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b6013df90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 -- 172.21.15.143:0/2481530352 >> 172.21.15.143:0/2481530352 conn(0x7f5b600b0bf0 msgr2=0x7f5b600b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:21.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 -- 172.21.15.143:0/2481530352 shutdown_connections 2024-01-15T18:10:21.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.762+0000 7f5b68400640 1 -- 172.21.15.143:0/2481530352 wait complete. 2024-01-15T18:10:21.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.766+0000 7f5b68400640 1 Processor -- start 2024-01-15T18:10:21.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.766+0000 7f5b68400640 1 -- start start 2024-01-15T18:10:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b68400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b600cdb90 0x7f5b6015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b68400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 0x7f5b601f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b68400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b601578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b68400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b601565f0 con 0x7f5b600cdb90 2024-01-15T18:10:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b68400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60156840 con 0x7f5b6013e850 2024-01-15T18:10:21.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b68400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b60156b20 con 0x7f5b601407e0 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 0x7f5b601f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 0x7f5b601f12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:36220/0 (socket says 172.21.15.143:36220) 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 -- 172.21.15.143:0/1917484406 learned_addr learned my addr 172.21.15.143:0/1917484406 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b66400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b601578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 -- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 msgr2=0x7f5b601578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b601578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 -- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b600cdb90 msgr2=0x7f5b6015e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b600cdb90 0x7f5b6015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 -- 172.21.15.143:0/1917484406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b58014660 con 0x7f5b6013e850 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b66400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b601578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b65000640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 0x7f5b601f12d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b4a0 tx=0x7f5b4800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48010610 con 0x7f5b6013e850 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b48010790 con 0x7f5b6013e850 2024-01-15T18:10:21.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.770+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48010b40 con 0x7f5b6013e850 2024-01-15T18:10:21.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.774+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b60156e00 con 0x7f5b6013e850 2024-01-15T18:10:21.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.774+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b601573c0 con 0x7f5b6013e850 2024-01-15T18:10:21.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.774+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b601575e0 con 0x7f5b6013e850 2024-01-15T18:10:21.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.778+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 18) v1 ==== 86874+0+0 (secure 0 0 0) 0x7f5b48010ca0 con 0x7f5b6013e850 2024-01-15T18:10:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.778+0000 7f5b5d600640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b380671a0 0x7f5b38069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.782+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5b480827c0 con 0x7f5b6013e850 2024-01-15T18:10:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.790+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5b48051600 con 0x7f5b6013e850 2024-01-15T18:10:21.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.818+0000 7f5b65a00640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b380671a0 0x7f5b38069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:21.838+0000 7f5b65a00640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b380671a0 0x7f5b38069660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5b580b6000 tx=0x7f5b58004ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.310+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5b60067680 con 0x7f5b6013e850 2024-01-15T18:10:22.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.314+0000 7f5b5d600640 1 -- 172.21.15.143:0/1917484406 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2873 (secure 0 0 0) 0x7f5b4801e090 con 0x7f5b6013e850 2024-01-15T18:10:22.323 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:22.324 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 348, 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:22.325 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:22.326 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1059, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 4291950640, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 8145276928, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 764948836352, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 99476253, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 89029188, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 24, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 28 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:22.327 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:22.328 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 8, 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:14.709192+0000", 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:10:22.329 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (7s)\n [==..........................] (remaining: 70s)", 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:10:22.330 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "3 ongoing cloning jobs; Avg progress = 22.22% (7s)\n [======......................] (remaining: 25s)", 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.22220000624656677, 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:22.331 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:22.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.330+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b380671a0 msgr2=0x7f5b38069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.330+0000 7f5b68400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b380671a0 0x7f5b38069660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f5b580b6000 tx=0x7f5b58004ac0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.330+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 msgr2=0x7f5b601f12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.330+0000 7f5b68400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 0x7f5b601f12d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800b4a0 tx=0x7f5b4800b960 comp rx=0 tx=0).stop 2024-01-15T18:10:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.334+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 shutdown_connections 2024-01-15T18:10:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.334+0000 7f5b68400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b380671a0 0x7f5b38069660 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.334+0000 7f5b68400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b601407e0 0x7f5b601578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.334+0000 7f5b68400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b6013e850 0x7f5b601f12d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.334+0000 7f5b68400640 1 --2- 172.21.15.143:0/1917484406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b600cdb90 0x7f5b6015e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.334+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 >> 172.21.15.143:0/1917484406 conn(0x7f5b600b0bf0 msgr2=0x7f5b6006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:22.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.338+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 shutdown_connections 2024-01-15T18:10:22.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:22.338+0000 7f5b68400640 1 -- 172.21.15.143:0/1917484406 wait complete. 2024-01-15T18:10:23.029 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:23.026+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:23.029 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:23.026+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:23.029 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:23.026+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:23.029 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:23.026+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:23.029 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:10:23.026+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:24.420 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.734+0000 7f2b45600640 1 Processor -- start 2024-01-15T18:10:24.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.734+0000 7f2b45600640 1 -- start start 2024-01-15T18:10:24.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.734+0000 7f2b45600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 0x7f2b4006d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.738+0000 7f2b3ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 0x7f2b4006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.738+0000 7f2b3ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 0x7f2b4006d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:54790/0 (socket says 172.21.15.143:54790) 2024-01-15T18:10:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.738+0000 7f2b3ea00640 1 -- 172.21.15.143:0/134658410 learned_addr learned my addr 172.21.15.143:0/134658410 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b45600640 1 --2- 172.21.15.143:0/134658410 >> v2:172.21.15.204:3300/0 conn(0x7f2b40165ec0 0x7f2b400ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b400cda20 con 0x7f2b400caf20 2024-01-15T18:10:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b400cdd00 con 0x7f2b40165ec0 2024-01-15T18:10:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400cdfe0 con 0x7f2b401614e0 2024-01-15T18:10:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3ea00640 1 -- 172.21.15.143:0/134658410 >> v2:172.21.15.204:3300/0 conn(0x7f2b40165ec0 msgr2=0x7f2b400ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:24.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3d600640 1 -- 172.21.15.143:0/134658410 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822959434 0 0) 0x7f2b400cda20 con 0x7f2b400caf20 2024-01-15T18:10:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3e000640 1 --2- 172.21.15.143:0/134658410 >> v2:172.21.15.204:3300/0 conn(0x7f2b40165ec0 0x7f2b400ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3ea00640 1 --2- 172.21.15.143:0/134658410 >> v2:172.21.15.204:3300/0 conn(0x7f2b40165ec0 0x7f2b400ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3ea00640 1 -- 172.21.15.143:0/134658410 >> v1:172.21.15.204:6789/0 conn(0x7f2b400caf20 legacy=0x7f2b4006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3ea00640 1 -- 172.21.15.143:0/134658410 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400ce2c0 con 0x7f2b401614e0 2024-01-15T18:10:24.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.742+0000 7f2b3e000640 1 --2- 172.21.15.143:0/134658410 >> v2:172.21.15.204:3300/0 conn(0x7f2b40165ec0 0x7f2b400ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.746+0000 7f2b3ea00640 1 --2- 172.21.15.143:0/134658410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 0x7f2b4006d140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009e50 tx=0x7f2b3001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd82f910cd8547f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.746+0000 7f2b3d600640 1 -- 172.21.15.143:0/134658410 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a5830 con 0x7f2b401614e0 2024-01-15T18:10:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.746+0000 7f2b3d600640 1 -- 172.21.15.143:0/134658410 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b300a5990 con 0x7f2b401614e0 2024-01-15T18:10:24.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.746+0000 7f2b3d600640 1 -- 172.21.15.143:0/134658410 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a5d20 con 0x7f2b401614e0 2024-01-15T18:10:24.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 msgr2=0x7f2b4006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 --2- 172.21.15.143:0/134658410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 0x7f2b4006d140 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009e50 tx=0x7f2b3001c970 comp rx=0 tx=0).stop 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 shutdown_connections 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 --2- 172.21.15.143:0/134658410 >> v2:172.21.15.204:3300/0 conn(0x7f2b40165ec0 0x7f2b400ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 --2- 172.21.15.143:0/134658410 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b401614e0 0x7f2b4006d140 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 >> 172.21.15.143:0/134658410 conn(0x7f2b400cfd20 msgr2=0x7f2b400d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 shutdown_connections 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.750+0000 7f2b45600640 1 -- 172.21.15.143:0/134658410 wait complete. 2024-01-15T18:10:24.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 Processor -- start 2024-01-15T18:10:24.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 -- start start 2024-01-15T18:10:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b400caf20 0x7f2b400b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b401614e0 0x7f2b40162030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 0x7f2b401fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40162570 con 0x7f2b401614e0 2024-01-15T18:10:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40162810 con 0x7f2b400caf20 2024-01-15T18:10:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.754+0000 7f2b45600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b40162af0 con 0x7f2b40165ec0 2024-01-15T18:10:24.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 0x7f2b401fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 0x7f2b401fc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:54796/0 (socket says 172.21.15.143:54796) 2024-01-15T18:10:24.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 -- 172.21.15.143:0/3165118142 learned_addr learned my addr 172.21.15.143:0/3165118142 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:24.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3ea00640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b400caf20 0x7f2b400b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 -- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b400caf20 msgr2=0x7f2b400b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b400caf20 0x7f2b400b4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 -- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b401614e0 msgr2=0x7f2b40162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b401614e0 0x7f2b40162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 -- 172.21.15.143:0/3165118142 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30009b30 con 0x7f2b40165ec0 2024-01-15T18:10:24.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b3f400640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 0x7f2b401fc8e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c450 tx=0x7f2b3800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:24.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38005480 con 0x7f2b40165ec0 2024-01-15T18:10:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b38003600 con 0x7f2b40165ec0 2024-01-15T18:10:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b40162dd0 con 0x7f2b40165ec0 2024-01-15T18:10:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380039b0 con 0x7f2b40165ec0 2024-01-15T18:10:24.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.758+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b40163390 con 0x7f2b40165ec0 2024-01-15T18:10:24.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.762+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b10005260 con 0x7f2b40165ec0 2024-01-15T18:10:24.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.766+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 18) v1 ==== 86874+0+0 (secure 0 0 0) 0x7f2b38003b10 con 0x7f2b40165ec0 2024-01-15T18:10:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.766+0000 7f2b36a00640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2b18067100 0x7f2b180695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.770+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2b38081fb0 con 0x7f2b40165ec0 2024-01-15T18:10:24.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.770+0000 7f2b3ea00640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2b18067100 0x7f2b180695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.770+0000 7f2b3ea00640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2b18067100 0x7f2b180695c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b300ae000 tx=0x7f2b300a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:24.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.770+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2b38050f60 con 0x7f2b40165ec0 2024-01-15T18:10:24.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:24.826+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f2b38051330 con 0x7f2b40165ec0 2024-01-15T18:10:25.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2b100057e0 con 0x7f2b40165ec0 2024-01-15T18:10:25.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b36a00640 1 -- 172.21.15.143:0/3165118142 <== mon.2 v2:172.21.15.204:3301/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f2b38075760 con 0x7f2b40165ec0 2024-01-15T18:10:25.184 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:25.185 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 351, 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:25.186 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.187 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1540, 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 6313705831, 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 12250972160, 2024-01-15T18:10:25.188 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 760843141120, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 262467378, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 257259961, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 63, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 70 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:25.189 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:25.190 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:25.191 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 19.049% (10s)\n [=====.......................] (remaining: 46s)", 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.19048571586608887, 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 33.335% (10s)\n [=========...................] (remaining: 21s)", 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.33335000276565552, 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:25.192 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:25.193 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:25.193 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2b18067100 msgr2=0x7f2b180695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2b18067100 0x7f2b180695c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b300ae000 tx=0x7f2b300a7070 comp rx=0 tx=0).stop 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 msgr2=0x7f2b401fc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 0x7f2b401fc8e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800c450 tx=0x7f2b3800c910 comp rx=0 tx=0).stop 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 shutdown_connections 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2b18067100 0x7f2b180695c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2b40165ec0 0x7f2b401fc8e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b401614e0 0x7f2b40162030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 --2- 172.21.15.143:0/3165118142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b400caf20 0x7f2b400b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 >> 172.21.15.143:0/3165118142 conn(0x7f2b400cfd20 msgr2=0x7f2b4006ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 shutdown_connections 2024-01-15T18:10:25.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:25.182+0000 7f2b45600640 1 -- 172.21.15.143:0/3165118142 wait complete. 2024-01-15T18:10:27.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:27.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 Processor -- start 2024-01-15T18:10:27.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 -- start start 2024-01-15T18:10:27.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f93440cde30 0x7f934415f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f93440cc6c0 0x7f93440b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93440ce290 con 0x7f934415ed40 2024-01-15T18:10:27.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93440cc2a0 con 0x7f93440cc6c0 2024-01-15T18:10:27.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9348e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934415f660 con 0x7f93440cde30 2024-01-15T18:10:27.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9343400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f934415ed40 0x7f93440cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37726/0 (socket says 172.21.15.143:37726) 2024-01-15T18:10:27.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.749+0000 7f9343400640 1 -- 172.21.15.143:0/3958813307 learned_addr learned my addr 172.21.15.143:0/3958813307 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:27.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.753+0000 7f9342a00640 1 --2- 172.21.15.143:0/3958813307 >> v2:172.21.15.204:3300/0 conn(0x7f93440cde30 0x7f934415f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.753+0000 7f9343e00640 1 --2- 172.21.15.143:0/3958813307 >> v2:172.21.15.143:3300/0 conn(0x7f93440cc6c0 0x7f93440b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.757+0000 7f9342a00640 1 -- 172.21.15.143:0/3958813307 >> v2:172.21.15.143:3300/0 conn(0x7f93440cc6c0 msgr2=0x7f93440b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:27.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.757+0000 7f9342a00640 1 --2- 172.21.15.143:0/3958813307 >> v2:172.21.15.143:3300/0 conn(0x7f93440cc6c0 0x7f93440b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.757+0000 7f9342a00640 1 -- 172.21.15.143:0/3958813307 >> v1:172.21.15.143:6789/0 conn(0x7f934415ed40 legacy=0x7f93440cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.757+0000 7f9342a00640 1 -- 172.21.15.143:0/3958813307 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440ce530 con 0x7f93440cde30 2024-01-15T18:10:27.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.777+0000 7f9342a00640 1 --2- 172.21.15.143:0/3958813307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93440cde30 0x7f934415f120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93440d1ef0 tx=0x7f932c01bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e3abcfbebd90db server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.777+0000 7f9342000640 1 -- 172.21.15.143:0/3958813307 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0a56d0 con 0x7f93440cde30 2024-01-15T18:10:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.777+0000 7f9342000640 1 -- 172.21.15.143:0/3958813307 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c0a5830 con 0x7f93440cde30 2024-01-15T18:10:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.777+0000 7f9342000640 1 -- 172.21.15.143:0/3958813307 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0a5bc0 con 0x7f93440cde30 2024-01-15T18:10:27.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 -- 172.21.15.143:0/3958813307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93440cde30 msgr2=0x7f934415f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 --2- 172.21.15.143:0/3958813307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93440cde30 0x7f934415f120 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f93440d1ef0 tx=0x7f932c01bee0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 -- 172.21.15.143:0/3958813307 shutdown_connections 2024-01-15T18:10:27.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 --2- 172.21.15.143:0/3958813307 >> v2:172.21.15.143:3300/0 conn(0x7f93440cc6c0 0x7f93440b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 --2- 172.21.15.143:0/3958813307 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f93440cde30 0x7f934415f120 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 -- 172.21.15.143:0/3958813307 >> 172.21.15.143:0/3958813307 conn(0x7f9344099a70 msgr2=0x7f9344099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:27.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.785+0000 7f9348e00640 1 -- 172.21.15.143:0/3958813307 shutdown_connections 2024-01-15T18:10:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.793+0000 7f9348e00640 1 -- 172.21.15.143:0/3958813307 wait complete. 2024-01-15T18:10:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.797+0000 7f9348e00640 1 Processor -- start 2024-01-15T18:10:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.797+0000 7f9348e00640 1 -- start start 2024-01-15T18:10:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9348e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 0x7f93440c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9348e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93440cde30 0x7f934415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 0x7f93440c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 0x7f93440c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39138/0 (socket says 172.21.15.143:39138) 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9348e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934415ed40 0x7f934415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9348e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344151fe0 con 0x7f93440cc6c0 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9348e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441522c0 con 0x7f934415ed40 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9348e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93441525a0 con 0x7f93440cde30 2024-01-15T18:10:27.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 -- 172.21.15.143:0/2841762333 learned_addr learned my addr 172.21.15.143:0/2841762333 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:27.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 -- 172.21.15.143:0/2841762333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93440cde30 msgr2=0x7f934415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93440cde30 0x7f934415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 -- 172.21.15.143:0/2841762333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934415ed40 msgr2=0x7f934415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934415ed40 0x7f934415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.801+0000 7f9343400640 1 -- 172.21.15.143:0/2841762333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009420 con 0x7f93440cc6c0 2024-01-15T18:10:27.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.805+0000 7f9343400640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 0x7f93440c7090 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f933400c780 tx=0x7f933400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.809+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340074e0 con 0x7f93440cc6c0 2024-01-15T18:10:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.809+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334007640 con 0x7f93440cc6c0 2024-01-15T18:10:27.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.809+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334004a30 con 0x7f93440cc6c0 2024-01-15T18:10:27.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.813+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9344152880 con 0x7f93440cc6c0 2024-01-15T18:10:27.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.813+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93440c75d0 con 0x7f93440cc6c0 2024-01-15T18:10:27.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.813+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344067c10 con 0x7f93440cc6c0 2024-01-15T18:10:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.837+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f9334004b90 con 0x7f93440cc6c0 2024-01-15T18:10:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.837+0000 7f933b400640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f931c06b640 0x7f931c06db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.837+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9334082b30 con 0x7f93440cc6c0 2024-01-15T18:10:27.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.841+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f93340515f0 con 0x7f93440cc6c0 2024-01-15T18:10:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.849+0000 7f9342a00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f931c06b640 0x7f931c06db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:27.853+0000 7f9342a00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f931c06b640 0x7f931c06db00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f932c0ae000 tx=0x7f932c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.190 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:28.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:28.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.545+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9344152d30 con 0x7f93440cc6c0 2024-01-15T18:10:28.561 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:28.561 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:28.562 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 354, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:28.563 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1804, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 7425196203, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 14255845376, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 758838267904, 2024-01-15T18:10:28.564 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 358104471, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 349755650, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 86, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 92 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:28.565 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:28.566 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 23.81% (12s)\n [======......................] (remaining: 39s)", 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.23810000717639923, 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:28.567 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 41.668% (12s)\n [===========.................] (remaining: 17s)", 2024-01-15T18:10:28.568 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.41667500138282776, 2024-01-15T18:10:28.568 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:28.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:28.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:28.568 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:28.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.553+0000 7f933b400640 1 -- 172.21.15.143:0/2841762333 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2425 (secure 0 0 0) 0x7f9334018030 con 0x7f93440cc6c0 2024-01-15T18:10:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.569+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f931c06b640 msgr2=0x7f931c06db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.569+0000 7f9348e00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f931c06b640 0x7f931c06db00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f932c0ae000 tx=0x7f932c008040 comp rx=0 tx=0).stop 2024-01-15T18:10:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.569+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 msgr2=0x7f93440c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.569+0000 7f9348e00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 0x7f93440c7090 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f933400c780 tx=0x7f933400cc40 comp rx=0 tx=0).stop 2024-01-15T18:10:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 shutdown_connections 2024-01-15T18:10:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f931c06b640 0x7f931c06db00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f934415ed40 0x7f934415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93440cde30 0x7f934415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 --2- 172.21.15.143:0/2841762333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f93440cc6c0 0x7f93440c7090 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 >> 172.21.15.143:0/2841762333 conn(0x7f9344099a70 msgr2=0x7f93440add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 shutdown_connections 2024-01-15T18:10:28.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:28.581+0000 7f9348e00640 1 -- 172.21.15.143:0/2841762333 wait complete. 2024-01-15T18:10:30.706 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:31.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.221+0000 7fec75400640 1 Processor -- start 2024-01-15T18:10:31.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.221+0000 7fec75400640 1 -- start start 2024-01-15T18:10:31.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.225+0000 7fec75400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fec70155930 0x7fec7015c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:31.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.225+0000 7fec75400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec70140c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:31.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.225+0000 7fec75400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec700ce140 con 0x7fec70155550 2024-01-15T18:10:31.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.225+0000 7fec75400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700cecc0 con 0x7fec70155930 2024-01-15T18:10:31.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.225+0000 7fec75400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec700cdf60 con 0x7fec70140800 2024-01-15T18:10:31.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.229+0000 7fec6de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fec70155930 0x7fec7015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:31.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.229+0000 7fec6de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fec70155930 0x7fec7015c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:54146/0 (socket says 172.21.15.143:54146) 2024-01-15T18:10:31.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.229+0000 7fec6de00640 1 -- 172.21.15.143:0/2080461687 learned_addr learned my addr 172.21.15.143:0/2080461687 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:31.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.229+0000 7fec6f200640 1 --2- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec70140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:31.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.233+0000 7fec6de00640 1 -- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 msgr2=0x7fec70140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:31.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.233+0000 7fec6de00640 1 --2- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec70140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.233+0000 7fec6de00640 1 -- 172.21.15.143:0/2080461687 >> v1:172.21.15.204:6789/0 conn(0x7fec70155550 legacy=0x7fec700cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:31.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.233+0000 7fec6de00640 1 -- 172.21.15.143:0/2080461687 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec700ce420 con 0x7fec70155930 2024-01-15T18:10:31.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.245+0000 7fec6de00640 1 --2- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70155930 0x7fec7015c270 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec60009870 tx=0x7fec6001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb871b423815160 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:31.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.245+0000 7fec6d400640 1 -- 172.21.15.143:0/2080461687 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600ac600 con 0x7fec70155930 2024-01-15T18:10:31.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.245+0000 7fec6d400640 1 -- 172.21.15.143:0/2080461687 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600a8050 con 0x7fec70155930 2024-01-15T18:10:31.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.245+0000 7fec6d400640 1 -- 172.21.15.143:0/2080461687 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600ac980 con 0x7fec70155930 2024-01-15T18:10:31.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.249+0000 7fec75400640 1 -- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70155930 msgr2=0x7fec7015c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:31.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.249+0000 7fec75400640 1 --2- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70155930 0x7fec7015c270 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fec60009870 tx=0x7fec6001c4d0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.253+0000 7fec75400640 1 -- 172.21.15.143:0/2080461687 shutdown_connections 2024-01-15T18:10:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.253+0000 7fec75400640 1 --2- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec70140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.253+0000 7fec75400640 1 --2- 172.21.15.143:0/2080461687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70155930 0x7fec7015c270 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.253+0000 7fec75400640 1 -- 172.21.15.143:0/2080461687 >> 172.21.15.143:0/2080461687 conn(0x7fec700b0bf0 msgr2=0x7fec700b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:31.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.265+0000 7fec75400640 1 -- 172.21.15.143:0/2080461687 shutdown_connections 2024-01-15T18:10:31.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.269+0000 7fec75400640 1 -- 172.21.15.143:0/2080461687 wait complete. 2024-01-15T18:10:31.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.277+0000 7fec75400640 1 Processor -- start 2024-01-15T18:10:31.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.277+0000 7fec75400640 1 -- start start 2024-01-15T18:10:31.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec75400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec7006c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:31.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec75400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 0x7fec70154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:31.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec75400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70154850 0x7fec7006e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:31.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec75400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70154ce0 con 0x7fec70155550 2024-01-15T18:10:31.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec75400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700ae8e0 con 0x7fec70154850 2024-01-15T18:10:31.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec75400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec700aebc0 con 0x7fec70140800 2024-01-15T18:10:31.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70154850 0x7fec7006e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:31.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 0x7fec70154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:31.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 0x7fec70154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39148/0 (socket says 172.21.15.143:39148) 2024-01-15T18:10:31.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6de00640 1 -- 172.21.15.143:0/1989749641 learned_addr learned my addr 172.21.15.143:0/1989749641 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:31.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6e800640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec7006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:31.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6de00640 1 -- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 msgr2=0x7fec7006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:31.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.281+0000 7fec6de00640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec7006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.285+0000 7fec6de00640 1 -- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70154850 msgr2=0x7fec7006e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:31.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.285+0000 7fec6de00640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70154850 0x7fec7006e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.285+0000 7fec6de00640 1 -- 172.21.15.143:0/1989749641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600094d0 con 0x7fec70155550 2024-01-15T18:10:31.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.297+0000 7fec6de00640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 0x7fec70154310 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fec600190e0 tx=0x7fec60019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.301+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600b7470 con 0x7fec70155550 2024-01-15T18:10:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.301+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec600b75d0 con 0x7fec70155550 2024-01-15T18:10:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.301+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec600b7940 con 0x7fec70155550 2024-01-15T18:10:31.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.301+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec700aeea0 con 0x7fec70155550 2024-01-15T18:10:31.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.301+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7006e630 con 0x7fec70155550 2024-01-15T18:10:31.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.305+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec700af350 con 0x7fec70155550 2024-01-15T18:10:31.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.309+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7fec600b3070 con 0x7fec70155550 2024-01-15T18:10:31.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.317+0000 7fec66a00640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec4806b5c0 0x7fec4806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:31.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.317+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fec6011da00 con 0x7fec70155550 2024-01-15T18:10:31.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.325+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec600ec500 con 0x7fec70155550 2024-01-15T18:10:31.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.333+0000 7fec6e800640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec4806b5c0 0x7fec4806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:31.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.341+0000 7fec6e800640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec4806b5c0 0x7fec4806da80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fec7006dee0 tx=0x7fec68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:31.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.917+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fec700af350 con 0x7fec70155550 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:31.928 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 358, 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.929 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:31.930 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 2204, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 9098866354, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 17812160512, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 755281952768, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 399226416, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 400296339, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 95, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 103 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.931 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:31.932 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:31.933 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 23.81% (12s)\n [======......................] (remaining: 39s)", 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.23810000717639923, 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 41.668% (12s)\n [===========.................] (remaining: 17s)", 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.41667500138282776, 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:31.934 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.921+0000 7fec66a00640 1 -- 172.21.15.143:0/1989749641 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7fec700af350 con 0x7fec70155550 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec4806b5c0 msgr2=0x7fec4806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec4806b5c0 0x7fec4806da80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fec7006dee0 tx=0x7fec68009040 comp rx=0 tx=0).stop 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 msgr2=0x7fec70154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 0x7fec70154310 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fec600190e0 tx=0x7fec60019270 comp rx=0 tx=0).stop 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 shutdown_connections 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec4806b5c0 0x7fec4806da80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec70154850 0x7fec7006e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec70155550 0x7fec70154310 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 --2- 172.21.15.143:0/1989749641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec70140800 0x7fec7006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.929+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 >> 172.21.15.143:0/1989749641 conn(0x7fec700b0bf0 msgr2=0x7fec700b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.941+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 shutdown_connections 2024-01-15T18:10:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:31.945+0000 7fec75400640 1 -- 172.21.15.143:0/1989749641 wait complete. 2024-01-15T18:10:34.009 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 Processor -- start 2024-01-15T18:10:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 -- start start 2024-01-15T18:10:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0a380b2aa0 0x7f0a38070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b3590 0x7f0a3816b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a38071d80 con 0x7f0a3806d3b0 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a38070ce0 con 0x7f0a380b2aa0 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.193+0000 7f0a3d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38070fc0 con 0x7f0a380b3590 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a35400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0a3806d3b0 0x7f0a380b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:51236/0 (socket says 172.21.15.143:51236) 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a35400640 1 -- 172.21.15.143:0/2460348986 learned_addr learned my addr 172.21.15.143:0/2460348986 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a36800640 1 --2- 172.21.15.143:0/2460348986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b3590 0x7f0a3816b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a35e00640 1 --2- 172.21.15.143:0/2460348986 >> v2:172.21.15.204:3300/0 conn(0x7f0a380b2aa0 0x7f0a38070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a36800640 1 -- 172.21.15.143:0/2460348986 >> v2:172.21.15.204:3300/0 conn(0x7f0a380b2aa0 msgr2=0x7f0a38070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a36800640 1 --2- 172.21.15.143:0/2460348986 >> v2:172.21.15.204:3300/0 conn(0x7f0a380b2aa0 0x7f0a38070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a36800640 1 -- 172.21.15.143:0/2460348986 >> v1:172.21.15.204:6789/0 conn(0x7f0a3806d3b0 legacy=0x7f0a380b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.197+0000 7f0a36800640 1 -- 172.21.15.143:0/2460348986 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380712a0 con 0x7f0a380b3590 2024-01-15T18:10:34.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a36800640 1 --2- 172.21.15.143:0/2460348986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b3590 0x7f0a3816b310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009d90 tx=0x7f0a3001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d44d619ef7fb25 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a34a00640 1 -- 172.21.15.143:0/2460348986 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300a5970 con 0x7f0a380b3590 2024-01-15T18:10:34.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a34a00640 1 -- 172.21.15.143:0/2460348986 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a300a5ad0 con 0x7f0a380b3590 2024-01-15T18:10:34.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a34a00640 1 -- 172.21.15.143:0/2460348986 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300a5e60 con 0x7f0a380b3590 2024-01-15T18:10:34.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 -- 172.21.15.143:0/2460348986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b3590 msgr2=0x7f0a3816b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 --2- 172.21.15.143:0/2460348986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b3590 0x7f0a3816b310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a30009d90 tx=0x7f0a3001c890 comp rx=0 tx=0).stop 2024-01-15T18:10:34.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 -- 172.21.15.143:0/2460348986 shutdown_connections 2024-01-15T18:10:34.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 --2- 172.21.15.143:0/2460348986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b3590 0x7f0a3816b310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 --2- 172.21.15.143:0/2460348986 >> v2:172.21.15.204:3300/0 conn(0x7f0a380b2aa0 0x7f0a38070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 -- 172.21.15.143:0/2460348986 >> 172.21.15.143:0/2460348986 conn(0x7f0a380b0bf0 msgr2=0x7f0a380b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 -- 172.21.15.143:0/2460348986 shutdown_connections 2024-01-15T18:10:34.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.201+0000 7f0a3d000640 1 -- 172.21.15.143:0/2460348986 wait complete. 2024-01-15T18:10:34.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 Processor -- start 2024-01-15T18:10:34.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 -- start start 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3806d3b0 0x7f0a38139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b2aa0 0x7f0a38139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 0x7f0a380a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380aa1d0 con 0x7f0a380b3590 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a381596b0 con 0x7f0a3806d3b0 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.205+0000 7f0a3d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3813a1f0 con 0x7f0a380b2aa0 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 0x7f0a380a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 0x7f0a380a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39160/0 (socket says 172.21.15.143:39160) 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 -- 172.21.15.143:0/1551986590 learned_addr learned my addr 172.21.15.143:0/1551986590 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 -- 172.21.15.143:0/1551986590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b2aa0 msgr2=0x7f0a38139760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b2aa0 0x7f0a38139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 -- 172.21.15.143:0/1551986590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3806d3b0 msgr2=0x7f0a38139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3806d3b0 0x7f0a38139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 -- 172.21.15.143:0/1551986590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a30009a70 con 0x7f0a380b3590 2024-01-15T18:10:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a36800640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 0x7f0a380a84d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0a300b0040 tx=0x7f0a30019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300ade20 con 0x7f0a380b3590 2024-01-15T18:10:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a30009450 con 0x7f0a380b3590 2024-01-15T18:10:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a380aa9a0 con 0x7f0a380b3590 2024-01-15T18:10:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300b9690 con 0x7f0a380b3590 2024-01-15T18:10:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.209+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a380aaf60 con 0x7f0a380b3590 2024-01-15T18:10:34.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.213+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38067680 con 0x7f0a380b3590 2024-01-15T18:10:34.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.213+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f0a300b97f0 con 0x7f0a380b3590 2024-01-15T18:10:34.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.217+0000 7f0a2e000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a10067160 0x7f0a10069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.217+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0a3011e650 con 0x7f0a380b3590 2024-01-15T18:10:34.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.217+0000 7f0a35e00640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a10067160 0x7f0a10069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.221+0000 7f0a35e00640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a10067160 0x7f0a10069620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a38148290 tx=0x7f0a28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.225+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0a300ed110 con 0x7f0a380b3590 2024-01-15T18:10:34.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.705+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0a3813bb90 con 0x7f0a380b3590 2024-01-15T18:10:34.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.709+0000 7f0a2e000640 1 -- 172.21.15.143:0/1551986590 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2410 (secure 0 0 0) 0x7f0a300f0730 con 0x7f0a380b3590 2024-01-15T18:10:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:34.713 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 361, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 3024, 2024-01-15T18:10:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 12538288229, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 24786907136, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 748307206144, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 682761180, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 686984380, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 163, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 174 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.717 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:34.719 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:34.720 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 42.857% (19s)\n [============................] (remaining: 25s)", 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.4285714328289032, 2024-01-15T18:10:34.721 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "2 ongoing cloning jobs; Avg progress = 50.0% (19s)\n [==============..............] (remaining: 19s)", 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.5, 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:34.722 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a10067160 msgr2=0x7f0a10069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a10067160 0x7f0a10069620 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a38148290 tx=0x7f0a28008040 comp rx=0 tx=0).stop 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 msgr2=0x7f0a380a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 0x7f0a380a84d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0a300b0040 tx=0x7f0a30019630 comp rx=0 tx=0).stop 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 shutdown_connections 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a10067160 0x7f0a10069620 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a380b3590 0x7f0a380a84d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380b2aa0 0x7f0a38139760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 --2- 172.21.15.143:0/1551986590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a3806d3b0 0x7f0a38139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 >> 172.21.15.143:0/1551986590 conn(0x7f0a380b0bf0 msgr2=0x7f0a38148450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 shutdown_connections 2024-01-15T18:10:34.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:34.717+0000 7f0a3d000640 1 -- 172.21.15.143:0/1551986590 wait complete. 2024-01-15T18:10:36.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:37.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.209+0000 7f0616c00640 1 Processor -- start 2024-01-15T18:10:37.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.209+0000 7f0616c00640 1 -- start start 2024-01-15T18:10:37.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f061015f150 0x7f061006c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:37.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f06101657f0 0x7f0610070280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:37.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061015f530 con 0x7f061006cdc0 2024-01-15T18:10:37.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061006e050 con 0x7f06101657f0 2024-01-15T18:10:37.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0610165ec0 con 0x7f061015f150 2024-01-15T18:10:37.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0615800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f061015f150 0x7f061006c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:37.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f06101657f0 0x7f0610070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:37.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0615800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f061015f150 0x7f061006c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:48492/0 (socket says 172.21.15.143:48492) 2024-01-15T18:10:37.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0616200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f06101657f0 0x7f0610070280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:35474/0 (socket says 172.21.15.143:35474) 2024-01-15T18:10:37.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.213+0000 7f0615800640 1 -- 172.21.15.143:0/3205166509 learned_addr learned my addr 172.21.15.143:0/3205166509 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:37.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.217+0000 7f0616200640 1 -- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f061015f150 msgr2=0x7f061006c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:37.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.217+0000 7f0616200640 1 --2- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f061015f150 0x7f061006c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.217+0000 7f0616200640 1 -- 172.21.15.143:0/3205166509 >> v1:172.21.15.204:6789/0 conn(0x7f061006cdc0 legacy=0x7f06101650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.217+0000 7f0616200640 1 -- 172.21.15.143:0/3205166509 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061006d3d0 con 0x7f06101657f0 2024-01-15T18:10:37.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.221+0000 7f0616200640 1 --2- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06101657f0 0x7f0610070280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097c0 tx=0x7f05f801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2457604537bd2fa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:37.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.221+0000 7f060fe00640 1 -- 172.21.15.143:0/3205166509 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f80ad660 con 0x7f06101657f0 2024-01-15T18:10:37.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.221+0000 7f060fe00640 1 -- 172.21.15.143:0/3205166509 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f80ad7c0 con 0x7f06101657f0 2024-01-15T18:10:37.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 -- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06101657f0 msgr2=0x7f0610070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 --2- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06101657f0 0x7f0610070280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097c0 tx=0x7f05f801c450 comp rx=0 tx=0).stop 2024-01-15T18:10:37.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 -- 172.21.15.143:0/3205166509 shutdown_connections 2024-01-15T18:10:37.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 --2- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06101657f0 0x7f0610070280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 --2- 172.21.15.143:0/3205166509 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f061015f150 0x7f061006c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 -- 172.21.15.143:0/3205166509 >> 172.21.15.143:0/3205166509 conn(0x7f061006a890 msgr2=0x7f061006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:37.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 -- 172.21.15.143:0/3205166509 shutdown_connections 2024-01-15T18:10:37.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.225+0000 7f0616c00640 1 -- 172.21.15.143:0/3205166509 wait complete. 2024-01-15T18:10:37.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.233+0000 7f0616c00640 1 Processor -- start 2024-01-15T18:10:37.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.233+0000 7f0616c00640 1 -- start start 2024-01-15T18:10:37.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f061006cdc0 0x7f061014a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:37.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061015f150 0x7f061014a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:37.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 0x7f06101545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061014c2e0 con 0x7f061006cdc0 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061014ad30 con 0x7f061015f150 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061014b010 con 0x7f06101657f0 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 0x7f06101545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 0x7f06101545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:48502/0 (socket says 172.21.15.143:48502) 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0616200640 1 -- 172.21.15.143:0/2503153426 learned_addr learned my addr 172.21.15.143:0/2503153426 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0614e00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061015f150 0x7f061014a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.237+0000 7f0615800640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f061006cdc0 0x7f061014a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.241+0000 7f0616200640 1 -- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061015f150 msgr2=0x7f061014a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.241+0000 7f0616200640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061015f150 0x7f061014a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.241+0000 7f0616200640 1 -- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f061006cdc0 msgr2=0x7f061014a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.241+0000 7f0616200640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f061006cdc0 0x7f061014a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.241+0000 7f0616200640 1 -- 172.21.15.143:0/2503153426 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009420 con 0x7f06101657f0 2024-01-15T18:10:37.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.241+0000 7f0615800640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f061006cdc0 0x7f061014a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:37.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.245+0000 7f0616200640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 0x7f06101545c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f05f801a080 tx=0x7f05f80191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:37.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.245+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8004500 con 0x7f06101657f0 2024-01-15T18:10:37.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.245+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8003620 con 0x7f06101657f0 2024-01-15T18:10:37.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.245+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8003990 con 0x7f06101657f0 2024-01-15T18:10:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.245+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061014b2f0 con 0x7f06101657f0 2024-01-15T18:10:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.245+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061014b8b0 con 0x7f06101657f0 2024-01-15T18:10:37.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.249+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f05f80b4050 con 0x7f06101657f0 2024-01-15T18:10:37.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.253+0000 7f060d600640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05e8067230 0x7f05e80696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:37.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.253+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f05f811d3a0 con 0x7f06101657f0 2024-01-15T18:10:37.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.253+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0610156860 con 0x7f06101657f0 2024-01-15T18:10:37.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.265+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0610156860 con 0x7f06101657f0 2024-01-15T18:10:37.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.277+0000 7f0615800640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05e8067230 0x7f05e80696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:37.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.309+0000 7f0615800640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05e8067230 0x7f05e80696f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0600009a50 tx=0x7f0600008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:37.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.677+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0610156860 con 0x7f06101657f0 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.677+0000 7f060d600640 1 -- 172.21.15.143:0/2503153426 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2410 (secure 0 0 0) 0x7f0610156860 con 0x7f06101657f0 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:37.682 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 364, 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:37.683 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:37.684 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 3243, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 13461209421, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 26653605888, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 746440507392, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 590500006, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 595428484, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 141, 2024-01-15T18:10:37.685 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 153 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:37.686 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:37.687 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 42.857% (19s)\n [============................] (remaining: 25s)", 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.4285714328289032, 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "2 ongoing cloning jobs; Avg progress = 50.0% (19s)\n [==============..............] (remaining: 19s)", 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.5, 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:37.688 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:37.689 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:37.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05e8067230 msgr2=0x7f05e80696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05e8067230 0x7f05e80696f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0600009a50 tx=0x7f0600008040 comp rx=0 tx=0).stop 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 msgr2=0x7f06101545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 0x7f06101545c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f05f801a080 tx=0x7f05f80191f0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 shutdown_connections 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05e8067230 0x7f05e80696f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06101657f0 0x7f06101545c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f061015f150 0x7f061014a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 --2- 172.21.15.143:0/2503153426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f061006cdc0 0x7f061014a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 >> 172.21.15.143:0/2503153426 conn(0x7f061006a890 msgr2=0x7f061015fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 shutdown_connections 2024-01-15T18:10:37.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:37.681+0000 7f0616c00640 1 -- 172.21.15.143:0/2503153426 wait complete. 2024-01-15T18:10:39.705 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:40.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 Processor -- start 2024-01-15T18:10:40.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 -- start start 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3388169ed0 0x7f338806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f338808aa10 0x7f33880b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338816a2b0 con 0x7f338806db90 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33880b3140 con 0x7f3388169ed0 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f3390400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338806caf0 con 0x7f338808aa10 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f338da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3388169ed0 0x7f338806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f338d000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f338806db90 0x7f338808a300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35292/0 (socket says 172.21.15.143:35292) 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:39.997+0000 7f338d000640 1 -- 172.21.15.143:0/3177330712 learned_addr learned my addr 172.21.15.143:0/3177330712 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.013+0000 7f338da00640 1 -- 172.21.15.143:0/3177330712 >> v2:172.21.15.204:3300/0 conn(0x7f338808aa10 msgr2=0x7f33880b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.001+0000 7f338e400640 1 --2- 172.21.15.143:0/3177330712 >> v2:172.21.15.204:3300/0 conn(0x7f338808aa10 0x7f33880b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.001+0000 7f3387e00640 1 -- 172.21.15.143:0/3177330712 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557170739 0 0) 0x7f338816a2b0 con 0x7f338806db90 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.013+0000 7f338da00640 1 --2- 172.21.15.143:0/3177330712 >> v2:172.21.15.204:3300/0 conn(0x7f338808aa10 0x7f33880b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.013+0000 7f338da00640 1 -- 172.21.15.143:0/3177330712 >> v1:172.21.15.143:6789/0 conn(0x7f338806db90 legacy=0x7f338808a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.013+0000 7f338da00640 1 -- 172.21.15.143:0/3177330712 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338806cdd0 con 0x7f3388169ed0 2024-01-15T18:10:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.013+0000 7f338e400640 1 --2- 172.21.15.143:0/3177330712 >> v2:172.21.15.204:3300/0 conn(0x7f338808aa10 0x7f33880b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.017+0000 7f338da00640 1 --2- 172.21.15.143:0/3177330712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3388169ed0 0x7f338806c3f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f33880cbc40 tx=0x7f337801d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3765b73f902d32fa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.017+0000 7f3387e00640 1 -- 172.21.15.143:0/3177330712 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780a76d0 con 0x7f3388169ed0 2024-01-15T18:10:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.017+0000 7f3387e00640 1 -- 172.21.15.143:0/3177330712 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33780a7830 con 0x7f3388169ed0 2024-01-15T18:10:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.017+0000 7f3387e00640 1 -- 172.21.15.143:0/3177330712 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33780a7bc0 con 0x7f3388169ed0 2024-01-15T18:10:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 -- 172.21.15.143:0/3177330712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3388169ed0 msgr2=0x7f338806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 --2- 172.21.15.143:0/3177330712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3388169ed0 0x7f338806c3f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f33880cbc40 tx=0x7f337801d610 comp rx=0 tx=0).stop 2024-01-15T18:10:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 -- 172.21.15.143:0/3177330712 shutdown_connections 2024-01-15T18:10:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 --2- 172.21.15.143:0/3177330712 >> v2:172.21.15.204:3300/0 conn(0x7f338808aa10 0x7f33880b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 --2- 172.21.15.143:0/3177330712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3388169ed0 0x7f338806c3f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 -- 172.21.15.143:0/3177330712 >> 172.21.15.143:0/3177330712 conn(0x7f33880aea10 msgr2=0x7f33880aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 -- 172.21.15.143:0/3177330712 shutdown_connections 2024-01-15T18:10:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.021+0000 7f3390400640 1 -- 172.21.15.143:0/3177330712 wait complete. 2024-01-15T18:10:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 Processor -- start 2024-01-15T18:10:40.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 -- start start 2024-01-15T18:10:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 0x7f3388166bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f338808aa10 0x7f33881595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3388169ed0 0x7f338820acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33881669a0 con 0x7f338808aa10 2024-01-15T18:10:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338820b3f0 con 0x7f338806db90 2024-01-15T18:10:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.025+0000 7f3390400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338820b6d0 con 0x7f3388169ed0 2024-01-15T18:10:40.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 0x7f3388166bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 0x7f3388166bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:35490/0 (socket says 172.21.15.143:35490) 2024-01-15T18:10:40.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338da00640 1 -- 172.21.15.143:0/1728469021 learned_addr learned my addr 172.21.15.143:0/1728469021 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:40.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338d000640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f338808aa10 0x7f33881595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338d000640 1 -- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3388169ed0 msgr2=0x7f338820acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338e400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3388169ed0 0x7f338820acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338d000640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3388169ed0 0x7f338820acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338d000640 1 -- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 msgr2=0x7f3388166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338d000640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 0x7f3388166bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338d000640 1 -- 172.21.15.143:0/1728469021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378007490 con 0x7f338808aa10 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338da00640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 0x7f3388166bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.029+0000 7f338e400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3388169ed0 0x7f338820acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:40.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f338d000640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f338808aa10 0x7f33881595f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f337000c430 tx=0x7f337000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700074e0 con 0x7f338808aa10 2024-01-15T18:10:40.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338820b9b0 con 0x7f338808aa10 2024-01-15T18:10:40.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338820bf70 con 0x7f338808aa10 2024-01-15T18:10:40.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370007660 con 0x7f338808aa10 2024-01-15T18:10:40.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370004a30 con 0x7f338808aa10 2024-01-15T18:10:40.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.033+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3358005260 con 0x7f338808aa10 2024-01-15T18:10:40.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.037+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f3370004b90 con 0x7f338808aa10 2024-01-15T18:10:40.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.037+0000 7f3385600640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3360067160 0x7f3360069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.037+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3370085610 con 0x7f338808aa10 2024-01-15T18:10:40.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.041+0000 7f338da00640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3360067160 0x7f3360069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.053+0000 7f338da00640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3360067160 0x7f3360069620 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f337801d650 tx=0x7f33780b2040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.053+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33700518b0 con 0x7f338808aa10 2024-01-15T18:10:40.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.461+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f33580050d0 con 0x7f338808aa10 2024-01-15T18:10:40.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.469+0000 7f3385600640 1 -- 172.21.15.143:0/1728469021 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2428 (secure 0 0 0) 0x7f3370054ed0 con 0x7f338808aa10 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:40.478 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 366, 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:40.479 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:40.480 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 3490, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 14493009305, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 28958171136, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 744135942144, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 587765740, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 588501302, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 140, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 152 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.481 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:40.482 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.483 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 52.381% (24s)\n [==============..............] (remaining: 22s)", 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.52381426095962524, 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "2 ongoing cloning jobs; Avg progress = 83.335% (24s)\n [=======================.....] (remaining: 4s)", 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.83335000276565552, 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:40.484 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3360067160 msgr2=0x7f3360069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3360067160 0x7f3360069620 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f337801d650 tx=0x7f33780b2040 comp rx=0 tx=0).stop 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f338808aa10 msgr2=0x7f33881595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f338808aa10 0x7f33881595f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f337000c430 tx=0x7f337000c8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 shutdown_connections 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3360067160 0x7f3360069620 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3388169ed0 0x7f338820acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f338808aa10 0x7f33881595f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 --2- 172.21.15.143:0/1728469021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f338806db90 0x7f3388166bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 >> 172.21.15.143:0/1728469021 conn(0x7f33880aea10 msgr2=0x7f3388168f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:40.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 shutdown_connections 2024-01-15T18:10:40.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:40.477+0000 7f3390400640 1 -- 172.21.15.143:0/1728469021 wait complete. 2024-01-15T18:10:42.539 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:10:42.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 Processor -- start 2024-01-15T18:10:42.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 -- start start 2024-01-15T18:10:42.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4324141660 0x7f43240ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f432415ec80 0x7f4324141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432415f310 con 0x7f4324142470 2024-01-15T18:10:42.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415fbb0 con 0x7f432415ec80 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4328c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432415f130 con 0x7f4324141660 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f432415ec80 0x7f4324141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f432415ec80 0x7f4324141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42956/0 (socket says 172.21.15.143:42956) 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 -- 172.21.15.143:0/3855492947 learned_addr learned my addr 172.21.15.143:0/3855492947 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4322800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4324142470 0x7f4324071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35296/0 (socket says 172.21.15.143:35296) 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321e00640 1 --2- 172.21.15.143:0/3855492947 >> v2:172.21.15.204:3300/0 conn(0x7f4324141660 0x7f43240ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 -- 172.21.15.143:0/3855492947 >> v2:172.21.15.204:3300/0 conn(0x7f4324141660 msgr2=0x7f43240ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:42.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 --2- 172.21.15.143:0/3855492947 >> v2:172.21.15.204:3300/0 conn(0x7f4324141660 0x7f43240ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 -- 172.21.15.143:0/3855492947 >> v1:172.21.15.143:6789/0 conn(0x7f4324142470 legacy=0x7f4324071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.673+0000 7f4321400640 1 -- 172.21.15.143:0/3855492947 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432415f5f0 con 0x7f432415ec80 2024-01-15T18:10:42.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4321400640 1 --2- 172.21.15.143:0/3855492947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432415ec80 0x7f4324141f30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f43240b1aa0 tx=0x7f430c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94410a1aa2fadec6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4320a00640 1 -- 172.21.15.143:0/3855492947 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0aa430 con 0x7f432415ec80 2024-01-15T18:10:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4320a00640 1 -- 172.21.15.143:0/3855492947 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c0aa590 con 0x7f432415ec80 2024-01-15T18:10:42.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4320a00640 1 -- 172.21.15.143:0/3855492947 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0aa920 con 0x7f432415ec80 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4328c00640 1 -- 172.21.15.143:0/3855492947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432415ec80 msgr2=0x7f4324141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4328c00640 1 --2- 172.21.15.143:0/3855492947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432415ec80 0x7f4324141f30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f43240b1aa0 tx=0x7f430c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4328c00640 1 -- 172.21.15.143:0/3855492947 shutdown_connections 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4328c00640 1 --2- 172.21.15.143:0/3855492947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432415ec80 0x7f4324141f30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4328c00640 1 --2- 172.21.15.143:0/3855492947 >> v2:172.21.15.204:3300/0 conn(0x7f4324141660 0x7f43240ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.677+0000 7f4328c00640 1 -- 172.21.15.143:0/3855492947 >> 172.21.15.143:0/3855492947 conn(0x7f432406c3f0 msgr2=0x7f432406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 -- 172.21.15.143:0/3855492947 shutdown_connections 2024-01-15T18:10:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 -- 172.21.15.143:0/3855492947 wait complete. 2024-01-15T18:10:42.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 Processor -- start 2024-01-15T18:10:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 -- start start 2024-01-15T18:10:42.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4324141660 0x7f432415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 0x7f4324139f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f432415ec80 0x7f432413a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43241442a0 con 0x7f4324142470 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324143690 con 0x7f432415ec80 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4328c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4324143970 con 0x7f4324141660 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 0x7f4324139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 0x7f4324139f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42958/0 (socket says 172.21.15.143:42958) 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 -- 172.21.15.143:0/2516415325 learned_addr learned my addr 172.21.15.143:0/2516415325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321e00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4324141660 0x7f432415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4322800640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f432415ec80 0x7f432413a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 -- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4324141660 msgr2=0x7f432415a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4324141660 0x7f432415a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 -- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f432415ec80 msgr2=0x7f432413a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f432415ec80 0x7f432413a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 -- 172.21.15.143:0/2516415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0094d0 con 0x7f4324142470 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.681+0000 7f4321400640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 0x7f4324139f30 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f430c0190e0 tx=0x7f430c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0aa430 con 0x7f4324142470 2024-01-15T18:10:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c0b3800 con 0x7f4324142470 2024-01-15T18:10:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43240cd150 con 0x7f4324142470 2024-01-15T18:10:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0b3bb0 con 0x7f4324142470 2024-01-15T18:10:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4324143f30 con 0x7f4324142470 2024-01-15T18:10:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324067500 con 0x7f4324142470 2024-01-15T18:10:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f430c0b3d10 con 0x7f4324142470 2024-01-15T18:10:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f431a000640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42fc067160 0x7f42fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f430c11cae0 con 0x7f4324142470 2024-01-15T18:10:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.685+0000 7f4321e00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42fc067160 0x7f42fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.689+0000 7f4321e00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42fc067160 0x7f42fc069620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f432415ea60 tx=0x7f431c0098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.689+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f430c0eb820 con 0x7f4324142470 2024-01-15T18:10:42.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.897+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f432406e2a0 con 0x7f4324142470 2024-01-15T18:10:42.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.897+0000 7f431a000640 1 -- 172.21.15.143:0/2516415325 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2403 (secure 0 0 0) 0x7f430c0eee40 con 0x7f4324142470 2024-01-15T18:10:42.902 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:42.903 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 369, 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 60, 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:10:42.904 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:10:42.905 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 3878, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 16107967341, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 32376791040, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 740717322240, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 574157176, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 583624688, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 138, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 150 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 36, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 4, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:10:42.906 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 4, 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6336 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:10:42.907 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 9, 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:10:24.725738+0000", 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "035f7356-1e57-4373-b6c5-6609a24ef30c": { 2024-01-15T18:10:42.908 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 57.143% (28s)\n [================............] (remaining: 21s)", 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.57142859697341919, 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: "87cbb1c5-c482-4025-b135-23b7436b7f07": { 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "Subvolume \"subvol_clone42257\" has been 100.0% cloned (28s)\n [============================] ", 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 1, 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:42.909 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42fc067160 msgr2=0x7f42fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42fc067160 0x7f42fc069620 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f432415ea60 tx=0x7f431c0098d0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 msgr2=0x7f4324139f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 0x7f4324139f30 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f430c0190e0 tx=0x7f430c01ad10 comp rx=0 tx=0).stop 2024-01-15T18:10:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 shutdown_connections 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f42fc067160 0x7f42fc069620 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f432415ec80 0x7f432413a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4324142470 0x7f4324139f30 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 --2- 172.21.15.143:0/2516415325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4324141660 0x7f432415a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 >> 172.21.15.143:0/2516415325 conn(0x7f432406c3f0 msgr2=0x7f43240aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 shutdown_connections 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:42.901+0000 7f4328c00640 1 -- 172.21.15.143:0/2516415325 wait complete. 2024-01-15T18:10:44.925 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:10:44.926 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42251 2024-01-15T18:10:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f7800640 1 Processor -- start 2024-01-15T18:10:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f7800640 1 -- start start 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f7800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd1f00ccf60 0x7fd1f0167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f7800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f006c5d0 con 0x7fd1f01607c0 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f7800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f00cffd0 con 0x7fd1f00cfb30 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f7800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f00d1ce0 con 0x7fd1f00ccf60 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f4e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd1f01607c0 0x7fd1f006b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35298/0 (socket says 172.21.15.143:35298) 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f4e00640 1 -- 172.21.15.143:0/3401280113 learned_addr learned my addr 172.21.15.143:0/3401280113 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f5800640 1 --2- 172.21.15.143:0/3401280113 >> v2:172.21.15.204:3300/0 conn(0x7fd1f00ccf60 0x7fd1f0167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485344763 0 0) 0x7fd1f006c5d0 con 0x7fd1f01607c0 2024-01-15T18:10:45.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d0003530 con 0x7fd1f01607c0 2024-01-15T18:10:45.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3390425152 0 0) 0x7fd1d0003530 con 0x7fd1f01607c0 2024-01-15T18:10:45.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 >> v2:172.21.15.204:3300/0 conn(0x7fd1f00ccf60 msgr2=0x7fd1f0167fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:45.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 --2- 172.21.15.143:0/3401280113 >> v2:172.21.15.204:3300/0 conn(0x7fd1f00ccf60 0x7fd1f0167fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 >> v1:172.21.15.204:6789/0 conn(0x7fd1f00cfb30 legacy=0x7fd1f006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00d02b0 con 0x7fd1f01607c0 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fd1e0001680 con 0x7fd1f01607c0 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 >> v1:172.21.15.143:6789/0 conn(0x7fd1f01607c0 legacy=0x7fd1f006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1d0003980 0x7fd1d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d00047d0 0x7fd1d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00d1ce0 con 0x7fd1d0005190 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d000c040 con 0x7fd1d00047d0 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d000c200 con 0x7fd1d0003980 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f5800640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1f4e00640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1d0003980 0x7fd1d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.069+0000 7fd1efe00640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d00047d0 0x7fd1d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 -- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1d0003980 msgr2=0x7fd1d0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1d0003980 0x7fd1d0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 -- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d00047d0 msgr2=0x7fd1d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:45.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d00047d0 0x7fd1d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 -- 172.21.15.143:0/3401280113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d0003530 con 0x7fd1d0005190 2024-01-15T18:10:45.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1efe00640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1d00047d0 0x7fd1d0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:45.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 -- 172.21.15.143:0/3401280113 reap_dead start 2024-01-15T18:10:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f5800640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1d000b990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009fc0 tx=0x7fd1e801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d334cddb1311e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e80a5cb0 con 0x7fd1d0005190 2024-01-15T18:10:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e800f040 con 0x7fd1d0005190 2024-01-15T18:10:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1ef400640 1 -- 172.21.15.143:0/3401280113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8003be0 con 0x7fd1d0005190 2024-01-15T18:10:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f7800640 1 -- 172.21.15.143:0/3401280113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 msgr2=0x7fd1d000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.073+0000 7fd1f7800640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1d000b990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009fc0 tx=0x7fd1e801cca0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- 172.21.15.143:0/3401280113 shutdown_connections 2024-01-15T18:10:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 --2- 172.21.15.143:0/3401280113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1d000b990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- 172.21.15.143:0/3401280113 >> 172.21.15.143:0/3401280113 conn(0x7fd1f00d0e20 msgr2=0x7fd1f00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- 172.21.15.143:0/3401280113 shutdown_connections 2024-01-15T18:10:45.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- 172.21.15.143:0/3401280113 wait complete. 2024-01-15T18:10:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 Processor -- start 2024-01-15T18:10:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- start start 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1f01f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1f01f0300 0x7fd1f00c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f0153c90 0x7fd1f0154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f0154680 con 0x7fd1d0005190 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f00c68c0 con 0x7fd1f0153c90 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f7800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f01f0780 con 0x7fd1f01f0300 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1f01f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1f01f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42976/0 (socket says 172.21.15.143:42976) 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 -- 172.21.15.143:0/317212071 learned_addr learned my addr 172.21.15.143:0/317212071 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f5800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f0153c90 0x7fd1f0154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1efe00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1f01f0300 0x7fd1f00c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 -- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1f01f0300 msgr2=0x7fd1f00c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1f01f0300 0x7fd1f00c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 -- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f0153c90 msgr2=0x7fd1f0154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f0153c90 0x7fd1f0154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.077+0000 7fd1f4e00640 1 -- 172.21.15.143:0/317212071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f00c5cb0 con 0x7fd1d0005190 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1f4e00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1f01f2d20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007290 tx=0x7fd1e0007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000f960 con 0x7fd1d0005190 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e000fae0 con 0x7fd1d0005190 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000fe90 con 0x7fd1d0005190 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f00c5f90 con 0x7fd1d0005190 2024-01-15T18:10:45.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f00c6550 con 0x7fd1d0005190 2024-01-15T18:10:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f006d4b0 con 0x7fd1d0005190 2024-01-15T18:10:45.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.081+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7fd1e001d460 con 0x7fd1d0005190 2024-01-15T18:10:45.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.085+0000 7fd1ecc00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1c8067190 0x7fd1c8069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.085+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd1e0082e90 con 0x7fd1d0005190 2024-01-15T18:10:45.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.085+0000 7fd1efe00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1c8067190 0x7fd1c8069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.085+0000 7fd1efe00640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1c8067190 0x7fd1c8069650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00cb090 tx=0x7fd1d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:45.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.089+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1e0051a00 con 0x7fd1d0005190 2024-01-15T18:10:45.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.313+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42251", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1f00cba30 con 0x7fd1c8067190 2024-01-15T18:10:45.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1ecc00640 1 -- 172.21.15.143:0/317212071 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fd1d8001400 con 0x7fd1c8067190 2024-01-15T18:10:45.364 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:45.365 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:45.365 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:10:45.365 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:45.365 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1c8067190 msgr2=0x7fd1c8069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1c8067190 0x7fd1c8069650 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00cb090 tx=0x7fd1d8009040 comp rx=0 tx=0).stop 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 msgr2=0x7fd1f01f2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1f01f2d20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0007290 tx=0x7fd1e0007750 comp rx=0 tx=0).stop 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 shutdown_connections 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1c8067190 0x7fd1c8069650 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1f0153c90 0x7fd1f0154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1f01f0300 0x7fd1f00c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 --2- 172.21.15.143:0/317212071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1d0005190 0x7fd1f01f2d20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 >> 172.21.15.143:0/317212071 conn(0x7fd1f00d0e20 msgr2=0x7fd1f00ce140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:45.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 shutdown_connections 2024-01-15T18:10:45.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.361+0000 7fd1f7800640 1 -- 172.21.15.143:0/317212071 wait complete. 2024-01-15T18:10:45.421 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:10:45.421 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42252 2024-01-15T18:10:45.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 Processor -- start 2024-01-15T18:10:45.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 -- start start 2024-01-15T18:10:45.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2c800d19f0 0x7f2c800cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c800c5220 0x7f2c800c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c800c4d90 con 0x7f2c800c5220 2024-01-15T18:10:45.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8015b640 con 0x7f2c800d19f0 2024-01-15T18:10:45.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c87000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800cf800 con 0x7f2c8015b260 2024-01-15T18:10:45.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.689+0000 7f2c85000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c800c5220 0x7f2c800c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2c800c5220 0x7f2c800c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42986/0 (socket says 172.21.15.143:42986) 2024-01-15T18:10:45.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 -- 172.21.15.143:0/1580175455 learned_addr learned my addr 172.21.15.143:0/1580175455 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:45.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 -- 172.21.15.143:0/1580175455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 msgr2=0x7f2c800cd810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:10:45.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 --2- 172.21.15.143:0/1580175455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800cd810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 -- 172.21.15.143:0/1580175455 >> v2:172.21.15.204:3300/0 conn(0x7f2c800d19f0 msgr2=0x7f2c800cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:45.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 --2- 172.21.15.143:0/1580175455 >> v2:172.21.15.204:3300/0 conn(0x7f2c800d19f0 0x7f2c800cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.693+0000 7f2c85000640 1 -- 172.21.15.143:0/1580175455 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800cfae0 con 0x7f2c800c5220 2024-01-15T18:10:45.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.697+0000 7f2c85000640 1 --2- 172.21.15.143:0/1580175455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800c5220 0x7f2c800c56b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2c80071670 tx=0x7f2c7801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d152dac9eb7332 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.697+0000 7f2c7ea00640 1 -- 172.21.15.143:0/1580175455 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780a5840 con 0x7f2c800c5220 2024-01-15T18:10:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.697+0000 7f2c7ea00640 1 -- 172.21.15.143:0/1580175455 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c780a59a0 con 0x7f2c800c5220 2024-01-15T18:10:45.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.697+0000 7f2c7ea00640 1 -- 172.21.15.143:0/1580175455 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780a5d30 con 0x7f2c800c5220 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.701+0000 7f2c87000640 1 -- 172.21.15.143:0/1580175455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800c5220 msgr2=0x7f2c800c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.701+0000 7f2c87000640 1 --2- 172.21.15.143:0/1580175455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800c5220 0x7f2c800c56b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2c80071670 tx=0x7f2c7801c9e0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.705+0000 7f2c87000640 1 -- 172.21.15.143:0/1580175455 shutdown_connections 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.705+0000 7f2c87000640 1 --2- 172.21.15.143:0/1580175455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800c5220 0x7f2c800c56b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.705+0000 7f2c87000640 1 --2- 172.21.15.143:0/1580175455 >> v2:172.21.15.204:3300/0 conn(0x7f2c800d19f0 0x7f2c800cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.705+0000 7f2c87000640 1 --2- 172.21.15.143:0/1580175455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.705+0000 7f2c87000640 1 -- 172.21.15.143:0/1580175455 >> 172.21.15.143:0/1580175455 conn(0x7f2c800d0ed0 msgr2=0x7f2c800d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:45.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.713+0000 7f2c87000640 1 -- 172.21.15.143:0/1580175455 shutdown_connections 2024-01-15T18:10:45.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.713+0000 7f2c87000640 1 -- 172.21.15.143:0/1580175455 wait complete. 2024-01-15T18:10:45.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.721+0000 7f2c87000640 1 Processor -- start 2024-01-15T18:10:45.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.721+0000 7f2c87000640 1 -- start start 2024-01-15T18:10:45.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.721+0000 7f2c87000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c800c5220 0x7f2c800ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.721+0000 7f2c87000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 0x7f2c80155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c87000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 0x7f2c80155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 0x7f2c80155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43002/0 (socket says 172.21.15.143:43002) 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 -- 172.21.15.143:0/2146432096 learned_addr learned my addr 172.21.15.143:0/2146432096 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801548c0 con 0x7f2c800d19f0 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80155c80 con 0x7f2c800c5220 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80153cd0 con 0x7f2c8015b260 2024-01-15T18:10:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c85000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7fe00640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c800c5220 0x7f2c800ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 -- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 msgr2=0x7f2c800b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800b64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 -- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c800c5220 msgr2=0x7f2c800ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c800c5220 0x7f2c800ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009b30 con 0x7f2c800d19f0 2024-01-15T18:10:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c85000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:45.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c7f400640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 0x7f2c80155740 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b550 tx=0x7f2c6800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:45.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68010870 con 0x7f2c800d19f0 2024-01-15T18:10:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c680109f0 con 0x7f2c800d19f0 2024-01-15T18:10:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68010da0 con 0x7f2c800d19f0 2024-01-15T18:10:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.725+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c80153f90 con 0x7f2c800d19f0 2024-01-15T18:10:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.729+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c80154550 con 0x7f2c800d19f0 2024-01-15T18:10:45.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.729+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c80154710 con 0x7f2c800d19f0 2024-01-15T18:10:45.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.737+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f2c68023480 con 0x7f2c800d19f0 2024-01-15T18:10:45.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.737+0000 7f2c77e00640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c58067230 0x7f2c580696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:45.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.737+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2c680833a0 con 0x7f2c800d19f0 2024-01-15T18:10:45.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.741+0000 7f2c7fe00640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c58067230 0x7f2c580696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.741+0000 7f2c7fe00640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c58067230 0x7f2c580696f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000a790 tx=0x7f2c70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:45.745+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2c68051dd0 con 0x7f2c800d19f0 2024-01-15T18:10:46.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.129+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42252", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c80067500 con 0x7f2c58067230 2024-01-15T18:10:46.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.589+0000 7f2c77e00640 1 -- 172.21.15.143:0/2146432096 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f2c80067500 con 0x7f2c58067230 2024-01-15T18:10:46.594 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:46.594 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:46.594 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:10:46.594 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:46.595 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:46.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c58067230 msgr2=0x7f2c580696f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:46.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c58067230 0x7f2c580696f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000a790 tx=0x7f2c70008040 comp rx=0 tx=0).stop 2024-01-15T18:10:46.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 msgr2=0x7f2c80155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:46.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 0x7f2c80155740 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b550 tx=0x7f2c6800ba10 comp rx=0 tx=0).stop 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 shutdown_connections 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2c58067230 0x7f2c580696f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2c8015b260 0x7f2c800b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c800d19f0 0x7f2c80155740 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 --2- 172.21.15.143:0/2146432096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2c800c5220 0x7f2c800ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 >> 172.21.15.143:0/2146432096 conn(0x7f2c800d0ed0 msgr2=0x7f2c80159480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 shutdown_connections 2024-01-15T18:10:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.593+0000 7f2c87000640 1 -- 172.21.15.143:0/2146432096 wait complete. 2024-01-15T18:10:46.656 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:10:46.656 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42253 2024-01-15T18:10:46.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.965+0000 7f6a07600640 1 Processor -- start 2024-01-15T18:10:46.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.965+0000 7f6a07600640 1 -- start start 2024-01-15T18:10:46.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.965+0000 7f6a07600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a00159170 0x7f6a0006d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a04c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a00159170 0x7f6a0006d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a04c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6a00159170 0x7f6a0006d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43014/0 (socket says 172.21.15.143:43014) 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a04c00640 1 -- 172.21.15.143:0/2597992756 learned_addr learned my addr 172.21.15.143:0/2597992756 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a0006cf30 con 0x7f6a000ce310 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a000cb2f0 con 0x7f6a000b1710 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a000cb0e0 con 0x7f6a00159170 2024-01-15T18:10:46.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f69ff400640 1 -- 172.21.15.143:0/2597992756 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964087011 0 0) 0x7f6a000cb2f0 con 0x7f6a000b1710 2024-01-15T18:10:46.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f69ff400640 1 -- 172.21.15.143:0/2597992756 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e0005740 con 0x7f6a000b1710 2024-01-15T18:10:46.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a04c00640 1 -- 172.21.15.143:0/2597992756 >> v1:172.21.15.204:6789/0 conn(0x7f6a000b1710 legacy=0x7f6a00070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:46.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a04c00640 1 -- 172.21.15.143:0/2597992756 >> v1:172.21.15.143:6789/0 conn(0x7f6a000ce310 legacy=0x7f6a0015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:46.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.969+0000 7f6a04c00640 1 -- 172.21.15.143:0/2597992756 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a000cb5d0 con 0x7f6a00159170 2024-01-15T18:10:46.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.977+0000 7f6a04c00640 1 --2- 172.21.15.143:0/2597992756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a00159170 0x7f6a0006d680 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009870 tx=0x7f69f801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1464e8598870b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:46.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.977+0000 7f69ff400640 1 -- 172.21.15.143:0/2597992756 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f80ab620 con 0x7f6a00159170 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.977+0000 7f69ff400640 1 -- 172.21.15.143:0/2597992756 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f80ab780 con 0x7f6a00159170 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.977+0000 7f69ff400640 1 -- 172.21.15.143:0/2597992756 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f80abb10 con 0x7f6a00159170 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.977+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a00159170 msgr2=0x7f6a0006d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.977+0000 7f6a07600640 1 --2- 172.21.15.143:0/2597992756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a00159170 0x7f6a0006d680 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009870 tx=0x7f69f801c400 comp rx=0 tx=0).stop 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.981+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 shutdown_connections 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.981+0000 7f6a07600640 1 --2- 172.21.15.143:0/2597992756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a00159170 0x7f6a0006d680 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.981+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 >> 172.21.15.143:0/2597992756 conn(0x7f6a000b0bf0 msgr2=0x7f6a000b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:46.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.981+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 shutdown_connections 2024-01-15T18:10:46.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.981+0000 7f6a07600640 1 -- 172.21.15.143:0/2597992756 wait complete. 2024-01-15T18:10:46.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.985+0000 7f6a07600640 1 Processor -- start 2024-01-15T18:10:46.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.985+0000 7f6a07600640 1 -- start start 2024-01-15T18:10:46.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a07600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a000b1710 0x7f6a00140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:46.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a07600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a000ce310 0x7f6a0013dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:46.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a07600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 0x7f6a001f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:46.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a07600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001f6f40 con 0x7f6a000ce310 2024-01-15T18:10:46.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a07600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a00141880 con 0x7f6a000b1710 2024-01-15T18:10:46.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a07600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a001f66a0 con 0x7f6a00159170 2024-01-15T18:10:46.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a05600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 0x7f6a001f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:46.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a05600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 0x7f6a001f94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:48554/0 (socket says 172.21.15.143:48554) 2024-01-15T18:10:46.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.989+0000 7f6a05600640 1 -- 172.21.15.143:0/410011286 learned_addr learned my addr 172.21.15.143:0/410011286 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:46.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.993+0000 7f6a05600640 1 -- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a000b1710 msgr2=0x7f6a00140fa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:10:46.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.993+0000 7f6a05600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a000b1710 0x7f6a00140fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.993+0000 7f6a05600640 1 -- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a000ce310 msgr2=0x7f6a0013dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:46.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.993+0000 7f6a05600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a000ce310 0x7f6a0013dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:46.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.993+0000 7f6a05600640 1 -- 172.21.15.143:0/410011286 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80094d0 con 0x7f6a00159170 2024-01-15T18:10:46.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.993+0000 7f6a05600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 0x7f6a001f94d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69f000c7b0 tx=0x7f69f000cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:47.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.997+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f0004b70 con 0x7f6a00159170 2024-01-15T18:10:47.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.997+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0004cf0 con 0x7f6a00159170 2024-01-15T18:10:47.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:46.997+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f000aa90 con 0x7f6a00159170 2024-01-15T18:10:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.001+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a001f6980 con 0x7f6a00159170 2024-01-15T18:10:47.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.001+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0013d6e0 con 0x7f6a00159170 2024-01-15T18:10:47.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.001+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a0013d540 con 0x7f6a00159170 2024-01-15T18:10:47.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.005+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f69f000abf0 con 0x7f6a00159170 2024-01-15T18:10:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.005+0000 7f69fcc00640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69d80671b0 0x7f69d8069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.005+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f69f00830e0 con 0x7f6a00159170 2024-01-15T18:10:47.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.009+0000 7f6a04c00640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69d80671b0 0x7f69d8069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.009+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f69f0051e60 con 0x7f6a00159170 2024-01-15T18:10:47.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.009+0000 7f6a04c00640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69d80671b0 0x7f69d8069670 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69f80af000 tx=0x7f69f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:47.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.233+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42253", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a00159550 con 0x7f69d80671b0 2024-01-15T18:10:47.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f69fcc00640 1 -- 172.21.15.143:0/410011286 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f69f80ba0c0 con 0x7f69d80671b0 2024-01-15T18:10:47.284 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:47.284 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:47.284 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:10:47.284 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:47.284 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69d80671b0 msgr2=0x7f69d8069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:47.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69d80671b0 0x7f69d8069670 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f69f80af000 tx=0x7f69f8008040 comp rx=0 tx=0).stop 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 msgr2=0x7f6a001f94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 0x7f6a001f94d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69f000c7b0 tx=0x7f69f000cc70 comp rx=0 tx=0).stop 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 shutdown_connections 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f69d80671b0 0x7f69d8069670 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6a00159170 0x7f6a001f94d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6a000ce310 0x7f6a0013dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 --2- 172.21.15.143:0/410011286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6a000b1710 0x7f6a00140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.281+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 >> 172.21.15.143:0/410011286 conn(0x7f6a000b0bf0 msgr2=0x7f6a000cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.285+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 shutdown_connections 2024-01-15T18:10:47.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.285+0000 7f6a07600640 1 -- 172.21.15.143:0/410011286 wait complete. 2024-01-15T18:10:47.312 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:10:47.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42254 2024-01-15T18:10:47.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d33a00640 1 Processor -- start 2024-01-15T18:10:47.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d33a00640 1 -- start start 2024-01-15T18:10:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d33a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 0x7f3d2c0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d33a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c162080 con 0x7f3d2c14b1e0 2024-01-15T18:10:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d33a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c161d70 con 0x7f3d2c070e00 2024-01-15T18:10:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d33a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c14b5c0 con 0x7f3d2c0cf990 2024-01-15T18:10:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d31000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3d2c14b1e0 0x7f3d2c0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42098/0 (socket says 172.21.15.143:42098) 2024-01-15T18:10:47.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d31000640 1 -- 172.21.15.143:0/69828089 learned_addr learned my addr 172.21.15.143:0/69828089 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224267592 0 0) 0x7f3d2c162080 con 0x7f3d2c14b1e0 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c003530 con 0x7f3d2c14b1e0 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.593+0000 7f3d2be00640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 0x7f3d2c0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3039087564 0 0) 0x7f3d0c003530 con 0x7f3d2c14b1e0 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 msgr2=0x7f3d2c0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 0x7f3d2c0708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 >> v1:172.21.15.204:6789/0 conn(0x7f3d2c070e00 legacy=0x7f3d2c162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:47.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c071580 con 0x7f3d2c14b1e0 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3d1c001680 con 0x7f3d2c14b1e0 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 >> v1:172.21.15.143:6789/0 conn(0x7f3d2c14b1e0 legacy=0x7f3d2c0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d0c0039b0 0x7f3d0c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d0c0047d0 0x7f3d0c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 0x7f3d0c0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c14b5c0 con 0x7f3d0c0039b0 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00baa0 con 0x7f3d0c0047d0 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00bc60 con 0x7f3d2c0cf990 2024-01-15T18:10:47.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31000640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d0c0039b0 0x7f3d0c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d2be00640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d0c0047d0 0x7f3d0c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31a00640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 0x7f3d0c0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31000640 1 -- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 msgr2=0x7f3d0c0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31000640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c0cf990 0x7f3d0c0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31000640 1 -- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d0c0047d0 msgr2=0x7f3d0c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31000640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d0c0047d0 0x7f3d0c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.597+0000 7f3d31000640 1 -- 172.21.15.143:0/69828089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c003530 con 0x7f3d0c0039b0 2024-01-15T18:10:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.601+0000 7f3d31000640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d0c0039b0 0x7f3d0c003db0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c210 tx=0x7f3d1c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb5b3d2878e87b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.601+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c005ae0 con 0x7f3d0c0039b0 2024-01-15T18:10:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.601+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c005c60 con 0x7f3d0c0039b0 2024-01-15T18:10:47.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.601+0000 7f3d2b400640 1 -- 172.21.15.143:0/69828089 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0a5820 con 0x7f3d0c0039b0 2024-01-15T18:10:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d33a00640 1 -- 172.21.15.143:0/69828089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d0c0039b0 msgr2=0x7f3d0c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:47.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d33a00640 1 --2- 172.21.15.143:0/69828089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d0c0039b0 0x7f3d0c003db0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c210 tx=0x7f3d1c01cc30 comp rx=0 tx=0).stop 2024-01-15T18:10:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d31a00640 1 -- 172.21.15.143:0/69828089 reap_dead start 2024-01-15T18:10:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d33a00640 1 -- 172.21.15.143:0/69828089 shutdown_connections 2024-01-15T18:10:47.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d33a00640 1 -- 172.21.15.143:0/69828089 >> 172.21.15.143:0/69828089 conn(0x7f3d2c06a890 msgr2=0x7f3d2c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:47.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d33a00640 1 -- 172.21.15.143:0/69828089 shutdown_connections 2024-01-15T18:10:47.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.605+0000 7f3d33a00640 1 -- 172.21.15.143:0/69828089 wait complete. 2024-01-15T18:10:47.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.609+0000 7f3d33a00640 1 Processor -- start 2024-01-15T18:10:47.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.613+0000 7f3d33a00640 1 -- start start 2024-01-15T18:10:47.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d33a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d2c163bd0 0x7f3d2c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d33a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 0x7f3d2c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d33a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 0x7f3d2c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d33a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c15d750 con 0x7f3d2c163bd0 2024-01-15T18:10:47.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d33a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0afbc0 con 0x7f3d2c15cd90 2024-01-15T18:10:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d33a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0afea0 con 0x7f3d2c163fb0 2024-01-15T18:10:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d2be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 0x7f3d2c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 0x7f3d2c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 0x7f3d2c15d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48422/0 (socket says 172.21.15.143:48422) 2024-01-15T18:10:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 -- 172.21.15.143:0/329588327 learned_addr learned my addr 172.21.15.143:0/329588327 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:47.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d2be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 0x7f3d2c0af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:34338/0 (socket says 172.21.15.143:34338) 2024-01-15T18:10:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 -- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 msgr2=0x7f3d2c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 0x7f3d2c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 -- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d2c163bd0 msgr2=0x7f3d2c15f0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:10:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d2c163bd0 0x7f3d2c15f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.617+0000 7f3d31a00640 1 -- 172.21.15.143:0/329588327 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c0b0180 con 0x7f3d2c15cd90 2024-01-15T18:10:47.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d2be00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 0x7f3d2c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:10:47.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d31a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 0x7f3d2c15d210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0cfe40 tx=0x7f3d2400e920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24018070 con 0x7f3d2c15cd90 2024-01-15T18:10:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d240135f0 con 0x7f3d2c15cd90 2024-01-15T18:10:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24013780 con 0x7f3d2c15cd90 2024-01-15T18:10:47.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c0b0460 con 0x7f3d2c15cd90 2024-01-15T18:10:47.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.621+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c0af8d0 con 0x7f3d2c15cd90 2024-01-15T18:10:47.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.625+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc005260 con 0x7f3d2c15cd90 2024-01-15T18:10:47.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.629+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f3d240138e0 con 0x7f3d2c15cd90 2024-01-15T18:10:47.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.629+0000 7f3d28c00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d04067260 0x7f3d04069720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:47.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.629+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3d24082f10 con 0x7f3d2c15cd90 2024-01-15T18:10:47.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.629+0000 7f3d31000640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d04067260 0x7f3d04069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:47.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.633+0000 7f3d31000640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d04067260 0x7f3d04069720 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0ab000 tx=0x7f3d1c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:47.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:47.633+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d240519d0 con 0x7f3d2c15cd90 2024-01-15T18:10:48.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.005+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42254", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cfc000bd0 con 0x7f3d04067260 2024-01-15T18:10:48.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.177+0000 7f3d28c00640 1 -- 172.21.15.143:0/329588327 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f3cfc000bd0 con 0x7f3d04067260 2024-01-15T18:10:48.180 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:48.181 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:48.181 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:10:48.181 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:48.181 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:48.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d04067260 msgr2=0x7f3d04069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:48.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d04067260 0x7f3d04069720 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0ab000 tx=0x7f3d1c0a7070 comp rx=0 tx=0).stop 2024-01-15T18:10:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 msgr2=0x7f3d2c15d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 0x7f3d2c15d210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0cfe40 tx=0x7f3d2400e920 comp rx=0 tx=0).stop 2024-01-15T18:10:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 shutdown_connections 2024-01-15T18:10:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d04067260 0x7f3d04069720 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d2c15cd90 0x7f3d2c15d210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d2c163fb0 0x7f3d2c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 --2- 172.21.15.143:0/329588327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d2c163bd0 0x7f3d2c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 >> 172.21.15.143:0/329588327 conn(0x7f3d2c06a890 msgr2=0x7f3d2c147de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:48.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.181+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 shutdown_connections 2024-01-15T18:10:48.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.189+0000 7f3d33a00640 1 -- 172.21.15.143:0/329588327 wait complete. 2024-01-15T18:10:48.275 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:10:48.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42255 2024-01-15T18:10:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.557+0000 7fcc59800640 1 Processor -- start 2024-01-15T18:10:48.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.557+0000 7fcc59800640 1 -- start start 2024-01-15T18:10:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc59800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc540ce690 0x7fcc540ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc59800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540b1da0 0x7fcc540b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc59800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcc540b3bf0 0x7fcc540b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc59800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5415dba0 con 0x7fcc540ce690 2024-01-15T18:10:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc59800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc54070bf0 con 0x7fcc540b3bf0 2024-01-15T18:10:48.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc59800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc540b2740 con 0x7fcc540b1da0 2024-01-15T18:10:48.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc52a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc540ce690 0x7fcc540ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:48.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc52a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc540ce690 0x7fcc540ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37348/0 (socket says 172.21.15.143:37348) 2024-01-15T18:10:48.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc52a00640 1 -- 172.21.15.143:0/1916697663 learned_addr learned my addr 172.21.15.143:0/1916697663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.561+0000 7fcc52a00640 1 -- 172.21.15.143:0/1916697663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540b1da0 msgr2=0x7fcc540b2200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:10:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.565+0000 7fcc52a00640 1 --2- 172.21.15.143:0/1916697663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540b1da0 0x7fcc540b2200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.565+0000 7fcc52a00640 1 -- 172.21.15.143:0/1916697663 >> v2:172.21.15.204:3300/0 conn(0x7fcc540b3bf0 msgr2=0x7fcc540b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.565+0000 7fcc52a00640 1 --2- 172.21.15.143:0/1916697663 >> v2:172.21.15.204:3300/0 conn(0x7fcc540b3bf0 0x7fcc540b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.565+0000 7fcc52a00640 1 -- 172.21.15.143:0/1916697663 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc540b4920 con 0x7fcc540ce690 2024-01-15T18:10:48.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc52a00640 1 --2- 172.21.15.143:0/1916697663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540ce690 0x7fcc540ac8e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc5415e870 tx=0x7fcc4401c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252badc3988bee70 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:48.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc51600640 1 -- 172.21.15.143:0/1916697663 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440a66f0 con 0x7fcc540ce690 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc51600640 1 -- 172.21.15.143:0/1916697663 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc440a6850 con 0x7fcc540ce690 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc51600640 1 -- 172.21.15.143:0/1916697663 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440a6be0 con 0x7fcc540ce690 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 -- 172.21.15.143:0/1916697663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540ce690 msgr2=0x7fcc540ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 --2- 172.21.15.143:0/1916697663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540ce690 0x7fcc540ac8e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcc5415e870 tx=0x7fcc4401c770 comp rx=0 tx=0).stop 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 -- 172.21.15.143:0/1916697663 shutdown_connections 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 --2- 172.21.15.143:0/1916697663 >> v2:172.21.15.204:3300/0 conn(0x7fcc540b3bf0 0x7fcc540b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 --2- 172.21.15.143:0/1916697663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540b1da0 0x7fcc540b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 --2- 172.21.15.143:0/1916697663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540ce690 0x7fcc540ac8e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 -- 172.21.15.143:0/1916697663 >> 172.21.15.143:0/1916697663 conn(0x7fcc5406c3f0 msgr2=0x7fcc5406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:48.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.573+0000 7fcc59800640 1 -- 172.21.15.143:0/1916697663 shutdown_connections 2024-01-15T18:10:48.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.577+0000 7fcc59800640 1 -- 172.21.15.143:0/1916697663 wait complete. 2024-01-15T18:10:48.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.577+0000 7fcc59800640 1 Processor -- start 2024-01-15T18:10:48.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.577+0000 7fcc59800640 1 -- start start 2024-01-15T18:10:48.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 0x7fcc54136f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc540b3bf0 0x7fcc5415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540ce690 0x7fcc5415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5415add0 con 0x7fcc540b1da0 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5413ed30 con 0x7fcc540b3bf0 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5413f010 con 0x7fcc540ce690 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 0x7fcc54136f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 0x7fcc54136f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37364/0 (socket says 172.21.15.143:37364) 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 -- 172.21.15.143:0/1482561668 learned_addr learned my addr 172.21.15.143:0/1482561668 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52000640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc540b3bf0 0x7fcc5415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc53400640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540ce690 0x7fcc5415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 -- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540ce690 msgr2=0x7fcc5415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540ce690 0x7fcc5415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 -- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc540b3bf0 msgr2=0x7fcc5415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc540b3bf0 0x7fcc5415a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 -- 172.21.15.143:0/1482561668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc440094d0 con 0x7fcc540b1da0 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc52a00640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 0x7fcc54136f00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcc440b0040 tx=0x7fcc44019510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440ae6a0 con 0x7fcc540b1da0 2024-01-15T18:10:48.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc440ae800 con 0x7fcc540b1da0 2024-01-15T18:10:48.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc440aebb0 con 0x7fcc540b1da0 2024-01-15T18:10:48.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5413f2f0 con 0x7fcc540b1da0 2024-01-15T18:10:48.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.581+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc54144620 con 0x7fcc540b1da0 2024-01-15T18:10:48.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.585+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc54067500 con 0x7fcc540b1da0 2024-01-15T18:10:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.589+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7fcc440aed10 con 0x7fcc540b1da0 2024-01-15T18:10:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.589+0000 7fcc4aa00640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc2c067160 0x7fcc2c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.589+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcc440aa070 con 0x7fcc540b1da0 2024-01-15T18:10:48.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.593+0000 7fcc52000640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc2c067160 0x7fcc2c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.593+0000 7fcc52000640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc2c067160 0x7fcc2c069620 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcc54161900 tx=0x7fcc3c0090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:48.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.601+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc440ed6e0 con 0x7fcc540b1da0 2024-01-15T18:10:48.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:48.985+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc5413eea0 con 0x7fcc2c067160 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.029+0000 7fcc4aa00640 1 -- 172.21.15.143:0/1482561668 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fcc5413eea0 con 0x7fcc2c067160 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:49.033 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:49.034 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:49.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc2c067160 msgr2=0x7fcc2c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:49.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc2c067160 0x7fcc2c069620 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcc54161900 tx=0x7fcc3c0090e0 comp rx=0 tx=0).stop 2024-01-15T18:10:49.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 msgr2=0x7fcc54136f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:49.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 0x7fcc54136f00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fcc440b0040 tx=0x7fcc44019510 comp rx=0 tx=0).stop 2024-01-15T18:10:49.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 shutdown_connections 2024-01-15T18:10:49.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc2c067160 0x7fcc2c069620 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:49.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc540ce690 0x7fcc5415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:49.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc540b3bf0 0x7fcc5415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:49.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 --2- 172.21.15.143:0/1482561668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc540b1da0 0x7fcc54136f00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:49.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 >> 172.21.15.143:0/1482561668 conn(0x7fcc5406c3f0 msgr2=0x7fcc5415edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:49.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 shutdown_connections 2024-01-15T18:10:49.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:49.037+0000 7fcc59800640 1 -- 172.21.15.143:0/1482561668 wait complete. 2024-01-15T18:10:49.100 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:10:50.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42255 2024-01-15T18:10:50.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.421+0000 7f33f8e00640 1 Processor -- start 2024-01-15T18:10:50.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.421+0000 7f33f8e00640 1 -- start start 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 0x7f33f406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f8e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f33f415b620 0x7f33f415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f8e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f406db40 con 0x7f33f4158a80 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f8e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f415c370 con 0x7f33f415b620 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f8e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40ce020 con 0x7f33f406d1a0 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 0x7f33f406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 0x7f33f406d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:34358/0 (socket says 172.21.15.143:34358) 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 -- 172.21.15.143:0/3258789415 learned_addr learned my addr 172.21.15.143:0/3258789415 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:50.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f3e00640 1 --2- 172.21.15.143:0/3258789415 >> v2:172.21.15.204:3300/0 conn(0x7f33f415b620 0x7f33f415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 -- 172.21.15.143:0/3258789415 >> v2:172.21.15.204:3300/0 conn(0x7f33f415b620 msgr2=0x7f33f415ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 --2- 172.21.15.143:0/3258789415 >> v2:172.21.15.204:3300/0 conn(0x7f33f415b620 0x7f33f415ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 -- 172.21.15.143:0/3258789415 >> v1:172.21.15.143:6789/0 conn(0x7f33f4158a80 legacy=0x7f33f40ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:50.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.425+0000 7f33f2a00640 1 -- 172.21.15.143:0/3258789415 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f40ce300 con 0x7f33f406d1a0 2024-01-15T18:10:50.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.429+0000 7f33f2a00640 1 --2- 172.21.15.143:0/3258789415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 0x7f33f406d600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009870 tx=0x7f33dc01c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3087939767b7e55f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:50.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f2000640 1 -- 172.21.15.143:0/3258789415 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0ad640 con 0x7f33f406d1a0 2024-01-15T18:10:50.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f2000640 1 -- 172.21.15.143:0/3258789415 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc0ad7a0 con 0x7f33f406d1a0 2024-01-15T18:10:50.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f2000640 1 -- 172.21.15.143:0/3258789415 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0adb30 con 0x7f33f406d1a0 2024-01-15T18:10:50.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 -- 172.21.15.143:0/3258789415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 msgr2=0x7f33f406d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:50.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 --2- 172.21.15.143:0/3258789415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 0x7f33f406d600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009870 tx=0x7f33dc01c3d0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 -- 172.21.15.143:0/3258789415 shutdown_connections 2024-01-15T18:10:50.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 --2- 172.21.15.143:0/3258789415 >> v2:172.21.15.204:3300/0 conn(0x7f33f415b620 0x7f33f415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 --2- 172.21.15.143:0/3258789415 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f406d1a0 0x7f33f406d600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 -- 172.21.15.143:0/3258789415 >> 172.21.15.143:0/3258789415 conn(0x7f33f40cf740 msgr2=0x7f33f40cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:50.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 -- 172.21.15.143:0/3258789415 shutdown_connections 2024-01-15T18:10:50.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.433+0000 7f33f8e00640 1 -- 172.21.15.143:0/3258789415 wait complete. 2024-01-15T18:10:50.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 Processor -- start 2024-01-15T18:10:50.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 -- start start 2024-01-15T18:10:50.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33f406d1a0 0x7f33f4149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:50.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f4158a80 0x7f33f4151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:50.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 0x7f33f4153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:50.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4139870 con 0x7f33f415b620 2024-01-15T18:10:50.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4138bf0 con 0x7f33f406d1a0 2024-01-15T18:10:50.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f8e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4138ed0 con 0x7f33f4158a80 2024-01-15T18:10:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 0x7f33f4153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 0x7f33f4153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37376/0 (socket says 172.21.15.143:37376) 2024-01-15T18:10:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f3e00640 1 -- 172.21.15.143:0/2905989719 learned_addr learned my addr 172.21.15.143:0/2905989719 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f2a00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f4158a80 0x7f33f4151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:50.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.441+0000 7f33f3400640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33f406d1a0 0x7f33f4149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:50.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3400640 1 -- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f4158a80 msgr2=0x7f33f4151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:50.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3400640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f4158a80 0x7f33f4151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3400640 1 -- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 msgr2=0x7f33f4153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3400640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 0x7f33f4153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3400640 1 -- 172.21.15.143:0/2905989719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0094d0 con 0x7f33f406d1a0 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 0x7f33f4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f3400640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33f406d1a0 0x7f33f4149840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c360 tx=0x7f33e400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4003600 con 0x7f33f406d1a0 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4003760 con 0x7f33f406d1a0 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f41391b0 con 0x7f33f406d1a0 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4003b50 con 0x7f33f406d1a0 2024-01-15T18:10:50.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f4149d80 con 0x7f33f406d1a0 2024-01-15T18:10:50.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.445+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f4067c10 con 0x7f33f406d1a0 2024-01-15T18:10:50.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.449+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f33e4003cb0 con 0x7f33f406d1a0 2024-01-15T18:10:50.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.449+0000 7f33eb400640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33cc067160 0x7f33cc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:50.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.449+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f33e4082010 con 0x7f33f406d1a0 2024-01-15T18:10:50.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.449+0000 7f33f2a00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33cc067160 0x7f33cc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:50.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.453+0000 7f33f2a00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33cc067160 0x7f33cc069620 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0af000 tx=0x7f33dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:50.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.453+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f33e4050bc0 con 0x7f33f406d1a0 2024-01-15T18:10:50.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.837+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7f33f406aca0 con 0x7f33cc067160 2024-01-15T18:10:50.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.893+0000 7f33eb400640 1 -- 172.21.15.143:0/2905989719 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f33dc0bc0c0 con 0x7f33cc067160 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:50.897 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:50.898 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:50.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33cc067160 msgr2=0x7f33cc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:50.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33cc067160 0x7f33cc069620 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0af000 tx=0x7f33dc009040 comp rx=0 tx=0).stop 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33f406d1a0 msgr2=0x7f33f4149840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33f406d1a0 0x7f33f4149840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f33e400c360 tx=0x7f33e400c820 comp rx=0 tx=0).stop 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 shutdown_connections 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f33cc067160 0x7f33cc069620 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f33f415b620 0x7f33f4153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f33f4158a80 0x7f33f4151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 --2- 172.21.15.143:0/2905989719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f33f406d1a0 0x7f33f4149840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:50.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 >> 172.21.15.143:0/2905989719 conn(0x7f33f40cf740 msgr2=0x7f33f40cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:50.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 shutdown_connections 2024-01-15T18:10:50.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:50.897+0000 7f33f8e00640 1 -- 172.21.15.143:0/2905989719 wait complete. 2024-01-15T18:10:50.972 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:10:51.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42255 2024-01-15T18:10:52.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.333+0000 7f46d3200640 1 Processor -- start 2024-01-15T18:10:52.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.333+0000 7f46d3200640 1 -- start start 2024-01-15T18:10:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46d3200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f46cc0cf280 0x7f46cc0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46d3200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc0cc730 con 0x7f46cc0cd1b0 2024-01-15T18:10:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46d3200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46cc070ee0 con 0x7f46cc143ac0 2024-01-15T18:10:52.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46d3200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46cc143ea0 con 0x7f46cc0cf280 2024-01-15T18:10:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46d1200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f46cc0cd1b0 0x7f46cc0cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42110/0 (socket says 172.21.15.143:42110) 2024-01-15T18:10:52.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46d1200640 1 -- 172.21.15.143:0/2030835662 learned_addr learned my addr 172.21.15.143:0/2030835662 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:52.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048644531 0 0) 0x7f46cc0cc730 con 0x7f46cc0cd1b0 2024-01-15T18:10:52.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac003530 con 0x7f46cc0cd1b0 2024-01-15T18:10:52.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2129627749 0 0) 0x7f46ac003530 con 0x7f46cc0cd1b0 2024-01-15T18:10:52.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 >> v2:172.21.15.204:3300/0 conn(0x7f46cc0cf280 msgr2=0x7f46cc0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:52.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.337+0000 7f46cb400640 1 --2- 172.21.15.143:0/2030835662 >> v2:172.21.15.204:3300/0 conn(0x7f46cc0cf280 0x7f46cc0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 --2- 172.21.15.143:0/2030835662 >> v2:172.21.15.204:3300/0 conn(0x7f46cc0cf280 0x7f46cc0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 >> v1:172.21.15.204:6789/0 conn(0x7f46cc143ac0 legacy=0x7f46cc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0711c0 con 0x7f46cc0cd1b0 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f46c4001a00 con 0x7f46cc0cd1b0 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 >> v1:172.21.15.143:6789/0 conn(0x7f46cc0cd1b0 legacy=0x7f46cc0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0039d0 0x7f46ac003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46ac0047d0 0x7f46ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46ac005390 0x7f46ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00c1d0 con 0x7f46ac005390 2024-01-15T18:10:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00c340 con 0x7f46ac0039d0 2024-01-15T18:10:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00c500 con 0x7f46ac0047d0 2024-01-15T18:10:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 --2- 172.21.15.143:0/2030835662 >> v2:172.21.15.204:3300/0 conn(0x7f46cc0cf280 0x7f46cc0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cbe00640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0039d0 0x7f46ac003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46ac0047d0 0x7f46ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46d1200640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46ac005390 0x7f46ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 -- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0039d0 msgr2=0x7f46ac003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0039d0 0x7f46ac003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 -- 172.21.15.143:0/2030835662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46ac005390 msgr2=0x7f46ac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46ac005390 0x7f46ac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46cb400640 1 -- 172.21.15.143:0/2030835662 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac003530 con 0x7f46ac0047d0 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46d1200640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46ac005390 0x7f46ac00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:52.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.341+0000 7f46d1200640 1 -- 172.21.15.143:0/2030835662 reap_dead start 2024-01-15T18:10:52.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.345+0000 7f46cb400640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46ac0047d0 0x7f46ac004c50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ccb0 tx=0x7f46bc01e8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35af005291c109b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.349+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0a78d0 con 0x7f46ac0047d0 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.349+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc0a7a30 con 0x7f46ac0047d0 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.349+0000 7f46caa00640 1 -- 172.21.15.143:0/2030835662 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0a7e20 con 0x7f46ac0047d0 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.349+0000 7f46d3200640 1 -- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46ac0047d0 msgr2=0x7f46ac004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.349+0000 7f46d3200640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46ac0047d0 0x7f46ac004c50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ccb0 tx=0x7f46bc01e8c0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.353+0000 7f46d3200640 1 -- 172.21.15.143:0/2030835662 shutdown_connections 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.353+0000 7f46d3200640 1 --2- 172.21.15.143:0/2030835662 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46ac0047d0 0x7f46ac004c50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.353+0000 7f46d3200640 1 -- 172.21.15.143:0/2030835662 >> 172.21.15.143:0/2030835662 conn(0x7f46cc06a890 msgr2=0x7f46cc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:52.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.357+0000 7f46d3200640 1 -- 172.21.15.143:0/2030835662 shutdown_connections 2024-01-15T18:10:52.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.357+0000 7f46d3200640 1 -- 172.21.15.143:0/2030835662 wait complete. 2024-01-15T18:10:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.361+0000 7f46d3200640 1 Processor -- start 2024-01-15T18:10:52.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.361+0000 7f46d3200640 1 -- start start 2024-01-15T18:10:52.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0047d0 0x7f46cc13adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46cc13b2f0 0x7f46cc15aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 0x7f46cc1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc13b770 con 0x7f46cc15b1e0 2024-01-15T18:10:52.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc15b9d0 con 0x7f46ac0047d0 2024-01-15T18:10:52.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc15bcb0 con 0x7f46cc13b2f0 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 0x7f46cc1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 0x7f46cc1f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37402/0 (socket says 172.21.15.143:37402) 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 -- 172.21.15.143:0/2479528377 learned_addr learned my addr 172.21.15.143:0/2479528377 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46cbe00640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0047d0 0x7f46cc13adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46cb400640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46cc13b2f0 0x7f46cc15aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 -- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46cc13b2f0 msgr2=0x7f46cc15aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46cc13b2f0 0x7f46cc15aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 -- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0047d0 msgr2=0x7f46cc13adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0047d0 0x7f46cc13adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 -- 172.21.15.143:0/2479528377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc15bf90 con 0x7f46cc15b1e0 2024-01-15T18:10:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46cbe00640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0047d0 0x7f46cc13adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:10:52.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.365+0000 7f46d1200640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 0x7f46cc1f4760 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001f50 tx=0x7f46c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:52.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.369+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c400eb90 con 0x7f46cc15b1e0 2024-01-15T18:10:52.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.369+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c400ecf0 con 0x7f46cc15b1e0 2024-01-15T18:10:52.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.369+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c4013640 con 0x7f46cc15b1e0 2024-01-15T18:10:52.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.369+0000 7f46d3200640 1 -- 172.21.15.143:0/2479528377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc15c270 con 0x7f46cc15b1e0 2024-01-15T18:10:52.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.369+0000 7f46d3200640 1 -- 172.21.15.143:0/2479528377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc15b660 con 0x7f46cc15b1e0 2024-01-15T18:10:52.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.373+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc0cb840 con 0x7f46cc15b1e0 2024-01-15T18:10:52.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.377+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f46c40137a0 con 0x7f46cc15b1e0 2024-01-15T18:10:52.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.377+0000 7f46c3e00640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f46a4067260 0x7f46a4069720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:52.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.377+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f46c4083da0 con 0x7f46cc15b1e0 2024-01-15T18:10:52.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.381+0000 7f46cbe00640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f46a4067260 0x7f46a4069720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.389+0000 7f46cbe00640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f46a4067260 0x7f46a4069720 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46cc1377c0 tx=0x7f46b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.397+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46c4052860 con 0x7f46cc15b1e0 2024-01-15T18:10:52.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.873+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7f46cc071c30 con 0x7f46a4067260 2024-01-15T18:10:52.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.917+0000 7f46c3e00640 1 -- 172.21.15.143:0/2479528377 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f46b4012df0 con 0x7f46a4067260 2024-01-15T18:10:52.920 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:52.920 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:52.920 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:52.920 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:52.920 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:52.920 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:52.921 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:52.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.929+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f46a4067260 msgr2=0x7f46a4069720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.929+0000 7f46c1600640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f46a4067260 0x7f46a4069720 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46cc1377c0 tx=0x7f46b4009040 comp rx=0 tx=0).stop 2024-01-15T18:10:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 msgr2=0x7f46cc1f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:52.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 0x7f46cc1f4760 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f46c4001f50 tx=0x7f46c400e530 comp rx=0 tx=0).stop 2024-01-15T18:10:52.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 shutdown_connections 2024-01-15T18:10:52.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f46a4067260 0x7f46a4069720 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46cc15b1e0 0x7f46cc1f4760 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46cc13b2f0 0x7f46cc15aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 --2- 172.21.15.143:0/2479528377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46ac0047d0 0x7f46cc13adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:52.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 >> 172.21.15.143:0/2479528377 conn(0x7f46cc06a890 msgr2=0x7f46cc0adba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:52.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 shutdown_connections 2024-01-15T18:10:52.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:52.933+0000 7f46c1600640 1 -- 172.21.15.143:0/2479528377 wait complete. 2024-01-15T18:10:53.030 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:10:54.031 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42255 2024-01-15T18:10:54.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.481+0000 7feb66c00640 1 Processor -- start 2024-01-15T18:10:54.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.481+0000 7feb66c00640 1 -- start start 2024-01-15T18:10:54.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feb60153550 0x7feb6006cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:54.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb600cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:54.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb60159c00 con 0x7feb60159690 2024-01-15T18:10:54.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb600b1e10 con 0x7feb60153550 2024-01-15T18:10:54.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb60159ee0 con 0x7feb60153930 2024-01-15T18:10:54.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb65800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7feb60153550 0x7feb6006cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:54.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7feb60159690 0x7feb600b38d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42120/0 (socket says 172.21.15.143:42120) 2024-01-15T18:10:54.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb66200640 1 -- 172.21.15.143:0/2670866288 learned_addr learned my addr 172.21.15.143:0/2670866288 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:54.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb64e00640 1 --2- 172.21.15.143:0/2670866288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb600cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:54.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb65800640 1 -- 172.21.15.143:0/2670866288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 msgr2=0x7feb600cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:54.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb65800640 1 --2- 172.21.15.143:0/2670866288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb600cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:54.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb65800640 1 -- 172.21.15.143:0/2670866288 >> v1:172.21.15.143:6789/0 conn(0x7feb60159690 legacy=0x7feb600b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:54.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.485+0000 7feb65800640 1 -- 172.21.15.143:0/2670866288 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6015a1c0 con 0x7feb60153550 2024-01-15T18:10:54.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.489+0000 7feb65800640 1 --2- 172.21.15.143:0/2670866288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb6006cda0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7feb480097c0 tx=0x7feb4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50659b9a3c002194 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:54.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb5fe00640 1 -- 172.21.15.143:0/2670866288 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480ab600 con 0x7feb60153550 2024-01-15T18:10:54.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb5fe00640 1 -- 172.21.15.143:0/2670866288 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb480a7050 con 0x7feb60153550 2024-01-15T18:10:54.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb5fe00640 1 -- 172.21.15.143:0/2670866288 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480ab980 con 0x7feb60153550 2024-01-15T18:10:54.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 -- 172.21.15.143:0/2670866288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 msgr2=0x7feb6006cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 --2- 172.21.15.143:0/2670866288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb6006cda0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7feb480097c0 tx=0x7feb4801c480 comp rx=0 tx=0).stop 2024-01-15T18:10:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 -- 172.21.15.143:0/2670866288 shutdown_connections 2024-01-15T18:10:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 --2- 172.21.15.143:0/2670866288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb600cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 --2- 172.21.15.143:0/2670866288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb6006cda0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:54.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 -- 172.21.15.143:0/2670866288 >> 172.21.15.143:0/2670866288 conn(0x7feb6006a890 msgr2=0x7feb6006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 -- 172.21.15.143:0/2670866288 shutdown_connections 2024-01-15T18:10:54.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.493+0000 7feb66c00640 1 -- 172.21.15.143:0/2670866288 wait complete. 2024-01-15T18:10:54.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 Processor -- start 2024-01-15T18:10:54.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 -- start start 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb601456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb60146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb60159690 0x7feb600cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60151050 con 0x7feb60153550 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb60145be0 con 0x7feb60159690 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb60151b60 con 0x7feb60153930 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb65800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb601456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:54.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb65800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb601456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37420/0 (socket says 172.21.15.143:37420) 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb65800640 1 -- 172.21.15.143:0/2279921376 learned_addr learned my addr 172.21.15.143:0/2279921376 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb64e00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb60146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.497+0000 7feb66200640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb60159690 0x7feb600cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb65800640 1 -- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 msgr2=0x7feb60146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb65800640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb60146080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb65800640 1 -- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb60159690 msgr2=0x7feb600cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb65800640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb60159690 0x7feb600cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:54.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb65800640 1 -- 172.21.15.143:0/2279921376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48009420 con 0x7feb60153550 2024-01-15T18:10:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb65800640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb601456a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7feb48019090 tx=0x7feb48019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480b4600 con 0x7feb60153550 2024-01-15T18:10:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb480b4760 con 0x7feb60153550 2024-01-15T18:10:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb480b4b10 con 0x7feb60153550 2024-01-15T18:10:54.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb60151230 con 0x7feb60153550 2024-01-15T18:10:54.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.501+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb601517f0 con 0x7feb60153550 2024-01-15T18:10:54.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.505+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb60072420 con 0x7feb60153550 2024-01-15T18:10:54.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.505+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7feb480b2040 con 0x7feb60153550 2024-01-15T18:10:54.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.505+0000 7feb5d600640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb380671b0 0x7feb38069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:54.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.505+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7feb4811cf30 con 0x7feb60153550 2024-01-15T18:10:54.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.517+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feb480eba30 con 0x7feb60153550 2024-01-15T18:10:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.557+0000 7feb64e00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb380671b0 0x7feb38069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:54.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.565+0000 7feb64e00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb380671b0 0x7feb38069670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb60159090 tx=0x7feb50015040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:54.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:54.989+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7feb6006e300 con 0x7feb380671b0 2024-01-15T18:10:55.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.017+0000 7feb5d600640 1 -- 172.21.15.143:0/2279921376 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7feb50004220 con 0x7feb380671b0 2024-01-15T18:10:55.022 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:55.022 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:55.022 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:55.022 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:55.022 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:55.022 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:55.023 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:55.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.021+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb380671b0 msgr2=0x7feb38069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.021+0000 7feb66c00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb380671b0 0x7feb38069670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb60159090 tx=0x7feb50015040 comp rx=0 tx=0).stop 2024-01-15T18:10:55.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 msgr2=0x7feb601456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:55.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb601456a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7feb48019090 tx=0x7feb48019220 comp rx=0 tx=0).stop 2024-01-15T18:10:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 shutdown_connections 2024-01-15T18:10:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb380671b0 0x7feb38069670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feb60159690 0x7feb600cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feb60153930 0x7feb60146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 --2- 172.21.15.143:0/2279921376 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb60153550 0x7feb601456a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:55.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 >> 172.21.15.143:0/2279921376 conn(0x7feb6006a890 msgr2=0x7feb600cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 shutdown_connections 2024-01-15T18:10:55.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:55.025+0000 7feb66c00640 1 -- 172.21.15.143:0/2279921376 wait complete. 2024-01-15T18:10:55.122 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:10:56.123 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42255 2024-01-15T18:10:56.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.429+0000 7f2db1800640 1 Processor -- start 2024-01-15T18:10:56.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.429+0000 7f2db1800640 1 -- start start 2024-01-15T18:10:56.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2db1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac0b4ac0 0x7f2dac164540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:56.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2db1800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac0b2a20 con 0x7f2dac157300 2024-01-15T18:10:56.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2db1800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac0b2d00 con 0x7f2dac159b70 2024-01-15T18:10:56.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2db1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac0b35a0 con 0x7f2dac0b4ac0 2024-01-15T18:10:56.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2dab400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac0b4ac0 0x7f2dac164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:56.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2daa000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f2dac159b70 0x7f2dac0b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:34534/0 (socket says 172.21.15.143:34534) 2024-01-15T18:10:56.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2daa000640 1 -- 172.21.15.143:0/293867718 learned_addr learned my addr 172.21.15.143:0/293867718 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:56.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973354399 0 0) 0x7f2dac0b2a20 con 0x7f2dac157300 2024-01-15T18:10:56.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d8c003530 con 0x7f2dac157300 2024-01-15T18:10:56.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283634827 0 0) 0x7f2dac0b2d00 con 0x7f2dac159b70 2024-01-15T18:10:56.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dac0b2a20 con 0x7f2dac159b70 2024-01-15T18:10:56.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2dab400640 1 -- 172.21.15.143:0/293867718 >> v1:172.21.15.204:6789/0 conn(0x7f2dac159b70 legacy=0x7f2dac0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2dab400640 1 -- 172.21.15.143:0/293867718 >> v1:172.21.15.143:6789/0 conn(0x7f2dac157300 legacy=0x7f2dac0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.433+0000 7f2dab400640 1 -- 172.21.15.143:0/293867718 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac0b2fe0 con 0x7f2dac0b4ac0 2024-01-15T18:10:56.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2dab400640 1 --2- 172.21.15.143:0/293867718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac0b4ac0 0x7f2dac164540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0097f0 tx=0x7f2d9c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bfcf6da09724df8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:56.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0aa610 con 0x7f2dac0b4ac0 2024-01-15T18:10:56.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0a6040 con 0x7f2dac0b4ac0 2024-01-15T18:10:56.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2da9600640 1 -- 172.21.15.143:0/293867718 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0aa990 con 0x7f2dac0b4ac0 2024-01-15T18:10:56.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 -- 172.21.15.143:0/293867718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac0b4ac0 msgr2=0x7f2dac164540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 --2- 172.21.15.143:0/293867718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac0b4ac0 0x7f2dac164540 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c0097f0 tx=0x7f2d9c01c4b0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 -- 172.21.15.143:0/293867718 shutdown_connections 2024-01-15T18:10:56.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 --2- 172.21.15.143:0/293867718 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac0b4ac0 0x7f2dac164540 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 -- 172.21.15.143:0/293867718 >> 172.21.15.143:0/293867718 conn(0x7f2dac06c3f0 msgr2=0x7f2dac06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:56.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 -- 172.21.15.143:0/293867718 shutdown_connections 2024-01-15T18:10:56.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.441+0000 7f2db1800640 1 -- 172.21.15.143:0/293867718 wait complete. 2024-01-15T18:10:56.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 Processor -- start 2024-01-15T18:10:56.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 -- start start 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dac0b4ac0 0x7f2dac151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 0x7f2dac14ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dac159b70 0x7f2dac135720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac159fd0 con 0x7f2dac159b70 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1336f0 con 0x7f2dac0b4ac0 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.445+0000 7f2db1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1339d0 con 0x7f2dac157300 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 0x7f2dac14ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2dab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dac159b70 0x7f2dac135720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 0x7f2dac14ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:34446/0 (socket says 172.21.15.143:34446) 2024-01-15T18:10:56.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 -- 172.21.15.143:0/2547035754 learned_addr learned my addr 172.21.15.143:0/2547035754 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:56.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daaa00640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dac0b4ac0 0x7f2dac151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 -- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dac0b4ac0 msgr2=0x7f2dac151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dac0b4ac0 0x7f2dac151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 -- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dac159b70 msgr2=0x7f2dac135720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dac159b70 0x7f2dac135720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 -- 172.21.15.143:0/2547035754 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c009420 con 0x7f2dac157300 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2daa000640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 0x7f2dac14ad80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001ec0 tx=0x7f2d9400c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d94005cf0 con 0x7f2dac157300 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d94005e50 con 0x7f2dac157300 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d940072d0 con 0x7f2dac157300 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac133cb0 con 0x7f2dac157300 2024-01-15T18:10:56.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac134270 con 0x7f2dac157300 2024-01-15T18:10:56.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.449+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dac067500 con 0x7f2dac157300 2024-01-15T18:10:56.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.453+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f2d94007430 con 0x7f2dac157300 2024-01-15T18:10:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.453+0000 7f2da2a00640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d840671b0 0x7f2d84069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.453+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2d94082370 con 0x7f2dac157300 2024-01-15T18:10:56.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.453+0000 7f2daaa00640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d840671b0 0x7f2d84069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:56.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.453+0000 7f2daaa00640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d840671b0 0x7f2d84069670 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2da4001570 tx=0x7f2da4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:56.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.457+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d94086e60 con 0x7f2dac157300 2024-01-15T18:10:56.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.657+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dac06e4e0 con 0x7f2d840671b0 2024-01-15T18:10:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.929+0000 7f2da2a00640 1 -- 172.21.15.143:0/2547035754 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f2dac06e4e0 con 0x7f2d840671b0 2024-01-15T18:10:56.933 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:56.933 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:56.933 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:10:56.933 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:56.934 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:56.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d840671b0 msgr2=0x7f2d84069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d840671b0 0x7f2d84069670 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2da4001570 tx=0x7f2da4009040 comp rx=0 tx=0).stop 2024-01-15T18:10:56.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 msgr2=0x7f2dac14ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:56.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 0x7f2dac14ad80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2d94001ec0 tx=0x7f2d9400c890 comp rx=0 tx=0).stop 2024-01-15T18:10:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 shutdown_connections 2024-01-15T18:10:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2d840671b0 0x7f2d84069670 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2dac159b70 0x7f2dac135720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2dac157300 0x7f2dac14ad80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 --2- 172.21.15.143:0/2547035754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2dac0b4ac0 0x7f2dac151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:56.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 >> 172.21.15.143:0/2547035754 conn(0x7f2dac06c3f0 msgr2=0x7f2dac0ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:56.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 shutdown_connections 2024-01-15T18:10:56.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:56.933+0000 7f2db1800640 1 -- 172.21.15.143:0/2547035754 wait complete. 2024-01-15T18:10:56.992 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:10:56.992 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42256 2024-01-15T18:10:57.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.297+0000 7fecdec00640 1 Processor -- start 2024-01-15T18:10:57.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.297+0000 7fecdec00640 1 -- start start 2024-01-15T18:10:57.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecdec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd80707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecdec00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd806cf30 con 0x7fecd80ce310 2024-01-15T18:10:57.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecdec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd80cb2f0 con 0x7fecd80b1710 2024-01-15T18:10:57.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecdec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80cb0e0 con 0x7fecd8070370 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecdcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd80707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd7400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fecd80ce310 0x7fecd8159170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54168/0 (socket says 172.21.15.143:54168) 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd7400640 1 -- 172.21.15.143:0/1112124224 learned_addr learned my addr 172.21.15.143:0/1112124224 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511048986 0 0) 0x7fecd806cf30 con 0x7fecd80ce310 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb8003530 con 0x7fecd80ce310 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4141682743 0 0) 0x7fecb8003530 con 0x7fecd80ce310 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 msgr2=0x7fecd80707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd80707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 >> v1:172.21.15.204:6789/0 conn(0x7fecd80b1710 legacy=0x7fecd806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80cb5d0 con 0x7fecd80ce310 2024-01-15T18:10:57.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fecc0001680 con 0x7fecd80ce310 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 >> v1:172.21.15.143:6789/0 conn(0x7fecd80ce310 legacy=0x7fecd8159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.301+0000 7fecd6a00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecb80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd6a00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecb8004d10 0x7fecb8005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80cb0e0 con 0x7fecb80039b0 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb800bbd0 con 0x7fecb8004d10 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb800bd90 con 0x7fecd8070370 2024-01-15T18:10:57.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7e00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7400640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecb80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecdcc00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecb8004d10 0x7fecb8005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7400640 1 -- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecb8004d10 msgr2=0x7fecb8005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7400640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecb8004d10 0x7fecb8005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7400640 1 -- 172.21.15.143:0/1112124224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecb80039b0 msgr2=0x7fecb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7400640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7400640 1 -- 172.21.15.143:0/1112124224 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb8003530 con 0x7fecd8070370 2024-01-15T18:10:57.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.305+0000 7fecd7e00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecb80039b0 0x7fecb8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:10:57.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.309+0000 7fecd7400640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecb80047d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c0e0 tx=0x7fecc001cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369c95c1e9e8dbf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:57.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.309+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd80cebe0 con 0x7fecd8070370 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.309+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc00058e0 con 0x7fecd8070370 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.309+0000 7fecd6a00640 1 -- 172.21.15.143:0/1112124224 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0005cd0 con 0x7fecd8070370 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 -- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 msgr2=0x7fecb80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 --2- 172.21.15.143:0/1112124224 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecb80047d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fecc000c0e0 tx=0x7fecc001cb30 comp rx=0 tx=0).stop 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdcc00640 1 -- 172.21.15.143:0/1112124224 reap_dead start 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 -- 172.21.15.143:0/1112124224 shutdown_connections 2024-01-15T18:10:57.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 -- 172.21.15.143:0/1112124224 >> 172.21.15.143:0/1112124224 conn(0x7fecd80b0bf0 msgr2=0x7fecd80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:57.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 -- 172.21.15.143:0/1112124224 shutdown_connections 2024-01-15T18:10:57.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 -- 172.21.15.143:0/1112124224 wait complete. 2024-01-15T18:10:57.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 Processor -- start 2024-01-15T18:10:57.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.313+0000 7fecdec00640 1 -- start start 2024-01-15T18:10:57.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd8137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 0x7fecd8137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecd81381e0 0x7fecd8139d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd8137980 con 0x7fecd8139790 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd81386f0 con 0x7fecd81381e0 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd81389d0 con 0x7fecd8070370 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 0x7fecd8137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 0x7fecd8137ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38612/0 (socket says 172.21.15.143:38612) 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 -- 172.21.15.143:0/2807156383 learned_addr learned my addr 172.21.15.143:0/2807156383 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7e00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd8137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdcc00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecd81381e0 0x7fecd8139d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 -- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 msgr2=0x7fecd8137440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd8137440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 -- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecd81381e0 msgr2=0x7fecd8139d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecd81381e0 0x7fecd8139d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 -- 172.21.15.143:0/2807156383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8138cb0 con 0x7fecd8139790 2024-01-15T18:10:57.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecd7400640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 0x7fecd8137ca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fecc0000c10 tx=0x7fecb8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:57.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc00b4070 con 0x7fecd8139790 2024-01-15T18:10:57.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc0005a50 con 0x7fecd8139790 2024-01-15T18:10:57.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0005dc0 con 0x7fecd8139790 2024-01-15T18:10:57.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd8138f90 con 0x7fecd8139790 2024-01-15T18:10:57.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd813a9f0 con 0x7fecd8139790 2024-01-15T18:10:57.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.317+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd8067680 con 0x7fecd8139790 2024-01-15T18:10:57.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.321+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7fecc00a5830 con 0x7fecd8139790 2024-01-15T18:10:57.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.321+0000 7feccfe00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fecb0067190 0x7fecb0069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:57.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.321+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fecc011c410 con 0x7fecd8139790 2024-01-15T18:10:57.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.321+0000 7fecd7e00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fecb0067190 0x7fecb0069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:57.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.325+0000 7fecd7e00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fecb0067190 0x7fecb0069650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fecd815ce70 tx=0x7fecc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:57.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.333+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fecc00eaf80 con 0x7fecd8139790 2024-01-15T18:10:57.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.773+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42256", "target": ["mon-mgr", ""]}) v1 -- 0x7fecd8138b40 con 0x7fecb0067190 2024-01-15T18:10:57.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.905+0000 7feccfe00640 1 -- 172.21.15.143:0/2807156383 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fecd8138b40 con 0x7fecb0067190 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:57.912 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:57.913 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fecb0067190 msgr2=0x7fecb0069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fecb0067190 0x7fecb0069650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fecd815ce70 tx=0x7fecc8009040 comp rx=0 tx=0).stop 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 msgr2=0x7fecd8137ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 0x7fecd8137ca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fecc0000c10 tx=0x7fecb8005800 comp rx=0 tx=0).stop 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 shutdown_connections 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fecb0067190 0x7fecb0069650 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecd81381e0 0x7fecd8139d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fecd8139790 0x7fecd8137ca0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 --2- 172.21.15.143:0/2807156383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fecd8070370 0x7fecd8137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:57.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.913+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 >> 172.21.15.143:0/2807156383 conn(0x7fecd80b0bf0 msgr2=0x7fecd8158370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:57.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.917+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 shutdown_connections 2024-01-15T18:10:57.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:57.917+0000 7fecdec00640 1 -- 172.21.15.143:0/2807156383 wait complete. 2024-01-15T18:10:58.053 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:10:58.296 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:58.302 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:59.054 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42256 2024-01-15T18:10:59.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.417+0000 7fd4a2200640 1 Processor -- start 2024-01-15T18:10:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.417+0000 7fd4a2200640 1 -- start start 2024-01-15T18:10:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.417+0000 7fd4a2200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd49c159380 0x7fd49c072180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.417+0000 7fd4a2200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd49c0ce950 0x7fd49c070a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.417+0000 7fd4a2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.421+0000 7fd49b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd49c159380 0x7fd49c072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:59.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.421+0000 7fd49b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd49c159380 0x7fd49c072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60446/0 (socket says 172.21.15.143:60446) 2024-01-15T18:10:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd4a2200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c0cfd80 con 0x7fd49c0ce950 2024-01-15T18:10:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd4a2200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c159760 con 0x7fd49c159380 2024-01-15T18:10:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd4a2200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c071cf0 con 0x7fd49c0cf3b0 2024-01-15T18:10:59.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49b400640 1 -- 172.21.15.143:0/207735780 learned_addr learned my addr 172.21.15.143:0/207735780 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:59.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49be00640 1 --2- 172.21.15.143:0/207735780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49aa00640 1 --2- 172.21.15.143:0/207735780 >> v2:172.21.15.143:3300/0 conn(0x7fd49c0ce950 0x7fd49c070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49be00640 1 -- 172.21.15.143:0/207735780 >> v2:172.21.15.204:3300/0 conn(0x7fd49c159380 msgr2=0x7fd49c072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49be00640 1 --2- 172.21.15.143:0/207735780 >> v2:172.21.15.204:3300/0 conn(0x7fd49c159380 0x7fd49c072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49be00640 1 -- 172.21.15.143:0/207735780 >> v2:172.21.15.143:3300/0 conn(0x7fd49c0ce950 msgr2=0x7fd49c070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49be00640 1 --2- 172.21.15.143:0/207735780 >> v2:172.21.15.143:3300/0 conn(0x7fd49c0ce950 0x7fd49c070a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.425+0000 7fd49be00640 1 -- 172.21.15.143:0/207735780 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c0ce520 con 0x7fd49c0cf3b0 2024-01-15T18:10:59.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.433+0000 7fd49be00640 1 --2- 172.21.15.143:0/207735780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c0cf840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd49c06d550 tx=0x7fd48401ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e303034829a85f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.437+0000 7fd49a000640 1 -- 172.21.15.143:0/207735780 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840a88f0 con 0x7fd49c0cf3b0 2024-01-15T18:10:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.437+0000 7fd49a000640 1 -- 172.21.15.143:0/207735780 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4840a8a50 con 0x7fd49c0cf3b0 2024-01-15T18:10:59.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.437+0000 7fd49a000640 1 -- 172.21.15.143:0/207735780 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4840a8de0 con 0x7fd49c0cf3b0 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 -- 172.21.15.143:0/207735780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 msgr2=0x7fd49c0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 --2- 172.21.15.143:0/207735780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c0cf840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd49c06d550 tx=0x7fd48401ec30 comp rx=0 tx=0).stop 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 -- 172.21.15.143:0/207735780 shutdown_connections 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 --2- 172.21.15.143:0/207735780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c0cf840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 --2- 172.21.15.143:0/207735780 >> v2:172.21.15.143:3300/0 conn(0x7fd49c0ce950 0x7fd49c070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 --2- 172.21.15.143:0/207735780 >> v2:172.21.15.204:3300/0 conn(0x7fd49c159380 0x7fd49c072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.441+0000 7fd4a2200640 1 -- 172.21.15.143:0/207735780 >> 172.21.15.143:0/207735780 conn(0x7fd49c06a890 msgr2=0x7fd49c06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:59.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.453+0000 7fd4a2200640 1 -- 172.21.15.143:0/207735780 shutdown_connections 2024-01-15T18:10:59.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.453+0000 7fd4a2200640 1 -- 172.21.15.143:0/207735780 wait complete. 2024-01-15T18:10:59.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.461+0000 7fd4a2200640 1 Processor -- start 2024-01-15T18:10:59.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.461+0000 7fd4a2200640 1 -- start start 2024-01-15T18:10:59.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.469+0000 7fd4a2200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 0x7fd49c150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.469+0000 7fd4a2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.469+0000 7fd4a2200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd49c159380 0x7fd49c06c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.469+0000 7fd4a2200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c15a670 con 0x7fd49c0ce950 2024-01-15T18:10:59.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 0x7fd49c150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:59.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 0x7fd49c150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38620/0 (socket says 172.21.15.143:38620) 2024-01-15T18:10:59.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 -- 172.21.15.143:0/1195563597 learned_addr learned my addr 172.21.15.143:0/1195563597 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:10:59.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c15a950 con 0x7fd49c159380 2024-01-15T18:10:59.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c15ac30 con 0x7fd49c0cf3b0 2024-01-15T18:10:59.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 -- 172.21.15.143:0/1195563597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 msgr2=0x7fd49c1515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 -- 172.21.15.143:0/1195563597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd49c159380 msgr2=0x7fd49c06c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd49c159380 0x7fd49c06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.473+0000 7fd49b400640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484009420 con 0x7fd49c0ce950 2024-01-15T18:10:59.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.477+0000 7fd49b400640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 0x7fd49c150330 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00ba20 tx=0x7fd48c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:59.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.477+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00da20 con 0x7fd49c0ce950 2024-01-15T18:10:59.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.477+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c00f950 con 0x7fd49c0ce950 2024-01-15T18:10:59.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.477+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c15af10 con 0x7fd49c0ce950 2024-01-15T18:10:59.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.477+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c15a2b0 con 0x7fd49c0ce950 2024-01-15T18:10:59.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.477+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00fce0 con 0x7fd49c0ce950 2024-01-15T18:10:59.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.481+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c005260 con 0x7fd49c0ce950 2024-01-15T18:10:59.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.481+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7fd48c014070 con 0x7fd49c0ce950 2024-01-15T18:10:59.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.481+0000 7fd493400640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4740671b0 0x7fd474069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:59.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.481+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd48c0824a0 con 0x7fd49c0ce950 2024-01-15T18:10:59.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.489+0000 7fd49aa00640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4740671b0 0x7fd474069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.493+0000 7fd49aa00640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4740671b0 0x7fd474069670 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd494007130 tx=0x7fd494007310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:59.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.493+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd48c050f60 con 0x7fd49c0ce950 2024-01-15T18:10:59.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.953+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42256", "target": ["mon-mgr", ""]}) v1 -- 0x7fd46c000bd0 con 0x7fd4740671b0 2024-01-15T18:10:59.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.977+0000 7fd493400640 1 -- 172.21.15.143:0/1195563597 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fd46c000bd0 con 0x7fd4740671b0 2024-01-15T18:10:59.980 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:10:59.980 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:10:59.980 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:10:59.980 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:10:59.981 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:10:59.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4740671b0 msgr2=0x7fd474069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:59.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4740671b0 0x7fd474069670 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd494007130 tx=0x7fd494007310 comp rx=0 tx=0).stop 2024-01-15T18:10:59.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 msgr2=0x7fd49c150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:59.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 0x7fd49c150330 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00ba20 tx=0x7fd48c00bee0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 shutdown_connections 2024-01-15T18:10:59.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4740671b0 0x7fd474069670 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd49c159380 0x7fd49c06c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd49c0cf3b0 0x7fd49c1515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 --2- 172.21.15.143:0/1195563597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd49c0ce950 0x7fd49c150330 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:59.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 >> 172.21.15.143:0/1195563597 conn(0x7fd49c06a890 msgr2=0x7fd49c1544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:59.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 shutdown_connections 2024-01-15T18:10:59.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:10:59.981+0000 7fd4a2200640 1 -- 172.21.15.143:0/1195563597 wait complete. 2024-01-15T18:11:00.062 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:11:01.063 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42256 2024-01-15T18:11:01.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 Processor -- start 2024-01-15T18:11:01.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 -- start start 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98e0150690 0x7f98e006a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e015a5e0 0x7f98e00b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e006d170 con 0x7f98e0070dd0 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e0150a70 con 0x7f98e0150690 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98e6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00b2a50 con 0x7f98e015a5e0 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98e0150690 0x7f98e006a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98e0150690 0x7f98e006a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38636/0 (socket says 172.21.15.143:38636) 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 -- 172.21.15.143:0/520367897 learned_addr learned my addr 172.21.15.143:0/520367897 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98df400640 1 --2- 172.21.15.143:0/520367897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e015a5e0 0x7f98e00b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 -- 172.21.15.143:0/520367897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e015a5e0 msgr2=0x7f98e00b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 --2- 172.21.15.143:0/520367897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e015a5e0 0x7f98e00b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 -- 172.21.15.143:0/520367897 >> v1:172.21.15.204:6789/0 conn(0x7f98e0070dd0 legacy=0x7f98e006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.201+0000 7f98dfe00640 1 -- 172.21.15.143:0/520367897 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e015aa20 con 0x7f98e0150690 2024-01-15T18:11:01.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98dfe00640 1 --2- 172.21.15.143:0/520367897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e0150690 0x7f98e006a490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009870 tx=0x7f98d001d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e07e6bf795d74c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:01.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98dea00640 1 -- 172.21.15.143:0/520367897 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00a9630 con 0x7f98e0150690 2024-01-15T18:11:01.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98dea00640 1 -- 172.21.15.143:0/520367897 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d00a9790 con 0x7f98e0150690 2024-01-15T18:11:01.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98dea00640 1 -- 172.21.15.143:0/520367897 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00a9b20 con 0x7f98e0150690 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 -- 172.21.15.143:0/520367897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e0150690 msgr2=0x7f98e006a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 --2- 172.21.15.143:0/520367897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e0150690 0x7f98e006a490 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009870 tx=0x7f98d001d620 comp rx=0 tx=0).stop 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 -- 172.21.15.143:0/520367897 shutdown_connections 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 --2- 172.21.15.143:0/520367897 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e015a5e0 0x7f98e00b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 --2- 172.21.15.143:0/520367897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e0150690 0x7f98e006a490 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 -- 172.21.15.143:0/520367897 >> 172.21.15.143:0/520367897 conn(0x7f98e00b1610 msgr2=0x7f98e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 -- 172.21.15.143:0/520367897 shutdown_connections 2024-01-15T18:11:01.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.205+0000 7f98e6a00640 1 -- 172.21.15.143:0/520367897 wait complete. 2024-01-15T18:11:01.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 Processor -- start 2024-01-15T18:11:01.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- start start 2024-01-15T18:11:01.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98e0070dd0 0x7f98e0149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e0150690 0x7f98e0138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 0x7f98e01390e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0149a90 con 0x7f98e015a5e0 2024-01-15T18:11:01.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0148540 con 0x7f98e0070dd0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e014a3f0 con 0x7f98e0150690 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 0x7f98e01390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 0x7f98e01390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38642/0 (socket says 172.21.15.143:38642) 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 -- 172.21.15.143:0/2789656428 learned_addr learned my addr 172.21.15.143:0/2789656428 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98df400640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e0150690 0x7f98e0138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98dfe00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98e0070dd0 0x7f98e0149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 -- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e0150690 msgr2=0x7f98e0138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e0150690 0x7f98e0138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 -- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98e0070dd0 msgr2=0x7f98e0149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98e0070dd0 0x7f98e0149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 -- 172.21.15.143:0/2789656428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00094d0 con 0x7f98e015a5e0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e4a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 0x7f98e01390e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c730 tx=0x7f98d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003b80 con 0x7f98e015a5e0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8003ce0 con 0x7f98e015a5e0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e0148e10 con 0x7f98e015a5e0 2024-01-15T18:11:01.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d800aa30 con 0x7f98e015a5e0 2024-01-15T18:11:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e01493d0 con 0x7f98e015a5e0 2024-01-15T18:11:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.209+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0074b00 con 0x7f98e015a5e0 2024-01-15T18:11:01.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.213+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f98d800ab90 con 0x7f98e015a5e0 2024-01-15T18:11:01.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.213+0000 7f98d7e00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f98b8067160 0x7f98b8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.213+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f98d8082ac0 con 0x7f98e015a5e0 2024-01-15T18:11:01.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.213+0000 7f98dfe00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f98b8067160 0x7f98b8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.213+0000 7f98dfe00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f98b8067160 0x7f98b8069620 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f98d00b0000 tx=0x7f98d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:01.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.217+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98d8051690 con 0x7f98e015a5e0 2024-01-15T18:11:01.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.397+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42256", "target": ["mon-mgr", ""]}) v1 -- 0x7f98e006e120 con 0x7f98b8067160 2024-01-15T18:11:01.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.549+0000 7f98d7e00640 1 -- 172.21.15.143:0/2789656428 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f98d00a9610 con 0x7f98b8067160 2024-01-15T18:11:01.553 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:01.553 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:11:01.553 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:11:01.553 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:01.553 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f98b8067160 msgr2=0x7f98b8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f98b8067160 0x7f98b8069620 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f98d00b0000 tx=0x7f98d0008040 comp rx=0 tx=0).stop 2024-01-15T18:11:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 msgr2=0x7f98e01390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 0x7f98e01390e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f98d800c730 tx=0x7f98d800cbf0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 shutdown_connections 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f98b8067160 0x7f98b8069620 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e015a5e0 0x7f98e01390e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98e0150690 0x7f98e0138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 --2- 172.21.15.143:0/2789656428 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98e0070dd0 0x7f98e0149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 >> 172.21.15.143:0/2789656428 conn(0x7f98e00b1610 msgr2=0x7f98e00b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 shutdown_connections 2024-01-15T18:11:01.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.553+0000 7f98e6a00640 1 -- 172.21.15.143:0/2789656428 wait complete. 2024-01-15T18:11:01.575 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:11:01.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42257 2024-01-15T18:11:01.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9093000640 1 Processor -- start 2024-01-15T18:11:01.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9093000640 1 -- start start 2024-01-15T18:11:01.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9093000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c072030 0x7f908c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9093000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c0ceb50 con 0x7f908c143780 2024-01-15T18:11:01.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9093000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f908c15c110 con 0x7f908c071bb0 2024-01-15T18:11:01.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9093000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0724c0 con 0x7f908c072030 2024-01-15T18:11:01.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f908c143780 0x7f908c0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54180/0 (socket says 172.21.15.143:54180) 2024-01-15T18:11:01.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 -- 172.21.15.143:0/2395730261 learned_addr learned my addr 172.21.15.143:0/2395730261 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044484963 0 0) 0x7f907c001120 con 0x7f908c143780 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9091000640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c072030 0x7f908c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f906c005740 con 0x7f908c143780 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148891741 0 0) 0x7f906c005740 con 0x7f908c143780 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c072030 msgr2=0x7f908c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c072030 0x7f908c15ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 >> v1:172.21.15.204:6789/0 conn(0x7f908c071bb0 legacy=0x7f908c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c15b7e0 con 0x7f908c143780 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f907c001830 con 0x7f908c143780 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 >> v1:172.21.15.143:6789/0 conn(0x7f908c143780 legacy=0x7f908c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906c005bc0 0x7f906c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c0069e0 0x7f906c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f906c0073a0 0x7f906c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c0724c0 con 0x7f906c005bc0 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c00e240 con 0x7f906c0069e0 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c00e400 con 0x7f906c0073a0 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9091000640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c072030 0x7f908c15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906c005bc0 0x7f906c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908b400640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c0069e0 0x7f906c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f9091000640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f906c0073a0 0x7f906c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 -- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f906c0073a0 msgr2=0x7f906c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f906c0073a0 0x7f906c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 -- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c0069e0 msgr2=0x7f906c006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c0069e0 0x7f906c006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.713+0000 7f908be00640 1 -- 172.21.15.143:0/2395730261 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c005740 con 0x7f906c005bc0 2024-01-15T18:11:01.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f908be00640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906c005bc0 0x7f906c005fe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f907c00c210 tx=0x7f907c01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f000f607de43b70 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:01.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c005ae0 con 0x7f906c005bc0 2024-01-15T18:11:01.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c005c40 con 0x7f906c005bc0 2024-01-15T18:11:01.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f908aa00640 1 -- 172.21.15.143:0/2395730261 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0a58e0 con 0x7f906c005bc0 2024-01-15T18:11:01.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9093000640 1 -- 172.21.15.143:0/2395730261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906c005bc0 msgr2=0x7f906c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9093000640 1 --2- 172.21.15.143:0/2395730261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f906c005bc0 0x7f906c005fe0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f907c00c210 tx=0x7f907c01cc00 comp rx=0 tx=0).stop 2024-01-15T18:11:01.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9091000640 1 -- 172.21.15.143:0/2395730261 reap_dead start 2024-01-15T18:11:01.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9093000640 1 -- 172.21.15.143:0/2395730261 shutdown_connections 2024-01-15T18:11:01.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9093000640 1 -- 172.21.15.143:0/2395730261 >> 172.21.15.143:0/2395730261 conn(0x7f908c06c3f0 msgr2=0x7f908c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:01.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9093000640 1 -- 172.21.15.143:0/2395730261 shutdown_connections 2024-01-15T18:11:01.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.717+0000 7f9093000640 1 -- 172.21.15.143:0/2395730261 wait complete. 2024-01-15T18:11:01.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 Processor -- start 2024-01-15T18:11:01.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 -- start start 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c005bc0 0x7f908c14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c134f00 0x7f908c1347d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 0x7f908c0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c159880 con 0x7f908c148460 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908c134d10 con 0x7f906c005bc0 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908c142a50 con 0x7f908c134f00 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 0x7f908c0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 0x7f908c0cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38660/0 (socket says 172.21.15.143:38660) 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 -- 172.21.15.143:0/2671653071 learned_addr learned my addr 172.21.15.143:0/2671653071 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f908b400640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c134f00 0x7f908c1347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f908be00640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c005bc0 0x7f908c14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 -- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c134f00 msgr2=0x7f908c1347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c134f00 0x7f908c1347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 -- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c005bc0 msgr2=0x7f908c14aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c005bc0 0x7f908c14aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 -- 172.21.15.143:0/2671653071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c135360 con 0x7f908c148460 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f908b400640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c134f00 0x7f908c1347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f908be00640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c005bc0 0x7f908c14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9091000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 0x7f908c0cd490 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f90840043b0 tx=0x7f9084004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084007ce0 con 0x7f908c148460 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908400f050 con 0x7f908c148460 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908c142fb0 con 0x7f908c148460 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.721+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9084014630 con 0x7f908c148460 2024-01-15T18:11:01.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.725+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908c136990 con 0x7f908c148460 2024-01-15T18:11:01.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.725+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f908c1367f0 con 0x7f908c148460 2024-01-15T18:11:01.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.725+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f9084014790 con 0x7f908c148460 2024-01-15T18:11:01.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.725+0000 7f9083e00640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9064067190 0x7f9064069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:01.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.725+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f90840832d0 con 0x7f908c148460 2024-01-15T18:11:01.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.725+0000 7f908be00640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9064067190 0x7f9064069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:01.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.729+0000 7f908be00640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9064067190 0x7f9064069650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f907c0ab000 tx=0x7f907c0a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:01.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.729+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f908c1367f0 con 0x7f908c148460 2024-01-15T18:11:01.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.913+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42257", "target": ["mon-mgr", ""]}) v1 -- 0x7f908c1367f0 con 0x7f9064067190 2024-01-15T18:11:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.921+0000 7f9083e00640 1 -- 172.21.15.143:0/2671653071 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f908c1367f0 con 0x7f9064067190 2024-01-15T18:11:01.926 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:01.926 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:11:01.926 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:11:01.926 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:01.927 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:01.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9064067190 msgr2=0x7f9064069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9064067190 0x7f9064069650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f907c0ab000 tx=0x7f907c0a7070 comp rx=0 tx=0).stop 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 msgr2=0x7f908c0cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 0x7f908c0cd490 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f90840043b0 tx=0x7f9084004870 comp rx=0 tx=0).stop 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 shutdown_connections 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9064067190 0x7f9064069650 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f908c148460 0x7f908c0cd490 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f908c134f00 0x7f908c1347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 --2- 172.21.15.143:0/2671653071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f906c005bc0 0x7f908c14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 >> 172.21.15.143:0/2671653071 conn(0x7f908c06c3f0 msgr2=0x7f908c0ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 shutdown_connections 2024-01-15T18:11:01.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:01.925+0000 7f9093000640 1 -- 172.21.15.143:0/2671653071 wait complete. 2024-01-15T18:11:01.949 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:11:02.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42257 2024-01-15T18:11:03.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.269+0000 7fa848800640 1 Processor -- start 2024-01-15T18:11:03.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.273+0000 7fa848800640 1 -- start start 2024-01-15T18:11:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.273+0000 7fa848800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa84014b1e0 0x7fa8400cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.273+0000 7fa848800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8400cf7c0 0x7fa8400708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.273+0000 7fa848800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8400cfc20 con 0x7fa840070e00 2024-01-15T18:11:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.273+0000 7fa848800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa840162080 con 0x7fa84014b1e0 2024-01-15T18:11:03.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.273+0000 7fa848800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa840161d70 con 0x7fa8400cf7c0 2024-01-15T18:11:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.277+0000 7fa846800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa840070e00 0x7fa8400b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58638/0 (socket says 172.21.15.143:58638) 2024-01-15T18:11:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.277+0000 7fa846800640 1 -- 172.21.15.143:0/2196276599 learned_addr learned my addr 172.21.15.143:0/2196276599 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.277+0000 7fa845400640 1 --2- 172.21.15.143:0/2196276599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8400cf7c0 0x7fa8400708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:03.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.277+0000 7fa845e00640 1 --2- 172.21.15.143:0/2196276599 >> v2:172.21.15.204:3300/0 conn(0x7fa84014b1e0 0x7fa8400cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.281+0000 7fa844a00640 1 -- 172.21.15.143:0/2196276599 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704483538 0 0) 0x7fa8400cfc20 con 0x7fa840070e00 2024-01-15T18:11:03.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.281+0000 7fa844a00640 1 -- 172.21.15.143:0/2196276599 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa820003530 con 0x7fa840070e00 2024-01-15T18:11:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.281+0000 7fa845400640 1 -- 172.21.15.143:0/2196276599 >> v2:172.21.15.204:3300/0 conn(0x7fa84014b1e0 msgr2=0x7fa8400cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.281+0000 7fa845400640 1 --2- 172.21.15.143:0/2196276599 >> v2:172.21.15.204:3300/0 conn(0x7fa84014b1e0 0x7fa8400cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.281+0000 7fa845400640 1 -- 172.21.15.143:0/2196276599 >> v1:172.21.15.204:6789/0 conn(0x7fa840070e00 legacy=0x7fa8400b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:03.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.281+0000 7fa845400640 1 -- 172.21.15.143:0/2196276599 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84014b5c0 con 0x7fa8400cf7c0 2024-01-15T18:11:03.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa845400640 1 --2- 172.21.15.143:0/2196276599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8400cf7c0 0x7fa8400708c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8280097c0 tx=0x7fa82801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9a18e4f5491138 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:03.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa844a00640 1 -- 172.21.15.143:0/2196276599 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280ab650 con 0x7fa8400cf7c0 2024-01-15T18:11:03.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa844a00640 1 -- 172.21.15.143:0/2196276599 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8280ab7b0 con 0x7fa8400cf7c0 2024-01-15T18:11:03.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa844a00640 1 -- 172.21.15.143:0/2196276599 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280abb40 con 0x7fa8400cf7c0 2024-01-15T18:11:03.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 -- 172.21.15.143:0/2196276599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8400cf7c0 msgr2=0x7fa8400708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:03.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 --2- 172.21.15.143:0/2196276599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8400cf7c0 0x7fa8400708c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa8280097c0 tx=0x7fa82801c490 comp rx=0 tx=0).stop 2024-01-15T18:11:03.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 -- 172.21.15.143:0/2196276599 shutdown_connections 2024-01-15T18:11:03.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 --2- 172.21.15.143:0/2196276599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa8400cf7c0 0x7fa8400708c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 --2- 172.21.15.143:0/2196276599 >> v2:172.21.15.204:3300/0 conn(0x7fa84014b1e0 0x7fa8400cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 -- 172.21.15.143:0/2196276599 >> 172.21.15.143:0/2196276599 conn(0x7fa84006a890 msgr2=0x7fa84006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:03.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.285+0000 7fa848800640 1 -- 172.21.15.143:0/2196276599 shutdown_connections 2024-01-15T18:11:03.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.289+0000 7fa848800640 1 -- 172.21.15.143:0/2196276599 wait complete. 2024-01-15T18:11:03.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.289+0000 7fa848800640 1 Processor -- start 2024-01-15T18:11:03.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.289+0000 7fa848800640 1 -- start start 2024-01-15T18:11:03.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa840070e00 0x7fa840146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:03.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8400cf7c0 0x7fa8401fa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:03.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa84014b1e0 0x7fa840160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:03.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400ad620 con 0x7fa840070e00 2024-01-15T18:11:03.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400aadd0 con 0x7fa8400cf7c0 2024-01-15T18:11:03.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8400ab0b0 con 0x7fa84014b1e0 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8400cf7c0 0x7fa8401fa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8400cf7c0 0x7fa8401fa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60484/0 (socket says 172.21.15.143:60484) 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845400640 1 -- 172.21.15.143:0/2040089763 learned_addr learned my addr 172.21.15.143:0/2040089763 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa840070e00 0x7fa840146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa846800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa84014b1e0 0x7fa840160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 -- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa84014b1e0 msgr2=0x7fa840160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa84014b1e0 0x7fa840160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 -- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8400cf7c0 msgr2=0x7fa8401fa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8400cf7c0 0x7fa8401fa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 -- 172.21.15.143:0/2040089763 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009420 con 0x7fa840070e00 2024-01-15T18:11:03.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa846800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa84014b1e0 0x7fa840160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa845e00640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa840070e00 0x7fa840146950 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa83000e870 tx=0x7fa83000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:03.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000ee60 con 0x7fa840070e00 2024-01-15T18:11:03.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8400ab390 con 0x7fa840070e00 2024-01-15T18:11:03.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.293+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8400ab950 con 0x7fa840070e00 2024-01-15T18:11:03.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.297+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000c940 con 0x7fa840070e00 2024-01-15T18:11:03.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.297+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000ccd0 con 0x7fa840070e00 2024-01-15T18:11:03.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.297+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa810005260 con 0x7fa840070e00 2024-01-15T18:11:03.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.301+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7fa830013470 con 0x7fa840070e00 2024-01-15T18:11:03.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.301+0000 7fa83e000640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa818067160 0x7fa818069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:03.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.301+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa8300856c0 con 0x7fa840070e00 2024-01-15T18:11:03.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.305+0000 7fa845400640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa818067160 0x7fa818069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:03.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.305+0000 7fa845400640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa818067160 0x7fa818069620 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa8280097c0 tx=0x7fa8280a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:03.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.313+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa830051960 con 0x7fa840070e00 2024-01-15T18:11:03.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.709+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42257", "target": ["mon-mgr", ""]}) v1 -- 0x7fa810000bd0 con 0x7fa818067160 2024-01-15T18:11:03.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa83e000640 1 -- 172.21.15.143:0/2040089763 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fa810000bd0 con 0x7fa818067160 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:03.740 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:03.741 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa818067160 msgr2=0x7fa818069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa818067160 0x7fa818069620 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa8280097c0 tx=0x7fa8280a7000 comp rx=0 tx=0).stop 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa840070e00 msgr2=0x7fa840146950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa840070e00 0x7fa840146950 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa83000e870 tx=0x7fa83000ed30 comp rx=0 tx=0).stop 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 shutdown_connections 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa818067160 0x7fa818069620 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa84014b1e0 0x7fa840160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa8400cf7c0 0x7fa8401fa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 --2- 172.21.15.143:0/2040089763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa840070e00 0x7fa840146950 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 >> 172.21.15.143:0/2040089763 conn(0x7fa84006a890 msgr2=0x7fa840143a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 shutdown_connections 2024-01-15T18:11:03.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:03.737+0000 7fa848800640 1 -- 172.21.15.143:0/2040089763 wait complete. 2024-01-15T18:11:03.804 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:11:04.805 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone42257 2024-01-15T18:11:04.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 Processor -- start 2024-01-15T18:11:04.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 -- start start 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f60dc141660 0x7f60dc0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60dc15ec80 0x7f60dc141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60dc15f310 con 0x7f60dc142470 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60dc15fbb0 con 0x7f60dc15ec80 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e3800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60dc15f130 con 0x7f60dc141660 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60dc15ec80 0x7f60dc141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60dc15ec80 0x7f60dc141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38690/0 (socket says 172.21.15.143:38690) 2024-01-15T18:11:04.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 -- 172.21.15.143:0/3015119696 learned_addr learned my addr 172.21.15.143:0/3015119696 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e1800640 1 --1- 172.21.15.143:0/3015119696 >> v1:172.21.15.143:6789/0 conn(0x7f60dc142470 0x7f60dc071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54184/0 (socket says 172.21.15.143:54184) 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60e0e00640 1 --2- 172.21.15.143:0/3015119696 >> v2:172.21.15.204:3300/0 conn(0x7f60dc141660 0x7f60dc0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 -- 172.21.15.143:0/3015119696 >> v2:172.21.15.204:3300/0 conn(0x7f60dc141660 msgr2=0x7f60dc0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 --2- 172.21.15.143:0/3015119696 >> v2:172.21.15.204:3300/0 conn(0x7f60dc141660 0x7f60dc0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 -- 172.21.15.143:0/3015119696 >> v1:172.21.15.143:6789/0 conn(0x7f60dc142470 legacy=0x7f60dc071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60dbe00640 1 -- 172.21.15.143:0/3015119696 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc15f5f0 con 0x7f60dc15ec80 2024-01-15T18:11:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.945+0000 7f60db400640 1 -- 172.21.15.143:0/3015119696 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19552250 0 0) 0x7f60dc15f310 con 0x7f60dc142470 2024-01-15T18:11:04.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60dbe00640 1 --2- 172.21.15.143:0/3015119696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc15ec80 0x7f60dc141f30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0b1aa0 tx=0x7f60c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dfdc22f85d482b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:04.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60db400640 1 -- 172.21.15.143:0/3015119696 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c40aa430 con 0x7f60dc15ec80 2024-01-15T18:11:04.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60db400640 1 -- 172.21.15.143:0/3015119696 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c40aa590 con 0x7f60dc15ec80 2024-01-15T18:11:04.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60db400640 1 -- 172.21.15.143:0/3015119696 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c40aa920 con 0x7f60dc15ec80 2024-01-15T18:11:04.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 -- 172.21.15.143:0/3015119696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc15ec80 msgr2=0x7f60dc141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:04.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 --2- 172.21.15.143:0/3015119696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc15ec80 0x7f60dc141f30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0b1aa0 tx=0x7f60c401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:11:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 -- 172.21.15.143:0/3015119696 shutdown_connections 2024-01-15T18:11:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 --2- 172.21.15.143:0/3015119696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc15ec80 0x7f60dc141f30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 --2- 172.21.15.143:0/3015119696 >> v2:172.21.15.204:3300/0 conn(0x7f60dc141660 0x7f60dc0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 -- 172.21.15.143:0/3015119696 >> 172.21.15.143:0/3015119696 conn(0x7f60dc06c3f0 msgr2=0x7f60dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 -- 172.21.15.143:0/3015119696 shutdown_connections 2024-01-15T18:11:04.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.949+0000 7f60e3800640 1 -- 172.21.15.143:0/3015119696 wait complete. 2024-01-15T18:11:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 Processor -- start 2024-01-15T18:11:04.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 -- start start 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60dc141660 0x7f60dc15e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 0x7f60dc159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60dc15ec80 0x7f60dc159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc1595d0 con 0x7f60dc142470 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc15bae0 con 0x7f60dc15ec80 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc15c660 con 0x7f60dc141660 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 0x7f60dc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 0x7f60dc159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38698/0 (socket says 172.21.15.143:38698) 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 -- 172.21.15.143:0/136222162 learned_addr learned my addr 172.21.15.143:0/136222162 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e0e00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60dc141660 0x7f60dc15e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e1800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60dc15ec80 0x7f60dc159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:04.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 -- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60dc141660 msgr2=0x7f60dc15e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60dc141660 0x7f60dc15e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 -- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60dc15ec80 msgr2=0x7f60dc159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60dc15ec80 0x7f60dc159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 -- 172.21.15.143:0/136222162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40094d0 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60dbe00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 0x7f60dc159770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f60c40ac040 tx=0x7f60c401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c40aac50 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c40b7070 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc0cd150 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c4004030 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.953+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc15c010 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.957+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc067500 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.957+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f60c4004190 con 0x7f60dc142470 2024-01-15T18:11:04.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.957+0000 7f60d8c00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b4067160 0x7f60b4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.957+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f60c411cfa0 con 0x7f60dc142470 2024-01-15T18:11:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.957+0000 7f60e0e00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b4067160 0x7f60b4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:04.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.957+0000 7f60e0e00640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b4067160 0x7f60b4069620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009b60 tx=0x7f60cc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:04.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:04.961+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f60c40ebb10 con 0x7f60dc142470 2024-01-15T18:11:05.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:05.141+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone42257", "target": ["mon-mgr", ""]}) v1 -- 0x7f60dc0ccfd0 con 0x7f60b4067160 2024-01-15T18:11:07.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.849+0000 7f60d8c00640 1 -- 172.21.15.143:0/136222162 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f60cc011ac0 con 0x7f60b4067160 2024-01-15T18:11:07.853 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:07.853 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:11:07.854 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:11:07.854 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:07.854 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b4067160 msgr2=0x7f60b4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b4067160 0x7f60b4069620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009b60 tx=0x7f60cc009190 comp rx=0 tx=0).stop 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 msgr2=0x7f60dc159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 0x7f60dc159770 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f60c40ac040 tx=0x7f60c401ad10 comp rx=0 tx=0).stop 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 shutdown_connections 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b4067160 0x7f60b4069620 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60dc15ec80 0x7f60dc159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60dc142470 0x7f60dc159770 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 --2- 172.21.15.143:0/136222162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60dc141660 0x7f60dc15e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 >> 172.21.15.143:0/136222162 conn(0x7f60dc06c3f0 msgr2=0x7f60dc139ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 shutdown_connections 2024-01-15T18:11:07.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:07.853+0000 7f60e3800640 1 -- 172.21.15.143:0/136222162 wait complete. 2024-01-15T18:11:07.880 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:11:08.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.017+0000 7f3bfbc00640 1 Processor -- start 2024-01-15T18:11:08.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bfbc00640 1 -- start start 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bfbc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3bf40ce560 0x7f3bf415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bfbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 0x7f3bf4072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bfbc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf415f310 con 0x7f3bf4141660 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bfbc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf415fbb0 con 0x7f3bf40ce560 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bfbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4072690 con 0x7f3bf4071d00 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3bf4141660 0x7f3bf4141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56888/0 (socket says 172.21.15.143:56888) 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9200640 1 -- 172.21.15.143:0/2017748761 learned_addr learned my addr 172.21.15.143:0/2017748761 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:08.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 0x7f3bf4072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3e00640 1 --2- 172.21.15.143:0/2017748761 >> v2:172.21.15.204:3300/0 conn(0x7f3bf40ce560 0x7f3bf415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322420042 0 0) 0x7f3bf415f310 con 0x7f3bf4141660 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd4003530 con 0x7f3bf4141660 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2402350230 0 0) 0x7f3bd4003530 con 0x7f3bf4141660 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 msgr2=0x7f3bf4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 0x7f3bf4072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 >> v2:172.21.15.204:3300/0 conn(0x7f3bf40ce560 msgr2=0x7f3bf415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 --2- 172.21.15.143:0/2017748761 >> v2:172.21.15.204:3300/0 conn(0x7f3bf40ce560 0x7f3bf415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf415f5f0 con 0x7f3bf4141660 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3be4001680 con 0x7f3bf4141660 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 >> v1:172.21.15.143:6789/0 conn(0x7f3bf4141660 legacy=0x7f3bf4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 0x7f3bd4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bd4003f00 0x7f3bd40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bd4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415fbb0 con 0x7f3bd4004d10 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf4072690 con 0x7f3bd4003f00 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd400b990 con 0x7f3bf4071d00 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bd4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9200640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 0x7f3bd4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf3e00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bd4003f00 0x7f3bd40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 -- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 msgr2=0x7f3bd4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf4071d00 0x7f3bd4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 -- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bd4003f00 msgr2=0x7f3bd40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bd4003f00 0x7f3bd40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.021+0000 7f3bf9c00640 1 -- 172.21.15.143:0/2017748761 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd4003530 con 0x7f3bd4004d10 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bf9c00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bd4005190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009fc0 tx=0x7f3bec01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d46a91e9f25002 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0a5c70 con 0x7f3bd4004d10 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec00f040 con 0x7f3bd4004d10 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bf3400640 1 -- 172.21.15.143:0/2017748761 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0ad440 con 0x7f3bd4004d10 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2017748761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 msgr2=0x7f3bd4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2017748761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bd4005190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009fc0 tx=0x7f3bec01cc00 comp rx=0 tx=0).stop 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bf9c00640 1 -- 172.21.15.143:0/2017748761 reap_dead start 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2017748761 shutdown_connections 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2017748761 >> 172.21.15.143:0/2017748761 conn(0x7f3bf406c3f0 msgr2=0x7f3bf406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2017748761 shutdown_connections 2024-01-15T18:11:08.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.025+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2017748761 wait complete. 2024-01-15T18:11:08.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 Processor -- start 2024-01-15T18:11:08.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 -- start start 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bf415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf41f64c0 0x7f3bf41f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bf413f400 0x7f3bf413f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf41f6e80 con 0x7f3bd4004d10 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf413fdc0 con 0x7f3bf413f400 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf415e360 con 0x7f3bf41f64c0 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bf415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bf415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46002/0 (socket says 172.21.15.143:46002) 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 -- 172.21.15.143:0/2952353732 learned_addr learned my addr 172.21.15.143:0/2952353732 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9c00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bf413f400 0x7f3bf413f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf3e00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf41f64c0 0x7f3bf41f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 -- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf41f64c0 msgr2=0x7f3bf41f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf41f64c0 0x7f3bf41f6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 -- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bf413f400 msgr2=0x7f3bf413f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bf413f400 0x7f3bf413f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 -- 172.21.15.143:0/2952353732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf41f7080 con 0x7f3bd4004d10 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf9200640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bf415aa70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3be4007310 tx=0x7f3be4007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400f960 con 0x7f3bd4004d10 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400fb00 con 0x7f3bd4004d10 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400feb0 con 0x7f3bd4004d10 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf41f7360 con 0x7f3bd4004d10 2024-01-15T18:11:08.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.029+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf41386e0 con 0x7f3bd4004d10 2024-01-15T18:11:08.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.033+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf4067500 con 0x7f3bd4004d10 2024-01-15T18:11:08.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.033+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f3be401c410 con 0x7f3bd4004d10 2024-01-15T18:11:08.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.033+0000 7f3bf0c00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3bcc067190 0x7f3bcc069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.033+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3be4082ca0 con 0x7f3bd4004d10 2024-01-15T18:11:08.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.033+0000 7f3bf3e00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3bcc067190 0x7f3bcc069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.033+0000 7f3bf3e00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3bcc067190 0x7f3bcc069650 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3bf415b3c0 tx=0x7f3bdc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.037+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3be4051730 con 0x7f3bd4004d10 2024-01-15T18:11:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.209+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3bf415f2c0 con 0x7f3bcc067190 2024-01-15T18:11:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.209+0000 7f3bf0c00640 1 -- 172.21.15.143:0/2952353732 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3bf415f2c0 con 0x7f3bcc067190 2024-01-15T18:11:08.214 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:11:08.214 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.214 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:11:08.214 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:08.214 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:11:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3bcc067190 msgr2=0x7f3bcc069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3bcc067190 0x7f3bcc069650 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3bf415b3c0 tx=0x7f3bdc009190 comp rx=0 tx=0).stop 2024-01-15T18:11:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 msgr2=0x7f3bf415aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bf415aa70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3be4007310 tx=0x7f3be4007710 comp rx=0 tx=0).stop 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 shutdown_connections 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3bcc067190 0x7f3bcc069650 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3bf413f400 0x7f3bf413f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3bf41f64c0 0x7f3bf41f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 --2- 172.21.15.143:0/2952353732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3bd4004d10 0x7f3bf415aa70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 >> 172.21.15.143:0/2952353732 conn(0x7f3bf406c3f0 msgr2=0x7f3bf406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 shutdown_connections 2024-01-15T18:11:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.213+0000 7f3bfbc00640 1 -- 172.21.15.143:0/2952353732 wait complete. 2024-01-15T18:11:08.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:11:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.369+0000 7f0df1400640 1 Processor -- start 2024-01-15T18:11:08.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0df1400640 1 -- start start 2024-01-15T18:11:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0df1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0df1400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0dec071470 0x7f0dec0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0df1400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec0b3370 con 0x7f0dec06bb70 2024-01-15T18:11:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0df1400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec15d5d0 con 0x7f0dec071470 2024-01-15T18:11:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0df1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec06a800 con 0x7f0dec15d1f0 2024-01-15T18:11:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec06b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42588/0 (socket says 172.21.15.143:42588) 2024-01-15T18:11:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 -- 172.21.15.143:0/3599506524 learned_addr learned my addr 172.21.15.143:0/3599506524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0deb200640 1 --2- 172.21.15.143:0/3599506524 >> v2:172.21.15.204:3300/0 conn(0x7f0dec071470 0x7f0dec0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 -- 172.21.15.143:0/3599506524 >> v2:172.21.15.204:3300/0 conn(0x7f0dec071470 msgr2=0x7f0dec0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0de9400640 1 -- 172.21.15.143:0/3599506524 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119672240 0 0) 0x7f0dec0b3370 con 0x7f0dec06bb70 2024-01-15T18:11:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 --2- 172.21.15.143:0/3599506524 >> v2:172.21.15.204:3300/0 conn(0x7f0dec071470 0x7f0dec0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 -- 172.21.15.143:0/3599506524 >> v1:172.21.15.204:6789/0 conn(0x7f0dec06bb70 legacy=0x7f0dec070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.373+0000 7f0dea800640 1 -- 172.21.15.143:0/3599506524 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec06aae0 con 0x7f0dec15d1f0 2024-01-15T18:11:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0dea800640 1 --2- 172.21.15.143:0/3599506524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec06b630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009870 tx=0x7f0ddc01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edb0e278392bc68 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0de9400640 1 -- 172.21.15.143:0/3599506524 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0aa670 con 0x7f0dec15d1f0 2024-01-15T18:11:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0de9400640 1 -- 172.21.15.143:0/3599506524 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0aa7d0 con 0x7f0dec15d1f0 2024-01-15T18:11:08.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0de9400640 1 -- 172.21.15.143:0/3599506524 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0aab60 con 0x7f0dec15d1f0 2024-01-15T18:11:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 -- 172.21.15.143:0/3599506524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 msgr2=0x7f0dec06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 --2- 172.21.15.143:0/3599506524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec06b630 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009870 tx=0x7f0ddc01c510 comp rx=0 tx=0).stop 2024-01-15T18:11:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 -- 172.21.15.143:0/3599506524 shutdown_connections 2024-01-15T18:11:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 --2- 172.21.15.143:0/3599506524 >> v2:172.21.15.204:3300/0 conn(0x7f0dec071470 0x7f0dec0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 --2- 172.21.15.143:0/3599506524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec06b630 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 -- 172.21.15.143:0/3599506524 >> 172.21.15.143:0/3599506524 conn(0x7f0dec0b1610 msgr2=0x7f0dec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 -- 172.21.15.143:0/3599506524 shutdown_connections 2024-01-15T18:11:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.377+0000 7f0df1400640 1 -- 172.21.15.143:0/3599506524 wait complete. 2024-01-15T18:11:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 Processor -- start 2024-01-15T18:11:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 -- start start 2024-01-15T18:11:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 0x7f0dec14ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec071470 0x7f0dec153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec14a2c0 con 0x7f0dec06bb70 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec154b30 con 0x7f0dec071470 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec15d7d0 con 0x7f0dec15d1f0 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 0x7f0dec14ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 0x7f0dec14ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46012/0 (socket says 172.21.15.143:46012) 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 -- 172.21.15.143:0/1070489592 learned_addr learned my addr 172.21.15.143:0/1070489592 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0deb200640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0de9e00640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec071470 0x7f0dec153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 -- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 msgr2=0x7f0dec149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 -- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec071470 msgr2=0x7f0dec153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec071470 0x7f0dec153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 -- 172.21.15.143:0/1070489592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0094d0 con 0x7f0dec06bb70 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0dea800640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 0x7f0dec14ad40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc01abc0 tx=0x7f0ddc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b3800 con 0x7f0dec06bb70 2024-01-15T18:11:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0b3980 con 0x7f0dec06bb70 2024-01-15T18:11:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0b3d30 con 0x7f0dec06bb70 2024-01-15T18:11:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec14a4a0 con 0x7f0dec06bb70 2024-01-15T18:11:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.381+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec14aa60 con 0x7f0dec06bb70 2024-01-15T18:11:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.385+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec074b00 con 0x7f0dec06bb70 2024-01-15T18:11:08.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.385+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f0ddc0b2380 con 0x7f0dec06bb70 2024-01-15T18:11:08.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.385+0000 7f0de2a00640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0dc4067160 0x7f0dc4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.385+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0ddc11d020 con 0x7f0dec06bb70 2024-01-15T18:11:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.385+0000 7f0de9e00640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0dc4067160 0x7f0dc4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.385+0000 7f0de9e00640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0dc4067160 0x7f0dc4069620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0d0540 tx=0x7f0dd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.389+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0ddc0ebd60 con 0x7f0dec06bb70 2024-01-15T18:11:08.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.561+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0dec160430 con 0x7f0dc4067160 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0de2a00640 1 -- 172.21.15.143:0/1070489592 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+381 (secure 0 0 0) 0x7f0dec160430 con 0x7f0dc4067160 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42257" 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.572 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42254" 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42255" 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42251" 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42252" 2024-01-15T18:11:08.573 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42256" 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone42253" 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:08.574 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0dc4067160 msgr2=0x7f0dc4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0dc4067160 0x7f0dc4069620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0d0540 tx=0x7f0dd4009040 comp rx=0 tx=0).stop 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 msgr2=0x7f0dec14ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 0x7f0dec14ad40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc01abc0 tx=0x7f0ddc01ad50 comp rx=0 tx=0).stop 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 shutdown_connections 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0dc4067160 0x7f0dc4069620 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0dec15d1f0 0x7f0dec149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0dec071470 0x7f0dec153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 --2- 172.21.15.143:0/1070489592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0dec06bb70 0x7f0dec14ad40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 >> 172.21.15.143:0/1070489592 conn(0x7f0dec0b1610 msgr2=0x7f0dec0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 shutdown_connections 2024-01-15T18:11:08.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.569+0000 7f0df1400640 1 -- 172.21.15.143:0/1070489592 wait complete. 2024-01-15T18:11:08.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42257 2024-01-15T18:11:08.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4eaba00640 1 Processor -- start 2024-01-15T18:11:08.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4eaba00640 1 -- start start 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4eaba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4ea40cf910 0x7f4ea40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4eaba00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40cbff0 con 0x7f4ea40b0310 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4eaba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea40d0c50 con 0x7f4ea406bfb0 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4eaba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40cb6c0 con 0x7f4ea40cf910 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4ea3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4ea40b0310 0x7f4ea40cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56892/0 (socket says 172.21.15.143:56892) 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4ea3e00640 1 -- 172.21.15.143:0/2693812154 learned_addr learned my addr 172.21.15.143:0/2693812154 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4ea9a00640 1 --2- 172.21.15.143:0/2693812154 >> v2:172.21.15.204:3300/0 conn(0x7f4ea40cf910 0x7f4ea40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392328160 0 0) 0x7f4ea40cbff0 con 0x7f4ea40b0310 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.733+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e84003530 con 0x7f4ea40b0310 2024-01-15T18:11:08.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 287572526 0 0) 0x7f4e84003530 con 0x7f4ea40b0310 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 >> v2:172.21.15.204:3300/0 conn(0x7f4ea40cf910 msgr2=0x7f4ea40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 --2- 172.21.15.143:0/2693812154 >> v2:172.21.15.204:3300/0 conn(0x7f4ea40cf910 0x7f4ea40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 >> v1:172.21.15.204:6789/0 conn(0x7f4ea406bfb0 legacy=0x7f4ea406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40cb9a0 con 0x7f4ea40b0310 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4e8c001680 con 0x7f4ea40b0310 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea9a00640 1 --2- 172.21.15.143:0/2693812154 >> v2:172.21.15.204:3300/0 conn(0x7f4ea40cf910 0x7f4ea40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 >> v1:172.21.15.143:6789/0 conn(0x7f4ea40b0310 legacy=0x7f4ea40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84003980 0x7f4e84003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e840047d0 0x7f4e84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84005190 0x7f4e8400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40cb6c0 con 0x7f4e840047d0 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8400c130 con 0x7f4e84003980 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8400c2f0 con 0x7f4e84005190 2024-01-15T18:11:08.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e840047d0 0x7f4e84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea9000640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84003980 0x7f4e84003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea9a00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84005190 0x7f4e8400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 -- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84005190 msgr2=0x7f4e8400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e84005190 0x7f4e8400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 -- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84003980 msgr2=0x7f4e84003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e84003980 0x7f4e84003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 -- 172.21.15.143:0/2693812154 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84003530 con 0x7f4e840047d0 2024-01-15T18:11:08.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea9a00640 1 -- 172.21.15.143:0/2693812154 reap_dead start 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e840047d0 0x7f4e84004c50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c210 tx=0x7f4e8c01cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d209b2bbad4cea9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c005ae0 con 0x7f4e840047d0 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c005c40 con 0x7f4e840047d0 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.737+0000 7f4ea3400640 1 -- 172.21.15.143:0/2693812154 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0a59c0 con 0x7f4e840047d0 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- 172.21.15.143:0/2693812154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e840047d0 msgr2=0x7f4e84004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e840047d0 0x7f4e84004c50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c210 tx=0x7f4e8c01cb70 comp rx=0 tx=0).stop 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- 172.21.15.143:0/2693812154 shutdown_connections 2024-01-15T18:11:08.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 --2- 172.21.15.143:0/2693812154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e840047d0 0x7f4e84004c50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- 172.21.15.143:0/2693812154 >> 172.21.15.143:0/2693812154 conn(0x7f4ea40d0ed0 msgr2=0x7f4ea40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:08.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- 172.21.15.143:0/2693812154 shutdown_connections 2024-01-15T18:11:08.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- 172.21.15.143:0/2693812154 wait complete. 2024-01-15T18:11:08.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 Processor -- start 2024-01-15T18:11:08.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- start start 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e840047d0 0x7f4ea4155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 0x7f4ea4158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ea4154870 0x7f4ea40af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4158d40 con 0x7f4ea41570c0 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4156020 con 0x7f4ea4154870 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.741+0000 7f4eaba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea4156300 con 0x7f4e840047d0 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 0x7f4ea4158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 0x7f4ea4158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46028/0 (socket says 172.21.15.143:46028) 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 -- 172.21.15.143:0/1001546240 learned_addr learned my addr 172.21.15.143:0/1001546240 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea9a00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ea4154870 0x7f4ea40af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea9000640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e840047d0 0x7f4ea4155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 -- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e840047d0 msgr2=0x7f4ea4155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e840047d0 0x7f4ea4155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 -- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ea4154870 msgr2=0x7f4ea40af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ea4154870 0x7f4ea40af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 -- 172.21.15.143:0/1001546240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea41565e0 con 0x7f4ea41570c0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea3e00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 0x7f4ea4158800 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c001940 tx=0x7f4e84005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0b4070 con 0x7f4ea41570c0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c001540 con 0x7f4ea41570c0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea41568c0 con 0x7f4ea41570c0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0a59a0 con 0x7f4ea41570c0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea4154d00 con 0x7f4ea41570c0 2024-01-15T18:11:08.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4067500 con 0x7f4ea41570c0 2024-01-15T18:11:08.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.745+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7f4e8c0a5b00 con 0x7f4ea41570c0 2024-01-15T18:11:08.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.749+0000 7f4ea0c00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e7c067160 0x7f4e7c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:08.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.749+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4e8c11c980 con 0x7f4ea41570c0 2024-01-15T18:11:08.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.749+0000 7f4ea9000640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e7c067160 0x7f4e7c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:08.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.749+0000 7f4ea9000640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e7c067160 0x7f4e7c069620 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4e94000c10 tx=0x7f4e94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:08.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.753+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e8c0eb4f0 con 0x7f4ea41570c0 2024-01-15T18:11:08.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.929+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42257", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ea40cf5d0 con 0x7f4e7c067160 2024-01-15T18:11:08.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.941+0000 7f4ea0c00640 1 -- 172.21.15.143:0/1001546240 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4e94001d40 con 0x7f4e7c067160 2024-01-15T18:11:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e7c067160 msgr2=0x7f4e7c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e7c067160 0x7f4e7c069620 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4e94000c10 tx=0x7f4e94009040 comp rx=0 tx=0).stop 2024-01-15T18:11:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 msgr2=0x7f4ea4158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 0x7f4ea4158800 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c001940 tx=0x7f4e84005800 comp rx=0 tx=0).stop 2024-01-15T18:11:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 shutdown_connections 2024-01-15T18:11:08.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e7c067160 0x7f4e7c069620 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4ea4154870 0x7f4ea40af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4ea41570c0 0x7f4ea4158800 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 --2- 172.21.15.143:0/1001546240 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e840047d0 0x7f4ea4155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 >> 172.21.15.143:0/1001546240 conn(0x7f4ea40d0ed0 msgr2=0x7f4ea40adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 shutdown_connections 2024-01-15T18:11:08.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f4eaba00640 1 -- 172.21.15.143:0/1001546240 wait complete. 2024-01-15T18:11:08.950 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.945+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.968 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.965+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.968 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.965+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.968 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.965+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.968 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.965+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.968 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:08.965+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:08.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:11:09.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 Processor -- start 2024-01-15T18:11:09.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 -- start start 2024-01-15T18:11:09.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff870159b70 0x7ff8700b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:09.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff8700b48f0 0x7ff870070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:09.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8700b4d20 con 0x7ff870157300 2024-01-15T18:11:09.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8700b35a0 con 0x7ff870159b70 2024-01-15T18:11:09.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff878800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff870159fd0 con 0x7ff8700b48f0 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff870159b70 0x7ff8700b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff870159b70 0x7ff8700b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46060/0 (socket says 172.21.15.143:46060) 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 -- 172.21.15.143:0/2761652631 learned_addr learned my addr 172.21.15.143:0/2761652631 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff876800640 1 --2- 172.21.15.143:0/2761652631 >> v2:172.21.15.204:3300/0 conn(0x7ff8700b48f0 0x7ff870070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 -- 172.21.15.143:0/2761652631 >> v2:172.21.15.204:3300/0 conn(0x7ff8700b48f0 msgr2=0x7ff870070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 --2- 172.21.15.143:0/2761652631 >> v2:172.21.15.204:3300/0 conn(0x7ff8700b48f0 0x7ff870070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 -- 172.21.15.143:0/2761652631 >> v1:172.21.15.204:6789/0 conn(0x7ff870157300 legacy=0x7ff8700b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:09.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.117+0000 7ff875400640 1 -- 172.21.15.143:0/2761652631 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8700b2a20 con 0x7ff870159b70 2024-01-15T18:11:09.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff875400640 1 --2- 172.21.15.143:0/2761652631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff8700b43b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff8700b1f70 tx=0x7ff85801d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17325cb4f9ffc4c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:09.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff874a00640 1 -- 172.21.15.143:0/2761652631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a6950 con 0x7ff870159b70 2024-01-15T18:11:09.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff874a00640 1 -- 172.21.15.143:0/2761652631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8580a6ab0 con 0x7ff870159b70 2024-01-15T18:11:09.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff874a00640 1 -- 172.21.15.143:0/2761652631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a6e40 con 0x7ff870159b70 2024-01-15T18:11:09.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 -- 172.21.15.143:0/2761652631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 msgr2=0x7ff8700b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:09.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 --2- 172.21.15.143:0/2761652631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff8700b43b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff8700b1f70 tx=0x7ff85801d8a0 comp rx=0 tx=0).stop 2024-01-15T18:11:09.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 -- 172.21.15.143:0/2761652631 shutdown_connections 2024-01-15T18:11:09.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 --2- 172.21.15.143:0/2761652631 >> v2:172.21.15.204:3300/0 conn(0x7ff8700b48f0 0x7ff870070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:09.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 --2- 172.21.15.143:0/2761652631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff8700b43b0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:09.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 -- 172.21.15.143:0/2761652631 >> 172.21.15.143:0/2761652631 conn(0x7ff87006c3f0 msgr2=0x7ff87006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:09.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 -- 172.21.15.143:0/2761652631 shutdown_connections 2024-01-15T18:11:09.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.121+0000 7ff878800640 1 -- 172.21.15.143:0/2761652631 wait complete. 2024-01-15T18:11:09.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 Processor -- start 2024-01-15T18:11:09.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 -- start start 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8700b48f0 0x7ff87014cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff870157300 0x7ff8701364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff870136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87014fc90 con 0x7ff870159b70 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87014f080 con 0x7ff8700b48f0 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87014f360 con 0x7ff870157300 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff870136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff870136fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46076/0 (socket says 172.21.15.143:46076) 2024-01-15T18:11:09.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 -- 172.21.15.143:0/1130232603 learned_addr learned my addr 172.21.15.143:0/1130232603 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:09.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff875e00640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8700b48f0 0x7ff87014cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:09.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff875400640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff870157300 0x7ff8701364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 -- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff870157300 msgr2=0x7ff8701364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff870157300 0x7ff8701364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 -- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8700b48f0 msgr2=0x7ff87014cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8700b48f0 0x7ff87014cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 -- 172.21.15.143:0/1130232603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85800a670 con 0x7ff870159b70 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff875e00640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8700b48f0 0x7ff87014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff876800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff870136fa0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff86800efc0 tx=0x7ff86800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868009b10 con 0x7ff870159b70 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86800f040 con 0x7ff870159b70 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8680146c0 con 0x7ff870159b70 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87014f640 con 0x7ff870159b70 2024-01-15T18:11:09.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.125+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff870136a10 con 0x7ff870159b70 2024-01-15T18:11:09.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.129+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87014fe00 con 0x7ff870159b70 2024-01-15T18:11:09.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.129+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) v1 ==== 86925+0+0 (secure 0 0 0) 0x7ff868014820 con 0x7ff870159b70 2024-01-15T18:11:09.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.129+0000 7ff86e000640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff848067160 0x7ff848069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:09.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.129+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff8680838b0 con 0x7ff870159b70 2024-01-15T18:11:09.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.129+0000 7ff875e00640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff848067160 0x7ff848069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:09.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.129+0000 7ff875e00640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff848067160 0x7ff848069620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff860007fd0 tx=0x7ff860009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:09.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.133+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff868052390 con 0x7ff870159b70 2024-01-15T18:11:09.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:09.309+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff870067500 con 0x7ff848067160 2024-01-15T18:11:11.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:11.789+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7ff868052760 con 0x7ff870159b70 2024-01-15T18:11:12.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.849+0000 7ff86e000640 1 -- 172.21.15.143:0/1130232603 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff870067500 con 0x7ff848067160 2024-01-15T18:11:12.853 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:12.853 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:11:12.853 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:12.853 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:11:12.853 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:12.853 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:11:12.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff848067160 msgr2=0x7ff848069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:12.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff848067160 0x7ff848069620 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff860007fd0 tx=0x7ff860009040 comp rx=0 tx=0).stop 2024-01-15T18:11:12.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 msgr2=0x7ff870136fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:12.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff870136fa0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff86800efc0 tx=0x7ff86800c490 comp rx=0 tx=0).stop 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 shutdown_connections 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff848067160 0x7ff848069620 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff870159b70 0x7ff870136fa0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff870157300 0x7ff8701364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 --2- 172.21.15.143:0/1130232603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8700b48f0 0x7ff87014cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 >> 172.21.15.143:0/1130232603 conn(0x7ff87006c3f0 msgr2=0x7ff8700ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 shutdown_connections 2024-01-15T18:11:12.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:12.853+0000 7ff878800640 1 -- 172.21.15.143:0/1130232603 wait complete. 2024-01-15T18:11:12.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:11:13.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.013+0000 7fdfdc600640 1 Processor -- start 2024-01-15T18:11:13.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.013+0000 7fdfdc600640 1 -- start start 2024-01-15T18:11:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfdc600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdfd40cda70 0x7fdfd40cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfdc600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf400 0x7fdfd40cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfdc600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdfd40cf800 0x7fdfd40b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfdc600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd415b5b0 con 0x7fdfd40cf800 2024-01-15T18:11:13.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfdc600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd40ccd30 con 0x7fdfd40cda70 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfdc600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40cba70 con 0x7fdfd40cf400 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdfd40cf800 0x7fdfd40b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdfd40cf800 0x7fdfd40b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46082/0 (socket says 172.21.15.143:46082) 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 -- 172.21.15.143:0/2793839439 learned_addr learned my addr 172.21.15.143:0/2793839439 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfd9200640 1 --2- 172.21.15.143:0/2793839439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf400 0x7fdfd40cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfd9c00640 1 --2- 172.21.15.143:0/2793839439 >> v2:172.21.15.204:3300/0 conn(0x7fdfd40cda70 0x7fdfd40cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 -- 172.21.15.143:0/2793839439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf400 msgr2=0x7fdfd40cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 --2- 172.21.15.143:0/2793839439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf400 0x7fdfd40cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 -- 172.21.15.143:0/2793839439 >> v2:172.21.15.204:3300/0 conn(0x7fdfd40cda70 msgr2=0x7fdfd40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 --2- 172.21.15.143:0/2793839439 >> v2:172.21.15.204:3300/0 conn(0x7fdfd40cda70 0x7fdfd40cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.017+0000 7fdfda600640 1 -- 172.21.15.143:0/2793839439 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40cbd10 con 0x7fdfd40cf800 2024-01-15T18:11:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfda600640 1 --2- 172.21.15.143:0/2793839439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cf800 0x7fdfd40b0740 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4071a00 tx=0x7fdfcc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a73c06e60ef8ed2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfd3e00640 1 -- 172.21.15.143:0/2793839439 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0a5820 con 0x7fdfd40cf800 2024-01-15T18:11:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfd3e00640 1 -- 172.21.15.143:0/2793839439 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc0a5980 con 0x7fdfd40cf800 2024-01-15T18:11:13.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfd3e00640 1 -- 172.21.15.143:0/2793839439 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0a5d10 con 0x7fdfd40cf800 2024-01-15T18:11:13.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 -- 172.21.15.143:0/2793839439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cf800 msgr2=0x7fdfd40b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2793839439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cf800 0x7fdfd40b0740 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4071a00 tx=0x7fdfcc01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 -- 172.21.15.143:0/2793839439 shutdown_connections 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2793839439 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cf800 0x7fdfd40b0740 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2793839439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf400 0x7fdfd40cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2793839439 >> v2:172.21.15.204:3300/0 conn(0x7fdfd40cda70 0x7fdfd40cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 -- 172.21.15.143:0/2793839439 >> 172.21.15.143:0/2793839439 conn(0x7fdfd40d0e20 msgr2=0x7fdfd40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 -- 172.21.15.143:0/2793839439 shutdown_connections 2024-01-15T18:11:13.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 -- 172.21.15.143:0/2793839439 wait complete. 2024-01-15T18:11:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 Processor -- start 2024-01-15T18:11:13.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.021+0000 7fdfdc600640 1 -- start start 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 0x7fdfd415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfd40cf400 0x7fdfd40b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf800 0x7fdfd415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40c8e80 con 0x7fdfd40cda70 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40c90f0 con 0x7fdfd40cf400 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40c93d0 con 0x7fdfd40cf800 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 0x7fdfd415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 0x7fdfd415b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46096/0 (socket says 172.21.15.143:46096) 2024-01-15T18:11:13.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 -- 172.21.15.143:0/2480617075 learned_addr learned my addr 172.21.15.143:0/2480617075 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9200640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfd40cf400 0x7fdfd40b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfda600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf800 0x7fdfd415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 -- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf800 msgr2=0x7fdfd415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf800 0x7fdfd415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 -- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfd40cf400 msgr2=0x7fdfd40b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfd40cf400 0x7fdfd40b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 -- 172.21.15.143:0/2480617075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc009b30 con 0x7fdfd40cda70 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd9c00640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 0x7fdfd415b790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400e870 tx=0x7fdfc400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400ee60 con 0x7fdfd40cda70 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc400c940 con 0x7fdfd40cda70 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd40c96b0 con 0x7fdfd40cda70 2024-01-15T18:11:13.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400ccf0 con 0x7fdfd40cda70 2024-01-15T18:11:13.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd40b66d0 con 0x7fdfd40cda70 2024-01-15T18:11:13.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd4067500 con 0x7fdfd40cda70 2024-01-15T18:11:13.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.025+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7fdfc4013470 con 0x7fdfd40cda70 2024-01-15T18:11:13.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.029+0000 7fdfd1600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfac067280 0x7fdfac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.029+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdfc4082fa0 con 0x7fdfd40cda70 2024-01-15T18:11:13.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.029+0000 7fdfd9200640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfac067280 0x7fdfac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.029+0000 7fdfd9200640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfac067280 0x7fdfac069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdfd415ea40 tx=0x7fdfbc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.033+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdfc4051a30 con 0x7fdfd40cda70 2024-01-15T18:11:13.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.209+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfd415c000 con 0x7fdfac067280 2024-01-15T18:11:13.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfd1600640 1 -- 172.21.15.143:0/2480617075 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdfbc0119d0 con 0x7fdfac067280 2024-01-15T18:11:13.239 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfac067280 msgr2=0x7fdfac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfac067280 0x7fdfac069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fdfd415ea40 tx=0x7fdfbc009190 comp rx=0 tx=0).stop 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 msgr2=0x7fdfd415b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 0x7fdfd415b790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400e870 tx=0x7fdfc400ed30 comp rx=0 tx=0).stop 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 shutdown_connections 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdfac067280 0x7fdfac069740 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdfd40cf800 0x7fdfd415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdfd40cf400 0x7fdfd40b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 --2- 172.21.15.143:0/2480617075 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdfd40cda70 0x7fdfd415b790 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 >> 172.21.15.143:0/2480617075 conn(0x7fdfd40d0e20 msgr2=0x7fdfd406f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:13.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 shutdown_connections 2024-01-15T18:11:13.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.237+0000 7fdfdc600640 1 -- 172.21.15.143:0/2480617075 wait complete. 2024-01-15T18:11:13.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:11:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9ec00640 1 Processor -- start 2024-01-15T18:11:13.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9ec00640 1 -- start start 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b980cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9ec00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b980ceb50 con 0x7f3b980722c0 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9ec00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b9815c110 con 0x7f3b980ce520 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9815b7e0 con 0x7f3b98143780 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9cc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3b980722c0 0x7f3b9815c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56906/0 (socket says 172.21.15.143:56906) 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b9cc00640 1 -- 172.21.15.143:0/690475383 learned_addr learned my addr 172.21.15.143:0/690475383 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:13.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b97e00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b980cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738957246 0 0) 0x7f3b900014a0 con 0x7f3b980722c0 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.397+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b78005740 con 0x7f3b980722c0 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90183834 0 0) 0x7f3b9815c110 con 0x7f3b980ce520 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b900014a0 con 0x7f3b980ce520 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4291906267 0 0) 0x7f3b78005740 con 0x7f3b980722c0 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 msgr2=0x7f3b980cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b980cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 >> v1:172.21.15.204:6789/0 conn(0x7f3b980ce520 legacy=0x7f3b98071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9815bac0 con 0x7f3b980722c0 2024-01-15T18:11:13.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3b90001b90 con 0x7f3b980722c0 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b980cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 >> v1:172.21.15.143:6789/0 conn(0x7f3b980722c0 legacy=0x7f3b9815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b78006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780069e0 0x7f3b78006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b780073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9815b7e0 con 0x7f3b78005d50 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7800dc50 con 0x7f3b780069e0 2024-01-15T18:11:13.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7800dde0 con 0x7f3b98143780 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b78006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97400640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780069e0 0x7f3b78006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b9cc00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b780073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 -- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 msgr2=0x7f3b780073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98143780 0x7f3b780073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 -- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780069e0 msgr2=0x7f3b78006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b780069e0 0x7f3b78006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 -- 172.21.15.143:0/690475383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88009420 con 0x7f3b78005d50 2024-01-15T18:11:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.401+0000 7f3b97e00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b78006150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009c80 tx=0x7f3b8801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9050a4707b777998 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b880a5740 con 0x7f3b78005d50 2024-01-15T18:11:13.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b880a58a0 con 0x7f3b78005d50 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b96a00640 1 -- 172.21.15.143:0/690475383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b880a5c50 con 0x7f3b78005d50 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- 172.21.15.143:0/690475383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 msgr2=0x7f3b78006150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/690475383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b78006150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009c80 tx=0x7f3b8801cb90 comp rx=0 tx=0).stop 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9cc00640 1 -- 172.21.15.143:0/690475383 reap_dead start 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- 172.21.15.143:0/690475383 shutdown_connections 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- 172.21.15.143:0/690475383 >> 172.21.15.143:0/690475383 conn(0x7f3b9806c3f0 msgr2=0x7f3b9806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- 172.21.15.143:0/690475383 shutdown_connections 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- 172.21.15.143:0/690475383 wait complete. 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 Processor -- start 2024-01-15T18:11:13.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- start start 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b98136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98141bc0 0x7f3b9814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b981595d0 0x7f3b98137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98142050 con 0x7f3b78005d50 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98136190 con 0x7f3b981595d0 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.405+0000 7f3b9ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b98136470 con 0x7f3b98141bc0 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b98136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b98136c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46118/0 (socket says 172.21.15.143:46118) 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 -- 172.21.15.143:0/2373862898 learned_addr learned my addr 172.21.15.143:0/2373862898 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:13.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97400640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98141bc0 0x7f3b9814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b9cc00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b981595d0 0x7f3b98137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 -- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98141bc0 msgr2=0x7f3b9814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98141bc0 0x7f3b9814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 -- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b981595d0 msgr2=0x7f3b98137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b981595d0 0x7f3b98137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 -- 172.21.15.143:0/2373862898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98136750 con 0x7f3b78005d50 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b97e00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b98136c70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009c80 tx=0x7f3b78007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b880be070 con 0x7f3b78005d50 2024-01-15T18:11:13.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8800f000 con 0x7f3b78005d50 2024-01-15T18:11:13.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004240 con 0x7f3b78005d50 2024-01-15T18:11:13.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b981426a0 con 0x7f3b78005d50 2024-01-15T18:11:13.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b98138080 con 0x7f3b78005d50 2024-01-15T18:11:13.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b98067500 con 0x7f3b78005d50 2024-01-15T18:11:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.409+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f3b880b4040 con 0x7f3b78005d50 2024-01-15T18:11:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.413+0000 7f3b8fe00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b700672b0 0x7f3b70069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.413+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3b8811ca90 con 0x7f3b78005d50 2024-01-15T18:11:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.413+0000 7f3b97400640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b700672b0 0x7f3b70069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.413+0000 7f3b97400640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b700672b0 0x7f3b70069770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3b98139540 tx=0x7f3b80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.417+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3b880eb5a0 con 0x7f3b78005d50 2024-01-15T18:11:13.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.589+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b98136010 con 0x7f3b700672b0 2024-01-15T18:11:13.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.605+0000 7f3b8fe00640 1 -- 172.21.15.143:0/2373862898 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3b80005020 con 0x7f3b700672b0 2024-01-15T18:11:13.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b700672b0 msgr2=0x7f3b70069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b700672b0 0x7f3b70069770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3b98139540 tx=0x7f3b80009040 comp rx=0 tx=0).stop 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 msgr2=0x7f3b98136c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b98136c70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009c80 tx=0x7f3b78007a10 comp rx=0 tx=0).stop 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 shutdown_connections 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b700672b0 0x7f3b70069770 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b981595d0 0x7f3b98137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b98141bc0 0x7f3b9814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 --2- 172.21.15.143:0/2373862898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b78005d50 0x7f3b98136c70 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 >> 172.21.15.143:0/2373862898 conn(0x7f3b9806c3f0 msgr2=0x7f3b980aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:13.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 shutdown_connections 2024-01-15T18:11:13.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.609+0000 7f3b9ec00640 1 -- 172.21.15.143:0/2373862898 wait complete. 2024-01-15T18:11:13.637 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42254 2024-01-15T18:11:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 Processor -- start 2024-01-15T18:11:13.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 -- start start 2024-01-15T18:11:13.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d815b260 0x7f85d80cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d80d19f0 0x7f85d80cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f85d80c5220 0x7f85d80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d80c4d90 con 0x7f85d815b260 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d815b640 con 0x7f85d80c5220 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.781+0000 7f85dda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80cf800 con 0x7f85d80d19f0 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d815b260 0x7f85d80cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d815b260 0x7f85d80cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46128/0 (socket says 172.21.15.143:46128) 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 -- 172.21.15.143:0/2355912153 learned_addr learned my addr 172.21.15.143:0/2355912153 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6000640 1 --2- 172.21.15.143:0/2355912153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d80d19f0 0x7f85d80cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d7400640 1 --2- 172.21.15.143:0/2355912153 >> v2:172.21.15.204:3300/0 conn(0x7f85d80c5220 0x7f85d80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 -- 172.21.15.143:0/2355912153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d80d19f0 msgr2=0x7f85d80cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 --2- 172.21.15.143:0/2355912153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d80d19f0 0x7f85d80cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 -- 172.21.15.143:0/2355912153 >> v2:172.21.15.204:3300/0 conn(0x7f85d80c5220 msgr2=0x7f85d80c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:13.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 --2- 172.21.15.143:0/2355912153 >> v2:172.21.15.204:3300/0 conn(0x7f85d80c5220 0x7f85d80c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 -- 172.21.15.143:0/2355912153 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d80cfae0 con 0x7f85d815b260 2024-01-15T18:11:13.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d6a00640 1 --2- 172.21.15.143:0/2355912153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d815b260 0x7f85d80cd810 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f85d8071260 tx=0x7f85c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d886bfc88b1a780c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d5600640 1 -- 172.21.15.143:0/2355912153 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80aa5f0 con 0x7f85d815b260 2024-01-15T18:11:13.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d5600640 1 -- 172.21.15.143:0/2355912153 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80aa750 con 0x7f85d815b260 2024-01-15T18:11:13.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.785+0000 7f85d5600640 1 -- 172.21.15.143:0/2355912153 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80aaae0 con 0x7f85d815b260 2024-01-15T18:11:13.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- 172.21.15.143:0/2355912153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d815b260 msgr2=0x7f85d80cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- 172.21.15.143:0/2355912153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d815b260 0x7f85d80cd810 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f85d8071260 tx=0x7f85c801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- 172.21.15.143:0/2355912153 shutdown_connections 2024-01-15T18:11:13.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- 172.21.15.143:0/2355912153 >> v2:172.21.15.204:3300/0 conn(0x7f85d80c5220 0x7f85d80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- 172.21.15.143:0/2355912153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d80d19f0 0x7f85d80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- 172.21.15.143:0/2355912153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d815b260 0x7f85d80cd810 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- 172.21.15.143:0/2355912153 >> 172.21.15.143:0/2355912153 conn(0x7f85d80d0ed0 msgr2=0x7f85d80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:13.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- 172.21.15.143:0/2355912153 shutdown_connections 2024-01-15T18:11:13.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- 172.21.15.143:0/2355912153 wait complete. 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 Processor -- start 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- start start 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80c5220 0x7f85d815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 0x7f85d8151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d815b260 0x7f85d809ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8158830 con 0x7f85d80d19f0 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d815aac0 con 0x7f85d80c5220 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.789+0000 7f85dda00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d809c200 con 0x7f85d815b260 2024-01-15T18:11:13.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 0x7f85d8151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 0x7f85d8151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46138/0 (socket says 172.21.15.143:46138) 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6000640 1 -- 172.21.15.143:0/1830267911 learned_addr learned my addr 172.21.15.143:0/1830267911 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80c5220 0x7f85d815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d7400640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d815b260 0x7f85d809ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 -- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d815b260 msgr2=0x7f85d809ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d815b260 0x7f85d809ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 -- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 msgr2=0x7f85d8151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 0x7f85d8151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 -- 172.21.15.143:0/1830267911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80094d0 con 0x7f85d80c5220 2024-01-15T18:11:13.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6000640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 0x7f85d8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:13.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85d6a00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80c5220 0x7f85d815a580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ab000 tx=0x7f85c801ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b3800 con 0x7f85d80c5220 2024-01-15T18:11:13.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80b3960 con 0x7f85d80c5220 2024-01-15T18:11:13.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d809bc40 con 0x7f85d80c5220 2024-01-15T18:11:13.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b3d10 con 0x7f85d80c5220 2024-01-15T18:11:13.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.793+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d81584c0 con 0x7f85d80c5220 2024-01-15T18:11:13.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.797+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d80748b0 con 0x7f85d80c5220 2024-01-15T18:11:13.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.797+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f85c80b2380 con 0x7f85d80c5220 2024-01-15T18:11:13.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.797+0000 7f85cea00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067300 0x7f85b00697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:13.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.797+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f85c80a7070 con 0x7f85d80c5220 2024-01-15T18:11:13.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.797+0000 7f85d6000640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067300 0x7f85b00697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:13.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.797+0000 7f85d6000640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067300 0x7f85b00697c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f85d809c670 tx=0x7f85c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:13.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.801+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85c80ebed0 con 0x7f85d80c5220 2024-01-15T18:11:13.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.977+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42254", "target": ["mon-mgr", ""]}) v1 -- 0x7f85d8067500 con 0x7f85b0067300 2024-01-15T18:11:14.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:13.997+0000 7f85cea00640 1 -- 172.21.15.143:0/1830267911 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f85d8067500 con 0x7f85b0067300 2024-01-15T18:11:14.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067300 msgr2=0x7f85b00697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067300 0x7f85b00697c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f85d809c670 tx=0x7f85c0008040 comp rx=0 tx=0).stop 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80c5220 msgr2=0x7f85d815a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80c5220 0x7f85d815a580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ab000 tx=0x7f85c801ad10 comp rx=0 tx=0).stop 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 shutdown_connections 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067300 0x7f85b00697c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d815b260 0x7f85d809ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80d19f0 0x7f85d8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 --2- 172.21.15.143:0/1830267911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80c5220 0x7f85d815a580 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 >> 172.21.15.143:0/1830267911 conn(0x7f85d80d0ed0 msgr2=0x7f85d81570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:14.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 shutdown_connections 2024-01-15T18:11:14.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.001+0000 7f85dda00640 1 -- 172.21.15.143:0/1830267911 wait complete. 2024-01-15T18:11:14.030 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42255 2024-01-15T18:11:14.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 Processor -- start 2024-01-15T18:11:14.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 -- start start 2024-01-15T18:11:14.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0cdf20 0x7fad6c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad6c0c6230 0x7fad6c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c0c4d90 con 0x7fad6c15b260 2024-01-15T18:11:14.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6c15b640 con 0x7fad6c0c6230 2024-01-15T18:11:14.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad71800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0cf800 con 0x7fad6c0cdf20 2024-01-15T18:11:14.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0cdf20 0x7fad6c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad6c0c6230 0x7fad6c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad6c0c6230 0x7fad6c0cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60366/0 (socket says 172.21.15.143:60366) 2024-01-15T18:11:14.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0cdf20 0x7fad6c0cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42742/0 (socket says 172.21.15.143:42742) 2024-01-15T18:11:14.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 -- 172.21.15.143:0/1212118231 learned_addr learned my addr 172.21.15.143:0/1212118231 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:14.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6aa00640 1 --1- 172.21.15.143:0/1212118231 >> v1:172.21.15.204:6789/0 conn(0x7fad6c15b260 0x7fad6c0cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42556/0 (socket says 172.21.15.143:42556) 2024-01-15T18:11:14.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 -- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0cdf20 msgr2=0x7fad6c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:14.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 --2- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0cdf20 0x7fad6c0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 -- 172.21.15.143:0/1212118231 >> v1:172.21.15.204:6789/0 conn(0x7fad6c15b260 legacy=0x7fad6c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.173+0000 7fad6b400640 1 -- 172.21.15.143:0/1212118231 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0cfae0 con 0x7fad6c0c6230 2024-01-15T18:11:14.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad6b400640 1 --2- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0c6230 0x7fad6c0cc020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fad64009e50 tx=0x7fad6401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f84ce5d122b86f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad69600640 1 -- 172.21.15.143:0/1212118231 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640a58a0 con 0x7fad6c0c6230 2024-01-15T18:11:14.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad69600640 1 -- 172.21.15.143:0/1212118231 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad640a5a00 con 0x7fad6c0c6230 2024-01-15T18:11:14.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad69600640 1 -- 172.21.15.143:0/1212118231 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640a5d90 con 0x7fad6c0c6230 2024-01-15T18:11:14.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 -- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0c6230 msgr2=0x7fad6c0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 --2- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0c6230 0x7fad6c0cc020 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fad64009e50 tx=0x7fad6401c9e0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 -- 172.21.15.143:0/1212118231 shutdown_connections 2024-01-15T18:11:14.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 --2- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0c6230 0x7fad6c0cc020 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 --2- 172.21.15.143:0/1212118231 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0cdf20 0x7fad6c0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 -- 172.21.15.143:0/1212118231 >> 172.21.15.143:0/1212118231 conn(0x7fad6c0d0ed0 msgr2=0x7fad6c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:14.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 -- 172.21.15.143:0/1212118231 shutdown_connections 2024-01-15T18:11:14.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.177+0000 7fad71800640 1 -- 172.21.15.143:0/1212118231 wait complete. 2024-01-15T18:11:14.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 Processor -- start 2024-01-15T18:11:14.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 -- start start 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0c6230 0x7fad6c154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0cdf20 0x7fad6c09ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 0x7fad6c09bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c09b1b0 con 0x7fad6c15b260 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0ca240 con 0x7fad6c0cdf20 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0ca4e0 con 0x7fad6c0c6230 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 0x7fad6c09bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 0x7fad6c09bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46148/0 (socket says 172.21.15.143:46148) 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 -- 172.21.15.143:0/1466644119 learned_addr learned my addr 172.21.15.143:0/1466644119 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6a000640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0cdf20 0x7fad6c09ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6aa00640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0c6230 0x7fad6c154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 -- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0c6230 msgr2=0x7fad6c154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0c6230 0x7fad6c154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 -- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0cdf20 msgr2=0x7fad6c09ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0cdf20 0x7fad6c09ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 -- 172.21.15.143:0/1466644119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64009b30 con 0x7fad6c15b260 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad6b400640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 0x7fad6c09bb80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fad640af040 tx=0x7fad64019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640b8670 con 0x7fad6c15b260 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad640b87f0 con 0x7fad6c15b260 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c0ca7c0 con 0x7fad6c15b260 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad640b8ba0 con 0x7fad6c15b260 2024-01-15T18:11:14.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.181+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c0cad80 con 0x7fad6c15b260 2024-01-15T18:11:14.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.185+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c0caf70 con 0x7fad6c15b260 2024-01-15T18:11:14.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.185+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7fad640b8d00 con 0x7fad6c15b260 2024-01-15T18:11:14.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.185+0000 7fad62a00640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44067280 0x7fad44069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.185+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fad6411eed0 con 0x7fad6c15b260 2024-01-15T18:11:14.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.185+0000 7fad6aa00640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44067280 0x7fad44069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.185+0000 7fad6aa00640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44067280 0x7fad44069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad5c000f70 tx=0x7fad5c007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.189+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad640eda10 con 0x7fad6c15b260 2024-01-15T18:11:14.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.361+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42255", "target": ["mon-mgr", ""]}) v1 -- 0x7fad6c0cd810 con 0x7fad44067280 2024-01-15T18:11:14.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad62a00640 1 -- 172.21.15.143:0/1466644119 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fad6c0cd810 con 0x7fad44067280 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44067280 msgr2=0x7fad44069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44067280 0x7fad44069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad5c000f70 tx=0x7fad5c007e50 comp rx=0 tx=0).stop 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 msgr2=0x7fad6c09bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 0x7fad6c09bb80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fad640af040 tx=0x7fad64019780 comp rx=0 tx=0).stop 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 shutdown_connections 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad44067280 0x7fad44069740 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad6c15b260 0x7fad6c09bb80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad6c0cdf20 0x7fad6c09ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 --2- 172.21.15.143:0/1466644119 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad6c0c6230 0x7fad6c154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 >> 172.21.15.143:0/1466644119 conn(0x7fad6c0d0ed0 msgr2=0x7fad6c0d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 shutdown_connections 2024-01-15T18:11:14.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.381+0000 7fad71800640 1 -- 172.21.15.143:0/1466644119 wait complete. 2024-01-15T18:11:14.409 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42251 2024-01-15T18:11:14.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.545+0000 7f71a1a00640 1 Processor -- start 2024-01-15T18:11:14.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.545+0000 7f71a1a00640 1 -- start start 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f71a1a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f719c159b70 0x7f719c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f71a1a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c0b4d20 con 0x7f719c157300 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f71a1a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c0b2a20 con 0x7f719c0b48f0 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f71a1a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c0b35a0 con 0x7f719c159b70 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f719c159b70 0x7f719c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f719c157300 0x7f719c0b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56922/0 (socket says 172.21.15.143:56922) 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719a000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f719c159b70 0x7f719c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46162/0 (socket says 172.21.15.143:46162) 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719aa00640 1 -- 172.21.15.143:0/1659887536 learned_addr learned my addr 172.21.15.143:0/1659887536 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719a000640 1 -- 172.21.15.143:0/1659887536 >> v1:172.21.15.204:6789/0 conn(0x7f719c0b48f0 legacy=0x7f719c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719a000640 1 -- 172.21.15.143:0/1659887536 >> v1:172.21.15.143:6789/0 conn(0x7f719c157300 legacy=0x7f719c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f719a000640 1 -- 172.21.15.143:0/1659887536 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0b2d00 con 0x7f719c159b70 2024-01-15T18:11:14.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.549+0000 7f7199600640 1 -- 172.21.15.143:0/1659887536 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667867171 0 0) 0x7f719c0b4d20 con 0x7f719c157300 2024-01-15T18:11:14.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f719a000640 1 --2- 172.21.15.143:0/1659887536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c159b70 0x7f719c0b43b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f71840097c0 tx=0x7f718401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f796777790ad373 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f7199600640 1 -- 172.21.15.143:0/1659887536 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840ab650 con 0x7f719c159b70 2024-01-15T18:11:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f7199600640 1 -- 172.21.15.143:0/1659887536 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71840ab7b0 con 0x7f719c159b70 2024-01-15T18:11:14.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f7199600640 1 -- 172.21.15.143:0/1659887536 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840abb40 con 0x7f719c159b70 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 -- 172.21.15.143:0/1659887536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c159b70 msgr2=0x7f719c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1659887536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c159b70 0x7f719c0b43b0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f71840097c0 tx=0x7f718401c450 comp rx=0 tx=0).stop 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 -- 172.21.15.143:0/1659887536 shutdown_connections 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1659887536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c159b70 0x7f719c0b43b0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 -- 172.21.15.143:0/1659887536 >> 172.21.15.143:0/1659887536 conn(0x7f719c06c3f0 msgr2=0x7f719c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 -- 172.21.15.143:0/1659887536 shutdown_connections 2024-01-15T18:11:14.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.553+0000 7f71a1a00640 1 -- 172.21.15.143:0/1659887536 wait complete. 2024-01-15T18:11:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 Processor -- start 2024-01-15T18:11:14.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- start start 2024-01-15T18:11:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c0b48f0 0x7f719c159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 0x7f719c145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f719c159b70 0x7f719c14e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c159fd0 con 0x7f719c157300 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c153ae0 con 0x7f719c0b48f0 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c14f940 con 0x7f719c159b70 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 0x7f719c145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 0x7f719c145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46174/0 (socket says 172.21.15.143:46174) 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 -- 172.21.15.143:0/1961013095 learned_addr learned my addr 172.21.15.143:0/1961013095 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719aa00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c0b48f0 0x7f719c159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719b400640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f719c159b70 0x7f719c14e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 -- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f719c159b70 msgr2=0x7f719c14e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f719c159b70 0x7f719c14e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 -- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c0b48f0 msgr2=0x7f719c159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c0b48f0 0x7f719c159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 -- 172.21.15.143:0/1961013095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7184009420 con 0x7f719c157300 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f719a000640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 0x7f719c145150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f718401a080 tx=0x7f718401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840b5810 con 0x7f719c157300 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71840b5990 con 0x7f719c157300 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840b5d40 con 0x7f719c157300 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c145690 con 0x7f719c157300 2024-01-15T18:11:14.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c14ed30 con 0x7f719c157300 2024-01-15T18:11:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.557+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c0b4100 con 0x7f719c157300 2024-01-15T18:11:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.561+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f71840c93f0 con 0x7f719c157300 2024-01-15T18:11:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.561+0000 7f7192a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7174067280 0x7f7174069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.561+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f718411e2b0 con 0x7f719c157300 2024-01-15T18:11:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.561+0000 7f719aa00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7174067280 0x7f7174069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.561+0000 7f719aa00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7174067280 0x7f7174069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f718c000f70 tx=0x7f718c007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.565+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f71840ecf10 con 0x7f719c157300 2024-01-15T18:11:14.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.737+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42251", "target": ["mon-mgr", ""]}) v1 -- 0x7f719c067500 con 0x7f7174067280 2024-01-15T18:11:14.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f7192a00640 1 -- 172.21.15.143:0/1961013095 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f719c067500 con 0x7f7174067280 2024-01-15T18:11:14.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7174067280 msgr2=0x7f7174069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7174067280 0x7f7174069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f718c000f70 tx=0x7f718c007240 comp rx=0 tx=0).stop 2024-01-15T18:11:14.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 msgr2=0x7f719c145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 0x7f719c145150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f718401a080 tx=0x7f718401ac90 comp rx=0 tx=0).stop 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 shutdown_connections 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7174067280 0x7f7174069740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f719c159b70 0x7f719c14e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f719c157300 0x7f719c145150 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 --2- 172.21.15.143:0/1961013095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f719c0b48f0 0x7f719c159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 >> 172.21.15.143:0/1961013095 conn(0x7f719c06c3f0 msgr2=0x7f719c0afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 shutdown_connections 2024-01-15T18:11:14.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.753+0000 7f71a1a00640 1 -- 172.21.15.143:0/1961013095 wait complete. 2024-01-15T18:11:14.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42252 2024-01-15T18:11:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b7e000640 1 Processor -- start 2024-01-15T18:11:14.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b7e000640 1 -- start start 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b7e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780cdf20 0x7f5b780cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b7e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b780c4d90 con 0x7f5b780c6230 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b7e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7815b640 con 0x7f5b7815b260 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b7e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780cf800 con 0x7f5b780cdf20 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b77e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5b780c6230 0x7f5b780cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56924/0 (socket says 172.21.15.143:56924) 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b77e00640 1 -- 172.21.15.143:0/4044039646 learned_addr learned my addr 172.21.15.143:0/4044039646 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:14.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76a00640 1 --2- 172.21.15.143:0/4044039646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780cdf20 0x7f5b780cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4140732443 0 0) 0x7f5b780c4d90 con 0x7f5b780c6230 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b58003530 con 0x7f5b780c6230 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356169772 0 0) 0x7f5b7815b640 con 0x7f5b7815b260 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b780c4d90 con 0x7f5b7815b260 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76a00640 1 -- 172.21.15.143:0/4044039646 >> v1:172.21.15.204:6789/0 conn(0x7f5b7815b260 legacy=0x7f5b780cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76a00640 1 -- 172.21.15.143:0/4044039646 >> v1:172.21.15.143:6789/0 conn(0x7f5b780c6230 legacy=0x7f5b780cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.925+0000 7f5b76a00640 1 -- 172.21.15.143:0/4044039646 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b780cfae0 con 0x7f5b780cdf20 2024-01-15T18:11:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b76a00640 1 --2- 172.21.15.143:0/4044039646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780cdf20 0x7f5b780cbae0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b600097f0 tx=0x7f5b6001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe374bb5c7bddb82 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b600aa5f0 con 0x7f5b780cdf20 2024-01-15T18:11:14.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b780ccf40 con 0x7f5b780cdf20 2024-01-15T18:11:14.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b76000640 1 -- 172.21.15.143:0/4044039646 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b600aa970 con 0x7f5b780cdf20 2024-01-15T18:11:14.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 -- 172.21.15.143:0/4044039646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780cdf20 msgr2=0x7f5b780cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 --2- 172.21.15.143:0/4044039646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780cdf20 0x7f5b780cbae0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5b600097f0 tx=0x7f5b6001c490 comp rx=0 tx=0).stop 2024-01-15T18:11:14.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 -- 172.21.15.143:0/4044039646 shutdown_connections 2024-01-15T18:11:14.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 --2- 172.21.15.143:0/4044039646 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780cdf20 0x7f5b780cbae0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 -- 172.21.15.143:0/4044039646 >> 172.21.15.143:0/4044039646 conn(0x7f5b780d0ed0 msgr2=0x7f5b780d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:14.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 -- 172.21.15.143:0/4044039646 shutdown_connections 2024-01-15T18:11:14.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.929+0000 7f5b7e000640 1 -- 172.21.15.143:0/4044039646 wait complete. 2024-01-15T18:11:14.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 Processor -- start 2024-01-15T18:11:14.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 -- start start 2024-01-15T18:11:14.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780c6230 0x7f5b7814b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b780cdf20 0x7f5b78157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 0x7f5b78157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780b66d0 con 0x7f5b7815b260 2024-01-15T18:11:14.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7814b7b0 con 0x7f5b780cdf20 2024-01-15T18:11:14.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b7e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b780b5b00 con 0x7f5b780c6230 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 0x7f5b78157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 0x7f5b78157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46176/0 (socket says 172.21.15.143:46176) 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 -- 172.21.15.143:0/3125310507 learned_addr learned my addr 172.21.15.143:0/3125310507 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b76a00640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b780cdf20 0x7f5b78157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77400640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780c6230 0x7f5b7814b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 -- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780c6230 msgr2=0x7f5b7814b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780c6230 0x7f5b7814b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 -- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b780cdf20 msgr2=0x7f5b78157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b780cdf20 0x7f5b78157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:14.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 -- 172.21.15.143:0/3125310507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b60009420 con 0x7f5b7815b260 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77400640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780c6230 0x7f5b7814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b77e00640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 0x7f5b78157f60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000cac0 tx=0x7f5b7000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.933+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000ac30 con 0x7f5b7815b260 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7000ad90 con 0x7f5b7815b260 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7814bc60 con 0x7f5b7815b260 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70012620 con 0x7f5b7815b260 2024-01-15T18:11:14.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b780b5d30 con 0x7f5b7815b260 2024-01-15T18:11:14.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b780b6810 con 0x7f5b7815b260 2024-01-15T18:11:14.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f5b70007070 con 0x7f5b7815b260 2024-01-15T18:11:14.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b6f400640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b50067280 0x7f5b50069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:14.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5b70082e70 con 0x7f5b7815b260 2024-01-15T18:11:14.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.937+0000 7f5b77400640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b50067280 0x7f5b50069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:14.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.941+0000 7f5b77400640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b50067280 0x7f5b50069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5b680011f0 tx=0x7f5b68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:14.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:14.941+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5b780b6810 con 0x7f5b7815b260 2024-01-15T18:11:15.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.117+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42252", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b780b6810 con 0x7f5b50067280 2024-01-15T18:11:15.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b6f400640 1 -- 172.21.15.143:0/3125310507 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5b780b6810 con 0x7f5b50067280 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b50067280 msgr2=0x7f5b50069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b50067280 0x7f5b50069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5b680011f0 tx=0x7f5b68009040 comp rx=0 tx=0).stop 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 msgr2=0x7f5b78157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 0x7f5b78157f60 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000cac0 tx=0x7f5b7000cf80 comp rx=0 tx=0).stop 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 shutdown_connections 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5b50067280 0x7f5b50069740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b7815b260 0x7f5b78157f60 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5b780cdf20 0x7f5b78157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 --2- 172.21.15.143:0/3125310507 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5b780c6230 0x7f5b7814b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 >> 172.21.15.143:0/3125310507 conn(0x7f5b780d0ed0 msgr2=0x7f5b780cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:15.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 shutdown_connections 2024-01-15T18:11:15.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.133+0000 7f5b7e000640 1 -- 172.21.15.143:0/3125310507 wait complete. 2024-01-15T18:11:15.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42256 2024-01-15T18:11:15.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.297+0000 7fe6c8400640 1 Processor -- start 2024-01-15T18:11:15.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.297+0000 7fe6c8400640 1 -- start start 2024-01-15T18:11:15.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c8400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00b24b0 0x7fe6c00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c8400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6c00b3bf0 0x7fe6c00b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c8400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6c0070bf0 con 0x7fe6c00ce690 2024-01-15T18:11:15.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c8400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c015dba0 con 0x7fe6c00b3bf0 2024-01-15T18:11:15.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c8400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c00b4920 con 0x7fe6c00b24b0 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c6400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe6c00b3bf0 0x7fe6c00b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c5a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe6c00ce690 0x7fe6c00b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56940/0 (socket says 172.21.15.143:56940) 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c5a00640 1 -- 172.21.15.143:0/4259692017 learned_addr learned my addr 172.21.15.143:0/4259692017 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c5000640 1 --2- 172.21.15.143:0/4259692017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00b24b0 0x7fe6c00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c6400640 1 -- 172.21.15.143:0/4259692017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00b24b0 msgr2=0x7fe6c00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c6400640 1 --2- 172.21.15.143:0/4259692017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00b24b0 0x7fe6c00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c6400640 1 -- 172.21.15.143:0/4259692017 >> v1:172.21.15.143:6789/0 conn(0x7fe6c00ce690 legacy=0x7fe6c00b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6c6400640 1 -- 172.21.15.143:0/4259692017 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c00b4c00 con 0x7fe6c00b3bf0 2024-01-15T18:11:15.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.301+0000 7fe6bfe00640 1 -- 172.21.15.143:0/4259692017 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1189806053 0 0) 0x7fe6c0070bf0 con 0x7fe6c00ce690 2024-01-15T18:11:15.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c6400640 1 --2- 172.21.15.143:0/4259692017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c00b4080 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009d90 tx=0x7fe6b801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861173452f09f602 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:15.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6bfe00640 1 -- 172.21.15.143:0/4259692017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80a5a00 con 0x7fe6c00b3bf0 2024-01-15T18:11:15.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6bfe00640 1 -- 172.21.15.143:0/4259692017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b80a5b60 con 0x7fe6c00b3bf0 2024-01-15T18:11:15.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6bfe00640 1 -- 172.21.15.143:0/4259692017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b80ad440 con 0x7fe6c00b3bf0 2024-01-15T18:11:15.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 -- 172.21.15.143:0/4259692017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b3bf0 msgr2=0x7fe6c00b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4259692017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c00b4080 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8009d90 tx=0x7fe6b801d380 comp rx=0 tx=0).stop 2024-01-15T18:11:15.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 -- 172.21.15.143:0/4259692017 shutdown_connections 2024-01-15T18:11:15.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4259692017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c00b4080 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4259692017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00b24b0 0x7fe6c00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 -- 172.21.15.143:0/4259692017 >> 172.21.15.143:0/4259692017 conn(0x7fe6c006c3f0 msgr2=0x7fe6c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:15.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 -- 172.21.15.143:0/4259692017 shutdown_connections 2024-01-15T18:11:15.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 -- 172.21.15.143:0/4259692017 wait complete. 2024-01-15T18:11:15.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 Processor -- start 2024-01-15T18:11:15.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.305+0000 7fe6c8400640 1 -- start start 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 0x7fe6c0146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c0159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00ce690 0x7fe6c0140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01628d0 con 0x7fe6c00b24b0 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c0146d80 con 0x7fe6c00b3bf0 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c01457a0 con 0x7fe6c00ce690 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 0x7fe6c0146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 0x7fe6c0146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46196/0 (socket says 172.21.15.143:46196) 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 -- 172.21.15.143:0/4009804665 learned_addr learned my addr 172.21.15.143:0/4009804665 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5000640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c0159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c6400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00ce690 0x7fe6c0140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 -- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00ce690 msgr2=0x7fe6c0140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00ce690 0x7fe6c0140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 -- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6c00b3bf0 msgr2=0x7fe6c0159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c0159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 -- 172.21.15.143:0/4009804665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b8009a70 con 0x7fe6c00b24b0 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c6400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00ce690 0x7fe6c0140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c5a00640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 0x7fe6c0146840 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000c480 tx=0x7fe6b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00074e0 con 0x7fe6c00b24b0 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0007660 con 0x7fe6c00b24b0 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c0145a80 con 0x7fe6c00b24b0 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0004a30 con 0x7fe6c00b24b0 2024-01-15T18:11:15.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c0146040 con 0x7fe6c00b24b0 2024-01-15T18:11:15.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.309+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c0067500 con 0x7fe6c00b24b0 2024-01-15T18:11:15.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.313+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7fe6b0004b90 con 0x7fe6c00b24b0 2024-01-15T18:11:15.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.313+0000 7fe6bd600640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe698067280 0x7fe698069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.313+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe6b0082420 con 0x7fe6c00b24b0 2024-01-15T18:11:15.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.313+0000 7fe6c5000640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe698067280 0x7fe698069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.313+0000 7fe6c5000640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe698067280 0x7fe698069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0146f90 tx=0x7fe6a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.317+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe6b00549a0 con 0x7fe6c00b24b0 2024-01-15T18:11:15.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.493+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42256", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6c006e260 con 0x7fe698067280 2024-01-15T18:11:15.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.505+0000 7fe6bd600640 1 -- 172.21.15.143:0/4009804665 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe6c006e260 con 0x7fe698067280 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe698067280 msgr2=0x7fe698069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe698067280 0x7fe698069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0146f90 tx=0x7fe6a8008040 comp rx=0 tx=0).stop 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 msgr2=0x7fe6c0146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 0x7fe6c0146840 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000c480 tx=0x7fe6b000c940 comp rx=0 tx=0).stop 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 shutdown_connections 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe698067280 0x7fe698069740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe6c00ce690 0x7fe6c0140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe6c00b3bf0 0x7fe6c0159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 --2- 172.21.15.143:0/4009804665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6c00b24b0 0x7fe6c0146840 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 >> 172.21.15.143:0/4009804665 conn(0x7fe6c006c3f0 msgr2=0x7fe6c015a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 shutdown_connections 2024-01-15T18:11:15.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.509+0000 7fe6c8400640 1 -- 172.21.15.143:0/4009804665 wait complete. 2024-01-15T18:11:15.535 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone42253 2024-01-15T18:11:15.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 Processor -- start 2024-01-15T18:11:15.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 -- start start 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c580cfb30 0x7f6c5806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c580cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c580cffd0 con 0x7f6c581607c0 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580d1ce0 con 0x7f6c580cfb30 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580d02b0 con 0x7f6c580cbf50 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c57400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6c580cfb30 0x7f6c5806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c57e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6c581607c0 0x7f6c5806b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56946/0 (socket says 172.21.15.143:56946) 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c57e00640 1 -- 172.21.15.143:0/850533098 learned_addr learned my addr 172.21.15.143:0/850533098 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.673+0000 7f6c5ce00640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c580cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c56a00640 1 -- 172.21.15.143:0/850533098 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444538857 0 0) 0x7f6c580cffd0 con 0x7f6c581607c0 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c56a00640 1 -- 172.21.15.143:0/850533098 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c38003530 con 0x7f6c581607c0 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c57400640 1 -- 172.21.15.143:0/850533098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 msgr2=0x7f6c580cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:15.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c57400640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c580cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c57400640 1 -- 172.21.15.143:0/850533098 >> v1:172.21.15.143:6789/0 conn(0x7f6c581607c0 legacy=0x7f6c5806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c57400640 1 -- 172.21.15.143:0/850533098 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580cb470 con 0x7f6c580cfb30 2024-01-15T18:11:15.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c5ce00640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c580cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:15.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c57400640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c5806bec0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6c400097c0 tx=0x7f6c4001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6cf614636d57ea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:15.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c56a00640 1 -- 172.21.15.143:0/850533098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400a8970 con 0x7f6c580cfb30 2024-01-15T18:11:15.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c56a00640 1 -- 172.21.15.143:0/850533098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400a8ad0 con 0x7f6c580cfb30 2024-01-15T18:11:15.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c56a00640 1 -- 172.21.15.143:0/850533098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400a8e60 con 0x7f6c580cfb30 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c5ee00640 1 -- 172.21.15.143:0/850533098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 msgr2=0x7f6c5806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.677+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c5806bec0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6c400097c0 tx=0x7f6c4001ec70 comp rx=0 tx=0).stop 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- 172.21.15.143:0/850533098 shutdown_connections 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c580cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/850533098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c5806bec0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- 172.21.15.143:0/850533098 >> 172.21.15.143:0/850533098 conn(0x7f6c580d0e20 msgr2=0x7f6c580d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- 172.21.15.143:0/850533098 shutdown_connections 2024-01-15T18:11:15.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- 172.21.15.143:0/850533098 wait complete. 2024-01-15T18:11:15.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 Processor -- start 2024-01-15T18:11:15.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- start start 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c58154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c58150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c581607c0 0x7f6c58156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580c8ca0 con 0x7f6c580cfb30 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c58151510 con 0x7f6c581607c0 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580c8090 con 0x7f6c580cbf50 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c58150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c58150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46204/0 (socket says 172.21.15.143:46204) 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 -- 172.21.15.143:0/2186563209 learned_addr learned my addr 172.21.15.143:0/2186563209 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:15.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c5ce00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c581607c0 0x7f6c58156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57e00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c58154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 -- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 msgr2=0x7f6c58154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c58154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 -- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c581607c0 msgr2=0x7f6c58156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c581607c0 0x7f6c58156960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.681+0000 7f6c57400640 1 -- 172.21.15.143:0/2186563209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c40009420 con 0x7f6c580cfb30 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c57400640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c58150fd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6c400b3040 tx=0x7f6c4001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400b9630 con 0x7f6c580cfb30 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400b97b0 con 0x7f6c580cfb30 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c580c8370 con 0x7f6c580cfb30 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400b9b60 con 0x7f6c580cfb30 2024-01-15T18:11:15.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c580c8930 con 0x7f6c580cfb30 2024-01-15T18:11:15.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5806d4b0 con 0x7f6c580cfb30 2024-01-15T18:11:15.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f6c400b7040 con 0x7f6c580cfb30 2024-01-15T18:11:15.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c4fe00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c30067280 0x7f6c30069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:15.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6c40120db0 con 0x7f6c580cfb30 2024-01-15T18:11:15.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.685+0000 7f6c57e00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c30067280 0x7f6c30069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:15.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.689+0000 7f6c57e00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c30067280 0x7f6c30069740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6c480011f0 tx=0x7f6c48007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:15.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.689+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6c400efa10 con 0x7f6c580cfb30 2024-01-15T18:11:15.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.869+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone42253", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c580ce6b0 con 0x7f6c30067280 2024-01-15T18:11:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.881+0000 7f6c4fe00640 1 -- 172.21.15.143:0/2186563209 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6c48005860 con 0x7f6c30067280 2024-01-15T18:11:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c30067280 msgr2=0x7f6c30069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c30067280 0x7f6c30069740 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6c480011f0 tx=0x7f6c48007d40 comp rx=0 tx=0).stop 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 msgr2=0x7f6c58150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c58150fd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6c400b3040 tx=0x7f6c4001ba10 comp rx=0 tx=0).stop 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 shutdown_connections 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6c30067280 0x7f6c30069740 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6c581607c0 0x7f6c58156960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6c580cfb30 0x7f6c58150fd0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 --2- 172.21.15.143:0/2186563209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6c580cbf50 0x7f6c58154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 >> 172.21.15.143:0/2186563209 conn(0x7f6c580d0e20 msgr2=0x7f6c580cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 shutdown_connections 2024-01-15T18:11:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:15.885+0000 7f6c5ee00640 1 -- 172.21.15.143:0/2186563209 wait complete. 2024-01-15T18:11:15.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:15.912 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:11:15.943 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-15T18:11:15.945 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:11:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.237+0000 7f21b3c00640 1 Processor -- start 2024-01-15T18:11:16.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.237+0000 7f21b3c00640 1 -- start start 2024-01-15T18:11:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.237+0000 7f21b3c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce110 0x7f21ac140e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.237+0000 7f21b3c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac071d20 con 0x7f21ac140600 2024-01-15T18:11:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.237+0000 7f21b3c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac0ce750 con 0x7f21ac1409e0 2024-01-15T18:11:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.237+0000 7f21b3c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0ced10 con 0x7f21ac0ce110 2024-01-15T18:11:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f21ac1409e0 0x7f21ac0cda00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42588/0 (socket says 172.21.15.143:42588) 2024-01-15T18:11:16.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 -- 172.21.15.143:0/1978737259 learned_addr learned my addr 172.21.15.143:0/1978737259 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21b1c00640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce110 0x7f21ac140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588878035 0 0) 0x7f21ac0ce750 con 0x7f21ac1409e0 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f218c014540 con 0x7f21ac1409e0 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5022625 0 0) 0x7f21ac071d20 con 0x7f21ac140600 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ac0ce750 con 0x7f21ac140600 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 802525927 0 0) 0x7f218c014540 con 0x7f21ac1409e0 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce110 msgr2=0x7f21ac140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce110 0x7f21ac140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 >> v1:172.21.15.143:6789/0 conn(0x7f21ac140600 legacy=0x7f21ac13df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0cea30 con 0x7f21ac1409e0 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f2194001680 con 0x7f21ac1409e0 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 >> v1:172.21.15.204:6789/0 conn(0x7f21ac1409e0 legacy=0x7f21ac0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:16.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c014b20 0x7f218c014f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f218c0157e0 0x7f218c015c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f218c0161a0 0x7f218c01c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0ced10 con 0x7f218c014b20 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c01cf90 con 0x7f218c0161a0 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c01d120 con 0x7f218c0157e0 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f218c0157e0 0x7f218c015c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:16.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 -- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f218c0161a0 msgr2=0x7f218c01c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:16.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f218c0161a0 0x7f218c01c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:16.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 -- 172.21.15.143:0/1978737259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c014b20 msgr2=0x7f218c014f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:16.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c014b20 0x7f218c014f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:16.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.241+0000 7f21abe00640 1 -- 172.21.15.143:0/1978737259 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0cea30 con 0x7f218c0157e0 2024-01-15T18:11:16.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.245+0000 7f21abe00640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f218c0157e0 0x7f218c015c60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f219400c210 tx=0x7f219401cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df002ef40e182569 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.245+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194005ae0 con 0x7f218c0157e0 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.245+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194005c40 con 0x7f218c0157e0 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.245+0000 7f21ab400640 1 -- 172.21.15.143:0/1978737259 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194007070 con 0x7f218c0157e0 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b3c00640 1 -- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f218c0157e0 msgr2=0x7f218c015c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b3c00640 1 --2- 172.21.15.143:0/1978737259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f218c0157e0 0x7f218c015c60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f219400c210 tx=0x7f219401cbc0 comp rx=0 tx=0).stop 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b1c00640 1 -- 172.21.15.143:0/1978737259 reap_dead start 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b3c00640 1 -- 172.21.15.143:0/1978737259 shutdown_connections 2024-01-15T18:11:16.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b3c00640 1 -- 172.21.15.143:0/1978737259 >> 172.21.15.143:0/1978737259 conn(0x7f21ac0b0bf0 msgr2=0x7f21ac0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b3c00640 1 -- 172.21.15.143:0/1978737259 shutdown_connections 2024-01-15T18:11:16.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.249+0000 7f21b3c00640 1 -- 172.21.15.143:0/1978737259 wait complete. 2024-01-15T18:11:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 Processor -- start 2024-01-15T18:11:16.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 -- start start 2024-01-15T18:11:16.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c0157e0 0x7f21ac1f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 0x7f21ac0a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac15ad00 0x7f21ac0a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac159bc0 con 0x7f218c0157e0 2024-01-15T18:11:16.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1f1870 con 0x7f21ac15a880 2024-01-15T18:11:16.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.253+0000 7f21b3c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0aa200 con 0x7f21ac15ad00 2024-01-15T18:11:16.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 0x7f21ac0a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:16.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 0x7f21ac0a9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60414/0 (socket says 172.21.15.143:60414) 2024-01-15T18:11:16.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 -- 172.21.15.143:0/3467759321 learned_addr learned my addr 172.21.15.143:0/3467759321 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:16.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21b1c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac15ad00 0x7f21ac0a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:16.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 -- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac15ad00 msgr2=0x7f21ac0a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:16.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac15ad00 0x7f21ac0a9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:16.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 -- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c0157e0 msgr2=0x7f21ac1f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:16.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c0157e0 0x7f21ac1f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:16.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 -- 172.21.15.143:0/3467759321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0a94a0 con 0x7f21ac15a880 2024-01-15T18:11:16.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21b1c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac15ad00 0x7f21ac0a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:16.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21abe00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 0x7f21ac0a9780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2194005fd0 tx=0x7f218c016810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:16.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940b4070 con 0x7f21ac15a880 2024-01-15T18:11:16.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194005e30 con 0x7f21ac15a880 2024-01-15T18:11:16.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940070a0 con 0x7f21ac15a880 2024-01-15T18:11:16.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac159da0 con 0x7f21ac15a880 2024-01-15T18:11:16.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.257+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac15a360 con 0x7f21ac15a880 2024-01-15T18:11:16.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.261+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac15a550 con 0x7f21ac15a880 2024-01-15T18:11:16.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.261+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 20) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f2194007200 con 0x7f21ac15a880 2024-01-15T18:11:16.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.265+0000 7f21a8c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067300 0x7f21840697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:16.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.265+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f219411cc00 con 0x7f21ac15a880 2024-01-15T18:11:16.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.265+0000 7f21b1200640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067300 0x7f21840697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:16.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.265+0000 7f21b1200640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067300 0x7f21840697c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21ac158cd0 tx=0x7f21a4007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:16.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.269+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21940eb690 con 0x7f21ac15a880 2024-01-15T18:11:16.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:16.509+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f21ac067680 con 0x7f2184067300 2024-01-15T18:11:16.734 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:11:16.729+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:11:16.823 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:11:17.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:17.749+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f21940eba60 con 0x7f21ac15a880 2024-01-15T18:11:19.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.757+0000 7f21a8c00640 1 -- 172.21.15.143:0/3467759321 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f21ac067680 con 0x7f2184067300 2024-01-15T18:11:19.763 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:11:19.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067300 msgr2=0x7f21840697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:19.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067300 0x7f21840697c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f21ac158cd0 tx=0x7f21a4007ce0 comp rx=0 tx=0).stop 2024-01-15T18:11:19.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 msgr2=0x7f21ac0a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:19.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 0x7f21ac0a9780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2194005fd0 tx=0x7f218c016810 comp rx=0 tx=0).stop 2024-01-15T18:11:19.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 shutdown_connections 2024-01-15T18:11:19.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067300 0x7f21840697c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:19.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac15ad00 0x7f21ac0a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:19.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac15a880 0x7f21ac0a9780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:19.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 --2- 172.21.15.143:0/3467759321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f218c0157e0 0x7f21ac1f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:19.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 >> 172.21.15.143:0/3467759321 conn(0x7f21ac0b0bf0 msgr2=0x7f21ac0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:19.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 shutdown_connections 2024-01-15T18:11:19.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:19.761+0000 7f21b3c00640 1 -- 172.21.15.143:0/3467759321 wait complete. 2024-01-15T18:11:19.790 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:11:19.790 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:11:19.873 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:11:19.976 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:11:19.976 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:11:19.976 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:19.992 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:20.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.269+0000 7f8c56400640 1 Processor -- start 2024-01-15T18:11:20.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.269+0000 7f8c56400640 1 -- start start 2024-01-15T18:11:20.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.269+0000 7f8c56400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c500cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c56400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8c500cd9e0 0x7f8c500b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4f400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8c5006db20 0x7f8c500b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:50550/0 (socket says 172.21.15.143:50550) 2024-01-15T18:11:20.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4f400640 1 -- 172.21.15.143:0/3835974747 learned_addr learned my addr 172.21.15.143:0/3835974747 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500cc790 con 0x7f8c5006db20 2024-01-15T18:11:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500b4ae0 con 0x7f8c500cd9e0 2024-01-15T18:11:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b4570 con 0x7f8c5015a440 2024-01-15T18:11:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4fe00640 1 --2- 172.21.15.143:0/3835974747 >> v2:172.21.15.143:3300/0 conn(0x7f8c500cd9e0 0x7f8c500b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4fe00640 1 -- 172.21.15.143:0/3835974747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 msgr2=0x7f8c500cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:20.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4fe00640 1 --2- 172.21.15.143:0/3835974747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c500cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4fe00640 1 -- 172.21.15.143:0/3835974747 >> v1:172.21.15.204:6789/0 conn(0x7f8c5006db20 legacy=0x7f8c500b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:20.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.273+0000 7f8c4fe00640 1 -- 172.21.15.143:0/3835974747 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500b4800 con 0x7f8c500cd9e0 2024-01-15T18:11:20.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.277+0000 7f8c4fe00640 1 --2- 172.21.15.143:0/3835974747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c500cd9e0 0x7f8c500b20b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8c400071d0 tx=0x7f8c4001d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770d670545995a74 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:20.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.277+0000 7f8c4e000640 1 -- 172.21.15.143:0/3835974747 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40008be0 con 0x7f8c500cd9e0 2024-01-15T18:11:20.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.277+0000 7f8c4e000640 1 -- 172.21.15.143:0/3835974747 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40008d40 con 0x7f8c500cd9e0 2024-01-15T18:11:20.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.277+0000 7f8c4e000640 1 -- 172.21.15.143:0/3835974747 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000a2f0 con 0x7f8c500cd9e0 2024-01-15T18:11:20.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c500cd9e0 msgr2=0x7f8c500b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 --2- 172.21.15.143:0/3835974747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c500cd9e0 0x7f8c500b20b0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8c400071d0 tx=0x7f8c4001d420 comp rx=0 tx=0).stop 2024-01-15T18:11:20.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 shutdown_connections 2024-01-15T18:11:20.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 --2- 172.21.15.143:0/3835974747 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c500cd9e0 0x7f8c500b20b0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 --2- 172.21.15.143:0/3835974747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c500cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 >> 172.21.15.143:0/3835974747 conn(0x7f8c500aea10 msgr2=0x7f8c500aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:20.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 shutdown_connections 2024-01-15T18:11:20.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.281+0000 7f8c56400640 1 -- 172.21.15.143:0/3835974747 wait complete. 2024-01-15T18:11:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 Processor -- start 2024-01-15T18:11:20.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 -- start start 2024-01-15T18:11:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 0x7f8c50152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c500cd9e0 0x7f8c5014bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c5014c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5014c100 con 0x7f8c5006db20 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c4f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 0x7f8c50152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c4f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 0x7f8c50152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49600/0 (socket says 172.21.15.143:49600) 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c4f400640 1 -- 172.21.15.143:0/3423311181 learned_addr learned my addr 172.21.15.143:0/3423311181 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5014c7a0 con 0x7f8c500cd9e0 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.285+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5014ca00 con 0x7f8c5015a440 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4fe00640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c5014c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4ea00640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c500cd9e0 0x7f8c5014bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4f400640 1 -- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 msgr2=0x7f8c5014c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4f400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c5014c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4f400640 1 -- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c500cd9e0 msgr2=0x7f8c5014bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4f400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c500cd9e0 0x7f8c5014bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4f400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4000ae50 con 0x7f8c5006db20 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c4f400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 0x7f8c50152b80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800b430 tx=0x7f8c3800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011830 con 0x7f8c5006db20 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c380119b0 con 0x7f8c5006db20 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38011d60 con 0x7f8c5006db20 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5014cce0 con 0x7f8c5006db20 2024-01-15T18:11:20.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5014d2a0 con 0x7f8c5006db20 2024-01-15T18:11:20.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.289+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50067500 con 0x7f8c5006db20 2024-01-15T18:11:20.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.301+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f8c380387b0 con 0x7f8c5006db20 2024-01-15T18:11:20.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.301+0000 7f8c47400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c280671f0 0x7f8c280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.301+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8c380560a0 con 0x7f8c5006db20 2024-01-15T18:11:20.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.301+0000 7f8c4ea00640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c280671f0 0x7f8c280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.305+0000 7f8c4ea00640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c280671f0 0x7f8c280696b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8c500b4480 tx=0x7f8c480094f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:20.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.305+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8c38055770 con 0x7f8c5006db20 2024-01-15T18:11:20.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.601+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c5006e6d0 con 0x7f8c5006db20 2024-01-15T18:11:20.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.601+0000 7f8c47400640 1 -- 172.21.15.143:0/3423311181 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f8c38015020 con 0x7f8c5006db20 2024-01-15T18:11:20.606 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:20.606 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":63,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:19.753876+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/3767686465":"2024-01-16T18:11:16.733260+0000","172.21.15.143:6836/3767686465":"2024-01-16T18:11:16.733260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c280671f0 msgr2=0x7f8c280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c280671f0 0x7f8c280696b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8c500b4480 tx=0x7f8c480094f0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 msgr2=0x7f8c50152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 0x7f8c50152b80 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800b430 tx=0x7f8c3800b8f0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 shutdown_connections 2024-01-15T18:11:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8c280671f0 0x7f8c280696b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8c5015a440 0x7f8c5014c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8c500cd9e0 0x7f8c5014bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 --2- 172.21.15.143:0/3423311181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c5006db20 0x7f8c50152b80 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 >> 172.21.15.143:0/3423311181 conn(0x7f8c500aea10 msgr2=0x7f8c50156780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 shutdown_connections 2024-01-15T18:11:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.605+0000 7f8c56400640 1 -- 172.21.15.143:0/3423311181 wait complete. 2024-01-15T18:11:20.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:11:20.629 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:11:20.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 Processor -- start 2024-01-15T18:11:20.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 -- start start 2024-01-15T18:11:20.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f81e015b260 0x7f81e00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f81e00d19f0 0x7f81e00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e00c4d90 con 0x7f81e00c5220 2024-01-15T18:11:20.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e015b640 con 0x7f81e00d19f0 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.765+0000 7f81e8800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e00cf800 con 0x7f81e015b260 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f81e00d19f0 0x7f81e00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f81e00d19f0 0x7f81e00cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49608/0 (socket says 172.21.15.143:49608) 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 -- 172.21.15.143:0/1846110463 learned_addr learned my addr 172.21.15.143:0/1846110463 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5e00640 1 --2- 172.21.15.143:0/1846110463 >> v2:172.21.15.204:3300/0 conn(0x7f81e015b260 0x7f81e00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 -- 172.21.15.143:0/1846110463 >> v2:172.21.15.204:3300/0 conn(0x7f81e015b260 msgr2=0x7f81e00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 --2- 172.21.15.143:0/1846110463 >> v2:172.21.15.204:3300/0 conn(0x7f81e015b260 0x7f81e00cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 -- 172.21.15.143:0/1846110463 >> v1:172.21.15.204:6789/0 conn(0x7f81e00c5220 legacy=0x7f81e00c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:20.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 -- 172.21.15.143:0/1846110463 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00cfae0 con 0x7f81e00d19f0 2024-01-15T18:11:20.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5e00640 1 --2- 172.21.15.143:0/1846110463 >> v2:172.21.15.204:3300/0 conn(0x7f81e015b260 0x7f81e00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:20.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e5400640 1 --2- 172.21.15.143:0/1846110463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00d19f0 0x7f81e00cdd50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f81e00681e0 tx=0x7f81c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebec850fba79291d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:20.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e4a00640 1 -- 172.21.15.143:0/1846110463 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c80ab600 con 0x7f81e00d19f0 2024-01-15T18:11:20.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e4a00640 1 -- 172.21.15.143:0/1846110463 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c80a7050 con 0x7f81e00d19f0 2024-01-15T18:11:20.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.769+0000 7f81e4a00640 1 -- 172.21.15.143:0/1846110463 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c80ab980 con 0x7f81e00d19f0 2024-01-15T18:11:20.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- 172.21.15.143:0/1846110463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00d19f0 msgr2=0x7f81e00cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 --2- 172.21.15.143:0/1846110463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00d19f0 0x7f81e00cdd50 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f81e00681e0 tx=0x7f81c801c490 comp rx=0 tx=0).stop 2024-01-15T18:11:20.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- 172.21.15.143:0/1846110463 shutdown_connections 2024-01-15T18:11:20.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 --2- 172.21.15.143:0/1846110463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00d19f0 0x7f81e00cdd50 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 --2- 172.21.15.143:0/1846110463 >> v2:172.21.15.204:3300/0 conn(0x7f81e015b260 0x7f81e00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- 172.21.15.143:0/1846110463 >> 172.21.15.143:0/1846110463 conn(0x7f81e00d0ed0 msgr2=0x7f81e00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:20.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- 172.21.15.143:0/1846110463 shutdown_connections 2024-01-15T18:11:20.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- 172.21.15.143:0/1846110463 wait complete. 2024-01-15T18:11:20.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 Processor -- start 2024-01-15T18:11:20.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- start start 2024-01-15T18:11:20.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 0x7f81e0152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e00d19f0 0x7f81e0151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81e015b260 0x7f81e0151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0151da0 con 0x7f81e00c5220 2024-01-15T18:11:20.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e009bab0 con 0x7f81e00d19f0 2024-01-15T18:11:20.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e8800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e0153790 con 0x7f81e015b260 2024-01-15T18:11:20.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.773+0000 7f81e5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 0x7f81e0152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 0x7f81e0152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49614/0 (socket says 172.21.15.143:49614) 2024-01-15T18:11:20.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 -- 172.21.15.143:0/2434402972 learned_addr learned my addr 172.21.15.143:0/2434402972 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e6800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81e015b260 0x7f81e0151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 -- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81e015b260 msgr2=0x7f81e0151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81e015b260 0x7f81e0151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 -- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e00d19f0 msgr2=0x7f81e0151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e00d19f0 0x7f81e0151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 -- 172.21.15.143:0/2434402972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c80094d0 con 0x7f81e00c5220 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e5e00640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 0x7f81e0152180 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c7f0 tx=0x7f81d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0007820 con 0x7f81e00c5220 2024-01-15T18:11:20.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0007980 con 0x7f81e00c5220 2024-01-15T18:11:20.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0007d30 con 0x7f81e00c5220 2024-01-15T18:11:20.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e009aea0 con 0x7f81e00c5220 2024-01-15T18:11:20.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e009b460 con 0x7f81e00c5220 2024-01-15T18:11:20.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e0067500 con 0x7f81e00c5220 2024-01-15T18:11:20.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.777+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f81d00113f0 con 0x7f81e00c5220 2024-01-15T18:11:20.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.781+0000 7f81de000640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81b80671f0 0x7f81b80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:20.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.781+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f81d00830b0 con 0x7f81e00c5220 2024-01-15T18:11:20.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.781+0000 7f81e5400640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81b80671f0 0x7f81b80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:20.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.781+0000 7f81e5400640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81b80671f0 0x7f81b80696b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f81c80ad000 tx=0x7f81c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:20.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.785+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81d0051cb0 con 0x7f81e00c5220 2024-01-15T18:11:20.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.985+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81e00b6d50 con 0x7f81e00c5220 2024-01-15T18:11:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.989+0000 7f81de000640 1 -- 172.21.15.143:0/2434402972 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81d001f0a0 con 0x7f81e00c5220 2024-01-15T18:11:20.994 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:11:20.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81b80671f0 msgr2=0x7f81b80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81b80671f0 0x7f81b80696b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f81c80ad000 tx=0x7f81c8008040 comp rx=0 tx=0).stop 2024-01-15T18:11:20.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 msgr2=0x7f81e0152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 0x7f81e0152180 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c7f0 tx=0x7f81d000ccb0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 shutdown_connections 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81b80671f0 0x7f81b80696b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f81e015b260 0x7f81e0151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f81e00d19f0 0x7f81e0151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 --2- 172.21.15.143:0/2434402972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e00c5220 0x7f81e0152180 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.993+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 >> 172.21.15.143:0/2434402972 conn(0x7f81e00d0ed0 msgr2=0x7f81e0157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:20.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.997+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 shutdown_connections 2024-01-15T18:11:20.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:20.997+0000 7f81e8800640 1 -- 172.21.15.143:0/2434402972 wait complete. 2024-01-15T18:11:21.035 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:11:21.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.277+0000 7fc856000640 1 Processor -- start 2024-01-15T18:11:21.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.277+0000 7fc856000640 1 -- start start 2024-01-15T18:11:21.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.281+0000 7fc856000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc85006a490 0x7fc8501597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:21.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc856000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc850159cf0 0x7fc8500b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc856000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85006b8a0 con 0x7fc8501593d0 2024-01-15T18:11:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc856000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85015a120 con 0x7fc85006a490 2024-01-15T18:11:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc856000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85006a920 con 0x7fc850159cf0 2024-01-15T18:11:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc85006a490 0x7fc8501597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:21.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc85006a490 0x7fc8501597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49616/0 (socket says 172.21.15.143:49616) 2024-01-15T18:11:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 -- 172.21.15.143:0/2717084724 learned_addr learned my addr 172.21.15.143:0/2717084724 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 -- 172.21.15.143:0/2717084724 >> v2:172.21.15.204:3300/0 conn(0x7fc850159cf0 msgr2=0x7fc8500b2290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:11:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 --2- 172.21.15.143:0/2717084724 >> v2:172.21.15.204:3300/0 conn(0x7fc850159cf0 0x7fc8500b2290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 -- 172.21.15.143:0/2717084724 >> v1:172.21.15.143:6789/0 conn(0x7fc8501593d0 legacy=0x7fc850072230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:11:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.285+0000 7fc84ea00640 1 -- 172.21.15.143:0/2717084724 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85006ac00 con 0x7fc85006a490 2024-01-15T18:11:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.293+0000 7fc84ea00640 1 --2- 172.21.15.143:0/2717084724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85006a490 0x7fc8501597b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc8500d0500 tx=0x7fc84001d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca9abb7bbeb3aa7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc84e000640 1 -- 172.21.15.143:0/2717084724 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840008be0 con 0x7fc85006a490 2024-01-15T18:11:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc84e000640 1 -- 172.21.15.143:0/2717084724 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840008d40 con 0x7fc85006a490 2024-01-15T18:11:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc84e000640 1 -- 172.21.15.143:0/2717084724 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84000a2f0 con 0x7fc85006a490 2024-01-15T18:11:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc856000640 1 -- 172.21.15.143:0/2717084724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85006a490 msgr2=0x7fc8501597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc856000640 1 --2- 172.21.15.143:0/2717084724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85006a490 0x7fc8501597b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc8500d0500 tx=0x7fc84001d4f0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc856000640 1 -- 172.21.15.143:0/2717084724 shutdown_connections 2024-01-15T18:11:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc856000640 1 --2- 172.21.15.143:0/2717084724 >> v2:172.21.15.204:3300/0 conn(0x7fc850159cf0 0x7fc8500b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc856000640 1 --2- 172.21.15.143:0/2717084724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc85006a490 0x7fc8501597b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.297+0000 7fc856000640 1 -- 172.21.15.143:0/2717084724 >> 172.21.15.143:0/2717084724 conn(0x7fc85006dc10 msgr2=0x7fc85006e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.301+0000 7fc856000640 1 -- 172.21.15.143:0/2717084724 shutdown_connections 2024-01-15T18:11:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.301+0000 7fc856000640 1 -- 172.21.15.143:0/2717084724 wait complete. 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 Processor -- start 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 -- start start 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8501593d0 0x7fc85008a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc850159cf0 0x7fc850156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 0x7fc850200180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc850203c60 con 0x7fc850159cf0 2024-01-15T18:11:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8502006c0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc856000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850203980 con 0x7fc8501593d0 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc854a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 0x7fc850200180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc854a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 0x7fc850200180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:52728/0 (socket says 172.21.15.143:52728) 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc854a00640 1 -- 172.21.15.143:0/1617419642 learned_addr learned my addr 172.21.15.143:0/1617419642 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.305+0000 7fc84f400640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8501593d0 0x7fc85008a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc854a00640 1 -- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8501593d0 msgr2=0x7fc85008a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc854a00640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8501593d0 0x7fc85008a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc854a00640 1 -- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc850159cf0 msgr2=0x7fc850156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc854a00640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc850159cf0 0x7fc850156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc854a00640 1 -- 172.21.15.143:0/1617419642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840007040 con 0x7fc8501ffcd0 2024-01-15T18:11:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc854a00640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 0x7fc850200180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc84800b970 tx=0x7fc84800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84800cb40 con 0x7fc8501ffcd0 2024-01-15T18:11:21.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84800cca0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848014680 con 0x7fc8501ffcd0 2024-01-15T18:11:21.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc850089640 con 0x7fc8501ffcd0 2024-01-15T18:11:21.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.309+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc850089c00 con 0x7fc8501ffcd0 2024-01-15T18:11:21.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.313+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8500894a0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.313+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fc84801d3b0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.313+0000 7fc847400640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc828067240 0x7fc828069700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:21.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.313+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fc848082da0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.317+0000 7fc84f400640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc828067240 0x7fc828069700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:21.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.317+0000 7fc84f400640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc828067240 0x7fc828069700 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc838009980 tx=0x7fc8380090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:21.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.321+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc8500894a0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.681+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8500894a0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.681+0000 7fc847400640 1 -- 172.21.15.143:0/1617419642 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v63) v1 ==== 80+0+124 (secure 0 0 0) 0x7fc8500894a0 con 0x7fc8501ffcd0 2024-01-15T18:11:21.687 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/3767686465 2024-01-16T18:11:16.733260+0000 2024-01-15T18:11:21.687 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/3767686465 2024-01-16T18:11:16.733260+0000 2024-01-15T18:11:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:11:21.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.685+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc828067240 msgr2=0x7fc828069700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:21.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.685+0000 7fc856000640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc828067240 0x7fc828069700 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc838009980 tx=0x7fc8380090e0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 msgr2=0x7fc850200180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:21.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 0x7fc850200180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc84800b970 tx=0x7fc84800be30 comp rx=0 tx=0).stop 2024-01-15T18:11:21.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 shutdown_connections 2024-01-15T18:11:21.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc828067240 0x7fc828069700 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8501ffcd0 0x7fc850200180 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc850159cf0 0x7fc850156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 --2- 172.21.15.143:0/1617419642 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc8501593d0 0x7fc85008a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:21.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.689+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 >> 172.21.15.143:0/1617419642 conn(0x7fc85006dc10 msgr2=0x7fc85006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:21.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.693+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 shutdown_connections 2024-01-15T18:11:21.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:21.693+0000 7fc856000640 1 -- 172.21.15.143:0/1617419642 wait complete. 2024-01-15T18:11:21.734 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:11:21.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:21.735 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:21.787 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:11:21.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:21.788 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:11:21.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[42049]: fuse finished with error 0 and tester_r 0 2024-01-15T18:11:21.989 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:11:21.989 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:21.999 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:22.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.213+0000 7f67cca00640 1 Processor -- start 2024-01-15T18:11:22.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67cca00640 1 -- start start 2024-01-15T18:11:22.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67cca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67c41613e0 0x7f67c40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67cca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40b4140 con 0x7f67c40b2290 2024-01-15T18:11:22.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67cca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c40b4420 con 0x7f67c406ada0 2024-01-15T18:11:22.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67cca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c40b4700 con 0x7f67c41613e0 2024-01-15T18:11:22.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67ca000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67c41613e0 0x7f67c40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67ca000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67c41613e0 0x7f67c40b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49646/0 (socket says 172.21.15.143:49646) 2024-01-15T18:11:22.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67ca000640 1 -- 172.21.15.143:0/1857844193 learned_addr learned my addr 172.21.15.143:0/1857844193 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:22.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67ca000640 1 -- 172.21.15.143:0/1857844193 >> v1:172.21.15.204:6789/0 conn(0x7f67c406ada0 legacy=0x7f67c4148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:22.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67ca000640 1 -- 172.21.15.143:0/1857844193 >> v1:172.21.15.143:6789/0 conn(0x7f67c40b2290 legacy=0x7f67c406a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:22.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.217+0000 7f67ca000640 1 -- 172.21.15.143:0/1857844193 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c40b49e0 con 0x7f67c41613e0 2024-01-15T18:11:22.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67ca000640 1 --2- 172.21.15.143:0/1857844193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c40b51e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009870 tx=0x7f67b401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49241b361e5dd13b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67c8c00640 1 -- 172.21.15.143:0/1857844193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40ab620 con 0x7f67c41613e0 2024-01-15T18:11:22.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67c8c00640 1 -- 172.21.15.143:0/1857844193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b40ab780 con 0x7f67c41613e0 2024-01-15T18:11:22.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67c8c00640 1 -- 172.21.15.143:0/1857844193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b40abb10 con 0x7f67c41613e0 2024-01-15T18:11:22.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 -- 172.21.15.143:0/1857844193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 msgr2=0x7f67c40b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 --2- 172.21.15.143:0/1857844193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c40b51e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009870 tx=0x7f67b401c400 comp rx=0 tx=0).stop 2024-01-15T18:11:22.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 -- 172.21.15.143:0/1857844193 shutdown_connections 2024-01-15T18:11:22.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 --2- 172.21.15.143:0/1857844193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c40b51e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 -- 172.21.15.143:0/1857844193 >> 172.21.15.143:0/1857844193 conn(0x7f67c4066720 msgr2=0x7f67c406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:22.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 -- 172.21.15.143:0/1857844193 shutdown_connections 2024-01-15T18:11:22.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.221+0000 7f67cca00640 1 -- 172.21.15.143:0/1857844193 wait complete. 2024-01-15T18:11:22.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 Processor -- start 2024-01-15T18:11:22.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 -- start start 2024-01-15T18:11:22.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c406ada0 0x7f67c40a87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c40b2290 0x7f67c413ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41455c0 con 0x7f67c41613e0 2024-01-15T18:11:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c40a8d20 con 0x7f67c406ada0 2024-01-15T18:11:22.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c415c300 con 0x7f67c40b2290 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67ca000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c406ada0 0x7f67c40a87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67c9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c40b2290 0x7f67c413ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c413e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49658/0 (socket says 172.21.15.143:49658) 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67ca000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c406ada0 0x7f67c40a87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:52744/0 (socket says 172.21.15.143:52744) 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 -- 172.21.15.143:0/1392748849 learned_addr learned my addr 172.21.15.143:0/1392748849 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 -- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c40b2290 msgr2=0x7f67c413ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c40b2290 0x7f67c413ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 -- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c406ada0 msgr2=0x7f67c40a87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c406ada0 0x7f67c40a87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 -- 172.21.15.143:0/1392748849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40094d0 con 0x7f67c41613e0 2024-01-15T18:11:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67c9600640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c40b2290 0x7f67c413ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.225+0000 7f67cb400640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c413e370 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b9a0 tx=0x7f67bc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00cb40 con 0x7f67c41613e0 2024-01-15T18:11:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c415b6f0 con 0x7f67c41613e0 2024-01-15T18:11:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67bc00cca0 con 0x7f67c41613e0 2024-01-15T18:11:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67bc014680 con 0x7f67c41613e0 2024-01-15T18:11:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c415bcb0 con 0x7f67c41613e0 2024-01-15T18:11:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c415c490 con 0x7f67c41613e0 2024-01-15T18:11:22.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f67bc01d3b0 con 0x7f67c41613e0 2024-01-15T18:11:22.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.229+0000 7f67c2000640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f679c0671f0 0x7f679c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.233+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f67bc082c60 con 0x7f67c41613e0 2024-01-15T18:11:22.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.233+0000 7f67ca000640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f679c0671f0 0x7f679c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.233+0000 7f67ca000640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f679c0671f0 0x7f679c0696b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f67b40af000 tx=0x7f67b40b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.237+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f67bc051730 con 0x7f67c41613e0 2024-01-15T18:11:22.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67c4067680 con 0x7f67c41613e0 2024-01-15T18:11:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67c2000640 1 -- 172.21.15.143:0/1392748849 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f67bc06ce60 con 0x7f67c41613e0 2024-01-15T18:11:22.395 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:22.396 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":63,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:19.753876+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/3767686465":"2024-01-16T18:11:16.733260+0000","172.21.15.143:6836/3767686465":"2024-01-16T18:11:16.733260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:22.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f679c0671f0 msgr2=0x7f679c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f679c0671f0 0x7f679c0696b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f67b40af000 tx=0x7f67b40b6040 comp rx=0 tx=0).stop 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 msgr2=0x7f67c413e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c413e370 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00b9a0 tx=0x7f67bc00be60 comp rx=0 tx=0).stop 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 shutdown_connections 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f679c0671f0 0x7f679c0696b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67c41613e0 0x7f67c413e370 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67c40b2290 0x7f67c413ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 --2- 172.21.15.143:0/1392748849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67c406ada0 0x7f67c40a87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 >> 172.21.15.143:0/1392748849 conn(0x7f67c4066720 msgr2=0x7f67c41445e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 shutdown_connections 2024-01-15T18:11:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.393+0000 7f67cca00640 1 -- 172.21.15.143:0/1392748849 wait complete. 2024-01-15T18:11:22.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:11:22.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:11:22.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.553+0000 7f3598200640 1 Processor -- start 2024-01-15T18:11:22.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.553+0000 7f3598200640 1 -- start start 2024-01-15T18:11:22.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3598200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3590071bb0 0x7f3590072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3598200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f359015c5a0 0x7f359015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3598200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35900ceb50 con 0x7f3590143780 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3598200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3590072570 con 0x7f359015c5a0 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3598200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359015c110 con 0x7f3590071bb0 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f359015c5a0 0x7f359015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f359015c5a0 0x7f359015c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49668/0 (socket says 172.21.15.143:49668) 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 -- 172.21.15.143:0/1259713999 learned_addr learned my addr 172.21.15.143:0/1259713999 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3594e00640 1 --2- 172.21.15.143:0/1259713999 >> v2:172.21.15.204:3300/0 conn(0x7f3590071bb0 0x7f3590072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 -- 172.21.15.143:0/1259713999 >> v2:172.21.15.204:3300/0 conn(0x7f3590071bb0 msgr2=0x7f3590072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 --2- 172.21.15.143:0/1259713999 >> v2:172.21.15.204:3300/0 conn(0x7f3590071bb0 0x7f3590072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 -- 172.21.15.143:0/1259713999 >> v1:172.21.15.204:6789/0 conn(0x7f3590143780 legacy=0x7f35900cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.557+0000 7f3596200640 1 -- 172.21.15.143:0/1259713999 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359015b7e0 con 0x7f359015c5a0 2024-01-15T18:11:22.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3596200640 1 --2- 172.21.15.143:0/1259713999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f359015c5a0 0x7f359015c9e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f359006beb0 tx=0x7f358801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b930310231053d5f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f358fe00640 1 -- 172.21.15.143:0/1259713999 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880a5850 con 0x7f359015c5a0 2024-01-15T18:11:22.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f358fe00640 1 -- 172.21.15.143:0/1259713999 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35880a59b0 con 0x7f359015c5a0 2024-01-15T18:11:22.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f358fe00640 1 -- 172.21.15.143:0/1259713999 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880a5d40 con 0x7f359015c5a0 2024-01-15T18:11:22.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 -- 172.21.15.143:0/1259713999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f359015c5a0 msgr2=0x7f359015c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 --2- 172.21.15.143:0/1259713999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f359015c5a0 0x7f359015c9e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f359006beb0 tx=0x7f358801c990 comp rx=0 tx=0).stop 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 -- 172.21.15.143:0/1259713999 shutdown_connections 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 --2- 172.21.15.143:0/1259713999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f359015c5a0 0x7f359015c9e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 --2- 172.21.15.143:0/1259713999 >> v2:172.21.15.204:3300/0 conn(0x7f3590071bb0 0x7f3590072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 -- 172.21.15.143:0/1259713999 >> 172.21.15.143:0/1259713999 conn(0x7f359006c3f0 msgr2=0x7f359006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 -- 172.21.15.143:0/1259713999 shutdown_connections 2024-01-15T18:11:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 -- 172.21.15.143:0/1259713999 wait complete. 2024-01-15T18:11:22.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 Processor -- start 2024-01-15T18:11:22.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.561+0000 7f3598200640 1 -- start start 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 0x7f35901f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3590143780 0x7f3590158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f359015c5a0 0x7f35901346c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35901351d0 con 0x7f3590071bb0 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3590138c70 con 0x7f3590143780 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3590138f50 con 0x7f359015c5a0 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 0x7f35901f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3594e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3590143780 0x7f3590158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 0x7f35901f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49680/0 (socket says 172.21.15.143:49680) 2024-01-15T18:11:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 -- 172.21.15.143:0/3766490759 learned_addr learned my addr 172.21.15.143:0/3766490759 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3596200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f359015c5a0 0x7f35901346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 -- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f359015c5a0 msgr2=0x7f35901346c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f359015c5a0 0x7f35901346c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 -- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3590143780 msgr2=0x7f3590158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3590143780 0x7f3590158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 -- 172.21.15.143:0/3766490759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588009b30 con 0x7f3590071bb0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3595800640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 0x7f35901f60d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f358000c450 tx=0x7f358000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580012830 con 0x7f3590071bb0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35800129b0 con 0x7f3590071bb0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3590139230 con 0x7f3590071bb0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580012d60 con 0x7f3590071bb0 2024-01-15T18:11:22.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35901598c0 con 0x7f3590071bb0 2024-01-15T18:11:22.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3590067500 con 0x7f3590071bb0 2024-01-15T18:11:22.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.565+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f358001c460 con 0x7f3590071bb0 2024-01-15T18:11:22.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.569+0000 7f358d600640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f35680671f0 0x7f35680696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.569+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3580082a20 con 0x7f3590071bb0 2024-01-15T18:11:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.569+0000 7f3594e00640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f35680671f0 0x7f35680696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.569+0000 7f3594e00640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f35680671f0 0x7f35680696b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f359013a9a0 tx=0x7f3578008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.573+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3580051760 con 0x7f3590071bb0 2024-01-15T18:11:22.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.721+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35900b3670 con 0x7f3590071bb0 2024-01-15T18:11:22.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.721+0000 7f358d600640 1 -- 172.21.15.143:0/3766490759 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3580054d80 con 0x7f3590071bb0 2024-01-15T18:11:22.725 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:11:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f35680671f0 msgr2=0x7f35680696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f35680671f0 0x7f35680696b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f359013a9a0 tx=0x7f3578008040 comp rx=0 tx=0).stop 2024-01-15T18:11:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 msgr2=0x7f35901f60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 0x7f35901f60d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f358000c450 tx=0x7f358000c910 comp rx=0 tx=0).stop 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 shutdown_connections 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f35680671f0 0x7f35680696b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f359015c5a0 0x7f35901346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3590143780 0x7f3590158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 --2- 172.21.15.143:0/3766490759 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3590071bb0 0x7f35901f60d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 >> 172.21.15.143:0/3766490759 conn(0x7f359006c3f0 msgr2=0x7f35900affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 shutdown_connections 2024-01-15T18:11:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.725+0000 7f3598200640 1 -- 172.21.15.143:0/3766490759 wait complete. 2024-01-15T18:11:22.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:11:22.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 Processor -- start 2024-01-15T18:11:22.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 -- start start 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a381613e0 0x7f8a380b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a380b2290 0x7f8a380b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a3806a690 0x7f8a3806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a380b2c10 con 0x7f8a381613e0 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a380b4140 con 0x7f8a3806a690 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a40000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380b4420 con 0x7f8a380b2290 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a381613e0 0x7f8a380b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a381613e0 0x7f8a380b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49692/0 (socket says 172.21.15.143:49692) 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 -- 172.21.15.143:0/331536079 learned_addr learned my addr 172.21.15.143:0/331536079 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 -- 172.21.15.143:0/331536079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a380b2290 msgr2=0x7f8a380b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3cc00640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a380b2290 0x7f8a380b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3ea00640 1 --2- 172.21.15.143:0/331536079 >> v2:172.21.15.204:3300/0 conn(0x7f8a3806a690 0x7f8a3806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a380b2290 0x7f8a380b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 -- 172.21.15.143:0/331536079 >> v2:172.21.15.204:3300/0 conn(0x7f8a3806a690 msgr2=0x7f8a3806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:22.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 --2- 172.21.15.143:0/331536079 >> v2:172.21.15.204:3300/0 conn(0x7f8a3806a690 0x7f8a3806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3d600640 1 -- 172.21.15.143:0/331536079 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a380b4700 con 0x7f8a381613e0 2024-01-15T18:11:22.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3ea00640 1 --2- 172.21.15.143:0/331536079 >> v2:172.21.15.204:3300/0 conn(0x7f8a3806a690 0x7f8a3806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:22.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.889+0000 7f8a3cc00640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a380b2290 0x7f8a380b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:22.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a3d600640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a381613e0 0x7f8a380b51e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8a380b32f0 tx=0x7f8a2801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4164b0266d3fa6a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a37e00640 1 -- 172.21.15.143:0/331536079 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a280ab600 con 0x7f8a381613e0 2024-01-15T18:11:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a37e00640 1 -- 172.21.15.143:0/331536079 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a280ab760 con 0x7f8a381613e0 2024-01-15T18:11:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a37e00640 1 -- 172.21.15.143:0/331536079 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a280abaf0 con 0x7f8a381613e0 2024-01-15T18:11:22.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 -- 172.21.15.143:0/331536079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a381613e0 msgr2=0x7f8a380b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a381613e0 0x7f8a380b51e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f8a380b32f0 tx=0x7f8a2801c440 comp rx=0 tx=0).stop 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 -- 172.21.15.143:0/331536079 shutdown_connections 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 --2- 172.21.15.143:0/331536079 >> v2:172.21.15.204:3300/0 conn(0x7f8a3806a690 0x7f8a3806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a380b2290 0x7f8a380b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 --2- 172.21.15.143:0/331536079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a381613e0 0x7f8a380b51e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 -- 172.21.15.143:0/331536079 >> 172.21.15.143:0/331536079 conn(0x7f8a38066720 msgr2=0x7f8a3806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:22.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 -- 172.21.15.143:0/331536079 shutdown_connections 2024-01-15T18:11:22.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.893+0000 7f8a40000640 1 -- 172.21.15.143:0/331536079 wait complete. 2024-01-15T18:11:22.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 Processor -- start 2024-01-15T18:11:22.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 -- start start 2024-01-15T18:11:22.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3806a690 0x7f8a380a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 0x7f8a38145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a381613e0 0x7f8a381461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a380aa6c0 con 0x7f8a380b2290 2024-01-15T18:11:22.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a38143820 con 0x7f8a3806a690 2024-01-15T18:11:22.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a38142c50 con 0x7f8a381613e0 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 0x7f8a38145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 0x7f8a38145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49706/0 (socket says 172.21.15.143:49706) 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 -- 172.21.15.143:0/458699243 learned_addr learned my addr 172.21.15.143:0/458699243 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3d600640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3806a690 0x7f8a380a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3ea00640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a381613e0 0x7f8a381461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 -- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a381613e0 msgr2=0x7f8a381461a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a381613e0 0x7f8a381461a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 -- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3806a690 msgr2=0x7f8a380a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3806a690 0x7f8a380a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 -- 172.21.15.143:0/458699243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280094d0 con 0x7f8a380b2290 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a3cc00640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 0x7f8a38145c60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c7c0 tx=0x7f8a2000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a20007a30 con 0x7f8a380b2290 2024-01-15T18:11:22.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a20007b90 con 0x7f8a380b2290 2024-01-15T18:11:22.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2000ce70 con 0x7f8a380b2290 2024-01-15T18:11:22.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a38142ef0 con 0x7f8a380b2290 2024-01-15T18:11:22.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.897+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a381434b0 con 0x7f8a380b2290 2024-01-15T18:11:22.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.901+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a38143990 con 0x7f8a380b2290 2024-01-15T18:11:22.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.901+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f8a200113f0 con 0x7f8a380b2290 2024-01-15T18:11:22.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.901+0000 7f8a35600640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a10067220 0x7f8a100696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:22.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.901+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8a20083080 con 0x7f8a380b2290 2024-01-15T18:11:22.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.901+0000 7f8a3d600640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a10067220 0x7f8a100696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:22.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.901+0000 7f8a3d600640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a10067220 0x7f8a100696e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8a280af000 tx=0x7f8a280b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:22.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:22.905+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a20051dc0 con 0x7f8a380b2290 2024-01-15T18:11:23.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.073+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a38067680 con 0x7f8a380b2290 2024-01-15T18:11:23.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.073+0000 7f8a35600640 1 -- 172.21.15.143:0/458699243 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v63) v1 ==== 80+0+124 (secure 0 0 0) 0x7f8a200553e0 con 0x7f8a380b2290 2024-01-15T18:11:23.078 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/3767686465 2024-01-16T18:11:16.733260+0000 2024-01-15T18:11:23.078 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/3767686465 2024-01-16T18:11:16.733260+0000 2024-01-15T18:11:23.079 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:11:23.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a10067220 msgr2=0x7f8a100696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a10067220 0x7f8a100696e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8a280af000 tx=0x7f8a280b6040 comp rx=0 tx=0).stop 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 msgr2=0x7f8a38145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 0x7f8a38145c60 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000c7c0 tx=0x7f8a2000cc80 comp rx=0 tx=0).stop 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 shutdown_connections 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a10067220 0x7f8a100696e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a381613e0 0x7f8a381461a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a380b2290 0x7f8a38145c60 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 --2- 172.21.15.143:0/458699243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a3806a690 0x7f8a380a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 >> 172.21.15.143:0/458699243 conn(0x7f8a38066720 msgr2=0x7f8a380b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 shutdown_connections 2024-01-15T18:11:23.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.077+0000 7f8a40000640 1 -- 172.21.15.143:0/458699243 wait complete. 2024-01-15T18:11:23.100 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:11:23.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:23.101 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:23.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33770]: fuse finished with error 0 and tester_r 0 2024-01-15T18:11:23.123 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:11:23.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:23.123 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:11:23.294 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:11:23.294 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:23.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:11:23.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:11:23.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:23.304 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:11:23.353 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:11:23.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:11:23.355 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:11:23.355 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:23.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:11:23.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:11:23.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:23.365 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:11:23.414 INFO:teuthology.orchestra.run.smithi204.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:11:23.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:11:23.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:11:23.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 Processor -- start 2024-01-15T18:11:23.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 -- start start 2024-01-15T18:11:23.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0b0310 0x7f885c0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f885c0cf740 0x7f885c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885c0cbff0 con 0x7f885c06bfb0 2024-01-15T18:11:23.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0d0c50 con 0x7f885c0cf740 2024-01-15T18:11:23.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8862e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cb6c0 con 0x7f885c0b0310 2024-01-15T18:11:23.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f885c0cf740 0x7f885c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f885c0cf740 0x7f885c0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49722/0 (socket says 172.21.15.143:49722) 2024-01-15T18:11:23.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 -- 172.21.15.143:0/605673485 learned_addr learned my addr 172.21.15.143:0/605673485 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:23.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f885b400640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0b0310 0x7f885c0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 -- 172.21.15.143:0/605673485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0b0310 msgr2=0x7f885c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:23.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0b0310 0x7f885c0cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 -- 172.21.15.143:0/605673485 >> v1:172.21.15.204:6789/0 conn(0x7f885c06bfb0 legacy=0x7f885c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:23.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f8860e00640 1 -- 172.21.15.143:0/605673485 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0cb9a0 con 0x7f885c0cf740 2024-01-15T18:11:23.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.549+0000 7f885b400640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0b0310 0x7f885c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:23.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8860e00640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0cf740 0x7f885c0cfbf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8854009e50 tx=0x7f885401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31e2a2eb653d5b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:23.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f885aa00640 1 -- 172.21.15.143:0/605673485 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88540a5810 con 0x7f885c0cf740 2024-01-15T18:11:23.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f885aa00640 1 -- 172.21.15.143:0/605673485 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88540a5970 con 0x7f885c0cf740 2024-01-15T18:11:23.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f885aa00640 1 -- 172.21.15.143:0/605673485 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88540a5d00 con 0x7f885c0cf740 2024-01-15T18:11:23.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 -- 172.21.15.143:0/605673485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0cf740 msgr2=0x7f885c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0cf740 0x7f885c0cfbf0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8854009e50 tx=0x7f885401c950 comp rx=0 tx=0).stop 2024-01-15T18:11:23.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 -- 172.21.15.143:0/605673485 shutdown_connections 2024-01-15T18:11:23.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0cf740 0x7f885c0cfbf0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 --2- 172.21.15.143:0/605673485 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0b0310 0x7f885c0cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 -- 172.21.15.143:0/605673485 >> 172.21.15.143:0/605673485 conn(0x7f885c0d0ed0 msgr2=0x7f885c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:23.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 -- 172.21.15.143:0/605673485 shutdown_connections 2024-01-15T18:11:23.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.553+0000 7f8862e00640 1 -- 172.21.15.143:0/605673485 wait complete. 2024-01-15T18:11:23.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 Processor -- start 2024-01-15T18:11:23.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 -- start start 2024-01-15T18:11:23.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 0x7f885c13edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0b0310 0x7f885c13f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0cf740 0x7f885c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c160810 con 0x7f885c06bfb0 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c13fc90 con 0x7f885c0b0310 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c15fc90 con 0x7f885c0cf740 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 0x7f885c13edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 0x7f885c13edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49724/0 (socket says 172.21.15.143:49724) 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 -- 172.21.15.143:0/2148314755 learned_addr learned my addr 172.21.15.143:0/2148314755 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885b400640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0b0310 0x7f885c13f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8860e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0cf740 0x7f885c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 -- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0cf740 msgr2=0x7f885c15eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0cf740 0x7f885c15eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 -- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0b0310 msgr2=0x7f885c13f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0b0310 0x7f885c13f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 -- 172.21.15.143:0/2148314755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8854009b30 con 0x7f885c06bfb0 2024-01-15T18:11:23.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f885be00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 0x7f885c13edb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c3c0 tx=0x7f884c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c0052d0 con 0x7f885c06bfb0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c005430 con 0x7f885c06bfb0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c15ff70 con 0x7f885c06bfb0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c003840 con 0x7f885c06bfb0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.557+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c160530 con 0x7f885c06bfb0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.561+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c067500 con 0x7f885c06bfb0 2024-01-15T18:11:23.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.561+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f884c0039a0 con 0x7f885c06bfb0 2024-01-15T18:11:23.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.561+0000 7f8853e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f88340671f0 0x7f88340696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.561+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f884c0821c0 con 0x7f885c06bfb0 2024-01-15T18:11:23.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.561+0000 7f885b400640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f88340671f0 0x7f88340696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.561+0000 7f885b400640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f88340671f0 0x7f88340696b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f885c1f9210 tx=0x7f8844008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:23.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.565+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f884c050ed0 con 0x7f885c06bfb0 2024-01-15T18:11:23.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.745+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f885c0ce660 con 0x7f885c06bfb0 2024-01-15T18:11:23.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.745+0000 7f8853e00640 1 -- 172.21.15.143:0/2148314755 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f884c017090 con 0x7f885c06bfb0 2024-01-15T18:11:23.750 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:23.751 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3162517776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3162517776},{"type":"v1","addr":"172.21.15.204:6834","nonce":3162517776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/348084006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":348084006},{"type":"v1","addr":"172.21.15.204:6835","nonce":348084006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2759637098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2759637098},{"type":"v1","addr":"172.21.15.143:6835","nonce":2759637098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2832770250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2832770250},{"type":"v1","addr":"172.21.15.143:6837","nonce":2832770250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2024-01-15T18:11:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 38 2024-01-15T18:11:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f88340671f0 msgr2=0x7f88340696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f88340671f0 0x7f88340696b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f885c1f9210 tx=0x7f8844008040 comp rx=0 tx=0).stop 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 msgr2=0x7f885c13edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 0x7f885c13edb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f884c00c3c0 tx=0x7f884c00c880 comp rx=0 tx=0).stop 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 shutdown_connections 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f88340671f0 0x7f88340696b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0cf740 0x7f885c15eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0b0310 0x7f885c13f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 --2- 172.21.15.143:0/2148314755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c06bfb0 0x7f885c13edb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 >> 172.21.15.143:0/2148314755 conn(0x7f885c0d0ed0 msgr2=0x7f885c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 shutdown_connections 2024-01-15T18:11:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.749+0000 7f8862e00640 1 -- 172.21.15.143:0/2148314755 wait complete. 2024-01-15T18:11:23.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:11:23.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 Processor -- start 2024-01-15T18:11:23.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 -- start start 2024-01-15T18:11:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff150157300 0x7ff150159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff1500b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff1500b48f0 0x7ff15015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1500b35a0 con 0x7ff150157300 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1500b4d80 con 0x7ff1500b48f0 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff157800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500b2a20 con 0x7ff1500b21d0 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff154e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff150157300 0x7ff150159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff154e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff150157300 0x7ff150159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49726/0 (socket says 172.21.15.143:49726) 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff154e00640 1 -- 172.21.15.143:0/2181110171 learned_addr learned my addr 172.21.15.143:0/2181110171 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff155800640 1 --2- 172.21.15.143:0/2181110171 >> v2:172.21.15.204:3300/0 conn(0x7ff1500b48f0 0x7ff15015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.909+0000 7ff14fe00640 1 --2- 172.21.15.143:0/2181110171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff1500b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff154e00640 1 -- 172.21.15.143:0/2181110171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 msgr2=0x7ff1500b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff154e00640 1 --2- 172.21.15.143:0/2181110171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff1500b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff154e00640 1 -- 172.21.15.143:0/2181110171 >> v2:172.21.15.204:3300/0 conn(0x7ff1500b48f0 msgr2=0x7ff15015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff154e00640 1 --2- 172.21.15.143:0/2181110171 >> v2:172.21.15.204:3300/0 conn(0x7ff1500b48f0 0x7ff15015d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff154e00640 1 -- 172.21.15.143:0/2181110171 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1500b2d00 con 0x7ff150157300 2024-01-15T18:11:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff154e00640 1 --2- 172.21.15.143:0/2181110171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff150157300 0x7ff150159b70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff15015a0b0 tx=0x7ff14001e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed35dcf321abe2a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff14f400640 1 -- 172.21.15.143:0/2181110171 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140004230 con 0x7ff150157300 2024-01-15T18:11:23.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff14f400640 1 -- 172.21.15.143:0/2181110171 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1400ab070 con 0x7ff150157300 2024-01-15T18:11:23.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.913+0000 7ff14f400640 1 -- 172.21.15.143:0/2181110171 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1400a7810 con 0x7ff150157300 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- 172.21.15.143:0/2181110171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff150157300 msgr2=0x7ff150159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- 172.21.15.143:0/2181110171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff150157300 0x7ff150159b70 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff15015a0b0 tx=0x7ff14001e980 comp rx=0 tx=0).stop 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- 172.21.15.143:0/2181110171 shutdown_connections 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- 172.21.15.143:0/2181110171 >> v2:172.21.15.204:3300/0 conn(0x7ff1500b48f0 0x7ff15015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- 172.21.15.143:0/2181110171 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff1500b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- 172.21.15.143:0/2181110171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff150157300 0x7ff150159b70 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- 172.21.15.143:0/2181110171 >> 172.21.15.143:0/2181110171 conn(0x7ff15006c3f0 msgr2=0x7ff15006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- 172.21.15.143:0/2181110171 shutdown_connections 2024-01-15T18:11:23.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- 172.21.15.143:0/2181110171 wait complete. 2024-01-15T18:11:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 Processor -- start 2024-01-15T18:11:23.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- start start 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff15014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 0x7ff150156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150157300 0x7ff1501491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff150156e80 con 0x7ff1500b48f0 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15014d720 con 0x7ff150157300 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff157800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15014da00 con 0x7ff1500b21d0 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 0x7ff150156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 0x7ff150156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49740/0 (socket says 172.21.15.143:49740) 2024-01-15T18:11:23.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 -- 172.21.15.143:0/2078092195 learned_addr learned my addr 172.21.15.143:0/2078092195 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff155800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150157300 0x7ff1501491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff154e00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff15014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 -- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 msgr2=0x7ff15014e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff15014e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 -- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150157300 msgr2=0x7ff1501491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150157300 0x7ff1501491f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.917+0000 7ff14fe00640 1 -- 172.21.15.143:0/2078092195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14000b780 con 0x7ff1500b48f0 2024-01-15T18:11:23.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14fe00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 0x7ff150156940 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff13800e930 tx=0x7ff13800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138009690 con 0x7ff1500b48f0 2024-01-15T18:11:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13800ca00 con 0x7ff1500b48f0 2024-01-15T18:11:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15014dce0 con 0x7ff1500b48f0 2024-01-15T18:11:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800cd70 con 0x7ff1500b48f0 2024-01-15T18:11:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15014e2a0 con 0x7ff1500b48f0 2024-01-15T18:11:23.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150067500 con 0x7ff1500b48f0 2024-01-15T18:11:23.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7ff138013470 con 0x7ff1500b48f0 2024-01-15T18:11:23.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14cc00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1280671f0 0x7ff1280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:23.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7ff138083060 con 0x7ff1500b48f0 2024-01-15T18:11:23.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.921+0000 7ff154e00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1280671f0 0x7ff1280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:23.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.925+0000 7ff154e00640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1280671f0 0x7ff1280696b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff1400b1000 tx=0x7ff1400b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:23.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:23.925+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff138051c70 con 0x7ff1500b48f0 2024-01-15T18:11:24.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.077+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7ff15006e4e0 con 0x7ff1500b48f0 2024-01-15T18:11:24.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.081+0000 7ff14cc00640 1 -- 172.21.15.143:0/2078092195 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v23) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff138055290 con 0x7ff1500b48f0 2024-01-15T18:11:24.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1280671f0 msgr2=0x7ff1280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1280671f0 0x7ff1280696b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff1400b1000 tx=0x7ff1400b3040 comp rx=0 tx=0).stop 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 msgr2=0x7ff150156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 0x7ff150156940 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff13800e930 tx=0x7ff13800edf0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 shutdown_connections 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1280671f0 0x7ff1280696b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff150157300 0x7ff1501491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1500b48f0 0x7ff150156940 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 --2- 172.21.15.143:0/2078092195 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1500b21d0 0x7ff15014e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 >> 172.21.15.143:0/2078092195 conn(0x7ff15006c3f0 msgr2=0x7ff15015cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 shutdown_connections 2024-01-15T18:11:24.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.085+0000 7ff157800640 1 -- 172.21.15.143:0/2078092195 wait complete. 2024-01-15T18:11:24.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:11:24.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 Processor -- start 2024-01-15T18:11:24.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 -- start start 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc370141660 0x7fc3700ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc370071d00 0x7fc370072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37015f310 con 0x7fc37015ec80 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37015fbb0 con 0x7fc370141660 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc377400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37015f130 con 0x7fc370071d00 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc36fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc37015ec80 0x7fc370141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58780/0 (socket says 172.21.15.143:58780) 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc36fe00640 1 -- 172.21.15.143:0/2297176436 learned_addr learned my addr 172.21.15.143:0/2297176436 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:24.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc375400640 1 --2- 172.21.15.143:0/2297176436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc370071d00 0x7fc370072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc374a00640 1 --2- 172.21.15.143:0/2297176436 >> v2:172.21.15.204:3300/0 conn(0x7fc370141660 0x7fc3700ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc36f400640 1 -- 172.21.15.143:0/2297176436 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786999467 0 0) 0x7fc37015f310 con 0x7fc37015ec80 2024-01-15T18:11:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc36f400640 1 -- 172.21.15.143:0/2297176436 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc350003530 con 0x7fc37015ec80 2024-01-15T18:11:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc375400640 1 -- 172.21.15.143:0/2297176436 >> v2:172.21.15.204:3300/0 conn(0x7fc370141660 msgr2=0x7fc3700ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc375400640 1 --2- 172.21.15.143:0/2297176436 >> v2:172.21.15.204:3300/0 conn(0x7fc370141660 0x7fc3700ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc375400640 1 -- 172.21.15.143:0/2297176436 >> v1:172.21.15.143:6789/0 conn(0x7fc37015ec80 legacy=0x7fc370141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc375400640 1 -- 172.21.15.143:0/2297176436 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37015f5f0 con 0x7fc370071d00 2024-01-15T18:11:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.245+0000 7fc374a00640 1 --2- 172.21.15.143:0/2297176436 >> v2:172.21.15.204:3300/0 conn(0x7fc370141660 0x7fc3700ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc375400640 1 --2- 172.21.15.143:0/2297176436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc370071d00 0x7fc370072150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc368009d90 tx=0x7fc36801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f6b0e2d5d35d0b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc36f400640 1 -- 172.21.15.143:0/2297176436 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680a5900 con 0x7fc370071d00 2024-01-15T18:11:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc36f400640 1 -- 172.21.15.143:0/2297176436 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3680a5a60 con 0x7fc370071d00 2024-01-15T18:11:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc36f400640 1 -- 172.21.15.143:0/2297176436 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680a5df0 con 0x7fc370071d00 2024-01-15T18:11:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc377400640 1 -- 172.21.15.143:0/2297176436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc370071d00 msgr2=0x7fc370072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc377400640 1 --2- 172.21.15.143:0/2297176436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc370071d00 0x7fc370072150 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc368009d90 tx=0x7fc36801c950 comp rx=0 tx=0).stop 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc377400640 1 -- 172.21.15.143:0/2297176436 shutdown_connections 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc377400640 1 --2- 172.21.15.143:0/2297176436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc370071d00 0x7fc370072150 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc377400640 1 --2- 172.21.15.143:0/2297176436 >> v2:172.21.15.204:3300/0 conn(0x7fc370141660 0x7fc3700ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.249+0000 7fc377400640 1 -- 172.21.15.143:0/2297176436 >> 172.21.15.143:0/2297176436 conn(0x7fc37006c3f0 msgr2=0x7fc37006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 -- 172.21.15.143:0/2297176436 shutdown_connections 2024-01-15T18:11:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 -- 172.21.15.143:0/2297176436 wait complete. 2024-01-15T18:11:24.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 Processor -- start 2024-01-15T18:11:24.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 -- start start 2024-01-15T18:11:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 0x7fc3701f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc370141660 0x7fc3700b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc37015ec80 0x7fc3700b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37015dfb0 con 0x7fc370071d00 2024-01-15T18:11:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc370136b00 con 0x7fc370141660 2024-01-15T18:11:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc377400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc370136de0 con 0x7fc37015ec80 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 0x7fc3701f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 0x7fc3701f8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49744/0 (socket says 172.21.15.143:49744) 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 -- 172.21.15.143:0/1708972126 learned_addr learned my addr 172.21.15.143:0/1708972126 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc36fe00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc370141660 0x7fc3700b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc375400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc37015ec80 0x7fc3700b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 -- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc37015ec80 msgr2=0x7fc3700b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc37015ec80 0x7fc3700b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 -- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc370141660 msgr2=0x7fc3700b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc370141660 0x7fc3700b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 -- 172.21.15.143:0/1708972126 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360009420 con 0x7fc370071d00 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.253+0000 7fc374a00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 0x7fc3701f8ff0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc36000ebe0 tx=0x7fc36000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36000eee0 con 0x7fc370071d00 2024-01-15T18:11:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36000cc30 con 0x7fc370071d00 2024-01-15T18:11:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368009a70 con 0x7fc370071d00 2024-01-15T18:11:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360013470 con 0x7fc370071d00 2024-01-15T18:11:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3700cd150 con 0x7fc370071d00 2024-01-15T18:11:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc370067500 con 0x7fc370071d00 2024-01-15T18:11:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fc3600135d0 con 0x7fc370071d00 2024-01-15T18:11:24.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36cc00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3480671f0 0x7fc3480696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fc3600831e0 con 0x7fc370071d00 2024-01-15T18:11:24.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.257+0000 7fc36fe00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3480671f0 0x7fc3480696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.261+0000 7fc36fe00640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3480671f0 0x7fc3480696b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc3701333b0 tx=0x7fc358007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.261+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc360051ca0 con 0x7fc370071d00 2024-01-15T18:11:24.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.413+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc37006e2a0 con 0x7fc370071d00 2024-01-15T18:11:24.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc36cc00640 1 -- 172.21.15.143:0/1708972126 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc3600552c0 con 0x7fc370071d00 2024-01-15T18:11:24.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3480671f0 msgr2=0x7fc3480696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3480671f0 0x7fc3480696b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc3701333b0 tx=0x7fc358007d40 comp rx=0 tx=0).stop 2024-01-15T18:11:24.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 msgr2=0x7fc3701f8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 0x7fc3701f8ff0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc36000ebe0 tx=0x7fc36000c520 comp rx=0 tx=0).stop 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 shutdown_connections 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3480671f0 0x7fc3480696b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc37015ec80 0x7fc3700b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc370141660 0x7fc3700b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 --2- 172.21.15.143:0/1708972126 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc370071d00 0x7fc3701f8ff0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 >> 172.21.15.143:0/1708972126 conn(0x7fc37006c3f0 msgr2=0x7fc3700ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 shutdown_connections 2024-01-15T18:11:24.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.421+0000 7fc377400640 1 -- 172.21.15.143:0/1708972126 wait complete. 2024-01-15T18:11:24.445 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-01-15T18:11:24.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 Processor -- start 2024-01-15T18:11:24.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 -- start start 2024-01-15T18:11:24.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c071300 0x7fcc9c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc9c157700 0x7fcc9c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9c157b90 con 0x7fcc9c070f20 2024-01-15T18:11:24.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc9c06bd40 con 0x7fcc9c157700 2024-01-15T18:11:24.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcca1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c06a490 con 0x7fcc9c071300 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc9c157700 0x7fcc9c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcc9c157700 0x7fcc9c15d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49756/0 (socket says 172.21.15.143:49756) 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 -- 172.21.15.143:0/273743931 learned_addr learned my addr 172.21.15.143:0/273743931 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9a000640 1 --2- 172.21.15.143:0/273743931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c071300 0x7fcc9c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 -- 172.21.15.143:0/273743931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c071300 msgr2=0x7fcc9c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 --2- 172.21.15.143:0/273743931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c071300 0x7fcc9c1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 -- 172.21.15.143:0/273743931 >> v1:172.21.15.204:6789/0 conn(0x7fcc9c070f20 legacy=0x7fcc9c06b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.581+0000 7fcc9b400640 1 -- 172.21.15.143:0/273743931 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c06a770 con 0x7fcc9c157700 2024-01-15T18:11:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcc9b400640 1 --2- 172.21.15.143:0/273743931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c15d6b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009e50 tx=0x7fcc9401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6705ad4bb1dbb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcc99600640 1 -- 172.21.15.143:0/273743931 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940a58a0 con 0x7fcc9c157700 2024-01-15T18:11:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcc99600640 1 -- 172.21.15.143:0/273743931 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc940a5a00 con 0x7fcc9c157700 2024-01-15T18:11:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcc99600640 1 -- 172.21.15.143:0/273743931 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940a5d90 con 0x7fcc9c157700 2024-01-15T18:11:24.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 -- 172.21.15.143:0/273743931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 msgr2=0x7fcc9c15d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 --2- 172.21.15.143:0/273743931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c15d6b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009e50 tx=0x7fcc9401c9e0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 -- 172.21.15.143:0/273743931 shutdown_connections 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 --2- 172.21.15.143:0/273743931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c15d6b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 --2- 172.21.15.143:0/273743931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c071300 0x7fcc9c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 -- 172.21.15.143:0/273743931 >> 172.21.15.143:0/273743931 conn(0x7fcc9c0b1610 msgr2=0x7fcc9c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 -- 172.21.15.143:0/273743931 shutdown_connections 2024-01-15T18:11:24.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.585+0000 7fcca1800640 1 -- 172.21.15.143:0/273743931 wait complete. 2024-01-15T18:11:24.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 Processor -- start 2024-01-15T18:11:24.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 -- start start 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c070f20 0x7fcc9c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9c071300 0x7fcc9c14f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c14fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c150690 con 0x7fcc9c157700 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c146ba0 con 0x7fcc9c071300 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcca1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9c1500d0 con 0x7fcc9c070f20 2024-01-15T18:11:24.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c14fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c14fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49772/0 (socket says 172.21.15.143:49772) 2024-01-15T18:11:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 -- 172.21.15.143:0/819784804 learned_addr learned my addr 172.21.15.143:0/819784804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9aa00640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c070f20 0x7fcc9c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9a000640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9c071300 0x7fcc9c14f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 -- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c070f20 msgr2=0x7fcc9c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c070f20 0x7fcc9c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 -- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9c071300 msgr2=0x7fcc9c14f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9c071300 0x7fcc9c14f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 -- 172.21.15.143:0/819784804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94009b30 con 0x7fcc9c157700 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.589+0000 7fcc9b400640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c14fa50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fcc9401a610 tx=0x7fcc94019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940b8670 con 0x7fcc9c157700 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc940b87d0 con 0x7fcc9c157700 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9c15e9f0 con 0x7fcc9c157700 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940b8b60 con 0x7fcc9c157700 2024-01-15T18:11:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9c146e40 con 0x7fcc9c157700 2024-01-15T18:11:24.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9c074b00 con 0x7fcc9c157700 2024-01-15T18:11:24.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fcc940b8cc0 con 0x7fcc9c157700 2024-01-15T18:11:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc92a00640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc740671f0 0x7fcc740696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fcc9411e210 con 0x7fcc9c157700 2024-01-15T18:11:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.593+0000 7fcc9aa00640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc740671f0 0x7fcc740696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.597+0000 7fcc9aa00640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc740671f0 0x7fcc740696b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c007fd0 tx=0x7fcc8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.597+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc940ecf10 con 0x7fcc9c157700 2024-01-15T18:11:24.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.749+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) v1 -- 0x7fcc9c15e840 con 0x7fcc9c157700 2024-01-15T18:11:24.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.753+0000 7fcc92a00640 1 -- 172.21.15.143:0/819784804 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v25) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcc940f0530 con 0x7fcc9c157700 2024-01-15T18:11:24.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.753+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc740671f0 msgr2=0x7fcc740696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.753+0000 7fcca1800640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc740671f0 0x7fcc740696b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c007fd0 tx=0x7fcc8c009040 comp rx=0 tx=0).stop 2024-01-15T18:11:24.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.753+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 msgr2=0x7fcc9c14fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.753+0000 7fcca1800640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c14fa50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fcc9401a610 tx=0x7fcc94019780 comp rx=0 tx=0).stop 2024-01-15T18:11:24.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 shutdown_connections 2024-01-15T18:11:24.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc740671f0 0x7fcc740696b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9c157700 0x7fcc9c14fa50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9c071300 0x7fcc9c14f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 --2- 172.21.15.143:0/819784804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9c070f20 0x7fcc9c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 >> 172.21.15.143:0/819784804 conn(0x7fcc9c0b1610 msgr2=0x7fcc9c06cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 shutdown_connections 2024-01-15T18:11:24.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.757+0000 7fcca1800640 1 -- 172.21.15.143:0/819784804 wait complete. 2024-01-15T18:11:24.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones' 2024-01-15T18:11:24.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee3dc00640 1 Processor -- start 2024-01-15T18:11:24.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee3dc00640 1 -- start start 2024-01-15T18:11:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee3dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee380cdf20 0x7fee380cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee3dc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee380c4d90 con 0x7fee380c6230 2024-01-15T18:11:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee3dc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3815b640 con 0x7fee3815b260 2024-01-15T18:11:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee3dc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee380cf800 con 0x7fee380cdf20 2024-01-15T18:11:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee36000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee380cdf20 0x7fee380cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee36000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fee380cdf20 0x7fee380cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49778/0 (socket says 172.21.15.143:49778) 2024-01-15T18:11:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee36000640 1 -- 172.21.15.143:0/324067803 learned_addr learned my addr 172.21.15.143:0/324067803 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee37400640 1 --1- 172.21.15.143:0/324067803 >> v1:172.21.15.143:6789/0 conn(0x7fee380c6230 0x7fee380cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58786/0 (socket says 172.21.15.143:58786) 2024-01-15T18:11:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee36000640 1 -- 172.21.15.143:0/324067803 >> v1:172.21.15.204:6789/0 conn(0x7fee3815b260 legacy=0x7fee380cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee36000640 1 -- 172.21.15.143:0/324067803 >> v1:172.21.15.143:6789/0 conn(0x7fee380c6230 legacy=0x7fee380cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.917+0000 7fee36000640 1 -- 172.21.15.143:0/324067803 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380cfae0 con 0x7fee380cdf20 2024-01-15T18:11:24.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee36000640 1 --2- 172.21.15.143:0/324067803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee380cbae0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fee20009870 tx=0x7fee2001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54340d4979d9d0b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee35600640 1 -- 172.21.15.143:0/324067803 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200aa670 con 0x7fee380cdf20 2024-01-15T18:11:24.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee35600640 1 -- 172.21.15.143:0/324067803 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380ccf40 con 0x7fee380cdf20 2024-01-15T18:11:24.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee35600640 1 -- 172.21.15.143:0/324067803 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200aa9f0 con 0x7fee380cdf20 2024-01-15T18:11:24.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 -- 172.21.15.143:0/324067803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 msgr2=0x7fee380cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 --2- 172.21.15.143:0/324067803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee380cbae0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fee20009870 tx=0x7fee2001c510 comp rx=0 tx=0).stop 2024-01-15T18:11:24.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 -- 172.21.15.143:0/324067803 shutdown_connections 2024-01-15T18:11:24.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 --2- 172.21.15.143:0/324067803 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee380cbae0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 -- 172.21.15.143:0/324067803 >> 172.21.15.143:0/324067803 conn(0x7fee380d0ed0 msgr2=0x7fee380d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:24.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 -- 172.21.15.143:0/324067803 shutdown_connections 2024-01-15T18:11:24.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.921+0000 7fee3dc00640 1 -- 172.21.15.143:0/324067803 wait complete. 2024-01-15T18:11:24.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 Processor -- start 2024-01-15T18:11:24.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 -- start start 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee380c6230 0x7fee380ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee3809cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee3815b260 0x7fee3814bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380ca050 con 0x7fee380cdf20 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380cb030 con 0x7fee380c6230 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee3dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380cb310 con 0x7fee3815b260 2024-01-15T18:11:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee3809cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee3809cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49794/0 (socket says 172.21.15.143:49794) 2024-01-15T18:11:24.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 -- 172.21.15.143:0/258507749 learned_addr learned my addr 172.21.15.143:0/258507749 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:24.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee37400640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee3815b260 0x7fee3814bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36a00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee380c6230 0x7fee380ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 -- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee3815b260 msgr2=0x7fee3814bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee3815b260 0x7fee3814bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 -- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee380c6230 msgr2=0x7fee380ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee380c6230 0x7fee380ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 -- 172.21.15.143:0/258507749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200094d0 con 0x7fee380cdf20 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee36000640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee3809cf10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fee20019120 tx=0x7fee2001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.925+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200b3800 con 0x7fee380cdf20 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee200b3980 con 0x7fee380cdf20 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee380c9a00 con 0x7fee380cdf20 2024-01-15T18:11:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee200b3d30 con 0x7fee380cdf20 2024-01-15T18:11:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee380caa20 con 0x7fee380cdf20 2024-01-15T18:11:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee380cb450 con 0x7fee380cdf20 2024-01-15T18:11:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fee200b2380 con 0x7fee380cdf20 2024-01-15T18:11:24.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee2ea00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee100671f0 0x7fee100696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:24.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fee2011d830 con 0x7fee380cdf20 2024-01-15T18:11:24.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.929+0000 7fee36a00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee100671f0 0x7fee100696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:24.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.933+0000 7fee36a00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee100671f0 0x7fee100696b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fee28007fd0 tx=0x7fee28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:24.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:24.933+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fee380cb450 con 0x7fee380cdf20 2024-01-15T18:11:25.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.137+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]} v 0) v1 -- 0x7fee380cb450 con 0x7fee380cdf20 2024-01-15T18:11:25.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.785+0000 7fee2ea00640 1 -- 172.21.15.143:0/258507749 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]}]=0 v326) v1 ==== 154+0+0 (secure 0 0 0) 0x7fee380cb450 con 0x7fee380cdf20 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee100671f0 msgr2=0x7fee100696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee100671f0 0x7fee100696b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fee28007fd0 tx=0x7fee28009040 comp rx=0 tx=0).stop 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 msgr2=0x7fee3809cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee3809cf10 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fee20019120 tx=0x7fee2001ad50 comp rx=0 tx=0).stop 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 shutdown_connections 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fee100671f0 0x7fee100696b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fee3815b260 0x7fee3814bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fee380cdf20 0x7fee3809cf10 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 --2- 172.21.15.143:0/258507749 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee380c6230 0x7fee380ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 >> 172.21.15.143:0/258507749 conn(0x7fee380d0ed0 msgr2=0x7fee380b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 shutdown_connections 2024-01-15T18:11:25.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.789+0000 7fee3dc00640 1 -- 172.21.15.143:0/258507749 wait complete. 2024-01-15T18:11:25.816 INFO:tasks.cephfs_test_runner:X = number of clones issued. ... ok 2024-01-15T18:11:25.817 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgress) X = number of clones issued. 2024-01-15T18:11:25.817 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:11:25.817 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:25.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.953+0000 7f6d21000640 1 Processor -- start 2024-01-15T18:11:25.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 -- start start 2024-01-15T18:11:25.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c157300 0x7f6d1c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d1c0b21d0 0x7f6d1c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d1c0b48f0 0x7f6d1c15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b35a0 con 0x7f6d1c0b21d0 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b4d80 con 0x7f6d1c0b48f0 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d21000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b2a20 con 0x7f6d1c157300 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d1c0b21d0 0x7f6d1c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d1c0b21d0 0x7f6d1c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49796/0 (socket says 172.21.15.143:49796) 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 -- 172.21.15.143:0/4093224913 learned_addr learned my addr 172.21.15.143:0/4093224913 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19e00640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c157300 0x7f6d1c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d1a800640 1 --2- 172.21.15.143:0/4093224913 >> v2:172.21.15.204:3300/0 conn(0x7f6d1c0b48f0 0x7f6d1c15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 -- 172.21.15.143:0/4093224913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c157300 msgr2=0x7f6d1c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c157300 0x7f6d1c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 -- 172.21.15.143:0/4093224913 >> v2:172.21.15.204:3300/0 conn(0x7f6d1c0b48f0 msgr2=0x7f6d1c15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 --2- 172.21.15.143:0/4093224913 >> v2:172.21.15.204:3300/0 conn(0x7f6d1c0b48f0 0x7f6d1c15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19400640 1 -- 172.21.15.143:0/4093224913 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0b2d00 con 0x7f6d1c0b21d0 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d19e00640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c157300 0x7f6d1c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:25.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.957+0000 7f6d1a800640 1 --2- 172.21.15.143:0/4093224913 >> v2:172.21.15.204:3300/0 conn(0x7f6d1c0b48f0 0x7f6d1c15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:25.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d19400640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c0b21d0 0x7f6d1c0b43b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c15a0b0 tx=0x7f6d0401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b5aa5af0b38f1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:25.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d18a00640 1 -- 172.21.15.143:0/4093224913 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d040a7c30 con 0x7f6d1c0b21d0 2024-01-15T18:11:25.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d18a00640 1 -- 172.21.15.143:0/4093224913 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d040a7d90 con 0x7f6d1c0b21d0 2024-01-15T18:11:25.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d18a00640 1 -- 172.21.15.143:0/4093224913 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d040af660 con 0x7f6d1c0b21d0 2024-01-15T18:11:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 -- 172.21.15.143:0/4093224913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c0b21d0 msgr2=0x7f6d1c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c0b21d0 0x7f6d1c0b43b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c15a0b0 tx=0x7f6d0401eba0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 -- 172.21.15.143:0/4093224913 shutdown_connections 2024-01-15T18:11:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 --2- 172.21.15.143:0/4093224913 >> v2:172.21.15.204:3300/0 conn(0x7f6d1c0b48f0 0x7f6d1c15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c0b21d0 0x7f6d1c0b43b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 --2- 172.21.15.143:0/4093224913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c157300 0x7f6d1c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 -- 172.21.15.143:0/4093224913 >> 172.21.15.143:0/4093224913 conn(0x7f6d1c06c3f0 msgr2=0x7f6d1c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 -- 172.21.15.143:0/4093224913 shutdown_connections 2024-01-15T18:11:25.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.961+0000 7f6d21000640 1 -- 172.21.15.143:0/4093224913 wait complete. 2024-01-15T18:11:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 Processor -- start 2024-01-15T18:11:25.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 -- start start 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c0b21d0 0x7f6d1c14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1c0b48f0 0x7f6d1c071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 0x7f6d1c146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0ad2f0 con 0x7f6d1c157300 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0ae150 con 0x7f6d1c0b48f0 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0ae5e0 con 0x7f6d1c0b21d0 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 0x7f6d1c146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 0x7f6d1c146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49800/0 (socket says 172.21.15.143:49800) 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 -- 172.21.15.143:0/4160289758 learned_addr learned my addr 172.21.15.143:0/4160289758 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:25.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d19e00640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c0b21d0 0x7f6d1c14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d19400640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1c0b48f0 0x7f6d1c071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 -- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c0b21d0 msgr2=0x7f6d1c14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c0b21d0 0x7f6d1c14d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 -- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1c0b48f0 msgr2=0x7f6d1c071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1c0b48f0 0x7f6d1c071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 -- 172.21.15.143:0/4160289758 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0400b770 con 0x7f6d1c157300 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d1a800640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 0x7f6d1c146570 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6d14007d60 tx=0x7f6d1400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d140039a0 con 0x7f6d1c157300 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1400f040 con 0x7f6d1c157300 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c0ad540 con 0x7f6d1c157300 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14003d40 con 0x7f6d1c157300 2024-01-15T18:11:25.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.965+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c0adb00 con 0x7f6d1c157300 2024-01-15T18:11:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.969+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c072710 con 0x7f6d1c157300 2024-01-15T18:11:25.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.969+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f6d14004030 con 0x7f6d1c157300 2024-01-15T18:11:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.969+0000 7f6d12000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf4067220 0x7f6cf40696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.969+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f6d14083060 con 0x7f6d1c157300 2024-01-15T18:11:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.969+0000 7f6d19e00640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf4067220 0x7f6cf40696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:25.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.969+0000 7f6d19e00640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf4067220 0x7f6cf40696e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c006e90 tx=0x7f6d0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:25.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:25.973+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d14051bb0 con 0x7f6d1c157300 2024-01-15T18:11:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.121+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c0ae2c0 con 0x7f6d1c157300 2024-01-15T18:11:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d12000640 1 -- 172.21.15.143:0/4160289758 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13067 (secure 0 0 0) 0x7f6d140551d0 con 0x7f6d1c157300 2024-01-15T18:11:26.127 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:26.127 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":63,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:19.753876+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/3767686465":"2024-01-16T18:11:16.733260+0000","172.21.15.143:6836/3767686465":"2024-01-16T18:11:16.733260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf4067220 msgr2=0x7f6cf40696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf4067220 0x7f6cf40696e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c006e90 tx=0x7f6d0c008040 comp rx=0 tx=0).stop 2024-01-15T18:11:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 msgr2=0x7f6d1c146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 0x7f6d1c146570 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6d14007d60 tx=0x7f6d1400d4a0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 shutdown_connections 2024-01-15T18:11:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf4067220 0x7f6cf40696e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d1c157300 0x7f6d1c146570 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d1c0b48f0 0x7f6d1c071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 --2- 172.21.15.143:0/4160289758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d1c0b21d0 0x7f6d1c14d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 >> 172.21.15.143:0/4160289758 conn(0x7f6d1c06c3f0 msgr2=0x7f6d1c15ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 shutdown_connections 2024-01-15T18:11:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.125+0000 7f6d21000640 1 -- 172.21.15.143:0/4160289758 wait complete. 2024-01-15T18:11:26.150 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:11:26.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:11:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.285+0000 7f4eb1e00640 1 Processor -- start 2024-01-15T18:11:26.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.285+0000 7f4eb1e00640 1 -- start start 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.285+0000 7f4eb1e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4eac0cfb30 0x7f4eac06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.285+0000 7f4eb1e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac0cffd0 con 0x7f4eac1607c0 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.285+0000 7f4eb1e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac0d02b0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.285+0000 7f4eb1e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eac0d1ce0 con 0x7f4eac0cfb30 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaaa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4eac1607c0 0x7f4eac06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58788/0 (socket says 172.21.15.143:58788) 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaaa00640 1 -- 172.21.15.143:0/664322550 learned_addr learned my addr 172.21.15.143:0/664322550 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaa000640 1 --2- 172.21.15.143:0/664322550 >> v2:172.21.15.204:3300/0 conn(0x7f4eac0cfb30 0x7f4eac06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4ea9600640 1 -- 172.21.15.143:0/664322550 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356166278 0 0) 0x7f4eac0cffd0 con 0x7f4eac1607c0 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4ea9600640 1 -- 172.21.15.143:0/664322550 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e8c003530 con 0x7f4eac1607c0 2024-01-15T18:11:26.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4ea9600640 1 -- 172.21.15.143:0/664322550 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1757339747 0 0) 0x7f4e8c003530 con 0x7f4eac1607c0 2024-01-15T18:11:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaa000640 1 -- 172.21.15.143:0/664322550 >> v1:172.21.15.204:6789/0 conn(0x7f4eac0cbf50 legacy=0x7f4eac0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaa000640 1 -- 172.21.15.143:0/664322550 >> v1:172.21.15.143:6789/0 conn(0x7f4eac1607c0 legacy=0x7f4eac06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaa000640 1 -- 172.21.15.143:0/664322550 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0cb470 con 0x7f4eac0cfb30 2024-01-15T18:11:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.289+0000 7f4eaa000640 1 --2- 172.21.15.143:0/664322550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac06bec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4e940097c0 tx=0x7f4e9401eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0fdc2784414d29 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4ea9600640 1 -- 172.21.15.143:0/664322550 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c001680 con 0x7f4eac0cfb30 2024-01-15T18:11:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4ea9600640 1 -- 172.21.15.143:0/664322550 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e940a8610 con 0x7f4eac0cfb30 2024-01-15T18:11:26.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4ea9600640 1 -- 172.21.15.143:0/664322550 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a89a0 con 0x7f4eac0cfb30 2024-01-15T18:11:26.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 -- 172.21.15.143:0/664322550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 msgr2=0x7f4eac06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/664322550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac06bec0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4e940097c0 tx=0x7f4e9401eb90 comp rx=0 tx=0).stop 2024-01-15T18:11:26.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 -- 172.21.15.143:0/664322550 shutdown_connections 2024-01-15T18:11:26.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/664322550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac06bec0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 -- 172.21.15.143:0/664322550 >> 172.21.15.143:0/664322550 conn(0x7f4eac0d0e20 msgr2=0x7f4eac0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:26.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 -- 172.21.15.143:0/664322550 shutdown_connections 2024-01-15T18:11:26.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 -- 172.21.15.143:0/664322550 wait complete. 2024-01-15T18:11:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 Processor -- start 2024-01-15T18:11:26.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.293+0000 7f4eb1e00640 1 -- start start 2024-01-15T18:11:26.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 0x7f4eac1f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac1607c0 0x7f4eac15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1f3fb0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac157930 con 0x7f4eac0cfb30 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac1f3340 con 0x7f4eac1607c0 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 0x7f4eac1f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 0x7f4eac1f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49802/0 (socket says 172.21.15.143:49802) 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 -- 172.21.15.143:0/2416157633 learned_addr learned my addr 172.21.15.143:0/2416157633 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaa000640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eab400640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac1607c0 0x7f4eac15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 -- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac1607c0 msgr2=0x7f4eac15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac1607c0 0x7f4eac15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 -- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 msgr2=0x7f4eac1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 -- 172.21.15.143:0/2416157633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e94009420 con 0x7f4eac0cbf50 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaa000640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eaaa00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 0x7f4eac1f2d60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c780 tx=0x7f4e9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c005ae0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c005c40 con 0x7f4eac0cbf50 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c007070 con 0x7f4eac0cbf50 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac1f3620 con 0x7f4eac0cbf50 2024-01-15T18:11:26.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac1f3be0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac06d4b0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.297+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f4e9c0071d0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.301+0000 7f4ea3400640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e84067220 0x7f4e840696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.301+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f4e9c0828f0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.301+0000 7f4eaa000640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e84067220 0x7f4e840696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.301+0000 7f4eaa000640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e84067220 0x7f4e840696e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4e940b2000 tx=0x7f4e94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:26.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.305+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e9c0516a0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.453+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4eac067500 con 0x7f4eac0cbf50 2024-01-15T18:11:26.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.453+0000 7f4ea3400640 1 -- 172.21.15.143:0/2416157633 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e9c054cc0 con 0x7f4eac0cbf50 2024-01-15T18:11:26.457 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e84067220 msgr2=0x7f4e840696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e84067220 0x7f4e840696e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4e940b2000 tx=0x7f4e94008040 comp rx=0 tx=0).stop 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 msgr2=0x7f4eac1f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 0x7f4eac1f2d60 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c780 tx=0x7f4e9c00cc40 comp rx=0 tx=0).stop 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 shutdown_connections 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4e84067220 0x7f4e840696e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4eac1607c0 0x7f4eac15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4eac0cfb30 0x7f4eac1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 --2- 172.21.15.143:0/2416157633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4eac0cbf50 0x7f4eac1f2d60 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 >> 172.21.15.143:0/2416157633 conn(0x7f4eac0d0e20 msgr2=0x7f4eac06fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 shutdown_connections 2024-01-15T18:11:26.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.457+0000 7f4eb1e00640 1 -- 172.21.15.143:0/2416157633 wait complete. 2024-01-15T18:11:26.480 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added' 2024-01-15T18:11:26.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.613+0000 7f8a49e00640 1 Processor -- start 2024-01-15T18:11:26.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.613+0000 7f8a49e00640 1 -- start start 2024-01-15T18:11:26.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a49e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a440b0490 0x7f8a4406bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a4406c460 0x7f8a440b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a49e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a440cb480 con 0x7f8a4415d3f0 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a49e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4406c8f0 con 0x7f8a440b0490 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a49e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440cb6d0 con 0x7f8a4406c460 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a440b0490 0x7f8a4406bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a440b0490 0x7f8a4406bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49808/0 (socket says 172.21.15.143:49808) 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 -- 172.21.15.143:0/2359870363 learned_addr learned my addr 172.21.15.143:0/2359870363 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a43400640 1 --2- 172.21.15.143:0/2359870363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a4406c460 0x7f8a440b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 -- 172.21.15.143:0/2359870363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a4406c460 msgr2=0x7f8a440b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:26.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 --2- 172.21.15.143:0/2359870363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a4406c460 0x7f8a440b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 -- 172.21.15.143:0/2359870363 >> v1:172.21.15.143:6789/0 conn(0x7f8a4415d3f0 legacy=0x7f8a440cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.617+0000 7f8a42000640 1 -- 172.21.15.143:0/2359870363 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440cb9b0 con 0x7f8a440b0490 2024-01-15T18:11:26.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a42000640 1 --2- 172.21.15.143:0/2359870363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a440b0490 0x7f8a4406bf20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009870 tx=0x7f8a2c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23743cdcbb1db6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:26.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a41600640 1 -- 172.21.15.143:0/2359870363 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0aa670 con 0x7f8a440b0490 2024-01-15T18:11:26.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a41600640 1 -- 172.21.15.143:0/2359870363 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c0aa7d0 con 0x7f8a440b0490 2024-01-15T18:11:26.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a41600640 1 -- 172.21.15.143:0/2359870363 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0aab60 con 0x7f8a440b0490 2024-01-15T18:11:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 -- 172.21.15.143:0/2359870363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a440b0490 msgr2=0x7f8a4406bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 --2- 172.21.15.143:0/2359870363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a440b0490 0x7f8a4406bf20 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009870 tx=0x7f8a2c01c510 comp rx=0 tx=0).stop 2024-01-15T18:11:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 -- 172.21.15.143:0/2359870363 shutdown_connections 2024-01-15T18:11:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 --2- 172.21.15.143:0/2359870363 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a4406c460 0x7f8a440b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 --2- 172.21.15.143:0/2359870363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a440b0490 0x7f8a4406bf20 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 -- 172.21.15.143:0/2359870363 >> 172.21.15.143:0/2359870363 conn(0x7f8a440d0e30 msgr2=0x7f8a440d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 -- 172.21.15.143:0/2359870363 shutdown_connections 2024-01-15T18:11:26.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 -- 172.21.15.143:0/2359870363 wait complete. 2024-01-15T18:11:26.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 Processor -- start 2024-01-15T18:11:26.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.621+0000 7f8a49e00640 1 -- start start 2024-01-15T18:11:26.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4406c460 0x7f8a4415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a440b0490 0x7f8a4415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 0x7f8a440ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4415f450 con 0x7f8a4415d3f0 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4415e8d0 con 0x7f8a4406c460 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4415ebb0 con 0x7f8a440b0490 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 0x7f8a440ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 0x7f8a440ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49824/0 (socket says 172.21.15.143:49824) 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 -- 172.21.15.143:0/1468637802 learned_addr learned my addr 172.21.15.143:0/1468637802 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a42000640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a440b0490 0x7f8a4415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a42a00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4406c460 0x7f8a4415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 -- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a440b0490 msgr2=0x7f8a4415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a440b0490 0x7f8a4415e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 -- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4406c460 msgr2=0x7f8a4415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4406c460 0x7f8a4415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 -- 172.21.15.143:0/1468637802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c0094d0 con 0x7f8a4415d3f0 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a42a00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4406c460 0x7f8a4415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:26.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a43400640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 0x7f8a440ae1e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00efc0 tx=0x7f8a3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c009ba0 con 0x7f8a4415d3f0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c00f040 con 0x7f8a4415d3f0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0146c0 con 0x7f8a4415d3f0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4415ee90 con 0x7f8a4415d3f0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a44152240 con 0x7f8a4415d3f0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.625+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a44067500 con 0x7f8a4415d3f0 2024-01-15T18:11:26.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.629+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f8a3c014820 con 0x7f8a4415d3f0 2024-01-15T18:11:26.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.629+0000 7f8a3b400640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a1c0671f0 0x7f8a1c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:26.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.629+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8a3c083720 con 0x7f8a4415d3f0 2024-01-15T18:11:26.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.629+0000 7f8a42a00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a1c0671f0 0x7f8a1c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:26.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.629+0000 7f8a42a00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a1c0671f0 0x7f8a1c0696b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000f70 tx=0x7f8a34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:26.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.633+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a3c052410 con 0x7f8a4415d3f0 2024-01-15T18:11:26.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:26.841+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]} v 0) v1 -- 0x7f8a440caf30 con 0x7f8a4415d3f0 2024-01-15T18:11:27.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.793+0000 7f8a3b400640 1 -- 172.21.15.143:0/1468637802 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]}]=0 v328) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8a3c0180f0 con 0x7f8a4415d3f0 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a1c0671f0 msgr2=0x7f8a1c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a1c0671f0 0x7f8a1c0696b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8a34000f70 tx=0x7f8a34009040 comp rx=0 tx=0).stop 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 msgr2=0x7f8a440ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 0x7f8a440ae1e0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00efc0 tx=0x7f8a3c00c490 comp rx=0 tx=0).stop 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 shutdown_connections 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a1c0671f0 0x7f8a1c0696b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4415d3f0 0x7f8a440ae1e0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a440b0490 0x7f8a4415e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 --2- 172.21.15.143:0/1468637802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a4406c460 0x7f8a4415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 >> 172.21.15.143:0/1468637802 conn(0x7f8a440d0e30 msgr2=0x7f8a440d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 shutdown_connections 2024-01-15T18:11:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.797+0000 7f8a49e00640 1 -- 172.21.15.143:0/1468637802 wait complete. 2024-01-15T18:11:27.825 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:11:27.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 Processor -- start 2024-01-15T18:11:27.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 -- start start 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f29f01607c0 0x7f29f006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29f00cfb30 0x7f29f006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00cffd0 con 0x7f29f006c230 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f00d1ce0 con 0x7f29f00cfb30 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.965+0000 7f29f8a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f00d02b0 con 0x7f29f01607c0 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29f00cfb30 0x7f29f006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f29f00cfb30 0x7f29f006bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55134/0 (socket says 172.21.15.143:55134) 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 -- 172.21.15.143:0/3914469781 learned_addr learned my addr 172.21.15.143:0/3914469781 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f6000640 1 --2- 172.21.15.143:0/3914469781 >> v2:172.21.15.204:3300/0 conn(0x7f29f01607c0 0x7f29f006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 -- 172.21.15.143:0/3914469781 >> v2:172.21.15.204:3300/0 conn(0x7f29f01607c0 msgr2=0x7f29f006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 --2- 172.21.15.143:0/3914469781 >> v2:172.21.15.204:3300/0 conn(0x7f29f01607c0 0x7f29f006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 -- 172.21.15.143:0/3914469781 >> v1:172.21.15.204:6789/0 conn(0x7f29f006c230 legacy=0x7f29f00cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:27.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 -- 172.21.15.143:0/3914469781 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f00cb470 con 0x7f29f00cfb30 2024-01-15T18:11:27.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f5600640 1 --2- 172.21.15.143:0/3914469781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f006bcf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f29f0071a00 tx=0x7f29d801ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9345608a92772a24 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.969+0000 7f29f4c00640 1 -- 172.21.15.143:0/3914469781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80a89b0 con 0x7f29f00cfb30 2024-01-15T18:11:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f4c00640 1 -- 172.21.15.143:0/3914469781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d80a8b10 con 0x7f29f00cfb30 2024-01-15T18:11:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f4c00640 1 -- 172.21.15.143:0/3914469781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80a8ea0 con 0x7f29f00cfb30 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- 172.21.15.143:0/3914469781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 msgr2=0x7f29f006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 --2- 172.21.15.143:0/3914469781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f006bcf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f29f0071a00 tx=0x7f29d801ecb0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- 172.21.15.143:0/3914469781 shutdown_connections 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 --2- 172.21.15.143:0/3914469781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f006bcf0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 --2- 172.21.15.143:0/3914469781 >> v2:172.21.15.204:3300/0 conn(0x7f29f01607c0 0x7f29f006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- 172.21.15.143:0/3914469781 >> 172.21.15.143:0/3914469781 conn(0x7f29f00d0e20 msgr2=0x7f29f00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:27.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- 172.21.15.143:0/3914469781 shutdown_connections 2024-01-15T18:11:27.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- 172.21.15.143:0/3914469781 wait complete. 2024-01-15T18:11:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 Processor -- start 2024-01-15T18:11:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- start start 2024-01-15T18:11:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f006c230 0x7f29f006e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f0157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29f01607c0 0x7f29f01547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0155bf0 con 0x7f29f00cfb30 2024-01-15T18:11:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01588a0 con 0x7f29f006c230 2024-01-15T18:11:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.973+0000 7f29f8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0158b80 con 0x7f29f01607c0 2024-01-15T18:11:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f0157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f0157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55150/0 (socket says 172.21.15.143:55150) 2024-01-15T18:11:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 -- 172.21.15.143:0/2207532209 learned_addr learned my addr 172.21.15.143:0/2207532209 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f6000640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f006c230 0x7f29f006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:27.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f6a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29f01607c0 0x7f29f01547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 -- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29f01607c0 msgr2=0x7f29f01547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29f01607c0 0x7f29f01547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 -- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f006c230 msgr2=0x7f29f006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f006c230 0x7f29f006e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 -- 172.21.15.143:0/2207532209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80094d0 con 0x7f29f00cfb30 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f5600640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f0157b60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f29d80b3040 tx=0x7f29d801ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80b0e80 con 0x7f29f00cfb30 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d80bb480 con 0x7f29f00cfb30 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80bb810 con 0x7f29f00cfb30 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f0155530 con 0x7f29f00cfb30 2024-01-15T18:11:27.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f0158500 con 0x7f29f00cfb30 2024-01-15T18:11:27.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f006d4b0 con 0x7f29f00cfb30 2024-01-15T18:11:27.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.977+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f29d80bb970 con 0x7f29f00cfb30 2024-01-15T18:11:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.981+0000 7f29ee000640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29c80671f0 0x7f29c80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.981+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f29d81213d0 con 0x7f29f00cfb30 2024-01-15T18:11:27.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.981+0000 7f29f6000640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29c80671f0 0x7f29c80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:27.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.981+0000 7f29f6000640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29c80671f0 0x7f29c80696b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29e00079c0 tx=0x7f29e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:27.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:27.985+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f29d80eff20 con 0x7f29f00cfb30 2024-01-15T18:11:28.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.133+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f29f006d4b0 con 0x7f29f00cfb30 2024-01-15T18:11:28.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.137+0000 7f29ee000640 1 -- 172.21.15.143:0/2207532209 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f29d80b50c0 con 0x7f29f00cfb30 2024-01-15T18:11:28.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29c80671f0 msgr2=0x7f29c80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29c80671f0 0x7f29c80696b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f29e00079c0 tx=0x7f29e0008040 comp rx=0 tx=0).stop 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 msgr2=0x7f29f0157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f0157b60 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f29d80b3040 tx=0x7f29d801ba50 comp rx=0 tx=0).stop 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 shutdown_connections 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f29c80671f0 0x7f29c80696b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29f01607c0 0x7f29f01547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f00cfb30 0x7f29f0157b60 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 --2- 172.21.15.143:0/2207532209 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29f006c230 0x7f29f006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 >> 172.21.15.143:0/2207532209 conn(0x7f29f00d0e20 msgr2=0x7f29f015d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 shutdown_connections 2024-01-15T18:11:28.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.141+0000 7f29f8a00640 1 -- 172.21.15.143:0/2207532209 wait complete. 2024-01-15T18:11:28.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:11:28.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:11:28.217 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:11:28.313 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:11:28.313 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:11:28.313 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:28.321 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:11:28.322 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:11:28.322 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:28.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:11:28.331 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:11:28.331 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:28.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:28.370 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:28.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:11:28.371 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:11:28.371 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:28.388 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:11:28.388 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:11:28.388 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:11:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 Processor -- start 2024-01-15T18:11:28.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 -- start start 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9c80713d0 0x7fd9c806ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd9c806bab0 0x7fd9c815d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c8160c90 con 0x7fd9c80b2c30 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c8160f70 con 0x7fd9c80713d0 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9ccc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c8161250 con 0x7fd9c806bab0 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9c5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9c80713d0 0x7fd9c806ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9c5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd9c80713d0 0x7fd9c806ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55152/0 (socket says 172.21.15.143:55152) 2024-01-15T18:11:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.545+0000 7fd9c5400640 1 -- 172.21.15.143:0/3386632825 learned_addr learned my addr 172.21.15.143:0/3386632825 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c6800640 1 --2- 172.21.15.143:0/3386632825 >> v2:172.21.15.204:3300/0 conn(0x7fd9c806bab0 0x7fd9c815d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c5400640 1 -- 172.21.15.143:0/3386632825 >> v2:172.21.15.204:3300/0 conn(0x7fd9c806bab0 msgr2=0x7fd9c815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c5400640 1 --2- 172.21.15.143:0/3386632825 >> v2:172.21.15.204:3300/0 conn(0x7fd9c806bab0 0x7fd9c815d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c5400640 1 -- 172.21.15.143:0/3386632825 >> v1:172.21.15.204:6789/0 conn(0x7fd9c80b2c30 legacy=0x7fd9c8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:28.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c5400640 1 -- 172.21.15.143:0/3386632825 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8161530 con 0x7fd9c80713d0 2024-01-15T18:11:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c5400640 1 --2- 172.21.15.143:0/3386632825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80713d0 0x7fd9c806ccc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80cf4e0 tx=0x7fd9b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfbb59a2eb40153 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c4a00640 1 -- 172.21.15.143:0/3386632825 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00ac410 con 0x7fd9c80713d0 2024-01-15T18:11:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c4a00640 1 -- 172.21.15.143:0/3386632825 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b00ac570 con 0x7fd9c80713d0 2024-01-15T18:11:28.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.549+0000 7fd9c4a00640 1 -- 172.21.15.143:0/3386632825 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00ac900 con 0x7fd9c80713d0 2024-01-15T18:11:28.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- 172.21.15.143:0/3386632825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80713d0 msgr2=0x7fd9c806ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/3386632825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80713d0 0x7fd9c806ccc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80cf4e0 tx=0x7fd9b001c490 comp rx=0 tx=0).stop 2024-01-15T18:11:28.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- 172.21.15.143:0/3386632825 shutdown_connections 2024-01-15T18:11:28.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/3386632825 >> v2:172.21.15.204:3300/0 conn(0x7fd9c806bab0 0x7fd9c815d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/3386632825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80713d0 0x7fd9c806ccc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- 172.21.15.143:0/3386632825 >> 172.21.15.143:0/3386632825 conn(0x7fd9c80b1610 msgr2=0x7fd9c80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:28.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- 172.21.15.143:0/3386632825 shutdown_connections 2024-01-15T18:11:28.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- 172.21.15.143:0/3386632825 wait complete. 2024-01-15T18:11:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 Processor -- start 2024-01-15T18:11:28.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- start start 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9c806bab0 0x7fd9c8150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c80713d0 0x7fd9c8155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 0x7fd9c814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c814ea90 con 0x7fd9c80b2c30 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8155d90 con 0x7fd9c80713d0 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9ccc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c814eca0 con 0x7fd9c806bab0 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 0x7fd9c814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 0x7fd9c814cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55164/0 (socket says 172.21.15.143:55164) 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 -- 172.21.15.143:0/1194689715 learned_addr learned my addr 172.21.15.143:0/1194689715 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c5400640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c80713d0 0x7fd9c8155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c5e00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9c806bab0 0x7fd9c8150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 -- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9c806bab0 msgr2=0x7fd9c8150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9c806bab0 0x7fd9c8150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 -- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c80713d0 msgr2=0x7fd9c8155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c80713d0 0x7fd9c8155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.553+0000 7fd9c6800640 1 -- 172.21.15.143:0/1194689715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00094d0 con 0x7fd9c80b2c30 2024-01-15T18:11:28.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9c6800640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 0x7fd9c814cf00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0007e40 tx=0x7fd9c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0014620 con 0x7fd9c80b2c30 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c000ee90 con 0x7fd9c80b2c30 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c814ef80 con 0x7fd9c80b2c30 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00149e0 con 0x7fd9c80b2c30 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c8155980 con 0x7fd9c80b2c30 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c8074b00 con 0x7fd9c80b2c30 2024-01-15T18:11:28.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fd9c0014b40 con 0x7fd9c80b2c30 2024-01-15T18:11:28.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9be000640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd9a0067220 0x7fd9a00696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fd9c0083580 con 0x7fd9c80b2c30 2024-01-15T18:11:28.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.557+0000 7fd9c5e00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd9a0067220 0x7fd9a00696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.561+0000 7fd9c5e00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd9a0067220 0x7fd9a00696e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b810 tx=0x7fd9b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:28.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.561+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd9c0052270 con 0x7fd9c80b2c30 2024-01-15T18:11:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.745+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9c8067500 con 0x7fd9c80b2c30 2024-01-15T18:11:28.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.745+0000 7fd9be000640 1 -- 172.21.15.143:0/1194689715 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fd9c0055890 con 0x7fd9c80b2c30 2024-01-15T18:11:28.748 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:28.749 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5678,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3162517776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3162517776},{"type":"v1","addr":"172.21.15.204:6834","nonce":3162517776}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/348084006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":348084006},{"type":"v1","addr":"172.21.15.204:6835","nonce":348084006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6333,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2759637098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2759637098},{"type":"v1","addr":"172.21.15.143:6835","nonce":2759637098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2832770250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2832770250},{"type":"v1","addr":"172.21.15.143:6837","nonce":2832770250}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2024-01-15T18:11:28.749 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 38 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd9a0067220 msgr2=0x7fd9a00696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd9a0067220 0x7fd9a00696e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800b810 tx=0x7fd9b8008040 comp rx=0 tx=0).stop 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 msgr2=0x7fd9c814cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 0x7fd9c814cf00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0007e40 tx=0x7fd9c000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 shutdown_connections 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd9a0067220 0x7fd9a00696e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd9c80b2c30 0x7fd9c814cf00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9c80713d0 0x7fd9c8155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 --2- 172.21.15.143:0/1194689715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd9c806bab0 0x7fd9c8150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 >> 172.21.15.143:0/1194689715 conn(0x7fd9c80b1610 msgr2=0x7fd9c806db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 shutdown_connections 2024-01-15T18:11:28.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.749+0000 7fd9ccc00640 1 -- 172.21.15.143:0/1194689715 wait complete. 2024-01-15T18:11:28.771 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:11:28.771 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:11:28.772 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:11:28.772 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:11:28.772 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:11:28.772 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:11:28.772 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:11:28.772 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:11:28.773 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:11:28.773 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:11:28.773 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:11:28.773 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:11:28.773 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:11:28.773 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:11:28.774 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:11:28.774 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:11:28.787 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:11:28.787 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:11:28.788 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:11:28.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:11:28.789 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:11:28.789 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:11:28.790 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:11:28.790 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:11:28.833 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:11:28.833 DEBUG:teuthology.parallel:result is None 2024-01-15T18:11:28.833 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:11:28.834 DEBUG:teuthology.parallel:result is None 2024-01-15T18:11:28.834 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:11:28.835 DEBUG:teuthology.parallel:result is None 2024-01-15T18:11:28.835 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:11:28.835 DEBUG:teuthology.parallel:result is None 2024-01-15T18:11:28.836 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:11:28.919 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:11:28.920 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:11:28.924 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:11:28.925 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:11:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.977+0000 7f829d200640 1 Processor -- start 2024-01-15T18:11:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.977+0000 7f829d200640 1 -- start start 2024-01-15T18:11:28.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f829d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cdef0 0x7f829815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f829d200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82980d02d0 con 0x7f82980cf150 2024-01-15T18:11:28.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f829d200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829815d6e0 con 0x7f829806c540 2024-01-15T18:11:28.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f829d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980cf6c0 con 0x7f82980cdef0 2024-01-15T18:11:28.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8296800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82980cf150 0x7f829815cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56624/0 (socket says 172.21.15.143:56624) 2024-01-15T18:11:28.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8296800640 1 -- 172.21.15.143:0/1485079124 learned_addr learned my addr 172.21.15.143:0/1485079124 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:28.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8295400640 1 --2- 172.21.15.143:0/1485079124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cdef0 0x7f829815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224825768 0 0) 0x7f82980d02d0 con 0x7f82980cf150 2024-01-15T18:11:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8278003530 con 0x7f82980cf150 2024-01-15T18:11:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260511327 0 0) 0x7f829815d6e0 con 0x7f829806c540 2024-01-15T18:11:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82980d02d0 con 0x7f829806c540 2024-01-15T18:11:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8295400640 1 -- 172.21.15.143:0/1485079124 >> v1:172.21.15.204:6789/0 conn(0x7f829806c540 legacy=0x7f82980cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8295400640 1 -- 172.21.15.143:0/1485079124 >> v1:172.21.15.143:6789/0 conn(0x7f82980cf150 legacy=0x7f829815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.981+0000 7f8295400640 1 -- 172.21.15.143:0/1485079124 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980cf9a0 con 0x7f82980cdef0 2024-01-15T18:11:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f8295400640 1 --2- 172.21.15.143:0/1485079124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cdef0 0x7f829815ca90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82800097f0 tx=0x7f828001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ef76b222f9205e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:28.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800ab600 con 0x7f82980cdef0 2024-01-15T18:11:28.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82800ab760 con 0x7f82980cdef0 2024-01-15T18:11:28.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f8294a00640 1 -- 172.21.15.143:0/1485079124 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82800abaf0 con 0x7f82980cdef0 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 -- 172.21.15.143:0/1485079124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cdef0 msgr2=0x7f829815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 --2- 172.21.15.143:0/1485079124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cdef0 0x7f829815ca90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f82800097f0 tx=0x7f828001c400 comp rx=0 tx=0).stop 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 -- 172.21.15.143:0/1485079124 shutdown_connections 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 --2- 172.21.15.143:0/1485079124 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cdef0 0x7f829815ca90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 -- 172.21.15.143:0/1485079124 >> 172.21.15.143:0/1485079124 conn(0x7f82980d0e20 msgr2=0x7f82980d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 -- 172.21.15.143:0/1485079124 shutdown_connections 2024-01-15T18:11:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.985+0000 7f829d200640 1 -- 172.21.15.143:0/1485079124 wait complete. 2024-01-15T18:11:28.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 Processor -- start 2024-01-15T18:11:28.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 -- start start 2024-01-15T18:11:28.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 0x7f829815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82980cdef0 0x7f82980aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cf150 0x7f82980c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980c6340 con 0x7f829806c540 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980c6620 con 0x7f82980cdef0 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980c6900 con 0x7f82980cf150 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 0x7f829815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 0x7f829815d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55216/0 (socket says 172.21.15.143:55216) 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 -- 172.21.15.143:0/3572294296 learned_addr learned my addr 172.21.15.143:0/3572294296 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295400640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82980cdef0 0x7f82980aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8296800640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cf150 0x7f82980c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 -- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cf150 msgr2=0x7f82980c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cf150 0x7f82980c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 -- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82980cdef0 msgr2=0x7f82980aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82980cdef0 0x7f82980aed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:28.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 -- 172.21.15.143:0/3572294296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8280009420 con 0x7f829806c540 2024-01-15T18:11:28.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f8295e00640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 0x7f829815d900 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f828800c490 tx=0x7f828800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:28.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82880052d0 con 0x7f829806c540 2024-01-15T18:11:28.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8288005430 con 0x7f829806c540 2024-01-15T18:11:28.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288012870 con 0x7f829806c540 2024-01-15T18:11:28.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980c6be0 con 0x7f829806c540 2024-01-15T18:11:28.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.989+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82980af2c0 con 0x7f829806c540 2024-01-15T18:11:28.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.993+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82980af4b0 con 0x7f829806c540 2024-01-15T18:11:28.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.993+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f82880129d0 con 0x7f829806c540 2024-01-15T18:11:28.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.993+0000 7f828e000640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82700671f0 0x7f82700696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:28.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.993+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f8288082740 con 0x7f829806c540 2024-01-15T18:11:28.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.993+0000 7f8295400640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82700671f0 0x7f82700696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:28.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.993+0000 7f8295400640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82700671f0 0x7f82700696b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f82800af000 tx=0x7f82800a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:28.997+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82980af4b0 con 0x7f829806c540 2024-01-15T18:11:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:29.161+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f82980af4b0 con 0x7f829806c540 2024-01-15T18:11:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.697+0000 7f828e000640 1 -- 172.21.15.143:0/3572294296 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f82980af4b0 con 0x7f829806c540 2024-01-15T18:11:30.701 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:11:30.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82700671f0 msgr2=0x7f82700696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82700671f0 0x7f82700696b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f82800af000 tx=0x7f82800a7000 comp rx=0 tx=0).stop 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 msgr2=0x7f829815d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 0x7f829815d900 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f828800c490 tx=0x7f828800c950 comp rx=0 tx=0).stop 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 shutdown_connections 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82700671f0 0x7f82700696b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82980cf150 0x7f82980c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82980cdef0 0x7f82980aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 --2- 172.21.15.143:0/3572294296 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f829806c540 0x7f829815d900 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 >> 172.21.15.143:0/3572294296 conn(0x7f82980d0e20 msgr2=0x7f82980b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:30.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 shutdown_connections 2024-01-15T18:11:30.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.701+0000 7f829d200640 1 -- 172.21.15.143:0/3572294296 wait complete. 2024-01-15T18:11:30.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:11:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 Processor -- start 2024-01-15T18:11:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 -- start start 2024-01-15T18:11:30.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980b0310 0x7fc3980cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3980cf740 0x7fc3980cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3980cbff0 con 0x7fc39806bfb0 2024-01-15T18:11:30.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3980d0c50 con 0x7fc3980cf740 2024-01-15T18:11:30.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc3a0000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980cb6c0 con 0x7fc3980b0310 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc39e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3980cf740 0x7fc3980cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc39cc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980b0310 0x7fc3980cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc39d600640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc39806bfb0 0x7fc39806c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:41594/0 (socket says 172.21.15.143:41594) 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc39e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3980cf740 0x7fc3980cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:39436/0 (socket says 172.21.15.143:39436) 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc39d600640 1 -- 172.21.15.143:0/3513652634 learned_addr learned my addr 172.21.15.143:0/3513652634 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.869+0000 7fc39e000640 1 -- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980b0310 msgr2=0x7fc3980cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc39e000640 1 --2- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980b0310 0x7fc3980cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc39e000640 1 -- 172.21.15.143:0/3513652634 >> v1:172.21.15.204:6789/0 conn(0x7fc39806bfb0 legacy=0x7fc39806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc39e000640 1 -- 172.21.15.143:0/3513652634 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3980cb9a0 con 0x7fc3980cf740 2024-01-15T18:11:30.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc39e000640 1 --2- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980cf740 0x7fc3980cfbf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc390009d90 tx=0x7fc39001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3b1b42e80b962a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:30.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc397e00640 1 -- 172.21.15.143:0/3513652634 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900a5950 con 0x7fc3980cf740 2024-01-15T18:11:30.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc397e00640 1 -- 172.21.15.143:0/3513652634 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3900a5ab0 con 0x7fc3980cf740 2024-01-15T18:11:30.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc397e00640 1 -- 172.21.15.143:0/3513652634 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3900a5e40 con 0x7fc3980cf740 2024-01-15T18:11:30.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc3a0000640 1 -- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980cf740 msgr2=0x7fc3980cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.873+0000 7fc3a0000640 1 --2- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980cf740 0x7fc3980cfbf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc390009d90 tx=0x7fc39001c890 comp rx=0 tx=0).stop 2024-01-15T18:11:30.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- 172.21.15.143:0/3513652634 shutdown_connections 2024-01-15T18:11:30.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 --2- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980cf740 0x7fc3980cfbf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 --2- 172.21.15.143:0/3513652634 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980b0310 0x7fc3980cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- 172.21.15.143:0/3513652634 >> 172.21.15.143:0/3513652634 conn(0x7fc3980d0ed0 msgr2=0x7fc3980d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:30.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- 172.21.15.143:0/3513652634 shutdown_connections 2024-01-15T18:11:30.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- 172.21.15.143:0/3513652634 wait complete. 2024-01-15T18:11:30.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 Processor -- start 2024-01-15T18:11:30.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- start start 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 0x7fc398160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980b0310 0x7fc398154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980cf740 0x7fc3980c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc398154de0 con 0x7fc39806bfb0 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39815c330 con 0x7fc3980b0310 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc3a0000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39815c610 con 0x7fc3980cf740 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc39d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 0x7fc398160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc39d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 0x7fc398160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55232/0 (socket says 172.21.15.143:55232) 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc39d600640 1 -- 172.21.15.143:0/2188121905 learned_addr learned my addr 172.21.15.143:0/2188121905 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc39cc00640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980b0310 0x7fc398154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.877+0000 7fc39e000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980cf740 0x7fc3980c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39d600640 1 -- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980cf740 msgr2=0x7fc3980c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39d600640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980cf740 0x7fc3980c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39d600640 1 -- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980b0310 msgr2=0x7fc398154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39d600640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980b0310 0x7fc398154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39d600640 1 -- 172.21.15.143:0/2188121905 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390009a70 con 0x7fc39806bfb0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39e000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980cf740 0x7fc3980c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39d600640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 0x7fc398160090 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c400 tx=0x7fc38800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388005cf0 con 0x7fc39806bfb0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc388005e50 con 0x7fc39806bfb0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc398155030 con 0x7fc39806bfb0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3880072d0 con 0x7fc39806bfb0 2024-01-15T18:11:30.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3981605d0 con 0x7fc39806bfb0 2024-01-15T18:11:30.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc398067500 con 0x7fc39806bfb0 2024-01-15T18:11:30.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fc388007430 con 0x7fc39806bfb0 2024-01-15T18:11:30.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc395600640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3700671f0 0x7fc3700696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fc3880821a0 con 0x7fc39806bfb0 2024-01-15T18:11:30.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.881+0000 7fc39cc00640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3700671f0 0x7fc3700696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.885+0000 7fc39cc00640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3700671f0 0x7fc3700696b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3980d1ac0 tx=0x7fc380009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:30.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:30.885+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc388050ee0 con 0x7fc39806bfb0 2024-01-15T18:11:31.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc3980ce660 con 0x7fc39806bfb0 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc395600640 1 -- 172.21.15.143:0/2188121905 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc388054500 con 0x7fc39806bfb0 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:11:31.095 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:11:31.096 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.097 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.098 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.099 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.100 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:11:31.101 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.102 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2024-01-15T18:11:31.103 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:11:31.104 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:11:31.105 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:11:31.106 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:31.107 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:11:31.108 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:31.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3700671f0 msgr2=0x7fc3700696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3700671f0 0x7fc3700696b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3980d1ac0 tx=0x7fc380009190 comp rx=0 tx=0).stop 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 msgr2=0x7fc398160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 0x7fc398160090 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc38800c400 tx=0x7fc38800c8c0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 shutdown_connections 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc3700671f0 0x7fc3700696b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3980cf740 0x7fc3980c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3980b0310 0x7fc398154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 --2- 172.21.15.143:0/2188121905 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc39806bfb0 0x7fc398160090 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 >> 172.21.15.143:0/2188121905 conn(0x7fc3980d0ed0 msgr2=0x7fc398156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 shutdown_connections 2024-01-15T18:11:31.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.093+0000 7fc3a0000640 1 -- 172.21.15.143:0/2188121905 wait complete. 2024-01-15T18:11:31.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:31.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.249+0000 7fd18e000640 1 Processor -- start 2024-01-15T18:11:31.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.249+0000 7fd18e000640 1 -- start start 2024-01-15T18:11:31.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd18e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd188143780 0x7fd1880cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd18e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880ceb50 con 0x7fd1880722c0 2024-01-15T18:11:31.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd18e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18815c110 con 0x7fd1880ce520 2024-01-15T18:11:31.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd18e000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18815b7e0 con 0x7fd188143780 2024-01-15T18:11:31.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd187e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd1880722c0 0x7fd18815c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56638/0 (socket says 172.21.15.143:56638) 2024-01-15T18:11:31.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd187e00640 1 -- 172.21.15.143:0/4054002046 learned_addr learned my addr 172.21.15.143:0/4054002046 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:31.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd187400640 1 --2- 172.21.15.143:0/4054002046 >> v2:172.21.15.204:3300/0 conn(0x7fd188143780 0x7fd1880cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd186000640 1 -- 172.21.15.143:0/4054002046 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512108679 0 0) 0x7fd1800014a0 con 0x7fd1880722c0 2024-01-15T18:11:31.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd186000640 1 -- 172.21.15.143:0/4054002046 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168005740 con 0x7fd1880722c0 2024-01-15T18:11:31.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd187400640 1 -- 172.21.15.143:0/4054002046 >> v1:172.21.15.204:6789/0 conn(0x7fd1880ce520 legacy=0x7fd188071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd186000640 1 -- 172.21.15.143:0/4054002046 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221742593 0 0) 0x7fd168005740 con 0x7fd1880722c0 2024-01-15T18:11:31.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd187400640 1 -- 172.21.15.143:0/4054002046 >> v1:172.21.15.143:6789/0 conn(0x7fd1880722c0 legacy=0x7fd18815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.253+0000 7fd187400640 1 -- 172.21.15.143:0/4054002046 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18815bac0 con 0x7fd188143780 2024-01-15T18:11:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd187400640 1 --2- 172.21.15.143:0/4054002046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd188143780 0x7fd1880cdfe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd1780097c0 tx=0x7fd17801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef81ba611de34ad9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd186000640 1 -- 172.21.15.143:0/4054002046 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180001b90 con 0x7fd188143780 2024-01-15T18:11:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd186000640 1 -- 172.21.15.143:0/4054002046 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1780a7050 con 0x7fd188143780 2024-01-15T18:11:31.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd186000640 1 -- 172.21.15.143:0/4054002046 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1780ab620 con 0x7fd188143780 2024-01-15T18:11:31.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 -- 172.21.15.143:0/4054002046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd188143780 msgr2=0x7fd1880cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 --2- 172.21.15.143:0/4054002046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd188143780 0x7fd1880cdfe0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd1780097c0 tx=0x7fd17801c3a0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 -- 172.21.15.143:0/4054002046 shutdown_connections 2024-01-15T18:11:31.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 --2- 172.21.15.143:0/4054002046 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd188143780 0x7fd1880cdfe0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 -- 172.21.15.143:0/4054002046 >> 172.21.15.143:0/4054002046 conn(0x7fd18806c3f0 msgr2=0x7fd18806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 -- 172.21.15.143:0/4054002046 shutdown_connections 2024-01-15T18:11:31.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.257+0000 7fd18e000640 1 -- 172.21.15.143:0/4054002046 wait complete. 2024-01-15T18:11:31.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 Processor -- start 2024-01-15T18:11:31.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- start start 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1880722c0 0x7fd1881f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1880ce520 0x7fd188159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 0x7fd188141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881427d0 con 0x7fd188143780 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18813d890 con 0x7fd1880ce520 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18813db70 con 0x7fd1880722c0 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 0x7fd188141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 0x7fd188141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55234/0 (socket says 172.21.15.143:55234) 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 -- 172.21.15.143:0/267861283 learned_addr learned my addr 172.21.15.143:0/267861283 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187400640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1880722c0 0x7fd1881f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd186a00640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1880ce520 0x7fd188159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 -- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1880722c0 msgr2=0x7fd1881f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1880722c0 0x7fd1881f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 -- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1880ce520 msgr2=0x7fd188159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1880ce520 0x7fd188159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 -- 172.21.15.143:0/267861283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd178009420 con 0x7fd188143780 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd187e00640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 0x7fd188141d80 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd180002240 tx=0x7fd18000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000ad50 con 0x7fd188143780 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180012400 con 0x7fd188143780 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18813de50 con 0x7fd188143780 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1800127b0 con 0x7fd188143780 2024-01-15T18:11:31.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1881422c0 con 0x7fd188143780 2024-01-15T18:11:31.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.261+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18813d6f0 con 0x7fd188143780 2024-01-15T18:11:31.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.265+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fd180012910 con 0x7fd188143780 2024-01-15T18:11:31.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.265+0000 7fd17f400640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1600671f0 0x7fd1600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.265+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd1800830d0 con 0x7fd188143780 2024-01-15T18:11:31.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.265+0000 7fd187400640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1600671f0 0x7fd1600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.265+0000 7fd187400640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1600671f0 0x7fd1600696b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd1780ad000 tx=0x7fd178008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.269+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd18813d6f0 con 0x7fd188143780 2024-01-15T18:11:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.421+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd18813e090 con 0x7fd188143780 2024-01-15T18:11:31.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.421+0000 7fd17f400640 1 -- 172.21.15.143:0/267861283 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+12936 (secure 0 0 0) 0x7fd18813e090 con 0x7fd188143780 2024-01-15T18:11:31.425 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:31.425 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":68,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:30.699346+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:31.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1600671f0 msgr2=0x7fd1600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1600671f0 0x7fd1600696b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd1780ad000 tx=0x7fd178008040 comp rx=0 tx=0).stop 2024-01-15T18:11:31.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 msgr2=0x7fd188141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 0x7fd188141d80 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd180002240 tx=0x7fd18000a430 comp rx=0 tx=0).stop 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 shutdown_connections 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1600671f0 0x7fd1600696b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd188143780 0x7fd188141d80 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1880ce520 0x7fd188159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 --2- 172.21.15.143:0/267861283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1880722c0 0x7fd1881f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 >> 172.21.15.143:0/267861283 conn(0x7fd18806c3f0 msgr2=0x7fd1880ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 shutdown_connections 2024-01-15T18:11:31.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.425+0000 7fd18e000640 1 -- 172.21.15.143:0/267861283 wait complete. 2024-01-15T18:11:31.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:11:31.447 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:11:31.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.581+0000 7f785e000640 1 Processor -- start 2024-01-15T18:11:31.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.581+0000 7f785e000640 1 -- start start 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f785e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7858071bb0 0x7f7858072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f785e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f785815c5a0 0x7f785815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f785e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78580ceb50 con 0x7f7858143780 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f785e000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7858072570 con 0x7f785815c5a0 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f785e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785815c110 con 0x7f7858071bb0 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7858143780 0x7f78580cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56652/0 (socket says 172.21.15.143:56652) 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857400640 1 -- 172.21.15.143:0/654498893 learned_addr learned my addr 172.21.15.143:0/654498893 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:31.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857e00640 1 --2- 172.21.15.143:0/654498893 >> v2:172.21.15.204:3300/0 conn(0x7f785815c5a0 0x7f785815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7856a00640 1 --2- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7858071bb0 0x7f7858072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7856000640 1 -- 172.21.15.143:0/654498893 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316699069 0 0) 0x7f7848001120 con 0x7f7858143780 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7856000640 1 -- 172.21.15.143:0/654498893 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7838003530 con 0x7f7858143780 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857e00640 1 -- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7858071bb0 msgr2=0x7f7858072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857e00640 1 --2- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7858071bb0 0x7f7858072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857e00640 1 -- 172.21.15.143:0/654498893 >> v1:172.21.15.143:6789/0 conn(0x7f7858143780 legacy=0x7f78580cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.585+0000 7f7857e00640 1 -- 172.21.15.143:0/654498893 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785815b7e0 con 0x7f785815c5a0 2024-01-15T18:11:31.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f7857e00640 1 --2- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f785815c5a0 0x7f785815ca30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7850009d90 tx=0x7f785001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e6535a733ee0e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f7856000640 1 -- 172.21.15.143:0/654498893 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500a5a10 con 0x7f785815c5a0 2024-01-15T18:11:31.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f7856000640 1 -- 172.21.15.143:0/654498893 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78500a5b70 con 0x7f785815c5a0 2024-01-15T18:11:31.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f7856000640 1 -- 172.21.15.143:0/654498893 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78500ad440 con 0x7f785815c5a0 2024-01-15T18:11:31.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 -- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f785815c5a0 msgr2=0x7f785815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 --2- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f785815c5a0 0x7f785815ca30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7850009d90 tx=0x7f785001c950 comp rx=0 tx=0).stop 2024-01-15T18:11:31.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 -- 172.21.15.143:0/654498893 shutdown_connections 2024-01-15T18:11:31.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 --2- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f785815c5a0 0x7f785815ca30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 --2- 172.21.15.143:0/654498893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7858071bb0 0x7f7858072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 -- 172.21.15.143:0/654498893 >> 172.21.15.143:0/654498893 conn(0x7f785806c3f0 msgr2=0x7f785806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 -- 172.21.15.143:0/654498893 shutdown_connections 2024-01-15T18:11:31.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.589+0000 7f785e000640 1 -- 172.21.15.143:0/654498893 wait complete. 2024-01-15T18:11:31.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 Processor -- start 2024-01-15T18:11:31.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- start start 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858071bb0 0x7f7858157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 0x7f7858137ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f785815c5a0 0x7f78581425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78581583d0 con 0x7f7858143780 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7858138db0 con 0x7f7858071bb0 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7858139090 con 0x7f785815c5a0 2024-01-15T18:11:31.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 0x7f7858137ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 0x7f7858137ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55238/0 (socket says 172.21.15.143:55238) 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 -- 172.21.15.143:0/2884248934 learned_addr learned my addr 172.21.15.143:0/2884248934 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7857400640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858071bb0 0x7f7858157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7857e00640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f785815c5a0 0x7f78581425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 -- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f785815c5a0 msgr2=0x7f78581425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f785815c5a0 0x7f78581425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 -- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858071bb0 msgr2=0x7f7858157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858071bb0 0x7f7858157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 -- 172.21.15.143:0/2884248934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7850009a70 con 0x7f7858143780 2024-01-15T18:11:31.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f7856a00640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 0x7f7858137ef0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f784000e930 tx=0x7f784000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840009690 con 0x7f7858143780 2024-01-15T18:11:31.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784000ca00 con 0x7f7858143780 2024-01-15T18:11:31.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784000cd70 con 0x7f7858143780 2024-01-15T18:11:31.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7858139370 con 0x7f7858143780 2024-01-15T18:11:31.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7858158120 con 0x7f7858143780 2024-01-15T18:11:31.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.593+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7858067500 con 0x7f7858143780 2024-01-15T18:11:31.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.597+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f7840013470 con 0x7f7858143780 2024-01-15T18:11:31.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.597+0000 7f784f400640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7830067220 0x7f78300696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.597+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f78400830c0 con 0x7f7858143780 2024-01-15T18:11:31.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.597+0000 7f7857400640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7830067220 0x7f78300696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.597+0000 7f7857400640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7830067220 0x7f78300696e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7848001280 tx=0x7f7848007bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.601+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7840051cd0 con 0x7f7858143780 2024-01-15T18:11:31.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.753+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f785813de40 con 0x7f7858143780 2024-01-15T18:11:31.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.753+0000 7f784f400640 1 -- 172.21.15.143:0/2884248934 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78400552f0 con 0x7f7858143780 2024-01-15T18:11:31.757 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7830067220 msgr2=0x7f78300696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7830067220 0x7f78300696e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7848001280 tx=0x7f7848007bc0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 msgr2=0x7f7858137ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 0x7f7858137ef0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f784000e930 tx=0x7f784000edf0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 shutdown_connections 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7830067220 0x7f78300696e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f785815c5a0 0x7f78581425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7858143780 0x7f7858137ef0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 --2- 172.21.15.143:0/2884248934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7858071bb0 0x7f7858157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 >> 172.21.15.143:0/2884248934 conn(0x7f785806c3f0 msgr2=0x7f78580adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 shutdown_connections 2024-01-15T18:11:31.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.757+0000 7f785e000640 1 -- 172.21.15.143:0/2884248934 wait complete. 2024-01-15T18:11:31.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:11:31.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 Processor -- start 2024-01-15T18:11:31.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 -- start start 2024-01-15T18:11:31.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f9c159b70 0x7f1f9c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c0b48f0 0x7f1f9c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0b2a20 con 0x7f1f9c157300 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c0b35a0 con 0x7f1f9c159b70 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.913+0000 7f1fa3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c159fd0 con 0x7f1f9c0b48f0 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1f9c159b70 0x7f1f9c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1fa0e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1f9c157300 0x7f1f9c0b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56668/0 (socket says 172.21.15.143:56668) 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1fa0e00640 1 -- 172.21.15.143:0/3044783372 learned_addr learned my addr 172.21.15.143:0/3044783372 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1fa1800640 1 --2- 172.21.15.143:0/3044783372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c0b48f0 0x7f1f9c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9be00640 1 -- 172.21.15.143:0/3044783372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c0b48f0 msgr2=0x7f1f9c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9be00640 1 --2- 172.21.15.143:0/3044783372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c0b48f0 0x7f1f9c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9be00640 1 -- 172.21.15.143:0/3044783372 >> v1:172.21.15.143:6789/0 conn(0x7f1f9c157300 legacy=0x7f1f9c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9be00640 1 -- 172.21.15.143:0/3044783372 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0b2d00 con 0x7f1f9c159b70 2024-01-15T18:11:31.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9be00640 1 --2- 172.21.15.143:0/3044783372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c159b70 0x7f1f9c0b43b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400a960 tx=0x7f1f8401d850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d4b12a10266c75 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9b400640 1 -- 172.21.15.143:0/3044783372 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840a67f0 con 0x7f1f9c159b70 2024-01-15T18:11:31.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9b400640 1 -- 172.21.15.143:0/3044783372 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f840a6950 con 0x7f1f9c159b70 2024-01-15T18:11:31.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.917+0000 7f1f9b400640 1 -- 172.21.15.143:0/3044783372 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840a6ce0 con 0x7f1f9c159b70 2024-01-15T18:11:31.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 -- 172.21.15.143:0/3044783372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c159b70 msgr2=0x7f1f9c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3044783372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c159b70 0x7f1f9c0b43b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400a960 tx=0x7f1f8401d850 comp rx=0 tx=0).stop 2024-01-15T18:11:31.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 -- 172.21.15.143:0/3044783372 shutdown_connections 2024-01-15T18:11:31.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3044783372 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c0b48f0 0x7f1f9c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3044783372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c159b70 0x7f1f9c0b43b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 -- 172.21.15.143:0/3044783372 >> 172.21.15.143:0/3044783372 conn(0x7f1f9c06c3f0 msgr2=0x7f1f9c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 -- 172.21.15.143:0/3044783372 shutdown_connections 2024-01-15T18:11:31.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 -- 172.21.15.143:0/3044783372 wait complete. 2024-01-15T18:11:31.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 Processor -- start 2024-01-15T18:11:31.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 -- start start 2024-01-15T18:11:31.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9c0b48f0 0x7f1f9c15f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.921+0000 7f1fa3800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 0x7f1f9c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c159b70 0x7f1f9c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c15e830 con 0x7f1f9c157300 2024-01-15T18:11:31.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c15eb10 con 0x7f1f9c0b48f0 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c15edf0 con 0x7f1f9c159b70 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 0x7f1f9c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 0x7f1f9c15f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55258/0 (socket says 172.21.15.143:55258) 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 -- 172.21.15.143:0/3178404392 learned_addr learned my addr 172.21.15.143:0/3178404392 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa1800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c159b70 0x7f1f9c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa0e00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9c0b48f0 0x7f1f9c15f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 -- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c159b70 msgr2=0x7f1f9c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c159b70 0x7f1f9c15da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 -- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9c0b48f0 msgr2=0x7f1f9c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9c0b48f0 0x7f1f9c15f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 -- 172.21.15.143:0/3178404392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8400a5c0 con 0x7f1f9c157300 2024-01-15T18:11:31.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa0e00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9c0b48f0 0x7f1f9c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f9be00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 0x7f1f9c15f750 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1f840b1040 tx=0x7f1f8401a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840ae910 con 0x7f1f9c157300 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f840aea90 con 0x7f1f9c157300 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840aee40 con 0x7f1f9c157300 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c151e00 con 0x7f1f9c157300 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c1f5c50 con 0x7f1f9c157300 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c067500 con 0x7f1f9c157300 2024-01-15T18:11:31.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.925+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f1f840b5040 con 0x7f1f9c157300 2024-01-15T18:11:31.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.929+0000 7f1f98c00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f740671f0 0x7f1f740696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.929+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f1f8411d470 con 0x7f1f9c157300 2024-01-15T18:11:31.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.929+0000 7f1fa0e00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f740671f0 0x7f1f740696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.929+0000 7f1fa0e00640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f740671f0 0x7f1f740696b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c005890 tx=0x7f1f8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:31.933+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1f840ebf10 con 0x7f1f9c157300 2024-01-15T18:11:32.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.081+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f9c164320 con 0x7f1f9c157300 2024-01-15T18:11:32.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.081+0000 7f1f98c00640 1 -- 172.21.15.143:0/3178404392 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1f840b3060 con 0x7f1f9c157300 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:32.086 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f740671f0 msgr2=0x7f1f740696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f740671f0 0x7f1f740696b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c005890 tx=0x7f1f8c009040 comp rx=0 tx=0).stop 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 msgr2=0x7f1f9c15f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 0x7f1f9c15f750 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1f840b1040 tx=0x7f1f8401a5f0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 shutdown_connections 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1f740671f0 0x7f1f740696b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f9c159b70 0x7f1f9c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f9c157300 0x7f1f9c15f750 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 --2- 172.21.15.143:0/3178404392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f9c0b48f0 0x7f1f9c15f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 >> 172.21.15.143:0/3178404392 conn(0x7f1f9c06c3f0 msgr2=0x7f1f9c1553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:32.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 shutdown_connections 2024-01-15T18:11:32.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.085+0000 7f1fa3800640 1 -- 172.21.15.143:0/3178404392 wait complete. 2024-01-15T18:11:32.107 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:11:32.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:11:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 Processor -- start 2024-01-15T18:11:32.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 -- start start 2024-01-15T18:11:32.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6d20143780 0x7f6d200cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d200ce520 0x7f6d20071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2015a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d20072580 con 0x7f6d200ce520 2024-01-15T18:11:32.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d200ceb50 con 0x7f6d20143780 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d25000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2015c110 con 0x7f6d200720f0 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d200ce520 0x7f6d20071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6d200ce520 0x7f6d20071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55270/0 (socket says 172.21.15.143:55270) 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 -- 172.21.15.143:0/4083781131 learned_addr learned my addr 172.21.15.143:0/4083781131 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1de00640 1 --2- 172.21.15.143:0/4083781131 >> v2:172.21.15.204:3300/0 conn(0x7f6d20143780 0x7f6d200cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1e800640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 -- 172.21.15.143:0/4083781131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 msgr2=0x7f6d2015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 -- 172.21.15.143:0/4083781131 >> v2:172.21.15.204:3300/0 conn(0x7f6d20143780 msgr2=0x7f6d200cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 --2- 172.21.15.143:0/4083781131 >> v2:172.21.15.204:3300/0 conn(0x7f6d20143780 0x7f6d200cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1d400640 1 -- 172.21.15.143:0/4083781131 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2015b7e0 con 0x7f6d200ce520 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1e800640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:32.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.245+0000 7f6d1de00640 1 --2- 172.21.15.143:0/4083781131 >> v2:172.21.15.204:3300/0 conn(0x7f6d20143780 0x7f6d200cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:32.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d1d400640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d20071bb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6d2006be30 tx=0x7f6d0801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6776f27cc6797fdc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:32.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d1ca00640 1 -- 172.21.15.143:0/4083781131 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080a7b70 con 0x7f6d200ce520 2024-01-15T18:11:32.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d1ca00640 1 -- 172.21.15.143:0/4083781131 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080a7cd0 con 0x7f6d200ce520 2024-01-15T18:11:32.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d1ca00640 1 -- 172.21.15.143:0/4083781131 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080af660 con 0x7f6d200ce520 2024-01-15T18:11:32.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 -- 172.21.15.143:0/4083781131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 msgr2=0x7f6d20071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d20071bb0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6d2006be30 tx=0x7f6d0801ebd0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 -- 172.21.15.143:0/4083781131 shutdown_connections 2024-01-15T18:11:32.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 --2- 172.21.15.143:0/4083781131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d20071bb0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 --2- 172.21.15.143:0/4083781131 >> v2:172.21.15.204:3300/0 conn(0x7f6d20143780 0x7f6d200cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 -- 172.21.15.143:0/4083781131 >> 172.21.15.143:0/4083781131 conn(0x7f6d2006c3f0 msgr2=0x7f6d2006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:32.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 -- 172.21.15.143:0/4083781131 shutdown_connections 2024-01-15T18:11:32.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.249+0000 7f6d25000640 1 -- 172.21.15.143:0/4083781131 wait complete. 2024-01-15T18:11:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 Processor -- start 2024-01-15T18:11:32.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 -- start start 2024-01-15T18:11:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d2013c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20143780 0x7f6d200c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2013f860 con 0x7f6d200ce520 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200c42a0 con 0x7f6d20143780 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2013f2d0 con 0x7f6d200720f0 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d2013c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d2013c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55278/0 (socket says 172.21.15.143:55278) 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 -- 172.21.15.143:0/1995382460 learned_addr learned my addr 172.21.15.143:0/1995382460 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1de00640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1e800640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20143780 0x7f6d200c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 -- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 msgr2=0x7f6d2013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2013ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 -- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20143780 msgr2=0x7f6d200c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20143780 0x7f6d200c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 -- 172.21.15.143:0/1995382460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0800b770 con 0x7f6d200ce520 2024-01-15T18:11:32.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d1d400640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d2013c6a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b2c0 tx=0x7f6d0801b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080a7b70 con 0x7f6d200ce520 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080ba440 con 0x7f6d200ce520 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080ba7f0 con 0x7f6d200ce520 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2013f580 con 0x7f6d200ce520 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.253+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d200c4820 con 0x7f6d200ce520 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.257+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d200c4a10 con 0x7f6d200ce520 2024-01-15T18:11:32.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.257+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f6d080b93e0 con 0x7f6d200ce520 2024-01-15T18:11:32.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.257+0000 7f6d16000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf80671f0 0x7f6cf80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.257+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f6d080a7e40 con 0x7f6d200ce520 2024-01-15T18:11:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.257+0000 7f6d1de00640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf80671f0 0x7f6cf80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:32.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.257+0000 7f6d1de00640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf80671f0 0x7f6cf80696b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000ba10 tx=0x7f6d10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:32.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.261+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6d080ec210 con 0x7f6d200ce520 2024-01-15T18:11:32.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.413+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6d20067500 con 0x7f6d200ce520 2024-01-15T18:11:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.825+0000 7f6d16000640 1 -- 172.21.15.143:0/1995382460 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6d080bf0b0 con 0x7f6d200ce520 2024-01-15T18:11:32.827 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:11:32.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.825+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf80671f0 msgr2=0x7f6cf80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.825+0000 7f6d25000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf80671f0 0x7f6cf80696b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000ba10 tx=0x7f6d10008040 comp rx=0 tx=0).stop 2024-01-15T18:11:32.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.825+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 msgr2=0x7f6d2013c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:32.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.825+0000 7f6d25000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d2013c6a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800b2c0 tx=0x7f6d0801b970 comp rx=0 tx=0).stop 2024-01-15T18:11:32.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 shutdown_connections 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cf80671f0 0x7f6cf80696b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d20143780 0x7f6d200c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6d200ce520 0x7f6d2013c6a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 --2- 172.21.15.143:0/1995382460 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6d200720f0 0x7f6d2013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 >> 172.21.15.143:0/1995382460 conn(0x7f6d2006c3f0 msgr2=0x7f6d200cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 shutdown_connections 2024-01-15T18:11:32.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:32.829+0000 7f6d25000640 1 -- 172.21.15.143:0/1995382460 wait complete. 2024-01-15T18:11:32.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:11:33.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 Processor -- start 2024-01-15T18:11:33.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 -- start start 2024-01-15T18:11:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8158a00 0x7f6ef80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ef815bf20 0x7f6ef80cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef815c3b0 con 0x7f6ef8070b50 2024-01-15T18:11:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef815f980 con 0x7f6ef815bf20 2024-01-15T18:11:33.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6efd000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef815ebc0 con 0x7f6ef8158a00 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ef815bf20 0x7f6ef80cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ef815bf20 0x7f6ef80cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55286/0 (socket says 172.21.15.143:55286) 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 -- 172.21.15.143:0/1488206355 learned_addr learned my addr 172.21.15.143:0/1488206355 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5e00640 1 --2- 172.21.15.143:0/1488206355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8158a00 0x7f6ef80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 -- 172.21.15.143:0/1488206355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8158a00 msgr2=0x7f6ef80ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 --2- 172.21.15.143:0/1488206355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8158a00 0x7f6ef80ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 -- 172.21.15.143:0/1488206355 >> v1:172.21.15.204:6789/0 conn(0x7f6ef8070b50 legacy=0x7f6ef8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:33.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.001+0000 7f6ef5400640 1 -- 172.21.15.143:0/1488206355 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef815eea0 con 0x7f6ef815bf20 2024-01-15T18:11:33.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6ef5400640 1 --2- 172.21.15.143:0/1488206355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef815bf20 0x7f6ef80cc3d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009870 tx=0x7f6ee001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfcb664fc64d5cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:33.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6ef4a00640 1 -- 172.21.15.143:0/1488206355 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00ad640 con 0x7f6ef815bf20 2024-01-15T18:11:33.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6ef4a00640 1 -- 172.21.15.143:0/1488206355 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee00ad7a0 con 0x7f6ef815bf20 2024-01-15T18:11:33.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6ef4a00640 1 -- 172.21.15.143:0/1488206355 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00adb30 con 0x7f6ef815bf20 2024-01-15T18:11:33.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 -- 172.21.15.143:0/1488206355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef815bf20 msgr2=0x7f6ef80cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:33.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 --2- 172.21.15.143:0/1488206355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef815bf20 0x7f6ef80cc3d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009870 tx=0x7f6ee001c480 comp rx=0 tx=0).stop 2024-01-15T18:11:33.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 -- 172.21.15.143:0/1488206355 shutdown_connections 2024-01-15T18:11:33.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 --2- 172.21.15.143:0/1488206355 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef815bf20 0x7f6ef80cc3d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 --2- 172.21.15.143:0/1488206355 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8158a00 0x7f6ef80ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 -- 172.21.15.143:0/1488206355 >> 172.21.15.143:0/1488206355 conn(0x7f6ef806a890 msgr2=0x7f6ef806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:33.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 -- 172.21.15.143:0/1488206355 shutdown_connections 2024-01-15T18:11:33.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.005+0000 7f6efd000640 1 -- 172.21.15.143:0/1488206355 wait complete. 2024-01-15T18:11:33.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 Processor -- start 2024-01-15T18:11:33.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- start start 2024-01-15T18:11:33.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8070b50 0x7f6ef815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8158a00 0x7f6ef80b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 0x7f6ef81f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef8154e60 con 0x7f6ef8152a60 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80b0660 con 0x7f6ef8158a00 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef815d650 con 0x7f6ef8070b50 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 0x7f6ef81f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 0x7f6ef81f9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55294/0 (socket says 172.21.15.143:55294) 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 -- 172.21.15.143:0/3402568417 learned_addr learned my addr 172.21.15.143:0/3402568417 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef5e00640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8070b50 0x7f6ef815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:33.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef5400640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8158a00 0x7f6ef80b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:33.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 -- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8070b50 msgr2=0x7f6ef815cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:33.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8070b50 0x7f6ef815cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 -- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8158a00 msgr2=0x7f6ef80b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8158a00 0x7f6ef80b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 -- 172.21.15.143:0/3402568417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee00094d0 con 0x7f6ef8152a60 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6ef6800640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 0x7f6ef81f9950 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c730 tx=0x7f6ef000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0003b80 con 0x7f6ef8152a60 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef815d930 con 0x7f6ef8152a60 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef0003ce0 con 0x7f6ef8152a60 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef000aa30 con 0x7f6ef8152a60 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef815def0 con 0x7f6ef8152a60 2024-01-15T18:11:33.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.009+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef80cb5d0 con 0x7f6ef8152a60 2024-01-15T18:11:33.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.013+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f6ef000ab90 con 0x7f6ef8152a60 2024-01-15T18:11:33.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.013+0000 7f6eee000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ed00671f0 0x7f6ed00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:33.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.013+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f6ef00829c0 con 0x7f6ef8152a60 2024-01-15T18:11:33.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.013+0000 7f6ef5e00640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ed00671f0 0x7f6ed00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:33.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.013+0000 7f6ef5e00640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ed00671f0 0x7f6ed00696b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ef815e390 tx=0x7f6ee8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:33.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.017+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ef0051680 con 0x7f6ef8152a60 2024-01-15T18:11:33.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.181+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6ef8154b40 con 0x7f6ef8152a60 2024-01-15T18:11:33.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6eee000640 1 -- 172.21.15.143:0/3402568417 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ef0054ca0 con 0x7f6ef8152a60 2024-01-15T18:11:33.835 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ed00671f0 msgr2=0x7f6ed00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ed00671f0 0x7f6ed00696b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6ef815e390 tx=0x7f6ee8008040 comp rx=0 tx=0).stop 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 msgr2=0x7f6ef81f9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 0x7f6ef81f9950 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000c730 tx=0x7f6ef000cbf0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 shutdown_connections 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ed00671f0 0x7f6ed00696b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ef8152a60 0x7f6ef81f9950 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8158a00 0x7f6ef80b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 --2- 172.21.15.143:0/3402568417 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ef8070b50 0x7f6ef815cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:33.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 >> 172.21.15.143:0/3402568417 conn(0x7f6ef806a890 msgr2=0x7f6ef806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:33.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 shutdown_connections 2024-01-15T18:11:33.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:33.833+0000 7f6efd000640 1 -- 172.21.15.143:0/3402568417 wait complete. 2024-01-15T18:11:33.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:11:34.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.017+0000 7f21bb600640 1 Processor -- start 2024-01-15T18:11:34.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21bb600640 1 -- start start 2024-01-15T18:11:34.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21bb600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21b40cfb30 0x7f21b406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:34.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21bb600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b40cffd0 con 0x7f21b41607c0 2024-01-15T18:11:34.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21bb600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b40d02b0 con 0x7f21b40cbf50 2024-01-15T18:11:34.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21bb600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b40d1ce0 con 0x7f21b40cfb30 2024-01-15T18:11:34.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b8c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f21b41607c0 0x7f21b406b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56670/0 (socket says 172.21.15.143:56670) 2024-01-15T18:11:34.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b8c00640 1 -- 172.21.15.143:0/1052588161 learned_addr learned my addr 172.21.15.143:0/1052588161 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:34.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3e00640 1 --2- 172.21.15.143:0/1052588161 >> v2:172.21.15.204:3300/0 conn(0x7f21b40cfb30 0x7f21b406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:34.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3400640 1 -- 172.21.15.143:0/1052588161 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687828007 0 0) 0x7f21b40cffd0 con 0x7f21b41607c0 2024-01-15T18:11:34.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3400640 1 -- 172.21.15.143:0/1052588161 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2194003530 con 0x7f21b41607c0 2024-01-15T18:11:34.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3e00640 1 -- 172.21.15.143:0/1052588161 >> v1:172.21.15.204:6789/0 conn(0x7f21b40cbf50 legacy=0x7f21b40cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3400640 1 -- 172.21.15.143:0/1052588161 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037159072 0 0) 0x7f21b40d02b0 con 0x7f21b40cbf50 2024-01-15T18:11:34.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3e00640 1 -- 172.21.15.143:0/1052588161 >> v1:172.21.15.143:6789/0 conn(0x7f21b41607c0 legacy=0x7f21b406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.021+0000 7f21b3e00640 1 -- 172.21.15.143:0/1052588161 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b40cb470 con 0x7f21b40cfb30 2024-01-15T18:11:34.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21b3e00640 1 --2- 172.21.15.143:0/1052588161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cfb30 0x7f21b406bec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f219c0097c0 tx=0x7f219c01eab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cff3c77a5b8ebeb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:34.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21b3400640 1 -- 172.21.15.143:0/1052588161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4001680 con 0x7f21b40cfb30 2024-01-15T18:11:34.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21b3400640 1 -- 172.21.15.143:0/1052588161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c0a8570 con 0x7f21b40cfb30 2024-01-15T18:11:34.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21b3400640 1 -- 172.21.15.143:0/1052588161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0a8900 con 0x7f21b40cfb30 2024-01-15T18:11:34.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 -- 172.21.15.143:0/1052588161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cfb30 msgr2=0x7f21b406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 --2- 172.21.15.143:0/1052588161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cfb30 0x7f21b406bec0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f219c0097c0 tx=0x7f219c01eab0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 -- 172.21.15.143:0/1052588161 shutdown_connections 2024-01-15T18:11:34.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 --2- 172.21.15.143:0/1052588161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cfb30 0x7f21b406bec0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 -- 172.21.15.143:0/1052588161 >> 172.21.15.143:0/1052588161 conn(0x7f21b40d0e20 msgr2=0x7f21b40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:34.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 -- 172.21.15.143:0/1052588161 shutdown_connections 2024-01-15T18:11:34.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.025+0000 7f21bb600640 1 -- 172.21.15.143:0/1052588161 wait complete. 2024-01-15T18:11:34.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 Processor -- start 2024-01-15T18:11:34.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- start start 2024-01-15T18:11:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cbf50 0x7f21b4152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 0x7f21b406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21b41607c0 0x7f21b406f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40b60d0 con 0x7f21b40cfb30 2024-01-15T18:11:34.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41f6170 con 0x7f21b40cbf50 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41f6cf0 con 0x7f21b41607c0 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 0x7f21b406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 0x7f21b406ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55306/0 (socket says 172.21.15.143:55306) 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 -- 172.21.15.143:0/1617175230 learned_addr learned my addr 172.21.15.143:0/1617175230 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b8c00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cbf50 0x7f21b4152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 -- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21b41607c0 msgr2=0x7f21b406f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21b41607c0 0x7f21b406f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 -- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cbf50 msgr2=0x7f21b4152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cbf50 0x7f21b4152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 -- 172.21.15.143:0/1617175230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c009420 con 0x7f21b40cfb30 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b8c00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cbf50 0x7f21b4152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b3e00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 0x7f21b406ed50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f219c0b3040 tx=0x7f219c01b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0c0070 con 0x7f21b40cfb30 2024-01-15T18:11:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c006e20 con 0x7f21b40cfb30 2024-01-15T18:11:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c0b0890 con 0x7f21b40cfb30 2024-01-15T18:11:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b40b26d0 con 0x7f21b40cfb30 2024-01-15T18:11:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b41f66a0 con 0x7f21b40cfb30 2024-01-15T18:11:34.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.029+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b406d4b0 con 0x7f21b40cfb30 2024-01-15T18:11:34.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.033+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f219c0b7040 con 0x7f21b40cfb30 2024-01-15T18:11:34.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.033+0000 7f21b0c00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f218c067220 0x7f218c0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:34.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.033+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f219c1204b0 con 0x7f21b40cfb30 2024-01-15T18:11:34.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.033+0000 7f21b8c00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f218c067220 0x7f218c0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:34.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.033+0000 7f21b8c00640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f218c067220 0x7f218c0696e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f21a4000f70 tx=0x7f21a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:34.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.037+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f219c0eef50 con 0x7f21b40cfb30 2024-01-15T18:11:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.221+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f21b406d4b0 con 0x7f21b40cfb30 2024-01-15T18:11:34.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.845+0000 7f21b0c00640 1 -- 172.21.15.143:0/1617175230 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f219c0b5020 con 0x7f21b40cfb30 2024-01-15T18:11:34.851 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 10 and data pool 11 2024-01-15T18:11:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f218c067220 msgr2=0x7f218c0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f218c067220 0x7f218c0696e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f21a4000f70 tx=0x7f21a4009040 comp rx=0 tx=0).stop 2024-01-15T18:11:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 msgr2=0x7f21b406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 0x7f21b406ed50 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f219c0b3040 tx=0x7f219c01b850 comp rx=0 tx=0).stop 2024-01-15T18:11:34.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 shutdown_connections 2024-01-15T18:11:34.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f218c067220 0x7f218c0696e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21b41607c0 0x7f21b406f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21b40cfb30 0x7f21b406ed50 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 --2- 172.21.15.143:0/1617175230 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21b40cbf50 0x7f21b4152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:34.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.853+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 >> 172.21.15.143:0/1617175230 conn(0x7f21b40d0e20 msgr2=0x7f21b406fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:34.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.857+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 shutdown_connections 2024-01-15T18:11:34.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:34.857+0000 7f21bb600640 1 -- 172.21.15.143:0/1617175230 wait complete. 2024-01-15T18:11:34.895 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:35.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4d6800640 1 Processor -- start 2024-01-15T18:11:35.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4d6800640 1 -- start start 2024-01-15T18:11:35.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4d6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d00ce560 0x7fb4d015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4d6800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d015f310 con 0x7fb4d0071d00 2024-01-15T18:11:35.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4d6800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d015f5f0 con 0x7fb4d0141660 2024-01-15T18:11:35.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4d6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d015fbb0 con 0x7fb4d00ce560 2024-01-15T18:11:35.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4cfe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb4d0071d00 0x7fb4d00b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56680/0 (socket says 172.21.15.143:56680) 2024-01-15T18:11:35.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4cfe00640 1 -- 172.21.15.143:0/1018549924 learned_addr learned my addr 172.21.15.143:0/1018549924 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:35.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4cea00640 1 --2- 172.21.15.143:0/1018549924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d00ce560 0x7fb4d015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313338907 0 0) 0x7fb4d015f310 con 0x7fb4d0071d00 2024-01-15T18:11:35.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b0003530 con 0x7fb4d0071d00 2024-01-15T18:11:35.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787269357 0 0) 0x7fb4d015f5f0 con 0x7fb4d0141660 2024-01-15T18:11:35.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.029+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d015f310 con 0x7fb4d0141660 2024-01-15T18:11:35.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4cea00640 1 -- 172.21.15.143:0/1018549924 >> v1:172.21.15.204:6789/0 conn(0x7fb4d0141660 legacy=0x7fb4d0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4cea00640 1 -- 172.21.15.143:0/1018549924 >> v1:172.21.15.143:6789/0 conn(0x7fb4d0071d00 legacy=0x7fb4d00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4cea00640 1 -- 172.21.15.143:0/1018549924 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d015f8d0 con 0x7fb4d00ce560 2024-01-15T18:11:35.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4cea00640 1 --2- 172.21.15.143:0/1018549924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d00ce560 0x7fb4d015ec80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80097f0 tx=0x7fb4b801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d784a1972caef8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80aa610 con 0x7fb4d00ce560 2024-01-15T18:11:35.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b80a6040 con 0x7fb4d00ce560 2024-01-15T18:11:35.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.033+0000 7fb4ce000640 1 -- 172.21.15.143:0/1018549924 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80aa970 con 0x7fb4d00ce560 2024-01-15T18:11:35.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- 172.21.15.143:0/1018549924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d00ce560 msgr2=0x7fb4d015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 --2- 172.21.15.143:0/1018549924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d00ce560 0x7fb4d015ec80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80097f0 tx=0x7fb4b801c490 comp rx=0 tx=0).stop 2024-01-15T18:11:35.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- 172.21.15.143:0/1018549924 shutdown_connections 2024-01-15T18:11:35.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 --2- 172.21.15.143:0/1018549924 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d00ce560 0x7fb4d015ec80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- 172.21.15.143:0/1018549924 >> 172.21.15.143:0/1018549924 conn(0x7fb4d006c3f0 msgr2=0x7fb4d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:35.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- 172.21.15.143:0/1018549924 shutdown_connections 2024-01-15T18:11:35.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- 172.21.15.143:0/1018549924 wait complete. 2024-01-15T18:11:35.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 Processor -- start 2024-01-15T18:11:35.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- start start 2024-01-15T18:11:35.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d0071d00 0x7fb4d00cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 0x7fb4d0159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d0141660 0x7fb4d015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00ce9e0 con 0x7fb4d00ce560 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d013bde0 con 0x7fb4d0141660 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4d6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00b2db0 con 0x7fb4d0071d00 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 0x7fb4d0159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 0x7fb4d0159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55314/0 (socket says 172.21.15.143:55314) 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 -- 172.21.15.143:0/2017976188 learned_addr learned my addr 172.21.15.143:0/2017976188 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cfe00640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d0141660 0x7fb4d015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cf400640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d0071d00 0x7fb4d00cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 -- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d0071d00 msgr2=0x7fb4d00cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d0071d00 0x7fb4d00cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 -- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d0141660 msgr2=0x7fb4d015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d0141660 0x7fb4d015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.037+0000 7fb4cea00640 1 -- 172.21.15.143:0/2017976188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8009420 con 0x7fb4d00ce560 2024-01-15T18:11:35.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4cea00640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 0x7fb4d0159bc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80ac040 tx=0x7fb4b801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8006e20 con 0x7fb4d00ce560 2024-01-15T18:11:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b80be410 con 0x7fb4d00ce560 2024-01-15T18:11:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80be780 con 0x7fb4d00ce560 2024-01-15T18:11:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d00cd150 con 0x7fb4d00ce560 2024-01-15T18:11:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d015ba80 con 0x7fb4d00ce560 2024-01-15T18:11:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d0067500 con 0x7fb4d00ce560 2024-01-15T18:11:35.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fb4b80be8e0 con 0x7fb4d00ce560 2024-01-15T18:11:35.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4c7400640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4a80671f0 0x7fb4a80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb4b811d820 con 0x7fb4d00ce560 2024-01-15T18:11:35.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.041+0000 7fb4cf400640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4a80671f0 0x7fb4a80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.045+0000 7fb4cf400640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4a80671f0 0x7fb4a80696b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0001720 tx=0x7fb4c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.045+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4b80ec680 con 0x7fb4d00ce560 2024-01-15T18:11:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.197+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4d006e2a0 con 0x7fb4d00ce560 2024-01-15T18:11:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.197+0000 7fb4c7400640 1 -- 172.21.15.143:0/2017976188 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16660 (secure 0 0 0) 0x7fb4b80c3020 con 0x7fb4d00ce560 2024-01-15T18:11:35.201 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:35.201 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:34.225038+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:11:32.460276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-15T18:11:33.227442+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4a80671f0 msgr2=0x7fb4a80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4a80671f0 0x7fb4a80696b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0001720 tx=0x7fb4c0009040 comp rx=0 tx=0).stop 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 msgr2=0x7fb4d0159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 0x7fb4d0159bc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80ac040 tx=0x7fb4b801acd0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 shutdown_connections 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4a80671f0 0x7fb4a80696b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d0141660 0x7fb4d015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d00ce560 0x7fb4d0159bc0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 --2- 172.21.15.143:0/2017976188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d0071d00 0x7fb4d00cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 >> 172.21.15.143:0/2017976188 conn(0x7fb4d006c3f0 msgr2=0x7fb4d00b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 shutdown_connections 2024-01-15T18:11:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.201+0000 7fb4d6800640 1 -- 172.21.15.143:0/2017976188 wait complete. 2024-01-15T18:11:35.222 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:35.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.353+0000 7fc96ca00640 1 Processor -- start 2024-01-15T18:11:35.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.353+0000 7fc96ca00640 1 -- start start 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc964157300 0x7fc964159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc9640b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96ca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9640b2a20 con 0x7fc9640b48f0 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96ca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9640b35a0 con 0x7fc964157300 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9640b4d80 con 0x7fc9640b21d0 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc969600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc9640b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc964157300 0x7fc964159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc964157300 0x7fc964159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:39526/0 (socket says 172.21.15.143:39526) 2024-01-15T18:11:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc969600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc9640b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42262/0 (socket says 172.21.15.143:42262) 2024-01-15T18:11:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 -- 172.21.15.143:0/2346437078 learned_addr learned my addr 172.21.15.143:0/2346437078 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 -- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 msgr2=0x7fc9640b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 --2- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc9640b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 -- 172.21.15.143:0/2346437078 >> v1:172.21.15.204:6789/0 conn(0x7fc9640b48f0 legacy=0x7fc96415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.357+0000 7fc96a000640 1 -- 172.21.15.143:0/2346437078 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9640b2d00 con 0x7fc964157300 2024-01-15T18:11:35.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96a000640 1 --2- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964159b70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc954009870 tx=0x7fc95401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31452a78a2baa62b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc968c00640 1 -- 172.21.15.143:0/2346437078 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540ab600 con 0x7fc964157300 2024-01-15T18:11:35.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc968c00640 1 -- 172.21.15.143:0/2346437078 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9540ab760 con 0x7fc964157300 2024-01-15T18:11:35.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc968c00640 1 -- 172.21.15.143:0/2346437078 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540abaf0 con 0x7fc964157300 2024-01-15T18:11:35.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 -- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 msgr2=0x7fc964159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 --2- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964159b70 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc954009870 tx=0x7fc95401c510 comp rx=0 tx=0).stop 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 -- 172.21.15.143:0/2346437078 shutdown_connections 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 --2- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc9640b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 --2- 172.21.15.143:0/2346437078 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964159b70 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 -- 172.21.15.143:0/2346437078 >> 172.21.15.143:0/2346437078 conn(0x7fc96406c3f0 msgr2=0x7fc96406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 -- 172.21.15.143:0/2346437078 shutdown_connections 2024-01-15T18:11:35.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 -- 172.21.15.143:0/2346437078 wait complete. 2024-01-15T18:11:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 Processor -- start 2024-01-15T18:11:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.361+0000 7fc96ca00640 1 -- start start 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc964154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 0x7fc964156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964167130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc964156250 con 0x7fc9640b48f0 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641555d0 con 0x7fc964157300 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9641558b0 con 0x7fc9640b21d0 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 0x7fc964156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 0x7fc964156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55320/0 (socket says 172.21.15.143:55320) 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 -- 172.21.15.143:0/3766874884 learned_addr learned my addr 172.21.15.143:0/3766874884 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96a000640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc964154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96aa00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964167130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 -- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 msgr2=0x7fc964154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc964154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 -- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 msgr2=0x7fc964167130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964167130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 -- 172.21.15.143:0/3766874884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540094d0 con 0x7fc9640b48f0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96aa00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964167130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc969600640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 0x7fc964156670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e930 tx=0x7fc94c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c009690 con 0x7fc9640b48f0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c00ca00 con 0x7fc9640b48f0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c00cd70 con 0x7fc9640b48f0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc964155b90 con 0x7fc9640b48f0 2024-01-15T18:11:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc964167670 con 0x7fc9640b48f0 2024-01-15T18:11:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.365+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc964156390 con 0x7fc9640b48f0 2024-01-15T18:11:35.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.369+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fc94c013470 con 0x7fc9640b48f0 2024-01-15T18:11:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.369+0000 7fc962000640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc93c0671f0 0x7fc93c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.369+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc94c0830b0 con 0x7fc9640b48f0 2024-01-15T18:11:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.369+0000 7fc96a000640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc93c0671f0 0x7fc93c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.369+0000 7fc96a000640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc93c0671f0 0x7fc93c0696b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9540af000 tx=0x7fc9540b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.373+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc964156390 con 0x7fc9640b48f0 2024-01-15T18:11:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.521+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc964156390 con 0x7fc9640b48f0 2024-01-15T18:11:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.521+0000 7fc962000640 1 -- 172.21.15.143:0/3766874884 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16660 (secure 0 0 0) 0x7fc964156390 con 0x7fc9640b48f0 2024-01-15T18:11:35.526 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:35.526 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":71,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:34.225038+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:11:32.460276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-15T18:11:33.227442+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc93c0671f0 msgr2=0x7fc93c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc93c0671f0 0x7fc93c0696b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc9540af000 tx=0x7fc9540b6040 comp rx=0 tx=0).stop 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 msgr2=0x7fc964156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 0x7fc964156670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00e930 tx=0x7fc94c00edf0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 shutdown_connections 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc93c0671f0 0x7fc93c0696b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc964157300 0x7fc964167130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9640b48f0 0x7fc964156670 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 --2- 172.21.15.143:0/3766874884 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc9640b21d0 0x7fc964154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 >> 172.21.15.143:0/3766874884 conn(0x7fc96406c3f0 msgr2=0x7fc96415b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 shutdown_connections 2024-01-15T18:11:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.525+0000 7fc96ca00640 1 -- 172.21.15.143:0/3766874884 wait complete. 2024-01-15T18:11:35.548 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:11:35.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 Processor -- start 2024-01-15T18:11:35.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 -- start start 2024-01-15T18:11:35.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40ce690 0x7f47e40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47e40b1da0 0x7f47e40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e4070bf0 con 0x7f47e40b3bf0 2024-01-15T18:11:35.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e415dba0 con 0x7f47e40b1da0 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47ebc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e40b2740 con 0x7f47e40ce690 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47e40b1da0 0x7f47e40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47e40b1da0 0x7f47e40b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55324/0 (socket says 172.21.15.143:55324) 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 -- 172.21.15.143:0/778202468 learned_addr learned my addr 172.21.15.143:0/778202468 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 -- 172.21.15.143:0/778202468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40ce690 msgr2=0x7f47e40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 --2- 172.21.15.143:0/778202468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40ce690 0x7f47e40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 -- 172.21.15.143:0/778202468 >> v1:172.21.15.204:6789/0 conn(0x7f47e40b3bf0 legacy=0x7f47e40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:35.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.685+0000 7f47e3e00640 1 -- 172.21.15.143:0/778202468 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e40b4920 con 0x7f47e40b1da0 2024-01-15T18:11:35.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47e3e00640 1 --2- 172.21.15.143:0/778202468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40b1da0 0x7f47e40b2200 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009870 tx=0x7f47cc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe6ca11521dbfef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47e3400640 1 -- 172.21.15.143:0/778202468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0aa5f0 con 0x7f47e40b1da0 2024-01-15T18:11:35.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47e3400640 1 -- 172.21.15.143:0/778202468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc0aa750 con 0x7f47e40b1da0 2024-01-15T18:11:35.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47e3400640 1 -- 172.21.15.143:0/778202468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0aaae0 con 0x7f47e40b1da0 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 -- 172.21.15.143:0/778202468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40b1da0 msgr2=0x7f47e40b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 --2- 172.21.15.143:0/778202468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40b1da0 0x7f47e40b2200 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009870 tx=0x7f47cc01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 -- 172.21.15.143:0/778202468 shutdown_connections 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 --2- 172.21.15.143:0/778202468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40b1da0 0x7f47e40b2200 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 --2- 172.21.15.143:0/778202468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40ce690 0x7f47e40ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 -- 172.21.15.143:0/778202468 >> 172.21.15.143:0/778202468 conn(0x7f47e406c3f0 msgr2=0x7f47e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 -- 172.21.15.143:0/778202468 shutdown_connections 2024-01-15T18:11:35.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.689+0000 7f47ebc00640 1 -- 172.21.15.143:0/778202468 wait complete. 2024-01-15T18:11:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 Processor -- start 2024-01-15T18:11:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 -- start start 2024-01-15T18:11:35.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47e40b1da0 0x7f47e41408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40b3bf0 0x7f47e4158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 0x7f47e415dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e415a530 con 0x7f47e40ce690 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e415a9c0 con 0x7f47e40b1da0 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e4159640 con 0x7f47e40b3bf0 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 0x7f47e415dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 0x7f47e415dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55332/0 (socket says 172.21.15.143:55332) 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 -- 172.21.15.143:0/2120931140 learned_addr learned my addr 172.21.15.143:0/2120931140 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9200640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47e40b1da0 0x7f47e41408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e3e00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40b3bf0 0x7f47e4158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 -- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40b3bf0 msgr2=0x7f47e4158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40b3bf0 0x7f47e4158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 -- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47e40b1da0 msgr2=0x7f47e41408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47e40b1da0 0x7f47e41408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:35.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 -- 172.21.15.143:0/2120931140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0094d0 con 0x7f47e40ce690 2024-01-15T18:11:35.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e3e00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40b3bf0 0x7f47e4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:35.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e9c00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 0x7f47e415dd10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c780 tx=0x7f47dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004b20 con 0x7f47e40ce690 2024-01-15T18:11:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc004c80 con 0x7f47e40ce690 2024-01-15T18:11:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc00aa90 con 0x7f47e40ce690 2024-01-15T18:11:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e4159920 con 0x7f47e40ce690 2024-01-15T18:11:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.693+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e4159ee0 con 0x7f47e40ce690 2024-01-15T18:11:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.697+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e415a6a0 con 0x7f47e40ce690 2024-01-15T18:11:35.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.697+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f47dc00abf0 con 0x7f47e40ce690 2024-01-15T18:11:35.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.697+0000 7f47e0c00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47bc0671f0 0x7f47bc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:35.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.697+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f47dc082e10 con 0x7f47e40ce690 2024-01-15T18:11:35.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.697+0000 7f47e9200640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47bc0671f0 0x7f47bc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:35.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.697+0000 7f47e9200640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47bc0671f0 0x7f47bc0696b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f47d4006fd0 tx=0x7f47d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:35.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.701+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47dc051b00 con 0x7f47e40ce690 2024-01-15T18:11:35.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:35.889+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f47e4067500 con 0x7f47e40ce690 2024-01-15T18:11:36.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.861+0000 7f47e0c00640 1 -- 172.21.15.143:0/2120931140 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f47dc055120 con 0x7f47e40ce690 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47bc0671f0 msgr2=0x7f47bc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47bc0671f0 0x7f47bc0696b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f47d4006fd0 tx=0x7f47d4008040 comp rx=0 tx=0).stop 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 msgr2=0x7f47e415dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 0x7f47e415dd10 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c780 tx=0x7f47dc00cc40 comp rx=0 tx=0).stop 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 shutdown_connections 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47bc0671f0 0x7f47bc0696b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47e40ce690 0x7f47e415dd10 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47e40b3bf0 0x7f47e4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 --2- 172.21.15.143:0/2120931140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47e40b1da0 0x7f47e41408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 >> 172.21.15.143:0/2120931140 conn(0x7f47e406c3f0 msgr2=0x7f47e415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 shutdown_connections 2024-01-15T18:11:36.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:36.865+0000 7f47ebc00640 1 -- 172.21.15.143:0/2120931140 wait complete. 2024-01-15T18:11:36.894 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:11:36.895 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:11:37.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.037+0000 7f4e2b800640 1 Processor -- start 2024-01-15T18:11:37.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.037+0000 7f4e2b800640 1 -- start start 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e240cdef0 0x7f4e2415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e2b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4e240cf150 0x7f4e2415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e2b800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e240d02d0 con 0x7f4e2406c540 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e2b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e240cf6c0 con 0x7f4e240cf150 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e2b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240cf9a0 con 0x7f4e240cdef0 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e28e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4e2406c540 0x7f4e240cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56686/0 (socket says 172.21.15.143:56686) 2024-01-15T18:11:37.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e28e00640 1 -- 172.21.15.143:0/1594301506 learned_addr learned my addr 172.21.15.143:0/1594301506 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:37.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e23e00640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e240cdef0 0x7f4e2415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:37.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e29800640 1 --2- 172.21.15.143:0/1594301506 >> v2:172.21.15.204:3300/0 conn(0x7f4e240cf150 0x7f4e2415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:37.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e23400640 1 -- 172.21.15.143:0/1594301506 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902121822 0 0) 0x7f4e240d02d0 con 0x7f4e2406c540 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e23400640 1 -- 172.21.15.143:0/1594301506 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e04003530 con 0x7f4e2406c540 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e23400640 1 -- 172.21.15.143:0/1594301506 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497355936 0 0) 0x7f4e04003530 con 0x7f4e2406c540 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e29800640 1 -- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e240cdef0 msgr2=0x7f4e2415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e29800640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e240cdef0 0x7f4e2415ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e29800640 1 -- 172.21.15.143:0/1594301506 >> v1:172.21.15.143:6789/0 conn(0x7f4e2406c540 legacy=0x7f4e240cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e29800640 1 -- 172.21.15.143:0/1594301506 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e240cfc80 con 0x7f4e240cf150 2024-01-15T18:11:37.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.041+0000 7f4e23e00640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e240cdef0 0x7f4e2415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:37.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e29800640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e2415cfd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009d90 tx=0x7f4e1c01c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e3269dc06be8e38 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:37.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e23400640 1 -- 172.21.15.143:0/1594301506 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2406cb30 con 0x7f4e240cf150 2024-01-15T18:11:37.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e23400640 1 -- 172.21.15.143:0/1594301506 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c0a55d0 con 0x7f4e240cf150 2024-01-15T18:11:37.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e23400640 1 -- 172.21.15.143:0/1594301506 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14001680 con 0x7f4e240cf150 2024-01-15T18:11:37.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 -- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 msgr2=0x7f4e2415cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:37.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e2415cfd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009d90 tx=0x7f4e1c01c7b0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 -- 172.21.15.143:0/1594301506 shutdown_connections 2024-01-15T18:11:37.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e2415cfd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 --2- 172.21.15.143:0/1594301506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e240cdef0 0x7f4e2415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 -- 172.21.15.143:0/1594301506 >> 172.21.15.143:0/1594301506 conn(0x7f4e240d0e20 msgr2=0x7f4e240d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:37.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 -- 172.21.15.143:0/1594301506 shutdown_connections 2024-01-15T18:11:37.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 -- 172.21.15.143:0/1594301506 wait complete. 2024-01-15T18:11:37.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 Processor -- start 2024-01-15T18:11:37.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.045+0000 7f4e2b800640 1 -- start start 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e2406c540 0x7f4e240b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 0x7f4e24141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e24142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240ca900 con 0x7f4e240cdef0 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c9cf0 con 0x7f4e240cf150 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e240c9fd0 con 0x7f4e2406c540 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 0x7f4e24141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 0x7f4e24141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55348/0 (socket says 172.21.15.143:55348) 2024-01-15T18:11:37.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 -- 172.21.15.143:0/2827610622 learned_addr learned my addr 172.21.15.143:0/2827610622 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e28e00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e2406c540 0x7f4e240b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e29800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e24142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 -- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e2406c540 msgr2=0x7f4e240b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e2406c540 0x7f4e240b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 -- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 msgr2=0x7f4e24142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e24142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 -- 172.21.15.143:0/2827610622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009420 con 0x7f4e240cdef0 2024-01-15T18:11:37.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e23e00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 0x7f4e24141f10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00ebe0 tx=0x7f4e0c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:37.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00eee0 con 0x7f4e240cdef0 2024-01-15T18:11:37.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c00cc30 con 0x7f4e240cdef0 2024-01-15T18:11:37.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c009a70 con 0x7f4e240cdef0 2024-01-15T18:11:37.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c013470 con 0x7f4e240cdef0 2024-01-15T18:11:37.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e240ca590 con 0x7f4e240cdef0 2024-01-15T18:11:37.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e24067500 con 0x7f4e240cdef0 2024-01-15T18:11:37.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.049+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f4e0c0135d0 con 0x7f4e240cdef0 2024-01-15T18:11:37.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.053+0000 7f4e20c00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4dfc067220 0x7f4dfc0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:37.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.053+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4e0c083070 con 0x7f4e240cdef0 2024-01-15T18:11:37.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.053+0000 7f4e28e00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4dfc067220 0x7f4dfc0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:37.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.053+0000 7f4e28e00640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4dfc067220 0x7f4dfc0696e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e14006b60 tx=0x7f4e14001dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:37.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.057+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4e0c051bc0 con 0x7f4e240cdef0 2024-01-15T18:11:37.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.241+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4e240d1940 con 0x7f4e240cdef0 2024-01-15T18:11:37.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.873+0000 7f4e20c00640 1 -- 172.21.15.143:0/2827610622 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4e0c0551e0 con 0x7f4e240cdef0 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4dfc067220 msgr2=0x7f4dfc0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4dfc067220 0x7f4dfc0696e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e14006b60 tx=0x7f4e14001dd0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 msgr2=0x7f4e24141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 0x7f4e24141f10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00ebe0 tx=0x7f4e0c00c520 comp rx=0 tx=0).stop 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 shutdown_connections 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4dfc067220 0x7f4dfc0696e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4e240cf150 0x7f4e24142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4e240cdef0 0x7f4e24141f10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 --2- 172.21.15.143:0/2827610622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4e2406c540 0x7f4e240b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 >> 172.21.15.143:0/2827610622 conn(0x7f4e240d0e20 msgr2=0x7f4e2406fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 shutdown_connections 2024-01-15T18:11:37.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:37.877+0000 7f4e2b800640 1 -- 172.21.15.143:0/2827610622 wait complete. 2024-01-15T18:11:37.902 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:11:37.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:11:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f258b800640 1 Processor -- start 2024-01-15T18:11:38.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f258b800640 1 -- start start 2024-01-15T18:11:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f258b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f25840cdf20 0x7f25840cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f258b800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840c4d90 con 0x7f258415b260 2024-01-15T18:11:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f258b800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258415b640 con 0x7f25840c6230 2024-01-15T18:11:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f258b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840cf800 con 0x7f25840cdf20 2024-01-15T18:11:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2588e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f258415b260 0x7f25840cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55646/0 (socket says 172.21.15.143:55646) 2024-01-15T18:11:38.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2588e00640 1 -- 172.21.15.143:0/3697249681 learned_addr learned my addr 172.21.15.143:0/3697249681 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2583e00640 1 --2- 172.21.15.143:0/3697249681 >> v2:172.21.15.204:3300/0 conn(0x7f25840cdf20 0x7f25840cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2583400640 1 -- 172.21.15.143:0/3697249681 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049209410 0 0) 0x7f25840c4d90 con 0x7f258415b260 2024-01-15T18:11:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2583400640 1 -- 172.21.15.143:0/3697249681 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2564003530 con 0x7f258415b260 2024-01-15T18:11:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2583e00640 1 -- 172.21.15.143:0/3697249681 >> v1:172.21.15.204:6789/0 conn(0x7f25840c6230 legacy=0x7f25840cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2583e00640 1 -- 172.21.15.143:0/3697249681 >> v1:172.21.15.143:6789/0 conn(0x7f258415b260 legacy=0x7f25840cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.041+0000 7f2583e00640 1 -- 172.21.15.143:0/3697249681 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840cfae0 con 0x7f25840cdf20 2024-01-15T18:11:38.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.045+0000 7f2583e00640 1 --2- 172.21.15.143:0/3697249681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840cdf20 0x7f25840cbae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f256c0097c0 tx=0x7f256c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f59145ee563a24 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.045+0000 7f2583400640 1 -- 172.21.15.143:0/3697249681 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0ab650 con 0x7f25840cdf20 2024-01-15T18:11:38.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.045+0000 7f2583400640 1 -- 172.21.15.143:0/3697249681 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25840ccf40 con 0x7f25840cdf20 2024-01-15T18:11:38.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.045+0000 7f2583400640 1 -- 172.21.15.143:0/3697249681 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0ab9d0 con 0x7f25840cdf20 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.045+0000 7f258b800640 1 -- 172.21.15.143:0/3697249681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840cdf20 msgr2=0x7f25840cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.045+0000 7f258b800640 1 --2- 172.21.15.143:0/3697249681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840cdf20 0x7f25840cbae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f256c0097c0 tx=0x7f256c01c450 comp rx=0 tx=0).stop 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- 172.21.15.143:0/3697249681 shutdown_connections 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 --2- 172.21.15.143:0/3697249681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840cdf20 0x7f25840cbae0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- 172.21.15.143:0/3697249681 >> 172.21.15.143:0/3697249681 conn(0x7f25840d0ed0 msgr2=0x7f25840d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- 172.21.15.143:0/3697249681 shutdown_connections 2024-01-15T18:11:38.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- 172.21.15.143:0/3697249681 wait complete. 2024-01-15T18:11:38.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 Processor -- start 2024-01-15T18:11:38.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- start start 2024-01-15T18:11:38.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840c6230 0x7f2584153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 0x7f2584150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f258415b260 0x7f2584152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841535e0 con 0x7f25840cdf20 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584153f40 con 0x7f25840c6230 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f258b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584151070 con 0x7f258415b260 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 0x7f2584150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 0x7f2584150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51688/0 (socket says 172.21.15.143:51688) 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 -- 172.21.15.143:0/2391027600 learned_addr learned my addr 172.21.15.143:0/2391027600 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2588e00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840c6230 0x7f2584153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2589800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f258415b260 0x7f2584152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 -- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f258415b260 msgr2=0x7f2584152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f258415b260 0x7f2584152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 -- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840c6230 msgr2=0x7f2584153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840c6230 0x7f2584153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.049+0000 7f2583e00640 1 -- 172.21.15.143:0/2391027600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c009420 con 0x7f25840cdf20 2024-01-15T18:11:38.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2583e00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 0x7f2584150b30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f256c0ad040 tx=0x7f256c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0abea0 con 0x7f25840cdf20 2024-01-15T18:11:38.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c0bf650 con 0x7f25840cdf20 2024-01-15T18:11:38.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c0bfa00 con 0x7f25840cdf20 2024-01-15T18:11:38.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2584152960 con 0x7f25840cdf20 2024-01-15T18:11:38.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2584152f20 con 0x7f25840cdf20 2024-01-15T18:11:38.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584153720 con 0x7f25840cdf20 2024-01-15T18:11:38.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f256c0b5400 con 0x7f25840cdf20 2024-01-15T18:11:38.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2580c00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f255c067220 0x7f255c0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.053+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f256c11d510 con 0x7f25840cdf20 2024-01-15T18:11:38.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.057+0000 7f2588e00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f255c067220 0x7f255c0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.057+0000 7f2588e00640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f255c067220 0x7f255c0696e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f25740011f0 tx=0x7f2574007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.057+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2584153720 con 0x7f25840cdf20 2024-01-15T18:11:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.237+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2584153720 con 0x7f25840cdf20 2024-01-15T18:11:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f2580c00640 1 -- 172.21.15.143:0/2391027600 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f2584153720 con 0x7f25840cdf20 2024-01-15T18:11:38.243 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:38.243 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3237385557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3237385557},{"type":"v1","addr":"172.21.15.204:6834","nonce":3237385557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/904040921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":904040921},{"type":"v1","addr":"172.21.15.204:6835","nonce":904040921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/64114877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":64114877},{"type":"v1","addr":"172.21.15.143:6836","nonce":64114877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:11:34.225370+0000","modified":"2024-01-15T18:11:37.246452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/1300573627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":1300573627},{"type":"v1","addr":"172.21.15.143:6837","nonce":1300573627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-01-15T18:11:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 44 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f255c067220 msgr2=0x7f255c0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f255c067220 0x7f255c0696e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f25740011f0 tx=0x7f2574007d40 comp rx=0 tx=0).stop 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 msgr2=0x7f2584150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 0x7f2584150b30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f256c0ad040 tx=0x7f256c01ac90 comp rx=0 tx=0).stop 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 shutdown_connections 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f255c067220 0x7f255c0696e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f258415b260 0x7f2584152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25840cdf20 0x7f2584150b30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 --2- 172.21.15.143:0/2391027600 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25840c6230 0x7f2584153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 >> 172.21.15.143:0/2391027600 conn(0x7f25840d0ed0 msgr2=0x7f25840cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 shutdown_connections 2024-01-15T18:11:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.241+0000 7f258b800640 1 -- 172.21.15.143:0/2391027600 wait complete. 2024-01-15T18:11:38.264 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:38.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 Processor -- start 2024-01-15T18:11:38.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 -- start start 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5824141660 0x7f58240ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5824071d00 0x7f5824072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582415f310 con 0x7f582415ec80 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582415fbb0 con 0x7f5824141660 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f582ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582415f130 con 0x7f5824071d00 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5824141660 0x7f58240ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5824141660 0x7f58240ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51702/0 (socket says 172.21.15.143:51702) 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 -- 172.21.15.143:0/1965390039 learned_addr learned my addr 172.21.15.143:0/1965390039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:38.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f582415ec80 0x7f5824141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55662/0 (socket says 172.21.15.143:55662) 2024-01-15T18:11:38.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5828c00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5824071d00 0x7f5824072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 -- 172.21.15.143:0/1965390039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5824071d00 msgr2=0x7f5824072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:38.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5824071d00 0x7f5824072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 -- 172.21.15.143:0/1965390039 >> v1:172.21.15.143:6789/0 conn(0x7f582415ec80 legacy=0x7f5824141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5823e00640 1 -- 172.21.15.143:0/1965390039 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582415f5f0 con 0x7f5824141660 2024-01-15T18:11:38.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.397+0000 7f5828c00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5824071d00 0x7f5824072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:38.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f5823e00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824141660 0x7f58240ce560 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5814009870 tx=0x7f581401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c729bc1def4b26 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f5822a00640 1 -- 172.21.15.143:0/1965390039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140a7070 con 0x7f5824141660 2024-01-15T18:11:38.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f5822a00640 1 -- 172.21.15.143:0/1965390039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58140ab430 con 0x7f5824141660 2024-01-15T18:11:38.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f5822a00640 1 -- 172.21.15.143:0/1965390039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140ab7c0 con 0x7f5824141660 2024-01-15T18:11:38.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 -- 172.21.15.143:0/1965390039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824141660 msgr2=0x7f58240ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824141660 0x7f58240ce560 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f5814009870 tx=0x7f581401c540 comp rx=0 tx=0).stop 2024-01-15T18:11:38.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 -- 172.21.15.143:0/1965390039 shutdown_connections 2024-01-15T18:11:38.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5824071d00 0x7f5824072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 --2- 172.21.15.143:0/1965390039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824141660 0x7f58240ce560 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 -- 172.21.15.143:0/1965390039 >> 172.21.15.143:0/1965390039 conn(0x7f582406c3f0 msgr2=0x7f582406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:38.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 -- 172.21.15.143:0/1965390039 shutdown_connections 2024-01-15T18:11:38.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.401+0000 7f582ac00640 1 -- 172.21.15.143:0/1965390039 wait complete. 2024-01-15T18:11:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 Processor -- start 2024-01-15T18:11:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 -- start start 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 0x7f582415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5824141660 0x7f582415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f582415ec80 0x7f582415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582415a6f0 con 0x7f5824071d00 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582415bc40 con 0x7f5824141660 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582415bf20 con 0x7f582415ec80 2024-01-15T18:11:38.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 0x7f582415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 0x7f582415a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51710/0 (socket says 172.21.15.143:51710) 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 -- 172.21.15.143:0/2461646071 learned_addr learned my addr 172.21.15.143:0/2461646071 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5828c00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f582415ec80 0x7f582415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823400640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5824141660 0x7f582415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 -- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f582415ec80 msgr2=0x7f582415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f582415ec80 0x7f582415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 -- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5824141660 msgr2=0x7f582415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5824141660 0x7f582415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 -- 172.21.15.143:0/2461646071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140094d0 con 0x7f5824071d00 2024-01-15T18:11:38.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f5823e00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 0x7f582415a1b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f58140ad040 tx=0x7f58140192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140bc070 con 0x7f5824071d00 2024-01-15T18:11:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58140b4630 con 0x7f5824071d00 2024-01-15T18:11:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58240cd150 con 0x7f5824071d00 2024-01-15T18:11:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58140b4a00 con 0x7f5824071d00 2024-01-15T18:11:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.405+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582415c4e0 con 0x7f5824071d00 2024-01-15T18:11:38.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.409+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5824067500 con 0x7f5824071d00 2024-01-15T18:11:38.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.409+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f58140b4b60 con 0x7f5824071d00 2024-01-15T18:11:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.409+0000 7f581be00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57fc0671f0 0x7f57fc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.409+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f581411c840 con 0x7f5824071d00 2024-01-15T18:11:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.409+0000 7f5823400640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57fc0671f0 0x7f57fc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.409+0000 7f5823400640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57fc0671f0 0x7f57fc0696b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f582413dd30 tx=0x7f580c007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.413+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58140eb3f0 con 0x7f5824071d00 2024-01-15T18:11:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.565+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f582415c060 con 0x7f5824071d00 2024-01-15T18:11:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.565+0000 7f581be00640 1 -- 172.21.15.143:0/2461646071 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16647 (secure 0 0 0) 0x7f582415c060 con 0x7f5824071d00 2024-01-15T18:11:38.569 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:38.569 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:11:35.848732+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:11:32.460276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-15T18:11:33.227442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:38.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57fc0671f0 msgr2=0x7f57fc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57fc0671f0 0x7f57fc0696b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f582413dd30 tx=0x7f580c007e50 comp rx=0 tx=0).stop 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 msgr2=0x7f582415a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 0x7f582415a1b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f58140ad040 tx=0x7f58140192e0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 shutdown_connections 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57fc0671f0 0x7f57fc0696b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f582415ec80 0x7f582415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5824141660 0x7f582415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 --2- 172.21.15.143:0/2461646071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5824071d00 0x7f582415a1b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 >> 172.21.15.143:0/2461646071 conn(0x7f582406c3f0 msgr2=0x7f58240af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 shutdown_connections 2024-01-15T18:11:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.569+0000 7f582ac00640 1 -- 172.21.15.143:0/2461646071 wait complete. 2024-01-15T18:11:38.590 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:11:38.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:11:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 Processor -- start 2024-01-15T18:11:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 -- start start 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa3c00b2c30 0x7fa3c006ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3c00713d0 0x7fa3c015d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c0160c90 con 0x7fa3c006bab0 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c0160f70 con 0x7fa3c00713d0 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3c5c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c0161250 con 0x7fa3c00b2c30 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3bf400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3c00713d0 0x7fa3c015d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3be000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa3c006bab0 0x7fa3c0070cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55678/0 (socket says 172.21.15.143:55678) 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.725+0000 7fa3be000640 1 -- 172.21.15.143:0/4191691026 learned_addr learned my addr 172.21.15.143:0/4191691026 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bea00640 1 --2- 172.21.15.143:0/4191691026 >> v2:172.21.15.204:3300/0 conn(0x7fa3c00b2c30 0x7fa3c006ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bf400640 1 -- 172.21.15.143:0/4191691026 >> v2:172.21.15.204:3300/0 conn(0x7fa3c00b2c30 msgr2=0x7fa3c006ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bf400640 1 --2- 172.21.15.143:0/4191691026 >> v2:172.21.15.204:3300/0 conn(0x7fa3c00b2c30 0x7fa3c006ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bf400640 1 -- 172.21.15.143:0/4191691026 >> v1:172.21.15.143:6789/0 conn(0x7fa3c006bab0 legacy=0x7fa3c0070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bf400640 1 -- 172.21.15.143:0/4191691026 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0161530 con 0x7fa3c00713d0 2024-01-15T18:11:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bea00640 1 --2- 172.21.15.143:0/4191691026 >> v2:172.21.15.204:3300/0 conn(0x7fa3c00b2c30 0x7fa3c006ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:38.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bf400640 1 --2- 172.21.15.143:0/4191691026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c00713d0 0x7fa3c015d9f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0066840 tx=0x7fa3b801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dd39fb7ca24281 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bd600640 1 -- 172.21.15.143:0/4191691026 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80a5830 con 0x7fa3c00713d0 2024-01-15T18:11:38.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bd600640 1 -- 172.21.15.143:0/4191691026 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80a5990 con 0x7fa3c00713d0 2024-01-15T18:11:38.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.729+0000 7fa3bd600640 1 -- 172.21.15.143:0/4191691026 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80a5d20 con 0x7fa3c00713d0 2024-01-15T18:11:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- 172.21.15.143:0/4191691026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c00713d0 msgr2=0x7fa3c015d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/4191691026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c00713d0 0x7fa3c015d9f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0066840 tx=0x7fa3b801c850 comp rx=0 tx=0).stop 2024-01-15T18:11:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- 172.21.15.143:0/4191691026 shutdown_connections 2024-01-15T18:11:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/4191691026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c00713d0 0x7fa3c015d9f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/4191691026 >> v2:172.21.15.204:3300/0 conn(0x7fa3c00b2c30 0x7fa3c006ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- 172.21.15.143:0/4191691026 >> 172.21.15.143:0/4191691026 conn(0x7fa3c00b1610 msgr2=0x7fa3c00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:38.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- 172.21.15.143:0/4191691026 shutdown_connections 2024-01-15T18:11:38.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- 172.21.15.143:0/4191691026 wait complete. 2024-01-15T18:11:38.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 Processor -- start 2024-01-15T18:11:38.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- start start 2024-01-15T18:11:38.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 0x7fa3c014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00713d0 0x7fa3c0146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3c00b2c30 0x7fa3c015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00b02a0 con 0x7fa3c006bab0 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00b0580 con 0x7fa3c00713d0 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3c5c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c014b0d0 con 0x7fa3c00b2c30 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 0x7fa3c014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 0x7fa3c014e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51730/0 (socket says 172.21.15.143:51730) 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 -- 172.21.15.143:0/2524357447 learned_addr learned my addr 172.21.15.143:0/2524357447 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3be000640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00713d0 0x7fa3c0146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bf400640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3c00b2c30 0x7fa3c015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 -- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3c00b2c30 msgr2=0x7fa3c015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3c00b2c30 0x7fa3c015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 -- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00713d0 msgr2=0x7fa3c0146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.733+0000 7fa3bea00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00713d0 0x7fa3c0146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3bea00640 1 -- 172.21.15.143:0/2524357447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b8009a70 con 0x7fa3c006bab0 2024-01-15T18:11:38.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3bea00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 0x7fa3c014e980 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d800 tx=0x7fa3b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000b6c0 con 0x7fa3c006bab0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000b820 con 0x7fa3c006bab0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c014b3b0 con 0x7fa3c006bab0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b000bbd0 con 0x7fa3c006bab0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c015ffd0 con 0x7fa3c006bab0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c0074b00 con 0x7fa3c006bab0 2024-01-15T18:11:38.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fa3b000bd30 con 0x7fa3c006bab0 2024-01-15T18:11:38.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3b6a00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3980671f0 0x7fa3980696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:38.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa3b0083370 con 0x7fa3c006bab0 2024-01-15T18:11:38.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.737+0000 7fa3be000640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3980671f0 0x7fa3980696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:38.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.741+0000 7fa3be000640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3980671f0 0x7fa3980696b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa3c01608e0 tx=0x7fa3a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:38.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.741+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3b0051e70 con 0x7fa3c006bab0 2024-01-15T18:11:38.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.893+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3c014af30 con 0x7fa3980671f0 2024-01-15T18:11:38.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.901+0000 7fa3b6a00640 1 -- 172.21.15.143:0/2524357447 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287707 (secure 0 0 0) 0x7fa3c014af30 con 0x7fa3980671f0 2024-01-15T18:11:38.909 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:38.910 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3980671f0 msgr2=0x7fa3980696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3980671f0 0x7fa3980696b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa3c01608e0 tx=0x7fa3a8009040 comp rx=0 tx=0).stop 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 msgr2=0x7fa3c014e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 0x7fa3c014e980 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000d800 tx=0x7fa3b000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 shutdown_connections 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3980671f0 0x7fa3980696b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3c00b2c30 0x7fa3c015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3c00713d0 0x7fa3c0146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 --2- 172.21.15.143:0/2524357447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3c006bab0 0x7fa3c014e980 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 >> 172.21.15.143:0/2524357447 conn(0x7fa3c00b1610 msgr2=0x7fa3c015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:38.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 shutdown_connections 2024-01-15T18:11:38.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:38.905+0000 7fa3c5c00640 1 -- 172.21.15.143:0/2524357447 wait complete. 2024-01-15T18:11:38.931 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-01-15T18:11:38.759723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406036,"kb_used_data":8024,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568684,"statfs":{"total":773094113280,"available":772678332416,"internally_reserved":0,"allocated":8216576,"data_stored":4134060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":407424560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.929730"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.875110+0000","last_change":"2024-01-15T18:11:34.869523+0000","last_active":"2024-01-15T18:11:34.875110+0000","last_peered":"2024-01-15T18:11:34.875110+0000","last_clean":"2024-01-15T18:11:34.875110+0000","last_became_active":"2024-01-15T18:11:34.869232+0000","last_became_peered":"2024-01-15T18:11:34.869232+0000","last_unstale":"2024-01-15T18:11:34.875110+0000","last_undegraded":"2024-01-15T18:11:34.875110+0000","last_fullsized":"2024-01-15T18:11:34.875110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:04.257797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855222+0000","last_change":"2024-01-15T18:11:34.855541+0000","last_active":"2024-01-15T18:11:35.855222+0000","last_peered":"2024-01-15T18:11:35.855222+0000","last_clean":"2024-01-15T18:11:35.855222+0000","last_became_active":"2024-01-15T18:11:34.855244+0000","last_became_peered":"2024-01-15T18:11:34.855244+0000","last_unstale":"2024-01-15T18:11:35.855222+0000","last_undegraded":"2024-01-15T18:11:35.855222+0000","last_fullsized":"2024-01-15T18:11:35.855222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:46:26.072190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854072+0000","last_change":"2024-01-15T18:11:34.862194+0000","last_active":"2024-01-15T18:11:35.854072+0000","last_peered":"2024-01-15T18:11:35.854072+0000","last_clean":"2024-01-15T18:11:35.854072+0000","last_became_active":"2024-01-15T18:11:34.861933+0000","last_became_peered":"2024-01-15T18:11:34.861933+0000","last_unstale":"2024-01-15T18:11:35.854072+0000","last_undegraded":"2024-01-15T18:11:35.854072+0000","last_fullsized":"2024-01-15T18:11:35.854072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:07:57.539145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.865725+0000","last_change":"2024-01-15T18:11:34.860373+0000","last_active":"2024-01-15T18:11:34.865725+0000","last_peered":"2024-01-15T18:11:34.865725+0000","last_clean":"2024-01-15T18:11:34.865725+0000","last_became_active":"2024-01-15T18:11:34.859670+0000","last_became_peered":"2024-01-15T18:11:34.859670+0000","last_unstale":"2024-01-15T18:11:34.865725+0000","last_undegraded":"2024-01-15T18:11:34.865725+0000","last_fullsized":"2024-01-15T18:11:34.865725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:10.071853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899931+0000","last_change":"2024-01-15T18:11:34.854369+0000","last_active":"2024-01-15T18:11:35.899931+0000","last_peered":"2024-01-15T18:11:35.899931+0000","last_clean":"2024-01-15T18:11:35.899931+0000","last_became_active":"2024-01-15T18:11:34.854103+0000","last_became_peered":"2024-01-15T18:11:34.854103+0000","last_unstale":"2024-01-15T18:11:35.899931+0000","last_undegraded":"2024-01-15T18:11:35.899931+0000","last_fullsized":"2024-01-15T18:11:35.899931+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:35.517210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854304+0000","last_change":"2024-01-15T18:11:34.855438+0000","last_active":"2024-01-15T18:11:35.854304+0000","last_peered":"2024-01-15T18:11:35.854304+0000","last_clean":"2024-01-15T18:11:35.854304+0000","last_became_active":"2024-01-15T18:11:34.855123+0000","last_became_peered":"2024-01-15T18:11:34.855123+0000","last_unstale":"2024-01-15T18:11:35.854304+0000","last_undegraded":"2024-01-15T18:11:35.854304+0000","last_fullsized":"2024-01-15T18:11:35.854304+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:08.521071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.866641+0000","last_change":"2024-01-15T18:11:34.860862+0000","last_active":"2024-01-15T18:11:34.866641+0000","last_peered":"2024-01-15T18:11:34.866641+0000","last_clean":"2024-01-15T18:11:34.866641+0000","last_became_active":"2024-01-15T18:11:34.860520+0000","last_became_peered":"2024-01-15T18:11:34.860520+0000","last_unstale":"2024-01-15T18:11:34.866641+0000","last_undegraded":"2024-01-15T18:11:34.866641+0000","last_fullsized":"2024-01-15T18:11:34.866641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:36:11.022504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854289+0000","last_change":"2024-01-15T18:11:34.854799+0000","last_active":"2024-01-15T18:11:35.854289+0000","last_peered":"2024-01-15T18:11:35.854289+0000","last_clean":"2024-01-15T18:11:35.854289+0000","last_became_active":"2024-01-15T18:11:34.854504+0000","last_became_peered":"2024-01-15T18:11:34.854504+0000","last_unstale":"2024-01-15T18:11:35.854289+0000","last_undegraded":"2024-01-15T18:11:35.854289+0000","last_fullsized":"2024-01-15T18:11:35.854289+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:27:03.032629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900191+0000","last_change":"2024-01-15T18:11:34.863452+0000","last_active":"2024-01-15T18:11:35.900191+0000","last_peered":"2024-01-15T18:11:35.900191+0000","last_clean":"2024-01-15T18:11:35.900191+0000","last_became_active":"2024-01-15T18:11:34.862835+0000","last_became_peered":"2024-01-15T18:11:34.862835+0000","last_unstale":"2024-01-15T18:11:35.900191+0000","last_undegraded":"2024-01-15T18:11:35.900191+0000","last_fullsized":"2024-01-15T18:11:35.900191+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:48:31.908622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854847+0000","last_change":"2024-01-15T18:11:34.857855+0000","last_active":"2024-01-15T18:11:35.854847+0000","last_peered":"2024-01-15T18:11:35.854847+0000","last_clean":"2024-01-15T18:11:35.854847+0000","last_became_active":"2024-01-15T18:11:34.857576+0000","last_became_peered":"2024-01-15T18:11:34.857576+0000","last_unstale":"2024-01-15T18:11:35.854847+0000","last_undegraded":"2024-01-15T18:11:35.854847+0000","last_fullsized":"2024-01-15T18:11:35.854847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.874975+0000","last_change":"2024-01-15T18:11:34.869432+0000","last_active":"2024-01-15T18:11:34.874975+0000","last_peered":"2024-01-15T18:11:34.874975+0000","last_clean":"2024-01-15T18:11:34.874975+0000","last_became_active":"2024-01-15T18:11:34.869044+0000","last_became_peered":"2024-01-15T18:11:34.869044+0000","last_unstale":"2024-01-15T18:11:34.874975+0000","last_undegraded":"2024-01-15T18:11:34.874975+0000","last_fullsized":"2024-01-15T18:11:34.874975+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:28.015798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.857794+0000","last_change":"2024-01-15T18:11:34.854639+0000","last_active":"2024-01-15T18:11:34.857794+0000","last_peered":"2024-01-15T18:11:34.857794+0000","last_clean":"2024-01-15T18:11:34.857794+0000","last_became_active":"2024-01-15T18:11:34.854085+0000","last_became_peered":"2024-01-15T18:11:34.854085+0000","last_unstale":"2024-01-15T18:11:34.857794+0000","last_undegraded":"2024-01-15T18:11:34.857794+0000","last_fullsized":"2024-01-15T18:11:34.857794+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:01.327919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927275+0000","last_change":"2024-01-15T18:11:34.863815+0000","last_active":"2024-01-15T18:11:36.927275+0000","last_peered":"2024-01-15T18:11:36.927275+0000","last_clean":"2024-01-15T18:11:36.927275+0000","last_became_active":"2024-01-15T18:11:34.863476+0000","last_became_peered":"2024-01-15T18:11:34.863476+0000","last_unstale":"2024-01-15T18:11:36.927275+0000","last_undegraded":"2024-01-15T18:11:36.927275+0000","last_fullsized":"2024-01-15T18:11:36.927275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:46.051721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900054+0000","last_change":"2024-01-15T18:11:34.858233+0000","last_active":"2024-01-15T18:11:35.900054+0000","last_peered":"2024-01-15T18:11:35.900054+0000","last_clean":"2024-01-15T18:11:35.900054+0000","last_became_active":"2024-01-15T18:11:34.857971+0000","last_became_peered":"2024-01-15T18:11:34.857971+0000","last_unstale":"2024-01-15T18:11:35.900054+0000","last_undegraded":"2024-01-15T18:11:35.900054+0000","last_fullsized":"2024-01-15T18:11:35.900054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:04:01.928940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927077+0000","last_change":"2024-01-15T18:11:34.868083+0000","last_active":"2024-01-15T18:11:36.927077+0000","last_peered":"2024-01-15T18:11:36.927077+0000","last_clean":"2024-01-15T18:11:36.927077+0000","last_became_active":"2024-01-15T18:11:34.867799+0000","last_became_peered":"2024-01-15T18:11:34.867799+0000","last_unstale":"2024-01-15T18:11:36.927077+0000","last_undegraded":"2024-01-15T18:11:36.927077+0000","last_fullsized":"2024-01-15T18:11:36.927077+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:45.043674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853173+0000","last_change":"2024-01-15T18:11:34.863389+0000","last_active":"2024-01-15T18:11:35.853173+0000","last_peered":"2024-01-15T18:11:35.853173+0000","last_clean":"2024-01-15T18:11:35.853173+0000","last_became_active":"2024-01-15T18:11:34.863056+0000","last_became_peered":"2024-01-15T18:11:34.863056+0000","last_unstale":"2024-01-15T18:11:35.853173+0000","last_undegraded":"2024-01-15T18:11:35.853173+0000","last_fullsized":"2024-01-15T18:11:35.853173+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:17.980905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900208+0000","last_change":"2024-01-15T18:11:34.863287+0000","last_active":"2024-01-15T18:11:35.900208+0000","last_peered":"2024-01-15T18:11:35.900208+0000","last_clean":"2024-01-15T18:11:35.900208+0000","last_became_active":"2024-01-15T18:11:34.862486+0000","last_became_peered":"2024-01-15T18:11:34.862486+0000","last_unstale":"2024-01-15T18:11:35.900208+0000","last_undegraded":"2024-01-15T18:11:35.900208+0000","last_fullsized":"2024-01-15T18:11:35.900208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:34.458274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.865272+0000","last_change":"2024-01-15T18:11:34.859323+0000","last_active":"2024-01-15T18:11:34.865272+0000","last_peered":"2024-01-15T18:11:34.865272+0000","last_clean":"2024-01-15T18:11:34.865272+0000","last_became_active":"2024-01-15T18:11:34.858632+0000","last_became_peered":"2024-01-15T18:11:34.858632+0000","last_unstale":"2024-01-15T18:11:34.865272+0000","last_undegraded":"2024-01-15T18:11:34.865272+0000","last_fullsized":"2024-01-15T18:11:34.865272+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:24.488504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855073+0000","last_change":"2024-01-15T18:11:34.855406+0000","last_active":"2024-01-15T18:11:35.855073+0000","last_peered":"2024-01-15T18:11:35.855073+0000","last_clean":"2024-01-15T18:11:35.855073+0000","last_became_active":"2024-01-15T18:11:34.855096+0000","last_became_peered":"2024-01-15T18:11:34.855096+0000","last_unstale":"2024-01-15T18:11:35.855073+0000","last_undegraded":"2024-01-15T18:11:35.855073+0000","last_fullsized":"2024-01-15T18:11:35.855073+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:49.139038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927248+0000","last_change":"2024-01-15T18:11:34.857130+0000","last_active":"2024-01-15T18:11:36.927248+0000","last_peered":"2024-01-15T18:11:36.927248+0000","last_clean":"2024-01-15T18:11:36.927248+0000","last_became_active":"2024-01-15T18:11:34.856869+0000","last_became_peered":"2024-01-15T18:11:34.856869+0000","last_unstale":"2024-01-15T18:11:36.927248+0000","last_undegraded":"2024-01-15T18:11:36.927248+0000","last_fullsized":"2024-01-15T18:11:36.927248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:43:00.875084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853922+0000","last_change":"2024-01-15T18:11:34.858353+0000","last_active":"2024-01-15T18:11:35.853922+0000","last_peered":"2024-01-15T18:11:35.853922+0000","last_clean":"2024-01-15T18:11:35.853922+0000","last_became_active":"2024-01-15T18:11:34.858047+0000","last_became_peered":"2024-01-15T18:11:34.858047+0000","last_unstale":"2024-01-15T18:11:35.853922+0000","last_undegraded":"2024-01-15T18:11:35.853922+0000","last_fullsized":"2024-01-15T18:11:35.853922+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:36:11.022504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853682+0000","last_change":"2024-01-15T18:11:34.852475+0000","last_active":"2024-01-15T18:11:35.853682+0000","last_peered":"2024-01-15T18:11:35.853682+0000","last_clean":"2024-01-15T18:11:35.853682+0000","last_became_active":"2024-01-15T18:11:34.852211+0000","last_became_peered":"2024-01-15T18:11:34.852211+0000","last_unstale":"2024-01-15T18:11:35.853682+0000","last_undegraded":"2024-01-15T18:11:35.853682+0000","last_fullsized":"2024-01-15T18:11:35.853682+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:28.015798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.868531+0000","last_change":"2024-01-15T18:11:34.865503+0000","last_active":"2024-01-15T18:11:34.868531+0000","last_peered":"2024-01-15T18:11:34.868531+0000","last_clean":"2024-01-15T18:11:34.868531+0000","last_became_active":"2024-01-15T18:11:34.865221+0000","last_became_peered":"2024-01-15T18:11:34.865221+0000","last_unstale":"2024-01-15T18:11:34.868531+0000","last_undegraded":"2024-01-15T18:11:34.868531+0000","last_fullsized":"2024-01-15T18:11:34.868531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:43:00.443491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.870584+0000","last_change":"2024-01-15T18:11:34.867365+0000","last_active":"2024-01-15T18:11:34.870584+0000","last_peered":"2024-01-15T18:11:34.870584+0000","last_clean":"2024-01-15T18:11:34.870584+0000","last_became_active":"2024-01-15T18:11:34.867073+0000","last_became_peered":"2024-01-15T18:11:34.867073+0000","last_unstale":"2024-01-15T18:11:34.870584+0000","last_undegraded":"2024-01-15T18:11:34.870584+0000","last_fullsized":"2024-01-15T18:11:34.870584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:38.520588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854096+0000","last_change":"2024-01-15T18:11:34.868146+0000","last_active":"2024-01-15T18:11:35.854096+0000","last_peered":"2024-01-15T18:11:35.854096+0000","last_clean":"2024-01-15T18:11:35.854096+0000","last_became_active":"2024-01-15T18:11:34.867692+0000","last_became_peered":"2024-01-15T18:11:34.867692+0000","last_unstale":"2024-01-15T18:11:35.854096+0000","last_undegraded":"2024-01-15T18:11:35.854096+0000","last_fullsized":"2024-01-15T18:11:35.854096+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:38.520588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.868147+0000","last_change":"2024-01-15T18:11:34.865848+0000","last_active":"2024-01-15T18:11:34.868147+0000","last_peered":"2024-01-15T18:11:34.868147+0000","last_clean":"2024-01-15T18:11:34.868147+0000","last_became_active":"2024-01-15T18:11:34.865552+0000","last_became_peered":"2024-01-15T18:11:34.865552+0000","last_unstale":"2024-01-15T18:11:34.868147+0000","last_undegraded":"2024-01-15T18:11:34.868147+0000","last_fullsized":"2024-01-15T18:11:34.868147+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:38.520588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854087+0000","last_change":"2024-01-15T18:11:34.861447+0000","last_active":"2024-01-15T18:11:35.854087+0000","last_peered":"2024-01-15T18:11:35.854087+0000","last_clean":"2024-01-15T18:11:35.854087+0000","last_became_active":"2024-01-15T18:11:34.861170+0000","last_became_peered":"2024-01-15T18:11:34.861170+0000","last_unstale":"2024-01-15T18:11:35.854087+0000","last_undegraded":"2024-01-15T18:11:35.854087+0000","last_fullsized":"2024-01-15T18:11:35.854087+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:18:57.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854013+0000","last_change":"2024-01-15T18:11:34.857549+0000","last_active":"2024-01-15T18:11:35.854013+0000","last_peered":"2024-01-15T18:11:35.854013+0000","last_clean":"2024-01-15T18:11:35.854013+0000","last_became_active":"2024-01-15T18:11:34.857249+0000","last_became_peered":"2024-01-15T18:11:34.857249+0000","last_unstale":"2024-01-15T18:11:35.854013+0000","last_undegraded":"2024-01-15T18:11:35.854013+0000","last_fullsized":"2024-01-15T18:11:35.854013+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:07.317112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853955+0000","last_change":"2024-01-15T18:11:34.858775+0000","last_active":"2024-01-15T18:11:35.853955+0000","last_peered":"2024-01-15T18:11:35.853955+0000","last_clean":"2024-01-15T18:11:35.853955+0000","last_became_active":"2024-01-15T18:11:34.858182+0000","last_became_peered":"2024-01-15T18:11:34.858182+0000","last_unstale":"2024-01-15T18:11:35.853955+0000","last_undegraded":"2024-01-15T18:11:35.853955+0000","last_fullsized":"2024-01-15T18:11:35.853955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:16:59.468515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.864636+0000","last_change":"2024-01-15T18:11:34.861558+0000","last_active":"2024-01-15T18:11:34.864636+0000","last_peered":"2024-01-15T18:11:34.864636+0000","last_clean":"2024-01-15T18:11:34.864636+0000","last_became_active":"2024-01-15T18:11:34.861372+0000","last_became_peered":"2024-01-15T18:11:34.861372+0000","last_unstale":"2024-01-15T18:11:34.864636+0000","last_undegraded":"2024-01-15T18:11:34.864636+0000","last_fullsized":"2024-01-15T18:11:34.864636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:43:00.443491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.869471+0000","last_change":"2024-01-15T18:11:34.867594+0000","last_active":"2024-01-15T18:11:34.869471+0000","last_peered":"2024-01-15T18:11:34.869471+0000","last_clean":"2024-01-15T18:11:34.869471+0000","last_became_active":"2024-01-15T18:11:34.866643+0000","last_became_peered":"2024-01-15T18:11:34.866643+0000","last_unstale":"2024-01-15T18:11:34.869471+0000","last_undegraded":"2024-01-15T18:11:34.869471+0000","last_fullsized":"2024-01-15T18:11:34.869471+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:28.015798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853488+0000","last_change":"2024-01-15T18:11:34.860213+0000","last_active":"2024-01-15T18:11:35.853488+0000","last_peered":"2024-01-15T18:11:35.853488+0000","last_clean":"2024-01-15T18:11:35.853488+0000","last_became_active":"2024-01-15T18:11:34.859972+0000","last_became_peered":"2024-01-15T18:11:34.859972+0000","last_unstale":"2024-01-15T18:11:35.853488+0000","last_undegraded":"2024-01-15T18:11:35.853488+0000","last_fullsized":"2024-01-15T18:11:35.853488+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:32:55.862337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.871573+0000","last_change":"2024-01-15T18:11:34.867793+0000","last_active":"2024-01-15T18:11:34.871573+0000","last_peered":"2024-01-15T18:11:34.871573+0000","last_clean":"2024-01-15T18:11:34.871573+0000","last_became_active":"2024-01-15T18:11:34.867508+0000","last_became_peered":"2024-01-15T18:11:34.867508+0000","last_unstale":"2024-01-15T18:11:34.871573+0000","last_undegraded":"2024-01-15T18:11:34.871573+0000","last_fullsized":"2024-01-15T18:11:34.871573+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:17.980905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.882360+0000","last_change":"2024-01-15T18:11:33.866809+0000","last_active":"2024-01-15T18:11:34.882360+0000","last_peered":"2024-01-15T18:11:34.882360+0000","last_clean":"2024-01-15T18:11:34.882360+0000","last_became_active":"2024-01-15T18:11:33.866344+0000","last_became_peered":"2024-01-15T18:11:33.866344+0000","last_unstale":"2024-01-15T18:11:34.882360+0000","last_undegraded":"2024-01-15T18:11:34.882360+0000","last_fullsized":"2024-01-15T18:11:34.882360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:42:59.865740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900257+0000","last_change":"2024-01-15T18:11:34.863089+0000","last_active":"2024-01-15T18:11:35.900257+0000","last_peered":"2024-01-15T18:11:35.900257+0000","last_clean":"2024-01-15T18:11:35.900257+0000","last_became_active":"2024-01-15T18:11:34.862135+0000","last_became_peered":"2024-01-15T18:11:34.862135+0000","last_unstale":"2024-01-15T18:11:35.900257+0000","last_undegraded":"2024-01-15T18:11:35.900257+0000","last_fullsized":"2024-01-15T18:11:35.900257+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:57.946188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927481+0000","last_change":"2024-01-15T18:11:33.859979+0000","last_active":"2024-01-15T18:11:36.927481+0000","last_peered":"2024-01-15T18:11:36.927481+0000","last_clean":"2024-01-15T18:11:36.927481+0000","last_became_active":"2024-01-15T18:11:33.859603+0000","last_became_peered":"2024-01-15T18:11:33.859603+0000","last_unstale":"2024-01-15T18:11:36.927481+0000","last_undegraded":"2024-01-15T18:11:36.927481+0000","last_fullsized":"2024-01-15T18:11:36.927481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:56.966417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855161+0000","last_change":"2024-01-15T18:11:34.869295+0000","last_active":"2024-01-15T18:11:35.855161+0000","last_peered":"2024-01-15T18:11:35.855161+0000","last_clean":"2024-01-15T18:11:35.855161+0000","last_became_active":"2024-01-15T18:11:34.868979+0000","last_became_peered":"2024-01-15T18:11:34.868979+0000","last_unstale":"2024-01-15T18:11:35.855161+0000","last_undegraded":"2024-01-15T18:11:35.855161+0000","last_fullsized":"2024-01-15T18:11:35.855161+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:13:31.715947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854227+0000","last_change":"2024-01-15T18:11:33.862349+0000","last_active":"2024-01-15T18:11:35.854227+0000","last_peered":"2024-01-15T18:11:35.854227+0000","last_clean":"2024-01-15T18:11:35.854227+0000","last_became_active":"2024-01-15T18:11:33.861891+0000","last_became_peered":"2024-01-15T18:11:33.861891+0000","last_unstale":"2024-01-15T18:11:35.854227+0000","last_undegraded":"2024-01-15T18:11:35.854227+0000","last_fullsized":"2024-01-15T18:11:35.854227+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:52.855208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.872100+0000","last_change":"2024-01-15T18:11:34.867858+0000","last_active":"2024-01-15T18:11:34.872100+0000","last_peered":"2024-01-15T18:11:34.872100+0000","last_clean":"2024-01-15T18:11:34.872100+0000","last_became_active":"2024-01-15T18:11:34.867538+0000","last_became_peered":"2024-01-15T18:11:34.867538+0000","last_unstale":"2024-01-15T18:11:34.872100+0000","last_undegraded":"2024-01-15T18:11:34.872100+0000","last_fullsized":"2024-01-15T18:11:34.872100+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:07.317112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855194+0000","last_change":"2024-01-15T18:11:33.859887+0000","last_active":"2024-01-15T18:11:35.855194+0000","last_peered":"2024-01-15T18:11:35.855194+0000","last_clean":"2024-01-15T18:11:35.855194+0000","last_became_active":"2024-01-15T18:11:33.859578+0000","last_became_peered":"2024-01-15T18:11:33.859578+0000","last_unstale":"2024-01-15T18:11:35.855194+0000","last_undegraded":"2024-01-15T18:11:35.855194+0000","last_fullsized":"2024-01-15T18:11:35.855194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:15.254859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.864392+0000","last_change":"2024-01-15T18:11:34.857363+0000","last_active":"2024-01-15T18:11:34.864392+0000","last_peered":"2024-01-15T18:11:34.864392+0000","last_clean":"2024-01-15T18:11:34.864392+0000","last_became_active":"2024-01-15T18:11:34.857077+0000","last_became_peered":"2024-01-15T18:11:34.857077+0000","last_unstale":"2024-01-15T18:11:34.864392+0000","last_undegraded":"2024-01-15T18:11:34.864392+0000","last_fullsized":"2024-01-15T18:11:34.864392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:00.840912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854046+0000","last_change":"2024-01-15T18:11:33.864259+0000","last_active":"2024-01-15T18:11:35.854046+0000","last_peered":"2024-01-15T18:11:35.854046+0000","last_clean":"2024-01-15T18:11:35.854046+0000","last_became_active":"2024-01-15T18:11:33.860183+0000","last_became_peered":"2024-01-15T18:11:33.860183+0000","last_unstale":"2024-01-15T18:11:35.854046+0000","last_undegraded":"2024-01-15T18:11:35.854046+0000","last_fullsized":"2024-01-15T18:11:35.854046+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:03:28.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854085+0000","last_change":"2024-01-15T18:11:34.856771+0000","last_active":"2024-01-15T18:11:35.854085+0000","last_peered":"2024-01-15T18:11:35.854085+0000","last_clean":"2024-01-15T18:11:35.854085+0000","last_became_active":"2024-01-15T18:11:34.856513+0000","last_became_peered":"2024-01-15T18:11:34.856513+0000","last_unstale":"2024-01-15T18:11:35.854085+0000","last_undegraded":"2024-01-15T18:11:35.854085+0000","last_fullsized":"2024-01-15T18:11:35.854085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:04.257797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854511+0000","last_change":"2024-01-15T18:11:33.859097+0000","last_active":"2024-01-15T18:11:35.854511+0000","last_peered":"2024-01-15T18:11:35.854511+0000","last_clean":"2024-01-15T18:11:35.854511+0000","last_became_active":"2024-01-15T18:11:33.858034+0000","last_became_peered":"2024-01-15T18:11:33.858034+0000","last_unstale":"2024-01-15T18:11:35.854511+0000","last_undegraded":"2024-01-15T18:11:35.854511+0000","last_fullsized":"2024-01-15T18:11:35.854511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:49:33.828519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899718+0000","last_change":"2024-01-15T18:11:34.852840+0000","last_active":"2024-01-15T18:11:35.899718+0000","last_peered":"2024-01-15T18:11:35.899718+0000","last_clean":"2024-01-15T18:11:35.899718+0000","last_became_active":"2024-01-15T18:11:34.852535+0000","last_became_peered":"2024-01-15T18:11:34.852535+0000","last_unstale":"2024-01-15T18:11:35.899718+0000","last_undegraded":"2024-01-15T18:11:35.899718+0000","last_fullsized":"2024-01-15T18:11:35.899718+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:19.936244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.841728+0000","last_change":"2024-01-15T18:11:33.861873+0000","last_active":"2024-01-15T18:11:34.841728+0000","last_peered":"2024-01-15T18:11:34.841728+0000","last_clean":"2024-01-15T18:11:34.841728+0000","last_became_active":"2024-01-15T18:11:33.861401+0000","last_became_peered":"2024-01-15T18:11:33.861401+0000","last_unstale":"2024-01-15T18:11:34.841728+0000","last_undegraded":"2024-01-15T18:11:34.841728+0000","last_fullsized":"2024-01-15T18:11:34.841728+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:56.936845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927099+0000","last_change":"2024-01-15T18:11:34.855795+0000","last_active":"2024-01-15T18:11:36.927099+0000","last_peered":"2024-01-15T18:11:36.927099+0000","last_clean":"2024-01-15T18:11:36.927099+0000","last_became_active":"2024-01-15T18:11:34.855472+0000","last_became_peered":"2024-01-15T18:11:34.855472+0000","last_unstale":"2024-01-15T18:11:36.927099+0000","last_undegraded":"2024-01-15T18:11:36.927099+0000","last_fullsized":"2024-01-15T18:11:36.927099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:40.410128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899745+0000","last_change":"2024-01-15T18:11:33.859623+0000","last_active":"2024-01-15T18:11:35.899745+0000","last_peered":"2024-01-15T18:11:35.899745+0000","last_clean":"2024-01-15T18:11:35.899745+0000","last_became_active":"2024-01-15T18:11:33.859343+0000","last_became_peered":"2024-01-15T18:11:33.859343+0000","last_unstale":"2024-01-15T18:11:35.899745+0000","last_undegraded":"2024-01-15T18:11:35.899745+0000","last_fullsized":"2024-01-15T18:11:35.899745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:51:06.976300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.875067+0000","last_change":"2024-01-15T18:11:34.870211+0000","last_active":"2024-01-15T18:11:34.875067+0000","last_peered":"2024-01-15T18:11:34.875067+0000","last_clean":"2024-01-15T18:11:34.875067+0000","last_became_active":"2024-01-15T18:11:34.869812+0000","last_became_peered":"2024-01-15T18:11:34.869812+0000","last_unstale":"2024-01-15T18:11:34.875067+0000","last_undegraded":"2024-01-15T18:11:34.875067+0000","last_fullsized":"2024-01-15T18:11:34.875067+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:07.317112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854860+0000","last_change":"2024-01-15T18:11:33.855424+0000","last_active":"2024-01-15T18:11:35.854860+0000","last_peered":"2024-01-15T18:11:35.854860+0000","last_clean":"2024-01-15T18:11:35.854860+0000","last_became_active":"2024-01-15T18:11:33.855134+0000","last_became_peered":"2024-01-15T18:11:33.855134+0000","last_unstale":"2024-01-15T18:11:35.854860+0000","last_undegraded":"2024-01-15T18:11:35.854860+0000","last_fullsized":"2024-01-15T18:11:35.854860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:53:59.831569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.877108+0000","last_change":"2024-01-15T18:11:34.871992+0000","last_active":"2024-01-15T18:11:34.877108+0000","last_peered":"2024-01-15T18:11:34.877108+0000","last_clean":"2024-01-15T18:11:34.877108+0000","last_became_active":"2024-01-15T18:11:34.871634+0000","last_became_peered":"2024-01-15T18:11:34.871634+0000","last_unstale":"2024-01-15T18:11:34.877108+0000","last_undegraded":"2024-01-15T18:11:34.877108+0000","last_fullsized":"2024-01-15T18:11:34.877108+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:40.410128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855298+0000","last_change":"2024-01-15T18:11:33.861737+0000","last_active":"2024-01-15T18:11:35.855298+0000","last_peered":"2024-01-15T18:11:35.855298+0000","last_clean":"2024-01-15T18:11:35.855298+0000","last_became_active":"2024-01-15T18:11:33.861301+0000","last_became_peered":"2024-01-15T18:11:33.861301+0000","last_unstale":"2024-01-15T18:11:35.855298+0000","last_undegraded":"2024-01-15T18:11:35.855298+0000","last_fullsized":"2024-01-15T18:11:35.855298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:23.479160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855097+0000","last_change":"2024-01-15T18:11:34.854796+0000","last_active":"2024-01-15T18:11:35.855097+0000","last_peered":"2024-01-15T18:11:35.855097+0000","last_clean":"2024-01-15T18:11:35.855097+0000","last_became_active":"2024-01-15T18:11:34.854416+0000","last_became_peered":"2024-01-15T18:11:34.854416+0000","last_unstale":"2024-01-15T18:11:35.855097+0000","last_undegraded":"2024-01-15T18:11:35.855097+0000","last_fullsized":"2024-01-15T18:11:35.855097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:38:26.768659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854158+0000","last_change":"2024-01-15T18:11:33.867024+0000","last_active":"2024-01-15T18:11:35.854158+0000","last_peered":"2024-01-15T18:11:35.854158+0000","last_clean":"2024-01-15T18:11:35.854158+0000","last_became_active":"2024-01-15T18:11:33.864649+0000","last_became_peered":"2024-01-15T18:11:33.864649+0000","last_unstale":"2024-01-15T18:11:35.854158+0000","last_undegraded":"2024-01-15T18:11:35.854158+0000","last_fullsized":"2024-01-15T18:11:35.854158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:40.227932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900018+0000","last_change":"2024-01-15T18:11:34.864115+0000","last_active":"2024-01-15T18:11:35.900018+0000","last_peered":"2024-01-15T18:11:35.900018+0000","last_clean":"2024-01-15T18:11:35.900018+0000","last_became_active":"2024-01-15T18:11:34.863852+0000","last_became_peered":"2024-01-15T18:11:34.863852+0000","last_unstale":"2024-01-15T18:11:35.900018+0000","last_undegraded":"2024-01-15T18:11:35.900018+0000","last_fullsized":"2024-01-15T18:11:35.900018+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:18:10.762967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.883349+0000","last_change":"2024-01-15T18:11:33.863747+0000","last_active":"2024-01-15T18:11:34.883349+0000","last_peered":"2024-01-15T18:11:34.883349+0000","last_clean":"2024-01-15T18:11:34.883349+0000","last_became_active":"2024-01-15T18:11:33.863483+0000","last_became_peered":"2024-01-15T18:11:33.863483+0000","last_unstale":"2024-01-15T18:11:34.883349+0000","last_undegraded":"2024-01-15T18:11:34.883349+0000","last_fullsized":"2024-01-15T18:11:34.883349+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:26:39.343577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.874836+0000","last_change":"2024-01-15T18:11:34.869388+0000","last_active":"2024-01-15T18:11:34.874836+0000","last_peered":"2024-01-15T18:11:34.874836+0000","last_clean":"2024-01-15T18:11:34.874836+0000","last_became_active":"2024-01-15T18:11:34.869120+0000","last_became_peered":"2024-01-15T18:11:34.869120+0000","last_unstale":"2024-01-15T18:11:34.874836+0000","last_undegraded":"2024-01-15T18:11:34.874836+0000","last_fullsized":"2024-01-15T18:11:34.874836+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:36:11.022504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.883267+0000","last_change":"2024-01-15T18:11:33.865339+0000","last_active":"2024-01-15T18:11:34.883267+0000","last_peered":"2024-01-15T18:11:34.883267+0000","last_clean":"2024-01-15T18:11:34.883267+0000","last_became_active":"2024-01-15T18:11:33.865054+0000","last_became_peered":"2024-01-15T18:11:33.865054+0000","last_unstale":"2024-01-15T18:11:34.883267+0000","last_undegraded":"2024-01-15T18:11:34.883267+0000","last_fullsized":"2024-01-15T18:11:34.883267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:22.156698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.867430+0000","last_change":"2024-01-15T18:11:34.863703+0000","last_active":"2024-01-15T18:11:34.867430+0000","last_peered":"2024-01-15T18:11:34.867430+0000","last_clean":"2024-01-15T18:11:34.867430+0000","last_became_active":"2024-01-15T18:11:34.863446+0000","last_became_peered":"2024-01-15T18:11:34.863446+0000","last_unstale":"2024-01-15T18:11:34.867430+0000","last_undegraded":"2024-01-15T18:11:34.867430+0000","last_fullsized":"2024-01-15T18:11:34.867430+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:17.980905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854623+0000","last_change":"2024-01-15T18:11:33.862604+0000","last_active":"2024-01-15T18:11:35.854623+0000","last_peered":"2024-01-15T18:11:35.854623+0000","last_clean":"2024-01-15T18:11:35.854623+0000","last_became_active":"2024-01-15T18:11:33.862222+0000","last_became_peered":"2024-01-15T18:11:33.862222+0000","last_unstale":"2024-01-15T18:11:35.854623+0000","last_undegraded":"2024-01-15T18:11:35.854623+0000","last_fullsized":"2024-01-15T18:11:35.854623+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:49.387944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.926984+0000","last_change":"2024-01-15T18:11:34.866361+0000","last_active":"2024-01-15T18:11:36.926984+0000","last_peered":"2024-01-15T18:11:36.926984+0000","last_clean":"2024-01-15T18:11:36.926984+0000","last_became_active":"2024-01-15T18:11:34.866000+0000","last_became_peered":"2024-01-15T18:11:34.866000+0000","last_unstale":"2024-01-15T18:11:36.926984+0000","last_undegraded":"2024-01-15T18:11:36.926984+0000","last_fullsized":"2024-01-15T18:11:36.926984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.854616+0000","last_change":"2024-01-15T18:11:33.859580+0000","last_active":"2024-01-15T18:11:34.854616+0000","last_peered":"2024-01-15T18:11:34.854616+0000","last_clean":"2024-01-15T18:11:34.854616+0000","last_became_active":"2024-01-15T18:11:33.859297+0000","last_became_peered":"2024-01-15T18:11:33.859297+0000","last_unstale":"2024-01-15T18:11:34.854616+0000","last_undegraded":"2024-01-15T18:11:34.854616+0000","last_fullsized":"2024-01-15T18:11:34.854616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:56.966417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899749+0000","last_change":"2024-01-15T18:11:34.859204+0000","last_active":"2024-01-15T18:11:35.899749+0000","last_peered":"2024-01-15T18:11:35.899749+0000","last_clean":"2024-01-15T18:11:35.899749+0000","last_became_active":"2024-01-15T18:11:34.858931+0000","last_became_peered":"2024-01-15T18:11:34.858931+0000","last_unstale":"2024-01-15T18:11:35.899749+0000","last_undegraded":"2024-01-15T18:11:35.899749+0000","last_fullsized":"2024-01-15T18:11:35.899749+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:00:52.077443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853269+0000","last_change":"2024-01-15T18:11:33.853663+0000","last_active":"2024-01-15T18:11:35.853269+0000","last_peered":"2024-01-15T18:11:35.853269+0000","last_clean":"2024-01-15T18:11:35.853269+0000","last_became_active":"2024-01-15T18:11:33.853355+0000","last_became_peered":"2024-01-15T18:11:33.853355+0000","last_unstale":"2024-01-15T18:11:35.853269+0000","last_undegraded":"2024-01-15T18:11:35.853269+0000","last_fullsized":"2024-01-15T18:11:35.853269+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:01.752782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854648+0000","last_change":"2024-01-15T18:11:34.847934+0000","last_active":"2024-01-15T18:11:35.854648+0000","last_peered":"2024-01-15T18:11:35.854648+0000","last_clean":"2024-01-15T18:11:35.854648+0000","last_became_active":"2024-01-15T18:11:34.847657+0000","last_became_peered":"2024-01-15T18:11:34.847657+0000","last_unstale":"2024-01-15T18:11:35.854648+0000","last_undegraded":"2024-01-15T18:11:35.854648+0000","last_fullsized":"2024-01-15T18:11:35.854648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:57.975760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855416+0000","last_change":"2024-01-15T18:11:33.860513+0000","last_active":"2024-01-15T18:11:35.855416+0000","last_peered":"2024-01-15T18:11:35.855416+0000","last_clean":"2024-01-15T18:11:35.855416+0000","last_became_active":"2024-01-15T18:11:33.860239+0000","last_became_peered":"2024-01-15T18:11:33.860239+0000","last_unstale":"2024-01-15T18:11:35.855416+0000","last_undegraded":"2024-01-15T18:11:35.855416+0000","last_fullsized":"2024-01-15T18:11:35.855416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:22.156698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.879487+0000","last_change":"2024-01-15T18:11:34.872903+0000","last_active":"2024-01-15T18:11:34.879487+0000","last_peered":"2024-01-15T18:11:34.879487+0000","last_clean":"2024-01-15T18:11:34.879487+0000","last_became_active":"2024-01-15T18:11:34.872651+0000","last_became_peered":"2024-01-15T18:11:34.872651+0000","last_unstale":"2024-01-15T18:11:34.879487+0000","last_undegraded":"2024-01-15T18:11:34.879487+0000","last_fullsized":"2024-01-15T18:11:34.879487+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:46.051721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854746+0000","last_change":"2024-01-15T18:11:33.860314+0000","last_active":"2024-01-15T18:11:35.854746+0000","last_peered":"2024-01-15T18:11:35.854746+0000","last_clean":"2024-01-15T18:11:35.854746+0000","last_became_active":"2024-01-15T18:11:33.858740+0000","last_became_peered":"2024-01-15T18:11:33.858740+0000","last_unstale":"2024-01-15T18:11:35.854746+0000","last_undegraded":"2024-01-15T18:11:35.854746+0000","last_fullsized":"2024-01-15T18:11:35.854746+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:59.434148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.857774+0000","last_change":"2024-01-15T18:11:34.852559+0000","last_active":"2024-01-15T18:11:34.857774+0000","last_peered":"2024-01-15T18:11:34.857774+0000","last_clean":"2024-01-15T18:11:34.857774+0000","last_became_active":"2024-01-15T18:11:34.852270+0000","last_became_peered":"2024-01-15T18:11:34.852270+0000","last_unstale":"2024-01-15T18:11:34.857774+0000","last_undegraded":"2024-01-15T18:11:34.857774+0000","last_fullsized":"2024-01-15T18:11:34.857774+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:10.006951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855069+0000","last_change":"2024-01-15T18:11:33.855744+0000","last_active":"2024-01-15T18:11:35.855069+0000","last_peered":"2024-01-15T18:11:35.855069+0000","last_clean":"2024-01-15T18:11:35.855069+0000","last_became_active":"2024-01-15T18:11:33.855184+0000","last_became_peered":"2024-01-15T18:11:33.855184+0000","last_unstale":"2024-01-15T18:11:35.855069+0000","last_undegraded":"2024-01-15T18:11:35.855069+0000","last_fullsized":"2024-01-15T18:11:35.855069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:13.615159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900057+0000","last_change":"2024-01-15T18:11:34.852702+0000","last_active":"2024-01-15T18:11:35.900057+0000","last_peered":"2024-01-15T18:11:35.900057+0000","last_clean":"2024-01-15T18:11:35.900057+0000","last_became_active":"2024-01-15T18:11:34.852441+0000","last_became_peered":"2024-01-15T18:11:34.852441+0000","last_unstale":"2024-01-15T18:11:35.900057+0000","last_undegraded":"2024-01-15T18:11:35.900057+0000","last_fullsized":"2024-01-15T18:11:35.900057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:01.622216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.857256+0000","last_change":"2024-01-15T18:11:33.867004+0000","last_active":"2024-01-15T18:11:34.857256+0000","last_peered":"2024-01-15T18:11:34.857256+0000","last_clean":"2024-01-15T18:11:34.857256+0000","last_became_active":"2024-01-15T18:11:33.866648+0000","last_became_peered":"2024-01-15T18:11:33.866648+0000","last_unstale":"2024-01-15T18:11:34.857256+0000","last_undegraded":"2024-01-15T18:11:34.857256+0000","last_fullsized":"2024-01-15T18:11:34.857256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:56.966417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853959+0000","last_change":"2024-01-15T18:11:34.853351+0000","last_active":"2024-01-15T18:11:35.853959+0000","last_peered":"2024-01-15T18:11:35.853959+0000","last_clean":"2024-01-15T18:11:35.853959+0000","last_became_active":"2024-01-15T18:11:34.853060+0000","last_became_peered":"2024-01-15T18:11:34.853060+0000","last_unstale":"2024-01-15T18:11:35.853959+0000","last_undegraded":"2024-01-15T18:11:35.853959+0000","last_fullsized":"2024-01-15T18:11:35.853959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:45.403716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854722+0000","last_change":"2024-01-15T18:11:33.858310+0000","last_active":"2024-01-15T18:11:35.854722+0000","last_peered":"2024-01-15T18:11:35.854722+0000","last_clean":"2024-01-15T18:11:35.854722+0000","last_became_active":"2024-01-15T18:11:33.857290+0000","last_became_peered":"2024-01-15T18:11:33.857290+0000","last_unstale":"2024-01-15T18:11:35.854722+0000","last_undegraded":"2024-01-15T18:11:35.854722+0000","last_fullsized":"2024-01-15T18:11:35.854722+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:16.142426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853775+0000","last_change":"2024-01-15T18:11:34.866053+0000","last_active":"2024-01-15T18:11:35.853775+0000","last_peered":"2024-01-15T18:11:35.853775+0000","last_clean":"2024-01-15T18:11:35.853775+0000","last_became_active":"2024-01-15T18:11:34.865784+0000","last_became_peered":"2024-01-15T18:11:34.865784+0000","last_unstale":"2024-01-15T18:11:35.853775+0000","last_undegraded":"2024-01-15T18:11:35.853775+0000","last_fullsized":"2024-01-15T18:11:35.853775+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:03.269758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.854764+0000","last_change":"2024-01-15T18:11:33.853075+0000","last_active":"2024-01-15T18:11:34.854764+0000","last_peered":"2024-01-15T18:11:34.854764+0000","last_clean":"2024-01-15T18:11:34.854764+0000","last_became_active":"2024-01-15T18:11:33.852820+0000","last_became_peered":"2024-01-15T18:11:33.852820+0000","last_unstale":"2024-01-15T18:11:34.854764+0000","last_undegraded":"2024-01-15T18:11:34.854764+0000","last_fullsized":"2024-01-15T18:11:34.854764+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:39.425569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853823+0000","last_change":"2024-01-15T18:11:34.868657+0000","last_active":"2024-01-15T18:11:35.853823+0000","last_peered":"2024-01-15T18:11:35.853823+0000","last_clean":"2024-01-15T18:11:35.853823+0000","last_became_active":"2024-01-15T18:11:34.868213+0000","last_became_peered":"2024-01-15T18:11:34.868213+0000","last_unstale":"2024-01-15T18:11:35.853823+0000","last_undegraded":"2024-01-15T18:11:35.853823+0000","last_fullsized":"2024-01-15T18:11:35.853823+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:30.094512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.926854+0000","last_change":"2024-01-15T18:11:33.862597+0000","last_active":"2024-01-15T18:11:36.926854+0000","last_peered":"2024-01-15T18:11:36.926854+0000","last_clean":"2024-01-15T18:11:36.926854+0000","last_became_active":"2024-01-15T18:11:33.862309+0000","last_became_peered":"2024-01-15T18:11:33.862309+0000","last_unstale":"2024-01-15T18:11:36.926854+0000","last_undegraded":"2024-01-15T18:11:36.926854+0000","last_fullsized":"2024-01-15T18:11:36.926854+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:16.971562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927410+0000","last_change":"2024-01-15T18:11:34.863810+0000","last_active":"2024-01-15T18:11:36.927410+0000","last_peered":"2024-01-15T18:11:36.927410+0000","last_clean":"2024-01-15T18:11:36.927410+0000","last_became_active":"2024-01-15T18:11:34.863549+0000","last_became_peered":"2024-01-15T18:11:34.863549+0000","last_unstale":"2024-01-15T18:11:36.927410+0000","last_undegraded":"2024-01-15T18:11:36.927410+0000","last_fullsized":"2024-01-15T18:11:36.927410+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.442608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855416+0000","last_change":"2024-01-15T18:11:33.860865+0000","last_active":"2024-01-15T18:11:35.855416+0000","last_peered":"2024-01-15T18:11:35.855416+0000","last_clean":"2024-01-15T18:11:35.855416+0000","last_became_active":"2024-01-15T18:11:33.859770+0000","last_became_peered":"2024-01-15T18:11:33.859770+0000","last_unstale":"2024-01-15T18:11:35.855416+0000","last_undegraded":"2024-01-15T18:11:35.855416+0000","last_fullsized":"2024-01-15T18:11:35.855416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:09.062510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.867177+0000","last_change":"2024-01-15T18:11:34.864463+0000","last_active":"2024-01-15T18:11:34.867177+0000","last_peered":"2024-01-15T18:11:34.867177+0000","last_clean":"2024-01-15T18:11:34.867177+0000","last_became_active":"2024-01-15T18:11:34.863829+0000","last_became_peered":"2024-01-15T18:11:34.863829+0000","last_unstale":"2024-01-15T18:11:34.867177+0000","last_undegraded":"2024-01-15T18:11:34.867177+0000","last_fullsized":"2024-01-15T18:11:34.867177+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:04.257797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.856421+0000","last_change":"2024-01-15T18:11:33.860012+0000","last_active":"2024-01-15T18:11:34.856421+0000","last_peered":"2024-01-15T18:11:34.856421+0000","last_clean":"2024-01-15T18:11:34.856421+0000","last_became_active":"2024-01-15T18:11:33.859664+0000","last_became_peered":"2024-01-15T18:11:33.859664+0000","last_unstale":"2024-01-15T18:11:34.856421+0000","last_undegraded":"2024-01-15T18:11:34.856421+0000","last_fullsized":"2024-01-15T18:11:34.856421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:44.034331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.871783+0000","last_change":"2024-01-15T18:11:34.867999+0000","last_active":"2024-01-15T18:11:34.871783+0000","last_peered":"2024-01-15T18:11:34.871783+0000","last_clean":"2024-01-15T18:11:34.871783+0000","last_became_active":"2024-01-15T18:11:34.867345+0000","last_became_peered":"2024-01-15T18:11:34.867345+0000","last_unstale":"2024-01-15T18:11:34.871783+0000","last_undegraded":"2024-01-15T18:11:34.871783+0000","last_fullsized":"2024-01-15T18:11:34.871783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:45.043674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900098+0000","last_change":"2024-01-15T18:11:33.860095+0000","last_active":"2024-01-15T18:11:35.900098+0000","last_peered":"2024-01-15T18:11:35.900098+0000","last_clean":"2024-01-15T18:11:35.900098+0000","last_became_active":"2024-01-15T18:11:33.859803+0000","last_became_peered":"2024-01-15T18:11:33.859803+0000","last_unstale":"2024-01-15T18:11:35.900098+0000","last_undegraded":"2024-01-15T18:11:35.900098+0000","last_fullsized":"2024-01-15T18:11:35.900098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:39:18.574104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.878195+0000","last_change":"2024-01-15T18:11:34.872270+0000","last_active":"2024-01-15T18:11:34.878195+0000","last_peered":"2024-01-15T18:11:34.878195+0000","last_clean":"2024-01-15T18:11:34.878195+0000","last_became_active":"2024-01-15T18:11:34.871924+0000","last_became_peered":"2024-01-15T18:11:34.871924+0000","last_unstale":"2024-01-15T18:11:34.878195+0000","last_undegraded":"2024-01-15T18:11:34.878195+0000","last_fullsized":"2024-01-15T18:11:34.878195+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927212+0000","last_change":"2024-01-15T18:11:33.859728+0000","last_active":"2024-01-15T18:11:36.927212+0000","last_peered":"2024-01-15T18:11:36.927212+0000","last_clean":"2024-01-15T18:11:36.927212+0000","last_became_active":"2024-01-15T18:11:33.859089+0000","last_became_peered":"2024-01-15T18:11:33.859089+0000","last_unstale":"2024-01-15T18:11:36.927212+0000","last_undegraded":"2024-01-15T18:11:36.927212+0000","last_fullsized":"2024-01-15T18:11:36.927212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:03.248453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900032+0000","last_change":"2024-01-15T18:11:34.860418+0000","last_active":"2024-01-15T18:11:35.900032+0000","last_peered":"2024-01-15T18:11:35.900032+0000","last_clean":"2024-01-15T18:11:35.900032+0000","last_became_active":"2024-01-15T18:11:34.858959+0000","last_became_peered":"2024-01-15T18:11:34.858959+0000","last_unstale":"2024-01-15T18:11:35.900032+0000","last_undegraded":"2024-01-15T18:11:35.900032+0000","last_fullsized":"2024-01-15T18:11:35.900032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:09.823321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899870+0000","last_change":"2024-01-15T18:11:33.862002+0000","last_active":"2024-01-15T18:11:35.899870+0000","last_peered":"2024-01-15T18:11:35.899870+0000","last_clean":"2024-01-15T18:11:35.899870+0000","last_became_active":"2024-01-15T18:11:33.860976+0000","last_became_peered":"2024-01-15T18:11:33.860976+0000","last_unstale":"2024-01-15T18:11:35.899870+0000","last_undegraded":"2024-01-15T18:11:35.899870+0000","last_fullsized":"2024-01-15T18:11:35.899870+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:14:50.034005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854659+0000","last_change":"2024-01-15T18:11:34.856213+0000","last_active":"2024-01-15T18:11:35.854659+0000","last_peered":"2024-01-15T18:11:35.854659+0000","last_clean":"2024-01-15T18:11:35.854659+0000","last_became_active":"2024-01-15T18:11:34.855971+0000","last_became_peered":"2024-01-15T18:11:34.855971+0000","last_unstale":"2024-01-15T18:11:35.854659+0000","last_undegraded":"2024-01-15T18:11:35.854659+0000","last_fullsized":"2024-01-15T18:11:35.854659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:30.028572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.841780+0000","last_change":"2024-01-15T18:11:33.865894+0000","last_active":"2024-01-15T18:11:34.841780+0000","last_peered":"2024-01-15T18:11:34.841780+0000","last_clean":"2024-01-15T18:11:34.841780+0000","last_became_active":"2024-01-15T18:11:33.865616+0000","last_became_peered":"2024-01-15T18:11:33.865616+0000","last_unstale":"2024-01-15T18:11:34.841780+0000","last_undegraded":"2024-01-15T18:11:34.841780+0000","last_fullsized":"2024-01-15T18:11:34.841780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:59.995378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.859101+0000","last_change":"2024-01-15T18:11:34.856755+0000","last_active":"2024-01-15T18:11:34.859101+0000","last_peered":"2024-01-15T18:11:34.859101+0000","last_clean":"2024-01-15T18:11:34.859101+0000","last_became_active":"2024-01-15T18:11:34.853080+0000","last_became_peered":"2024-01-15T18:11:34.853080+0000","last_unstale":"2024-01-15T18:11:34.859101+0000","last_undegraded":"2024-01-15T18:11:34.859101+0000","last_fullsized":"2024-01-15T18:11:34.859101+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:04:44.408867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853051+0000","last_change":"2024-01-15T18:11:33.853901+0000","last_active":"2024-01-15T18:11:35.853051+0000","last_peered":"2024-01-15T18:11:35.853051+0000","last_clean":"2024-01-15T18:11:35.853051+0000","last_became_active":"2024-01-15T18:11:33.853596+0000","last_became_peered":"2024-01-15T18:11:33.853596+0000","last_unstale":"2024-01-15T18:11:35.853051+0000","last_undegraded":"2024-01-15T18:11:35.853051+0000","last_fullsized":"2024-01-15T18:11:35.853051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:05:58.805708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853242+0000","last_change":"2024-01-15T18:11:34.860207+0000","last_active":"2024-01-15T18:11:35.853242+0000","last_peered":"2024-01-15T18:11:35.853242+0000","last_clean":"2024-01-15T18:11:35.853242+0000","last_became_active":"2024-01-15T18:11:34.859934+0000","last_became_peered":"2024-01-15T18:11:34.859934+0000","last_unstale":"2024-01-15T18:11:35.853242+0000","last_undegraded":"2024-01-15T18:11:35.853242+0000","last_fullsized":"2024-01-15T18:11:35.853242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:09:34.365283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853578+0000","last_change":"2024-01-15T18:11:33.867063+0000","last_active":"2024-01-15T18:11:35.853578+0000","last_peered":"2024-01-15T18:11:35.853578+0000","last_clean":"2024-01-15T18:11:35.853578+0000","last_became_active":"2024-01-15T18:11:33.866447+0000","last_became_peered":"2024-01-15T18:11:33.866447+0000","last_unstale":"2024-01-15T18:11:35.853578+0000","last_undegraded":"2024-01-15T18:11:35.853578+0000","last_fullsized":"2024-01-15T18:11:35.853578+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:59:16.269693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853940+0000","last_change":"2024-01-15T18:11:34.868394+0000","last_active":"2024-01-15T18:11:35.853940+0000","last_peered":"2024-01-15T18:11:35.853940+0000","last_clean":"2024-01-15T18:11:35.853940+0000","last_became_active":"2024-01-15T18:11:34.867881+0000","last_became_peered":"2024-01-15T18:11:34.867881+0000","last_unstale":"2024-01-15T18:11:35.853940+0000","last_undegraded":"2024-01-15T18:11:35.853940+0000","last_fullsized":"2024-01-15T18:11:35.853940+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:04.600320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854638+0000","last_change":"2024-01-15T18:11:33.864317+0000","last_active":"2024-01-15T18:11:35.854638+0000","last_peered":"2024-01-15T18:11:35.854638+0000","last_clean":"2024-01-15T18:11:35.854638+0000","last_became_active":"2024-01-15T18:11:33.864012+0000","last_became_peered":"2024-01-15T18:11:33.864012+0000","last_unstale":"2024-01-15T18:11:35.854638+0000","last_undegraded":"2024-01-15T18:11:35.854638+0000","last_fullsized":"2024-01-15T18:11:35.854638+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:56.936845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:11:33.854260+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:11:33.854260+0000","last_peered":"2024-01-15T18:11:33.854260+0000","last_clean":"2024-01-15T18:11:33.854260+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:11:33.854260+0000","last_undegraded":"2024-01-15T18:11:33.854260+0000","last_fullsized":"2024-01-15T18:11:33.854260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46500,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325340,"statfs":{"total":96636764160,"available":96589148160,"internally_reserved":0,"allocated":1466368,"data_stored":956383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0289999999999999,"5min":1.0089999999999999,"15min":0.77000000000000002},"min":{"1min":0.439,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":15.619,"5min":15.619,"15min":15.619},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":0.90400000000000003,"15min":0.79200000000000004},"min":{"1min":0.371,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.8820000000000001,"5min":13.744,"15min":13.744},"last":0.69999999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5680000000000001,"5min":1.175,"15min":0.89100000000000001},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":2.577,"5min":1.1779999999999999,"15min":0.89600000000000002},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":16.721,"5min":16.721,"15min":16.721},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":4.4169999999999998,"5min":1.504,"15min":0.97899999999999998},"min":{"1min":0.51700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":30.846,"5min":30.846,"15min":30.846},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.3660000000000001,"15min":1.0049999999999999},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":10.098000000000001,"5min":39.695,"15min":39.695},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":3.274,"5min":1.498,"15min":1.0640000000000001},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":18.754999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":5.4630000000000001,"5min":1.788,"15min":1.143},"min":{"1min":0.51800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":42.692,"5min":42.692,"15min":42.692},"last":0.65600000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.629,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.952,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63500000000000001,"15min":0.72499999999999998},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.032,"5min":0.71099999999999997,"15min":0.754},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.75600000000000001,"15min":0.80400000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.747},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":3.3450000000000002,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58412,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313428,"statfs":{"total":96636764160,"available":96576950272,"internally_reserved":0,"allocated":884736,"data_stored":371819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3980000000000001,"5min":1.3799999999999999,"15min":0.878},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":15.257999999999999,"5min":22.754000000000001,"15min":22.754000000000001},"last":0.5},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.2130000000000001,"15min":0.81899999999999995},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":8.8030000000000008,"5min":12.449,"15min":12.449},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.1419999999999999,"15min":0.80200000000000005},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.33900000000000002},"max":{"1min":4.6260000000000003,"5min":14.246,"15min":14.246},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.9730000000000001,"5min":1.1850000000000001,"15min":0.86799999999999999},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":10.535,"5min":10.535,"15min":10.535},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.1949999999999998,"5min":1.72,"15min":1.054},"min":{"1min":0.65300000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":13.239000000000001,"5min":42.825000000000003,"15min":42.825000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.8680000000000001,"15min":1.0620000000000001},"min":{"1min":0.63700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":14.340999999999999,"5min":42.851999999999997,"15min":42.851999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.8079999999999998,"5min":1.601,"15min":0.999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":17.359999999999999,"5min":24.199000000000002,"15min":24.199000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":3.04,"5min":1.427,"15min":0.95099999999999996},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.38300000000000001},"max":{"1min":16.760000000000002,"5min":16.760000000000002,"15min":16.760000000000002},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4370000000000001,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.626},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.444,"5min":12.491,"15min":12.491},"last":0.73099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.46600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":9.4819999999999993,"15min":9.4819999999999993},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.1379999999999999,"15min":0.92300000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.0390000000000001,"5min":12.945,"15min":12.945},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.754,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1899999999999999,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":2.2970000000000002,"5min":1.0800000000000001,"15min":0.88},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":27.651,"5min":27.651,"15min":27.651},"last":0.751}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58400,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313440,"statfs":{"total":96636764160,"available":96576962560,"internally_reserved":0,"allocated":868352,"data_stored":365489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.9470000000000001,"5min":0.92500000000000004,"15min":0.71599999999999997},"min":{"1min":0.502,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":11.519,"5min":11.519,"15min":11.519},"last":0.40400000000000003},{"interface":"front","average":{"1min":2.6600000000000001,"5min":1.131,"15min":0.79100000000000004},"min":{"1min":0.53100000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":12.952,"5min":12.952,"15min":12.952},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":1.024,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":13.358000000000001,"5min":13.358000000000001,"15min":13.358000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":2.71,"5min":1.2170000000000001,"15min":0.86699999999999999},"min":{"1min":0.496,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":21.856999999999999,"5min":21.856999999999999,"15min":21.856999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":0.95199999999999996,"15min":0.77300000000000002},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":6.3239999999999998,"5min":6.3239999999999998,"15min":6.3239999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.774,"5min":0.98099999999999998,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.435,"15min":0.434},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.628}]},{"osd":3,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.5750000000000002,"5min":1.1559999999999999,"15min":0.88400000000000001},"min":{"1min":0.53200000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":13.317,"5min":13.317,"15min":13.317},"last":0.64700000000000002},{"interface":"front","average":{"1min":3.327,"5min":1.29,"15min":0.88},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":19.638000000000002,"5min":19.638000000000002,"15min":19.638000000000002},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.6259999999999999,"5min":11.093,"15min":11.093},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.76500000000000001,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.149,"5min":0.69799999999999995,"15min":0.73399999999999999},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":7.2240000000000002,"5min":7.2240000000000002,"15min":7.2240000000000002},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.77400000000000002},"min":{"1min":0.45100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.319,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.78300000000000003,"15min":0.81399999999999995},"min":{"1min":0.55100000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.751}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45936,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325904,"statfs":{"total":96636764160,"available":96589725696,"internally_reserved":0,"allocated":880640,"data_stored":371797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.21,"15min":0.80900000000000005},"min":{"1min":0.42099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":9.7590000000000003,"5min":34.802,"15min":34.802},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.98299999999999998,"15min":0.73899999999999999},"min":{"1min":0.35399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.98,"5min":22.335999999999999,"15min":22.335999999999999},"last":0.79100000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.365,"15min":0.93500000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.625,"5min":38.198,"15min":38.198},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.3069999999999999,"15min":0.89300000000000002},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.605,"5min":38.220999999999997,"15min":38.220999999999997},"last":1.022}]},{"osd":2,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.0109999999999999,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":7.6989999999999998,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.628},{"interface":"front","average":{"1min":1.2929999999999999,"5min":0.92500000000000004,"15min":0.79900000000000004},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5.2329999999999997,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.82099999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.0009999999999999,"15min":0.78200000000000003},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4720000000000004,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.8819999999999999,"5min":1.1020000000000001,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":10.206,"5min":10.206,"15min":10.206},"last":1}]},{"osd":5,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4359999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.119},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78600000000000003,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.522,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":0.96499999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.78700000000000003,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":7.5439999999999996,"5min":10.284000000000001,"15min":10.284000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.71999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.792,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.93600000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.82399999999999995,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.6870000000000003,"5min":11.457000000000001,"15min":11.457000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.24,"5min":0.85199999999999998,"15min":0.82099999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.67,"5min":11.67,"15min":11.67},"last":1.0409999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":884736,"data_stored":371839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.125,"15min":0.67100000000000004},"min":{"1min":0.27000000000000002,"5min":0.255,"15min":0.255},"max":{"1min":8.1660000000000004,"5min":25.771999999999998,"15min":25.771999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":2.8610000000000002,"5min":1.4259999999999999,"15min":0.76600000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":23.074999999999999,"5min":28.332000000000001,"15min":28.332000000000001},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":3.0840000000000001,"5min":1.5229999999999999,"15min":0.79200000000000004},"min":{"1min":0.32900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":20.321000000000002,"5min":40.457999999999998,"15min":40.457999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":1.589,"5min":1.579,"15min":0.88500000000000001},"min":{"1min":0.374,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.383,"5min":27.427,"15min":27.427},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":3.2869999999999999,"5min":1.417,"15min":0.78500000000000003},"min":{"1min":0.36199999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":23.036999999999999,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":2.02,"5min":1.1819999999999999,"15min":0.746},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.2960000000000003,"5min":12.538,"15min":12.538},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":1.23,"15min":0.89200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":22.998000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":1.03},{"interface":"front","average":{"1min":3.681,"5min":1.4550000000000001,"15min":0.98299999999999998},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.377},"max":{"1min":20.399000000000001,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.751}]},{"osd":5,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.107,"15min":0.871},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.9710000000000001,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":2.9980000000000002,"5min":1.5029999999999999,"15min":1.0149999999999999},"min":{"1min":0.55500000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":22.908999999999999,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":3.2789999999999999,"5min":1.458,"15min":0.98899999999999999},"min":{"1min":0.53700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":20.445,"5min":20.445,"15min":20.445},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.762,"5min":1.1419999999999999,"15min":0.93300000000000005},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":5.9249999999999998,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.73099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":2.871,"5min":1.456,"15min":0.98399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":23.111999999999998,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.96099999999999997},{"interface":"front","average":{"1min":5.6580000000000004,"5min":1.964,"15min":1.1839999999999999},"min":{"1min":0.46400000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":44.817999999999998,"5min":44.817999999999998,"15min":44.817999999999998},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58420,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313420,"statfs":{"total":96636764160,"available":96576942080,"internally_reserved":0,"allocated":888832,"data_stored":373339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.4569999999999999,"5min":1.944,"15min":0.94199999999999995},"min":{"1min":0.309,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":12.695,"5min":29.527999999999999,"15min":29.527999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":3.0739999999999998,"5min":2.3650000000000002,"15min":1.129},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":14.538,"5min":34.122999999999998,"15min":34.122999999999998},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":3.2410000000000001,"5min":3.3769999999999998,"15min":1.4490000000000001},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30199999999999999},"max":{"1min":34.029000000000003,"5min":113.149,"15min":113.149},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.8340000000000001,"5min":1.7450000000000001,"15min":0.90700000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":13.464,"5min":41.302,"15min":41.302},"last":0.32200000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.7200000000000002,"5min":1.9890000000000001,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":19.206,"5min":46.960000000000001,"15min":46.960000000000001},"last":0.379},{"interface":"front","average":{"1min":1.3260000000000001,"5min":2.4830000000000001,"15min":1.177},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":6.7830000000000004,"5min":115.693,"15min":115.693},"last":0.47299999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.8420000000000001,"15min":1.1419999999999999},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.4580000000000002,"5min":24.366,"15min":24.366},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.456,"5min":1.746,"15min":1.0640000000000001},"min":{"1min":0.55200000000000005,"5min":0.443,"15min":0.443},"max":{"1min":6.7460000000000004,"5min":20.489000000000001,"15min":20.489000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.9039999999999999,"5min":1.9470000000000001,"15min":1.125},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":19.282,"5min":29.645,"15min":29.645},"last":0.64400000000000002},{"interface":"front","average":{"1min":2.8969999999999998,"5min":2.0270000000000001,"15min":1.2170000000000001},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.496},"max":{"1min":26.084,"5min":26.084,"15min":26.084},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.786,"15min":1.1220000000000001},"min":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.9199999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":3.2210000000000001,"5min":2.0430000000000001,"15min":1.208},"min":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":19.321000000000002,"5min":29.690000000000001,"15min":29.690000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.621,"15min":1.095},"min":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":6.8600000000000003,"5min":20.446999999999999,"15min":20.446999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":2.8780000000000001,"5min":2.0390000000000001,"15min":1.2569999999999999},"min":{"1min":0.57699999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":19.361999999999998,"5min":29.785,"15min":29.785},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45936,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325904,"statfs":{"total":96636764160,"available":96589725696,"internally_reserved":0,"allocated":884736,"data_stored":367627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3290000000000002,"5min":1.4399999999999999,"15min":0.93999999999999995},"min":{"1min":0.33200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":7.1580000000000004,"5min":39.948,"15min":39.948},"last":0.318},{"interface":"front","average":{"1min":3.4340000000000002,"5min":1.577,"15min":1.0269999999999999},"min":{"1min":0.36799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":35.790999999999997,"5min":35.790999999999997,"15min":35.790999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.484,"15min":0.96899999999999997},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":11.346,"5min":49.229999999999997,"15min":49.229999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":2.4670000000000001,"5min":1.008,"15min":0.83299999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":7.4100000000000001,"5min":10.349,"15min":10.349},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":3.3719999999999999,"5min":1.7669999999999999,"15min":1.095},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":23.521999999999998,"5min":25.427,"15min":25.427},"last":0.36399999999999999},{"interface":"front","average":{"1min":2.8929999999999998,"5min":1.252,"15min":0.99099999999999999},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":10.077999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":3,"5min":1.512,"15min":1.1899999999999999},"min":{"1min":0.58099999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":23.695,"5min":23.695,"15min":23.695},"last":0.59999999999999998},{"interface":"front","average":{"1min":2.5750000000000002,"5min":1.222,"15min":1.05},"min":{"1min":0.68100000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":10.038,"5min":12.997,"15min":12.997},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.9969999999999999,"5min":1.526,"15min":1.111},"min":{"1min":0.61799999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":23.626999999999999,"5min":23.626999999999999,"15min":23.626999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":2.5840000000000001,"5min":1.2509999999999999,"15min":1.1579999999999999},"min":{"1min":0.76500000000000001,"5min":0.496,"15min":0.496},"max":{"1min":9.9610000000000003,"5min":13.054,"15min":13.054},"last":0.66900000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":3.0680000000000001,"5min":1.472,"15min":1.1519999999999999},"min":{"1min":0.65700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":23.760000000000002,"5min":23.760000000000002,"15min":23.760000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":2.323,"5min":1.2450000000000001,"15min":1.109},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":11.643000000000001,"5min":11.712999999999999,"15min":11.712999999999999},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":1.2929999999999999,"15min":1.143},"min":{"1min":0.67800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":11.833,"5min":11.833,"15min":11.833},"last":0.79700000000000004},{"interface":"front","average":{"1min":3.1000000000000001,"5min":1.357,"15min":1.1619999999999999},"min":{"1min":0.71599999999999997,"5min":0.502,"15min":0.502},"max":{"1min":14.606,"5min":14.606,"15min":14.606},"last":0.76100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46492,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325348,"statfs":{"total":96636764160,"available":96589156352,"internally_reserved":0,"allocated":1458176,"data_stored":955767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":3.6899999999999999,"5min":1.76,"15min":1.1739999999999999},"min":{"1min":0.26600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":16.149000000000001,"5min":31.977,"15min":31.977},"last":0.434},{"interface":"front","average":{"1min":2.4239999999999999,"5min":1.512,"15min":0.81599999999999995},"min":{"1min":0.374,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":9.9589999999999996,"5min":36.145000000000003,"15min":36.145000000000003},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":4.2450000000000001,"5min":1.8169999999999999,"15min":0.94199999999999995},"min":{"1min":0.45100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":36.386000000000003,"5min":41.387999999999998,"15min":41.387999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":2.5819999999999999,"5min":1.575,"15min":0.96999999999999997},"min":{"1min":0.38100000000000001,"5min":0.25,"15min":0.25},"max":{"1min":16.556000000000001,"5min":33.219999999999999,"15min":33.219999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.5760000000000001,"5min":1.399,"15min":1.0960000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":9.8740000000000006,"5min":31.943999999999999,"15min":31.943999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":4.4980000000000002,"5min":1.6339999999999999,"15min":0.92000000000000004},"min":{"1min":0.55200000000000005,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":45.128,"5min":45.128,"15min":45.128},"last":0.49199999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.2589999999999999,"15min":1.0429999999999999},"min":{"1min":0.56699999999999995,"5min":0.372,"15min":0.372},"max":{"1min":9.4760000000000009,"5min":32.932000000000002,"15min":32.932000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":2.5329999999999999,"5min":1.5329999999999999,"15min":1.1539999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":14.831,"5min":31.782,"15min":31.782},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":1.5389999999999999,"15min":1.1739999999999999},"min":{"1min":0.57899999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":11.247999999999999,"5min":31.858000000000001,"15min":31.858000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.1930000000000001,"15min":1.046},"min":{"1min":0.61099999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":9.7080000000000002,"5min":33.151000000000003,"15min":33.151000000000003},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.1579999999999999,"15min":0.93600000000000005},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":6.5670000000000002,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.714,"5min":1.22,"15min":1.0669999999999999},"min":{"1min":0.64400000000000002,"5min":0.498,"15min":0.498},"max":{"1min":9.8550000000000004,"5min":33.098999999999997,"15min":33.098999999999997},"last":0.59999999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1339999999999999,"5min":1.2230000000000001,"15min":0.93600000000000005},"min":{"1min":0.65100000000000002,"5min":0.504,"15min":0.501},"max":{"1min":9.266,"5min":9.6910000000000007,"15min":9.6910000000000007},"last":0.61799999999999999},{"interface":"front","average":{"1min":3.0329999999999999,"5min":1.647,"15min":1.2310000000000001},"min":{"1min":0.61899999999999999,"5min":0.504,"15min":0.504},"max":{"1min":10.377000000000001,"5min":31.905999999999999,"15min":31.905999999999999},"last":0.68200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:11:38.936 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:11:39.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.105+0000 7f2322000640 1 Processor -- start 2024-01-15T18:11:39.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.105+0000 7f2322000640 1 -- start start 2024-01-15T18:11:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f2322000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f231c143780 0x7f231c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f2322000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f231c0722c0 0x7f231c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f2322000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231c0ceb50 con 0x7f231c0ce520 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f2322000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c15c110 con 0x7f231c143780 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f2322000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c15b7e0 con 0x7f231c0722c0 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f231c143780 0x7f231c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f231c0ce520 0x7f231c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55692/0 (socket says 172.21.15.143:55692) 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f231c143780 0x7f231c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51740/0 (socket says 172.21.15.143:51740) 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231aa00640 1 -- 172.21.15.143:0/1736851077 learned_addr learned my addr 172.21.15.143:0/1736851077 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231be00640 1 --2- 172.21.15.143:0/1736851077 >> v2:172.21.15.204:3300/0 conn(0x7f231c0722c0 0x7f231c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231b400640 1 -- 172.21.15.143:0/1736851077 >> v2:172.21.15.204:3300/0 conn(0x7f231c0722c0 msgr2=0x7f231c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231b400640 1 --2- 172.21.15.143:0/1736851077 >> v2:172.21.15.204:3300/0 conn(0x7f231c0722c0 0x7f231c15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231b400640 1 -- 172.21.15.143:0/1736851077 >> v1:172.21.15.143:6789/0 conn(0x7f231c0ce520 legacy=0x7f231c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.109+0000 7f231b400640 1 -- 172.21.15.143:0/1736851077 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c15bac0 con 0x7f231c143780 2024-01-15T18:11:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f231b400640 1 --2- 172.21.15.143:0/1736851077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c0cdfe0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f231c070a20 tx=0x7f230c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d790a52f546db78 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f231a000640 1 -- 172.21.15.143:0/1736851077 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0ab600 con 0x7f231c143780 2024-01-15T18:11:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f231a000640 1 -- 172.21.15.143:0/1736851077 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c0a7050 con 0x7f231c143780 2024-01-15T18:11:39.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f231a000640 1 -- 172.21.15.143:0/1736851077 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c0ab980 con 0x7f231c143780 2024-01-15T18:11:39.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 -- 172.21.15.143:0/1736851077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 msgr2=0x7f231c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 --2- 172.21.15.143:0/1736851077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c0cdfe0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f231c070a20 tx=0x7f230c01c460 comp rx=0 tx=0).stop 2024-01-15T18:11:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 -- 172.21.15.143:0/1736851077 shutdown_connections 2024-01-15T18:11:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 --2- 172.21.15.143:0/1736851077 >> v2:172.21.15.204:3300/0 conn(0x7f231c0722c0 0x7f231c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 --2- 172.21.15.143:0/1736851077 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c0cdfe0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 -- 172.21.15.143:0/1736851077 >> 172.21.15.143:0/1736851077 conn(0x7f231c06c3f0 msgr2=0x7f231c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 -- 172.21.15.143:0/1736851077 shutdown_connections 2024-01-15T18:11:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 -- 172.21.15.143:0/1736851077 wait complete. 2024-01-15T18:11:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 Processor -- start 2024-01-15T18:11:39.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.113+0000 7f2322000640 1 -- start start 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f231c0722c0 0x7f231c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c0ce520 0x7f231c13f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c1f5f50 con 0x7f231c143780 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c140410 con 0x7f231c0ce520 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c137bc0 con 0x7f231c0722c0 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c13f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51744/0 (socket says 172.21.15.143:51744) 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 -- 172.21.15.143:0/804360399 learned_addr learned my addr 172.21.15.143:0/804360399 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231aa00640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c0ce520 0x7f231c13f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231b400640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f231c0722c0 0x7f231c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 -- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f231c0722c0 msgr2=0x7f231c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f231c0722c0 0x7f231c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 -- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c0ce520 msgr2=0x7f231c13f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c0ce520 0x7f231c13f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 -- 172.21.15.143:0/804360399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c009420 con 0x7f231c143780 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f231be00640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c13f9f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2314007dc0 tx=0x7f231400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314014420 con 0x7f231c143780 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23140145a0 con 0x7f231c143780 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c132d30 con 0x7f231c143780 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314014950 con 0x7f231c143780 2024-01-15T18:11:39.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c1332f0 con 0x7f231c143780 2024-01-15T18:11:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.117+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c067500 con 0x7f231c143780 2024-01-15T18:11:39.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.121+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f2314014ab0 con 0x7f231c143780 2024-01-15T18:11:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.121+0000 7f2313400640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f22f40671f0 0x7f22f40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.121+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f23140834c0 con 0x7f231c143780 2024-01-15T18:11:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.121+0000 7f231b400640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f22f40671f0 0x7f22f40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.121+0000 7f231b400640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f22f40671f0 0x7f22f40696b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f230c0ad000 tx=0x7f230c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.125+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2314052200 con 0x7f231c143780 2024-01-15T18:11:39.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.273+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f231c0b49c0 con 0x7f22f40671f0 2024-01-15T18:11:39.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.281+0000 7f2313400640 1 -- 172.21.15.143:0/804360399 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287707 (secure 0 0 0) 0x7f231c0b49c0 con 0x7f22f40671f0 2024-01-15T18:11:39.286 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:39.287 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:11:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f22f40671f0 msgr2=0x7f22f40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f22f40671f0 0x7f22f40696b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f230c0ad000 tx=0x7f230c008040 comp rx=0 tx=0).stop 2024-01-15T18:11:39.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 msgr2=0x7f231c13f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c13f9f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2314007dc0 tx=0x7f231400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 shutdown_connections 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f22f40671f0 0x7f22f40696b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f231c143780 0x7f231c13f9f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f231c0ce520 0x7f231c13f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 --2- 172.21.15.143:0/804360399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f231c0722c0 0x7f231c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 >> 172.21.15.143:0/804360399 conn(0x7f231c06c3f0 msgr2=0x7f231c070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 shutdown_connections 2024-01-15T18:11:39.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.285+0000 7f2322000640 1 -- 172.21.15.143:0/804360399 wait complete. 2024-01-15T18:11:39.307 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-01-15T18:11:38.759723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":406036,"kb_used_data":8024,"kb_used_omap":12,"kb_used_meta":397875,"kb_avail":754568684,"statfs":{"total":773094113280,"available":772678332416,"internally_reserved":0,"allocated":8216576,"data_stored":4134060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":407424560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.929730"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.875110+0000","last_change":"2024-01-15T18:11:34.869523+0000","last_active":"2024-01-15T18:11:34.875110+0000","last_peered":"2024-01-15T18:11:34.875110+0000","last_clean":"2024-01-15T18:11:34.875110+0000","last_became_active":"2024-01-15T18:11:34.869232+0000","last_became_peered":"2024-01-15T18:11:34.869232+0000","last_unstale":"2024-01-15T18:11:34.875110+0000","last_undegraded":"2024-01-15T18:11:34.875110+0000","last_fullsized":"2024-01-15T18:11:34.875110+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:04.257797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855222+0000","last_change":"2024-01-15T18:11:34.855541+0000","last_active":"2024-01-15T18:11:35.855222+0000","last_peered":"2024-01-15T18:11:35.855222+0000","last_clean":"2024-01-15T18:11:35.855222+0000","last_became_active":"2024-01-15T18:11:34.855244+0000","last_became_peered":"2024-01-15T18:11:34.855244+0000","last_unstale":"2024-01-15T18:11:35.855222+0000","last_undegraded":"2024-01-15T18:11:35.855222+0000","last_fullsized":"2024-01-15T18:11:35.855222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:46:26.072190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854072+0000","last_change":"2024-01-15T18:11:34.862194+0000","last_active":"2024-01-15T18:11:35.854072+0000","last_peered":"2024-01-15T18:11:35.854072+0000","last_clean":"2024-01-15T18:11:35.854072+0000","last_became_active":"2024-01-15T18:11:34.861933+0000","last_became_peered":"2024-01-15T18:11:34.861933+0000","last_unstale":"2024-01-15T18:11:35.854072+0000","last_undegraded":"2024-01-15T18:11:35.854072+0000","last_fullsized":"2024-01-15T18:11:35.854072+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:07:57.539145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.865725+0000","last_change":"2024-01-15T18:11:34.860373+0000","last_active":"2024-01-15T18:11:34.865725+0000","last_peered":"2024-01-15T18:11:34.865725+0000","last_clean":"2024-01-15T18:11:34.865725+0000","last_became_active":"2024-01-15T18:11:34.859670+0000","last_became_peered":"2024-01-15T18:11:34.859670+0000","last_unstale":"2024-01-15T18:11:34.865725+0000","last_undegraded":"2024-01-15T18:11:34.865725+0000","last_fullsized":"2024-01-15T18:11:34.865725+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:10.071853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899931+0000","last_change":"2024-01-15T18:11:34.854369+0000","last_active":"2024-01-15T18:11:35.899931+0000","last_peered":"2024-01-15T18:11:35.899931+0000","last_clean":"2024-01-15T18:11:35.899931+0000","last_became_active":"2024-01-15T18:11:34.854103+0000","last_became_peered":"2024-01-15T18:11:34.854103+0000","last_unstale":"2024-01-15T18:11:35.899931+0000","last_undegraded":"2024-01-15T18:11:35.899931+0000","last_fullsized":"2024-01-15T18:11:35.899931+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:50:35.517210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854304+0000","last_change":"2024-01-15T18:11:34.855438+0000","last_active":"2024-01-15T18:11:35.854304+0000","last_peered":"2024-01-15T18:11:35.854304+0000","last_clean":"2024-01-15T18:11:35.854304+0000","last_became_active":"2024-01-15T18:11:34.855123+0000","last_became_peered":"2024-01-15T18:11:34.855123+0000","last_unstale":"2024-01-15T18:11:35.854304+0000","last_undegraded":"2024-01-15T18:11:35.854304+0000","last_fullsized":"2024-01-15T18:11:35.854304+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:08.521071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.866641+0000","last_change":"2024-01-15T18:11:34.860862+0000","last_active":"2024-01-15T18:11:34.866641+0000","last_peered":"2024-01-15T18:11:34.866641+0000","last_clean":"2024-01-15T18:11:34.866641+0000","last_became_active":"2024-01-15T18:11:34.860520+0000","last_became_peered":"2024-01-15T18:11:34.860520+0000","last_unstale":"2024-01-15T18:11:34.866641+0000","last_undegraded":"2024-01-15T18:11:34.866641+0000","last_fullsized":"2024-01-15T18:11:34.866641+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:36:11.022504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854289+0000","last_change":"2024-01-15T18:11:34.854799+0000","last_active":"2024-01-15T18:11:35.854289+0000","last_peered":"2024-01-15T18:11:35.854289+0000","last_clean":"2024-01-15T18:11:35.854289+0000","last_became_active":"2024-01-15T18:11:34.854504+0000","last_became_peered":"2024-01-15T18:11:34.854504+0000","last_unstale":"2024-01-15T18:11:35.854289+0000","last_undegraded":"2024-01-15T18:11:35.854289+0000","last_fullsized":"2024-01-15T18:11:35.854289+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:27:03.032629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900191+0000","last_change":"2024-01-15T18:11:34.863452+0000","last_active":"2024-01-15T18:11:35.900191+0000","last_peered":"2024-01-15T18:11:35.900191+0000","last_clean":"2024-01-15T18:11:35.900191+0000","last_became_active":"2024-01-15T18:11:34.862835+0000","last_became_peered":"2024-01-15T18:11:34.862835+0000","last_unstale":"2024-01-15T18:11:35.900191+0000","last_undegraded":"2024-01-15T18:11:35.900191+0000","last_fullsized":"2024-01-15T18:11:35.900191+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:48:31.908622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854847+0000","last_change":"2024-01-15T18:11:34.857855+0000","last_active":"2024-01-15T18:11:35.854847+0000","last_peered":"2024-01-15T18:11:35.854847+0000","last_clean":"2024-01-15T18:11:35.854847+0000","last_became_active":"2024-01-15T18:11:34.857576+0000","last_became_peered":"2024-01-15T18:11:34.857576+0000","last_unstale":"2024-01-15T18:11:35.854847+0000","last_undegraded":"2024-01-15T18:11:35.854847+0000","last_fullsized":"2024-01-15T18:11:35.854847+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.874975+0000","last_change":"2024-01-15T18:11:34.869432+0000","last_active":"2024-01-15T18:11:34.874975+0000","last_peered":"2024-01-15T18:11:34.874975+0000","last_clean":"2024-01-15T18:11:34.874975+0000","last_became_active":"2024-01-15T18:11:34.869044+0000","last_became_peered":"2024-01-15T18:11:34.869044+0000","last_unstale":"2024-01-15T18:11:34.874975+0000","last_undegraded":"2024-01-15T18:11:34.874975+0000","last_fullsized":"2024-01-15T18:11:34.874975+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:28.015798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.857794+0000","last_change":"2024-01-15T18:11:34.854639+0000","last_active":"2024-01-15T18:11:34.857794+0000","last_peered":"2024-01-15T18:11:34.857794+0000","last_clean":"2024-01-15T18:11:34.857794+0000","last_became_active":"2024-01-15T18:11:34.854085+0000","last_became_peered":"2024-01-15T18:11:34.854085+0000","last_unstale":"2024-01-15T18:11:34.857794+0000","last_undegraded":"2024-01-15T18:11:34.857794+0000","last_fullsized":"2024-01-15T18:11:34.857794+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:01.327919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927275+0000","last_change":"2024-01-15T18:11:34.863815+0000","last_active":"2024-01-15T18:11:36.927275+0000","last_peered":"2024-01-15T18:11:36.927275+0000","last_clean":"2024-01-15T18:11:36.927275+0000","last_became_active":"2024-01-15T18:11:34.863476+0000","last_became_peered":"2024-01-15T18:11:34.863476+0000","last_unstale":"2024-01-15T18:11:36.927275+0000","last_undegraded":"2024-01-15T18:11:36.927275+0000","last_fullsized":"2024-01-15T18:11:36.927275+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:46.051721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900054+0000","last_change":"2024-01-15T18:11:34.858233+0000","last_active":"2024-01-15T18:11:35.900054+0000","last_peered":"2024-01-15T18:11:35.900054+0000","last_clean":"2024-01-15T18:11:35.900054+0000","last_became_active":"2024-01-15T18:11:34.857971+0000","last_became_peered":"2024-01-15T18:11:34.857971+0000","last_unstale":"2024-01-15T18:11:35.900054+0000","last_undegraded":"2024-01-15T18:11:35.900054+0000","last_fullsized":"2024-01-15T18:11:35.900054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:04:01.928940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927077+0000","last_change":"2024-01-15T18:11:34.868083+0000","last_active":"2024-01-15T18:11:36.927077+0000","last_peered":"2024-01-15T18:11:36.927077+0000","last_clean":"2024-01-15T18:11:36.927077+0000","last_became_active":"2024-01-15T18:11:34.867799+0000","last_became_peered":"2024-01-15T18:11:34.867799+0000","last_unstale":"2024-01-15T18:11:36.927077+0000","last_undegraded":"2024-01-15T18:11:36.927077+0000","last_fullsized":"2024-01-15T18:11:36.927077+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:45.043674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853173+0000","last_change":"2024-01-15T18:11:34.863389+0000","last_active":"2024-01-15T18:11:35.853173+0000","last_peered":"2024-01-15T18:11:35.853173+0000","last_clean":"2024-01-15T18:11:35.853173+0000","last_became_active":"2024-01-15T18:11:34.863056+0000","last_became_peered":"2024-01-15T18:11:34.863056+0000","last_unstale":"2024-01-15T18:11:35.853173+0000","last_undegraded":"2024-01-15T18:11:35.853173+0000","last_fullsized":"2024-01-15T18:11:35.853173+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:17.980905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900208+0000","last_change":"2024-01-15T18:11:34.863287+0000","last_active":"2024-01-15T18:11:35.900208+0000","last_peered":"2024-01-15T18:11:35.900208+0000","last_clean":"2024-01-15T18:11:35.900208+0000","last_became_active":"2024-01-15T18:11:34.862486+0000","last_became_peered":"2024-01-15T18:11:34.862486+0000","last_unstale":"2024-01-15T18:11:35.900208+0000","last_undegraded":"2024-01-15T18:11:35.900208+0000","last_fullsized":"2024-01-15T18:11:35.900208+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:34.458274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.865272+0000","last_change":"2024-01-15T18:11:34.859323+0000","last_active":"2024-01-15T18:11:34.865272+0000","last_peered":"2024-01-15T18:11:34.865272+0000","last_clean":"2024-01-15T18:11:34.865272+0000","last_became_active":"2024-01-15T18:11:34.858632+0000","last_became_peered":"2024-01-15T18:11:34.858632+0000","last_unstale":"2024-01-15T18:11:34.865272+0000","last_undegraded":"2024-01-15T18:11:34.865272+0000","last_fullsized":"2024-01-15T18:11:34.865272+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:24.488504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855073+0000","last_change":"2024-01-15T18:11:34.855406+0000","last_active":"2024-01-15T18:11:35.855073+0000","last_peered":"2024-01-15T18:11:35.855073+0000","last_clean":"2024-01-15T18:11:35.855073+0000","last_became_active":"2024-01-15T18:11:34.855096+0000","last_became_peered":"2024-01-15T18:11:34.855096+0000","last_unstale":"2024-01-15T18:11:35.855073+0000","last_undegraded":"2024-01-15T18:11:35.855073+0000","last_fullsized":"2024-01-15T18:11:35.855073+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:49.139038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927248+0000","last_change":"2024-01-15T18:11:34.857130+0000","last_active":"2024-01-15T18:11:36.927248+0000","last_peered":"2024-01-15T18:11:36.927248+0000","last_clean":"2024-01-15T18:11:36.927248+0000","last_became_active":"2024-01-15T18:11:34.856869+0000","last_became_peered":"2024-01-15T18:11:34.856869+0000","last_unstale":"2024-01-15T18:11:36.927248+0000","last_undegraded":"2024-01-15T18:11:36.927248+0000","last_fullsized":"2024-01-15T18:11:36.927248+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:43:00.875084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853922+0000","last_change":"2024-01-15T18:11:34.858353+0000","last_active":"2024-01-15T18:11:35.853922+0000","last_peered":"2024-01-15T18:11:35.853922+0000","last_clean":"2024-01-15T18:11:35.853922+0000","last_became_active":"2024-01-15T18:11:34.858047+0000","last_became_peered":"2024-01-15T18:11:34.858047+0000","last_unstale":"2024-01-15T18:11:35.853922+0000","last_undegraded":"2024-01-15T18:11:35.853922+0000","last_fullsized":"2024-01-15T18:11:35.853922+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:36:11.022504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853682+0000","last_change":"2024-01-15T18:11:34.852475+0000","last_active":"2024-01-15T18:11:35.853682+0000","last_peered":"2024-01-15T18:11:35.853682+0000","last_clean":"2024-01-15T18:11:35.853682+0000","last_became_active":"2024-01-15T18:11:34.852211+0000","last_became_peered":"2024-01-15T18:11:34.852211+0000","last_unstale":"2024-01-15T18:11:35.853682+0000","last_undegraded":"2024-01-15T18:11:35.853682+0000","last_fullsized":"2024-01-15T18:11:35.853682+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:28.015798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.868531+0000","last_change":"2024-01-15T18:11:34.865503+0000","last_active":"2024-01-15T18:11:34.868531+0000","last_peered":"2024-01-15T18:11:34.868531+0000","last_clean":"2024-01-15T18:11:34.868531+0000","last_became_active":"2024-01-15T18:11:34.865221+0000","last_became_peered":"2024-01-15T18:11:34.865221+0000","last_unstale":"2024-01-15T18:11:34.868531+0000","last_undegraded":"2024-01-15T18:11:34.868531+0000","last_fullsized":"2024-01-15T18:11:34.868531+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:43:00.443491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.870584+0000","last_change":"2024-01-15T18:11:34.867365+0000","last_active":"2024-01-15T18:11:34.870584+0000","last_peered":"2024-01-15T18:11:34.870584+0000","last_clean":"2024-01-15T18:11:34.870584+0000","last_became_active":"2024-01-15T18:11:34.867073+0000","last_became_peered":"2024-01-15T18:11:34.867073+0000","last_unstale":"2024-01-15T18:11:34.870584+0000","last_undegraded":"2024-01-15T18:11:34.870584+0000","last_fullsized":"2024-01-15T18:11:34.870584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:38.520588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854096+0000","last_change":"2024-01-15T18:11:34.868146+0000","last_active":"2024-01-15T18:11:35.854096+0000","last_peered":"2024-01-15T18:11:35.854096+0000","last_clean":"2024-01-15T18:11:35.854096+0000","last_became_active":"2024-01-15T18:11:34.867692+0000","last_became_peered":"2024-01-15T18:11:34.867692+0000","last_unstale":"2024-01-15T18:11:35.854096+0000","last_undegraded":"2024-01-15T18:11:35.854096+0000","last_fullsized":"2024-01-15T18:11:35.854096+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:38.520588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.868147+0000","last_change":"2024-01-15T18:11:34.865848+0000","last_active":"2024-01-15T18:11:34.868147+0000","last_peered":"2024-01-15T18:11:34.868147+0000","last_clean":"2024-01-15T18:11:34.868147+0000","last_became_active":"2024-01-15T18:11:34.865552+0000","last_became_peered":"2024-01-15T18:11:34.865552+0000","last_unstale":"2024-01-15T18:11:34.868147+0000","last_undegraded":"2024-01-15T18:11:34.868147+0000","last_fullsized":"2024-01-15T18:11:34.868147+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:38.520588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854087+0000","last_change":"2024-01-15T18:11:34.861447+0000","last_active":"2024-01-15T18:11:35.854087+0000","last_peered":"2024-01-15T18:11:35.854087+0000","last_clean":"2024-01-15T18:11:35.854087+0000","last_became_active":"2024-01-15T18:11:34.861170+0000","last_became_peered":"2024-01-15T18:11:34.861170+0000","last_unstale":"2024-01-15T18:11:35.854087+0000","last_undegraded":"2024-01-15T18:11:35.854087+0000","last_fullsized":"2024-01-15T18:11:35.854087+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:18:57.354607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854013+0000","last_change":"2024-01-15T18:11:34.857549+0000","last_active":"2024-01-15T18:11:35.854013+0000","last_peered":"2024-01-15T18:11:35.854013+0000","last_clean":"2024-01-15T18:11:35.854013+0000","last_became_active":"2024-01-15T18:11:34.857249+0000","last_became_peered":"2024-01-15T18:11:34.857249+0000","last_unstale":"2024-01-15T18:11:35.854013+0000","last_undegraded":"2024-01-15T18:11:35.854013+0000","last_fullsized":"2024-01-15T18:11:35.854013+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:07.317112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853955+0000","last_change":"2024-01-15T18:11:34.858775+0000","last_active":"2024-01-15T18:11:35.853955+0000","last_peered":"2024-01-15T18:11:35.853955+0000","last_clean":"2024-01-15T18:11:35.853955+0000","last_became_active":"2024-01-15T18:11:34.858182+0000","last_became_peered":"2024-01-15T18:11:34.858182+0000","last_unstale":"2024-01-15T18:11:35.853955+0000","last_undegraded":"2024-01-15T18:11:35.853955+0000","last_fullsized":"2024-01-15T18:11:35.853955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:16:59.468515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.864636+0000","last_change":"2024-01-15T18:11:34.861558+0000","last_active":"2024-01-15T18:11:34.864636+0000","last_peered":"2024-01-15T18:11:34.864636+0000","last_clean":"2024-01-15T18:11:34.864636+0000","last_became_active":"2024-01-15T18:11:34.861372+0000","last_became_peered":"2024-01-15T18:11:34.861372+0000","last_unstale":"2024-01-15T18:11:34.864636+0000","last_undegraded":"2024-01-15T18:11:34.864636+0000","last_fullsized":"2024-01-15T18:11:34.864636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:43:00.443491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.869471+0000","last_change":"2024-01-15T18:11:34.867594+0000","last_active":"2024-01-15T18:11:34.869471+0000","last_peered":"2024-01-15T18:11:34.869471+0000","last_clean":"2024-01-15T18:11:34.869471+0000","last_became_active":"2024-01-15T18:11:34.866643+0000","last_became_peered":"2024-01-15T18:11:34.866643+0000","last_unstale":"2024-01-15T18:11:34.869471+0000","last_undegraded":"2024-01-15T18:11:34.869471+0000","last_fullsized":"2024-01-15T18:11:34.869471+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:28.015798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853488+0000","last_change":"2024-01-15T18:11:34.860213+0000","last_active":"2024-01-15T18:11:35.853488+0000","last_peered":"2024-01-15T18:11:35.853488+0000","last_clean":"2024-01-15T18:11:35.853488+0000","last_became_active":"2024-01-15T18:11:34.859972+0000","last_became_peered":"2024-01-15T18:11:34.859972+0000","last_unstale":"2024-01-15T18:11:35.853488+0000","last_undegraded":"2024-01-15T18:11:35.853488+0000","last_fullsized":"2024-01-15T18:11:35.853488+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:32:55.862337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.871573+0000","last_change":"2024-01-15T18:11:34.867793+0000","last_active":"2024-01-15T18:11:34.871573+0000","last_peered":"2024-01-15T18:11:34.871573+0000","last_clean":"2024-01-15T18:11:34.871573+0000","last_became_active":"2024-01-15T18:11:34.867508+0000","last_became_peered":"2024-01-15T18:11:34.867508+0000","last_unstale":"2024-01-15T18:11:34.871573+0000","last_undegraded":"2024-01-15T18:11:34.871573+0000","last_fullsized":"2024-01-15T18:11:34.871573+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:17.980905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.882360+0000","last_change":"2024-01-15T18:11:33.866809+0000","last_active":"2024-01-15T18:11:34.882360+0000","last_peered":"2024-01-15T18:11:34.882360+0000","last_clean":"2024-01-15T18:11:34.882360+0000","last_became_active":"2024-01-15T18:11:33.866344+0000","last_became_peered":"2024-01-15T18:11:33.866344+0000","last_unstale":"2024-01-15T18:11:34.882360+0000","last_undegraded":"2024-01-15T18:11:34.882360+0000","last_fullsized":"2024-01-15T18:11:34.882360+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:42:59.865740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900257+0000","last_change":"2024-01-15T18:11:34.863089+0000","last_active":"2024-01-15T18:11:35.900257+0000","last_peered":"2024-01-15T18:11:35.900257+0000","last_clean":"2024-01-15T18:11:35.900257+0000","last_became_active":"2024-01-15T18:11:34.862135+0000","last_became_peered":"2024-01-15T18:11:34.862135+0000","last_unstale":"2024-01-15T18:11:35.900257+0000","last_undegraded":"2024-01-15T18:11:35.900257+0000","last_fullsized":"2024-01-15T18:11:35.900257+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:57.946188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927481+0000","last_change":"2024-01-15T18:11:33.859979+0000","last_active":"2024-01-15T18:11:36.927481+0000","last_peered":"2024-01-15T18:11:36.927481+0000","last_clean":"2024-01-15T18:11:36.927481+0000","last_became_active":"2024-01-15T18:11:33.859603+0000","last_became_peered":"2024-01-15T18:11:33.859603+0000","last_unstale":"2024-01-15T18:11:36.927481+0000","last_undegraded":"2024-01-15T18:11:36.927481+0000","last_fullsized":"2024-01-15T18:11:36.927481+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:56.966417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855161+0000","last_change":"2024-01-15T18:11:34.869295+0000","last_active":"2024-01-15T18:11:35.855161+0000","last_peered":"2024-01-15T18:11:35.855161+0000","last_clean":"2024-01-15T18:11:35.855161+0000","last_became_active":"2024-01-15T18:11:34.868979+0000","last_became_peered":"2024-01-15T18:11:34.868979+0000","last_unstale":"2024-01-15T18:11:35.855161+0000","last_undegraded":"2024-01-15T18:11:35.855161+0000","last_fullsized":"2024-01-15T18:11:35.855161+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:13:31.715947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854227+0000","last_change":"2024-01-15T18:11:33.862349+0000","last_active":"2024-01-15T18:11:35.854227+0000","last_peered":"2024-01-15T18:11:35.854227+0000","last_clean":"2024-01-15T18:11:35.854227+0000","last_became_active":"2024-01-15T18:11:33.861891+0000","last_became_peered":"2024-01-15T18:11:33.861891+0000","last_unstale":"2024-01-15T18:11:35.854227+0000","last_undegraded":"2024-01-15T18:11:35.854227+0000","last_fullsized":"2024-01-15T18:11:35.854227+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:52.855208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.872100+0000","last_change":"2024-01-15T18:11:34.867858+0000","last_active":"2024-01-15T18:11:34.872100+0000","last_peered":"2024-01-15T18:11:34.872100+0000","last_clean":"2024-01-15T18:11:34.872100+0000","last_became_active":"2024-01-15T18:11:34.867538+0000","last_became_peered":"2024-01-15T18:11:34.867538+0000","last_unstale":"2024-01-15T18:11:34.872100+0000","last_undegraded":"2024-01-15T18:11:34.872100+0000","last_fullsized":"2024-01-15T18:11:34.872100+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:07.317112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855194+0000","last_change":"2024-01-15T18:11:33.859887+0000","last_active":"2024-01-15T18:11:35.855194+0000","last_peered":"2024-01-15T18:11:35.855194+0000","last_clean":"2024-01-15T18:11:35.855194+0000","last_became_active":"2024-01-15T18:11:33.859578+0000","last_became_peered":"2024-01-15T18:11:33.859578+0000","last_unstale":"2024-01-15T18:11:35.855194+0000","last_undegraded":"2024-01-15T18:11:35.855194+0000","last_fullsized":"2024-01-15T18:11:35.855194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:15.254859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.864392+0000","last_change":"2024-01-15T18:11:34.857363+0000","last_active":"2024-01-15T18:11:34.864392+0000","last_peered":"2024-01-15T18:11:34.864392+0000","last_clean":"2024-01-15T18:11:34.864392+0000","last_became_active":"2024-01-15T18:11:34.857077+0000","last_became_peered":"2024-01-15T18:11:34.857077+0000","last_unstale":"2024-01-15T18:11:34.864392+0000","last_undegraded":"2024-01-15T18:11:34.864392+0000","last_fullsized":"2024-01-15T18:11:34.864392+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:00.840912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854046+0000","last_change":"2024-01-15T18:11:33.864259+0000","last_active":"2024-01-15T18:11:35.854046+0000","last_peered":"2024-01-15T18:11:35.854046+0000","last_clean":"2024-01-15T18:11:35.854046+0000","last_became_active":"2024-01-15T18:11:33.860183+0000","last_became_peered":"2024-01-15T18:11:33.860183+0000","last_unstale":"2024-01-15T18:11:35.854046+0000","last_undegraded":"2024-01-15T18:11:35.854046+0000","last_fullsized":"2024-01-15T18:11:35.854046+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:03:28.899740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854085+0000","last_change":"2024-01-15T18:11:34.856771+0000","last_active":"2024-01-15T18:11:35.854085+0000","last_peered":"2024-01-15T18:11:35.854085+0000","last_clean":"2024-01-15T18:11:35.854085+0000","last_became_active":"2024-01-15T18:11:34.856513+0000","last_became_peered":"2024-01-15T18:11:34.856513+0000","last_unstale":"2024-01-15T18:11:35.854085+0000","last_undegraded":"2024-01-15T18:11:35.854085+0000","last_fullsized":"2024-01-15T18:11:35.854085+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:04.257797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854511+0000","last_change":"2024-01-15T18:11:33.859097+0000","last_active":"2024-01-15T18:11:35.854511+0000","last_peered":"2024-01-15T18:11:35.854511+0000","last_clean":"2024-01-15T18:11:35.854511+0000","last_became_active":"2024-01-15T18:11:33.858034+0000","last_became_peered":"2024-01-15T18:11:33.858034+0000","last_unstale":"2024-01-15T18:11:35.854511+0000","last_undegraded":"2024-01-15T18:11:35.854511+0000","last_fullsized":"2024-01-15T18:11:35.854511+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:49:33.828519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899718+0000","last_change":"2024-01-15T18:11:34.852840+0000","last_active":"2024-01-15T18:11:35.899718+0000","last_peered":"2024-01-15T18:11:35.899718+0000","last_clean":"2024-01-15T18:11:35.899718+0000","last_became_active":"2024-01-15T18:11:34.852535+0000","last_became_peered":"2024-01-15T18:11:34.852535+0000","last_unstale":"2024-01-15T18:11:35.899718+0000","last_undegraded":"2024-01-15T18:11:35.899718+0000","last_fullsized":"2024-01-15T18:11:35.899718+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:19.936244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.841728+0000","last_change":"2024-01-15T18:11:33.861873+0000","last_active":"2024-01-15T18:11:34.841728+0000","last_peered":"2024-01-15T18:11:34.841728+0000","last_clean":"2024-01-15T18:11:34.841728+0000","last_became_active":"2024-01-15T18:11:33.861401+0000","last_became_peered":"2024-01-15T18:11:33.861401+0000","last_unstale":"2024-01-15T18:11:34.841728+0000","last_undegraded":"2024-01-15T18:11:34.841728+0000","last_fullsized":"2024-01-15T18:11:34.841728+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:56.936845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927099+0000","last_change":"2024-01-15T18:11:34.855795+0000","last_active":"2024-01-15T18:11:36.927099+0000","last_peered":"2024-01-15T18:11:36.927099+0000","last_clean":"2024-01-15T18:11:36.927099+0000","last_became_active":"2024-01-15T18:11:34.855472+0000","last_became_peered":"2024-01-15T18:11:34.855472+0000","last_unstale":"2024-01-15T18:11:36.927099+0000","last_undegraded":"2024-01-15T18:11:36.927099+0000","last_fullsized":"2024-01-15T18:11:36.927099+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:40.410128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899745+0000","last_change":"2024-01-15T18:11:33.859623+0000","last_active":"2024-01-15T18:11:35.899745+0000","last_peered":"2024-01-15T18:11:35.899745+0000","last_clean":"2024-01-15T18:11:35.899745+0000","last_became_active":"2024-01-15T18:11:33.859343+0000","last_became_peered":"2024-01-15T18:11:33.859343+0000","last_unstale":"2024-01-15T18:11:35.899745+0000","last_undegraded":"2024-01-15T18:11:35.899745+0000","last_fullsized":"2024-01-15T18:11:35.899745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:51:06.976300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.875067+0000","last_change":"2024-01-15T18:11:34.870211+0000","last_active":"2024-01-15T18:11:34.875067+0000","last_peered":"2024-01-15T18:11:34.875067+0000","last_clean":"2024-01-15T18:11:34.875067+0000","last_became_active":"2024-01-15T18:11:34.869812+0000","last_became_peered":"2024-01-15T18:11:34.869812+0000","last_unstale":"2024-01-15T18:11:34.875067+0000","last_undegraded":"2024-01-15T18:11:34.875067+0000","last_fullsized":"2024-01-15T18:11:34.875067+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:07.317112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854860+0000","last_change":"2024-01-15T18:11:33.855424+0000","last_active":"2024-01-15T18:11:35.854860+0000","last_peered":"2024-01-15T18:11:35.854860+0000","last_clean":"2024-01-15T18:11:35.854860+0000","last_became_active":"2024-01-15T18:11:33.855134+0000","last_became_peered":"2024-01-15T18:11:33.855134+0000","last_unstale":"2024-01-15T18:11:35.854860+0000","last_undegraded":"2024-01-15T18:11:35.854860+0000","last_fullsized":"2024-01-15T18:11:35.854860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:53:59.831569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.877108+0000","last_change":"2024-01-15T18:11:34.871992+0000","last_active":"2024-01-15T18:11:34.877108+0000","last_peered":"2024-01-15T18:11:34.877108+0000","last_clean":"2024-01-15T18:11:34.877108+0000","last_became_active":"2024-01-15T18:11:34.871634+0000","last_became_peered":"2024-01-15T18:11:34.871634+0000","last_unstale":"2024-01-15T18:11:34.877108+0000","last_undegraded":"2024-01-15T18:11:34.877108+0000","last_fullsized":"2024-01-15T18:11:34.877108+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:40.410128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855298+0000","last_change":"2024-01-15T18:11:33.861737+0000","last_active":"2024-01-15T18:11:35.855298+0000","last_peered":"2024-01-15T18:11:35.855298+0000","last_clean":"2024-01-15T18:11:35.855298+0000","last_became_active":"2024-01-15T18:11:33.861301+0000","last_became_peered":"2024-01-15T18:11:33.861301+0000","last_unstale":"2024-01-15T18:11:35.855298+0000","last_undegraded":"2024-01-15T18:11:35.855298+0000","last_fullsized":"2024-01-15T18:11:35.855298+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:23.479160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855097+0000","last_change":"2024-01-15T18:11:34.854796+0000","last_active":"2024-01-15T18:11:35.855097+0000","last_peered":"2024-01-15T18:11:35.855097+0000","last_clean":"2024-01-15T18:11:35.855097+0000","last_became_active":"2024-01-15T18:11:34.854416+0000","last_became_peered":"2024-01-15T18:11:34.854416+0000","last_unstale":"2024-01-15T18:11:35.855097+0000","last_undegraded":"2024-01-15T18:11:35.855097+0000","last_fullsized":"2024-01-15T18:11:35.855097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:38:26.768659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854158+0000","last_change":"2024-01-15T18:11:33.867024+0000","last_active":"2024-01-15T18:11:35.854158+0000","last_peered":"2024-01-15T18:11:35.854158+0000","last_clean":"2024-01-15T18:11:35.854158+0000","last_became_active":"2024-01-15T18:11:33.864649+0000","last_became_peered":"2024-01-15T18:11:33.864649+0000","last_unstale":"2024-01-15T18:11:35.854158+0000","last_undegraded":"2024-01-15T18:11:35.854158+0000","last_fullsized":"2024-01-15T18:11:35.854158+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:40.227932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900018+0000","last_change":"2024-01-15T18:11:34.864115+0000","last_active":"2024-01-15T18:11:35.900018+0000","last_peered":"2024-01-15T18:11:35.900018+0000","last_clean":"2024-01-15T18:11:35.900018+0000","last_became_active":"2024-01-15T18:11:34.863852+0000","last_became_peered":"2024-01-15T18:11:34.863852+0000","last_unstale":"2024-01-15T18:11:35.900018+0000","last_undegraded":"2024-01-15T18:11:35.900018+0000","last_fullsized":"2024-01-15T18:11:35.900018+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:18:10.762967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":17,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.883349+0000","last_change":"2024-01-15T18:11:33.863747+0000","last_active":"2024-01-15T18:11:34.883349+0000","last_peered":"2024-01-15T18:11:34.883349+0000","last_clean":"2024-01-15T18:11:34.883349+0000","last_became_active":"2024-01-15T18:11:33.863483+0000","last_became_peered":"2024-01-15T18:11:33.863483+0000","last_unstale":"2024-01-15T18:11:34.883349+0000","last_undegraded":"2024-01-15T18:11:34.883349+0000","last_fullsized":"2024-01-15T18:11:34.883349+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:26:39.343577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.874836+0000","last_change":"2024-01-15T18:11:34.869388+0000","last_active":"2024-01-15T18:11:34.874836+0000","last_peered":"2024-01-15T18:11:34.874836+0000","last_clean":"2024-01-15T18:11:34.874836+0000","last_became_active":"2024-01-15T18:11:34.869120+0000","last_became_peered":"2024-01-15T18:11:34.869120+0000","last_unstale":"2024-01-15T18:11:34.874836+0000","last_undegraded":"2024-01-15T18:11:34.874836+0000","last_fullsized":"2024-01-15T18:11:34.874836+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:36:11.022504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.883267+0000","last_change":"2024-01-15T18:11:33.865339+0000","last_active":"2024-01-15T18:11:34.883267+0000","last_peered":"2024-01-15T18:11:34.883267+0000","last_clean":"2024-01-15T18:11:34.883267+0000","last_became_active":"2024-01-15T18:11:33.865054+0000","last_became_peered":"2024-01-15T18:11:33.865054+0000","last_unstale":"2024-01-15T18:11:34.883267+0000","last_undegraded":"2024-01-15T18:11:34.883267+0000","last_fullsized":"2024-01-15T18:11:34.883267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:22.156698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.867430+0000","last_change":"2024-01-15T18:11:34.863703+0000","last_active":"2024-01-15T18:11:34.867430+0000","last_peered":"2024-01-15T18:11:34.867430+0000","last_clean":"2024-01-15T18:11:34.867430+0000","last_became_active":"2024-01-15T18:11:34.863446+0000","last_became_peered":"2024-01-15T18:11:34.863446+0000","last_unstale":"2024-01-15T18:11:34.867430+0000","last_undegraded":"2024-01-15T18:11:34.867430+0000","last_fullsized":"2024-01-15T18:11:34.867430+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:17.980905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854623+0000","last_change":"2024-01-15T18:11:33.862604+0000","last_active":"2024-01-15T18:11:35.854623+0000","last_peered":"2024-01-15T18:11:35.854623+0000","last_clean":"2024-01-15T18:11:35.854623+0000","last_became_active":"2024-01-15T18:11:33.862222+0000","last_became_peered":"2024-01-15T18:11:33.862222+0000","last_unstale":"2024-01-15T18:11:35.854623+0000","last_undegraded":"2024-01-15T18:11:35.854623+0000","last_fullsized":"2024-01-15T18:11:35.854623+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:49.387944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.926984+0000","last_change":"2024-01-15T18:11:34.866361+0000","last_active":"2024-01-15T18:11:36.926984+0000","last_peered":"2024-01-15T18:11:36.926984+0000","last_clean":"2024-01-15T18:11:36.926984+0000","last_became_active":"2024-01-15T18:11:34.866000+0000","last_became_peered":"2024-01-15T18:11:34.866000+0000","last_unstale":"2024-01-15T18:11:36.926984+0000","last_undegraded":"2024-01-15T18:11:36.926984+0000","last_fullsized":"2024-01-15T18:11:36.926984+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.854616+0000","last_change":"2024-01-15T18:11:33.859580+0000","last_active":"2024-01-15T18:11:34.854616+0000","last_peered":"2024-01-15T18:11:34.854616+0000","last_clean":"2024-01-15T18:11:34.854616+0000","last_became_active":"2024-01-15T18:11:33.859297+0000","last_became_peered":"2024-01-15T18:11:33.859297+0000","last_unstale":"2024-01-15T18:11:34.854616+0000","last_undegraded":"2024-01-15T18:11:34.854616+0000","last_fullsized":"2024-01-15T18:11:34.854616+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:56.966417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899749+0000","last_change":"2024-01-15T18:11:34.859204+0000","last_active":"2024-01-15T18:11:35.899749+0000","last_peered":"2024-01-15T18:11:35.899749+0000","last_clean":"2024-01-15T18:11:35.899749+0000","last_became_active":"2024-01-15T18:11:34.858931+0000","last_became_peered":"2024-01-15T18:11:34.858931+0000","last_unstale":"2024-01-15T18:11:35.899749+0000","last_undegraded":"2024-01-15T18:11:35.899749+0000","last_fullsized":"2024-01-15T18:11:35.899749+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:00:52.077443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":20,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853269+0000","last_change":"2024-01-15T18:11:33.853663+0000","last_active":"2024-01-15T18:11:35.853269+0000","last_peered":"2024-01-15T18:11:35.853269+0000","last_clean":"2024-01-15T18:11:35.853269+0000","last_became_active":"2024-01-15T18:11:33.853355+0000","last_became_peered":"2024-01-15T18:11:33.853355+0000","last_unstale":"2024-01-15T18:11:35.853269+0000","last_undegraded":"2024-01-15T18:11:35.853269+0000","last_fullsized":"2024-01-15T18:11:35.853269+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:01.752782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854648+0000","last_change":"2024-01-15T18:11:34.847934+0000","last_active":"2024-01-15T18:11:35.854648+0000","last_peered":"2024-01-15T18:11:35.854648+0000","last_clean":"2024-01-15T18:11:35.854648+0000","last_became_active":"2024-01-15T18:11:34.847657+0000","last_became_peered":"2024-01-15T18:11:34.847657+0000","last_unstale":"2024-01-15T18:11:35.854648+0000","last_undegraded":"2024-01-15T18:11:35.854648+0000","last_fullsized":"2024-01-15T18:11:35.854648+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:57.975760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855416+0000","last_change":"2024-01-15T18:11:33.860513+0000","last_active":"2024-01-15T18:11:35.855416+0000","last_peered":"2024-01-15T18:11:35.855416+0000","last_clean":"2024-01-15T18:11:35.855416+0000","last_became_active":"2024-01-15T18:11:33.860239+0000","last_became_peered":"2024-01-15T18:11:33.860239+0000","last_unstale":"2024-01-15T18:11:35.855416+0000","last_undegraded":"2024-01-15T18:11:35.855416+0000","last_fullsized":"2024-01-15T18:11:35.855416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:22.156698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.879487+0000","last_change":"2024-01-15T18:11:34.872903+0000","last_active":"2024-01-15T18:11:34.879487+0000","last_peered":"2024-01-15T18:11:34.879487+0000","last_clean":"2024-01-15T18:11:34.879487+0000","last_became_active":"2024-01-15T18:11:34.872651+0000","last_became_peered":"2024-01-15T18:11:34.872651+0000","last_unstale":"2024-01-15T18:11:34.879487+0000","last_undegraded":"2024-01-15T18:11:34.879487+0000","last_fullsized":"2024-01-15T18:11:34.879487+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:46.051721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854746+0000","last_change":"2024-01-15T18:11:33.860314+0000","last_active":"2024-01-15T18:11:35.854746+0000","last_peered":"2024-01-15T18:11:35.854746+0000","last_clean":"2024-01-15T18:11:35.854746+0000","last_became_active":"2024-01-15T18:11:33.858740+0000","last_became_peered":"2024-01-15T18:11:33.858740+0000","last_unstale":"2024-01-15T18:11:35.854746+0000","last_undegraded":"2024-01-15T18:11:35.854746+0000","last_fullsized":"2024-01-15T18:11:35.854746+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:59.434148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.857774+0000","last_change":"2024-01-15T18:11:34.852559+0000","last_active":"2024-01-15T18:11:34.857774+0000","last_peered":"2024-01-15T18:11:34.857774+0000","last_clean":"2024-01-15T18:11:34.857774+0000","last_became_active":"2024-01-15T18:11:34.852270+0000","last_became_peered":"2024-01-15T18:11:34.852270+0000","last_unstale":"2024-01-15T18:11:34.857774+0000","last_undegraded":"2024-01-15T18:11:34.857774+0000","last_fullsized":"2024-01-15T18:11:34.857774+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:10.006951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855069+0000","last_change":"2024-01-15T18:11:33.855744+0000","last_active":"2024-01-15T18:11:35.855069+0000","last_peered":"2024-01-15T18:11:35.855069+0000","last_clean":"2024-01-15T18:11:35.855069+0000","last_became_active":"2024-01-15T18:11:33.855184+0000","last_became_peered":"2024-01-15T18:11:33.855184+0000","last_unstale":"2024-01-15T18:11:35.855069+0000","last_undegraded":"2024-01-15T18:11:35.855069+0000","last_fullsized":"2024-01-15T18:11:35.855069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:13.615159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900057+0000","last_change":"2024-01-15T18:11:34.852702+0000","last_active":"2024-01-15T18:11:35.900057+0000","last_peered":"2024-01-15T18:11:35.900057+0000","last_clean":"2024-01-15T18:11:35.900057+0000","last_became_active":"2024-01-15T18:11:34.852441+0000","last_became_peered":"2024-01-15T18:11:34.852441+0000","last_unstale":"2024-01-15T18:11:35.900057+0000","last_undegraded":"2024-01-15T18:11:35.900057+0000","last_fullsized":"2024-01-15T18:11:35.900057+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:01.622216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.857256+0000","last_change":"2024-01-15T18:11:33.867004+0000","last_active":"2024-01-15T18:11:34.857256+0000","last_peered":"2024-01-15T18:11:34.857256+0000","last_clean":"2024-01-15T18:11:34.857256+0000","last_became_active":"2024-01-15T18:11:33.866648+0000","last_became_peered":"2024-01-15T18:11:33.866648+0000","last_unstale":"2024-01-15T18:11:34.857256+0000","last_undegraded":"2024-01-15T18:11:34.857256+0000","last_fullsized":"2024-01-15T18:11:34.857256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:56.966417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853959+0000","last_change":"2024-01-15T18:11:34.853351+0000","last_active":"2024-01-15T18:11:35.853959+0000","last_peered":"2024-01-15T18:11:35.853959+0000","last_clean":"2024-01-15T18:11:35.853959+0000","last_became_active":"2024-01-15T18:11:34.853060+0000","last_became_peered":"2024-01-15T18:11:34.853060+0000","last_unstale":"2024-01-15T18:11:35.853959+0000","last_undegraded":"2024-01-15T18:11:35.853959+0000","last_fullsized":"2024-01-15T18:11:35.853959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:45.403716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854722+0000","last_change":"2024-01-15T18:11:33.858310+0000","last_active":"2024-01-15T18:11:35.854722+0000","last_peered":"2024-01-15T18:11:35.854722+0000","last_clean":"2024-01-15T18:11:35.854722+0000","last_became_active":"2024-01-15T18:11:33.857290+0000","last_became_peered":"2024-01-15T18:11:33.857290+0000","last_unstale":"2024-01-15T18:11:35.854722+0000","last_undegraded":"2024-01-15T18:11:35.854722+0000","last_fullsized":"2024-01-15T18:11:35.854722+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:16.142426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853775+0000","last_change":"2024-01-15T18:11:34.866053+0000","last_active":"2024-01-15T18:11:35.853775+0000","last_peered":"2024-01-15T18:11:35.853775+0000","last_clean":"2024-01-15T18:11:35.853775+0000","last_became_active":"2024-01-15T18:11:34.865784+0000","last_became_peered":"2024-01-15T18:11:34.865784+0000","last_unstale":"2024-01-15T18:11:35.853775+0000","last_undegraded":"2024-01-15T18:11:35.853775+0000","last_fullsized":"2024-01-15T18:11:35.853775+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:03.269758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.854764+0000","last_change":"2024-01-15T18:11:33.853075+0000","last_active":"2024-01-15T18:11:34.854764+0000","last_peered":"2024-01-15T18:11:34.854764+0000","last_clean":"2024-01-15T18:11:34.854764+0000","last_became_active":"2024-01-15T18:11:33.852820+0000","last_became_peered":"2024-01-15T18:11:33.852820+0000","last_unstale":"2024-01-15T18:11:34.854764+0000","last_undegraded":"2024-01-15T18:11:34.854764+0000","last_fullsized":"2024-01-15T18:11:34.854764+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:39.425569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853823+0000","last_change":"2024-01-15T18:11:34.868657+0000","last_active":"2024-01-15T18:11:35.853823+0000","last_peered":"2024-01-15T18:11:35.853823+0000","last_clean":"2024-01-15T18:11:35.853823+0000","last_became_active":"2024-01-15T18:11:34.868213+0000","last_became_peered":"2024-01-15T18:11:34.868213+0000","last_unstale":"2024-01-15T18:11:35.853823+0000","last_undegraded":"2024-01-15T18:11:35.853823+0000","last_fullsized":"2024-01-15T18:11:35.853823+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:30.094512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.926854+0000","last_change":"2024-01-15T18:11:33.862597+0000","last_active":"2024-01-15T18:11:36.926854+0000","last_peered":"2024-01-15T18:11:36.926854+0000","last_clean":"2024-01-15T18:11:36.926854+0000","last_became_active":"2024-01-15T18:11:33.862309+0000","last_became_peered":"2024-01-15T18:11:33.862309+0000","last_unstale":"2024-01-15T18:11:36.926854+0000","last_undegraded":"2024-01-15T18:11:36.926854+0000","last_fullsized":"2024-01-15T18:11:36.926854+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:16.971562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927410+0000","last_change":"2024-01-15T18:11:34.863810+0000","last_active":"2024-01-15T18:11:36.927410+0000","last_peered":"2024-01-15T18:11:36.927410+0000","last_clean":"2024-01-15T18:11:36.927410+0000","last_became_active":"2024-01-15T18:11:34.863549+0000","last_became_peered":"2024-01-15T18:11:34.863549+0000","last_unstale":"2024-01-15T18:11:36.927410+0000","last_undegraded":"2024-01-15T18:11:36.927410+0000","last_fullsized":"2024-01-15T18:11:36.927410+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.442608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.855416+0000","last_change":"2024-01-15T18:11:33.860865+0000","last_active":"2024-01-15T18:11:35.855416+0000","last_peered":"2024-01-15T18:11:35.855416+0000","last_clean":"2024-01-15T18:11:35.855416+0000","last_became_active":"2024-01-15T18:11:33.859770+0000","last_became_peered":"2024-01-15T18:11:33.859770+0000","last_unstale":"2024-01-15T18:11:35.855416+0000","last_undegraded":"2024-01-15T18:11:35.855416+0000","last_fullsized":"2024-01-15T18:11:35.855416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:09.062510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.867177+0000","last_change":"2024-01-15T18:11:34.864463+0000","last_active":"2024-01-15T18:11:34.867177+0000","last_peered":"2024-01-15T18:11:34.867177+0000","last_clean":"2024-01-15T18:11:34.867177+0000","last_became_active":"2024-01-15T18:11:34.863829+0000","last_became_peered":"2024-01-15T18:11:34.863829+0000","last_unstale":"2024-01-15T18:11:34.867177+0000","last_undegraded":"2024-01-15T18:11:34.867177+0000","last_fullsized":"2024-01-15T18:11:34.867177+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:04.257797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.856421+0000","last_change":"2024-01-15T18:11:33.860012+0000","last_active":"2024-01-15T18:11:34.856421+0000","last_peered":"2024-01-15T18:11:34.856421+0000","last_clean":"2024-01-15T18:11:34.856421+0000","last_became_active":"2024-01-15T18:11:33.859664+0000","last_became_peered":"2024-01-15T18:11:33.859664+0000","last_unstale":"2024-01-15T18:11:34.856421+0000","last_undegraded":"2024-01-15T18:11:34.856421+0000","last_fullsized":"2024-01-15T18:11:34.856421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:44.034331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.871783+0000","last_change":"2024-01-15T18:11:34.867999+0000","last_active":"2024-01-15T18:11:34.871783+0000","last_peered":"2024-01-15T18:11:34.871783+0000","last_clean":"2024-01-15T18:11:34.871783+0000","last_became_active":"2024-01-15T18:11:34.867345+0000","last_became_peered":"2024-01-15T18:11:34.867345+0000","last_unstale":"2024-01-15T18:11:34.871783+0000","last_undegraded":"2024-01-15T18:11:34.871783+0000","last_fullsized":"2024-01-15T18:11:34.871783+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:45.043674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900098+0000","last_change":"2024-01-15T18:11:33.860095+0000","last_active":"2024-01-15T18:11:35.900098+0000","last_peered":"2024-01-15T18:11:35.900098+0000","last_clean":"2024-01-15T18:11:35.900098+0000","last_became_active":"2024-01-15T18:11:33.859803+0000","last_became_peered":"2024-01-15T18:11:33.859803+0000","last_unstale":"2024-01-15T18:11:35.900098+0000","last_undegraded":"2024-01-15T18:11:35.900098+0000","last_fullsized":"2024-01-15T18:11:35.900098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:39:18.574104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.878195+0000","last_change":"2024-01-15T18:11:34.872270+0000","last_active":"2024-01-15T18:11:34.878195+0000","last_peered":"2024-01-15T18:11:34.878195+0000","last_clean":"2024-01-15T18:11:34.878195+0000","last_became_active":"2024-01-15T18:11:34.871924+0000","last_became_peered":"2024-01-15T18:11:34.871924+0000","last_unstale":"2024-01-15T18:11:34.878195+0000","last_undegraded":"2024-01-15T18:11:34.878195+0000","last_fullsized":"2024-01-15T18:11:34.878195+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.434912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:36.927212+0000","last_change":"2024-01-15T18:11:33.859728+0000","last_active":"2024-01-15T18:11:36.927212+0000","last_peered":"2024-01-15T18:11:36.927212+0000","last_clean":"2024-01-15T18:11:36.927212+0000","last_became_active":"2024-01-15T18:11:33.859089+0000","last_became_peered":"2024-01-15T18:11:33.859089+0000","last_unstale":"2024-01-15T18:11:36.927212+0000","last_undegraded":"2024-01-15T18:11:36.927212+0000","last_fullsized":"2024-01-15T18:11:36.927212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:03.248453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.900032+0000","last_change":"2024-01-15T18:11:34.860418+0000","last_active":"2024-01-15T18:11:35.900032+0000","last_peered":"2024-01-15T18:11:35.900032+0000","last_clean":"2024-01-15T18:11:35.900032+0000","last_became_active":"2024-01-15T18:11:34.858959+0000","last_became_peered":"2024-01-15T18:11:34.858959+0000","last_unstale":"2024-01-15T18:11:35.900032+0000","last_undegraded":"2024-01-15T18:11:35.900032+0000","last_fullsized":"2024-01-15T18:11:35.900032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:09.823321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.899870+0000","last_change":"2024-01-15T18:11:33.862002+0000","last_active":"2024-01-15T18:11:35.899870+0000","last_peered":"2024-01-15T18:11:35.899870+0000","last_clean":"2024-01-15T18:11:35.899870+0000","last_became_active":"2024-01-15T18:11:33.860976+0000","last_became_peered":"2024-01-15T18:11:33.860976+0000","last_unstale":"2024-01-15T18:11:35.899870+0000","last_undegraded":"2024-01-15T18:11:35.899870+0000","last_fullsized":"2024-01-15T18:11:35.899870+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:14:50.034005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854659+0000","last_change":"2024-01-15T18:11:34.856213+0000","last_active":"2024-01-15T18:11:35.854659+0000","last_peered":"2024-01-15T18:11:35.854659+0000","last_clean":"2024-01-15T18:11:35.854659+0000","last_became_active":"2024-01-15T18:11:34.855971+0000","last_became_peered":"2024-01-15T18:11:34.855971+0000","last_unstale":"2024-01-15T18:11:35.854659+0000","last_undegraded":"2024-01-15T18:11:35.854659+0000","last_fullsized":"2024-01-15T18:11:35.854659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:30.028572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.841780+0000","last_change":"2024-01-15T18:11:33.865894+0000","last_active":"2024-01-15T18:11:34.841780+0000","last_peered":"2024-01-15T18:11:34.841780+0000","last_clean":"2024-01-15T18:11:34.841780+0000","last_became_active":"2024-01-15T18:11:33.865616+0000","last_became_peered":"2024-01-15T18:11:33.865616+0000","last_unstale":"2024-01-15T18:11:34.841780+0000","last_undegraded":"2024-01-15T18:11:34.841780+0000","last_fullsized":"2024-01-15T18:11:34.841780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:59.995378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:11:34.859101+0000","last_change":"2024-01-15T18:11:34.856755+0000","last_active":"2024-01-15T18:11:34.859101+0000","last_peered":"2024-01-15T18:11:34.859101+0000","last_clean":"2024-01-15T18:11:34.859101+0000","last_became_active":"2024-01-15T18:11:34.853080+0000","last_became_peered":"2024-01-15T18:11:34.853080+0000","last_unstale":"2024-01-15T18:11:34.859101+0000","last_undegraded":"2024-01-15T18:11:34.859101+0000","last_fullsized":"2024-01-15T18:11:34.859101+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:04:44.408867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853051+0000","last_change":"2024-01-15T18:11:33.853901+0000","last_active":"2024-01-15T18:11:35.853051+0000","last_peered":"2024-01-15T18:11:35.853051+0000","last_clean":"2024-01-15T18:11:35.853051+0000","last_became_active":"2024-01-15T18:11:33.853596+0000","last_became_peered":"2024-01-15T18:11:33.853596+0000","last_unstale":"2024-01-15T18:11:35.853051+0000","last_undegraded":"2024-01-15T18:11:35.853051+0000","last_fullsized":"2024-01-15T18:11:35.853051+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:05:58.805708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853242+0000","last_change":"2024-01-15T18:11:34.860207+0000","last_active":"2024-01-15T18:11:35.853242+0000","last_peered":"2024-01-15T18:11:35.853242+0000","last_clean":"2024-01-15T18:11:35.853242+0000","last_became_active":"2024-01-15T18:11:34.859934+0000","last_became_peered":"2024-01-15T18:11:34.859934+0000","last_unstale":"2024-01-15T18:11:35.853242+0000","last_undegraded":"2024-01-15T18:11:35.853242+0000","last_fullsized":"2024-01-15T18:11:35.853242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:09:34.365283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853578+0000","last_change":"2024-01-15T18:11:33.867063+0000","last_active":"2024-01-15T18:11:35.853578+0000","last_peered":"2024-01-15T18:11:35.853578+0000","last_clean":"2024-01-15T18:11:35.853578+0000","last_became_active":"2024-01-15T18:11:33.866447+0000","last_became_peered":"2024-01-15T18:11:33.866447+0000","last_unstale":"2024-01-15T18:11:35.853578+0000","last_undegraded":"2024-01-15T18:11:35.853578+0000","last_fullsized":"2024-01-15T18:11:35.853578+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:59:16.269693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.853940+0000","last_change":"2024-01-15T18:11:34.868394+0000","last_active":"2024-01-15T18:11:35.853940+0000","last_peered":"2024-01-15T18:11:35.853940+0000","last_clean":"2024-01-15T18:11:35.853940+0000","last_became_active":"2024-01-15T18:11:34.867881+0000","last_became_peered":"2024-01-15T18:11:34.867881+0000","last_unstale":"2024-01-15T18:11:35.853940+0000","last_undegraded":"2024-01-15T18:11:35.853940+0000","last_fullsized":"2024-01-15T18:11:35.853940+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:33.829835+0000","last_clean_scrub_stamp":"2024-01-15T18:11:33.829835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:04.600320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:11:35.854638+0000","last_change":"2024-01-15T18:11:33.864317+0000","last_active":"2024-01-15T18:11:35.854638+0000","last_peered":"2024-01-15T18:11:35.854638+0000","last_clean":"2024-01-15T18:11:35.854638+0000","last_became_active":"2024-01-15T18:11:33.864012+0000","last_became_peered":"2024-01-15T18:11:33.864012+0000","last_unstale":"2024-01-15T18:11:35.854638+0000","last_undegraded":"2024-01-15T18:11:35.854638+0000","last_fullsized":"2024-01-15T18:11:35.854638+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:11:32.820491+0000","last_clean_scrub_stamp":"2024-01-15T18:11:32.820491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:01:56.936845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":207,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:11:33.854260+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:11:33.854260+0000","last_peered":"2024-01-15T18:11:33.854260+0000","last_clean":"2024-01-15T18:11:33.854260+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:11:33.854260+0000","last_undegraded":"2024-01-15T18:11:33.854260+0000","last_fullsized":"2024-01-15T18:11:33.854260+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46500,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325340,"statfs":{"total":96636764160,"available":96589148160,"internally_reserved":0,"allocated":1466368,"data_stored":956383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0289999999999999,"5min":1.0089999999999999,"15min":0.77000000000000002},"min":{"1min":0.439,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":15.619,"5min":15.619,"15min":15.619},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":0.90400000000000003,"15min":0.79200000000000004},"min":{"1min":0.371,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":3.8820000000000001,"5min":13.744,"15min":13.744},"last":0.69999999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5680000000000001,"5min":1.175,"15min":0.89100000000000001},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":16.356000000000002,"5min":16.356000000000002,"15min":16.356000000000002},"last":0.76500000000000001},{"interface":"front","average":{"1min":2.577,"5min":1.1779999999999999,"15min":0.89600000000000002},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":16.721,"5min":16.721,"15min":16.721},"last":0.67500000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":4.4169999999999998,"5min":1.504,"15min":0.97899999999999998},"min":{"1min":0.51700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":30.846,"5min":30.846,"15min":30.846},"last":0.73099999999999998},{"interface":"front","average":{"1min":1.8879999999999999,"5min":1.3660000000000001,"15min":1.0049999999999999},"min":{"1min":0.504,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":10.098000000000001,"5min":39.695,"15min":39.695},"last":0.59499999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":3.274,"5min":1.498,"15min":1.0640000000000001},"min":{"1min":0.50800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":18.754999999999999,"5min":26.463999999999999,"15min":26.463999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":5.4630000000000001,"5min":1.788,"15min":1.143},"min":{"1min":0.51800000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":42.692,"5min":42.692,"15min":42.692},"last":0.65600000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.629,"15min":0.66100000000000003},"min":{"1min":0.39600000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.36699999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.952,"5min":4.3710000000000004,"15min":4.3710000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.63500000000000001,"15min":0.72499999999999998},"min":{"1min":0.374,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.032,"5min":0.71099999999999997,"15min":0.754},"min":{"1min":0.44600000000000001,"5min":0.44,"15min":0.44},"max":{"1min":4.6269999999999998,"5min":4.6269999999999998,"15min":4.6269999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:11:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.75600000000000001,"15min":0.80400000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":3.3079999999999998,"5min":3.3079999999999998,"15min":3.3079999999999998},"last":0.747},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.439,"5min":0.376,"15min":0.376},"max":{"1min":3.3450000000000002,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.61499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58412,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313428,"statfs":{"total":96636764160,"available":96576950272,"internally_reserved":0,"allocated":884736,"data_stored":371819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3980000000000001,"5min":1.3799999999999999,"15min":0.878},"min":{"1min":0.45400000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":15.257999999999999,"5min":22.754000000000001,"15min":22.754000000000001},"last":0.5},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.2130000000000001,"15min":0.81899999999999995},"min":{"1min":0.41399999999999998,"5min":0.37,"15min":0.37},"max":{"1min":8.8030000000000008,"5min":12.449,"15min":12.449},"last":0.52800000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.1419999999999999,"15min":0.80200000000000005},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.33900000000000002},"max":{"1min":4.6260000000000003,"5min":14.246,"15min":14.246},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.9730000000000001,"5min":1.1850000000000001,"15min":0.86799999999999999},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":10.535,"5min":10.535,"15min":10.535},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.1949999999999998,"5min":1.72,"15min":1.054},"min":{"1min":0.65300000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":13.239000000000001,"5min":42.825000000000003,"15min":42.825000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":2.2210000000000001,"5min":1.8680000000000001,"15min":1.0620000000000001},"min":{"1min":0.63700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":14.340999999999999,"5min":42.851999999999997,"15min":42.851999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.8079999999999998,"5min":1.601,"15min":0.999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":17.359999999999999,"5min":24.199000000000002,"15min":24.199000000000002},"last":0.69399999999999995},{"interface":"front","average":{"1min":3.04,"5min":1.427,"15min":0.95099999999999996},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.38300000000000001},"max":{"1min":16.760000000000002,"5min":16.760000000000002,"15min":16.760000000000002},"last":0.67200000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.68400000000000005,"15min":0.65500000000000003},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4370000000000001,"5min":7.1609999999999996,"15min":7.1609999999999996},"last":0.626},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.91600000000000004,"15min":0.77000000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.444,"5min":12.491,"15min":12.491},"last":0.73099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77900000000000003,"15min":0.76500000000000001},"min":{"1min":0.46600000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.2330000000000001,"5min":9.4819999999999993,"15min":9.4819999999999993},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":1.1379999999999999,"15min":0.92300000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":2.0390000000000001,"5min":12.945,"15min":12.945},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.754,"15min":0.78300000000000003},"min":{"1min":0.47599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.1899999999999999,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":2.2970000000000002,"5min":1.0800000000000001,"15min":0.88},"min":{"1min":0.45600000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":27.651,"5min":27.651,"15min":27.651},"last":0.751}]}]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58400,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313440,"statfs":{"total":96636764160,"available":96576962560,"internally_reserved":0,"allocated":868352,"data_stored":365489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.9470000000000001,"5min":0.92500000000000004,"15min":0.71599999999999997},"min":{"1min":0.502,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":11.519,"5min":11.519,"15min":11.519},"last":0.40400000000000003},{"interface":"front","average":{"1min":2.6600000000000001,"5min":1.131,"15min":0.79100000000000004},"min":{"1min":0.53100000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":12.952,"5min":12.952,"15min":12.952},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.9119999999999999,"5min":1.024,"15min":0.79100000000000004},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":13.358000000000001,"5min":13.358000000000001,"15min":13.358000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":2.71,"5min":1.2170000000000001,"15min":0.86699999999999999},"min":{"1min":0.496,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":21.856999999999999,"5min":21.856999999999999,"15min":21.856999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":1.665,"5min":0.95199999999999996,"15min":0.77300000000000002},"min":{"1min":0.45800000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":6.3239999999999998,"5min":6.3239999999999998,"15min":6.3239999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.774,"5min":0.98099999999999998,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.435,"15min":0.434},"max":{"1min":8.0220000000000002,"5min":8.0220000000000002,"15min":8.0220000000000002},"last":0.628}]},{"osd":3,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.5750000000000002,"5min":1.1559999999999999,"15min":0.88400000000000001},"min":{"1min":0.53200000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":13.317,"5min":13.317,"15min":13.317},"last":0.64700000000000002},{"interface":"front","average":{"1min":3.327,"5min":1.29,"15min":0.88},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":19.638000000000002,"5min":19.638000000000002,"15min":19.638000000000002},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.70099999999999996},"min":{"1min":0.38400000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.6259999999999999,"5min":11.093,"15min":11.093},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.67100000000000004,"15min":0.68300000000000005},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":3.3889999999999998,"5min":3.3889999999999998,"15min":3.3889999999999998},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.76500000000000001,"15min":0.77400000000000002},"min":{"1min":0.48899999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":3.4249999999999998,"5min":3.4249999999999998,"15min":3.4249999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.149,"5min":0.69799999999999995,"15min":0.73399999999999999},"min":{"1min":0.38300000000000001,"5min":0.308,"15min":0.308},"max":{"1min":7.2240000000000002,"5min":7.2240000000000002,"15min":7.2240000000000002},"last":0.67400000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.77400000000000002},"min":{"1min":0.45100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.319,"5min":4.1799999999999997,"15min":4.1799999999999997},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.0549999999999999,"5min":0.78300000000000003,"15min":0.81399999999999995},"min":{"1min":0.55100000000000005,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.1900000000000004,"5min":4.1900000000000004,"15min":4.1900000000000004},"last":0.751}]}]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45936,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325904,"statfs":{"total":96636764160,"available":96589725696,"internally_reserved":0,"allocated":880640,"data_stored":371797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.21,"15min":0.80900000000000005},"min":{"1min":0.42099999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":9.7590000000000003,"5min":34.802,"15min":34.802},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.98299999999999998,"15min":0.73899999999999999},"min":{"1min":0.35399999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.98,"5min":22.335999999999999,"15min":22.335999999999999},"last":0.79100000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.365,"15min":0.93500000000000005},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":7.625,"5min":38.198,"15min":38.198},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.3069999999999999,"15min":0.89300000000000002},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.605,"5min":38.220999999999997,"15min":38.220999999999997},"last":1.022}]},{"osd":2,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.0109999999999999,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":7.6989999999999998,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.628},{"interface":"front","average":{"1min":1.2929999999999999,"5min":0.92500000000000004,"15min":0.79900000000000004},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":5.2329999999999997,"5min":5.9279999999999999,"15min":5.9279999999999999},"last":0.82099999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.0009999999999999,"15min":0.78200000000000003},"min":{"1min":0.42899999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":7.4720000000000004,"5min":8.6839999999999993,"15min":8.6839999999999993},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.8819999999999999,"5min":1.1020000000000001,"15min":0.82799999999999996},"min":{"1min":0.49199999999999999,"5min":0.439,"15min":0.439},"max":{"1min":10.206,"5min":10.206,"15min":10.206},"last":1}]},{"osd":5,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.64700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4359999999999999,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.119},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78600000000000003,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.522,"5min":8.7880000000000003,"15min":8.7880000000000003},"last":0.96499999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.78700000000000003,"15min":0.73199999999999998},"min":{"1min":0.40699999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":7.5439999999999996,"5min":10.284000000000001,"15min":10.284000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67400000000000004,"15min":0.71999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.792,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.93600000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:10:59 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.82399999999999995,"15min":0.76100000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":5.6870000000000003,"5min":11.457000000000001,"15min":11.457000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.24,"5min":0.85199999999999998,"15min":0.82099999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.67,"5min":11.67,"15min":11.67},"last":1.0409999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":884736,"data_stored":371839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.125,"15min":0.67100000000000004},"min":{"1min":0.27000000000000002,"5min":0.255,"15min":0.255},"max":{"1min":8.1660000000000004,"5min":25.771999999999998,"15min":25.771999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":2.8610000000000002,"5min":1.4259999999999999,"15min":0.76600000000000001},"min":{"1min":0.37,"5min":0.246,"15min":0.246},"max":{"1min":23.074999999999999,"5min":28.332000000000001,"15min":28.332000000000001},"last":0.54000000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":3.0840000000000001,"5min":1.5229999999999999,"15min":0.79200000000000004},"min":{"1min":0.32900000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":20.321000000000002,"5min":40.457999999999998,"15min":40.457999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":1.589,"5min":1.579,"15min":0.88500000000000001},"min":{"1min":0.374,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":6.383,"5min":27.427,"15min":27.427},"last":0.49299999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":3.2869999999999999,"5min":1.417,"15min":0.78500000000000003},"min":{"1min":0.36199999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":23.036999999999999,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":2.02,"5min":1.1819999999999999,"15min":0.746},"min":{"1min":0.39800000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":6.2960000000000003,"5min":12.538,"15min":12.538},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":2.101,"5min":1.23,"15min":0.89200000000000002},"min":{"1min":0.45400000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":22.998000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":1.03},{"interface":"front","average":{"1min":3.681,"5min":1.4550000000000001,"15min":0.98299999999999998},"min":{"1min":0.57399999999999995,"5min":0.377,"15min":0.377},"max":{"1min":20.399000000000001,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.751}]},{"osd":5,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":1.107,"15min":0.871},"min":{"1min":0.48999999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":5.9710000000000001,"5min":7.0810000000000004,"15min":7.0810000000000004},"last":0.85899999999999999},{"interface":"front","average":{"1min":2.9980000000000002,"5min":1.5029999999999999,"15min":1.0149999999999999},"min":{"1min":0.55500000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":22.908999999999999,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":3.2789999999999999,"5min":1.458,"15min":0.98899999999999999},"min":{"1min":0.53700000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":20.445,"5min":20.445,"15min":20.445},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.762,"5min":1.1419999999999999,"15min":0.93300000000000005},"min":{"1min":0.58999999999999997,"5min":0.495,"15min":0.495},"max":{"1min":5.9249999999999998,"5min":7.1399999999999997,"15min":7.1399999999999997},"last":0.73099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":2.871,"5min":1.456,"15min":0.98399999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":23.111999999999998,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.96099999999999997},{"interface":"front","average":{"1min":5.6580000000000004,"5min":1.964,"15min":1.1839999999999999},"min":{"1min":0.46400000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":44.817999999999998,"5min":44.817999999999998,"15min":44.817999999999998},"last":0.64900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574934,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58420,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313420,"statfs":{"total":96636764160,"available":96576942080,"internally_reserved":0,"allocated":888832,"data_stored":373339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.4569999999999999,"5min":1.944,"15min":0.94199999999999995},"min":{"1min":0.309,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":12.695,"5min":29.527999999999999,"15min":29.527999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":3.0739999999999998,"5min":2.3650000000000002,"15min":1.129},"min":{"1min":0.432,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":14.538,"5min":34.122999999999998,"15min":34.122999999999998},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":3.2410000000000001,"5min":3.3769999999999998,"15min":1.4490000000000001},"min":{"1min":0.35599999999999998,"5min":0.30499999999999999,"15min":0.30199999999999999},"max":{"1min":34.029000000000003,"5min":113.149,"15min":113.149},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.8340000000000001,"5min":1.7450000000000001,"15min":0.90700000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":13.464,"5min":41.302,"15min":41.302},"last":0.32200000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.7200000000000002,"5min":1.9890000000000001,"15min":0.96099999999999997},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":19.206,"5min":46.960000000000001,"15min":46.960000000000001},"last":0.379},{"interface":"front","average":{"1min":1.3260000000000001,"5min":2.4830000000000001,"15min":1.177},"min":{"1min":0.41599999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":6.7830000000000004,"5min":115.693,"15min":115.693},"last":0.47299999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.8420000000000001,"15min":1.1419999999999999},"min":{"1min":0.53600000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":6.4580000000000002,"5min":24.366,"15min":24.366},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.456,"5min":1.746,"15min":1.0640000000000001},"min":{"1min":0.55200000000000005,"5min":0.443,"15min":0.443},"max":{"1min":6.7460000000000004,"5min":20.489000000000001,"15min":20.489000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.9039999999999999,"5min":1.9470000000000001,"15min":1.125},"min":{"1min":0.55300000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":19.282,"5min":29.645,"15min":29.645},"last":0.64400000000000002},{"interface":"front","average":{"1min":2.8969999999999998,"5min":2.0270000000000001,"15min":1.2170000000000001},"min":{"1min":0.56699999999999995,"5min":0.51400000000000001,"15min":0.496},"max":{"1min":26.084,"5min":26.084,"15min":26.084},"last":0.73799999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.786,"15min":1.1220000000000001},"min":{"1min":0.54800000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":6.9199999999999999,"5min":20.370999999999999,"15min":20.370999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":3.2210000000000001,"5min":2.0430000000000001,"15min":1.208},"min":{"1min":0.50700000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":19.321000000000002,"5min":29.690000000000001,"15min":29.690000000000001},"last":0.82599999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.621,"15min":1.095},"min":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":6.8600000000000003,"5min":20.446999999999999,"15min":20.446999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":2.8780000000000001,"5min":2.0390000000000001,"15min":1.2569999999999999},"min":{"1min":0.57699999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":19.361999999999998,"5min":29.785,"15min":29.785},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45936,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325904,"statfs":{"total":96636764160,"available":96589725696,"internally_reserved":0,"allocated":884736,"data_stored":367627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.3290000000000002,"5min":1.4399999999999999,"15min":0.93999999999999995},"min":{"1min":0.33200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":7.1580000000000004,"5min":39.948,"15min":39.948},"last":0.318},{"interface":"front","average":{"1min":3.4340000000000002,"5min":1.577,"15min":1.0269999999999999},"min":{"1min":0.36799999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":35.790999999999997,"5min":35.790999999999997,"15min":35.790999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.254,"5min":1.484,"15min":0.96899999999999997},"min":{"1min":0.34200000000000003,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":11.346,"5min":49.229999999999997,"15min":49.229999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":2.4670000000000001,"5min":1.008,"15min":0.83299999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":7.4100000000000001,"5min":10.349,"15min":10.349},"last":0.49299999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":3.3719999999999999,"5min":1.7669999999999999,"15min":1.095},"min":{"1min":0.40000000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":23.521999999999998,"5min":25.427,"15min":25.427},"last":0.36399999999999999},{"interface":"front","average":{"1min":2.8929999999999998,"5min":1.252,"15min":0.99099999999999999},"min":{"1min":0.46200000000000002,"5min":0.378,"15min":0.378},"max":{"1min":10.077999999999999,"5min":13.119999999999999,"15min":13.119999999999999},"last":0.42999999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":3,"5min":1.512,"15min":1.1899999999999999},"min":{"1min":0.58099999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":23.695,"5min":23.695,"15min":23.695},"last":0.59999999999999998},{"interface":"front","average":{"1min":2.5750000000000002,"5min":1.222,"15min":1.05},"min":{"1min":0.68100000000000005,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":10.038,"5min":12.997,"15min":12.997},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.9969999999999999,"5min":1.526,"15min":1.111},"min":{"1min":0.61799999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":23.626999999999999,"5min":23.626999999999999,"15min":23.626999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":2.5840000000000001,"5min":1.2509999999999999,"15min":1.1579999999999999},"min":{"1min":0.76500000000000001,"5min":0.496,"15min":0.496},"max":{"1min":9.9610000000000003,"5min":13.054,"15min":13.054},"last":0.66900000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":3.0680000000000001,"5min":1.472,"15min":1.1519999999999999},"min":{"1min":0.65700000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":23.760000000000002,"5min":23.760000000000002,"15min":23.760000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":2.323,"5min":1.2450000000000001,"15min":1.109},"min":{"1min":0.59799999999999998,"5min":0.441,"15min":0.441},"max":{"1min":11.643000000000001,"5min":11.712999999999999,"15min":11.712999999999999},"last":0.83399999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":2.492,"5min":1.2929999999999999,"15min":1.143},"min":{"1min":0.67800000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":11.833,"5min":11.833,"15min":11.833},"last":0.79700000000000004},{"interface":"front","average":{"1min":3.1000000000000001,"5min":1.357,"15min":1.1619999999999999},"min":{"1min":0.71599999999999997,"5min":0.502,"15min":0.502},"max":{"1min":14.606,"5min":14.606,"15min":14.606},"last":0.76100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46492,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325348,"statfs":{"total":96636764160,"available":96589156352,"internally_reserved":0,"allocated":1458176,"data_stored":955767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":3.6899999999999999,"5min":1.76,"15min":1.1739999999999999},"min":{"1min":0.26600000000000001,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":16.149000000000001,"5min":31.977,"15min":31.977},"last":0.434},{"interface":"front","average":{"1min":2.4239999999999999,"5min":1.512,"15min":0.81599999999999995},"min":{"1min":0.374,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":9.9589999999999996,"5min":36.145000000000003,"15min":36.145000000000003},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":4.2450000000000001,"5min":1.8169999999999999,"15min":0.94199999999999995},"min":{"1min":0.45100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":36.386000000000003,"5min":41.387999999999998,"15min":41.387999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":2.5819999999999999,"5min":1.575,"15min":0.96999999999999997},"min":{"1min":0.38100000000000001,"5min":0.25,"15min":0.25},"max":{"1min":16.556000000000001,"5min":33.219999999999999,"15min":33.219999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.5760000000000001,"5min":1.399,"15min":1.0960000000000001},"min":{"1min":0.40699999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":9.8740000000000006,"5min":31.943999999999999,"15min":31.943999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":4.4980000000000002,"5min":1.6339999999999999,"15min":0.92000000000000004},"min":{"1min":0.55200000000000005,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":45.128,"5min":45.128,"15min":45.128},"last":0.49199999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.635,"5min":1.2589999999999999,"15min":1.0429999999999999},"min":{"1min":0.56699999999999995,"5min":0.372,"15min":0.372},"max":{"1min":9.4760000000000009,"5min":32.932000000000002,"15min":32.932000000000002},"last":0.51300000000000001},{"interface":"front","average":{"1min":2.5329999999999999,"5min":1.5329999999999999,"15min":1.1539999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.44},"max":{"1min":14.831,"5min":31.782,"15min":31.782},"last":0.64300000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.395,"5min":1.5389999999999999,"15min":1.1739999999999999},"min":{"1min":0.57899999999999996,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":11.247999999999999,"5min":31.858000000000001,"15min":31.858000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.1930000000000001,"15min":1.046},"min":{"1min":0.61099999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":9.7080000000000002,"5min":33.151000000000003,"15min":33.151000000000003},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":1.726,"5min":1.1579999999999999,"15min":0.93600000000000005},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":6.5670000000000002,"5min":9.6440000000000001,"15min":9.6440000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.714,"5min":1.22,"15min":1.0669999999999999},"min":{"1min":0.64400000000000002,"5min":0.498,"15min":0.498},"max":{"1min":9.8550000000000004,"5min":33.098999999999997,"15min":33.098999999999997},"last":0.59999999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:10:54 2024","interfaces":[{"interface":"back","average":{"1min":2.1339999999999999,"5min":1.2230000000000001,"15min":0.93600000000000005},"min":{"1min":0.65100000000000002,"5min":0.504,"15min":0.501},"max":{"1min":9.266,"5min":9.6910000000000007,"15min":9.6910000000000007},"last":0.61799999999999999},{"interface":"front","average":{"1min":3.0329999999999999,"5min":1.647,"15min":1.2310000000000001},"min":{"1min":0.61899999999999999,"5min":0.504,"15min":0.504},"max":{"1min":10.377000000000001,"5min":31.905999999999999,"15min":31.905999999999999},"last":0.68200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:11:39.311 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:11:39.311 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:11:39.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 Processor -- start 2024-01-15T18:11:39.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 -- start start 2024-01-15T18:11:39.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88741436f0 0x7f88740ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f88740ae5c0 0x7f88740b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88740721b0 con 0x7f88740b39d0 2024-01-15T18:11:39.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88740b3e40 con 0x7f88740ae5c0 2024-01-15T18:11:39.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8878e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8874072640 con 0x7f88741436f0 2024-01-15T18:11:39.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8872800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f88740b39d0 0x7f8874160d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55702/0 (socket says 172.21.15.143:55702) 2024-01-15T18:11:39.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8872800640 1 -- 172.21.15.143:0/719144696 learned_addr learned my addr 172.21.15.143:0/719144696 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8871400640 1 --2- 172.21.15.143:0/719144696 >> v2:172.21.15.204:3300/0 conn(0x7f88740ae5c0 0x7f88740b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8871e00640 1 --2- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88741436f0 0x7f88740ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8870a00640 1 -- 172.21.15.143:0/719144696 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580047075 0 0) 0x7f88740721b0 con 0x7f88740b39d0 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8870a00640 1 -- 172.21.15.143:0/719144696 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8854003530 con 0x7f88740b39d0 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8871400640 1 -- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88741436f0 msgr2=0x7f88740ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8871400640 1 --2- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88741436f0 0x7f88740ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8871400640 1 -- 172.21.15.143:0/719144696 >> v1:172.21.15.143:6789/0 conn(0x7f88740b39d0 legacy=0x7f8874160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.493+0000 7f8871400640 1 -- 172.21.15.143:0/719144696 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740715a0 con 0x7f88740ae5c0 2024-01-15T18:11:39.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.497+0000 7f8871400640 1 --2- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f88740b3490 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f885c0097c0 tx=0x7f885c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e59b52645a12ba8f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.497+0000 7f8870a00640 1 -- 172.21.15.143:0/719144696 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0ab600 con 0x7f88740ae5c0 2024-01-15T18:11:39.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.497+0000 7f8870a00640 1 -- 172.21.15.143:0/719144696 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c0a7050 con 0x7f88740ae5c0 2024-01-15T18:11:39.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.497+0000 7f8870a00640 1 -- 172.21.15.143:0/719144696 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0ab980 con 0x7f88740ae5c0 2024-01-15T18:11:39.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 msgr2=0x7f88740b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 --2- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f88740b3490 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f885c0097c0 tx=0x7f885c01c450 comp rx=0 tx=0).stop 2024-01-15T18:11:39.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- 172.21.15.143:0/719144696 shutdown_connections 2024-01-15T18:11:39.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 --2- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f88740b3490 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 --2- 172.21.15.143:0/719144696 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88741436f0 0x7f88740ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- 172.21.15.143:0/719144696 >> 172.21.15.143:0/719144696 conn(0x7f887406a890 msgr2=0x7f887406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:39.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- 172.21.15.143:0/719144696 shutdown_connections 2024-01-15T18:11:39.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- 172.21.15.143:0/719144696 wait complete. 2024-01-15T18:11:39.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 Processor -- start 2024-01-15T18:11:39.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- start start 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f8874142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88740b39d0 0x7f8874136670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 0x7f88740afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887415b060 con 0x7f88741436f0 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88741674f0 con 0x7f88740ae5c0 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8878e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88741677d0 con 0x7f88740b39d0 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 0x7f88740afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 0x7f88740afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51760/0 (socket says 172.21.15.143:51760) 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 -- 172.21.15.143:0/2512289584 learned_addr learned my addr 172.21.15.143:0/2512289584 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8871e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f8874142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8871400640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88740b39d0 0x7f8874136670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 -- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88740b39d0 msgr2=0x7f8874136670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88740b39d0 0x7f8874136670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 -- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 msgr2=0x7f8874142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f8874142b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8872800640 1 -- 172.21.15.143:0/2512289584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c009420 con 0x7f88741436f0 2024-01-15T18:11:39.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.501+0000 7f8871e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f8874142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:39.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f8872800640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 0x7f88740afc50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f886c00cb40 tx=0x7f886c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c00aca0 con 0x7f88741436f0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c00ae00 con 0x7f88741436f0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887415b2d0 con 0x7f88741436f0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c012620 con 0x7f88741436f0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8874136bb0 con 0x7f88741436f0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887415b510 con 0x7f88741436f0 2024-01-15T18:11:39.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.505+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f886c007070 con 0x7f88741436f0 2024-01-15T18:11:39.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.509+0000 7f886a000640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f884c0671f0 0x7f884c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.509+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f886c085680 con 0x7f88741436f0 2024-01-15T18:11:39.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.509+0000 7f8871e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f884c0671f0 0x7f884c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.509+0000 7f8871e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f884c0671f0 0x7f884c0696b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8864009b10 tx=0x7f8864008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.513+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f887415b510 con 0x7f88741436f0 2024-01-15T18:11:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.713+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f887415b510 con 0x7f88741436f0 2024-01-15T18:11:39.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f886a000640 1 -- 172.21.15.143:0/2512289584 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f887415b510 con 0x7f88741436f0 2024-01-15T18:11:39.718 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:11:39.719 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:11:39.719 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:11:39.719 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:11:39.719 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:11:39.719 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f884c0671f0 msgr2=0x7f884c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f884c0671f0 0x7f884c0696b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8864009b10 tx=0x7f8864008040 comp rx=0 tx=0).stop 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 msgr2=0x7f88740afc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 0x7f88740afc50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f886c00cb40 tx=0x7f886c00a060 comp rx=0 tx=0).stop 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 shutdown_connections 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f884c0671f0 0x7f884c0696b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88741436f0 0x7f88740afc50 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f88740b39d0 0x7f8874136670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 --2- 172.21.15.143:0/2512289584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88740ae5c0 0x7f8874142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 >> 172.21.15.143:0/2512289584 conn(0x7f887406a890 msgr2=0x7f887413f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 shutdown_connections 2024-01-15T18:11:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.717+0000 7f8878e00640 1 -- 172.21.15.143:0/2512289584 wait complete. 2024-01-15T18:11:39.740 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:11:39.822 INFO:tasks.ceph.mds.c.smithi143.stderr:2024-01-15T18:11:39.817+0000 7f1e23a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:11:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.873+0000 7f0781a00640 1 Processor -- start 2024-01-15T18:11:39.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.873+0000 7f0781a00640 1 -- start start 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f0781a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f077c071bb0 0x7f077c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f0781a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f077c15c5a0 0x7f077c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f0781a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c0ceb50 con 0x7f077c143780 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f0781a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c072570 con 0x7f077c15c5a0 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f0781a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c15c110 con 0x7f077c071bb0 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f077c15c5a0 0x7f077c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f077c15c5a0 0x7f077c15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51776/0 (socket says 172.21.15.143:51776) 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 -- 172.21.15.143:0/3386798244 learned_addr learned my addr 172.21.15.143:0/3386798244 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077a000640 1 --2- 172.21.15.143:0/3386798244 >> v2:172.21.15.204:3300/0 conn(0x7f077c071bb0 0x7f077c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 -- 172.21.15.143:0/3386798244 >> v2:172.21.15.204:3300/0 conn(0x7f077c071bb0 msgr2=0x7f077c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 --2- 172.21.15.143:0/3386798244 >> v2:172.21.15.204:3300/0 conn(0x7f077c071bb0 0x7f077c072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 -- 172.21.15.143:0/3386798244 >> v1:172.21.15.204:6789/0 conn(0x7f077c143780 legacy=0x7f077c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 -- 172.21.15.143:0/3386798244 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c15b7e0 con 0x7f077c15c5a0 2024-01-15T18:11:39.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.877+0000 7f077b400640 1 --2- 172.21.15.143:0/3386798244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c15c9e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f077c06beb0 tx=0x7f077401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a61d614d0f24a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0779600640 1 -- 172.21.15.143:0/3386798244 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077400b2d0 con 0x7f077c15c5a0 2024-01-15T18:11:39.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0779600640 1 -- 172.21.15.143:0/3386798244 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400b430 con 0x7f077c15c5a0 2024-01-15T18:11:39.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0779600640 1 -- 172.21.15.143:0/3386798244 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740a7cc0 con 0x7f077c15c5a0 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 -- 172.21.15.143:0/3386798244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 msgr2=0x7f077c15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 --2- 172.21.15.143:0/3386798244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c15c9e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f077c06beb0 tx=0x7f077401ebd0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 -- 172.21.15.143:0/3386798244 shutdown_connections 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 --2- 172.21.15.143:0/3386798244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c15c9e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 --2- 172.21.15.143:0/3386798244 >> v2:172.21.15.204:3300/0 conn(0x7f077c071bb0 0x7f077c072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 -- 172.21.15.143:0/3386798244 >> 172.21.15.143:0/3386798244 conn(0x7f077c06c3f0 msgr2=0x7f077c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 -- 172.21.15.143:0/3386798244 shutdown_connections 2024-01-15T18:11:39.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.881+0000 7f0781a00640 1 -- 172.21.15.143:0/3386798244 wait complete. 2024-01-15T18:11:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 Processor -- start 2024-01-15T18:11:39.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 -- start start 2024-01-15T18:11:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f077c071bb0 0x7f077c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f077c143780 0x7f077c13fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c13eda0 con 0x7f077c15c5a0 2024-01-15T18:11:39.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c13f080 con 0x7f077c071bb0 2024-01-15T18:11:39.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0781a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c13f360 con 0x7f077c143780 2024-01-15T18:11:39.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51780/0 (socket says 172.21.15.143:51780) 2024-01-15T18:11:39.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 -- 172.21.15.143:0/2218593122 learned_addr learned my addr 172.21.15.143:0/2218593122 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 -- 172.21.15.143:0/2218593122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f077c143780 msgr2=0x7f077c13fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f077c143780 0x7f077c13fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 -- 172.21.15.143:0/2218593122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f077c071bb0 msgr2=0x7f077c158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f077c071bb0 0x7f077c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 -- 172.21.15.143:0/2218593122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077400bd40 con 0x7f077c15c5a0 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f077b400640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c0afb90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f077401b7e0 tx=0x7f077401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.885+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774009800 con 0x7f077c15c5a0 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774009980 con 0x7f077c15c5a0 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774009d30 con 0x7f077c15c5a0 2024-01-15T18:11:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c13f640 con 0x7f077c15c5a0 2024-01-15T18:11:39.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c13e900 con 0x7f077c15c5a0 2024-01-15T18:11:39.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c13f4a0 con 0x7f077c15c5a0 2024-01-15T18:11:39.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f07740c3460 con 0x7f077c15c5a0 2024-01-15T18:11:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0772a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07540671f0 0x7f07540696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f077411e2d0 con 0x7f077c15c5a0 2024-01-15T18:11:39.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.889+0000 7f077aa00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07540671f0 0x7f07540696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:39.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.893+0000 7f077aa00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07540671f0 0x7f07540696b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f076c000f70 tx=0x7f076c0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:39.893+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f077c13f4a0 con 0x7f077c15c5a0 2024-01-15T18:11:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.081+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f077c13f4a0 con 0x7f077c15c5a0 2024-01-15T18:11:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.081+0000 7f0772a00640 1 -- 172.21.15.143:0/2218593122 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f077c13f4a0 con 0x7f077c15c5a0 2024-01-15T18:11:40.085 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:11:40.085 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3237385557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3237385557},{"type":"v1","addr":"172.21.15.204:6834","nonce":3237385557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/904040921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":904040921},{"type":"v1","addr":"172.21.15.204:6835","nonce":904040921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/64114877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":64114877},{"type":"v1","addr":"172.21.15.143:6836","nonce":64114877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:11:34.225370+0000","modified":"2024-01-15T18:11:37.246452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/1300573627","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":1300573627},{"type":"v1","addr":"172.21.15.143:6837","nonce":1300573627}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-01-15T18:11:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 44 2024-01-15T18:11:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07540671f0 msgr2=0x7f07540696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07540671f0 0x7f07540696b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f076c000f70 tx=0x7f076c0079c0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 msgr2=0x7f077c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c0afb90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f077401b7e0 tx=0x7f077401b970 comp rx=0 tx=0).stop 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 shutdown_connections 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f07540671f0 0x7f07540696b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f077c15c5a0 0x7f077c0afb90 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f077c143780 0x7f077c13fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 --2- 172.21.15.143:0/2218593122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f077c071bb0 0x7f077c158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 >> 172.21.15.143:0/2218593122 conn(0x7f077c06c3f0 msgr2=0x7f077c06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 shutdown_connections 2024-01-15T18:11:40.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.085+0000 7f0781a00640 1 -- 172.21.15.143:0/2218593122 wait complete. 2024-01-15T18:11:40.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:11:34.225370+0000', 'modified': '2024-01-15T18:11:37.246452+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6996': {'gid': 6996, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6837/1300573627', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6835', 'nonce': 1300573627}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 1300573627}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:11:40.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:11:40.510 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-15T18:11:40.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 Processor -- start 2024-01-15T18:11:40.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 -- start start 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe7640ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe76415ec80 0x7fe764141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe764142470 0x7fe764071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76415fbb0 con 0x7fe764142470 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76415f100 con 0x7fe76415ec80 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76415f310 con 0x7fe764141660 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe764142470 0x7fe764071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe764142470 0x7fe764071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51786/0 (socket says 172.21.15.143:51786) 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 -- 172.21.15.143:0/794128074 learned_addr learned my addr 172.21.15.143:0/794128074 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe769200640 1 --2- 172.21.15.143:0/794128074 >> v2:172.21.15.204:3300/0 conn(0x7fe76415ec80 0x7fe764141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe769c00640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe7640ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 -- 172.21.15.143:0/794128074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 msgr2=0x7fe7640ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe7640ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 -- 172.21.15.143:0/794128074 >> v2:172.21.15.204:3300/0 conn(0x7fe76415ec80 msgr2=0x7fe764141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 --2- 172.21.15.143:0/794128074 >> v2:172.21.15.204:3300/0 conn(0x7fe76415ec80 0x7fe764141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe76a600640 1 -- 172.21.15.143:0/794128074 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76415f5f0 con 0x7fe764142470 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe769c00640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe7640ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:40.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.649+0000 7fe769200640 1 --2- 172.21.15.143:0/794128074 >> v2:172.21.15.204:3300/0 conn(0x7fe76415ec80 0x7fe764141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:40.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76a600640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe764071d00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe76406bee0 tx=0x7fe75c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d230fe8fd488c8b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe763e00640 1 -- 172.21.15.143:0/794128074 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0a5730 con 0x7fe764142470 2024-01-15T18:11:40.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe763e00640 1 -- 172.21.15.143:0/794128074 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c0a5890 con 0x7fe764142470 2024-01-15T18:11:40.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe763e00640 1 -- 172.21.15.143:0/794128074 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0a5c20 con 0x7fe764142470 2024-01-15T18:11:40.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 -- 172.21.15.143:0/794128074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 msgr2=0x7fe764071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe764071d00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe76406bee0 tx=0x7fe75c01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 -- 172.21.15.143:0/794128074 shutdown_connections 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe764071d00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 --2- 172.21.15.143:0/794128074 >> v2:172.21.15.204:3300/0 conn(0x7fe76415ec80 0x7fe764141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 --2- 172.21.15.143:0/794128074 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe7640ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 -- 172.21.15.143:0/794128074 >> 172.21.15.143:0/794128074 conn(0x7fe76406c3f0 msgr2=0x7fe76406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 -- 172.21.15.143:0/794128074 shutdown_connections 2024-01-15T18:11:40.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.653+0000 7fe76c600640 1 -- 172.21.15.143:0/794128074 wait complete. 2024-01-15T18:11:40.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 Processor -- start 2024-01-15T18:11:40.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 -- start start 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe764159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe7641388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76415ec80 0x7fe7640ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640b1ea0 con 0x7fe764142470 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764139350 con 0x7fe76415ec80 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640b20f0 con 0x7fe764141660 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe7641388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe7641388b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51802/0 (socket says 172.21.15.143:51802) 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 -- 172.21.15.143:0/3570917349 learned_addr learned my addr 172.21.15.143:0/3570917349 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769c00640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe764159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76a600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76415ec80 0x7fe7640ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 -- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 msgr2=0x7fe764159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe764159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 -- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76415ec80 msgr2=0x7fe7640ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76415ec80 0x7fe7640ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 -- 172.21.15.143:0/3570917349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c009b30 con 0x7fe764142470 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe769200640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe7641388b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c7f0 tx=0x7fe74c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe761600640 1 -- 172.21.15.143:0/3570917349 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c007820 con 0x7fe764142470 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe761600640 1 -- 172.21.15.143:0/3570917349 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c007980 con 0x7fe764142470 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7640cd150 con 0x7fe764142470 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe761600640 1 -- 172.21.15.143:0/3570917349 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c007d30 con 0x7fe764142470 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.657+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7640b26b0 con 0x7fe764142470 2024-01-15T18:11:40.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe761600640 1 -- 172.21.15.143:0/3570917349 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7fe74c0113f0 con 0x7fe764142470 2024-01-15T18:11:40.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe761600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe73c0671f0 0x7fe73c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe761600640 1 -- 172.21.15.143:0/3570917349 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe74c082ff0 con 0x7fe764142470 2024-01-15T18:11:40.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe769c00640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe73c0671f0 0x7fe73c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 Processor -- start 2024-01-15T18:11:40.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 -- start start 2024-01-15T18:11:40.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe769c00640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe73c0671f0 0x7fe73c0696b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe754006b20 tx=0x7fe754008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf417c90 0x5574cf418070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf4185b0 0x5574cf3cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf238130 con 0x5574cf3ce320 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf2259a0 con 0x5574cf4185b0 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574cf222ed0 con 0x5574cf417c90 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51806/0 (socket says 172.21.15.143:51806) 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe76a600640 1 -- 172.21.15.143:0/542757593 learned_addr learned my addr 172.21.15.143:0/542757593 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe769200640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf4185b0 0x5574cf3cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.661+0000 7fe769c00640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf417c90 0x5574cf418070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769200640 1 -- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf417c90 msgr2=0x5574cf418070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769200640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf417c90 0x5574cf418070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769200640 1 -- 172.21.15.143:0/542757593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 msgr2=0x5574cf3ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769200640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769200640 1 -- 172.21.15.143:0/542757593 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574cef16890 con 0x5574cf4185b0 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76a600640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769c00640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf417c90 0x5574cf418070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe769200640 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf4185b0 0x5574cf3cdde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe74c082f00 tx=0x7fe74c055940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af77115aa66b8bd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe75a000640 1 -- 172.21.15.143:0/542757593 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c06d070 con 0x5574cf4185b0 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe75a000640 1 -- 172.21.15.143:0/542757593 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c05dc40 con 0x5574cf4185b0 2024-01-15T18:11:40.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe75a000640 1 -- 172.21.15.143:0/542757593 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c052050 con 0x5574cf4185b0 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf4185b0 msgr2=0x5574cf3cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf4185b0 0x5574cf3cdde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe74c082f00 tx=0x7fe74c055940 comp rx=0 tx=0).stop 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- 172.21.15.143:0/542757593 shutdown_connections 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf4185b0 0x5574cf3cdde0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- 172.21.15.143:0/542757593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf417c90 0x5574cf418070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- 172.21.15.143:0/542757593 >> 172.21.15.143:0/542757593 conn(0x5574cf3ca2d0 msgr2=0x5574cf3cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- 172.21.15.143:0/542757593 shutdown_connections 2024-01-15T18:11:40.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- 172.21.15.143:0/542757593 wait complete. 2024-01-15T18:11:40.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- start start 2024-01-15T18:11:40.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 Processor -- start 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf3d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf3e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf21e7c0 con 0x5574cf3ce320 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf1d6bf0 con 0x5574cf417c90 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.665+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574cf1dca90 con 0x5574cf4185b0 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 10 client.0 _ll_register_callbacks cb 0x5574cf3b93c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:11:40.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51820/0 (socket says 172.21.15.143:51820) 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 -- 172.21.15.143:0/380019542 learned_addr learned my addr 172.21.15.143:0/380019542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769200640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf3d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76a600640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf3e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 -- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 msgr2=0x5574cf3e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf3e6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 -- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 msgr2=0x5574cf3d3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf3d3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 -- 172.21.15.143:0/380019542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe76415f5f0 con 0x5574cf3ce320 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769c00640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3d2d40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe7540139d0 tx=0x7fe7540046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9a4a6d36b20bd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 1 -- 172.21.15.143:0/380019542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400e9e0 con 0x5574cf3ce320 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574cf1e0470 con 0x5574cf3ce320 2024-01-15T18:11:40.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 1 -- 172.21.15.143:0/380019542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe754014b90 con 0x5574cf3ce320 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 1 -- 172.21.15.143:0/380019542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe754014d50 con 0x5574cf3ce320 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 10 client.7128 fetch_fsmap learned FSMap version 44 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x7fe75400e850 con 0x5574cf3ce320 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 1 -- 172.21.15.143:0/380019542 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5574cf1e0470 con 0x5574cf3ce320 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 1 -- 172.21.15.143:0/380019542 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe754011070 con 0x5574cf3ce320 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 10 client.7128 fetch_fsmap finished waiting for FSMap version 44 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 10 client.7128 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 20 client.7128 populate_metadata read hostname 'smithi143' 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf3e7320 0x5574cf429f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 4 client.7128 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 --> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574cf1e3b50 con 0x5574cf3e7320 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769200640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf3e7320 0x5574cf429f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe769200640 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf3e7320 0x5574cf429f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de0db12531b83cd server_cookie=54de9a56f1547fcd in_seq=0 out_seq=0 2024-01-15T18:11:40.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 10 client.7128 ms_handle_connect on v2:172.21.15.143:6835/1300573627 2024-01-15T18:11:40.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 1 -- 172.21.15.143:0/380019542 <== mds.0 v2:172.21.15.143:6835/1300573627 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574cf1e3b50 con 0x5574cf3e7320 2024-01-15T18:11:40.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 10 client.7128 handle_command_reply: tid=0 2024-01-15T18:11:40.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 client.7128 shutdown 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 msgr2=0x5574cf3d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3d2d40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fe7540139d0 tx=0x7fe7540046d0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 shutdown_connections 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf3e7320 0x5574cf429f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf3e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf3d3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 --2- 172.21.15.143:0/380019542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf3d2d40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe752a00640 0 client.7128 ms_handle_reset on v2:172.21.15.143:6835/1300573627 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.669+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 >> 172.21.15.143:0/380019542 conn(0x5574cf3ca2d0 msgr2=0x5574cf3cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.673+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 shutdown_connections 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.673+0000 7fe76fa77000 1 -- 172.21.15.143:0/380019542 wait complete. 2024-01-15T18:11:40.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.673+0000 7fe76fa77000 20 client.7128 trim_cache size 0 max 16384 2024-01-15T18:11:40.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.685+0000 7fe76fa77000 1 Processor -- start 2024-01-15T18:11:40.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.685+0000 7fe76fa77000 1 -- start start 2024-01-15T18:11:40.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf416c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf415b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf416070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ceed7940 con 0x5574cf3ce320 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf21bbc0 con 0x5574cf417c90 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574cf21a130 con 0x5574cf4185b0 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf416c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf416c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51824/0 (socket says 172.21.15.143:51824) 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 -- 172.21.15.143:0/45549365 learned_addr learned my addr 172.21.15.143:0/45549365 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769200640 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf415b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76a600640 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf416070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 -- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 msgr2=0x5574cf416070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf416070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 -- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 msgr2=0x5574cf415b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf415b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 -- 172.21.15.143:0/45549365 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640b26b0 con 0x5574cf3ce320 2024-01-15T18:11:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe769c00640 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf416c90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe754008010 tx=0x7fe754021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550789eeefc489e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe75aa00640 1 -- 172.21.15.143:0/45549365 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400e9e0 con 0x5574cf3ce320 2024-01-15T18:11:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe75aa00640 1 -- 172.21.15.143:0/45549365 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754037070 con 0x5574cf3ce320 2024-01-15T18:11:40.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe75aa00640 1 -- 172.21.15.143:0/45549365 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754029a00 con 0x5574cf3ce320 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- 172.21.15.143:0/45549365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 msgr2=0x5574cf416c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf416c90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fe754008010 tx=0x7fe754021000 comp rx=0 tx=0).stop 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- 172.21.15.143:0/45549365 shutdown_connections 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf416070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf417c90 0x5574cf415b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 --2- 172.21.15.143:0/45549365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf3ce320 0x5574cf416c90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- 172.21.15.143:0/45549365 >> 172.21.15.143:0/45549365 conn(0x5574cf3ca2d0 msgr2=0x5574cf425280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- 172.21.15.143:0/45549365 shutdown_connections 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- 172.21.15.143:0/45549365 wait complete. 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 -- start start 2024-01-15T18:11:40.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.689+0000 7fe76fa77000 1 Processor -- start 2024-01-15T18:11:40.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf3ce320 0x5574cf3cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 0x5574cf42afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf42b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf1d5a10 con 0x5574cf417c90 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574cf21a5c0 con 0x5574cf3ce320 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574cf2260c0 con 0x5574cf4185b0 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 10 client.0 _ll_register_callbacks cb 0x5574cf26cc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 0x5574cf42afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 0x5574cf42afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51838/0 (socket says 172.21.15.143:51838) 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 -- 172.21.15.143:0/226330885 learned_addr learned my addr 172.21.15.143:0/226330885 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf3ce320 0x5574cf3cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76a600640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf42b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 -- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 msgr2=0x5574cf42b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf42b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 -- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf3ce320 msgr2=0x5574cf3cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf3ce320 0x5574cf3cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 -- 172.21.15.143:0/226330885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75400e850 con 0x5574cf417c90 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769c00640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf3ce320 0x5574cf3cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:40.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769200640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 0x5574cf42afa0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe74c059c50 tx=0x7fe74c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428805f75660693d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 1 -- 172.21.15.143:0/226330885 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c081040 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 1 -- 172.21.15.143:0/226330885 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe74c01bd70 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5574cf21d110 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 1 -- 172.21.15.143:0/226330885 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe74c06d040 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 10 client.7134 fetch_fsmap learned FSMap version 44 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x5574cf1df190 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 1 -- 172.21.15.143:0/226330885 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c077020 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 1 -- 172.21.15.143:0/226330885 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe74c052050 con 0x5574cf417c90 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 10 client.7134 fetch_fsmap finished waiting for FSMap version 44 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 10 client.7134 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 20 client.7134 populate_metadata read hostname 'smithi143' 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf478010 0x5574cf4783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 4 client.7134 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 --> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5574cf24c9a0 con 0x5574cf478010 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769c00640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf478010 0x5574cf4783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe769c00640 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf478010 0x5574cf4783f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39f321e82ad7177 server_cookie=55a039123d865547 in_seq=0 out_seq=0 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 10 client.7134 ms_handle_connect on v2:172.21.15.143:6835/1300573627 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 1 -- 172.21.15.143:0/226330885 <== mds.0 v2:172.21.15.143:6835/1300573627 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5574cf24c9a0 con 0x5574cf478010 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 10 client.7134 handle_command_reply: tid=0 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 client.7134 shutdown 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 msgr2=0x5574cf42afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 0x5574cf42afa0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fe74c059c50 tx=0x7fe74c0046d0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 shutdown_connections 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:6835/1300573627,v1:172.21.15.143:6837/1300573627] conn(0x5574cf478010 0x5574cf4783f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5574cf4185b0 0x5574cf42b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5574cf417c90 0x5574cf42afa0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe752a00640 0 client.7134 ms_handle_reset on v2:172.21.15.143:6835/1300573627 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 --2- 172.21.15.143:0/226330885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574cf3ce320 0x5574cf3cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.693+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 >> 172.21.15.143:0/226330885 conn(0x5574cf3ca2d0 msgr2=0x5574cf3cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 shutdown_connections 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76fa77000 1 -- 172.21.15.143:0/226330885 wait complete. 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76fa77000 20 client.7134 trim_cache size 0 max 16384 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 6996, 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:11:40.702 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8469413589999997, 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 44, 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 73, 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 72, 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 11.773374208 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe73c0671f0 msgr2=0x7fe73c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe73c0671f0 0x7fe73c0696b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe754006b20 tx=0x7fe754008040 comp rx=0 tx=0).stop 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 msgr2=0x7fe7641388b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe7641388b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00c7f0 tx=0x7fe74c00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 shutdown_connections 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe73c0671f0 0x7fe73c0696b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe76415ec80 0x7fe7640ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe764142470 0x7fe7641388b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 --2- 172.21.15.143:0/3570917349 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe764141660 0x7fe764159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 >> 172.21.15.143:0/3570917349 conn(0x7fe76406c3f0 msgr2=0x7fe7640afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 shutdown_connections 2024-01-15T18:11:40.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.697+0000 7fe76c600640 1 -- 172.21.15.143:0/3570917349 wait complete. 2024-01-15T18:11:40.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:11:40.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 Processor -- start 2024-01-15T18:11:40.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 -- start start 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f403415aef0 0x7f403406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40340b29a0 0x7f40340ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403415f7c0 con 0x7f403406c560 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403406c310 con 0x7f403415aef0 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403415ebb0 con 0x7f40340b29a0 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4039400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f403415aef0 0x7f403406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4038a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f403406c560 0x7f40340b2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55708/0 (socket says 172.21.15.143:55708) 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4038a00640 1 -- 172.21.15.143:0/1673272267 learned_addr learned my addr 172.21.15.143:0/1673272267 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403a800640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40340b29a0 0x7f40340ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4039400640 1 -- 172.21.15.143:0/1673272267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40340b29a0 msgr2=0x7f40340ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4039400640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40340b29a0 0x7f40340ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4033e00640 1 -- 172.21.15.143:0/1673272267 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147355529 0 0) 0x7f403415f7c0 con 0x7f403406c560 2024-01-15T18:11:40.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4039400640 1 -- 172.21.15.143:0/1673272267 >> v1:172.21.15.143:6789/0 conn(0x7f403406c560 legacy=0x7f40340b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f4039400640 1 -- 172.21.15.143:0/1673272267 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403415ee90 con 0x7f403415aef0 2024-01-15T18:11:40.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.853+0000 7f403a800640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40340b29a0 0x7f40340ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:40.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f4039400640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403415aef0 0x7f403406b320 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f40240097c0 tx=0x7f402401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e05f632fab606a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f4033e00640 1 -- 172.21.15.143:0/1673272267 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240ac630 con 0x7f403415aef0 2024-01-15T18:11:40.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f4033e00640 1 -- 172.21.15.143:0/1673272267 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240ac790 con 0x7f403415aef0 2024-01-15T18:11:40.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f4033e00640 1 -- 172.21.15.143:0/1673272267 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240acb20 con 0x7f403415aef0 2024-01-15T18:11:40.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f403be00640 1 -- 172.21.15.143:0/1673272267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403415aef0 msgr2=0x7f403406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f403be00640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403415aef0 0x7f403406b320 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f40240097c0 tx=0x7f402401c450 comp rx=0 tx=0).stop 2024-01-15T18:11:40.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f403be00640 1 -- 172.21.15.143:0/1673272267 shutdown_connections 2024-01-15T18:11:40.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f403be00640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40340b29a0 0x7f40340ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f403be00640 1 --2- 172.21.15.143:0/1673272267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403415aef0 0x7f403406b320 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.857+0000 7f403be00640 1 -- 172.21.15.143:0/1673272267 >> 172.21.15.143:0/1673272267 conn(0x7f4034066720 msgr2=0x7f403406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:40.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 -- 172.21.15.143:0/1673272267 shutdown_connections 2024-01-15T18:11:40.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 -- 172.21.15.143:0/1673272267 wait complete. 2024-01-15T18:11:40.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 Processor -- start 2024-01-15T18:11:40.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 -- start start 2024-01-15T18:11:40.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 0x7f40340b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40340b29a0 0x7f4034165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f403415aef0 0x7f403414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403415bb50 con 0x7f403406c560 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4034165860 con 0x7f40340b29a0 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40340b4c70 con 0x7f403415aef0 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 0x7f40340b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 0x7f40340b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51862/0 (socket says 172.21.15.143:51862) 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 -- 172.21.15.143:0/1869374741 learned_addr learned my addr 172.21.15.143:0/1869374741 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4038a00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40340b29a0 0x7f4034165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403a800640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f403415aef0 0x7f403414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 -- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f403415aef0 msgr2=0x7f403414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f403415aef0 0x7f403414d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 -- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40340b29a0 msgr2=0x7f4034165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40340b29a0 0x7f4034165320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 -- 172.21.15.143:0/1869374741 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4024009420 con 0x7f403406c560 2024-01-15T18:11:40.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4038a00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40340b29a0 0x7f4034165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:40.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f403a800640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f403415aef0 0x7f403414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:40.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.861+0000 7f4039400640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 0x7f40340b4690 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f40240af040 tx=0x7f402401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240b8650 con 0x7f403406c560 2024-01-15T18:11:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240b87d0 con 0x7f403406c560 2024-01-15T18:11:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240b8b80 con 0x7f403406c560 2024-01-15T18:11:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40340b4f50 con 0x7f403406c560 2024-01-15T18:11:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40340b5510 con 0x7f403406c560 2024-01-15T18:11:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4034067680 con 0x7f403406c560 2024-01-15T18:11:40.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f40240b8ce0 con 0x7f403406c560 2024-01-15T18:11:40.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4031600640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f400c0671f0 0x7f400c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:40.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f402411d780 con 0x7f403406c560 2024-01-15T18:11:40.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.865+0000 7f4038a00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f400c0671f0 0x7f400c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:40.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.869+0000 7f4038a00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f400c0671f0 0x7f400c0696b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f403414fd20 tx=0x7f401c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:40.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:40.869+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40240ec390 con 0x7f403406c560 2024-01-15T18:11:41.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.053+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f403415bc90 con 0x7f403406c560 2024-01-15T18:11:41.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f4031600640 1 -- 172.21.15.143:0/1869374741 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f40240ef9b0 con 0x7f403406c560 2024-01-15T18:11:41.058 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:11:41.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f400c0671f0 msgr2=0x7f400c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f400c0671f0 0x7f400c0696b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f403414fd20 tx=0x7f401c009040 comp rx=0 tx=0).stop 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 msgr2=0x7f40340b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 0x7f40340b4690 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f40240af040 tx=0x7f402401ac90 comp rx=0 tx=0).stop 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 shutdown_connections 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f400c0671f0 0x7f400c0696b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f403415aef0 0x7f403414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40340b29a0 0x7f4034165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 --2- 172.21.15.143:0/1869374741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f403406c560 0x7f40340b4690 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 >> 172.21.15.143:0/1869374741 conn(0x7f4034066720 msgr2=0x7f4034150030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 shutdown_connections 2024-01-15T18:11:41.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41.057+0000 7f403be00640 1 -- 172.21.15.143:0/1869374741 wait complete. 2024-01-15T18:11:41.081 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:11:41.081 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:11:41.082 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:11:41.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.082 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:11:41.092 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 42042sec preferred_lft 42042sec 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.093 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:11:41.094 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:41.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.095 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:11:41.151 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:11:41.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.153 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:11:41.208 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:11:41.208 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:11:41.208 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:11:41.208 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:11:41.208 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:11:41.208 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:11:41.209 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:11:41.209 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:11:41.209 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:11:41.209 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:11:41.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.209 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:11:41.209 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:11:41.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:11:41.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:11:41.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:11:41.209 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:11:41.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.298 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:11:41.350 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:11:41.350 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:11:41.351 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:11:41.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:11:41.413 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:11:41.413 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:11:41.413 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:41.466 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:11:41.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:11:41.466 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:41.517 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:11:41.577 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:11:41.626 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:11:41.627 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:11:41.628 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:11:41.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:11:41.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:11:41.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:11:41.629 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:11:41.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.630 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:11:41.681 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:11:41.681 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:11:41.724 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:11:41.744 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:11:41.795 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:11:41.796 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:11:41.797 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:11:41.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.798 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:11:41.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:11:41.817+0000 7f896df36580 -1 init, newargv = 0x55e4507615e0 newargc=15 2024-01-15T18:11:41.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[44865]: starting ceph client 2024-01-15T18:11:41.826 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[44865]: starting fuse 2024-01-15T18:11:41.828 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:11:41.828 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:11:41.828 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:11:41.829 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:11:41.830 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:11:41.950 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.44865.asok 2024-01-15T18:11:41.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:11:41.961 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44865.asok 2024-01-15T18:11:41.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:41.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44865.asok status 2024-01-15T18:11:42.131 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:11:42.131 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:11:42.131 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:11:42.131 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:11:42.131 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "44865", 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7149, 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 7149 2024-01-15T18:11:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2932680777 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "172.21.15.143:0", 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2932680777 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.7149 172.21.15.143:0/2932680777", 2024-01-15T18:11:42.133 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "172.21.15.143:0/2932680777", 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 44, 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 74, 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:11:42.134 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:11:42.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.147 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:11:42.147 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:42.156 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:11:42.156 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:42.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.157 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:42.217 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:11:42.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.389+0000 7faa34a00640 1 Processor -- start 2024-01-15T18:11:42.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.389+0000 7faa34a00640 1 -- start start 2024-01-15T18:11:42.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa34a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa2c141660 0x7faa2c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:42.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa34a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c15f310 con 0x7faa2c071d00 2024-01-15T18:11:42.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa34a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2c15f5f0 con 0x7faa2c15ec80 2024-01-15T18:11:42.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa34a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa2c15fbb0 con 0x7faa2c141660 2024-01-15T18:11:42.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa2c141660 0x7faa2c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:42.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faa2c071d00 0x7faa2c0b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55714/0 (socket says 172.21.15.143:55714) 2024-01-15T18:11:42.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32a00640 1 -- 172.21.15.143:0/3746289584 learned_addr learned my addr 172.21.15.143:0/3746289584 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:42.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32000640 1 -- 172.21.15.143:0/3746289584 >> v1:172.21.15.204:6789/0 conn(0x7faa2c15ec80 legacy=0x7faa2c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32000640 1 -- 172.21.15.143:0/3746289584 >> v1:172.21.15.143:6789/0 conn(0x7faa2c071d00 legacy=0x7faa2c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32000640 1 -- 172.21.15.143:0/3746289584 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c15f8d0 con 0x7faa2c141660 2024-01-15T18:11:42.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa30c00640 1 -- 172.21.15.143:0/3746289584 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900827225 0 0) 0x7faa2c15f310 con 0x7faa2c071d00 2024-01-15T18:11:42.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa32000640 1 --2- 172.21.15.143:0/3746289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c0ce560 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0097c0 tx=0x7faa1c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d8f2fd4d106756 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:42.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.393+0000 7faa30c00640 1 -- 172.21.15.143:0/3746289584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0ab670 con 0x7faa2c141660 2024-01-15T18:11:42.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa30c00640 1 -- 172.21.15.143:0/3746289584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1c0ab7d0 con 0x7faa2c141660 2024-01-15T18:11:42.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa30c00640 1 -- 172.21.15.143:0/3746289584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0abb60 con 0x7faa2c141660 2024-01-15T18:11:42.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- 172.21.15.143:0/3746289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 msgr2=0x7faa2c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 --2- 172.21.15.143:0/3746289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c0ce560 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0097c0 tx=0x7faa1c01c450 comp rx=0 tx=0).stop 2024-01-15T18:11:42.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- 172.21.15.143:0/3746289584 shutdown_connections 2024-01-15T18:11:42.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 --2- 172.21.15.143:0/3746289584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c0ce560 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- 172.21.15.143:0/3746289584 >> 172.21.15.143:0/3746289584 conn(0x7faa2c06c3f0 msgr2=0x7faa2c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:42.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- 172.21.15.143:0/3746289584 shutdown_connections 2024-01-15T18:11:42.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- 172.21.15.143:0/3746289584 wait complete. 2024-01-15T18:11:42.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 Processor -- start 2024-01-15T18:11:42.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- start start 2024-01-15T18:11:42.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa2c071d00 0x7faa2c1f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:42.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c13e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:42.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa2c15ec80 0x7faa2c1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:42.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.397+0000 7faa34a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c15f130 con 0x7faa2c141660 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa34a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c1438e0 con 0x7faa2c071d00 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa34a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c15e7c0 con 0x7faa2c15ec80 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c13e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c13e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51884/0 (socket says 172.21.15.143:51884) 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 -- 172.21.15.143:0/2286064725 learned_addr learned my addr 172.21.15.143:0/2286064725 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa32000640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa2c071d00 0x7faa2c1f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa32a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa2c15ec80 0x7faa2c1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 -- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa2c15ec80 msgr2=0x7faa2c1f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa2c15ec80 0x7faa2c1f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 -- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa2c071d00 msgr2=0x7faa2c1f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa2c071d00 0x7faa2c1f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 -- 172.21.15.143:0/2286064725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c009420 con 0x7faa2c141660 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa31600640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c13e270 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c3f0 tx=0x7faa1400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140052d0 con 0x7faa2c141660 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14005430 con 0x7faa2c141660 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14003840 con 0x7faa2c141660 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c0cd150 con 0x7faa2c141660 2024-01-15T18:11:42.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c143290 con 0x7faa2c141660 2024-01-15T18:11:42.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.401+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2c143a90 con 0x7faa2c141660 2024-01-15T18:11:42.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.405+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7faa140039a0 con 0x7faa2c141660 2024-01-15T18:11:42.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.405+0000 7faa2a000640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa04067220 0x7faa040696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:42.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.405+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faa14082510 con 0x7faa2c141660 2024-01-15T18:11:42.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.405+0000 7faa32000640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa04067220 0x7faa040696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:42.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.405+0000 7faa32000640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa04067220 0x7faa040696e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0af000 tx=0x7faa1c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:42.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.409+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa14051250 con 0x7faa2c141660 2024-01-15T18:11:42.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.589+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faa2c143a90 con 0x7faa2c141660 2024-01-15T18:11:42.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa2a000640 1 -- 172.21.15.143:0/2286064725 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7faa2c143a90 con 0x7faa2c141660 2024-01-15T18:11:42.594 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa04067220 msgr2=0x7faa040696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa04067220 0x7faa040696e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0af000 tx=0x7faa1c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 msgr2=0x7faa2c13e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c13e270 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c3f0 tx=0x7faa1400c8b0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 shutdown_connections 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faa04067220 0x7faa040696e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa2c15ec80 0x7faa2c1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa2c141660 0x7faa2c13e270 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 --2- 172.21.15.143:0/2286064725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa2c071d00 0x7faa2c1f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:42.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 >> 172.21.15.143:0/2286064725 conn(0x7faa2c06c3f0 msgr2=0x7faa2c06d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:42.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 shutdown_connections 2024-01-15T18:11:42.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:42.593+0000 7faa34a00640 1 -- 172.21.15.143:0/2286064725 wait complete. 2024-01-15T18:11:42.617 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:11:42.617 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:11:42.617 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:11:42.617 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:11:42.618 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:42.618 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:11:42.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:11:42.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:11:42.618 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:11:42.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.618 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:11:42.629 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:11:42.629 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:42.630 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 42035sec preferred_lft 42035sec 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:11:42.631 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:11:42.632 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:11:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.633 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:11:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:11:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:11:42.633 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:11:42.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.633 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:11:42.694 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:11:42.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.697 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:11:42.750 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:11:42.751 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:11:42.751 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:11:42.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.751 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:11:42.751 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:11:42.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:11:42.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:11:42.751 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:11:42.752 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:11:42.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.846 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:11:42.857 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:11:42.857 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:11:42.857 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:11:42.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:42.857 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:11:42.922 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:11:42.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:11:42.923 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:43.041 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:11:43.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:11:43.042 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:43.050 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:11:43.114 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:11:43.167 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:11:43.167 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:11:43.168 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:11:43.169 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:11:43.170 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:11:43.171 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:11:43.171 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:11:43.171 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:11:43.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:43.172 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:11:43.222 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:11:43.222 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:11:43.269 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:11:43.291 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:11:43.343 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:11:43.343 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:11:43.343 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:11:43.343 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:11:43.344 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:11:43.345 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:11:43.346 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:11:43.346 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:11:43.346 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:11:43.346 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:11:43.346 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:11:43.346 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:11:43.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:43.347 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:11:43.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:11:43.362+0000 7fa684bc6580 -1 init, newargv = 0x55fb2205b340 newargc=15 2024-01-15T18:11:43.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33935]: starting ceph client 2024-01-15T18:11:43.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33935]: starting fuse 2024-01-15T18:11:43.372 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:11:43.372 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:11:43.373 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:11:43.374 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:11:43.375 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:11:43.497 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.33935.asok 2024-01-15T18:11:43.501 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:11:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:11:43.509 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33935.asok 2024-01-15T18:11:43.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:43.509 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33935.asok status 2024-01-15T18:11:43.677 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "33935", 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:11:43.678 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 7012, 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 7012 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:11:43.679 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 697849429 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 697849429 2024-01-15T18:11:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.7012 192.168.0.1:0/697849429", 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/697849429", 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 44, 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 74, 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:11:43.681 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:11:43.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:43.693 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:11:43.693 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:43.703 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:11:43.703 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:43.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:43.703 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:43.764 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:11:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 Processor -- start 2024-01-15T18:11:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 -- start start 2024-01-15T18:11:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f49040ce560 0x7f490415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f4904072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490415f310 con 0x7f4904141660 2024-01-15T18:11:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490415fbb0 con 0x7f49040ce560 2024-01-15T18:11:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4904072690 con 0x7f4904071d00 2024-01-15T18:11:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4904141660 0x7f4904141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55730/0 (socket says 172.21.15.143:55730) 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490a000640 1 -- 172.21.15.143:0/4249944051 learned_addr learned my addr 172.21.15.143:0/4249944051 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f4909600640 1 --2- 172.21.15.143:0/4249944051 >> v2:172.21.15.204:3300/0 conn(0x7f49040ce560 0x7f490415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.913+0000 7f490aa00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f4904072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217665117 0 0) 0x7f490415f310 con 0x7f4904141660 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e4003530 con 0x7f4904141660 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4186082046 0 0) 0x7f48e4003530 con 0x7f4904141660 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 msgr2=0x7f4904072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f4904072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 >> v2:172.21.15.204:3300/0 conn(0x7f49040ce560 msgr2=0x7f490415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 --2- 172.21.15.143:0/4249944051 >> v2:172.21.15.204:3300/0 conn(0x7f49040ce560 0x7f490415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490415f5f0 con 0x7f4904141660 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f48f4001680 con 0x7f4904141660 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4909600640 1 --2- 172.21.15.143:0/4249944051 >> v2:172.21.15.204:3300/0 conn(0x7f49040ce560 0x7f490415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f4904072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 >> v1:172.21.15.143:6789/0 conn(0x7f4904141660 legacy=0x7f4904141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f48e4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4003ec0 0x7f48e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48e4004d10 0x7f48e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490415fbb0 con 0x7f48e4004d10 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4904072690 con 0x7f48e4003ec0 2024-01-15T18:11:43.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e400b990 con 0x7f4904071d00 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48e4004d10 0x7f48e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4909600640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4003ec0 0x7f48e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490a000640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f48e4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 -- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 msgr2=0x7f48e4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4904071d00 0x7f48e4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 -- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4003ec0 msgr2=0x7f48e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4003ec0 0x7f48e40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f490aa00640 1 -- 172.21.15.143:0/4249944051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc009a70 con 0x7f48e4004d10 2024-01-15T18:11:43.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.917+0000 7f4909600640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4003ec0 0x7f48e40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490aa00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48e4004d10 0x7f48e4005190 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00e350 tx=0x7f48fc01cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476506fcd5c88a51 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0a5df0 con 0x7f48e4004d10 2024-01-15T18:11:43.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc00f040 con 0x7f48e4004d10 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f4908c00640 1 -- 172.21.15.143:0/4249944051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0ad640 con 0x7f48e4004d10 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 -- 172.21.15.143:0/4249944051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48e4004d10 msgr2=0x7f48e4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 --2- 172.21.15.143:0/4249944051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48e4004d10 0x7f48e4005190 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00e350 tx=0x7f48fc01cde0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490aa00640 1 -- 172.21.15.143:0/4249944051 reap_dead start 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 -- 172.21.15.143:0/4249944051 shutdown_connections 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 -- 172.21.15.143:0/4249944051 >> 172.21.15.143:0/4249944051 conn(0x7f490406c3f0 msgr2=0x7f490406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 -- 172.21.15.143:0/4249944051 shutdown_connections 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 -- 172.21.15.143:0/4249944051 wait complete. 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 Processor -- start 2024-01-15T18:11:43.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.921+0000 7f490ca00640 1 -- start start 2024-01-15T18:11:43.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 0x7f490415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49041f2d10 0x7f490415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4904143900 0x7f4904143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49041392c0 con 0x7f4904143900 2024-01-15T18:11:43.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49041442c0 con 0x7f48e4004d10 2024-01-15T18:11:43.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490415d690 con 0x7f49041f2d10 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4904143900 0x7f4904143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 0x7f490415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 0x7f490415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58298/0 (socket says 172.21.15.143:58298) 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 -- 172.21.15.143:0/1358583830 learned_addr learned my addr 172.21.15.143:0/1358583830 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4904143900 0x7f4904143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51908/0 (socket says 172.21.15.143:51908) 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f4909600640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49041f2d10 0x7f490415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 -- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49041f2d10 msgr2=0x7f490415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49041f2d10 0x7f490415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 -- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4904143900 msgr2=0x7f4904143d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4904143900 0x7f4904143d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:43.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 -- 172.21.15.143:0/1358583830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4904138360 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490a000640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 0x7f490415abe0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f48f4007310 tx=0x7f48f40077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f400f9c0 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f400fb60 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4904138640 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4012350 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4904138c00 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4904139400 con 0x7f48e4004d10 2024-01-15T18:11:43.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.925+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f48f40124b0 con 0x7f48e4004d10 2024-01-15T18:11:43.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.929+0000 7f4902000640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48dc067220 0x7f48dc0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:43.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.929+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f48f4082c00 con 0x7f48e4004d10 2024-01-15T18:11:43.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.929+0000 7f4909600640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48dc067220 0x7f48dc0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:43.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.929+0000 7f4909600640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48dc067220 0x7f48dc0696e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f490415dff0 tx=0x7f48ec009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:43.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:43.933+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4904139400 con 0x7f48e4004d10 2024-01-15T18:11:44.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.113+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4904139400 con 0x7f48e4004d10 2024-01-15T18:11:44.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f4902000640 1 -- 172.21.15.143:0/1358583830 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 136+0+0 (secure 0 0 0) 0x7f4904139400 con 0x7f48e4004d10 2024-01-15T18:11:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48dc067220 msgr2=0x7f48dc0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48dc067220 0x7f48dc0696e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f490415dff0 tx=0x7f48ec009190 comp rx=0 tx=0).stop 2024-01-15T18:11:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 msgr2=0x7f490415abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 0x7f490415abe0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f48f4007310 tx=0x7f48f40077d0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 shutdown_connections 2024-01-15T18:11:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48dc067220 0x7f48dc0696e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4904143900 0x7f4904143d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49041f2d10 0x7f490415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 --2- 172.21.15.143:0/1358583830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f48e4004d10 0x7f490415abe0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 >> 172.21.15.143:0/1358583830 conn(0x7f490406c3f0 msgr2=0x7f490406d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:44.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 shutdown_connections 2024-01-15T18:11:44.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.673+0000 7f490ca00640 1 -- 172.21.15.143:0/1358583830 wait complete. 2024-01-15T18:11:44.699 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:11:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.833+0000 7f31f2200640 1 Processor -- start 2024-01-15T18:11:44.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.833+0000 7f31f2200640 1 -- start start 2024-01-15T18:11:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31f2200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31ec15b260 0x7f31ec0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31f2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec0d19f0 0x7f31ec0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:44.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31f2200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec0c4d90 con 0x7f31ec0c5220 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31f2200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ec15b640 con 0x7f31ec15b260 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31f2200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0cf800 con 0x7f31ec0d19f0 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31ec15b260 0x7f31ec0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31ec15b260 0x7f31ec0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51918/0 (socket says 172.21.15.143:51918) 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 -- 172.21.15.143:0/3101637665 learned_addr learned my addr 172.21.15.143:0/3101637665 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eaa00640 1 --2- 172.21.15.143:0/3101637665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec0d19f0 0x7f31ec0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 -- 172.21.15.143:0/3101637665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec0d19f0 msgr2=0x7f31ec0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 --2- 172.21.15.143:0/3101637665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec0d19f0 0x7f31ec0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 -- 172.21.15.143:0/3101637665 >> v1:172.21.15.204:6789/0 conn(0x7f31ec0c5220 legacy=0x7f31ec0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:44.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.837+0000 7f31eb400640 1 -- 172.21.15.143:0/3101637665 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0cfae0 con 0x7f31ec15b260 2024-01-15T18:11:44.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31eb400640 1 --2- 172.21.15.143:0/3101637665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec15b260 0x7f31ec0cd810 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009870 tx=0x7f31dc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99ac3b198e3d581 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31ea000640 1 -- 172.21.15.143:0/3101637665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0ab620 con 0x7f31ec15b260 2024-01-15T18:11:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31ea000640 1 -- 172.21.15.143:0/3101637665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc0ab780 con 0x7f31ec15b260 2024-01-15T18:11:44.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31ea000640 1 -- 172.21.15.143:0/3101637665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0abb10 con 0x7f31ec15b260 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 -- 172.21.15.143:0/3101637665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec15b260 msgr2=0x7f31ec0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 --2- 172.21.15.143:0/3101637665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec15b260 0x7f31ec0cd810 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009870 tx=0x7f31dc01c510 comp rx=0 tx=0).stop 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 -- 172.21.15.143:0/3101637665 shutdown_connections 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 --2- 172.21.15.143:0/3101637665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec0d19f0 0x7f31ec0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 --2- 172.21.15.143:0/3101637665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec15b260 0x7f31ec0cd810 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 -- 172.21.15.143:0/3101637665 >> 172.21.15.143:0/3101637665 conn(0x7f31ec0d0ed0 msgr2=0x7f31ec0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 -- 172.21.15.143:0/3101637665 shutdown_connections 2024-01-15T18:11:44.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 -- 172.21.15.143:0/3101637665 wait complete. 2024-01-15T18:11:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 Processor -- start 2024-01-15T18:11:44.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.841+0000 7f31f2200640 1 -- start start 2024-01-15T18:11:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31ec0c5220 0x7f31ec151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 0x7f31ec14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec15b260 0x7f31ec0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec14f3a0 con 0x7f31ec0d19f0 2024-01-15T18:11:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1517b0 con 0x7f31ec0c5220 2024-01-15T18:11:44.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec158630 con 0x7f31ec15b260 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 0x7f31ec14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 0x7f31ec14dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51930/0 (socket says 172.21.15.143:51930) 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31ec0c5220 0x7f31ec151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eaa00640 1 -- 172.21.15.143:0/1944199158 learned_addr learned my addr 172.21.15.143:0/1944199158 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31ebe00640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec15b260 0x7f31ec0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 -- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec15b260 msgr2=0x7f31ec0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec15b260 0x7f31ec0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 -- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 msgr2=0x7f31ec14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 0x7f31ec14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 -- 172.21.15.143:0/1944199158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc0094d0 con 0x7f31ec0c5220 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31ebe00640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec15b260 0x7f31ec0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eaa00640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 0x7f31ec14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31eb400640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31ec0c5220 0x7f31ec151270 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0ac000 tx=0x7f31dc01ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:44.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0be3f0 con 0x7f31ec0c5220 2024-01-15T18:11:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc0be570 con 0x7f31ec0c5220 2024-01-15T18:11:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec165490 con 0x7f31ec0c5220 2024-01-15T18:11:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0be920 con 0x7f31ec0c5220 2024-01-15T18:11:44.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec14ede0 con 0x7f31ec0c5220 2024-01-15T18:11:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec067500 con 0x7f31ec0c5220 2024-01-15T18:11:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.845+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f31dc0b43f0 con 0x7f31ec0c5220 2024-01-15T18:11:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.849+0000 7f31e3400640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31c40671f0 0x7f31c40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.849+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f31dc11cd90 con 0x7f31ec0c5220 2024-01-15T18:11:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.849+0000 7f31eaa00640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31c40671f0 0x7f31c40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:44.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.849+0000 7f31eaa00640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31c40671f0 0x7f31c40696b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0ce5a0 tx=0x7f31d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:44.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:44.853+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31dc0ebbf0 con 0x7f31ec0c5220 2024-01-15T18:11:45.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.021+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f31ec0d1340 con 0x7f31c40671f0 2024-01-15T18:11:45.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.021+0000 7f31e3400640 1 -- 172.21.15.143:0/1944199158 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f31ec0d1340 con 0x7f31c40671f0 2024-01-15T18:11:45.026 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:11:45.026 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:11:45.026 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:11:45.026 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:11:45.026 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:11:45.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31c40671f0 msgr2=0x7f31c40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31c40671f0 0x7f31c40696b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0ce5a0 tx=0x7f31d4008040 comp rx=0 tx=0).stop 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31ec0c5220 msgr2=0x7f31ec151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31ec0c5220 0x7f31ec151270 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0ac000 tx=0x7f31dc01ad50 comp rx=0 tx=0).stop 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 shutdown_connections 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31c40671f0 0x7f31c40696b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31ec15b260 0x7f31ec0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31ec0d19f0 0x7f31ec14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 --2- 172.21.15.143:0/1944199158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31ec0c5220 0x7f31ec151270 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 >> 172.21.15.143:0/1944199158 conn(0x7f31ec0d0ed0 msgr2=0x7f31ec0c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 shutdown_connections 2024-01-15T18:11:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.025+0000 7f31f2200640 1 -- 172.21.15.143:0/1944199158 wait complete. 2024-01-15T18:11:45.048 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:11:45.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96e2600640 1 Processor -- start 2024-01-15T18:11:45.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96e2600640 1 -- start start 2024-01-15T18:11:45.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96e2600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f96dc0cfb30 0x7f96dc06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96e2600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0cffd0 con 0x7f96dc0cbf50 2024-01-15T18:11:45.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96e2600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc0d02b0 con 0x7f96dc1607c0 2024-01-15T18:11:45.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96e2600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc0d1ce0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.185+0000 7f96dbe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f96dc0cbf50 0x7f96dc0cc3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55742/0 (socket says 172.21.15.143:55742) 2024-01-15T18:11:45.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96dbe00640 1 -- 172.21.15.143:0/2919960106 learned_addr learned my addr 172.21.15.143:0/2919960106 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:45.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96daa00640 1 --2- 172.21.15.143:0/2919960106 >> v2:172.21.15.204:3300/0 conn(0x7f96dc0cfb30 0x7f96dc06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96da000640 1 -- 172.21.15.143:0/2919960106 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213118766 0 0) 0x7f96dc0cffd0 con 0x7f96dc0cbf50 2024-01-15T18:11:45.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96da000640 1 -- 172.21.15.143:0/2919960106 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc003530 con 0x7f96dc0cbf50 2024-01-15T18:11:45.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96da000640 1 -- 172.21.15.143:0/2919960106 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489944020 0 0) 0x7f96dc0d02b0 con 0x7f96dc1607c0 2024-01-15T18:11:45.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96daa00640 1 -- 172.21.15.143:0/2919960106 >> v1:172.21.15.204:6789/0 conn(0x7f96dc1607c0 legacy=0x7f96dc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96daa00640 1 -- 172.21.15.143:0/2919960106 >> v1:172.21.15.143:6789/0 conn(0x7f96dc0cbf50 legacy=0x7f96dc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96daa00640 1 -- 172.21.15.143:0/2919960106 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc0cb470 con 0x7f96dc0cfb30 2024-01-15T18:11:45.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96daa00640 1 --2- 172.21.15.143:0/2919960106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cfb30 0x7f96dc06bec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96c40097c0 tx=0x7f96c401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5abe48659a45e74 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96da000640 1 -- 172.21.15.143:0/2919960106 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40a8970 con 0x7f96dc0cfb30 2024-01-15T18:11:45.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96da000640 1 -- 172.21.15.143:0/2919960106 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c40a8ad0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.189+0000 7f96da000640 1 -- 172.21.15.143:0/2919960106 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40a8e60 con 0x7f96dc0cfb30 2024-01-15T18:11:45.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- 172.21.15.143:0/2919960106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cfb30 msgr2=0x7f96dc06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 --2- 172.21.15.143:0/2919960106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cfb30 0x7f96dc06bec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96c40097c0 tx=0x7f96c401ec70 comp rx=0 tx=0).stop 2024-01-15T18:11:45.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- 172.21.15.143:0/2919960106 shutdown_connections 2024-01-15T18:11:45.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 --2- 172.21.15.143:0/2919960106 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cfb30 0x7f96dc06bec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- 172.21.15.143:0/2919960106 >> 172.21.15.143:0/2919960106 conn(0x7f96dc0d0e20 msgr2=0x7f96dc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- 172.21.15.143:0/2919960106 shutdown_connections 2024-01-15T18:11:45.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- 172.21.15.143:0/2919960106 wait complete. 2024-01-15T18:11:45.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 Processor -- start 2024-01-15T18:11:45.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- start start 2024-01-15T18:11:45.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cbf50 0x7f96dc158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 0x7f96dc15a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96dc1607c0 0x7f96dc15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0ca9f0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1587a0 con 0x7f96dc0cbf50 2024-01-15T18:11:45.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96e2600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1f35d0 con 0x7f96dc1607c0 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96daa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 0x7f96dc15a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96daa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 0x7f96dc15a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51940/0 (socket says 172.21.15.143:51940) 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96daa00640 1 -- 172.21.15.143:0/910441606 learned_addr learned my addr 172.21.15.143:0/910441606 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96db400640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cbf50 0x7f96dc158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.193+0000 7f96dbe00640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96dc1607c0 0x7f96dc15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96daa00640 1 -- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96dc1607c0 msgr2=0x7f96dc15edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96daa00640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96dc1607c0 0x7f96dc15edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96daa00640 1 -- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cbf50 msgr2=0x7f96dc158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96daa00640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cbf50 0x7f96dc158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96daa00640 1 -- 172.21.15.143:0/910441606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4009420 con 0x7f96dc0cfb30 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96daa00640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 0x7f96dc15a870 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f96c40b3040 tx=0x7f96c401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40bf070 con 0x7f96dc0cfb30 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c40b0bf0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40b9410 con 0x7f96dc0cfb30 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc1f38b0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc15a070 con 0x7f96dc0cfb30 2024-01-15T18:11:45.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc06d4b0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.197+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f96c40b7040 con 0x7f96dc0cfb30 2024-01-15T18:11:45.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.201+0000 7f96d3400640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96b40671f0 0x7f96b40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.201+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f96c4120060 con 0x7f96dc0cfb30 2024-01-15T18:11:45.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.201+0000 7f96db400640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96b40671f0 0x7f96b40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.201+0000 7f96db400640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96b40671f0 0x7f96b40696b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f96cc007fd0 tx=0x7f96cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.205+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96c40eebb0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f96dc067500 con 0x7f96dc0cfb30 2024-01-15T18:11:45.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96d3400640 1 -- 172.21.15.143:0/910441606 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f96c40b50c0 con 0x7f96dc0cfb30 2024-01-15T18:11:45.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96b40671f0 msgr2=0x7f96b40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96b40671f0 0x7f96b40696b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f96cc007fd0 tx=0x7f96cc009040 comp rx=0 tx=0).stop 2024-01-15T18:11:45.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 msgr2=0x7f96dc15a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 0x7f96dc15a870 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f96c40b3040 tx=0x7f96c401ba10 comp rx=0 tx=0).stop 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 shutdown_connections 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96b40671f0 0x7f96b40696b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96dc1607c0 0x7f96dc15edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96dc0cfb30 0x7f96dc15a870 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 --2- 172.21.15.143:0/910441606 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96dc0cbf50 0x7f96dc158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.353+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 >> 172.21.15.143:0/910441606 conn(0x7f96dc0d0e20 msgr2=0x7f96dc15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.357+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 shutdown_connections 2024-01-15T18:11:45.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.357+0000 7f96e2600640 1 -- 172.21.15.143:0/910441606 wait complete. 2024-01-15T18:11:45.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:11:45.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 Processor -- start 2024-01-15T18:11:45.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 -- start start 2024-01-15T18:11:45.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efd4c06aba0 0x7efd4c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 0x7efd4c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4c0ae830 con 0x7efd4c0ae450 2024-01-15T18:11:45.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c0ad900 con 0x7efd4c06aba0 2024-01-15T18:11:45.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd53c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c15e6c0 con 0x7efd4c15de90 2024-01-15T18:11:45.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 0x7efd4c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd4be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efd4c06aba0 0x7efd4c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 0x7efd4c0ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:32948/0 (socket says 172.21.15.143:32948) 2024-01-15T18:11:45.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 -- 172.21.15.143:0/129486028 learned_addr learned my addr 172.21.15.143:0/129486028 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:45.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51200640 1 --1- 172.21.15.143:0/129486028 >> v1:172.21.15.204:6789/0 conn(0x7efd4c0ae450 0x7efd4c06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58466/0 (socket says 172.21.15.143:58466) 2024-01-15T18:11:45.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 -- 172.21.15.143:0/129486028 >> v2:172.21.15.204:3300/0 conn(0x7efd4c06aba0 msgr2=0x7efd4c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 --2- 172.21.15.143:0/129486028 >> v2:172.21.15.204:3300/0 conn(0x7efd4c06aba0 0x7efd4c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 -- 172.21.15.143:0/129486028 >> v1:172.21.15.204:6789/0 conn(0x7efd4c0ae450 legacy=0x7efd4c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.517+0000 7efd51c00640 1 -- 172.21.15.143:0/129486028 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c160e20 con 0x7efd4c15de90 2024-01-15T18:11:45.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd51c00640 1 --2- 172.21.15.143:0/129486028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 0x7efd4c0ca370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd44009e50 tx=0x7efd4401c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a9a383afb68d94 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd4b400640 1 -- 172.21.15.143:0/129486028 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440a58a0 con 0x7efd4c15de90 2024-01-15T18:11:45.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd4b400640 1 -- 172.21.15.143:0/129486028 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd440a5a00 con 0x7efd4c15de90 2024-01-15T18:11:45.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd4b400640 1 -- 172.21.15.143:0/129486028 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440a5d90 con 0x7efd4c15de90 2024-01-15T18:11:45.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd53c00640 1 -- 172.21.15.143:0/129486028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 msgr2=0x7efd4c0ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd53c00640 1 --2- 172.21.15.143:0/129486028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 0x7efd4c0ca370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd44009e50 tx=0x7efd4401c9e0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.521+0000 7efd53c00640 1 -- 172.21.15.143:0/129486028 shutdown_connections 2024-01-15T18:11:45.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 --2- 172.21.15.143:0/129486028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c15de90 0x7efd4c0ca370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 --2- 172.21.15.143:0/129486028 >> v2:172.21.15.204:3300/0 conn(0x7efd4c06aba0 0x7efd4c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- 172.21.15.143:0/129486028 >> 172.21.15.143:0/129486028 conn(0x7efd4c0cfd20 msgr2=0x7efd4c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- 172.21.15.143:0/129486028 shutdown_connections 2024-01-15T18:11:45.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- 172.21.15.143:0/129486028 wait complete. 2024-01-15T18:11:45.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 Processor -- start 2024-01-15T18:11:45.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- start start 2024-01-15T18:11:45.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 0x7efd4c14f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c0ae450 0x7efd4c160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd4c15de90 0x7efd4c1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c14f930 con 0x7efd4c06aba0 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c152aa0 con 0x7efd4c15de90 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd53c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c152d80 con 0x7efd4c0ae450 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 0x7efd4c14f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 0x7efd4c14f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51944/0 (socket says 172.21.15.143:51944) 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 -- 172.21.15.143:0/849296395 learned_addr learned my addr 172.21.15.143:0/849296395 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd4c15de90 0x7efd4c1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd4be00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c0ae450 0x7efd4c160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 -- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c0ae450 msgr2=0x7efd4c160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c0ae450 0x7efd4c160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 -- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd4c15de90 msgr2=0x7efd4c1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd4c15de90 0x7efd4c1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 -- 172.21.15.143:0/849296395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44009b30 con 0x7efd4c06aba0 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.525+0000 7efd51200640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 0x7efd4c14f3f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c490 tx=0x7efd3c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c007480 con 0x7efd4c06aba0 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c0075e0 con 0x7efd4c06aba0 2024-01-15T18:11:45.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c004a10 con 0x7efd4c06aba0 2024-01-15T18:11:45.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c153060 con 0x7efd4c06aba0 2024-01-15T18:11:45.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c153620 con 0x7efd4c06aba0 2024-01-15T18:11:45.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c067500 con 0x7efd4c06aba0 2024-01-15T18:11:45.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.529+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7efd3c004b70 con 0x7efd4c06aba0 2024-01-15T18:11:45.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.533+0000 7efd48c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efd240671f0 0x7efd240696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.533+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efd3c081dc0 con 0x7efd4c06aba0 2024-01-15T18:11:45.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.533+0000 7efd4be00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efd240671f0 0x7efd240696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.533+0000 7efd4be00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efd240671f0 0x7efd240696b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efd4c071470 tx=0x7efd34009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.537+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efd3c050b60 con 0x7efd4c06aba0 2024-01-15T18:11:45.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.709+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7efd4c161be0 con 0x7efd240671f0 2024-01-15T18:11:45.715 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:45.713+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:45.715 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:45.713+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:45.715 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:45.713+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:45.715 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:45.713+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:45.715 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:11:45.713+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:45.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.741+0000 7efd48c00640 1 -- 172.21.15.143:0/849296395 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efd34011a30 con 0x7efd240671f0 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efd240671f0 msgr2=0x7efd240696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efd240671f0 0x7efd240696b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efd4c071470 tx=0x7efd34009190 comp rx=0 tx=0).stop 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 msgr2=0x7efd4c14f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 0x7efd4c14f3f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c490 tx=0x7efd3c00c950 comp rx=0 tx=0).stop 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 shutdown_connections 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efd240671f0 0x7efd240696b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd4c15de90 0x7efd4c1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efd4c0ae450 0x7efd4c160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 --2- 172.21.15.143:0/849296395 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd4c06aba0 0x7efd4c14f3f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 >> 172.21.15.143:0/849296395 conn(0x7efd4c0cfd20 msgr2=0x7efd4c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 shutdown_connections 2024-01-15T18:11:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.745+0000 7efd53c00640 1 -- 172.21.15.143:0/849296395 wait complete. 2024-01-15T18:11:45.766 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:11:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f673a200640 1 Processor -- start 2024-01-15T18:11:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f673a200640 1 -- start start 2024-01-15T18:11:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f673a200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f67340cde50 0x7f67340d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f673a200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67340ce7c0 con 0x7f67340cf410 2024-01-15T18:11:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f673a200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67340ce290 con 0x7f67340c5600 2024-01-15T18:11:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f673a200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67340cef50 con 0x7f67340cde50 2024-01-15T18:11:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6733400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f67340cf410 0x7f67340cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55754/0 (socket says 172.21.15.143:55754) 2024-01-15T18:11:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6733400640 1 -- 172.21.15.143:0/934944689 learned_addr learned my addr 172.21.15.143:0/934944689 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 --2- 172.21.15.143:0/934944689 >> v2:172.21.15.204:3300/0 conn(0x7f67340cde50 0x7f67340d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 710041219 0 0) 0x7f6724001120 con 0x7f67340cf410 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6714003530 con 0x7f67340cf410 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543380555 0 0) 0x7f6714003530 con 0x7f67340cf410 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 >> v2:172.21.15.204:3300/0 conn(0x7f67340cde50 msgr2=0x7f67340d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 --2- 172.21.15.143:0/934944689 >> v2:172.21.15.204:3300/0 conn(0x7f67340cde50 0x7f67340d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 >> v1:172.21.15.204:6789/0 conn(0x7f67340c5600 legacy=0x7f67340c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67340ce4e0 con 0x7f67340cf410 2024-01-15T18:11:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f6724001810 con 0x7f67340cf410 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 >> v1:172.21.15.143:6789/0 conn(0x7f67340cf410 legacy=0x7f67340cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6714003980 0x7f6714003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6714004380 0x7f6714004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714005050 0x7f671400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340cef50 con 0x7f6714004380 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671400bfb0 con 0x7f6714003980 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671400c170 con 0x7f6714005050 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6714004380 0x7f6714004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6733400640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6714003980 0x7f6714003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6733e00640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714005050 0x7f671400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 -- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714005050 msgr2=0x7f671400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714005050 0x7f671400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 -- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6714003980 msgr2=0x7f6714003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6714003980 0x7f6714003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6732a00640 1 -- 172.21.15.143:0/934944689 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6714003530 con 0x7f6714004380 2024-01-15T18:11:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.905+0000 7f6733e00640 1 -- 172.21.15.143:0/934944689 reap_dead start 2024-01-15T18:11:45.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f6732a00640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6714004380 0x7f6714004850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f671c009ae0 tx=0x7f671c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2d957796ac6dfb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0a5640 con 0x7f6714004380 2024-01-15T18:11:45.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0a57a0 con 0x7f6714004380 2024-01-15T18:11:45.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f6732000640 1 -- 172.21.15.143:0/934944689 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0a5b50 con 0x7f6714004380 2024-01-15T18:11:45.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f673a200640 1 -- 172.21.15.143:0/934944689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6714004380 msgr2=0x7f6714004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f673a200640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6714004380 0x7f6714004850 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f671c009ae0 tx=0x7f671c01ca50 comp rx=0 tx=0).stop 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f673a200640 1 -- 172.21.15.143:0/934944689 shutdown_connections 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f673a200640 1 --2- 172.21.15.143:0/934944689 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6714004380 0x7f6714004850 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.909+0000 7f673a200640 1 -- 172.21.15.143:0/934944689 >> 172.21.15.143:0/934944689 conn(0x7f67340d0e30 msgr2=0x7f67340d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 -- 172.21.15.143:0/934944689 shutdown_connections 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 -- 172.21.15.143:0/934944689 wait complete. 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 Processor -- start 2024-01-15T18:11:45.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 -- start start 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714004380 0x7f67340ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673409ddd0 0x7f673406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 0x7f67340cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67340cad70 con 0x7f67340ccc40 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673409acf0 con 0x7f673409ddd0 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f673a200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673409e540 con 0x7f6714004380 2024-01-15T18:11:45.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 0x7f67340cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 0x7f67340cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51990/0 (socket says 172.21.15.143:51990) 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 -- 172.21.15.143:0/3124952698 learned_addr learned my addr 172.21.15.143:0/3124952698 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733400640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714004380 0x7f67340ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6732a00640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673409ddd0 0x7f673406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 -- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714004380 msgr2=0x7f67340ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714004380 0x7f67340ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 -- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673409ddd0 msgr2=0x7f673406f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673409ddd0 0x7f673406f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 -- 172.21.15.143:0/3124952698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673409e820 con 0x7f67340ccc40 2024-01-15T18:11:45.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.913+0000 7f6733e00640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 0x7f67340cd0d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f67340d0730 tx=0x7f672c00f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00fda0 con 0x7f67340ccc40 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c015410 con 0x7f67340ccc40 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c0157c0 con 0x7f67340ccc40 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673409af60 con 0x7f67340ccc40 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673409eb00 con 0x7f67340ccc40 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67340b2110 con 0x7f67340ccc40 2024-01-15T18:11:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 21) v1 ==== 86976+0+0 (secure 0 0 0) 0x7f672c015920 con 0x7f67340ccc40 2024-01-15T18:11:45.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f672b400640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f670c067220 0x7f670c0696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f672c0861e0 con 0x7f67340ccc40 2024-01-15T18:11:45.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.917+0000 7f6733400640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f670c067220 0x7f670c0696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.921+0000 7f6733400640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f670c067220 0x7f670c0696e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6724001060 tx=0x7f6724000f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:45.921+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f672c0587c0 con 0x7f67340ccc40 2024-01-15T18:11:46.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.097+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f67340cb5d0 con 0x7f670c067220 2024-01-15T18:11:46.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f672b400640 1 -- 172.21.15.143:0/3124952698 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f67340cb5d0 con 0x7f670c067220 2024-01-15T18:11:46.103 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a 2024-01-15T18:11:46.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f670c067220 msgr2=0x7f670c0696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f670c067220 0x7f670c0696e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6724001060 tx=0x7f6724000f90 comp rx=0 tx=0).stop 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 msgr2=0x7f67340cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 0x7f67340cd0d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f67340d0730 tx=0x7f672c00f540 comp rx=0 tx=0).stop 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 shutdown_connections 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f670c067220 0x7f670c0696e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67340ccc40 0x7f67340cd0d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f673409ddd0 0x7f673406f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 --2- 172.21.15.143:0/3124952698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6714004380 0x7f67340ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 >> 172.21.15.143:0/3124952698 conn(0x7f67340d0e30 msgr2=0x7f67340d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:46.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 shutdown_connections 2024-01-15T18:11:46.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:11:46.101+0000 7f673a200640 1 -- 172.21.15.143:0/3124952698 wait complete. 2024-01-15T18:11:46.125 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a 2024-01-15T18:11:46.126 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:11:46.126 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:46.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:46.136 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:11:53.315 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:11:53.315 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:11:53.315 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.11835 s, 151 MB/s 2024-01-15T18:11:53.317 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:11:53.317 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:53.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:53.326 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:11:58.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:58.505 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:00.234 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:12:00.234 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:12:00.234 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85328 s, 157 MB/s 2024-01-15T18:12:00.235 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:12:00.235 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:00.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:00.244 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:12:07.142 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:12:07.142 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:12:07.142 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.84096 s, 157 MB/s 2024-01-15T18:12:07.143 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 10 2024-01-15T18:12:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 Processor -- start 2024-01-15T18:12:07.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 -- start start 2024-01-15T18:12:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf815d1f0 0x7f6cf806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6cf806bb70 0x7f6cf8070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf80b3370 con 0x7f6cf80712a0 2024-01-15T18:12:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf80716f0 con 0x7f6cf806bb70 2024-01-15T18:12:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cffe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf815d5d0 con 0x7f6cf815d1f0 2024-01-15T18:12:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6cf806bb70 0x7f6cf8070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6cf806bb70 0x7f6cf8070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43874/0 (socket says 172.21.15.143:43874) 2024-01-15T18:12:07.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 -- 172.21.15.143:0/3479236179 learned_addr learned my addr 172.21.15.143:0/3479236179 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfd400640 1 --2- 172.21.15.143:0/3479236179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf815d1f0 0x7f6cf806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 -- 172.21.15.143:0/3479236179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf815d1f0 msgr2=0x7f6cf806b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 --2- 172.21.15.143:0/3479236179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf815d1f0 0x7f6cf806b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 -- 172.21.15.143:0/3479236179 >> v1:172.21.15.204:6789/0 conn(0x7f6cf80712a0 legacy=0x7f6cf806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.281+0000 7f6cfca00640 1 -- 172.21.15.143:0/3479236179 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf806a800 con 0x7f6cf806bb70 2024-01-15T18:12:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cfca00640 1 --2- 172.21.15.143:0/3479236179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf806bb70 0x7f6cf8070d60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009870 tx=0x7f6ce001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65188d0249427885 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cf7e00640 1 -- 172.21.15.143:0/3479236179 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00aa670 con 0x7f6cf806bb70 2024-01-15T18:12:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cf7e00640 1 -- 172.21.15.143:0/3479236179 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce00aa7d0 con 0x7f6cf806bb70 2024-01-15T18:12:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cf7e00640 1 -- 172.21.15.143:0/3479236179 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00aab60 con 0x7f6cf806bb70 2024-01-15T18:12:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 -- 172.21.15.143:0/3479236179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf806bb70 msgr2=0x7f6cf8070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 --2- 172.21.15.143:0/3479236179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf806bb70 0x7f6cf8070d60 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009870 tx=0x7f6ce001c510 comp rx=0 tx=0).stop 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 -- 172.21.15.143:0/3479236179 shutdown_connections 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 --2- 172.21.15.143:0/3479236179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf806bb70 0x7f6cf8070d60 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 --2- 172.21.15.143:0/3479236179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf815d1f0 0x7f6cf806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 -- 172.21.15.143:0/3479236179 >> 172.21.15.143:0/3479236179 conn(0x7f6cf80b1610 msgr2=0x7f6cf80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 -- 172.21.15.143:0/3479236179 shutdown_connections 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.285+0000 7f6cffe00640 1 -- 172.21.15.143:0/3479236179 wait complete. 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 Processor -- start 2024-01-15T18:12:07.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 -- start start 2024-01-15T18:12:07.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf806bb70 0x7f6cf8153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 0x7f6cf814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cf815d1f0 0x7f6cf8159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8155610 con 0x7f6cf80712a0 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8150370 con 0x7f6cf815d1f0 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf8150650 con 0x7f6cf806bb70 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 0x7f6cf814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 0x7f6cf814fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43880/0 (socket says 172.21.15.143:43880) 2024-01-15T18:12:07.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 -- 172.21.15.143:0/4158013987 learned_addr learned my addr 172.21.15.143:0/4158013987 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfde00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cf815d1f0 0x7f6cf8159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfd400640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf806bb70 0x7f6cf8153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 -- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf806bb70 msgr2=0x7f6cf8153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf806bb70 0x7f6cf8153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 -- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cf815d1f0 msgr2=0x7f6cf8159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cf815d1f0 0x7f6cf8159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 -- 172.21.15.143:0/4158013987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00094d0 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cfca00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 0x7f6cf814fb40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6ce001a140 tx=0x7f6ce001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00b3800 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce00b3960 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00b3d10 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf8150930 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.289+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf8150080 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.293+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf8074b00 con 0x7f6cf80712a0 2024-01-15T18:12:07.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.293+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 22) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f6ce00b2380 con 0x7f6cf80712a0 2024-01-15T18:12:07.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.293+0000 7f6cf5600640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cd0067280 0x7f6cd0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.293+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6ce011d870 con 0x7f6cf80712a0 2024-01-15T18:12:07.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.293+0000 7f6cfd400640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cd0067280 0x7f6cd0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.293+0000 7f6cfd400640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cd0067280 0x7f6cd0069740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80b48a0 tx=0x7f6ce8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.297+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ce00ec3b0 con 0x7f6cf80712a0 2024-01-15T18:12:07.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.449+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f6cf8150b20 con 0x7f6cf80712a0 2024-01-15T18:12:07.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.453+0000 7f6cf5600640 1 -- 172.21.15.143:0/4158013987 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v27)=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6cf8150b20 con 0x7f6cf80712a0 2024-01-15T18:12:07.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cd0067280 msgr2=0x7f6cd0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cd0067280 0x7f6cd0069740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80b48a0 tx=0x7f6ce8008040 comp rx=0 tx=0).stop 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 msgr2=0x7f6cf814fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 0x7f6cf814fb40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6ce001a140 tx=0x7f6ce001ad50 comp rx=0 tx=0).stop 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 shutdown_connections 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cd0067280 0x7f6cd0069740 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6cf815d1f0 0x7f6cf8159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cf80712a0 0x7f6cf814fb40 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 --2- 172.21.15.143:0/4158013987 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6cf806bb70 0x7f6cf8153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 >> 172.21.15.143:0/4158013987 conn(0x7f6cf80b1610 msgr2=0x7f6cf8151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 shutdown_connections 2024-01-15T18:12:07.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.457+0000 7f6cffe00640 1 -- 172.21.15.143:0/4158013987 wait complete. 2024-01-15T18:12:07.478 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:12:07.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 Processor -- start 2024-01-15T18:12:07.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 -- start start 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa1780ade20 0x7fa1780ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780725b0 0x7fa17815b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17815b540 con 0x7fa1780ae740 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780b03d0 con 0x7fa1780ade20 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa17e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780af950 con 0x7fa1780725b0 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa177400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa1780ade20 0x7fa1780ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa177e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa1780ae740 0x7fa17815bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46738/0 (socket says 172.21.15.143:46738) 2024-01-15T18:12:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.617+0000 7fa177e00640 1 -- 172.21.15.143:0/573123222 learned_addr learned my addr 172.21.15.143:0/573123222 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:07.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa176a00640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780725b0 0x7fa17815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa177400640 1 -- 172.21.15.143:0/573123222 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780725b0 msgr2=0x7fa17815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa177400640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780725b0 0x7fa17815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa177400640 1 -- 172.21.15.143:0/573123222 >> v1:172.21.15.143:6789/0 conn(0x7fa1780ae740 legacy=0x7fa17815bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa177400640 1 -- 172.21.15.143:0/573123222 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780afc30 con 0x7fa1780ade20 2024-01-15T18:12:07.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa176000640 1 -- 172.21.15.143:0/573123222 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210125302 0 0) 0x7fa17815b540 con 0x7fa1780ae740 2024-01-15T18:12:07.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa176a00640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780725b0 0x7fa17815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa177400640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780ade20 0x7fa1780ae200 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa1680097c0 tx=0x7fa17815cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c180061a72409dd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa176000640 1 -- 172.21.15.143:0/573123222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680ac620 con 0x7fa1780ade20 2024-01-15T18:12:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa176000640 1 -- 172.21.15.143:0/573123222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1680ac780 con 0x7fa1780ade20 2024-01-15T18:12:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa176000640 1 -- 172.21.15.143:0/573123222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680acb10 con 0x7fa1780ade20 2024-01-15T18:12:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa17e000640 1 -- 172.21.15.143:0/573123222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780ade20 msgr2=0x7fa1780ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.621+0000 7fa17e000640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780ade20 0x7fa1780ae200 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa1680097c0 tx=0x7fa17815cc80 comp rx=0 tx=0).stop 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- 172.21.15.143:0/573123222 shutdown_connections 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780725b0 0x7fa17815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 --2- 172.21.15.143:0/573123222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780ade20 0x7fa1780ae200 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- 172.21.15.143:0/573123222 >> 172.21.15.143:0/573123222 conn(0x7fa178071a90 msgr2=0x7fa178071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- 172.21.15.143:0/573123222 shutdown_connections 2024-01-15T18:12:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- 172.21.15.143:0/573123222 wait complete. 2024-01-15T18:12:07.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 Processor -- start 2024-01-15T18:12:07.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- start start 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 0x7fa17809b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780ade20 0x7fa1781f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1780ae740 0x7fa1781f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17809c270 con 0x7fa1780725b0 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17809b9d0 con 0x7fa1780ae740 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa17e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17809bcb0 con 0x7fa1780ade20 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 0x7fa17809b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 0x7fa17809b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43900/0 (socket says 172.21.15.143:43900) 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 -- 172.21.15.143:0/4281859853 learned_addr learned my addr 172.21.15.143:0/4281859853 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa176a00640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780ade20 0x7fa1781f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177e00640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1780ae740 0x7fa1781f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 -- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780ade20 msgr2=0x7fa1781f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780ade20 0x7fa1781f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 -- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1780ae740 msgr2=0x7fa1781f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1780ae740 0x7fa1781f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.625+0000 7fa177400640 1 -- 172.21.15.143:0/4281859853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009420 con 0x7fa1780725b0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa177400640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 0x7fa17809b180 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa168019ff0 tx=0x7fa168019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680b5610 con 0x7fa1780725b0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1680b5770 con 0x7fa1780725b0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17809bf90 con 0x7fa1780725b0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680b5b20 con 0x7fa1780725b0 2024-01-15T18:12:07.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780b3630 con 0x7fa1780725b0 2024-01-15T18:12:07.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1781553f0 con 0x7fa1780725b0 2024-01-15T18:12:07.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 22) v1 ==== 87027+0+0 (secure 0 0 0) 0x7fa1680b3040 con 0x7fa1780725b0 2024-01-15T18:12:07.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa16f400640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa150067280 0x7fa150069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa16811d1d0 con 0x7fa1780725b0 2024-01-15T18:12:07.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa176a00640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa150067280 0x7fa150069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.629+0000 7fa176a00640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa150067280 0x7fa150069740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa1781f8e20 tx=0x7fa160008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.633+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa1781553f0 con 0x7fa1780725b0 2024-01-15T18:12:07.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.809+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa178067500 con 0x7fa150067280 2024-01-15T18:12:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa16f400640 1 -- 172.21.15.143:0/4281859853 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa178067500 con 0x7fa150067280 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa150067280 msgr2=0x7fa150069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa150067280 0x7fa150069740 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa1781f8e20 tx=0x7fa160008040 comp rx=0 tx=0).stop 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 msgr2=0x7fa17809b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 0x7fa17809b180 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa168019ff0 tx=0x7fa168019160 comp rx=0 tx=0).stop 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 shutdown_connections 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa150067280 0x7fa150069740 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1780ae740 0x7fa1781f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1780ade20 0x7fa1781f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 --2- 172.21.15.143:0/4281859853 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1780725b0 0x7fa17809b180 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 >> 172.21.15.143:0/4281859853 conn(0x7fa178071a90 msgr2=0x7fa17809c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 shutdown_connections 2024-01-15T18:12:07.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.821+0000 7fa17e000640 1 -- 172.21.15.143:0/4281859853 wait complete. 2024-01-15T18:12:07.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.981+0000 7fad9d200640 1 Processor -- start 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.981+0000 7fad9d200640 1 -- start start 2024-01-15T18:12:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad9d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad98157300 0x7fad98159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad9d200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad980b4ac0 0x7fad98070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad9d200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad980b2a20 con 0x7fad980b21d0 2024-01-15T18:12:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad9d200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad980b35a0 con 0x7fad980b4ac0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad9d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad980b2d00 con 0x7fad98157300 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad95400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fad980b21d0 0x7fad980b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46752/0 (socket says 172.21.15.143:46752) 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad95400640 1 -- 172.21.15.143:0/694054140 learned_addr learned my addr 172.21.15.143:0/694054140 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad96800640 1 --2- 172.21.15.143:0/694054140 >> v2:172.21.15.204:3300/0 conn(0x7fad980b4ac0 0x7fad98070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad95e00640 1 --2- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad98157300 0x7fad98159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad94a00640 1 -- 172.21.15.143:0/694054140 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249832781 0 0) 0x7fad980b2a20 con 0x7fad980b21d0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad94a00640 1 -- 172.21.15.143:0/694054140 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad78004640 con 0x7fad980b21d0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad96800640 1 -- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad98157300 msgr2=0x7fad98159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad96800640 1 --2- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad98157300 0x7fad98159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad94a00640 1 -- 172.21.15.143:0/694054140 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448432427 0 0) 0x7fad78004640 con 0x7fad980b21d0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad96800640 1 -- 172.21.15.143:0/694054140 >> v1:172.21.15.143:6789/0 conn(0x7fad980b21d0 legacy=0x7fad980b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.985+0000 7fad96800640 1 -- 172.21.15.143:0/694054140 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad980b2fe0 con 0x7fad980b4ac0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad96800640 1 --2- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad980b4ac0 0x7fad98070e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fad90009d90 tx=0x7fad9001c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d19c951b3793524d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad94a00640 1 -- 172.21.15.143:0/694054140 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80001680 con 0x7fad980b4ac0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad94a00640 1 -- 172.21.15.143:0/694054140 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad900a5660 con 0x7fad980b4ac0 2024-01-15T18:12:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad94a00640 1 -- 172.21.15.143:0/694054140 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad900a59f0 con 0x7fad980b4ac0 2024-01-15T18:12:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 -- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad980b4ac0 msgr2=0x7fad98070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 --2- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad980b4ac0 0x7fad98070e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fad90009d90 tx=0x7fad9001c840 comp rx=0 tx=0).stop 2024-01-15T18:12:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 -- 172.21.15.143:0/694054140 shutdown_connections 2024-01-15T18:12:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 --2- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad980b4ac0 0x7fad98070e80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 --2- 172.21.15.143:0/694054140 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad98157300 0x7fad98159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 -- 172.21.15.143:0/694054140 >> 172.21.15.143:0/694054140 conn(0x7fad9806c3f0 msgr2=0x7fad9806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 -- 172.21.15.143:0/694054140 shutdown_connections 2024-01-15T18:12:07.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.989+0000 7fad9d200640 1 -- 172.21.15.143:0/694054140 wait complete. 2024-01-15T18:12:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 Processor -- start 2024-01-15T18:12:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- start start 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad980b21d0 0x7fad9806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 0x7fad980adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98157300 0x7fad9813c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad980aca80 con 0x7fad980b4ac0 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad980acd20 con 0x7fad98157300 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad980ad000 con 0x7fad980b21d0 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 0x7fad980adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 0x7fad980adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43904/0 (socket says 172.21.15.143:43904) 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 -- 172.21.15.143:0/1163645726 learned_addr learned my addr 172.21.15.143:0/1163645726 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95e00640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad980b21d0 0x7fad9806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad96800640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98157300 0x7fad9813c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 -- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad980b21d0 msgr2=0x7fad9806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad980b21d0 0x7fad9806d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 -- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98157300 msgr2=0x7fad9813c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98157300 0x7fad9813c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 -- 172.21.15.143:0/1163645726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad90009a70 con 0x7fad980b4ac0 2024-01-15T18:12:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad95400640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 0x7fad980adee0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fad8000c780 tx=0x7fad8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80005cf0 con 0x7fad980b4ac0 2024-01-15T18:12:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80005e50 con 0x7fad980b4ac0 2024-01-15T18:12:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad980ad2e0 con 0x7fad980b4ac0 2024-01-15T18:12:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad800072b0 con 0x7fad980b4ac0 2024-01-15T18:12:07.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad98150ff0 con 0x7fad980b4ac0 2024-01-15T18:12:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.993+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad98067500 con 0x7fad980b4ac0 2024-01-15T18:12:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.997+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 22) v1 ==== 87027+0+0 (secure 0 0 0) 0x7fad80007410 con 0x7fad980b4ac0 2024-01-15T18:12:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.997+0000 7fad8e000640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad70067280 0x7fad70069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.997+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fad80082c10 con 0x7fad980b4ac0 2024-01-15T18:12:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.997+0000 7fad95e00640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad70067280 0x7fad70069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:07.997+0000 7fad95e00640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad70067280 0x7fad70069740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad88009ac0 tx=0x7fad88009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:08.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.005+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad80051710 con 0x7fad980b4ac0 2024-01-15T18:12:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.177+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fad980ad790 con 0x7fad70067280 2024-01-15T18:12:08.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.181+0000 7fad8e000640 1 -- 172.21.15.143:0/1163645726 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fad980ad790 con 0x7fad70067280 2024-01-15T18:12:08.185 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad70067280 msgr2=0x7fad70069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad70067280 0x7fad70069740 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fad88009ac0 tx=0x7fad88009190 comp rx=0 tx=0).stop 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 msgr2=0x7fad980adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 0x7fad980adee0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fad8000c780 tx=0x7fad8000cc40 comp rx=0 tx=0).stop 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 shutdown_connections 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad70067280 0x7fad70069740 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad98157300 0x7fad9813c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad980b4ac0 0x7fad980adee0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 --2- 172.21.15.143:0/1163645726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad980b21d0 0x7fad9806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 >> 172.21.15.143:0/1163645726 conn(0x7fad9806c3f0 msgr2=0x7fad980b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:08.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 shutdown_connections 2024-01-15T18:12:08.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:08.185+0000 7fad9d200640 1 -- 172.21.15.143:0/1163645726 wait complete. 2024-01-15T18:12:08.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:08.206 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a) 2024-01-15T18:12:08.219 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a 2024-01-15T18:12:08.219 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:12:08.219 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:08.220 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:10.221 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a) 2024-01-15T18:12:10.235 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/aa5d8f4d-b7e7-4e80-8813-7f51e8e5e57a 2024-01-15T18:12:10.235 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:12:10.235 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:10.236 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:12:10.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99311 2024-01-15T18:12:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.425+0000 7f6eeb800640 1 Processor -- start 2024-01-15T18:12:10.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.425+0000 7f6eeb800640 1 -- start start 2024-01-15T18:12:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6eeb800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ee4141660 0x7f6ee40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6eeb800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ee4071d00 0x7f6ee4072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6eeb800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee4072650 con 0x7f6ee415ec80 2024-01-15T18:12:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6eeb800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee415fbb0 con 0x7f6ee4071d00 2024-01-15T18:12:10.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6eeb800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee415f130 con 0x7f6ee4141660 2024-01-15T18:12:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ee4071d00 0x7f6ee4072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6ee4071d00 0x7f6ee4072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43906/0 (socket says 172.21.15.143:43906) 2024-01-15T18:12:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 -- 172.21.15.143:0/2456302809 learned_addr learned my addr 172.21.15.143:0/2456302809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:10.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee8e00640 1 --2- 172.21.15.143:0/2456302809 >> v2:172.21.15.204:3300/0 conn(0x7f6ee4141660 0x7f6ee40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 -- 172.21.15.143:0/2456302809 >> v2:172.21.15.204:3300/0 conn(0x7f6ee4141660 msgr2=0x7f6ee40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 --2- 172.21.15.143:0/2456302809 >> v2:172.21.15.204:3300/0 conn(0x7f6ee4141660 0x7f6ee40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 -- 172.21.15.143:0/2456302809 >> v1:172.21.15.204:6789/0 conn(0x7f6ee415ec80 legacy=0x7f6ee4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 -- 172.21.15.143:0/2456302809 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee415f310 con 0x7f6ee4071d00 2024-01-15T18:12:10.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee8e00640 1 --2- 172.21.15.143:0/2456302809 >> v2:172.21.15.204:3300/0 conn(0x7f6ee4141660 0x7f6ee40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.429+0000 7f6ee9800640 1 --2- 172.21.15.143:0/2456302809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee4072110 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40b1aa0 tx=0x7f6edc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9e6def8c111ca1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6ee3400640 1 -- 172.21.15.143:0/2456302809 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0a5860 con 0x7f6ee4071d00 2024-01-15T18:12:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6ee3400640 1 -- 172.21.15.143:0/2456302809 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0a59c0 con 0x7f6ee4071d00 2024-01-15T18:12:10.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6ee3400640 1 -- 172.21.15.143:0/2456302809 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0a5d50 con 0x7f6ee4071d00 2024-01-15T18:12:10.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- 172.21.15.143:0/2456302809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 msgr2=0x7f6ee4072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2456302809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee4072110 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40b1aa0 tx=0x7f6edc01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- 172.21.15.143:0/2456302809 shutdown_connections 2024-01-15T18:12:10.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2456302809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee4072110 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2456302809 >> v2:172.21.15.204:3300/0 conn(0x7f6ee4141660 0x7f6ee40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- 172.21.15.143:0/2456302809 >> 172.21.15.143:0/2456302809 conn(0x7f6ee406c3f0 msgr2=0x7f6ee406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:10.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- 172.21.15.143:0/2456302809 shutdown_connections 2024-01-15T18:12:10.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- 172.21.15.143:0/2456302809 wait complete. 2024-01-15T18:12:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 Processor -- start 2024-01-15T18:12:10.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- start start 2024-01-15T18:12:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee4141660 0x7f6ee415aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ee415ec80 0x7f6ee415da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee413ff60 con 0x7f6ee4071d00 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee413f3e0 con 0x7f6ee4141660 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.433+0000 7f6eeb800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee413f6c0 con 0x7f6ee415ec80 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee415c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43910/0 (socket says 172.21.15.143:43910) 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 -- 172.21.15.143:0/2295510416 learned_addr learned my addr 172.21.15.143:0/2295510416 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee3e00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee4141660 0x7f6ee415aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee9800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ee415ec80 0x7f6ee415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 -- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ee415ec80 msgr2=0x7f6ee415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ee415ec80 0x7f6ee415da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 -- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee4141660 msgr2=0x7f6ee415aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee4141660 0x7f6ee415aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 -- 172.21.15.143:0/2295510416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009b30 con 0x7f6ee4071d00 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee8e00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee415c170 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400e810 tx=0x7f6ed400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400c8d0 con 0x7f6ee4071d00 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed400ca30 con 0x7f6ee4071d00 2024-01-15T18:12:10.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400cdc0 con 0x7f6ee4071d00 2024-01-15T18:12:10.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee413f9a0 con 0x7f6ee4071d00 2024-01-15T18:12:10.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41402e0 con 0x7f6ee4071d00 2024-01-15T18:12:10.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee4067500 con 0x7f6ee4071d00 2024-01-15T18:12:10.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.437+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 22) v1 ==== 87027+0+0 (secure 0 0 0) 0x7f6ed4013410 con 0x7f6ee4071d00 2024-01-15T18:12:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.441+0000 7f6ee0c00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ebc0672b0 0x7f6ebc069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.441+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6ed4082c80 con 0x7f6ee4071d00 2024-01-15T18:12:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.441+0000 7f6ee3e00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ebc0672b0 0x7f6ebc069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.441+0000 7f6ee3e00640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ebc0672b0 0x7f6ebc069770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc001810 tx=0x7f6ecc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.445+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6ed4051820 con 0x7f6ee4071d00 2024-01-15T18:12:10.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.621+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99311", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ee413f830 con 0x7f6ebc0672b0 2024-01-15T18:12:10.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.669+0000 7f6ee0c00640 1 -- 172.21.15.143:0/2295510416 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6ee413f830 con 0x7f6ebc0672b0 2024-01-15T18:12:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.669+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ebc0672b0 msgr2=0x7f6ebc069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.669+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ebc0672b0 0x7f6ebc069770 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc001810 tx=0x7f6ecc009040 comp rx=0 tx=0).stop 2024-01-15T18:12:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.669+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 msgr2=0x7f6ee415c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.669+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee415c170 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400e810 tx=0x7f6ed400ecd0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 shutdown_connections 2024-01-15T18:12:10.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6ebc0672b0 0x7f6ebc069770 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ee415ec80 0x7f6ee415da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ee4141660 0x7f6ee415aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 --2- 172.21.15.143:0/2295510416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ee4071d00 0x7f6ee415c170 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 >> 172.21.15.143:0/2295510416 conn(0x7f6ee406c3f0 msgr2=0x7f6ee40ac9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 shutdown_connections 2024-01-15T18:12:10.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f6eeb800640 1 -- 172.21.15.143:0/2295510416 wait complete. 2024-01-15T18:12:10.675 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.675 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.676 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.676 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.676 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.673+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.693 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.689+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.693 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.689+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.693 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.689+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.693 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.689+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.693 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:10.689+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:10.698 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99312 2024-01-15T18:12:10.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.845+0000 7fd679a00640 1 Processor -- start 2024-01-15T18:12:10.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.845+0000 7fd679a00640 1 -- start start 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.845+0000 7fd679a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd67415b260 0x7fd6740cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.845+0000 7fd679a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740ccaf0 0x7fd6740cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd679a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6740c4d90 con 0x7fd6740d19f0 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd679a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67415b640 con 0x7fd67415b260 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd679a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740cf800 con 0x7fd6740ccaf0 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd672000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd6740d19f0 0x7fd6740c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46760/0 (socket says 172.21.15.143:46760) 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd672000640 1 -- 172.21.15.143:0/3338834494 learned_addr learned my addr 172.21.15.143:0/3338834494 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd672a00640 1 --2- 172.21.15.143:0/3338834494 >> v2:172.21.15.204:3300/0 conn(0x7fd67415b260 0x7fd6740cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd673400640 1 --2- 172.21.15.143:0/3338834494 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740ccaf0 0x7fd6740cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd671600640 1 -- 172.21.15.143:0/3338834494 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161218352 0 0) 0x7fd6740c4d90 con 0x7fd6740d19f0 2024-01-15T18:12:10.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd671600640 1 -- 172.21.15.143:0/3338834494 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd654003530 con 0x7fd6740d19f0 2024-01-15T18:12:10.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd673400640 1 -- 172.21.15.143:0/3338834494 >> v2:172.21.15.204:3300/0 conn(0x7fd67415b260 msgr2=0x7fd6740cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:10.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd673400640 1 --2- 172.21.15.143:0/3338834494 >> v2:172.21.15.204:3300/0 conn(0x7fd67415b260 0x7fd6740cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd673400640 1 -- 172.21.15.143:0/3338834494 >> v1:172.21.15.143:6789/0 conn(0x7fd6740d19f0 legacy=0x7fd6740c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd673400640 1 -- 172.21.15.143:0/3338834494 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740cfae0 con 0x7fd6740ccaf0 2024-01-15T18:12:10.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd672a00640 1 --2- 172.21.15.143:0/3338834494 >> v2:172.21.15.204:3300/0 conn(0x7fd67415b260 0x7fd6740cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:12:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.849+0000 7fd673400640 1 --2- 172.21.15.143:0/3338834494 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740ccaf0 0x7fd6740cdd50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009d90 tx=0x7fd66c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=768de30f603652d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd671600640 1 -- 172.21.15.143:0/3338834494 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0a5900 con 0x7fd6740ccaf0 2024-01-15T18:12:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd671600640 1 -- 172.21.15.143:0/3338834494 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c0a5a60 con 0x7fd6740ccaf0 2024-01-15T18:12:10.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd671600640 1 -- 172.21.15.143:0/3338834494 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0a5dd0 con 0x7fd6740ccaf0 2024-01-15T18:12:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- 172.21.15.143:0/3338834494 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740ccaf0 msgr2=0x7fd6740cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 --2- 172.21.15.143:0/3338834494 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740ccaf0 0x7fd6740cdd50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009d90 tx=0x7fd66c01c950 comp rx=0 tx=0).stop 2024-01-15T18:12:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- 172.21.15.143:0/3338834494 shutdown_connections 2024-01-15T18:12:10.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 --2- 172.21.15.143:0/3338834494 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740ccaf0 0x7fd6740cdd50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 --2- 172.21.15.143:0/3338834494 >> v2:172.21.15.204:3300/0 conn(0x7fd67415b260 0x7fd6740cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- 172.21.15.143:0/3338834494 >> 172.21.15.143:0/3338834494 conn(0x7fd6740d0ed0 msgr2=0x7fd6740d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- 172.21.15.143:0/3338834494 shutdown_connections 2024-01-15T18:12:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- 172.21.15.143:0/3338834494 wait complete. 2024-01-15T18:12:10.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 Processor -- start 2024-01-15T18:12:10.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- start start 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 0x7fd6741522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740d19f0 0x7fd6740b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67415b260 0x7fd674152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740b4bd0 con 0x7fd6740ccaf0 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740b4610 con 0x7fd67415b260 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.853+0000 7fd679a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740b48f0 con 0x7fd6740d19f0 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 0x7fd6741522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 0x7fd6741522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43974/0 (socket says 172.21.15.143:43974) 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 -- 172.21.15.143:0/3376573259 learned_addr learned my addr 172.21.15.143:0/3376573259 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd673400640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67415b260 0x7fd674152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672000640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740d19f0 0x7fd6740b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 -- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740d19f0 msgr2=0x7fd6740b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740d19f0 0x7fd6740b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 -- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67415b260 msgr2=0x7fd674152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67415b260 0x7fd674152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 -- 172.21.15.143:0/3376573259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664009420 con 0x7fd6740ccaf0 2024-01-15T18:12:10.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672000640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740d19f0 0x7fd6740b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:10.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd672a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 0x7fd6741522e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd66400e980 tx=0x7fd66400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66400ca00 con 0x7fd6740ccaf0 2024-01-15T18:12:10.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66400cb60 con 0x7fd6740ccaf0 2024-01-15T18:12:10.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c009a70 con 0x7fd6740ccaf0 2024-01-15T18:12:10.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664013470 con 0x7fd6740ccaf0 2024-01-15T18:12:10.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6740c98d0 con 0x7fd6740ccaf0 2024-01-15T18:12:10.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6740c9450 con 0x7fd6740ccaf0 2024-01-15T18:12:10.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.857+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7fd6640135d0 con 0x7fd6740ccaf0 2024-01-15T18:12:10.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.861+0000 7fd66aa00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd64c0673a0 0x7fd64c069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.861+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd664082fd0 con 0x7fd6740ccaf0 2024-01-15T18:12:10.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.861+0000 7fd672000640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd64c0673a0 0x7fd64c069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.861+0000 7fd672000640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd64c0673a0 0x7fd64c069860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd6740b5610 tx=0x7fd65c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:10.865+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd6740c9450 con 0x7fd6740ccaf0 2024-01-15T18:12:11.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.041+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99312", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6740b4a60 con 0x7fd64c0673a0 2024-01-15T18:12:11.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd66aa00640 1 -- 172.21.15.143:0/3376573259 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd6740b4a60 con 0x7fd64c0673a0 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd64c0673a0 msgr2=0x7fd64c069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd64c0673a0 0x7fd64c069860 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd6740b5610 tx=0x7fd65c009040 comp rx=0 tx=0).stop 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 msgr2=0x7fd6741522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 0x7fd6741522e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd66400e980 tx=0x7fd66400ee40 comp rx=0 tx=0).stop 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 shutdown_connections 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd64c0673a0 0x7fd64c069860 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd67415b260 0x7fd674152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd6740d19f0 0x7fd6740b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 --2- 172.21.15.143:0/3376573259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6740ccaf0 0x7fd6741522e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.093+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 >> 172.21.15.143:0/3376573259 conn(0x7fd6740d0ed0 msgr2=0x7fd6740cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:11.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.097+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 shutdown_connections 2024-01-15T18:12:11.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.097+0000 7fd679a00640 1 -- 172.21.15.143:0/3376573259 wait complete. 2024-01-15T18:12:11.119 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99313 2024-01-15T18:12:11.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.257+0000 7fb2df600640 1 Processor -- start 2024-01-15T18:12:11.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.257+0000 7fb2df600640 1 -- start start 2024-01-15T18:12:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2df600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb2d8141660 0x7fb2d80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2df600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2df600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2d8142470 0x7fb2d8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2df600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d815fbb0 con 0x7fb2d8142470 2024-01-15T18:12:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2df600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d815f100 con 0x7fb2d8141660 2024-01-15T18:12:11.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2df600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815f310 con 0x7fb2d815ec80 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2d8142470 0x7fb2d8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb2d8142470 0x7fb2d8071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43976/0 (socket says 172.21.15.143:43976) 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 -- 172.21.15.143:0/1950742034 learned_addr learned my addr 172.21.15.143:0/1950742034 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2d7e00640 1 --2- 172.21.15.143:0/1950742034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dcc00640 1 --2- 172.21.15.143:0/1950742034 >> v2:172.21.15.204:3300/0 conn(0x7fb2d8141660 0x7fb2d80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 -- 172.21.15.143:0/1950742034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 msgr2=0x7fb2d8141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 --2- 172.21.15.143:0/1950742034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 -- 172.21.15.143:0/1950742034 >> v2:172.21.15.204:3300/0 conn(0x7fb2d8141660 msgr2=0x7fb2d80ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 --2- 172.21.15.143:0/1950742034 >> v2:172.21.15.204:3300/0 conn(0x7fb2d8141660 0x7fb2d80ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.261+0000 7fb2dd600640 1 -- 172.21.15.143:0/1950742034 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d815f5f0 con 0x7fb2d8142470 2024-01-15T18:12:11.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2dd600640 1 --2- 172.21.15.143:0/1950742034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d8071d00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80b1aa0 tx=0x7fb2d001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0b865e171844bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2d7400640 1 -- 172.21.15.143:0/1950742034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a5730 con 0x7fb2d8142470 2024-01-15T18:12:11.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2d7400640 1 -- 172.21.15.143:0/1950742034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d00a5890 con 0x7fb2d8142470 2024-01-15T18:12:11.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2d7400640 1 -- 172.21.15.143:0/1950742034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a5c20 con 0x7fb2d8142470 2024-01-15T18:12:11.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 -- 172.21.15.143:0/1950742034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 msgr2=0x7fb2d8071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 --2- 172.21.15.143:0/1950742034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d8071d00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80b1aa0 tx=0x7fb2d001c9a0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 -- 172.21.15.143:0/1950742034 shutdown_connections 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 --2- 172.21.15.143:0/1950742034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d8071d00 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 --2- 172.21.15.143:0/1950742034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 --2- 172.21.15.143:0/1950742034 >> v2:172.21.15.204:3300/0 conn(0x7fb2d8141660 0x7fb2d80ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 -- 172.21.15.143:0/1950742034 >> 172.21.15.143:0/1950742034 conn(0x7fb2d806c3f0 msgr2=0x7fb2d806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 -- 172.21.15.143:0/1950742034 shutdown_connections 2024-01-15T18:12:11.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 -- 172.21.15.143:0/1950742034 wait complete. 2024-01-15T18:12:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 Processor -- start 2024-01-15T18:12:11.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.265+0000 7fb2df600640 1 -- start start 2024-01-15T18:12:11.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d8141660 0x7fb2d80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d813a0f0 con 0x7fb2d8142470 2024-01-15T18:12:11.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80b2150 con 0x7fb2d8141660 2024-01-15T18:12:11.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80b2430 con 0x7fb2d815ec80 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43980/0 (socket says 172.21.15.143:43980) 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 -- 172.21.15.143:0/3310324477 learned_addr learned my addr 172.21.15.143:0/3310324477 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2dd600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2dcc00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d8141660 0x7fb2d80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 -- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 msgr2=0x7fb2d8139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 -- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d8141660 msgr2=0x7fb2d80b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d8141660 0x7fb2d80b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 -- 172.21.15.143:0/3310324477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0009b30 con 0x7fb2d8142470 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2dd600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2dcc00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d8141660 0x7fb2d80b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:11.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d7e00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d815e430 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000d5f0 tx=0x7fb2c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00125f0 con 0x7fb2d8142470 2024-01-15T18:12:11.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0012750 con 0x7fb2d8142470 2024-01-15T18:12:11.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d80cd150 con 0x7fb2d8142470 2024-01-15T18:12:11.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0012ae0 con 0x7fb2d8142470 2024-01-15T18:12:11.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d80b29f0 con 0x7fb2d8142470 2024-01-15T18:12:11.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8067500 con 0x7fb2d8142470 2024-01-15T18:12:11.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.269+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7fb2c0012c40 con 0x7fb2d8142470 2024-01-15T18:12:11.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.273+0000 7fb2d4c00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2b00673a0 0x7fb2b0069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.273+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb2c00831f0 con 0x7fb2d8142470 2024-01-15T18:12:11.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.273+0000 7fb2dcc00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2b00673a0 0x7fb2b0069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.273+0000 7fb2dcc00640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2b00673a0 0x7fb2b0069860 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8004660 tx=0x7fb2c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.277+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2c0051ee0 con 0x7fb2d8142470 2024-01-15T18:12:11.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:11.453+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99313", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d80ccfd0 con 0x7fb2b00673a0 2024-01-15T18:12:13.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.989+0000 7fb2d4c00640 1 -- 172.21.15.143:0/3310324477 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb2c8003f00 con 0x7fb2b00673a0 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2b00673a0 msgr2=0x7fb2b0069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2b00673a0 0x7fb2b0069860 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8004660 tx=0x7fb2c8008040 comp rx=0 tx=0).stop 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 msgr2=0x7fb2d815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d815e430 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000d5f0 tx=0x7fb2c000dab0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 shutdown_connections 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2b00673a0 0x7fb2b0069860 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb2d815ec80 0x7fb2d8139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2d8142470 0x7fb2d815e430 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 --2- 172.21.15.143:0/3310324477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2d8141660 0x7fb2d80b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 >> 172.21.15.143:0/3310324477 conn(0x7fb2d806c3f0 msgr2=0x7fb2d80af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:13.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 shutdown_connections 2024-01-15T18:12:13.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:13.993+0000 7fb2df600640 1 -- 172.21.15.143:0/3310324477 wait complete. 2024-01-15T18:12:14.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99314 2024-01-15T18:12:14.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.157+0000 7f9d57e00640 1 Processor -- start 2024-01-15T18:12:14.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.157+0000 7f9d57e00640 1 -- start start 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d57e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9d50071bb0 0x7f9d50072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d57e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d5015c5a0 0x7f9d5015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d57e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d500ceb50 con 0x7f9d50143780 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d57e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d50072570 con 0x7f9d5015c5a0 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d57e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5015c110 con 0x7f9d50071bb0 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d5015c5a0 0x7f9d5015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9d5015c5a0 0x7f9d5015c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43994/0 (socket says 172.21.15.143:43994) 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 -- 172.21.15.143:0/1707355524 learned_addr learned my addr 172.21.15.143:0/1707355524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d54a00640 1 --2- 172.21.15.143:0/1707355524 >> v2:172.21.15.204:3300/0 conn(0x7f9d50071bb0 0x7f9d50072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 -- 172.21.15.143:0/1707355524 >> v2:172.21.15.204:3300/0 conn(0x7f9d50071bb0 msgr2=0x7f9d50072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:14.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 --2- 172.21.15.143:0/1707355524 >> v2:172.21.15.204:3300/0 conn(0x7f9d50071bb0 0x7f9d50072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 -- 172.21.15.143:0/1707355524 >> v1:172.21.15.204:6789/0 conn(0x7f9d50143780 legacy=0x7f9d500cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d55e00640 1 -- 172.21.15.143:0/1707355524 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5015b7e0 con 0x7f9d5015c5a0 2024-01-15T18:12:14.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.161+0000 7f9d54a00640 1 --2- 172.21.15.143:0/1707355524 >> v2:172.21.15.204:3300/0 conn(0x7f9d50071bb0 0x7f9d50072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:14.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d55e00640 1 --2- 172.21.15.143:0/1707355524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d5015c5a0 0x7f9d5015c9e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9d5006beb0 tx=0x7f9d4801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ca48dc1dc46620 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:14.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d4fe00640 1 -- 172.21.15.143:0/1707355524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480a5890 con 0x7f9d5015c5a0 2024-01-15T18:12:14.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d4fe00640 1 -- 172.21.15.143:0/1707355524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d480a59f0 con 0x7f9d5015c5a0 2024-01-15T18:12:14.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d4fe00640 1 -- 172.21.15.143:0/1707355524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480a5d80 con 0x7f9d5015c5a0 2024-01-15T18:12:14.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 -- 172.21.15.143:0/1707355524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d5015c5a0 msgr2=0x7f9d5015c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1707355524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d5015c5a0 0x7f9d5015c9e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9d5006beb0 tx=0x7f9d4801c9d0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 -- 172.21.15.143:0/1707355524 shutdown_connections 2024-01-15T18:12:14.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1707355524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d5015c5a0 0x7f9d5015c9e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1707355524 >> v2:172.21.15.204:3300/0 conn(0x7f9d50071bb0 0x7f9d50072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 -- 172.21.15.143:0/1707355524 >> 172.21.15.143:0/1707355524 conn(0x7f9d5006c3f0 msgr2=0x7f9d5006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:14.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 -- 172.21.15.143:0/1707355524 shutdown_connections 2024-01-15T18:12:14.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 -- 172.21.15.143:0/1707355524 wait complete. 2024-01-15T18:12:14.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.165+0000 7f9d57e00640 1 Processor -- start 2024-01-15T18:12:14.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- start start 2024-01-15T18:12:14.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d50071bb0 0x7f9d5013bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 0x7f9d500b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5015c5a0 0x7f9d500b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d500b2280 con 0x7f9d50143780 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d500b3120 con 0x7f9d50071bb0 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d500b3400 con 0x7f9d5015c5a0 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 0x7f9d500b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 0x7f9d500b2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43998/0 (socket says 172.21.15.143:43998) 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 -- 172.21.15.143:0/1740273557 learned_addr learned my addr 172.21.15.143:0/1740273557 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d55400640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d50071bb0 0x7f9d5013bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d55e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5015c5a0 0x7f9d500b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 -- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5015c5a0 msgr2=0x7f9d500b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5015c5a0 0x7f9d500b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 -- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d50071bb0 msgr2=0x7f9d5013bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d50071bb0 0x7f9d5013bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 -- 172.21.15.143:0/1740273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48009b30 con 0x7f9d50143780 2024-01-15T18:12:14.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d54a00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 0x7f9d500b2560 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800e810 tx=0x7f9d3800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800c8d0 con 0x7f9d50143780 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3800ca30 con 0x7f9d50143780 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d500b36e0 con 0x7f9d50143780 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800cdc0 con 0x7f9d50143780 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d500b3ca0 con 0x7f9d50143780 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.169+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d50067500 con 0x7f9d50143780 2024-01-15T18:12:14.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.173+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7f9d38013410 con 0x7f9d50143780 2024-01-15T18:12:14.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.173+0000 7f9d4d600640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d280673d0 0x7f9d28069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.173+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9d380856f0 con 0x7f9d50143780 2024-01-15T18:12:14.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.173+0000 7f9d55400640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d280673d0 0x7f9d28069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.173+0000 7f9d55400640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d280673d0 0x7f9d28069890 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9d40007fd0 tx=0x7f9d40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:14.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.177+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9d380517a0 con 0x7f9d50143780 2024-01-15T18:12:14.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:14.353+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d500b0330 con 0x7f9d280673d0 2024-01-15T18:12:18.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:18.997+0000 7f9d4d600640 1 -- 172.21.15.143:0/1740273557 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9d500b0330 con 0x7f9d280673d0 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d280673d0 msgr2=0x7f9d28069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d280673d0 0x7f9d28069890 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9d40007fd0 tx=0x7f9d40009040 comp rx=0 tx=0).stop 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 msgr2=0x7f9d500b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 0x7f9d500b2560 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800e810 tx=0x7f9d3800ecd0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 shutdown_connections 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d280673d0 0x7f9d28069890 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9d5015c5a0 0x7f9d500b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d50143780 0x7f9d500b2560 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 --2- 172.21.15.143:0/1740273557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9d50071bb0 0x7f9d5013bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 >> 172.21.15.143:0/1740273557 conn(0x7f9d5006c3f0 msgr2=0x7f9d50070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:19.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 shutdown_connections 2024-01-15T18:12:19.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.001+0000 7f9d57e00640 1 -- 172.21.15.143:0/1740273557 wait complete. 2024-01-15T18:12:19.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99315 2024-01-15T18:12:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66cb000640 1 Processor -- start 2024-01-15T18:12:19.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66cb000640 1 -- start start 2024-01-15T18:12:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66cb000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f66c4143780 0x7f66c40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66cb000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40ceb50 con 0x7f66c40722c0 2024-01-15T18:12:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66cb000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c415c110 con 0x7f66c40ce520 2024-01-15T18:12:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66cb000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c415b7e0 con 0x7f66c4143780 2024-01-15T18:12:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f66c40722c0 0x7f66c415c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45660/0 (socket says 172.21.15.143:45660) 2024-01-15T18:12:19.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 -- 172.21.15.143:0/2925699698 learned_addr learned my addr 172.21.15.143:0/2925699698 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c3e00640 1 --2- 172.21.15.143:0/2925699698 >> v2:172.21.15.204:3300/0 conn(0x7f66c4143780 0x7f66c40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582840345 0 0) 0x7f66bc0014a0 con 0x7f66c40722c0 2024-01-15T18:12:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a4005740 con 0x7f66c40722c0 2024-01-15T18:12:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068988016 0 0) 0x7f66c415c110 con 0x7f66c40ce520 2024-01-15T18:12:19.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66bc0014a0 con 0x7f66c40ce520 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361356117 0 0) 0x7f66a4005740 con 0x7f66c40722c0 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 >> v2:172.21.15.204:3300/0 conn(0x7f66c4143780 msgr2=0x7f66c40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 --2- 172.21.15.143:0/2925699698 >> v2:172.21.15.204:3300/0 conn(0x7f66c4143780 0x7f66c40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 >> v1:172.21.15.204:6789/0 conn(0x7f66c40ce520 legacy=0x7f66c4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c415bac0 con 0x7f66c40722c0 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f66bc001b90 con 0x7f66c40722c0 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 >> v1:172.21.15.143:6789/0 conn(0x7f66c40722c0 legacy=0x7f66c415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a4005d20 0x7f66a4006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a40069e0 0x7f66a4006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a40073a0 0x7f66a400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c415b7e0 con 0x7f66a40073a0 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a400e220 con 0x7f66a40069e0 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a400e3e0 con 0x7f66a4005d20 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a40073a0 0x7f66a400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c3e00640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a4005d20 0x7f66a4006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c3400640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a40069e0 0x7f66a4006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 -- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a4005d20 msgr2=0x7f66a4006120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a4005d20 0x7f66a4006120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 -- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a40069e0 msgr2=0x7f66a4006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66a40069e0 0x7f66a4006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.168+0000 7f66c9000640 1 -- 172.21.15.143:0/2925699698 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4005740 con 0x7f66a40073a0 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66c9000640 1 -- 172.21.15.143:0/2925699698 reap_dead start 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66c9000640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a40073a0 0x7f66a400dba0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a790 tx=0x7f66bc01fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c73ef98cd32cbe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0aa630 con 0x7f66a40073a0 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc0aa790 con 0x7f66a40073a0 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66c2a00640 1 -- 172.21.15.143:0/2925699698 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0aab40 con 0x7f66a40073a0 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 -- 172.21.15.143:0/2925699698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a40073a0 msgr2=0x7f66a400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a40073a0 0x7f66a400dba0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a790 tx=0x7f66bc01fa00 comp rx=0 tx=0).stop 2024-01-15T18:12:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 -- 172.21.15.143:0/2925699698 shutdown_connections 2024-01-15T18:12:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 --2- 172.21.15.143:0/2925699698 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66a40073a0 0x7f66a400dba0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 -- 172.21.15.143:0/2925699698 >> 172.21.15.143:0/2925699698 conn(0x7f66c406c3f0 msgr2=0x7f66c406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 -- 172.21.15.143:0/2925699698 shutdown_connections 2024-01-15T18:12:19.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.172+0000 7f66cb000640 1 -- 172.21.15.143:0/2925699698 wait complete. 2024-01-15T18:12:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 Processor -- start 2024-01-15T18:12:19.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 -- start start 2024-01-15T18:12:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a40073a0 0x7f66c41592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c41f1f00 0x7f66c413d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 0x7f66c4141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41597f0 con 0x7f66c41f2380 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4158530 con 0x7f66c41f1f00 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66cb000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41587d0 con 0x7f66a40073a0 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 0x7f66c4141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 0x7f66c4141180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37942/0 (socket says 172.21.15.143:37942) 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 -- 172.21.15.143:0/1854387037 learned_addr learned my addr 172.21.15.143:0/1854387037 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c3400640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c41f1f00 0x7f66c413d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c3e00640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a40073a0 0x7f66c41592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 -- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a40073a0 msgr2=0x7f66c41592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a40073a0 0x7f66c41592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 -- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c41f1f00 msgr2=0x7f66c413d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c41f1f00 0x7f66c413d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 -- 172.21.15.143:0/1854387037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4158ab0 con 0x7f66c41f2380 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c3e00640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a40073a0 0x7f66c41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66c9000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 0x7f66c4141180 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a790 tx=0x7f66a4007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:19.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.176+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0aa630 con 0x7f66c41f2380 2024-01-15T18:12:19.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc004b70 con 0x7f66c41f2380 2024-01-15T18:12:19.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0aa900 con 0x7f66c41f2380 2024-01-15T18:12:19.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c4158d90 con 0x7f66c41f2380 2024-01-15T18:12:19.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c40ad760 con 0x7f66c41f2380 2024-01-15T18:12:19.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c4067500 con 0x7f66c41f2380 2024-01-15T18:12:19.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7f66bc0aaa60 con 0x7f66c41f2380 2024-01-15T18:12:19.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66bbe00640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f669c0673d0 0x7f669c069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:19.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f66bc11fd20 con 0x7f66c41f2380 2024-01-15T18:12:19.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.180+0000 7f66c3e00640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f669c0673d0 0x7f669c069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:19.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.184+0000 7f66c3e00640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f669c0673d0 0x7f669c069890 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009b60 tx=0x7f66b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:19.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.184+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66bc0eea20 con 0x7f66c41f2380 2024-01-15T18:12:19.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:19.364+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99315", "target": ["mon-mgr", ""]}) v1 -- 0x7f66c40b4ce0 con 0x7f669c0673d0 2024-01-15T18:12:23.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.992+0000 7f66bbe00640 1 -- 172.21.15.143:0/1854387037 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f66b40097b0 con 0x7f669c0673d0 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f669c0673d0 msgr2=0x7f669c069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f669c0673d0 0x7f669c069890 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009b60 tx=0x7f66b4008040 comp rx=0 tx=0).stop 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 msgr2=0x7f66c4141180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 0x7f66c4141180 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a790 tx=0x7f66a4007a10 comp rx=0 tx=0).stop 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 shutdown_connections 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f669c0673d0 0x7f669c069890 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66c41f2380 0x7f66c4141180 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66c41f1f00 0x7f66c413d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 --2- 172.21.15.143:0/1854387037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f66a40073a0 0x7f66c41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 >> 172.21.15.143:0/1854387037 conn(0x7f66c406c3f0 msgr2=0x7f66c40b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:23.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 shutdown_connections 2024-01-15T18:12:23.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:23.996+0000 7f66cb000640 1 -- 172.21.15.143:0/1854387037 wait complete. 2024-01-15T18:12:24.026 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99316 2024-01-15T18:12:24.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.176+0000 7fea40600640 1 Processor -- start 2024-01-15T18:12:24.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.176+0000 7fea40600640 1 -- start start 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea40600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea38072030 0x7fea3815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea40600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea380ceb50 con 0x7fea38071bb0 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea40600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3815c110 con 0x7fea38143780 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea40600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3815b7e0 con 0x7fea38072030 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea3e600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea38072030 0x7fea3815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea3e600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea38072030 0x7fea3815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37950/0 (socket says 172.21.15.143:37950) 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea3e600640 1 -- 172.21.15.143:0/2990035419 learned_addr learned my addr 172.21.15.143:0/2990035419 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea3e600640 1 -- 172.21.15.143:0/2990035419 >> v1:172.21.15.204:6789/0 conn(0x7fea38143780 legacy=0x7fea380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea3e600640 1 -- 172.21.15.143:0/2990035419 >> v1:172.21.15.143:6789/0 conn(0x7fea38071bb0 legacy=0x7fea3815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.180+0000 7fea3e600640 1 -- 172.21.15.143:0/2990035419 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3815bac0 con 0x7fea38072030 2024-01-15T18:12:24.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea3e600640 1 --2- 172.21.15.143:0/2990035419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38072030 0x7fea3815ccb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fea3000d150 tx=0x7fea3001eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c721c61fd91cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea37e00640 1 -- 172.21.15.143:0/2990035419 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000b2d0 con 0x7fea38072030 2024-01-15T18:12:24.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea37e00640 1 -- 172.21.15.143:0/2990035419 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3000b430 con 0x7fea38072030 2024-01-15T18:12:24.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea37e00640 1 -- 172.21.15.143:0/2990035419 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300a7c40 con 0x7fea38072030 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 -- 172.21.15.143:0/2990035419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38072030 msgr2=0x7fea3815ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 --2- 172.21.15.143:0/2990035419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38072030 0x7fea3815ccb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fea3000d150 tx=0x7fea3001eb90 comp rx=0 tx=0).stop 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 -- 172.21.15.143:0/2990035419 shutdown_connections 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 --2- 172.21.15.143:0/2990035419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38072030 0x7fea3815ccb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 -- 172.21.15.143:0/2990035419 >> 172.21.15.143:0/2990035419 conn(0x7fea3806c3f0 msgr2=0x7fea3806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 -- 172.21.15.143:0/2990035419 shutdown_connections 2024-01-15T18:12:24.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 -- 172.21.15.143:0/2990035419 wait complete. 2024-01-15T18:12:24.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 Processor -- start 2024-01-15T18:12:24.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.184+0000 7fea40600640 1 -- start start 2024-01-15T18:12:24.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38071bb0 0x7fea3813b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea38072030 0x7fea38139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea38143780 0x7fea381594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3813bd80 con 0x7fea38071bb0 2024-01-15T18:12:24.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3813a7a0 con 0x7fea38072030 2024-01-15T18:12:24.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3813aa80 con 0x7fea38143780 2024-01-15T18:12:24.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea38072030 0x7fea38139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea38143780 0x7fea381594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea38072030 0x7fea38139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58678/0 (socket says 172.21.15.143:58678) 2024-01-15T18:12:24.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3d200640 1 -- 172.21.15.143:0/3515099018 learned_addr learned my addr 172.21.15.143:0/3515099018 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 -- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea38072030 msgr2=0x7fea38139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea38072030 0x7fea38139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 -- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38071bb0 msgr2=0x7fea3813b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38071bb0 0x7fea3813b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 -- 172.21.15.143:0/3515099018 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3000bd40 con 0x7fea38143780 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea3e600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea38143780 0x7fea381594d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea300ad000 tx=0x7fea3001b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300bd070 con 0x7fea38143780 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30009600 con 0x7fea38143780 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300099b0 con 0x7fea38143780 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3813ad60 con 0x7fea38143780 2024-01-15T18:12:24.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.188+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3813b320 con 0x7fea38143780 2024-01-15T18:12:24.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.192+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3813b510 con 0x7fea38143780 2024-01-15T18:12:24.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.192+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7fea30009b10 con 0x7fea38143780 2024-01-15T18:12:24.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.192+0000 7fea35600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fea100673a0 0x7fea10069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.192+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fea3011d8e0 con 0x7fea38143780 2024-01-15T18:12:24.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.192+0000 7fea3dc00640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fea100673a0 0x7fea10069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.192+0000 7fea3dc00640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fea100673a0 0x7fea10069860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fea28009670 tx=0x7fea28006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.196+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea300ec5d0 con 0x7fea38143780 2024-01-15T18:12:24.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.376+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99316", "target": ["mon-mgr", ""]}) v1 -- 0x7fea38070aa0 con 0x7fea100673a0 2024-01-15T18:12:24.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.428+0000 7fea35600640 1 -- 172.21.15.143:0/3515099018 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fea28009280 con 0x7fea100673a0 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fea100673a0 msgr2=0x7fea10069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fea100673a0 0x7fea10069860 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fea28009670 tx=0x7fea28006bd0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea38143780 msgr2=0x7fea381594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea38143780 0x7fea381594d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea300ad000 tx=0x7fea3001b930 comp rx=0 tx=0).stop 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 shutdown_connections 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fea100673a0 0x7fea10069860 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea38143780 0x7fea381594d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea38072030 0x7fea38139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 --2- 172.21.15.143:0/3515099018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea38071bb0 0x7fea3813b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 >> 172.21.15.143:0/3515099018 conn(0x7fea3806c3f0 msgr2=0x7fea3813f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 shutdown_connections 2024-01-15T18:12:24.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.432+0000 7fea40600640 1 -- 172.21.15.143:0/3515099018 wait complete. 2024-01-15T18:12:24.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99317 2024-01-15T18:12:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 Processor -- start 2024-01-15T18:12:24.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 -- start start 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f084c15b610 0x7f084c140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f084c071920 0x7f084c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f084c140e00 con 0x7f084c06a490 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c13e420 con 0x7f084c15b610 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f0852800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c1416a0 con 0x7f084c071920 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f084b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f084c15b610 0x7f084c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f084b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f084c15b610 0x7f084c140700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37968/0 (socket says 172.21.15.143:37968) 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f084b400640 1 -- 172.21.15.143:0/2995897464 learned_addr learned my addr 172.21.15.143:0/2995897464 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.592+0000 7f084be00640 1 --1- 172.21.15.143:0/2995897464 >> v1:172.21.15.143:6789/0 conn(0x7f084c06a490 0x7f084c15b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45678/0 (socket says 172.21.15.143:45678) 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084aa00640 1 --2- 172.21.15.143:0/2995897464 >> v2:172.21.15.204:3300/0 conn(0x7f084c071920 0x7f084c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084b400640 1 -- 172.21.15.143:0/2995897464 >> v2:172.21.15.204:3300/0 conn(0x7f084c071920 msgr2=0x7f084c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084b400640 1 --2- 172.21.15.143:0/2995897464 >> v2:172.21.15.204:3300/0 conn(0x7f084c071920 0x7f084c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084b400640 1 -- 172.21.15.143:0/2995897464 >> v1:172.21.15.143:6789/0 conn(0x7f084c06a490 legacy=0x7f084c15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084b400640 1 -- 172.21.15.143:0/2995897464 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c1410e0 con 0x7f084c15b610 2024-01-15T18:12:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084b400640 1 --2- 172.21.15.143:0/2995897464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c15b610 0x7f084c140700 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f083c009870 tx=0x7f083c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1232440a7b595eda server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084a000640 1 -- 172.21.15.143:0/2995897464 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c0ab610 con 0x7f084c15b610 2024-01-15T18:12:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084a000640 1 -- 172.21.15.143:0/2995897464 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c0ab770 con 0x7f084c15b610 2024-01-15T18:12:24.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.596+0000 7f084a000640 1 -- 172.21.15.143:0/2995897464 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c0abb00 con 0x7f084c15b610 2024-01-15T18:12:24.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- 172.21.15.143:0/2995897464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c15b610 msgr2=0x7f084c140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 --2- 172.21.15.143:0/2995897464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c15b610 0x7f084c140700 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f083c009870 tx=0x7f083c01c500 comp rx=0 tx=0).stop 2024-01-15T18:12:24.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- 172.21.15.143:0/2995897464 shutdown_connections 2024-01-15T18:12:24.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 --2- 172.21.15.143:0/2995897464 >> v2:172.21.15.204:3300/0 conn(0x7f084c071920 0x7f084c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 --2- 172.21.15.143:0/2995897464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c15b610 0x7f084c140700 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- 172.21.15.143:0/2995897464 >> 172.21.15.143:0/2995897464 conn(0x7f084c0b1b90 msgr2=0x7f084c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- 172.21.15.143:0/2995897464 shutdown_connections 2024-01-15T18:12:24.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- 172.21.15.143:0/2995897464 wait complete. 2024-01-15T18:12:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 Processor -- start 2024-01-15T18:12:24.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- start start 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f084c06a490 0x7f084c1381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 0x7f084c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f084c15b610 0x7f084c151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c13c330 con 0x7f084c071920 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c13c610 con 0x7f084c06a490 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f0852800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c13c8f0 con 0x7f084c15b610 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f084aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 0x7f084c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f084aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 0x7f084c151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37970/0 (socket says 172.21.15.143:37970) 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f084aa00640 1 -- 172.21.15.143:0/3728102552 learned_addr learned my addr 172.21.15.143:0/3728102552 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:24.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f084b400640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f084c06a490 0x7f084c1381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.600+0000 7f084be00640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f084c15b610 0x7f084c151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084aa00640 1 -- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f084c15b610 msgr2=0x7f084c151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084aa00640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f084c15b610 0x7f084c151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084aa00640 1 -- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f084c06a490 msgr2=0x7f084c1381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084aa00640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f084c06a490 0x7f084c1381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084aa00640 1 -- 172.21.15.143:0/3728102552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c0094d0 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084aa00640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 0x7f084c151a30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f083400d800 tx=0x7f083400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400b6c0 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083400b820 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c13cbd0 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400bb90 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084c13d190 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c067500 con 0x7f084c071920 2024-01-15T18:12:24.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7f083400bcf0 con 0x7f084c071920 2024-01-15T18:12:24.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0843400640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08240673d0 0x7f0824069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0834083210 con 0x7f084c071920 2024-01-15T18:12:24.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.604+0000 7f084b400640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08240673d0 0x7f0824069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.608+0000 7f084b400640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08240673d0 0x7f0824069890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f083c0af000 tx=0x7f083c0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.612+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0834051ec0 con 0x7f084c071920 2024-01-15T18:12:24.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.792+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99317", "target": ["mon-mgr", ""]}) v1 -- 0x7f084c153920 con 0x7f08240673d0 2024-01-15T18:12:24.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0843400640 1 -- 172.21.15.143:0/3728102552 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f084c153920 con 0x7f08240673d0 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08240673d0 msgr2=0x7f0824069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08240673d0 0x7f0824069890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f083c0af000 tx=0x7f083c0b6040 comp rx=0 tx=0).stop 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 msgr2=0x7f084c151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 0x7f084c151a30 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f083400d800 tx=0x7f083400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 shutdown_connections 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f08240673d0 0x7f0824069890 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f084c15b610 0x7f084c151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f084c071920 0x7f084c151a30 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 --2- 172.21.15.143:0/3728102552 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f084c06a490 0x7f084c1381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 >> 172.21.15.143:0/3728102552 conn(0x7f084c0b1b90 msgr2=0x7f084c0cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 shutdown_connections 2024-01-15T18:12:24.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:24.848+0000 7f0852800640 1 -- 172.21.15.143:0/3728102552 wait complete. 2024-01-15T18:12:24.871 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:12:24.872 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99314 2024-01-15T18:12:25.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5743a00640 1 Processor -- start 2024-01-15T18:12:25.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5743a00640 1 -- start start 2024-01-15T18:12:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5743a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c072030 0x7f573c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5743a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0ceb50 con 0x7f573c143780 2024-01-15T18:12:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5743a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c15c110 con 0x7f573c071bb0 2024-01-15T18:12:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5743a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0724c0 con 0x7f573c072030 2024-01-15T18:12:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5741000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f573c143780 0x7f573c0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45692/0 (socket says 172.21.15.143:45692) 2024-01-15T18:12:25.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5741000640 1 -- 172.21.15.143:0/2302018048 learned_addr learned my addr 172.21.15.143:0/2302018048 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f5741a00640 1 --2- 172.21.15.143:0/2302018048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c072030 0x7f573c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744829788 0 0) 0x7f572c001120 con 0x7f573c143780 2024-01-15T18:12:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c005740 con 0x7f573c143780 2024-01-15T18:12:25.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.008+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764348143 0 0) 0x7f573c15c110 con 0x7f573c071bb0 2024-01-15T18:12:25.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f572c001120 con 0x7f573c071bb0 2024-01-15T18:12:25.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129263994 0 0) 0x7f571c005740 con 0x7f573c143780 2024-01-15T18:12:25.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f5741a00640 1 -- 172.21.15.143:0/2302018048 >> v1:172.21.15.204:6789/0 conn(0x7f573c071bb0 legacy=0x7f573c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f5741a00640 1 -- 172.21.15.143:0/2302018048 >> v1:172.21.15.143:6789/0 conn(0x7f573c143780 legacy=0x7f573c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f5741a00640 1 -- 172.21.15.143:0/2302018048 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c15b7e0 con 0x7f573c072030 2024-01-15T18:12:25.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f5741a00640 1 --2- 172.21.15.143:0/2302018048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c072030 0x7f573c15ccb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5734009dc0 tx=0x7f573401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00a36179ca571a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c001830 con 0x7f573c072030 2024-01-15T18:12:25.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57340a56d0 con 0x7f573c072030 2024-01-15T18:12:25.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.012+0000 7f573b400640 1 -- 172.21.15.143:0/2302018048 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57340a5a60 con 0x7f573c072030 2024-01-15T18:12:25.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- 172.21.15.143:0/2302018048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c072030 msgr2=0x7f573c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 --2- 172.21.15.143:0/2302018048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c072030 0x7f573c15ccb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5734009dc0 tx=0x7f573401c890 comp rx=0 tx=0).stop 2024-01-15T18:12:25.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- 172.21.15.143:0/2302018048 shutdown_connections 2024-01-15T18:12:25.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 --2- 172.21.15.143:0/2302018048 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c072030 0x7f573c15ccb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- 172.21.15.143:0/2302018048 >> 172.21.15.143:0/2302018048 conn(0x7f573c06c3f0 msgr2=0x7f573c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:25.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- 172.21.15.143:0/2302018048 shutdown_connections 2024-01-15T18:12:25.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- 172.21.15.143:0/2302018048 wait complete. 2024-01-15T18:12:25.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 Processor -- start 2024-01-15T18:12:25.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- start start 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c071bb0 0x7f573c141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f573c072030 0x7f573c140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 0x7f573c13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c142160 con 0x7f573c143780 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c140b80 con 0x7f573c072030 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5743a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c140e60 con 0x7f573c071bb0 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5741a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 0x7f573c13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5741a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 0x7f573c13e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37982/0 (socket says 172.21.15.143:37982) 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5741a00640 1 -- 172.21.15.143:0/3416658294 learned_addr learned my addr 172.21.15.143:0/3416658294 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f5741000640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c071bb0 0x7f573c141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.016+0000 7f573be00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f573c072030 0x7f573c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5741a00640 1 -- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c071bb0 msgr2=0x7f573c141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5741a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c071bb0 0x7f573c141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5741a00640 1 -- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f573c072030 msgr2=0x7f573c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5741a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f573c072030 0x7f573c140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5741a00640 1 -- 172.21.15.143:0/3416658294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5734009a70 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5741a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 0x7f573c13e1b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f57340b0040 tx=0x7f5734019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57340a5ea0 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734003bc0 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734009450 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c141140 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c141700 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c067500 con 0x7f573c143780 2024-01-15T18:12:25.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7f57340b4050 con 0x7f573c143780 2024-01-15T18:12:25.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5738c00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57140673a0 0x7f5714069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.020+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f573411d480 con 0x7f573c143780 2024-01-15T18:12:25.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.024+0000 7f5741000640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57140673a0 0x7f5714069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.024+0000 7f5741000640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57140673a0 0x7f5714069860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f572c001680 tx=0x7f572c0015d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.028+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f57340ec1c0 con 0x7f573c143780 2024-01-15T18:12:25.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.204+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7f573c140fd0 con 0x7f57140673a0 2024-01-15T18:12:25.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5738c00640 1 -- 172.21.15.143:0/3416658294 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f573c140fd0 con 0x7f57140673a0 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:25.214 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57140673a0 msgr2=0x7f5714069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57140673a0 0x7f5714069860 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f572c001680 tx=0x7f572c0015d0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 msgr2=0x7f573c13e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 0x7f573c13e1b0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f57340b0040 tx=0x7f5734019630 comp rx=0 tx=0).stop 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 shutdown_connections 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f57140673a0 0x7f5714069860 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f573c143780 0x7f573c13e1b0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f573c072030 0x7f573c140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 --2- 172.21.15.143:0/3416658294 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f573c071bb0 0x7f573c141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 >> 172.21.15.143:0/3416658294 conn(0x7f573c06c3f0 msgr2=0x7f573c0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:25.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 shutdown_connections 2024-01-15T18:12:25.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:25.212+0000 7f5743a00640 1 -- 172.21.15.143:0/3416658294 wait complete. 2024-01-15T18:12:25.235 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:12:26.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99314 2024-01-15T18:12:26.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445f400640 1 Processor -- start 2024-01-15T18:12:26.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445f400640 1 -- start start 2024-01-15T18:12:26.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f44580cf910 0x7f44580cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445f400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44580cbff0 con 0x7f445806bfb0 2024-01-15T18:12:26.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44580d0c50 con 0x7f44580b0310 2024-01-15T18:12:26.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445f400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44580cb6c0 con 0x7f44580cf910 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445ca00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f445806bfb0 0x7f445806c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45708/0 (socket says 172.21.15.143:45708) 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445ca00640 1 -- 172.21.15.143:0/3790563951 learned_addr learned my addr 172.21.15.143:0/3790563951 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f445d400640 1 --2- 172.21.15.143:0/3790563951 >> v2:172.21.15.204:3300/0 conn(0x7f44580cf910 0x7f44580cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411676921 0 0) 0x7f44580cbff0 con 0x7f445806bfb0 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4438003530 con 0x7f445806bfb0 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 863293215 0 0) 0x7f4438003530 con 0x7f445806bfb0 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 >> v2:172.21.15.204:3300/0 conn(0x7f44580cf910 msgr2=0x7f44580cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 --2- 172.21.15.143:0/3790563951 >> v2:172.21.15.204:3300/0 conn(0x7f44580cf910 0x7f44580cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 >> v1:172.21.15.204:6789/0 conn(0x7f44580b0310 legacy=0x7f44580cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580cb9a0 con 0x7f445806bfb0 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4448001680 con 0x7f445806bfb0 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 >> v1:172.21.15.143:6789/0 conn(0x7f445806bfb0 legacy=0x7f445806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.384+0000 7f4457400640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4438003980 0x7f4438003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4438004380 0x7f4438004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44380050f0 0x7f443800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580cb6c0 con 0x7f4438004380 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443800c030 con 0x7f4438003980 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443800c1f0 con 0x7f44380050f0 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4438004380 0x7f4438004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f445ca00640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4438003980 0x7f4438003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f445d400640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44380050f0 0x7f443800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 -- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44380050f0 msgr2=0x7f443800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44380050f0 0x7f443800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 -- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4438003980 msgr2=0x7f4438003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4438003980 0x7f4438003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 -- 172.21.15.143:0/3790563951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4438003530 con 0x7f4438004380 2024-01-15T18:12:26.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f445d400640 1 -- 172.21.15.143:0/3790563951 reap_dead start 2024-01-15T18:12:26.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457e00640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4438004380 0x7f4438004850 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f44580b06f0 tx=0x7f444001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8b27e58ba65385 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400ab600 con 0x7f4438004380 2024-01-15T18:12:26.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44400a7050 con 0x7f4438004380 2024-01-15T18:12:26.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.388+0000 7f4457400640 1 -- 172.21.15.143:0/3790563951 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400ab9e0 con 0x7f4438004380 2024-01-15T18:12:26.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- 172.21.15.143:0/3790563951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4438004380 msgr2=0x7f4438004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4438004380 0x7f4438004850 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f44580b06f0 tx=0x7f444001c450 comp rx=0 tx=0).stop 2024-01-15T18:12:26.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- 172.21.15.143:0/3790563951 shutdown_connections 2024-01-15T18:12:26.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 --2- 172.21.15.143:0/3790563951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4438004380 0x7f4438004850 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- 172.21.15.143:0/3790563951 >> 172.21.15.143:0/3790563951 conn(0x7f44580d0ed0 msgr2=0x7f44580d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:26.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- 172.21.15.143:0/3790563951 shutdown_connections 2024-01-15T18:12:26.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- 172.21.15.143:0/3790563951 wait complete. 2024-01-15T18:12:26.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 Processor -- start 2024-01-15T18:12:26.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- start start 2024-01-15T18:12:26.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4438004380 0x7f44580cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 0x7f445809c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44581f6280 0x7f44581f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445815c760 con 0x7f445809e6e0 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44580c8040 con 0x7f44581f6280 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.392+0000 7f445f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44580c82e0 con 0x7f4438004380 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 0x7f445809c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 0x7f445809c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38008/0 (socket says 172.21.15.143:38008) 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 -- 172.21.15.143:0/2019021010 learned_addr learned my addr 172.21.15.143:0/2019021010 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f445ca00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4438004380 0x7f44580cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f445d400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44581f6280 0x7f44581f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 -- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4438004380 msgr2=0x7f44580cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4438004380 0x7f44580cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 -- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44581f6280 msgr2=0x7f44581f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44581f6280 0x7f44581f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 -- 172.21.15.143:0/2019021010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580c85c0 con 0x7f445809e6e0 2024-01-15T18:12:26.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4457e00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 0x7f445809c930 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f44400ae040 tx=0x7f4438005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440007240 con 0x7f445809e6e0 2024-01-15T18:12:26.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44400073c0 con 0x7f445809e6e0 2024-01-15T18:12:26.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400b4400 con 0x7f445809e6e0 2024-01-15T18:12:26.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44580c88a0 con 0x7f445809e6e0 2024-01-15T18:12:26.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445809eb60 con 0x7f445809e6e0 2024-01-15T18:12:26.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4458067500 con 0x7f445809e6e0 2024-01-15T18:12:26.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.396+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7f44400b2040 con 0x7f445809e6e0 2024-01-15T18:12:26.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.400+0000 7f4454c00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44300673d0 0x7f4430069890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.400+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f444011c750 con 0x7f445809e6e0 2024-01-15T18:12:26.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.400+0000 7f445ca00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44300673d0 0x7f4430069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.400+0000 7f445ca00640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44300673d0 0x7f4430069890 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4458067bb0 tx=0x7f4448000ff0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.404+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44400eb490 con 0x7f445809e6e0 2024-01-15T18:12:26.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.584+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7f445815ccd0 con 0x7f44300673d0 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.592+0000 7f4454c00640 1 -- 172.21.15.143:0/2019021010 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f445815ccd0 con 0x7f44300673d0 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:12:26.596 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:12:26.597 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:12:26.597 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:26.597 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:26.597 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:26.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44300673d0 msgr2=0x7f4430069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44300673d0 0x7f4430069890 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4458067bb0 tx=0x7f4448000ff0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 msgr2=0x7f445809c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 0x7f445809c930 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f44400ae040 tx=0x7f4438005800 comp rx=0 tx=0).stop 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 shutdown_connections 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44300673d0 0x7f4430069890 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44581f6280 0x7f44581f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f445809e6e0 0x7f445809c930 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 --2- 172.21.15.143:0/2019021010 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4438004380 0x7f44580cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 >> 172.21.15.143:0/2019021010 conn(0x7f44580d0ed0 msgr2=0x7f44580adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 shutdown_connections 2024-01-15T18:12:26.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:26.596+0000 7f445f400640 1 -- 172.21.15.143:0/2019021010 wait complete. 2024-01-15T18:12:26.618 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:12:27.619 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99314 2024-01-15T18:12:27.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.764+0000 7f59c3000640 1 Processor -- start 2024-01-15T18:12:27.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.764+0000 7f59c3000640 1 -- start start 2024-01-15T18:12:27.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c3000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc0cfb30 0x7f59bc06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:27.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c3000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59bc0cbf50 0x7f59bc0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:27.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c3000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0cffd0 con 0x7f59bc1607c0 2024-01-15T18:12:27.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c3000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59bc0d1ce0 con 0x7f59bc0cbf50 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c3000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0d02b0 con 0x7f59bc0cfb30 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59bc0cbf50 0x7f59bc0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59bc0cbf50 0x7f59bc0cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52680/0 (socket says 172.21.15.143:52680) 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 -- 172.21.15.143:0/3549674610 learned_addr learned my addr 172.21.15.143:0/3549674610 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59bb400640 1 --2- 172.21.15.143:0/3549674610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc0cfb30 0x7f59bc06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 -- 172.21.15.143:0/3549674610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc0cfb30 msgr2=0x7f59bc06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 --2- 172.21.15.143:0/3549674610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc0cfb30 0x7f59bc06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 -- 172.21.15.143:0/3549674610 >> v1:172.21.15.204:6789/0 conn(0x7f59bc1607c0 legacy=0x7f59bc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:27.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.768+0000 7f59c1000640 1 -- 172.21.15.143:0/3549674610 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0cb470 con 0x7f59bc0cbf50 2024-01-15T18:12:27.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c1000640 1 --2- 172.21.15.143:0/3549674610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cbf50 0x7f59bc0cc390 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009e50 tx=0x7f59b401f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f50726c97e5fea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:27.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59baa00640 1 -- 172.21.15.143:0/3549674610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40af400 con 0x7f59bc0cbf50 2024-01-15T18:12:27.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59baa00640 1 -- 172.21.15.143:0/3549674610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b40ab050 con 0x7f59bc0cbf50 2024-01-15T18:12:27.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59baa00640 1 -- 172.21.15.143:0/3549674610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40af780 con 0x7f59bc0cbf50 2024-01-15T18:12:27.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 -- 172.21.15.143:0/3549674610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cbf50 msgr2=0x7f59bc0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 --2- 172.21.15.143:0/3549674610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cbf50 0x7f59bc0cc390 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009e50 tx=0x7f59b401f1f0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 -- 172.21.15.143:0/3549674610 shutdown_connections 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 --2- 172.21.15.143:0/3549674610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cbf50 0x7f59bc0cc390 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 --2- 172.21.15.143:0/3549674610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc0cfb30 0x7f59bc06bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 -- 172.21.15.143:0/3549674610 >> 172.21.15.143:0/3549674610 conn(0x7f59bc0d0e20 msgr2=0x7f59bc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 -- 172.21.15.143:0/3549674610 shutdown_connections 2024-01-15T18:12:27.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 -- 172.21.15.143:0/3549674610 wait complete. 2024-01-15T18:12:27.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.772+0000 7f59c3000640 1 Processor -- start 2024-01-15T18:12:27.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- start start 2024-01-15T18:12:27.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59bc0cbf50 0x7f59bc153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 0x7f59bc1f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc1607c0 0x7f59bc1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1572e0 con 0x7f59bc0cfb30 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc153b90 con 0x7f59bc0cbf50 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc1566d0 con 0x7f59bc1607c0 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 0x7f59bc1f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 0x7f59bc1f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52682/0 (socket says 172.21.15.143:52682) 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 -- 172.21.15.143:0/3909549170 learned_addr learned my addr 172.21.15.143:0/3909549170 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c1000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc1607c0 0x7f59bc1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bbe00640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59bc0cbf50 0x7f59bc153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 -- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc1607c0 msgr2=0x7f59bc1f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc1607c0 0x7f59bc1f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 -- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59bc0cbf50 msgr2=0x7f59bc153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:27.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59bc0cbf50 0x7f59bc153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 -- 172.21.15.143:0/3909549170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b4009b30 con 0x7f59bc0cfb30 2024-01-15T18:12:27.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59bb400640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 0x7f59bc1f6210 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f59a400e930 tx=0x7f59a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4009690 con 0x7f59bc0cfb30 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc1569b0 con 0x7f59bc0cfb30 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a400c9a0 con 0x7f59bc0cfb30 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a400cd10 con 0x7f59bc0cfb30 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc156f70 con 0x7f59bc0cfb30 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.776+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc06d4b0 con 0x7f59bc0cfb30 2024-01-15T18:12:27.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.780+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7f59a4013470 con 0x7f59bc0cfb30 2024-01-15T18:12:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.780+0000 7f59b3e00640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59940673a0 0x7f5994069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.780+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f59a4082fc0 con 0x7f59bc0cfb30 2024-01-15T18:12:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.780+0000 7f59bbe00640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59940673a0 0x7f5994069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.780+0000 7f59bbe00640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59940673a0 0x7f5994069860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0ca130 tx=0x7f59ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:27.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.784+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f59a4051d40 con 0x7f59bc0cfb30 2024-01-15T18:12:27.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.968+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7f59bc0ce6b0 con 0x7f59940673a0 2024-01-15T18:12:27.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.972+0000 7f59b3e00640 1 -- 172.21.15.143:0/3909549170 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f59ac001d60 con 0x7f59940673a0 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:27.975 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59940673a0 msgr2=0x7f5994069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59940673a0 0x7f5994069860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f59bc0ca130 tx=0x7f59ac009040 comp rx=0 tx=0).stop 2024-01-15T18:12:27.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 msgr2=0x7f59bc1f6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 0x7f59bc1f6210 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f59a400e930 tx=0x7f59a400edf0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 shutdown_connections 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59940673a0 0x7f5994069860 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59bc1607c0 0x7f59bc1f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59bc0cfb30 0x7f59bc1f6210 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 --2- 172.21.15.143:0/3909549170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59bc0cbf50 0x7f59bc153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 >> 172.21.15.143:0/3909549170 conn(0x7f59bc0d0e20 msgr2=0x7f59bc15dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 shutdown_connections 2024-01-15T18:12:27.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:27.976+0000 7f59c3000640 1 -- 172.21.15.143:0/3909549170 wait complete. 2024-01-15T18:12:27.997 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:12:28.566 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:28.570 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:28.952 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:28.948+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:28.953 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:28.948+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:28.953 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:28.948+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:28.953 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:28.948+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:28.953 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:28.948+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:28.997 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99314 2024-01-15T18:12:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fc800640 1 Processor -- start 2024-01-15T18:12:29.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fc800640 1 -- start start 2024-01-15T18:12:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 0x7fe1f40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fc800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40b3370 con 0x7fe1f415d1f0 2024-01-15T18:12:29.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fc800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f415d5d0 con 0x7fe1f4070d60 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f406a800 con 0x7fe1f40711f0 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f9e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe1f415d1f0 0x7fe1f406b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35456/0 (socket says 172.21.15.143:35456) 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f9e00640 1 -- 172.21.15.143:0/1116314811 learned_addr learned my addr 172.21.15.143:0/1116314811 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 0x7fe1f40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259378105 0 0) 0x7fe1f40b3370 con 0x7fe1f415d1f0 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d4003530 con 0x7fe1f415d1f0 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894646569 0 0) 0x7fe1f415d5d0 con 0x7fe1f4070d60 2024-01-15T18:12:29.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f40b3370 con 0x7fe1f4070d60 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360802974 0 0) 0x7fe1d4003530 con 0x7fe1f415d1f0 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 msgr2=0x7fe1f40ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 0x7fe1f40ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 >> v1:172.21.15.204:6789/0 conn(0x7fe1f4070d60 legacy=0x7fe1f40acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f406aae0 con 0x7fe1f415d1f0 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fe1e4001680 con 0x7fe1f415d1f0 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 >> v1:172.21.15.143:6789/0 conn(0x7fe1f415d1f0 legacy=0x7fe1f406b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 0x7fe1d4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40040a0 0x7fe1d40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4004d10 0x7fe1d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f406a800 con 0x7fe1d4004d10 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400ba70 con 0x7fe1d40040a0 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400bc30 con 0x7fe1f40711f0 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4004d10 0x7fe1d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f9400640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40040a0 0x7fe1d40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1f9e00640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 0x7fe1d4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 -- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 msgr2=0x7fe1d4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f40711f0 0x7fe1d4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 -- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40040a0 msgr2=0x7fe1d40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40040a0 0x7fe1d40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:29.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.136+0000 7fe1fa800640 1 -- 172.21.15.143:0/1116314811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4003530 con 0x7fe1d4004d10 2024-01-15T18:12:29.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1fa800640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4004d10 0x7fe1d4005190 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009fc0 tx=0x7fe1ec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486483e1dc8bd489 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:29.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0a5c70 con 0x7fe1d4004d10 2024-01-15T18:12:29.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec00f040 con 0x7fe1d4004d10 2024-01-15T18:12:29.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1f8a00640 1 -- 172.21.15.143:0/1116314811 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0ad440 con 0x7fe1d4004d10 2024-01-15T18:12:29.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1fc800640 1 -- 172.21.15.143:0/1116314811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4004d10 msgr2=0x7fe1d4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:29.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1fc800640 1 --2- 172.21.15.143:0/1116314811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4004d10 0x7fe1d4005190 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009fc0 tx=0x7fe1ec01cc60 comp rx=0 tx=0).stop 2024-01-15T18:12:29.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.140+0000 7fe1fa800640 1 -- 172.21.15.143:0/1116314811 reap_dead start 2024-01-15T18:12:29.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- 172.21.15.143:0/1116314811 shutdown_connections 2024-01-15T18:12:29.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- 172.21.15.143:0/1116314811 >> 172.21.15.143:0/1116314811 conn(0x7fe1f40b1610 msgr2=0x7fe1f40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:29.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- 172.21.15.143:0/1116314811 shutdown_connections 2024-01-15T18:12:29.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- 172.21.15.143:0/1116314811 wait complete. 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 Processor -- start 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- start start 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4004d10 0x7fe1f413c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 0x7fe1f415ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f4160500 0x7fe1f414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40acb60 con 0x7fe1f41f6be0 2024-01-15T18:12:29.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40ace40 con 0x7fe1f4160500 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40ad120 con 0x7fe1d4004d10 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 0x7fe1f415ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 0x7fe1f415ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52724/0 (socket says 172.21.15.143:52724) 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 -- 172.21.15.143:0/2367950315 learned_addr learned my addr 172.21.15.143:0/2367950315 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1fa800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f4160500 0x7fe1f414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9e00640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4004d10 0x7fe1f413c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 -- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4004d10 msgr2=0x7fe1f413c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4004d10 0x7fe1f413c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 -- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f4160500 msgr2=0x7fe1f414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f4160500 0x7fe1f414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 -- 172.21.15.143:0/2367950315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40ad400 con 0x7fe1f41f6be0 2024-01-15T18:12:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.144+0000 7fe1f9400640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 0x7fe1f415ffc0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00f770 tx=0x7fe1dc00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0014b0 con 0x7fe1f41f6be0 2024-01-15T18:12:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc00fe80 con 0x7fe1f41f6be0 2024-01-15T18:12:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f40ad6e0 con 0x7fe1f41f6be0 2024-01-15T18:12:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00d730 con 0x7fe1f41f6be0 2024-01-15T18:12:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f414e450 con 0x7fe1f41f6be0 2024-01-15T18:12:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f4074b00 con 0x7fe1f41f6be0 2024-01-15T18:12:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 23) v1 ==== 87129+0+0 (secure 0 0 0) 0x7fe1dc00d890 con 0x7fe1f41f6be0 2024-01-15T18:12:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f2000640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc0673a0 0x7fe1cc069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe1dc082e90 con 0x7fe1f41f6be0 2024-01-15T18:12:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f9e00640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc0673a0 0x7fe1cc069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.148+0000 7fe1f9e00640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc0673a0 0x7fe1cc069860 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40054a0 tx=0x7fe1e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.152+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1dc051b20 con 0x7fe1f41f6be0 2024-01-15T18:12:29.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:29.332+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1f406b380 con 0x7fe1cc0673a0 2024-01-15T18:12:31.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:31.708+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 24) v1 ==== 87180+0+0 (secure 0 0 0) 0x7fe1dc051ef0 con 0x7fe1f41f6be0 2024-01-15T18:12:33.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.936+0000 7fe1f2000640 1 -- 172.21.15.143:0/2367950315 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fe1e4013ad0 con 0x7fe1cc0673a0 2024-01-15T18:12:33.939 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:33.939 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:33.940 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:33.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc0673a0 msgr2=0x7fe1cc069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc0673a0 0x7fe1cc069860 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40054a0 tx=0x7fe1e4009040 comp rx=0 tx=0).stop 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 msgr2=0x7fe1f415ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 0x7fe1f415ffc0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc00f770 tx=0x7fe1dc00fc30 comp rx=0 tx=0).stop 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 shutdown_connections 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc0673a0 0x7fe1cc069860 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f4160500 0x7fe1f414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1f41f6be0 0x7fe1f415ffc0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 --2- 172.21.15.143:0/2367950315 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4004d10 0x7fe1f413c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 >> 172.21.15.143:0/2367950315 conn(0x7fe1f40b1610 msgr2=0x7fe1f415c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:33.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 shutdown_connections 2024-01-15T18:12:33.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:33.940+0000 7fe1fc800640 1 -- 172.21.15.143:0/2367950315 wait complete. 2024-01-15T18:12:33.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:33.948+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:33.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:33.948+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:33.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:33.948+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:33.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:33.948+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:33.951 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:12:33.948+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:33.975 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:12:34.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99314 2024-01-15T18:12:35.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.388+0000 7fc87b000640 1 Processor -- start 2024-01-15T18:12:35.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.388+0000 7fc87b000640 1 -- start start 2024-01-15T18:12:35.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc87b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 0x7fc8740ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:35.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc87b000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc87415f0c0 0x7fc87415f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:35.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc87b000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87415bb60 con 0x7fc874158a50 2024-01-15T18:12:35.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc87b000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87415b970 con 0x7fc87415f0c0 2024-01-15T18:12:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc87b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87415be40 con 0x7fc87415b510 2024-01-15T18:12:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc879c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc874158a50 0x7fc8740716f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35460/0 (socket says 172.21.15.143:35460) 2024-01-15T18:12:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc879200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 0x7fc8740ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc879200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 0x7fc8740ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42434/0 (socket says 172.21.15.143:42434) 2024-01-15T18:12:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc879200640 1 -- 172.21.15.143:0/377311368 learned_addr learned my addr 172.21.15.143:0/377311368 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:35.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.392+0000 7fc87a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc87415f0c0 0x7fc87415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:35.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.396+0000 7fc879200640 1 -- 172.21.15.143:0/377311368 >> v2:172.21.15.204:3300/0 conn(0x7fc87415f0c0 msgr2=0x7fc87415f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:35.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.396+0000 7fc879200640 1 --2- 172.21.15.143:0/377311368 >> v2:172.21.15.204:3300/0 conn(0x7fc87415f0c0 0x7fc87415f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:35.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.396+0000 7fc879200640 1 -- 172.21.15.143:0/377311368 >> v1:172.21.15.143:6789/0 conn(0x7fc874158a50 legacy=0x7fc8740716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:35.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.396+0000 7fc879200640 1 -- 172.21.15.143:0/377311368 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87415c120 con 0x7fc87415b510 2024-01-15T18:12:35.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.404+0000 7fc879200640 1 --2- 172.21.15.143:0/377311368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 0x7fc8740ce5d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009e50 tx=0x7fc86c01c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74164750f4f0cfbc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:35.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.404+0000 7fc873e00640 1 -- 172.21.15.143:0/377311368 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0a5760 con 0x7fc87415b510 2024-01-15T18:12:35.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.404+0000 7fc873e00640 1 -- 172.21.15.143:0/377311368 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c0a58c0 con 0x7fc87415b510 2024-01-15T18:12:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.404+0000 7fc87b000640 1 -- 172.21.15.143:0/377311368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 msgr2=0x7fc8740ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.404+0000 7fc87b000640 1 --2- 172.21.15.143:0/377311368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 0x7fc8740ce5d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009e50 tx=0x7fc86c01c9f0 comp rx=0 tx=0).stop 2024-01-15T18:12:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.408+0000 7fc87b000640 1 -- 172.21.15.143:0/377311368 shutdown_connections 2024-01-15T18:12:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.408+0000 7fc87b000640 1 --2- 172.21.15.143:0/377311368 >> v2:172.21.15.204:3300/0 conn(0x7fc87415f0c0 0x7fc87415f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.408+0000 7fc87b000640 1 --2- 172.21.15.143:0/377311368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415b510 0x7fc8740ce5d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:35.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.408+0000 7fc87b000640 1 -- 172.21.15.143:0/377311368 >> 172.21.15.143:0/377311368 conn(0x7fc8740b0b10 msgr2=0x7fc8740b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:35.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.408+0000 7fc87b000640 1 -- 172.21.15.143:0/377311368 shutdown_connections 2024-01-15T18:12:35.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.408+0000 7fc87b000640 1 -- 172.21.15.143:0/377311368 wait complete. 2024-01-15T18:12:35.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.412+0000 7fc87b000640 1 Processor -- start 2024-01-15T18:12:35.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.412+0000 7fc87b000640 1 -- start start 2024-01-15T18:12:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc874158a50 0x7fc874157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc87415b510 0x7fc87413b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415f0c0 0x7fc874157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874166ea0 con 0x7fc87415b510 2024-01-15T18:12:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874167140 con 0x7fc874158a50 2024-01-15T18:12:35.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc874167420 con 0x7fc87415f0c0 2024-01-15T18:12:35.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415f0c0 0x7fc874157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:35.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.416+0000 7fc87a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415f0c0 0x7fc874157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42440/0 (socket says 172.21.15.143:42440) 2024-01-15T18:12:35.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.420+0000 7fc87a600640 1 -- 172.21.15.143:0/589007748 learned_addr learned my addr 172.21.15.143:0/589007748 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:35.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.420+0000 7fc879c00640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc874158a50 0x7fc874157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:35.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.420+0000 7fc879200640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc87415b510 0x7fc87413b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:35.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.424+0000 7fc879200640 1 -- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415f0c0 msgr2=0x7fc874157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:35.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.424+0000 7fc879200640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415f0c0 0x7fc874157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:35.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.424+0000 7fc879200640 1 -- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc874158a50 msgr2=0x7fc874157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:35.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.424+0000 7fc879200640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc874158a50 0x7fc874157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:35.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.424+0000 7fc879200640 1 -- 172.21.15.143:0/589007748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c009b30 con 0x7fc87415b510 2024-01-15T18:12:35.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.424+0000 7fc879200640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc87415b510 0x7fc87413b6e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0ab000 tx=0x7fc86c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.428+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0a5760 con 0x7fc87415b510 2024-01-15T18:12:35.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.432+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c0090d0 con 0x7fc87415b510 2024-01-15T18:12:35.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.432+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0b86b0 con 0x7fc87415b510 2024-01-15T18:12:35.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.432+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874071f80 con 0x7fc87415b510 2024-01-15T18:12:35.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.436+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741679e0 con 0x7fc87415b510 2024-01-15T18:12:35.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.436+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874167bd0 con 0x7fc87415b510 2024-01-15T18:12:35.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.440+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fc86c0b8810 con 0x7fc87415b510 2024-01-15T18:12:35.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.440+0000 7fc871600640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc84c067510 0x7fc84c0699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:35.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.444+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc86c1272f0 con 0x7fc87415b510 2024-01-15T18:12:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.444+0000 7fc879c00640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc84c067510 0x7fc84c0699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:35.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.444+0000 7fc879c00640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc84c067510 0x7fc84c0699d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc85c007240 tx=0x7fc85c007e70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:35.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.452+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc86c125060 con 0x7fc87415b510 2024-01-15T18:12:35.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.916+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7fc874071e00 con 0x7fc84c067510 2024-01-15T18:12:35.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:35.988+0000 7fc871600640 1 -- 172.21.15.143:0/589007748 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fc85c005e20 con 0x7fc84c067510 2024-01-15T18:12:35.993 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:35.993 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:12:35.993 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:12:35.993 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:12:35.994 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:12:35.995 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:35.995 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:35.995 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc84c067510 msgr2=0x7fc84c0699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc84c067510 0x7fc84c0699d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc85c007240 tx=0x7fc85c007e70 comp rx=0 tx=0).stop 2024-01-15T18:12:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc87415b510 msgr2=0x7fc87413b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc87415b510 0x7fc87413b6e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0ab000 tx=0x7fc86c019790 comp rx=0 tx=0).stop 2024-01-15T18:12:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 shutdown_connections 2024-01-15T18:12:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc84c067510 0x7fc84c0699d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc87415f0c0 0x7fc874157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc87415b510 0x7fc87413b6e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 --2- 172.21.15.143:0/589007748 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc874158a50 0x7fc874157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.000+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 >> 172.21.15.143:0/589007748 conn(0x7fc8740b0b10 msgr2=0x7fc8740cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.004+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 shutdown_connections 2024-01-15T18:12:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.004+0000 7fc87b000640 1 -- 172.21.15.143:0/589007748 wait complete. 2024-01-15T18:12:36.097 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:12:36.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99317 2024-01-15T18:12:36.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.492+0000 7f7b7b000640 1 Processor -- start 2024-01-15T18:12:36.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.492+0000 7f7b7b000640 1 -- start start 2024-01-15T18:12:36.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.496+0000 7f7b7b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b740ce7c0 0x7f7b740b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:36.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.496+0000 7f7b7b000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740ce410 con 0x7f7b74159270 2024-01-15T18:12:36.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.496+0000 7f7b7b000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b740b3510 con 0x7f7b7406d010 2024-01-15T18:12:36.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.496+0000 7f7b7b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740b28e0 con 0x7f7b740ce7c0 2024-01-15T18:12:36.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.496+0000 7f7b7a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b740ce7c0 0x7f7b740b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:36.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.500+0000 7f7b79c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7b74159270 0x7f7b7406c900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35476/0 (socket says 172.21.15.143:35476) 2024-01-15T18:12:36.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.500+0000 7f7b79c00640 1 -- 172.21.15.143:0/3599002763 learned_addr learned my addr 172.21.15.143:0/3599002763 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:36.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.500+0000 7f7b73e00640 1 -- 172.21.15.143:0/3599002763 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382588979 0 0) 0x7f7b740b3510 con 0x7f7b7406d010 2024-01-15T18:12:36.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.500+0000 7f7b7a600640 1 -- 172.21.15.143:0/3599002763 >> v1:172.21.15.204:6789/0 conn(0x7f7b7406d010 legacy=0x7f7b740cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.500+0000 7f7b7a600640 1 -- 172.21.15.143:0/3599002763 >> v1:172.21.15.143:6789/0 conn(0x7f7b74159270 legacy=0x7f7b7406c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.504+0000 7f7b7a600640 1 -- 172.21.15.143:0/3599002763 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b740b3740 con 0x7f7b740ce7c0 2024-01-15T18:12:36.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.512+0000 7f7b7a600640 1 --2- 172.21.15.143:0/3599002763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b740ce7c0 0x7f7b740b2fd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00c310 tx=0x7f7b6c01dc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf278b551de2116 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:36.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.516+0000 7f7b73e00640 1 -- 172.21.15.143:0/3599002763 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0a6c80 con 0x7f7b740ce7c0 2024-01-15T18:12:36.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.516+0000 7f7b73e00640 1 -- 172.21.15.143:0/3599002763 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0a6de0 con 0x7f7b740ce7c0 2024-01-15T18:12:36.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.516+0000 7f7b73e00640 1 -- 172.21.15.143:0/3599002763 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00a2f0 con 0x7f7b740ce7c0 2024-01-15T18:12:36.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 -- 172.21.15.143:0/3599002763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b740ce7c0 msgr2=0x7f7b740b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 --2- 172.21.15.143:0/3599002763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b740ce7c0 0x7f7b740b2fd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00c310 tx=0x7f7b6c01dc40 comp rx=0 tx=0).stop 2024-01-15T18:12:36.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 -- 172.21.15.143:0/3599002763 shutdown_connections 2024-01-15T18:12:36.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 --2- 172.21.15.143:0/3599002763 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b740ce7c0 0x7f7b740b2fd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 -- 172.21.15.143:0/3599002763 >> 172.21.15.143:0/3599002763 conn(0x7f7b7406a890 msgr2=0x7f7b7406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:36.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 -- 172.21.15.143:0/3599002763 shutdown_connections 2024-01-15T18:12:36.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.520+0000 7f7b7b000640 1 -- 172.21.15.143:0/3599002763 wait complete. 2024-01-15T18:12:36.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.528+0000 7f7b7b000640 1 Processor -- start 2024-01-15T18:12:36.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.528+0000 7f7b7b000640 1 -- start start 2024-01-15T18:12:36.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.528+0000 7f7b7b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b7406d010 0x7f7b74155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:36.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b7b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b740ce7c0 0x7f7b7414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:36.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b7b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b74159270 0x7f7b740c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:36.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b7406d010 0x7f7b74155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:36.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b7b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74155c50 con 0x7f7b740ce7c0 2024-01-15T18:12:36.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b7b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74154670 con 0x7f7b74159270 2024-01-15T18:12:36.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b7b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74154950 con 0x7f7b7406d010 2024-01-15T18:12:36.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b7406d010 0x7f7b74155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42464/0 (socket says 172.21.15.143:42464) 2024-01-15T18:12:36.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79c00640 1 -- 172.21.15.143:0/537227274 learned_addr learned my addr 172.21.15.143:0/537227274 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:36.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b740ce7c0 0x7f7b7414ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:36.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 -- 172.21.15.143:0/537227274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b7406d010 msgr2=0x7f7b74155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b7406d010 0x7f7b74155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 -- 172.21.15.143:0/537227274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b74159270 msgr2=0x7f7b740c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:36.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b74159270 0x7f7b740c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:36.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 -- 172.21.15.143:0/537227274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c00c050 con 0x7f7b740ce7c0 2024-01-15T18:12:36.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b79200640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b740ce7c0 0x7f7b7414ad60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7b640011f0 tx=0x7f7b6400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:36.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640054e0 con 0x7f7b740ce7c0 2024-01-15T18:12:36.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b64003600 con 0x7f7b740ce7c0 2024-01-15T18:12:36.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64003a10 con 0x7f7b740ce7c0 2024-01-15T18:12:36.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.532+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b74154c30 con 0x7f7b740ce7c0 2024-01-15T18:12:36.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.536+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b741551f0 con 0x7f7b740ce7c0 2024-01-15T18:12:36.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.540+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b44005260 con 0x7f7b740ce7c0 2024-01-15T18:12:36.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.544+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f7b64003b70 con 0x7f7b740ce7c0 2024-01-15T18:12:36.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.548+0000 7f7b71600640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7b4c0674c0 0x7f7b4c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:36.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.556+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7b64003e20 con 0x7f7b740ce7c0 2024-01-15T18:12:36.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.560+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7b64050fe0 con 0x7f7b740ce7c0 2024-01-15T18:12:36.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.564+0000 7f7b79c00640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7b4c0674c0 0x7f7b4c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:36.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:36.572+0000 7f7b79c00640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7b4c0674c0 0x7f7b4c069980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000f70 tx=0x7f7b5c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:37.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.060+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99317", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b44000bd0 con 0x7f7b4c0674c0 2024-01-15T18:12:37.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.124+0000 7f7b71600640 1 -- 172.21.15.143:0/537227274 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f7b44000bd0 con 0x7f7b4c0674c0 2024-01-15T18:12:37.125 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:37.125 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.126 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.128+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7b4c0674c0 msgr2=0x7f7b4c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.128+0000 7f7b7b000640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7b4c0674c0 0x7f7b4c069980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000f70 tx=0x7f7b5c00b040 comp rx=0 tx=0).stop 2024-01-15T18:12:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.128+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b740ce7c0 msgr2=0x7f7b7414ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.128+0000 7f7b7b000640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b740ce7c0 0x7f7b7414ad60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f7b640011f0 tx=0x7f7b6400c8c0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 shutdown_connections 2024-01-15T18:12:37.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7b4c0674c0 0x7f7b4c069980 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7b74159270 0x7f7b740c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b740ce7c0 0x7f7b7414ad60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 --2- 172.21.15.143:0/537227274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7b7406d010 0x7f7b74155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 >> 172.21.15.143:0/537227274 conn(0x7f7b7406a890 msgr2=0x7f7b74159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 shutdown_connections 2024-01-15T18:12:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.132+0000 7f7b7b000640 1 -- 172.21.15.143:0/537227274 wait complete. 2024-01-15T18:12:37.194 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:12:37.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 Processor -- start 2024-01-15T18:12:37.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 -- start start 2024-01-15T18:12:37.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fdb4c141660 0x7fdb4c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:37.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb4c071d00 0x7fdb4c072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:37.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c072650 con 0x7fdb4c15ec80 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4c15fbb0 con 0x7fdb4c071d00 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb53400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4c15f130 con 0x7fdb4c141660 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb51400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb4c071d00 0x7fdb4c072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb51400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdb4c071d00 0x7fdb4c072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52502/0 (socket says 172.21.15.143:52502) 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb4be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdb4c15ec80 0x7fdb4c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43870/0 (socket says 172.21.15.143:43870) 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.328+0000 7fdb51400640 1 -- 172.21.15.143:0/350976045 learned_addr learned my addr 172.21.15.143:0/350976045 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:37.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb50a00640 1 --2- 172.21.15.143:0/350976045 >> v2:172.21.15.204:3300/0 conn(0x7fdb4c141660 0x7fdb4c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:37.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb51400640 1 -- 172.21.15.143:0/350976045 >> v2:172.21.15.204:3300/0 conn(0x7fdb4c141660 msgr2=0x7fdb4c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:37.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb51400640 1 --2- 172.21.15.143:0/350976045 >> v2:172.21.15.204:3300/0 conn(0x7fdb4c141660 0x7fdb4c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb51400640 1 -- 172.21.15.143:0/350976045 >> v1:172.21.15.143:6789/0 conn(0x7fdb4c15ec80 legacy=0x7fdb4c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb51400640 1 -- 172.21.15.143:0/350976045 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c15f310 con 0x7fdb4c071d00 2024-01-15T18:12:37.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb50a00640 1 --2- 172.21.15.143:0/350976045 >> v2:172.21.15.204:3300/0 conn(0x7fdb4c141660 0x7fdb4c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:37.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb51400640 1 --2- 172.21.15.143:0/350976045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c071d00 0x7fdb4c072110 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0b1aa0 tx=0x7fdb4401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a073e6cfb0b97ec server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:37.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb4b400640 1 -- 172.21.15.143:0/350976045 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a5860 con 0x7fdb4c071d00 2024-01-15T18:12:37.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb4b400640 1 -- 172.21.15.143:0/350976045 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb440a59c0 con 0x7fdb4c071d00 2024-01-15T18:12:37.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.332+0000 7fdb4b400640 1 -- 172.21.15.143:0/350976045 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a5d50 con 0x7fdb4c071d00 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- 172.21.15.143:0/350976045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c071d00 msgr2=0x7fdb4c072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 --2- 172.21.15.143:0/350976045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c071d00 0x7fdb4c072110 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0b1aa0 tx=0x7fdb4401c9a0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- 172.21.15.143:0/350976045 shutdown_connections 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 --2- 172.21.15.143:0/350976045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c071d00 0x7fdb4c072110 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 --2- 172.21.15.143:0/350976045 >> v2:172.21.15.204:3300/0 conn(0x7fdb4c141660 0x7fdb4c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- 172.21.15.143:0/350976045 >> 172.21.15.143:0/350976045 conn(0x7fdb4c06c3f0 msgr2=0x7fdb4c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- 172.21.15.143:0/350976045 shutdown_connections 2024-01-15T18:12:37.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- 172.21.15.143:0/350976045 wait complete. 2024-01-15T18:12:37.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 Processor -- start 2024-01-15T18:12:37.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- start start 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c071d00 0x7fdb4c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 0x7fdb4c158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb4c15ec80 0x7fdb4c13ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c15a4e0 con 0x7fdb4c141660 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c137d10 con 0x7fdb4c071d00 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb53400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c137ff0 con 0x7fdb4c15ec80 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 0x7fdb4c158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 0x7fdb4c158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52506/0 (socket says 172.21.15.143:52506) 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 -- 172.21.15.143:0/2262916994 learned_addr learned my addr 172.21.15.143:0/2262916994 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb51400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb4c15ec80 0x7fdb4c13ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb50a00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c071d00 0x7fdb4c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 -- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb4c15ec80 msgr2=0x7fdb4c13ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb4c15ec80 0x7fdb4c13ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 -- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c071d00 msgr2=0x7fdb4c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c071d00 0x7fdb4c15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.336+0000 7fdb4be00640 1 -- 172.21.15.143:0/2262916994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009b30 con 0x7fdb4c141660 2024-01-15T18:12:37.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb4be00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 0x7fdb4c158970 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c490 tx=0x7fdb3400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:37.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34007480 con 0x7fdb4c141660 2024-01-15T18:12:37.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb340075e0 con 0x7fdb4c141660 2024-01-15T18:12:37.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34004a10 con 0x7fdb4c141660 2024-01-15T18:12:37.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c1382d0 con 0x7fdb4c141660 2024-01-15T18:12:37.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c138890 con 0x7fdb4c141660 2024-01-15T18:12:37.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c067500 con 0x7fdb4c141660 2024-01-15T18:12:37.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fdb34004b70 con 0x7fdb4c141660 2024-01-15T18:12:37.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb48c00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdb240674c0 0x7fdb24069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:37.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdb34082020 con 0x7fdb4c141660 2024-01-15T18:12:37.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.340+0000 7fdb50a00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdb240674c0 0x7fdb24069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:37.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.344+0000 7fdb50a00640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdb240674c0 0x7fdb24069980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c06dee0 tx=0x7fdb3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:37.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.344+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdb34086e60 con 0x7fdb4c141660 2024-01-15T18:12:37.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdb4c072650 con 0x7fdb4c141660 2024-01-15T18:12:37.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb48c00640 1 -- 172.21.15.143:0/2262916994 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2801 (secure 0 0 0) 0x7fdb4c072650 con 0x7fdb4c141660 2024-01-15T18:12:37.550 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:37.550 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:37.550 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:37.551 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 483, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:37.552 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:37.553 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1040, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 4216420255, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 8512155648, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 764581957632, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 87662519, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 82790333, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 21, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 25 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:37.554 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:37.555 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 10, 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:30.772610+0000", 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:12:37.556 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:12:37.557 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:37.558 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdb240674c0 msgr2=0x7fdb24069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdb240674c0 0x7fdb24069980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c06dee0 tx=0x7fdb3c008040 comp rx=0 tx=0).stop 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 msgr2=0x7fdb4c158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 0x7fdb4c158970 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c490 tx=0x7fdb3400c950 comp rx=0 tx=0).stop 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 shutdown_connections 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdb240674c0 0x7fdb24069980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdb4c15ec80 0x7fdb4c13ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb4c141660 0x7fdb4c158970 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 --2- 172.21.15.143:0/2262916994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c071d00 0x7fdb4c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:37.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 >> 172.21.15.143:0/2262916994 conn(0x7fdb4c06c3f0 msgr2=0x7fdb4c0afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:37.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 shutdown_connections 2024-01-15T18:12:37.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:37.548+0000 7fdb53400640 1 -- 172.21.15.143:0/2262916994 wait complete. 2024-01-15T18:12:38.573 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:38.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.716+0000 7fe3ea000640 1 Processor -- start 2024-01-15T18:12:38.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.716+0000 7fe3ea000640 1 -- start start 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3ea000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3e40b0310 0x7fe3e40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40cf740 0x7fe3e40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3ea000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40cbff0 con 0x7fe3e406bfb0 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3ea000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e40d0c50 con 0x7fe3e40b0310 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3ea000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40cb6c0 con 0x7fe3e40cf740 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3e40b0310 0x7fe3e40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3e40b0310 0x7fe3e40cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52522/0 (socket says 172.21.15.143:52522) 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 -- 172.21.15.143:0/559536814 learned_addr learned my addr 172.21.15.143:0/559536814 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:38.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e3400640 1 --1- 172.21.15.143:0/559536814 >> v1:172.21.15.143:6789/0 conn(0x7fe3e406bfb0 0x7fe3e406c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43874/0 (socket says 172.21.15.143:43874) 2024-01-15T18:12:38.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/559536814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40cf740 0x7fe3e40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:38.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 -- 172.21.15.143:0/559536814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40cf740 msgr2=0x7fe3e40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:38.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 --2- 172.21.15.143:0/559536814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40cf740 0x7fe3e40cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 -- 172.21.15.143:0/559536814 >> v1:172.21.15.143:6789/0 conn(0x7fe3e406bfb0 legacy=0x7fe3e406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:38.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.720+0000 7fe3e2a00640 1 -- 172.21.15.143:0/559536814 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40cb9a0 con 0x7fe3e40b0310 2024-01-15T18:12:38.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3e2a00640 1 --2- 172.21.15.143:0/559536814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e40cf200 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009870 tx=0x7fe3cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250b89021c7249e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:38.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3e2000640 1 -- 172.21.15.143:0/559536814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0ab680 con 0x7fe3e40b0310 2024-01-15T18:12:38.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3e2000640 1 -- 172.21.15.143:0/559536814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0ab7e0 con 0x7fe3e40b0310 2024-01-15T18:12:38.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3e2000640 1 -- 172.21.15.143:0/559536814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0abb70 con 0x7fe3e40b0310 2024-01-15T18:12:38.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 -- 172.21.15.143:0/559536814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 msgr2=0x7fe3e40cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:38.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 --2- 172.21.15.143:0/559536814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e40cf200 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009870 tx=0x7fe3cc01c480 comp rx=0 tx=0).stop 2024-01-15T18:12:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 -- 172.21.15.143:0/559536814 shutdown_connections 2024-01-15T18:12:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 --2- 172.21.15.143:0/559536814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40cf740 0x7fe3e40cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 --2- 172.21.15.143:0/559536814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e40cf200 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 -- 172.21.15.143:0/559536814 >> 172.21.15.143:0/559536814 conn(0x7fe3e40d0ed0 msgr2=0x7fe3e40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 -- 172.21.15.143:0/559536814 shutdown_connections 2024-01-15T18:12:38.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 -- 172.21.15.143:0/559536814 wait complete. 2024-01-15T18:12:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 Processor -- start 2024-01-15T18:12:38.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.724+0000 7fe3ea000640 1 -- start start 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e406bfb0 0x7fe3e414ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e40cf740 0x7fe3e414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41507d0 con 0x7fe3e40b0310 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4150ab0 con 0x7fe3e40cf740 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4150d90 con 0x7fe3e406bfb0 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e415b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52524/0 (socket says 172.21.15.143:52524) 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 -- 172.21.15.143:0/357438889 learned_addr learned my addr 172.21.15.143:0/357438889 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e40cf740 0x7fe3e414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:38.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e3400640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e406bfb0 0x7fe3e414ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 -- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e406bfb0 msgr2=0x7fe3e414ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e406bfb0 0x7fe3e414ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 -- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e40cf740 msgr2=0x7fe3e414f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e40cf740 0x7fe3e414f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 -- 172.21.15.143:0/357438889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0094d0 con 0x7fe3e40b0310 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3e2a00640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e415b300 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097c0 tx=0x7fe3cc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0b4800 con 0x7fe3e40b0310 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0b4980 con 0x7fe3e40b0310 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0b4d30 con 0x7fe3e40b0310 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e4151070 con 0x7fe3e40b0310 2024-01-15T18:12:38.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e415b840 con 0x7fe3e40b0310 2024-01-15T18:12:38.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.728+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e415ba30 con 0x7fe3e40b0310 2024-01-15T18:12:38.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.732+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fe3cc0c7440 con 0x7fe3e40b0310 2024-01-15T18:12:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.732+0000 7fe3db400640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc0674f0 0x7fe3bc0699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.732+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe3cc11dda0 con 0x7fe3e40b0310 2024-01-15T18:12:38.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.732+0000 7fe3e3400640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc0674f0 0x7fe3bc0699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:38.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.732+0000 7fe3e3400640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc0674f0 0x7fe3bc0699b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000f70 tx=0x7fe3d4007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:38.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.736+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3cc0ec880 con 0x7fe3e40b0310 2024-01-15T18:12:38.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe3e40ce660 con 0x7fe3e40b0310 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3db400640 1 -- 172.21.15.143:0/357438889 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2801 (secure 0 0 0) 0x7fe3cc0efea0 con 0x7fe3e40b0310 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:38.954 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 485, 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.955 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.956 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1040, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 4216420255, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 8512155648, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 764581957632, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 87662519, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 82790333, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 21, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 25 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:38.957 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:38.958 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 10, 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:30.772610+0000", 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:12:38.959 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:38.960 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0, 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:38.961 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc0674f0 msgr2=0x7fe3bc0699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc0674f0 0x7fe3bc0699b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4000f70 tx=0x7fe3d4007e50 comp rx=0 tx=0).stop 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 msgr2=0x7fe3e415b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e415b300 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097c0 tx=0x7fe3cc01acc0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 shutdown_connections 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc0674f0 0x7fe3bc0699b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e40cf740 0x7fe3e414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40b0310 0x7fe3e415b300 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 --2- 172.21.15.143:0/357438889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e406bfb0 0x7fe3e414ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 >> 172.21.15.143:0/357438889 conn(0x7fe3e40d0ed0 msgr2=0x7fe3e40d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:38.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 shutdown_connections 2024-01-15T18:12:38.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:38.952+0000 7fe3ea000640 1 -- 172.21.15.143:0/357438889 wait complete. 2024-01-15T18:12:39.977 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:40.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 Processor -- start 2024-01-15T18:12:40.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 -- start start 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc0b2c30 0x7f74fc06ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f74fc06bab0 0x7f74fc070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74fc06e5c0 0x7f74fc071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74fc160c90 con 0x7f74fc06e5c0 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74fc160f70 con 0x7f74fc06bab0 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.120+0000 7f7501c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc161250 con 0x7f74fc0b2c30 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74fc06e5c0 0x7f74fc071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f74fc06e5c0 0x7f74fc071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52540/0 (socket says 172.21.15.143:52540) 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 -- 172.21.15.143:0/2508750067 learned_addr learned my addr 172.21.15.143:0/2508750067 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74faa00640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc0b2c30 0x7f74fc06ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fa000640 1 --2- 172.21.15.143:0/2508750067 >> v2:172.21.15.204:3300/0 conn(0x7f74fc06bab0 0x7f74fc070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 -- 172.21.15.143:0/2508750067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc0b2c30 msgr2=0x7f74fc06ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc0b2c30 0x7f74fc06ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 -- 172.21.15.143:0/2508750067 >> v2:172.21.15.204:3300/0 conn(0x7f74fc06bab0 msgr2=0x7f74fc070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 --2- 172.21.15.143:0/2508750067 >> v2:172.21.15.204:3300/0 conn(0x7f74fc06bab0 0x7f74fc070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 -- 172.21.15.143:0/2508750067 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc161530 con 0x7f74fc06e5c0 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fa000640 1 --2- 172.21.15.143:0/2508750067 >> v2:172.21.15.204:3300/0 conn(0x7f74fc06bab0 0x7f74fc070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:40.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74faa00640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc0b2c30 0x7f74fc06ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:40.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74fb400640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc06e5c0 0x7f74fc071200 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0718c0 tx=0x7f74f401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce3e00e358a053b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:40.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74f9600640 1 -- 172.21.15.143:0/2508750067 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f40a5730 con 0x7f74fc06e5c0 2024-01-15T18:12:40.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74f9600640 1 -- 172.21.15.143:0/2508750067 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f40a5890 con 0x7f74fc06e5c0 2024-01-15T18:12:40.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.124+0000 7f74f9600640 1 -- 172.21.15.143:0/2508750067 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f40a5c20 con 0x7f74fc06e5c0 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- 172.21.15.143:0/2508750067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc06e5c0 msgr2=0x7f74fc071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc06e5c0 0x7f74fc071200 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f74fc0718c0 tx=0x7f74f401c9a0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- 172.21.15.143:0/2508750067 shutdown_connections 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc06e5c0 0x7f74fc071200 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- 172.21.15.143:0/2508750067 >> v2:172.21.15.204:3300/0 conn(0x7f74fc06bab0 0x7f74fc070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- 172.21.15.143:0/2508750067 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc0b2c30 0x7f74fc06ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- 172.21.15.143:0/2508750067 >> 172.21.15.143:0/2508750067 conn(0x7f74fc0b1610 msgr2=0x7f74fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- 172.21.15.143:0/2508750067 shutdown_connections 2024-01-15T18:12:40.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- 172.21.15.143:0/2508750067 wait complete. 2024-01-15T18:12:40.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 Processor -- start 2024-01-15T18:12:40.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- start start 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74fc06bab0 0x7f74fc14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc06e5c0 0x7f74fc1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 0x7f74fc15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc0cc850 con 0x7f74fc0b2c30 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc152210 con 0x7f74fc06bab0 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.128+0000 7f7501c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74fc1524f0 con 0x7f74fc06e5c0 2024-01-15T18:12:40.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 0x7f74fc15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 0x7f74fc15bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52548/0 (socket says 172.21.15.143:52548) 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 -- 172.21.15.143:0/2943529327 learned_addr learned my addr 172.21.15.143:0/2943529327 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fa000640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc06e5c0 0x7f74fc1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74faa00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74fc06bab0 0x7f74fc14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 -- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc06e5c0 msgr2=0x7f74fc1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc06e5c0 0x7f74fc1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 -- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74fc06bab0 msgr2=0x7f74fc14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74fc06bab0 0x7f74fc14d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 -- 172.21.15.143:0/2943529327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f4009b30 con 0x7f74fc0b2c30 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74fb400640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 0x7f74fc15bec0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f74f401b050 tx=0x7f74f4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f40b8670 con 0x7f74fc0b2c30 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74f40b87d0 con 0x7f74fc0b2c30 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f40b8b80 con 0x7f74fc0b2c30 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74fc1527d0 con 0x7f74fc0b2c30 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74fc152d90 con 0x7f74fc0b2c30 2024-01-15T18:12:40.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74fc074b00 con 0x7f74fc0b2c30 2024-01-15T18:12:40.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.132+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f74f40b8ce0 con 0x7f74fc0b2c30 2024-01-15T18:12:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.136+0000 7f74f2a00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f74d40674c0 0x7f74d4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.136+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f74f411e480 con 0x7f74fc0b2c30 2024-01-15T18:12:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.136+0000 7f74faa00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f74d40674c0 0x7f74d4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:40.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.136+0000 7f74faa00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f74d40674c0 0x7f74d4069980 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009830 tx=0x7f74ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:40.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.140+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f74f40ed230 con 0x7f74fc0b2c30 2024-01-15T18:12:40.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f74fc0cc990 con 0x7f74fc0b2c30 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f74f2a00640 1 -- 172.21.15.143:0/2943529327 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2879 (secure 0 0 0) 0x7f74f40f0850 con 0x7f74fc0b2c30 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.342 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 486, 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:40.343 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:40.344 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1252, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 5101517172, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 10001186816, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 763092926464, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 163441324, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 156482612, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 39, 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 45 2024-01-15T18:12:40.345 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:40.346 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 10, 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:30.772610+0000", 2024-01-15T18:12:40.347 INFO:teuthology.orchestra.run.smithi143.stdout: "services": { 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "daemons": { 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "summary": "", 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "c": { 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "start_epoch": 0, 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "start_stamp": "0.000000", 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 0, 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "(unrecognized address family 0)/0", 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": {}, 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: "task_status": {} 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.348 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.16664999723434448, 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:40.349 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:40.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f74d40674c0 msgr2=0x7f74d4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:40.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f74d40674c0 0x7f74d4069980 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f74ec009830 tx=0x7f74ec008040 comp rx=0 tx=0).stop 2024-01-15T18:12:40.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 msgr2=0x7f74fc15bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:40.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 0x7f74fc15bec0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f74f401b050 tx=0x7f74f4019740 comp rx=0 tx=0).stop 2024-01-15T18:12:40.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 shutdown_connections 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f74d40674c0 0x7f74d4069980 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74fc0b2c30 0x7f74fc15bec0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f74fc06e5c0 0x7f74fc1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 --2- 172.21.15.143:0/2943529327 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f74fc06bab0 0x7f74fc14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 >> 172.21.15.143:0/2943529327 conn(0x7f74fc0b1610 msgr2=0x7f74fc06db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 shutdown_connections 2024-01-15T18:12:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:40.340+0000 7f7501c00640 1 -- 172.21.15.143:0/2943529327 wait complete. 2024-01-15T18:12:41.366 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:41.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fcbe00640 1 Processor -- start 2024-01-15T18:12:41.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fcbe00640 1 -- start start 2024-01-15T18:12:41.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fcbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fc40adaf0 0x7f8fc40c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:41.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fcbe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40adf80 con 0x7f8fc40c43e0 2024-01-15T18:12:41.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fcbe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40c47f0 con 0x7f8fc4161880 2024-01-15T18:12:41.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fcbe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc4161c60 con 0x7f8fc40adaf0 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fc40adaf0 0x7f8fc40c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc9e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8fc40c43e0 0x7f8fc40cfce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43876/0 (socket says 172.21.15.143:43876) 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8fc40adaf0 0x7f8fc40c3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52554/0 (socket says 172.21.15.143:52554) 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc9e00640 1 -- 172.21.15.143:0/3299336046 learned_addr learned my addr 172.21.15.143:0/3299336046 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc8a00640 1 -- 172.21.15.143:0/3299336046 >> v1:172.21.15.204:6789/0 conn(0x7f8fc4161880 legacy=0x7f8fc406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc8a00640 1 -- 172.21.15.143:0/3299336046 >> v1:172.21.15.143:6789/0 conn(0x7f8fc40c43e0 legacy=0x7f8fc40cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc3e00640 1 -- 172.21.15.143:0/3299336046 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843112525 0 0) 0x7f8fc40adf80 con 0x7f8fc40c43e0 2024-01-15T18:12:41.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.512+0000 7f8fc8a00640 1 -- 172.21.15.143:0/3299336046 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40cec40 con 0x7f8fc40adaf0 2024-01-15T18:12:41.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fc8a00640 1 --2- 172.21.15.143:0/3299336046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc40adaf0 0x7f8fc40c3ea0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0097c0 tx=0x7f8fac01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e2c7b401c1383e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:41.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fc3e00640 1 -- 172.21.15.143:0/3299336046 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0ab650 con 0x7f8fc40adaf0 2024-01-15T18:12:41.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fc3e00640 1 -- 172.21.15.143:0/3299336046 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac0ab7b0 con 0x7f8fc40adaf0 2024-01-15T18:12:41.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fc3e00640 1 -- 172.21.15.143:0/3299336046 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac0abb40 con 0x7f8fc40adaf0 2024-01-15T18:12:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3299336046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc40adaf0 msgr2=0x7f8fc40c3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3299336046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc40adaf0 0x7f8fc40c3ea0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8fac0097c0 tx=0x7f8fac01c450 comp rx=0 tx=0).stop 2024-01-15T18:12:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3299336046 shutdown_connections 2024-01-15T18:12:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3299336046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc40adaf0 0x7f8fc40c3ea0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3299336046 >> 172.21.15.143:0/3299336046 conn(0x7f8fc40d0ed0 msgr2=0x7f8fc40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3299336046 shutdown_connections 2024-01-15T18:12:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.516+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3299336046 wait complete. 2024-01-15T18:12:41.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 Processor -- start 2024-01-15T18:12:41.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 -- start start 2024-01-15T18:12:41.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc40adaf0 0x7f8fc406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:41.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8fc40c43e0 0x7f8fc4155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:41.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 0x7f8fc41fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:41.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc406bae0 con 0x7f8fc4161880 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40ce0c0 con 0x7f8fc40adaf0 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c8790 con 0x7f8fc40c43e0 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 0x7f8fc41fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 0x7f8fc41fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52556/0 (socket says 172.21.15.143:52556) 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 -- 172.21.15.143:0/3722527909 learned_addr learned my addr 172.21.15.143:0/3722527909 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9400640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc40adaf0 0x7f8fc406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc8a00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8fc40c43e0 0x7f8fc4155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 -- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8fc40c43e0 msgr2=0x7f8fc4155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8fc40c43e0 0x7f8fc4155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 -- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc40adaf0 msgr2=0x7f8fc406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc40adaf0 0x7f8fc406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 -- 172.21.15.143:0/3722527909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac009420 con 0x7f8fc4161880 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc9e00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 0x7f8fc41fd410 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c810 tx=0x7f8fbc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00a640 con 0x7f8fc4161880 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc00a7c0 con 0x7f8fc4161880 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc41609c0 con 0x7f8fc4161880 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00ab70 con 0x7f8fc4161880 2024-01-15T18:12:41.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.520+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40c8420 con 0x7f8fc4161880 2024-01-15T18:12:41.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.524+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40c7fa0 con 0x7f8fc4161880 2024-01-15T18:12:41.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.524+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f8fbc00acd0 con 0x7f8fc4161880 2024-01-15T18:12:41.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.524+0000 7f8fc1600640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f9c0674c0 0x7f8f9c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:41.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.524+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8fbc082870 con 0x7f8fc4161880 2024-01-15T18:12:41.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.524+0000 7f8fc9400640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f9c0674c0 0x7f8f9c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:41.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.524+0000 7f8fc9400640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f9c0674c0 0x7f8f9c069980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000f70 tx=0x7f8fb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:41.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.528+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8fc40c7fa0 con 0x7f8fc4161880 2024-01-15T18:12:41.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.732+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8fc40c7fa0 con 0x7f8fc4161880 2024-01-15T18:12:41.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.732+0000 7f8fc1600640 1 -- 172.21.15.143:0/3722527909 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f8fc40c7fa0 con 0x7f8fc4161880 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:41.737 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:41.738 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 488, 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:41.739 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:41.740 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1252, 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 5101517172, 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 10001186816, 2024-01-15T18:12:41.741 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 763092926464, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 163441066, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 156468371, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 39, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 41 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:41.742 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:41.743 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:41.744 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:41.745 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.16664999723434448, 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:41.746 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:41.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f9c0674c0 msgr2=0x7f8f9c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f9c0674c0 0x7f8f9c069980 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4000f70 tx=0x7f8fb4009040 comp rx=0 tx=0).stop 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 msgr2=0x7f8fc41fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 0x7f8fc41fd410 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c810 tx=0x7f8fbc00ccd0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 shutdown_connections 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f9c0674c0 0x7f8f9c069980 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fc4161880 0x7f8fc41fd410 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8fc40c43e0 0x7f8fc4155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 --2- 172.21.15.143:0/3722527909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8fc40adaf0 0x7f8fc406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 >> 172.21.15.143:0/3722527909 conn(0x7f8fc40d0ed0 msgr2=0x7f8fc415a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 shutdown_connections 2024-01-15T18:12:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:41.736+0000 7f8fcbe00640 1 -- 172.21.15.143:0/3722527909 wait complete. 2024-01-15T18:12:42.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:42.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceb0200640 1 Processor -- start 2024-01-15T18:12:42.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceb0200640 1 -- start start 2024-01-15T18:12:42.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceb0200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcea806c630 0x7fcea80b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:42.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceb0200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea80cb480 con 0x7fcea80b0490 2024-01-15T18:12:42.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceb0200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea80cb6d0 con 0x7fcea815d3f0 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceb0200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea80cb9b0 con 0x7fcea806c630 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceae200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcea806c630 0x7fcea80b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceace00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcea80b0490 0x7fcea806bf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43880/0 (socket says 172.21.15.143:43880) 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceace00640 1 -- 172.21.15.143:0/1678758245 learned_addr learned my addr 172.21.15.143:0/1678758245 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceae200640 1 -- 172.21.15.143:0/1678758245 >> v1:172.21.15.204:6789/0 conn(0x7fcea815d3f0 legacy=0x7fcea80cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceae200640 1 -- 172.21.15.143:0/1678758245 >> v1:172.21.15.143:6789/0 conn(0x7fcea80b0490 legacy=0x7fcea806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fceae200640 1 -- 172.21.15.143:0/1678758245 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea80cbc90 con 0x7fcea806c630 2024-01-15T18:12:42.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.904+0000 7fcea7e00640 1 -- 172.21.15.143:0/1678758245 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377228822 0 0) 0x7fcea80cb480 con 0x7fcea80b0490 2024-01-15T18:12:42.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceae200640 1 --2- 172.21.15.143:0/1678758245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80b1960 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009d90 tx=0x7fcea001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c54a8ffa2f54fb8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fcea7e00640 1 -- 172.21.15.143:0/1678758245 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea00a5a00 con 0x7fcea806c630 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fcea7e00640 1 -- 172.21.15.143:0/1678758245 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea00a5b60 con 0x7fcea806c630 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fcea7e00640 1 -- 172.21.15.143:0/1678758245 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea00ad440 con 0x7fcea806c630 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 -- 172.21.15.143:0/1678758245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 msgr2=0x7fcea80b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 --2- 172.21.15.143:0/1678758245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80b1960 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009d90 tx=0x7fcea001c920 comp rx=0 tx=0).stop 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 -- 172.21.15.143:0/1678758245 shutdown_connections 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 --2- 172.21.15.143:0/1678758245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80b1960 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:42.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 -- 172.21.15.143:0/1678758245 >> 172.21.15.143:0/1678758245 conn(0x7fcea80d0e30 msgr2=0x7fcea80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:42.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 -- 172.21.15.143:0/1678758245 shutdown_connections 2024-01-15T18:12:42.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.908+0000 7fceb0200640 1 -- 172.21.15.143:0/1678758245 wait complete. 2024-01-15T18:12:42.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 Processor -- start 2024-01-15T18:12:42.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 -- start start 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcea80b0490 0x7fcea8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcea815d3f0 0x7fcea8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea815e030 con 0x7fcea806c630 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea815c0f0 con 0x7fcea815d3f0 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea8160160 con 0x7fcea80b0490 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52574/0 (socket says 172.21.15.143:52574) 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 -- 172.21.15.143:0/96079082 learned_addr learned my addr 172.21.15.143:0/96079082 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:42.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceae200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcea815d3f0 0x7fcea8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceace00640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcea80b0490 0x7fcea8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 -- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcea80b0490 msgr2=0x7fcea8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcea80b0490 0x7fcea8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 -- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcea815d3f0 msgr2=0x7fcea8153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcea815d3f0 0x7fcea8153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 -- 172.21.15.143:0/96079082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0009a70 con 0x7fcea806c630 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcead800640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80cc5f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c500 tx=0x7fce9800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98005220 con 0x7fcea806c630 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce98005380 con 0x7fcea806c630 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98007270 con 0x7fcea806c630 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea8160440 con 0x7fcea806c630 2024-01-15T18:12:42.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.912+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea815d9e0 con 0x7fcea806c630 2024-01-15T18:12:42.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.916+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea8067500 con 0x7fcea806c630 2024-01-15T18:12:42.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.916+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fce980073d0 con 0x7fcea806c630 2024-01-15T18:12:42.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.916+0000 7fcea5600640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fce800674f0 0x7fce800699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:42.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.916+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fce98082370 con 0x7fcea806c630 2024-01-15T18:12:42.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.916+0000 7fceace00640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fce800674f0 0x7fce800699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:42.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.916+0000 7fceace00640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fce800674f0 0x7fce800699b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcea81f9410 tx=0x7fce90007240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:42.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:42.920+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fce98050dd0 con 0x7fcea806c630 2024-01-15T18:12:43.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.124+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcea80667b0 con 0x7fcea806c630 2024-01-15T18:12:43.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.124+0000 7fcea5600640 1 -- 172.21.15.143:0/96079082 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7fce980543f0 con 0x7fcea806c630 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:43.127 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 489, 2024-01-15T18:12:43.128 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:43.129 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1252, 2024-01-15T18:12:43.130 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 5101517172, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 10001186816, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 763092926464, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 163441066, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 156468371, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 39, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 41 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:43.131 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:43.132 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:43.133 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.16664999723434448, 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:43.134 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:43.135 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:43.135 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:43.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.124+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fce800674f0 msgr2=0x7fce800699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.124+0000 7fceb0200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fce800674f0 0x7fce800699b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcea81f9410 tx=0x7fce90007240 comp rx=0 tx=0).stop 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.124+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 msgr2=0x7fcea80cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.124+0000 7fceb0200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80cc5f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c500 tx=0x7fce9800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 shutdown_connections 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fce800674f0 0x7fce800699b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcea815d3f0 0x7fcea8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcea80b0490 0x7fcea8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 --2- 172.21.15.143:0/96079082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcea806c630 0x7fcea80cc5f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 >> 172.21.15.143:0/96079082 conn(0x7fcea80d0e30 msgr2=0x7fcea80b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 shutdown_connections 2024-01-15T18:12:43.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:43.128+0000 7fceb0200640 1 -- 172.21.15.143:0/96079082 wait complete. 2024-01-15T18:12:44.150 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:44.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.532+0000 7fed15200640 1 Processor -- start 2024-01-15T18:12:44.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.532+0000 7fed15200640 1 -- start start 2024-01-15T18:12:44.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.540+0000 7fed15200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fed1014b110 0x7fed100cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.540+0000 7fed15200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed100cc8f0 0x7fed100b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.540+0000 7fed15200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed10071f70 0x7fed100b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.540+0000 7fed15200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed100b30a0 con 0x7fed100cc8f0 2024-01-15T18:12:44.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.540+0000 7fed15200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed10070340 con 0x7fed1014b110 2024-01-15T18:12:44.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.540+0000 7fed15200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed10070620 con 0x7fed10071f70 2024-01-15T18:12:44.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed10071f70 0x7fed100b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed10071f70 0x7fed100b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46790/0 (socket says 172.21.15.143:46790) 2024-01-15T18:12:44.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0e800640 1 -- 172.21.15.143:0/992226455 learned_addr learned my addr 172.21.15.143:0/992226455 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:44.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0d400640 1 --2- 172.21.15.143:0/992226455 >> v2:172.21.15.143:3300/0 conn(0x7fed100cc8f0 0x7fed100b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0de00640 1 --2- 172.21.15.143:0/992226455 >> v2:172.21.15.204:3300/0 conn(0x7fed1014b110 0x7fed100cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0d400640 1 -- 172.21.15.143:0/992226455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed10071f70 msgr2=0x7fed100b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:44.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0d400640 1 --2- 172.21.15.143:0/992226455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed10071f70 0x7fed100b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0d400640 1 -- 172.21.15.143:0/992226455 >> v2:172.21.15.204:3300/0 conn(0x7fed1014b110 msgr2=0x7fed100cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:44.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.544+0000 7fed0d400640 1 --2- 172.21.15.143:0/992226455 >> v2:172.21.15.204:3300/0 conn(0x7fed1014b110 0x7fed100cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.548+0000 7fed0d400640 1 -- 172.21.15.143:0/992226455 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed10070900 con 0x7fed100cc8f0 2024-01-15T18:12:44.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.548+0000 7fed0de00640 1 --2- 172.21.15.143:0/992226455 >> v2:172.21.15.204:3300/0 conn(0x7fed1014b110 0x7fed100cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:44.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.556+0000 7fed0d400640 1 --2- 172.21.15.143:0/992226455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed100b3a40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fed1006fe80 tx=0x7fed0001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96ee02836c50524 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:44.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.556+0000 7fed0ca00640 1 -- 172.21.15.143:0/992226455 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000ac410 con 0x7fed100cc8f0 2024-01-15T18:12:44.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.556+0000 7fed0ca00640 1 -- 172.21.15.143:0/992226455 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed100b2e90 con 0x7fed100cc8f0 2024-01-15T18:12:44.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.556+0000 7fed0ca00640 1 -- 172.21.15.143:0/992226455 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000ac790 con 0x7fed100cc8f0 2024-01-15T18:12:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.560+0000 7fed15200640 1 -- 172.21.15.143:0/992226455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 msgr2=0x7fed100b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.560+0000 7fed15200640 1 --2- 172.21.15.143:0/992226455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed100b3a40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fed1006fe80 tx=0x7fed0001c3d0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 -- 172.21.15.143:0/992226455 shutdown_connections 2024-01-15T18:12:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 --2- 172.21.15.143:0/992226455 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed10071f70 0x7fed100b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 --2- 172.21.15.143:0/992226455 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed100b3a40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 --2- 172.21.15.143:0/992226455 >> v2:172.21.15.204:3300/0 conn(0x7fed1014b110 0x7fed100cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 -- 172.21.15.143:0/992226455 >> 172.21.15.143:0/992226455 conn(0x7fed1006c3f0 msgr2=0x7fed1006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 -- 172.21.15.143:0/992226455 shutdown_connections 2024-01-15T18:12:44.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.564+0000 7fed15200640 1 -- 172.21.15.143:0/992226455 wait complete. 2024-01-15T18:12:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.568+0000 7fed15200640 1 Processor -- start 2024-01-15T18:12:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.568+0000 7fed15200640 1 -- start start 2024-01-15T18:12:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed15200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 0x7fed1015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed15200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed1015f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed15200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed1014b110 0x7fed1015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 0x7fed1015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 0x7fed1015f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48768/0 (socket says 172.21.15.143:48768) 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0de00640 1 -- 172.21.15.143:0/4147807514 learned_addr learned my addr 172.21.15.143:0/4147807514 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed101453c0 con 0x7fed100cc8f0 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed101615f0 con 0x7fed10071f70 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed10142550 con 0x7fed1014b110 2024-01-15T18:12:44.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0d400640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed1015f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0e800640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed1014b110 0x7fed1015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0de00640 1 -- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed1014b110 msgr2=0x7fed1015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.572+0000 7fed0de00640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed1014b110 0x7fed1015fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.576+0000 7fed0de00640 1 -- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 msgr2=0x7fed1015f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.576+0000 7fed0de00640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed1015f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.576+0000 7fed0de00640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00009420 con 0x7fed10071f70 2024-01-15T18:12:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.576+0000 7fed0de00640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 0x7fed1015f410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fed08007ef0 tx=0x7fed0800d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:44.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.580+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08003ba0 con 0x7fed10071f70 2024-01-15T18:12:44.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.580+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0800f040 con 0x7fed10071f70 2024-01-15T18:12:44.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.580+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800de80 con 0x7fed10071f70 2024-01-15T18:12:44.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.580+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed10142830 con 0x7fed10071f70 2024-01-15T18:12:44.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.580+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed10144fe0 con 0x7fed10071f70 2024-01-15T18:12:44.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.584+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece0005260 con 0x7fed10071f70 2024-01-15T18:12:44.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.584+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fed08004030 con 0x7fed10071f70 2024-01-15T18:12:44.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.588+0000 7fed06000640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fece8067590 0x7fece8069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:44.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.588+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fed080832c0 con 0x7fed10071f70 2024-01-15T18:12:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.596+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fed08051dc0 con 0x7fed10071f70 2024-01-15T18:12:44.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.596+0000 7fed0d400640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fece8067590 0x7fece8069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:44.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:44.600+0000 7fed0d400640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fece8067590 0x7fece8069a50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fed000ae000 tx=0x7fed00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.056+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fece00057e0 con 0x7fed10071f70 2024-01-15T18:12:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.060+0000 7fed06000640 1 -- 172.21.15.143:0/4147807514 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7fed0801a020 con 0x7fed10071f70 2024-01-15T18:12:45.064 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:45.064 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:45.064 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:45.064 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:45.065 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 491, 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:45.066 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.067 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:45.068 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1315, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 5369961708, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 11146256384, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 761947856896, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 178809096, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 178818816, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 43, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 46 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:45.069 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:45.070 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:45.071 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.072 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.16664999723434448, 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:45.073 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:45.074 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.072+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fece8067590 msgr2=0x7fece8069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.072+0000 7fed15200640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fece8067590 0x7fece8069a50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fed000ae000 tx=0x7fed00008040 comp rx=0 tx=0).stop 2024-01-15T18:12:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.072+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 msgr2=0x7fed1015f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:45.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.072+0000 7fed15200640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 0x7fed1015f410 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fed08007ef0 tx=0x7fed0800d4a0 comp rx=0 tx=0).stop 2024-01-15T18:12:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.076+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 shutdown_connections 2024-01-15T18:12:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.076+0000 7fed15200640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fece8067590 0x7fece8069a50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.076+0000 7fed15200640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fed1014b110 0x7fed1015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:45.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.076+0000 7fed15200640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed100cc8f0 0x7fed1015f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.076+0000 7fed15200640 1 --2- 172.21.15.143:0/4147807514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed10071f70 0x7fed1015f410 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:45.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.076+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 >> 172.21.15.143:0/4147807514 conn(0x7fed1006c3f0 msgr2=0x7fed100afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.080+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 shutdown_connections 2024-01-15T18:12:45.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:45.080+0000 7fed15200640 1 -- 172.21.15.143:0/4147807514 wait complete. 2024-01-15T18:12:46.144 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:46.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.528+0000 7f4619a00640 1 Processor -- start 2024-01-15T18:12:46.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.528+0000 7f4619a00640 1 -- start start 2024-01-15T18:12:46.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.528+0000 7f4619a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 0x7f46140b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:46.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.528+0000 7f4619a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f46140b33f0 0x7f461415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:46.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4613400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 0x7f46140b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:46.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4613400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 0x7f46140b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46816/0 (socket says 172.21.15.143:46816) 2024-01-15T18:12:46.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4613400640 1 -- 172.21.15.143:0/2290019235 learned_addr learned my addr 172.21.15.143:0/2290019235 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:46.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619000640 1 --2- 172.21.15.143:0/2290019235 >> v2:172.21.15.204:3300/0 conn(0x7f46140b33f0 0x7f461415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:46.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461415bc40 con 0x7f4614144740 2024-01-15T18:12:46.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461415c8f0 con 0x7f46140b33f0 2024-01-15T18:12:46.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461415ba60 con 0x7f461415b5e0 2024-01-15T18:12:46.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619000640 1 -- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 msgr2=0x7f46140b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:46.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619000640 1 --2- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 0x7f46140b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:46.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619000640 1 -- 172.21.15.143:0/2290019235 >> v1:172.21.15.143:6789/0 conn(0x7f4614144740 legacy=0x7f46140ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:46.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4619000640 1 -- 172.21.15.143:0/2290019235 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461415bf20 con 0x7f46140b33f0 2024-01-15T18:12:46.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.532+0000 7f4613400640 1 --2- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 0x7f46140b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:12:46.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.540+0000 7f4619000640 1 --2- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f461415cd10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f46040097c0 tx=0x7f460401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aead9ae254736d57 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:46.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.540+0000 7f4612a00640 1 -- 172.21.15.143:0/2290019235 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040ad640 con 0x7f46140b33f0 2024-01-15T18:12:46.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.540+0000 7f4612a00640 1 -- 172.21.15.143:0/2290019235 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040ad7a0 con 0x7f46140b33f0 2024-01-15T18:12:46.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.540+0000 7f4612a00640 1 -- 172.21.15.143:0/2290019235 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040adb30 con 0x7f46140b33f0 2024-01-15T18:12:46.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 msgr2=0x7f461415cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:46.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 --2- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f461415cd10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f46040097c0 tx=0x7f460401c480 comp rx=0 tx=0).stop 2024-01-15T18:12:46.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 shutdown_connections 2024-01-15T18:12:46.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 --2- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f461415cd10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:46.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 --2- 172.21.15.143:0/2290019235 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f461415b5e0 0x7f46140b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:46.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 >> 172.21.15.143:0/2290019235 conn(0x7f46140b0aa0 msgr2=0x7f46140b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:46.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 shutdown_connections 2024-01-15T18:12:46.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.548+0000 7f4619a00640 1 -- 172.21.15.143:0/2290019235 wait complete. 2024-01-15T18:12:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.552+0000 7f4619a00640 1 Processor -- start 2024-01-15T18:12:46.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.552+0000 7f4619a00640 1 -- start start 2024-01-15T18:12:46.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4619a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f4614158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4619a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4614144740 0x7f461415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f4614158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f4614158c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48784/0 (socket says 172.21.15.143:48784) 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4619a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f461415b5e0 0x7f461413f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4614144740 0x7f461415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4614144740 0x7f461415ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46832/0 (socket says 172.21.15.143:46832) 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 -- 172.21.15.143:0/1349011758 learned_addr learned my addr 172.21.15.143:0/1349011758 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461406c850 con 0x7f461415b5e0 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4614143af0 con 0x7f46140b33f0 2024-01-15T18:12:46.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4614143dd0 con 0x7f4614144740 2024-01-15T18:12:46.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 -- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4614144740 msgr2=0x7f461415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:46.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4614144740 0x7f461415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:46.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 -- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f461415b5e0 msgr2=0x7f461413f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:46.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f461415b5e0 0x7f461413f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:46.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.556+0000 7f4613e00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45fc0094d0 con 0x7f46140b33f0 2024-01-15T18:12:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.564+0000 7f4613e00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f4614158c20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c7f0 tx=0x7f460c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:46.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.564+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c003b80 con 0x7f46140b33f0 2024-01-15T18:12:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.564+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4604009420 con 0x7f46140b33f0 2024-01-15T18:12:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.564+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461415b170 con 0x7f46140b33f0 2024-01-15T18:12:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.568+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461406cce0 con 0x7f46140b33f0 2024-01-15T18:12:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.568+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c003ce0 con 0x7f46140b33f0 2024-01-15T18:12:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.568+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c00aa90 con 0x7f46140b33f0 2024-01-15T18:12:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.572+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f460c00abf0 con 0x7f46140b33f0 2024-01-15T18:12:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.576+0000 7f460be00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45ec0674c0 0x7f45ec069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.576+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f460c083810 con 0x7f46140b33f0 2024-01-15T18:12:46.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.584+0000 7f4613400640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45ec0674c0 0x7f45ec069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:46.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.584+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f460c052240 con 0x7f46140b33f0 2024-01-15T18:12:46.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:46.588+0000 7f4613400640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45ec0674c0 0x7f45ec069980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f461415a9f0 tx=0x7f45fc009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:47.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.092+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f461406cce0 con 0x7f46140b33f0 2024-01-15T18:12:47.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.092+0000 7f460be00640 1 -- 172.21.15.143:0/1349011758 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f460c055860 con 0x7f46140b33f0 2024-01-15T18:12:47.096 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:47.097 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:47.098 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 493, 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:47.099 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:47.100 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1543, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 6326263020, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 12656074752, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 760438038528, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 271760120, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 258496236, 2024-01-15T18:12:47.101 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 65, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 65 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:47.102 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.103 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.104 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:47.105 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.16664999723434448, 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:47.106 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:47.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45ec0674c0 msgr2=0x7f45ec069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45ec0674c0 0x7f45ec069980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f461415a9f0 tx=0x7f45fc009240 comp rx=0 tx=0).stop 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 msgr2=0x7f4614158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f4614158c20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c7f0 tx=0x7f460c00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 shutdown_connections 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45ec0674c0 0x7f45ec069980 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f461415b5e0 0x7f461413f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4614144740 0x7f461415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 --2- 172.21.15.143:0/1349011758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46140b33f0 0x7f4614158c20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:47.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 >> 172.21.15.143:0/1349011758 conn(0x7f46140b0aa0 msgr2=0x7f46140cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:47.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 shutdown_connections 2024-01-15T18:12:47.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:47.096+0000 7f4619a00640 1 -- 172.21.15.143:0/1349011758 wait complete. 2024-01-15T18:12:48.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:48.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.552+0000 7f21c8800640 1 Processor -- start 2024-01-15T18:12:48.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.556+0000 7f21c8800640 1 -- start start 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c8800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f21c00afdb0 0x7f21c01592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c8800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c00b3a10 con 0x7f21c0159810 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c8800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c006a490 con 0x7f21c00af860 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c8800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c00b2a10 con 0x7f21c00afdb0 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c5e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f21c00af860 0x7f21c00b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:53612/0 (socket says 172.21.15.143:53612) 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c5e00640 1 -- 172.21.15.143:0/3128692093 learned_addr learned my addr 172.21.15.143:0/3128692093 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:48.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c5400640 1 --2- 172.21.15.143:0/3128692093 >> v2:172.21.15.204:3300/0 conn(0x7f21c00afdb0 0x7f21c01592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c4a00640 1 -- 172.21.15.143:0/3128692093 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119924628 0 0) 0x7f21c006a490 con 0x7f21c00af860 2024-01-15T18:12:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c4a00640 1 -- 172.21.15.143:0/3128692093 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21a0003530 con 0x7f21c00af860 2024-01-15T18:12:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c5400640 1 -- 172.21.15.143:0/3128692093 >> v1:172.21.15.204:6789/0 conn(0x7f21c00af860 legacy=0x7f21c00b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c5400640 1 -- 172.21.15.143:0/3128692093 >> v1:172.21.15.143:6789/0 conn(0x7f21c0159810 legacy=0x7f21c0071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:48.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.560+0000 7f21c5400640 1 -- 172.21.15.143:0/3128692093 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c01602b0 con 0x7f21c00afdb0 2024-01-15T18:12:48.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.564+0000 7f21c5400640 1 --2- 172.21.15.143:0/3128692093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c01592d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f21a80097c0 tx=0x7f21a801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83411dd3c3182bd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.564+0000 7f21c4a00640 1 -- 172.21.15.143:0/3128692093 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80ab610 con 0x7f21c00afdb0 2024-01-15T18:12:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.564+0000 7f21c4a00640 1 -- 172.21.15.143:0/3128692093 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a80ab770 con 0x7f21c00afdb0 2024-01-15T18:12:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.564+0000 7f21c4a00640 1 -- 172.21.15.143:0/3128692093 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80abb00 con 0x7f21c00afdb0 2024-01-15T18:12:48.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.568+0000 7f21c8800640 1 -- 172.21.15.143:0/3128692093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 msgr2=0x7f21c01592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:48.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.568+0000 7f21c8800640 1 --2- 172.21.15.143:0/3128692093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c01592d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f21a80097c0 tx=0x7f21a801c410 comp rx=0 tx=0).stop 2024-01-15T18:12:48.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.572+0000 7f21c8800640 1 -- 172.21.15.143:0/3128692093 shutdown_connections 2024-01-15T18:12:48.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.572+0000 7f21c8800640 1 --2- 172.21.15.143:0/3128692093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c01592d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:48.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.572+0000 7f21c8800640 1 -- 172.21.15.143:0/3128692093 >> 172.21.15.143:0/3128692093 conn(0x7f21c0066720 msgr2=0x7f21c006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:48.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.572+0000 7f21c8800640 1 -- 172.21.15.143:0/3128692093 shutdown_connections 2024-01-15T18:12:48.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.572+0000 7f21c8800640 1 -- 172.21.15.143:0/3128692093 wait complete. 2024-01-15T18:12:48.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.576+0000 7f21c8800640 1 Processor -- start 2024-01-15T18:12:48.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.580+0000 7f21c8800640 1 -- start start 2024-01-15T18:12:48.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.584+0000 7f21c8800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21c00af860 0x7f21c0164c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.584+0000 7f21c8800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c0202550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.584+0000 7f21c8800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0157560 0x7f21c0162c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.584+0000 7f21c5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c0202550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:48.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.584+0000 7f21c5400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c0202550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:50774/0 (socket says 172.21.15.143:50774) 2024-01-15T18:12:48.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c5400640 1 -- 172.21.15.143:0/3631567962 learned_addr learned my addr 172.21.15.143:0/3631567962 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:48.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c015b750 con 0x7f21c0157560 2024-01-15T18:12:48.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0157f40 con 0x7f21c00afdb0 2024-01-15T18:12:48.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0202a90 con 0x7f21c00af860 2024-01-15T18:12:48.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0157560 0x7f21c0162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:48.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c5e00640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21c00af860 0x7f21c0164c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 -- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21c00af860 msgr2=0x7f21c0164c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21c00af860 0x7f21c0164c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 -- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 msgr2=0x7f21c0202550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c0202550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009420 con 0x7f21c0157560 2024-01-15T18:12:48.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c5400640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c0202550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:48.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.588+0000 7f21c7200640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0157560 0x7f21c0162c60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f21b800be20 tx=0x7f21b800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.592+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b800ce70 con 0x7f21c0157560 2024-01-15T18:12:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.592+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8014440 con 0x7f21c0157560 2024-01-15T18:12:48.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.592+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8014810 con 0x7f21c0157560 2024-01-15T18:12:48.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.592+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c015add0 con 0x7f21c0157560 2024-01-15T18:12:48.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.596+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c015b370 con 0x7f21c0157560 2024-01-15T18:12:48.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.596+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c015b890 con 0x7f21c0157560 2024-01-15T18:12:48.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.600+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f21b801d3b0 con 0x7f21c0157560 2024-01-15T18:12:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.604+0000 7f21be000640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2198067510 0x7f21980699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:48.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.604+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21b8083260 con 0x7f21c0157560 2024-01-15T18:12:48.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.604+0000 7f21c5e00640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2198067510 0x7f21980699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:48.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.604+0000 7f21c5e00640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2198067510 0x7f21980699d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f21b00011f0 tx=0x7f21b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:48.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:48.620+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21c015b890 con 0x7f21c0157560 2024-01-15T18:12:49.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.104+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f21c015b890 con 0x7f21c0157560 2024-01-15T18:12:49.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.104+0000 7f21be000640 1 -- 172.21.15.143:0/3631567962 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f21c015b890 con 0x7f21c0157560 2024-01-15T18:12:49.108 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:49.108 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:49.108 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:49.108 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:49.108 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:49.109 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 495, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:49.110 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:49.111 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1543, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 6326263020, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 12656074752, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 760438038528, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 271760120, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 258496236, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 65, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 65 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:49.112 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.113 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:49.114 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.095228567719459534, 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.16664999723434448, 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:49.115 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:49.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2198067510 msgr2=0x7f21980699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:49.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2198067510 0x7f21980699d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f21b00011f0 tx=0x7f21b0009040 comp rx=0 tx=0).stop 2024-01-15T18:12:49.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0157560 msgr2=0x7f21c0162c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:49.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0157560 0x7f21c0162c60 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f21b800be20 tx=0x7f21b800bf10 comp rx=0 tx=0).stop 2024-01-15T18:12:49.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 shutdown_connections 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2198067510 0x7f21980699d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0157560 0x7f21c0162c60 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21c00afdb0 0x7f21c0202550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 --2- 172.21.15.143:0/3631567962 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21c00af860 0x7f21c0164c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.108+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 >> 172.21.15.143:0/3631567962 conn(0x7f21c0066720 msgr2=0x7f21c00b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.112+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 shutdown_connections 2024-01-15T18:12:49.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:49.112+0000 7f21c8800640 1 -- 172.21.15.143:0/3631567962 wait complete. 2024-01-15T18:12:50.141 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:50.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbefa800640 1 Processor -- start 2024-01-15T18:12:50.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbefa800640 1 -- start start 2024-01-15T18:12:50.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbefa800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbef406c6a0 0x7fbef40b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:50.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbefa800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40cf2b0 con 0x7fbef415d490 2024-01-15T18:12:50.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbefa800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40cf530 con 0x7fbef40b2550 2024-01-15T18:12:50.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbefa800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef406bb70 con 0x7fbef406c6a0 2024-01-15T18:12:50.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbef3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbef415d490 0x7fbef40cb500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35364/0 (socket says 172.21.15.143:35364) 2024-01-15T18:12:50.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbef3400640 1 -- 172.21.15.143:0/4050749339 learned_addr learned my addr 172.21.15.143:0/4050749339 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:50.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbef3e00640 1 --2- 172.21.15.143:0/4050749339 >> v2:172.21.15.143:3300/0 conn(0x7fbef406c6a0 0x7fbef40b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:50.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbef3e00640 1 -- 172.21.15.143:0/4050749339 >> v1:172.21.15.204:6789/0 conn(0x7fbef40b2550 legacy=0x7fbef406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbef3e00640 1 -- 172.21.15.143:0/4050749339 >> v1:172.21.15.143:6789/0 conn(0x7fbef415d490 legacy=0x7fbef40cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.400+0000 7fbef3e00640 1 -- 172.21.15.143:0/4050749339 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40cf7f0 con 0x7fbef406c6a0 2024-01-15T18:12:50.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbef3e00640 1 --2- 172.21.15.143:0/4050749339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef406c6a0 0x7fbef40b1f70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009d90 tx=0x7fbeec01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fd205b9abc7dd4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:50.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbef2000640 1 -- 172.21.15.143:0/4050749339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0a5a30 con 0x7fbef406c6a0 2024-01-15T18:12:50.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbef2000640 1 -- 172.21.15.143:0/4050749339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec0a5b90 con 0x7fbef406c6a0 2024-01-15T18:12:50.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbef2000640 1 -- 172.21.15.143:0/4050749339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0ad440 con 0x7fbef406c6a0 2024-01-15T18:12:50.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 -- 172.21.15.143:0/4050749339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef406c6a0 msgr2=0x7fbef40b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 --2- 172.21.15.143:0/4050749339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef406c6a0 0x7fbef40b1f70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009d90 tx=0x7fbeec01c950 comp rx=0 tx=0).stop 2024-01-15T18:12:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 -- 172.21.15.143:0/4050749339 shutdown_connections 2024-01-15T18:12:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 --2- 172.21.15.143:0/4050749339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef406c6a0 0x7fbef40b1f70 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 -- 172.21.15.143:0/4050749339 >> 172.21.15.143:0/4050749339 conn(0x7fbef40d0eb0 msgr2=0x7fbef40d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 -- 172.21.15.143:0/4050749339 shutdown_connections 2024-01-15T18:12:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.404+0000 7fbefa800640 1 -- 172.21.15.143:0/4050749339 wait complete. 2024-01-15T18:12:50.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 Processor -- start 2024-01-15T18:12:50.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 -- start start 2024-01-15T18:12:50.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef406c6a0 0x7fbef40caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:50.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 0x7fbef415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef415d490 0x7fbef414c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40ca5b0 con 0x7fbef40b2550 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef415c3a0 con 0x7fbef415d490 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c8010 con 0x7fbef406c6a0 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 0x7fbef415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 0x7fbef415b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33084/0 (socket says 172.21.15.143:33084) 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 -- 172.21.15.143:0/1917232482 learned_addr learned my addr 172.21.15.143:0/1917232482 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef3e00640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef415d490 0x7fbef414c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:50.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef3400640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef406c6a0 0x7fbef40caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:50.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 -- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef406c6a0 msgr2=0x7fbef40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef406c6a0 0x7fbef40caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 -- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef415d490 msgr2=0x7fbef414c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef415d490 0x7fbef414c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 -- 172.21.15.143:0/1917232482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec009a70 con 0x7fbef40b2550 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbef2a00640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 0x7fbef415b000 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00c500 tx=0x7fbedc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc005220 con 0x7fbef40b2550 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc005380 con 0x7fbef40b2550 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc007270 con 0x7fbef40b2550 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef415b540 con 0x7fbef40b2550 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.408+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef414d490 con 0x7fbef40b2550 2024-01-15T18:12:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.412+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4067500 con 0x7fbef40b2550 2024-01-15T18:12:50.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.412+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fbedc0073d0 con 0x7fbef40b2550 2024-01-15T18:12:50.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.412+0000 7fbeeb400640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbecc0674c0 0x7fbecc069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:50.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.412+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbedc082850 con 0x7fbef40b2550 2024-01-15T18:12:50.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.412+0000 7fbef3400640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbecc0674c0 0x7fbecc069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:50.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.412+0000 7fbef3400640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbecc0674c0 0x7fbecc069980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbee4005890 tx=0x7fbee4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:50.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.416+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbedc051310 con 0x7fbef40b2550 2024-01-15T18:12:50.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbef40c8150 con 0x7fbef40b2550 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbeeb400640 1 -- 172.21.15.143:0/1917232482 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7fbedc054930 con 0x7fbef40b2550 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:50.622 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 496, 2024-01-15T18:12:50.623 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.624 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1781, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 7324592557, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 14321369088, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 758772744192, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 263800212, 2024-01-15T18:12:50.625 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 258925877, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 63, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 67 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:50.626 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:50.627 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 19.046% (23s)\n [=====.......................] (remaining: 97s)", 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.19045713543891907, 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 33.33% (23s)\n [=========...................] (remaining: 46s)", 2024-01-15T18:12:50.628 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.33329999446868896, 2024-01-15T18:12:50.629 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:50.629 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:50.629 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:50.629 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbecc0674c0 msgr2=0x7fbecc069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbecc0674c0 0x7fbecc069980 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbee4005890 tx=0x7fbee4009040 comp rx=0 tx=0).stop 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 msgr2=0x7fbef415b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 0x7fbef415b000 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00c500 tx=0x7fbedc00c9c0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 shutdown_connections 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbecc0674c0 0x7fbecc069980 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbef415d490 0x7fbef414c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbef40b2550 0x7fbef415b000 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 --2- 172.21.15.143:0/1917232482 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbef406c6a0 0x7fbef40caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 >> 172.21.15.143:0/1917232482 conn(0x7fbef40d0eb0 msgr2=0x7fbef4160460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 shutdown_connections 2024-01-15T18:12:50.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:50.620+0000 7fbefa800640 1 -- 172.21.15.143:0/1917232482 wait complete. 2024-01-15T18:12:51.645 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:12:51.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 Processor -- start 2024-01-15T18:12:51.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 -- start start 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff61015b260 0x7ff6100cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6100d19f0 0x7ff6100cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100c5220 0x7ff6100c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6100c4d90 con 0x7ff6100d19f0 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61015b640 con 0x7ff61015b260 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff614c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100cf800 con 0x7ff6100c5220 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6100d19f0 0x7ff6100cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff6100d19f0 0x7ff6100cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33100/0 (socket says 172.21.15.143:33100) 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 -- 172.21.15.143:0/341337142 learned_addr learned my addr 172.21.15.143:0/341337142 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:51.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60e800640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100c5220 0x7ff6100c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60de00640 1 --2- 172.21.15.143:0/341337142 >> v2:172.21.15.204:3300/0 conn(0x7ff61015b260 0x7ff6100cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 -- 172.21.15.143:0/341337142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100c5220 msgr2=0x7ff6100c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100c5220 0x7ff6100c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 -- 172.21.15.143:0/341337142 >> v2:172.21.15.204:3300/0 conn(0x7ff61015b260 msgr2=0x7ff6100cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 --2- 172.21.15.143:0/341337142 >> v2:172.21.15.204:3300/0 conn(0x7ff61015b260 0x7ff6100cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60d400640 1 -- 172.21.15.143:0/341337142 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100cfae0 con 0x7ff6100d19f0 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60e800640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100c5220 0x7ff6100c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:51.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.792+0000 7ff60de00640 1 --2- 172.21.15.143:0/341337142 >> v2:172.21.15.204:3300/0 conn(0x7ff61015b260 0x7ff6100cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff60d400640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100d19f0 0x7ff6100cdd50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff6100681e0 tx=0x7ff5f801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3a46fe052f4eb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff60ca00640 1 -- 172.21.15.143:0/341337142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80aa5f0 con 0x7ff6100d19f0 2024-01-15T18:12:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff60ca00640 1 -- 172.21.15.143:0/341337142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f80aa750 con 0x7ff6100d19f0 2024-01-15T18:12:51.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff60ca00640 1 -- 172.21.15.143:0/341337142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80aaae0 con 0x7ff6100d19f0 2024-01-15T18:12:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 -- 172.21.15.143:0/341337142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100d19f0 msgr2=0x7ff6100cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100d19f0 0x7ff6100cdd50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff6100681e0 tx=0x7ff5f801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 -- 172.21.15.143:0/341337142 shutdown_connections 2024-01-15T18:12:51.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100c5220 0x7ff6100c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 --2- 172.21.15.143:0/341337142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100d19f0 0x7ff6100cdd50 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 --2- 172.21.15.143:0/341337142 >> v2:172.21.15.204:3300/0 conn(0x7ff61015b260 0x7ff6100cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 -- 172.21.15.143:0/341337142 >> 172.21.15.143:0/341337142 conn(0x7ff6100d0ed0 msgr2=0x7ff6100d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 -- 172.21.15.143:0/341337142 shutdown_connections 2024-01-15T18:12:51.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.796+0000 7ff614c00640 1 -- 172.21.15.143:0/341337142 wait complete. 2024-01-15T18:12:51.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 Processor -- start 2024-01-15T18:12:51.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 -- start start 2024-01-15T18:12:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 0x7ff6100c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100d19f0 0x7ff6101f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61015b260 0x7ff610153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6101f8150 con 0x7ff6100c5220 2024-01-15T18:12:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6101f7030 con 0x7ff61015b260 2024-01-15T18:12:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6101f7230 con 0x7ff6100d19f0 2024-01-15T18:12:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 0x7ff6100c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 0x7ff6100c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33102/0 (socket says 172.21.15.143:33102) 2024-01-15T18:12:51.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 -- 172.21.15.143:0/767927073 learned_addr learned my addr 172.21.15.143:0/767927073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60e800640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61015b260 0x7ff610153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60d400640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100d19f0 0x7ff6101f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 -- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100d19f0 msgr2=0x7ff6101f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100d19f0 0x7ff6101f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 -- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61015b260 msgr2=0x7ff610153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:51.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61015b260 0x7ff610153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 -- 172.21.15.143:0/767927073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f80094d0 con 0x7ff6100c5220 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60e800640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61015b260 0x7ff610153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60d400640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100d19f0 0x7ff6101f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff60de00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 0x7ff6100c8980 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff60000d800 tx=0x7ff60000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6000096c0 con 0x7ff6100c5220 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600009820 con 0x7ff6100c5220 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6101f74d0 con 0x7ff6100c5220 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600009bd0 con 0x7ff6100c5220 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.800+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6101f7a90 con 0x7ff6100c5220 2024-01-15T18:12:51.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.804+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff610067500 con 0x7ff6100c5220 2024-01-15T18:12:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.804+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7ff600009d30 con 0x7ff6100c5220 2024-01-15T18:12:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.804+0000 7ff606000640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5e80674c0 0x7ff5e8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.804+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff600083800 con 0x7ff6100c5220 2024-01-15T18:12:51.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.804+0000 7ff60d400640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5e80674c0 0x7ff5e8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:51.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.804+0000 7ff60d400640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5e80674c0 0x7ff5e8069980 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80ae000 tx=0x7ff5f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:51.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:51.808+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff600052230 con 0x7ff6100c5220 2024-01-15T18:12:52.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.012+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff6100b6d50 con 0x7ff6100c5220 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.012+0000 7ff606000640 1 -- 172.21.15.143:0/767927073 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2409 (secure 0 0 0) 0x7ff600055850 con 0x7ff6100c5220 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:12:52.016 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 498, 2024-01-15T18:12:52.017 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:12:52.018 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 1781, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 7324592557, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 14321369088, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 758772744192, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 187984481, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 185197790, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 44, 2024-01-15T18:12:52.019 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 47 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:12:52.020 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:12:52.021 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 28.571% (26s)\n [========....................] (remaining: 65s)", 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.28571429848670959, 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 50.0% (26s)\n [==============..............] (remaining: 26s)", 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.5, 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:12:52.022 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:12:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5e80674c0 msgr2=0x7ff5e8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5e80674c0 0x7ff5e8069980 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80ae000 tx=0x7ff5f8008040 comp rx=0 tx=0).stop 2024-01-15T18:12:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 msgr2=0x7ff6100c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 0x7ff6100c8980 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff60000d800 tx=0x7ff60000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:12:52.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 shutdown_connections 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff5e80674c0 0x7ff5e8069980 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff61015b260 0x7ff610153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff6100d19f0 0x7ff6101f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 --2- 172.21.15.143:0/767927073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff6100c5220 0x7ff6100c8980 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 >> 172.21.15.143:0/767927073 conn(0x7ff6100d0ed0 msgr2=0x7ff61015a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 shutdown_connections 2024-01-15T18:12:52.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:52.016+0000 7ff614c00640 1 -- 172.21.15.143:0/767927073 wait complete. 2024-01-15T18:12:53.039 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99318 2024-01-15T18:12:53.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.188+0000 7f1bf4e00640 1 Processor -- start 2024-01-15T18:12:53.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.188+0000 7f1bf4e00640 1 -- start start 2024-01-15T18:12:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.188+0000 7f1bf4e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bf0141660 0x7f1bf00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bf4e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1bf0071d00 0x7f1bf0072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bf4e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bf0072650 con 0x7f1bf015ec80 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bf4e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf015fbb0 con 0x7f1bf0141660 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bf4e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf015f130 con 0x7f1bf0071d00 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bf0141660 0x7f1bf00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1bf0141660 0x7f1bf00ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33118/0 (socket says 172.21.15.143:33118) 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 -- 172.21.15.143:0/3381342794 learned_addr learned my addr 172.21.15.143:0/3381342794 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 -- 172.21.15.143:0/3381342794 >> v2:172.21.15.204:3300/0 conn(0x7f1bf0071d00 msgr2=0x7f1bf0072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bee800640 1 --2- 172.21.15.143:0/3381342794 >> v2:172.21.15.204:3300/0 conn(0x7f1bf0071d00 0x7f1bf0072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 --2- 172.21.15.143:0/3381342794 >> v2:172.21.15.204:3300/0 conn(0x7f1bf0071d00 0x7f1bf0072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 -- 172.21.15.143:0/3381342794 >> v1:172.21.15.204:6789/0 conn(0x7f1bf015ec80 legacy=0x7f1bf0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:53.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 -- 172.21.15.143:0/3381342794 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf015f310 con 0x7f1bf0141660 2024-01-15T18:12:53.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1bede00640 1 --2- 172.21.15.143:0/3381342794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf0141660 0x7f1bf00ce560 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1bf006e250 tx=0x7f1be001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625d59f8b407d879 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:53.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.192+0000 7f1beca00640 1 -- 172.21.15.143:0/3381342794 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be00aa5f0 con 0x7f1bf0141660 2024-01-15T18:12:53.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1beca00640 1 -- 172.21.15.143:0/3381342794 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be00aa750 con 0x7f1bf0141660 2024-01-15T18:12:53.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1beca00640 1 -- 172.21.15.143:0/3381342794 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be00aaae0 con 0x7f1bf0141660 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- 172.21.15.143:0/3381342794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf0141660 msgr2=0x7f1bf00ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/3381342794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf0141660 0x7f1bf00ce560 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1bf006e250 tx=0x7f1be001c490 comp rx=0 tx=0).stop 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- 172.21.15.143:0/3381342794 shutdown_connections 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/3381342794 >> v2:172.21.15.204:3300/0 conn(0x7f1bf0071d00 0x7f1bf0072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/3381342794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf0141660 0x7f1bf00ce560 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- 172.21.15.143:0/3381342794 >> 172.21.15.143:0/3381342794 conn(0x7f1bf006c3f0 msgr2=0x7f1bf006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- 172.21.15.143:0/3381342794 shutdown_connections 2024-01-15T18:12:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- 172.21.15.143:0/3381342794 wait complete. 2024-01-15T18:12:53.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 Processor -- start 2024-01-15T18:12:53.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- start start 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1bf0071d00 0x7f1bf015e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bf0141660 0x7f1bf0134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 0x7f1bf0135300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf0136210 con 0x7f1bf015ec80 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf01358e0 con 0x7f1bf0141660 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.196+0000 7f1bf4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf0135bc0 con 0x7f1bf0071d00 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 0x7f1bf0135300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 0x7f1bf0135300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33124/0 (socket says 172.21.15.143:33124) 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 -- 172.21.15.143:0/97512725 learned_addr learned my addr 172.21.15.143:0/97512725 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bed400640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bf0141660 0x7f1bf0134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bede00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1bf0071d00 0x7f1bf015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 -- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1bf0071d00 msgr2=0x7f1bf015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1bf0071d00 0x7f1bf015e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 -- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bf0141660 msgr2=0x7f1bf0134dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bf0141660 0x7f1bf0134dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 -- 172.21.15.143:0/97512725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00094d0 con 0x7f1bf015ec80 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bee800640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 0x7f1bf0135300 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f1be800cc80 tx=0x7f1be800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be800ce30 con 0x7f1bf015ec80 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be80039a0 con 0x7f1bf015ec80 2024-01-15T18:12:53.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8003d30 con 0x7f1bf015ec80 2024-01-15T18:12:53.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf0135ea0 con 0x7f1bf015ec80 2024-01-15T18:12:53.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf015e670 con 0x7f1bf015ec80 2024-01-15T18:12:53.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf015e8f0 con 0x7f1bf015ec80 2024-01-15T18:12:53.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.200+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f1be8003e90 con 0x7f1bf015ec80 2024-01-15T18:12:53.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.204+0000 7f1be6000640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1bc80674c0 0x7f1bc8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.204+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1be80832d0 con 0x7f1bf015ec80 2024-01-15T18:12:53.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.204+0000 7f1bede00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1bc80674c0 0x7f1bc8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.204+0000 7f1bede00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1bc80674c0 0x7f1bc8069980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1be00ae000 tx=0x7f1be00b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:53.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.208+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1be8051db0 con 0x7f1bf015ec80 2024-01-15T18:12:53.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.384+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99318", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bf0135a50 con 0x7f1bc80674c0 2024-01-15T18:12:53.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1be6000640 1 -- 172.21.15.143:0/97512725 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1bf0135a50 con 0x7f1bc80674c0 2024-01-15T18:12:53.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1bc80674c0 msgr2=0x7f1bc8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1bc80674c0 0x7f1bc8069980 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1be00ae000 tx=0x7f1be00b5040 comp rx=0 tx=0).stop 2024-01-15T18:12:53.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 msgr2=0x7f1bf0135300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 0x7f1bf0135300 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f1be800cc80 tx=0x7f1be800a5c0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 shutdown_connections 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1bc80674c0 0x7f1bc8069980 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1bf015ec80 0x7f1bf0135300 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1bf0141660 0x7f1bf0134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 --2- 172.21.15.143:0/97512725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1bf0071d00 0x7f1bf015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 >> 172.21.15.143:0/97512725 conn(0x7f1bf006c3f0 msgr2=0x7f1bf0070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 shutdown_connections 2024-01-15T18:12:53.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.444+0000 7f1bf4e00640 1 -- 172.21.15.143:0/97512725 wait complete. 2024-01-15T18:12:53.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone99319 2024-01-15T18:12:53.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.604+0000 7f31a6200640 1 Processor -- start 2024-01-15T18:12:53.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.604+0000 7f31a6200640 1 -- start start 2024-01-15T18:12:53.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f31a6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a015d130 0x7f31a00aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f31a6200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31a015d510 0x7f31a006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f31a6200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f31a006a490 0x7f31a006a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f31a6200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a006ae40 con 0x7f31a015d510 2024-01-15T18:12:53.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f31a6200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a0160eb0 con 0x7f31a006a490 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f31a6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0161190 con 0x7f31a015d130 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31a015d510 0x7f31a006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f31a015d510 0x7f31a006e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33132/0 (socket says 172.21.15.143:33132) 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 -- 172.21.15.143:0/966933627 learned_addr learned my addr 172.21.15.143:0/966933627 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319fe00640 1 --2- 172.21.15.143:0/966933627 >> v2:172.21.15.204:3300/0 conn(0x7f31a006a490 0x7f31a006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319f400640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a015d130 0x7f31a00aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 -- 172.21.15.143:0/966933627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a015d130 msgr2=0x7f31a00aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a015d130 0x7f31a00aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 -- 172.21.15.143:0/966933627 >> v2:172.21.15.204:3300/0 conn(0x7f31a006a490 msgr2=0x7f31a006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 --2- 172.21.15.143:0/966933627 >> v2:172.21.15.204:3300/0 conn(0x7f31a006a490 0x7f31a006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319ea00640 1 -- 172.21.15.143:0/966933627 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0161470 con 0x7f31a015d510 2024-01-15T18:12:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319f400640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a015d130 0x7f31a00aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:53.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.608+0000 7f319fe00640 1 --2- 172.21.15.143:0/966933627 >> v2:172.21.15.204:3300/0 conn(0x7f31a006a490 0x7f31a006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:53.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f319ea00640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d510 0x7f31a006e060 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b4f50 tx=0x7f318801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2174688040f477 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:53.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f319e000640 1 -- 172.21.15.143:0/966933627 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880aa640 con 0x7f31a015d510 2024-01-15T18:12:53.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f319e000640 1 -- 172.21.15.143:0/966933627 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31880aa7a0 con 0x7f31a015d510 2024-01-15T18:12:53.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f319e000640 1 -- 172.21.15.143:0/966933627 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880aab30 con 0x7f31a015d510 2024-01-15T18:12:53.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 -- 172.21.15.143:0/966933627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d510 msgr2=0x7f31a006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d510 0x7f31a006e060 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b4f50 tx=0x7f318801c500 comp rx=0 tx=0).stop 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 -- 172.21.15.143:0/966933627 shutdown_connections 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 --2- 172.21.15.143:0/966933627 >> v2:172.21.15.204:3300/0 conn(0x7f31a006a490 0x7f31a006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d510 0x7f31a006e060 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 --2- 172.21.15.143:0/966933627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a015d130 0x7f31a00aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 -- 172.21.15.143:0/966933627 >> 172.21.15.143:0/966933627 conn(0x7f31a00b1b90 msgr2=0x7f31a00b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 -- 172.21.15.143:0/966933627 shutdown_connections 2024-01-15T18:12:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 -- 172.21.15.143:0/966933627 wait complete. 2024-01-15T18:12:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 Processor -- start 2024-01-15T18:12:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.612+0000 7f31a6200640 1 -- start start 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a006a490 0x7f31a01602b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 0x7f31a0139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a015d510 0x7f31a01583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0157f60 con 0x7f31a015d130 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0157350 con 0x7f31a015d510 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0157630 con 0x7f31a006a490 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 0x7f31a0139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 0x7f31a0139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33134/0 (socket says 172.21.15.143:33134) 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 -- 172.21.15.143:0/2192828753 learned_addr learned my addr 172.21.15.143:0/2192828753 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319fe00640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a015d510 0x7f31a01583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319f400640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a006a490 0x7f31a01602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 -- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a006a490 msgr2=0x7f31a01602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a006a490 0x7f31a01602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 -- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a015d510 msgr2=0x7f31a01583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a015d510 0x7f31a01583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 -- 172.21.15.143:0/2192828753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880094d0 con 0x7f31a015d130 2024-01-15T18:12:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f319ea00640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 0x7f31a0139340 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f318801abb0 tx=0x7f318801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880bd5f0 con 0x7f31a015d130 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31880bd750 con 0x7f31a015d130 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a0157910 con 0x7f31a015d130 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31880bdb00 con 0x7f31a015d130 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a01607f0 con 0x7f31a015d130 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.616+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a0158110 con 0x7f31a015d130 2024-01-15T18:12:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.620+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f31880a8050 con 0x7f31a015d130 2024-01-15T18:12:53.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.620+0000 7f3197400640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31780674c0 0x7f3178069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:53.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.620+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f318811daa0 con 0x7f31a015d130 2024-01-15T18:12:53.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.620+0000 7f319f400640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31780674c0 0x7f3178069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:53.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.620+0000 7f319f400640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31780674c0 0x7f3178069980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f31a00cf400 tx=0x7f3190008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:53.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.624+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f31a0158110 con 0x7f31a015d130 2024-01-15T18:12:53.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.808+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone99319", "target": ["mon-mgr", ""]}) v1 -- 0x7f31a0067500 con 0x7f31780674c0 2024-01-15T18:12:53.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.864+0000 7f3197400640 1 -- 172.21.15.143:0/2192828753 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f31a0067500 con 0x7f31780674c0 2024-01-15T18:12:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31780674c0 msgr2=0x7f3178069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31780674c0 0x7f3178069980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f31a00cf400 tx=0x7f3190008040 comp rx=0 tx=0).stop 2024-01-15T18:12:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 msgr2=0x7f31a0139340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 0x7f31a0139340 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f318801abb0 tx=0x7f318801ad40 comp rx=0 tx=0).stop 2024-01-15T18:12:53.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 shutdown_connections 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f31780674c0 0x7f3178069980 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f31a015d510 0x7f31a01583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f31a015d130 0x7f31a0139340 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 --2- 172.21.15.143:0/2192828753 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f31a006a490 0x7f31a01602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 >> 172.21.15.143:0/2192828753 conn(0x7f31a00b1b90 msgr2=0x7f31a00afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 shutdown_connections 2024-01-15T18:12:53.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:53.868+0000 7f31a6200640 1 -- 172.21.15.143:0/2192828753 wait complete. 2024-01-15T18:12:53.890 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993110 2024-01-15T18:12:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 Processor -- start 2024-01-15T18:12:54.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 -- start start 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200141660 0x7f82000ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8200071d00 0x7f8200072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820015f310 con 0x7f820015ec80 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820015fbb0 con 0x7f8200071d00 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8208a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8200072690 con 0x7f8200141660 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8206a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8200071d00 0x7f8200072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8205600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f820015ec80 0x7f8200141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35366/0 (socket says 172.21.15.143:35366) 2024-01-15T18:12:54.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8205600640 1 -- 172.21.15.143:0/2896096607 learned_addr learned my addr 172.21.15.143:0/2896096607 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8206000640 1 --2- 172.21.15.143:0/2896096607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200141660 0x7f82000ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8206a00640 1 -- 172.21.15.143:0/2896096607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200141660 msgr2=0x7f82000ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8206a00640 1 --2- 172.21.15.143:0/2896096607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200141660 0x7f82000ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8204c00640 1 -- 172.21.15.143:0/2896096607 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812964206 0 0) 0x7f820015f310 con 0x7f820015ec80 2024-01-15T18:12:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8206a00640 1 -- 172.21.15.143:0/2896096607 >> v1:172.21.15.143:6789/0 conn(0x7f820015ec80 legacy=0x7f8200141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.036+0000 7f8206a00640 1 -- 172.21.15.143:0/2896096607 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820015f5f0 con 0x7f8200071d00 2024-01-15T18:12:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8206a00640 1 --2- 172.21.15.143:0/2896096607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8200071d00 0x7f8200072150 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009d90 tx=0x7f81f801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d556a7eb0302574 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8204c00640 1 -- 172.21.15.143:0/2896096607 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f80a5900 con 0x7f8200071d00 2024-01-15T18:12:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8204c00640 1 -- 172.21.15.143:0/2896096607 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f80a5a60 con 0x7f8200071d00 2024-01-15T18:12:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8204c00640 1 -- 172.21.15.143:0/2896096607 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f80a5df0 con 0x7f8200071d00 2024-01-15T18:12:54.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 -- 172.21.15.143:0/2896096607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8200071d00 msgr2=0x7f8200072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 --2- 172.21.15.143:0/2896096607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8200071d00 0x7f8200072150 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f81f8009d90 tx=0x7f81f801c950 comp rx=0 tx=0).stop 2024-01-15T18:12:54.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 -- 172.21.15.143:0/2896096607 shutdown_connections 2024-01-15T18:12:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 --2- 172.21.15.143:0/2896096607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8200071d00 0x7f8200072150 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 --2- 172.21.15.143:0/2896096607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200141660 0x7f82000ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 -- 172.21.15.143:0/2896096607 >> 172.21.15.143:0/2896096607 conn(0x7f820006c3f0 msgr2=0x7f820006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 -- 172.21.15.143:0/2896096607 shutdown_connections 2024-01-15T18:12:54.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.040+0000 7f8208a00640 1 -- 172.21.15.143:0/2896096607 wait complete. 2024-01-15T18:12:54.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 Processor -- start 2024-01-15T18:12:54.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 -- start start 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200071d00 0x7f82000a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8200141660 0x7f820015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 0x7f8200158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820015abd0 con 0x7f820015ec80 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820015c620 con 0x7f8200141660 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820015c900 con 0x7f8200071d00 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 0x7f8200158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 0x7f8200158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33152/0 (socket says 172.21.15.143:33152) 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 -- 172.21.15.143:0/658301647 learned_addr learned my addr 172.21.15.143:0/658301647 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206000640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200071d00 0x7f82000a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8205600640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8200141660 0x7f820015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 -- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200071d00 msgr2=0x7f82000a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200071d00 0x7f82000a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 -- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8200141660 msgr2=0x7f820015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8200141660 0x7f820015e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 -- 172.21.15.143:0/658301647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009a70 con 0x7f820015ec80 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8206a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 0x7f8200158730 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f81f801a580 tx=0x7f81f80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8003ba0 con 0x7f820015ec80 2024-01-15T18:12:54.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8003d00 con 0x7f820015ec80 2024-01-15T18:12:54.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82000cd150 con 0x7f820015ec80 2024-01-15T18:12:54.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8004230 con 0x7f820015ec80 2024-01-15T18:12:54.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.044+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820015cec0 con 0x7f820015ec80 2024-01-15T18:12:54.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.048+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8200067500 con 0x7f820015ec80 2024-01-15T18:12:54.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.048+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f81f80b4070 con 0x7f820015ec80 2024-01-15T18:12:54.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.048+0000 7f81fe000640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81d80674c0 0x7f81d8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.048+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f81f811dba0 con 0x7f820015ec80 2024-01-15T18:12:54.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.048+0000 7f8206000640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81d80674c0 0x7f81d8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.048+0000 7f8206000640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81d80674c0 0x7f81d8069980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f82000ad7f0 tx=0x7f81f0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.052+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f81f80ec640 con 0x7f820015ec80 2024-01-15T18:12:54.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.232+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993110", "target": ["mon-mgr", ""]}) v1 -- 0x7f8200066fd0 con 0x7f81d80674c0 2024-01-15T18:12:54.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.280+0000 7f81fe000640 1 -- 172.21.15.143:0/658301647 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8200066fd0 con 0x7f81d80674c0 2024-01-15T18:12:54.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81d80674c0 msgr2=0x7f81d8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81d80674c0 0x7f81d8069980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f82000ad7f0 tx=0x7f81f0009190 comp rx=0 tx=0).stop 2024-01-15T18:12:54.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 msgr2=0x7f8200158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 0x7f8200158730 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f81f801a580 tx=0x7f81f80196f0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 shutdown_connections 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81d80674c0 0x7f81d8069980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f820015ec80 0x7f8200158730 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8200141660 0x7f820015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 --2- 172.21.15.143:0/658301647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8200071d00 0x7f82000a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 >> 172.21.15.143:0/658301647 conn(0x7f820006c3f0 msgr2=0x7f8200140700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 shutdown_connections 2024-01-15T18:12:54.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.284+0000 7f8208a00640 1 -- 172.21.15.143:0/658301647 wait complete. 2024-01-15T18:12:54.304 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993111 2024-01-15T18:12:54.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 Processor -- start 2024-01-15T18:12:54.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 -- start start 2024-01-15T18:12:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3d0071d00 0x7fa3d0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d015f310 con 0x7fa3d015ec80 2024-01-15T18:12:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d015fbb0 con 0x7fa3d0071d00 2024-01-15T18:12:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3d6000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0072690 con 0x7fa3d0141660 2024-01-15T18:12:54.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.444+0000 7fa3cfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3d0071d00 0x7fa3d0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa3d0071d00 0x7fa3d0072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33164/0 (socket says 172.21.15.143:33164) 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 -- 172.21.15.143:0/1851470412 learned_addr learned my addr 172.21.15.143:0/1851470412 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cf400640 1 --2- 172.21.15.143:0/1851470412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 -- 172.21.15.143:0/1851470412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 msgr2=0x7fa3d00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 --2- 172.21.15.143:0/1851470412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 -- 172.21.15.143:0/1851470412 >> v1:172.21.15.204:6789/0 conn(0x7fa3d015ec80 legacy=0x7fa3d0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 -- 172.21.15.143:0/1851470412 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d015f5f0 con 0x7fa3d0071d00 2024-01-15T18:12:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3cfe00640 1 --2- 172.21.15.143:0/1851470412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d0072150 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009e50 tx=0x7fa3c801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b172f1091e4c9b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3ce000640 1 -- 172.21.15.143:0/1851470412 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80a57d0 con 0x7fa3d0071d00 2024-01-15T18:12:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3ce000640 1 -- 172.21.15.143:0/1851470412 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c80a5930 con 0x7fa3d0071d00 2024-01-15T18:12:54.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.448+0000 7fa3ce000640 1 -- 172.21.15.143:0/1851470412 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c80a5cc0 con 0x7fa3d0071d00 2024-01-15T18:12:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- 172.21.15.143:0/1851470412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 msgr2=0x7fa3d0072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1851470412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d0072150 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009e50 tx=0x7fa3c801ca10 comp rx=0 tx=0).stop 2024-01-15T18:12:54.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- 172.21.15.143:0/1851470412 shutdown_connections 2024-01-15T18:12:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1851470412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d0072150 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1851470412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- 172.21.15.143:0/1851470412 >> 172.21.15.143:0/1851470412 conn(0x7fa3d006c3f0 msgr2=0x7fa3d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- 172.21.15.143:0/1851470412 shutdown_connections 2024-01-15T18:12:54.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- 172.21.15.143:0/1851470412 wait complete. 2024-01-15T18:12:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 Processor -- start 2024-01-15T18:12:54.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- start start 2024-01-15T18:12:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d015ec80 0x7fa3d015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013ecc0 con 0x7fa3d0071d00 2024-01-15T18:12:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d013de50 con 0x7fa3d015ec80 2024-01-15T18:12:54.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3d6000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d01f6230 con 0x7fa3d0141660 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d015e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33168/0 (socket says 172.21.15.143:33168) 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.452+0000 7fa3cf400640 1 -- 172.21.15.143:0/1846301392 learned_addr learned my addr 172.21.15.143:0/1846301392 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cfe00640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d015ec80 0x7fa3d015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cea00640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cf400640 1 -- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 msgr2=0x7fa3d013d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cf400640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d013d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cf400640 1 -- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d015ec80 msgr2=0x7fa3d015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cf400640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d015ec80 0x7fa3d015b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cf400640 1 -- 172.21.15.143:0/1846301392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c8009b30 con 0x7fa3d0071d00 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cfe00640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d015ec80 0x7fa3d015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cf400640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d015e190 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000e870 tx=0x7fa3c000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0009510 con 0x7fa3d0071d00 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c000eec0 con 0x7fa3d0071d00 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d00cd150 con 0x7fa3d0071d00 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000cb40 con 0x7fa3d0071d00 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d013e390 con 0x7fa3d0071d00 2024-01-15T18:12:54.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0067500 con 0x7fa3d0071d00 2024-01-15T18:12:54.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fa3c000cca0 con 0x7fa3d0071d00 2024-01-15T18:12:54.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3c7400640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3a80674c0 0x7fa3a8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa3c0083190 con 0x7fa3d0071d00 2024-01-15T18:12:54.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.456+0000 7fa3cea00640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3a80674c0 0x7fa3a8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.460+0000 7fa3cea00640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3a80674c0 0x7fa3a8069980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0165510 tx=0x7fa3b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.460+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa3c0051c20 con 0x7fa3d0071d00 2024-01-15T18:12:54.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.640+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993111", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d00ccfd0 con 0x7fa3a80674c0 2024-01-15T18:12:54.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.692+0000 7fa3c7400640 1 -- 172.21.15.143:0/1846301392 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa3b8001ce0 con 0x7fa3a80674c0 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3a80674c0 msgr2=0x7fa3a8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3a80674c0 0x7fa3a8069980 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0165510 tx=0x7fa3b8009040 comp rx=0 tx=0).stop 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 msgr2=0x7fa3d015e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d015e190 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000e870 tx=0x7fa3c000ec70 comp rx=0 tx=0).stop 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 shutdown_connections 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa3a80674c0 0x7fa3a8069980 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3d015ec80 0x7fa3d015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa3d0141660 0x7fa3d013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 --2- 172.21.15.143:0/1846301392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa3d0071d00 0x7fa3d015e190 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 >> 172.21.15.143:0/1846301392 conn(0x7fa3d006c3f0 msgr2=0x7fa3d00af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:54.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 shutdown_connections 2024-01-15T18:12:54.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.696+0000 7fa3d6000640 1 -- 172.21.15.143:0/1846301392 wait complete. 2024-01-15T18:12:54.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993112 2024-01-15T18:12:54.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.860+0000 7f73db600640 1 Processor -- start 2024-01-15T18:12:54.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.860+0000 7f73db600640 1 -- start start 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73db600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73d415aef0 0x7f73d406b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73db600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f73d406c560 0x7f73d40b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73db600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d415f7c0 con 0x7f73d40b27d0 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73db600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d40b2c60 con 0x7f73d415aef0 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73db600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d406c310 con 0x7f73d406c560 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73d415aef0 0x7f73d406b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73d415aef0 0x7f73d406b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33184/0 (socket says 172.21.15.143:33184) 2024-01-15T18:12:54.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 -- 172.21.15.143:0/3990956508 learned_addr learned my addr 172.21.15.143:0/3990956508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d3e00640 1 --2- 172.21.15.143:0/3990956508 >> v2:172.21.15.204:3300/0 conn(0x7f73d406c560 0x7f73d40b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 -- 172.21.15.143:0/3990956508 >> v2:172.21.15.204:3300/0 conn(0x7f73d406c560 msgr2=0x7f73d40b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 --2- 172.21.15.143:0/3990956508 >> v2:172.21.15.204:3300/0 conn(0x7f73d406c560 0x7f73d40b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 -- 172.21.15.143:0/3990956508 >> v1:172.21.15.204:6789/0 conn(0x7f73d40b27d0 legacy=0x7f73d4071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d8c00640 1 -- 172.21.15.143:0/3990956508 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d415ebb0 con 0x7f73d415aef0 2024-01-15T18:12:54.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.864+0000 7f73d3e00640 1 --2- 172.21.15.143:0/3990956508 >> v2:172.21.15.204:3300/0 conn(0x7f73d406c560 0x7f73d40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:54.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73d8c00640 1 --2- 172.21.15.143:0/3990956508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d406b320 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f73d406a8b0 tx=0x7f73c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79089b78644859e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73d3400640 1 -- 172.21.15.143:0/3990956508 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40ac640 con 0x7f73d415aef0 2024-01-15T18:12:54.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73d3400640 1 -- 172.21.15.143:0/3990956508 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c40ac7a0 con 0x7f73d415aef0 2024-01-15T18:12:54.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73d3400640 1 -- 172.21.15.143:0/3990956508 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c40acb30 con 0x7f73d415aef0 2024-01-15T18:12:54.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 -- 172.21.15.143:0/3990956508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 msgr2=0x7f73d406b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 --2- 172.21.15.143:0/3990956508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d406b320 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f73d406a8b0 tx=0x7f73c401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 -- 172.21.15.143:0/3990956508 shutdown_connections 2024-01-15T18:12:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 --2- 172.21.15.143:0/3990956508 >> v2:172.21.15.204:3300/0 conn(0x7f73d406c560 0x7f73d40b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 --2- 172.21.15.143:0/3990956508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d406b320 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 -- 172.21.15.143:0/3990956508 >> 172.21.15.143:0/3990956508 conn(0x7f73d4066720 msgr2=0x7f73d406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 -- 172.21.15.143:0/3990956508 shutdown_connections 2024-01-15T18:12:54.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 -- 172.21.15.143:0/3990956508 wait complete. 2024-01-15T18:12:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 Processor -- start 2024-01-15T18:12:54.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.868+0000 7f73db600640 1 -- start start 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73d406c560 0x7f73d4156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73d40b27d0 0x7f73d4146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d41472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41523a0 con 0x7f73d415aef0 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d4152f20 con 0x7f73d40b27d0 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d41533b0 con 0x7f73d406c560 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d41472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73d8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73d406c560 0x7f73d4156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d41472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33194/0 (socket says 172.21.15.143:33194) 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 -- 172.21.15.143:0/8631928 learned_addr learned my addr 172.21.15.143:0/8631928 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73d3e00640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73d40b27d0 0x7f73d4146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 -- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73d406c560 msgr2=0x7f73d4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73d406c560 0x7f73d4156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 -- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73d40b27d0 msgr2=0x7f73d4146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73d40b27d0 0x7f73d4146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 -- 172.21.15.143:0/8631928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40094d0 con 0x7f73d415aef0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73da000640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d41472d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c760 tx=0x7f73cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc003b80 con 0x7f73d415aef0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc003ce0 con 0x7f73d415aef0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d41525f0 con 0x7f73d415aef0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00aa30 con 0x7f73d415aef0 2024-01-15T18:12:54.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d4152bb0 con 0x7f73d415aef0 2024-01-15T18:12:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.872+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d41530b0 con 0x7f73d415aef0 2024-01-15T18:12:54.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.876+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f73cc00ab90 con 0x7f73d415aef0 2024-01-15T18:12:54.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.876+0000 7f73d0c00640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f73ac0674c0 0x7f73ac069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:54.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.876+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f73cc082cc0 con 0x7f73d415aef0 2024-01-15T18:12:54.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.876+0000 7f73d8c00640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f73ac0674c0 0x7f73ac069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:54.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.876+0000 7f73d8c00640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f73ac0674c0 0x7f73ac069980 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f73c40ae000 tx=0x7f73c40a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:54.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:54.880+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f73cc0517a0 con 0x7f73d415aef0 2024-01-15T18:12:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.060+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993112", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d406dc10 con 0x7f73ac0674c0 2024-01-15T18:12:55.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.108+0000 7f73d0c00640 1 -- 172.21.15.143:0/8631928 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f73d406dc10 con 0x7f73ac0674c0 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f73ac0674c0 msgr2=0x7f73ac069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f73ac0674c0 0x7f73ac069980 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f73c40ae000 tx=0x7f73c40a7000 comp rx=0 tx=0).stop 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 msgr2=0x7f73d41472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d41472d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c760 tx=0x7f73cc00cc20 comp rx=0 tx=0).stop 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 shutdown_connections 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f73ac0674c0 0x7f73ac069980 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73d415aef0 0x7f73d41472d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f73d40b27d0 0x7f73d4146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 --2- 172.21.15.143:0/8631928 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f73d406c560 0x7f73d4156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 >> 172.21.15.143:0/8631928 conn(0x7f73d4066720 msgr2=0x7f73d4150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 shutdown_connections 2024-01-15T18:12:55.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.112+0000 7f73db600640 1 -- 172.21.15.143:0/8631928 wait complete. 2024-01-15T18:12:55.134 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993113 2024-01-15T18:12:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.272+0000 7f37f4400640 1 Processor -- start 2024-01-15T18:12:55.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.272+0000 7f37f4400640 1 -- start start 2024-01-15T18:12:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f4400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37ec0cda70 0x7f37ec0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f4400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37ec0cf400 0x7f37ec0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f4400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ec15b5b0 con 0x7f37ec0cf800 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f4400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0ccd30 con 0x7f37ec0cf400 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f4400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ec0cba70 con 0x7f37ec0cda70 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37ec0cf400 0x7f37ec0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f37ec0cf400 0x7f37ec0cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33202/0 (socket says 172.21.15.143:33202) 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 -- 172.21.15.143:0/693987099 learned_addr learned my addr 172.21.15.143:0/693987099 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1a00640 1 --2- 172.21.15.143:0/693987099 >> v2:172.21.15.204:3300/0 conn(0x7f37ec0cda70 0x7f37ec0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 -- 172.21.15.143:0/693987099 >> v2:172.21.15.204:3300/0 conn(0x7f37ec0cda70 msgr2=0x7f37ec0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:55.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 --2- 172.21.15.143:0/693987099 >> v2:172.21.15.204:3300/0 conn(0x7f37ec0cda70 0x7f37ec0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 -- 172.21.15.143:0/693987099 >> v1:172.21.15.204:6789/0 conn(0x7f37ec0cf800 legacy=0x7f37ec0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.276+0000 7f37f1000640 1 -- 172.21.15.143:0/693987099 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec0cbd10 con 0x7f37ec0cf400 2024-01-15T18:12:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f1000640 1 --2- 172.21.15.143:0/693987099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cf400 0x7f37ec0cde50 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f37ec071a00 tx=0x7f37d401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2af38fa10d1cedd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37ebe00640 1 -- 172.21.15.143:0/693987099 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40aa430 con 0x7f37ec0cf400 2024-01-15T18:12:55.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37ebe00640 1 -- 172.21.15.143:0/693987099 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d40aa590 con 0x7f37ec0cf400 2024-01-15T18:12:55.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37ebe00640 1 -- 172.21.15.143:0/693987099 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d40aa920 con 0x7f37ec0cf400 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 -- 172.21.15.143:0/693987099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cf400 msgr2=0x7f37ec0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 --2- 172.21.15.143:0/693987099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cf400 0x7f37ec0cde50 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f37ec071a00 tx=0x7f37d401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 -- 172.21.15.143:0/693987099 shutdown_connections 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 --2- 172.21.15.143:0/693987099 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cf400 0x7f37ec0cde50 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 --2- 172.21.15.143:0/693987099 >> v2:172.21.15.204:3300/0 conn(0x7f37ec0cda70 0x7f37ec0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 -- 172.21.15.143:0/693987099 >> 172.21.15.143:0/693987099 conn(0x7f37ec0d0e20 msgr2=0x7f37ec0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 -- 172.21.15.143:0/693987099 shutdown_connections 2024-01-15T18:12:55.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 -- 172.21.15.143:0/693987099 wait complete. 2024-01-15T18:12:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 Processor -- start 2024-01-15T18:12:55.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.280+0000 7f37f4400640 1 -- start start 2024-01-15T18:12:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 0x7f37ec0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37ec0cf400 0x7f37ec0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37ec0cf800 0x7f37ec0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec0c8580 con 0x7f37ec0cda70 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1f6390 con 0x7f37ec0cf400 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ec1f6bf0 con 0x7f37ec0cf800 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 0x7f37ec0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 0x7f37ec0ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33204/0 (socket says 172.21.15.143:33204) 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 -- 172.21.15.143:0/2111058464 learned_addr learned my addr 172.21.15.143:0/2111058464 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1000640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37ec0cf400 0x7f37ec0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f2400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37ec0cf800 0x7f37ec0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 -- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37ec0cf800 msgr2=0x7f37ec0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37ec0cf800 0x7f37ec0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 -- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37ec0cf400 msgr2=0x7f37ec0b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37ec0cf400 0x7f37ec0b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 -- 172.21.15.143:0/2111058464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40094d0 con 0x7f37ec0cda70 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f2400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37ec0cf800 0x7f37ec0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f1a00640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 0x7f37ec0ca8d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d5f0 tx=0x7f37dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0125f0 con 0x7f37ec0cda70 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc012750 con 0x7f37ec0cda70 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc012ae0 con 0x7f37ec0cda70 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec0c9480 con 0x7f37ec0cda70 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ec0c82a0 con 0x7f37ec0cda70 2024-01-15T18:12:55.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.284+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ec067500 con 0x7f37ec0cda70 2024-01-15T18:12:55.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.288+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f37dc012c40 con 0x7f37ec0cda70 2024-01-15T18:12:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.288+0000 7f37e9600640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37c40674f0 0x7f37c40699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.288+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f37dc083430 con 0x7f37ec0cda70 2024-01-15T18:12:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.288+0000 7f37f1000640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37c40674f0 0x7f37c40699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.288+0000 7f37f1000640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37c40674f0 0x7f37c40699b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f37d40ae000 tx=0x7f37d40b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:55.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.292+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37dc051ec0 con 0x7f37ec0cda70 2024-01-15T18:12:55.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.468+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993113", "target": ["mon-mgr", ""]}) v1 -- 0x7f37ec0cfd10 con 0x7f37c40674f0 2024-01-15T18:12:55.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37e9600640 1 -- 172.21.15.143:0/2111058464 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f37d40aa960 con 0x7f37c40674f0 2024-01-15T18:12:55.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37c40674f0 msgr2=0x7f37c40699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37c40674f0 0x7f37c40699b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f37d40ae000 tx=0x7f37d40b5040 comp rx=0 tx=0).stop 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 msgr2=0x7f37ec0ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 0x7f37ec0ca8d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00d5f0 tx=0x7f37dc00dab0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 shutdown_connections 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37c40674f0 0x7f37c40699b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37ec0cf800 0x7f37ec0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37ec0cf400 0x7f37ec0b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 --2- 172.21.15.143:0/2111058464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37ec0cda70 0x7f37ec0ca8d0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.520+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 >> 172.21.15.143:0/2111058464 conn(0x7f37ec0d0e20 msgr2=0x7f37ec06cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.524+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 shutdown_connections 2024-01-15T18:12:55.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.524+0000 7f37f4400640 1 -- 172.21.15.143:0/2111058464 wait complete. 2024-01-15T18:12:55.544 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993114 2024-01-15T18:12:55.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d2c00640 1 Processor -- start 2024-01-15T18:12:55.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d2c00640 1 -- start start 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d2c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f86cc141660 0x7f86cc0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d2c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86cc15f310 con 0x7f86cc071d00 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d2c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86cc15f5f0 con 0x7f86cc15ec80 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d2c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc15fbb0 con 0x7f86cc141660 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86cbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f86cc141660 0x7f86cc0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d0c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f86cc071d00 0x7f86cc0b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35378/0 (socket says 172.21.15.143:35378) 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86d0c00640 1 -- 172.21.15.143:0/1729543693 learned_addr learned my addr 172.21.15.143:0/1729543693 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86cbe00640 1 -- 172.21.15.143:0/1729543693 >> v1:172.21.15.204:6789/0 conn(0x7f86cc15ec80 legacy=0x7f86cc141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86cbe00640 1 -- 172.21.15.143:0/1729543693 >> v1:172.21.15.143:6789/0 conn(0x7f86cc071d00 legacy=0x7f86cc0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.684+0000 7f86cbe00640 1 -- 172.21.15.143:0/1729543693 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc15f8d0 con 0x7f86cc141660 2024-01-15T18:12:55.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86cbe00640 1 --2- 172.21.15.143:0/1729543693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc141660 0x7f86cc0ce560 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0097c0 tx=0x7f86bc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a48cd247a06af51 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:55.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86caa00640 1 -- 172.21.15.143:0/1729543693 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0ab670 con 0x7f86cc141660 2024-01-15T18:12:55.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86caa00640 1 -- 172.21.15.143:0/1729543693 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc0ab7d0 con 0x7f86cc141660 2024-01-15T18:12:55.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86caa00640 1 -- 172.21.15.143:0/1729543693 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0abb60 con 0x7f86cc141660 2024-01-15T18:12:55.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86d2c00640 1 -- 172.21.15.143:0/1729543693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc141660 msgr2=0x7f86cc0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86d2c00640 1 --2- 172.21.15.143:0/1729543693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc141660 0x7f86cc0ce560 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0097c0 tx=0x7f86bc01c450 comp rx=0 tx=0).stop 2024-01-15T18:12:55.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86d2c00640 1 -- 172.21.15.143:0/1729543693 shutdown_connections 2024-01-15T18:12:55.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86d2c00640 1 --2- 172.21.15.143:0/1729543693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc141660 0x7f86cc0ce560 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.688+0000 7f86d2c00640 1 -- 172.21.15.143:0/1729543693 >> 172.21.15.143:0/1729543693 conn(0x7f86cc06c3f0 msgr2=0x7f86cc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:55.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 -- 172.21.15.143:0/1729543693 shutdown_connections 2024-01-15T18:12:55.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 -- 172.21.15.143:0/1729543693 wait complete. 2024-01-15T18:12:55.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 Processor -- start 2024-01-15T18:12:55.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 -- start start 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 0x7f86cc147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc141660 0x7f86cc138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86cc15ec80 0x7f86cc0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc15f130 con 0x7f86cc071d00 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1487d0 con 0x7f86cc141660 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d2c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc148ab0 con 0x7f86cc15ec80 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 0x7f86cc147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 0x7f86cc147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33212/0 (socket says 172.21.15.143:33212) 2024-01-15T18:12:55.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 -- 172.21.15.143:0/140816527 learned_addr learned my addr 172.21.15.143:0/140816527 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 -- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86cc15ec80 msgr2=0x7f86cc0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d0c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86cc15ec80 0x7f86cc0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cb400640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc141660 0x7f86cc138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86cc15ec80 0x7f86cc0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 -- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc141660 msgr2=0x7f86cc138720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc141660 0x7f86cc138720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 -- 172.21.15.143:0/140816527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc009420 con 0x7f86cc071d00 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86d0c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86cc15ec80 0x7f86cc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cb400640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc141660 0x7f86cc138720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:55.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.692+0000 7f86cbe00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 0x7f86cc147e90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0ad040 tx=0x7f86bc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0b5810 con 0x7f86cc071d00 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc0b5990 con 0x7f86cc071d00 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc0cd150 con 0x7f86cc071d00 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0b5d40 con 0x7f86cc071d00 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc149070 con 0x7f86cc071d00 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc072710 con 0x7f86cc071d00 2024-01-15T18:12:55.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f86bc0bf410 con 0x7f86cc071d00 2024-01-15T18:12:55.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86c3e00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86a40674c0 0x7f86a4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:55.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f86bc11df00 con 0x7f86cc071d00 2024-01-15T18:12:55.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.696+0000 7f86cb400640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86a40674c0 0x7f86a4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:55.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.700+0000 7f86cb400640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86a40674c0 0x7f86a4069980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86cc067cd0 tx=0x7f86b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:55.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.700+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86bc0ecab0 con 0x7f86cc071d00 2024-01-15T18:12:55.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.884+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993114", "target": ["mon-mgr", ""]}) v1 -- 0x7f86cc148940 con 0x7f86a40674c0 2024-01-15T18:12:55.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86c3e00640 1 -- 172.21.15.143:0/140816527 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f86cc148940 con 0x7f86a40674c0 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86a40674c0 msgr2=0x7f86a4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86a40674c0 0x7f86a4069980 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f86cc067cd0 tx=0x7f86b4009040 comp rx=0 tx=0).stop 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 msgr2=0x7f86cc147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 0x7f86cc147e90 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f86bc0ad040 tx=0x7f86bc01ac90 comp rx=0 tx=0).stop 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 shutdown_connections 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86a40674c0 0x7f86a4069980 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f86cc15ec80 0x7f86cc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc141660 0x7f86cc138720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 --2- 172.21.15.143:0/140816527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86cc071d00 0x7f86cc147e90 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:55.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 >> 172.21.15.143:0/140816527 conn(0x7f86cc06c3f0 msgr2=0x7f86cc0af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:55.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 shutdown_connections 2024-01-15T18:12:55.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:55.936+0000 7f86d2c00640 1 -- 172.21.15.143:0/140816527 wait complete. 2024-01-15T18:12:55.961 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993115 2024-01-15T18:12:56.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.100+0000 7f0627600640 1 Processor -- start 2024-01-15T18:12:56.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.100+0000 7f0627600640 1 -- start start 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0627600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0620141660 0x7f06200ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0627600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062015f310 con 0x7f0620071d00 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0627600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062015f5f0 con 0x7f062015ec80 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0627600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062015fbb0 con 0x7f0620141660 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0625600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0620071d00 0x7f06200b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35388/0 (socket says 172.21.15.143:35388) 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0625600640 1 -- 172.21.15.143:0/1541711892 learned_addr learned my addr 172.21.15.143:0/1541711892 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:56.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0624c00640 1 --2- 172.21.15.143:0/1541711892 >> v2:172.21.15.204:3300/0 conn(0x7f0620141660 0x7f06200ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858845555 0 0) 0x7f062015f310 con 0x7f0620071d00 2024-01-15T18:12:56.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0600003530 con 0x7f0620071d00 2024-01-15T18:12:56.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476629745 0 0) 0x7f062015f5f0 con 0x7f062015ec80 2024-01-15T18:12:56.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f062015f310 con 0x7f062015ec80 2024-01-15T18:12:56.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0624c00640 1 -- 172.21.15.143:0/1541711892 >> v1:172.21.15.204:6789/0 conn(0x7f062015ec80 legacy=0x7f0620141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0624c00640 1 -- 172.21.15.143:0/1541711892 >> v1:172.21.15.143:6789/0 conn(0x7f0620071d00 legacy=0x7f06200b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4171923575 0 0) 0x7f0600003530 con 0x7f0620071d00 2024-01-15T18:12:56.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.104+0000 7f0624c00640 1 -- 172.21.15.143:0/1541711892 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062015f8d0 con 0x7f0620141660 2024-01-15T18:12:56.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0624c00640 1 --2- 172.21.15.143:0/1541711892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0620141660 0x7f06200ce560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f06100097c0 tx=0x7f061001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bd0be661bc63dc9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100ab670 con 0x7f0620141660 2024-01-15T18:12:56.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06100ab7d0 con 0x7f0620141660 2024-01-15T18:12:56.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f061f400640 1 -- 172.21.15.143:0/1541711892 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100abb60 con 0x7f0620141660 2024-01-15T18:12:56.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 -- 172.21.15.143:0/1541711892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0620141660 msgr2=0x7f06200ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 --2- 172.21.15.143:0/1541711892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0620141660 0x7f06200ce560 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f06100097c0 tx=0x7f061001c450 comp rx=0 tx=0).stop 2024-01-15T18:12:56.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 -- 172.21.15.143:0/1541711892 shutdown_connections 2024-01-15T18:12:56.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 --2- 172.21.15.143:0/1541711892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0620141660 0x7f06200ce560 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 -- 172.21.15.143:0/1541711892 >> 172.21.15.143:0/1541711892 conn(0x7f062006c3f0 msgr2=0x7f062006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:56.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 -- 172.21.15.143:0/1541711892 shutdown_connections 2024-01-15T18:12:56.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 -- 172.21.15.143:0/1541711892 wait complete. 2024-01-15T18:12:56.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 Processor -- start 2024-01-15T18:12:56.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.108+0000 7f0627600640 1 -- start start 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0620071d00 0x7f06201569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 0x7f06200b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062015ec80 0x7f06200acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062015f130 con 0x7f0620141660 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200b2540 con 0x7f062015ec80 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06200b2800 con 0x7f0620071d00 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 0x7f06200b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 0x7f06200b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33224/0 (socket says 172.21.15.143:33224) 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 -- 172.21.15.143:0/2905421559 learned_addr learned my addr 172.21.15.143:0/2905421559 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0625600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062015ec80 0x7f06200acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0624c00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0620071d00 0x7f06201569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 -- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0620071d00 msgr2=0x7f06201569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0620071d00 0x7f06201569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 -- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062015ec80 msgr2=0x7f06200acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062015ec80 0x7f06200acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 -- 172.21.15.143:0/2905421559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610009420 con 0x7f0620141660 2024-01-15T18:12:56.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061fe00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 0x7f06200b1ea0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f060800c430 tx=0x7f060800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080052d0 con 0x7f0620141660 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0608005450 con 0x7f0620141660 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06200cd150 con 0x7f0620141660 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608003840 con 0x7f0620141660 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06200b2dc0 con 0x7f0620141660 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.112+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0620067500 con 0x7f0620141660 2024-01-15T18:12:56.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.116+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f06080039a0 con 0x7f0620141660 2024-01-15T18:12:56.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.116+0000 7f061cc00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f80674c0 0x7f05f8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.116+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0608082580 con 0x7f0620141660 2024-01-15T18:12:56.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.116+0000 7f0624c00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f80674c0 0x7f05f8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.116+0000 7f0624c00640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f80674c0 0x7f05f8069980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f06100af000 tx=0x7f06100a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.120+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06080510c0 con 0x7f0620141660 2024-01-15T18:12:56.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.300+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993115", "target": ["mon-mgr", ""]}) v1 -- 0x7f06200ccfd0 con 0x7f05f80674c0 2024-01-15T18:12:56.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.352+0000 7f061cc00640 1 -- 172.21.15.143:0/2905421559 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0610004020 con 0x7f05f80674c0 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f80674c0 msgr2=0x7f05f8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f80674c0 0x7f05f8069980 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f06100af000 tx=0x7f06100a7000 comp rx=0 tx=0).stop 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 msgr2=0x7f06200b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 0x7f06200b1ea0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f060800c430 tx=0x7f060800c8f0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 shutdown_connections 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f80674c0 0x7f05f8069980 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f062015ec80 0x7f06200acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0620141660 0x7f06200b1ea0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 --2- 172.21.15.143:0/2905421559 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0620071d00 0x7f06201569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 >> 172.21.15.143:0/2905421559 conn(0x7f062006c3f0 msgr2=0x7f06200afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 shutdown_connections 2024-01-15T18:12:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.356+0000 7f0627600640 1 -- 172.21.15.143:0/2905421559 wait complete. 2024-01-15T18:12:56.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993116 2024-01-15T18:12:56.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.516+0000 7f147fc00640 1 Processor -- start 2024-01-15T18:12:56.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.516+0000 7f147fc00640 1 -- start start 2024-01-15T18:12:56.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147fc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1478071d00 0x7f1478072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147fc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147815f310 con 0x7f147815ec80 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147fc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147815fbb0 con 0x7f1478071d00 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1478072690 con 0x7f1478141660 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:54808/0 (socket says 172.21.15.143:54808) 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 -- 172.21.15.143:0/2872715505 learned_addr learned my addr 172.21.15.143:0/2872715505 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147dc00640 1 --2- 172.21.15.143:0/2872715505 >> v2:172.21.15.204:3300/0 conn(0x7f1478071d00 0x7f1478072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 -- 172.21.15.143:0/2872715505 >> v2:172.21.15.204:3300/0 conn(0x7f1478071d00 msgr2=0x7f1478072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 --2- 172.21.15.143:0/2872715505 >> v2:172.21.15.204:3300/0 conn(0x7f1478071d00 0x7f1478072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 -- 172.21.15.143:0/2872715505 >> v1:172.21.15.204:6789/0 conn(0x7f147815ec80 legacy=0x7f1478141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.520+0000 7f147d200640 1 -- 172.21.15.143:0/2872715505 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147815f5f0 con 0x7f1478141660 2024-01-15T18:12:56.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147d200640 1 --2- 172.21.15.143:0/2872715505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780ce560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1468009870 tx=0x7f146801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4afb8e1669720d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f1477400640 1 -- 172.21.15.143:0/2872715505 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14680a7070 con 0x7f1478141660 2024-01-15T18:12:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f1477400640 1 -- 172.21.15.143:0/2872715505 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14680ab430 con 0x7f1478141660 2024-01-15T18:12:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f1477400640 1 -- 172.21.15.143:0/2872715505 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14680ab7c0 con 0x7f1478141660 2024-01-15T18:12:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 -- 172.21.15.143:0/2872715505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 msgr2=0x7f14780ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 --2- 172.21.15.143:0/2872715505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780ce560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1468009870 tx=0x7f146801c540 comp rx=0 tx=0).stop 2024-01-15T18:12:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 -- 172.21.15.143:0/2872715505 shutdown_connections 2024-01-15T18:12:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 --2- 172.21.15.143:0/2872715505 >> v2:172.21.15.204:3300/0 conn(0x7f1478071d00 0x7f1478072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 --2- 172.21.15.143:0/2872715505 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780ce560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 -- 172.21.15.143:0/2872715505 >> 172.21.15.143:0/2872715505 conn(0x7f147806c3f0 msgr2=0x7f147806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 -- 172.21.15.143:0/2872715505 shutdown_connections 2024-01-15T18:12:56.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.524+0000 7f147fc00640 1 -- 172.21.15.143:0/2872715505 wait complete. 2024-01-15T18:12:56.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 Processor -- start 2024-01-15T18:12:56.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- start start 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1478071d00 0x7f1478158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 0x7f1478158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781585d0 con 0x7f147815ec80 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478158aa0 con 0x7f1478071d00 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147813f570 con 0x7f1478141660 2024-01-15T18:12:56.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 0x7f1478158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 0x7f1478158090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33238/0 (socket says 172.21.15.143:33238) 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 -- 172.21.15.143:0/995093719 learned_addr learned my addr 172.21.15.143:0/995093719 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f1477e00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147d200640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1478071d00 0x7f1478158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 -- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 msgr2=0x7f14780b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 -- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1478071d00 msgr2=0x7f1478158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1478071d00 0x7f1478158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 -- 172.21.15.143:0/995093719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680094d0 con 0x7f147815ec80 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147dc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 0x7f1478158090 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f147000ec10 tx=0x7f147000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000cd20 con 0x7f147815ec80 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147000f040 con 0x7f147815ec80 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14700146c0 con 0x7f147815ec80 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14780cd150 con 0x7f147815ec80 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147813fb30 con 0x7f147815ec80 2024-01-15T18:12:56.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.528+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1478067500 con 0x7f147815ec80 2024-01-15T18:12:56.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.532+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f1470014a30 con 0x7f147815ec80 2024-01-15T18:12:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.532+0000 7f1474c00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14500674c0 0x7f1450069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.532+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1470083dd0 con 0x7f147815ec80 2024-01-15T18:12:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.532+0000 7f147d200640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14500674c0 0x7f1450069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.532+0000 7f147d200640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14500674c0 0x7f1450069980 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f14680af000 tx=0x7f1468008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.536+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1470052910 con 0x7f147815ec80 2024-01-15T18:12:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.716+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993116", "target": ["mon-mgr", ""]}) v1 -- 0x7f1478144d10 con 0x7f14500674c0 2024-01-15T18:12:56.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f1474c00640 1 -- 172.21.15.143:0/995093719 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1478144d10 con 0x7f14500674c0 2024-01-15T18:12:56.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14500674c0 msgr2=0x7f1450069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14500674c0 0x7f1450069980 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f14680af000 tx=0x7f1468008040 comp rx=0 tx=0).stop 2024-01-15T18:12:56.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 msgr2=0x7f1478158090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 0x7f1478158090 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f147000ec10 tx=0x7f147000c490 comp rx=0 tx=0).stop 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 shutdown_connections 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14500674c0 0x7f1450069980 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f147815ec80 0x7f1478158090 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1478141660 0x7f14780b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 --2- 172.21.15.143:0/995093719 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1478071d00 0x7f1478158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 >> 172.21.15.143:0/995093719 conn(0x7f147806c3f0 msgr2=0x7f14780afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 shutdown_connections 2024-01-15T18:12:56.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.772+0000 7f147fc00640 1 -- 172.21.15.143:0/995093719 wait complete. 2024-01-15T18:12:56.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993117 2024-01-15T18:12:56.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe863400640 1 Processor -- start 2024-01-15T18:12:56.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe863400640 1 -- start start 2024-01-15T18:12:56.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe863400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe85c06c540 0x7fe85c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe863400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0d02d0 con 0x7fe85c0cf150 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe863400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe85c0cf6c0 con 0x7fe85c06b7b0 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe863400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe85c0cf9a0 con 0x7fe85c06c540 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe85c0cf150 0x7fe85c15ca90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35394/0 (socket says 172.21.15.143:35394) 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85be00640 1 -- 172.21.15.143:0/2488642192 learned_addr learned my addr 172.21.15.143:0/2488642192 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe860a00640 1 --2- 172.21.15.143:0/2488642192 >> v2:172.21.15.204:3300/0 conn(0x7fe85c06c540 0x7fe85c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552427016 0 0) 0x7fe85c0d02d0 con 0x7fe85c0cf150 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe83c003530 con 0x7fe85c0cf150 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396950669 0 0) 0x7fe85c0cf6c0 con 0x7fe85c06b7b0 2024-01-15T18:12:56.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c0d02d0 con 0x7fe85c06b7b0 2024-01-15T18:12:56.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2136351977 0 0) 0x7fe83c003530 con 0x7fe85c0cf150 2024-01-15T18:12:56.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe860a00640 1 -- 172.21.15.143:0/2488642192 >> v1:172.21.15.204:6789/0 conn(0x7fe85c06b7b0 legacy=0x7fe85c0cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe860a00640 1 -- 172.21.15.143:0/2488642192 >> v1:172.21.15.143:6789/0 conn(0x7fe85c0cf150 legacy=0x7fe85c15ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.940+0000 7fe860a00640 1 -- 172.21.15.143:0/2488642192 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c0cfc80 con 0x7fe85c06c540 2024-01-15T18:12:56.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.944+0000 7fe860a00640 1 --2- 172.21.15.143:0/2488642192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c06c540 0x7fe85c0cd7e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0097c0 tx=0x7fe84c01c250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720148ba9c983298 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.944+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844001680 con 0x7fe85c06c540 2024-01-15T18:12:56.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.944+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c0a7050 con 0x7fe85c06c540 2024-01-15T18:12:56.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.944+0000 7fe85b400640 1 -- 172.21.15.143:0/2488642192 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0ab600 con 0x7fe85c06c540 2024-01-15T18:12:56.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.944+0000 7fe863400640 1 -- 172.21.15.143:0/2488642192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c06c540 msgr2=0x7fe85c0cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.944+0000 7fe863400640 1 --2- 172.21.15.143:0/2488642192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c06c540 0x7fe85c0cd7e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0097c0 tx=0x7fe84c01c250 comp rx=0 tx=0).stop 2024-01-15T18:12:56.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- 172.21.15.143:0/2488642192 shutdown_connections 2024-01-15T18:12:56.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 --2- 172.21.15.143:0/2488642192 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c06c540 0x7fe85c0cd7e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- 172.21.15.143:0/2488642192 >> 172.21.15.143:0/2488642192 conn(0x7fe85c0d0e20 msgr2=0x7fe85c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:56.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- 172.21.15.143:0/2488642192 shutdown_connections 2024-01-15T18:12:56.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- 172.21.15.143:0/2488642192 wait complete. 2024-01-15T18:12:56.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 Processor -- start 2024-01-15T18:12:56.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- start start 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe85c06b7b0 0x7fe85c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 0x7fe85c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c0cf150 0x7fe85c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0c6e50 con 0x7fe85c06c540 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0c6240 con 0x7fe85c0cf150 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe863400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c0c6520 con 0x7fe85c06b7b0 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 0x7fe85c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 0x7fe85c0c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33254/0 (socket says 172.21.15.143:33254) 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 -- 172.21.15.143:0/3775835726 learned_addr learned my addr 172.21.15.143:0/3775835726 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe861400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c0cf150 0x7fe85c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe860a00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe85c06b7b0 0x7fe85c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 -- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe85c06b7b0 msgr2=0x7fe85c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe85c06b7b0 0x7fe85c0c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 -- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c0cf150 msgr2=0x7fe85c0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:56.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c0cf150 0x7fe85c0b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 -- 172.21.15.143:0/3775835726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c009420 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.948+0000 7fe85be00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 0x7fe85c0c72e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe84400c750 tx=0x7fe84400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844005ae0 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe844005c40 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85c0c6800 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe844007070 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85c0c5f50 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c0cedc0 con 0x7fe85c06c540 2024-01-15T18:12:56.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fe8440071d0 con 0x7fe85c06c540 2024-01-15T18:12:56.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe858c00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8340674c0 0x7fe834069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe844082c80 con 0x7fe85c06c540 2024-01-15T18:12:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.952+0000 7fe860a00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8340674c0 0x7fe834069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:56.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.956+0000 7fe860a00640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8340674c0 0x7fe834069980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0ad000 tx=0x7fe84c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:56.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:56.956+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe844051760 con 0x7fe85c06c540 2024-01-15T18:12:57.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.172+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993117", "target": ["mon-mgr", ""]}) v1 -- 0x7fe85c0cc420 con 0x7fe8340674c0 2024-01-15T18:12:57.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.596+0000 7fe858c00640 1 -- 172.21.15.143:0/3775835726 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe84c004300 con 0x7fe8340674c0 2024-01-15T18:12:57.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8340674c0 msgr2=0x7fe834069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:57.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8340674c0 0x7fe834069980 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0ad000 tx=0x7fe84c0a6000 comp rx=0 tx=0).stop 2024-01-15T18:12:57.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 msgr2=0x7fe85c0c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:57.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 0x7fe85c0c72e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe84400c750 tx=0x7fe84400cc10 comp rx=0 tx=0).stop 2024-01-15T18:12:57.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 shutdown_connections 2024-01-15T18:12:57.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8340674c0 0x7fe834069980 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:57.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe85c0cf150 0x7fe85c0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:57.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe85c06c540 0x7fe85c0c72e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:57.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 --2- 172.21.15.143:0/3775835726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe85c06b7b0 0x7fe85c0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:57.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 >> 172.21.15.143:0/3775835726 conn(0x7fe85c0d0e20 msgr2=0x7fe85c0d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:57.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 shutdown_connections 2024-01-15T18:12:57.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:57.608+0000 7fe863400640 1 -- 172.21.15.143:0/3775835726 wait complete. 2024-01-15T18:12:57.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993118 2024-01-15T18:12:58.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 Processor -- start 2024-01-15T18:12:58.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 -- start start 2024-01-15T18:12:58.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55fc0cf990 0x7f55fc0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f55fc070e00 0x7f55fc162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc162080 con 0x7f55fc14b1e0 2024-01-15T18:12:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc161d70 con 0x7f55fc0cf990 2024-01-15T18:12:58.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.032+0000 7f5601800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc14b5c0 con 0x7f55fc070e00 2024-01-15T18:12:58.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fa000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55fc0cf990 0x7f55fc0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:58.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fb400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f55fc070e00 0x7f55fc162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:58.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55faa00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f55fc14b1e0 0x7f55fc0cf280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:35258/0 (socket says 172.21.15.143:35258) 2024-01-15T18:12:58.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fa000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55fc0cf990 0x7f55fc0708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47538/0 (socket says 172.21.15.143:47538) 2024-01-15T18:12:58.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55faa00640 1 -- 172.21.15.143:0/626362766 learned_addr learned my addr 172.21.15.143:0/626362766 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fa000640 1 -- 172.21.15.143:0/626362766 >> v2:172.21.15.204:3300/0 conn(0x7f55fc070e00 msgr2=0x7f55fc162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fa000640 1 --2- 172.21.15.143:0/626362766 >> v2:172.21.15.204:3300/0 conn(0x7f55fc070e00 0x7f55fc162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fa000640 1 -- 172.21.15.143:0/626362766 >> v1:172.21.15.204:6789/0 conn(0x7f55fc14b1e0 legacy=0x7f55fc0cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fa000640 1 -- 172.21.15.143:0/626362766 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc071580 con 0x7f55fc0cf990 2024-01-15T18:12:58.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.036+0000 7f55fb400640 1 --2- 172.21.15.143:0/626362766 >> v2:172.21.15.204:3300/0 conn(0x7f55fc070e00 0x7f55fc162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:12:58.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.040+0000 7f55fa000640 1 --2- 172.21.15.143:0/626362766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc0708c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0b4590 tx=0x7f55e401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed20b3b7476e5d44 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:58.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f55f9600640 1 -- 172.21.15.143:0/626362766 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40a7070 con 0x7f55fc0cf990 2024-01-15T18:12:58.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f55f9600640 1 -- 172.21.15.143:0/626362766 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e40ab430 con 0x7f55fc0cf990 2024-01-15T18:12:58.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f55f9600640 1 -- 172.21.15.143:0/626362766 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40ab7c0 con 0x7f55fc0cf990 2024-01-15T18:12:58.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f5601800640 1 -- 172.21.15.143:0/626362766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 msgr2=0x7f55fc0708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:58.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f5601800640 1 --2- 172.21.15.143:0/626362766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc0708c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f55fc0b4590 tx=0x7f55e401c430 comp rx=0 tx=0).stop 2024-01-15T18:12:58.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f5601800640 1 -- 172.21.15.143:0/626362766 shutdown_connections 2024-01-15T18:12:58.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f5601800640 1 --2- 172.21.15.143:0/626362766 >> v2:172.21.15.204:3300/0 conn(0x7f55fc070e00 0x7f55fc162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:58.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f5601800640 1 --2- 172.21.15.143:0/626362766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc0708c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:58.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.044+0000 7f5601800640 1 -- 172.21.15.143:0/626362766 >> 172.21.15.143:0/626362766 conn(0x7f55fc06a890 msgr2=0x7f55fc06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:58.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.048+0000 7f5601800640 1 -- 172.21.15.143:0/626362766 shutdown_connections 2024-01-15T18:12:58.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.048+0000 7f5601800640 1 -- 172.21.15.143:0/626362766 wait complete. 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 Processor -- start 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 -- start start 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 0x7f55fc14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55fc14b1e0 0x7f55fc149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc14c4b0 con 0x7f55fc0cf990 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc14c790 con 0x7f55fc14b1e0 2024-01-15T18:12:58.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.052+0000 7f5601800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc14ca70 con 0x7f55fc070e00 2024-01-15T18:12:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.056+0000 7f55faa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 0x7f55fc14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.056+0000 7f55faa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 0x7f55fc14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60874/0 (socket says 172.21.15.143:60874) 2024-01-15T18:12:58.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.056+0000 7f55faa00640 1 -- 172.21.15.143:0/1407130677 learned_addr learned my addr 172.21.15.143:0/1407130677 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:58.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.056+0000 7f55fb400640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55fc14b1e0 0x7f55fc149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:58.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.056+0000 7f55fa000640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:58.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.056+0000 7f55fb400640 1 -- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 msgr2=0x7f55fc14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:58.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55fb400640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 0x7f55fc14d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55fb400640 1 -- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 msgr2=0x7f55fc14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55fb400640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55fb400640 1 -- 172.21.15.143:0/1407130677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec009420 con 0x7f55fc14b1e0 2024-01-15T18:12:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55faa00640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 0x7f55fc14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:12:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55fa000640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:58.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55fb400640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55fc14b1e0 0x7f55fc149240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00e930 tx=0x7f55ec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec004020 con 0x7f55fc14b1e0 2024-01-15T18:12:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec0041a0 con 0x7f55fc14b1e0 2024-01-15T18:12:58.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00c940 con 0x7f55fc14b1e0 2024-01-15T18:12:58.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e4009420 con 0x7f55fc14b1e0 2024-01-15T18:12:58.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.060+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc14d030 con 0x7f55fc14b1e0 2024-01-15T18:12:58.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.064+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f55ec00caa0 con 0x7f55fc14b1e0 2024-01-15T18:12:58.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.068+0000 7f55f2a00640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55d40674f0 0x7f55d40699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:58.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.068+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f55ec082760 con 0x7f55fc14b1e0 2024-01-15T18:12:58.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.068+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc14c8d0 con 0x7f55fc14b1e0 2024-01-15T18:12:58.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.076+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55fc14c8d0 con 0x7f55fc14b1e0 2024-01-15T18:12:58.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.080+0000 7f55faa00640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55d40674f0 0x7f55d40699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:58.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.080+0000 7f55faa00640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55d40674f0 0x7f55d40699b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f55f4005c10 tx=0x7f55f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:58.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:58.556+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993118", "target": ["mon-mgr", ""]}) v1 -- 0x7f55fc14c8d0 con 0x7f55d40674f0 2024-01-15T18:12:58.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:58.915 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:59.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.336+0000 7f55f2a00640 1 -- 172.21.15.143:0/1407130677 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f55fc14c8d0 con 0x7f55d40674f0 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.344+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55d40674f0 msgr2=0x7f55d40699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.344+0000 7f5601800640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55d40674f0 0x7f55d40699b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f55f4005c10 tx=0x7f55f4009040 comp rx=0 tx=0).stop 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.344+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55fc14b1e0 msgr2=0x7f55fc149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.344+0000 7f5601800640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55fc14b1e0 0x7f55fc149240 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00e930 tx=0x7f55ec00edf0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 shutdown_connections 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55d40674f0 0x7f55d40699b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55fc14b1e0 0x7f55fc149240 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55fc0cf990 0x7f55fc14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 --2- 172.21.15.143:0/1407130677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55fc070e00 0x7f55fc14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 >> 172.21.15.143:0/1407130677 conn(0x7f55fc06a890 msgr2=0x7f55fc16bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:59.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 shutdown_connections 2024-01-15T18:12:59.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.348+0000 7f5601800640 1 -- 172.21.15.143:0/1407130677 wait complete. 2024-01-15T18:12:59.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993119 2024-01-15T18:12:59.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.824+0000 7fb3d7800640 1 Processor -- start 2024-01-15T18:12:59.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.824+0000 7fb3d7800640 1 -- start start 2024-01-15T18:12:59.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3d7800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb3d014afa0 0x7fb3d006ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3d7800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d00cc800 con 0x7fb3d00b3f80 2024-01-15T18:12:59.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3d7800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d014b380 con 0x7fb3d006d340 2024-01-15T18:12:59.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3d7800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d006d7c0 con 0x7fb3d014afa0 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3d4e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb3d014afa0 0x7fb3d006ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3cfe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fb3d006d340 0x7fb3d00b3870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:35264/0 (socket says 172.21.15.143:35264) 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.828+0000 7fb3cfe00640 1 -- 172.21.15.143:0/2940640121 learned_addr learned my addr 172.21.15.143:0/2940640121 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.832+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881188987 0 0) 0x7fb3d014b380 con 0x7fb3d006d340 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.832+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b0003530 con 0x7fb3d006d340 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 943154854 0 0) 0x7fb3b0003530 con 0x7fb3d006d340 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 >> v2:172.21.15.204:3300/0 conn(0x7fb3d014afa0 msgr2=0x7fb3d006ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 --2- 172.21.15.143:0/2940640121 >> v2:172.21.15.204:3300/0 conn(0x7fb3d014afa0 0x7fb3d006ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 >> v1:172.21.15.143:6789/0 conn(0x7fb3d00b3f80 legacy=0x7fb3d00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 --> v1:172.21.15.204:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d006daa0 con 0x7fb3d006d340 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 <== mon.1 v1:172.21.15.204:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fb3c0001660 con 0x7fb3d006d340 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 >> v1:172.21.15.204:6789/0 conn(0x7fb3d006d340 legacy=0x7fb3d00b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b0003980 0x7fb3b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b00047d0 0x7fb3b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b0005190 0x7fb3b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d006d7c0 con 0x7fb3b0003980 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b000c170 con 0x7fb3b00047d0 2024-01-15T18:12:59.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b000c330 con 0x7fb3b0005190 2024-01-15T18:12:59.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3d4e00640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b0003980 0x7fb3b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cfe00640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b00047d0 0x7fb3b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3d5800640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b0005190 0x7fb3b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cfe00640 1 -- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b0005190 msgr2=0x7fb3b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cfe00640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b0005190 0x7fb3b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cfe00640 1 -- 172.21.15.143:0/2940640121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b0003980 msgr2=0x7fb3b0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cfe00640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3b0003980 0x7fb3b0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.836+0000 7fb3cfe00640 1 -- 172.21.15.143:0/2940640121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d006daa0 con 0x7fb3b00047d0 2024-01-15T18:12:59.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.840+0000 7fb3d5800640 1 -- 172.21.15.143:0/2940640121 reap_dead start 2024-01-15T18:12:59.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.848+0000 7fb3cfe00640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b00047d0 0x7fb3b0004c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c1b0 tx=0x7fb3c001cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f7d1e9720d2c03f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:59.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.852+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00a9030 con 0x7fb3b00047d0 2024-01-15T18:12:59.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.852+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c00073d0 con 0x7fb3b00047d0 2024-01-15T18:12:59.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.852+0000 7fb3cf400640 1 -- 172.21.15.143:0/2940640121 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0007780 con 0x7fb3b00047d0 2024-01-15T18:12:59.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.852+0000 7fb3d7800640 1 -- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b00047d0 msgr2=0x7fb3b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:59.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.852+0000 7fb3d7800640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b00047d0 0x7fb3b0004c50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000c1b0 tx=0x7fb3c001cc20 comp rx=0 tx=0).stop 2024-01-15T18:12:59.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.860+0000 7fb3d7800640 1 -- 172.21.15.143:0/2940640121 shutdown_connections 2024-01-15T18:12:59.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.860+0000 7fb3d7800640 1 --2- 172.21.15.143:0/2940640121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3b00047d0 0x7fb3b0004c50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.860+0000 7fb3d7800640 1 -- 172.21.15.143:0/2940640121 >> 172.21.15.143:0/2940640121 conn(0x7fb3d00aea10 msgr2=0x7fb3d00aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:59.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.864+0000 7fb3d7800640 1 -- 172.21.15.143:0/2940640121 shutdown_connections 2024-01-15T18:12:59.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.864+0000 7fb3d7800640 1 -- 172.21.15.143:0/2940640121 wait complete. 2024-01-15T18:12:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.868+0000 7fb3d7800640 1 Processor -- start 2024-01-15T18:12:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.868+0000 7fb3d7800640 1 -- start start 2024-01-15T18:12:59.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d7800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b00047d0 0x7fb3d015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d7800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d0147290 0x7fb3d00ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b00047d0 0x7fb3d015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b00047d0 0x7fb3d015e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60892/0 (socket says 172.21.15.143:60892) 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d7800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3d01fa8a0 0x7fb3d0160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d7800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0147800 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d7800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0147ae0 con 0x7fb3d0147290 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d7800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0147dc0 con 0x7fb3b00047d0 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d4e00640 1 -- 172.21.15.143:0/1630340299 learned_addr learned my addr 172.21.15.143:0/1630340299 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3d5800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3d01fa8a0 0x7fb3d0160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.872+0000 7fb3cfe00640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d0147290 0x7fb3d00ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d5800640 1 -- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b00047d0 msgr2=0x7fb3d015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d5800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b00047d0 0x7fb3d015e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d5800640 1 -- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d0147290 msgr2=0x7fb3d00ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d5800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d0147290 0x7fb3d00ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d5800640 1 -- 172.21.15.143:0/1630340299 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d01480a0 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3cfe00640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d0147290 0x7fb3d00ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d5800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3d01fa8a0 0x7fb3d0160a80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00b0a70 tx=0x7fb3b800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800fdf0 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0148380 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80050d0 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8005480 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.876+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d00adb60 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.880+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d0147f00 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.880+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fb3b8003600 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.880+0000 7fb3ccc00640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb3a80674c0 0x7fb3a8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:59.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.884+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb3b8082af0 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.896+0000 7fb3d4e00640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb3a80674c0 0x7fb3a8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:59.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.896+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb3d0147f00 con 0x7fb3d01fa8a0 2024-01-15T18:12:59.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:12:59.900+0000 7fb3d4e00640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb3a80674c0 0x7fb3a8069980 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009fd0 tx=0x7fb3c8009760 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:00.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.244+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993119", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3d00add50 con 0x7fb3a80674c0 2024-01-15T18:13:00.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.292+0000 7fb3ccc00640 1 -- 172.21.15.143:0/1630340299 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb3d00add50 con 0x7fb3a80674c0 2024-01-15T18:13:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb3a80674c0 msgr2=0x7fb3a8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:00.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb3a80674c0 0x7fb3a8069980 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8009fd0 tx=0x7fb3c8009760 comp rx=0 tx=0).stop 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3d01fa8a0 msgr2=0x7fb3d0160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3d01fa8a0 0x7fb3d0160a80 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00b0a70 tx=0x7fb3b800fc00 comp rx=0 tx=0).stop 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 shutdown_connections 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb3a80674c0 0x7fb3a8069980 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb3d01fa8a0 0x7fb3d0160a80 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3d0147290 0x7fb3d00ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 --2- 172.21.15.143:0/1630340299 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3b00047d0 0x7fb3d015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 >> 172.21.15.143:0/1630340299 conn(0x7fb3d00aea10 msgr2=0x7fb3d0146060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 shutdown_connections 2024-01-15T18:13:00.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.296+0000 7fb3d7800640 1 -- 172.21.15.143:0/1630340299 wait complete. 2024-01-15T18:13:00.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone993120 2024-01-15T18:13:00.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.572+0000 7f9a15e00640 1 Processor -- start 2024-01-15T18:13:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.580+0000 7f9a15e00640 1 -- start start 2024-01-15T18:13:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.580+0000 7f9a15e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9a100cb430 0x7f9a100b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a10071990 0x7f9a1006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a10165930 con 0x7f9a10162490 2024-01-15T18:13:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a100b38a0 con 0x7f9a10071990 2024-01-15T18:13:00.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a101628d0 con 0x7f9a100cb430 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a10071990 0x7f9a1006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9a10071990 0x7f9a1006c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47582/0 (socket says 172.21.15.143:47582) 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 -- 172.21.15.143:0/2670213947 learned_addr learned my addr 172.21.15.143:0/2670213947 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 -- 172.21.15.143:0/2670213947 >> v2:172.21.15.204:3300/0 conn(0x7f9a100cb430 msgr2=0x7f9a100b2b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 --2- 172.21.15.143:0/2670213947 >> v2:172.21.15.204:3300/0 conn(0x7f9a100cb430 0x7f9a100b2b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 -- 172.21.15.143:0/2670213947 >> v1:172.21.15.204:6789/0 conn(0x7f9a10162490 legacy=0x7f9a10071280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.584+0000 7f9a15400640 1 -- 172.21.15.143:0/2670213947 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100b3140 con 0x7f9a10071990 2024-01-15T18:13:00.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.588+0000 7f9a15400640 1 --2- 172.21.15.143:0/2670213947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a10071990 0x7f9a1006c7f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9a100b2af0 tx=0x7f9a0001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a01e0299a7b2ec server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:00.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.592+0000 7f9a0f400640 1 -- 172.21.15.143:0/2670213947 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000a5760 con 0x7f9a10071990 2024-01-15T18:13:00.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.592+0000 7f9a0f400640 1 -- 172.21.15.143:0/2670213947 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a000a58c0 con 0x7f9a10071990 2024-01-15T18:13:00.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.592+0000 7f9a0f400640 1 -- 172.21.15.143:0/2670213947 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000a5c50 con 0x7f9a10071990 2024-01-15T18:13:00.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.596+0000 7f9a15e00640 1 -- 172.21.15.143:0/2670213947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a10071990 msgr2=0x7f9a1006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:00.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.596+0000 7f9a15e00640 1 --2- 172.21.15.143:0/2670213947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a10071990 0x7f9a1006c7f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f9a100b2af0 tx=0x7f9a0001c9b0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.596+0000 7f9a15e00640 1 -- 172.21.15.143:0/2670213947 shutdown_connections 2024-01-15T18:13:00.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.596+0000 7f9a15e00640 1 --2- 172.21.15.143:0/2670213947 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a10071990 0x7f9a1006c7f0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.596+0000 7f9a15e00640 1 --2- 172.21.15.143:0/2670213947 >> v2:172.21.15.204:3300/0 conn(0x7f9a100cb430 0x7f9a100b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.596+0000 7f9a15e00640 1 -- 172.21.15.143:0/2670213947 >> 172.21.15.143:0/2670213947 conn(0x7f9a1006a890 msgr2=0x7f9a1006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:00.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.604+0000 7f9a15e00640 1 -- 172.21.15.143:0/2670213947 shutdown_connections 2024-01-15T18:13:00.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.604+0000 7f9a15e00640 1 -- 172.21.15.143:0/2670213947 wait complete. 2024-01-15T18:13:00.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.608+0000 7f9a15e00640 1 Processor -- start 2024-01-15T18:13:00.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.608+0000 7f9a15e00640 1 -- start start 2024-01-15T18:13:00.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.608+0000 7f9a15e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 0x7f9a1015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a15e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a100cb430 0x7f9a10149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a15e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a10162490 0x7f9a101414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a15e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1014aab0 con 0x7f9a100cb430 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a15e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a10149740 con 0x7f9a10162490 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a15e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a10149a10 con 0x7f9a10071990 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 0x7f9a1015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 0x7f9a1015ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60898/0 (socket says 172.21.15.143:60898) 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 -- 172.21.15.143:0/3190710618 learned_addr learned my addr 172.21.15.143:0/3190710618 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:00.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a0fe00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a100cb430 0x7f9a10149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:00.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a15400640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a10162490 0x7f9a101414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:00.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 -- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a10162490 msgr2=0x7f9a101414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a10162490 0x7f9a101414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 -- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a100cb430 msgr2=0x7f9a10149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a100cb430 0x7f9a10149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a14a00640 1 -- 172.21.15.143:0/3190710618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00009b30 con 0x7f9a10071990 2024-01-15T18:13:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.612+0000 7f9a0fe00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a100cb430 0x7f9a10149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:13:00.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a14a00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 0x7f9a1015ece0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800b4d0 tx=0x7f9a0800b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:00.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08010870 con 0x7f9a10071990 2024-01-15T18:13:00.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a10149cf0 con 0x7f9a10071990 2024-01-15T18:13:00.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1014a2b0 con 0x7f9a10071990 2024-01-15T18:13:00.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a080109d0 con 0x7f9a10071990 2024-01-15T18:13:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08010d80 con 0x7f9a10071990 2024-01-15T18:13:00.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.616+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0005260 con 0x7f9a10071990 2024-01-15T18:13:00.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.624+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f9a08023480 con 0x7f9a10071990 2024-01-15T18:13:00.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.628+0000 7f9a0cc00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f99e80674c0 0x7f99e8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:00.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.628+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9a08083230 con 0x7f9a10071990 2024-01-15T18:13:00.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.636+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9a08051f70 con 0x7f9a10071990 2024-01-15T18:13:00.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.636+0000 7f9a0fe00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f99e80674c0 0x7f99e8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:00.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:00.652+0000 7f9a0fe00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f99e80674c0 0x7f99e8069980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9a10151690 tx=0x7f99f8009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:01.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.084+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone993120", "target": ["mon-mgr", ""]}) v1 -- 0x7f99e0000c90 con 0x7f99e80674c0 2024-01-15T18:13:01.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.532+0000 7f9a0cc00640 1 -- 172.21.15.143:0/3190710618 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f99e0000c90 con 0x7f99e80674c0 2024-01-15T18:13:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.536+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f99e80674c0 msgr2=0x7f99e8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:01.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.536+0000 7f9a15e00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f99e80674c0 0x7f99e8069980 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9a10151690 tx=0x7f99f8009060 comp rx=0 tx=0).stop 2024-01-15T18:13:01.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.536+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 msgr2=0x7f9a1015ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:01.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.536+0000 7f9a15e00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 0x7f9a1015ece0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800b4d0 tx=0x7f9a0800b990 comp rx=0 tx=0).stop 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 shutdown_connections 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f99e80674c0 0x7f99e8069980 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a10162490 0x7f9a101414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9a100cb430 0x7f9a10149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 --2- 172.21.15.143:0/3190710618 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9a10071990 0x7f9a1015ece0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 >> 172.21.15.143:0/3190710618 conn(0x7f9a1006a890 msgr2=0x7f9a100cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 shutdown_connections 2024-01-15T18:13:01.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.540+0000 7f9a15e00640 1 -- 172.21.15.143:0/3190710618 wait complete. 2024-01-15T18:13:01.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:13:01.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.980+0000 7f7c4ae00640 1 Processor -- start 2024-01-15T18:13:01.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.980+0000 7f7c4ae00640 1 -- start start 2024-01-15T18:13:01.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c4ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c4406d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:01.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c4ae00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7c4406a490 0x7f7c4406a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c43400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f7c44165ec0 0x7f7c440ca9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:35268/0 (socket says 172.21.15.143:35268) 2024-01-15T18:13:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c4ae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c4406adf0 con 0x7f7c44165ec0 2024-01-15T18:13:01.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c4ae00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c440cda20 con 0x7f7c4406a490 2024-01-15T18:13:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c4ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440cdd00 con 0x7f7c441614e0 2024-01-15T18:13:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c43400640 1 -- 172.21.15.143:0/1187691248 learned_addr learned my addr 172.21.15.143:0/1187691248 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:13:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c48e00640 1 --2- 172.21.15.143:0/1187691248 >> v2:172.21.15.204:3300/0 conn(0x7f7c4406a490 0x7f7c4406a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:01.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c43e00640 1 --2- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c4406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:01.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c42a00640 1 -- 172.21.15.143:0/1187691248 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666052217 0 0) 0x7f7c4406adf0 con 0x7f7c44165ec0 2024-01-15T18:13:01.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c42a00640 1 -- 172.21.15.143:0/1187691248 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c24003530 con 0x7f7c44165ec0 2024-01-15T18:13:01.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c42a00640 1 -- 172.21.15.143:0/1187691248 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794865775 0 0) 0x7f7c24003530 con 0x7f7c44165ec0 2024-01-15T18:13:01.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c48e00640 1 -- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 msgr2=0x7f7c4406d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:01.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c48e00640 1 --2- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c4406d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:01.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c48e00640 1 -- 172.21.15.143:0/1187691248 >> v1:172.21.15.204:6789/0 conn(0x7f7c44165ec0 legacy=0x7f7c440ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:01.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:01.988+0000 7f7c48e00640 1 -- 172.21.15.143:0/1187691248 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c440cdfe0 con 0x7f7c4406a490 2024-01-15T18:13:02.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.000+0000 7f7c48e00640 1 --2- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c4406a490 0x7f7c4406a8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009d90 tx=0x7f7c3c01c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33363f679021023f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:02.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.000+0000 7f7c42a00640 1 -- 172.21.15.143:0/1187691248 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c001680 con 0x7f7c4406a490 2024-01-15T18:13:02.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.000+0000 7f7c42a00640 1 -- 172.21.15.143:0/1187691248 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c0a55b0 con 0x7f7c4406a490 2024-01-15T18:13:02.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.000+0000 7f7c42a00640 1 -- 172.21.15.143:0/1187691248 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0a5940 con 0x7f7c4406a490 2024-01-15T18:13:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.004+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c4406a490 msgr2=0x7f7c4406a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:02.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.004+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c4406a490 0x7f7c4406a8b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009d90 tx=0x7f7c3c01c7b0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.008+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1187691248 shutdown_connections 2024-01-15T18:13:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.008+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c4406a490 0x7f7c4406a8b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.008+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1187691248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c4406d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.008+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1187691248 >> 172.21.15.143:0/1187691248 conn(0x7f7c440cfd20 msgr2=0x7f7c440d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:02.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.012+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1187691248 shutdown_connections 2024-01-15T18:13:02.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.012+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1187691248 wait complete. 2024-01-15T18:13:02.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.020+0000 7f7c4ae00640 1 Processor -- start 2024-01-15T18:13:02.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.020+0000 7f7c4ae00640 1 -- start start 2024-01-15T18:13:02.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c4ae00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c4406a490 0x7f7c4406db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:02.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c4ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c442020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:02.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c4ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 0x7f7c44163ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:02.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c4406a490 0x7f7c4406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c43e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c4406a490 0x7f7c4406db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47600/0 (socket says 172.21.15.143:47600) 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c4ae00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44163810 con 0x7f7c4406a490 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c4ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44202610 con 0x7f7c44165ec0 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c4ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c440c6290 con 0x7f7c441614e0 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c48e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 0x7f7c44163ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c48e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 0x7f7c44163ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48730/0 (socket says 172.21.15.143:48730) 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c48e00640 1 -- 172.21.15.143:0/1292893543 learned_addr learned my addr 172.21.15.143:0/1292893543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.024+0000 7f7c43400640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c442020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:02.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c48e00640 1 -- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 msgr2=0x7f7c442020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c48e00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c442020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c48e00640 1 -- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c4406a490 msgr2=0x7f7c4406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c48e00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c4406a490 0x7f7c4406db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c48e00640 1 -- 172.21.15.143:0/1292893543 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c009a70 con 0x7f7c44165ec0 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c48e00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 0x7f7c44163ca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c01d050 tx=0x7f7c3c019550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0ad640 con 0x7f7c44165ec0 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3c0ad7a0 con 0x7f7c44165ec0 2024-01-15T18:13:02.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.028+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c0adb30 con 0x7f7c44165ec0 2024-01-15T18:13:02.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.032+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c440c6570 con 0x7f7c44165ec0 2024-01-15T18:13:02.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.032+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c441641e0 con 0x7f7c44165ec0 2024-01-15T18:13:02.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.036+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c441639c0 con 0x7f7c44165ec0 2024-01-15T18:13:02.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.040+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f7c3c0b4070 con 0x7f7c44165ec0 2024-01-15T18:13:02.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.040+0000 7f7c3be00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c1c067590 0x7f7c1c069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:02.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.044+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7c3c11df30 con 0x7f7c44165ec0 2024-01-15T18:13:02.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.056+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c441639c0 con 0x7f7c44165ec0 2024-01-15T18:13:02.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.060+0000 7f7c43e00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c1c067590 0x7f7c1c069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:02.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.064+0000 7f7c43e00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c1c067590 0x7f7c1c069a50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009af0 tx=0x7f7c34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:02.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.552+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7c44163980 con 0x7f7c44165ec0 2024-01-15T18:13:02.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.556+0000 7f7c3be00640 1 -- 172.21.15.143:0/1292893543 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f7c3c0b2990 con 0x7f7c44165ec0 2024-01-15T18:13:02.558 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:13:02.558 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:13:02.558 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:13:02.558 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:13:02.558 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:13:02.558 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:13:02.559 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 508, 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:13:02.560 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 74, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:13:02.561 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 2355, 2024-01-15T18:13:02.562 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 9669631595, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 19728687104, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 753365426176, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 185832695, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 195304127, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 44, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 56 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:13:02.563 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 45, 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 5, 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:13:02.564 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 5, 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "c", 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 6996 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:13:02.565 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 11, 2024-01-15T18:13:02.566 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:12:40.782196+0000", 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": { 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "4d084b21-4678-4506-adc6-0f7b02c121a2": { 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "18 ongoing+pending cloning jobs; Avg progress = 14.816% (36s)\n [====........................] (remaining: 3m)", 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.14815555512905121, 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "68b86d30-057d-4358-8e21-5f124c115b7d": { 2024-01-15T18:13:02.567 INFO:teuthology.orchestra.run.smithi143.stdout: "message": "4 ongoing cloning jobs; Avg progress = 66.67% (36s)\n [==================..........] (remaining: 18s)", 2024-01-15T18:13:02.568 INFO:teuthology.orchestra.run.smithi143.stdout: "progress": 0.66670000553131104, 2024-01-15T18:13:02.568 INFO:teuthology.orchestra.run.smithi143.stdout: "add_to_ceph_s": true 2024-01-15T18:13:02.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:13:02.568 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:13:02.568 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:13:02.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.560+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c1c067590 msgr2=0x7f7c1c069a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:02.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.560+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c1c067590 0x7f7c1c069a50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7c34009af0 tx=0x7f7c34008040 comp rx=0 tx=0).stop 2024-01-15T18:13:02.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.560+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 msgr2=0x7f7c44163ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:02.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.560+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 0x7f7c44163ca0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c01d050 tx=0x7f7c3c019550 comp rx=0 tx=0).stop 2024-01-15T18:13:02.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.564+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 shutdown_connections 2024-01-15T18:13:02.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.564+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c1c067590 0x7f7c1c069a50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.564+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7c44165ec0 0x7f7c44163ca0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.564+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7c441614e0 0x7f7c442020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.564+0000 7f7c4ae00640 1 --2- 172.21.15.143:0/1292893543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c4406a490 0x7f7c4406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:02.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.564+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 >> 172.21.15.143:0/1292893543 conn(0x7f7c440cfd20 msgr2=0x7f7c4406ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:02.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.568+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 shutdown_connections 2024-01-15T18:13:02.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:02.568+0000 7f7c4ae00640 1 -- 172.21.15.143:0/1292893543 wait complete. 2024-01-15T18:13:02.641 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-01-15T18:13:02.641 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.1481555551290512; onpen_p = 0.2857142984867096 2024-01-15T18:13:02.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99311 2024-01-15T18:13:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.072+0000 7febb1800640 1 Processor -- start 2024-01-15T18:13:03.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.072+0000 7febb1800640 1 -- start start 2024-01-15T18:13:03.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febb1800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7febac14b1e0 0x7febac0cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febb1800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febac0cf7c0 0x7febac0708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febb1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac070e00 0x7febac0b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febb1800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0cfc20 con 0x7febac0cf7c0 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febb1800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac162080 con 0x7febac14b1e0 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febb1800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac161d70 con 0x7febac070e00 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7febac14b1e0 0x7febac0cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7febac14b1e0 0x7febac0cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48732/0 (socket says 172.21.15.143:48732) 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 -- 172.21.15.143:0/2128422918 learned_addr learned my addr 172.21.15.143:0/2128422918 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 -- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac070e00 msgr2=0x7febac0b1ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 --2- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac070e00 0x7febac0b1ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 -- 172.21.15.143:0/2128422918 >> v2:172.21.15.143:3300/0 conn(0x7febac0cf7c0 msgr2=0x7febac0708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 --2- 172.21.15.143:0/2128422918 >> v2:172.21.15.143:3300/0 conn(0x7febac0cf7c0 0x7febac0708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.076+0000 7febaaa00640 1 -- 172.21.15.143:0/2128422918 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac14b5c0 con 0x7febac14b1e0 2024-01-15T18:13:03.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.084+0000 7febaaa00640 1 --2- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac0cf280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7febac06e480 tx=0x7feb9402d240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a30a8f71887eaf9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:03.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.088+0000 7feba9600640 1 -- 172.21.15.143:0/2128422918 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb940bc600 con 0x7febac14b1e0 2024-01-15T18:13:03.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.088+0000 7feba9600640 1 -- 172.21.15.143:0/2128422918 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb940b8050 con 0x7febac14b1e0 2024-01-15T18:13:03.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.088+0000 7feba9600640 1 -- 172.21.15.143:0/2128422918 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb940bc600 con 0x7febac14b1e0 2024-01-15T18:13:03.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.096+0000 7febb1800640 1 -- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 msgr2=0x7febac0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:03.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.096+0000 7febb1800640 1 --2- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac0cf280 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7febac06e480 tx=0x7feb9402d240 comp rx=0 tx=0).stop 2024-01-15T18:13:03.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 -- 172.21.15.143:0/2128422918 shutdown_connections 2024-01-15T18:13:03.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 --2- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac070e00 0x7febac0b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 --2- 172.21.15.143:0/2128422918 >> v2:172.21.15.143:3300/0 conn(0x7febac0cf7c0 0x7febac0708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 --2- 172.21.15.143:0/2128422918 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac0cf280 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 -- 172.21.15.143:0/2128422918 >> 172.21.15.143:0/2128422918 conn(0x7febac06a890 msgr2=0x7febac06aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:03.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 -- 172.21.15.143:0/2128422918 shutdown_connections 2024-01-15T18:13:03.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.104+0000 7febb1800640 1 -- 172.21.15.143:0/2128422918 wait complete. 2024-01-15T18:13:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.112+0000 7febb1800640 1 Processor -- start 2024-01-15T18:13:03.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.112+0000 7febb1800640 1 -- start start 2024-01-15T18:13:03.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febb1800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac070e00 0x7febac161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febb1800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac0cf7c0 0x7febac149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febb1800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac1495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febaaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac070e00 0x7febac161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febb1800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0aafa0 con 0x7febac070e00 2024-01-15T18:13:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febb1800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac131270 con 0x7febac14b1e0 2024-01-15T18:13:03.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febaaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac070e00 0x7febac161470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47618/0 (socket says 172.21.15.143:47618) 2024-01-15T18:13:03.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febaaa00640 1 -- 172.21.15.143:0/454132972 learned_addr learned my addr 172.21.15.143:0/454132972 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:03.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febaa000640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac0cf7c0 0x7febac149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.116+0000 7febb1800640 1 -- 172.21.15.143:0/454132972 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac131550 con 0x7febac0cf7c0 2024-01-15T18:13:03.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febab400640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac1495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febaa000640 1 -- 172.21.15.143:0/454132972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 msgr2=0x7febac1495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:03.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febaa000640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac1495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febaa000640 1 -- 172.21.15.143:0/454132972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac070e00 msgr2=0x7febac161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febaa000640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febac070e00 0x7febac161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febaa000640 1 -- 172.21.15.143:0/454132972 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940094d0 con 0x7febac0cf7c0 2024-01-15T18:13:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febab400640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febac14b1e0 0x7febac1495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:13:03.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.120+0000 7febaa000640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febac0cf7c0 0x7febac149080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feba400bd70 tx=0x7feba400be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:03.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.124+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4014440 con 0x7febac0cf7c0 2024-01-15T18:13:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.124+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40145c0 con 0x7febac0cf7c0 2024-01-15T18:13:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.124+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4014970 con 0x7febac0cf7c0 2024-01-15T18:13:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.124+0000 7febb1800640 1 -- 172.21.15.143:0/454132972 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac131830 con 0x7febac0cf7c0 2024-01-15T18:13:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.124+0000 7febb1800640 1 -- 172.21.15.143:0/454132972 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac134760 con 0x7febac0cf7c0 2024-01-15T18:13:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.128+0000 7febb1800640 1 -- 172.21.15.143:0/454132972 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac131d50 con 0x7febac0cf7c0 2024-01-15T18:13:03.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.128+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7feba4014ad0 con 0x7febac0cf7c0 2024-01-15T18:13:03.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.132+0000 7feba2a00640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb84067510 0x7feb840699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.136+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7feba4055940 con 0x7febac0cf7c0 2024-01-15T18:13:03.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.136+0000 7febaaa00640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb84067510 0x7feb840699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.140+0000 7febaaa00640 1 --2- 172.21.15.143:0/454132972 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feb84067510 0x7feb840699d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7feb9402b8f0 tx=0x7feb940b7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:03.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.140+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feba4052410 con 0x7febac0cf7c0 2024-01-15T18:13:03.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.532+0000 7febb1800640 1 -- 172.21.15.143:0/454132972 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99311", "target": ["mon-mgr", ""]}) v1 -- 0x7febac1313e0 con 0x7feb84067510 2024-01-15T18:13:03.559 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:03.556+0000 7f3caa200640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-01-15T18:13:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.556+0000 7feba2a00640 1 -- 172.21.15.143:0/454132972 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) v1 ==== 60+0+0 (secure 0 0 0) 0x7febac1313e0 con 0x7feb84067510 2024-01-15T18:13:03.559 INFO:teuthology.orchestra.run.smithi143.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-01-15T18:13:03.568 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-15T18:13:03.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99312 2024-01-15T18:13:03.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 Processor -- start 2024-01-15T18:13:03.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 -- start start 2024-01-15T18:13:03.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3b48158bc0 0x7f3b48071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f3f0 0x7f3b4806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3b4806f880 0x7f3b4806a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b4815c4f0 con 0x7f3b4806f880 2024-01-15T18:13:03.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b48158fa0 con 0x7f3b48158bc0 2024-01-15T18:13:03.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b50000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4815b970 con 0x7f3b4806f3f0 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3b4806f880 0x7f3b4806a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3b4806f880 0x7f3b4806a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47624/0 (socket says 172.21.15.143:47624) 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 -- 172.21.15.143:0/2692856819 learned_addr learned my addr 172.21.15.143:0/2692856819 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4d600640 1 --2- 172.21.15.143:0/2692856819 >> v2:172.21.15.204:3300/0 conn(0x7f3b48158bc0 0x7f3b48071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4cc00640 1 --2- 172.21.15.143:0/2692856819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f3f0 0x7f3b4806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 -- 172.21.15.143:0/2692856819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f3f0 msgr2=0x7f3b4806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 --2- 172.21.15.143:0/2692856819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f3f0 0x7f3b4806a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 -- 172.21.15.143:0/2692856819 >> v2:172.21.15.204:3300/0 conn(0x7f3b48158bc0 msgr2=0x7f3b48071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 --2- 172.21.15.143:0/2692856819 >> v2:172.21.15.204:3300/0 conn(0x7f3b48158bc0 0x7f3b48071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.708+0000 7f3b4e000640 1 -- 172.21.15.143:0/2692856819 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4815bc50 con 0x7f3b4806f880 2024-01-15T18:13:03.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b4e000640 1 --2- 172.21.15.143:0/2692856819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f880 0x7f3b4806a9d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3b4806cc40 tx=0x7f3b4001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39af90d703dea19e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:03.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b47e00640 1 -- 172.21.15.143:0/2692856819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400a5700 con 0x7f3b4806f880 2024-01-15T18:13:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b47e00640 1 -- 172.21.15.143:0/2692856819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b400a5860 con 0x7f3b4806f880 2024-01-15T18:13:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b47e00640 1 -- 172.21.15.143:0/2692856819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400a5bf0 con 0x7f3b4806f880 2024-01-15T18:13:03.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 -- 172.21.15.143:0/2692856819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f880 msgr2=0x7f3b4806a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 --2- 172.21.15.143:0/2692856819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f880 0x7f3b4806a9d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3b4806cc40 tx=0x7f3b4001c880 comp rx=0 tx=0).stop 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 -- 172.21.15.143:0/2692856819 shutdown_connections 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 --2- 172.21.15.143:0/2692856819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f880 0x7f3b4806a9d0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 --2- 172.21.15.143:0/2692856819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f3f0 0x7f3b4806a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 --2- 172.21.15.143:0/2692856819 >> v2:172.21.15.204:3300/0 conn(0x7f3b48158bc0 0x7f3b48071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 -- 172.21.15.143:0/2692856819 >> 172.21.15.143:0/2692856819 conn(0x7f3b480cfd20 msgr2=0x7f3b480d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 -- 172.21.15.143:0/2692856819 shutdown_connections 2024-01-15T18:13:03.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.712+0000 7f3b50000640 1 -- 172.21.15.143:0/2692856819 wait complete. 2024-01-15T18:13:03.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 Processor -- start 2024-01-15T18:13:03.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 -- start start 2024-01-15T18:13:03.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f3f0 0x7f3b48155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f880 0x7f3b480cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b48158bc0 0x7f3b480ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480ad480 con 0x7f3b4806f3f0 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480ad720 con 0x7f3b48158bc0 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480ada00 con 0x7f3b4806f880 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f3f0 0x7f3b48155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f3f0 0x7f3b48155ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47628/0 (socket says 172.21.15.143:47628) 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 -- 172.21.15.143:0/2165841157 learned_addr learned my addr 172.21.15.143:0/2165841157 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4cc00640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f880 0x7f3b480cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4e000640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b48158bc0 0x7f3b480ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 -- 172.21.15.143:0/2165841157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f880 msgr2=0x7f3b480cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3b4806f880 0x7f3b480cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 -- 172.21.15.143:0/2165841157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b48158bc0 msgr2=0x7f3b480ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b48158bc0 0x7f3b480ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 -- 172.21.15.143:0/2165841157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b40009b30 con 0x7f3b4806f3f0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b4d600640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3b4806f3f0 0x7f3b48155ea0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800e930 tx=0x7f3b3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38009690 con 0x7f3b4806f3f0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3800ca00 con 0x7f3b4806f3f0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 -- 172.21.15.143:0/2165841157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b480adce0 con 0x7f3b4806f3f0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3800cd70 con 0x7f3b4806f3f0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.716+0000 7f3b50000640 1 -- 172.21.15.143:0/2165841157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b481563e0 con 0x7f3b4806f3f0 2024-01-15T18:13:03.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.720+0000 7f3b50000640 1 -- 172.21.15.143:0/2165841157 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b48067500 con 0x7f3b4806f3f0 2024-01-15T18:13:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.720+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f3b38013470 con 0x7f3b4806f3f0 2024-01-15T18:13:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.720+0000 7f3b45600640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b200674c0 0x7f3b20069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.720+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3b38083340 con 0x7f3b4806f3f0 2024-01-15T18:13:03.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.720+0000 7f3b4cc00640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b200674c0 0x7f3b20069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:03.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.720+0000 7f3b4cc00640 1 --2- 172.21.15.143:0/2165841157 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3b200674c0 0x7f3b20069980 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3b481f9180 tx=0x7f3b30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:03.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.724+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3b38051e30 con 0x7f3b4806f3f0 2024-01-15T18:13:03.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:03.908+0000 7f3b50000640 1 -- 172.21.15.143:0/2165841157 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99312", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b480ae190 con 0x7f3b200674c0 2024-01-15T18:13:09.012 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:09.008+0000 7f3caa200640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-01-15T18:13:09.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.008+0000 7f3b45600640 1 -- 172.21.15.143:0/2165841157 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f3b30009620 con 0x7f3b200674c0 2024-01-15T18:13:09.013 INFO:teuthology.orchestra.run.smithi143.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-01-15T18:13:09.021 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-15T18:13:09.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99313 2024-01-15T18:13:09.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 Processor -- start 2024-01-15T18:13:09.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 -- start start 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fa406bfb0 0x7f0fa406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40b0310 0x7f0fa40cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa40cbff0 con 0x7f0fa40cc9c0 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa40d0c50 con 0x7f0fa406bfb0 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fac400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa406c8d0 con 0x7f0fa40b0310 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fa406bfb0 0x7f0fa406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fa406bfb0 0x7f0fa406c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38200/0 (socket says 172.21.15.143:38200) 2024-01-15T18:13:09.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2182072611 learned_addr learned my addr 172.21.15.143:0/2182072611 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:09.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9000640 1 --2- 172.21.15.143:0/2182072611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40b0310 0x7f0fa40cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:09.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2182072611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40b0310 msgr2=0x7f0fa40cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:09.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 --2- 172.21.15.143:0/2182072611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40b0310 0x7f0fa40cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:09.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2182072611 >> v1:172.21.15.204:6789/0 conn(0x7f0fa40cc9c0 legacy=0x7f0fa40cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:09.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.164+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2182072611 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40cb6c0 con 0x7f0fa406bfb0 2024-01-15T18:13:09.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fa9a00640 1 --2- 172.21.15.143:0/2182072611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa406c390 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009870 tx=0x7f0f9401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d62944292f26d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:09.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fa3e00640 1 -- 172.21.15.143:0/2182072611 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940ab680 con 0x7f0fa406bfb0 2024-01-15T18:13:09.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fa3e00640 1 -- 172.21.15.143:0/2182072611 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f940ab7e0 con 0x7f0fa406bfb0 2024-01-15T18:13:09.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fa3e00640 1 -- 172.21.15.143:0/2182072611 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940abb70 con 0x7f0fa406bfb0 2024-01-15T18:13:09.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 -- 172.21.15.143:0/2182072611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 msgr2=0x7f0fa406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:09.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 --2- 172.21.15.143:0/2182072611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa406c390 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009870 tx=0x7f0f9401c480 comp rx=0 tx=0).stop 2024-01-15T18:13:09.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 -- 172.21.15.143:0/2182072611 shutdown_connections 2024-01-15T18:13:09.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 --2- 172.21.15.143:0/2182072611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40b0310 0x7f0fa40cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:09.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 --2- 172.21.15.143:0/2182072611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa406c390 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:09.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 -- 172.21.15.143:0/2182072611 >> 172.21.15.143:0/2182072611 conn(0x7f0fa40d0ed0 msgr2=0x7f0fa40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:09.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 -- 172.21.15.143:0/2182072611 shutdown_connections 2024-01-15T18:13:09.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.168+0000 7f0fac400640 1 -- 172.21.15.143:0/2182072611 wait complete. 2024-01-15T18:13:09.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 Processor -- start 2024-01-15T18:13:09.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- start start 2024-01-15T18:13:09.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:09.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fa40b0310 0x7f0fa415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40cc9c0 0x7f0fa40af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa415f780 con 0x7f0fa406bfb0 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa415fa60 con 0x7f0fa40b0310 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa415fd40 con 0x7f0fa40cc9c0 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa415f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38212/0 (socket says 172.21.15.143:38212) 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2143994314 learned_addr learned my addr 172.21.15.143:0/2143994314 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0faa400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40cc9c0 0x7f0fa40af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9000640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fa40b0310 0x7f0fa415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:09.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40cc9c0 msgr2=0x7f0fa40af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40cc9c0 0x7f0fa40af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fa40b0310 msgr2=0x7f0fa415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fa40b0310 0x7f0fa415bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 -- 172.21.15.143:0/2143994314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940094d0 con 0x7f0fa406bfb0 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa9a00640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa415f0a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0f940097c0 tx=0x7f0f9401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940b4800 con 0x7f0fa406bfb0 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f940b4980 con 0x7f0fa406bfb0 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f940b4d30 con 0x7f0fa406bfb0 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa4160020 con 0x7f0fa406bfb0 2024-01-15T18:13:09.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa415c2e0 con 0x7f0fa406bfb0 2024-01-15T18:13:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.172+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa4067500 con 0x7f0fa406bfb0 2024-01-15T18:13:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.176+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f0f940c7440 con 0x7f0fa406bfb0 2024-01-15T18:13:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.176+0000 7f0fa1600640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f7c0674c0 0x7f0f7c069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.176+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0f9411ddb0 con 0x7f0fa406bfb0 2024-01-15T18:13:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.176+0000 7f0fa9000640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f7c0674c0 0x7f0f7c069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:09.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.176+0000 7f0fa9000640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f7c0674c0 0x7f0f7c069980 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40cb690 tx=0x7f0f8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:09.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.180+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0f940ec890 con 0x7f0fa406bfb0 2024-01-15T18:13:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:09.360+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99313", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fa4066ec0 con 0x7f0f7c0674c0 2024-01-15T18:13:10.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.408+0000 7f0fa1600640 1 -- 172.21.15.143:0/2143994314 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0fa4066ec0 con 0x7f0f7c0674c0 2024-01-15T18:13:10.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f7c0674c0 msgr2=0x7f0f7c069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:10.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f7c0674c0 0x7f0f7c069980 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40cb690 tx=0x7f0f8c008040 comp rx=0 tx=0).stop 2024-01-15T18:13:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 msgr2=0x7f0fa415f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa415f0a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0f940097c0 tx=0x7f0f9401acc0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 shutdown_connections 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f7c0674c0 0x7f0f7c069980 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fa40cc9c0 0x7f0fa40af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fa40b0310 0x7f0fa415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 --2- 172.21.15.143:0/2143994314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa406bfb0 0x7f0fa415f0a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 >> 172.21.15.143:0/2143994314 conn(0x7f0fa40d0ed0 msgr2=0x7f0fa40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 shutdown_connections 2024-01-15T18:13:10.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.416+0000 7f0fac400640 1 -- 172.21.15.143:0/2143994314 wait complete. 2024-01-15T18:13:10.468 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99314 2024-01-15T18:13:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.760+0000 7f4f13a00640 1 Processor -- start 2024-01-15T18:13:10.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.760+0000 7f4f13a00640 1 -- start start 2024-01-15T18:13:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f13a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c0ce2d0 0x7f4f0c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:10.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f13a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f0c15b110 0x7f4f0c15da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f13a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0c0cb980 con 0x7f4f0c071140 2024-01-15T18:13:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f13a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0c0cc220 con 0x7f4f0c15b110 2024-01-15T18:13:10.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f13a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c15a4a0 con 0x7f4f0c0ce2d0 2024-01-15T18:13:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f11a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f0c15b110 0x7f4f0c15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f11a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4f0c15b110 0x7f4f0c15da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:54550/0 (socket says 172.21.15.143:54550) 2024-01-15T18:13:10.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.764+0000 7f4f11a00640 1 -- 172.21.15.143:0/196506614 learned_addr learned my addr 172.21.15.143:0/196506614 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.768+0000 7f4f0be00640 1 --2- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c0ce2d0 0x7f4f0c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.768+0000 7f4f11a00640 1 -- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c0ce2d0 msgr2=0x7f4f0c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.768+0000 7f4f11a00640 1 --2- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c0ce2d0 0x7f4f0c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.768+0000 7f4f11a00640 1 -- 172.21.15.143:0/196506614 >> v1:172.21.15.143:6789/0 conn(0x7f4f0c071140 legacy=0x7f4f0c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:10.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.768+0000 7f4f11a00640 1 -- 172.21.15.143:0/196506614 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0cbc60 con 0x7f4f0c15b110 2024-01-15T18:13:10.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.772+0000 7f4f11a00640 1 --2- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15da90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009e50 tx=0x7f4f0401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97435b25b11f31b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.772+0000 7f4f0b400640 1 -- 172.21.15.143:0/196506614 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040a57c0 con 0x7f4f0c15b110 2024-01-15T18:13:10.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.772+0000 7f4f0b400640 1 -- 172.21.15.143:0/196506614 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f040a5920 con 0x7f4f0c15b110 2024-01-15T18:13:10.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.772+0000 7f4f0b400640 1 -- 172.21.15.143:0/196506614 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040a5cb0 con 0x7f4f0c15b110 2024-01-15T18:13:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 -- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c15b110 msgr2=0x7f4f0c15da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:10.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 --2- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15da90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009e50 tx=0x7f4f0401c9e0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 -- 172.21.15.143:0/196506614 shutdown_connections 2024-01-15T18:13:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 --2- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15da90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 --2- 172.21.15.143:0/196506614 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c0ce2d0 0x7f4f0c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 -- 172.21.15.143:0/196506614 >> 172.21.15.143:0/196506614 conn(0x7f4f0c0cfd20 msgr2=0x7f4f0c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:10.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.776+0000 7f4f13a00640 1 -- 172.21.15.143:0/196506614 shutdown_connections 2024-01-15T18:13:10.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.780+0000 7f4f13a00640 1 -- 172.21.15.143:0/196506614 wait complete. 2024-01-15T18:13:10.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.780+0000 7f4f13a00640 1 Processor -- start 2024-01-15T18:13:10.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 -- start start 2024-01-15T18:13:10.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c071140 0x7f4f0c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:10.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c0ce2d0 0x7f4f0c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:10.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:10.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0c73b0 con 0x7f4f0c15b110 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0b5310 con 0x7f4f0c0ce2d0 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f13a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0c0b5ae0 con 0x7f4f0c071140 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c071140 0x7f4f0c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f0be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c0ce2d0 0x7f4f0c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38224/0 (socket says 172.21.15.143:38224) 2024-01-15T18:13:10.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 -- 172.21.15.143:0/732790701 learned_addr learned my addr 172.21.15.143:0/732790701 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:10.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 -- 172.21.15.143:0/732790701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c071140 msgr2=0x7f4f0c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:10.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c071140 0x7f4f0c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 -- 172.21.15.143:0/732790701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c0ce2d0 msgr2=0x7f4f0c0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:10.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c0ce2d0 0x7f4f0c0c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:10.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.784+0000 7f4f11a00640 1 -- 172.21.15.143:0/732790701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04009b30 con 0x7f4f0c15b110 2024-01-15T18:13:10.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f11a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15c570 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4f0401a610 tx=0x7f4f04019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:10.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040b8640 con 0x7f4f0c15b110 2024-01-15T18:13:10.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0c0b5520 con 0x7f4f0c15b110 2024-01-15T18:13:10.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f040b87a0 con 0x7f4f0c15b110 2024-01-15T18:13:10.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f040b8b50 con 0x7f4f0c15b110 2024-01-15T18:13:10.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0c0c93a0 con 0x7f4f0c15b110 2024-01-15T18:13:10.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.788+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0c06f0b0 con 0x7f4f0c15b110 2024-01-15T18:13:10.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.792+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f4f040b8cb0 con 0x7f4f0c15b110 2024-01-15T18:13:10.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.792+0000 7f4f08c00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ee4067510 0x7f4ee40699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:10.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.792+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4f0411e490 con 0x7f4f0c15b110 2024-01-15T18:13:10.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.796+0000 7f4f11000640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ee4067510 0x7f4ee40699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:10.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.800+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f040ed2f0 con 0x7f4f0c15b110 2024-01-15T18:13:10.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:10.800+0000 7f4f11000640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ee4067510 0x7f4ee40699d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c1fe310 tx=0x7f4efc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:11.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.124+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f0c06f710 con 0x7f4ee4067510 2024-01-15T18:13:11.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f08c00640 1 -- 172.21.15.143:0/732790701 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4f0c06f710 con 0x7f4ee4067510 2024-01-15T18:13:11.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ee4067510 msgr2=0x7f4ee40699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:11.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ee4067510 0x7f4ee40699d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c1fe310 tx=0x7f4efc009040 comp rx=0 tx=0).stop 2024-01-15T18:13:11.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 msgr2=0x7f4f0c15c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:11.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15c570 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4f0401a610 tx=0x7f4f04019780 comp rx=0 tx=0).stop 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 shutdown_connections 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ee4067510 0x7f4ee40699d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f0c15b110 0x7f4f0c15c570 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f0c0ce2d0 0x7f4f0c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 --2- 172.21.15.143:0/732790701 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f0c071140 0x7f4f0c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 >> 172.21.15.143:0/732790701 conn(0x7f4f0c0cfd20 msgr2=0x7f4f0c162ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 shutdown_connections 2024-01-15T18:13:11.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.276+0000 7f4f13a00640 1 -- 172.21.15.143:0/732790701 wait complete. 2024-01-15T18:13:11.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99315 2024-01-15T18:13:11.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 Processor -- start 2024-01-15T18:13:11.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 -- start start 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54181607c0 0x7f541806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54180cbf50 0x7f54180cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54180cffd0 con 0x7f54180cfb30 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54180d1ce0 con 0x7f54180cbf50 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f541d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180d02b0 con 0x7f54181607c0 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5417400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54180cbf50 0x7f54180cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5416000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f54180cfb30 0x7f541806bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34264/0 (socket says 172.21.15.143:34264) 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5416000640 1 -- 172.21.15.143:0/226996364 learned_addr learned my addr 172.21.15.143:0/226996364 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5417400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54180cbf50 0x7f54180cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38226/0 (socket says 172.21.15.143:38226) 2024-01-15T18:13:11.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5416a00640 1 --2- 172.21.15.143:0/226996364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54181607c0 0x7f541806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:11.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5417400640 1 -- 172.21.15.143:0/226996364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54181607c0 msgr2=0x7f541806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:11.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5417400640 1 --2- 172.21.15.143:0/226996364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54181607c0 0x7f541806b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5417400640 1 -- 172.21.15.143:0/226996364 >> v1:172.21.15.143:6789/0 conn(0x7f54180cfb30 legacy=0x7f541806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:11.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5417400640 1 -- 172.21.15.143:0/226996364 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54180cb470 con 0x7f54180cbf50 2024-01-15T18:13:11.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.440+0000 7f5415600640 1 -- 172.21.15.143:0/226996364 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14014125 0 0) 0x7f54180cffd0 con 0x7f54180cfb30 2024-01-15T18:13:11.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f5417400640 1 --2- 172.21.15.143:0/226996364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54180cbf50 0x7f54180cc390 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f541001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e557a5b8253984f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:11.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f5415600640 1 -- 172.21.15.143:0/226996364 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54100af600 con 0x7f54180cbf50 2024-01-15T18:13:11.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f5415600640 1 -- 172.21.15.143:0/226996364 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54100ab050 con 0x7f54180cbf50 2024-01-15T18:13:11.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f5415600640 1 -- 172.21.15.143:0/226996364 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54100af980 con 0x7f54180cbf50 2024-01-15T18:13:11.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 -- 172.21.15.143:0/226996364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54180cbf50 msgr2=0x7f54180cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:11.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 --2- 172.21.15.143:0/226996364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54180cbf50 0x7f54180cc390 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f541001f130 comp rx=0 tx=0).stop 2024-01-15T18:13:11.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 -- 172.21.15.143:0/226996364 shutdown_connections 2024-01-15T18:13:11.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 --2- 172.21.15.143:0/226996364 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54180cbf50 0x7f54180cc390 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 --2- 172.21.15.143:0/226996364 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54181607c0 0x7f541806b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 -- 172.21.15.143:0/226996364 >> 172.21.15.143:0/226996364 conn(0x7f54180d0e20 msgr2=0x7f54180d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:11.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 -- 172.21.15.143:0/226996364 shutdown_connections 2024-01-15T18:13:11.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.444+0000 7f541d600640 1 -- 172.21.15.143:0/226996364 wait complete. 2024-01-15T18:13:11.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 Processor -- start 2024-01-15T18:13:11.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- start start 2024-01-15T18:13:11.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54180cbf50 0x7f54180b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:11.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54180cfb30 0x7f54180b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 0x7f541814f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180b3620 con 0x7f54181607c0 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180b38c0 con 0x7f54180cfb30 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180b3ba0 con 0x7f54180cbf50 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 0x7f541814f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 0x7f541814f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38234/0 (socket says 172.21.15.143:38234) 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 -- 172.21.15.143:0/480368713 learned_addr learned my addr 172.21.15.143:0/480368713 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5416a00640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54180cbf50 0x7f54180b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5416000640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54180cfb30 0x7f54180b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 -- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54180cbf50 msgr2=0x7f54180b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54180cbf50 0x7f54180b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 -- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54180cfb30 msgr2=0x7f54180b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54180cfb30 0x7f54180b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 -- 172.21.15.143:0/480368713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410009a70 con 0x7f54181607c0 2024-01-15T18:13:11.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5416000640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54180cfb30 0x7f54180b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f5417400640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 0x7f541814f290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f54100b2040 tx=0x7f541001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410009450 con 0x7f54181607c0 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54100b9410 con 0x7f54181607c0 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54100b97c0 con 0x7f54181607c0 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54180c4920 con 0x7f54181607c0 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54180c4ee0 con 0x7f54181607c0 2024-01-15T18:13:11.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.448+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541806d4b0 con 0x7f54181607c0 2024-01-15T18:13:11.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.452+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f54100b6050 con 0x7f54181607c0 2024-01-15T18:13:11.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.452+0000 7f540ea00640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f53f00674c0 0x7f53f0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:11.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.452+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5410121500 con 0x7f54181607c0 2024-01-15T18:13:11.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.452+0000 7f5416a00640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f53f00674c0 0x7f53f0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:11.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.452+0000 7f5416a00640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f53f00674c0 0x7f53f0069980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5408009ac0 tx=0x7f5408009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:11.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.456+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54100eff30 con 0x7f54181607c0 2024-01-15T18:13:11.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:11.636+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99315", "target": ["mon-mgr", ""]}) v1 -- 0x7f54180ce6b0 con 0x7f53f00674c0 2024-01-15T18:13:28.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.960+0000 7f540ea00640 1 -- 172.21.15.143:0/480368713 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f54080119e0 con 0x7f53f00674c0 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f53f00674c0 msgr2=0x7f53f0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f53f00674c0 0x7f53f0069980 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5408009ac0 tx=0x7f5408009190 comp rx=0 tx=0).stop 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 msgr2=0x7f541814f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 0x7f541814f290 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f54100b2040 tx=0x7f541001bed0 comp rx=0 tx=0).stop 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 shutdown_connections 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f53f00674c0 0x7f53f0069980 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54181607c0 0x7f541814f290 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:28.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54180cfb30 0x7f54180b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 --2- 172.21.15.143:0/480368713 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54180cbf50 0x7f54180b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 >> 172.21.15.143:0/480368713 conn(0x7f54180d0e20 msgr2=0x7f541806f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 shutdown_connections 2024-01-15T18:13:28.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:28.964+0000 7f541d600640 1 -- 172.21.15.143:0/480368713 wait complete. 2024-01-15T18:13:28.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99316 2024-01-15T18:13:29.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:13:29.046 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:13:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.148+0000 7fad97400640 1 Processor -- start 2024-01-15T18:13:29.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.148+0000 7fad97400640 1 -- start start 2024-01-15T18:13:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad97400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fad90143780 0x7fad900cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad97400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad900ce520 0x7fad90071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad97400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad900ceb50 con 0x7fad900720f0 2024-01-15T18:13:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad97400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9015c110 con 0x7fad900ce520 2024-01-15T18:13:29.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad97400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad90072530 con 0x7fad90143780 2024-01-15T18:13:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad8fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad900ce520 0x7fad90071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad95400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fad900720f0 0x7fad9015a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52670/0 (socket says 172.21.15.143:52670) 2024-01-15T18:13:29.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad95400640 1 -- 172.21.15.143:0/265332699 learned_addr learned my addr 172.21.15.143:0/265332699 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:13:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad94a00640 1 --2- 172.21.15.143:0/265332699 >> v2:172.21.15.204:3300/0 conn(0x7fad90143780 0x7fad900cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad8fe00640 1 -- 172.21.15.143:0/265332699 >> v2:172.21.15.204:3300/0 conn(0x7fad90143780 msgr2=0x7fad900cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad8fe00640 1 --2- 172.21.15.143:0/265332699 >> v2:172.21.15.204:3300/0 conn(0x7fad90143780 0x7fad900cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad8fe00640 1 -- 172.21.15.143:0/265332699 >> v1:172.21.15.143:6789/0 conn(0x7fad900720f0 legacy=0x7fad9015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad8f400640 1 -- 172.21.15.143:0/265332699 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285272976 0 0) 0x7fad880014a0 con 0x7fad900720f0 2024-01-15T18:13:29.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.152+0000 7fad8fe00640 1 -- 172.21.15.143:0/265332699 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9015b7e0 con 0x7fad900ce520 2024-01-15T18:13:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad8fe00640 1 --2- 172.21.15.143:0/265332699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad900ce520 0x7fad90071bb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fad9006be30 tx=0x7fad7801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ea0e23b2e7048 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad8f400640 1 -- 172.21.15.143:0/265332699 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad780ab600 con 0x7fad900ce520 2024-01-15T18:13:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad8f400640 1 -- 172.21.15.143:0/265332699 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad780a7050 con 0x7fad900ce520 2024-01-15T18:13:29.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad8f400640 1 -- 172.21.15.143:0/265332699 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad780ab980 con 0x7fad900ce520 2024-01-15T18:13:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 -- 172.21.15.143:0/265332699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad900ce520 msgr2=0x7fad90071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 --2- 172.21.15.143:0/265332699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad900ce520 0x7fad90071bb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fad9006be30 tx=0x7fad7801c460 comp rx=0 tx=0).stop 2024-01-15T18:13:29.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 -- 172.21.15.143:0/265332699 shutdown_connections 2024-01-15T18:13:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 --2- 172.21.15.143:0/265332699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad900ce520 0x7fad90071bb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 --2- 172.21.15.143:0/265332699 >> v2:172.21.15.204:3300/0 conn(0x7fad90143780 0x7fad900cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 -- 172.21.15.143:0/265332699 >> 172.21.15.143:0/265332699 conn(0x7fad9006c3f0 msgr2=0x7fad9006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 -- 172.21.15.143:0/265332699 shutdown_connections 2024-01-15T18:13:29.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 -- 172.21.15.143:0/265332699 wait complete. 2024-01-15T18:13:29.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 Processor -- start 2024-01-15T18:13:29.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.156+0000 7fad97400640 1 -- start start 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad900720f0 0x7fad90139fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad900ce520 0x7fad90156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 0x7fad90158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad900b1ec0 con 0x7fad90143780 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9013a510 con 0x7fad900720f0 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad90139b40 con 0x7fad900ce520 2024-01-15T18:13:29.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 0x7fad90158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 0x7fad90158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57744/0 (socket says 172.21.15.143:57744) 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 -- 172.21.15.143:0/11944589 learned_addr learned my addr 172.21.15.143:0/11944589 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad94a00640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad900720f0 0x7fad90139fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad8fe00640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad900ce520 0x7fad90156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 -- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad900ce520 msgr2=0x7fad90156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad900ce520 0x7fad90156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 -- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad900720f0 msgr2=0x7fad90139fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad900720f0 0x7fad90139fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 -- 172.21.15.143:0/11944589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78009420 con 0x7fad90143780 2024-01-15T18:13:29.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad95400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 0x7fad90158850 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c730 tx=0x7fad8800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800ce40 con 0x7fad90143780 2024-01-15T18:13:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8800a5e0 con 0x7fad90143780 2024-01-15T18:13:29.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800a990 con 0x7fad90143780 2024-01-15T18:13:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad90139550 con 0x7fad90143780 2024-01-15T18:13:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad900b23b0 con 0x7fad90143780 2024-01-15T18:13:29.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.160+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad90139cb0 con 0x7fad90143780 2024-01-15T18:13:29.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.164+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 25) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fad8800aaf0 con 0x7fad90143780 2024-01-15T18:13:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.164+0000 7fad8cc00640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad680674c0 0x7fad68069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.164+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fad880825d0 con 0x7fad90143780 2024-01-15T18:13:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.164+0000 7fad94a00640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad680674c0 0x7fad68069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:29.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.164+0000 7fad94a00640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad680674c0 0x7fad68069980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fad800079c0 tx=0x7fad80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:29.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.168+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad880510f0 con 0x7fad90143780 2024-01-15T18:13:29.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:29.352+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99316", "target": ["mon-mgr", ""]}) v1 -- 0x7fad900ceb50 con 0x7fad680674c0 2024-01-15T18:13:29.967 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:29.964+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:29.967 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:29.964+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:29.967 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:29.964+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:29.967 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:29.964+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:29.967 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:13:29.964+0000 7f3cb0600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:31.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:31.752+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 26) v1 ==== 87282+0+0 (secure 0 0 0) 0x7fad880514c0 con 0x7fad90143780 2024-01-15T18:13:39.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.940+0000 7fad8cc00640 1 -- 172.21.15.143:0/11944589 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fad900ceb50 con 0x7fad680674c0 2024-01-15T18:13:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad680674c0 msgr2=0x7fad68069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad680674c0 0x7fad68069980 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fad800079c0 tx=0x7fad80008040 comp rx=0 tx=0).stop 2024-01-15T18:13:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 msgr2=0x7fad90158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 0x7fad90158850 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c730 tx=0x7fad8800cbf0 comp rx=0 tx=0).stop 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 shutdown_connections 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fad680674c0 0x7fad68069980 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad90143780 0x7fad90158850 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad900ce520 0x7fad90156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 --2- 172.21.15.143:0/11944589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad900720f0 0x7fad90139fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 >> 172.21.15.143:0/11944589 conn(0x7fad9006c3f0 msgr2=0x7fad900aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 shutdown_connections 2024-01-15T18:13:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:39.948+0000 7fad97400640 1 -- 172.21.15.143:0/11944589 wait complete. 2024-01-15T18:13:40.042 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99317 2024-01-15T18:13:40.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.460+0000 7f3d55000640 1 Processor -- start 2024-01-15T18:13:40.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.460+0000 7f3d55000640 1 -- start start 2024-01-15T18:13:40.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d55000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3d500ce2d0 0x7f3d5015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:40.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d55000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d500ce780 con 0x7f3d50071140 2024-01-15T18:13:40.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d55000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d500cb980 con 0x7f3d5015b110 2024-01-15T18:13:40.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d55000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d500cc220 con 0x7f3d500ce2d0 2024-01-15T18:13:40.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4e800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f3d5015b110 0x7f3d5015da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:40658/0 (socket says 172.21.15.143:40658) 2024-01-15T18:13:40.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4e800640 1 -- 172.21.15.143:0/4097472198 learned_addr learned my addr 172.21.15.143:0/4097472198 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:13:40.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4de00640 1 --1- 172.21.15.143:0/4097472198 >> v1:172.21.15.143:6789/0 conn(0x7f3d50071140 0x7f3d50071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36872/0 (socket says 172.21.15.143:36872) 2024-01-15T18:13:40.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4d400640 1 --2- 172.21.15.143:0/4097472198 >> v2:172.21.15.143:3300/0 conn(0x7f3d500ce2d0 0x7f3d5015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:40.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4ca00640 1 -- 172.21.15.143:0/4097472198 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630412255 0 0) 0x7f3d500cb980 con 0x7f3d5015b110 2024-01-15T18:13:40.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4ca00640 1 -- 172.21.15.143:0/4097472198 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d30003530 con 0x7f3d5015b110 2024-01-15T18:13:40.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4d400640 1 -- 172.21.15.143:0/4097472198 >> v1:172.21.15.204:6789/0 conn(0x7f3d5015b110 legacy=0x7f3d5015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:40.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4ca00640 1 -- 172.21.15.143:0/4097472198 <== mon.1 v1:172.21.15.204:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1204636947 0 0) 0x7f3d30003530 con 0x7f3d5015b110 2024-01-15T18:13:40.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4d400640 1 -- 172.21.15.143:0/4097472198 >> v1:172.21.15.143:6789/0 conn(0x7f3d50071140 legacy=0x7f3d50071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:40.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.464+0000 7f3d4d400640 1 -- 172.21.15.143:0/4097472198 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d500cbc60 con 0x7f3d500ce2d0 2024-01-15T18:13:40.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.472+0000 7f3d4d400640 1 --2- 172.21.15.143:0/4097472198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d500ce2d0 0x7f3d5015abd0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009740 tx=0x7f3d3801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b3a7d39ed275d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.476+0000 7f3d4ca00640 1 -- 172.21.15.143:0/4097472198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48001930 con 0x7f3d500ce2d0 2024-01-15T18:13:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.476+0000 7f3d4ca00640 1 -- 172.21.15.143:0/4097472198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d380a7050 con 0x7f3d500ce2d0 2024-01-15T18:13:40.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.476+0000 7f3d4ca00640 1 -- 172.21.15.143:0/4097472198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d380ab620 con 0x7f3d500ce2d0 2024-01-15T18:13:40.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.480+0000 7f3d55000640 1 -- 172.21.15.143:0/4097472198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d500ce2d0 msgr2=0x7f3d5015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:40.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.480+0000 7f3d55000640 1 --2- 172.21.15.143:0/4097472198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d500ce2d0 0x7f3d5015abd0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3d38009740 tx=0x7f3d3801c340 comp rx=0 tx=0).stop 2024-01-15T18:13:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.480+0000 7f3d55000640 1 -- 172.21.15.143:0/4097472198 shutdown_connections 2024-01-15T18:13:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.480+0000 7f3d55000640 1 --2- 172.21.15.143:0/4097472198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d500ce2d0 0x7f3d5015abd0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:40.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.480+0000 7f3d55000640 1 -- 172.21.15.143:0/4097472198 >> 172.21.15.143:0/4097472198 conn(0x7f3d500cfd20 msgr2=0x7f3d500d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:40.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.484+0000 7f3d55000640 1 -- 172.21.15.143:0/4097472198 shutdown_connections 2024-01-15T18:13:40.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.484+0000 7f3d55000640 1 -- 172.21.15.143:0/4097472198 wait complete. 2024-01-15T18:13:40.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.488+0000 7f3d55000640 1 Processor -- start 2024-01-15T18:13:40.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 -- start start 2024-01-15T18:13:40.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 0x7f3d50205c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d500ce2d0 0x7f3d500c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d5015b110 0x7f3d500c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500c8310 con 0x7f3d50071140 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500c9170 con 0x7f3d5015b110 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.492+0000 7f3d55000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d500c3150 con 0x7f3d500ce2d0 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 0x7f3d50205c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 0x7f3d50205c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45960/0 (socket says 172.21.15.143:45960) 2024-01-15T18:13:40.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 -- 172.21.15.143:0/141183584 learned_addr learned my addr 172.21.15.143:0/141183584 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:40.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4e800640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d5015b110 0x7f3d500c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4d400640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d500ce2d0 0x7f3d500c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 -- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d500ce2d0 msgr2=0x7f3d500c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d500ce2d0 0x7f3d500c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 -- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d5015b110 msgr2=0x7f3d500c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d5015b110 0x7f3d500c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 -- 172.21.15.143:0/141183584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d38009420 con 0x7f3d50071140 2024-01-15T18:13:40.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4e800640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d5015b110 0x7f3d500c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:13:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d4de00640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 0x7f3d50205c20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000c3a0 tx=0x7f3d4000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d400074e0 con 0x7f3d50071140 2024-01-15T18:13:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d500c8560 con 0x7f3d50071140 2024-01-15T18:13:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d500c8b20 con 0x7f3d50071140 2024-01-15T18:13:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d40007640 con 0x7f3d50071140 2024-01-15T18:13:40.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.496+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d40004a30 con 0x7f3d50071140 2024-01-15T18:13:40.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.500+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d20005260 con 0x7f3d50071140 2024-01-15T18:13:40.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.500+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f3d40004b90 con 0x7f3d50071140 2024-01-15T18:13:40.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.504+0000 7f3d46000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d28067550 0x7f3d28069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:40.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.504+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3d40082a80 con 0x7f3d50071140 2024-01-15T18:13:40.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.512+0000 7f3d4d400640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d28067550 0x7f3d28069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:40.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.516+0000 7f3d4d400640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d28067550 0x7f3d28069a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3d380ad000 tx=0x7f3d380a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:40.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:40.532+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3d40051390 con 0x7f3d50071140 2024-01-15T18:13:41.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.008+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99317", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d20000bd0 con 0x7f3d28067550 2024-01-15T18:13:41.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.308+0000 7f3d46000640 1 -- 172.21.15.143:0/141183584 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d20000bd0 con 0x7f3d28067550 2024-01-15T18:13:41.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d28067550 msgr2=0x7f3d28069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d28067550 0x7f3d28069a10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3d380ad000 tx=0x7f3d380a6000 comp rx=0 tx=0).stop 2024-01-15T18:13:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 msgr2=0x7f3d50205c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 0x7f3d50205c20 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3d4000c3a0 tx=0x7f3d4000c860 comp rx=0 tx=0).stop 2024-01-15T18:13:41.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 shutdown_connections 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3d28067550 0x7f3d28069a10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d5015b110 0x7f3d500c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3d500ce2d0 0x7f3d500c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 --2- 172.21.15.143:0/141183584 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d50071140 0x7f3d50205c20 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 >> 172.21.15.143:0/141183584 conn(0x7f3d500cfd20 msgr2=0x7f3d5006e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 shutdown_connections 2024-01-15T18:13:41.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.312+0000 7f3d55000640 1 -- 172.21.15.143:0/141183584 wait complete. 2024-01-15T18:13:41.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99318 2024-01-15T18:13:41.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.612+0000 7f0562000640 1 Processor -- start 2024-01-15T18:13:41.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.612+0000 7f0562000640 1 -- start start 2024-01-15T18:13:41.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.612+0000 7f0562000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f055c144a50 0x7f055c06a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:41.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.612+0000 7f0562000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f055c0ae320 0x7f055c06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f0562000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c072010 con 0x7f055c06b0e0 2024-01-15T18:13:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f0562000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c06fbe0 con 0x7f055c144a50 2024-01-15T18:13:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f0562000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c071400 con 0x7f055c0ae320 2024-01-15T18:13:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055aa00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f055c06b0e0 0x7f055c15eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:40674/0 (socket says 172.21.15.143:40674) 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f055c0ae320 0x7f055c06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055aa00640 1 -- 172.21.15.143:0/2488115328 learned_addr learned my addr 172.21.15.143:0/2488115328 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055b400640 1 --2- 172.21.15.143:0/2488115328 >> v2:172.21.15.143:3300/0 conn(0x7f055c144a50 0x7f055c06a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055be00640 1 -- 172.21.15.143:0/2488115328 >> v2:172.21.15.143:3300/0 conn(0x7f055c144a50 msgr2=0x7f055c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055be00640 1 --2- 172.21.15.143:0/2488115328 >> v2:172.21.15.143:3300/0 conn(0x7f055c144a50 0x7f055c06a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055be00640 1 -- 172.21.15.143:0/2488115328 >> v1:172.21.15.204:6789/0 conn(0x7f055c06b0e0 legacy=0x7f055c15eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.616+0000 7f055be00640 1 -- 172.21.15.143:0/2488115328 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0716e0 con 0x7f055c0ae320 2024-01-15T18:13:41.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.620+0000 7f055be00640 1 --2- 172.21.15.143:0/2488115328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 0x7f055c06a9d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f055c0b2cf0 tx=0x7f055401c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef05e31f1b0386f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:41.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.620+0000 7f055a000640 1 -- 172.21.15.143:0/2488115328 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05540a5800 con 0x7f055c0ae320 2024-01-15T18:13:41.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.620+0000 7f055a000640 1 -- 172.21.15.143:0/2488115328 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05540a5960 con 0x7f055c0ae320 2024-01-15T18:13:41.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.620+0000 7f055a000640 1 -- 172.21.15.143:0/2488115328 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05540a5cf0 con 0x7f055c0ae320 2024-01-15T18:13:41.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 -- 172.21.15.143:0/2488115328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 msgr2=0x7f055c06a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:41.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 --2- 172.21.15.143:0/2488115328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 0x7f055c06a9d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f055c0b2cf0 tx=0x7f055401c7a0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 -- 172.21.15.143:0/2488115328 shutdown_connections 2024-01-15T18:13:41.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 --2- 172.21.15.143:0/2488115328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 0x7f055c06a9d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 --2- 172.21.15.143:0/2488115328 >> v2:172.21.15.143:3300/0 conn(0x7f055c144a50 0x7f055c06a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 -- 172.21.15.143:0/2488115328 >> 172.21.15.143:0/2488115328 conn(0x7f055c0b1960 msgr2=0x7f055c0b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:41.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 -- 172.21.15.143:0/2488115328 shutdown_connections 2024-01-15T18:13:41.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.624+0000 7f0562000640 1 -- 172.21.15.143:0/2488115328 wait complete. 2024-01-15T18:13:41.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 Processor -- start 2024-01-15T18:13:41.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 -- start start 2024-01-15T18:13:41.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 0x7f055c1f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:41.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 0x7f055c1fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:41.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f055c144a50 0x7f055c137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:41.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c138a40 con 0x7f055c06b0e0 2024-01-15T18:13:41.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c13b9c0 con 0x7f055c0ae320 2024-01-15T18:13:41.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.628+0000 7f0562000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c13c540 con 0x7f055c144a50 2024-01-15T18:13:41.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 0x7f055c1f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:41.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 0x7f055c1f7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:45980/0 (socket says 172.21.15.143:45980) 2024-01-15T18:13:41.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 -- 172.21.15.143:0/1107645348 learned_addr learned my addr 172.21.15.143:0/1107645348 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:41.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 -- 172.21.15.143:0/1107645348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f055c144a50 msgr2=0x7f055c137f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T18:13:41.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f055c144a50 0x7f055c137f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 -- 172.21.15.143:0/1107645348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 msgr2=0x7f055c1fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:41.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 0x7f055c1fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:41.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 -- 172.21.15.143:0/1107645348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0554009a70 con 0x7f055c06b0e0 2024-01-15T18:13:41.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f055b400640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 0x7f055c1f7520 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e930 tx=0x7f054c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:41.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c009690 con 0x7f055c06b0e0 2024-01-15T18:13:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c1f7a60 con 0x7f055c06b0e0 2024-01-15T18:13:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.632+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c13bef0 con 0x7f055c06b0e0 2024-01-15T18:13:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.636+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055c067c10 con 0x7f055c06b0e0 2024-01-15T18:13:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.636+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c0059a0 con 0x7f055c06b0e0 2024-01-15T18:13:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.636+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c005d10 con 0x7f055c06b0e0 2024-01-15T18:13:41.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.640+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f054c00c980 con 0x7f055c06b0e0 2024-01-15T18:13:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.640+0000 7f0553400640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0534067550 0x7f0534069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:41.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.640+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f054c0832c0 con 0x7f055c06b0e0 2024-01-15T18:13:41.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.648+0000 7f055aa00640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0534067550 0x7f0534069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:41.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.648+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f054c051d70 con 0x7f055c06b0e0 2024-01-15T18:13:41.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.656+0000 7f055aa00640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0534067550 0x7f0534069a10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0544001290 tx=0x7f0544001be0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:41.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:41.884+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99318", "target": ["mon-mgr", ""]}) v1 -- 0x7f055c071ca0 con 0x7f0534067550 2024-01-15T18:13:52.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.104+0000 7f0553400640 1 -- 172.21.15.143:0/1107645348 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f054400eeb0 con 0x7f0534067550 2024-01-15T18:13:52.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0534067550 msgr2=0x7f0534069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0534067550 0x7f0534069a10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0544001290 tx=0x7f0544001be0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 msgr2=0x7f055c1f7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 0x7f055c1f7520 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f054c00e930 tx=0x7f054c00edf0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 shutdown_connections 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0534067550 0x7f0534069a10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f055c144a50 0x7f055c137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f055c0ae320 0x7f055c1fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 --2- 172.21.15.143:0/1107645348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f055c06b0e0 0x7f055c1f7520 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 >> 172.21.15.143:0/1107645348 conn(0x7f055c0b1960 msgr2=0x7f055c15dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 shutdown_connections 2024-01-15T18:13:52.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.108+0000 7f0562000640 1 -- 172.21.15.143:0/1107645348 wait complete. 2024-01-15T18:13:52.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone99319 2024-01-15T18:13:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f824c000640 1 Processor -- start 2024-01-15T18:13:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f824c000640 1 -- start start 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f824c000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f82440cdf20 0x7f82440cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f824c000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440c4d90 con 0x7f824415b260 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f824c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824415b640 con 0x7f82440c6230 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f824c000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440cf800 con 0x7f82440cdf20 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8249600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f824415b260 0x7f82440cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48448/0 (socket says 172.21.15.143:48448) 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8249600640 1 -- 172.21.15.143:0/2496176622 learned_addr learned my addr 172.21.15.143:0/2496176622 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:13:52.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8248c00640 1 --2- 172.21.15.143:0/2496176622 >> v2:172.21.15.204:3300/0 conn(0x7f82440cdf20 0x7f82440cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:52.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343307903 0 0) 0x7f82440c4d90 con 0x7f824415b260 2024-01-15T18:13:52.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8224003530 con 0x7f824415b260 2024-01-15T18:13:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785417276 0 0) 0x7f824415b640 con 0x7f82440c6230 2024-01-15T18:13:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82440c4d90 con 0x7f82440c6230 2024-01-15T18:13:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2490794820 0 0) 0x7f8224003530 con 0x7f824415b260 2024-01-15T18:13:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8248c00640 1 -- 172.21.15.143:0/2496176622 >> v1:172.21.15.204:6789/0 conn(0x7f82440c6230 legacy=0x7f82440cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8248c00640 1 -- 172.21.15.143:0/2496176622 >> v1:172.21.15.143:6789/0 conn(0x7f824415b260 legacy=0x7f82440cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.392+0000 7f8248c00640 1 -- 172.21.15.143:0/2496176622 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440cfae0 con 0x7f82440cdf20 2024-01-15T18:13:52.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f8248c00640 1 --2- 172.21.15.143:0/2496176622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f82440cbae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097c0 tx=0x7f822c01c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1ad2b89195907c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:52.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234001680 con 0x7f82440cdf20 2024-01-15T18:13:52.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c0a7050 con 0x7f82440cdf20 2024-01-15T18:13:52.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f8243e00640 1 -- 172.21.15.143:0/2496176622 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c0ab620 con 0x7f82440cdf20 2024-01-15T18:13:52.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f824c000640 1 -- 172.21.15.143:0/2496176622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 msgr2=0x7f82440cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f824c000640 1 --2- 172.21.15.143:0/2496176622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f82440cbae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f822c0097c0 tx=0x7f822c01c370 comp rx=0 tx=0).stop 2024-01-15T18:13:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f824c000640 1 -- 172.21.15.143:0/2496176622 shutdown_connections 2024-01-15T18:13:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f824c000640 1 --2- 172.21.15.143:0/2496176622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f82440cbae0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.396+0000 7f824c000640 1 -- 172.21.15.143:0/2496176622 >> 172.21.15.143:0/2496176622 conn(0x7f82440d0ed0 msgr2=0x7f82440d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 -- 172.21.15.143:0/2496176622 shutdown_connections 2024-01-15T18:13:52.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 -- 172.21.15.143:0/2496176622 wait complete. 2024-01-15T18:13:52.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 Processor -- start 2024-01-15T18:13:52.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 -- start start 2024-01-15T18:13:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 0x7f82441580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f824415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f824415b260 0x7f8244140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824413f890 con 0x7f82440c6230 2024-01-15T18:13:52.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824413fb70 con 0x7f82440cdf20 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824413fe50 con 0x7f824415b260 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 0x7f82441580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 0x7f82441580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46946/0 (socket says 172.21.15.143:46946) 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 -- 172.21.15.143:0/929393153 learned_addr learned my addr 172.21.15.143:0/929393153 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8248c00640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f824415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f824a000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f824415b260 0x7f8244140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 -- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f824415b260 msgr2=0x7f8244140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f824415b260 0x7f8244140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 -- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 msgr2=0x7f824415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f824415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 -- 172.21.15.143:0/929393153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c009420 con 0x7f82440c6230 2024-01-15T18:13:52.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.400+0000 7f8249600640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 0x7f82441580f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f823400c780 tx=0x7f823400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234005cf0 con 0x7f82440c6230 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8244140130 con 0x7f82440c6230 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8234005e50 con 0x7f82440c6230 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340072b0 con 0x7f82440c6230 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8244158630 con 0x7f82440c6230 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8244067500 con 0x7f82440c6230 2024-01-15T18:13:52.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f8234007410 con 0x7f82440c6230 2024-01-15T18:13:52.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8241600640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f821c067550 0x7f821c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:52.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8234082800 con 0x7f82440c6230 2024-01-15T18:13:52.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8248c00640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f821c067550 0x7f821c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:52.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.404+0000 7f8248c00640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f821c067550 0x7f821c069a10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f822c0ad000 tx=0x7f822c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:52.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.408+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82340512a0 con 0x7f82440c6230 2024-01-15T18:13:52.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:13:52.588+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone99319", "target": ["mon-mgr", ""]}) v1 -- 0x7f824413fce0 con 0x7f821c067550 2024-01-15T18:13:59.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:13:59.118 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:03.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.436+0000 7f8241600640 1 -- 172.21.15.143:0/929393153 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f824413fce0 con 0x7f821c067550 2024-01-15T18:14:03.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f821c067550 msgr2=0x7f821c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f821c067550 0x7f821c069a10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f822c0ad000 tx=0x7f822c008040 comp rx=0 tx=0).stop 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 msgr2=0x7f82441580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 0x7f82441580f0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f823400c780 tx=0x7f823400cc40 comp rx=0 tx=0).stop 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 shutdown_connections 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f821c067550 0x7f821c069a10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f824415b260 0x7f8244140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82440cdf20 0x7f824415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 --2- 172.21.15.143:0/929393153 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82440c6230 0x7f82441580f0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.444+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 >> 172.21.15.143:0/929393153 conn(0x7f82440d0ed0 msgr2=0x7f82440cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:03.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.448+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 shutdown_connections 2024-01-15T18:14:03.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.448+0000 7f824c000640 1 -- 172.21.15.143:0/929393153 wait complete. 2024-01-15T18:14:03.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993110 2024-01-15T18:14:03.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 Processor -- start 2024-01-15T18:14:03.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 -- start start 2024-01-15T18:14:03.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f32b414b1e0 0x7f32b40cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:03.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b40708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b40cfc20 con 0x7f32b4070e00 2024-01-15T18:14:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b4162080 con 0x7f32b414b1e0 2024-01-15T18:14:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.944+0000 7f32bba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4161d70 con 0x7f32b40cf7c0 2024-01-15T18:14:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b9a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f32b4070e00 0x7f32b40b1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:35736/0 (socket says 172.21.15.143:35736) 2024-01-15T18:14:03.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b9a00640 1 -- 172.21.15.143:0/976726206 learned_addr learned my addr 172.21.15.143:0/976726206 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:14:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b3e00640 1 --2- 172.21.15.143:0/976726206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b40708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b9000640 1 --2- 172.21.15.143:0/976726206 >> v2:172.21.15.143:3300/0 conn(0x7f32b414b1e0 0x7f32b40cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b3e00640 1 -- 172.21.15.143:0/976726206 >> v2:172.21.15.143:3300/0 conn(0x7f32b414b1e0 msgr2=0x7f32b40cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b3e00640 1 --2- 172.21.15.143:0/976726206 >> v2:172.21.15.143:3300/0 conn(0x7f32b414b1e0 0x7f32b40cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b3e00640 1 -- 172.21.15.143:0/976726206 >> v1:172.21.15.204:6789/0 conn(0x7f32b4070e00 legacy=0x7f32b40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:03.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.948+0000 7f32b3e00640 1 -- 172.21.15.143:0/976726206 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b414b5c0 con 0x7f32b40cf7c0 2024-01-15T18:14:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32b3e00640 1 --2- 172.21.15.143:0/976726206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b40708c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f32a40097c0 tx=0x7f32a401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8970542235881059 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32b3400640 1 -- 172.21.15.143:0/976726206 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a40ab650 con 0x7f32b40cf7c0 2024-01-15T18:14:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32b3400640 1 -- 172.21.15.143:0/976726206 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a40ab7b0 con 0x7f32b40cf7c0 2024-01-15T18:14:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32b3400640 1 -- 172.21.15.143:0/976726206 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a40abb40 con 0x7f32b40cf7c0 2024-01-15T18:14:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 -- 172.21.15.143:0/976726206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 msgr2=0x7f32b40708c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:03.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 --2- 172.21.15.143:0/976726206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b40708c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f32a40097c0 tx=0x7f32a401c450 comp rx=0 tx=0).stop 2024-01-15T18:14:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 -- 172.21.15.143:0/976726206 shutdown_connections 2024-01-15T18:14:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 --2- 172.21.15.143:0/976726206 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b40708c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 --2- 172.21.15.143:0/976726206 >> v2:172.21.15.143:3300/0 conn(0x7f32b414b1e0 0x7f32b40cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 -- 172.21.15.143:0/976726206 >> 172.21.15.143:0/976726206 conn(0x7f32b406a890 msgr2=0x7f32b406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:03.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 -- 172.21.15.143:0/976726206 shutdown_connections 2024-01-15T18:14:03.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.952+0000 7f32bba00640 1 -- 172.21.15.143:0/976726206 wait complete. 2024-01-15T18:14:03.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 Processor -- start 2024-01-15T18:14:03.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 -- start start 2024-01-15T18:14:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b4070e00 0x7f32b4143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 0x7f32b40b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4142bb0 con 0x7f32b414b1e0 2024-01-15T18:14:03.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4144480 con 0x7f32b4070e00 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32bba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40ad040 con 0x7f32b40cf7c0 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 0x7f32b40b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 0x7f32b40b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47502/0 (socket says 172.21.15.143:47502) 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 -- 172.21.15.143:0/2152826354 learned_addr learned my addr 172.21.15.143:0/2152826354 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b3e00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 -- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 msgr2=0x7f32b41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b41432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 -- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b4070e00 msgr2=0x7f32b4143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b4070e00 0x7f32b4143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:03.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.956+0000 7f32b9a00640 1 -- 172.21.15.143:0/2152826354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4009420 con 0x7f32b414b1e0 2024-01-15T18:14:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32b9a00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 0x7f32b40b0410 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00cbc0 tx=0x7f32ac00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00ad40 con 0x7f32b414b1e0 2024-01-15T18:14:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac012400 con 0x7f32b414b1e0 2024-01-15T18:14:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00aea0 con 0x7f32b414b1e0 2024-01-15T18:14:03.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b4144690 con 0x7f32b414b1e0 2024-01-15T18:14:03.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b4144c50 con 0x7f32b414b1e0 2024-01-15T18:14:03.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.960+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4067680 con 0x7f32b414b1e0 2024-01-15T18:14:03.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.964+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f32ac007070 con 0x7f32b414b1e0 2024-01-15T18:14:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.964+0000 7f32b0c00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f328c0675a0 0x7f328c069a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.964+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f32ac082fb0 con 0x7f32b414b1e0 2024-01-15T18:14:03.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.964+0000 7f32b9000640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f328c0675a0 0x7f328c069a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:03.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.968+0000 7f32b9000640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f328c0675a0 0x7f328c069a60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f32b4143e70 tx=0x7f329c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:03.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:03.968+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f32ac051a70 con 0x7f32b414b1e0 2024-01-15T18:14:04.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:04.332+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993110", "target": ["mon-mgr", ""]}) v1 -- 0x7f32b406e130 con 0x7f328c0675a0 2024-01-15T18:14:09.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:09.756+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f32ac051e40 con 0x7f32b414b1e0 2024-01-15T18:14:14.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.347+0000 7f32b0c00640 1 -- 172.21.15.143:0/2152826354 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32b406e130 con 0x7f328c0675a0 2024-01-15T18:14:14.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f328c0675a0 msgr2=0x7f328c069a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:14.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f328c0675a0 0x7f328c069a60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f32b4143e70 tx=0x7f329c008040 comp rx=0 tx=0).stop 2024-01-15T18:14:14.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 msgr2=0x7f32b40b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:14.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 0x7f32b40b0410 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00cbc0 tx=0x7f32ac00a430 comp rx=0 tx=0).stop 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 shutdown_connections 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f328c0675a0 0x7f328c069a60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f32b414b1e0 0x7f32b40b0410 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f32b40cf7c0 0x7f32b41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 --2- 172.21.15.143:0/2152826354 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32b4070e00 0x7f32b4143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 >> 172.21.15.143:0/2152826354 conn(0x7f32b406a890 msgr2=0x7f32b40a8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 shutdown_connections 2024-01-15T18:14:14.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.351+0000 7f32bba00640 1 -- 172.21.15.143:0/2152826354 wait complete. 2024-01-15T18:14:14.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993111 2024-01-15T18:14:14.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.703+0000 7f62a1600640 1 Processor -- start 2024-01-15T18:14:14.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.703+0000 7f62a1600640 1 -- start start 2024-01-15T18:14:14.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f62a1600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f629c0cb4e0 0x7f629c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:14.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f62a1600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f629c0adad0 0x7f629c0adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:14.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f62a1600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c0cd2b0 con 0x7f629c0c3df0 2024-01-15T18:14:14.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f62a1600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c0cc210 con 0x7f629c0adad0 2024-01-15T18:14:14.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f62a1600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c0cc4f0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f629c0cb4e0 0x7f629c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:14.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f629c0cb4e0 0x7f629c0cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:43284/0 (socket says 172.21.15.143:43284) 2024-01-15T18:14:14.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 -- 172.21.15.143:0/2267707619 learned_addr learned my addr 172.21.15.143:0/2267707619 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 -- 172.21.15.143:0/2267707619 >> v2:172.21.15.143:3300/0 conn(0x7f629c0adad0 msgr2=0x7f629c0adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 --2- 172.21.15.143:0/2267707619 >> v2:172.21.15.143:3300/0 conn(0x7f629c0adad0 0x7f629c0adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 -- 172.21.15.143:0/2267707619 >> v1:172.21.15.143:6789/0 conn(0x7f629c0c3df0 legacy=0x7f629c0c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:14.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.707+0000 7f629b400640 1 -- 172.21.15.143:0/2267707619 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0cc7d0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.719+0000 7f629b400640 1 --2- 172.21.15.143:0/2267707619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0cb4e0 0x7f629c0cb960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f629c06d780 tx=0x7f628401c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f17679e96ebd7a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:14.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.719+0000 7f629aa00640 1 -- 172.21.15.143:0/2267707619 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840aa5f0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.719+0000 7f629aa00640 1 -- 172.21.15.143:0/2267707619 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62840aa750 con 0x7f629c0cb4e0 2024-01-15T18:14:14.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.719+0000 7f629aa00640 1 -- 172.21.15.143:0/2267707619 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840aaae0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 -- 172.21.15.143:0/2267707619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0cb4e0 msgr2=0x7f629c0cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:14.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 --2- 172.21.15.143:0/2267707619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0cb4e0 0x7f629c0cb960 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f629c06d780 tx=0x7f628401c4b0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 -- 172.21.15.143:0/2267707619 shutdown_connections 2024-01-15T18:14:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 --2- 172.21.15.143:0/2267707619 >> v2:172.21.15.143:3300/0 conn(0x7f629c0adad0 0x7f629c0adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 --2- 172.21.15.143:0/2267707619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0cb4e0 0x7f629c0cb960 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 -- 172.21.15.143:0/2267707619 >> 172.21.15.143:0/2267707619 conn(0x7f629c0d0e20 msgr2=0x7f629c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.723+0000 7f62a1600640 1 -- 172.21.15.143:0/2267707619 shutdown_connections 2024-01-15T18:14:14.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.727+0000 7f62a1600640 1 -- 172.21.15.143:0/2267707619 wait complete. 2024-01-15T18:14:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 Processor -- start 2024-01-15T18:14:14.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 -- start start 2024-01-15T18:14:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0adad0 0x7f629c15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0c3df0 0x7f629c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 0x7f629c0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0b1560 con 0x7f629c0adad0 2024-01-15T18:14:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0b1280 con 0x7f629c0c3df0 2024-01-15T18:14:14.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0c4ac0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.731+0000 7f62a0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 0x7f629c0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:14.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 0x7f629c0c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:38974/0 (socket says 172.21.15.143:38974) 2024-01-15T18:14:14.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 -- 172.21.15.143:0/4240027500 learned_addr learned my addr 172.21.15.143:0/4240027500 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:14.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f629b400640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0c3df0 0x7f629c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 -- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0c3df0 msgr2=0x7f629c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0c3df0 0x7f629c0b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 -- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0adad0 msgr2=0x7f629c15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0adad0 0x7f629c15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:14.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 -- 172.21.15.143:0/4240027500 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62840094d0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.735+0000 7f62a0c00640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 0x7f629c0c55a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f628c00b970 tx=0x7f628c00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:14.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.739+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c00cb40 con 0x7f629c0cb4e0 2024-01-15T18:14:14.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.739+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c00ccc0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.739+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c014680 con 0x7f629c0cb4e0 2024-01-15T18:14:14.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.739+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c0c4da0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.739+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c15a290 con 0x7f629c0cb4e0 2024-01-15T18:14:14.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.743+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c0c4920 con 0x7f629c0cb4e0 2024-01-15T18:14:14.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.747+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f628c0147e0 con 0x7f629c0cb4e0 2024-01-15T18:14:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.747+0000 7f6293e00640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6274067510 0x7f62740699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:14.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.747+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f628c082e60 con 0x7f629c0cb4e0 2024-01-15T18:14:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.747+0000 7f629be00640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6274067510 0x7f62740699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.751+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f628c055360 con 0x7f629c0cb4e0 2024-01-15T18:14:14.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:14.751+0000 7f629be00640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6274067510 0x7f62740699d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6294007fd0 tx=0x7f62940079a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:15.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.123+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993111", "target": ["mon-mgr", ""]}) v1 -- 0x7f629c06c7c0 con 0x7f6274067510 2024-01-15T18:14:15.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.623+0000 7f6293e00640 1 -- 172.21.15.143:0/4240027500 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f629c06c7c0 con 0x7f6274067510 2024-01-15T18:14:15.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6274067510 msgr2=0x7f62740699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:15.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6274067510 0x7f62740699d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6294007fd0 tx=0x7f62940079a0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 msgr2=0x7f629c0c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:15.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 0x7f629c0c55a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f628c00b970 tx=0x7f628c00be30 comp rx=0 tx=0).stop 2024-01-15T18:14:15.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 shutdown_connections 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6274067510 0x7f62740699d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f629c0cb4e0 0x7f629c0c55a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f629c0c3df0 0x7f629c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 --2- 172.21.15.143:0/4240027500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c0adad0 0x7f629c15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 >> 172.21.15.143:0/4240027500 conn(0x7f629c0d0e20 msgr2=0x7f629c06ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 shutdown_connections 2024-01-15T18:14:15.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.627+0000 7f62a1600640 1 -- 172.21.15.143:0/4240027500 wait complete. 2024-01-15T18:14:15.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993112 2024-01-15T18:14:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 Processor -- start 2024-01-15T18:14:15.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 -- start start 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a7c15b260 0x7f8a7c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c0d19f0 0x7f8a7c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7c0c4d90 con 0x7f8a7c0c5220 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7c15b640 con 0x7f8a7c15b260 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a83800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c0cf800 con 0x7f8a7c0d19f0 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a80e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a7c15b260 0x7f8a7c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a80e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a7c15b260 0x7f8a7c0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59286/0 (socket says 172.21.15.143:59286) 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a80e00640 1 -- 172.21.15.143:0/1584822373 learned_addr learned my addr 172.21.15.143:0/1584822373 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.915+0000 7f8a7be00640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c0d19f0 0x7f8a7c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a80e00640 1 -- 172.21.15.143:0/1584822373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c0d19f0 msgr2=0x7f8a7c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a80e00640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c0d19f0 0x7f8a7c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a80e00640 1 -- 172.21.15.143:0/1584822373 >> v1:172.21.15.204:6789/0 conn(0x7f8a7c0c5220 legacy=0x7f8a7c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:15.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a80e00640 1 -- 172.21.15.143:0/1584822373 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c0cfae0 con 0x7f8a7c15b260 2024-01-15T18:14:15.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a7be00640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c0d19f0 0x7f8a7c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:14:15.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a80e00640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c15b260 0x7f8a7c0cd810 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009870 tx=0x7f8a6c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3d48f5040d2993 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:15.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a7b400640 1 -- 172.21.15.143:0/1584822373 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0ab620 con 0x7f8a7c15b260 2024-01-15T18:14:15.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a7b400640 1 -- 172.21.15.143:0/1584822373 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c0ab780 con 0x7f8a7c15b260 2024-01-15T18:14:15.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a7b400640 1 -- 172.21.15.143:0/1584822373 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0abb10 con 0x7f8a7c15b260 2024-01-15T18:14:15.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a83800640 1 -- 172.21.15.143:0/1584822373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c15b260 msgr2=0x7f8a7c0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:15.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.919+0000 7f8a83800640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c15b260 0x7f8a7c0cd810 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009870 tx=0x7f8a6c01c510 comp rx=0 tx=0).stop 2024-01-15T18:14:15.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- 172.21.15.143:0/1584822373 shutdown_connections 2024-01-15T18:14:15.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c0d19f0 0x7f8a7c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 --2- 172.21.15.143:0/1584822373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c15b260 0x7f8a7c0cd810 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- 172.21.15.143:0/1584822373 >> 172.21.15.143:0/1584822373 conn(0x7f8a7c0d0ed0 msgr2=0x7f8a7c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:15.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- 172.21.15.143:0/1584822373 shutdown_connections 2024-01-15T18:14:15.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- 172.21.15.143:0/1584822373 wait complete. 2024-01-15T18:14:15.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 Processor -- start 2024-01-15T18:14:15.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- start start 2024-01-15T18:14:15.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 0x7f8a7c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c0d19f0 0x7f8a7c1513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c15b260 0x7f8a7c151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c1568e0 con 0x7f8a7c0d19f0 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c158230 con 0x7f8a7c0c5220 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a83800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7c156600 con 0x7f8a7c15b260 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 0x7f8a7c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 0x7f8a7c157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:43302/0 (socket says 172.21.15.143:43302) 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 -- 172.21.15.143:0/1037932265 learned_addr learned my addr 172.21.15.143:0/1037932265 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a7be00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c0d19f0 0x7f8a7c1513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a81800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c15b260 0x7f8a7c151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 -- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c15b260 msgr2=0x7f8a7c151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c15b260 0x7f8a7c151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 -- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c0d19f0 msgr2=0x7f8a7c1513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:15.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c0d19f0 0x7f8a7c1513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:15.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a80e00640 1 -- 172.21.15.143:0/1037932265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6c0094d0 con 0x7f8a7c0c5220 2024-01-15T18:14:15.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.923+0000 7f8a7be00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c0d19f0 0x7f8a7c1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a80e00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 0x7f8a7c157cf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0ad040 tx=0x7f8a6c01ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0be3f0 con 0x7f8a7c0c5220 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6c0be570 con 0x7f8a7c0c5220 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a7c0b2570 con 0x7f8a7c0c5220 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c0be920 con 0x7f8a7c0c5220 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a7c0b2b30 con 0x7f8a7c0c5220 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7c0b2d20 con 0x7f8a7c0c5220 2024-01-15T18:14:15.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f8a6c0b43f0 con 0x7f8a7c0c5220 2024-01-15T18:14:15.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a78c00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a540674f0 0x7f8a540699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:15.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8a6c11d450 con 0x7f8a7c0c5220 2024-01-15T18:14:15.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.927+0000 7f8a7be00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a540674f0 0x7f8a540699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:15.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.931+0000 7f8a7be00640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a540674f0 0x7f8a540699b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0b6290 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:15.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:15.931+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a6c126080 con 0x7f8a7c0c5220 2024-01-15T18:14:16.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:16.123+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993112", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a7c067500 con 0x7f8a540674f0 2024-01-15T18:14:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.743+0000 7f8a78c00640 1 -- 172.21.15.143:0/1037932265 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8a7c067500 con 0x7f8a540674f0 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.743+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a540674f0 msgr2=0x7f8a540699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.743+0000 7f8a83800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a540674f0 0x7f8a540699b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0b6290 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.743+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 msgr2=0x7f8a7c157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.743+0000 7f8a83800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 0x7f8a7c157cf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c0ad040 tx=0x7f8a6c01ad50 comp rx=0 tx=0).stop 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 shutdown_connections 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a540674f0 0x7f8a540699b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a7c15b260 0x7f8a7c151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a7c0d19f0 0x7f8a7c1513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 --2- 172.21.15.143:0/1037932265 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a7c0c5220 0x7f8a7c157cf0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 >> 172.21.15.143:0/1037932265 conn(0x7f8a7c0d0ed0 msgr2=0x7f8a7c15a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 shutdown_connections 2024-01-15T18:14:24.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.747+0000 7f8a83800640 1 -- 172.21.15.143:0/1037932265 wait complete. 2024-01-15T18:14:24.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993113 2024-01-15T18:14:24.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcfe00640 1 Processor -- start 2024-01-15T18:14:24.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcfe00640 1 -- start start 2024-01-15T18:14:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcfe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5bc8071140 0x7f5bc80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcfe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc80ce810 con 0x7f5bc8071520 2024-01-15T18:14:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcfe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc80cb980 con 0x7f5bc815da90 2024-01-15T18:14:24.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcfe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80cc220 con 0x7f5bc8071140 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcca00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5bc8071520 0x7f5bc815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39926/0 (socket says 172.21.15.143:39926) 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcca00640 1 -- 172.21.15.143:0/3233746314 learned_addr learned my addr 172.21.15.143:0/3233746314 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bcd400640 1 --2- 172.21.15.143:0/3233746314 >> v2:172.21.15.204:3300/0 conn(0x7f5bc8071140 0x7f5bc80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203160853 0 0) 0x7f5bc80ce810 con 0x7f5bc8071520 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba8003530 con 0x7f5bc8071520 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 798839886 0 0) 0x7f5ba8003530 con 0x7f5bc8071520 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 >> v2:172.21.15.204:3300/0 conn(0x7f5bc8071140 msgr2=0x7f5bc80ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 --2- 172.21.15.143:0/3233746314 >> v2:172.21.15.204:3300/0 conn(0x7f5bc8071140 0x7f5bc80ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 >> v1:172.21.15.204:6789/0 conn(0x7f5bc815da90 legacy=0x7f5bc8164e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80cbc60 con 0x7f5bc8071520 2024-01-15T18:14:24.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f5bb0001680 con 0x7f5bc8071520 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.911+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 >> v1:172.21.15.143:6789/0 conn(0x7f5bc8071520 legacy=0x7f5bc815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ba8003980 0x7f5ba8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba80047d0 0x7f5ba8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ba8005190 0x7f5ba800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80cc220 con 0x7f5ba8003980 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba800c030 con 0x7f5ba80047d0 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba800e400 con 0x7f5ba8005190 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 --2- 172.21.15.143:0/3233746314 >> v2:172.21.15.204:3300/0 conn(0x7f5bc8071140 0x7f5bc80ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ba8003980 0x7f5ba8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcca00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba80047d0 0x7f5ba8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcde00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ba8005190 0x7f5ba800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 -- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ba8005190 msgr2=0x7f5ba800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ba8005190 0x7f5ba800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 -- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba80047d0 msgr2=0x7f5ba8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:24.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba80047d0 0x7f5ba8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 -- 172.21.15.143:0/3233746314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8003530 con 0x7f5ba8003980 2024-01-15T18:14:24.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcde00640 1 -- 172.21.15.143:0/3233746314 reap_dead start 2024-01-15T18:14:24.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bcd400640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ba8003980 0x7f5ba8003da0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009c90 tx=0x7f5bb801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f47709d5947ea3c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:24.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8004020 con 0x7f5ba8003980 2024-01-15T18:14:24.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8004180 con 0x7f5ba8003980 2024-01-15T18:14:24.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.915+0000 7f5bc7e00640 1 -- 172.21.15.143:0/3233746314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80a5640 con 0x7f5ba8003980 2024-01-15T18:14:24.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- 172.21.15.143:0/3233746314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ba8003980 msgr2=0x7f5ba8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ba8003980 0x7f5ba8003da0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009c90 tx=0x7f5bb801cbd0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- 172.21.15.143:0/3233746314 shutdown_connections 2024-01-15T18:14:24.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/3233746314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ba8003980 0x7f5ba8003da0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- 172.21.15.143:0/3233746314 >> 172.21.15.143:0/3233746314 conn(0x7f5bc80cfd20 msgr2=0x7f5bc80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:24.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- 172.21.15.143:0/3233746314 shutdown_connections 2024-01-15T18:14:24.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- 172.21.15.143:0/3233746314 wait complete. 2024-01-15T18:14:24.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 Processor -- start 2024-01-15T18:14:24.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- start start 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba8003980 0x7f5bc81e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc8144660 0x7f5bc80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 0x7f5bc80b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80b5970 con 0x7f5bc80b4150 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80b5040 con 0x7f5ba8003980 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcfe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80b5320 con 0x7f5bc8144660 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 0x7f5bc80b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 0x7f5bc80b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33134/0 (socket says 172.21.15.143:33134) 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcde00640 1 -- 172.21.15.143:0/949463750 learned_addr learned my addr 172.21.15.143:0/949463750 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcd400640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba8003980 0x7f5bc81e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.919+0000 7f5bcca00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc8144660 0x7f5bc80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcde00640 1 -- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc8144660 msgr2=0x7f5bc80c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcde00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc8144660 0x7f5bc80c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcde00640 1 -- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba8003980 msgr2=0x7f5bc81e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:24.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcde00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba8003980 0x7f5bc81e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:24.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcde00640 1 -- 172.21.15.143:0/949463750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80b5600 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcde00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 0x7f5bc80b45d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5bc815de70 tx=0x7f5bc000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc000ee90 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc0003980 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0003d70 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc81e79d0 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc80b4b10 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc80b5ab0 con 0x7f5bc80b4150 2024-01-15T18:14:24.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f5bc00050d0 con 0x7f5bc80b4150 2024-01-15T18:14:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bc5600640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ba00674c0 0x7f5ba0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5bc0082cf0 con 0x7f5bc80b4150 2024-01-15T18:14:24.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.923+0000 7f5bcd400640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ba00674c0 0x7f5ba0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:24.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.927+0000 7f5bcd400640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ba00674c0 0x7f5ba0069980 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ae000 tx=0x7f5bb80046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:24.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:24.931+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5bc80b5ab0 con 0x7f5bc80b4150 2024-01-15T18:14:25.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.111+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993113", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bc81579e0 con 0x7f5ba00674c0 2024-01-15T18:14:25.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.983+0000 7f5bc5600640 1 -- 172.21.15.143:0/949463750 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5bc81579e0 con 0x7f5ba00674c0 2024-01-15T18:14:25.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ba00674c0 msgr2=0x7f5ba0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:25.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ba00674c0 0x7f5ba0069980 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80ae000 tx=0x7f5bb80046d0 comp rx=0 tx=0).stop 2024-01-15T18:14:25.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 msgr2=0x7f5bc80b45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:25.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 0x7f5bc80b45d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5bc815de70 tx=0x7f5bc000eb50 comp rx=0 tx=0).stop 2024-01-15T18:14:25.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 shutdown_connections 2024-01-15T18:14:25.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5ba00674c0 0x7f5ba0069980 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bc80b4150 0x7f5bc80b45d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5bc8144660 0x7f5bc80c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 --2- 172.21.15.143:0/949463750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ba8003980 0x7f5bc81e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 >> 172.21.15.143:0/949463750 conn(0x7f5bc80cfd20 msgr2=0x7f5bc80d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 shutdown_connections 2024-01-15T18:14:25.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:25.987+0000 7f5bcfe00640 1 -- 172.21.15.143:0/949463750 wait complete. 2024-01-15T18:14:26.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993114 2024-01-15T18:14:26.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.147+0000 7f97d7a00640 1 Processor -- start 2024-01-15T18:14:26.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.147+0000 7f97d7a00640 1 -- start start 2024-01-15T18:14:26.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.147+0000 7f97d7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d00ce560 0x7f97d015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:26.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97d7a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015f310 con 0x7f97d0071d00 2024-01-15T18:14:26.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97d7a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015f5f0 con 0x7f97d0141660 2024-01-15T18:14:26.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97d7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015fbb0 con 0x7f97d00ce560 2024-01-15T18:14:26.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97d5a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f97d0071d00 0x7f97d00b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39936/0 (socket says 172.21.15.143:39936) 2024-01-15T18:14:26.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97d5a00640 1 -- 172.21.15.143:0/260177453 learned_addr learned my addr 172.21.15.143:0/260177453 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cfe00640 1 --2- 172.21.15.143:0/260177453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d00ce560 0x7f97d015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164521858 0 0) 0x7f97d015f310 con 0x7f97d0071d00 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b0003530 con 0x7f97d0071d00 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556628586 0 0) 0x7f97d015f5f0 con 0x7f97d0141660 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97d015f310 con 0x7f97d0141660 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cfe00640 1 -- 172.21.15.143:0/260177453 >> v1:172.21.15.204:6789/0 conn(0x7f97d0141660 legacy=0x7f97d0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180059362 0 0) 0x7f97b0003530 con 0x7f97d0071d00 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cfe00640 1 -- 172.21.15.143:0/260177453 >> v1:172.21.15.143:6789/0 conn(0x7f97d0071d00 legacy=0x7f97d00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:26.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.151+0000 7f97cfe00640 1 -- 172.21.15.143:0/260177453 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d015f8d0 con 0x7f97d00ce560 2024-01-15T18:14:26.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97cfe00640 1 --2- 172.21.15.143:0/260177453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d00ce560 0x7f97d015ec80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097f0 tx=0x7f97b801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f68591dc1c438c99 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:26.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c8001a00 con 0x7f97d00ce560 2024-01-15T18:14:26.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b80a7050 con 0x7f97d00ce560 2024-01-15T18:14:26.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97cf400640 1 -- 172.21.15.143:0/260177453 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8004250 con 0x7f97d00ce560 2024-01-15T18:14:26.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 -- 172.21.15.143:0/260177453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d00ce560 msgr2=0x7f97d015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:26.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 --2- 172.21.15.143:0/260177453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d00ce560 0x7f97d015ec80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97b80097f0 tx=0x7f97b801c3b0 comp rx=0 tx=0).stop 2024-01-15T18:14:26.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 -- 172.21.15.143:0/260177453 shutdown_connections 2024-01-15T18:14:26.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 --2- 172.21.15.143:0/260177453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d00ce560 0x7f97d015ec80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:26.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 -- 172.21.15.143:0/260177453 >> 172.21.15.143:0/260177453 conn(0x7f97d006c3f0 msgr2=0x7f97d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:26.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 -- 172.21.15.143:0/260177453 shutdown_connections 2024-01-15T18:14:26.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 -- 172.21.15.143:0/260177453 wait complete. 2024-01-15T18:14:26.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 Processor -- start 2024-01-15T18:14:26.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.155+0000 7f97d7a00640 1 -- start start 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 0x7f97d015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97d00ce560 0x7f97d01387b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d0141660 0x7f97d0138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00ce9e0 con 0x7f97d0071d00 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01339d0 con 0x7f97d00ce560 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d015a8c0 con 0x7f97d0141660 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 0x7f97d015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:26.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 0x7f97d015a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33140/0 (socket says 172.21.15.143:33140) 2024-01-15T18:14:26.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 -- 172.21.15.143:0/886917576 learned_addr learned my addr 172.21.15.143:0/886917576 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97cfe00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97d00ce560 0x7f97d01387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d0141660 0x7f97d0138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 -- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d0141660 msgr2=0x7f97d0138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d0141660 0x7f97d0138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 -- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97d00ce560 msgr2=0x7f97d01387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97d00ce560 0x7f97d01387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 -- 172.21.15.143:0/886917576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8009420 con 0x7f97d0071d00 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97cfe00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97d00ce560 0x7f97d01387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d0141660 0x7f97d0138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:14:26.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d5000640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 0x7f97d015a380 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c430 tx=0x7f97c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00052d0 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c0005450 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0003840 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d00cd150 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d01436a0 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d0066f60 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.159+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f97c00039a0 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.163+0000 7f97ccc00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97a80674c0 0x7f97a8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.163+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f97c0082660 con 0x7f97d0071d00 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.163+0000 7f97cfe00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97a80674c0 0x7f97a8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:26.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.163+0000 7f97cfe00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97a80674c0 0x7f97a8069980 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f97b80ad000 tx=0x7f97b80a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:26.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.167+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f97c0051090 con 0x7f97d0071d00 2024-01-15T18:14:26.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:26.347+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993114", "target": ["mon-mgr", ""]}) v1 -- 0x7f97d00ccfd0 con 0x7f97a80674c0 2024-01-15T18:14:29.208 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:29.212 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:35.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.631+0000 7f97ccc00640 1 -- 172.21.15.143:0/886917576 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f97b80ba0c0 con 0x7f97a80674c0 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97a80674c0 msgr2=0x7f97a8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97a80674c0 0x7f97a8069980 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f97b80ad000 tx=0x7f97b80a6000 comp rx=0 tx=0).stop 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 msgr2=0x7f97d015a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 0x7f97d015a380 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f97c000c430 tx=0x7f97c000c8f0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 shutdown_connections 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f97a80674c0 0x7f97a8069980 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f97d0141660 0x7f97d0138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f97d00ce560 0x7f97d01387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 --2- 172.21.15.143:0/886917576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f97d0071d00 0x7f97d015a380 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 >> 172.21.15.143:0/886917576 conn(0x7f97d006c3f0 msgr2=0x7f97d0070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:35.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 shutdown_connections 2024-01-15T18:14:35.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.635+0000 7f97d7a00640 1 -- 172.21.15.143:0/886917576 wait complete. 2024-01-15T18:14:35.653 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993115 2024-01-15T18:14:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff11d800640 1 Processor -- start 2024-01-15T18:14:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff11d800640 1 -- start start 2024-01-15T18:14:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff11d800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff118159b70 0x7ff1180b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff11d800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1180b4d20 con 0x7ff1180b48f0 2024-01-15T18:14:35.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff11d800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1180b2a20 con 0x7ff118157300 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff11d800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1180b35a0 con 0x7ff118159b70 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff116000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff118159b70 0x7ff1180b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff117400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff1180b48f0 0x7ff118070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49798/0 (socket says 172.21.15.143:49798) 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff117400640 1 -- 172.21.15.143:0/1081851216 learned_addr learned my addr 172.21.15.143:0/1081851216 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff116000640 1 -- 172.21.15.143:0/1081851216 >> v1:172.21.15.204:6789/0 conn(0x7ff118157300 legacy=0x7ff1180b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff116000640 1 -- 172.21.15.143:0/1081851216 >> v1:172.21.15.143:6789/0 conn(0x7ff1180b48f0 legacy=0x7ff118070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff115600640 1 -- 172.21.15.143:0/1081851216 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838109208 0 0) 0x7ff1180b4d20 con 0x7ff1180b48f0 2024-01-15T18:14:35.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.807+0000 7ff116000640 1 -- 172.21.15.143:0/1081851216 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180b2d00 con 0x7ff118159b70 2024-01-15T18:14:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff116000640 1 --2- 172.21.15.143:0/1081851216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118159b70 0x7ff1180b43b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff10001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf3f7d3e259ec3d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff115600640 1 -- 172.21.15.143:0/1081851216 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000ab650 con 0x7ff118159b70 2024-01-15T18:14:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff115600640 1 -- 172.21.15.143:0/1081851216 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1000ab7b0 con 0x7ff118159b70 2024-01-15T18:14:35.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff115600640 1 -- 172.21.15.143:0/1081851216 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000abb40 con 0x7ff118159b70 2024-01-15T18:14:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff11d800640 1 -- 172.21.15.143:0/1081851216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118159b70 msgr2=0x7ff1180b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff11d800640 1 --2- 172.21.15.143:0/1081851216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118159b70 0x7ff1180b43b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff10001c450 comp rx=0 tx=0).stop 2024-01-15T18:14:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff11d800640 1 -- 172.21.15.143:0/1081851216 shutdown_connections 2024-01-15T18:14:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff11d800640 1 --2- 172.21.15.143:0/1081851216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118159b70 0x7ff1180b43b0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.811+0000 7ff11d800640 1 -- 172.21.15.143:0/1081851216 >> 172.21.15.143:0/1081851216 conn(0x7ff11806c3f0 msgr2=0x7ff11806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 -- 172.21.15.143:0/1081851216 shutdown_connections 2024-01-15T18:14:35.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 -- 172.21.15.143:0/1081851216 wait complete. 2024-01-15T18:14:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 Processor -- start 2024-01-15T18:14:35.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 -- start start 2024-01-15T18:14:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1180b48f0 0x7ff118142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 0x7ff11814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff118159b70 0x7ff1181415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff118159fd0 con 0x7ff118157300 2024-01-15T18:14:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11814ee50 con 0x7ff1180b48f0 2024-01-15T18:14:35.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff11d800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11814f0b0 con 0x7ff118159b70 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 0x7ff11814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 0x7ff11814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57864/0 (socket says 172.21.15.143:57864) 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 -- 172.21.15.143:0/2788912652 learned_addr learned my addr 172.21.15.143:0/2788912652 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff117400640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff118159b70 0x7ff1181415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116a00640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1180b48f0 0x7ff118142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 -- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff118159b70 msgr2=0x7ff1181415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff118159b70 0x7ff1181415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 -- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1180b48f0 msgr2=0x7ff118142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1180b48f0 0x7ff118142cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:35.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.815+0000 7ff116000640 1 -- 172.21.15.143:0/2788912652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff100009420 con 0x7ff118157300 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff116000640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 0x7ff11814e910 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff1000ad040 tx=0x7ff10001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b5810 con 0x7ff118157300 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1000b5970 con 0x7ff118157300 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b5d20 con 0x7ff118157300 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11814f390 con 0x7ff118157300 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11814f950 con 0x7ff118157300 2024-01-15T18:14:35.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff118067500 con 0x7ff118157300 2024-01-15T18:14:35.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7ff1000bf410 con 0x7ff118157300 2024-01-15T18:14:35.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff10ea00640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0f00674c0 0x7ff0f0069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:35.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff10011db90 con 0x7ff118157300 2024-01-15T18:14:35.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.819+0000 7ff116a00640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0f00674c0 0x7ff0f0069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:35.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.823+0000 7ff116a00640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0f00674c0 0x7ff0f0069980 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff1181f6ff0 tx=0x7ff108009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:35.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:35.823+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff1000ec790 con 0x7ff118157300 2024-01-15T18:14:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:36.007+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993115", "target": ["mon-mgr", ""]}) v1 -- 0x7ff118150160 con 0x7ff0f00674c0 2024-01-15T18:14:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.903+0000 7ff10ea00640 1 -- 172.21.15.143:0/2788912652 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff118150160 con 0x7ff0f00674c0 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0f00674c0 msgr2=0x7ff0f0069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0f00674c0 0x7ff0f0069980 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff1181f6ff0 tx=0x7ff108009040 comp rx=0 tx=0).stop 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 msgr2=0x7ff11814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 0x7ff11814e910 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7ff1000ad040 tx=0x7ff10001ac90 comp rx=0 tx=0).stop 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 shutdown_connections 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff0f00674c0 0x7ff0f0069980 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff118159b70 0x7ff1181415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff118157300 0x7ff11814e910 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 --2- 172.21.15.143:0/2788912652 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff1180b48f0 0x7ff118142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 >> 172.21.15.143:0/2788912652 conn(0x7ff11806c3f0 msgr2=0x7ff11814d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 shutdown_connections 2024-01-15T18:14:45.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:45.907+0000 7ff11d800640 1 -- 172.21.15.143:0/2788912652 wait complete. 2024-01-15T18:14:45.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993116 2024-01-15T18:14:46.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.067+0000 7f447e800640 1 Processor -- start 2024-01-15T18:14:46.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f447e800640 1 -- start start 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f447e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f447806aba0 0x7f44780aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f447e800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4478160eb0 con 0x7f447815d130 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f447e800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4478161190 con 0x7f447815d510 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f447e800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4478161470 con 0x7f447806aba0 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f447815d130 0x7f447806a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43206/0 (socket says 172.21.15.143:43206) 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 -- 172.21.15.143:0/1452838919 learned_addr learned my addr 172.21.15.143:0/1452838919 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476a00640 1 --2- 172.21.15.143:0/1452838919 >> v2:172.21.15.204:3300/0 conn(0x7f447806aba0 0x7f44780aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264906109 0 0) 0x7f4478160eb0 con 0x7f447815d130 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4458003530 con 0x7f447815d130 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862824151 0 0) 0x7f4458003530 con 0x7f447815d130 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 >> v2:172.21.15.204:3300/0 conn(0x7f447806aba0 msgr2=0x7f44780aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 --2- 172.21.15.143:0/1452838919 >> v2:172.21.15.204:3300/0 conn(0x7f447806aba0 0x7f44780aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 >> v1:172.21.15.204:6789/0 conn(0x7f447815d510 legacy=0x7f44780b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:46.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478161750 con 0x7f447815d130 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4468001680 con 0x7f447815d130 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 >> v1:172.21.15.143:6789/0 conn(0x7f447815d130 legacy=0x7f447806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4458003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44580047d0 0x7f4458004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4458005190 0x7f445800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478161470 con 0x7f4458003980 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c030 con 0x7f44580047d0 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445800c1f0 con 0x7f4458005190 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4458003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476a00640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44580047d0 0x7f4458004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477e00640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4458005190 0x7f445800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 -- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4458005190 msgr2=0x7f445800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4458005190 0x7f445800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 -- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44580047d0 msgr2=0x7f4458004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44580047d0 0x7f4458004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:46.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477400640 1 -- 172.21.15.143:0/1452838919 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4458003530 con 0x7f4458003980 2024-01-15T18:14:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4476a00640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44580047d0 0x7f4458004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:14:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.071+0000 7f4477e00640 1 -- 172.21.15.143:0/1452838919 reap_dead start 2024-01-15T18:14:46.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f4477400640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4458003da0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f446800c240 tx=0x7f446801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7883c7c84589f0f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4468005ae0 con 0x7f4458003980 2024-01-15T18:14:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468005c40 con 0x7f4458003980 2024-01-15T18:14:46.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f4476000640 1 -- 172.21.15.143:0/1452838919 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680a5a40 con 0x7f4458003980 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 -- 172.21.15.143:0/1452838919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 msgr2=0x7f4458003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4458003da0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f446800c240 tx=0x7f446801cc40 comp rx=0 tx=0).stop 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 -- 172.21.15.143:0/1452838919 shutdown_connections 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 --2- 172.21.15.143:0/1452838919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4458003da0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 -- 172.21.15.143:0/1452838919 >> 172.21.15.143:0/1452838919 conn(0x7f44780b1b90 msgr2=0x7f44780b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 -- 172.21.15.143:0/1452838919 shutdown_connections 2024-01-15T18:14:46.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.075+0000 7f447e800640 1 -- 172.21.15.143:0/1452838919 wait complete. 2024-01-15T18:14:46.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 Processor -- start 2024-01-15T18:14:46.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 -- start start 2024-01-15T18:14:46.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4478153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447814de90 0x7f4478155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4478151240 0x7f447815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478156220 con 0x7f4458003980 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4478155ff0 con 0x7f447814de90 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447815e970 con 0x7f4478151240 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4478153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4478153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52550/0 (socket says 172.21.15.143:52550) 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 -- 172.21.15.143:0/1573833771 learned_addr learned my addr 172.21.15.143:0/1573833771 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477e00640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4478151240 0x7f447815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4476a00640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447814de90 0x7f4478155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 -- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4478151240 msgr2=0x7f447815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4478151240 0x7f447815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 -- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447814de90 msgr2=0x7f4478155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447814de90 0x7f4478155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 -- 172.21.15.143:0/1573833771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447815ec10 con 0x7f4458003980 2024-01-15T18:14:46.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f4477400640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4478153300 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f446801ca70 tx=0x7f4458005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:46.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680b4070 con 0x7f4458003980 2024-01-15T18:14:46.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4468001650 con 0x7f4458003980 2024-01-15T18:14:46.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44680a5a20 con 0x7f4458003980 2024-01-15T18:14:46.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.079+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447815eef0 con 0x7f4458003980 2024-01-15T18:14:46.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447815fcf0 con 0x7f4458003980 2024-01-15T18:14:46.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44780cfbe0 con 0x7f4458003980 2024-01-15T18:14:46.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f44680a5b80 con 0x7f4458003980 2024-01-15T18:14:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f446f400640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44500674c0 0x7f4450069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f446811c860 con 0x7f4458003980 2024-01-15T18:14:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f4476a00640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44500674c0 0x7f4450069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.083+0000 7f4476a00640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44500674c0 0x7f4450069980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44781511f0 tx=0x7f4460008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.087+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44680eb340 con 0x7f4458003980 2024-01-15T18:14:46.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:46.267+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993116", "target": ["mon-mgr", ""]}) v1 -- 0x7f4478067500 con 0x7f44500674c0 2024-01-15T18:14:48.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.807+0000 7f446f400640 1 -- 172.21.15.143:0/1573833771 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4478067500 con 0x7f44500674c0 2024-01-15T18:14:48.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44500674c0 msgr2=0x7f4450069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:48.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44500674c0 0x7f4450069980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44781511f0 tx=0x7f4460008040 comp rx=0 tx=0).stop 2024-01-15T18:14:48.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 msgr2=0x7f4478153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4478153300 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f446801ca70 tx=0x7f4458005800 comp rx=0 tx=0).stop 2024-01-15T18:14:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 shutdown_connections 2024-01-15T18:14:48.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44500674c0 0x7f4450069980 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4478151240 0x7f447815f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f447814de90 0x7f4478155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 --2- 172.21.15.143:0/1573833771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4458003980 0x7f4478153300 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.811+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 >> 172.21.15.143:0/1573833771 conn(0x7f44780b1b90 msgr2=0x7f44780adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.815+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 shutdown_connections 2024-01-15T18:14:48.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:48.815+0000 7f447e800640 1 -- 172.21.15.143:0/1573833771 wait complete. 2024-01-15T18:14:48.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993117 2024-01-15T18:14:49.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 Processor -- start 2024-01-15T18:14:49.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 -- start start 2024-01-15T18:14:49.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59e006bfb0 0x7f59e006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:49.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e00b0310 0x7f59e00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e00cbff0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e00d0c50 con 0x7f59e006bfb0 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59e6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e006c8d0 con 0x7f59e00b0310 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59e006bfb0 0x7f59e006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f59e006bfb0 0x7f59e006c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38218/0 (socket says 172.21.15.143:38218) 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 -- 172.21.15.143:0/2509088370 learned_addr learned my addr 172.21.15.143:0/2509088370 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59dfe00640 1 --1- 172.21.15.143:0/2509088370 >> v1:172.21.15.143:6789/0 conn(0x7f59e00cc9c0 0x7f59e00cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49618/0 (socket says 172.21.15.143:49618) 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59dea00640 1 --2- 172.21.15.143:0/2509088370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e00b0310 0x7f59e00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:49.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 -- 172.21.15.143:0/2509088370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e00b0310 msgr2=0x7f59e00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:14:49.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 --2- 172.21.15.143:0/2509088370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e00b0310 0x7f59e00cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:49.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 -- 172.21.15.143:0/2509088370 >> v1:172.21.15.143:6789/0 conn(0x7f59e00cc9c0 legacy=0x7f59e00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:49.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59de000640 1 -- 172.21.15.143:0/2509088370 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710111090 0 0) 0x7f59e00cbff0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.055+0000 7f59df400640 1 -- 172.21.15.143:0/2509088370 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e00cb6c0 con 0x7f59e006bfb0 2024-01-15T18:14:49.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59df400640 1 --2- 172.21.15.143:0/2509088370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e006bfb0 0x7f59e006c390 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009870 tx=0x7f59d001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78bccb7982ddd01 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:49.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59de000640 1 -- 172.21.15.143:0/2509088370 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00ab680 con 0x7f59e006bfb0 2024-01-15T18:14:49.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59de000640 1 -- 172.21.15.143:0/2509088370 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d00ab7e0 con 0x7f59e006bfb0 2024-01-15T18:14:49.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59de000640 1 -- 172.21.15.143:0/2509088370 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00abb70 con 0x7f59e006bfb0 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59e6800640 1 -- 172.21.15.143:0/2509088370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e006bfb0 msgr2=0x7f59e006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59e6800640 1 --2- 172.21.15.143:0/2509088370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e006bfb0 0x7f59e006c390 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009870 tx=0x7f59d001c480 comp rx=0 tx=0).stop 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59e6800640 1 -- 172.21.15.143:0/2509088370 shutdown_connections 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59e6800640 1 --2- 172.21.15.143:0/2509088370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e00b0310 0x7f59e00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59e6800640 1 --2- 172.21.15.143:0/2509088370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e006bfb0 0x7f59e006c390 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.059+0000 7f59e6800640 1 -- 172.21.15.143:0/2509088370 >> 172.21.15.143:0/2509088370 conn(0x7f59e00d0ed0 msgr2=0x7f59e00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:14:49.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 -- 172.21.15.143:0/2509088370 shutdown_connections 2024-01-15T18:14:49.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 -- 172.21.15.143:0/2509088370 wait complete. 2024-01-15T18:14:49.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 Processor -- start 2024-01-15T18:14:49.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 -- start start 2024-01-15T18:14:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e006bfb0 0x7f59e00c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e00b0310 0x7f59e01592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e00cc9c0 0x7f59e01597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00cade0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00c9d40 con 0x7f59e00b0310 2024-01-15T18:14:49.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59e6800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e00ca020 con 0x7f59e006bfb0 2024-01-15T18:14:49.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59df400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e006bfb0 0x7f59e00c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:49.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e00cc9c0 0x7f59e01597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59df400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e006bfb0 0x7f59e00c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:60842/0 (socket says 172.21.15.143:60842) 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59df400640 1 -- 172.21.15.143:0/3642194144 learned_addr learned my addr 172.21.15.143:0/3642194144 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dea00640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e00b0310 0x7f59e01592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 -- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e006bfb0 msgr2=0x7f59e00c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e006bfb0 0x7f59e00c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 -- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e00b0310 msgr2=0x7f59e01592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e00b0310 0x7f59e01592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 -- 172.21.15.143:0/3642194144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00094d0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.063+0000 7f59dfe00640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e00cc9c0 0x7f59e01597f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f59d8002240 tx=0x7f59d800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800ce30 con 0x7f59e00cc9c0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d800a640 con 0x7f59e00cc9c0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e00ca300 con 0x7f59e00cc9c0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d800a9f0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e00ca8c0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e00caae0 con 0x7f59e00cc9c0 2024-01-15T18:14:49.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.067+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f59d800ab50 con 0x7f59e00cc9c0 2024-01-15T18:14:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.071+0000 7f59d7400640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59b80674c0 0x7f59b8069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:14:49.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.071+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f59d8082830 con 0x7f59e00cc9c0 2024-01-15T18:14:49.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.071+0000 7f59df400640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59b80674c0 0x7f59b8069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:14:49.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.071+0000 7f59df400640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59b80674c0 0x7f59b8069980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f59d001a0b0 tx=0x7f59d00b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:14:49.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.071+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f59d8051310 con 0x7f59e00cc9c0 2024-01-15T18:14:49.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:14:49.255+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993117", "target": ["mon-mgr", ""]}) v1 -- 0x7f59e006b7b0 con 0x7f59b80674c0 2024-01-15T18:14:59.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:59.271 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59d7400640 1 -- 172.21.15.143:0/3642194144 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f59d00ab950 con 0x7f59b80674c0 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59b80674c0 msgr2=0x7f59b8069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59b80674c0 0x7f59b8069980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f59d001a0b0 tx=0x7f59d00b6040 comp rx=0 tx=0).stop 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e00cc9c0 msgr2=0x7f59e01597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e00cc9c0 0x7f59e01597f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f59d8002240 tx=0x7f59d800cbe0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 shutdown_connections 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59b80674c0 0x7f59b8069980 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f59e00cc9c0 0x7f59e01597f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59e00b0310 0x7f59e01592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 --2- 172.21.15.143:0/3642194144 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f59e006bfb0 0x7f59e00c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.035+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 >> 172.21.15.143:0/3642194144 conn(0x7f59e00d0ed0 msgr2=0x7f59e00adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.039+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 shutdown_connections 2024-01-15T18:15:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.039+0000 7f59e6800640 1 -- 172.21.15.143:0/3642194144 wait complete. 2024-01-15T18:15:04.055 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993118 2024-01-15T18:15:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb583400640 1 Processor -- start 2024-01-15T18:15:04.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb583400640 1 -- start start 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb583400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb57c0cf7e0 0x7fb57c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb583400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c15b5b0 con 0x7fb57c0cda70 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb583400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57c0cba70 con 0x7fb57c0cf400 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb583400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57c0cbd10 con 0x7fb57c0cf7e0 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb580a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb57c0cda70 0x7fb57c0cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35096/0 (socket says 172.21.15.143:35096) 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.199+0000 7fb580a00640 1 -- 172.21.15.143:0/2844069904 learned_addr learned my addr 172.21.15.143:0/2844069904 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb581400640 1 --2- 172.21.15.143:0/2844069904 >> v2:172.21.15.204:3300/0 conn(0x7fb57c0cf7e0 0x7fb57c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865929885 0 0) 0x7fb57c15b5b0 con 0x7fb57c0cda70 2024-01-15T18:15:04.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb55c003530 con 0x7fb57c0cda70 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702258572 0 0) 0x7fb55c003530 con 0x7fb57c0cda70 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 >> v2:172.21.15.204:3300/0 conn(0x7fb57c0cf7e0 msgr2=0x7fb57c0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 --2- 172.21.15.143:0/2844069904 >> v2:172.21.15.204:3300/0 conn(0x7fb57c0cf7e0 0x7fb57c0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 >> v1:172.21.15.204:6789/0 conn(0x7fb57c0cf400 legacy=0x7fb57c0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0cbff0 con 0x7fb57c0cda70 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fb56c001680 con 0x7fb57c0cda70 2024-01-15T18:15:04.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 >> v1:172.21.15.143:6789/0 conn(0x7fb57c0cda70 legacy=0x7fb57c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c003980 0x7fb55c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb55c004380 0x7fb55c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb55c005050 0x7fb55c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0cbd10 con 0x7fb55c004380 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c00bfb0 con 0x7fb55c005050 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55c00c170 con 0x7fb55c003980 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57be00640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb55c004380 0x7fb55c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb581400640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb55c005050 0x7fb55c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb580a00640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c003980 0x7fb55c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57be00640 1 -- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c003980 msgr2=0x7fb55c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57be00640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c003980 0x7fb55c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57be00640 1 -- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb55c005050 msgr2=0x7fb55c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57be00640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb55c005050 0x7fb55c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb57be00640 1 -- 172.21.15.143:0/2844069904 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c003530 con 0x7fb55c004380 2024-01-15T18:15:04.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.203+0000 7fb581400640 1 -- 172.21.15.143:0/2844069904 reap_dead start 2024-01-15T18:15:04.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb57be00640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb55c004380 0x7fb55c004850 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb564001860 tx=0x7fb56401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcfe5b9b4a80f0e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:04.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640a8850 con 0x7fb55c004380 2024-01-15T18:15:04.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5640a89b0 con 0x7fb55c004380 2024-01-15T18:15:04.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb57b400640 1 -- 172.21.15.143:0/2844069904 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5640a8da0 con 0x7fb55c004380 2024-01-15T18:15:04.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 -- 172.21.15.143:0/2844069904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb55c004380 msgr2=0x7fb55c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb55c004380 0x7fb55c004850 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fb564001860 tx=0x7fb56401ec30 comp rx=0 tx=0).stop 2024-01-15T18:15:04.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 -- 172.21.15.143:0/2844069904 shutdown_connections 2024-01-15T18:15:04.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 --2- 172.21.15.143:0/2844069904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb55c004380 0x7fb55c004850 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 -- 172.21.15.143:0/2844069904 >> 172.21.15.143:0/2844069904 conn(0x7fb57c0d0e20 msgr2=0x7fb57c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:04.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 -- 172.21.15.143:0/2844069904 shutdown_connections 2024-01-15T18:15:04.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 -- 172.21.15.143:0/2844069904 wait complete. 2024-01-15T18:15:04.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 Processor -- start 2024-01-15T18:15:04.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.207+0000 7fb583400640 1 -- start start 2024-01-15T18:15:04.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c004380 0x7fb57c06cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb57c1f2870 0x7fb57c1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 0x7fb57c0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0c4710 con 0x7fb57c0c9a80 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c06fa90 con 0x7fb57c1f2870 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c0c4920 con 0x7fb55c004380 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 0x7fb57c0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 0x7fb57c0c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59294/0 (socket says 172.21.15.143:59294) 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 -- 172.21.15.143:0/3914942992 learned_addr learned my addr 172.21.15.143:0/3914942992 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb57be00640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb57c1f2870 0x7fb57c1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb580a00640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c004380 0x7fb57c06cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 -- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c004380 msgr2=0x7fb57c06cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c004380 0x7fb57c06cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 -- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb57c1f2870 msgr2=0x7fb57c1f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:04.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb57c1f2870 0x7fb57c1f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:04.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 -- 172.21.15.143:0/3914942992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c0c4c00 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb581400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 0x7fb57c0c9f00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb57c072310 tx=0x7fb57400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400e040 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57400f040 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57400fec0 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57c0c4ee0 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57c06d280 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57c067500 con 0x7fb57c0c9a80 2024-01-15T18:15:04.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.211+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7fb574015430 con 0x7fb57c0c9a80 2024-01-15T18:15:04.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.215+0000 7fb578c00640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5540674f0 0x7fb5540699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:04.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.215+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb574083970 con 0x7fb57c0c9a80 2024-01-15T18:15:04.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.215+0000 7fb580a00640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5540674f0 0x7fb5540699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:04.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.215+0000 7fb580a00640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5540674f0 0x7fb5540699b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb56c001cc0 tx=0x7fb56c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:04.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.219+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb574052540 con 0x7fb57c0c9a80 2024-01-15T18:15:04.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:04.399+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993118", "target": ["mon-mgr", ""]}) v1 -- 0x7fb57c0cfd10 con 0x7fb5540674f0 2024-01-15T18:15:08.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.851+0000 7fb578c00640 1 -- 172.21.15.143:0/3914942992 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb56c011910 con 0x7fb5540674f0 2024-01-15T18:15:08.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5540674f0 msgr2=0x7fb5540699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:08.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5540674f0 0x7fb5540699b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb56c001cc0 tx=0x7fb56c007040 comp rx=0 tx=0).stop 2024-01-15T18:15:08.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 msgr2=0x7fb57c0c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:08.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 0x7fb57c0c9f00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fb57c072310 tx=0x7fb57400f1b0 comp rx=0 tx=0).stop 2024-01-15T18:15:08.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 shutdown_connections 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb5540674f0 0x7fb5540699b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb57c0c9a80 0x7fb57c0c9f00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb57c1f2870 0x7fb57c1f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 --2- 172.21.15.143:0/3914942992 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb55c004380 0x7fb57c06cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 >> 172.21.15.143:0/3914942992 conn(0x7fb57c0d0e20 msgr2=0x7fb57c0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 shutdown_connections 2024-01-15T18:15:08.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:08.855+0000 7fb583400640 1 -- 172.21.15.143:0/3914942992 wait complete. 2024-01-15T18:15:08.888 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993119 2024-01-15T18:15:09.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 Processor -- start 2024-01-15T18:15:09.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 -- start start 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa20157300 0x7faa20159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa200b4ac0 0x7faa20070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa200b2a20 con 0x7faa200b21d0 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200b35a0 con 0x7faa200b4ac0 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa28800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200b2d00 con 0x7faa20157300 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa26800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faa200b4ac0 0x7faa20070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa25400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faa200b21d0 0x7faa200b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55138/0 (socket says 172.21.15.143:55138) 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa25400640 1 -- 172.21.15.143:0/4079425931 learned_addr learned my addr 172.21.15.143:0/4079425931 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa25e00640 1 --2- 172.21.15.143:0/4079425931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa20157300 0x7faa20159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa26800640 1 -- 172.21.15.143:0/4079425931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa20157300 msgr2=0x7faa20159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:09.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa26800640 1 --2- 172.21.15.143:0/4079425931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa20157300 0x7faa20159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:09.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa26800640 1 -- 172.21.15.143:0/4079425931 >> v1:172.21.15.143:6789/0 conn(0x7faa200b21d0 legacy=0x7faa200b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:09.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa24a00640 1 -- 172.21.15.143:0/4079425931 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367238435 0 0) 0x7faa200b2a20 con 0x7faa200b21d0 2024-01-15T18:15:09.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.035+0000 7faa26800640 1 -- 172.21.15.143:0/4079425931 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200b2fe0 con 0x7faa200b4ac0 2024-01-15T18:15:09.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa26800640 1 --2- 172.21.15.143:0/4079425931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa20070e80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7faa18009d90 tx=0x7faa1801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e98d41c2cf5a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:09.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa24a00640 1 -- 172.21.15.143:0/4079425931 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180a5a00 con 0x7faa200b4ac0 2024-01-15T18:15:09.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa24a00640 1 -- 172.21.15.143:0/4079425931 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180a5b60 con 0x7faa200b4ac0 2024-01-15T18:15:09.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa24a00640 1 -- 172.21.15.143:0/4079425931 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180ad440 con 0x7faa200b4ac0 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 -- 172.21.15.143:0/4079425931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 msgr2=0x7faa20070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 --2- 172.21.15.143:0/4079425931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa20070e80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7faa18009d90 tx=0x7faa1801c920 comp rx=0 tx=0).stop 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 -- 172.21.15.143:0/4079425931 shutdown_connections 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 --2- 172.21.15.143:0/4079425931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa20070e80 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 --2- 172.21.15.143:0/4079425931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa20157300 0x7faa20159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 -- 172.21.15.143:0/4079425931 >> 172.21.15.143:0/4079425931 conn(0x7faa2006c3f0 msgr2=0x7faa2006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 -- 172.21.15.143:0/4079425931 shutdown_connections 2024-01-15T18:15:09.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.039+0000 7faa28800640 1 -- 172.21.15.143:0/4079425931 wait complete. 2024-01-15T18:15:09.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 Processor -- start 2024-01-15T18:15:09.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 -- start start 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa200b21d0 0x7faa201431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa2014de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa20157300 0x7faa2014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2014dc30 con 0x7faa200b4ac0 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200ac8e0 con 0x7faa20157300 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa20158cd0 con 0x7faa200b21d0 2024-01-15T18:15:09.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa2014de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa2014de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54010/0 (socket says 172.21.15.143:54010) 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 -- 172.21.15.143:0/1973936499 learned_addr learned my addr 172.21.15.143:0/1973936499 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa26800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa20157300 0x7faa2014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25e00640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa200b21d0 0x7faa201431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 -- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa200b21d0 msgr2=0x7faa201431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa200b21d0 0x7faa201431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 -- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa20157300 msgr2=0x7faa2014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa20157300 0x7faa2014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 -- 172.21.15.143:0/1973936499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18009a70 con 0x7faa200b4ac0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25e00640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa200b21d0 0x7faa201431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa25400640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa2014de50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7faa080011f0 tx=0x7faa0800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080074e0 con 0x7faa200b4ac0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa08007640 con 0x7faa200b4ac0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa20158fb0 con 0x7faa200b4ac0 2024-01-15T18:15:09.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08004a10 con 0x7faa200b4ac0 2024-01-15T18:15:09.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.043+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa20159570 con 0x7faa200b4ac0 2024-01-15T18:15:09.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.047+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa200aca90 con 0x7faa200b4ac0 2024-01-15T18:15:09.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.047+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7faa08004b70 con 0x7faa200b4ac0 2024-01-15T18:15:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.047+0000 7faa1e000640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa9f80674f0 0x7fa9f80699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.047+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7faa08082180 con 0x7faa200b4ac0 2024-01-15T18:15:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.047+0000 7faa25e00640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa9f80674f0 0x7fa9f80699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:09.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.047+0000 7faa25e00640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa9f80674f0 0x7fa9f80699b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faa10009eb0 tx=0x7faa10009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:09.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.051+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faa200aca90 con 0x7faa200b4ac0 2024-01-15T18:15:09.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:09.235+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993119", "target": ["mon-mgr", ""]}) v1 -- 0x7faa20159700 con 0x7fa9f80674f0 2024-01-15T18:15:18.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa1e000640 1 -- 172.21.15.143:0/1973936499 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7faa20159700 con 0x7fa9f80674f0 2024-01-15T18:15:18.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa9f80674f0 msgr2=0x7fa9f80699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa9f80674f0 0x7fa9f80699b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faa10009eb0 tx=0x7faa10009190 comp rx=0 tx=0).stop 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 msgr2=0x7faa2014de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa2014de50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7faa080011f0 tx=0x7faa0800c9b0 comp rx=0 tx=0).stop 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 shutdown_connections 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa9f80674f0 0x7fa9f80699b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa20157300 0x7faa2014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faa200b4ac0 0x7faa2014de50 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 --2- 172.21.15.143:0/1973936499 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faa200b21d0 0x7faa201431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.987+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 >> 172.21.15.143:0/1973936499 conn(0x7faa2006c3f0 msgr2=0x7faa200afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.991+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 shutdown_connections 2024-01-15T18:15:18.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:18.991+0000 7faa28800640 1 -- 172.21.15.143:0/1973936499 wait complete. 2024-01-15T18:15:19.007 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone993120 2024-01-15T18:15:19.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 Processor -- start 2024-01-15T18:15:19.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 -- start start 2024-01-15T18:15:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5978141660 0x7f59780ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597815ec80 0x7f5978141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5978142470 0x7f5978071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597815fbb0 con 0x7f5978142470 2024-01-15T18:15:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597815f100 con 0x7f5978141660 2024-01-15T18:15:19.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f5980600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597815f310 con 0x7f597815ec80 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5978142470 0x7f5978071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5978142470 0x7f5978071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33192/0 (socket says 172.21.15.143:33192) 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 -- 172.21.15.143:0/206044627 learned_addr learned my addr 172.21.15.143:0/206044627 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597d200640 1 --2- 172.21.15.143:0/206044627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597815ec80 0x7f5978141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597dc00640 1 --2- 172.21.15.143:0/206044627 >> v2:172.21.15.204:3300/0 conn(0x7f5978141660 0x7f59780ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 -- 172.21.15.143:0/206044627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597815ec80 msgr2=0x7f5978141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 --2- 172.21.15.143:0/206044627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597815ec80 0x7f5978141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 -- 172.21.15.143:0/206044627 >> v2:172.21.15.204:3300/0 conn(0x7f5978141660 msgr2=0x7f59780ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 --2- 172.21.15.143:0/206044627 >> v2:172.21.15.204:3300/0 conn(0x7f5978141660 0x7f59780ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.151+0000 7f597e600640 1 -- 172.21.15.143:0/206044627 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597815f5f0 con 0x7f5978142470 2024-01-15T18:15:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f597e600640 1 --2- 172.21.15.143:0/206044627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f5978071d00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f59780b1aa0 tx=0x7f597001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc64a78e36d9cd40 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5977e00640 1 -- 172.21.15.143:0/206044627 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700a5730 con 0x7f5978142470 2024-01-15T18:15:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5977e00640 1 -- 172.21.15.143:0/206044627 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59700a5890 con 0x7f5978142470 2024-01-15T18:15:19.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5977e00640 1 -- 172.21.15.143:0/206044627 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700a5c20 con 0x7f5978142470 2024-01-15T18:15:19.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 -- 172.21.15.143:0/206044627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 msgr2=0x7f5978071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:19.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 --2- 172.21.15.143:0/206044627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f5978071d00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f59780b1aa0 tx=0x7f597001c9a0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 -- 172.21.15.143:0/206044627 shutdown_connections 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 --2- 172.21.15.143:0/206044627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f5978071d00 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 --2- 172.21.15.143:0/206044627 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f597815ec80 0x7f5978141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 --2- 172.21.15.143:0/206044627 >> v2:172.21.15.204:3300/0 conn(0x7f5978141660 0x7f59780ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.155+0000 7f5980600640 1 -- 172.21.15.143:0/206044627 >> 172.21.15.143:0/206044627 conn(0x7f597806c3f0 msgr2=0x7f597806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 -- 172.21.15.143:0/206044627 shutdown_connections 2024-01-15T18:15:19.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 -- 172.21.15.143:0/206044627 wait complete. 2024-01-15T18:15:19.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 Processor -- start 2024-01-15T18:15:19.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 -- start start 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5978141660 0x7f59780b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f597815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597815ec80 0x7f5978139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597813a0f0 con 0x7f5978142470 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780b2150 con 0x7f597815ec80 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f5980600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780b2430 con 0x7f5978141660 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f597815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f597815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33208/0 (socket says 172.21.15.143:33208) 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 -- 172.21.15.143:0/1103393054 learned_addr learned my addr 172.21.15.143:0/1103393054 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597dc00640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5978141660 0x7f59780b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597e600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597815ec80 0x7f5978139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 -- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5978141660 msgr2=0x7f59780b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:19.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5978141660 0x7f59780b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 -- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597815ec80 msgr2=0x7f5978139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597815ec80 0x7f5978139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 -- 172.21.15.143:0/1103393054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009b30 con 0x7f5978142470 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597e600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597815ec80 0x7f5978139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.159+0000 7f597d200640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f597815e430 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f596000d5f0 tx=0x7f596000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59600125f0 con 0x7f5978142470 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5960012750 con 0x7f5978142470 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960012ae0 con 0x7f5978142470 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59780cd150 con 0x7f5978142470 2024-01-15T18:15:19.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59780b29f0 con 0x7f5978142470 2024-01-15T18:15:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5978067500 con 0x7f5978142470 2024-01-15T18:15:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f5960012c40 con 0x7f5978142470 2024-01-15T18:15:19.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5975600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59500674c0 0x7f5950069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5960083460 con 0x7f5978142470 2024-01-15T18:15:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f597dc00640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59500674c0 0x7f5950069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:19.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.163+0000 7f597dc00640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59500674c0 0x7f5950069980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f59680045f0 tx=0x7f5968008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:19.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.167+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5960051e90 con 0x7f5978142470 2024-01-15T18:15:19.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:19.347+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone993120", "target": ["mon-mgr", ""]}) v1 -- 0x7f59780ccfd0 con 0x7f59500674c0 2024-01-15T18:15:25.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.935+0000 7f5975600640 1 -- 172.21.15.143:0/1103393054 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5968004240 con 0x7f59500674c0 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59500674c0 msgr2=0x7f5950069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59500674c0 0x7f5950069980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f59680045f0 tx=0x7f5968008040 comp rx=0 tx=0).stop 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 msgr2=0x7f597815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f597815e430 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f596000d5f0 tx=0x7f596000dab0 comp rx=0 tx=0).stop 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 shutdown_connections 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f59500674c0 0x7f5950069980 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f597815ec80 0x7f5978139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5978142470 0x7f597815e430 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 --2- 172.21.15.143:0/1103393054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5978141660 0x7f59780b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 >> 172.21.15.143:0/1103393054 conn(0x7f597806c3f0 msgr2=0x7f59780af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:25.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 shutdown_connections 2024-01-15T18:15:25.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:25.939+0000 7f5980600640 1 -- 172.21.15.143:0/1103393054 wait complete. 2024-01-15T18:15:25.957 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:25.958 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99311 2024-01-15T18:15:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 Processor -- start 2024-01-15T18:15:26.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 -- start start 2024-01-15T18:15:26.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cda70 0x7f0b0c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b0c0cf800 0x7f0b0c0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c15b5b0 con 0x7f0b0c0cf400 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0ccd30 con 0x7f0b0c0cf800 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b13600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0cba70 con 0x7f0b0c0cda70 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b0c0cf800 0x7f0b0c0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0b0c0cf800 0x7f0b0c0b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33210/0 (socket says 172.21.15.143:33210) 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 -- 172.21.15.143:0/2174026676 learned_addr learned my addr 172.21.15.143:0/2174026676 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b10c00640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cda70 0x7f0b0c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 -- 172.21.15.143:0/2174026676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cda70 msgr2=0x7f0b0c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cda70 0x7f0b0c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 -- 172.21.15.143:0/2174026676 >> v1:172.21.15.204:6789/0 conn(0x7f0b0c0cf400 legacy=0x7f0b0c0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b11600640 1 -- 172.21.15.143:0/2174026676 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c0cbd10 con 0x7f0b0c0cf800 2024-01-15T18:15:26.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.103+0000 7f0b10c00640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cda70 0x7f0b0c0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:26.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b11600640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf800 0x7f0b0c0b0740 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009e50 tx=0x7f0b0401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b1814ac479ce25 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b0b400640 1 -- 172.21.15.143:0/2174026676 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040a5860 con 0x7f0b0c0cf800 2024-01-15T18:15:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b0b400640 1 -- 172.21.15.143:0/2174026676 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b040a59c0 con 0x7f0b0c0cf800 2024-01-15T18:15:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b0b400640 1 -- 172.21.15.143:0/2174026676 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040a5d50 con 0x7f0b0c0cf800 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 -- 172.21.15.143:0/2174026676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf800 msgr2=0x7f0b0c0b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf800 0x7f0b0c0b0740 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009e50 tx=0x7f0b0401c9e0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 -- 172.21.15.143:0/2174026676 shutdown_connections 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf800 0x7f0b0c0b0740 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 --2- 172.21.15.143:0/2174026676 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cda70 0x7f0b0c0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 -- 172.21.15.143:0/2174026676 >> 172.21.15.143:0/2174026676 conn(0x7f0b0c0d0e20 msgr2=0x7f0b0c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 -- 172.21.15.143:0/2174026676 shutdown_connections 2024-01-15T18:15:26.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.107+0000 7f0b13600640 1 -- 172.21.15.143:0/2174026676 wait complete. 2024-01-15T18:15:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 Processor -- start 2024-01-15T18:15:26.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 -- start start 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c0cda70 0x7f0b0c06fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 0x7f0b0c14ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cf800 0x7f0b0c155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c153320 con 0x7f0b0c0cf400 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c153600 con 0x7f0b0c0cda70 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1538e0 con 0x7f0b0c0cf800 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 0x7f0b0c14ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c0cda70 0x7f0b0c06fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:26.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b11600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cf800 0x7f0b0c155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:26.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b10c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c0cda70 0x7f0b0c06fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:34910/0 (socket says 172.21.15.143:34910) 2024-01-15T18:15:26.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b10c00640 1 -- 172.21.15.143:0/2947295651 learned_addr learned my addr 172.21.15.143:0/2947295651 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:26.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 0x7f0b0c14ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33214/0 (socket says 172.21.15.143:33214) 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 -- 172.21.15.143:0/2947295651 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cf800 msgr2=0x7f0b0c155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cf800 0x7f0b0c155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 -- 172.21.15.143:0/2947295651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c0cda70 msgr2=0x7f0b0c06fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c0cda70 0x7f0b0c06fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 -- 172.21.15.143:0/2947295651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009b30 con 0x7f0b0c0cf400 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b0be00640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 0x7f0b0c14ad90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0af4000f70 tx=0x7f0af400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40052d0 con 0x7f0b0c0cf400 2024-01-15T18:15:26.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af4005430 con 0x7f0b0c0cf400 2024-01-15T18:15:26.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4003840 con 0x7f0b0c0cf400 2024-01-15T18:15:26.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c153bc0 con 0x7f0b0c0cf400 2024-01-15T18:15:26.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.111+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c14b2d0 con 0x7f0b0c0cf400 2024-01-15T18:15:26.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.115+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c067500 con 0x7f0b0c0cf400 2024-01-15T18:15:26.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.115+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 27) v1 ==== 87231+0+0 (secure 0 0 0) 0x7f0af40039a0 con 0x7f0b0c0cf400 2024-01-15T18:15:26.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.115+0000 7f0b08c00640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae40674c0 0x7f0ae4069980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:26.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.115+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0af4082580 con 0x7f0b0c0cf400 2024-01-15T18:15:26.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.115+0000 7f0b10c00640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae40674c0 0x7f0ae4069980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:26.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.115+0000 7f0b10c00640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae40674c0 0x7f0ae4069980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00bb70 tx=0x7f0afc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:26.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.119+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0af40b6280 con 0x7f0b0c0cf400 2024-01-15T18:15:26.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.299+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99311", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b0c153770 con 0x7f0ae40674c0 2024-01-15T18:15:26.301 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:26.303+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:26.301 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:26.303+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:26.301 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:26.303+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:26.301 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:26.303+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:26.301 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:26.303+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:26.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.947+0000 7f0b08c00640 1 -- 172.21.15.143:0/2947295651 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f0b0c153770 con 0x7f0ae40674c0 2024-01-15T18:15:26.943 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:26.943 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:26.943 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:15:26.943 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:26.944 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.947+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae40674c0 msgr2=0x7f0ae4069980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.947+0000 7f0b13600640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae40674c0 0x7f0ae4069980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00bb70 tx=0x7f0afc008040 comp rx=0 tx=0).stop 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.947+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 msgr2=0x7f0b0c14ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.947+0000 7f0b13600640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 0x7f0b0c14ad90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0af4000f70 tx=0x7f0af400c890 comp rx=0 tx=0).stop 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 shutdown_connections 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0ae40674c0 0x7f0ae4069980 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0b0c0cf800 0x7f0b0c155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0b0c0cf400 0x7f0b0c14ad90 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 --2- 172.21.15.143:0/2947295651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0b0c0cda70 0x7f0b0c06fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:26.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 >> 172.21.15.143:0/2947295651 conn(0x7f0b0c0d0e20 msgr2=0x7f0b0c06f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:26.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 shutdown_connections 2024-01-15T18:15:26.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:26.951+0000 7f0b13600640 1 -- 172.21.15.143:0/2947295651 wait complete. 2024-01-15T18:15:26.972 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:26.972 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99312 2024-01-15T18:15:27.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.115+0000 7f0519a00640 1 Processor -- start 2024-01-15T18:15:27.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.115+0000 7f0519a00640 1 -- start start 2024-01-15T18:15:27.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0519a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f05140ce560 0x7f051415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0519a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415f310 con 0x7f0514071d00 2024-01-15T18:15:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0519a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415f5f0 con 0x7f0514141660 2024-01-15T18:15:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0519a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051415fbb0 con 0x7f05140ce560 2024-01-15T18:15:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0513400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0514071d00 0x7f05140b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56142/0 (socket says 172.21.15.143:56142) 2024-01-15T18:15:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0513400640 1 -- 172.21.15.143:0/3444917922 learned_addr learned my addr 172.21.15.143:0/3444917922 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:27.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512000640 1 --2- 172.21.15.143:0/3444917922 >> v2:172.21.15.204:3300/0 conn(0x7f05140ce560 0x7f051415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822168283 0 0) 0x7f051415f310 con 0x7f0514071d00 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f4003530 con 0x7f0514071d00 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488015847 0 0) 0x7f051415f5f0 con 0x7f0514141660 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051415f310 con 0x7f0514141660 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955096954 0 0) 0x7f04f4003530 con 0x7f0514071d00 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 >> v2:172.21.15.204:3300/0 conn(0x7f05140ce560 msgr2=0x7f051415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 --2- 172.21.15.143:0/3444917922 >> v2:172.21.15.204:3300/0 conn(0x7f05140ce560 0x7f051415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 >> v1:172.21.15.204:6789/0 conn(0x7f0514141660 legacy=0x7f0514141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051415f8d0 con 0x7f0514071d00 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f050c001a00 con 0x7f0514071d00 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512000640 1 --2- 172.21.15.143:0/3444917922 >> v2:172.21.15.204:3300/0 conn(0x7f05140ce560 0x7f051415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 >> v1:172.21.15.143:6789/0 conn(0x7f0514071d00 legacy=0x7f05140b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f4003b10 0x7f04f4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f40047d0 0x7f04f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04f4005190 0x7f04f400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051415fbb0 con 0x7f04f4003b10 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f400c010 con 0x7f04f4005190 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f400c1d0 con 0x7f04f40047d0 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512a00640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f4003b10 0x7f04f4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512000640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f40047d0 0x7f04f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0513400640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04f4005190 0x7f04f400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512a00640 1 -- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f40047d0 msgr2=0x7f04f4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512a00640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f40047d0 0x7f04f4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512a00640 1 -- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04f4005190 msgr2=0x7f04f400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512a00640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04f4005190 0x7f04f400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0512a00640 1 -- 172.21.15.143:0/3444917922 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f4003530 con 0x7f04f4003b10 2024-01-15T18:15:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.119+0000 7f0513400640 1 -- 172.21.15.143:0/3444917922 reap_dead start 2024-01-15T18:15:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0512a00640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f4003b10 0x7f04f4003f10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f05040018d0 tx=0x7f050401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e1a9bd9519c4b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05040052d0 con 0x7f04f4003b10 2024-01-15T18:15:27.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05040a7050 con 0x7f04f4003b10 2024-01-15T18:15:27.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0511600640 1 -- 172.21.15.143:0/3444917922 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504003600 con 0x7f04f4003b10 2024-01-15T18:15:27.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 -- 172.21.15.143:0/3444917922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f4003b10 msgr2=0x7f04f4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f4003b10 0x7f04f4003f10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f05040018d0 tx=0x7f050401c540 comp rx=0 tx=0).stop 2024-01-15T18:15:27.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 -- 172.21.15.143:0/3444917922 shutdown_connections 2024-01-15T18:15:27.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 --2- 172.21.15.143:0/3444917922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04f4003b10 0x7f04f4003f10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 -- 172.21.15.143:0/3444917922 >> 172.21.15.143:0/3444917922 conn(0x7f051406c3f0 msgr2=0x7f051406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:27.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 -- 172.21.15.143:0/3444917922 shutdown_connections 2024-01-15T18:15:27.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.123+0000 7f0519a00640 1 -- 172.21.15.143:0/3444917922 wait complete. 2024-01-15T18:15:27.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 Processor -- start 2024-01-15T18:15:27.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 -- start start 2024-01-15T18:15:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f4003b10 0x7f051413fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 0x7f051413c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514139d70 0x7f051413a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051413c670 con 0x7f051413bcb0 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051413dcf0 con 0x7f0514139d70 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051413ab40 con 0x7f04f4003b10 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 0x7f051413c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 0x7f051413c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33246/0 (socket says 172.21.15.143:33246) 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 -- 172.21.15.143:0/260786931 learned_addr learned my addr 172.21.15.143:0/260786931 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0513400640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514139d70 0x7f051413a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f4003b10 0x7f051413fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 -- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f4003b10 msgr2=0x7f051413fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f4003b10 0x7f051413fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 -- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514139d70 msgr2=0x7f051413a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514139d70 0x7f051413a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 -- 172.21.15.143:0/260786931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140cd150 con 0x7f051413bcb0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0513400640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514139d70 0x7f051413a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0512000640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 0x7f051413c130 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f04fc011c00 tx=0x7f04fc00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc011cf0 con 0x7f051413bcb0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc011e50 con 0x7f051413bcb0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04fc004270 con 0x7f051413bcb0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051413ae20 con 0x7f051413bcb0 2024-01-15T18:15:27.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.127+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051413b3e0 con 0x7f051413bcb0 2024-01-15T18:15:27.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.131+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051413e020 con 0x7f051413bcb0 2024-01-15T18:15:27.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.131+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f04fc0043d0 con 0x7f051413bcb0 2024-01-15T18:15:27.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.131+0000 7f050aa00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04ec067550 0x7f04ec069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.131+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f04fc083460 con 0x7f051413bcb0 2024-01-15T18:15:27.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.131+0000 7f0512a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04ec067550 0x7f04ec069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.131+0000 7f0512a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04ec067550 0x7f04ec069a10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f05040ac000 tx=0x7f05040a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.135+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f04fc0b7280 con 0x7f051413bcb0 2024-01-15T18:15:27.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.315+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99312", "target": ["mon-mgr", ""]}) v1 -- 0x7f05140ccfd0 con 0x7f04ec067550 2024-01-15T18:15:27.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.323+0000 7f050aa00640 1 -- 172.21.15.143:0/260786931 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f05040052b0 con 0x7f04ec067550 2024-01-15T18:15:27.321 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:27.321 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:27.321 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:15:27.321 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:27.321 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04ec067550 msgr2=0x7f04ec069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04ec067550 0x7f04ec069a10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f05040ac000 tx=0x7f05040a6000 comp rx=0 tx=0).stop 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 msgr2=0x7f051413c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 0x7f051413c130 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f04fc011c00 tx=0x7f04fc00f610 comp rx=0 tx=0).stop 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 shutdown_connections 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04ec067550 0x7f04ec069a10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0514139d70 0x7f051413a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f051413bcb0 0x7f051413c130 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 --2- 172.21.15.143:0/260786931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04f4003b10 0x7f051413fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 >> 172.21.15.143:0/260786931 conn(0x7f051406c3f0 msgr2=0x7f05140ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:27.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 shutdown_connections 2024-01-15T18:15:27.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.327+0000 7f0519a00640 1 -- 172.21.15.143:0/260786931 wait complete. 2024-01-15T18:15:27.343 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:27.344 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99313 2024-01-15T18:15:27.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 Processor -- start 2024-01-15T18:15:27.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 -- start start 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff510161880 0x7ff5100adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5100c45b0 0x7ff5100cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100cec40 con 0x7ff51006b5a0 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff510161c60 con 0x7ff5100c45b0 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff516200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5100cef20 con 0x7ff510161880 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5100c45b0 0x7ff5100cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5100c45b0 0x7ff5100cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35120/0 (socket says 172.21.15.143:35120) 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 -- 172.21.15.143:0/2769673856 learned_addr learned my addr 172.21.15.143:0/2769673856 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50f400640 1 --2- 172.21.15.143:0/2769673856 >> v2:172.21.15.204:3300/0 conn(0x7ff510161880 0x7ff5100adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 -- 172.21.15.143:0/2769673856 >> v2:172.21.15.204:3300/0 conn(0x7ff510161880 msgr2=0x7ff5100adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:27.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 --2- 172.21.15.143:0/2769673856 >> v2:172.21.15.204:3300/0 conn(0x7ff510161880 0x7ff5100adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 -- 172.21.15.143:0/2769673856 >> v1:172.21.15.204:6789/0 conn(0x7ff51006b5a0 legacy=0x7ff5100c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.487+0000 7ff50fe00640 1 -- 172.21.15.143:0/2769673856 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100cf200 con 0x7ff5100c45b0 2024-01-15T18:15:27.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff50fe00640 1 --2- 172.21.15.143:0/2769673856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5100c45b0 0x7ff5100cfce0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff510072740 tx=0x7ff50801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b29b37d90e97511 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff50e000640 1 -- 172.21.15.143:0/2769673856 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080a57a0 con 0x7ff5100c45b0 2024-01-15T18:15:27.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff50e000640 1 -- 172.21.15.143:0/2769673856 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5080a5900 con 0x7ff5100c45b0 2024-01-15T18:15:27.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff50e000640 1 -- 172.21.15.143:0/2769673856 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080a5c90 con 0x7ff5100c45b0 2024-01-15T18:15:27.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 -- 172.21.15.143:0/2769673856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5100c45b0 msgr2=0x7ff5100cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 --2- 172.21.15.143:0/2769673856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5100c45b0 0x7ff5100cfce0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff510072740 tx=0x7ff50801c9d0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 -- 172.21.15.143:0/2769673856 shutdown_connections 2024-01-15T18:15:27.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 --2- 172.21.15.143:0/2769673856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5100c45b0 0x7ff5100cfce0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 --2- 172.21.15.143:0/2769673856 >> v2:172.21.15.204:3300/0 conn(0x7ff510161880 0x7ff5100adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 -- 172.21.15.143:0/2769673856 >> 172.21.15.143:0/2769673856 conn(0x7ff5100d0ed0 msgr2=0x7ff5100d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:27.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 -- 172.21.15.143:0/2769673856 shutdown_connections 2024-01-15T18:15:27.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.491+0000 7ff516200640 1 -- 172.21.15.143:0/2769673856 wait complete. 2024-01-15T18:15:27.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 Processor -- start 2024-01-15T18:15:27.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 -- start start 2024-01-15T18:15:27.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff51006b5a0 0x7ff51015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5100c45b0 0x7ff510157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 0x7ff5101584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c5d30 con 0x7ff510161880 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c6010 con 0x7ff51006b5a0 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100c62f0 con 0x7ff5100c45b0 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 0x7ff5101584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff51006b5a0 0x7ff51015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 0x7ff5101584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35122/0 (socket says 172.21.15.143:35122) 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 -- 172.21.15.143:0/56653881 learned_addr learned my addr 172.21.15.143:0/56653881 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:27.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50ea00640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5100c45b0 0x7ff510157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 -- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5100c45b0 msgr2=0x7ff510157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5100c45b0 0x7ff510157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 -- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff51006b5a0 msgr2=0x7ff51015e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff51006b5a0 0x7ff51015e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 -- 172.21.15.143:0/56653881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff508009b30 con 0x7ff510161880 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50f400640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff51006b5a0 0x7ff51015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff50fe00640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 0x7ff5101584f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff5080195e0 tx=0x7ff508019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080b8670 con 0x7ff510161880 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5080b87d0 con 0x7ff510161880 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5080b8bc0 con 0x7ff510161880 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100c65d0 con 0x7ff510161880 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.495+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51015e7e0 con 0x7ff510161880 2024-01-15T18:15:27.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.499+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51015e9d0 con 0x7ff510161880 2024-01-15T18:15:27.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.499+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7ff5080b8d20 con 0x7ff510161880 2024-01-15T18:15:27.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.499+0000 7ff507400640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e8067550 0x7ff4e8069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.499+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff50811ebe0 con 0x7ff510161880 2024-01-15T18:15:27.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.499+0000 7ff50f400640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e8067550 0x7ff4e8069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.499+0000 7ff50f400640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e8067550 0x7ff4e8069a10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff500006aa0 tx=0x7ff500008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.503+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff5080ed920 con 0x7ff510161880 2024-01-15T18:15:27.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.683+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99313", "target": ["mon-mgr", ""]}) v1 -- 0x7ff510067500 con 0x7ff4e8067550 2024-01-15T18:15:27.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff507400640 1 -- 172.21.15.143:0/56653881 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7ff510067500 con 0x7ff4e8067550 2024-01-15T18:15:27.686 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:27.687 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e8067550 msgr2=0x7ff4e8069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e8067550 0x7ff4e8069a10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff500006aa0 tx=0x7ff500008040 comp rx=0 tx=0).stop 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 msgr2=0x7ff5101584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 0x7ff5101584f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff5080195e0 tx=0x7ff508019770 comp rx=0 tx=0).stop 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 shutdown_connections 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e8067550 0x7ff4e8069a10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff510161880 0x7ff5101584f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff5100c45b0 0x7ff510157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 --2- 172.21.15.143:0/56653881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff51006b5a0 0x7ff51015e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 >> 172.21.15.143:0/56653881 conn(0x7ff5100d0ed0 msgr2=0x7ff51006f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 shutdown_connections 2024-01-15T18:15:27.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.691+0000 7ff516200640 1 -- 172.21.15.143:0/56653881 wait complete. 2024-01-15T18:15:27.709 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:27.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99314 2024-01-15T18:15:27.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.851+0000 7f777fe00640 1 Processor -- start 2024-01-15T18:15:27.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.851+0000 7f777fe00640 1 -- start start 2024-01-15T18:15:27.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 0x7f77780cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f77780ce520 0x7f7778071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77780ceb50 con 0x7f77780720f0 2024-01-15T18:15:27.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7778072580 con 0x7f77780ce520 2024-01-15T18:15:27.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777815c110 con 0x7f7778143780 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 0x7f77780cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777ca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f77780ce520 0x7f7778071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777de00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f77780720f0 0x7f777815a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43842/0 (socket says 172.21.15.143:43842) 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 0x7f77780cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:37580/0 (socket says 172.21.15.143:37580) 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777de00640 1 -- 172.21.15.143:0/35197128 learned_addr learned my addr 172.21.15.143:0/35197128 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f7777e00640 1 -- 172.21.15.143:0/35197128 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733018280 0 0) 0x7f77700014a0 con 0x7f77780720f0 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f7777e00640 1 -- 172.21.15.143:0/35197128 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7758003530 con 0x7f77780720f0 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777d400640 1 -- 172.21.15.143:0/35197128 >> v2:172.21.15.204:3300/0 conn(0x7f77780ce520 msgr2=0x7f7778071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777d400640 1 --2- 172.21.15.143:0/35197128 >> v2:172.21.15.204:3300/0 conn(0x7f77780ce520 0x7f7778071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777d400640 1 -- 172.21.15.143:0/35197128 >> v1:172.21.15.204:6789/0 conn(0x7f77780720f0 legacy=0x7f777815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777d400640 1 -- 172.21.15.143:0/35197128 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777815b7e0 con 0x7f7778143780 2024-01-15T18:15:27.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.855+0000 7f777ca00640 1 --2- 172.21.15.143:0/35197128 >> v2:172.21.15.204:3300/0 conn(0x7f77780ce520 0x7f7778071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777d400640 1 --2- 172.21.15.143:0/35197128 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 0x7f77780cdfe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77680097c0 tx=0x7f776801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d77c9d9d0ec4d7d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f7777e00640 1 -- 172.21.15.143:0/35197128 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680ad640 con 0x7f7778143780 2024-01-15T18:15:27.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f7777e00640 1 -- 172.21.15.143:0/35197128 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77680ad7a0 con 0x7f7778143780 2024-01-15T18:15:27.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f7777e00640 1 -- 172.21.15.143:0/35197128 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680adb30 con 0x7f7778143780 2024-01-15T18:15:27.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 -- 172.21.15.143:0/35197128 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 msgr2=0x7f77780cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 --2- 172.21.15.143:0/35197128 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 0x7f77780cdfe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f77680097c0 tx=0x7f776801c480 comp rx=0 tx=0).stop 2024-01-15T18:15:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 -- 172.21.15.143:0/35197128 shutdown_connections 2024-01-15T18:15:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 --2- 172.21.15.143:0/35197128 >> v2:172.21.15.204:3300/0 conn(0x7f77780ce520 0x7f7778071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 --2- 172.21.15.143:0/35197128 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7778143780 0x7f77780cdfe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 -- 172.21.15.143:0/35197128 >> 172.21.15.143:0/35197128 conn(0x7f777806c3f0 msgr2=0x7f777806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 -- 172.21.15.143:0/35197128 shutdown_connections 2024-01-15T18:15:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.859+0000 7f777fe00640 1 -- 172.21.15.143:0/35197128 wait complete. 2024-01-15T18:15:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 Processor -- start 2024-01-15T18:15:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- start start 2024-01-15T18:15:27.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 0x7f7778132690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77780ce520 0x7f7778136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778143780 0x7f7778137250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778139b10 con 0x7f77780720f0 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77781554f0 con 0x7f7778143780 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77781548e0 con 0x7f77780ce520 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 0x7f7778132690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 0x7f7778132690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35128/0 (socket says 172.21.15.143:35128) 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 -- 172.21.15.143:0/2397955999 learned_addr learned my addr 172.21.15.143:0/2397955999 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777de00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778143780 0x7f7778137250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777ca00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77780ce520 0x7f7778136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 -- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77780ce520 msgr2=0x7f7778136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77780ce520 0x7f7778136d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 -- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778143780 msgr2=0x7f7778137250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778143780 0x7f7778137250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 -- 172.21.15.143:0/2397955999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776000b6c0 con 0x7f77780720f0 2024-01-15T18:15:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777d400640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 0x7f7778132690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f77680b0040 tx=0x7f7768019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680b6610 con 0x7f77780720f0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77680a6040 con 0x7f77780720f0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7768009420 con 0x7f77780720f0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680b69b0 con 0x7f77780720f0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7778154ea0 con 0x7f77780720f0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.863+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7778067500 con 0x7f77780720f0 2024-01-15T18:15:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.867+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f77680b4040 con 0x7f77780720f0 2024-01-15T18:15:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.867+0000 7f7775600640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7750067550 0x7f7750069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.867+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f776811ce60 con 0x7f77780720f0 2024-01-15T18:15:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.867+0000 7f777ca00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7750067550 0x7f7750069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:27.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.867+0000 7f777ca00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7750067550 0x7f7750069a10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f77780b4320 tx=0x7f776000b3a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:27.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:27.871+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f77680ebaf0 con 0x7f77780720f0 2024-01-15T18:15:28.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.055+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7f7778155630 con 0x7f7750067550 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.063+0000 7f7775600640 1 -- 172.21.15.143:0/2397955999 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f776000bb70 con 0x7f7750067550 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:28.060 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:28.061 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:28.061 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:28.061 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:28.061 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:28.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7750067550 msgr2=0x7f7750069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7750067550 0x7f7750069a10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f77780b4320 tx=0x7f776000b3a0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 msgr2=0x7f7778132690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 0x7f7778132690 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f77680b0040 tx=0x7f7768019220 comp rx=0 tx=0).stop 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 shutdown_connections 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7750067550 0x7f7750069a10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7778143780 0x7f7778137250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f77780ce520 0x7f7778136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 --2- 172.21.15.143:0/2397955999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77780720f0 0x7f7778132690 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 >> 172.21.15.143:0/2397955999 conn(0x7f777806c3f0 msgr2=0x7f777806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 shutdown_connections 2024-01-15T18:15:28.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.067+0000 7f777fe00640 1 -- 172.21.15.143:0/2397955999 wait complete. 2024-01-15T18:15:28.082 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:28.083 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99315 2024-01-15T18:15:28.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 Processor -- start 2024-01-15T18:15:28.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 -- start start 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8264141660 0x7f82640ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8264071d00 0x7f8264072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8264072650 con 0x7f826415ec80 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826415fbb0 con 0x7f8264071d00 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f826b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826415f130 con 0x7f8264141660 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.227+0000 7f8269800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8264071d00 0x7f8264072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8264071d00 0x7f8264072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35144/0 (socket says 172.21.15.143:35144) 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 -- 172.21.15.143:0/582354327 learned_addr learned my addr 172.21.15.143:0/582354327 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8268e00640 1 --2- 172.21.15.143:0/582354327 >> v2:172.21.15.204:3300/0 conn(0x7f8264141660 0x7f82640ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 -- 172.21.15.143:0/582354327 >> v2:172.21.15.204:3300/0 conn(0x7f8264141660 msgr2=0x7f82640ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 --2- 172.21.15.143:0/582354327 >> v2:172.21.15.204:3300/0 conn(0x7f8264141660 0x7f82640ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 -- 172.21.15.143:0/582354327 >> v1:172.21.15.204:6789/0 conn(0x7f826415ec80 legacy=0x7f8264141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 -- 172.21.15.143:0/582354327 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826415f310 con 0x7f8264071d00 2024-01-15T18:15:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8269800640 1 --2- 172.21.15.143:0/582354327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f8264072110 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f82640b1aa0 tx=0x7f825c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dbe11909c29c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8263400640 1 -- 172.21.15.143:0/582354327 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0a5860 con 0x7f8264071d00 2024-01-15T18:15:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8263400640 1 -- 172.21.15.143:0/582354327 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c0a59c0 con 0x7f8264071d00 2024-01-15T18:15:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.231+0000 7f8263400640 1 -- 172.21.15.143:0/582354327 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0a5d50 con 0x7f8264071d00 2024-01-15T18:15:28.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- 172.21.15.143:0/582354327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 msgr2=0x7f8264072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 --2- 172.21.15.143:0/582354327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f8264072110 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f82640b1aa0 tx=0x7f825c01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- 172.21.15.143:0/582354327 shutdown_connections 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 --2- 172.21.15.143:0/582354327 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f8264072110 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 --2- 172.21.15.143:0/582354327 >> v2:172.21.15.204:3300/0 conn(0x7f8264141660 0x7f82640ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- 172.21.15.143:0/582354327 >> 172.21.15.143:0/582354327 conn(0x7f826406c3f0 msgr2=0x7f826406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- 172.21.15.143:0/582354327 shutdown_connections 2024-01-15T18:15:28.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- 172.21.15.143:0/582354327 wait complete. 2024-01-15T18:15:28.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 Processor -- start 2024-01-15T18:15:28.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- start start 2024-01-15T18:15:28.233 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f826415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8264141660 0x7f8264139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f826415ec80 0x7f826413e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640b2230 con 0x7f8264071d00 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82641396b0 con 0x7f826415ec80 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f826b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8264138ce0 con 0x7f8264141660 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f8268e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f826415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f8268e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f826415ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35154/0 (socket says 172.21.15.143:35154) 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f8268e00640 1 -- 172.21.15.143:0/753497920 learned_addr learned my addr 172.21.15.143:0/753497920 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f8269800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f826415ec80 0x7f826413e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.235+0000 7f8263e00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8264141660 0x7f8264139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8268e00640 1 -- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8264141660 msgr2=0x7f8264139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8268e00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8264141660 0x7f8264139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8268e00640 1 -- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f826415ec80 msgr2=0x7f826413e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8268e00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f826415ec80 0x7f826413e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8268e00640 1 -- 172.21.15.143:0/753497920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c009b30 con 0x7f8264071d00 2024-01-15T18:15:28.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8268e00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f826415ab50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f825400d830 tx=0x7f825400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400b660 con 0x7f8264071d00 2024-01-15T18:15:28.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400b7c0 con 0x7f8264071d00 2024-01-15T18:15:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400bb30 con 0x7f8264071d00 2024-01-15T18:15:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82641380d0 con 0x7f8264071d00 2024-01-15T18:15:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8264138690 con 0x7f8264071d00 2024-01-15T18:15:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8264067500 con 0x7f8264071d00 2024-01-15T18:15:28.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f825401e460 con 0x7f8264071d00 2024-01-15T18:15:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.239+0000 7f8260c00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f823c067550 0x7f823c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.243+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8254083430 con 0x7f8264071d00 2024-01-15T18:15:28.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.243+0000 7f8263e00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f823c067550 0x7f823c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.243+0000 7f8263e00640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f823c067550 0x7f823c069a10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8264140e20 tx=0x7f824c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.243+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82540b7280 con 0x7f8264071d00 2024-01-15T18:15:28.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.427+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99315", "target": ["mon-mgr", ""]}) v1 -- 0x7f8264142640 con 0x7f823c067550 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.435+0000 7f8260c00640 1 -- 172.21.15.143:0/753497920 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f824c001510 con 0x7f823c067550 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:28.432 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:28.433 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f823c067550 msgr2=0x7f823c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f823c067550 0x7f823c069a10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8264140e20 tx=0x7f824c009040 comp rx=0 tx=0).stop 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 msgr2=0x7f826415ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f826415ab50 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f825400d830 tx=0x7f825400dcf0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 shutdown_connections 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f823c067550 0x7f823c069a10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f826415ec80 0x7f826413e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8264141660 0x7f8264139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 --2- 172.21.15.143:0/753497920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8264071d00 0x7f826415ab50 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 >> 172.21.15.143:0/753497920 conn(0x7f826406c3f0 msgr2=0x7f82640ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 shutdown_connections 2024-01-15T18:15:28.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.439+0000 7f826b800640 1 -- 172.21.15.143:0/753497920 wait complete. 2024-01-15T18:15:28.454 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:28.454 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99316 2024-01-15T18:15:28.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb470000640 1 Processor -- start 2024-01-15T18:15:28.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb470000640 1 -- start start 2024-01-15T18:15:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb470000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4680ce690 0x7fb4680ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb470000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb468070bf0 con 0x7fb4680b3bf0 2024-01-15T18:15:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb470000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4680b4920 con 0x7fb4680b1da0 2024-01-15T18:15:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb470000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46815dba0 con 0x7fb4680ce690 2024-01-15T18:15:28.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb46d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4680ce690 0x7fb4680ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb46d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4680ce690 0x7fb4680ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35170/0 (socket says 172.21.15.143:35170) 2024-01-15T18:15:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb46d600640 1 -- 172.21.15.143:0/2192429543 learned_addr learned my addr 172.21.15.143:0/2192429543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb46d600640 1 -- 172.21.15.143:0/2192429543 >> v1:172.21.15.204:6789/0 conn(0x7fb4680b1da0 legacy=0x7fb4680b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb46d600640 1 -- 172.21.15.143:0/2192429543 >> v1:172.21.15.143:6789/0 conn(0x7fb4680b3bf0 legacy=0x7fb4680b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.599+0000 7fb46d600640 1 -- 172.21.15.143:0/2192429543 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4680b4c00 con 0x7fb4680ce690 2024-01-15T18:15:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb46d600640 1 --2- 172.21.15.143:0/2192429543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680ce690 0x7fb4680ac8e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb458009870 tx=0x7fb45801cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32097e41525d8c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb467e00640 1 -- 172.21.15.143:0/2192429543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580aa650 con 0x7fb4680ce690 2024-01-15T18:15:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb467e00640 1 -- 172.21.15.143:0/2192429543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580aa7b0 con 0x7fb4680ce690 2024-01-15T18:15:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb467e00640 1 -- 172.21.15.143:0/2192429543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580aab40 con 0x7fb4680ce690 2024-01-15T18:15:28.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 -- 172.21.15.143:0/2192429543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680ce690 msgr2=0x7fb4680ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 --2- 172.21.15.143:0/2192429543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680ce690 0x7fb4680ac8e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb458009870 tx=0x7fb45801cf30 comp rx=0 tx=0).stop 2024-01-15T18:15:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 -- 172.21.15.143:0/2192429543 shutdown_connections 2024-01-15T18:15:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 --2- 172.21.15.143:0/2192429543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680ce690 0x7fb4680ac8e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 -- 172.21.15.143:0/2192429543 >> 172.21.15.143:0/2192429543 conn(0x7fb46806c3f0 msgr2=0x7fb46806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 -- 172.21.15.143:0/2192429543 shutdown_connections 2024-01-15T18:15:28.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.603+0000 7fb470000640 1 -- 172.21.15.143:0/2192429543 wait complete. 2024-01-15T18:15:28.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 Processor -- start 2024-01-15T18:15:28.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 -- start start 2024-01-15T18:15:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 0x7fb468145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4680b3bf0 0x7fb468159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4680ce690 0x7fb468139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb468146250 con 0x7fb4680b1da0 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb468146530 con 0x7fb4680ce690 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468146810 con 0x7fb4680b3bf0 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 0x7fb468145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 0x7fb468145950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35186/0 (socket says 172.21.15.143:35186) 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 -- 172.21.15.143:0/3639539236 learned_addr learned my addr 172.21.15.143:0/3639539236 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46e000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4680ce690 0x7fb468139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46cc00640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4680b3bf0 0x7fb468159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 -- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4680b3bf0 msgr2=0x7fb468159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4680b3bf0 0x7fb468159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 -- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4680ce690 msgr2=0x7fb468139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4680ce690 0x7fb468139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 -- 172.21.15.143:0/3639539236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580094d0 con 0x7fb4680b1da0 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46cc00640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4680b3bf0 0x7fb468159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46e000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4680ce690 0x7fb468139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:28.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb46d600640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 0x7fb468145950 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ac040 tx=0x7fb45801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b38c0 con 0x7fb4680b1da0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580b3a20 con 0x7fb4680b1da0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb468146af0 con 0x7fb4680b1da0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580b3dd0 con 0x7fb4680b1da0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.607+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46815a230 con 0x7fb4680b1da0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.611+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4680ae650 con 0x7fb4680b1da0 2024-01-15T18:15:28.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.611+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7fb4580c6390 con 0x7fb4680b1da0 2024-01-15T18:15:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.611+0000 7fb465600640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067550 0x7fb440069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.611+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb45811dcb0 con 0x7fb4680b1da0 2024-01-15T18:15:28.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.611+0000 7fb46cc00640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067550 0x7fb440069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.611+0000 7fb46cc00640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067550 0x7fb440069a10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb46815ebf0 tx=0x7fb450009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.615+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4580ec6c0 con 0x7fb4680b1da0 2024-01-15T18:15:28.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.799+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99316", "target": ["mon-mgr", ""]}) v1 -- 0x7fb46815da30 con 0x7fb440067550 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.811+0000 7fb465600640 1 -- 172.21.15.143:0/3639539236 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fb46815da30 con 0x7fb440067550 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:28.809 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:28.810 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:28.810 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:28.810 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:28.810 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:28.810 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067550 msgr2=0x7fb440069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067550 0x7fb440069a10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb46815ebf0 tx=0x7fb450009040 comp rx=0 tx=0).stop 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 msgr2=0x7fb468145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 0x7fb468145950 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fb4580ac040 tx=0x7fb45801ad10 comp rx=0 tx=0).stop 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 shutdown_connections 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067550 0x7fb440069a10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4680ce690 0x7fb468139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4680b3bf0 0x7fb468159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 --2- 172.21.15.143:0/3639539236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4680b1da0 0x7fb468145950 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 >> 172.21.15.143:0/3639539236 conn(0x7fb46806c3f0 msgr2=0x7fb46806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 shutdown_connections 2024-01-15T18:15:28.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.815+0000 7fb470000640 1 -- 172.21.15.143:0/3639539236 wait complete. 2024-01-15T18:15:28.831 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:28.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99317 2024-01-15T18:15:28.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb3a00640 1 Processor -- start 2024-01-15T18:15:28.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb3a00640 1 -- start start 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9bac06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb3a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0cffd0 con 0x7f9bac1607c0 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bac0d02b0 con 0x7f9bac0cbf50 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0d1ce0 con 0x7f9bac0cfb30 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9bac1607c0 0x7f9bac06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48234/0 (socket says 172.21.15.143:48234) 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1000640 1 -- 172.21.15.143:0/785455991 learned_addr learned my addr 172.21.15.143:0/785455991 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9babe00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9bac06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159653566 0 0) 0x7f9bac0cffd0 con 0x7f9bac1607c0 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b8c003530 con 0x7f9bac1607c0 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962758278 0 0) 0x7f9bac0d02b0 con 0x7f9bac0cbf50 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bac0cffd0 con 0x7f9bac0cbf50 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4035376342 0 0) 0x7f9b8c003530 con 0x7f9bac1607c0 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 msgr2=0x7f9bac06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:28.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9bac06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 >> v1:172.21.15.204:6789/0 conn(0x7f9bac0cbf50 legacy=0x7f9bac0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac0cb470 con 0x7f9bac1607c0 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f9b9c001680 con 0x7f9bac1607c0 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 >> v1:172.21.15.143:6789/0 conn(0x7f9bac1607c0 legacy=0x7f9bac06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c003b40 0x7f9b8c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9b8c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9b8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac0d1ce0 con 0x7f9b8c004d10 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c00ba40 con 0x7f9b8c003b40 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c00bc00 con 0x7f9bac0cfb30 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1000640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c003b40 0x7f9b8c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1a00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9b8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9babe00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9b8c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1a00640 1 -- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 msgr2=0x7f9b8c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1a00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9b8c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1a00640 1 -- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c003b40 msgr2=0x7f9b8c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1a00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b8c003b40 0x7f9b8c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9bb1a00640 1 -- 172.21.15.143:0/785455991 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c003530 con 0x7f9b8c004d10 2024-01-15T18:15:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.979+0000 7f9babe00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac0cfb30 0x7f9b8c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bb1a00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9b8c005190 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001fd0 tx=0x7f9ba401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be39321b412f5ec3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4003b80 con 0x7f9b8c004d10 2024-01-15T18:15:28.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba4003d00 con 0x7f9b8c004d10 2024-01-15T18:15:28.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bab400640 1 -- 172.21.15.143:0/785455991 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4005310 con 0x7f9b8c004d10 2024-01-15T18:15:28.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bb3a00640 1 -- 172.21.15.143:0/785455991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 msgr2=0x7f9b8c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/785455991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9b8c005190 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001fd0 tx=0x7f9ba401c8e0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bb1a00640 1 -- 172.21.15.143:0/785455991 reap_dead start 2024-01-15T18:15:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bb3a00640 1 -- 172.21.15.143:0/785455991 shutdown_connections 2024-01-15T18:15:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.983+0000 7f9bb3a00640 1 -- 172.21.15.143:0/785455991 >> 172.21.15.143:0/785455991 conn(0x7f9bac0d0e20 msgr2=0x7f9bac0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 -- 172.21.15.143:0/785455991 shutdown_connections 2024-01-15T18:15:28.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 -- 172.21.15.143:0/785455991 wait complete. 2024-01-15T18:15:28.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 Processor -- start 2024-01-15T18:15:28.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 -- start start 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9bac1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac155bf0 0x7f9bac1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bac153d20 0x7f9bac1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac154710 con 0x7f9b8c004d10 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac146d00 con 0x7f9bac153d20 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac1560d0 con 0x7f9bac155bf0 2024-01-15T18:15:28.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9bac1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9bac1f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35212/0 (socket says 172.21.15.143:35212) 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 -- 172.21.15.143:0/2938559025 learned_addr learned my addr 172.21.15.143:0/2938559025 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9babe00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac155bf0 0x7f9bac1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bac153d20 0x7f9bac1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 -- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac155bf0 msgr2=0x7f9bac1467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac155bf0 0x7f9bac1467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 -- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bac153d20 msgr2=0x7f9bac1541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bac153d20 0x7f9bac1541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 -- 172.21.15.143:0/2938559025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac1563b0 con 0x7f9b8c004d10 2024-01-15T18:15:28.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.987+0000 7f9bb1000640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9bac1f5c60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c007310 tx=0x7f9b9c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c0015e0 con 0x7f9b8c004d10 2024-01-15T18:15:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c00f9c0 con 0x7f9b8c004d10 2024-01-15T18:15:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00fd70 con 0x7f9b8c004d10 2024-01-15T18:15:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac156690 con 0x7f9b8c004d10 2024-01-15T18:15:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac156c50 con 0x7f9b8c004d10 2024-01-15T18:15:28.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac06d4b0 con 0x7f9b8c004d10 2024-01-15T18:15:28.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f9b9c01d460 con 0x7f9b8c004d10 2024-01-15T18:15:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.991+0000 7f9ba8c00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b84067550 0x7f9b84069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:28.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.995+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9b9c082f00 con 0x7f9b8c004d10 2024-01-15T18:15:28.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.995+0000 7f9babe00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b84067550 0x7f9b84069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:28.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.995+0000 7f9babe00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b84067550 0x7f9b84069a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9bac06c3f0 tx=0x7f9b94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:28.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:28.999+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b9c051c40 con 0x7f9b8c004d10 2024-01-15T18:15:29.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.183+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99317", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bac0ce6b0 con 0x7f9b84067550 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.191+0000 7f9ba8c00640 1 -- 172.21.15.143:0/2938559025 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f9b94014ac0 con 0x7f9b84067550 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:29.188 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:29.189 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:29.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b84067550 msgr2=0x7f9b84069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b84067550 0x7f9b84069a10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9bac06c3f0 tx=0x7f9b94008040 comp rx=0 tx=0).stop 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 msgr2=0x7f9bac1f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9bac1f5c60 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c007310 tx=0x7f9b9c0077d0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 shutdown_connections 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b84067550 0x7f9b84069a10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bac153d20 0x7f9bac1541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bac155bf0 0x7f9bac1467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 --2- 172.21.15.143:0/2938559025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b8c004d10 0x7f9bac1f5c60 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 >> 172.21.15.143:0/2938559025 conn(0x7f9bac0d0e20 msgr2=0x7f9bac06b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 shutdown_connections 2024-01-15T18:15:29.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.195+0000 7f9bb3a00640 1 -- 172.21.15.143:0/2938559025 wait complete. 2024-01-15T18:15:29.210 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:29.210 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99318 2024-01-15T18:15:29.321 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:29.324 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:29.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 Processor -- start 2024-01-15T18:15:29.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 -- start start 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1048071140 0x7f10480ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f104815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10480ce810 con 0x7f104815b110 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480cc220 con 0x7f1048071140 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104815a4a0 con 0x7f1048071520 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f104815b110 0x7f104815da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48244/0 (socket says 172.21.15.143:48244) 2024-01-15T18:15:29.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104de00640 1 -- 172.21.15.143:0/284500835 learned_addr learned my addr 172.21.15.143:0/284500835 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:29.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 --2- 172.21.15.143:0/284500835 >> v2:172.21.15.204:3300/0 conn(0x7f1048071140 0x7f10480ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104ca00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f104815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214909622 0 0) 0x7f10480ce810 con 0x7f104815b110 2024-01-15T18:15:29.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1028003530 con 0x7f104815b110 2024-01-15T18:15:29.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3581958050 0 0) 0x7f1028003530 con 0x7f104815b110 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 msgr2=0x7f104815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f104815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 >> v2:172.21.15.204:3300/0 conn(0x7f1048071140 msgr2=0x7f10480ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 --2- 172.21.15.143:0/284500835 >> v2:172.21.15.204:3300/0 conn(0x7f1048071140 0x7f10480ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480cb980 con 0x7f104815b110 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f1040001a00 con 0x7f104815b110 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104ca00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f104815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 >> v1:172.21.15.143:6789/0 conn(0x7f104815b110 legacy=0x7f104815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1028003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280047d0 0x7f1028004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f1028005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480cc220 con 0x7f1028003980 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104815a4a0 con 0x7f10280047d0 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102800dba0 con 0x7f1048071520 2024-01-15T18:15:29.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1028003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104de00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f1028005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104ca00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280047d0 0x7f1028004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 -- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 msgr2=0x7f1028005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1048071520 0x7f1028005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 -- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280047d0 msgr2=0x7f1028004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280047d0 0x7f1028004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.355+0000 7f104d400640 1 -- 172.21.15.143:0/284500835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030009420 con 0x7f1028003980 2024-01-15T18:15:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104d400640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1028003da0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1038009ae0 tx=0x7f103801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d421c2649ba93d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380a6480 con 0x7f1028003980 2024-01-15T18:15:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10380a65e0 con 0x7f1028003980 2024-01-15T18:15:29.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f1047e00640 1 -- 172.21.15.143:0/284500835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380a6990 con 0x7f1028003980 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104fe00640 1 -- 172.21.15.143:0/284500835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 msgr2=0x7f1028003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104fe00640 1 --2- 172.21.15.143:0/284500835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1028003da0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1038009ae0 tx=0x7f103801ca60 comp rx=0 tx=0).stop 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104de00640 1 -- 172.21.15.143:0/284500835 reap_dead start 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104fe00640 1 -- 172.21.15.143:0/284500835 shutdown_connections 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104fe00640 1 -- 172.21.15.143:0/284500835 >> 172.21.15.143:0/284500835 conn(0x7f10480cfd20 msgr2=0x7f10480d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104fe00640 1 -- 172.21.15.143:0/284500835 shutdown_connections 2024-01-15T18:15:29.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.359+0000 7f104fe00640 1 -- 172.21.15.143:0/284500835 wait complete. 2024-01-15T18:15:29.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 Processor -- start 2024-01-15T18:15:29.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 -- start start 2024-01-15T18:15:29.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1048149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10480c4910 0x7f10480c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10480c96c0 0x7f10480c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480c6070 con 0x7f1028003980 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104814a240 con 0x7f10480c96c0 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10480ca080 con 0x7f10480c4910 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1048149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1048149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35232/0 (socket says 172.21.15.143:35232) 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 -- 172.21.15.143:0/3352864198 learned_addr learned my addr 172.21.15.143:0/3352864198 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104de00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10480c96c0 0x7f10480c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104ca00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10480c4910 0x7f10480c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 -- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10480c4910 msgr2=0x7f10480c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10480c4910 0x7f10480c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 -- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10480c96c0 msgr2=0x7f10480c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10480c96c0 0x7f10480c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 -- 172.21.15.143:0/3352864198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10480c53f0 con 0x7f1028003980 2024-01-15T18:15:29.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104d400640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1048149d00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1038009ae0 tx=0x7f1028005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:29.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380b5070 con 0x7f1028003980 2024-01-15T18:15:29.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103800f000 con 0x7f1028003980 2024-01-15T18:15:29.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10480c56d0 con 0x7f1028003980 2024-01-15T18:15:29.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380a7460 con 0x7f1028003980 2024-01-15T18:15:29.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.363+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10480c5c90 con 0x7f1028003980 2024-01-15T18:15:29.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.367+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048067500 con 0x7f1028003980 2024-01-15T18:15:29.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.367+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f10380b9070 con 0x7f1028003980 2024-01-15T18:15:29.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.367+0000 7f1045600640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1020067550 0x7f1020069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.367+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f103811cce0 con 0x7f1028003980 2024-01-15T18:15:29.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.367+0000 7f104ca00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1020067550 0x7f1020069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.367+0000 7f104ca00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1020067550 0x7f1020069a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f10480c6a30 tx=0x7f1030008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:29.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.371+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f10380eb800 con 0x7f1028003980 2024-01-15T18:15:29.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.555+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99318", "target": ["mon-mgr", ""]}) v1 -- 0x7f1048158970 con 0x7f1020067550 2024-01-15T18:15:29.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.563+0000 7f1045600640 1 -- 172.21.15.143:0/3352864198 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f1048158970 con 0x7f1020067550 2024-01-15T18:15:29.559 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:29.560 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:29.561 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1020067550 msgr2=0x7f1020069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1020067550 0x7f1020069a10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f10480c6a30 tx=0x7f1030008040 comp rx=0 tx=0).stop 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 msgr2=0x7f1048149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1048149d00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1038009ae0 tx=0x7f1028005800 comp rx=0 tx=0).stop 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 shutdown_connections 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1020067550 0x7f1020069a10 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10480c96c0 0x7f10480c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10480c4910 0x7f10480c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 --2- 172.21.15.143:0/3352864198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1028003980 0x7f1048149d00 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 >> 172.21.15.143:0/3352864198 conn(0x7f10480cfd20 msgr2=0x7f10480d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:29.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 shutdown_connections 2024-01-15T18:15:29.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.567+0000 7f104fe00640 1 -- 172.21.15.143:0/3352864198 wait complete. 2024-01-15T18:15:29.582 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:29.582 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone99319 2024-01-15T18:15:29.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.723+0000 7f520ca00640 1 Processor -- start 2024-01-15T18:15:29.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.723+0000 7f520ca00640 1 -- start start 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5204143780 0x7f52040cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52040722c0 0x7f520415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520ca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52040ceb50 con 0x7f52040ce520 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520ca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520415c110 con 0x7f52040722c0 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520415b7e0 con 0x7f5204143780 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52040722c0 0x7f520415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52040722c0 0x7f520415c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35242/0 (socket says 172.21.15.143:35242) 2024-01-15T18:15:29.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 -- 172.21.15.143:0/1669556967 learned_addr learned my addr 172.21.15.143:0/1669556967 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:29.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520a000640 1 --2- 172.21.15.143:0/1669556967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5204143780 0x7f52040cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 -- 172.21.15.143:0/1669556967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5204143780 msgr2=0x7f52040cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:29.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 --2- 172.21.15.143:0/1669556967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5204143780 0x7f52040cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 -- 172.21.15.143:0/1669556967 >> v1:172.21.15.204:6789/0 conn(0x7f52040ce520 legacy=0x7f5204071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.727+0000 7f520aa00640 1 -- 172.21.15.143:0/1669556967 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520415bac0 con 0x7f52040722c0 2024-01-15T18:15:29.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520aa00640 1 --2- 172.21.15.143:0/1669556967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040722c0 0x7f520415c5a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00d150 tx=0x7f51fc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfe8a4a8c59d21b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:29.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f5208c00640 1 -- 172.21.15.143:0/1669556967 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00b2d0 con 0x7f52040722c0 2024-01-15T18:15:29.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f5208c00640 1 -- 172.21.15.143:0/1669556967 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc00b430 con 0x7f52040722c0 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f5208c00640 1 -- 172.21.15.143:0/1669556967 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0a7c10 con 0x7f52040722c0 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 -- 172.21.15.143:0/1669556967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040722c0 msgr2=0x7f520415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 --2- 172.21.15.143:0/1669556967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040722c0 0x7f520415c5a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00d150 tx=0x7f51fc01ec10 comp rx=0 tx=0).stop 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 -- 172.21.15.143:0/1669556967 shutdown_connections 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 --2- 172.21.15.143:0/1669556967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040722c0 0x7f520415c5a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 --2- 172.21.15.143:0/1669556967 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5204143780 0x7f52040cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 -- 172.21.15.143:0/1669556967 >> 172.21.15.143:0/1669556967 conn(0x7f520406c3f0 msgr2=0x7f520406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 -- 172.21.15.143:0/1669556967 shutdown_connections 2024-01-15T18:15:29.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 -- 172.21.15.143:0/1669556967 wait complete. 2024-01-15T18:15:29.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 Processor -- start 2024-01-15T18:15:29.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.731+0000 7f520ca00640 1 -- start start 2024-01-15T18:15:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52040722c0 0x7f5204131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 0x7f5204158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5204143780 0x7f52040b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040ce350 con 0x7f52040ce520 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5204159480 con 0x7f5204143780 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5204159980 con 0x7f52040722c0 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 0x7f5204158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 0x7f5204158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35258/0 (socket says 172.21.15.143:35258) 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 -- 172.21.15.143:0/847231586 learned_addr learned my addr 172.21.15.143:0/847231586 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520a000640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52040722c0 0x7f5204131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520aa00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5204143780 0x7f52040b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 -- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52040722c0 msgr2=0x7f5204131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52040722c0 0x7f5204131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 -- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5204143780 msgr2=0x7f52040b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5204143780 0x7f52040b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 -- 172.21.15.143:0/847231586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc00bd40 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5209600640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 0x7f5204158f40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c560 tx=0x7f51ec00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec005220 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec005380 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec007270 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52040ce070 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52041317a0 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.735+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204131320 con 0x7f52040ce520 2024-01-15T18:15:29.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.739+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f51ec0073d0 con 0x7f52040ce520 2024-01-15T18:15:29.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.739+0000 7f5202000640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51dc067550 0x7f51dc069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:29.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.739+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f51ec082680 con 0x7f52040ce520 2024-01-15T18:15:29.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.739+0000 7f520a000640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51dc067550 0x7f51dc069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:29.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.739+0000 7f520a000640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51dc067550 0x7f51dc069a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5204136860 tx=0x7f51f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:29.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.743+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5204131320 con 0x7f52040ce520 2024-01-15T18:15:29.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.923+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone99319", "target": ["mon-mgr", ""]}) v1 -- 0x7f52040ceb50 con 0x7f51dc067550 2024-01-15T18:15:29.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f5202000640 1 -- 172.21.15.143:0/847231586 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f52040ceb50 con 0x7f51dc067550 2024-01-15T18:15:29.929 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:29.930 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51dc067550 msgr2=0x7f51dc069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51dc067550 0x7f51dc069a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5204136860 tx=0x7f51f4008040 comp rx=0 tx=0).stop 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 msgr2=0x7f5204158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 0x7f5204158f40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c560 tx=0x7f51ec00ca20 comp rx=0 tx=0).stop 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 shutdown_connections 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51dc067550 0x7f51dc069a10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5204143780 0x7f52040b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52040ce520 0x7f5204158f40 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 --2- 172.21.15.143:0/847231586 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f52040722c0 0x7f5204131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:29.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 >> 172.21.15.143:0/847231586 conn(0x7f520406c3f0 msgr2=0x7f52040af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:29.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 shutdown_connections 2024-01-15T18:15:29.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:29.935+0000 7f520ca00640 1 -- 172.21.15.143:0/847231586 wait complete. 2024-01-15T18:15:29.952 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:29.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993110 2024-01-15T18:15:30.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 Processor -- start 2024-01-15T18:15:30.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 -- start start 2024-01-15T18:15:30.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a0141660 0x7f34a00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f34a015ec80 0x7f34a0141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34a015f310 con 0x7f34a0142470 2024-01-15T18:15:30.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a015fbb0 con 0x7f34a015ec80 2024-01-15T18:15:30.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a015f100 con 0x7f34a0141660 2024-01-15T18:15:30.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a5000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f34a0142470 0x7f34a0071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48252/0 (socket says 172.21.15.143:48252) 2024-01-15T18:15:30.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.095+0000 7f34a5000640 1 -- 172.21.15.143:0/297099162 learned_addr learned my addr 172.21.15.143:0/297099162 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:30.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349fe00640 1 --2- 172.21.15.143:0/297099162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a0141660 0x7f34a00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349f400640 1 --2- 172.21.15.143:0/297099162 >> v2:172.21.15.204:3300/0 conn(0x7f34a015ec80 0x7f34a0141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349ea00640 1 -- 172.21.15.143:0/297099162 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516830839 0 0) 0x7f34a015f310 con 0x7f34a0142470 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349ea00640 1 -- 172.21.15.143:0/297099162 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3480003530 con 0x7f34a0142470 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349fe00640 1 -- 172.21.15.143:0/297099162 >> v2:172.21.15.204:3300/0 conn(0x7f34a015ec80 msgr2=0x7f34a0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349fe00640 1 --2- 172.21.15.143:0/297099162 >> v2:172.21.15.204:3300/0 conn(0x7f34a015ec80 0x7f34a0141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349fe00640 1 -- 172.21.15.143:0/297099162 >> v1:172.21.15.143:6789/0 conn(0x7f34a0142470 legacy=0x7f34a0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349fe00640 1 -- 172.21.15.143:0/297099162 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a015f5f0 con 0x7f34a0141660 2024-01-15T18:15:30.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349f400640 1 --2- 172.21.15.143:0/297099162 >> v2:172.21.15.204:3300/0 conn(0x7f34a015ec80 0x7f34a0141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.099+0000 7f349fe00640 1 --2- 172.21.15.143:0/297099162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a0141660 0x7f34a00ce560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34900097c0 tx=0x7f349001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da8020b339c8bf01 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f349ea00640 1 -- 172.21.15.143:0/297099162 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34900ab650 con 0x7f34a0141660 2024-01-15T18:15:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f349ea00640 1 -- 172.21.15.143:0/297099162 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34900ab7b0 con 0x7f34a0141660 2024-01-15T18:15:30.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f349ea00640 1 -- 172.21.15.143:0/297099162 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34900abb20 con 0x7f34a0141660 2024-01-15T18:15:30.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 -- 172.21.15.143:0/297099162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a0141660 msgr2=0x7f34a00ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 --2- 172.21.15.143:0/297099162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a0141660 0x7f34a00ce560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f34900097c0 tx=0x7f349001c450 comp rx=0 tx=0).stop 2024-01-15T18:15:30.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 -- 172.21.15.143:0/297099162 shutdown_connections 2024-01-15T18:15:30.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 --2- 172.21.15.143:0/297099162 >> v2:172.21.15.204:3300/0 conn(0x7f34a015ec80 0x7f34a0141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 --2- 172.21.15.143:0/297099162 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a0141660 0x7f34a00ce560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 -- 172.21.15.143:0/297099162 >> 172.21.15.143:0/297099162 conn(0x7f34a006c3f0 msgr2=0x7f34a006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:30.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 -- 172.21.15.143:0/297099162 shutdown_connections 2024-01-15T18:15:30.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 -- 172.21.15.143:0/297099162 wait complete. 2024-01-15T18:15:30.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 Processor -- start 2024-01-15T18:15:30.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 -- start start 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 0x7f34a00b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.103+0000 7f34a7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0142470 0x7f34a00705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a015ec80 0x7f34a00b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0142d50 con 0x7f34a0141660 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0143030 con 0x7f34a0142470 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a0143310 con 0x7f34a015ec80 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 0x7f34a00b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 0x7f34a00b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35262/0 (socket says 172.21.15.143:35262) 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 -- 172.21.15.143:0/1244696778 learned_addr learned my addr 172.21.15.143:0/1244696778 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349f400640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0142470 0x7f34a00705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a5000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a015ec80 0x7f34a00b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 -- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a015ec80 msgr2=0x7f34a00b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a015ec80 0x7f34a00b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 -- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0142470 msgr2=0x7f34a00705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0142470 0x7f34a00705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 -- 172.21.15.143:0/1244696778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3488009420 con 0x7f34a0141660 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349f400640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0142470 0x7f34a00705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a5000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a015ec80 0x7f34a00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f349fe00640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 0x7f34a00b38b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3490000c10 tx=0x7f34900191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490004450 con 0x7f34a0141660 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34900ab920 con 0x7f34a0141660 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3490009420 con 0x7f34a0141660 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34900abc90 con 0x7f34a0141660 2024-01-15T18:15:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a00cd150 con 0x7f34a0141660 2024-01-15T18:15:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a0067500 con 0x7f34a0141660 2024-01-15T18:15:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.107+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f3490003620 con 0x7f34a0141660 2024-01-15T18:15:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.111+0000 7f3497e00640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3478067550 0x7f3478069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.111+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f349011d380 con 0x7f34a0141660 2024-01-15T18:15:30.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.111+0000 7f349f400640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3478067550 0x7f3478069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.111+0000 7f349f400640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3478067550 0x7f3478069a10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f34a0140ef0 tx=0x7f3488008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.115+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34900ebdf0 con 0x7f34a0141660 2024-01-15T18:15:30.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.291+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993110", "target": ["mon-mgr", ""]}) v1 -- 0x7f34a015f380 con 0x7f3478067550 2024-01-15T18:15:30.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.299+0000 7f3497e00640 1 -- 172.21.15.143:0/1244696778 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f34a015f380 con 0x7f3478067550 2024-01-15T18:15:30.297 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:30.297 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:30.298 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3478067550 msgr2=0x7f3478069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3478067550 0x7f3478069a10 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f34a0140ef0 tx=0x7f3488008040 comp rx=0 tx=0).stop 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 msgr2=0x7f34a00b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 0x7f34a00b38b0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3490000c10 tx=0x7f34900191f0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 shutdown_connections 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3478067550 0x7f3478069a10 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34a015ec80 0x7f34a00b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34a0142470 0x7f34a00705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 --2- 172.21.15.143:0/1244696778 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34a0141660 0x7f34a00b38b0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 >> 172.21.15.143:0/1244696778 conn(0x7f34a006c3f0 msgr2=0x7f34a00b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 shutdown_connections 2024-01-15T18:15:30.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.303+0000 7f34a7000640 1 -- 172.21.15.143:0/1244696778 wait complete. 2024-01-15T18:15:30.319 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:30.319 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993111 2024-01-15T18:15:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff24aa00640 1 Processor -- start 2024-01-15T18:15:30.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff24aa00640 1 -- start start 2024-01-15T18:15:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff24aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff244159b70 0x7ff2440b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff24aa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440b4d20 con 0x7ff244157300 2024-01-15T18:15:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff24aa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2440b2a20 con 0x7ff2440b48f0 2024-01-15T18:15:30.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff24aa00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2440b35a0 con 0x7ff244159b70 2024-01-15T18:15:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff243400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff244159b70 0x7ff2440b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff243400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff244159b70 0x7ff2440b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35278/0 (socket says 172.21.15.143:35278) 2024-01-15T18:15:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff243400640 1 -- 172.21.15.143:0/453620579 learned_addr learned my addr 172.21.15.143:0/453620579 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff243400640 1 -- 172.21.15.143:0/453620579 >> v1:172.21.15.204:6789/0 conn(0x7ff2440b48f0 legacy=0x7ff244070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff243400640 1 -- 172.21.15.143:0/453620579 >> v1:172.21.15.143:6789/0 conn(0x7ff244157300 legacy=0x7ff2440b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.463+0000 7ff243400640 1 -- 172.21.15.143:0/453620579 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2440b2d00 con 0x7ff244159b70 2024-01-15T18:15:30.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff243400640 1 --2- 172.21.15.143:0/453620579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff2440b43b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00bb10 tx=0x7ff22c01ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a508e702c2cbc4b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff242a00640 1 -- 172.21.15.143:0/453620579 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0a7cb0 con 0x7ff244159b70 2024-01-15T18:15:30.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff242a00640 1 -- 172.21.15.143:0/453620579 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22c0a7e10 con 0x7ff244159b70 2024-01-15T18:15:30.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff242a00640 1 -- 172.21.15.143:0/453620579 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c0af660 con 0x7ff244159b70 2024-01-15T18:15:30.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 -- 172.21.15.143:0/453620579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 msgr2=0x7ff2440b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 --2- 172.21.15.143:0/453620579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff2440b43b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00bb10 tx=0x7ff22c01ebe0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 -- 172.21.15.143:0/453620579 shutdown_connections 2024-01-15T18:15:30.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 --2- 172.21.15.143:0/453620579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff2440b43b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 -- 172.21.15.143:0/453620579 >> 172.21.15.143:0/453620579 conn(0x7ff24406c3f0 msgr2=0x7ff24406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:30.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 -- 172.21.15.143:0/453620579 shutdown_connections 2024-01-15T18:15:30.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.467+0000 7ff24aa00640 1 -- 172.21.15.143:0/453620579 wait complete. 2024-01-15T18:15:30.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 Processor -- start 2024-01-15T18:15:30.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 -- start start 2024-01-15T18:15:30.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2440b48f0 0x7ff24413d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff244157300 0x7ff24414d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff24406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244159fd0 con 0x7ff244159b70 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24413daf0 con 0x7ff244157300 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff244150720 con 0x7ff2440b48f0 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff24406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff24406c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35294/0 (socket says 172.21.15.143:35294) 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 -- 172.21.15.143:0/1377834513 learned_addr learned my addr 172.21.15.143:0/1377834513 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff243400640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff244157300 0x7ff24414d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff243e00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2440b48f0 0x7ff24413d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 -- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2440b48f0 msgr2=0x7ff24413d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2440b48f0 0x7ff24413d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 -- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff244157300 msgr2=0x7ff24414d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff244157300 0x7ff24414d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 -- 172.21.15.143:0/1377834513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c00b770 con 0x7ff244159b70 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff243400640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff244157300 0x7ff24414d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff248a00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff24406c7d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00c6c0 tx=0x7ff23c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c003b80 con 0x7ff244159b70 2024-01-15T18:15:30.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c003ce0 con 0x7ff244159b70 2024-01-15T18:15:30.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff244150160 con 0x7ff244159b70 2024-01-15T18:15:30.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0052f0 con 0x7ff244159b70 2024-01-15T18:15:30.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.471+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24415aa20 con 0x7ff244159b70 2024-01-15T18:15:30.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.475+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff244067500 con 0x7ff244159b70 2024-01-15T18:15:30.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.475+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7ff23c005450 con 0x7ff244159b70 2024-01-15T18:15:30.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.475+0000 7ff23be00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff21c067550 0x7ff21c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.475+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff23c082a90 con 0x7ff244159b70 2024-01-15T18:15:30.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.475+0000 7ff243e00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff21c067550 0x7ff21c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.475+0000 7ff243e00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff21c067550 0x7ff21c069a10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff234007fd0 tx=0x7ff234009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.479+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff23c0517d0 con 0x7ff244159b70 2024-01-15T18:15:30.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.659+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993111", "target": ["mon-mgr", ""]}) v1 -- 0x7ff24406e4e0 con 0x7ff21c067550 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.667+0000 7ff23be00640 1 -- 172.21.15.143:0/1377834513 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7ff24406e4e0 con 0x7ff21c067550 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:30.663 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:30.664 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:30.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.667+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff21c067550 msgr2=0x7ff21c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.667+0000 7ff24aa00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff21c067550 0x7ff21c069a10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff234007fd0 tx=0x7ff234009040 comp rx=0 tx=0).stop 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.667+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 msgr2=0x7ff24406c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.667+0000 7ff24aa00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff24406c7d0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00c6c0 tx=0x7ff23c00cb80 comp rx=0 tx=0).stop 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.667+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 shutdown_connections 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff21c067550 0x7ff21c069a10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff244159b70 0x7ff24406c7d0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff244157300 0x7ff24414d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 --2- 172.21.15.143:0/1377834513 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2440b48f0 0x7ff24413d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 >> 172.21.15.143:0/1377834513 conn(0x7ff24406c3f0 msgr2=0x7ff2440b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 shutdown_connections 2024-01-15T18:15:30.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.671+0000 7ff24aa00640 1 -- 172.21.15.143:0/1377834513 wait complete. 2024-01-15T18:15:30.686 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:30.686 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993112 2024-01-15T18:15:30.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 Processor -- start 2024-01-15T18:15:30.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 -- start start 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52740cdf20 0x7f52740cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f52740c6230 0x7f52740cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52740c4d90 con 0x7f527415b260 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527415b640 con 0x7f52740cdf20 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5279c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52740cf800 con 0x7f52740c6230 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52740cdf20 0x7f52740cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f52740cdf20 0x7f52740cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35310/0 (socket says 172.21.15.143:35310) 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 -- 172.21.15.143:0/2726592528 learned_addr learned my addr 172.21.15.143:0/2726592528 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5273400640 1 --2- 172.21.15.143:0/2726592528 >> v2:172.21.15.204:3300/0 conn(0x7f52740c6230 0x7f52740cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 -- 172.21.15.143:0/2726592528 >> v2:172.21.15.204:3300/0 conn(0x7f52740c6230 msgr2=0x7f52740cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 --2- 172.21.15.143:0/2726592528 >> v2:172.21.15.204:3300/0 conn(0x7f52740c6230 0x7f52740cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 -- 172.21.15.143:0/2726592528 >> v1:172.21.15.204:6789/0 conn(0x7f527415b260 legacy=0x7f52740cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:30.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.831+0000 7f5272000640 1 -- 172.21.15.143:0/2726592528 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740cfae0 con 0x7f52740cdf20 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5272000640 1 --2- 172.21.15.143:0/2726592528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f52740cbae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f527406b3d0 tx=0x7f525c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a161792f0732c750 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5271600640 1 -- 172.21.15.143:0/2726592528 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0aa5f0 con 0x7f52740cdf20 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5271600640 1 -- 172.21.15.143:0/2726592528 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c0aa750 con 0x7f52740cdf20 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5271600640 1 -- 172.21.15.143:0/2726592528 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0aaae0 con 0x7f52740cdf20 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 -- 172.21.15.143:0/2726592528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 msgr2=0x7f52740cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 --2- 172.21.15.143:0/2726592528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f52740cbae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f527406b3d0 tx=0x7f525c01c490 comp rx=0 tx=0).stop 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 -- 172.21.15.143:0/2726592528 shutdown_connections 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 --2- 172.21.15.143:0/2726592528 >> v2:172.21.15.204:3300/0 conn(0x7f52740c6230 0x7f52740cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 --2- 172.21.15.143:0/2726592528 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f52740cbae0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 -- 172.21.15.143:0/2726592528 >> 172.21.15.143:0/2726592528 conn(0x7f52740d0ed0 msgr2=0x7f52740d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 -- 172.21.15.143:0/2726592528 shutdown_connections 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.835+0000 7f5279c00640 1 -- 172.21.15.143:0/2726592528 wait complete. 2024-01-15T18:15:30.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 Processor -- start 2024-01-15T18:15:30.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 -- start start 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52740c6230 0x7f5274154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f5274157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f527415b260 0x7f5274158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c98c0 con 0x7f52740cdf20 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c9020 con 0x7f52740c6230 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5279c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c9300 con 0x7f527415b260 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f5274157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f5274157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35318/0 (socket says 172.21.15.143:35318) 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 -- 172.21.15.143:0/3802716420 learned_addr learned my addr 172.21.15.143:0/3802716420 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5273400640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f527415b260 0x7f5274158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272a00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52740c6230 0x7f5274154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 -- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f527415b260 msgr2=0x7f5274158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f527415b260 0x7f5274158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 -- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52740c6230 msgr2=0x7f5274154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52740c6230 0x7f5274154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 -- 172.21.15.143:0/3802716420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0094d0 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.839+0000 7f5272000640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f5274157de0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f525c01a0c0 tx=0x7f525c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0b3800 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c0b3980 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0b3d30 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52740c95e0 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5274154950 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5274155230 con 0x7f52740cdf20 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f525c0c6440 con 0x7f52740cdf20 2024-01-15T18:15:30.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f526aa00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f524c067550 0x7f524c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f525c0a7070 con 0x7f52740cdf20 2024-01-15T18:15:30.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.843+0000 7f5272a00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f524c067550 0x7f524c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.847+0000 7f5272a00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f524c067550 0x7f524c069a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5264007fd0 tx=0x7f5264007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:30.847+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5274155230 con 0x7f52740cdf20 2024-01-15T18:15:31.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.031+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993112", "target": ["mon-mgr", ""]}) v1 -- 0x7f52740cb730 con 0x7f524c067550 2024-01-15T18:15:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.039+0000 7f526aa00640 1 -- 172.21.15.143:0/3802716420 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f52740cb730 con 0x7f524c067550 2024-01-15T18:15:31.036 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:31.036 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:31.037 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:31.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f524c067550 msgr2=0x7f524c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f524c067550 0x7f524c069a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5264007fd0 tx=0x7f5264007e40 comp rx=0 tx=0).stop 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 msgr2=0x7f5274157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f5274157de0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f525c01a0c0 tx=0x7f525c01acd0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 shutdown_connections 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f524c067550 0x7f524c069a10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f527415b260 0x7f5274158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52740cdf20 0x7f5274157de0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 --2- 172.21.15.143:0/3802716420 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f52740c6230 0x7f5274154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 >> 172.21.15.143:0/3802716420 conn(0x7f52740d0ed0 msgr2=0x7f52740cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 shutdown_connections 2024-01-15T18:15:31.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.043+0000 7f5279c00640 1 -- 172.21.15.143:0/3802716420 wait complete. 2024-01-15T18:15:31.058 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:31.058 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993113 2024-01-15T18:15:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.199+0000 7f3503400640 1 Processor -- start 2024-01-15T18:15:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.199+0000 7f3503400640 1 -- start start 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3503400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f34fc06bfb0 0x7f34fc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3503400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc0d0210 0x7f34fc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3503400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc0cbff0 con 0x7f34fc0b0310 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3503400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0d0c50 con 0x7f34fc06bfb0 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3503400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc06c8d0 con 0x7f34fc0d0210 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f34fc06bfb0 0x7f34fc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f34fc06bfb0 0x7f34fc06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35332/0 (socket says 172.21.15.143:35332) 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 -- 172.21.15.143:0/1346298842 learned_addr learned my addr 172.21.15.143:0/1346298842 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3501400640 1 --2- 172.21.15.143:0/1346298842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc0d0210 0x7f34fc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 -- 172.21.15.143:0/1346298842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc0d0210 msgr2=0x7f34fc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 --2- 172.21.15.143:0/1346298842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc0d0210 0x7f34fc0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 -- 172.21.15.143:0/1346298842 >> v1:172.21.15.204:6789/0 conn(0x7f34fc0b0310 legacy=0x7f34fc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.203+0000 7f3500a00640 1 -- 172.21.15.143:0/1346298842 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc0cb6c0 con 0x7f34fc06bfb0 2024-01-15T18:15:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3500a00640 1 --2- 172.21.15.143:0/1346298842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc06bfb0 0x7f34fc06c390 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009870 tx=0x7f34ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7656836e7a70e35 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f34fb400640 1 -- 172.21.15.143:0/1346298842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0ad640 con 0x7f34fc06bfb0 2024-01-15T18:15:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f34fb400640 1 -- 172.21.15.143:0/1346298842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec0ad7a0 con 0x7f34fc06bfb0 2024-01-15T18:15:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f34fb400640 1 -- 172.21.15.143:0/1346298842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0adb30 con 0x7f34fc06bfb0 2024-01-15T18:15:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 -- 172.21.15.143:0/1346298842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc06bfb0 msgr2=0x7f34fc06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 --2- 172.21.15.143:0/1346298842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc06bfb0 0x7f34fc06c390 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009870 tx=0x7f34ec01c480 comp rx=0 tx=0).stop 2024-01-15T18:15:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 -- 172.21.15.143:0/1346298842 shutdown_connections 2024-01-15T18:15:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 --2- 172.21.15.143:0/1346298842 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc0d0210 0x7f34fc0ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 --2- 172.21.15.143:0/1346298842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc06bfb0 0x7f34fc06c390 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 -- 172.21.15.143:0/1346298842 >> 172.21.15.143:0/1346298842 conn(0x7f34fc0d0ed0 msgr2=0x7f34fc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 -- 172.21.15.143:0/1346298842 shutdown_connections 2024-01-15T18:15:31.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 -- 172.21.15.143:0/1346298842 wait complete. 2024-01-15T18:15:31.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 Processor -- start 2024-01-15T18:15:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.207+0000 7f3503400640 1 -- start start 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc06bfb0 0x7f34fc0ae770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 0x7f34fc0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34fc0d0210 0x7f34fc0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0c6890 con 0x7f34fc0b0310 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc155bc0 con 0x7f34fc0d0210 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0c62d0 con 0x7f34fc06bfb0 2024-01-15T18:15:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 0x7f34fc0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 0x7f34fc0aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35348/0 (socket says 172.21.15.143:35348) 2024-01-15T18:15:31.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3501400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34fc0d0210 0x7f34fc0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 -- 172.21.15.143:0/3702598780 learned_addr learned my addr 172.21.15.143:0/3702598780 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:31.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3500a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc06bfb0 0x7f34fc0ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 -- 172.21.15.143:0/3702598780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc06bfb0 msgr2=0x7f34fc0ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc06bfb0 0x7f34fc0ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 -- 172.21.15.143:0/3702598780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34fc0d0210 msgr2=0x7f34fc0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34fc0d0210 0x7f34fc0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 -- 172.21.15.143:0/3702598780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0094d0 con 0x7f34fc0b0310 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34fbe00640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 0x7f34fc0aecb0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c390 tx=0x7f34e400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e40052d0 con 0x7f34fc0b0310 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e4005430 con 0x7f34fc0b0310 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4003820 con 0x7f34fc0b0310 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc0c65b0 con 0x7f34fc0b0310 2024-01-15T18:15:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1560f0 con 0x7f34fc0b0310 2024-01-15T18:15:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc067500 con 0x7f34fc0b0310 2024-01-15T18:15:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.211+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f34e4003980 con 0x7f34fc0b0310 2024-01-15T18:15:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.215+0000 7f34f8c00640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34d4067550 0x7f34d4069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.215+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f34e4082590 con 0x7f34fc0b0310 2024-01-15T18:15:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.215+0000 7f3500a00640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34d4067550 0x7f34d4069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.215+0000 7f3500a00640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34d4067550 0x7f34d4069a10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0af000 tx=0x7f34ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.219+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34e4051050 con 0x7f34fc0b0310 2024-01-15T18:15:31.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.399+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993113", "target": ["mon-mgr", ""]}) v1 -- 0x7f34fc0ce690 con 0x7f34d4067550 2024-01-15T18:15:31.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f34f8c00640 1 -- 172.21.15.143:0/3702598780 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f34ec0a6040 con 0x7f34d4067550 2024-01-15T18:15:31.405 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:31.406 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34d4067550 msgr2=0x7f34d4069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34d4067550 0x7f34d4069a10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0af000 tx=0x7f34ec008040 comp rx=0 tx=0).stop 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 msgr2=0x7f34fc0aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 0x7f34fc0aecb0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f34e400c390 tx=0x7f34e400c850 comp rx=0 tx=0).stop 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 shutdown_connections 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34d4067550 0x7f34d4069a10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f34fc0d0210 0x7f34fc0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34fc0b0310 0x7f34fc0aecb0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 --2- 172.21.15.143:0/3702598780 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f34fc06bfb0 0x7f34fc0ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 >> 172.21.15.143:0/3702598780 conn(0x7f34fc0d0ed0 msgr2=0x7f34fc06ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 shutdown_connections 2024-01-15T18:15:31.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.411+0000 7f3503400640 1 -- 172.21.15.143:0/3702598780 wait complete. 2024-01-15T18:15:31.428 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:31.428 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993114 2024-01-15T18:15:31.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 Processor -- start 2024-01-15T18:15:31.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 -- start start 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff8d81577c0 0x7ff8d80b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8d80b2d80 0x7ff8d80b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d80b1290 0x7ff8d80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d80b16e0 con 0x7ff8d80b2d80 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d806e230 con 0x7ff8d81577c0 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d815dc00 con 0x7ff8d80b1290 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8d80b2d80 0x7ff8d80b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8d80b2d80 0x7ff8d80b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35352/0 (socket says 172.21.15.143:35352) 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 -- 172.21.15.143:0/2017413677 learned_addr learned my addr 172.21.15.143:0/2017413677 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7e00640 1 --2- 172.21.15.143:0/2017413677 >> v2:172.21.15.204:3300/0 conn(0x7ff8d81577c0 0x7ff8d80b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8dca00640 1 --2- 172.21.15.143:0/2017413677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d80b1290 0x7ff8d80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 -- 172.21.15.143:0/2017413677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d80b1290 msgr2=0x7ff8d80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 --2- 172.21.15.143:0/2017413677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d80b1290 0x7ff8d80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 -- 172.21.15.143:0/2017413677 >> v2:172.21.15.204:3300/0 conn(0x7ff8d81577c0 msgr2=0x7ff8d80b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 --2- 172.21.15.143:0/2017413677 >> v2:172.21.15.204:3300/0 conn(0x7ff8d81577c0 0x7ff8d80b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7400640 1 -- 172.21.15.143:0/2017413677 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d815dee0 con 0x7ff8d80b2d80 2024-01-15T18:15:31.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.571+0000 7ff8d7e00640 1 --2- 172.21.15.143:0/2017413677 >> v2:172.21.15.204:3300/0 conn(0x7ff8d81577c0 0x7ff8d80b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8d7400640 1 --2- 172.21.15.143:0/2017413677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d80b0d50 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80ad960 tx=0x7ff8c001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe988a8726ff38f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8d6a00640 1 -- 172.21.15.143:0/2017413677 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00ab600 con 0x7ff8d80b2d80 2024-01-15T18:15:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8d6a00640 1 -- 172.21.15.143:0/2017413677 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00a7050 con 0x7ff8d80b2d80 2024-01-15T18:15:31.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8d6a00640 1 -- 172.21.15.143:0/2017413677 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00ab980 con 0x7ff8d80b2d80 2024-01-15T18:15:31.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 -- 172.21.15.143:0/2017413677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 msgr2=0x7ff8d80b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 --2- 172.21.15.143:0/2017413677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d80b0d50 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80ad960 tx=0x7ff8c001c240 comp rx=0 tx=0).stop 2024-01-15T18:15:31.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 -- 172.21.15.143:0/2017413677 shutdown_connections 2024-01-15T18:15:31.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 --2- 172.21.15.143:0/2017413677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d80b1290 0x7ff8d80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 --2- 172.21.15.143:0/2017413677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d80b0d50 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 --2- 172.21.15.143:0/2017413677 >> v2:172.21.15.204:3300/0 conn(0x7ff8d81577c0 0x7ff8d80b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 -- 172.21.15.143:0/2017413677 >> 172.21.15.143:0/2017413677 conn(0x7ff8d806a660 msgr2=0x7ff8d806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 -- 172.21.15.143:0/2017413677 shutdown_connections 2024-01-15T18:15:31.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.575+0000 7ff8dea00640 1 -- 172.21.15.143:0/2017413677 wait complete. 2024-01-15T18:15:31.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 Processor -- start 2024-01-15T18:15:31.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 -- start start 2024-01-15T18:15:31.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d80b1290 0x7ff8d8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d8148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d81577c0 0x7ff8d81492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8147f60 con 0x7ff8d80b2d80 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8071c60 con 0x7ff8d80b1290 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8070d70 con 0x7ff8d81577c0 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d8148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d8148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35370/0 (socket says 172.21.15.143:35370) 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 -- 172.21.15.143:0/308025881 learned_addr learned my addr 172.21.15.143:0/308025881 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7e00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d80b1290 0x7ff8d8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dca00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d81577c0 0x7ff8d81492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 -- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d81577c0 msgr2=0x7ff8d81492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d81577c0 0x7ff8d81492d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 -- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d80b1290 msgr2=0x7ff8d8155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d80b1290 0x7ff8d8155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 -- 172.21.15.143:0/308025881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00094d0 con 0x7ff8d80b2d80 2024-01-15T18:15:31.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dca00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d81577c0 0x7ff8d81492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:31.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8d7400640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d8148d90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff8c001a8f0 tx=0x7ff8c0018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00b4620 con 0x7ff8d80b2d80 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c00b4780 con 0x7ff8d80b2d80 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d806c990 con 0x7ff8d80b2d80 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00b4b30 con 0x7ff8d80b2d80 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.579+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d8071330 con 0x7ff8d80b2d80 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.583+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d8067500 con 0x7ff8d80b2d80 2024-01-15T18:15:31.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.583+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7ff8c00b2070 con 0x7ff8d80b2d80 2024-01-15T18:15:31.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.583+0000 7ff8cfe00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b0067550 0x7ff8b0069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.583+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff8c011c280 con 0x7ff8d80b2d80 2024-01-15T18:15:31.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.583+0000 7ff8d7e00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b0067550 0x7ff8b0069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.583+0000 7ff8d7e00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b0067550 0x7ff8b0069a10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800c900 tx=0x7ff8c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.587+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8c00eaf10 con 0x7ff8d80b2d80 2024-01-15T18:15:31.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.767+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993114", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8d8157ba0 con 0x7ff8b0067550 2024-01-15T18:15:31.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.775+0000 7ff8cfe00640 1 -- 172.21.15.143:0/308025881 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7ff8c800c590 con 0x7ff8b0067550 2024-01-15T18:15:31.772 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:31.772 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:31.773 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b0067550 msgr2=0x7ff8b0069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b0067550 0x7ff8b0069a10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800c900 tx=0x7ff8c8008040 comp rx=0 tx=0).stop 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 msgr2=0x7ff8d8148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d8148d90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff8c001a8f0 tx=0x7ff8c0018fe0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 shutdown_connections 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b0067550 0x7ff8b0069a10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8d81577c0 0x7ff8d81492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8d80b2d80 0x7ff8d8148d90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 --2- 172.21.15.143:0/308025881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8d80b1290 0x7ff8d8155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 >> 172.21.15.143:0/308025881 conn(0x7ff8d806a660 msgr2=0x7ff8d81537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 shutdown_connections 2024-01-15T18:15:31.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.779+0000 7ff8dea00640 1 -- 172.21.15.143:0/308025881 wait complete. 2024-01-15T18:15:31.797 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:31.797 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993115 2024-01-15T18:15:31.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.939+0000 7f9b48600640 1 Processor -- start 2024-01-15T18:15:31.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.939+0000 7f9b48600640 1 -- start start 2024-01-15T18:15:31.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b48600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9b40071bb0 0x7f9b40072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b48600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9b4015c5a0 0x7f9b4015c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b48600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b400ceb50 con 0x7f9b40143780 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b48600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b40072570 con 0x7f9b4015c5a0 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b48600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4015c110 con 0x7f9b40071bb0 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b45c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9b40143780 0x7f9b400cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48266/0 (socket says 172.21.15.143:48266) 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b45c00640 1 -- 172.21.15.143:0/3555169579 learned_addr learned my addr 172.21.15.143:0/3555169579 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b46600640 1 --2- 172.21.15.143:0/3555169579 >> v2:172.21.15.143:3300/0 conn(0x7f9b4015c5a0 0x7f9b4015c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b45200640 1 --2- 172.21.15.143:0/3555169579 >> v2:172.21.15.204:3300/0 conn(0x7f9b40071bb0 0x7f9b40072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b46600640 1 -- 172.21.15.143:0/3555169579 >> v2:172.21.15.204:3300/0 conn(0x7f9b40071bb0 msgr2=0x7f9b40072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b46600640 1 --2- 172.21.15.143:0/3555169579 >> v2:172.21.15.204:3300/0 conn(0x7f9b40071bb0 0x7f9b40072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b46600640 1 -- 172.21.15.143:0/3555169579 >> v1:172.21.15.143:6789/0 conn(0x7f9b40143780 legacy=0x7f9b400cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.943+0000 7f9b46600640 1 -- 172.21.15.143:0/3555169579 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4015b7e0 con 0x7f9b4015c5a0 2024-01-15T18:15:31.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b46600640 1 --2- 172.21.15.143:0/3555169579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b4015c5a0 0x7f9b4015c9e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9b4006beb0 tx=0x7f9b3801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0527805ab6fdc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b3fe00640 1 -- 172.21.15.143:0/3555169579 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b380a59d0 con 0x7f9b4015c5a0 2024-01-15T18:15:31.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b3fe00640 1 -- 172.21.15.143:0/3555169579 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b380a5b30 con 0x7f9b4015c5a0 2024-01-15T18:15:31.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b3fe00640 1 -- 172.21.15.143:0/3555169579 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b380ad440 con 0x7f9b4015c5a0 2024-01-15T18:15:31.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 -- 172.21.15.143:0/3555169579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b4015c5a0 msgr2=0x7f9b4015c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 --2- 172.21.15.143:0/3555169579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b4015c5a0 0x7f9b4015c9e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9b4006beb0 tx=0x7f9b3801c910 comp rx=0 tx=0).stop 2024-01-15T18:15:31.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 -- 172.21.15.143:0/3555169579 shutdown_connections 2024-01-15T18:15:31.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 --2- 172.21.15.143:0/3555169579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b4015c5a0 0x7f9b4015c9e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 --2- 172.21.15.143:0/3555169579 >> v2:172.21.15.204:3300/0 conn(0x7f9b40071bb0 0x7f9b40072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 -- 172.21.15.143:0/3555169579 >> 172.21.15.143:0/3555169579 conn(0x7f9b4006c3f0 msgr2=0x7f9b4006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 -- 172.21.15.143:0/3555169579 shutdown_connections 2024-01-15T18:15:31.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 -- 172.21.15.143:0/3555169579 wait complete. 2024-01-15T18:15:31.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 Processor -- start 2024-01-15T18:15:31.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.947+0000 7f9b48600640 1 -- start start 2024-01-15T18:15:31.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 0x7f9b40133490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b40143780 0x7f9b401f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4015c5a0 0x7f9b40070e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40131380 con 0x7f9b40071bb0 2024-01-15T18:15:31.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40131660 con 0x7f9b4015c5a0 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b40131940 con 0x7f9b40143780 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 0x7f9b40133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 0x7f9b40133490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35384/0 (socket says 172.21.15.143:35384) 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 -- 172.21.15.143:0/4099628715 learned_addr learned my addr 172.21.15.143:0/4099628715 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45200640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b40143780 0x7f9b401f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b46600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4015c5a0 0x7f9b40070e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 -- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b40143780 msgr2=0x7f9b401f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b40143780 0x7f9b401f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 -- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4015c5a0 msgr2=0x7f9b40070e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4015c5a0 0x7f9b40070e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 -- 172.21.15.143:0/4099628715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38009a70 con 0x7f9b40071bb0 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b45c00640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 0x7f9b40133490 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c500 tx=0x7f9b3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30005cf0 con 0x7f9b40071bb0 2024-01-15T18:15:31.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30005e50 con 0x7f9b40071bb0 2024-01-15T18:15:31.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b40131c20 con 0x7f9b40071bb0 2024-01-15T18:15:31.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b300072f0 con 0x7f9b40071bb0 2024-01-15T18:15:31.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4013c8b0 con 0x7f9b40071bb0 2024-01-15T18:15:31.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.951+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b40132140 con 0x7f9b40071bb0 2024-01-15T18:15:31.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.955+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f9b30007450 con 0x7f9b40071bb0 2024-01-15T18:15:31.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.955+0000 7f9b3d600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b18067550 0x7f9b18069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.955+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9b300824a0 con 0x7f9b40071bb0 2024-01-15T18:15:31.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.955+0000 7f9b45200640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b18067550 0x7f9b18069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.955+0000 7f9b45200640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b18067550 0x7f9b18069a10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9b400b35d0 tx=0x7f9b28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:31.959+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9b300b6280 con 0x7f9b40071bb0 2024-01-15T18:15:32.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.139+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993115", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b4015cdb0 con 0x7f9b18067550 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b3d600640 1 -- 172.21.15.143:0/4099628715 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f9b4015cdb0 con 0x7f9b18067550 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:32.147 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:32.148 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b18067550 msgr2=0x7f9b18069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b18067550 0x7f9b18069a10 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9b400b35d0 tx=0x7f9b28008040 comp rx=0 tx=0).stop 2024-01-15T18:15:32.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 msgr2=0x7f9b40133490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 0x7f9b40133490 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c500 tx=0x7f9b3000c9c0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 shutdown_connections 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9b18067550 0x7f9b18069a10 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9b4015c5a0 0x7f9b40070e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9b40143780 0x7f9b401f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 --2- 172.21.15.143:0/4099628715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9b40071bb0 0x7f9b40133490 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.151+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 >> 172.21.15.143:0/4099628715 conn(0x7f9b4006c3f0 msgr2=0x7f9b400b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.155+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 shutdown_connections 2024-01-15T18:15:32.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.155+0000 7f9b48600640 1 -- 172.21.15.143:0/4099628715 wait complete. 2024-01-15T18:15:32.169 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:32.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993116 2024-01-15T18:15:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.311+0000 7f505fc00640 1 Processor -- start 2024-01-15T18:15:32.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.311+0000 7f505fc00640 1 -- start start 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f50580cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505fc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f505806b7b0 0x7f50580cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505fc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50580d02d0 con 0x7f50580cf150 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505fc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50580cf6c0 con 0x7f505806b7b0 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50580cf9a0 con 0x7f505806c540 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f505806b7b0 0x7f50580cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f505806b7b0 0x7f50580cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35392/0 (socket says 172.21.15.143:35392) 2024-01-15T18:15:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 -- 172.21.15.143:0/1525290817 learned_addr learned my addr 172.21.15.143:0/1525290817 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505d200640 1 --2- 172.21.15.143:0/1525290817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f50580cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 -- 172.21.15.143:0/1525290817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 msgr2=0x7f50580cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 --2- 172.21.15.143:0/1525290817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f50580cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 -- 172.21.15.143:0/1525290817 >> v1:172.21.15.204:6789/0 conn(0x7f50580cf150 legacy=0x7f505815ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.315+0000 7f505dc00640 1 -- 172.21.15.143:0/1525290817 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580cfc80 con 0x7f505806b7b0 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505dc00640 1 --2- 172.21.15.143:0/1525290817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505806b7b0 0x7f50580cdd20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5050009e50 tx=0x7f505001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a088f53c7d174933 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f5057400640 1 -- 172.21.15.143:0/1525290817 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500a5830 con 0x7f505806b7b0 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f5057400640 1 -- 172.21.15.143:0/1525290817 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50500a5990 con 0x7f505806b7b0 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f5057400640 1 -- 172.21.15.143:0/1525290817 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500a5d20 con 0x7f505806b7b0 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 -- 172.21.15.143:0/1525290817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505806b7b0 msgr2=0x7f50580cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 --2- 172.21.15.143:0/1525290817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505806b7b0 0x7f50580cdd20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5050009e50 tx=0x7f505001c950 comp rx=0 tx=0).stop 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 -- 172.21.15.143:0/1525290817 shutdown_connections 2024-01-15T18:15:32.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 --2- 172.21.15.143:0/1525290817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505806b7b0 0x7f50580cdd20 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 --2- 172.21.15.143:0/1525290817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f50580cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 -- 172.21.15.143:0/1525290817 >> 172.21.15.143:0/1525290817 conn(0x7f50580d0e20 msgr2=0x7f50580d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:32.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 -- 172.21.15.143:0/1525290817 shutdown_connections 2024-01-15T18:15:32.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 -- 172.21.15.143:0/1525290817 wait complete. 2024-01-15T18:15:32.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 Processor -- start 2024-01-15T18:15:32.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.319+0000 7f505fc00640 1 -- start start 2024-01-15T18:15:32.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505806b7b0 0x7f5058158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f505815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 0x7f505806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058159270 con 0x7f50580cf150 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058160a60 con 0x7f505806b7b0 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505815fca0 con 0x7f505806c540 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 0x7f505806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 0x7f505806ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35404/0 (socket says 172.21.15.143:35404) 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f5057e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f505815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 -- 172.21.15.143:0/1951529899 learned_addr learned my addr 172.21.15.143:0/1951529899 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505806b7b0 0x7f5058158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 -- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 msgr2=0x7f505815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f505815dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 -- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505806b7b0 msgr2=0x7f5058158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505806b7b0 0x7f5058158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 -- 172.21.15.143:0/1951529899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5050009b30 con 0x7f50580cf150 2024-01-15T18:15:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505d200640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505806b7b0 0x7f5058158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:32.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505dc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 0x7f505806ec10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f50500af040 tx=0x7f50500196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500b8670 con 0x7f50580cf150 2024-01-15T18:15:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50500b87d0 con 0x7f50580cf150 2024-01-15T18:15:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500b8b80 con 0x7f50580cf150 2024-01-15T18:15:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505815ff80 con 0x7f50580cf150 2024-01-15T18:15:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5058160540 con 0x7f50580cf150 2024-01-15T18:15:32.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505815fde0 con 0x7f50580cf150 2024-01-15T18:15:32.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.323+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f50500b8ce0 con 0x7f50580cf150 2024-01-15T18:15:32.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.327+0000 7f5054c00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5030067550 0x7f5030069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.327+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f505011e580 con 0x7f50580cf150 2024-01-15T18:15:32.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.327+0000 7f505d200640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5030067550 0x7f5030069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.327+0000 7f505d200640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5030067550 0x7f5030069a10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5058159a50 tx=0x7f5048009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:32.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.331+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f505815fde0 con 0x7f50580cf150 2024-01-15T18:15:32.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.511+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993116", "target": ["mon-mgr", ""]}) v1 -- 0x7f50580cbcf0 con 0x7f5030067550 2024-01-15T18:15:32.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.519+0000 7f5054c00640 1 -- 172.21.15.143:0/1951529899 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f5048011e90 con 0x7f5030067550 2024-01-15T18:15:32.516 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:32.516 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:32.516 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:32.516 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:32.516 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:32.517 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:32.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5030067550 msgr2=0x7f5030069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5030067550 0x7f5030069a10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5058159a50 tx=0x7f5048009190 comp rx=0 tx=0).stop 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 msgr2=0x7f505806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 0x7f505806ec10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f50500af040 tx=0x7f50500196f0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 shutdown_connections 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5030067550 0x7f5030069a10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50580cf150 0x7f505806ec10 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f505806c540 0x7f505815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 --2- 172.21.15.143:0/1951529899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f505806b7b0 0x7f5058158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 >> 172.21.15.143:0/1951529899 conn(0x7f50580d0e20 msgr2=0x7f50580af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 shutdown_connections 2024-01-15T18:15:32.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.523+0000 7f505fc00640 1 -- 172.21.15.143:0/1951529899 wait complete. 2024-01-15T18:15:32.538 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:32.538 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993117 2024-01-15T18:15:32.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e25600640 1 Processor -- start 2024-01-15T18:15:32.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e25600640 1 -- start start 2024-01-15T18:15:32.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e25600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9e200b24b0 0x7f9e200ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e25600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e20070bf0 con 0x7f9e200b3bf0 2024-01-15T18:15:32.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e25600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e200b4920 con 0x7f9e200ce690 2024-01-15T18:15:32.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e25600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2015dba0 con 0x7f9e200b24b0 2024-01-15T18:15:32.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e1e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9e200b24b0 0x7f9e200ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.679+0000 7f9e1e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9e200b24b0 0x7f9e200ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35420/0 (socket says 172.21.15.143:35420) 2024-01-15T18:15:32.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1e000640 1 -- 172.21.15.143:0/2164148816 learned_addr learned my addr 172.21.15.143:0/2164148816 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:32.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1e000640 1 -- 172.21.15.143:0/2164148816 >> v1:172.21.15.204:6789/0 conn(0x7f9e200ce690 legacy=0x7f9e200b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1e000640 1 -- 172.21.15.143:0/2164148816 >> v1:172.21.15.143:6789/0 conn(0x7f9e200b3bf0 legacy=0x7f9e200b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1e000640 1 -- 172.21.15.143:0/2164148816 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e200b4c00 con 0x7f9e200b24b0 2024-01-15T18:15:32.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1e000640 1 --2- 172.21.15.143:0/2164148816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200b24b0 0x7f9e200ac8e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009870 tx=0x7f9e0801cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c81d1bfabde4d95 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:32.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1d600640 1 -- 172.21.15.143:0/2164148816 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a95f0 con 0x7f9e200b24b0 2024-01-15T18:15:32.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1d600640 1 -- 172.21.15.143:0/2164148816 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e080a9750 con 0x7f9e200b24b0 2024-01-15T18:15:32.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.683+0000 7f9e1d600640 1 -- 172.21.15.143:0/2164148816 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a9ae0 con 0x7f9e200b24b0 2024-01-15T18:15:32.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- 172.21.15.143:0/2164148816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200b24b0 msgr2=0x7f9e200ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 --2- 172.21.15.143:0/2164148816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200b24b0 0x7f9e200ac8e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009870 tx=0x7f9e0801cf70 comp rx=0 tx=0).stop 2024-01-15T18:15:32.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- 172.21.15.143:0/2164148816 shutdown_connections 2024-01-15T18:15:32.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 --2- 172.21.15.143:0/2164148816 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200b24b0 0x7f9e200ac8e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- 172.21.15.143:0/2164148816 >> 172.21.15.143:0/2164148816 conn(0x7f9e2006c3f0 msgr2=0x7f9e2006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:32.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- 172.21.15.143:0/2164148816 shutdown_connections 2024-01-15T18:15:32.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- 172.21.15.143:0/2164148816 wait complete. 2024-01-15T18:15:32.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 Processor -- start 2024-01-15T18:15:32.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- start start 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e200b24b0 0x7f9e20158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e200b3bf0 0x7f9e201589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 0x7f9e200ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e201404a0 con 0x7f9e200ce690 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e200af330 con 0x7f9e200b24b0 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e25600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e200af610 con 0x7f9e200b3bf0 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.687+0000 7f9e1f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 0x7f9e200ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 0x7f9e200ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35428/0 (socket says 172.21.15.143:35428) 2024-01-15T18:15:32.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 -- 172.21.15.143:0/1794494160 learned_addr learned my addr 172.21.15.143:0/1794494160 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1ea00640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e200b24b0 0x7f9e20158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1e000640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e200b3bf0 0x7f9e201589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 -- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e200b3bf0 msgr2=0x7f9e201589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e200b3bf0 0x7f9e201589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 -- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e200b24b0 msgr2=0x7f9e20158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e200b24b0 0x7f9e20158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 -- 172.21.15.143:0/1794494160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080094d0 con 0x7f9e200ce690 2024-01-15T18:15:32.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e1f400640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 0x7f9e200ae870 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800c6c0 tx=0x7f9e1800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:32.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18003b80 con 0x7f9e200ce690 2024-01-15T18:15:32.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e18003ce0 con 0x7f9e200ce690 2024-01-15T18:15:32.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e180052f0 con 0x7f9e200ce690 2024-01-15T18:15:32.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e20140130 con 0x7f9e200ce690 2024-01-15T18:15:32.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e200aedb0 con 0x7f9e200ce690 2024-01-15T18:15:32.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2013ff90 con 0x7f9e200ce690 2024-01-15T18:15:32.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.691+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f9e18005450 con 0x7f9e200ce690 2024-01-15T18:15:32.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.695+0000 7f9e16a00640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9df8067550 0x7f9df8069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:32.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.695+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9e18082a30 con 0x7f9e200ce690 2024-01-15T18:15:32.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.695+0000 7f9e1ea00640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9df8067550 0x7f9df8069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:32.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.695+0000 7f9e1ea00640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9df8067550 0x7f9df8069a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9e10007fd0 tx=0x7f9e10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:32.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.699+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9e2013ff90 con 0x7f9e200ce690 2024-01-15T18:15:32.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.883+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993117", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e2013ff90 con 0x7f9df8067550 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e16a00640 1 -- 172.21.15.143:0/1794494160 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f9e2013ff90 con 0x7f9df8067550 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:32.887 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:32.888 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:32.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9df8067550 msgr2=0x7f9df8069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9df8067550 0x7f9df8069a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9e10007fd0 tx=0x7f9e10009040 comp rx=0 tx=0).stop 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 msgr2=0x7f9e200ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 0x7f9e200ae870 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800c6c0 tx=0x7f9e1800cb80 comp rx=0 tx=0).stop 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 shutdown_connections 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9df8067550 0x7f9df8069a10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e200ce690 0x7f9e200ae870 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e200b3bf0 0x7f9e201589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 --2- 172.21.15.143:0/1794494160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e200b24b0 0x7f9e20158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.891+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 >> 172.21.15.143:0/1794494160 conn(0x7f9e2006c3f0 msgr2=0x7f9e20146760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.895+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 shutdown_connections 2024-01-15T18:15:32.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:32.895+0000 7f9e25600640 1 -- 172.21.15.143:0/1794494160 wait complete. 2024-01-15T18:15:32.910 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:32.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993118 2024-01-15T18:15:33.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 Processor -- start 2024-01-15T18:15:33.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 -- start start 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5c0157300 0x7fd5c0159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd5c00b21d0 0x7fd5c00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00b35a0 con 0x7fd5c0157300 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00b4d60 con 0x7fd5c00b21d0 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00b2a20 con 0x7fd5c00b48f0 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5c0157300 0x7fd5c0159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd5c0157300 0x7fd5c0159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35434/0 (socket says 172.21.15.143:35434) 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2563713980 learned_addr learned my addr 172.21.15.143:0/2563713980 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c5000640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bf400640 1 --2- 172.21.15.143:0/2563713980 >> v2:172.21.15.204:3300/0 conn(0x7fd5c00b21d0 0x7fd5c00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2563713980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 msgr2=0x7fd5c015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2563713980 >> v2:172.21.15.204:3300/0 conn(0x7fd5c00b21d0 msgr2=0x7fd5c00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 --2- 172.21.15.143:0/2563713980 >> v2:172.21.15.204:3300/0 conn(0x7fd5c00b21d0 0x7fd5c00b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2563713980 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00b2d00 con 0x7fd5c0157300 2024-01-15T18:15:33.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.055+0000 7fd5c5000640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:33.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5bfe00640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c0157300 0x7fd5c0159b70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b4fe0 tx=0x7fd5b001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df89fe0d3ae7ce30 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5bea00640 1 -- 172.21.15.143:0/2563713980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00aa5f0 con 0x7fd5c0157300 2024-01-15T18:15:33.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5bea00640 1 -- 172.21.15.143:0/2563713980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b00aa750 con 0x7fd5c0157300 2024-01-15T18:15:33.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5bea00640 1 -- 172.21.15.143:0/2563713980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00aaae0 con 0x7fd5c0157300 2024-01-15T18:15:33.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 -- 172.21.15.143:0/2563713980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c0157300 msgr2=0x7fd5c0159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c0157300 0x7fd5c0159b70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b4fe0 tx=0x7fd5b001c4d0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 -- 172.21.15.143:0/2563713980 shutdown_connections 2024-01-15T18:15:33.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2563713980 >> v2:172.21.15.204:3300/0 conn(0x7fd5c00b21d0 0x7fd5c00b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2563713980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c0157300 0x7fd5c0159b70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.059+0000 7fd5c7000640 1 -- 172.21.15.143:0/2563713980 >> 172.21.15.143:0/2563713980 conn(0x7fd5c006c3f0 msgr2=0x7fd5c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:33.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 -- 172.21.15.143:0/2563713980 shutdown_connections 2024-01-15T18:15:33.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 -- 172.21.15.143:0/2563713980 wait complete. 2024-01-15T18:15:33.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 Processor -- start 2024-01-15T18:15:33.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 -- start start 2024-01-15T18:15:33.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 0x7fd5c015ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c0157300 0x7fd5c015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0154d50 con 0x7fd5c00b21d0 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0160630 con 0x7fd5c0157300 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0160910 con 0x7fd5c00b48f0 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 0x7fd5c015ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 0x7fd5c015ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35444/0 (socket says 172.21.15.143:35444) 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2781713881 learned_addr learned my addr 172.21.15.143:0/2781713881 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5c5000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c0157300 0x7fd5c015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bf400640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 msgr2=0x7fd5c015bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c0157300 msgr2=0x7fd5c015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c0157300 0x7fd5c015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 -- 172.21.15.143:0/2781713881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00094d0 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.063+0000 7fd5bfe00640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 0x7fd5c015ffa0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00ac040 tx=0x7fd5b001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00aa5f0 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b00b3820 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00b3bd0 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c0162340 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c0160bf0 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c01621a0 con 0x7fd5c00b21d0 2024-01-15T18:15:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7fd5b00b3d30 con 0x7fd5c00b21d0 2024-01-15T18:15:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5b7e00640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd598067580 0x7fd598069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd5b00a7070 con 0x7fd5c00b21d0 2024-01-15T18:15:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5bf400640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd598067580 0x7fd598069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.067+0000 7fd5bf400640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd598067580 0x7fd598069a40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0155e60 tx=0x7fd5a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.071+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd5c01621a0 con 0x7fd5c00b21d0 2024-01-15T18:15:33.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.251+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993118", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5c00721a0 con 0x7fd598067580 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.259+0000 7fd5b7e00640 1 -- 172.21.15.143:0/2781713881 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fd5a8004260 con 0x7fd598067580 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:33.256 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:33.257 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:33.257 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:33.257 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:33.257 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:33.257 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:33.257 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:33.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd598067580 msgr2=0x7fd598069a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd598067580 0x7fd598069a40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0155e60 tx=0x7fd5a8008040 comp rx=0 tx=0).stop 2024-01-15T18:15:33.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 msgr2=0x7fd5c015ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 0x7fd5c015ffa0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fd5b00ac040 tx=0x7fd5b001ad10 comp rx=0 tx=0).stop 2024-01-15T18:15:33.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 shutdown_connections 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd598067580 0x7fd598069a40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5c0157300 0x7fd5c015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd5c00b48f0 0x7fd5c015bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 --2- 172.21.15.143:0/2781713881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5c00b21d0 0x7fd5c015ffa0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 >> 172.21.15.143:0/2781713881 conn(0x7fd5c006c3f0 msgr2=0x7fd5c0070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 shutdown_connections 2024-01-15T18:15:33.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.263+0000 7fd5c7000640 1 -- 172.21.15.143:0/2781713881 wait complete. 2024-01-15T18:15:33.278 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:33.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993119 2024-01-15T18:15:33.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 Processor -- start 2024-01-15T18:15:33.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 -- start start 2024-01-15T18:15:33.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cda70 0x7f62100cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f62100cf400 0x7f62100cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621015b5b0 con 0x7f62100cf800 2024-01-15T18:15:33.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100ccd30 con 0x7f62100cf400 2024-01-15T18:15:33.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f6214e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100cba70 con 0x7f62100cda70 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cda70 0x7f62100cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f62100cf400 0x7f62100cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620e800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f62100cf800 0x7f62100b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43946/0 (socket says 172.21.15.143:43946) 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620e800640 1 -- 172.21.15.143:0/139801386 learned_addr learned my addr 172.21.15.143:0/139801386 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620ca00640 1 -- 172.21.15.143:0/139801386 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390005887 0 0) 0x7f621015b5b0 con 0x7f62100cf800 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620ca00640 1 -- 172.21.15.143:0/139801386 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f0003530 con 0x7f62100cf800 2024-01-15T18:15:33.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620d400640 1 -- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cda70 msgr2=0x7f62100cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:33.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620d400640 1 --2- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cda70 0x7f62100cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620d400640 1 -- 172.21.15.143:0/139801386 >> v1:172.21.15.204:6789/0 conn(0x7f62100cf800 legacy=0x7f62100b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620d400640 1 -- 172.21.15.143:0/139801386 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100cbd10 con 0x7f62100cf400 2024-01-15T18:15:33.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.423+0000 7f620de00640 1 --2- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cda70 0x7f62100cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:33.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f620d400640 1 --2- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cf400 0x7f62100cde50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f61f80097c0 tx=0x7f61f801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb02b6ac5261820 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f620ca00640 1 -- 172.21.15.143:0/139801386 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f80ab650 con 0x7f62100cf400 2024-01-15T18:15:33.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f620ca00640 1 -- 172.21.15.143:0/139801386 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f80ab7b0 con 0x7f62100cf400 2024-01-15T18:15:33.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f620ca00640 1 -- 172.21.15.143:0/139801386 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f80abb40 con 0x7f62100cf400 2024-01-15T18:15:33.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 -- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cf400 msgr2=0x7f62100cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 --2- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cf400 0x7f62100cde50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f61f80097c0 tx=0x7f61f801c450 comp rx=0 tx=0).stop 2024-01-15T18:15:33.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 -- 172.21.15.143:0/139801386 shutdown_connections 2024-01-15T18:15:33.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 --2- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cf400 0x7f62100cde50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 --2- 172.21.15.143:0/139801386 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cda70 0x7f62100cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 -- 172.21.15.143:0/139801386 >> 172.21.15.143:0/139801386 conn(0x7f62100d0e20 msgr2=0x7f62100d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:33.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 -- 172.21.15.143:0/139801386 shutdown_connections 2024-01-15T18:15:33.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.427+0000 7f6214e00640 1 -- 172.21.15.143:0/139801386 wait complete. 2024-01-15T18:15:33.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 Processor -- start 2024-01-15T18:15:33.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 -- start start 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cda70 0x7f6210153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cf400 0x7f62100af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 0x7f621006cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62101f7110 con 0x7f62100cf800 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62101f60f0 con 0x7f62100cda70 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62101f63d0 con 0x7f62100cf400 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 0x7f621006cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 0x7f621006cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35452/0 (socket says 172.21.15.143:35452) 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 -- 172.21.15.143:0/3197545217 learned_addr learned my addr 172.21.15.143:0/3197545217 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620d400640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cf400 0x7f62100af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620de00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cda70 0x7f6210153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 -- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cf400 msgr2=0x7f62100af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cf400 0x7f62100af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 -- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cda70 msgr2=0x7f6210153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cda70 0x7f6210153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 -- 172.21.15.143:0/3197545217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6200009420 con 0x7f62100cf800 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620de00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cda70 0x7f6210153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f620e800640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 0x7f621006cb90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f620800cb40 tx=0x7f620800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800aca0 con 0x7f62100cf800 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620800ae00 con 0x7f62100cf800 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8009420 con 0x7f62100cf800 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208012620 con 0x7f62100cf800 2024-01-15T18:15:33.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.431+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62101f6990 con 0x7f62100cf800 2024-01-15T18:15:33.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.435+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6210067500 con 0x7f62100cf800 2024-01-15T18:15:33.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.435+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f6208007070 con 0x7f62100cf800 2024-01-15T18:15:33.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.435+0000 7f6206000640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61e8067550 0x7f61e8069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.435+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f62080830a0 con 0x7f62100cf800 2024-01-15T18:15:33.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.435+0000 7f620de00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61e8067550 0x7f61e8069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.435+0000 7f620de00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61e8067550 0x7f61e8069a10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6200009ca0 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.439+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6208051b60 con 0x7f62100cf800 2024-01-15T18:15:33.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.619+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993119", "target": ["mon-mgr", ""]}) v1 -- 0x7f62100cfd10 con 0x7f61e8067550 2024-01-15T18:15:33.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.627+0000 7f6206000640 1 -- 172.21.15.143:0/3197545217 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f6200011e90 con 0x7f61e8067550 2024-01-15T18:15:33.625 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:33.625 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:33.625 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:33.625 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:33.626 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:33.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61e8067550 msgr2=0x7f61e8069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61e8067550 0x7f61e8069a10 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6200009ca0 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 msgr2=0x7f621006cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 0x7f621006cb90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f620800cb40 tx=0x7f620800a060 comp rx=0 tx=0).stop 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 shutdown_connections 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61e8067550 0x7f61e8069a10 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62100cf800 0x7f621006cb90 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f62100cf400 0x7f62100af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 --2- 172.21.15.143:0/3197545217 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62100cda70 0x7f6210153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 >> 172.21.15.143:0/3197545217 conn(0x7f62100d0e20 msgr2=0x7f621006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 shutdown_connections 2024-01-15T18:15:33.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.631+0000 7f6214e00640 1 -- 172.21.15.143:0/3197545217 wait complete. 2024-01-15T18:15:33.647 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:15:33.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone993120 2024-01-15T18:15:33.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 Processor -- start 2024-01-15T18:15:33.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 -- start start 2024-01-15T18:15:33.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 0x7f83e40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f83e40b13e0 0x7f83e406c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e406ca60 con 0x7f83e415b4e0 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e415b950 con 0x7f83e4144700 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.791+0000 7f83eaa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e406cc90 con 0x7f83e40b13e0 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 0x7f83e40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 0x7f83e40705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35468/0 (socket says 172.21.15.143:35468) 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e3e00640 1 -- 172.21.15.143:0/564490514 learned_addr learned my addr 172.21.15.143:0/564490514 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:33.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e8a00640 1 --2- 172.21.15.143:0/564490514 >> v2:172.21.15.204:3300/0 conn(0x7f83e40b13e0 0x7f83e406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e8a00640 1 -- 172.21.15.143:0/564490514 >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 msgr2=0x7f83e40705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e8a00640 1 --2- 172.21.15.143:0/564490514 >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 0x7f83e40705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e8a00640 1 -- 172.21.15.143:0/564490514 >> v1:172.21.15.143:6789/0 conn(0x7f83e415b4e0 legacy=0x7f83e40b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e8a00640 1 -- 172.21.15.143:0/564490514 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e406cf70 con 0x7f83e40b13e0 2024-01-15T18:15:33.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e3e00640 1 --2- 172.21.15.143:0/564490514 >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 0x7f83e40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:33.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e8a00640 1 --2- 172.21.15.143:0/564490514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e40b13e0 0x7f83e406c520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83e406e190 tx=0x7f83dc01c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f57df5646b096c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.795+0000 7f83e2a00640 1 -- 172.21.15.143:0/564490514 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0a5940 con 0x7f83e40b13e0 2024-01-15T18:15:33.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83e2a00640 1 -- 172.21.15.143:0/564490514 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc0a5aa0 con 0x7f83e40b13e0 2024-01-15T18:15:33.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83e2a00640 1 -- 172.21.15.143:0/564490514 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0a5e30 con 0x7f83e40b13e0 2024-01-15T18:15:33.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 -- 172.21.15.143:0/564490514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e40b13e0 msgr2=0x7f83e406c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 --2- 172.21.15.143:0/564490514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e40b13e0 0x7f83e406c520 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83e406e190 tx=0x7f83dc01c880 comp rx=0 tx=0).stop 2024-01-15T18:15:33.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 -- 172.21.15.143:0/564490514 shutdown_connections 2024-01-15T18:15:33.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 --2- 172.21.15.143:0/564490514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e40b13e0 0x7f83e406c520 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 --2- 172.21.15.143:0/564490514 >> v2:172.21.15.143:3300/0 conn(0x7f83e4144700 0x7f83e40705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 -- 172.21.15.143:0/564490514 >> 172.21.15.143:0/564490514 conn(0x7f83e406a660 msgr2=0x7f83e406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:33.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 -- 172.21.15.143:0/564490514 shutdown_connections 2024-01-15T18:15:33.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 -- 172.21.15.143:0/564490514 wait complete. 2024-01-15T18:15:33.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 Processor -- start 2024-01-15T18:15:33.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 -- start start 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.799+0000 7f83eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 0x7f83e40b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83e4144700 0x7f83e40b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e415b4e0 0x7f83e414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40b3670 con 0x7f83e40b13e0 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e414d170 con 0x7f83e415b4e0 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e414cc20 con 0x7f83e4144700 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 0x7f83e40b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 0x7f83e40b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35474/0 (socket says 172.21.15.143:35474) 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 -- 172.21.15.143:0/2519810031 learned_addr learned my addr 172.21.15.143:0/2519810031 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e8a00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e415b4e0 0x7f83e414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3400640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83e4144700 0x7f83e40b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 -- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83e4144700 msgr2=0x7f83e40b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83e4144700 0x7f83e40b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 -- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e415b4e0 msgr2=0x7f83e414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e415b4e0 0x7f83e414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 -- 172.21.15.143:0/2519810031 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d40094d0 con 0x7f83e40b13e0 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3400640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83e4144700 0x7f83e40b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83e3e00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 0x7f83e40b1ab0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f83d400e9d0 tx=0x7f83d400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d400ca60 con 0x7f83e40b13e0 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d400cbc0 con 0x7f83e40b13e0 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d4013470 con 0x7f83e40b13e0 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc009a70 con 0x7f83e40b13e0 2024-01-15T18:15:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e40b2370 con 0x7f83e40b13e0 2024-01-15T18:15:33.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e4067500 con 0x7f83e40b13e0 2024-01-15T18:15:33.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.803+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f83d40135d0 con 0x7f83e40b13e0 2024-01-15T18:15:33.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.807+0000 7f83dbe00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f83bc067550 0x7f83bc069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:33.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.807+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f83d4083300 con 0x7f83e40b13e0 2024-01-15T18:15:33.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.807+0000 7f83e3400640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f83bc067550 0x7f83bc069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:33.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.807+0000 7f83e3400640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f83bc067550 0x7f83bc069a10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f83e40cf930 tx=0x7f83cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:33.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.811+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83d4051dc0 con 0x7f83e40b13e0 2024-01-15T18:15:33.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:33.995+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone993120", "target": ["mon-mgr", ""]}) v1 -- 0x7f83e40ce5e0 con 0x7f83bc067550 2024-01-15T18:15:34.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83dbe00640 1 -- 172.21.15.143:0/2519810031 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f83cc001350 con 0x7f83bc067550 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:15:34.002 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:34.003 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:34.003 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f83bc067550 msgr2=0x7f83bc069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f83bc067550 0x7f83bc069a10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f83e40cf930 tx=0x7f83cc009040 comp rx=0 tx=0).stop 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 msgr2=0x7f83e40b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 0x7f83e40b1ab0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f83d400e9d0 tx=0x7f83d400ee90 comp rx=0 tx=0).stop 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 shutdown_connections 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f83bc067550 0x7f83bc069a10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f83e415b4e0 0x7f83e414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f83e4144700 0x7f83e40b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 --2- 172.21.15.143:0/2519810031 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83e40b13e0 0x7f83e40b1ab0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 >> 172.21.15.143:0/2519810031 conn(0x7f83e406a660 msgr2=0x7f83e40afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 shutdown_connections 2024-01-15T18:15:34.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.007+0000 7f83eaa00640 1 -- 172.21.15.143:0/2519810031 wait complete. 2024-01-15T18:15:34.024 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:15:34.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8484400640 1 Processor -- start 2024-01-15T18:15:34.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8484400640 1 -- start start 2024-01-15T18:15:34.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8484400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f847c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8484400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0ceb50 con 0x7f847c0722c0 2024-01-15T18:15:34.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8484400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c15c110 con 0x7f847c0ce520 2024-01-15T18:15:34.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8484400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15b7e0 con 0x7f847c143780 2024-01-15T18:15:34.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8482400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f847c0722c0 0x7f847c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48282/0 (socket says 172.21.15.143:48282) 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8482400640 1 -- 172.21.15.143:0/426724946 learned_addr learned my addr 172.21.15.143:0/426724946 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f847c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136398497 0 0) 0x7f84740014a0 con 0x7f847c0722c0 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f845c003530 con 0x7f847c0722c0 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 135309130 0 0) 0x7f847c15c110 con 0x7f847c0ce520 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84740014a0 con 0x7f847c0ce520 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321343065 0 0) 0x7f845c003530 con 0x7f847c0722c0 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 msgr2=0x7f847c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f847c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 >> v1:172.21.15.204:6789/0 conn(0x7f847c0ce520 legacy=0x7f847c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c15bac0 con 0x7f847c0722c0 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8474001b90 con 0x7f847c0722c0 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f847c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 >> v1:172.21.15.143:6789/0 conn(0x7f847c0722c0 legacy=0x7f847c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f845c003b40 0x7f845c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845c0047d0 0x7f845c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f845c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15b7e0 con 0x7f845c003b40 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c00dbd0 con 0x7f845c0047d0 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c00dd90 con 0x7f847c143780 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f845c003b40 0x7f845c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481000640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845c0047d0 0x7f845c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8482400640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f845c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 -- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 msgr2=0x7f845c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f847c143780 0x7f845c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 -- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845c0047d0 msgr2=0x7f845c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845c0047d0 0x7f845c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481a00640 1 -- 172.21.15.143:0/426724946 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c009420 con 0x7f845c003b40 2024-01-15T18:15:34.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.167+0000 7f8481000640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f845c0047d0 0x7f845c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:34.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8481a00640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f845c003b40 0x7f845c003f40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f846c009c80 tx=0x7f846c01cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aca3077155a62e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0a5740 con 0x7f845c003b40 2024-01-15T18:15:34.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c0a58a0 con 0x7f845c003b40 2024-01-15T18:15:34.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f847be00640 1 -- 172.21.15.143:0/426724946 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0a5c50 con 0x7f845c003b40 2024-01-15T18:15:34.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8484400640 1 -- 172.21.15.143:0/426724946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f845c003b40 msgr2=0x7f845c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8484400640 1 --2- 172.21.15.143:0/426724946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f845c003b40 0x7f845c003f40 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f846c009c80 tx=0x7f846c01cb90 comp rx=0 tx=0).stop 2024-01-15T18:15:34.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8482400640 1 -- 172.21.15.143:0/426724946 reap_dead start 2024-01-15T18:15:34.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8484400640 1 -- 172.21.15.143:0/426724946 shutdown_connections 2024-01-15T18:15:34.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8484400640 1 -- 172.21.15.143:0/426724946 >> 172.21.15.143:0/426724946 conn(0x7f847c06c3f0 msgr2=0x7f847c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8484400640 1 -- 172.21.15.143:0/426724946 shutdown_connections 2024-01-15T18:15:34.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.171+0000 7f8484400640 1 -- 172.21.15.143:0/426724946 wait complete. 2024-01-15T18:15:34.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 Processor -- start 2024-01-15T18:15:34.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 -- start start 2024-01-15T18:15:34.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845c003b40 0x7f847c1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 0x7f847c142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f847c13f500 0x7f847c13f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c136ff0 con 0x7f847c141cb0 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c139410 con 0x7f847c13f500 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8484400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c13fec0 con 0x7f845c003b40 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 0x7f847c142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 0x7f847c142130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35492/0 (socket says 172.21.15.143:35492) 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 -- 172.21.15.143:0/3768484801 learned_addr learned my addr 172.21.15.143:0/3768484801 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481a00640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845c003b40 0x7f847c1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8482400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f847c13f500 0x7f847c13f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 -- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845c003b40 msgr2=0x7f847c1f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845c003b40 0x7f847c1f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 -- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f847c13f500 msgr2=0x7f847c13f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f847c13f500 0x7f847c13f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 -- 172.21.15.143:0/3768484801 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c1f6020 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.175+0000 7f8481000640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 0x7f847c142130 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f846400e780 tx=0x7f846400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464017070 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640050d0 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464005240 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c138790 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c138d50 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c067500 con 0x7f847c141cb0 2024-01-15T18:15:34.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f8464007020 con 0x7f847c141cb0 2024-01-15T18:15:34.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8479600640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8454067550 0x7f8454069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f84640825c0 con 0x7f847c141cb0 2024-01-15T18:15:34.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.179+0000 7f8481a00640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8454067550 0x7f8454069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.183+0000 7f8481a00640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8454067550 0x7f8454069a10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f846c0af000 tx=0x7f846c0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.183+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8464051300 con 0x7f847c141cb0 2024-01-15T18:15:34.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.351+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f847c15d710 con 0x7f8454067550 2024-01-15T18:15:34.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.351+0000 7f8479600640 1 -- 172.21.15.143:0/3768484801 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f846c00f080 con 0x7f8454067550 2024-01-15T18:15:34.349 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:15:34.349 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.350 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:15:34.350 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:34.350 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8454067550 msgr2=0x7f8454069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8454067550 0x7f8454069a10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f846c0af000 tx=0x7f846c0046d0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 msgr2=0x7f847c142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 0x7f847c142130 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f846400e780 tx=0x7f846400ec40 comp rx=0 tx=0).stop 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 shutdown_connections 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8454067550 0x7f8454069a10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f847c13f500 0x7f847c13f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f847c141cb0 0x7f847c142130 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 --2- 172.21.15.143:0/3768484801 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f845c003b40 0x7f847c1f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 >> 172.21.15.143:0/3768484801 conn(0x7f847c06c3f0 msgr2=0x7f847c15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 shutdown_connections 2024-01-15T18:15:34.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.355+0000 7f8484400640 1 -- 172.21.15.143:0/3768484801 wait complete. 2024-01-15T18:15:34.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:15:34.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e2da00640 1 Processor -- start 2024-01-15T18:15:34.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e2da00640 1 -- start start 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e2da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7e280cf910 0x7f7e280cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e2da00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e280cbff0 con 0x7f7e2806bfb0 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e2da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e280d0c50 con 0x7f7e280b0310 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e2da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e280cb6c0 con 0x7f7e280cf910 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e26a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7e2806bfb0 0x7f7e2806c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48284/0 (socket says 172.21.15.143:48284) 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e26a00640 1 -- 172.21.15.143:0/692342212 learned_addr learned my addr 172.21.15.143:0/692342212 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e27400640 1 --2- 172.21.15.143:0/692342212 >> v2:172.21.15.204:3300/0 conn(0x7f7e280cf910 0x7f7e280cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114095113 0 0) 0x7f7e280cbff0 con 0x7f7e2806bfb0 2024-01-15T18:15:34.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e08003530 con 0x7f7e2806bfb0 2024-01-15T18:15:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212390892 0 0) 0x7f7e280d0c50 con 0x7f7e280b0310 2024-01-15T18:15:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e280cbff0 con 0x7f7e280b0310 2024-01-15T18:15:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3300805733 0 0) 0x7f7e08003530 con 0x7f7e2806bfb0 2024-01-15T18:15:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e27400640 1 -- 172.21.15.143:0/692342212 >> v1:172.21.15.204:6789/0 conn(0x7f7e280b0310 legacy=0x7f7e280cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e27400640 1 -- 172.21.15.143:0/692342212 >> v1:172.21.15.143:6789/0 conn(0x7f7e2806bfb0 legacy=0x7f7e2806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.515+0000 7f7e27400640 1 -- 172.21.15.143:0/692342212 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280cb9a0 con 0x7f7e280cf910 2024-01-15T18:15:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e27400640 1 --2- 172.21.15.143:0/692342212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cf910 0x7f7e280cfdc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009d90 tx=0x7f7e2001c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d3484b2111f2d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18001680 con 0x7f7e280cf910 2024-01-15T18:15:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e200a5520 con 0x7f7e280cf910 2024-01-15T18:15:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e25600640 1 -- 172.21.15.143:0/692342212 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e200a58b0 con 0x7f7e280cf910 2024-01-15T18:15:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 -- 172.21.15.143:0/692342212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cf910 msgr2=0x7f7e280cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 --2- 172.21.15.143:0/692342212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cf910 0x7f7e280cfdc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009d90 tx=0x7f7e2001c7b0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 -- 172.21.15.143:0/692342212 shutdown_connections 2024-01-15T18:15:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 --2- 172.21.15.143:0/692342212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280cf910 0x7f7e280cfdc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 -- 172.21.15.143:0/692342212 >> 172.21.15.143:0/692342212 conn(0x7f7e280d0ed0 msgr2=0x7f7e280d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 -- 172.21.15.143:0/692342212 shutdown_connections 2024-01-15T18:15:34.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.519+0000 7f7e2da00640 1 -- 172.21.15.143:0/692342212 wait complete. 2024-01-15T18:15:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 Processor -- start 2024-01-15T18:15:34.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 -- start start 2024-01-15T18:15:34.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e2806bfb0 0x7f7e280cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280b0310 0x7f7e280af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 0x7f7e280c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e28154420 con 0x7f7e280cf910 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280c9410 con 0x7f7e280b0310 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e280c96f0 con 0x7f7e2806bfb0 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 0x7f7e280c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 0x7f7e280c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35508/0 (socket says 172.21.15.143:35508) 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 -- 172.21.15.143:0/3346937360 learned_addr learned my addr 172.21.15.143:0/3346937360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e26a00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e2806bfb0 0x7f7e280cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e26000640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280b0310 0x7f7e280af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 -- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e2806bfb0 msgr2=0x7f7e280cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e2806bfb0 0x7f7e280cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 -- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280b0310 msgr2=0x7f7e280af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280b0310 0x7f7e280af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 -- 172.21.15.143:0/3346937360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e20009a70 con 0x7f7e280cf910 2024-01-15T18:15:34.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e26000640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280b0310 0x7f7e280af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:34.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e27400640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 0x7f7e280c88f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7e200b0040 tx=0x7f7e20019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20003ba0 con 0x7f7e280cf910 2024-01-15T18:15:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e20003d00 con 0x7f7e280cf910 2024-01-15T18:15:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20004260 con 0x7f7e280cf910 2024-01-15T18:15:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e28153800 con 0x7f7e280cf910 2024-01-15T18:15:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.523+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e28153d20 con 0x7f7e280cf910 2024-01-15T18:15:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.527+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e28153a20 con 0x7f7e280cf910 2024-01-15T18:15:34.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.527+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f7e200b4050 con 0x7f7e280cf910 2024-01-15T18:15:34.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.527+0000 7f7e1ea00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e00067580 0x7f7e00069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.527+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7e2011dff0 con 0x7f7e280cf910 2024-01-15T18:15:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.527+0000 7f7e26a00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e00067580 0x7f7e00069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.531+0000 7f7e26a00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e00067580 0x7f7e00069a40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000f70 tx=0x7f7e18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.531+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7e200ecb20 con 0x7f7e280cf910 2024-01-15T18:15:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.703+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e280c9580 con 0x7f7e00067580 2024-01-15T18:15:34.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e1ea00640 1 -- 172.21.15.143:0/3346937360 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1029 (secure 0 0 0) 0x7f7e280c9580 con 0x7f7e00067580 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99311" 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99314" 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993115" 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99316" 2024-01-15T18:15:34.711 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99312" 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993116" 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99315" 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:15:34.712 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993113" 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993111" 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.713 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993117" 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99319" 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993120" 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993118" 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993112" 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.714 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993114" 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99318" 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993110" 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone993119" 2024-01-15T18:15:34.715 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99317" 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone99313" 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:34.716 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:15:34.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e00067580 msgr2=0x7f7e00069a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e00067580 0x7f7e00069a40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e18000f70 tx=0x7f7e18009040 comp rx=0 tx=0).stop 2024-01-15T18:15:34.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 msgr2=0x7f7e280c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 0x7f7e280c88f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7e200b0040 tx=0x7f7e20019550 comp rx=0 tx=0).stop 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 shutdown_connections 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7e00067580 0x7f7e00069a40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7e280cf910 0x7f7e280c88f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e280b0310 0x7f7e280af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 --2- 172.21.15.143:0/3346937360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7e2806bfb0 0x7f7e280cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 >> 172.21.15.143:0/3346937360 conn(0x7f7e280d0ed0 msgr2=0x7f7e2806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 shutdown_connections 2024-01-15T18:15:34.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.715+0000 7f7e2da00640 1 -- 172.21.15.143:0/3346937360 wait complete. 2024-01-15T18:15:34.733 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99311 2024-01-15T18:15:34.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 Processor -- start 2024-01-15T18:15:34.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 -- start start 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa69015b260 0x7fa6900cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6900d19f0 0x7fa6900cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6900c4d90 con 0x7fa6900c5220 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69015b640 con 0x7fa6900d19f0 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa695200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6900cf800 con 0x7fa69015b260 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6900d19f0 0x7fa6900cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6900d19f0 0x7fa6900cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35516/0 (socket says 172.21.15.143:35516) 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 -- 172.21.15.143:0/1061248524 learned_addr learned my addr 172.21.15.143:0/1061248524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:34.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68de00640 1 --2- 172.21.15.143:0/1061248524 >> v2:172.21.15.204:3300/0 conn(0x7fa69015b260 0x7fa6900cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 -- 172.21.15.143:0/1061248524 >> v2:172.21.15.204:3300/0 conn(0x7fa69015b260 msgr2=0x7fa6900cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:34.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 --2- 172.21.15.143:0/1061248524 >> v2:172.21.15.204:3300/0 conn(0x7fa69015b260 0x7fa6900cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 -- 172.21.15.143:0/1061248524 >> v1:172.21.15.204:6789/0 conn(0x7fa6900c5220 legacy=0x7fa6900c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.883+0000 7fa68d400640 1 -- 172.21.15.143:0/1061248524 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6900cfae0 con 0x7fa6900d19f0 2024-01-15T18:15:34.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa68d400640 1 --2- 172.21.15.143:0/1061248524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa6900cdd50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa6900681e0 tx=0x7fa67801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6c704ebed1a624 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa68ca00640 1 -- 172.21.15.143:0/1061248524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780ab600 con 0x7fa6900d19f0 2024-01-15T18:15:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa68ca00640 1 -- 172.21.15.143:0/1061248524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6780a7050 con 0x7fa6900d19f0 2024-01-15T18:15:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa68ca00640 1 -- 172.21.15.143:0/1061248524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780ab980 con 0x7fa6900d19f0 2024-01-15T18:15:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 -- 172.21.15.143:0/1061248524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 msgr2=0x7fa6900cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 --2- 172.21.15.143:0/1061248524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa6900cdd50 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fa6900681e0 tx=0x7fa67801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 -- 172.21.15.143:0/1061248524 shutdown_connections 2024-01-15T18:15:34.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 --2- 172.21.15.143:0/1061248524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa6900cdd50 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 --2- 172.21.15.143:0/1061248524 >> v2:172.21.15.204:3300/0 conn(0x7fa69015b260 0x7fa6900cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 -- 172.21.15.143:0/1061248524 >> 172.21.15.143:0/1061248524 conn(0x7fa6900d0ed0 msgr2=0x7fa6900d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 -- 172.21.15.143:0/1061248524 shutdown_connections 2024-01-15T18:15:34.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.887+0000 7fa695200640 1 -- 172.21.15.143:0/1061248524 wait complete. 2024-01-15T18:15:34.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 Processor -- start 2024-01-15T18:15:34.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 -- start start 2024-01-15T18:15:34.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6900c5220 0x7fa6901f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa690154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa69015b260 0x7fa69009b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69009ea90 con 0x7fa6900d19f0 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6900b4100 con 0x7fa69015b260 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69009c060 con 0x7fa6900c5220 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa690154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa690154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35520/0 (socket says 172.21.15.143:35520) 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 -- 172.21.15.143:0/2995337703 learned_addr learned my addr 172.21.15.143:0/2995337703 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68e800640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa69015b260 0x7fa69009b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68de00640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6900c5220 0x7fa6901f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 -- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6900c5220 msgr2=0x7fa6901f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6900c5220 0x7fa6901f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 -- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa69015b260 msgr2=0x7fa69009b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa69015b260 0x7fa69009b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 -- 172.21.15.143:0/2995337703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780094d0 con 0x7fa6900d19f0 2024-01-15T18:15:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa68d400640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa690154fe0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa6780ae040 tx=0x7fa67801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780b6420 con 0x7fa6900d19f0 2024-01-15T18:15:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6780b65a0 con 0x7fa6900d19f0 2024-01-15T18:15:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69009c340 con 0x7fa6900d19f0 2024-01-15T18:15:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780b6950 con 0x7fa6900d19f0 2024-01-15T18:15:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.891+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6900b3d90 con 0x7fa6900d19f0 2024-01-15T18:15:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.895+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6900b4290 con 0x7fa6900d19f0 2024-01-15T18:15:34.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.895+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7fa6780b2040 con 0x7fa6900d19f0 2024-01-15T18:15:34.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.895+0000 7fa686000640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa668067550 0x7fa668069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.895+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa67811d860 con 0x7fa6900d19f0 2024-01-15T18:15:34.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.895+0000 7fa68de00640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa668067550 0x7fa668069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.895+0000 7fa68de00640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa668067550 0x7fa668069a10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa6900cb0f0 tx=0x7fa680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:34.899+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6780ec270 con 0x7fa6900d19f0 2024-01-15T18:15:35.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.075+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99311", "target": ["mon-mgr", ""]}) v1 -- 0x7fa690067500 con 0x7fa668067550 2024-01-15T18:15:35.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa686000640 1 -- 172.21.15.143:0/2995337703 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa690067500 con 0x7fa668067550 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa668067550 msgr2=0x7fa668069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa668067550 0x7fa668069a10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa6900cb0f0 tx=0x7fa680008040 comp rx=0 tx=0).stop 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 msgr2=0x7fa690154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa690154fe0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fa6780ae040 tx=0x7fa67801ad10 comp rx=0 tx=0).stop 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 shutdown_connections 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa668067550 0x7fa668069a10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa69015b260 0x7fa69009b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6900d19f0 0x7fa690154fe0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 --2- 172.21.15.143:0/2995337703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6900c5220 0x7fa6901f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 >> 172.21.15.143:0/2995337703 conn(0x7fa6900d0ed0 msgr2=0x7fa69006b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 shutdown_connections 2024-01-15T18:15:35.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.091+0000 7fa695200640 1 -- 172.21.15.143:0/2995337703 wait complete. 2024-01-15T18:15:35.090 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.095+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.090 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.095+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.090 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.095+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.090 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.095+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.090 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.095+0000 7f3cade00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.107 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.111+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.107 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.111+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.107 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.111+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.107 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.111+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.107 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:15:35.111+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:35.114 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99314 2024-01-15T18:15:35.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.259+0000 7fc1c8800640 1 Processor -- start 2024-01-15T18:15:35.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.259+0000 7fc1c8800640 1 -- start start 2024-01-15T18:15:35.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c8800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc1c00b29a0 0x7fc1c006de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c8800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c015e8a0 con 0x7fc1c0070d80 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c8800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c00b3270 con 0x7fc1c00b2510 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c8800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c00b2f90 con 0x7fc1c00b29a0 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c5e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc1c0070d80 0x7fc1c0071160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48292/0 (socket says 172.21.15.143:48292) 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c5e00640 1 -- 172.21.15.143:0/1545937385 learned_addr learned my addr 172.21.15.143:0/1545937385 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c6800640 1 --2- 172.21.15.143:0/1545937385 >> v2:172.21.15.204:3300/0 conn(0x7fc1c00b29a0 0x7fc1c006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978775390 0 0) 0x7fc1c015e8a0 con 0x7fc1c0070d80 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1a0003530 con 0x7fc1c0070d80 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71505296 0 0) 0x7fc1c00b3270 con 0x7fc1c00b2510 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c015e8a0 con 0x7fc1c00b2510 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1455875382 0 0) 0x7fc1a0003530 con 0x7fc1c0070d80 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c6800640 1 -- 172.21.15.143:0/1545937385 >> v1:172.21.15.204:6789/0 conn(0x7fc1c00b2510 legacy=0x7fc1c006d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c6800640 1 -- 172.21.15.143:0/1545937385 >> v1:172.21.15.143:6789/0 conn(0x7fc1c0070d80 legacy=0x7fc1c0071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.263+0000 7fc1c6800640 1 -- 172.21.15.143:0/1545937385 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c006cc50 con 0x7fc1c00b29a0 2024-01-15T18:15:35.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c6800640 1 --2- 172.21.15.143:0/1545937385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b29a0 0x7fc1c006de40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009d90 tx=0x7fc1b801c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bbe7a59ddacc04f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:35.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0001680 con 0x7fc1c00b29a0 2024-01-15T18:15:35.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b80a55f0 con 0x7fc1c00b29a0 2024-01-15T18:15:35.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c4a00640 1 -- 172.21.15.143:0/1545937385 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80a5980 con 0x7fc1c00b29a0 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 -- 172.21.15.143:0/1545937385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b29a0 msgr2=0x7fc1c006de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1545937385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b29a0 0x7fc1c006de40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009d90 tx=0x7fc1b801c7b0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 -- 172.21.15.143:0/1545937385 shutdown_connections 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1545937385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b29a0 0x7fc1c006de40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 -- 172.21.15.143:0/1545937385 >> 172.21.15.143:0/1545937385 conn(0x7fc1c00b1610 msgr2=0x7fc1c00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 -- 172.21.15.143:0/1545937385 shutdown_connections 2024-01-15T18:15:35.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.267+0000 7fc1c8800640 1 -- 172.21.15.143:0/1545937385 wait complete. 2024-01-15T18:15:35.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 Processor -- start 2024-01-15T18:15:35.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- start start 2024-01-15T18:15:35.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 0x7fc1c0139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b2510 0x7fc1c013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c00b29a0 0x7fc1c013f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c015d080 con 0x7fc1c0070d80 2024-01-15T18:15:35.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c015c470 con 0x7fc1c00b2510 2024-01-15T18:15:35.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c015c750 con 0x7fc1c00b29a0 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 0x7fc1c0139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 0x7fc1c0139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35540/0 (socket says 172.21.15.143:35540) 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 -- 172.21.15.143:0/1330271981 learned_addr learned my addr 172.21.15.143:0/1330271981 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c6800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c00b29a0 0x7fc1c013f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5400640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b2510 0x7fc1c013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 -- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c00b29a0 msgr2=0x7fc1c013f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c00b29a0 0x7fc1c013f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 -- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b2510 msgr2=0x7fc1c013f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b2510 0x7fc1c013f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 -- 172.21.15.143:0/1330271981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8009a70 con 0x7fc1c0070d80 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c5e00640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 0x7fc1c0139920 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c780 tx=0x7fc1b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0005cf0 con 0x7fc1c0070d80 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b0005e50 con 0x7fc1c0070d80 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00072b0 con 0x7fc1c0070d80 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c015ca30 con 0x7fc1c0070d80 2024-01-15T18:15:35.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c015c140 con 0x7fc1c0070d80 2024-01-15T18:15:35.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.271+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c0074b00 con 0x7fc1c0070d80 2024-01-15T18:15:35.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.275+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7fc1b0007410 con 0x7fc1c0070d80 2024-01-15T18:15:35.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.275+0000 7fc1be000640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc198067550 0x7fc198069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.275+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc1b0082800 con 0x7fc1c0070d80 2024-01-15T18:15:35.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.275+0000 7fc1c5400640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc198067550 0x7fc198069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.275+0000 7fc1c5400640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc198067550 0x7fc198069a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc1c01f8de0 tx=0x7fc1a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:35.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.279+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1b00512c0 con 0x7fc1c0070d80 2024-01-15T18:15:35.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.451+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99314", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1c015c8c0 con 0x7fc198067550 2024-01-15T18:15:35.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1be000640 1 -- 172.21.15.143:0/1330271981 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc1c015c8c0 con 0x7fc198067550 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc198067550 msgr2=0x7fc198069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc198067550 0x7fc198069a10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc1c01f8de0 tx=0x7fc1a8009040 comp rx=0 tx=0).stop 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 msgr2=0x7fc1c0139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 0x7fc1c0139920 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc1b000c780 tx=0x7fc1b000cc40 comp rx=0 tx=0).stop 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 shutdown_connections 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc198067550 0x7fc198069a10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1c00b29a0 0x7fc1c013f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1c00b2510 0x7fc1c013f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 --2- 172.21.15.143:0/1330271981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1c0070d80 0x7fc1c0139920 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 >> 172.21.15.143:0/1330271981 conn(0x7fc1c00b1610 msgr2=0x7fc1c00b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:35.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 shutdown_connections 2024-01-15T18:15:35.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.467+0000 7fc1c8800640 1 -- 172.21.15.143:0/1330271981 wait complete. 2024-01-15T18:15:35.489 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993115 2024-01-15T18:15:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 Processor -- start 2024-01-15T18:15:35.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 -- start start 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b24b0 0x7f35080ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f35080b3bf0 0x7f35080b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3508070bf0 con 0x7f35080ce690 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350815dba0 con 0x7f35080b3bf0 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35080b4920 con 0x7f35080b24b0 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f35080b3bf0 0x7f35080b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f3507e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f35080ce690 0x7f35080b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48300/0 (socket says 172.21.15.143:48300) 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f3507e00640 1 -- 172.21.15.143:0/121132805 learned_addr learned my addr 172.21.15.143:0/121132805 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f3507400640 1 --2- 172.21.15.143:0/121132805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b24b0 0x7f35080ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ce00640 1 -- 172.21.15.143:0/121132805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b24b0 msgr2=0x7f35080ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:35.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ce00640 1 --2- 172.21.15.143:0/121132805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b24b0 0x7f35080ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ce00640 1 -- 172.21.15.143:0/121132805 >> v1:172.21.15.143:6789/0 conn(0x7f35080ce690 legacy=0x7f35080b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f350ce00640 1 -- 172.21.15.143:0/121132805 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080b4c00 con 0x7f35080b3bf0 2024-01-15T18:15:35.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.635+0000 7f3506a00640 1 -- 172.21.15.143:0/121132805 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611471308 0 0) 0x7f3508070bf0 con 0x7f35080ce690 2024-01-15T18:15:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ce00640 1 --2- 172.21.15.143:0/121132805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b3bf0 0x7f35080b4080 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3500009d90 tx=0x7f350001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3494f733ed8fca44 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f3506a00640 1 -- 172.21.15.143:0/121132805 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000a5a00 con 0x7f35080b3bf0 2024-01-15T18:15:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f3506a00640 1 -- 172.21.15.143:0/121132805 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35000a5b60 con 0x7f35080b3bf0 2024-01-15T18:15:35.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f3506a00640 1 -- 172.21.15.143:0/121132805 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000ad440 con 0x7f35080b3bf0 2024-01-15T18:15:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 -- 172.21.15.143:0/121132805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b3bf0 msgr2=0x7f35080b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 --2- 172.21.15.143:0/121132805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b3bf0 0x7f35080b4080 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3500009d90 tx=0x7f350001d380 comp rx=0 tx=0).stop 2024-01-15T18:15:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 -- 172.21.15.143:0/121132805 shutdown_connections 2024-01-15T18:15:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 --2- 172.21.15.143:0/121132805 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b3bf0 0x7f35080b4080 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 --2- 172.21.15.143:0/121132805 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b24b0 0x7f35080ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 -- 172.21.15.143:0/121132805 >> 172.21.15.143:0/121132805 conn(0x7f350806c3f0 msgr2=0x7f350806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.639+0000 7f350ee00640 1 -- 172.21.15.143:0/121132805 shutdown_connections 2024-01-15T18:15:35.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 -- 172.21.15.143:0/121132805 wait complete. 2024-01-15T18:15:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 Processor -- start 2024-01-15T18:15:35.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 -- start start 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 0x7f3508146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b3bf0 0x7f3508159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35080ce690 0x7f3508140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35081628d0 con 0x7f35080b24b0 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508146d80 con 0x7f35080ce690 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35081457a0 con 0x7f35080b3bf0 2024-01-15T18:15:35.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 0x7f3508146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 0x7f3508146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35566/0 (socket says 172.21.15.143:35566) 2024-01-15T18:15:35.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 -- 172.21.15.143:0/2850878925 learned_addr learned my addr 172.21.15.143:0/2850878925 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507400640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b3bf0 0x7f3508159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f350ce00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35080ce690 0x7f3508140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 -- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b3bf0 msgr2=0x7f3508159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b3bf0 0x7f3508159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 -- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35080ce690 msgr2=0x7f3508140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35080ce690 0x7f3508140220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 -- 172.21.15.143:0/2850878925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500009a70 con 0x7f35080b24b0 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f3507e00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 0x7f3508146840 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c480 tx=0x7f34f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.643+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80074e0 con 0x7f35080b24b0 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f8007660 con 0x7f35080b24b0 2024-01-15T18:15:35.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004a30 con 0x7f35080b24b0 2024-01-15T18:15:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3508145a80 con 0x7f35080b24b0 2024-01-15T18:15:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3508146040 con 0x7f35080b24b0 2024-01-15T18:15:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3508067500 con 0x7f35080b24b0 2024-01-15T18:15:35.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f34f8004b90 con 0x7f35080b24b0 2024-01-15T18:15:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f34ffe00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34e0067550 0x7f34e0069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f34f80821d0 con 0x7f35080b24b0 2024-01-15T18:15:35.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f3507400640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34e0067550 0x7f34e0069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:35.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.647+0000 7f3507400640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34e0067550 0x7f34e0069a10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3508146f90 tx=0x7f34f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:35.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.651+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f34f8054720 con 0x7f35080b24b0 2024-01-15T18:15:35.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.823+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993115", "target": ["mon-mgr", ""]}) v1 -- 0x7f350806e260 con 0x7f34e0067550 2024-01-15T18:15:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.839+0000 7f34ffe00640 1 -- 172.21.15.143:0/2850878925 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f350806e260 con 0x7f34e0067550 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.839+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34e0067550 msgr2=0x7f34e0069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.839+0000 7f350ee00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34e0067550 0x7f34e0069a10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3508146f90 tx=0x7f34f0008040 comp rx=0 tx=0).stop 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.839+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 msgr2=0x7f3508146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.839+0000 7f350ee00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 0x7f3508146840 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f34f800c480 tx=0x7f34f800c940 comp rx=0 tx=0).stop 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 shutdown_connections 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f34e0067550 0x7f34e0069a10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35080ce690 0x7f3508140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35080b3bf0 0x7f3508159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 --2- 172.21.15.143:0/2850878925 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35080b24b0 0x7f3508146840 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 >> 172.21.15.143:0/2850878925 conn(0x7f350806c3f0 msgr2=0x7f350815a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 shutdown_connections 2024-01-15T18:15:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:35.843+0000 7f350ee00640 1 -- 172.21.15.143:0/2850878925 wait complete. 2024-01-15T18:15:35.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99316 2024-01-15T18:15:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 Processor -- start 2024-01-15T18:15:36.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 -- start start 2024-01-15T18:15:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d4143780 0x7f03d40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03d40ce520 0x7f03d4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d40ceb50 con 0x7f03d40720f0 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d4072580 con 0x7f03d40ce520 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d9200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d415c110 con 0x7f03d4143780 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03d40ce520 0x7f03d4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d1400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f03d40ce520 0x7f03d4071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35572/0 (socket says 172.21.15.143:35572) 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.007+0000 7f03d1400640 1 -- 172.21.15.143:0/1394833525 learned_addr learned my addr 172.21.15.143:0/1394833525 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d1e00640 1 --2- 172.21.15.143:0/1394833525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d4143780 0x7f03d40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d1400640 1 -- 172.21.15.143:0/1394833525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d4143780 msgr2=0x7f03d40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d1400640 1 --2- 172.21.15.143:0/1394833525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d4143780 0x7f03d40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d1400640 1 -- 172.21.15.143:0/1394833525 >> v1:172.21.15.143:6789/0 conn(0x7f03d40720f0 legacy=0x7f03d415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d1400640 1 -- 172.21.15.143:0/1394833525 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d415b7e0 con 0x7f03d40ce520 2024-01-15T18:15:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d1400640 1 --2- 172.21.15.143:0/1394833525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d4071bb0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009870 tx=0x7f03bc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62eaf7e054c4f475 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d0a00640 1 -- 172.21.15.143:0/1394833525 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0ab650 con 0x7f03d40ce520 2024-01-15T18:15:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d0a00640 1 -- 172.21.15.143:0/1394833525 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc0ab7b0 con 0x7f03d40ce520 2024-01-15T18:15:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.011+0000 7f03d0a00640 1 -- 172.21.15.143:0/1394833525 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0abb40 con 0x7f03d40ce520 2024-01-15T18:15:36.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- 172.21.15.143:0/1394833525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 msgr2=0x7f03d4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 --2- 172.21.15.143:0/1394833525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d4071bb0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f03bc009870 tx=0x7f03bc01c540 comp rx=0 tx=0).stop 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- 172.21.15.143:0/1394833525 shutdown_connections 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 --2- 172.21.15.143:0/1394833525 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d4071bb0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 --2- 172.21.15.143:0/1394833525 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d4143780 0x7f03d40cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- 172.21.15.143:0/1394833525 >> 172.21.15.143:0/1394833525 conn(0x7f03d406c3f0 msgr2=0x7f03d406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- 172.21.15.143:0/1394833525 shutdown_connections 2024-01-15T18:15:36.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- 172.21.15.143:0/1394833525 wait complete. 2024-01-15T18:15:36.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 Processor -- start 2024-01-15T18:15:36.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- start start 2024-01-15T18:15:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d40720f0 0x7f03d413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d41399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d4143780 0x7f03d4156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d413e020 con 0x7f03d40ce520 2024-01-15T18:15:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d413cf80 con 0x7f03d4143780 2024-01-15T18:15:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d9200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d413d260 con 0x7f03d40720f0 2024-01-15T18:15:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d41399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d41399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35588/0 (socket says 172.21.15.143:35588) 2024-01-15T18:15:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.015+0000 7f03d1400640 1 -- 172.21.15.143:0/3815695510 learned_addr learned my addr 172.21.15.143:0/3815695510 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1e00640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d40720f0 0x7f03d413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d2800640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d4143780 0x7f03d4156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1400640 1 -- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d40720f0 msgr2=0x7f03d413c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1400640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d40720f0 0x7f03d413c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1400640 1 -- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d4143780 msgr2=0x7f03d4156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1400640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d4143780 0x7f03d4156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1400640 1 -- 172.21.15.143:0/3815695510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc0094d0 con 0x7f03d40ce520 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d1400640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d41399f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0ad040 tx=0x7f03bc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0b4800 con 0x7f03d40ce520 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc0b4960 con 0x7f03d40ce520 2024-01-15T18:15:36.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d413d540 con 0x7f03d40ce520 2024-01-15T18:15:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc0b4cf0 con 0x7f03d40ce520 2024-01-15T18:15:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d413db00 con 0x7f03d40ce520 2024-01-15T18:15:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d413dcf0 con 0x7f03d40ce520 2024-01-15T18:15:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.019+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f03bc0be3f0 con 0x7f03d40ce520 2024-01-15T18:15:36.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.023+0000 7f03ca000640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03ac067550 0x7f03ac069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.023+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f03bc11d830 con 0x7f03d40ce520 2024-01-15T18:15:36.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.023+0000 7f03d1e00640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03ac067550 0x7f03ac069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.023+0000 7f03d1e00640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03ac067550 0x7f03ac069a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f03d413e230 tx=0x7f03c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.027+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f03bc0ec690 con 0x7f03d40ce520 2024-01-15T18:15:36.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.203+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99316", "target": ["mon-mgr", ""]}) v1 -- 0x7f03d413d3d0 con 0x7f03ac067550 2024-01-15T18:15:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.215+0000 7f03ca000640 1 -- 172.21.15.143:0/3815695510 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f03d413d3d0 con 0x7f03ac067550 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03ac067550 msgr2=0x7f03ac069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03ac067550 0x7f03ac069a10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f03d413e230 tx=0x7f03c4008040 comp rx=0 tx=0).stop 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 msgr2=0x7f03d41399f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d41399f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0ad040 tx=0x7f03bc01ad80 comp rx=0 tx=0).stop 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 shutdown_connections 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f03ac067550 0x7f03ac069a10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03d4143780 0x7f03d4156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03d40ce520 0x7f03d41399f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 --2- 172.21.15.143:0/3815695510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f03d40720f0 0x7f03d413c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 >> 172.21.15.143:0/3815695510 conn(0x7f03d406c3f0 msgr2=0x7f03d40aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 shutdown_connections 2024-01-15T18:15:36.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.219+0000 7f03d9200640 1 -- 172.21.15.143:0/3815695510 wait complete. 2024-01-15T18:15:36.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99312 2024-01-15T18:15:36.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 Processor -- start 2024-01-15T18:15:36.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 -- start start 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9e240b24b0 0x7f9e240ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e240b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e24070bf0 con 0x7f9e240ce690 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2415dba0 con 0x7f9e240b24b0 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e2a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240b4920 con 0x7f9e240b3bf0 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e23400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9e240ce690 0x7f9e240b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:48312/0 (socket says 172.21.15.143:48312) 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e23400640 1 -- 172.21.15.143:0/2222010312 learned_addr learned my addr 172.21.15.143:0/2222010312 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22a00640 1 --2- 172.21.15.143:0/2222010312 >> v2:172.21.15.204:3300/0 conn(0x7f9e240b24b0 0x7f9e240ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e23e00640 1 --2- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e240b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22000640 1 -- 172.21.15.143:0/2222010312 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3980594351 0 0) 0x7f9e24070bf0 con 0x7f9e240ce690 2024-01-15T18:15:36.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22000640 1 -- 172.21.15.143:0/2222010312 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e04003530 con 0x7f9e240ce690 2024-01-15T18:15:36.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22000640 1 -- 172.21.15.143:0/2222010312 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3774373162 0 0) 0x7f9e04003530 con 0x7f9e240ce690 2024-01-15T18:15:36.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22a00640 1 -- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 msgr2=0x7f9e240b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:36.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22a00640 1 --2- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e240b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22a00640 1 -- 172.21.15.143:0/2222010312 >> v1:172.21.15.143:6789/0 conn(0x7f9e240ce690 legacy=0x7f9e240b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.383+0000 7f9e22a00640 1 -- 172.21.15.143:0/2222010312 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240b4c00 con 0x7f9e240b24b0 2024-01-15T18:15:36.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.387+0000 7f9e22a00640 1 --2- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e240ac8e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0097c0 tx=0x7f9e0c01cdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=194179abce308b47 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.387+0000 7f9e22000640 1 -- 172.21.15.143:0/2222010312 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14001680 con 0x7f9e240b24b0 2024-01-15T18:15:36.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.387+0000 7f9e22000640 1 -- 172.21.15.143:0/2222010312 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c0a7050 con 0x7f9e240b24b0 2024-01-15T18:15:36.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.387+0000 7f9e22000640 1 -- 172.21.15.143:0/2222010312 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c0ab620 con 0x7f9e240b24b0 2024-01-15T18:15:36.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 msgr2=0x7f9e240ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e240ac8e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0097c0 tx=0x7f9e0c01cdd0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- 172.21.15.143:0/2222010312 shutdown_connections 2024-01-15T18:15:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e240b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2222010312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e240ac8e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- 172.21.15.143:0/2222010312 >> 172.21.15.143:0/2222010312 conn(0x7f9e2406c3f0 msgr2=0x7f9e2406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- 172.21.15.143:0/2222010312 shutdown_connections 2024-01-15T18:15:36.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- 172.21.15.143:0/2222010312 wait complete. 2024-01-15T18:15:36.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 Processor -- start 2024-01-15T18:15:36.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- start start 2024-01-15T18:15:36.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e24158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e2415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 0x7f9e240ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2413f3f0 con 0x7f9e240ce690 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24144400 con 0x7f9e240b24b0 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e2a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e24159040 con 0x7f9e240b3bf0 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e23e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 0x7f9e240ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e23e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 0x7f9e240ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35592/0 (socket says 172.21.15.143:35592) 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e23e00640 1 -- 172.21.15.143:0/2975051860 learned_addr learned my addr 172.21.15.143:0/2975051860 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e22a00640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e2415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.391+0000 7f9e23400640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e24158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23e00640 1 -- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 msgr2=0x7f9e2415b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23e00640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e2415b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23e00640 1 -- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 msgr2=0x7f9e24158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23e00640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e24158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23e00640 1 -- 172.21.15.143:0/2975051860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c009420 con 0x7f9e240ce690 2024-01-15T18:15:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23400640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e24158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23e00640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 0x7f9e240ada60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00efc0 tx=0x7f9e1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c009b10 con 0x7f9e240ce690 2024-01-15T18:15:36.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c00f040 con 0x7f9e240ce690 2024-01-15T18:15:36.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c0146c0 con 0x7f9e240ce690 2024-01-15T18:15:36.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2413f640 con 0x7f9e240ce690 2024-01-15T18:15:36.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e24144650 con 0x7f9e240ce690 2024-01-15T18:15:36.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e24067500 con 0x7f9e240ce690 2024-01-15T18:15:36.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f9e1c014820 con 0x7f9e240ce690 2024-01-15T18:15:36.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e1b400640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dfc067550 0x7f9dfc069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9e1c083810 con 0x7f9e240ce690 2024-01-15T18:15:36.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.395+0000 7f9e23400640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dfc067550 0x7f9dfc069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.399+0000 7f9e23400640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dfc067550 0x7f9dfc069a10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e14005890 tx=0x7f9e14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.399+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9e1c052500 con 0x7f9e240ce690 2024-01-15T18:15:36.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.575+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99312", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e2413faf0 con 0x7f9dfc067550 2024-01-15T18:15:36.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.587+0000 7f9e1b400640 1 -- 172.21.15.143:0/2975051860 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9e140058e0 con 0x7f9dfc067550 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dfc067550 msgr2=0x7f9dfc069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dfc067550 0x7f9dfc069a10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e14005890 tx=0x7f9e14009040 comp rx=0 tx=0).stop 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 msgr2=0x7f9e240ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 0x7f9e240ada60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c00efc0 tx=0x7f9e1c00c490 comp rx=0 tx=0).stop 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 shutdown_connections 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9dfc067550 0x7f9dfc069a10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e240ce690 0x7f9e240ada60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e240b3bf0 0x7f9e2415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 --2- 172.21.15.143:0/2975051860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e240b24b0 0x7f9e24158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 >> 172.21.15.143:0/2975051860 conn(0x7f9e2406c3f0 msgr2=0x7f9e2415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 shutdown_connections 2024-01-15T18:15:36.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.591+0000 7f9e2a600640 1 -- 172.21.15.143:0/2975051860 wait complete. 2024-01-15T18:15:36.609 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993116 2024-01-15T18:15:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 Processor -- start 2024-01-15T18:15:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 -- start start 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9354143430 0x7f93541462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9354143c80 0x7f9354146800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93540b37a0 con 0x7f93540b2290 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935406ca70 con 0x7f9354143430 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f935c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9354143810 con 0x7f9354143c80 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9354143430 0x7f93541462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9354143430 0x7f93541462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35606/0 (socket says 172.21.15.143:35606) 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 -- 172.21.15.143:0/3548121055 learned_addr learned my addr 172.21.15.143:0/3548121055 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359200640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9354143c80 0x7f9354146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 -- 172.21.15.143:0/3548121055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9354143c80 msgr2=0x7f9354146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9354143c80 0x7f9354146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 -- 172.21.15.143:0/3548121055 >> v1:172.21.15.204:6789/0 conn(0x7f93540b2290 legacy=0x7f93540b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359c00640 1 -- 172.21.15.143:0/3548121055 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b3a80 con 0x7f9354143430 2024-01-15T18:15:36.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.755+0000 7f9359200640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9354143c80 0x7f9354146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:36.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f9359c00640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f93541462c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f93440098a0 tx=0x7f934401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42d2e9453e88c7f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f9353e00640 1 -- 172.21.15.143:0/3548121055 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440aa650 con 0x7f9354143430 2024-01-15T18:15:36.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f9353e00640 1 -- 172.21.15.143:0/3548121055 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93440aa7b0 con 0x7f9354143430 2024-01-15T18:15:36.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f9353e00640 1 -- 172.21.15.143:0/3548121055 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93440aab40 con 0x7f9354143430 2024-01-15T18:15:36.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f935c600640 1 -- 172.21.15.143:0/3548121055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 msgr2=0x7f93541462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f935c600640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f93541462c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f93440098a0 tx=0x7f934401c4f0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f935c600640 1 -- 172.21.15.143:0/3548121055 shutdown_connections 2024-01-15T18:15:36.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f935c600640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9354143c80 0x7f9354146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f935c600640 1 --2- 172.21.15.143:0/3548121055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f93541462c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.759+0000 7f935c600640 1 -- 172.21.15.143:0/3548121055 >> 172.21.15.143:0/3548121055 conn(0x7f9354066720 msgr2=0x7f935406dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 -- 172.21.15.143:0/3548121055 shutdown_connections 2024-01-15T18:15:36.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 -- 172.21.15.143:0/3548121055 wait complete. 2024-01-15T18:15:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 Processor -- start 2024-01-15T18:15:36.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 -- start start 2024-01-15T18:15:36.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93540b2290 0x7f93541417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f935413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9354143c80 0x7f935413d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354141ce0 con 0x7f9354143430 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354130bc0 con 0x7f9354143c80 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9354130e80 con 0x7f93540b2290 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f935413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f935413ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35618/0 (socket says 172.21.15.143:35618) 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 -- 172.21.15.143:0/1467676756 learned_addr learned my addr 172.21.15.143:0/1467676756 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f935b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9354143c80 0x7f935413d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359c00640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93540b2290 0x7f93541417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 -- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93540b2290 msgr2=0x7f93541417a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93540b2290 0x7f93541417a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 -- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9354143c80 msgr2=0x7f935413d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9354143c80 0x7f935413d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.763+0000 7f9359200640 1 -- 172.21.15.143:0/1467676756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440094d0 con 0x7f9354143430 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9359200640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f935413ccd0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c7f0 tx=0x7f933c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c007820 con 0x7f9354143430 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c007980 con 0x7f9354143430 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93540b2c30 con 0x7f9354143430 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c007d30 con 0x7f9354143430 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9354131440 con 0x7f9354143430 2024-01-15T18:15:36.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9354067680 con 0x7f9354143430 2024-01-15T18:15:36.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f933c0113f0 con 0x7f9354143430 2024-01-15T18:15:36.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9351600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f932c067550 0x7f932c069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f933c083340 con 0x7f9354143430 2024-01-15T18:15:36.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.767+0000 7f9359c00640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f932c067550 0x7f932c069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.771+0000 7f9359c00640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f932c067550 0x7f932c069a10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f93440ae000 tx=0x7f93440b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.771+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f933c0b7280 con 0x7f9354143430 2024-01-15T18:15:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.951+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993116", "target": ["mon-mgr", ""]}) v1 -- 0x7f93540b2ab0 con 0x7f932c067550 2024-01-15T18:15:36.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.963+0000 7f9351600640 1 -- 172.21.15.143:0/1467676756 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f93440aa9a0 con 0x7f932c067550 2024-01-15T18:15:36.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.963+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f932c067550 msgr2=0x7f932c069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.963+0000 7f935c600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f932c067550 0x7f932c069a10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f93440ae000 tx=0x7f93440b5040 comp rx=0 tx=0).stop 2024-01-15T18:15:36.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.963+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 msgr2=0x7f935413ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.963+0000 7f935c600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f935413ccd0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f933c00c7f0 tx=0x7f933c00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.963+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 shutdown_connections 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f932c067550 0x7f932c069a10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9354143c80 0x7f935413d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9354143430 0x7f935413ccd0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 --2- 172.21.15.143:0/1467676756 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f93540b2290 0x7f93541417a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 >> 172.21.15.143:0/1467676756 conn(0x7f9354066720 msgr2=0x7f93540b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 shutdown_connections 2024-01-15T18:15:36.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:36.967+0000 7f935c600640 1 -- 172.21.15.143:0/1467676756 wait complete. 2024-01-15T18:15:36.985 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99315 2024-01-15T18:15:37.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.135+0000 7f2afc400640 1 Processor -- start 2024-01-15T18:15:37.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.135+0000 7f2afc400640 1 -- start start 2024-01-15T18:15:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2afc400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2af4071e00 0x7f2af40c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2afc400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2af406b5a0 0x7f2af406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2afc400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af406bf30 con 0x7f2af415d570 2024-01-15T18:15:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2afc400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40ce6d0 con 0x7f2af406b5a0 2024-01-15T18:15:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2afc400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40c5f10 con 0x7f2af4071e00 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2af4071e00 0x7f2af40c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2af4071e00 0x7f2af40c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:53378/0 (socket says 172.21.15.143:53378) 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 -- 172.21.15.143:0/4105823897 learned_addr learned my addr 172.21.15.143:0/4105823897 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2afa400640 1 --2- 172.21.15.143:0/4105823897 >> v2:172.21.15.143:3300/0 conn(0x7f2af406b5a0 0x7f2af406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 -- 172.21.15.143:0/4105823897 >> v2:172.21.15.143:3300/0 conn(0x7f2af406b5a0 msgr2=0x7f2af406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 --2- 172.21.15.143:0/4105823897 >> v2:172.21.15.143:3300/0 conn(0x7f2af406b5a0 0x7f2af406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 -- 172.21.15.143:0/4105823897 >> v1:172.21.15.204:6789/0 conn(0x7f2af415d570 legacy=0x7f2af40cf140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:15:37.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.139+0000 7f2af9000640 1 -- 172.21.15.143:0/4105823897 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40ce940 con 0x7f2af4071e00 2024-01-15T18:15:37.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.143+0000 7f2af9000640 1 --2- 172.21.15.143:0/4105823897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af4071e00 0x7f2af40c55f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2af40748d0 tx=0x7f2adc01eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb235573acf3dcae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2af3e00640 1 -- 172.21.15.143:0/4105823897 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0a7c50 con 0x7f2af4071e00 2024-01-15T18:15:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2af3e00640 1 -- 172.21.15.143:0/4105823897 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc0a7db0 con 0x7f2af4071e00 2024-01-15T18:15:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2af3e00640 1 -- 172.21.15.143:0/4105823897 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0af660 con 0x7f2af4071e00 2024-01-15T18:15:37.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 -- 172.21.15.143:0/4105823897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af4071e00 msgr2=0x7f2af40c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 --2- 172.21.15.143:0/4105823897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af4071e00 0x7f2af40c55f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2af40748d0 tx=0x7f2adc01eb60 comp rx=0 tx=0).stop 2024-01-15T18:15:37.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 -- 172.21.15.143:0/4105823897 shutdown_connections 2024-01-15T18:15:37.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 --2- 172.21.15.143:0/4105823897 >> v2:172.21.15.143:3300/0 conn(0x7f2af406b5a0 0x7f2af406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 --2- 172.21.15.143:0/4105823897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af4071e00 0x7f2af40c55f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 -- 172.21.15.143:0/4105823897 >> 172.21.15.143:0/4105823897 conn(0x7f2af40cc800 msgr2=0x7f2af40ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:37.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 -- 172.21.15.143:0/4105823897 shutdown_connections 2024-01-15T18:15:37.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.147+0000 7f2afc400640 1 -- 172.21.15.143:0/4105823897 wait complete. 2024-01-15T18:15:37.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 Processor -- start 2024-01-15T18:15:37.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 -- start start 2024-01-15T18:15:37.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af406b5a0 0x7f2af414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 0x7f2af4159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af415d570 0x7f2af415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40b35a0 con 0x7f2af4071e00 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af415b940 con 0x7f2af406b5a0 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.151+0000 7f2afc400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af414a920 con 0x7f2af415d570 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 0x7f2af4159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 0x7f2af4159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35642/0 (socket says 172.21.15.143:35642) 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 -- 172.21.15.143:0/1076937399 learned_addr learned my addr 172.21.15.143:0/1076937399 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9a00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af406b5a0 0x7f2af414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2afa400640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af415d570 0x7f2af415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 -- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af415d570 msgr2=0x7f2af415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af415d570 0x7f2af415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 -- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af406b5a0 msgr2=0x7f2af414a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af406b5a0 0x7f2af414a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 -- 172.21.15.143:0/1076937399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc00b770 con 0x7f2af4071e00 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af9000640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 0x7f2af4159e90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2adc01c790 tx=0x7f2adc01b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0afd10 con 0x7f2af4071e00 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc0afe70 con 0x7f2af4071e00 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0b8660 con 0x7f2af4071e00 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2afc400640 1 -- 172.21.15.143:0/1076937399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af40b5bb0 con 0x7f2af4071e00 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.155+0000 7f2afc400640 1 -- 172.21.15.143:0/1076937399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af40b2ee0 con 0x7f2af4071e00 2024-01-15T18:15:37.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.159+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af40b36e0 con 0x7f2af4071e00 2024-01-15T18:15:37.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.159+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 28) v1 ==== 87282+0+0 (secure 0 0 0) 0x7f2adc0b6070 con 0x7f2af4071e00 2024-01-15T18:15:37.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.163+0000 7f2af1600640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2acc067620 0x7f2acc069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.163+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2adc11def0 con 0x7f2af4071e00 2024-01-15T18:15:37.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.163+0000 7f2af9a00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2acc067620 0x7f2acc069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.163+0000 7f2af9a00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2acc067620 0x7f2acc069ae0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4006fd0 tx=0x7f2ae4009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:37.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.167+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2adc0ecb80 con 0x7f2af4071e00 2024-01-15T18:15:37.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.507+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99315", "target": ["mon-mgr", ""]}) v1 -- 0x7f2af40c4c40 con 0x7f2acc067620 2024-01-15T18:15:37.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.543+0000 7f2af1600640 1 -- 172.21.15.143:0/1076937399 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2af40c4c40 con 0x7f2acc067620 2024-01-15T18:15:37.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2acc067620 msgr2=0x7f2acc069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2acc067620 0x7f2acc069ae0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4006fd0 tx=0x7f2ae4009060 comp rx=0 tx=0).stop 2024-01-15T18:15:37.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 msgr2=0x7f2af4159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 0x7f2af4159e90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f2adc01c790 tx=0x7f2adc01b900 comp rx=0 tx=0).stop 2024-01-15T18:15:37.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 shutdown_connections 2024-01-15T18:15:37.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2acc067620 0x7f2acc069ae0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2af415d570 0x7f2af415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2af4071e00 0x7f2af4159e90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 --2- 172.21.15.143:0/1076937399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2af406b5a0 0x7f2af414a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 >> 172.21.15.143:0/1076937399 conn(0x7f2af40cc800 msgr2=0x7f2af40ae820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:37.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 shutdown_connections 2024-01-15T18:15:37.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.551+0000 7f2aeaa00640 1 -- 172.21.15.143:0/1076937399 wait complete. 2024-01-15T18:15:37.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:15:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 Processor -- start 2024-01-15T18:15:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 -- start start 2024-01-15T18:15:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f714c06d3b0 0x7f714c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b33c0 0x7f714c16b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714c070a80 con 0x7f714c0b2aa0 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714c071d80 con 0x7f714c06d3b0 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7152e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c070ce0 con 0x7f714c0b33c0 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f7150e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b33c0 0x7f714c16b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f714b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f714c06d3b0 0x7f714c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f714be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f714c0b2aa0 0x7f714c070370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52294/0 (socket says 172.21.15.143:52294) 2024-01-15T18:15:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.839+0000 7f714be00640 1 -- 172.21.15.143:0/4140773376 learned_addr learned my addr 172.21.15.143:0/4140773376 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:37.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.843+0000 7f7150e00640 1 -- 172.21.15.143:0/4140773376 >> v2:172.21.15.204:3300/0 conn(0x7f714c06d3b0 msgr2=0x7f714c0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:37.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.843+0000 7f7150e00640 1 --2- 172.21.15.143:0/4140773376 >> v2:172.21.15.204:3300/0 conn(0x7f714c06d3b0 0x7f714c0b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.843+0000 7f7150e00640 1 -- 172.21.15.143:0/4140773376 >> v1:172.21.15.143:6789/0 conn(0x7f714c0b2aa0 legacy=0x7f714c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.843+0000 7f714aa00640 1 -- 172.21.15.143:0/4140773376 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112702945 0 0) 0x7f714c070a80 con 0x7f714c0b2aa0 2024-01-15T18:15:37.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.843+0000 7f7150e00640 1 -- 172.21.15.143:0/4140773376 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c070fc0 con 0x7f714c0b33c0 2024-01-15T18:15:37.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.843+0000 7f714b400640 1 --2- 172.21.15.143:0/4140773376 >> v2:172.21.15.204:3300/0 conn(0x7f714c06d3b0 0x7f714c0b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:37.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7150e00640 1 --2- 172.21.15.143:0/4140773376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b33c0 0x7f714c16b310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7144009d90 tx=0x7f714401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c599b410c5e61e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:37.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f714aa00640 1 -- 172.21.15.143:0/4140773376 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71440a5a00 con 0x7f714c0b33c0 2024-01-15T18:15:37.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f714aa00640 1 -- 172.21.15.143:0/4140773376 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71440a5b60 con 0x7f714c0b33c0 2024-01-15T18:15:37.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f714aa00640 1 -- 172.21.15.143:0/4140773376 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71440ad440 con 0x7f714c0b33c0 2024-01-15T18:15:37.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7152e00640 1 -- 172.21.15.143:0/4140773376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b33c0 msgr2=0x7f714c16b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7152e00640 1 --2- 172.21.15.143:0/4140773376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b33c0 0x7f714c16b310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7144009d90 tx=0x7f714401c920 comp rx=0 tx=0).stop 2024-01-15T18:15:37.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7152e00640 1 -- 172.21.15.143:0/4140773376 shutdown_connections 2024-01-15T18:15:37.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7152e00640 1 --2- 172.21.15.143:0/4140773376 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b33c0 0x7f714c16b310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7152e00640 1 --2- 172.21.15.143:0/4140773376 >> v2:172.21.15.204:3300/0 conn(0x7f714c06d3b0 0x7f714c0b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.847+0000 7f7152e00640 1 -- 172.21.15.143:0/4140773376 >> 172.21.15.143:0/4140773376 conn(0x7f714c0b0bf0 msgr2=0x7f714c0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:37.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.851+0000 7f7152e00640 1 -- 172.21.15.143:0/4140773376 shutdown_connections 2024-01-15T18:15:37.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.851+0000 7f7152e00640 1 -- 172.21.15.143:0/4140773376 wait complete. 2024-01-15T18:15:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 Processor -- start 2024-01-15T18:15:37.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 -- start start 2024-01-15T18:15:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f714c06d3b0 0x7f714c0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b2aa0 0x7f714c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f714c0b33c0 0x7f714c144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c145e70 con 0x7f714c06d3b0 2024-01-15T18:15:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c1fdbb0 con 0x7f714c0b33c0 2024-01-15T18:15:37.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7152e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c06d060 con 0x7f714c0b2aa0 2024-01-15T18:15:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f714c06d3b0 0x7f714c0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f714c06d3b0 0x7f714c0afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51828/0 (socket says 172.21.15.143:51828) 2024-01-15T18:15:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714be00640 1 -- 172.21.15.143:0/3478619022 learned_addr learned my addr 172.21.15.143:0/3478619022 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714b400640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b2aa0 0x7f714c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f7150e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f714c0b33c0 0x7f714c144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714b400640 1 -- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f714c0b33c0 msgr2=0x7f714c144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714b400640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f714c0b33c0 0x7f714c144940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.855+0000 7f714b400640 1 -- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f714c06d3b0 msgr2=0x7f714c0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f714b400640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f714c06d3b0 0x7f714c0afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f714b400640 1 -- 172.21.15.143:0/3478619022 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7144009a70 con 0x7f714c0b2aa0 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f714b400640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b2aa0 0x7f714c15a2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f713c00d600 tx=0x7f713c00dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c014620 con 0x7f714c0b2aa0 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c014780 con 0x7f714c0b2aa0 2024-01-15T18:15:37.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c014b30 con 0x7f714c0b2aa0 2024-01-15T18:15:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c1488d0 con 0x7f714c0b2aa0 2024-01-15T18:15:37.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714c1fe0a0 con 0x7f714c0b2aa0 2024-01-15T18:15:37.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.859+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c067680 con 0x7f714c0b2aa0 2024-01-15T18:15:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.863+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f713c014c90 con 0x7f714c0b2aa0 2024-01-15T18:15:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.863+0000 7f7143e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71240676c0 0x7f7124069b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:37.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.863+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f713c083640 con 0x7f714c0b2aa0 2024-01-15T18:15:37.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.867+0000 7f714be00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71240676c0 0x7f7124069b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:37.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.867+0000 7f714be00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71240676c0 0x7f7124069b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7134007b10 tx=0x7f7134007a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:37.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:37.871+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f713c0520d0 con 0x7f714c0b2aa0 2024-01-15T18:15:38.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.127+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f714c070ad0 con 0x7f71240676c0 2024-01-15T18:15:38.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.939+0000 7f7143e00640 1 -- 172.21.15.143:0/3478619022 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f714c070ad0 con 0x7f71240676c0 2024-01-15T18:15:38.937 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:38.937 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:15:38.937 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:38.937 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:15:38.937 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:38.937 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71240676c0 msgr2=0x7f7124069b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71240676c0 0x7f7124069b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7134007b10 tx=0x7f7134007a60 comp rx=0 tx=0).stop 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b2aa0 msgr2=0x7f714c15a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b2aa0 0x7f714c15a2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f713c00d600 tx=0x7f713c00dac0 comp rx=0 tx=0).stop 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 shutdown_connections 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71240676c0 0x7f7124069b80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f714c0b33c0 0x7f714c144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:38.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f714c0b2aa0 0x7f714c15a2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:38.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 --2- 172.21.15.143:0/3478619022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f714c06d3b0 0x7f714c0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:38.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 >> 172.21.15.143:0/3478619022 conn(0x7f714c0b0bf0 msgr2=0x7f714c06c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:38.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 shutdown_connections 2024-01-15T18:15:38.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:38.943+0000 7f7152e00640 1 -- 172.21.15.143:0/3478619022 wait complete. 2024-01-15T18:15:38.960 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:15:39.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f93c00640 1 Processor -- start 2024-01-15T18:15:39.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f93c00640 1 -- start start 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f93c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f8c141660 0x7f7f8c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f93c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c15f310 con 0x7f7f8c15ec80 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f93c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c15f5f0 con 0x7f7f8c071d00 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f93c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c15fbb0 con 0x7f7f8c141660 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f91200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7f8c141660 0x7f7f8c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f8be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7f8c15ec80 0x7f7f8c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52298/0 (socket says 172.21.15.143:52298) 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f8be00640 1 -- 172.21.15.143:0/3284979927 learned_addr learned my addr 172.21.15.143:0/3284979927 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:39.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f8b400640 1 -- 172.21.15.143:0/3284979927 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453882122 0 0) 0x7f7f8c15f5f0 con 0x7f7f8c071d00 2024-01-15T18:15:39.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f91200640 1 -- 172.21.15.143:0/3284979927 >> v1:172.21.15.204:6789/0 conn(0x7f7f8c071d00 legacy=0x7f7f8c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f91200640 1 -- 172.21.15.143:0/3284979927 >> v1:172.21.15.143:6789/0 conn(0x7f7f8c15ec80 legacy=0x7f7f8c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.107+0000 7f7f91200640 1 -- 172.21.15.143:0/3284979927 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c15f8d0 con 0x7f7f8c141660 2024-01-15T18:15:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f91200640 1 --2- 172.21.15.143:0/3284979927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c141660 0x7f7f8c0ce560 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0097c0 tx=0x7f7f7c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d570e1f67658989 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f8b400640 1 -- 172.21.15.143:0/3284979927 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0ab600 con 0x7f7f8c141660 2024-01-15T18:15:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f8b400640 1 -- 172.21.15.143:0/3284979927 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c0a7050 con 0x7f7f8c141660 2024-01-15T18:15:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f8b400640 1 -- 172.21.15.143:0/3284979927 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0ab980 con 0x7f7f8c141660 2024-01-15T18:15:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 -- 172.21.15.143:0/3284979927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c141660 msgr2=0x7f7f8c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 --2- 172.21.15.143:0/3284979927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c141660 0x7f7f8c0ce560 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0097c0 tx=0x7f7f7c01c310 comp rx=0 tx=0).stop 2024-01-15T18:15:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 -- 172.21.15.143:0/3284979927 shutdown_connections 2024-01-15T18:15:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 --2- 172.21.15.143:0/3284979927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c141660 0x7f7f8c0ce560 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 -- 172.21.15.143:0/3284979927 >> 172.21.15.143:0/3284979927 conn(0x7f7f8c06c3f0 msgr2=0x7f7f8c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:39.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 -- 172.21.15.143:0/3284979927 shutdown_connections 2024-01-15T18:15:39.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.111+0000 7f7f93c00640 1 -- 172.21.15.143:0/3284979927 wait complete. 2024-01-15T18:15:39.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 Processor -- start 2024-01-15T18:15:39.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 -- start start 2024-01-15T18:15:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 0x7f7f8c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c141660 0x7f7f8c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f8c15ec80 0x7f7f8c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15f130 con 0x7f7f8c071d00 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15b2e0 con 0x7f7f8c15ec80 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0b3610 con 0x7f7f8c141660 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 0x7f7f8c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 0x7f7f8c159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51854/0 (socket says 172.21.15.143:51854) 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 -- 172.21.15.143:0/1741548894 learned_addr learned my addr 172.21.15.143:0/1741548894 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f8be00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c141660 0x7f7f8c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f8c15ec80 0x7f7f8c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 -- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c141660 msgr2=0x7f7f8c15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c141660 0x7f7f8c15a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 -- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f8c15ec80 msgr2=0x7f7f8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f8c15ec80 0x7f7f8c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 -- 172.21.15.143:0/1741548894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c009420 con 0x7f7f8c071d00 2024-01-15T18:15:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f91200640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 0x7f7f8c159ef0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c018f20 tx=0x7f7f7c01ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0aa8b0 con 0x7f7f8c071d00 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7c0aaa30 con 0x7f7f8c071d00 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c0b38f0 con 0x7f7f8c071d00 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0aade0 con 0x7f7f8c071d00 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.115+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c0b3eb0 con 0x7f7f8c071d00 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.119+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c067500 con 0x7f7f8c071d00 2024-01-15T18:15:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.119+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f7f7c0c2460 con 0x7f7f8c071d00 2024-01-15T18:15:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.119+0000 7f7f88c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7f640676a0 0x7f7f64069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.119+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7f7c11dca0 con 0x7f7f8c071d00 2024-01-15T18:15:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.119+0000 7f7f8be00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7f640676a0 0x7f7f64069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.119+0000 7f7f8be00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7f640676a0 0x7f7f64069b60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009fd0 tx=0x7f7f7400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.123+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7f7c0ec730 con 0x7f7f8c071d00 2024-01-15T18:15:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.379+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f8c142640 con 0x7f7f640676a0 2024-01-15T18:15:39.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.407+0000 7f7f88c00640 1 -- 172.21.15.143:0/1741548894 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7f74016450 con 0x7f7f640676a0 2024-01-15T18:15:39.403 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7f640676a0 msgr2=0x7f7f64069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7f640676a0 0x7f7f64069b60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009fd0 tx=0x7f7f7400b040 comp rx=0 tx=0).stop 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 msgr2=0x7f7f8c159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 0x7f7f8c159ef0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c018f20 tx=0x7f7f7c01ab50 comp rx=0 tx=0).stop 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 shutdown_connections 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7f640676a0 0x7f7f64069b60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7f8c15ec80 0x7f7f8c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7f8c141660 0x7f7f8c15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 --2- 172.21.15.143:0/1741548894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7f8c071d00 0x7f7f8c159ef0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 >> 172.21.15.143:0/1741548894 conn(0x7f7f8c06c3f0 msgr2=0x7f7f8c0af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 shutdown_connections 2024-01-15T18:15:39.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.411+0000 7f7f93c00640 1 -- 172.21.15.143:0/1741548894 wait complete. 2024-01-15T18:15:39.426 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:15:39.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.567+0000 7f174d000640 1 Processor -- start 2024-01-15T18:15:39.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.567+0000 7f174d000640 1 -- start start 2024-01-15T18:15:39.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f174d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 0x7f174806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f174d000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17480cc810 con 0x7f1748070cc0 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f174d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174815f8c0 con 0x7f174806dff0 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f174d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174806e460 con 0x7f174815bf40 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1748070cc0 0x7f17480710a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52314/0 (socket says 172.21.15.143:52314) 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 -- 172.21.15.143:0/693597830 learned_addr learned my addr 172.21.15.143:0/693597830 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1746800640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 0x7f174806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10233349 0 0) 0x7f1738001120 con 0x7f1748070cc0 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1728003530 con 0x7f1748070cc0 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2440734928 0 0) 0x7f1728003530 con 0x7f1748070cc0 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 msgr2=0x7f174806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 0x7f174806a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 >> v1:172.21.15.204:6789/0 conn(0x7f174806dff0 legacy=0x7f174815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17480cbf70 con 0x7f1748070cc0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f1738001810 con 0x7f1748070cc0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 >> v1:172.21.15.143:6789/0 conn(0x7f1748070cc0 legacy=0x7f17480710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f1728003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17280047d0 0x7f1728004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 0x7f1728005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172800ba40 con 0x7f17280039b0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172800bbb0 con 0x7f17280047d0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172800bd70 con 0x7f174815bf40 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f1728003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1746800640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 0x7f1728005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745400640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17280047d0 0x7f1728004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 -- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 msgr2=0x7f1728005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f174815bf40 0x7f1728005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 -- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17280047d0 msgr2=0x7f1728004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17280047d0 0x7f1728004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745e00640 1 -- 172.21.15.143:0/693597830 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728003530 con 0x7f17280039b0 2024-01-15T18:15:39.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.571+0000 7f1745400640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17280047d0 0x7f1728004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:39.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f1745e00640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f1728003dd0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f173800c210 tx=0x7f173801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caee10ff7b870900 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738005ae0 con 0x7f17280039b0 2024-01-15T18:15:39.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738005c60 con 0x7f17280039b0 2024-01-15T18:15:39.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f1744a00640 1 -- 172.21.15.143:0/693597830 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738007070 con 0x7f17280039b0 2024-01-15T18:15:39.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 -- 172.21.15.143:0/693597830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 msgr2=0x7f1728003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 --2- 172.21.15.143:0/693597830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f1728003dd0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f173800c210 tx=0x7f173801cc00 comp rx=0 tx=0).stop 2024-01-15T18:15:39.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f1746800640 1 -- 172.21.15.143:0/693597830 reap_dead start 2024-01-15T18:15:39.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 -- 172.21.15.143:0/693597830 shutdown_connections 2024-01-15T18:15:39.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 -- 172.21.15.143:0/693597830 >> 172.21.15.143:0/693597830 conn(0x7f17480b1610 msgr2=0x7f17480b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:39.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 -- 172.21.15.143:0/693597830 shutdown_connections 2024-01-15T18:15:39.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 -- 172.21.15.143:0/693597830 wait complete. 2024-01-15T18:15:39.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 Processor -- start 2024-01-15T18:15:39.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.575+0000 7f174d000640 1 -- start start 2024-01-15T18:15:39.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f17481565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17481509c0 0x7f17480cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1748150e20 0x7f17480ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17481505a0 con 0x7f17280039b0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748153f90 con 0x7f1748150e20 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174814fc00 con 0x7f17481509c0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f17481565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f17481565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51862/0 (socket says 172.21.15.143:51862) 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 -- 172.21.15.143:0/547334886 learned_addr learned my addr 172.21.15.143:0/547334886 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1746800640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1748150e20 0x7f17480ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745400640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17481509c0 0x7f17480cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 -- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17481509c0 msgr2=0x7f17480cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17481509c0 0x7f17480cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 -- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1748150e20 msgr2=0x7f17480ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1748150e20 0x7f17480ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 -- 172.21.15.143:0/547334886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174814fee0 con 0x7f17280039b0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f1745e00640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f17481565b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f173801ca30 tx=0x7f1728005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380b4070 con 0x7f17280039b0 2024-01-15T18:15:39.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738005e50 con 0x7f17280039b0 2024-01-15T18:15:39.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17380015d0 con 0x7f17280039b0 2024-01-15T18:15:39.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17481501c0 con 0x7f17280039b0 2024-01-15T18:15:39.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748156cc0 con 0x7f17280039b0 2024-01-15T18:15:39.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.579+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1748074b00 con 0x7f17280039b0 2024-01-15T18:15:39.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.583+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f1738007090 con 0x7f17280039b0 2024-01-15T18:15:39.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.583+0000 7f173e000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1720067670 0x7f1720069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.583+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f173811c8f0 con 0x7f17280039b0 2024-01-15T18:15:39.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.583+0000 7f1745400640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1720067670 0x7f1720069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.583+0000 7f1745400640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1720067670 0x7f1720069b30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1748154fd0 tx=0x7f1730009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.587+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f17380eb2d0 con 0x7f17280039b0 2024-01-15T18:15:39.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.759+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f174815c350 con 0x7f1720067670 2024-01-15T18:15:39.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.775+0000 7f173e000640 1 -- 172.21.15.143:0/547334886 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1730001360 con 0x7f1720067670 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1720067670 msgr2=0x7f1720069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1720067670 0x7f1720069b30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1748154fd0 tx=0x7f1730009040 comp rx=0 tx=0).stop 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 msgr2=0x7f17481565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f17481565b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f173801ca30 tx=0x7f1728005800 comp rx=0 tx=0).stop 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 shutdown_connections 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1720067670 0x7f1720069b30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1748150e20 0x7f17480ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17481509c0 0x7f17480cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 --2- 172.21.15.143:0/547334886 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17280039b0 0x7f17481565b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 >> 172.21.15.143:0/547334886 conn(0x7f17480b1610 msgr2=0x7f17480afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:39.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 shutdown_connections 2024-01-15T18:15:39.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.779+0000 7f174d000640 1 -- 172.21.15.143:0/547334886 wait complete. 2024-01-15T18:15:39.802 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993113 2024-01-15T18:15:39.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb12000640 1 Processor -- start 2024-01-15T18:15:39.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb12000640 1 -- start start 2024-01-15T18:15:39.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb12000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb0c0cdf20 0x7ffb0c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb12000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c0c4d90 con 0x7ffb0c0c6230 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb12000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c15b640 con 0x7ffb0c15b260 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb12000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c0cf800 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb0c0cdf20 0x7ffb0c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffb0c0cdf20 0x7ffb0c0cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51868/0 (socket says 172.21.15.143:51868) 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0aa00640 1 -- 172.21.15.143:0/4083327486 learned_addr learned my addr 172.21.15.143:0/4083327486 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0b400640 1 --1- 172.21.15.143:0/4083327486 >> v1:172.21.15.204:6789/0 conn(0x7ffb0c15b260 0x7ffb0c0cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:57842/0 (socket says 172.21.15.143:57842) 2024-01-15T18:15:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0aa00640 1 -- 172.21.15.143:0/4083327486 >> v1:172.21.15.204:6789/0 conn(0x7ffb0c15b260 legacy=0x7ffb0c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0aa00640 1 -- 172.21.15.143:0/4083327486 >> v1:172.21.15.143:6789/0 conn(0x7ffb0c0c6230 legacy=0x7ffb0c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.955+0000 7ffb0aa00640 1 -- 172.21.15.143:0/4083327486 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0cfae0 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb0aa00640 1 --2- 172.21.15.143:0/4083327486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0cbae0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969a6de90ee46660 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb0a000640 1 -- 172.21.15.143:0/4083327486 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40aa670 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb0a000640 1 -- 172.21.15.143:0/4083327486 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0ccf40 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb0a000640 1 -- 172.21.15.143:0/4083327486 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40aa9f0 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 -- 172.21.15.143:0/4083327486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 msgr2=0x7ffb0c0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 --2- 172.21.15.143:0/4083327486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0cbae0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf401c510 comp rx=0 tx=0).stop 2024-01-15T18:15:39.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 -- 172.21.15.143:0/4083327486 shutdown_connections 2024-01-15T18:15:39.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 --2- 172.21.15.143:0/4083327486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0cbae0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 -- 172.21.15.143:0/4083327486 >> 172.21.15.143:0/4083327486 conn(0x7ffb0c0d0ed0 msgr2=0x7ffb0c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:39.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 -- 172.21.15.143:0/4083327486 shutdown_connections 2024-01-15T18:15:39.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.959+0000 7ffb12000640 1 -- 172.21.15.143:0/4083327486 wait complete. 2024-01-15T18:15:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 Processor -- start 2024-01-15T18:15:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 -- start start 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c0c6230 0x7ffb0c14f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb0c15b260 0x7ffb0c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0c99e0 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0c8dd0 con 0x7ffb0c0c6230 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0c90b0 con 0x7ffb0c15b260 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51882/0 (socket says 172.21.15.143:51882) 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 -- 172.21.15.143:0/3574365724 learned_addr learned my addr 172.21.15.143:0/3574365724 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0b400640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c0c6230 0x7ffb0c14f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0be00640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb0c15b260 0x7ffb0c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 -- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb0c15b260 msgr2=0x7ffb0c0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb0c15b260 0x7ffb0c0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 -- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c0c6230 msgr2=0x7ffb0c14f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c0c6230 0x7ffb0c14f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 -- 172.21.15.143:0/3574365724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40094d0 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0be00640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb0c15b260 0x7ffb0c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0b400640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c0c6230 0x7ffb0c14f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:39.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb0aa00640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0b5690 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4019120 tx=0x7ffaf401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40b3800 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf40b3980 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40b3d30 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0c9390 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.963+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c14f7f0 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.967+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0c9b50 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.967+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7ffaf40b2380 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.967+0000 7ffb03400640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffae4067670 0x7ffae4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.967+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffaf411d850 con 0x7ffb0c0cdf20 2024-01-15T18:15:39.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.967+0000 7ffb0b400640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffae4067670 0x7ffae4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.967+0000 7ffb0b400640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffae4067670 0x7ffae4069b30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ffafc000f70 tx=0x7ffafc007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:39.971+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffaf40ec340 con 0x7ffb0c0cdf20 2024-01-15T18:15:40.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.151+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993113", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb0c067500 con 0x7ffae4067670 2024-01-15T18:15:40.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb03400640 1 -- 172.21.15.143:0/3574365724 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffb0c067500 con 0x7ffae4067670 2024-01-15T18:15:40.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffae4067670 msgr2=0x7ffae4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffae4067670 0x7ffae4069b30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ffafc000f70 tx=0x7ffafc007d50 comp rx=0 tx=0).stop 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 msgr2=0x7ffb0c0b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0b5690 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4019120 tx=0x7ffaf401ad50 comp rx=0 tx=0).stop 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 shutdown_connections 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffae4067670 0x7ffae4069b30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb0c15b260 0x7ffb0c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb0c0cdf20 0x7ffb0c0b5690 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 --2- 172.21.15.143:0/3574365724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb0c0c6230 0x7ffb0c14f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 >> 172.21.15.143:0/3574365724 conn(0x7ffb0c0d0ed0 msgr2=0x7ffb0c0cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:40.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 shutdown_connections 2024-01-15T18:15:40.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.171+0000 7ffb12000640 1 -- 172.21.15.143:0/3574365724 wait complete. 2024-01-15T18:15:40.193 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993111 2024-01-15T18:15:40.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.339+0000 7f4fd1c00640 1 Processor -- start 2024-01-15T18:15:40.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fd1c00640 1 -- start start 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fd1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc0cdf20 0x7f4fcc0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fd1c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4fcc0c6230 0x7f4fcc0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fd1c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc0c4d90 con 0x7f4fcc15b260 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fd1c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc15b640 con 0x7f4fcc0c6230 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fd1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0cf800 con 0x7f4fcc0cdf20 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4fcc0c6230 0x7f4fcc0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4fcc0c6230 0x7f4fcc0cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51894/0 (socket says 172.21.15.143:51894) 2024-01-15T18:15:40.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 -- 172.21.15.143:0/3117901 learned_addr learned my addr 172.21.15.143:0/3117901 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:40.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fca000640 1 --2- 172.21.15.143:0/3117901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc0cdf20 0x7f4fcc0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 -- 172.21.15.143:0/3117901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc0cdf20 msgr2=0x7f4fcc0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:40.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 --2- 172.21.15.143:0/3117901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc0cdf20 0x7f4fcc0cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 -- 172.21.15.143:0/3117901 >> v1:172.21.15.204:6789/0 conn(0x7f4fcc15b260 legacy=0x7f4fcc0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.343+0000 7f4fcb400640 1 -- 172.21.15.143:0/3117901 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0cfae0 con 0x7f4fcc0c6230 2024-01-15T18:15:40.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fcb400640 1 --2- 172.21.15.143:0/3117901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc0cc020 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009e50 tx=0x7f4fc401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a70c07157141f45 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:40.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fc9600640 1 -- 172.21.15.143:0/3117901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40a58a0 con 0x7f4fcc0c6230 2024-01-15T18:15:40.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fc9600640 1 -- 172.21.15.143:0/3117901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc40a5a00 con 0x7f4fcc0c6230 2024-01-15T18:15:40.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fc9600640 1 -- 172.21.15.143:0/3117901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40a5d90 con 0x7f4fcc0c6230 2024-01-15T18:15:40.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 -- 172.21.15.143:0/3117901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 msgr2=0x7f4fcc0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/3117901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc0cc020 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009e50 tx=0x7f4fc401c9e0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 -- 172.21.15.143:0/3117901 shutdown_connections 2024-01-15T18:15:40.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/3117901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc0cc020 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/3117901 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc0cdf20 0x7f4fcc0cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 -- 172.21.15.143:0/3117901 >> 172.21.15.143:0/3117901 conn(0x7f4fcc0d0ed0 msgr2=0x7f4fcc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:40.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 -- 172.21.15.143:0/3117901 shutdown_connections 2024-01-15T18:15:40.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.347+0000 7f4fd1c00640 1 -- 172.21.15.143:0/3117901 wait complete. 2024-01-15T18:15:40.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 Processor -- start 2024-01-15T18:15:40.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- start start 2024-01-15T18:15:40.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fcc0cdf20 0x7f4fcc153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc15b260 0x7f4fcc09ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1f5fb0 con 0x7f4fcc0c6230 2024-01-15T18:15:40.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0c8040 con 0x7f4fcc0cdf20 2024-01-15T18:15:40.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0c8320 con 0x7f4fcc15b260 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51910/0 (socket says 172.21.15.143:51910) 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 -- 172.21.15.143:0/2522482114 learned_addr learned my addr 172.21.15.143:0/2522482114 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fca000640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fcc0cdf20 0x7f4fcc153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcb400640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc15b260 0x7f4fcc09ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 -- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc15b260 msgr2=0x7f4fcc09ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc15b260 0x7f4fcc09ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 -- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fcc0cdf20 msgr2=0x7f4fcc153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fcc0cdf20 0x7f4fcc153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 -- 172.21.15.143:0/2522482114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4009b30 con 0x7f4fcc0c6230 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fcaa00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc157f30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c3c0 tx=0x7f4fbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0052d0 con 0x7f4fcc0c6230 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc005430 con 0x7f4fcc0c6230 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc0c8600 con 0x7f4fcc0c6230 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc003840 con 0x7f4fcc0c6230 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc154360 con 0x7f4fcc0c6230 2024-01-15T18:15:40.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.351+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc0c8b20 con 0x7f4fcc0c6230 2024-01-15T18:15:40.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.355+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f4fbc0039a0 con 0x7f4fcc0c6230 2024-01-15T18:15:40.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.355+0000 7f4fc2a00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fa4067670 0x7f4fa4069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.355+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4fbc0825d0 con 0x7f4fcc0c6230 2024-01-15T18:15:40.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.355+0000 7f4fca000640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fa4067670 0x7f4fa4069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.355+0000 7f4fca000640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fa4067670 0x7f4fa4069b30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc06d5f0 tx=0x7f4fb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:40.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.359+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4fcc0c8b20 con 0x7f4fcc0c6230 2024-01-15T18:15:40.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.535+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993111", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fcc0c81b0 con 0x7f4fa4067670 2024-01-15T18:15:40.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.551+0000 7f4fc2a00640 1 -- 172.21.15.143:0/2522482114 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4fcc0c81b0 con 0x7f4fa4067670 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fa4067670 msgr2=0x7f4fa4069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fa4067670 0x7f4fa4069b30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc06d5f0 tx=0x7f4fb4008040 comp rx=0 tx=0).stop 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 msgr2=0x7f4fcc157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc157f30 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00c3c0 tx=0x7f4fbc00c880 comp rx=0 tx=0).stop 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 shutdown_connections 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4fa4067670 0x7f4fa4069b30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fcc15b260 0x7f4fcc09ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fcc0cdf20 0x7f4fcc153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 --2- 172.21.15.143:0/2522482114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fcc0c6230 0x7f4fcc157f30 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 >> 172.21.15.143:0/2522482114 conn(0x7f4fcc0d0ed0 msgr2=0x7f4fcc0d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 shutdown_connections 2024-01-15T18:15:40.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.555+0000 7f4fd1c00640 1 -- 172.21.15.143:0/2522482114 wait complete. 2024-01-15T18:15:40.575 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993117 2024-01-15T18:15:40.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 Processor -- start 2024-01-15T18:15:40.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 -- start start 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c15b260 0x7ffb1c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ffb1c0d19f0 0x7ffb1c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb1c0c4d90 con 0x7ffb1c0c5220 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb1c15b640 con 0x7ffb1c0d19f0 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb22800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0cf800 con 0x7ffb1c15b260 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb1be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ffb1c0c5220 0x7ffb1c0c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52326/0 (socket says 172.21.15.143:52326) 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb1be00640 1 -- 172.21.15.143:0/1574116687 learned_addr learned my addr 172.21.15.143:0/1574116687 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb1b400640 1 --2- 172.21.15.143:0/1574116687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c15b260 0x7ffb1c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb1aa00640 1 --2- 172.21.15.143:0/1574116687 >> v2:172.21.15.204:3300/0 conn(0x7ffb1c0d19f0 0x7ffb1c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb1a000640 1 -- 172.21.15.143:0/1574116687 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008803330 0 0) 0x7ffb1c0c4d90 con 0x7ffb1c0c5220 2024-01-15T18:15:40.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.719+0000 7ffb1a000640 1 -- 172.21.15.143:0/1574116687 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc003530 con 0x7ffb1c0c5220 2024-01-15T18:15:40.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1b400640 1 -- 172.21.15.143:0/1574116687 >> v2:172.21.15.204:3300/0 conn(0x7ffb1c0d19f0 msgr2=0x7ffb1c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:40.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1a000640 1 -- 172.21.15.143:0/1574116687 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800315945 0 0) 0x7ffafc003530 con 0x7ffb1c0c5220 2024-01-15T18:15:40.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1b400640 1 --2- 172.21.15.143:0/1574116687 >> v2:172.21.15.204:3300/0 conn(0x7ffb1c0d19f0 0x7ffb1c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1b400640 1 -- 172.21.15.143:0/1574116687 >> v1:172.21.15.143:6789/0 conn(0x7ffb1c0c5220 legacy=0x7ffb1c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1b400640 1 -- 172.21.15.143:0/1574116687 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0cfae0 con 0x7ffb1c15b260 2024-01-15T18:15:40.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1b400640 1 --2- 172.21.15.143:0/1574116687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c15b260 0x7ffb1c0cd810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097c0 tx=0x7ffb0c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c288ec4e4c97913 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:40.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1a000640 1 -- 172.21.15.143:0/1574116687 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14001a00 con 0x7ffb1c15b260 2024-01-15T18:15:40.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1a000640 1 -- 172.21.15.143:0/1574116687 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0a7050 con 0x7ffb1c15b260 2024-01-15T18:15:40.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.723+0000 7ffb1a000640 1 -- 172.21.15.143:0/1574116687 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0ab620 con 0x7ffb1c15b260 2024-01-15T18:15:40.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- 172.21.15.143:0/1574116687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c15b260 msgr2=0x7ffb1c0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 --2- 172.21.15.143:0/1574116687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c15b260 0x7ffb1c0cd810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0097c0 tx=0x7ffb0c01c370 comp rx=0 tx=0).stop 2024-01-15T18:15:40.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- 172.21.15.143:0/1574116687 shutdown_connections 2024-01-15T18:15:40.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 --2- 172.21.15.143:0/1574116687 >> v2:172.21.15.204:3300/0 conn(0x7ffb1c0d19f0 0x7ffb1c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 --2- 172.21.15.143:0/1574116687 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c15b260 0x7ffb1c0cd810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- 172.21.15.143:0/1574116687 >> 172.21.15.143:0/1574116687 conn(0x7ffb1c0d0ed0 msgr2=0x7ffb1c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:40.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- 172.21.15.143:0/1574116687 shutdown_connections 2024-01-15T18:15:40.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- 172.21.15.143:0/1574116687 wait complete. 2024-01-15T18:15:40.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 Processor -- start 2024-01-15T18:15:40.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- start start 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 0x7ffb1c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c0d19f0 0x7ffb1c159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb1c15b260 0x7ffb1c141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b2890 con 0x7ffb1c0c5220 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b23a0 con 0x7ffb1c15b260 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb22800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1c0b25b0 con 0x7ffb1c0d19f0 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 0x7ffb1c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb1b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 0x7ffb1c157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51924/0 (socket says 172.21.15.143:51924) 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb1b400640 1 -- 172.21.15.143:0/2547768825 learned_addr learned my addr 172.21.15.143:0/2547768825 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb1be00640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb1c15b260 0x7ffb1c141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.727+0000 7ffb1aa00640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c0d19f0 0x7ffb1c159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1b400640 1 -- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c0d19f0 msgr2=0x7ffb1c159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1b400640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c0d19f0 0x7ffb1c159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1b400640 1 -- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb1c15b260 msgr2=0x7ffb1c141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1b400640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb1c15b260 0x7ffb1c141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1b400640 1 -- 172.21.15.143:0/2547768825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c009420 con 0x7ffb1c0c5220 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1b400640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 0x7ffb1c157cb0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c01aa20 tx=0x7ffb0c019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0b4400 con 0x7ffb1c0c5220 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c0b4580 con 0x7ffb1c0c5220 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0b4930 con 0x7ffb1c0c5220 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1c155460 con 0x7ffb1c0c5220 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1c159c60 con 0x7ffb1c0c5220 2024-01-15T18:15:40.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1c1552c0 con 0x7ffb1c0c5220 2024-01-15T18:15:40.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7ffb0c0b2040 con 0x7ffb1c0c5220 2024-01-15T18:15:40.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb13400640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffaf40676a0 0x7ffaf4069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:40.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffb0c11c820 con 0x7ffb1c0c5220 2024-01-15T18:15:40.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.731+0000 7ffb1aa00640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffaf40676a0 0x7ffaf4069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:40.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.735+0000 7ffb1aa00640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffaf40676a0 0x7ffaf4069b60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c1f9800 tx=0x7ffb04009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:40.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.735+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ffb1c1552c0 con 0x7ffb1c0c5220 2024-01-15T18:15:40.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.915+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993117", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb1c1546f0 con 0x7ffaf40676a0 2024-01-15T18:15:40.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb13400640 1 -- 172.21.15.143:0/2547768825 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ffb1c1546f0 con 0x7ffaf40676a0 2024-01-15T18:15:40.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffaf40676a0 msgr2=0x7ffaf4069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffaf40676a0 0x7ffaf4069b60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c1f9800 tx=0x7ffb04009190 comp rx=0 tx=0).stop 2024-01-15T18:15:40.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 msgr2=0x7ffb1c157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:40.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 0x7ffb1c157cb0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c01aa20 tx=0x7ffb0c019110 comp rx=0 tx=0).stop 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 shutdown_connections 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ffaf40676a0 0x7ffaf4069b60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb1c15b260 0x7ffb1c141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ffb1c0d19f0 0x7ffb1c159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 --2- 172.21.15.143:0/2547768825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffb1c0c5220 0x7ffb1c157cb0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.927+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 >> 172.21.15.143:0/2547768825 conn(0x7ffb1c0d0ed0 msgr2=0x7ffb1c153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.931+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 shutdown_connections 2024-01-15T18:15:40.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:40.931+0000 7ffb22800640 1 -- 172.21.15.143:0/2547768825 wait complete. 2024-01-15T18:15:40.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99319 2024-01-15T18:15:41.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 Processor -- start 2024-01-15T18:15:41.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 -- start start 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0e415ec80 0x7fb0e4141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e415f310 con 0x7fb0e4142470 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e415fbb0 con 0x7fb0e415ec80 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0eb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e415f100 con 0x7fb0e4141660 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0e415ec80 0x7fb0e4141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0e415ec80 0x7fb0e4141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51936/0 (socket says 172.21.15.143:51936) 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 -- 172.21.15.143:0/2198988097 learned_addr learned my addr 172.21.15.143:0/2198988097 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e8a00640 1 --2- 172.21.15.143:0/2198988097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 -- 172.21.15.143:0/2198988097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 msgr2=0x7fb0e40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 --2- 172.21.15.143:0/2198988097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 -- 172.21.15.143:0/2198988097 >> v1:172.21.15.143:6789/0 conn(0x7fb0e4142470 legacy=0x7fb0e4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.095+0000 7fb0e3e00640 1 -- 172.21.15.143:0/2198988097 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e415f5f0 con 0x7fb0e415ec80 2024-01-15T18:15:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.099+0000 7fb0e3e00640 1 --2- 172.21.15.143:0/2198988097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e415ec80 0x7fb0e4141f30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009870 tx=0x7fb0cc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16f1c13857fd183 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.099+0000 7fb0e3400640 1 -- 172.21.15.143:0/2198988097 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0aa670 con 0x7fb0e415ec80 2024-01-15T18:15:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.099+0000 7fb0e3400640 1 -- 172.21.15.143:0/2198988097 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc0aa7d0 con 0x7fb0e415ec80 2024-01-15T18:15:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.099+0000 7fb0e3400640 1 -- 172.21.15.143:0/2198988097 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0aab60 con 0x7fb0e415ec80 2024-01-15T18:15:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.099+0000 7fb0eb400640 1 -- 172.21.15.143:0/2198988097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e415ec80 msgr2=0x7fb0e4141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.099+0000 7fb0eb400640 1 --2- 172.21.15.143:0/2198988097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e415ec80 0x7fb0e4141f30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009870 tx=0x7fb0cc01c510 comp rx=0 tx=0).stop 2024-01-15T18:15:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- 172.21.15.143:0/2198988097 shutdown_connections 2024-01-15T18:15:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 --2- 172.21.15.143:0/2198988097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e415ec80 0x7fb0e4141f30 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 --2- 172.21.15.143:0/2198988097 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- 172.21.15.143:0/2198988097 >> 172.21.15.143:0/2198988097 conn(0x7fb0e406c3f0 msgr2=0x7fb0e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- 172.21.15.143:0/2198988097 shutdown_connections 2024-01-15T18:15:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- 172.21.15.143:0/2198988097 wait complete. 2024-01-15T18:15:41.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 Processor -- start 2024-01-15T18:15:41.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- start start 2024-01-15T18:15:41.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40a8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 0x7fb0e413cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e415ec80 0x7fb0e4159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e413a7f0 con 0x7fb0e4142470 2024-01-15T18:15:41.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e413d2f0 con 0x7fb0e415ec80 2024-01-15T18:15:41.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0eb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e415dfc0 con 0x7fb0e4141660 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 0x7fb0e413cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 0x7fb0e413cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51944/0 (socket says 172.21.15.143:51944) 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 -- 172.21.15.143:0/435183585 learned_addr learned my addr 172.21.15.143:0/435183585 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e8a00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 -- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 msgr2=0x7fb0e40a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 -- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e415ec80 msgr2=0x7fb0e4159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e9400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e415ec80 0x7fb0e4159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e415ec80 0x7fb0e4159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e3e00640 1 -- 172.21.15.143:0/435183585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc0094d0 con 0x7fb0e4142470 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e8a00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.103+0000 7fb0e9400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e415ec80 0x7fb0e4159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e3e00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 0x7fb0e413cdb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc01a140 tx=0x7fb0cc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0b3800 con 0x7fb0e4142470 2024-01-15T18:15:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc0b3980 con 0x7fb0e4142470 2024-01-15T18:15:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e40cd150 con 0x7fb0e4142470 2024-01-15T18:15:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc0b3d30 con 0x7fb0e4142470 2024-01-15T18:15:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e413ad20 con 0x7fb0e4142470 2024-01-15T18:15:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e4067500 con 0x7fb0e4142470 2024-01-15T18:15:41.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7fb0cc0b2380 con 0x7fb0e4142470 2024-01-15T18:15:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e0c00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0bc067670 0x7fb0bc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb0cc11d330 con 0x7fb0e4142470 2024-01-15T18:15:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.107+0000 7fb0e8a00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0bc067670 0x7fb0bc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.111+0000 7fb0e8a00640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0bc067670 0x7fb0bc069b30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009840 tx=0x7fb0d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.111+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0cc0ebc80 con 0x7fb0e4142470 2024-01-15T18:15:41.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.287+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99319", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0e40ccfd0 con 0x7fb0bc067670 2024-01-15T18:15:41.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.299+0000 7fb0e0c00640 1 -- 172.21.15.143:0/435183585 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb0d4009e20 con 0x7fb0bc067670 2024-01-15T18:15:41.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0bc067670 msgr2=0x7fb0bc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0bc067670 0x7fb0bc069b30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009840 tx=0x7fb0d4008040 comp rx=0 tx=0).stop 2024-01-15T18:15:41.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 msgr2=0x7fb0e413cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 0x7fb0e413cdb0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc01a140 tx=0x7fb0cc01ad50 comp rx=0 tx=0).stop 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 shutdown_connections 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb0bc067670 0x7fb0bc069b30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0e415ec80 0x7fb0e4159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0e4142470 0x7fb0e413cdb0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 --2- 172.21.15.143:0/435183585 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0e4141660 0x7fb0e40a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 >> 172.21.15.143:0/435183585 conn(0x7fb0e406c3f0 msgr2=0x7fb0e4140620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 shutdown_connections 2024-01-15T18:15:41.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.303+0000 7fb0eb400640 1 -- 172.21.15.143:0/435183585 wait complete. 2024-01-15T18:15:41.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993120 2024-01-15T18:15:41.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 Processor -- start 2024-01-15T18:15:41.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 -- start start 2024-01-15T18:15:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fddbc141660 0x7fddbc0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 0x7fddbc072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddbc15f310 con 0x7fddbc15ec80 2024-01-15T18:15:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddbc15fbb0 con 0x7fddbc141660 2024-01-15T18:15:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddc1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc15f130 con 0x7fddbc071d00 2024-01-15T18:15:41.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fddbc15ec80 0x7fddbc141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52348/0 (socket says 172.21.15.143:52348) 2024-01-15T18:15:41.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9e00640 1 -- 172.21.15.143:0/3871647820 learned_addr learned my addr 172.21.15.143:0/3871647820 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddbb200640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 0x7fddbc072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddba800640 1 --2- 172.21.15.143:0/3871647820 >> v2:172.21.15.204:3300/0 conn(0x7fddbc141660 0x7fddbc0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069769627 0 0) 0x7fddbc15f310 con 0x7fddbc15ec80 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd9c003530 con 0x7fddbc15ec80 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1608159353 0 0) 0x7fdd9c003530 con 0x7fddbc15ec80 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 msgr2=0x7fddbc072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 0x7fddbc072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 >> v2:172.21.15.204:3300/0 conn(0x7fddbc141660 msgr2=0x7fddbc0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 --2- 172.21.15.143:0/3871647820 >> v2:172.21.15.204:3300/0 conn(0x7fddbc141660 0x7fddbc0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc15f5f0 con 0x7fddbc15ec80 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fdda4001680 con 0x7fddbc15ec80 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 >> v1:172.21.15.143:6789/0 conn(0x7fddbc15ec80 legacy=0x7fddbc141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 0x7fdd9c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd9c003f00 0x7fdd9c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.471+0000 7fddb9400640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fdd9c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc15fbb0 con 0x7fdd9c004d10 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc15f130 con 0x7fdd9c003f00 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c00b990 con 0x7fddbc071d00 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fdd9c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddba800640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 0x7fdd9c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9e00640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd9c003f00 0x7fdd9c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 -- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 msgr2=0x7fdd9c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc071d00 0x7fdd9c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 -- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd9c003f00 msgr2=0x7fdd9c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:41.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd9c003f00 0x7fdd9c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 -- 172.21.15.143:0/3871647820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c003530 con 0x7fdd9c004d10 2024-01-15T18:15:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddbb200640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fdd9c005190 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009fc0 tx=0x7fddb401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69f5e98177c8c93 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb40a5c70 con 0x7fdd9c004d10 2024-01-15T18:15:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb400f040 con 0x7fdd9c004d10 2024-01-15T18:15:41.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.475+0000 7fddb9400640 1 -- 172.21.15.143:0/3871647820 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb40ad440 con 0x7fdd9c004d10 2024-01-15T18:15:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- 172.21.15.143:0/3871647820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 msgr2=0x7fdd9c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 --2- 172.21.15.143:0/3871647820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fdd9c005190 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fddb4009fc0 tx=0x7fddb401cc00 comp rx=0 tx=0).stop 2024-01-15T18:15:41.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddbb200640 1 -- 172.21.15.143:0/3871647820 reap_dead start 2024-01-15T18:15:41.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- 172.21.15.143:0/3871647820 shutdown_connections 2024-01-15T18:15:41.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- 172.21.15.143:0/3871647820 >> 172.21.15.143:0/3871647820 conn(0x7fddbc06c3f0 msgr2=0x7fddbc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:41.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- 172.21.15.143:0/3871647820 shutdown_connections 2024-01-15T18:15:41.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- 172.21.15.143:0/3871647820 wait complete. 2024-01-15T18:15:41.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 Processor -- start 2024-01-15T18:15:41.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- start start 2024-01-15T18:15:41.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fddbc143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddbc0b3660 0x7fddbc0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc137580 0x7fddbc137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc1f3d90 con 0x7fdd9c004d10 2024-01-15T18:15:41.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc1f2ef0 con 0x7fddbc0b3660 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddc1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc0b4000 con 0x7fddbc137580 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddba800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fddbc143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddba800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fddbc143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51950/0 (socket says 172.21.15.143:51950) 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddba800640 1 -- 172.21.15.143:0/68954647 learned_addr learned my addr 172.21.15.143:0/68954647 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddb9e00640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddbc0b3660 0x7fddbc0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.479+0000 7fddbb200640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc137580 0x7fddbc137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddba800640 1 -- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc137580 msgr2=0x7fddbc137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddba800640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc137580 0x7fddbc137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddba800640 1 -- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddbc0b3660 msgr2=0x7fddbc0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddba800640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddbc0b3660 0x7fddbc0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddba800640 1 -- 172.21.15.143:0/68954647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddbc1f3110 con 0x7fdd9c004d10 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddba800640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fddbc143cc0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fddac011ad0 tx=0x7fddac011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac00fbb0 con 0x7fdd9c004d10 2024-01-15T18:15:41.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac00fd10 con 0x7fdd9c004d10 2024-01-15T18:15:41.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddbc1f33f0 con 0x7fdd9c004d10 2024-01-15T18:15:41.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac014680 con 0x7fdd9c004d10 2024-01-15T18:15:41.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddbc1f39b0 con 0x7fdd9c004d10 2024-01-15T18:15:41.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddbc067500 con 0x7fdd9c004d10 2024-01-15T18:15:41.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.483+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7fddac01d400 con 0x7fdd9c004d10 2024-01-15T18:15:41.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.487+0000 7fddb2a00640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd94067670 0x7fdd94069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.487+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fddac083940 con 0x7fdd9c004d10 2024-01-15T18:15:41.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.487+0000 7fddb9e00640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd94067670 0x7fdd94069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.487+0000 7fddb9e00640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd94067670 0x7fdd94069b30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0b3580 tx=0x7fdda4007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.487+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fddac0523d0 con 0x7fdd9c004d10 2024-01-15T18:15:41.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.667+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993120", "target": ["mon-mgr", ""]}) v1 -- 0x7fddbc141a40 con 0x7fdd94067670 2024-01-15T18:15:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.679+0000 7fddb2a00640 1 -- 172.21.15.143:0/68954647 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdda4007e60 con 0x7fdd94067670 2024-01-15T18:15:41.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd94067670 msgr2=0x7fdd94069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd94067670 0x7fdd94069b30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0b3580 tx=0x7fdda4007900 comp rx=0 tx=0).stop 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 msgr2=0x7fddbc143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fddbc143cc0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fddac011ad0 tx=0x7fddac011f90 comp rx=0 tx=0).stop 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 shutdown_connections 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdd94067670 0x7fdd94069b30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fddbc137580 0x7fddbc137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fddbc0b3660 0x7fddbc0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 --2- 172.21.15.143:0/68954647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd9c004d10 0x7fddbc143cc0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 >> 172.21.15.143:0/68954647 conn(0x7fddbc06c3f0 msgr2=0x7fddbc0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 shutdown_connections 2024-01-15T18:15:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.683+0000 7fddc1400640 1 -- 172.21.15.143:0/68954647 wait complete. 2024-01-15T18:15:41.703 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993118 2024-01-15T18:15:41.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 Processor -- start 2024-01-15T18:15:41.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 -- start start 2024-01-15T18:15:41.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47d4071140 0x7f47d406a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f47d406a9d0 0x7f47d40ca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d415c4f0 con 0x7f47d4158bc0 2024-01-15T18:15:41.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40715d0 con 0x7f47d4071140 2024-01-15T18:15:41.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d9600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d406ae60 con 0x7f47d406a9d0 2024-01-15T18:15:41.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f47d4071140 0x7f47d406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f47d4158bc0 0x7f47d406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52352/0 (socket says 172.21.15.143:52352) 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2a00640 1 -- 172.21.15.143:0/2118776357 learned_addr learned my addr 172.21.15.143:0/2118776357 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d3400640 1 --2- 172.21.15.143:0/2118776357 >> v2:172.21.15.204:3300/0 conn(0x7f47d406a9d0 0x7f47d40ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2000640 1 -- 172.21.15.143:0/2118776357 >> v2:172.21.15.204:3300/0 conn(0x7f47d406a9d0 msgr2=0x7f47d40ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2000640 1 --2- 172.21.15.143:0/2118776357 >> v2:172.21.15.204:3300/0 conn(0x7f47d406a9d0 0x7f47d40ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2000640 1 -- 172.21.15.143:0/2118776357 >> v1:172.21.15.143:6789/0 conn(0x7f47d4158bc0 legacy=0x7f47d406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d2000640 1 -- 172.21.15.143:0/2118776357 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4158fa0 con 0x7f47d4071140 2024-01-15T18:15:41.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.851+0000 7f47d3400640 1 --2- 172.21.15.143:0/2118776357 >> v2:172.21.15.204:3300/0 conn(0x7f47d406a9d0 0x7f47d40ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:41.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d2000640 1 --2- 172.21.15.143:0/2118776357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d406a490 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f47d4072570 tx=0x7f47bc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a73df5990df55 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d1600640 1 -- 172.21.15.143:0/2118776357 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0ac630 con 0x7f47d4071140 2024-01-15T18:15:41.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d1600640 1 -- 172.21.15.143:0/2118776357 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc0ac790 con 0x7f47d4071140 2024-01-15T18:15:41.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d1600640 1 -- 172.21.15.143:0/2118776357 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0acb20 con 0x7f47d4071140 2024-01-15T18:15:41.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 -- 172.21.15.143:0/2118776357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 msgr2=0x7f47d406a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 --2- 172.21.15.143:0/2118776357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d406a490 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f47d4072570 tx=0x7f47bc01c3d0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 -- 172.21.15.143:0/2118776357 shutdown_connections 2024-01-15T18:15:41.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 --2- 172.21.15.143:0/2118776357 >> v2:172.21.15.204:3300/0 conn(0x7f47d406a9d0 0x7f47d40ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 --2- 172.21.15.143:0/2118776357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d406a490 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 -- 172.21.15.143:0/2118776357 >> 172.21.15.143:0/2118776357 conn(0x7f47d40cfd20 msgr2=0x7f47d40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:41.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 -- 172.21.15.143:0/2118776357 shutdown_connections 2024-01-15T18:15:41.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.855+0000 7f47d9600640 1 -- 172.21.15.143:0/2118776357 wait complete. 2024-01-15T18:15:41.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 Processor -- start 2024-01-15T18:15:41.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 -- start start 2024-01-15T18:15:41.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47d406a9d0 0x7f47d4151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d40b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4158bc0 0x7f47d40b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4151e90 con 0x7f47d4071140 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4163340 con 0x7f47d4158bc0 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40ceff0 con 0x7f47d406a9d0 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d40b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d40b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51956/0 (socket says 172.21.15.143:51956) 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 -- 172.21.15.143:0/1186850641 learned_addr learned my addr 172.21.15.143:0/1186850641 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2a00640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47d406a9d0 0x7f47d4151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d3400640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4158bc0 0x7f47d40b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 -- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47d406a9d0 msgr2=0x7f47d4151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47d406a9d0 0x7f47d4151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 -- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4158bc0 msgr2=0x7f47d40b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4158bc0 0x7f47d40b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 -- 172.21.15.143:0/1186850641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc009420 con 0x7f47d4071140 2024-01-15T18:15:41.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d2000640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d40b28e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0af040 tx=0x7f47bc019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b5820 con 0x7f47d4071140 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc0b59a0 con 0x7f47d4071140 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b5d50 con 0x7f47d4071140 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d40cf2d0 con 0x7f47d4071140 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.859+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d40ce300 con 0x7f47d4071140 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.863+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d40cf4c0 con 0x7f47d4071140 2024-01-15T18:15:41.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.863+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f47bc0b3040 con 0x7f47d4071140 2024-01-15T18:15:41.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.863+0000 7f47caa00640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47ac067670 0x7f47ac069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:41.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.863+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f47bc11d930 con 0x7f47d4071140 2024-01-15T18:15:41.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.863+0000 7f47d2a00640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47ac067670 0x7f47ac069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:41.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.863+0000 7f47d2a00640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47ac067670 0x7f47ac069b30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005890 tx=0x7f47c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:41.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:41.867+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f47bc0ec3c0 con 0x7f47d4071140 2024-01-15T18:15:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.043+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993118", "target": ["mon-mgr", ""]}) v1 -- 0x7f47d41631c0 con 0x7f47ac067670 2024-01-15T18:15:42.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.059+0000 7f47caa00640 1 -- 172.21.15.143:0/1186850641 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f47c4001470 con 0x7f47ac067670 2024-01-15T18:15:42.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.059+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47ac067670 msgr2=0x7f47ac069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.059+0000 7f47d9600640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47ac067670 0x7f47ac069b30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005890 tx=0x7f47c4009040 comp rx=0 tx=0).stop 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.059+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 msgr2=0x7f47d40b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.059+0000 7f47d9600640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d40b28e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0af040 tx=0x7f47bc019170 comp rx=0 tx=0).stop 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 shutdown_connections 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f47ac067670 0x7f47ac069b30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47d4158bc0 0x7f47d40b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47d4071140 0x7f47d40b28e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 --2- 172.21.15.143:0/1186850641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f47d406a9d0 0x7f47d4151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 >> 172.21.15.143:0/1186850641 conn(0x7f47d40cfd20 msgr2=0x7f47d415c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 shutdown_connections 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.063+0000 7f47d9600640 1 -- 172.21.15.143:0/1186850641 wait complete. 2024-01-15T18:15:42.085 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993112 2024-01-15T18:15:42.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.295+0000 7fe743800640 1 Processor -- start 2024-01-15T18:15:42.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.295+0000 7fe743800640 1 -- start start 2024-01-15T18:15:42.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe743800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe73c0cf910 0x7fe73c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe743800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0cbff0 con 0x7fe73c0b0310 2024-01-15T18:15:42.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe743800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0d0c50 con 0x7fe73c06bfb0 2024-01-15T18:15:42.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe743800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0cb6c0 con 0x7fe73c0cf910 2024-01-15T18:15:42.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe73be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe73c0b0310 0x7fe73c0cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52362/0 (socket says 172.21.15.143:52362) 2024-01-15T18:15:42.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe73be00640 1 -- 172.21.15.143:0/2471702927 learned_addr learned my addr 172.21.15.143:0/2471702927 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:42.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe741800640 1 --2- 172.21.15.143:0/2471702927 >> v2:172.21.15.204:3300/0 conn(0x7fe73c0cf910 0x7fe73c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe741800640 1 -- 172.21.15.143:0/2471702927 >> v1:172.21.15.204:6789/0 conn(0x7fe73c06bfb0 legacy=0x7fe73c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:42.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe741800640 1 -- 172.21.15.143:0/2471702927 >> v1:172.21.15.143:6789/0 conn(0x7fe73c0b0310 legacy=0x7fe73c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.299+0000 7fe741800640 1 -- 172.21.15.143:0/2471702927 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0cb9a0 con 0x7fe73c0cf910 2024-01-15T18:15:42.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.307+0000 7fe741800640 1 --2- 172.21.15.143:0/2471702927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0cf910 0x7fe73c0cfdc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe734009d90 tx=0x7fe73401c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d358efb39b9507 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.307+0000 7fe73b400640 1 -- 172.21.15.143:0/2471702927 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340a66a0 con 0x7fe73c0cf910 2024-01-15T18:15:42.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.307+0000 7fe73b400640 1 -- 172.21.15.143:0/2471702927 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7340a6800 con 0x7fe73c0cf910 2024-01-15T18:15:42.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.307+0000 7fe73b400640 1 -- 172.21.15.143:0/2471702927 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340a6b90 con 0x7fe73c0cf910 2024-01-15T18:15:42.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.307+0000 7fe743800640 1 -- 172.21.15.143:0/2471702927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0cf910 msgr2=0x7fe73c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.307+0000 7fe743800640 1 --2- 172.21.15.143:0/2471702927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0cf910 0x7fe73c0cfdc0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe734009d90 tx=0x7fe73401c6e0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 -- 172.21.15.143:0/2471702927 shutdown_connections 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 --2- 172.21.15.143:0/2471702927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0cf910 0x7fe73c0cfdc0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 -- 172.21.15.143:0/2471702927 >> 172.21.15.143:0/2471702927 conn(0x7fe73c0d0ed0 msgr2=0x7fe73c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 -- 172.21.15.143:0/2471702927 shutdown_connections 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 -- 172.21.15.143:0/2471702927 wait complete. 2024-01-15T18:15:42.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 Processor -- start 2024-01-15T18:15:42.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.311+0000 7fe743800640 1 -- start start 2024-01-15T18:15:42.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe743800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe73c06bfb0 0x7fe73c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe743800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0b0310 0x7fe73c0add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe743800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe73c0cf910 0x7fe73c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe743800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c14b3f0 con 0x7fe73c0cf910 2024-01-15T18:15:42.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe743800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0af270 con 0x7fe73c0b0310 2024-01-15T18:15:42.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe743800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0ae660 con 0x7fe73c06bfb0 2024-01-15T18:15:42.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe740e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe73c06bfb0 0x7fe73c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe740e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe73c06bfb0 0x7fe73c14aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:50144/0 (socket says 172.21.15.143:50144) 2024-01-15T18:15:42.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe740e00640 1 -- 172.21.15.143:0/4230947534 learned_addr learned my addr 172.21.15.143:0/4230947534 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:42.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe73c0cf910 0x7fe73c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe73be00640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0b0310 0x7fe73c0add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 -- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe73c06bfb0 msgr2=0x7fe73c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe73c06bfb0 0x7fe73c14aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 -- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0b0310 msgr2=0x7fe73c0add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0b0310 0x7fe73c0add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 -- 172.21.15.143:0/4230947534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe734009a70 con 0x7fe73c0cf910 2024-01-15T18:15:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.315+0000 7fe741800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe73c0cf910 0x7fe73c0b18a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe7340ab000 tx=0x7fe734019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340b7070 con 0x7fe73c0cf910 2024-01-15T18:15:42.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c0ae940 con 0x7fe73c0cf910 2024-01-15T18:15:42.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c0aef00 con 0x7fe73c0cf910 2024-01-15T18:15:42.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7340a7690 con 0x7fe73c0cf910 2024-01-15T18:15:42.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7340a7a00 con 0x7fe73c0cf910 2024-01-15T18:15:42.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c0c5ae0 con 0x7fe73c0cf910 2024-01-15T18:15:42.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.319+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7fe7340a7b60 con 0x7fe73c0cf910 2024-01-15T18:15:42.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.323+0000 7fe738c00640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe714067670 0x7fe714069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.323+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe73411db70 con 0x7fe73c0cf910 2024-01-15T18:15:42.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.323+0000 7fe740e00640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe714067670 0x7fe714069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.323+0000 7fe740e00640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe714067670 0x7fe714069b30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0c41c0 tx=0x7fe72c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.327+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe7340ec500 con 0x7fe73c0cf910 2024-01-15T18:15:42.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.646+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993112", "target": ["mon-mgr", ""]}) v1 -- 0x7fe73c16b6b0 con 0x7fe714067670 2024-01-15T18:15:42.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.674+0000 7fe738c00640 1 -- 172.21.15.143:0/4230947534 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe72c009e00 con 0x7fe714067670 2024-01-15T18:15:42.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe714067670 msgr2=0x7fe714069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe714067670 0x7fe714069b30 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0c41c0 tx=0x7fe72c008040 comp rx=0 tx=0).stop 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe73c0cf910 msgr2=0x7fe73c0b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe73c0cf910 0x7fe73c0b18a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe7340ab000 tx=0x7fe734019480 comp rx=0 tx=0).stop 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 shutdown_connections 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe714067670 0x7fe714069b30 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe73c0cf910 0x7fe73c0b18a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe73c0b0310 0x7fe73c0add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 --2- 172.21.15.143:0/4230947534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe73c06bfb0 0x7fe73c14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 >> 172.21.15.143:0/4230947534 conn(0x7fe73c0d0ed0 msgr2=0x7fe73c06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:42.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 shutdown_connections 2024-01-15T18:15:42.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.678+0000 7fe743800640 1 -- 172.21.15.143:0/4230947534 wait complete. 2024-01-15T18:15:42.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993114 2024-01-15T18:15:42.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.946+0000 7f784e600640 1 Processor -- start 2024-01-15T18:15:42.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.946+0000 7f784e600640 1 -- start start 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f784e600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f784814b1e0 0x7f78480cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f784e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f7848162510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f784e600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78480cfc20 con 0x7f78480cf7c0 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f784e600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7848162080 con 0x7f784814b1e0 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f784e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7848161d70 con 0x7f7848070fd0 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7846a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f78480cf7c0 0x7f78480708c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:57868/0 (socket says 172.21.15.143:57868) 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7846a00640 1 -- 172.21.15.143:0/1429569055 learned_addr learned my addr 172.21.15.143:0/1429569055 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7847400640 1 --2- 172.21.15.143:0/1429569055 >> v2:172.21.15.143:3300/0 conn(0x7f784814b1e0 0x7f78480cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7847e00640 1 --2- 172.21.15.143:0/1429569055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f7848162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7846000640 1 -- 172.21.15.143:0/1429569055 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902806564 0 0) 0x7f78480cfc20 con 0x7f78480cf7c0 2024-01-15T18:15:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7847e00640 1 -- 172.21.15.143:0/1429569055 >> v2:172.21.15.143:3300/0 conn(0x7f784814b1e0 msgr2=0x7f78480cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7847e00640 1 --2- 172.21.15.143:0/1429569055 >> v2:172.21.15.143:3300/0 conn(0x7f784814b1e0 0x7f78480cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7847e00640 1 -- 172.21.15.143:0/1429569055 >> v1:172.21.15.204:6789/0 conn(0x7f78480cf7c0 legacy=0x7f78480708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.950+0000 7f7847e00640 1 -- 172.21.15.143:0/1429569055 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784814b5c0 con 0x7f7848070fd0 2024-01-15T18:15:42.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.954+0000 7f7847e00640 1 --2- 172.21.15.143:0/1429569055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f7848162510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7840009d90 tx=0x7f784001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7966696682530d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f7846000640 1 -- 172.21.15.143:0/1429569055 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400a59e0 con 0x7f7848070fd0 2024-01-15T18:15:42.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f7846000640 1 -- 172.21.15.143:0/1429569055 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400a5b40 con 0x7f7848070fd0 2024-01-15T18:15:42.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f7846000640 1 -- 172.21.15.143:0/1429569055 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400ad440 con 0x7f7848070fd0 2024-01-15T18:15:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 -- 172.21.15.143:0/1429569055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 msgr2=0x7f7848162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 --2- 172.21.15.143:0/1429569055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f7848162510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7840009d90 tx=0x7f784001c920 comp rx=0 tx=0).stop 2024-01-15T18:15:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 -- 172.21.15.143:0/1429569055 shutdown_connections 2024-01-15T18:15:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 --2- 172.21.15.143:0/1429569055 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f7848162510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 --2- 172.21.15.143:0/1429569055 >> v2:172.21.15.143:3300/0 conn(0x7f784814b1e0 0x7f78480cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 -- 172.21.15.143:0/1429569055 >> 172.21.15.143:0/1429569055 conn(0x7f784806a890 msgr2=0x7f784806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:42.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 -- 172.21.15.143:0/1429569055 shutdown_connections 2024-01-15T18:15:42.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.958+0000 7f784e600640 1 -- 172.21.15.143:0/1429569055 wait complete. 2024-01-15T18:15:42.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.962+0000 7f784e600640 1 Processor -- start 2024-01-15T18:15:42.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.962+0000 7f784e600640 1 -- start start 2024-01-15T18:15:42.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f784e600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f784816bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f784e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78480cf7c0 0x7f7848146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f784e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 0x7f7848142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f784e600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78480a93e0 con 0x7f78480cf7c0 2024-01-15T18:15:42.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f784e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78480a96c0 con 0x7f784814b1e0 2024-01-15T18:15:42.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f784e600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78480a99a0 con 0x7f7848070fd0 2024-01-15T18:15:42.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f7847e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 0x7f7848142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f7847e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 0x7f7848142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:47266/0 (socket says 172.21.15.143:47266) 2024-01-15T18:15:42.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.966+0000 7f7847e00640 1 -- 172.21.15.143:0/2269333866 learned_addr learned my addr 172.21.15.143:0/2269333866 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:42.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.970+0000 7f7847400640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f784816bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.970+0000 7f7847e00640 1 -- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 msgr2=0x7f784816bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.970+0000 7f7847e00640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f784816bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.970+0000 7f7847e00640 1 -- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78480cf7c0 msgr2=0x7f7848146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.970+0000 7f7847e00640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78480cf7c0 0x7f7848146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.970+0000 7f7847e00640 1 -- 172.21.15.143:0/2269333866 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009a70 con 0x7f784814b1e0 2024-01-15T18:15:42.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.974+0000 7f7847e00640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 0x7f7848142de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f784001afd0 tx=0x7f78400196c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.974+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400aece0 con 0x7f784814b1e0 2024-01-15T18:15:42.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.974+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400aee40 con 0x7f784814b1e0 2024-01-15T18:15:42.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.974+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400b7610 con 0x7f784814b1e0 2024-01-15T18:15:42.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.974+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78480a9c80 con 0x7f784814b1e0 2024-01-15T18:15:42.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.974+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7848146d40 con 0x7f784814b1e0 2024-01-15T18:15:42.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.978+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784814ef60 con 0x7f784814b1e0 2024-01-15T18:15:42.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.978+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f78400b4050 con 0x7f784814b1e0 2024-01-15T18:15:42.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.978+0000 7f783f400640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7820067670 0x7f7820069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.978+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f784011e510 con 0x7f784814b1e0 2024-01-15T18:15:42.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.978+0000 7f7847400640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7820067670 0x7f7820069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.982+0000 7f7847400640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7820067670 0x7f7820069b30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7838009af0 tx=0x7f7838008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:42.990+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f784814ef60 con 0x7f784814b1e0 2024-01-15T18:15:43.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.318+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993114", "target": ["mon-mgr", ""]}) v1 -- 0x7f7848146f60 con 0x7f7820067670 2024-01-15T18:15:43.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.346+0000 7f783f400640 1 -- 172.21.15.143:0/2269333866 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7848146f60 con 0x7f7820067670 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7820067670 msgr2=0x7f7820069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7820067670 0x7f7820069b30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7838009af0 tx=0x7f7838008040 comp rx=0 tx=0).stop 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 msgr2=0x7f7848142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 0x7f7848142de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f784001afd0 tx=0x7f78400196c0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 shutdown_connections 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7820067670 0x7f7820069b30 secure :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7838009af0 tx=0x7f7838008040 comp rx=0 tx=0).stop 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f784814b1e0 0x7f7848142de0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78480cf7c0 0x7f7848146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 --2- 172.21.15.143:0/2269333866 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7848070fd0 0x7f784816bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.350+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 >> 172.21.15.143:0/2269333866 conn(0x7f784806a890 msgr2=0x7f7848144b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:43.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.354+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 shutdown_connections 2024-01-15T18:15:43.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.354+0000 7f784e600640 1 -- 172.21.15.143:0/2269333866 wait complete. 2024-01-15T18:15:43.397 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99318 2024-01-15T18:15:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.566+0000 7f7d0c200640 1 Processor -- start 2024-01-15T18:15:43.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.566+0000 7f7d0c200640 1 -- start start 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0c200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7d040ce560 0x7f7d0415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0c200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d04071d00 0x7f7d04072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0c200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d04072650 con 0x7f7d04141660 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0c200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0415fbb0 con 0x7f7d04071d00 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0c200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0415f310 con 0x7f7d040ce560 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d04071d00 0x7f7d04072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7d04071d00 0x7f7d04072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51996/0 (socket says 172.21.15.143:51996) 2024-01-15T18:15:43.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 -- 172.21.15.143:0/507275708 learned_addr learned my addr 172.21.15.143:0/507275708 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d08e00640 1 --2- 172.21.15.143:0/507275708 >> v2:172.21.15.204:3300/0 conn(0x7f7d040ce560 0x7f7d0415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 -- 172.21.15.143:0/507275708 >> v2:172.21.15.204:3300/0 conn(0x7f7d040ce560 msgr2=0x7f7d0415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 --2- 172.21.15.143:0/507275708 >> v2:172.21.15.204:3300/0 conn(0x7f7d040ce560 0x7f7d0415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 -- 172.21.15.143:0/507275708 >> v1:172.21.15.204:6789/0 conn(0x7f7d04141660 legacy=0x7f7d04141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d0a200640 1 -- 172.21.15.143:0/507275708 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0415f5f0 con 0x7f7d04071d00 2024-01-15T18:15:43.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.570+0000 7f7d08e00640 1 --2- 172.21.15.143:0/507275708 >> v2:172.21.15.204:3300/0 conn(0x7f7d040ce560 0x7f7d0415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0a200640 1 --2- 172.21.15.143:0/507275708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04071d00 0x7f7d04072110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406f940 tx=0x7f7cfc01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21609eaa17490bc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d03e00640 1 -- 172.21.15.143:0/507275708 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0a5740 con 0x7f7d04071d00 2024-01-15T18:15:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d03e00640 1 -- 172.21.15.143:0/507275708 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0a58a0 con 0x7f7d04071d00 2024-01-15T18:15:43.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d03e00640 1 -- 172.21.15.143:0/507275708 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0a5c30 con 0x7f7d04071d00 2024-01-15T18:15:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 -- 172.21.15.143:0/507275708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04071d00 msgr2=0x7f7d04072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 --2- 172.21.15.143:0/507275708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04071d00 0x7f7d04072110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406f940 tx=0x7f7cfc01c990 comp rx=0 tx=0).stop 2024-01-15T18:15:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 -- 172.21.15.143:0/507275708 shutdown_connections 2024-01-15T18:15:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 --2- 172.21.15.143:0/507275708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04071d00 0x7f7d04072110 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 --2- 172.21.15.143:0/507275708 >> v2:172.21.15.204:3300/0 conn(0x7f7d040ce560 0x7f7d0415ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 -- 172.21.15.143:0/507275708 >> 172.21.15.143:0/507275708 conn(0x7f7d0406c3f0 msgr2=0x7f7d0406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 -- 172.21.15.143:0/507275708 shutdown_connections 2024-01-15T18:15:43.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 -- 172.21.15.143:0/507275708 wait complete. 2024-01-15T18:15:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 Processor -- start 2024-01-15T18:15:43.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.574+0000 7f7d0c200640 1 -- start start 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d04071d00 0x7f7d0413f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d040ce560 0x7f7d0413c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 0x7f7d04159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0415d8b0 con 0x7f7d04141660 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0415db90 con 0x7f7d040ce560 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0415de70 con 0x7f7d04071d00 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 0x7f7d04159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 0x7f7d04159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52010/0 (socket says 172.21.15.143:52010) 2024-01-15T18:15:43.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 -- 172.21.15.143:0/3260390488 learned_addr learned my addr 172.21.15.143:0/3260390488 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d08e00640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d040ce560 0x7f7d0413c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d09800640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d04071d00 0x7f7d0413f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 -- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d04071d00 msgr2=0x7f7d0413f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d04071d00 0x7f7d0413f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 -- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d040ce560 msgr2=0x7f7d0413c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d040ce560 0x7f7d0413c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 -- 172.21.15.143:0/3260390488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009b30 con 0x7f7d04141660 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0a200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 0x7f7d04159c20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc01a5c0 tx=0x7f7cfc019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0b8670 con 0x7f7d04141660 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0b87d0 con 0x7f7d04141660 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d040cd150 con 0x7f7d04141660 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0b8b60 con 0x7f7d04141660 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0415e430 con 0x7f7d04141660 2024-01-15T18:15:43.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.578+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0415e620 con 0x7f7d04141660 2024-01-15T18:15:43.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.582+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f7cfc0b8cc0 con 0x7f7d04141660 2024-01-15T18:15:43.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.582+0000 7f7d01600640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7cdc067670 0x7f7cdc069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.582+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7cfc11e640 con 0x7f7d04141660 2024-01-15T18:15:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.582+0000 7f7d09800640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7cdc067670 0x7f7cdc069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.582+0000 7f7d09800640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7cdc067670 0x7f7cdc069b30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7d04139fa0 tx=0x7f7cf4007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:43.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.586+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7cfc0ed1f0 con 0x7f7d04141660 2024-01-15T18:15:43.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.758+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99318", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d040ccfd0 con 0x7f7cdc067670 2024-01-15T18:15:43.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.770+0000 7f7d01600640 1 -- 172.21.15.143:0/3260390488 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7cf4003bf0 con 0x7f7cdc067670 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7cdc067670 msgr2=0x7f7cdc069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7cdc067670 0x7f7cdc069b30 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7d04139fa0 tx=0x7f7cf4007e40 comp rx=0 tx=0).stop 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 msgr2=0x7f7d04159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 0x7f7d04159c20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc01a5c0 tx=0x7f7cfc019730 comp rx=0 tx=0).stop 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 shutdown_connections 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7cdc067670 0x7f7cdc069b30 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d04141660 0x7f7d04159c20 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7d040ce560 0x7f7d0413c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 --2- 172.21.15.143:0/3260390488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7d04071d00 0x7f7d0413f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 >> 172.21.15.143:0/3260390488 conn(0x7f7d0406c3f0 msgr2=0x7f7d040ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:43.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 shutdown_connections 2024-01-15T18:15:43.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.774+0000 7f7d0c200640 1 -- 172.21.15.143:0/3260390488 wait complete. 2024-01-15T18:15:43.794 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993110 2024-01-15T18:15:43.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 Processor -- start 2024-01-15T18:15:43.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 -- start start 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc36406aba0 0x7fc3640aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36415d510 0x7fc36406e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc364160eb0 con 0x7fc36415d130 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc364161190 con 0x7fc36406aba0 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.946+0000 7fc369800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc364161470 con 0x7fc36415d510 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc36415d130 0x7fc36406a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52376/0 (socket says 172.21.15.143:52376) 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362a00640 1 -- 172.21.15.143:0/1181045382 learned_addr learned my addr 172.21.15.143:0/1181045382 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362000640 1 --2- 172.21.15.143:0/1181045382 >> v2:172.21.15.204:3300/0 conn(0x7fc36406aba0 0x7fc3640aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc363400640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36415d510 0x7fc36406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc361600640 1 -- 172.21.15.143:0/1181045382 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601981277 0 0) 0x7fc364160eb0 con 0x7fc36415d130 2024-01-15T18:15:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc361600640 1 -- 172.21.15.143:0/1181045382 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc344003530 con 0x7fc36415d130 2024-01-15T18:15:43.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362000640 1 -- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36415d510 msgr2=0x7fc36406e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:43.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362000640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36415d510 0x7fc36406e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362000640 1 -- 172.21.15.143:0/1181045382 >> v1:172.21.15.143:6789/0 conn(0x7fc36415d130 legacy=0x7fc36406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362000640 1 -- 172.21.15.143:0/1181045382 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364161750 con 0x7fc36406aba0 2024-01-15T18:15:43.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc363400640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36415d510 0x7fc36406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc362000640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36406aba0 0x7fc3640aeed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0097c0 tx=0x7fc34c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49c1c94f7c3381c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc361600640 1 -- 172.21.15.143:0/1181045382 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0a6040 con 0x7fc36406aba0 2024-01-15T18:15:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc361600640 1 -- 172.21.15.143:0/1181045382 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c0aa600 con 0x7fc36406aba0 2024-01-15T18:15:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.950+0000 7fc361600640 1 -- 172.21.15.143:0/1181045382 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0aa990 con 0x7fc36406aba0 2024-01-15T18:15:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36406aba0 msgr2=0x7fc3640aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36406aba0 0x7fc3640aeed0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0097c0 tx=0x7fc34c01c480 comp rx=0 tx=0).stop 2024-01-15T18:15:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- 172.21.15.143:0/1181045382 shutdown_connections 2024-01-15T18:15:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36415d510 0x7fc36406e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 --2- 172.21.15.143:0/1181045382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36406aba0 0x7fc3640aeed0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- 172.21.15.143:0/1181045382 >> 172.21.15.143:0/1181045382 conn(0x7fc3640b1b90 msgr2=0x7fc3640b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- 172.21.15.143:0/1181045382 shutdown_connections 2024-01-15T18:15:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- 172.21.15.143:0/1181045382 wait complete. 2024-01-15T18:15:43.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 Processor -- start 2024-01-15T18:15:43.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- start start 2024-01-15T18:15:43.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36406aba0 0x7fc36414af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 0x7fc364153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36415d510 0x7fc36414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640adf50 con 0x7fc36415d130 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640ae130 con 0x7fc36415d510 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc369800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3640ae410 con 0x7fc36406aba0 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc362000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 0x7fc364153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc362000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 0x7fc364153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52024/0 (socket says 172.21.15.143:52024) 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.954+0000 7fc362000640 1 -- 172.21.15.143:0/3420121540 learned_addr learned my addr 172.21.15.143:0/3420121540 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362a00640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36406aba0 0x7fc36414af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc363400640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36415d510 0x7fc36414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362000640 1 -- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36406aba0 msgr2=0x7fc36414af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362000640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36406aba0 0x7fc36414af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362000640 1 -- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36415d510 msgr2=0x7fc36414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362000640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36415d510 0x7fc36414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362000640 1 -- 172.21.15.143:0/3420121540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c009a70 con 0x7fc36415d130 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc362000640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 0x7fc364153990 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc34c01a0b0 tx=0x7fc34c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0b3800 con 0x7fc36415d130 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c0b3960 con 0x7fc36415d130 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c009420 con 0x7fc36415d130 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c0b3cd0 con 0x7fc36415d130 2024-01-15T18:15:43.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3640ae9d0 con 0x7fc36415d130 2024-01-15T18:15:43.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc364067500 con 0x7fc36415d130 2024-01-15T18:15:43.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.958+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7fc34c0bc3e0 con 0x7fc36415d130 2024-01-15T18:15:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.962+0000 7fc35aa00640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc33c067670 0x7fc33c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.962+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc34c11d4d0 con 0x7fc36415d130 2024-01-15T18:15:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.962+0000 7fc362a00640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc33c067670 0x7fc33c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:43.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.962+0000 7fc362a00640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc33c067670 0x7fc33c069b30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc3540011f0 tx=0x7fc354007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:43.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:43.966+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc34c0ec000 con 0x7fc36415d130 2024-01-15T18:15:44.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.150+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993110", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3640b0670 con 0x7fc33c067670 2024-01-15T18:15:44.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.174+0000 7fc35aa00640 1 -- 172.21.15.143:0/3420121540 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc3640b0670 con 0x7fc33c067670 2024-01-15T18:15:44.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.178+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc33c067670 msgr2=0x7fc33c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.178+0000 7fc369800640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc33c067670 0x7fc33c069b30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc3540011f0 tx=0x7fc354007d40 comp rx=0 tx=0).stop 2024-01-15T18:15:44.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.178+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 msgr2=0x7fc364153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.178+0000 7fc369800640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 0x7fc364153990 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fc34c01a0b0 tx=0x7fc34c01acc0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 shutdown_connections 2024-01-15T18:15:44.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc33c067670 0x7fc33c069b30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc36415d510 0x7fc36414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc36415d130 0x7fc364153990 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 --2- 172.21.15.143:0/3420121540 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc36406aba0 0x7fc36414af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 >> 172.21.15.143:0/3420121540 conn(0x7fc3640b1b90 msgr2=0x7fc364071260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:44.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 shutdown_connections 2024-01-15T18:15:44.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.182+0000 7fc369800640 1 -- 172.21.15.143:0/3420121540 wait complete. 2024-01-15T18:15:44.220 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone993119 2024-01-15T18:15:44.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 Processor -- start 2024-01-15T18:15:44.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 -- start start 2024-01-15T18:15:44.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c141660 0x7f869c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f869c15ec80 0x7f869c141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f869c15f310 con 0x7f869c142470 2024-01-15T18:15:44.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c15fbb0 con 0x7f869c15ec80 2024-01-15T18:15:44.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f86a1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c15f100 con 0x7f869c141660 2024-01-15T18:15:44.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c141660 0x7f869c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f869c15ec80 0x7f869c141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869b400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f869c142470 0x7f869c071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:57880/0 (socket says 172.21.15.143:57880) 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869b400640 1 -- 172.21.15.143:0/4024465766 learned_addr learned my addr 172.21.15.143:0/4024465766 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869aa00640 1 -- 172.21.15.143:0/4024465766 >> v2:172.21.15.204:3300/0 conn(0x7f869c15ec80 msgr2=0x7f869c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f8699600640 1 -- 172.21.15.143:0/4024465766 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402215390 0 0) 0x7f869c15f310 con 0x7f869c142470 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869aa00640 1 --2- 172.21.15.143:0/4024465766 >> v2:172.21.15.204:3300/0 conn(0x7f869c15ec80 0x7f869c141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869aa00640 1 -- 172.21.15.143:0/4024465766 >> v1:172.21.15.204:6789/0 conn(0x7f869c142470 legacy=0x7f869c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869aa00640 1 -- 172.21.15.143:0/4024465766 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c15f5f0 con 0x7f869c141660 2024-01-15T18:15:44.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.394+0000 7f869a000640 1 --2- 172.21.15.143:0/4024465766 >> v2:172.21.15.204:3300/0 conn(0x7f869c15ec80 0x7f869c141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:44.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f869aa00640 1 --2- 172.21.15.143:0/4024465766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c141660 0x7f869c0ce560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f868c0097c0 tx=0x7f868c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f768d150276c82b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f8699600640 1 -- 172.21.15.143:0/4024465766 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0ab650 con 0x7f869c141660 2024-01-15T18:15:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f8699600640 1 -- 172.21.15.143:0/4024465766 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c0ab7b0 con 0x7f869c141660 2024-01-15T18:15:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f8699600640 1 -- 172.21.15.143:0/4024465766 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0abb40 con 0x7f869c141660 2024-01-15T18:15:44.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f86a1600640 1 -- 172.21.15.143:0/4024465766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c141660 msgr2=0x7f869c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f86a1600640 1 --2- 172.21.15.143:0/4024465766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c141660 0x7f869c0ce560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f868c0097c0 tx=0x7f868c01c450 comp rx=0 tx=0).stop 2024-01-15T18:15:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f86a1600640 1 -- 172.21.15.143:0/4024465766 shutdown_connections 2024-01-15T18:15:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f86a1600640 1 --2- 172.21.15.143:0/4024465766 >> v2:172.21.15.204:3300/0 conn(0x7f869c15ec80 0x7f869c141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f86a1600640 1 --2- 172.21.15.143:0/4024465766 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c141660 0x7f869c0ce560 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.398+0000 7f86a1600640 1 -- 172.21.15.143:0/4024465766 >> 172.21.15.143:0/4024465766 conn(0x7f869c06c3f0 msgr2=0x7f869c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 -- 172.21.15.143:0/4024465766 shutdown_connections 2024-01-15T18:15:44.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 -- 172.21.15.143:0/4024465766 wait complete. 2024-01-15T18:15:44.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 Processor -- start 2024-01-15T18:15:44.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 -- start start 2024-01-15T18:15:44.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f869c141660 0x7f869c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c142470 0x7f869c070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 0x7f869c1392f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c15adc0 con 0x7f869c15ec80 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c15e2f0 con 0x7f869c141660 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f86a1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c15d700 con 0x7f869c142470 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 0x7f869c1392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 0x7f869c1392f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52030/0 (socket says 172.21.15.143:52030) 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 -- 172.21.15.143:0/1581554328 learned_addr learned my addr 172.21.15.143:0/1581554328 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869a000640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c142470 0x7f869c070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869aa00640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f869c141660 0x7f869c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 -- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c142470 msgr2=0x7f869c070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c142470 0x7f869c070370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 -- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f869c141660 msgr2=0x7f869c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f869c141660 0x7f869c0711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 -- 172.21.15.143:0/1581554328 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684009420 con 0x7f869c15ec80 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.402+0000 7f869b400640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 0x7f869c1392f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8694001570 tx=0x7f869400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400cea0 con 0x7f869c15ec80 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869400a5e0 con 0x7f869c15ec80 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400a990 con 0x7f869c15ec80 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c009420 con 0x7f869c15ec80 2024-01-15T18:15:44.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c0cd150 con 0x7f869c15ec80 2024-01-15T18:15:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c15e4a0 con 0x7f869c15ec80 2024-01-15T18:15:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f869400aaf0 con 0x7f869c15ec80 2024-01-15T18:15:44.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f8692a00640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8674067670 0x7f8674069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8694082600 con 0x7f869c15ec80 2024-01-15T18:15:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.406+0000 7f869aa00640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8674067670 0x7f8674069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.410+0000 7f869aa00640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8674067670 0x7f8674069b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f868c0af000 tx=0x7f868c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.410+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f869c15e4a0 con 0x7f869c15ec80 2024-01-15T18:15:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.582+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone993119", "target": ["mon-mgr", ""]}) v1 -- 0x7f869c072410 con 0x7f8674067670 2024-01-15T18:15:44.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.594+0000 7f8692a00640 1 -- 172.21.15.143:0/1581554328 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f868c0ab920 con 0x7f8674067670 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8674067670 msgr2=0x7f8674069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8674067670 0x7f8674069b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f868c0af000 tx=0x7f868c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 msgr2=0x7f869c1392f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 0x7f869c1392f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8694001570 tx=0x7f869400cc50 comp rx=0 tx=0).stop 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 shutdown_connections 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8674067670 0x7f8674069b30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f869c15ec80 0x7f869c1392f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f869c142470 0x7f869c070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 --2- 172.21.15.143:0/1581554328 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f869c141660 0x7f869c0711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 >> 172.21.15.143:0/1581554328 conn(0x7f869c06c3f0 msgr2=0x7f869c0afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:44.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 shutdown_connections 2024-01-15T18:15:44.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.598+0000 7f86a1600640 1 -- 172.21.15.143:0/1581554328 wait complete. 2024-01-15T18:15:44.619 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99317 2024-01-15T18:15:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 Processor -- start 2024-01-15T18:15:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 -- start start 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd1580cfb30 0x7fd15806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1580cbf50 0x7fd1580cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580cffd0 con 0x7fd1581607c0 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580d1ce0 con 0x7fd1580cbf50 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1580d02b0 con 0x7fd1580cfb30 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1580cbf50 0x7fd1580cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd1580cbf50 0x7fd1580cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52034/0 (socket says 172.21.15.143:52034) 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15de00640 1 -- 172.21.15.143:0/2982666281 learned_addr learned my addr 172.21.15.143:0/2982666281 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.762+0000 7fd15ca00640 1 --2- 172.21.15.143:0/2982666281 >> v2:172.21.15.204:3300/0 conn(0x7fd1580cfb30 0x7fd15806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd15de00640 1 -- 172.21.15.143:0/2982666281 >> v2:172.21.15.204:3300/0 conn(0x7fd1580cfb30 msgr2=0x7fd15806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd15de00640 1 --2- 172.21.15.143:0/2982666281 >> v2:172.21.15.204:3300/0 conn(0x7fd1580cfb30 0x7fd15806bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd15de00640 1 -- 172.21.15.143:0/2982666281 >> v1:172.21.15.204:6789/0 conn(0x7fd1581607c0 legacy=0x7fd15806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd15de00640 1 -- 172.21.15.143:0/2982666281 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1580cb470 con 0x7fd1580cbf50 2024-01-15T18:15:44.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd15de00640 1 --2- 172.21.15.143:0/2982666281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1580cbf50 0x7fd1580cc3c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd15809a980 tx=0x7fd15001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192e6e3409d0d283 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd157e00640 1 -- 172.21.15.143:0/2982666281 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500af400 con 0x7fd1580cbf50 2024-01-15T18:15:44.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd157e00640 1 -- 172.21.15.143:0/2982666281 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1500ab050 con 0x7fd1580cbf50 2024-01-15T18:15:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd157e00640 1 -- 172.21.15.143:0/2982666281 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500af780 con 0x7fd1580cbf50 2024-01-15T18:15:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.766+0000 7fd15fe00640 1 -- 172.21.15.143:0/2982666281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1580cbf50 msgr2=0x7fd1580cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 --2- 172.21.15.143:0/2982666281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1580cbf50 0x7fd1580cc3c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd15809a980 tx=0x7fd15001f1c0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- 172.21.15.143:0/2982666281 shutdown_connections 2024-01-15T18:15:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 --2- 172.21.15.143:0/2982666281 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1580cbf50 0x7fd1580cc3c0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 --2- 172.21.15.143:0/2982666281 >> v2:172.21.15.204:3300/0 conn(0x7fd1580cfb30 0x7fd15806bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- 172.21.15.143:0/2982666281 >> 172.21.15.143:0/2982666281 conn(0x7fd1580d0e20 msgr2=0x7fd1580d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- 172.21.15.143:0/2982666281 shutdown_connections 2024-01-15T18:15:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- 172.21.15.143:0/2982666281 wait complete. 2024-01-15T18:15:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 Processor -- start 2024-01-15T18:15:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- start start 2024-01-15T18:15:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1580cbf50 0x7fd1581571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1580cfb30 0x7fd1581f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 0x7fd1580c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580b19e0 con 0x7fd1581607c0 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580b1420 con 0x7fd1580cfb30 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580b1700 con 0x7fd1580cbf50 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 0x7fd1580c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 0x7fd1580c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52048/0 (socket says 172.21.15.143:52048) 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1580cfb30 0x7fd1581f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15de00640 1 -- 172.21.15.143:0/3600883411 learned_addr learned my addr 172.21.15.143:0/3600883411 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.770+0000 7fd15d400640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1580cbf50 0x7fd1581571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15ca00640 1 -- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1580cbf50 msgr2=0x7fd1581571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15ca00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1580cbf50 0x7fd1581571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15ca00640 1 -- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 msgr2=0x7fd1580c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15ca00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 0x7fd1580c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15ca00640 1 -- 172.21.15.143:0/3600883411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd150009b30 con 0x7fd1580cfb30 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15de00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 0x7fd1580c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15ca00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1580cfb30 0x7fd1581f33d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d5c0 tx=0x7fd14000da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1400125f0 con 0x7fd1580cfb30 2024-01-15T18:15:44.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd140012750 con 0x7fd1580cfb30 2024-01-15T18:15:44.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140012b00 con 0x7fd1580cfb30 2024-01-15T18:15:44.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1581588d0 con 0x7fd1580cfb30 2024-01-15T18:15:44.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1581f3910 con 0x7fd1580cfb30 2024-01-15T18:15:44.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15806d4b0 con 0x7fd1580cfb30 2024-01-15T18:15:44.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7fd140012c60 con 0x7fd1580cfb30 2024-01-15T18:15:44.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd155600640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd130067670 0x7fd130069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd140083540 con 0x7fd1580cfb30 2024-01-15T18:15:44.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.774+0000 7fd15d400640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd130067670 0x7fd130069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.778+0000 7fd15d400640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd130067670 0x7fd130069b30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd1580b1040 tx=0x7fd148009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.782+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd140051fd0 con 0x7fd1580cfb30 2024-01-15T18:15:44.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.962+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99317", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1580ce6b0 con 0x7fd130067670 2024-01-15T18:15:44.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.974+0000 7fd155600640 1 -- 172.21.15.143:0/3600883411 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd148001470 con 0x7fd130067670 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd130067670 msgr2=0x7fd130069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd130067670 0x7fd130069b30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd1580b1040 tx=0x7fd148009040 comp rx=0 tx=0).stop 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1580cfb30 msgr2=0x7fd1581f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1580cfb30 0x7fd1581f33d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd14000d5c0 tx=0x7fd14000da80 comp rx=0 tx=0).stop 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 shutdown_connections 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd130067670 0x7fd130069b30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1581607c0 0x7fd1580c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd1580cfb30 0x7fd1581f33d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 --2- 172.21.15.143:0/3600883411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd1580cbf50 0x7fd1581571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 >> 172.21.15.143:0/3600883411 conn(0x7fd1580d0e20 msgr2=0x7fd15815e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 shutdown_connections 2024-01-15T18:15:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:44.978+0000 7fd15fe00640 1 -- 172.21.15.143:0/3600883411 wait complete. 2024-01-15T18:15:44.998 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone99313 2024-01-15T18:15:45.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.158+0000 7f156ae00640 1 Processor -- start 2024-01-15T18:15:45.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.158+0000 7f156ae00640 1 -- start start 2024-01-15T18:15:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.158+0000 7f156ae00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f15640b20b0 0x7f15640b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.158+0000 7f156ae00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15640cc790 con 0x7f156406db20 2024-01-15T18:15:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.158+0000 7f156ae00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15640b4ae0 con 0x7f156415a440 2024-01-15T18:15:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.158+0000 7f156ae00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15640b4570 con 0x7f15640b20b0 2024-01-15T18:15:45.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1563e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f156406db20 0x7f15640b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:52382/0 (socket says 172.21.15.143:52382) 2024-01-15T18:15:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1563e00640 1 -- 172.21.15.143:0/341856452 learned_addr learned my addr 172.21.15.143:0/341856452 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1568e00640 1 --2- 172.21.15.143:0/341856452 >> v2:172.21.15.204:3300/0 conn(0x7f15640b20b0 0x7f15640b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1562a00640 1 -- 172.21.15.143:0/341856452 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102280821 0 0) 0x7f15640cc790 con 0x7f156406db20 2024-01-15T18:15:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1562a00640 1 -- 172.21.15.143:0/341856452 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1544003530 con 0x7f156406db20 2024-01-15T18:15:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1568e00640 1 -- 172.21.15.143:0/341856452 >> v1:172.21.15.204:6789/0 conn(0x7f156415a440 legacy=0x7f15640cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1568e00640 1 -- 172.21.15.143:0/341856452 >> v1:172.21.15.143:6789/0 conn(0x7f156406db20 legacy=0x7f15640b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.162+0000 7f1568e00640 1 -- 172.21.15.143:0/341856452 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15640b4800 con 0x7f15640b20b0 2024-01-15T18:15:45.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.166+0000 7f1568e00640 1 --2- 172.21.15.143:0/341856452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15640b20b0 0x7f15640b2520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f155c009c80 tx=0x7f155c01da10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1024dbd28724948f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:45.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.166+0000 7f1562a00640 1 -- 172.21.15.143:0/341856452 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0a6ad0 con 0x7f15640b20b0 2024-01-15T18:15:45.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.166+0000 7f1562a00640 1 -- 172.21.15.143:0/341856452 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c0a6c30 con 0x7f15640b20b0 2024-01-15T18:15:45.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.166+0000 7f1562a00640 1 -- 172.21.15.143:0/341856452 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c0ae440 con 0x7f15640b20b0 2024-01-15T18:15:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.166+0000 7f156ae00640 1 -- 172.21.15.143:0/341856452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15640b20b0 msgr2=0x7f15640b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.166+0000 7f156ae00640 1 --2- 172.21.15.143:0/341856452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15640b20b0 0x7f15640b2520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f155c009c80 tx=0x7f155c01da10 comp rx=0 tx=0).stop 2024-01-15T18:15:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.170+0000 7f156ae00640 1 -- 172.21.15.143:0/341856452 shutdown_connections 2024-01-15T18:15:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.170+0000 7f156ae00640 1 --2- 172.21.15.143:0/341856452 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f15640b20b0 0x7f15640b2520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.170+0000 7f156ae00640 1 -- 172.21.15.143:0/341856452 >> 172.21.15.143:0/341856452 conn(0x7f15640aea10 msgr2=0x7f15640aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.170+0000 7f156ae00640 1 -- 172.21.15.143:0/341856452 shutdown_connections 2024-01-15T18:15:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.170+0000 7f156ae00640 1 -- 172.21.15.143:0/341856452 wait complete. 2024-01-15T18:15:45.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 Processor -- start 2024-01-15T18:15:45.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 -- start start 2024-01-15T18:15:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f156406db20 0x7f15641648b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 0x7f156414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156415a440 0x7f15641555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15641551d0 con 0x7f156415a440 2024-01-15T18:15:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156414fd80 con 0x7f156406db20 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f156ae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1564154300 con 0x7f15640b20b0 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f1563400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 0x7f156414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f1563400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 0x7f156414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:50222/0 (socket says 172.21.15.143:50222) 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f1563400640 1 -- 172.21.15.143:0/3430802154 learned_addr learned my addr 172.21.15.143:0/3430802154 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.174+0000 7f1563e00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f156406db20 0x7f15641648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f1563400640 1 -- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f156406db20 msgr2=0x7f15641648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f1563400640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f156406db20 0x7f15641648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f1563400640 1 -- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156415a440 msgr2=0x7f15641555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f1563400640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156415a440 0x7f15641555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f1563400640 1 -- 172.21.15.143:0/3430802154 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c00d040 con 0x7f15640b20b0 2024-01-15T18:15:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f1563400640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 0x7f156414f840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f154c00c3c0 tx=0x7f154c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0052d0 con 0x7f15640b20b0 2024-01-15T18:15:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c005430 con 0x7f15640b20b0 2024-01-15T18:15:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c003840 con 0x7f15640b20b0 2024-01-15T18:15:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1564154550 con 0x7f15640b20b0 2024-01-15T18:15:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1564154b10 con 0x7f15640b20b0 2024-01-15T18:15:45.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.178+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1564155310 con 0x7f15640b20b0 2024-01-15T18:15:45.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.182+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f154c0039a0 con 0x7f15640b20b0 2024-01-15T18:15:45.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.182+0000 7f155be00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f153c067670 0x7f153c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.182+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f154c082660 con 0x7f15640b20b0 2024-01-15T18:15:45.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.182+0000 7f1563e00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f153c067670 0x7f153c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.182+0000 7f1563e00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f153c067670 0x7f153c069b30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f15540049e0 tx=0x7f1554006020 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:45.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.190+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f154c0510f0 con 0x7f15640b20b0 2024-01-15T18:15:45.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.414+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone99313", "target": ["mon-mgr", ""]}) v1 -- 0x7f1564164ec0 con 0x7f153c067670 2024-01-15T18:15:45.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.426+0000 7f155be00640 1 -- 172.21.15.143:0/3430802154 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1564164ec0 con 0x7f153c067670 2024-01-15T18:15:45.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f153c067670 msgr2=0x7f153c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f153c067670 0x7f153c069b30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f15540049e0 tx=0x7f1554006020 comp rx=0 tx=0).stop 2024-01-15T18:15:45.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 msgr2=0x7f156414f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 0x7f156414f840 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f154c00c3c0 tx=0x7f154c00c880 comp rx=0 tx=0).stop 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 shutdown_connections 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f153c067670 0x7f153c069b30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f156415a440 0x7f15641555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f15640b20b0 0x7f156414f840 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 --2- 172.21.15.143:0/3430802154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f156406db20 0x7f15641648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 >> 172.21.15.143:0/3430802154 conn(0x7f15640aea10 msgr2=0x7f15641531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 shutdown_connections 2024-01-15T18:15:45.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.430+0000 7f156ae00640 1 -- 172.21.15.143:0/3430802154 wait complete. 2024-01-15T18:15:45.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:45.449 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:15:45.466 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-15T18:15:45.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:15:45.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 Processor -- start 2024-01-15T18:15:45.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 -- start start 2024-01-15T18:15:45.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fb8143780 0x7f2fb80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fb80722c0 0x7f2fb815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb80ceb50 con 0x7f2fb80ce520 2024-01-15T18:15:45.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb815c110 con 0x7f2fb8143780 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fbe200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb815b7e0 con 0x7f2fb80722c0 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fb7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fb8143780 0x7f2fb80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fb7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fb8143780 0x7f2fb80cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52066/0 (socket says 172.21.15.143:52066) 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fb7400640 1 -- 172.21.15.143:0/1695620799 learned_addr learned my addr 172.21.15.143:0/1695620799 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.650+0000 7f2fb7e00640 1 --2- 172.21.15.143:0/1695620799 >> v2:172.21.15.204:3300/0 conn(0x7f2fb80722c0 0x7f2fb815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb7400640 1 -- 172.21.15.143:0/1695620799 >> v2:172.21.15.204:3300/0 conn(0x7f2fb80722c0 msgr2=0x7f2fb815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb7400640 1 --2- 172.21.15.143:0/1695620799 >> v2:172.21.15.204:3300/0 conn(0x7f2fb80722c0 0x7f2fb815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb7400640 1 -- 172.21.15.143:0/1695620799 >> v1:172.21.15.204:6789/0 conn(0x7f2fb80ce520 legacy=0x7f2fb8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb7400640 1 -- 172.21.15.143:0/1695620799 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb815bac0 con 0x7f2fb8143780 2024-01-15T18:15:45.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb7400640 1 --2- 172.21.15.143:0/1695620799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb8143780 0x7f2fb80cdfe0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8070a20 tx=0x7f2fa801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0d881ab490ee3f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:45.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb6000640 1 -- 172.21.15.143:0/1695620799 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80ab610 con 0x7f2fb8143780 2024-01-15T18:15:45.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb6000640 1 -- 172.21.15.143:0/1695620799 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa80ab770 con 0x7f2fb8143780 2024-01-15T18:15:45.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fb6000640 1 -- 172.21.15.143:0/1695620799 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa80abb00 con 0x7f2fb8143780 2024-01-15T18:15:45.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fbe200640 1 -- 172.21.15.143:0/1695620799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb8143780 msgr2=0x7f2fb80cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.654+0000 7f2fbe200640 1 --2- 172.21.15.143:0/1695620799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb8143780 0x7f2fb80cdfe0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8070a20 tx=0x7f2fa801c500 comp rx=0 tx=0).stop 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- 172.21.15.143:0/1695620799 shutdown_connections 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 --2- 172.21.15.143:0/1695620799 >> v2:172.21.15.204:3300/0 conn(0x7f2fb80722c0 0x7f2fb815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 --2- 172.21.15.143:0/1695620799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb8143780 0x7f2fb80cdfe0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- 172.21.15.143:0/1695620799 >> 172.21.15.143:0/1695620799 conn(0x7f2fb806c3f0 msgr2=0x7f2fb806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- 172.21.15.143:0/1695620799 shutdown_connections 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- 172.21.15.143:0/1695620799 wait complete. 2024-01-15T18:15:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 Processor -- start 2024-01-15T18:15:45.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- start start 2024-01-15T18:15:45.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 0x7f2fb8159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80ce520 0x7f2fb8159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fb8143780 0x7f2fb81f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb813c640 con 0x7f2fb80ce520 2024-01-15T18:15:45.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80b2580 con 0x7f2fb80722c0 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fbe200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80b2860 con 0x7f2fb8143780 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80ce520 0x7f2fb8159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 0x7f2fb8159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 0x7f2fb8159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:47368/0 (socket says 172.21.15.143:47368) 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb7400640 1 -- 172.21.15.143:0/112687934 learned_addr learned my addr 172.21.15.143:0/112687934 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb7e00640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fb8143780 0x7f2fb81f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb6a00640 1 -- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fb8143780 msgr2=0x7f2fb81f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb6a00640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fb8143780 0x7f2fb81f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb6a00640 1 -- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 msgr2=0x7f2fb8159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.658+0000 7f2fb6a00640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 0x7f2fb8159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fb6a00640 1 -- 172.21.15.143:0/112687934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa80094d0 con 0x7f2fb80ce520 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fb7400640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 0x7f2fb8159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:45.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fb6a00640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80ce520 0x7f2fb8159020 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0001ec0 tx=0x7f2fa000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:45.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa00052d0 con 0x7f2fb80ce520 2024-01-15T18:15:45.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa0005430 con 0x7f2fb80ce520 2024-01-15T18:15:45.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0003840 con 0x7f2fb80ce520 2024-01-15T18:15:45.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb8159560 con 0x7f2fb80ce520 2024-01-15T18:15:45.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb813cb70 con 0x7f2fb80ce520 2024-01-15T18:15:45.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb813cd00 con 0x7f2fb80ce520 2024-01-15T18:15:45.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 29) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f2fa00039a0 con 0x7f2fb80ce520 2024-01-15T18:15:45.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2faf400640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f900676a0 0x7f2f90069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:45.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f2fa0082440 con 0x7f2fb80ce520 2024-01-15T18:15:45.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.662+0000 7f2fb7400640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f900676a0 0x7f2f90069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:45.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.666+0000 7f2fb7400640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f900676a0 0x7f2f90069b60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2fa801c540 tx=0x7f2fa80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:45.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.666+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2fa0050f10 con 0x7f2fb80ce520 2024-01-15T18:15:45.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:45.838+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fb80ceb50 con 0x7f2f900676a0 2024-01-15T18:15:46.363 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:15:46.362+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:15:46.457 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:15:47.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:47.874+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f2fa0082020 con 0x7f2fb80ce520 2024-01-15T18:15:49.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.386+0000 7f2faf400640 1 -- 172.21.15.143:0/112687934 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f2fb80ceb50 con 0x7f2f900676a0 2024-01-15T18:15:49.384 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:15:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f900676a0 msgr2=0x7f2f90069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f900676a0 0x7f2f90069b60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2fa801c540 tx=0x7f2fa80b6040 comp rx=0 tx=0).stop 2024-01-15T18:15:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80ce520 msgr2=0x7f2fb8159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80ce520 0x7f2fb8159020 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0001ec0 tx=0x7f2fa000c890 comp rx=0 tx=0).stop 2024-01-15T18:15:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 shutdown_connections 2024-01-15T18:15:49.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f900676a0 0x7f2f90069b60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fb8143780 0x7f2fb81f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fb80ce520 0x7f2fb8159020 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 --2- 172.21.15.143:0/112687934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fb80722c0 0x7f2fb8159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 >> 172.21.15.143:0/112687934 conn(0x7f2fb806c3f0 msgr2=0x7f2fb813eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 shutdown_connections 2024-01-15T18:15:49.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.390+0000 7f2fbe200640 1 -- 172.21.15.143:0/112687934 wait complete. 2024-01-15T18:15:49.410 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:15:49.410 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:15:49.488 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:15:49.631 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:15:49.631 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:15:49.631 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:49.641 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:15:49.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dd1a00640 1 Processor -- start 2024-01-15T18:15:49.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dd1a00640 1 -- start start 2024-01-15T18:15:49.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dd1a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7dcc15b260 0x7f7dcc0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dd1a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc0c4d90 con 0x7f7dcc0ccaf0 2024-01-15T18:15:49.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dd1a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dcc15b640 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dd1a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dcc0cf800 con 0x7f7dcc15b260 2024-01-15T18:15:49.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dcaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7dcc15b260 0x7f7dcc0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dcaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7dcc15b260 0x7f7dcc0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36778/0 (socket says 172.21.15.143:36778) 2024-01-15T18:15:49.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dcaa00640 1 -- 172.21.15.143:0/2173595522 learned_addr learned my addr 172.21.15.143:0/2173595522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:49.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dcaa00640 1 -- 172.21.15.143:0/2173595522 >> v1:172.21.15.204:6789/0 conn(0x7f7dcc0d19f0 legacy=0x7f7dcc0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dcaa00640 1 -- 172.21.15.143:0/2173595522 >> v1:172.21.15.143:6789/0 conn(0x7f7dcc0ccaf0 legacy=0x7f7dcc0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.826+0000 7f7dcaa00640 1 -- 172.21.15.143:0/2173595522 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc0cfae0 con 0x7f7dcc15b260 2024-01-15T18:15:49.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dcaa00640 1 --2- 172.21.15.143:0/2173595522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc0cd810 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009870 tx=0x7f7dbc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc8bef3c9c17428 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:49.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dc9600640 1 -- 172.21.15.143:0/2173595522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0ab600 con 0x7f7dcc15b260 2024-01-15T18:15:49.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dc9600640 1 -- 172.21.15.143:0/2173595522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dbc0a7050 con 0x7f7dcc15b260 2024-01-15T18:15:49.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dc9600640 1 -- 172.21.15.143:0/2173595522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0ab980 con 0x7f7dcc15b260 2024-01-15T18:15:49.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 -- 172.21.15.143:0/2173595522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 msgr2=0x7f7dcc0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/2173595522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc0cd810 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009870 tx=0x7f7dbc01c540 comp rx=0 tx=0).stop 2024-01-15T18:15:49.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 -- 172.21.15.143:0/2173595522 shutdown_connections 2024-01-15T18:15:49.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/2173595522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc0cd810 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 -- 172.21.15.143:0/2173595522 >> 172.21.15.143:0/2173595522 conn(0x7f7dcc0d0ed0 msgr2=0x7f7dcc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:49.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 -- 172.21.15.143:0/2173595522 shutdown_connections 2024-01-15T18:15:49.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.830+0000 7f7dd1a00640 1 -- 172.21.15.143:0/2173595522 wait complete. 2024-01-15T18:15:49.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 Processor -- start 2024-01-15T18:15:49.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 -- start start 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dcc0ccaf0 0x7f7dcc151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dcc0d19f0 0x7f7dcc159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06cd90 con 0x7f7dcc15b260 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc152820 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dcc06c180 con 0x7f7dcc0ccaf0 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dcb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dcc0d19f0 0x7f7dcc159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.832 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dcb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36784/0 (socket says 172.21.15.143:36784) 2024-01-15T18:15:49.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dcb400640 1 -- 172.21.15.143:0/3480050637 learned_addr learned my addr 172.21.15.143:0/3480050637 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:49.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dcaa00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dcc0ccaf0 0x7f7dcc151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 -- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dcc0ccaf0 msgr2=0x7f7dcc151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dcc0ccaf0 0x7f7dcc151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 -- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 msgr2=0x7f7dcc159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 -- 172.21.15.143:0/3480050637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0094d0 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dcaa00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dcc0ccaf0 0x7f7dcc151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dcb400640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dca000640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dcc0d19f0 0x7f7dcc159250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7db4000f70 tx=0x7f7db400c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db40052d0 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db4005430 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4003840 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dcc06c460 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.834+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dcc06ca20 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.838+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc067500 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.838+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f7db40039a0 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.838+0000 7f7dc2a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7da40675e0 0x7f7da4069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.838+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f7db4082310 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.838+0000 7f7dcaa00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7da40675e0 0x7f7da4069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.838+0000 7f7dcaa00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7da40675e0 0x7f7da4069aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc071260 tx=0x7f7dbc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.842+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7db4050fc0 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.994+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dcc0cb400 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.994+0000 7f7dc2a00640 1 -- 172.21.15.143:0/3480050637 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13113 (secure 0 0 0) 0x7f7db4017070 con 0x7f7dcc0d19f0 2024-01-15T18:15:49.993 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:49.993 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":78,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:15:49.381489+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/1300573627":"2024-01-16T18:15:46.367103+0000","172.21.15.143:6835/1300573627":"2024-01-16T18:15:46.367103+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:15:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7da40675e0 msgr2=0x7f7da4069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7da40675e0 0x7f7da4069aa0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc071260 tx=0x7f7dbc0a6000 comp rx=0 tx=0).stop 2024-01-15T18:15:49.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dcc0d19f0 msgr2=0x7f7dcc159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dcc0d19f0 0x7f7dcc159250 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7db4000f70 tx=0x7f7db400c820 comp rx=0 tx=0).stop 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 shutdown_connections 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7da40675e0 0x7f7da4069aa0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7dcc15b260 0x7f7dcc159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7dcc0d19f0 0x7f7dcc159250 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 --2- 172.21.15.143:0/3480050637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7dcc0ccaf0 0x7f7dcc151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 >> 172.21.15.143:0/3480050637 conn(0x7f7dcc0d0ed0 msgr2=0x7f7dcc0c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 shutdown_connections 2024-01-15T18:15:49.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:49.998+0000 7f7dd1a00640 1 -- 172.21.15.143:0/3480050637 wait complete. 2024-01-15T18:15:50.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:15:50.016 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:15:50.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1551c00640 1 Processor -- start 2024-01-15T18:15:50.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1551c00640 1 -- start start 2024-01-15T18:15:50.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1551c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 0x7f154c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1551c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0cbff0 con 0x7f154c06bfb0 2024-01-15T18:15:50.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1551c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f154c0d0c50 con 0x7f154c0b0310 2024-01-15T18:15:50.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1551c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0cb6c0 con 0x7f154c0cf910 2024-01-15T18:15:50.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f154aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f154c06bfb0 0x7f154c06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33406/0 (socket says 172.21.15.143:33406) 2024-01-15T18:15:50.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f154aa00640 1 -- 172.21.15.143:0/935375039 learned_addr learned my addr 172.21.15.143:0/935375039 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:50.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f154b400640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 0x7f154c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1233254429 0 0) 0x7f154c0cbff0 con 0x7f154c06bfb0 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f152c003530 con 0x7f154c06bfb0 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135209159 0 0) 0x7f154c0d0c50 con 0x7f154c0b0310 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.162+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f154c0cbff0 con 0x7f154c0b0310 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1933873537 0 0) 0x7f152c003530 con 0x7f154c06bfb0 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 msgr2=0x7f154c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 0x7f154c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 >> v1:172.21.15.204:6789/0 conn(0x7f154c0b0310 legacy=0x7f154c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0cb9a0 con 0x7f154c06bfb0 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f153c001680 con 0x7f154c06bfb0 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 >> v1:172.21.15.143:6789/0 conn(0x7f154c06bfb0 legacy=0x7f154c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 0x7f152c003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f152c0040f0 0x7f152c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c004d10 0x7f152c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c0cb6c0 con 0x7f152c0040f0 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c00bb50 con 0x7f152c004d10 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c00bd10 con 0x7f154c0cf910 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f152c0040f0 0x7f152c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154aa00640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 0x7f152c003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154b400640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c004d10 0x7f152c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 -- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 msgr2=0x7f152c003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0cf910 0x7f152c003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 -- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c004d10 msgr2=0x7f152c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c004d10 0x7f152c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 -- 172.21.15.143:0/935375039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c003530 con 0x7f152c0040f0 2024-01-15T18:15:50.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f154a000640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f152c0040f0 0x7f152c0047d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f154c0b06f0 tx=0x7f153401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c53edaee2e402c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:50.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15340052d0 con 0x7f152c0040f0 2024-01-15T18:15:50.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1534005450 con 0x7f152c0040f0 2024-01-15T18:15:50.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.166+0000 7f1549600640 1 -- 172.21.15.143:0/935375039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534003840 con 0x7f152c0040f0 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- 172.21.15.143:0/935375039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f152c0040f0 msgr2=0x7f152c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 --2- 172.21.15.143:0/935375039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f152c0040f0 0x7f152c0047d0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f154c0b06f0 tx=0x7f153401c510 comp rx=0 tx=0).stop 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f154b400640 1 -- 172.21.15.143:0/935375039 reap_dead start 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- 172.21.15.143:0/935375039 shutdown_connections 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- 172.21.15.143:0/935375039 >> 172.21.15.143:0/935375039 conn(0x7f154c0d0ed0 msgr2=0x7f154c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- 172.21.15.143:0/935375039 shutdown_connections 2024-01-15T18:15:50.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- 172.21.15.143:0/935375039 wait complete. 2024-01-15T18:15:50.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 Processor -- start 2024-01-15T18:15:50.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- start start 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c0040f0 0x7f154c0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0c8030 0x7f154c06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 0x7f154c06f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c15a0a0 con 0x7f154c153ec0 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154c159b70 con 0x7f152c0040f0 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.170+0000 7f1551c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c159dc0 con 0x7f154c0c8030 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 0x7f154c06f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 0x7f154c06f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36806/0 (socket says 172.21.15.143:36806) 2024-01-15T18:15:50.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 -- 172.21.15.143:0/297023317 learned_addr learned my addr 172.21.15.143:0/297023317 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154aa00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c0040f0 0x7f154c0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154a000640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0c8030 0x7f154c06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 -- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0c8030 msgr2=0x7f154c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0c8030 0x7f154c06f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 -- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c0040f0 msgr2=0x7f154c0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c0040f0 0x7f154c0c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 -- 172.21.15.143:0/297023317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0c78a0 con 0x7f154c153ec0 2024-01-15T18:15:50.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f154b400640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 0x7f154c06f9b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f154c071bb0 tx=0x7f154400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:50.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154400e040 con 0x7f154c153ec0 2024-01-15T18:15:50.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154400f040 con 0x7f154c153ec0 2024-01-15T18:15:50.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544015430 con 0x7f154c153ec0 2024-01-15T18:15:50.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154c0c7b80 con 0x7f154c153ec0 2024-01-15T18:15:50.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154c153b60 con 0x7f154c153ec0 2024-01-15T18:15:50.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154c0c7700 con 0x7f154c153ec0 2024-01-15T18:15:50.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.174+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f1544015590 con 0x7f154c153ec0 2024-01-15T18:15:50.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.178+0000 7f1542a00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15240675e0 0x7f1524069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.178+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f1544083a30 con 0x7f154c153ec0 2024-01-15T18:15:50.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.178+0000 7f154aa00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15240675e0 0x7f1524069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.178+0000 7f154aa00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15240675e0 0x7f1524069aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f154c1543e0 tx=0x7f153c001010 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:50.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.182+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f154c0c7700 con 0x7f154c153ec0 2024-01-15T18:15:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.330+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f154c0c7700 con 0x7f154c153ec0 2024-01-15T18:15:50.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1542a00640 1 -- 172.21.15.143:0/297023317 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f154c0c7700 con 0x7f154c153ec0 2024-01-15T18:15:50.330 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15240675e0 msgr2=0x7f1524069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15240675e0 0x7f1524069aa0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f154c1543e0 tx=0x7f153c001010 comp rx=0 tx=0).stop 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 msgr2=0x7f154c06f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 0x7f154c06f9b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f154c071bb0 tx=0x7f154400f580 comp rx=0 tx=0).stop 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 shutdown_connections 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f15240675e0 0x7f1524069aa0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f154c153ec0 0x7f154c06f9b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f154c0c8030 0x7f154c06f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 --2- 172.21.15.143:0/297023317 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f152c0040f0 0x7f154c0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 >> 172.21.15.143:0/297023317 conn(0x7f154c0d0ed0 msgr2=0x7f154c0d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:50.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 shutdown_connections 2024-01-15T18:15:50.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.334+0000 7f1551c00640 1 -- 172.21.15.143:0/297023317 wait complete. 2024-01-15T18:15:50.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:15:50.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 Processor -- start 2024-01-15T18:15:50.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 -- start start 2024-01-15T18:15:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c401607c0 0x7f3c4006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400cffd0 con 0x7f3c400cfb30 2024-01-15T18:15:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c400d1ce0 con 0x7f3c401607c0 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c47e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400d02b0 con 0x7f3c400cbf50 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c401607c0 0x7f3c4006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3c401607c0 0x7f3c4006b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36818/0 (socket says 172.21.15.143:36818) 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 -- 172.21.15.143:0/3149954631 learned_addr learned my addr 172.21.15.143:0/3149954631 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45e00640 1 --2- 172.21.15.143:0/3149954631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 -- 172.21.15.143:0/3149954631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 msgr2=0x7f3c400cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 --2- 172.21.15.143:0/3149954631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 -- 172.21.15.143:0/3149954631 >> v1:172.21.15.204:6789/0 conn(0x7f3c400cfb30 legacy=0x7f3c4006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c3fe00640 1 -- 172.21.15.143:0/3149954631 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112293606 0 0) 0x7f3c400cffd0 con 0x7f3c400cfb30 2024-01-15T18:15:50.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.494+0000 7f3c45400640 1 -- 172.21.15.143:0/3149954631 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400cb470 con 0x7f3c401607c0 2024-01-15T18:15:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c45400640 1 --2- 172.21.15.143:0/3149954631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c4006b7b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009870 tx=0x7f3c3001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55693e4f0b4fce0b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c3fe00640 1 -- 172.21.15.143:0/3149954631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c300a8a30 con 0x7f3c401607c0 2024-01-15T18:15:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c3fe00640 1 -- 172.21.15.143:0/3149954631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c300a8b90 con 0x7f3c401607c0 2024-01-15T18:15:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c3fe00640 1 -- 172.21.15.143:0/3149954631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c300b0440 con 0x7f3c401607c0 2024-01-15T18:15:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 -- 172.21.15.143:0/3149954631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 msgr2=0x7f3c4006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 --2- 172.21.15.143:0/3149954631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c4006b7b0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009870 tx=0x7f3c3001ed30 comp rx=0 tx=0).stop 2024-01-15T18:15:50.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 -- 172.21.15.143:0/3149954631 shutdown_connections 2024-01-15T18:15:50.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 --2- 172.21.15.143:0/3149954631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 --2- 172.21.15.143:0/3149954631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c4006b7b0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 -- 172.21.15.143:0/3149954631 >> 172.21.15.143:0/3149954631 conn(0x7f3c400d0e20 msgr2=0x7f3c400d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:50.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 -- 172.21.15.143:0/3149954631 shutdown_connections 2024-01-15T18:15:50.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.498+0000 7f3c47e00640 1 -- 172.21.15.143:0/3149954631 wait complete. 2024-01-15T18:15:50.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 Processor -- start 2024-01-15T18:15:50.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 -- start start 2024-01-15T18:15:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c400cfb30 0x7f3c4015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c400cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400cb250 con 0x7f3c401607c0 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c9500 con 0x7f3c400cfb30 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c97e0 con 0x7f3c400cbf50 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c400cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c400cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36822/0 (socket says 172.21.15.143:36822) 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 -- 172.21.15.143:0/1170020945 learned_addr learned my addr 172.21.15.143:0/1170020945 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c44a00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c400cfb30 0x7f3c4015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45400640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 -- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 msgr2=0x7f3c400cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 -- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c400cfb30 msgr2=0x7f3c4015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c400cfb30 0x7f3c4015c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 -- 172.21.15.143:0/1170020945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300094d0 con 0x7f3c401607c0 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c44a00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c400cfb30 0x7f3c4015c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c45e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c400cd760 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800efc0 tx=0x7f3c3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38014480 con 0x7f3c401607c0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800f040 con 0x7f3c401607c0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c400c9ac0 con 0x7f3c401607c0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38014820 con 0x7f3c401607c0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.502+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4015c8a0 con 0x7f3c401607c0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.506+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4006d4b0 con 0x7f3c401607c0 2024-01-15T18:15:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.506+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f3c38014980 con 0x7f3c401607c0 2024-01-15T18:15:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.506+0000 7f3c3d600640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c18067610 0x7f3c18069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:50.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.506+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f3c38083ba0 con 0x7f3c401607c0 2024-01-15T18:15:50.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.506+0000 7f3c45400640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c18067610 0x7f3c18069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:50.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.506+0000 7f3c45400640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c18067610 0x7f3c18069ad0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c300b2000 tx=0x7f3c30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:50.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.510+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3c380525a0 con 0x7f3c401607c0 2024-01-15T18:15:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3c400c9f70 con 0x7f3c401607c0 2024-01-15T18:15:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c3d600640 1 -- 172.21.15.143:0/1170020945 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) v1 ==== 80+0+124 (secure 0 0 0) 0x7f3c38018020 con 0x7f3c401607c0 2024-01-15T18:15:50.667 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/1300573627 2024-01-16T18:15:46.367103+0000 2024-01-15T18:15:50.668 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/1300573627 2024-01-16T18:15:46.367103+0000 2024-01-15T18:15:50.668 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:15:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c18067610 msgr2=0x7f3c18069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c18067610 0x7f3c18069ad0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c300b2000 tx=0x7f3c30009040 comp rx=0 tx=0).stop 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 msgr2=0x7f3c400cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c400cd760 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800efc0 tx=0x7f3c3800c490 comp rx=0 tx=0).stop 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 shutdown_connections 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c18067610 0x7f3c18069ad0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c401607c0 0x7f3c400cd760 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c400cfb30 0x7f3c4015c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 --2- 172.21.15.143:0/1170020945 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c400cbf50 0x7f3c400cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.670+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 >> 172.21.15.143:0/1170020945 conn(0x7f3c400d0e20 msgr2=0x7f3c400cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.674+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 shutdown_connections 2024-01-15T18:15:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:50.674+0000 7f3c47e00640 1 -- 172.21.15.143:0/1170020945 wait complete. 2024-01-15T18:15:50.690 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:15:50.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:50.690 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:15:50.709 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[44865]: fuse finished with error 0 and tester_r 0 2024-01-15T18:15:50.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:15:50.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:50.710 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:15:50.978 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:15:50.979 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:50.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:15:51.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.242+0000 7f2203800640 1 Processor -- start 2024-01-15T18:15:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.242+0000 7f2203800640 1 -- start start 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2203800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21fc0b0310 0x7f21fc0cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2203800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc0cf740 0x7f21fc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2203800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21fc0cbff0 con 0x7f21fc06bfb0 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2203800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21fc0d0c50 con 0x7f21fc0b0310 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2203800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0cb6c0 con 0x7f21fc0cf740 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21fc0b0310 0x7f21fc0cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21fc0b0310 0x7f21fc0cf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36838/0 (socket says 172.21.15.143:36838) 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 -- 172.21.15.143:0/4215702848 learned_addr learned my addr 172.21.15.143:0/4215702848 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2201800640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc0cf740 0x7f21fc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 -- 172.21.15.143:0/4215702848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc0cf740 msgr2=0x7f21fc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc0cf740 0x7f21fc0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 -- 172.21.15.143:0/4215702848 >> v1:172.21.15.143:6789/0 conn(0x7f21fc06bfb0 legacy=0x7f21fc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f21fbe00640 1 -- 172.21.15.143:0/4215702848 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21fc0cb9a0 con 0x7f21fc0b0310 2024-01-15T18:15:51.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.246+0000 7f2201800640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc0cf740 0x7f21fc0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:51.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f21fbe00640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0cf200 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009870 tx=0x7f21e401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ff4547b45c66 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f21fb400640 1 -- 172.21.15.143:0/4215702848 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40ab680 con 0x7f21fc0b0310 2024-01-15T18:15:51.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f21fb400640 1 -- 172.21.15.143:0/4215702848 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e40ab7e0 con 0x7f21fc0b0310 2024-01-15T18:15:51.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f21fb400640 1 -- 172.21.15.143:0/4215702848 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40abb70 con 0x7f21fc0b0310 2024-01-15T18:15:51.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 -- 172.21.15.143:0/4215702848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 msgr2=0x7f21fc0cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0cf200 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009870 tx=0x7f21e401c480 comp rx=0 tx=0).stop 2024-01-15T18:15:51.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 -- 172.21.15.143:0/4215702848 shutdown_connections 2024-01-15T18:15:51.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc0cf740 0x7f21fc0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 --2- 172.21.15.143:0/4215702848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0cf200 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 -- 172.21.15.143:0/4215702848 >> 172.21.15.143:0/4215702848 conn(0x7f21fc0d0ed0 msgr2=0x7f21fc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:51.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 -- 172.21.15.143:0/4215702848 shutdown_connections 2024-01-15T18:15:51.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 -- 172.21.15.143:0/4215702848 wait complete. 2024-01-15T18:15:51.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.250+0000 7f2203800640 1 Processor -- start 2024-01-15T18:15:51.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 -- start start 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc06bfb0 0x7f21fc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21fc0cf740 0x7f21fc154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc156420 con 0x7f21fc0b0310 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc0c4a00 con 0x7f21fc0cf740 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc1571e0 con 0x7f21fc06bfb0 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0c76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36852/0 (socket says 172.21.15.143:36852) 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 -- 172.21.15.143:0/923204611 learned_addr learned my addr 172.21.15.143:0/923204611 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:51.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2201800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21fc0cf740 0x7f21fc154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2200e00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc06bfb0 0x7f21fc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 -- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc06bfb0 msgr2=0x7f21fc1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc06bfb0 0x7f21fc1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 -- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21fc0cf740 msgr2=0x7f21fc154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21fc0cf740 0x7f21fc154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 -- 172.21.15.143:0/923204611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40094d0 con 0x7f21fc0b0310 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21fbe00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0c76d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f21e40097c0 tx=0x7f21e401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40b4800 con 0x7f21fc0b0310 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e40b4980 con 0x7f21fc0b0310 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e40b4d30 con 0x7f21fc0b0310 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc1574c0 con 0x7f21fc0b0310 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.254+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc1591e0 con 0x7f21fc0b0310 2024-01-15T18:15:51.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.258+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21fc159040 con 0x7f21fc0b0310 2024-01-15T18:15:51.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.258+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f21e40b3380 con 0x7f21fc0b0310 2024-01-15T18:15:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.258+0000 7f21f8c00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21d40675e0 0x7f21d4069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.258+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f21e411d830 con 0x7f21fc0b0310 2024-01-15T18:15:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.258+0000 7f2200e00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21d40675e0 0x7f21d4069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.258+0000 7f2200e00640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21d40675e0 0x7f21d4069aa0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0afc90 tx=0x7f21ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.266+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21fc159040 con 0x7f21fc0b0310 2024-01-15T18:15:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21fc159040 con 0x7f21fc0b0310 2024-01-15T18:15:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f21f8c00640 1 -- 172.21.15.143:0/923204611 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13113 (secure 0 0 0) 0x7f21fc159040 con 0x7f21fc0b0310 2024-01-15T18:15:51.439 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:51.439 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":78,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:15:49.381489+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/1300573627":"2024-01-16T18:15:46.367103+0000","172.21.15.143:6835/1300573627":"2024-01-16T18:15:46.367103+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21d40675e0 msgr2=0x7f21d4069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21d40675e0 0x7f21d4069aa0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f21fc0afc90 tx=0x7f21ec009040 comp rx=0 tx=0).stop 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 msgr2=0x7f21fc0c76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0c76d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f21e40097c0 tx=0x7f21e401acc0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 shutdown_connections 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f21d40675e0 0x7f21d4069aa0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21fc0cf740 0x7f21fc154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21fc0b0310 0x7f21fc0c76d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 --2- 172.21.15.143:0/923204611 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21fc06bfb0 0x7f21fc1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 >> 172.21.15.143:0/923204611 conn(0x7f21fc0d0ed0 msgr2=0x7f21fc0d12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 shutdown_connections 2024-01-15T18:15:51.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.442+0000 7f2203800640 1 -- 172.21.15.143:0/923204611 wait complete. 2024-01-15T18:15:51.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:15:51.462 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:15:51.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.602+0000 7fdd0ea00640 1 Processor -- start 2024-01-15T18:15:51.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.602+0000 7fdd0ea00640 1 -- start start 2024-01-15T18:15:51.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.602+0000 7fdd0ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd0806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd0ea00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd080cffd0 con 0x7fdd080cbf50 2024-01-15T18:15:51.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd0ea00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd080d02b0 con 0x7fdd081607c0 2024-01-15T18:15:51.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd0ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080d1ce0 con 0x7fdd080cfb30 2024-01-15T18:15:51.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd0ca00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdd080cbf50 0x7fdd080cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33424/0 (socket says 172.21.15.143:33424) 2024-01-15T18:15:51.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd0ca00640 1 -- 172.21.15.143:0/707535893 learned_addr learned my addr 172.21.15.143:0/707535893 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:51.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd07400640 1 --2- 172.21.15.143:0/707535893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd0806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456574267 0 0) 0x7fdd080cffd0 con 0x7fdd080cbf50 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce8003530 con 0x7fdd080cbf50 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121197719 0 0) 0x7fdd080d02b0 con 0x7fdd081607c0 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd080cffd0 con 0x7fdd081607c0 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3732122926 0 0) 0x7fdce8003530 con 0x7fdd080cbf50 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd07400640 1 -- 172.21.15.143:0/707535893 >> v1:172.21.15.204:6789/0 conn(0x7fdd081607c0 legacy=0x7fdd0806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd07400640 1 -- 172.21.15.143:0/707535893 >> v1:172.21.15.143:6789/0 conn(0x7fdd080cbf50 legacy=0x7fdd080cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd07400640 1 -- 172.21.15.143:0/707535893 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080cb470 con 0x7fdd080cfb30 2024-01-15T18:15:51.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.606+0000 7fdd07400640 1 --2- 172.21.15.143:0/707535893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd0806bec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00097f0 tx=0x7fdcf001ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acdfb32af15b89f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00001a00 con 0x7fdd080cfb30 2024-01-15T18:15:51.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf00a8650 con 0x7fdd080cfb30 2024-01-15T18:15:51.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd06a00640 1 -- 172.21.15.143:0/707535893 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00a89e0 con 0x7fdd080cfb30 2024-01-15T18:15:51.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- 172.21.15.143:0/707535893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 msgr2=0x7fdd0806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/707535893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd0806bec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00097f0 tx=0x7fdcf001ebd0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- 172.21.15.143:0/707535893 shutdown_connections 2024-01-15T18:15:51.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/707535893 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd0806bec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- 172.21.15.143:0/707535893 >> 172.21.15.143:0/707535893 conn(0x7fdd080d0e20 msgr2=0x7fdd080d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:51.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- 172.21.15.143:0/707535893 shutdown_connections 2024-01-15T18:15:51.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- 172.21.15.143:0/707535893 wait complete. 2024-01-15T18:15:51.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 Processor -- start 2024-01-15T18:15:51.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- start start 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd080cbf50 0x7fdd080c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd081f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 0x7fdd080c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0814fe10 con 0x7fdd081607c0 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080c9870 con 0x7fdd080cbf50 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.610+0000 7fdd0ea00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080c8bf0 con 0x7fdd080cfb30 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 0x7fdd080c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd07e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd080cbf50 0x7fdd080c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 0x7fdd080c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36864/0 (socket says 172.21.15.143:36864) 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 -- 172.21.15.143:0/1904729380 learned_addr learned my addr 172.21.15.143:0/1904729380 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:51.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd07400640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd081f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 -- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 msgr2=0x7fdd081f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd081f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 -- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd080cbf50 msgr2=0x7fdd080c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd080cbf50 0x7fdd080c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 -- 172.21.15.143:0/1904729380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf0009420 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ca00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 0x7fdd080c81a0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdd00001f50 tx=0x7fdd0000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00012400 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd00012580 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd080c8ed0 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00012930 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd080c9490 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0806d4b0 con 0x7fdd081607c0 2024-01-15T18:15:51.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.614+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fdd00012a90 con 0x7fdd081607c0 2024-01-15T18:15:51.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.618+0000 7fdcffe00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdce00675e0 0x7fdce0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.618+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fdd00083520 con 0x7fdd081607c0 2024-01-15T18:15:51.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.618+0000 7fdd07e00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdce00675e0 0x7fdce0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.618+0000 7fdd07e00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdce00675e0 0x7fdce0069aa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80011f0 tx=0x7fdcf8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.622+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdd00052260 con 0x7fdd081607c0 2024-01-15T18:15:51.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.770+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd080c99b0 con 0x7fdd081607c0 2024-01-15T18:15:51.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.770+0000 7fdcffe00640 1 -- 172.21.15.143:0/1904729380 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd00055880 con 0x7fdd081607c0 2024-01-15T18:15:51.769 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:15:51.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdce00675e0 msgr2=0x7fdce0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdce00675e0 0x7fdce0069aa0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80011f0 tx=0x7fdcf8009040 comp rx=0 tx=0).stop 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 msgr2=0x7fdd080c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 0x7fdd080c81a0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fdd00001f50 tx=0x7fdd0000a430 comp rx=0 tx=0).stop 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 shutdown_connections 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdce00675e0 0x7fdce0069aa0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd081607c0 0x7fdd080c81a0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdd080cfb30 0x7fdd081f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 --2- 172.21.15.143:0/1904729380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd080cbf50 0x7fdd080c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 >> 172.21.15.143:0/1904729380 conn(0x7fdd080d0e20 msgr2=0x7fdd0806c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 shutdown_connections 2024-01-15T18:15:51.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.774+0000 7fdd0ea00640 1 -- 172.21.15.143:0/1904729380 wait complete. 2024-01-15T18:15:51.791 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:15:51.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 Processor -- start 2024-01-15T18:15:51.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 -- start start 2024-01-15T18:15:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91141607c0 0x7f911406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f91140cbf50 0x7f91140cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91140cffd0 con 0x7f91140cfb30 2024-01-15T18:15:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140d1ce0 con 0x7f91141607c0 2024-01-15T18:15:51.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f911ac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140d02b0 con 0x7f91140cbf50 2024-01-15T18:15:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91141607c0 0x7f911406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f91141607c0 0x7f911406b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36870/0 (socket says 172.21.15.143:36870) 2024-01-15T18:15:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 -- 172.21.15.143:0/1953566037 learned_addr learned my addr 172.21.15.143:0/1953566037 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113400640 1 --1- 172.21.15.143:0/1953566037 >> v1:172.21.15.143:6789/0 conn(0x7f91140cfb30 0x7f911406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33426/0 (socket says 172.21.15.143:33426) 2024-01-15T18:15:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9118c00640 1 --2- 172.21.15.143:0/1953566037 >> v2:172.21.15.204:3300/0 conn(0x7f91140cbf50 0x7f91140cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 -- 172.21.15.143:0/1953566037 >> v2:172.21.15.204:3300/0 conn(0x7f91140cbf50 msgr2=0x7f91140cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 --2- 172.21.15.143:0/1953566037 >> v2:172.21.15.204:3300/0 conn(0x7f91140cbf50 0x7f91140cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 -- 172.21.15.143:0/1953566037 >> v1:172.21.15.143:6789/0 conn(0x7f91140cfb30 legacy=0x7f911406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9112a00640 1 -- 172.21.15.143:0/1953566037 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029259388 0 0) 0x7f91140cffd0 con 0x7f91140cfb30 2024-01-15T18:15:51.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.934+0000 7f9113e00640 1 -- 172.21.15.143:0/1953566037 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140cb470 con 0x7f91141607c0 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f9113e00640 1 --2- 172.21.15.143:0/1953566037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91141607c0 0x7f911406b7b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f91140cef70 tx=0x7f910401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16feb44ffbe5685d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f9112a00640 1 -- 172.21.15.143:0/1953566037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040a89f0 con 0x7f91141607c0 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f9112a00640 1 -- 172.21.15.143:0/1953566037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91040a8b50 con 0x7f91141607c0 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f9112a00640 1 -- 172.21.15.143:0/1953566037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040b0440 con 0x7f91141607c0 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 -- 172.21.15.143:0/1953566037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91141607c0 msgr2=0x7f911406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 --2- 172.21.15.143:0/1953566037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91141607c0 0x7f911406b7b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f91140cef70 tx=0x7f910401ecf0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 -- 172.21.15.143:0/1953566037 shutdown_connections 2024-01-15T18:15:51.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 --2- 172.21.15.143:0/1953566037 >> v2:172.21.15.204:3300/0 conn(0x7f91140cbf50 0x7f91140cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 --2- 172.21.15.143:0/1953566037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91141607c0 0x7f911406b7b0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 -- 172.21.15.143:0/1953566037 >> 172.21.15.143:0/1953566037 conn(0x7f91140d0e20 msgr2=0x7f91140d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:51.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 -- 172.21.15.143:0/1953566037 shutdown_connections 2024-01-15T18:15:51.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.938+0000 7f911ac00640 1 -- 172.21.15.143:0/1953566037 wait complete. 2024-01-15T18:15:51.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 Processor -- start 2024-01-15T18:15:51.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- start start 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91140cbf50 0x7f911415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 0x7f911409b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91141607c0 0x7f911415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91141f62c0 con 0x7f91140cfb30 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911409c200 con 0x7f91140cbf50 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91141f6510 con 0x7f91141607c0 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 0x7f911409b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 0x7f911409b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36876/0 (socket says 172.21.15.143:36876) 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 -- 172.21.15.143:0/659400291 learned_addr learned my addr 172.21.15.143:0/659400291 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113e00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91140cbf50 0x7f911415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9118c00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91141607c0 0x7f911415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 -- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91141607c0 msgr2=0x7f911415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91141607c0 0x7f911415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 -- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91140cbf50 msgr2=0x7f911415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91140cbf50 0x7f911415a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 -- 172.21.15.143:0/659400291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040094d0 con 0x7f91140cfb30 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f9113400640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 0x7f911409b890 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0011f0 tx=0x7f90fc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0074e0 con 0x7f91140cfb30 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc007640 con 0x7f91140cfb30 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91141f67f0 con 0x7f91140cfb30 2024-01-15T18:15:51.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc004a10 con 0x7f91140cfb30 2024-01-15T18:15:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911415ab10 con 0x7f91140cfb30 2024-01-15T18:15:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.942+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911406d4b0 con 0x7f91140cfb30 2024-01-15T18:15:51.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.946+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f90fc004b70 con 0x7f91140cfb30 2024-01-15T18:15:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.946+0000 7f910be00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90ec0675e0 0x7f90ec069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.946+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f90fc0823b0 con 0x7f91140cfb30 2024-01-15T18:15:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.946+0000 7f9113e00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90ec0675e0 0x7f90ec069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:51.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.946+0000 7f9113e00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90ec0675e0 0x7f90ec069aa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f91040b2000 tx=0x7f91040aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:51.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:51.950+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90fc0510f0 con 0x7f91140cfb30 2024-01-15T18:15:52.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.110+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f91141f6d10 con 0x7f91140cfb30 2024-01-15T18:15:52.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f910be00640 1 -- 172.21.15.143:0/659400291 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) v1 ==== 80+0+124 (secure 0 0 0) 0x7f91141f6d10 con 0x7f91140cfb30 2024-01-15T18:15:52.110 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/1300573627 2024-01-16T18:15:46.367103+0000 2024-01-15T18:15:52.110 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/1300573627 2024-01-16T18:15:46.367103+0000 2024-01-15T18:15:52.110 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90ec0675e0 msgr2=0x7f90ec069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90ec0675e0 0x7f90ec069aa0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f91040b2000 tx=0x7f91040aa070 comp rx=0 tx=0).stop 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 msgr2=0x7f911409b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 0x7f911409b890 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0011f0 tx=0x7f90fc00c9b0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 shutdown_connections 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f90ec0675e0 0x7f90ec069aa0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f91141607c0 0x7f911415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f91140cfb30 0x7f911409b890 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 --2- 172.21.15.143:0/659400291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f91140cbf50 0x7f911415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 >> 172.21.15.143:0/659400291 conn(0x7f91140d0e20 msgr2=0x7f91140cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 shutdown_connections 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.114+0000 7f911ac00640 1 -- 172.21.15.143:0/659400291 wait complete. 2024-01-15T18:15:52.133 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:15:52.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:52.133 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:15:52.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[33935]: fuse finished with error 0 and tester_r 0 2024-01-15T18:15:52.156 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:15:52.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:52.156 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:15:52.328 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:15:52.328 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:52.337 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:15:52.337 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:15:52.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:52.337 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:15:52.388 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:15:52.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:15:52.389 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:15:52.389 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:52.399 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:15:52.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:15:52.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:15:52.399 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:15:52.449 INFO:teuthology.orchestra.run.smithi204.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:15:52.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:15:52.450 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:15:52.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.586+0000 7fc80f400640 1 Processor -- start 2024-01-15T18:15:52.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80f400640 1 -- start start 2024-01-15T18:15:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808071bb0 0x7fc808072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc80815c5a0 0x7fc80815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8080ceb50 con 0x7fc808143780 2024-01-15T18:15:52.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80f400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc808072570 con 0x7fc80815c5a0 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80815c110 con 0x7fc808071bb0 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc80815c5a0 0x7fc80815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc80815c5a0 0x7fc80815ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36888/0 (socket says 172.21.15.143:36888) 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 -- 172.21.15.143:0/849010855 learned_addr learned my addr 172.21.15.143:0/849010855 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc807e00640 1 --2- 172.21.15.143:0/849010855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808071bb0 0x7fc808072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 -- 172.21.15.143:0/849010855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808071bb0 msgr2=0x7fc808072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 --2- 172.21.15.143:0/849010855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808071bb0 0x7fc808072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 -- 172.21.15.143:0/849010855 >> v1:172.21.15.204:6789/0 conn(0x7fc808143780 legacy=0x7fc8080cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.590+0000 7fc80d400640 1 -- 172.21.15.143:0/849010855 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80815b7e0 con 0x7fc80815c5a0 2024-01-15T18:15:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80d400640 1 --2- 172.21.15.143:0/849010855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc80815ca30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc800009e50 tx=0x7fc80001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fca61aac0af2d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc807400640 1 -- 172.21.15.143:0/849010855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000a58e0 con 0x7fc80815c5a0 2024-01-15T18:15:52.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc807400640 1 -- 172.21.15.143:0/849010855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000a5a40 con 0x7fc80815c5a0 2024-01-15T18:15:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc807400640 1 -- 172.21.15.143:0/849010855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000a5dd0 con 0x7fc80815c5a0 2024-01-15T18:15:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 -- 172.21.15.143:0/849010855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 msgr2=0x7fc80815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 --2- 172.21.15.143:0/849010855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc80815ca30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc800009e50 tx=0x7fc80001ca10 comp rx=0 tx=0).stop 2024-01-15T18:15:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 -- 172.21.15.143:0/849010855 shutdown_connections 2024-01-15T18:15:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 --2- 172.21.15.143:0/849010855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc80815ca30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 --2- 172.21.15.143:0/849010855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808071bb0 0x7fc808072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 -- 172.21.15.143:0/849010855 >> 172.21.15.143:0/849010855 conn(0x7fc80806c3f0 msgr2=0x7fc80806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 -- 172.21.15.143:0/849010855 shutdown_connections 2024-01-15T18:15:52.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.594+0000 7fc80f400640 1 -- 172.21.15.143:0/849010855 wait complete. 2024-01-15T18:15:52.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 Processor -- start 2024-01-15T18:15:52.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- start start 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc808071bb0 0x7fc808141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808143780 0x7fc808158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc808136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808141050 con 0x7fc80815c5a0 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808142d50 con 0x7fc808071bb0 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc808141260 con 0x7fc808143780 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc808136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc807e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808143780 0x7fc808158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc808071bb0 0x7fc808141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc808136150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36890/0 (socket says 172.21.15.143:36890) 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 -- 172.21.15.143:0/3779109986 learned_addr learned my addr 172.21.15.143:0/3779109986 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 -- 172.21.15.143:0/3779109986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808143780 msgr2=0x7fc808158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808143780 0x7fc808158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 -- 172.21.15.143:0/3779109986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc808071bb0 msgr2=0x7fc808141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc808071bb0 0x7fc808141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 -- 172.21.15.143:0/3779109986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc800009b30 con 0x7fc80815c5a0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80d400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc808136150 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc80001a640 tx=0x7fc8000197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000b8640 con 0x7fc80815c5a0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000b87a0 con 0x7fc80815c5a0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc808141540 con 0x7fc80815c5a0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000b8b30 con 0x7fc80815c5a0 2024-01-15T18:15:52.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8081590c0 con 0x7fc80815c5a0 2024-01-15T18:15:52.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.598+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808067500 con 0x7fc80815c5a0 2024-01-15T18:15:52.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.602+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fc8000b8c90 con 0x7fc80815c5a0 2024-01-15T18:15:52.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.602+0000 7fc804c00640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7e00675e0 0x7fc7e0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.602+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fc80011e4a0 con 0x7fc80815c5a0 2024-01-15T18:15:52.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.602+0000 7fc80ca00640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7e00675e0 0x7fc7e0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.602+0000 7fc80ca00640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7e00675e0 0x7fc7e0069aa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8001810 tx=0x7fc7f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.606+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc8000ed160 con 0x7fc80815c5a0 2024-01-15T18:15:52.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8081419f0 con 0x7fc80815c5a0 2024-01-15T18:15:52.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc804c00640 1 -- 172.21.15.143:0/3779109986 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fc8000f0780 con 0x7fc80815c5a0 2024-01-15T18:15:52.783 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:52.783 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3237385557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3237385557},{"type":"v1","addr":"172.21.15.204:6834","nonce":3237385557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/904040921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":904040921},{"type":"v1","addr":"172.21.15.204:6835","nonce":904040921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/64114877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":64114877},{"type":"v1","addr":"172.21.15.143:6836","nonce":64114877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/639432785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":639432785},{"type":"v1","addr":"172.21.15.143:6837","nonce":639432785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2024-01-15T18:15:52.783 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 47 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7e00675e0 msgr2=0x7fc7e0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7e00675e0 0x7fc7e0069aa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8001810 tx=0x7fc7f8009040 comp rx=0 tx=0).stop 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 msgr2=0x7fc808136150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc808136150 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc80001a640 tx=0x7fc8000197b0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 shutdown_connections 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7e00675e0 0x7fc7e0069aa0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc80815c5a0 0x7fc808136150 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc808143780 0x7fc808158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 --2- 172.21.15.143:0/3779109986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc808071bb0 0x7fc808141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 >> 172.21.15.143:0/3779109986 conn(0x7fc80806c3f0 msgr2=0x7fc8080acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 shutdown_connections 2024-01-15T18:15:52.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.786+0000 7fc80f400640 1 -- 172.21.15.143:0/3779109986 wait complete. 2024-01-15T18:15:52.805 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:15:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.946+0000 7f2091800640 1 Processor -- start 2024-01-15T18:15:52.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.946+0000 7f2091800640 1 -- start start 2024-01-15T18:15:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f2091800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f208c159b70 0x7f208c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f2091800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f2091800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c0b2a20 con 0x7f208c157300 2024-01-15T18:15:52.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f2091800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c0b35a0 con 0x7f208c159b70 2024-01-15T18:15:52.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f2091800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c159fd0 con 0x7f208c0b48f0 2024-01-15T18:15:52.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f208c159b70 0x7f208c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f208c159b70 0x7f208c0b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:35446/0 (socket says 172.21.15.143:35446) 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 -- 172.21.15.143:0/2072687881 learned_addr learned my addr 172.21.15.143:0/2072687881 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208aa00640 1 --1- 172.21.15.143:0/2072687881 >> v1:172.21.15.204:6789/0 conn(0x7f208c157300 0x7f208c0b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:34812/0 (socket says 172.21.15.143:34812) 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 -- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 msgr2=0x7f208c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 --2- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 -- 172.21.15.143:0/2072687881 >> v1:172.21.15.204:6789/0 conn(0x7f208c157300 legacy=0x7f208c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208a000640 1 -- 172.21.15.143:0/2072687881 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c0b2d00 con 0x7f208c159b70 2024-01-15T18:15:52.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.950+0000 7f208b400640 1 --2- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:52.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f208a000640 1 --2- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c159b70 0x7f208c0b43b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2074009870 tx=0x7f207401c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21806e3367849967 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2089600640 1 -- 172.21.15.143:0/2072687881 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20740a7070 con 0x7f208c159b70 2024-01-15T18:15:52.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2089600640 1 -- 172.21.15.143:0/2072687881 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20740ab430 con 0x7f208c159b70 2024-01-15T18:15:52.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2089600640 1 -- 172.21.15.143:0/2072687881 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20740ab7c0 con 0x7f208c159b70 2024-01-15T18:15:52.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 -- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c159b70 msgr2=0x7f208c0b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 --2- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c159b70 0x7f208c0b43b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2074009870 tx=0x7f207401c540 comp rx=0 tx=0).stop 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 -- 172.21.15.143:0/2072687881 shutdown_connections 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 --2- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 --2- 172.21.15.143:0/2072687881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c159b70 0x7f208c0b43b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 -- 172.21.15.143:0/2072687881 >> 172.21.15.143:0/2072687881 conn(0x7f208c06c3f0 msgr2=0x7f208c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 -- 172.21.15.143:0/2072687881 shutdown_connections 2024-01-15T18:15:52.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.954+0000 7f2091800640 1 -- 172.21.15.143:0/2072687881 wait complete. 2024-01-15T18:15:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 Processor -- start 2024-01-15T18:15:52.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- start start 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c157300 0x7f208c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 0x7f208c156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0ade40 con 0x7f208c159b70 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c14c090 con 0x7f208c157300 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0ad1c0 con 0x7f208c0b48f0 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 0x7f208c156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 0x7f208c156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36902/0 (socket says 172.21.15.143:36902) 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 -- 172.21.15.143:0/830097751 learned_addr learned my addr 172.21.15.143:0/830097751 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208a000640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c157300 0x7f208c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208aa00640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c14bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 -- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 msgr2=0x7f208c14bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c14bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 -- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c157300 msgr2=0x7f208c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c157300 0x7f208c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 -- 172.21.15.143:0/830097751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208400ab80 con 0x7f208c159b70 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f208b400640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 0x7f208c156300 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2084009e80 tx=0x7f20840075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084018070 con 0x7f208c159b70 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2084010040 con 0x7f208c159b70 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084003ba0 con 0x7f208c159b70 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20740094d0 con 0x7f208c159b70 2024-01-15T18:15:52.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c0ad780 con 0x7f208c159b70 2024-01-15T18:15:52.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.958+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c0adf80 con 0x7f208c159b70 2024-01-15T18:15:52.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.962+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f208401d410 con 0x7f208c159b70 2024-01-15T18:15:52.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.962+0000 7f2082a00640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2064067610 0x7f2064069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.962+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f2084083340 con 0x7f208c159b70 2024-01-15T18:15:52.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.962+0000 7f208aa00640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2064067610 0x7f2064069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.962+0000 7f208aa00640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2064067610 0x7f2064069ad0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f207c000f70 tx=0x7f207c007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:52.966+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f208c0adf80 con 0x7f208c159b70 2024-01-15T18:15:53.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.114+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f208c0adf80 con 0x7f208c159b70 2024-01-15T18:15:53.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.118+0000 7f2082a00640 1 -- 172.21.15.143:0/830097751 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v28) v1 ==== 120+0+0 (secure 0 0 0) 0x7f208c0adf80 con 0x7f208c159b70 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2064067610 msgr2=0x7f2064069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2064067610 0x7f2064069ad0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f207c000f70 tx=0x7f207c007c90 comp rx=0 tx=0).stop 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 msgr2=0x7f208c156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 0x7f208c156300 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2084009e80 tx=0x7f20840075c0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 shutdown_connections 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2064067610 0x7f2064069ad0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f208c159b70 0x7f208c156300 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f208c157300 0x7f208c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 --2- 172.21.15.143:0/830097751 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f208c0b48f0 0x7f208c14bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 >> 172.21.15.143:0/830097751 conn(0x7f208c06c3f0 msgr2=0x7f208c06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:53.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 shutdown_connections 2024-01-15T18:15:53.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.122+0000 7f2091800640 1 -- 172.21.15.143:0/830097751 wait complete. 2024-01-15T18:15:53.140 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:15:53.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 Processor -- start 2024-01-15T18:15:53.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 -- start start 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0a380cf400 0x7f0a380cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 0x7f0a380b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3815b5b0 con 0x7f0a380cda70 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a380cba70 con 0x7f0a380cf400 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a40600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380cbd10 con 0x7f0a380ce390 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0a380cda70 0x7f0a380cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33442/0 (socket says 172.21.15.143:33442) 2024-01-15T18:15:53.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3dc00640 1 -- 172.21.15.143:0/468270300 learned_addr learned my addr 172.21.15.143:0/468270300 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3d200640 1 --2- 172.21.15.143:0/468270300 >> v2:172.21.15.204:3300/0 conn(0x7f0a380cf400 0x7f0a380cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006987086 0 0) 0x7f0a3815b5b0 con 0x7f0a380cda70 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3e600640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 0x7f0a380b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a18003530 con 0x7f0a380cda70 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3714632724 0 0) 0x7f0a18003530 con 0x7f0a380cda70 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 msgr2=0x7f0a380b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:53.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 0x7f0a380b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 >> v2:172.21.15.204:3300/0 conn(0x7f0a380cf400 msgr2=0x7f0a380cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 --2- 172.21.15.143:0/468270300 >> v2:172.21.15.204:3300/0 conn(0x7f0a380cf400 0x7f0a380cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380cbff0 con 0x7f0a380cda70 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f0a28001680 con 0x7f0a380cda70 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 >> v1:172.21.15.143:6789/0 conn(0x7f0a380cda70 legacy=0x7f0a380cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18003980 0x7f0a18003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 0x7f0a180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a18004d10 0x7f0a18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380cba70 con 0x7f0a18004d10 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380cbd10 con 0x7f0a18003980 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1800b990 con 0x7f0a380ce390 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3e600640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a18004d10 0x7f0a18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3d200640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 0x7f0a180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.282+0000 7f0a3dc00640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18003980 0x7f0a18003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 -- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 msgr2=0x7f0a180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ce390 0x7f0a180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 -- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18003980 msgr2=0x7f0a18003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18003980 0x7f0a18003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 -- 172.21.15.143:0/468270300 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18003530 con 0x7f0a18004d10 2024-01-15T18:15:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3dc00640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18003980 0x7f0a18003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:53.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a18004d10 0x7f0a18005190 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000e170 tx=0x7f0a3001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864dd89ca85b080b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300a5c50 con 0x7f0a18004d10 2024-01-15T18:15:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3000f040 con 0x7f0a18004d10 2024-01-15T18:15:53.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a37e00640 1 -- 172.21.15.143:0/468270300 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300ad440 con 0x7f0a18004d10 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a40600640 1 -- 172.21.15.143:0/468270300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a18004d10 msgr2=0x7f0a18005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a40600640 1 --2- 172.21.15.143:0/468270300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a18004d10 0x7f0a18005190 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000e170 tx=0x7f0a3001cc40 comp rx=0 tx=0).stop 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.286+0000 7f0a3e600640 1 -- 172.21.15.143:0/468270300 reap_dead start 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- 172.21.15.143:0/468270300 shutdown_connections 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- 172.21.15.143:0/468270300 >> 172.21.15.143:0/468270300 conn(0x7f0a380d0e20 msgr2=0x7f0a380d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- 172.21.15.143:0/468270300 shutdown_connections 2024-01-15T18:15:53.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- 172.21.15.143:0/468270300 wait complete. 2024-01-15T18:15:53.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 Processor -- start 2024-01-15T18:15:53.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- start start 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18004d10 0x7f0a381f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 0x7f0a380b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ca1b0 0x7f0a380ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380cab70 con 0x7f0a381496f0 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b4300 con 0x7f0a18004d10 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a40600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380b45e0 con 0x7f0a380ca1b0 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 0x7f0a380b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 0x7f0a380b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36920/0 (socket says 172.21.15.143:36920) 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 -- 172.21.15.143:0/59341915 learned_addr learned my addr 172.21.15.143:0/59341915 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3dc00640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18004d10 0x7f0a381f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3e600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ca1b0 0x7f0a380ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 -- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ca1b0 msgr2=0x7f0a380ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ca1b0 0x7f0a380ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 -- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18004d10 msgr2=0x7f0a381f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18004d10 0x7f0a381f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.290+0000 7f0a3d200640 1 -- 172.21.15.143:0/59341915 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380b48c0 con 0x7f0a381496f0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a3d200640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 0x7f0a380b3a20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0a20011aa0 tx=0x7f0a20011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000fb80 con 0x7f0a381496f0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2000fce0 con 0x7f0a381496f0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a20014680 con 0x7f0a381496f0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a380b4ba0 con 0x7f0a381496f0 2024-01-15T18:15:53.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a381f0930 con 0x7f0a381496f0 2024-01-15T18:15:53.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a38067500 con 0x7f0a381496f0 2024-01-15T18:15:53.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f0a200147e0 con 0x7f0a381496f0 2024-01-15T18:15:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a35600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a100675e0 0x7f0a10069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f0a200839f0 con 0x7f0a381496f0 2024-01-15T18:15:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.294+0000 7f0a3dc00640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a100675e0 0x7f0a10069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.298+0000 7f0a3dc00640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a100675e0 0x7f0a10069aa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0a28000c10 tx=0x7f0a28007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:53.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.298+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0a20052420 con 0x7f0a381496f0 2024-01-15T18:15:53.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.450+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0a380cf1e0 con 0x7f0a381496f0 2024-01-15T18:15:53.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.454+0000 7f0a35600640 1 -- 172.21.15.143:0/59341915 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0a20055a40 con 0x7f0a381496f0 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a100675e0 msgr2=0x7f0a10069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a100675e0 0x7f0a10069aa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0a28000c10 tx=0x7f0a28007040 comp rx=0 tx=0).stop 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 msgr2=0x7f0a380b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 0x7f0a380b3a20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f0a20011aa0 tx=0x7f0a20011f60 comp rx=0 tx=0).stop 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 shutdown_connections 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0a100675e0 0x7f0a10069aa0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0a380ca1b0 0x7f0a380ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0a381496f0 0x7f0a380b3a20 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 --2- 172.21.15.143:0/59341915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a18004d10 0x7f0a381f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 >> 172.21.15.143:0/59341915 conn(0x7f0a380d0e20 msgr2=0x7f0a3806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:53.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 shutdown_connections 2024-01-15T18:15:53.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.458+0000 7f0a40600640 1 -- 172.21.15.143:0/59341915 wait complete. 2024-01-15T18:15:53.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added' 2024-01-15T18:15:53.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4db600640 1 Processor -- start 2024-01-15T18:15:53.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4db600640 1 -- start start 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4db600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4d40ccf60 0x7fb4d4167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4db600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d406c5d0 con 0x7fb4d41607c0 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4db600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d40cffd0 con 0x7fb4d40cfb30 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4db600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d40d1ce0 con 0x7fb4d40ccf60 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d8c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb4d41607c0 0x7fb4d406b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33452/0 (socket says 172.21.15.143:33452) 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d8c00640 1 -- 172.21.15.143:0/1155424368 learned_addr learned my addr 172.21.15.143:0/1155424368 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:53.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d9600640 1 --2- 172.21.15.143:0/1155424368 >> v2:172.21.15.204:3300/0 conn(0x7fb4d40ccf60 0x7fb4d4167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d3400640 1 -- 172.21.15.143:0/1155424368 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368364550 0 0) 0x7fb4d406c5d0 con 0x7fb4d41607c0 2024-01-15T18:15:53.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d3400640 1 -- 172.21.15.143:0/1155424368 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b4003530 con 0x7fb4d41607c0 2024-01-15T18:15:53.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d9600640 1 -- 172.21.15.143:0/1155424368 >> v1:172.21.15.204:6789/0 conn(0x7fb4d40cfb30 legacy=0x7fb4d406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d9600640 1 -- 172.21.15.143:0/1155424368 >> v1:172.21.15.143:6789/0 conn(0x7fb4d41607c0 legacy=0x7fb4d406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.614+0000 7fb4d9600640 1 -- 172.21.15.143:0/1155424368 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d40d02b0 con 0x7fb4d40ccf60 2024-01-15T18:15:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4d9600640 1 --2- 172.21.15.143:0/1155424368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40ccf60 0x7fb4d4167fb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009d90 tx=0x7fb4cc01f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a96e9699110a8e36 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4d3400640 1 -- 172.21.15.143:0/1155424368 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0af600 con 0x7fb4d40ccf60 2024-01-15T18:15:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4d3400640 1 -- 172.21.15.143:0/1155424368 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0ab050 con 0x7fb4d40ccf60 2024-01-15T18:15:53.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4d3400640 1 -- 172.21.15.143:0/1155424368 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0af980 con 0x7fb4d40ccf60 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 -- 172.21.15.143:0/1155424368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40ccf60 msgr2=0x7fb4d4167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 --2- 172.21.15.143:0/1155424368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40ccf60 0x7fb4d4167fb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009d90 tx=0x7fb4cc01f130 comp rx=0 tx=0).stop 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 -- 172.21.15.143:0/1155424368 shutdown_connections 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 --2- 172.21.15.143:0/1155424368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40ccf60 0x7fb4d4167fb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 -- 172.21.15.143:0/1155424368 >> 172.21.15.143:0/1155424368 conn(0x7fb4d40d0e20 msgr2=0x7fb4d40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 -- 172.21.15.143:0/1155424368 shutdown_connections 2024-01-15T18:15:53.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.618+0000 7fb4db600640 1 -- 172.21.15.143:0/1155424368 wait complete. 2024-01-15T18:15:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 Processor -- start 2024-01-15T18:15:53.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 -- start start 2024-01-15T18:15:53.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40ccf60 0x7fb4d41f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40cfb30 0x7fb4d41f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 0x7fb4d4159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d41f3ab0 con 0x7fb4d41607c0 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d41f7020 con 0x7fb4d40cfb30 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d41f29a0 con 0x7fb4d40ccf60 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 0x7fb4d4159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 0x7fb4d4159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36930/0 (socket says 172.21.15.143:36930) 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 -- 172.21.15.143:0/2836827424 learned_addr learned my addr 172.21.15.143:0/2836827424 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d3e00640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40cfb30 0x7fb4d41f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d8c00640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40ccf60 0x7fb4d41f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 -- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40ccf60 msgr2=0x7fb4d41f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40ccf60 0x7fb4d41f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 -- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40cfb30 msgr2=0x7fb4d41f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40cfb30 0x7fb4d41f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 -- 172.21.15.143:0/2836827424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc009a70 con 0x7fb4d41607c0 2024-01-15T18:15:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d8c00640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40ccf60 0x7fb4d41f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:53.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d9600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 0x7fb4d4159d30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0b2040 tx=0x7fb4cc01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc009450 con 0x7fb4d41607c0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0b9410 con 0x7fb4d41607c0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0b97c0 con 0x7fb4d41607c0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d41f64a0 con 0x7fb4d41607c0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.622+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d41f6a60 con 0x7fb4d41607c0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.626+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d406d4b0 con 0x7fb4d41607c0 2024-01-15T18:15:53.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.626+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fb4cc0b6050 con 0x7fb4d41607c0 2024-01-15T18:15:53.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.626+0000 7fb4d0c00640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4ac0675e0 0x7fb4ac069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:53.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.626+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fb4cc1212b0 con 0x7fb4d41607c0 2024-01-15T18:15:53.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.626+0000 7fb4d8c00640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4ac0675e0 0x7fb4ac069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:53.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.626+0000 7fb4d8c00640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4ac0675e0 0x7fb4ac069aa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40011f0 tx=0x7fb4c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:53.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.630+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4cc0eff40 con 0x7fb4d41607c0 2024-01-15T18:15:53.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.834+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]} v 0) v1 -- 0x7fb4d4067500 con 0x7fb4d41607c0 2024-01-15T18:15:53.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4d0c00640 1 -- 172.21.15.143:0/2836827424 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]}]=0 v512) v1 ==== 166+0+0 (secure 0 0 0) 0x7fb4cc0b4070 con 0x7fb4d41607c0 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4ac0675e0 msgr2=0x7fb4ac069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4ac0675e0 0x7fb4ac069aa0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40011f0 tx=0x7fb4c4009040 comp rx=0 tx=0).stop 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 msgr2=0x7fb4d4159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 0x7fb4d4159d30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc0b2040 tx=0x7fb4cc01bed0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 shutdown_connections 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb4ac0675e0 0x7fb4ac069aa0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4d41607c0 0x7fb4d4159d30 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4d40cfb30 0x7fb4d41f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 --2- 172.21.15.143:0/2836827424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4d40ccf60 0x7fb4d41f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:53.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 >> 172.21.15.143:0/2836827424 conn(0x7fb4d40d0e20 msgr2=0x7fb4d40cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:53.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 shutdown_connections 2024-01-15T18:15:53.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:53.910+0000 7fb4db600640 1 -- 172.21.15.143:0/2836827424 wait complete. 2024-01-15T18:15:53.928 INFO:tasks.cephfs_test_runner:X = number of clones issued. ... ok 2024-01-15T18:15:53.928 INFO:tasks.cephfs_test_runner:Starting test: test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgress) Test that the command "ceph fs clone status" prints progress stats 2024-01-15T18:15:53.929 INFO:tasks.cephfs_test_runner:test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:15:53.929 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:15:54.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 Processor -- start 2024-01-15T18:15:54.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 -- start start 2024-01-15T18:15:54.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8954143780 0x7f89540cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f89540722c0 0x7f895415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540ceb50 con 0x7f89540ce520 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895415c110 con 0x7f89540722c0 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f895a600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895415b7e0 con 0x7f8954143780 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8952a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f89540ce520 0x7f8954071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33462/0 (socket says 172.21.15.143:33462) 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8952a00640 1 -- 172.21.15.143:0/1741040901 learned_addr learned my addr 172.21.15.143:0/1741040901 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8953e00640 1 --2- 172.21.15.143:0/1741040901 >> v2:172.21.15.143:3300/0 conn(0x7f89540722c0 0x7f895415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8953400640 1 --2- 172.21.15.143:0/1741040901 >> v2:172.21.15.204:3300/0 conn(0x7f8954143780 0x7f89540cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8952000640 1 -- 172.21.15.143:0/1741040901 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516116503 0 0) 0x7f893c001120 con 0x7f89540ce520 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8952000640 1 -- 172.21.15.143:0/1741040901 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8934003530 con 0x7f89540ce520 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8953e00640 1 -- 172.21.15.143:0/1741040901 >> v2:172.21.15.204:3300/0 conn(0x7f8954143780 msgr2=0x7f89540cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:54.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8953e00640 1 --2- 172.21.15.143:0/1741040901 >> v2:172.21.15.204:3300/0 conn(0x7f8954143780 0x7f89540cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8953e00640 1 -- 172.21.15.143:0/1741040901 >> v1:172.21.15.143:6789/0 conn(0x7f89540ce520 legacy=0x7f8954071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.070+0000 7f8953e00640 1 -- 172.21.15.143:0/1741040901 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895415bac0 con 0x7f89540722c0 2024-01-15T18:15:54.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f8953e00640 1 --2- 172.21.15.143:0/1741040901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f895415c5a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f895406be30 tx=0x7f894c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e98bbaa60388e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f8952000640 1 -- 172.21.15.143:0/1741040901 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0a58e0 con 0x7f89540722c0 2024-01-15T18:15:54.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f8952000640 1 -- 172.21.15.143:0/1741040901 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c0a5a40 con 0x7f89540722c0 2024-01-15T18:15:54.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f8952000640 1 -- 172.21.15.143:0/1741040901 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0a5dd0 con 0x7f89540722c0 2024-01-15T18:15:54.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 -- 172.21.15.143:0/1741040901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 msgr2=0x7f895415c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 --2- 172.21.15.143:0/1741040901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f895415c5a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f895406be30 tx=0x7f894c01c910 comp rx=0 tx=0).stop 2024-01-15T18:15:54.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 -- 172.21.15.143:0/1741040901 shutdown_connections 2024-01-15T18:15:54.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 --2- 172.21.15.143:0/1741040901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f895415c5a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 --2- 172.21.15.143:0/1741040901 >> v2:172.21.15.204:3300/0 conn(0x7f8954143780 0x7f89540cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 -- 172.21.15.143:0/1741040901 >> 172.21.15.143:0/1741040901 conn(0x7f895406c3f0 msgr2=0x7f895406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:54.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 -- 172.21.15.143:0/1741040901 shutdown_connections 2024-01-15T18:15:54.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.074+0000 7f895a600640 1 -- 172.21.15.143:0/1741040901 wait complete. 2024-01-15T18:15:54.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 Processor -- start 2024-01-15T18:15:54.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 -- start start 2024-01-15T18:15:54.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f8954150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89540ce520 0x7f895406d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8954143780 0x7f895406d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954152f20 con 0x7f89540722c0 2024-01-15T18:15:54.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954152310 con 0x7f89540ce520 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89541525f0 con 0x7f8954143780 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f8954150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f8954150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36940/0 (socket says 172.21.15.143:36940) 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 -- 172.21.15.143:0/3693009677 learned_addr learned my addr 172.21.15.143:0/3693009677 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8952a00640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89540ce520 0x7f895406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953e00640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8954143780 0x7f895406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 -- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8954143780 msgr2=0x7f895406d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8954143780 0x7f895406d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 -- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89540ce520 msgr2=0x7f895406d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89540ce520 0x7f895406d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 -- 172.21.15.143:0/3693009677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c009a70 con 0x7f89540722c0 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953e00640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8954143780 0x7f895406d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f8953400640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f8954150d00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f894400d5f0 tx=0x7f894400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440125f0 con 0x7f89540722c0 2024-01-15T18:15:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944012750 con 0x7f89540722c0 2024-01-15T18:15:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944012ae0 con 0x7f89540722c0 2024-01-15T18:15:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89541528d0 con 0x7f89540722c0 2024-01-15T18:15:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.078+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541f59e0 con 0x7f89540722c0 2024-01-15T18:15:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.082+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8954067500 con 0x7f89540722c0 2024-01-15T18:15:54.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.082+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f8944012c40 con 0x7f89540722c0 2024-01-15T18:15:54.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.082+0000 7f894b400640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f892c0675e0 0x7f892c069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.082+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f8944083540 con 0x7f89540722c0 2024-01-15T18:15:54.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.082+0000 7f8952a00640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f892c0675e0 0x7f892c069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.082+0000 7f8952a00640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f892c0675e0 0x7f892c069aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8954157820 tx=0x7f893c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.086+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8944051f40 con 0x7f89540722c0 2024-01-15T18:15:54.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89540b42d0 con 0x7f89540722c0 2024-01-15T18:15:54.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f894b400640 1 -- 172.21.15.143:0/3693009677 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13113 (secure 0 0 0) 0x7f8944055560 con 0x7f89540722c0 2024-01-15T18:15:54.235 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:54.235 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":78,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:15:49.381489+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/1300573627":"2024-01-16T18:15:46.367103+0000","172.21.15.143:6835/1300573627":"2024-01-16T18:15:46.367103+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:15:54.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f892c0675e0 msgr2=0x7f892c069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f892c0675e0 0x7f892c069aa0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8954157820 tx=0x7f893c009040 comp rx=0 tx=0).stop 2024-01-15T18:15:54.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 msgr2=0x7f8954150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f8954150d00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f894400d5f0 tx=0x7f894400dab0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 shutdown_connections 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f892c0675e0 0x7f892c069aa0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8954143780 0x7f895406d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89540ce520 0x7f895406d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 --2- 172.21.15.143:0/3693009677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89540722c0 0x7f8954150d00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.238+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 >> 172.21.15.143:0/3693009677 conn(0x7f895406c3f0 msgr2=0x7f8954071aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.242+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 shutdown_connections 2024-01-15T18:15:54.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.242+0000 7f895a600640 1 -- 172.21.15.143:0/3693009677 wait complete. 2024-01-15T18:15:54.257 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:15:54.258 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:15:54.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 Processor -- start 2024-01-15T18:15:54.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 -- start start 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d34143780 0x7f1d340cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340ce520 0x7f1d34071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d340ceb50 con 0x7f1d340720f0 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d34072580 con 0x7f1d34143780 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d3c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3415c110 con 0x7f1d340ce520 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d34143780 0x7f1d340cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d34143780 0x7f1d340cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36944/0 (socket says 172.21.15.143:36944) 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 -- 172.21.15.143:0/4249322752 learned_addr learned my addr 172.21.15.143:0/4249322752 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39000640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340ce520 0x7f1d34071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 -- 172.21.15.143:0/4249322752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340ce520 msgr2=0x7f1d34071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340ce520 0x7f1d34071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 -- 172.21.15.143:0/4249322752 >> v1:172.21.15.204:6789/0 conn(0x7f1d340720f0 legacy=0x7f1d3415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:54.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39a00640 1 -- 172.21.15.143:0/4249322752 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3415b7e0 con 0x7f1d34143780 2024-01-15T18:15:54.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.402+0000 7f1d39000640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340ce520 0x7f1d34071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d39a00640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d34143780 0x7f1d340cdfe0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009870 tx=0x7f1d2401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ea2755f1d2c734 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d33e00640 1 -- 172.21.15.143:0/4249322752 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d240ac630 con 0x7f1d34143780 2024-01-15T18:15:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d33e00640 1 -- 172.21.15.143:0/4249322752 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d240ac790 con 0x7f1d34143780 2024-01-15T18:15:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d33e00640 1 -- 172.21.15.143:0/4249322752 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d240acb20 con 0x7f1d34143780 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 -- 172.21.15.143:0/4249322752 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d34143780 msgr2=0x7f1d340cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d34143780 0x7f1d340cdfe0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009870 tx=0x7f1d2401c500 comp rx=0 tx=0).stop 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 -- 172.21.15.143:0/4249322752 shutdown_connections 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340ce520 0x7f1d34071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 --2- 172.21.15.143:0/4249322752 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d34143780 0x7f1d340cdfe0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 -- 172.21.15.143:0/4249322752 >> 172.21.15.143:0/4249322752 conn(0x7f1d3406c3f0 msgr2=0x7f1d3406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 -- 172.21.15.143:0/4249322752 shutdown_connections 2024-01-15T18:15:54.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.406+0000 7f1d3c400640 1 -- 172.21.15.143:0/4249322752 wait complete. 2024-01-15T18:15:54.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 Processor -- start 2024-01-15T18:15:54.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 -- start start 2024-01-15T18:15:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340720f0 0x7f1d34133830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 0x7f1d3406cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d34143780 0x7f1d3406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d34133d70 con 0x7f1d340ce520 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340ad960 con 0x7f1d34143780 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d340acde0 con 0x7f1d340720f0 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 0x7f1d3406cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 0x7f1d3406cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36950/0 (socket says 172.21.15.143:36950) 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 -- 172.21.15.143:0/243146666 learned_addr learned my addr 172.21.15.143:0/243146666 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3a400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d34143780 0x7f1d3406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39a00640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340720f0 0x7f1d34133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 -- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340720f0 msgr2=0x7f1d34133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340720f0 0x7f1d34133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 -- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d34143780 msgr2=0x7f1d3406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d34143780 0x7f1d3406d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 -- 172.21.15.143:0/243146666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d240094d0 con 0x7f1d340ce520 2024-01-15T18:15:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d39000640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 0x7f1d3406cf60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00c7c0 tx=0x7f1d1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c007a30 con 0x7f1d340ce520 2024-01-15T18:15:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c007b90 con 0x7f1d340ce520 2024-01-15T18:15:54.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00ce70 con 0x7f1d340ce520 2024-01-15T18:15:54.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d340ad0c0 con 0x7f1d340ce520 2024-01-15T18:15:54.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.410+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d340ad680 con 0x7f1d340ce520 2024-01-15T18:15:54.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.414+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d34067500 con 0x7f1d340ce520 2024-01-15T18:15:54.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.414+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f1d1c0113f0 con 0x7f1d340ce520 2024-01-15T18:15:54.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.414+0000 7f1d31600640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1d0c067610 0x7f1d0c069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.414+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f1d1c083230 con 0x7f1d340ce520 2024-01-15T18:15:54.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.414+0000 7f1d39a00640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1d0c067610 0x7f1d0c069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.414+0000 7f1d39a00640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1d0c067610 0x7f1d0c069ad0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ae000 tx=0x7f1d240a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.418+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1d1c051fd0 con 0x7f1d340ce520 2024-01-15T18:15:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.570+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d34140780 con 0x7f1d340ce520 2024-01-15T18:15:54.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d31600640 1 -- 172.21.15.143:0/243146666 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d1c0555f0 con 0x7f1d340ce520 2024-01-15T18:15:54.569 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:15:54.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1d0c067610 msgr2=0x7f1d0c069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1d0c067610 0x7f1d0c069ad0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ae000 tx=0x7f1d240a7000 comp rx=0 tx=0).stop 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 msgr2=0x7f1d3406cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 0x7f1d3406cf60 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00c7c0 tx=0x7f1d1c00cc80 comp rx=0 tx=0).stop 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 shutdown_connections 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1d0c067610 0x7f1d0c069ad0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1d34143780 0x7f1d3406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d340ce520 0x7f1d3406cf60 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 --2- 172.21.15.143:0/243146666 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1d340720f0 0x7f1d34133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 >> 172.21.15.143:0/243146666 conn(0x7f1d3406c3f0 msgr2=0x7f1d3413f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 shutdown_connections 2024-01-15T18:15:54.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.574+0000 7f1d3c400640 1 -- 172.21.15.143:0/243146666 wait complete. 2024-01-15T18:15:54.591 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed' 2024-01-15T18:15:54.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.730+0000 7f7534000640 1 Processor -- start 2024-01-15T18:15:54.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.730+0000 7f7534000640 1 -- start start 2024-01-15T18:15:54.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7534000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f752c0d0150 0x7f752c0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7534000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 0x7f752c0c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7534000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c0cefa0 con 0x7f752c0b3610 2024-01-15T18:15:54.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7534000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c0c9fa0 con 0x7f752c0d0150 2024-01-15T18:15:54.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7534000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0b3ef0 con 0x7f752c0b45d0 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7530c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f752c0d0150 0x7f752c0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 0x7f752c0c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 0x7f752c0c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:50360/0 (socket says 172.21.15.143:50360) 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 -- 172.21.15.143:0/2316596724 learned_addr learned my addr 172.21.15.143:0/2316596724 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7531600640 1 --1- 172.21.15.143:0/2316596724 >> v1:172.21.15.204:6789/0 conn(0x7f752c0b3610 0x7f752c0cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:34838/0 (socket says 172.21.15.143:34838) 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 -- 172.21.15.143:0/2316596724 >> v2:172.21.15.204:3300/0 conn(0x7f752c0d0150 msgr2=0x7f752c0c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 --2- 172.21.15.143:0/2316596724 >> v2:172.21.15.204:3300/0 conn(0x7f752c0d0150 0x7f752c0c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 -- 172.21.15.143:0/2316596724 >> v1:172.21.15.204:6789/0 conn(0x7f752c0b3610 legacy=0x7f752c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7532000640 1 -- 172.21.15.143:0/2316596724 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c0d0f90 con 0x7f752c0b45d0 2024-01-15T18:15:54.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.734+0000 7f7530c00640 1 --2- 172.21.15.143:0/2316596724 >> v2:172.21.15.204:3300/0 conn(0x7f752c0d0150 0x7f752c0c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:54.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7532000640 1 --2- 172.21.15.143:0/2316596724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 0x7f752c0c70c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e50 tx=0x7f752401f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a1509cbae2aba1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f752be00640 1 -- 172.21.15.143:0/2316596724 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240af400 con 0x7f752c0b45d0 2024-01-15T18:15:54.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f752be00640 1 -- 172.21.15.143:0/2316596724 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75240ab050 con 0x7f752c0b45d0 2024-01-15T18:15:54.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f752be00640 1 -- 172.21.15.143:0/2316596724 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240af780 con 0x7f752c0b45d0 2024-01-15T18:15:54.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 -- 172.21.15.143:0/2316596724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 msgr2=0x7f752c0c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 --2- 172.21.15.143:0/2316596724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 0x7f752c0c70c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e50 tx=0x7f752401f1f0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 -- 172.21.15.143:0/2316596724 shutdown_connections 2024-01-15T18:15:54.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 --2- 172.21.15.143:0/2316596724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0b45d0 0x7f752c0c70c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 --2- 172.21.15.143:0/2316596724 >> v2:172.21.15.204:3300/0 conn(0x7f752c0d0150 0x7f752c0c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 -- 172.21.15.143:0/2316596724 >> 172.21.15.143:0/2316596724 conn(0x7f752c0d3030 msgr2=0x7f752c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:54.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 -- 172.21.15.143:0/2316596724 shutdown_connections 2024-01-15T18:15:54.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 -- 172.21.15.143:0/2316596724 wait complete. 2024-01-15T18:15:54.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 Processor -- start 2024-01-15T18:15:54.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.738+0000 7f7534000640 1 -- start start 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f752c0b3610 0x7f752c0b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 0x7f752c157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0d0150 0x7f752c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0b8220 con 0x7f752c0b45d0 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0b8640 con 0x7f752c0b3610 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0b75a0 con 0x7f752c0d0150 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 0x7f752c157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 0x7f752c157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36962/0 (socket says 172.21.15.143:36962) 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 -- 172.21.15.143:0/2864463325 learned_addr learned my addr 172.21.15.143:0/2864463325 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7531600640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f752c0b3610 0x7f752c0b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7532000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0d0150 0x7f752c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 -- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0d0150 msgr2=0x7f752c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0d0150 0x7f752c1577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 -- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f752c0b3610 msgr2=0x7f752c0b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f752c0b3610 0x7f752c0b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 -- 172.21.15.143:0/2864463325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514009420 con 0x7f752c0b45d0 2024-01-15T18:15:54.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7530c00640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 0x7f752c157290 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f751400e980 tx=0x7f751400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751400ca00 con 0x7f752c0b45d0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751400cb60 con 0x7f752c0b45d0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7524009b30 con 0x7f752c0b45d0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514013470 con 0x7f752c0b45d0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c0b7b60 con 0x7f752c0b45d0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.742+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c067500 con 0x7f752c0b45d0 2024-01-15T18:15:54.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.746+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f75140135d0 con 0x7f752c0b45d0 2024-01-15T18:15:54.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.746+0000 7f7529600640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75040675e0 0x7f7504069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.746+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f7514083040 con 0x7f752c0b45d0 2024-01-15T18:15:54.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.746+0000 7f7531600640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75040675e0 0x7f7504069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.746+0000 7f7531600640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75040675e0 0x7f7504069aa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f751c000f70 tx=0x7f751c007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.750+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7514051d80 con 0x7f752c0b45d0 2024-01-15T18:15:54.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:54.954+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]} v 0) v1 -- 0x7f752c0cd610 con 0x7f752c0b45d0 2024-01-15T18:15:55.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.918+0000 7f7529600640 1 -- 172.21.15.143:0/2864463325 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]}]=0 v514) v1 ==== 148+0+0 (secure 0 0 0) 0x7f75140210d0 con 0x7f752c0b45d0 2024-01-15T18:15:55.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75040675e0 msgr2=0x7f7504069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:55.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75040675e0 0x7f7504069aa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f751c000f70 tx=0x7f751c007e50 comp rx=0 tx=0).stop 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 msgr2=0x7f752c157290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 0x7f752c157290 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f751400e980 tx=0x7f751400ee40 comp rx=0 tx=0).stop 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 shutdown_connections 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f75040675e0 0x7f7504069aa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f752c0d0150 0x7f752c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c0b45d0 0x7f752c157290 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 --2- 172.21.15.143:0/2864463325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f752c0b3610 0x7f752c0b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 >> 172.21.15.143:0/2864463325 conn(0x7f752c0d3030 msgr2=0x7f752c0cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 shutdown_connections 2024-01-15T18:15:55.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:55.922+0000 7f7534000640 1 -- 172.21.15.143:0/2864463325 wait complete. 2024-01-15T18:15:55.943 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:15:56.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 Processor -- start 2024-01-15T18:15:56.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 -- start start 2024-01-15T18:15:56.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9fe40ce690 0x7f9fe40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40b1da0 0x7f9fe40b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9fe40b3bf0 0x7f9fe40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe415dba0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe4070bf0 con 0x7f9fe40ce690 2024-01-15T18:15:56.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40b2740 con 0x7f9fe40b1da0 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9fe40b3bf0 0x7f9fe40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9fe40b3bf0 0x7f9fe40b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36970/0 (socket says 172.21.15.143:36970) 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 -- 172.21.15.143:0/124062547 learned_addr learned my addr 172.21.15.143:0/124062547 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3400640 1 --2- 172.21.15.143:0/124062547 >> v2:172.21.15.204:3300/0 conn(0x7f9fe40ce690 0x7f9fe40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe2a00640 1 --2- 172.21.15.143:0/124062547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40b1da0 0x7f9fe40b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 -- 172.21.15.143:0/124062547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40b1da0 msgr2=0x7f9fe40b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 --2- 172.21.15.143:0/124062547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40b1da0 0x7f9fe40b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 -- 172.21.15.143:0/124062547 >> v2:172.21.15.204:3300/0 conn(0x7f9fe40ce690 msgr2=0x7f9fe40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 --2- 172.21.15.143:0/124062547 >> v2:172.21.15.204:3300/0 conn(0x7f9fe40ce690 0x7f9fe40ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3e00640 1 -- 172.21.15.143:0/124062547 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40b4920 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.090+0000 7f9fe3400640 1 --2- 172.21.15.143:0/124062547 >> v2:172.21.15.204:3300/0 conn(0x7f9fe40ce690 0x7f9fe40ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:56.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fe3e00640 1 --2- 172.21.15.143:0/124062547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe40b4080 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b1aa0 tx=0x7f9fdc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ded7a36ac2fd940 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:56.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fe2000640 1 -- 172.21.15.143:0/124062547 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0a5820 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fe2000640 1 -- 172.21.15.143:0/124062547 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc0a5980 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fe2000640 1 -- 172.21.15.143:0/124062547 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0a5d10 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 -- 172.21.15.143:0/124062547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 msgr2=0x7f9fe40b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 --2- 172.21.15.143:0/124062547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe40b4080 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b1aa0 tx=0x7f9fdc01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 -- 172.21.15.143:0/124062547 shutdown_connections 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 --2- 172.21.15.143:0/124062547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe40b4080 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 --2- 172.21.15.143:0/124062547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40b1da0 0x7f9fe40b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 --2- 172.21.15.143:0/124062547 >> v2:172.21.15.204:3300/0 conn(0x7f9fe40ce690 0x7f9fe40ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 -- 172.21.15.143:0/124062547 >> 172.21.15.143:0/124062547 conn(0x7f9fe406c3f0 msgr2=0x7f9fe406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 -- 172.21.15.143:0/124062547 shutdown_connections 2024-01-15T18:15:56.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.094+0000 7f9fea400640 1 -- 172.21.15.143:0/124062547 wait complete. 2024-01-15T18:15:56.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 Processor -- start 2024-01-15T18:15:56.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 -- start start 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe40b1da0 0x7f9fe4159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40ce690 0x7f9fe4142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe413fcf0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe413e2d0 con 0x7f9fe40b1da0 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe413e5b0 con 0x7f9fe40ce690 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe413d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36984/0 (socket says 172.21.15.143:36984) 2024-01-15T18:15:56.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 -- 172.21.15.143:0/2667643348 learned_addr learned my addr 172.21.15.143:0/2667643348 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe3400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe40b1da0 0x7f9fe4159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe3e00640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40ce690 0x7f9fe4142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 -- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40ce690 msgr2=0x7f9fe4142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40ce690 0x7f9fe4142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 -- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe40b1da0 msgr2=0x7f9fe4159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe40b1da0 0x7f9fe4159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 -- 172.21.15.143:0/2667643348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc009b30 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe3e00640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40ce690 0x7f9fe4142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe3400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe40b1da0 0x7f9fe4159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:56.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fe2a00640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe413d8f0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00d800 tx=0x7f9fcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:56.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0096a0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc009800 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc009b70 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe413e890 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.098+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe413f010 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.102+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe40acae0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.102+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f9fcc009cd0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.102+0000 7f9fdb400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fbc0675e0 0x7f9fbc069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.102+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f9fcc083600 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.102+0000 7f9fe3400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fbc0675e0 0x7f9fbc069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.102+0000 7f9fe3400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fbc0675e0 0x7f9fbc069aa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400b990 tx=0x7f9fd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:56.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.106+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9fcc0522d0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.254+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9fe4071e40 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.258+0000 7f9fdb400640 1 -- 172.21.15.143:0/2667643348 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9fcc0558f0 con 0x7f9fe40b3bf0 2024-01-15T18:15:56.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fbc0675e0 msgr2=0x7f9fbc069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fbc0675e0 0x7f9fbc069aa0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9fd400b990 tx=0x7f9fd4008040 comp rx=0 tx=0).stop 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 msgr2=0x7f9fe413d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe413d8f0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00d800 tx=0x7f9fcc00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 shutdown_connections 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9fbc0675e0 0x7f9fbc069aa0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fe40ce690 0x7f9fe4142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fe40b3bf0 0x7f9fe413d8f0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 --2- 172.21.15.143:0/2667643348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fe40b1da0 0x7f9fe4159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 >> 172.21.15.143:0/2667643348 conn(0x7f9fe406c3f0 msgr2=0x7f9fe415dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 shutdown_connections 2024-01-15T18:15:56.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.262+0000 7f9fea400640 1 -- 172.21.15.143:0/2667643348 wait complete. 2024-01-15T18:15:56.279 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:15:56.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:15:56.368 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:15:56.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:15:56.444 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:15:56.444 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:56.453 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:15:56.453 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:15:56.453 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:56.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:15:56.462 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:15:56.462 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:56.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:15:56.505 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:15:56.505 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:15:56.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:15:56.513 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:15:56.514 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:15:56.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe90da00640 1 Processor -- start 2024-01-15T18:15:56.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe90da00640 1 -- start start 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe90da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe908072410 0x7fe90815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe90da00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90815f310 con 0x7fe9080ce560 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe90da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90815f5f0 con 0x7fe908141660 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe90da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90815fbb0 con 0x7fe908072410 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe906000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe9080ce560 0x7fe908071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33466/0 (socket says 172.21.15.143:33466) 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe906000640 1 -- 172.21.15.143:0/2110283041 learned_addr learned my addr 172.21.15.143:0/2110283041 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.682+0000 7fe907400640 1 --2- 172.21.15.143:0/2110283041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe908072410 0x7fe90815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954415570 0 0) 0x7fe90815f310 con 0x7fe9080ce560 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e8003530 con 0x7fe9080ce560 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053486688 0 0) 0x7fe90815f5f0 con 0x7fe908141660 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90815f310 con 0x7fe908141660 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1172673265 0 0) 0x7fe8e8003530 con 0x7fe9080ce560 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe907400640 1 -- 172.21.15.143:0/2110283041 >> v1:172.21.15.204:6789/0 conn(0x7fe908141660 legacy=0x7fe908141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe907400640 1 -- 172.21.15.143:0/2110283041 >> v1:172.21.15.143:6789/0 conn(0x7fe9080ce560 legacy=0x7fe908071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe907400640 1 -- 172.21.15.143:0/2110283041 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90815f8d0 con 0x7fe908072410 2024-01-15T18:15:56.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe907400640 1 --2- 172.21.15.143:0/2110283041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe908072410 0x7fe90815ec80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe900009dc0 tx=0x7fe90001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ed989e83c8f66c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0001680 con 0x7fe908072410 2024-01-15T18:15:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9000a55e0 con 0x7fe908072410 2024-01-15T18:15:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.686+0000 7fe905600640 1 -- 172.21.15.143:0/2110283041 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000a5950 con 0x7fe908072410 2024-01-15T18:15:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- 172.21.15.143:0/2110283041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe908072410 msgr2=0x7fe90815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 --2- 172.21.15.143:0/2110283041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe908072410 0x7fe90815ec80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe900009dc0 tx=0x7fe90001c890 comp rx=0 tx=0).stop 2024-01-15T18:15:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- 172.21.15.143:0/2110283041 shutdown_connections 2024-01-15T18:15:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 --2- 172.21.15.143:0/2110283041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe908072410 0x7fe90815ec80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- 172.21.15.143:0/2110283041 >> 172.21.15.143:0/2110283041 conn(0x7fe90806c3f0 msgr2=0x7fe90806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- 172.21.15.143:0/2110283041 shutdown_connections 2024-01-15T18:15:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- 172.21.15.143:0/2110283041 wait complete. 2024-01-15T18:15:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 Processor -- start 2024-01-15T18:15:56.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- start start 2024-01-15T18:15:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 0x7fe90813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9080ce560 0x7fe908138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe908141660 0x7fe908157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe908139e80 con 0x7fe908072410 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9081398c0 con 0x7fe908141660 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe90da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe908139ba0 con 0x7fe9080ce560 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe906a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 0x7fe90813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe906a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 0x7fe90813f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36986/0 (socket says 172.21.15.143:36986) 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe906a00640 1 -- 172.21.15.143:0/1826871963 learned_addr learned my addr 172.21.15.143:0/1826871963 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe907400640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe908141660 0x7fe908157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.690+0000 7fe906000640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9080ce560 0x7fe908138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe906a00640 1 -- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9080ce560 msgr2=0x7fe908138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe906a00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9080ce560 0x7fe908138fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe906a00640 1 -- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe908141660 msgr2=0x7fe908157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe906a00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe908141660 0x7fe908157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe906a00640 1 -- 172.21.15.143:0/1826871963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe900009a70 con 0x7fe908072410 2024-01-15T18:15:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe906a00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 0x7fe90813f780 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800c430 tx=0x7fe8f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f80052d0 con 0x7fe908072410 2024-01-15T18:15:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8005450 con 0x7fe908072410 2024-01-15T18:15:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8003840 con 0x7fe908072410 2024-01-15T18:15:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9080cd150 con 0x7fe908072410 2024-01-15T18:15:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90813f260 con 0x7fe908072410 2024-01-15T18:15:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90813ede0 con 0x7fe908072410 2024-01-15T18:15:56.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fe8f80039a0 con 0x7fe908072410 2024-01-15T18:15:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe8fea00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8e00675e0 0x7fe8e0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.694+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe8f80823c0 con 0x7fe908072410 2024-01-15T18:15:56.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.698+0000 7fe906000640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8e00675e0 0x7fe8e0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:56.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.698+0000 7fe906000640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8e00675e0 0x7fe8e0069aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0001ec0 tx=0x7fe8f0001a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:56.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.698+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe90813ede0 con 0x7fe908072410 2024-01-15T18:15:56.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe90813ede0 con 0x7fe908072410 2024-01-15T18:15:56.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe8fea00640 1 -- 172.21.15.143:0/1826871963 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fe90813ede0 con 0x7fe908072410 2024-01-15T18:15:56.879 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:56.879 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6158,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/3237385557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3237385557},{"type":"v1","addr":"172.21.15.204:6834","nonce":3237385557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6868,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/904040921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":904040921},{"type":"v1","addr":"172.21.15.204:6835","nonce":904040921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6993,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/64114877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":64114877},{"type":"v1","addr":"172.21.15.143:6836","nonce":64114877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7905,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/639432785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":639432785},{"type":"v1","addr":"172.21.15.143:6837","nonce":639432785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2024-01-15T18:15:56.879 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 47 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8e00675e0 msgr2=0x7fe8e0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8e00675e0 0x7fe8e0069aa0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0001ec0 tx=0x7fe8f0001a60 comp rx=0 tx=0).stop 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 msgr2=0x7fe90813f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 0x7fe90813f780 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800c430 tx=0x7fe8f800c8f0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 shutdown_connections 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe8e00675e0 0x7fe8e0069aa0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe908141660 0x7fe908157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9080ce560 0x7fe908138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 --2- 172.21.15.143:0/1826871963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe908072410 0x7fe90813f780 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:56.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 >> 172.21.15.143:0/1826871963 conn(0x7fe90806c3f0 msgr2=0x7fe908141f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:56.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 shutdown_connections 2024-01-15T18:15:56.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:56.882+0000 7fe90da00640 1 -- 172.21.15.143:0/1826871963 wait complete. 2024-01-15T18:15:56.902 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:15:56.902 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:15:56.902 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:15:56.902 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:15:56.902 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:15:56.902 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:15:56.903 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:15:56.903 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:15:56.903 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:15:56.903 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:15:56.903 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:15:56.903 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:15:56.903 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:15:56.903 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:15:56.904 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:15:56.904 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:15:56.915 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:15:56.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:15:56.916 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:15:56.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:15:56.917 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:15:56.917 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:15:56.918 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:15:56.918 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:15:56.963 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:15:56.964 DEBUG:teuthology.parallel:result is None 2024-01-15T18:15:56.964 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:15:56.964 DEBUG:teuthology.parallel:result is None 2024-01-15T18:15:56.965 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:15:56.965 DEBUG:teuthology.parallel:result is None 2024-01-15T18:15:56.966 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:15:56.966 DEBUG:teuthology.parallel:result is None 2024-01-15T18:15:56.966 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:15:57.051 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:15:57.052 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:15:57.052 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:15:57.052 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:15:57.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 Processor -- start 2024-01-15T18:15:57.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 -- start start 2024-01-15T18:15:57.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b741607c0 0x7f4b7406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b740cbf50 0x7f4b740cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b740cffd0 con 0x7f4b740cfb30 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740d1ce0 con 0x7f4b740cbf50 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7c800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740d02b0 con 0x7f4b741607c0 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4b740cbf50 0x7f4b740cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b79400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4b740cfb30 0x7f4b7406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33490/0 (socket says 172.21.15.143:33490) 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b79400640 1 -- 172.21.15.143:0/2070759043 learned_addr learned my addr 172.21.15.143:0/2070759043 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b79e00640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b741607c0 0x7f4b7406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7a800640 1 -- 172.21.15.143:0/2070759043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b741607c0 msgr2=0x7f4b7406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7a800640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b741607c0 0x7f4b7406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7a800640 1 -- 172.21.15.143:0/2070759043 >> v1:172.21.15.143:6789/0 conn(0x7f4b740cfb30 legacy=0x7f4b7406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b78a00640 1 -- 172.21.15.143:0/2070759043 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909245104 0 0) 0x7f4b740cffd0 con 0x7f4b740cfb30 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b7a800640 1 -- 172.21.15.143:0/2070759043 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b740cb470 con 0x7f4b740cbf50 2024-01-15T18:15:57.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.114+0000 7f4b79e00640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b741607c0 0x7f4b7406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:57.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7a800640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b740cbf50 0x7f4b740cc390 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009d90 tx=0x7f4b6c01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43cc90d9474235de server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b78a00640 1 -- 172.21.15.143:0/2070759043 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0af600 con 0x7f4b740cbf50 2024-01-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b78a00640 1 -- 172.21.15.143:0/2070759043 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0ab050 con 0x7f4b740cbf50 2024-01-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b78a00640 1 -- 172.21.15.143:0/2070759043 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0af980 con 0x7f4b740cbf50 2024-01-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 -- 172.21.15.143:0/2070759043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b740cbf50 msgr2=0x7f4b740cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b740cbf50 0x7f4b740cc390 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c009d90 tx=0x7f4b6c01f130 comp rx=0 tx=0).stop 2024-01-15T18:15:57.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 -- 172.21.15.143:0/2070759043 shutdown_connections 2024-01-15T18:15:57.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b740cbf50 0x7f4b740cc390 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 --2- 172.21.15.143:0/2070759043 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b741607c0 0x7f4b7406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 -- 172.21.15.143:0/2070759043 >> 172.21.15.143:0/2070759043 conn(0x7f4b740d0e20 msgr2=0x7f4b740d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:57.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 -- 172.21.15.143:0/2070759043 shutdown_connections 2024-01-15T18:15:57.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.118+0000 7f4b7c800640 1 -- 172.21.15.143:0/2070759043 wait complete. 2024-01-15T18:15:57.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 Processor -- start 2024-01-15T18:15:57.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 -- start start 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b740cbf50 0x7f4b740c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b740cfb30 0x7f4b740c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 0x7f4b740c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740ae7d0 con 0x7f4b741607c0 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740adc00 con 0x7f4b740cbf50 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b740adea0 con 0x7f4b740cfb30 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 0x7f4b740c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 0x7f4b740c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37034/0 (socket says 172.21.15.143:37034) 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 -- 172.21.15.143:0/3452032072 learned_addr learned my addr 172.21.15.143:0/3452032072 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b79400640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b740cfb30 0x7f4b740c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b79e00640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b740cbf50 0x7f4b740c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 -- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b740cfb30 msgr2=0x7f4b740c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b740cfb30 0x7f4b740c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 -- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b740cbf50 msgr2=0x7f4b740c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b740cbf50 0x7f4b740c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 -- 172.21.15.143:0/3452032072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b64009420 con 0x7f4b741607c0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b79e00640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b740cbf50 0x7f4b740c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7a800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 0x7f4b740c8430 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0b2040 tx=0x7f4b6c01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c009450 con 0x7f4b741607c0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c0b9410 con 0x7f4b741607c0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0b97a0 con 0x7f4b741607c0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.122+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c009a70 con 0x7f4b741607c0 2024-01-15T18:15:57.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b740ae460 con 0x7f4b741607c0 2024-01-15T18:15:57.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7406d4b0 con 0x7f4b741607c0 2024-01-15T18:15:57.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f4b6c0b6050 con 0x7f4b741607c0 2024-01-15T18:15:57.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b72000640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4b4c0675e0 0x7f4b4c069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f4b6c121270 con 0x7f4b741607c0 2024-01-15T18:15:57.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b79e00640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4b4c0675e0 0x7f4b4c069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.126+0000 7f4b79e00640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4b4c0675e0 0x7f4b4c069aa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4b64011310 tx=0x7f4b64009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:57.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.130+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4b6c0f3a40 con 0x7f4b741607c0 2024-01-15T18:15:57.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:57.294+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4b74067500 con 0x7f4b741607c0 2024-01-15T18:15:58.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.278+0000 7f4b72000640 1 -- 172.21.15.143:0/3452032072 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4b6c0b4020 con 0x7f4b741607c0 2024-01-15T18:15:58.276 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:15:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4b4c0675e0 msgr2=0x7f4b4c069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4b4c0675e0 0x7f4b4c069aa0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4b64011310 tx=0x7f4b64009190 comp rx=0 tx=0).stop 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 msgr2=0x7f4b740c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 0x7f4b740c8430 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0b2040 tx=0x7f4b6c01bed0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 shutdown_connections 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4b4c0675e0 0x7f4b4c069aa0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b741607c0 0x7f4b740c8430 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b740cfb30 0x7f4b740c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 --2- 172.21.15.143:0/3452032072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b740cbf50 0x7f4b740c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 >> 172.21.15.143:0/3452032072 conn(0x7f4b740d0e20 msgr2=0x7f4b740ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 shutdown_connections 2024-01-15T18:15:58.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.282+0000 7f4b7c800640 1 -- 172.21.15.143:0/3452032072 wait complete. 2024-01-15T18:15:58.305 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:15:58.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 Processor -- start 2024-01-15T18:15:58.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 -- start start 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13f815b260 0x7f13f80cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80d19f0 0x7f13f80cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f80c4d90 con 0x7f13f80c5220 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f815b640 con 0x7f13f815b260 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13fdc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80cf800 con 0x7f13f80d19f0 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13f815b260 0x7f13f80cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f13f815b260 0x7f13f80cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33468/0 (socket says 172.21.15.143:33468) 2024-01-15T18:15:58.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 -- 172.21.15.143:0/3879828453 learned_addr learned my addr 172.21.15.143:0/3879828453 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6000640 1 --2- 172.21.15.143:0/3879828453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80d19f0 0x7f13f80cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 -- 172.21.15.143:0/3879828453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80d19f0 msgr2=0x7f13f80cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 --2- 172.21.15.143:0/3879828453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80d19f0 0x7f13f80cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 -- 172.21.15.143:0/3879828453 >> v1:172.21.15.204:6789/0 conn(0x7f13f80c5220 legacy=0x7f13f80c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.450+0000 7f13f6a00640 1 -- 172.21.15.143:0/3879828453 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80cfae0 con 0x7f13f815b260 2024-01-15T18:15:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13f6a00640 1 --2- 172.21.15.143:0/3879828453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f815b260 0x7f13f80cd810 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009870 tx=0x7f13e801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da75a0135713631 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13f5600640 1 -- 172.21.15.143:0/3879828453 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e80ab620 con 0x7f13f815b260 2024-01-15T18:15:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13f5600640 1 -- 172.21.15.143:0/3879828453 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e80ab780 con 0x7f13f815b260 2024-01-15T18:15:58.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13f5600640 1 -- 172.21.15.143:0/3879828453 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e80abb10 con 0x7f13f815b260 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 -- 172.21.15.143:0/3879828453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f815b260 msgr2=0x7f13f80cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 --2- 172.21.15.143:0/3879828453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f815b260 0x7f13f80cd810 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009870 tx=0x7f13e801c510 comp rx=0 tx=0).stop 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 -- 172.21.15.143:0/3879828453 shutdown_connections 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 --2- 172.21.15.143:0/3879828453 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80d19f0 0x7f13f80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 --2- 172.21.15.143:0/3879828453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f815b260 0x7f13f80cd810 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 -- 172.21.15.143:0/3879828453 >> 172.21.15.143:0/3879828453 conn(0x7f13f80d0ed0 msgr2=0x7f13f80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 -- 172.21.15.143:0/3879828453 shutdown_connections 2024-01-15T18:15:58.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.454+0000 7f13fdc00640 1 -- 172.21.15.143:0/3879828453 wait complete. 2024-01-15T18:15:58.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 Processor -- start 2024-01-15T18:15:58.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 -- start start 2024-01-15T18:15:58.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80c5220 0x7f13f8157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 0x7f13f80b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13f815b260 0x7f13f80b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8150d70 con 0x7f13f80d19f0 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8153000 con 0x7f13f815b260 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f8158220 con 0x7f13f80c5220 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 0x7f13f80b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 0x7f13f80b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33470/0 (socket says 172.21.15.143:33470) 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 -- 172.21.15.143:0/4249577636 learned_addr learned my addr 172.21.15.143:0/4249577636 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f7400640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13f815b260 0x7f13f80b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6a00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80c5220 0x7f13f8157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 -- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80c5220 msgr2=0x7f13f8157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80c5220 0x7f13f8157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 -- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13f815b260 msgr2=0x7f13f80b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13f815b260 0x7f13f80b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 -- 172.21.15.143:0/4249577636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80094d0 con 0x7f13f80d19f0 2024-01-15T18:15:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13f6000640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 0x7f13f80b2440 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d5f0 tx=0x7f13e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:58.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00125f0 con 0x7f13f80d19f0 2024-01-15T18:15:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e0012750 con 0x7f13f80d19f0 2024-01-15T18:15:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e0012ae0 con 0x7f13f80d19f0 2024-01-15T18:15:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f806c550 con 0x7f13f80d19f0 2024-01-15T18:15:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.458+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f806cb10 con 0x7f13f80d19f0 2024-01-15T18:15:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.462+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f8150eb0 con 0x7f13f80d19f0 2024-01-15T18:15:58.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.462+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f13e0012c40 con 0x7f13f80d19f0 2024-01-15T18:15:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.462+0000 7f13eea00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13d00675e0 0x7f13d0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.462+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f13e0083450 con 0x7f13f80d19f0 2024-01-15T18:15:58.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.462+0000 7f13f6a00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13d00675e0 0x7f13d0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.462+0000 7f13f6a00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13d00675e0 0x7f13d0069aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f13e80af000 tx=0x7f13e80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:58.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.466+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f13f8150eb0 con 0x7f13f80d19f0 2024-01-15T18:15:58.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.666+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f13f8150eb0 con 0x7f13f80d19f0 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.666+0000 7f13eea00640 1 -- 172.21.15.143:0/4249577636 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f13f8150eb0 con 0x7f13f80d19f0 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.665 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.666 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.667 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.668 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.669 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.670 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.671 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:15:58.672 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.673 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.674 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.675 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.676 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:15:58.677 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:15:58.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13d00675e0 msgr2=0x7f13d0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13d00675e0 0x7f13d0069aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f13e80af000 tx=0x7f13e80b6040 comp rx=0 tx=0).stop 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 msgr2=0x7f13f80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 0x7f13f80b2440 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f13e000d5f0 tx=0x7f13e000dab0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 shutdown_connections 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f13d00675e0 0x7f13d0069aa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f13f815b260 0x7f13f80b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f80d19f0 0x7f13f80b2440 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 --2- 172.21.15.143:0/4249577636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f13f80c5220 0x7f13f8157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 >> 172.21.15.143:0/4249577636 conn(0x7f13f80d0ed0 msgr2=0x7f13f80c89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 shutdown_connections 2024-01-15T18:15:58.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.670+0000 7f13fdc00640 1 -- 172.21.15.143:0/4249577636 wait complete. 2024-01-15T18:15:58.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:15:58.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 Processor -- start 2024-01-15T18:15:58.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 -- start start 2024-01-15T18:15:58.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabd8141660 0x7fabd80ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fabd815ec80 0x7fabd8141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8142470 0x7fabd8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd815fbb0 con 0x7fabd8141660 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd815f130 con 0x7fabd815ec80 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabdcc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815f310 con 0x7fabd8142470 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.826+0000 7fabd5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabd8141660 0x7fabd80ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabd8141660 0x7fabd80ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33478/0 (socket says 172.21.15.143:33478) 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 -- 172.21.15.143:0/3739355474 learned_addr learned my addr 172.21.15.143:0/3739355474 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd6800640 1 --2- 172.21.15.143:0/3739355474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8142470 0x7fabd8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5400640 1 --2- 172.21.15.143:0/3739355474 >> v2:172.21.15.204:3300/0 conn(0x7fabd815ec80 0x7fabd8141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 -- 172.21.15.143:0/3739355474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8142470 msgr2=0x7fabd8071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 --2- 172.21.15.143:0/3739355474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8142470 0x7fabd8071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 -- 172.21.15.143:0/3739355474 >> v2:172.21.15.204:3300/0 conn(0x7fabd815ec80 msgr2=0x7fabd8141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 --2- 172.21.15.143:0/3739355474 >> v2:172.21.15.204:3300/0 conn(0x7fabd815ec80 0x7fabd8141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 -- 172.21.15.143:0/3739355474 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd815f5f0 con 0x7fabd8141660 2024-01-15T18:15:58.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd5e00640 1 --2- 172.21.15.143:0/3739355474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8141660 0x7fabd80ce560 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fabd806f940 tx=0x7fabc801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ae5751b11697c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:58.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd4a00640 1 -- 172.21.15.143:0/3739355474 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80ac600 con 0x7fabd8141660 2024-01-15T18:15:58.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd4a00640 1 -- 172.21.15.143:0/3739355474 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc80a8050 con 0x7fabd8141660 2024-01-15T18:15:58.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.830+0000 7fabd4a00640 1 -- 172.21.15.143:0/3739355474 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80ac980 con 0x7fabd8141660 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- 172.21.15.143:0/3739355474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8141660 msgr2=0x7fabd80ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- 172.21.15.143:0/3739355474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8141660 0x7fabd80ce560 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fabd806f940 tx=0x7fabc801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- 172.21.15.143:0/3739355474 shutdown_connections 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- 172.21.15.143:0/3739355474 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8142470 0x7fabd8071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- 172.21.15.143:0/3739355474 >> v2:172.21.15.204:3300/0 conn(0x7fabd815ec80 0x7fabd8141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- 172.21.15.143:0/3739355474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8141660 0x7fabd80ce560 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- 172.21.15.143:0/3739355474 >> 172.21.15.143:0/3739355474 conn(0x7fabd806c3f0 msgr2=0x7fabd806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- 172.21.15.143:0/3739355474 shutdown_connections 2024-01-15T18:15:58.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- 172.21.15.143:0/3739355474 wait complete. 2024-01-15T18:15:58.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 Processor -- start 2024-01-15T18:15:58.831 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- start start 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8141660 0x7fabd813d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 0x7fabd815a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabd815ec80 0x7fabd815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80ac140 con 0x7fabd8142470 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd815bf10 con 0x7fabd815ec80 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabdcc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80ab850 con 0x7fabd8141660 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabd5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 0x7fabd815a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabd5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 0x7fabd815a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33494/0 (socket says 172.21.15.143:33494) 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabd5400640 1 -- 172.21.15.143:0/857815266 learned_addr learned my addr 172.21.15.143:0/857815266 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:58.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabd5e00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8141660 0x7fabd813d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.834+0000 7fabd6800640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabd815ec80 0x7fabd815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd5400640 1 -- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8141660 msgr2=0x7fabd813d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd5400640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8141660 0x7fabd813d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd5400640 1 -- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabd815ec80 msgr2=0x7fabd815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd5400640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabd815ec80 0x7fabd815b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd5400640 1 -- 172.21.15.143:0/857815266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80094d0 con 0x7fabd8142470 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd6800640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabd815ec80 0x7fabd815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabd5400640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 0x7fabd815a940 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fabc000d5f0 tx=0x7fabc000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc00125f0 con 0x7fabd8142470 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc0012750 con 0x7fabd8142470 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0012ae0 con 0x7fabd8142470 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd80cd150 con 0x7fabd8142470 2024-01-15T18:15:58.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd80abaf0 con 0x7fabd8142470 2024-01-15T18:15:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd8067500 con 0x7fabd8142470 2024-01-15T18:15:58.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.838+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fabc0012c40 con 0x7fabd8142470 2024-01-15T18:15:58.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.842+0000 7fabce000640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fabb00675e0 0x7fabb0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:58.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.842+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fabc0083450 con 0x7fabd8142470 2024-01-15T18:15:58.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.842+0000 7fabd5e00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fabb00675e0 0x7fabb0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:58.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.842+0000 7fabd5e00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fabb00675e0 0x7fabb0069aa0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fabc80ae000 tx=0x7fabc80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:58.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.846+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabc0051f00 con 0x7fabd8142470 2024-01-15T18:15:58.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.998+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabd806e2a0 con 0x7fabd8142470 2024-01-15T18:15:58.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.998+0000 7fabce000640 1 -- 172.21.15.143:0/857815266 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12982 (secure 0 0 0) 0x7fabc0007380 con 0x7fabd8142470 2024-01-15T18:15:58.995 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:58.995 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":83,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:15:58.279647+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:15:58.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.998+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fabb00675e0 msgr2=0x7fabb0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.998+0000 7fabdcc00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fabb00675e0 0x7fabb0069aa0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fabc80ae000 tx=0x7fabc80a7000 comp rx=0 tx=0).stop 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.998+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 msgr2=0x7fabd815a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:58.998+0000 7fabdcc00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 0x7fabd815a940 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fabc000d5f0 tx=0x7fabc000dab0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 shutdown_connections 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fabb00675e0 0x7fabb0069aa0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabd815ec80 0x7fabd815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabd8142470 0x7fabd815a940 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 --2- 172.21.15.143:0/857815266 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabd8141660 0x7fabd813d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 >> 172.21.15.143:0/857815266 conn(0x7fabd806c3f0 msgr2=0x7fabd80affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 shutdown_connections 2024-01-15T18:15:58.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.002+0000 7fabdcc00640 1 -- 172.21.15.143:0/857815266 wait complete. 2024-01-15T18:15:59.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:15:59.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:15:59.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.162+0000 7f30eb400640 1 Processor -- start 2024-01-15T18:15:59.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.162+0000 7f30eb400640 1 -- start start 2024-01-15T18:15:59.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30eb400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f30e406bfb0 0x7f30e406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30e40b0310 0x7f30e40cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30eb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40cc9c0 0x7f30e40cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30eb400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40cd3b0 con 0x7f30e40b0310 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30eb400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40cbff0 con 0x7f30e406bfb0 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30eb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40d0c50 con 0x7f30e40cc9c0 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30e40b0310 0x7f30e40cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f30e40b0310 0x7f30e40cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33508/0 (socket says 172.21.15.143:33508) 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 -- 172.21.15.143:0/3196291290 learned_addr learned my addr 172.21.15.143:0/3196291290 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e8a00640 1 --2- 172.21.15.143:0/3196291290 >> v2:172.21.15.204:3300/0 conn(0x7f30e406bfb0 0x7f30e406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e9400640 1 --2- 172.21.15.143:0/3196291290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40cc9c0 0x7f30e40cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 -- 172.21.15.143:0/3196291290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40cc9c0 msgr2=0x7f30e40cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 --2- 172.21.15.143:0/3196291290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40cc9c0 0x7f30e40cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 -- 172.21.15.143:0/3196291290 >> v2:172.21.15.204:3300/0 conn(0x7f30e406bfb0 msgr2=0x7f30e406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 --2- 172.21.15.143:0/3196291290 >> v2:172.21.15.204:3300/0 conn(0x7f30e406bfb0 0x7f30e406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.166+0000 7f30e3e00640 1 -- 172.21.15.143:0/3196291290 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e406c8d0 con 0x7f30e40b0310 2024-01-15T18:15:59.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30e3e00640 1 --2- 172.21.15.143:0/3196291290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e40b0310 0x7f30e40cc480 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f30e40716c0 tx=0x7f30cc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e1dae0f24d1d55 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30e3400640 1 -- 172.21.15.143:0/3196291290 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc0ab600 con 0x7f30e40b0310 2024-01-15T18:15:59.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30e3400640 1 -- 172.21.15.143:0/3196291290 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc0ab760 con 0x7f30e40b0310 2024-01-15T18:15:59.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30e3400640 1 -- 172.21.15.143:0/3196291290 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc0abaf0 con 0x7f30e40b0310 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 -- 172.21.15.143:0/3196291290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e40b0310 msgr2=0x7f30e40cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 --2- 172.21.15.143:0/3196291290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e40b0310 0x7f30e40cc480 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f30e40716c0 tx=0x7f30cc01c440 comp rx=0 tx=0).stop 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 -- 172.21.15.143:0/3196291290 shutdown_connections 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 --2- 172.21.15.143:0/3196291290 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40cc9c0 0x7f30e40cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 --2- 172.21.15.143:0/3196291290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e40b0310 0x7f30e40cc480 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 --2- 172.21.15.143:0/3196291290 >> v2:172.21.15.204:3300/0 conn(0x7f30e406bfb0 0x7f30e406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 -- 172.21.15.143:0/3196291290 >> 172.21.15.143:0/3196291290 conn(0x7f30e40d0ed0 msgr2=0x7f30e40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 -- 172.21.15.143:0/3196291290 shutdown_connections 2024-01-15T18:15:59.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 -- 172.21.15.143:0/3196291290 wait complete. 2024-01-15T18:15:59.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 Processor -- start 2024-01-15T18:15:59.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.170+0000 7f30eb400640 1 -- start start 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 0x7f30e4157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40b0310 0x7f30e40b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e40cc9c0 0x7f30e40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40c8e20 con 0x7f30e406bfb0 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41524a0 con 0x7f30e40cc9c0 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40b1ce0 con 0x7f30e40b0310 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 0x7f30e4157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 0x7f30e4157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33510/0 (socket says 172.21.15.143:33510) 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 -- 172.21.15.143:0/1547028176 learned_addr learned my addr 172.21.15.143:0/1547028176 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e9400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e40cc9c0 0x7f30e40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e3e00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40b0310 0x7f30e40b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 -- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40b0310 msgr2=0x7f30e40b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40b0310 0x7f30e40b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 -- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e40cc9c0 msgr2=0x7f30e40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e40cc9c0 0x7f30e40b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 -- 172.21.15.143:0/1547028176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc0094d0 con 0x7f30e406bfb0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e8a00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 0x7f30e4157260 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f30d400d5f0 tx=0x7f30d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d40125f0 con 0x7f30e406bfb0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d4012750 con 0x7f30e406bfb0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4012b00 con 0x7f30e406bfb0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40b1fc0 con 0x7f30e406bfb0 2024-01-15T18:15:59.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e41530a0 con 0x7f30e406bfb0 2024-01-15T18:15:59.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4067500 con 0x7f30e406bfb0 2024-01-15T18:15:59.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.174+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f30d4012c60 con 0x7f30e406bfb0 2024-01-15T18:15:59.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.178+0000 7f30e0c00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30bc0675e0 0x7f30bc069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.178+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f30d40834b0 con 0x7f30e406bfb0 2024-01-15T18:15:59.174 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.178+0000 7f30e3e00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30bc0675e0 0x7f30bc069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.178+0000 7f30e3e00640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30bc0675e0 0x7f30bc069aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0af000 tx=0x7f30cc0b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.178 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.182+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f30d4051fc0 con 0x7f30e406bfb0 2024-01-15T18:15:59.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.330+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30e40cd190 con 0x7f30e406bfb0 2024-01-15T18:15:59.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.330+0000 7f30e0c00640 1 -- 172.21.15.143:0/1547028176 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30d40555e0 con 0x7f30e406bfb0 2024-01-15T18:15:59.329 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30bc0675e0 msgr2=0x7f30bc069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30bc0675e0 0x7f30bc069aa0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0af000 tx=0x7f30cc0b6040 comp rx=0 tx=0).stop 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 msgr2=0x7f30e4157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 0x7f30e4157260 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f30d400d5f0 tx=0x7f30d400dab0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 shutdown_connections 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f30bc0675e0 0x7f30bc069aa0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e40cc9c0 0x7f30e40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f30e40b0310 0x7f30e40b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 --2- 172.21.15.143:0/1547028176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f30e406bfb0 0x7f30e4157260 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 >> 172.21.15.143:0/1547028176 conn(0x7f30e40d0ed0 msgr2=0x7f30e406f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:59.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 shutdown_connections 2024-01-15T18:15:59.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.334+0000 7f30eb400640 1 -- 172.21.15.143:0/1547028176 wait complete. 2024-01-15T18:15:59.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:15:59.436 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:59.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:59.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd70200640 1 Processor -- start 2024-01-15T18:15:59.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd70200640 1 -- start start 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd70200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcd680b0310 0x7fcd680cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd70200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680cbff0 con 0x7fcd680cf740 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd70200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680d0c50 con 0x7fcd6806bfb0 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd70200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680cb6c0 con 0x7fcd680b0310 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd6e200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcd680cf740 0x7fcd680cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51880/0 (socket says 172.21.15.143:51880) 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd6e200640 1 -- 172.21.15.143:0/3197553510 learned_addr learned my addr 172.21.15.143:0/3197553510 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd6ce00640 1 --2- 172.21.15.143:0/3197553510 >> v2:172.21.15.204:3300/0 conn(0x7fcd680b0310 0x7fcd680cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492884338 0 0) 0x7fcd680cbff0 con 0x7fcd680cf740 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd48003530 con 0x7fcd680cf740 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808921084 0 0) 0x7fcd680d0c50 con 0x7fcd6806bfb0 2024-01-15T18:15:59.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd680cbff0 con 0x7fcd6806bfb0 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3302803515 0 0) 0x7fcd48003530 con 0x7fcd680cf740 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 >> v2:172.21.15.204:3300/0 conn(0x7fcd680b0310 msgr2=0x7fcd680cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 --2- 172.21.15.143:0/3197553510 >> v2:172.21.15.204:3300/0 conn(0x7fcd680b0310 0x7fcd680cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 >> v1:172.21.15.204:6789/0 conn(0x7fcd6806bfb0 legacy=0x7fcd6806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680cb9a0 con 0x7fcd680cf740 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fcd60001a00 con 0x7fcd680cf740 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 >> v1:172.21.15.143:6789/0 conn(0x7fcd680cf740 legacy=0x7fcd680cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd48003b10 0x7fcd48003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd480047d0 0x7fcd48004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd48005190 0x7fcd4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680cb6c0 con 0x7fcd480047d0 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4800c0c0 con 0x7fcd48005190 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.494+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4800c280 con 0x7fcd48003b10 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd480047d0 0x7fcd48004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6e200640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd48005190 0x7fcd4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6d800640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd48003b10 0x7fcd48003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 -- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd48003b10 msgr2=0x7fcd48003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd48003b10 0x7fcd48003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 -- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd48005190 msgr2=0x7fcd4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd48005190 0x7fcd4800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 -- 172.21.15.143:0/3197553510 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd48003530 con 0x7fcd480047d0 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6e200640 1 -- 172.21.15.143:0/3197553510 reap_dead start 2024-01-15T18:15:59.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6d800640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd48003b10 0x7fcd48003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:59.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd6ce00640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd480047d0 0x7fcd48004c50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009ab0 tx=0x7fcd5001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b6b590b3bab63f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500a64f0 con 0x7fcd480047d0 2024-01-15T18:15:59.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd500a6650 con 0x7fcd480047d0 2024-01-15T18:15:59.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.498+0000 7fcd67e00640 1 -- 172.21.15.143:0/3197553510 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd500a6a00 con 0x7fcd480047d0 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- 172.21.15.143:0/3197553510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd480047d0 msgr2=0x7fcd48004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd480047d0 0x7fcd48004c50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009ab0 tx=0x7fcd5001ca00 comp rx=0 tx=0).stop 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- 172.21.15.143:0/3197553510 shutdown_connections 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 --2- 172.21.15.143:0/3197553510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd480047d0 0x7fcd48004c50 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- 172.21.15.143:0/3197553510 >> 172.21.15.143:0/3197553510 conn(0x7fcd680d0ed0 msgr2=0x7fcd680d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- 172.21.15.143:0/3197553510 shutdown_connections 2024-01-15T18:15:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- 172.21.15.143:0/3197553510 wait complete. 2024-01-15T18:15:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 Processor -- start 2024-01-15T18:15:59.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- start start 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd480047d0 0x7fcd681530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd68156160 0x7fcd681565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 0x7fcd680c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd681596c0 con 0x7fcd680c6410 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68153bc0 con 0x7fcd480047d0 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd70200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd68156b20 con 0x7fcd68156160 2024-01-15T18:15:59.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.502+0000 7fcd6e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 0x7fcd680c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 0x7fcd680c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33534/0 (socket says 172.21.15.143:33534) 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 -- 172.21.15.143:0/246030392 learned_addr learned my addr 172.21.15.143:0/246030392 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6ce00640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd68156160 0x7fcd681565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6d800640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd480047d0 0x7fcd681530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 -- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd68156160 msgr2=0x7fcd681565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd68156160 0x7fcd681565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 -- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd480047d0 msgr2=0x7fcd681530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd480047d0 0x7fcd681530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 -- 172.21.15.143:0/246030392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680c6dd0 con 0x7fcd680c6410 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6d800640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd480047d0 0x7fcd681530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd6e200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 0x7fcd680c6890 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcd680718d0 tx=0x7fcd60007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000aad0 con 0x7fcd680c6410 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000ac50 con 0x7fcd680c6410 2024-01-15T18:15:59.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd68153620 con 0x7fcd680c6410 2024-01-15T18:15:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd600133f0 con 0x7fcd680c6410 2024-01-15T18:15:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd680c7350 con 0x7fcd680c6410 2024-01-15T18:15:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd680c7540 con 0x7fcd680c6410 2024-01-15T18:15:59.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fcd60013550 con 0x7fcd680c6410 2024-01-15T18:15:59.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd65600640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd40067610 0x7fcd40069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.506+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fcd600835d0 con 0x7fcd680c6410 2024-01-15T18:15:59.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.510+0000 7fcd6d800640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd40067610 0x7fcd40069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.510+0000 7fcd6d800640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd40067610 0x7fcd40069ad0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcd680c4990 tx=0x7fcd58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.510+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcd680c7540 con 0x7fcd680c6410 2024-01-15T18:15:59.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.662+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcd680c7540 con 0x7fcd680c6410 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.662+0000 7fcd65600640 1 -- 172.21.15.143:0/246030392 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcd680c7540 con 0x7fcd680c6410 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:15:59.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd40067610 msgr2=0x7fcd40069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd40067610 0x7fcd40069ad0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcd680c4990 tx=0x7fcd58009040 comp rx=0 tx=0).stop 2024-01-15T18:15:59.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 msgr2=0x7fcd680c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 0x7fcd680c6890 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fcd680718d0 tx=0x7fcd60007ae0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 shutdown_connections 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd40067610 0x7fcd40069ad0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd680c6410 0x7fcd680c6890 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd68156160 0x7fcd681565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 --2- 172.21.15.143:0/246030392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd480047d0 0x7fcd681530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 >> 172.21.15.143:0/246030392 conn(0x7fcd680d0ed0 msgr2=0x7fcd680cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 shutdown_connections 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.666+0000 7fcd70200640 1 -- 172.21.15.143:0/246030392 wait complete. 2024-01-15T18:15:59.684 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:15:59.684 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:15:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.870+0000 7fe86b000640 1 Processor -- start 2024-01-15T18:15:59.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.870+0000 7fe86b000640 1 -- start start 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe86b000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe864143780 0x7fe8640cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe86b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe864071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe86b000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8640ceb50 con 0x7fe8640720f0 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe86b000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe864072580 con 0x7fe864143780 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe86b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86415c110 con 0x7fe8640ce520 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe864143780 0x7fe8640cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe869000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe8640720f0 0x7fe86415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51894/0 (socket says 172.21.15.143:51894) 2024-01-15T18:15:59.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe869000640 1 -- 172.21.15.143:0/3809047468 learned_addr learned my addr 172.21.15.143:0/3809047468 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:15:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863400640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe864071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863e00640 1 -- 172.21.15.143:0/3809047468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 msgr2=0x7fe864071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863e00640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe864071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863e00640 1 -- 172.21.15.143:0/3809047468 >> v1:172.21.15.143:6789/0 conn(0x7fe8640720f0 legacy=0x7fe86415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863e00640 1 -- 172.21.15.143:0/3809047468 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86415b7e0 con 0x7fe864143780 2024-01-15T18:15:59.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.874+0000 7fe863400640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe864071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:59.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe863e00640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe864143780 0x7fe8640cdfe0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe8540097c0 tx=0x7fe85401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd94ee7803182cef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe862a00640 1 -- 172.21.15.143:0/3809047468 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540ad640 con 0x7fe864143780 2024-01-15T18:15:59.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe862a00640 1 -- 172.21.15.143:0/3809047468 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8540ad7a0 con 0x7fe864143780 2024-01-15T18:15:59.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe862a00640 1 -- 172.21.15.143:0/3809047468 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540adb30 con 0x7fe864143780 2024-01-15T18:15:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 -- 172.21.15.143:0/3809047468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe864143780 msgr2=0x7fe8640cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe864143780 0x7fe8640cdfe0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe8540097c0 tx=0x7fe85401c480 comp rx=0 tx=0).stop 2024-01-15T18:15:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 -- 172.21.15.143:0/3809047468 shutdown_connections 2024-01-15T18:15:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe864071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 --2- 172.21.15.143:0/3809047468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe864143780 0x7fe8640cdfe0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 -- 172.21.15.143:0/3809047468 >> 172.21.15.143:0/3809047468 conn(0x7fe86406c3f0 msgr2=0x7fe86406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 -- 172.21.15.143:0/3809047468 shutdown_connections 2024-01-15T18:15:59.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 -- 172.21.15.143:0/3809047468 wait complete. 2024-01-15T18:15:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 Processor -- start 2024-01-15T18:15:59.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.878+0000 7fe86b000640 1 -- start start 2024-01-15T18:15:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 0x7fe864159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe8641391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe864143780 0x7fe86412f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe864139700 con 0x7fe8640720f0 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe864136d20 con 0x7fe864143780 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640adad0 con 0x7fe8640ce520 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 0x7fe864159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 0x7fe864159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33556/0 (socket says 172.21.15.143:33556) 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 -- 172.21.15.143:0/936389082 learned_addr learned my addr 172.21.15.143:0/936389082 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863400640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe8641391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe869000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe864143780 0x7fe86412f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 -- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 msgr2=0x7fe8641391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe8641391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 -- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe864143780 msgr2=0x7fe86412f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe864143780 0x7fe86412f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 -- 172.21.15.143:0/936389082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe854009420 con 0x7fe8640720f0 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863400640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe8641391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe863e00640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 0x7fe864159b00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe8540b0040 tx=0x7fe854019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540b6820 con 0x7fe8640720f0 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8540a6040 con 0x7fe8640720f0 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86413ae50 con 0x7fe8640720f0 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8540b6be0 con 0x7fe8640720f0 2024-01-15T18:15:59.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86413ab00 con 0x7fe8640720f0 2024-01-15T18:15:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.882+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86413ac90 con 0x7fe8640720f0 2024-01-15T18:15:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.886+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fe8540b4040 con 0x7fe8640720f0 2024-01-15T18:15:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.886+0000 7fe85be00640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe83c0675e0 0x7fe83c069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.886+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fe85411d290 con 0x7fe8640720f0 2024-01-15T18:15:59.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.886+0000 7fe863400640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe83c0675e0 0x7fe83c069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.886+0000 7fe863400640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe83c0675e0 0x7fe83c069aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:15:59.890+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe8540ebcf0 con 0x7fe8640720f0 2024-01-15T18:16:00.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.042+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe864067500 con 0x7fe8640720f0 2024-01-15T18:16:00.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.938+0000 7fe85be00640 1 -- 172.21.15.143:0/936389082 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe8540b2070 con 0x7fe8640720f0 2024-01-15T18:16:00.937 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe83c0675e0 msgr2=0x7fe83c069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe83c0675e0 0x7fe83c069aa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c008040 comp rx=0 tx=0).stop 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 msgr2=0x7fe864159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 0x7fe864159b00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe8540b0040 tx=0x7fe854019220 comp rx=0 tx=0).stop 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 shutdown_connections 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe83c0675e0 0x7fe83c069aa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe864143780 0x7fe86412f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8640ce520 0x7fe8641391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 --2- 172.21.15.143:0/936389082 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8640720f0 0x7fe864159b00 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 >> 172.21.15.143:0/936389082 conn(0x7fe86406c3f0 msgr2=0x7fe8640cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:00.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 shutdown_connections 2024-01-15T18:16:00.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:00.942+0000 7fe86b000640 1 -- 172.21.15.143:0/936389082 wait complete. 2024-01-15T18:16:00.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:16:01.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.118+0000 7f8658400640 1 Processor -- start 2024-01-15T18:16:01.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 -- start start 2024-01-15T18:16:01.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8650158a00 0x7f86500ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f865015bf20 0x7f86500cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f86500cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865015f980 con 0x7f865015bf20 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865015c3b0 con 0x7f8650158a00 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8658400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865015ebc0 con 0x7f8650070b50 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f865015bf20 0x7f86500cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f865015bf20 0x7f86500cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33568/0 (socket says 172.21.15.143:33568) 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 -- 172.21.15.143:0/170281830 learned_addr learned my addr 172.21.15.143:0/170281830 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655a00640 1 --2- 172.21.15.143:0/170281830 >> v2:172.21.15.204:3300/0 conn(0x7f8650158a00 0x7f86500ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8656400640 1 --2- 172.21.15.143:0/170281830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f86500cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 -- 172.21.15.143:0/170281830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 msgr2=0x7f86500cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 --2- 172.21.15.143:0/170281830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f86500cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 -- 172.21.15.143:0/170281830 >> v2:172.21.15.204:3300/0 conn(0x7f8650158a00 msgr2=0x7f86500ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 --2- 172.21.15.143:0/170281830 >> v2:172.21.15.204:3300/0 conn(0x7f8650158a00 0x7f86500ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.122+0000 7f8655000640 1 -- 172.21.15.143:0/170281830 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865015eea0 con 0x7f865015bf20 2024-01-15T18:16:01.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8655000640 1 --2- 172.21.15.143:0/170281830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015bf20 0x7f86500cc3d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f865006e350 tx=0x7f863801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a48c187f44d7b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:01.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f864fe00640 1 -- 172.21.15.143:0/170281830 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380ab600 con 0x7f865015bf20 2024-01-15T18:16:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f864fe00640 1 -- 172.21.15.143:0/170281830 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86380ab760 con 0x7f865015bf20 2024-01-15T18:16:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f864fe00640 1 -- 172.21.15.143:0/170281830 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380abaf0 con 0x7f865015bf20 2024-01-15T18:16:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 -- 172.21.15.143:0/170281830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015bf20 msgr2=0x7f86500cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:01.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 --2- 172.21.15.143:0/170281830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015bf20 0x7f86500cc3d0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f865006e350 tx=0x7f863801c440 comp rx=0 tx=0).stop 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 -- 172.21.15.143:0/170281830 shutdown_connections 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 --2- 172.21.15.143:0/170281830 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f86500cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 --2- 172.21.15.143:0/170281830 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015bf20 0x7f86500cc3d0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 --2- 172.21.15.143:0/170281830 >> v2:172.21.15.204:3300/0 conn(0x7f8650158a00 0x7f86500ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 -- 172.21.15.143:0/170281830 >> 172.21.15.143:0/170281830 conn(0x7f865006a890 msgr2=0x7f865006aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 -- 172.21.15.143:0/170281830 shutdown_connections 2024-01-15T18:16:01.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.126+0000 7f8658400640 1 -- 172.21.15.143:0/170281830 wait complete. 2024-01-15T18:16:01.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 Processor -- start 2024-01-15T18:16:01.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 -- start start 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f8650146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 0x7f86500b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015c5d0 0x7f865006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865015d7c0 con 0x7f865015c5d0 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8650147510 con 0x7f8650158a00 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865015cb40 con 0x7f8650070b50 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8656400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015c5d0 0x7f865006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 0x7f86500b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f8650146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 0x7f86500b2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:55058/0 (socket says 172.21.15.143:55058) 2024-01-15T18:16:01.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 -- 172.21.15.143:0/473867997 learned_addr learned my addr 172.21.15.143:0/473867997 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 -- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 msgr2=0x7f8650146fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f8650146fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 -- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015c5d0 msgr2=0x7f865006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015c5d0 0x7f865006d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 -- 172.21.15.143:0/473867997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86380094d0 con 0x7f8650158a00 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8656400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015c5d0 0x7f865006d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8655000640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 0x7f86500b2900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f863801aaf0 tx=0x7f863801ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380b4800 con 0x7f8650158a00 2024-01-15T18:16:01.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86380b4980 con 0x7f8650158a00 2024-01-15T18:16:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380b4d30 con 0x7f8650158a00 2024-01-15T18:16:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865015ce20 con 0x7f8650158a00 2024-01-15T18:16:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.130+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865015d3e0 con 0x7f8650158a00 2024-01-15T18:16:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.134+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86500cb5f0 con 0x7f8650158a00 2024-01-15T18:16:01.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.134+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f86380c7440 con 0x7f8650158a00 2024-01-15T18:16:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.134+0000 7f864d600640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86280675e0 0x7f8628069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.134+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f863811d7b0 con 0x7f8650158a00 2024-01-15T18:16:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.134+0000 7f8655a00640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86280675e0 0x7f8628069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:01.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.134+0000 7f8655a00640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86280675e0 0x7f8628069aa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f865006de90 tx=0x7f8640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:01.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.138+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86380ec550 con 0x7f8650158a00 2024-01-15T18:16:01.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.290+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f865015d900 con 0x7f8650158a00 2024-01-15T18:16:01.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.954+0000 7f864d600640 1 -- 172.21.15.143:0/473867997 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f865015d900 con 0x7f8650158a00 2024-01-15T18:16:01.951 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:16:01.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.958+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86280675e0 msgr2=0x7f8628069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:01.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.958+0000 7f8658400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86280675e0 0x7f8628069aa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f865006de90 tx=0x7f8640008040 comp rx=0 tx=0).stop 2024-01-15T18:16:01.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.958+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 msgr2=0x7f86500b2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:01.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.958+0000 7f8658400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 0x7f86500b2900 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f863801aaf0 tx=0x7f863801ac80 comp rx=0 tx=0).stop 2024-01-15T18:16:01.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 shutdown_connections 2024-01-15T18:16:01.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86280675e0 0x7f8628069aa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f865015c5d0 0x7f865006d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8650158a00 0x7f86500b2900 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 --2- 172.21.15.143:0/473867997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8650070b50 0x7f8650146fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:01.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 >> 172.21.15.143:0/473867997 conn(0x7f865006a890 msgr2=0x7f8650157c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:01.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 shutdown_connections 2024-01-15T18:16:01.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:01.962+0000 7f8658400640 1 -- 172.21.15.143:0/473867997 wait complete. 2024-01-15T18:16:01.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:16:02.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcda800640 1 Processor -- start 2024-01-15T18:16:02.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcda800640 1 -- start start 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcda800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cfb30 0x7efcd406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcda800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40cffd0 con 0x7efcd40cbf50 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcda800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd40d02b0 con 0x7efcd41607c0 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcda800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd40d1ce0 con 0x7efcd40cfb30 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efcd40cbf50 0x7efcd40cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51900/0 (socket says 172.21.15.143:51900) 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd3e00640 1 -- 172.21.15.143:0/1356223496 learned_addr learned my addr 172.21.15.143:0/1356223496 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2a00640 1 --2- 172.21.15.143:0/1356223496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cfb30 0x7efcd406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103199468 0 0) 0x7efcd40cffd0 con 0x7efcd40cbf50 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb4003530 con 0x7efcd40cbf50 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895646216 0 0) 0x7efcd40d02b0 con 0x7efcd41607c0 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcd40cffd0 con 0x7efcd41607c0 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2a00640 1 -- 172.21.15.143:0/1356223496 >> v1:172.21.15.204:6789/0 conn(0x7efcd41607c0 legacy=0x7efcd406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2a00640 1 -- 172.21.15.143:0/1356223496 >> v1:172.21.15.143:6789/0 conn(0x7efcd40cbf50 legacy=0x7efcd40cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.134+0000 7efcd2a00640 1 -- 172.21.15.143:0/1356223496 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd40cb470 con 0x7efcd40cfb30 2024-01-15T18:16:02.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcd2a00640 1 --2- 172.21.15.143:0/1356223496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cfb30 0x7efcd406bec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0097f0 tx=0x7efcbc01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a913ae5c71ec76dc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0a89b0 con 0x7efcd40cfb30 2024-01-15T18:16:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc0a8b10 con 0x7efcd40cfb30 2024-01-15T18:16:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcd2000640 1 -- 172.21.15.143:0/1356223496 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc0a8ea0 con 0x7efcd40cfb30 2024-01-15T18:16:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 -- 172.21.15.143:0/1356223496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cfb30 msgr2=0x7efcd406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 --2- 172.21.15.143:0/1356223496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cfb30 0x7efcd406bec0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efcbc0097f0 tx=0x7efcbc01ecb0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 -- 172.21.15.143:0/1356223496 shutdown_connections 2024-01-15T18:16:02.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 --2- 172.21.15.143:0/1356223496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cfb30 0x7efcd406bec0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 -- 172.21.15.143:0/1356223496 >> 172.21.15.143:0/1356223496 conn(0x7efcd40d0e20 msgr2=0x7efcd40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:02.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 -- 172.21.15.143:0/1356223496 shutdown_connections 2024-01-15T18:16:02.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.138+0000 7efcda800640 1 -- 172.21.15.143:0/1356223496 wait complete. 2024-01-15T18:16:02.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 Processor -- start 2024-01-15T18:16:02.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 -- start start 2024-01-15T18:16:02.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cbf50 0x7efcd41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd40cfb30 0x7efcd415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 0x7efcd41f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4158580 con 0x7efcd41607c0 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4157cb0 con 0x7efcd40cfb30 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd414eb40 con 0x7efcd40cbf50 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 0x7efcd41f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 0x7efcd41f6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33590/0 (socket says 172.21.15.143:33590) 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 -- 172.21.15.143:0/851989213 learned_addr learned my addr 172.21.15.143:0/851989213 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd2a00640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd40cfb30 0x7efcd415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3400640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cbf50 0x7efcd41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 -- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cbf50 msgr2=0x7efcd41589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cbf50 0x7efcd41589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 -- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd40cfb30 msgr2=0x7efcd415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd40cfb30 0x7efcd415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 -- 172.21.15.143:0/851989213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc009420 con 0x7efcd41607c0 2024-01-15T18:16:02.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3400640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cbf50 0x7efcd41589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:02.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcd3e00640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 0x7efcd41f6b10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efccc00cba0 tx=0x7efccc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:02.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00ace0 con 0x7efcd41607c0 2024-01-15T18:16:02.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc00ae40 con 0x7efcd41607c0 2024-01-15T18:16:02.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc012620 con 0x7efcd41607c0 2024-01-15T18:16:02.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.142+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd415ae60 con 0x7efcd41607c0 2024-01-15T18:16:02.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.146+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd41581a0 con 0x7efcd41607c0 2024-01-15T18:16:02.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.146+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd406d4b0 con 0x7efcd41607c0 2024-01-15T18:16:02.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.146+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7efccc007070 con 0x7efcd41607c0 2024-01-15T18:16:02.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.146+0000 7efccb400640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efcac0675e0 0x7efcac069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.146+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5665+0+0 (secure 0 0 0) 0x7efccc083280 con 0x7efcd41607c0 2024-01-15T18:16:02.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.146+0000 7efcd3400640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efcac0675e0 0x7efcac069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.150+0000 7efcd3400640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efcac0675e0 0x7efcac069aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efcc40011f0 tx=0x7efcc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:02.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.150+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efccc051fc0 con 0x7efcd41607c0 2024-01-15T18:16:02.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.330+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efcd41586c0 con 0x7efcd41607c0 2024-01-15T18:16:02.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.974+0000 7efccb400640 1 -- 172.21.15.143:0/851989213 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7efcd41586c0 con 0x7efcd41607c0 2024-01-15T18:16:02.981 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 12 and data pool 13 2024-01-15T18:16:02.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.982+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efcac0675e0 msgr2=0x7efcac069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.982+0000 7efcda800640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efcac0675e0 0x7efcac069aa0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7efcc40011f0 tx=0x7efcc4009040 comp rx=0 tx=0).stop 2024-01-15T18:16:02.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.982+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 msgr2=0x7efcd41f6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.982+0000 7efcda800640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 0x7efcd41f6b10 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7efccc00cba0 tx=0x7efccc00a430 comp rx=0 tx=0).stop 2024-01-15T18:16:02.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.986+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 shutdown_connections 2024-01-15T18:16:02.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.986+0000 7efcda800640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efcac0675e0 0x7efcac069aa0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.986+0000 7efcda800640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcd41607c0 0x7efcd41f6b10 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.986+0000 7efcda800640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efcd40cfb30 0x7efcd415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.986+0000 7efcda800640 1 --2- 172.21.15.143:0/851989213 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efcd40cbf50 0x7efcd41589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.986+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 >> 172.21.15.143:0/851989213 conn(0x7efcd40d0e20 msgr2=0x7efcd406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:02.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.990+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 shutdown_connections 2024-01-15T18:16:02.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:02.990+0000 7efcda800640 1 -- 172.21.15.143:0/851989213 wait complete. 2024-01-15T18:16:03.017 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:16:03.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5573c00640 1 Processor -- start 2024-01-15T18:16:03.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5573c00640 1 -- start start 2024-01-15T18:16:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5573c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f556c157300 0x7f556c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5573c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c0b2a20 con 0x7f556c0b21d0 2024-01-15T18:16:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5573c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c0b2d00 con 0x7f556c0b4ac0 2024-01-15T18:16:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5573c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c0b35a0 con 0x7f556c157300 2024-01-15T18:16:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f556c0b21d0 0x7f556c0b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51916/0 (socket says 172.21.15.143:51916) 2024-01-15T18:16:03.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556be00640 1 -- 172.21.15.143:0/882042728 learned_addr learned my addr 172.21.15.143:0/882042728 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f5571200640 1 --2- 172.21.15.143:0/882042728 >> v2:172.21.15.204:3300/0 conn(0x7f556c157300 0x7f556c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247007380 0 0) 0x7f556c0b2a20 con 0x7f556c0b21d0 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f554c005740 con 0x7f556c0b21d0 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611022362 0 0) 0x7f554c005740 con 0x7f556c0b21d0 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 >> v2:172.21.15.204:3300/0 conn(0x7f556c157300 msgr2=0x7f556c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 --2- 172.21.15.143:0/882042728 >> v2:172.21.15.204:3300/0 conn(0x7f556c157300 0x7f556c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 >> v1:172.21.15.204:6789/0 conn(0x7f556c0b4ac0 legacy=0x7f556c070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0b2fe0 con 0x7f556c0b21d0 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f5554001680 con 0x7f556c0b21d0 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 >> v1:172.21.15.143:6789/0 conn(0x7f556c0b21d0 legacy=0x7f556c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554c005b90 0x7f554c005f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c006590 0x7f554c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554c007260 0x7f554c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0b35a0 con 0x7f554c007260 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00e1c0 con 0x7f554c006590 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.158+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00e380 con 0x7f554c005b90 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f556be00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c006590 0x7f554c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554c007260 0x7f554c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571200640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554c005b90 0x7f554c005f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 -- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554c005b90 msgr2=0x7f554c005f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f554c005b90 0x7f554c005f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 -- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c006590 msgr2=0x7f554c006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c006590 0x7f554c006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 -- 172.21.15.143:0/882042728 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c005740 con 0x7f554c007260 2024-01-15T18:16:03.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 -- 172.21.15.143:0/882042728 reap_dead start 2024-01-15T18:16:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f5571c00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554c007260 0x7f554c00dba0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5564001f60 tx=0x7f556401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb603f3c6a36f05 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564003b80 con 0x7f554c007260 2024-01-15T18:16:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564003ce0 con 0x7f554c007260 2024-01-15T18:16:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.162+0000 7f556b400640 1 -- 172.21.15.143:0/882042728 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640052f0 con 0x7f554c007260 2024-01-15T18:16:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- 172.21.15.143:0/882042728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554c007260 msgr2=0x7f554c00dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554c007260 0x7f554c00dba0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5564001f60 tx=0x7f556401c860 comp rx=0 tx=0).stop 2024-01-15T18:16:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- 172.21.15.143:0/882042728 shutdown_connections 2024-01-15T18:16:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 --2- 172.21.15.143:0/882042728 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f554c007260 0x7f554c00dba0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- 172.21.15.143:0/882042728 >> 172.21.15.143:0/882042728 conn(0x7f556c06c3f0 msgr2=0x7f556c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- 172.21.15.143:0/882042728 shutdown_connections 2024-01-15T18:16:03.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- 172.21.15.143:0/882042728 wait complete. 2024-01-15T18:16:03.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 Processor -- start 2024-01-15T18:16:03.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- start start 2024-01-15T18:16:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c007260 0x7f556c1e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f556c158f00 0x7f556c1e3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 0x7f556c1f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1e3ac0 con 0x7f556c158330 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1e24e0 con 0x7f554c007260 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5573c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c1e27c0 con 0x7f556c158f00 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5571c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 0x7f556c1f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5571c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 0x7f556c1f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33606/0 (socket says 172.21.15.143:33606) 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5571c00640 1 -- 172.21.15.143:0/839642065 learned_addr learned my addr 172.21.15.143:0/839642065 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f556be00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f556c158f00 0x7f556c1e3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.166+0000 7f5571200640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c007260 0x7f556c1e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5571c00640 1 -- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f556c158f00 msgr2=0x7f556c1e3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5571c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f556c158f00 0x7f556c1e3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5571c00640 1 -- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c007260 msgr2=0x7f556c1e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5571c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c007260 0x7f556c1e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5571c00640 1 -- 172.21.15.143:0/839642065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c1e2aa0 con 0x7f556c158330 2024-01-15T18:16:03.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5571c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 0x7f556c1f2340 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f556401d100 tx=0x7f554c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564005830 con 0x7f556c158330 2024-01-15T18:16:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55640b4470 con 0x7f556c158330 2024-01-15T18:16:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c1e2d80 con 0x7f556c158330 2024-01-15T18:16:03.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55640b4820 con 0x7f556c158330 2024-01-15T18:16:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c159380 con 0x7f556c158330 2024-01-15T18:16:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c1e32a0 con 0x7f556c158330 2024-01-15T18:16:03.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.170+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f55640b4980 con 0x7f556c158330 2024-01-15T18:16:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.174+0000 7f5568c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5544067610 0x7f5544069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.174+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f55640b0070 con 0x7f556c158330 2024-01-15T18:16:03.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.174+0000 7f5571200640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5544067610 0x7f5544069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.174+0000 7f5571200640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5544067610 0x7f5544069ad0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f555c009b60 tx=0x7f555c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.178+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f556c1e32a0 con 0x7f556c158330 2024-01-15T18:16:03.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.326+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f556c1e32a0 con 0x7f556c158330 2024-01-15T18:16:03.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5568c00640 1 -- 172.21.15.143:0/839642065 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16697 (secure 0 0 0) 0x7f556c1e32a0 con 0x7f556c158330 2024-01-15T18:16:03.326 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:03.326 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":86,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:16:02.335403+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:16:00.088960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-15T18:16:01.336678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5544067610 msgr2=0x7f5544069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5544067610 0x7f5544069ad0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f555c009b60 tx=0x7f555c009190 comp rx=0 tx=0).stop 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 msgr2=0x7f556c1f2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 0x7f556c1f2340 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f556401d100 tx=0x7f554c007a10 comp rx=0 tx=0).stop 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 shutdown_connections 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5544067610 0x7f5544069ad0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f556c158330 0x7f556c1f2340 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f556c158f00 0x7f556c1e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 --2- 172.21.15.143:0/839642065 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f554c007260 0x7f556c1e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 >> 172.21.15.143:0/839642065 conn(0x7f556c06c3f0 msgr2=0x7f556c15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:03.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 shutdown_connections 2024-01-15T18:16:03.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.330+0000 7f5573c00640 1 -- 172.21.15.143:0/839642065 wait complete. 2024-01-15T18:16:03.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:16:03.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 Processor -- start 2024-01-15T18:16:03.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 -- start start 2024-01-15T18:16:03.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f6ce00d19f0 0x7f6ce00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce00c4d90 con 0x7f6ce00c5220 2024-01-15T18:16:03.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce015b640 con 0x7f6ce00d19f0 2024-01-15T18:16:03.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00cf800 con 0x7f6ce015b260 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce5a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f6ce00c5220 0x7f6ce00c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42752/0 (socket says 172.21.15.143:42752) 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce5a00640 1 -- 172.21.15.143:0/3486179600 learned_addr learned my addr 172.21.15.143:0/3486179600 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.486+0000 7f6ce5000640 1 --2- 172.21.15.143:0/3486179600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6cdfe00640 1 --2- 172.21.15.143:0/3486179600 >> v2:172.21.15.204:3300/0 conn(0x7f6ce00d19f0 0x7f6ce00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6cdf400640 1 -- 172.21.15.143:0/3486179600 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865109093 0 0) 0x7f6ce00c4d90 con 0x7f6ce00c5220 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6cdf400640 1 -- 172.21.15.143:0/3486179600 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc0003530 con 0x7f6ce00c5220 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6ce5000640 1 -- 172.21.15.143:0/3486179600 >> v2:172.21.15.204:3300/0 conn(0x7f6ce00d19f0 msgr2=0x7f6ce00cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6ce5000640 1 --2- 172.21.15.143:0/3486179600 >> v2:172.21.15.204:3300/0 conn(0x7f6ce00d19f0 0x7f6ce00cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6ce5000640 1 -- 172.21.15.143:0/3486179600 >> v1:172.21.15.204:6789/0 conn(0x7f6ce00c5220 legacy=0x7f6ce00c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6ce5000640 1 -- 172.21.15.143:0/3486179600 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce00cfae0 con 0x7f6ce015b260 2024-01-15T18:16:03.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6ce5000640 1 --2- 172.21.15.143:0/3486179600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00097c0 tx=0x7f6cd001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1601103fe79fcab9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6cdf400640 1 -- 172.21.15.143:0/3486179600 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00ad660 con 0x7f6ce015b260 2024-01-15T18:16:03.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6cdf400640 1 -- 172.21.15.143:0/3486179600 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd00ad7c0 con 0x7f6ce015b260 2024-01-15T18:16:03.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.490+0000 7f6cdf400640 1 -- 172.21.15.143:0/3486179600 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00adb50 con 0x7f6ce015b260 2024-01-15T18:16:03.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3486179600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 msgr2=0x7f6ce00cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3486179600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00097c0 tx=0x7f6cd001c450 comp rx=0 tx=0).stop 2024-01-15T18:16:03.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3486179600 shutdown_connections 2024-01-15T18:16:03.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3486179600 >> v2:172.21.15.204:3300/0 conn(0x7f6ce00d19f0 0x7f6ce00cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3486179600 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3486179600 >> 172.21.15.143:0/3486179600 conn(0x7f6ce00d0ed0 msgr2=0x7f6ce00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:03.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3486179600 shutdown_connections 2024-01-15T18:16:03.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3486179600 wait complete. 2024-01-15T18:16:03.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 Processor -- start 2024-01-15T18:16:03.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- start start 2024-01-15T18:16:03.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 0x7f6ce009ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce00d19f0 0x7f6ce0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00ae7e0 con 0x7f6ce00c5220 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00aeac0 con 0x7f6ce00d19f0 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.494+0000 7f6ce7a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce00aeda0 con 0x7f6ce015b260 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 0x7f6ce009ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 0x7f6ce009ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33614/0 (socket says 172.21.15.143:33614) 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 -- 172.21.15.143:0/3480933760 learned_addr learned my addr 172.21.15.143:0/3480933760 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdfe00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce00d19f0 0x7f6ce0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 -- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 msgr2=0x7f6ce00cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 -- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce00d19f0 msgr2=0x7f6ce0152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce00d19f0 0x7f6ce0152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 -- 172.21.15.143:0/3480933760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd0009420 con 0x7f6ce00c5220 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdfe00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce00d19f0 0x7f6ce0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:03.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce5000640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 0x7f6ce009ac70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6cd001ab00 tx=0x7f6cd00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00b6600 con 0x7f6ce00c5220 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd00b6780 con 0x7f6ce00c5220 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce00af080 con 0x7f6ce00c5220 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00b6b30 con 0x7f6ce00c5220 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce0152fe0 con 0x7f6ce00c5220 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce0067500 con 0x7f6ce00c5220 2024-01-15T18:16:03.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f6cd00b4040 con 0x7f6ce00c5220 2024-01-15T18:16:03.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.498+0000 7f6cdcc00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cb80675e0 0x7f6cb8069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.502+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6cd011d1e0 con 0x7f6ce00c5220 2024-01-15T18:16:03.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.502+0000 7f6cdfe00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cb80675e0 0x7f6cb8069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.502+0000 7f6cdfe00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cb80675e0 0x7f6cb8069aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00b6be0 tx=0x7f6cc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.506+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6cd00ebc40 con 0x7f6ce00c5220 2024-01-15T18:16:03.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.654+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ce00af530 con 0x7f6ce00c5220 2024-01-15T18:16:03.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.654+0000 7f6cdcc00640 1 -- 172.21.15.143:0/3480933760 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16697 (secure 0 0 0) 0x7f6cd00b2090 con 0x7f6ce00c5220 2024-01-15T18:16:03.653 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:03.653 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":86,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:16:02.335403+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:16:00.088960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-15T18:16:01.336678+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:16:03.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cb80675e0 msgr2=0x7f6cb8069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cb80675e0 0x7f6cb8069aa0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6ce00b6be0 tx=0x7f6cc8008040 comp rx=0 tx=0).stop 2024-01-15T18:16:03.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 msgr2=0x7f6ce009ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 0x7f6ce009ac70 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6cd001ab00 tx=0x7f6cd00191f0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 shutdown_connections 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6cb80675e0 0x7f6cb8069aa0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6ce015b260 0x7f6ce00cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ce00d19f0 0x7f6ce0152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 --2- 172.21.15.143:0/3480933760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ce00c5220 0x7f6ce009ac70 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 >> 172.21.15.143:0/3480933760 conn(0x7f6ce00d0ed0 msgr2=0x7f6ce0153d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 shutdown_connections 2024-01-15T18:16:03.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.658+0000 7f6ce7a00640 1 -- 172.21.15.143:0/3480933760 wait complete. 2024-01-15T18:16:03.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:16:03.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.818+0000 7febfe000640 1 Processor -- start 2024-01-15T18:16:03.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.818+0000 7febfe000640 1 -- start start 2024-01-15T18:16:03.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.818+0000 7febfe000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febfe000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7febf80c5600 0x7febf80c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febfe000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf80ce7c0 con 0x7febf80cf410 2024-01-15T18:16:03.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febfe000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf80cef50 con 0x7febf80c5600 2024-01-15T18:16:03.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febfe000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80ce290 con 0x7febf80cde50 2024-01-15T18:16:03.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf7e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7febf80c5600 0x7febf80c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf7400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7febf80cf410 0x7febf80cf7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42764/0 (socket says 172.21.15.143:42764) 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf80d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42746/0 (socket says 172.21.15.143:42746) 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf7400640 1 -- 172.21.15.143:0/2451931281 learned_addr learned my addr 172.21.15.143:0/2451931281 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 -- 172.21.15.143:0/2451931281 >> v2:172.21.15.204:3300/0 conn(0x7febf80c5600 msgr2=0x7febf80c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 --2- 172.21.15.143:0/2451931281 >> v2:172.21.15.204:3300/0 conn(0x7febf80c5600 0x7febf80c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 -- 172.21.15.143:0/2451931281 >> v1:172.21.15.204:6789/0 conn(0x7febf80cf410 legacy=0x7febf80cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 -- 172.21.15.143:0/2451931281 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80ce4e0 con 0x7febf80cde50 2024-01-15T18:16:03.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf7e00640 1 --2- 172.21.15.143:0/2451931281 >> v2:172.21.15.204:3300/0 conn(0x7febf80c5600 0x7febf80c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:03.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.822+0000 7febf6a00640 1 --2- 172.21.15.143:0/2451931281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf80d1950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febe00097c0 tx=0x7febe001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a01ce03f0f1bc0a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febf6000640 1 -- 172.21.15.143:0/2451931281 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00ab600 con 0x7febf80cde50 2024-01-15T18:16:03.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febf6000640 1 -- 172.21.15.143:0/2451931281 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe00a7050 con 0x7febf80cde50 2024-01-15T18:16:03.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febf6000640 1 -- 172.21.15.143:0/2451931281 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00ab980 con 0x7febf80cde50 2024-01-15T18:16:03.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 -- 172.21.15.143:0/2451931281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 msgr2=0x7febf80d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 --2- 172.21.15.143:0/2451931281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf80d1950 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febe00097c0 tx=0x7febe001c480 comp rx=0 tx=0).stop 2024-01-15T18:16:03.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 -- 172.21.15.143:0/2451931281 shutdown_connections 2024-01-15T18:16:03.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 --2- 172.21.15.143:0/2451931281 >> v2:172.21.15.204:3300/0 conn(0x7febf80c5600 0x7febf80c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 --2- 172.21.15.143:0/2451931281 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf80d1950 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 -- 172.21.15.143:0/2451931281 >> 172.21.15.143:0/2451931281 conn(0x7febf80d0e30 msgr2=0x7febf80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:03.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 -- 172.21.15.143:0/2451931281 shutdown_connections 2024-01-15T18:16:03.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 -- 172.21.15.143:0/2451931281 wait complete. 2024-01-15T18:16:03.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 Processor -- start 2024-01-15T18:16:03.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.826+0000 7febfe000640 1 -- start start 2024-01-15T18:16:03.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 0x7febf815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf81571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febf80cf410 0x7febf8163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8165c50 con 0x7febf80c5600 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8165970 con 0x7febf80cf410 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf81648d0 con 0x7febf80cde50 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 0x7febf815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 0x7febf815b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33620/0 (socket says 172.21.15.143:33620) 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 -- 172.21.15.143:0/2675776910 learned_addr learned my addr 172.21.15.143:0/2675776910 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf6a00640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf81571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7e00640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febf80cf410 0x7febf8163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 -- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 msgr2=0x7febf81571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf81571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 -- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febf80cf410 msgr2=0x7febf8163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:03.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febf80cf410 0x7febf8163b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 -- 172.21.15.143:0/2675776910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf0009a70 con 0x7febf80c5600 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febf7400640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 0x7febf815b110 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7febe800c400 tx=0x7febe800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8005cf0 con 0x7febf80c5600 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8005e50 con 0x7febf80c5600 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80072d0 con 0x7febf80c5600 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe0009420 con 0x7febf80c5600 2024-01-15T18:16:03.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf8164e90 con 0x7febf80c5600 2024-01-15T18:16:03.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf8067500 con 0x7febf80c5600 2024-01-15T18:16:03.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.830+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7febe8007430 con 0x7febf80c5600 2024-01-15T18:16:03.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.834+0000 7febef400640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7febd00675e0 0x7febd0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:03.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.834+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7febe80820b0 con 0x7febf80c5600 2024-01-15T18:16:03.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.834+0000 7febf6a00640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7febd00675e0 0x7febd0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:03.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.834+0000 7febf6a00640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7febd00675e0 0x7febd0069aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7febe00ad000 tx=0x7febe0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:03.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:03.838+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7febe8050df0 con 0x7febf80c5600 2024-01-15T18:16:04.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.022+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7febf80ce7c0 con 0x7febf80c5600 2024-01-15T18:16:04.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febef400640 1 -- 172.21.15.143:0/2675776910 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7febe8017020 con 0x7febf80c5600 2024-01-15T18:16:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7febd00675e0 msgr2=0x7febd0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7febd00675e0 0x7febd0069aa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7febe00ad000 tx=0x7febe0008040 comp rx=0 tx=0).stop 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 msgr2=0x7febf815b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 0x7febf815b110 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7febe800c400 tx=0x7febe800c800 comp rx=0 tx=0).stop 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 shutdown_connections 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7febd00675e0 0x7febd0069aa0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7febf80cf410 0x7febf8163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7febf80cde50 0x7febf81571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 --2- 172.21.15.143:0/2675776910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febf80c5600 0x7febf815b110 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 >> 172.21.15.143:0/2675776910 conn(0x7febf80d0e30 msgr2=0x7febf8157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 shutdown_connections 2024-01-15T18:16:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:04.990+0000 7febfe000640 1 -- 172.21.15.143:0/2675776910 wait complete. 2024-01-15T18:16:05.011 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:16:05.012 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:16:05.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.158+0000 7fd461200640 1 Processor -- start 2024-01-15T18:16:05.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.158+0000 7fd461200640 1 -- start start 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd461200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd45c15d1f0 0x7fd45c06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd461200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c06bb70 0x7fd45c070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd461200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45c0b3370 con 0x7fd45c0712a0 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd461200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c0716f0 con 0x7fd45c15d1f0 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd461200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c15d5d0 con 0x7fd45c06bb70 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd45c15d1f0 0x7fd45c06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd45c15d1f0 0x7fd45c06b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33632/0 (socket says 172.21.15.143:33632) 2024-01-15T18:16:05.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 -- 172.21.15.143:0/3182191997 learned_addr learned my addr 172.21.15.143:0/3182191997 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459400640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c06bb70 0x7fd45c070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 -- 172.21.15.143:0/3182191997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c06bb70 msgr2=0x7fd45c070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c06bb70 0x7fd45c070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 -- 172.21.15.143:0/3182191997 >> v1:172.21.15.204:6789/0 conn(0x7fd45c0712a0 legacy=0x7fd45c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 -- 172.21.15.143:0/3182191997 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c06a800 con 0x7fd45c15d1f0 2024-01-15T18:16:05.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459400640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c06bb70 0x7fd45c070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:05.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.162+0000 7fd459e00640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c15d1f0 0x7fd45c06b630 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249ced7862e1b58a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:05.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd458a00640 1 -- 172.21.15.143:0/3182191997 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0aa5f0 con 0x7fd45c15d1f0 2024-01-15T18:16:05.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd458a00640 1 -- 172.21.15.143:0/3182191997 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0aa750 con 0x7fd45c15d1f0 2024-01-15T18:16:05.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd458a00640 1 -- 172.21.15.143:0/3182191997 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0aaae0 con 0x7fd45c15d1f0 2024-01-15T18:16:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 -- 172.21.15.143:0/3182191997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c15d1f0 msgr2=0x7fd45c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c15d1f0 0x7fd45c06b630 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 -- 172.21.15.143:0/3182191997 shutdown_connections 2024-01-15T18:16:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c06bb70 0x7fd45c070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 --2- 172.21.15.143:0/3182191997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c15d1f0 0x7fd45c06b630 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 -- 172.21.15.143:0/3182191997 >> 172.21.15.143:0/3182191997 conn(0x7fd45c0b1610 msgr2=0x7fd45c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 -- 172.21.15.143:0/3182191997 shutdown_connections 2024-01-15T18:16:05.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 -- 172.21.15.143:0/3182191997 wait complete. 2024-01-15T18:16:05.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 Processor -- start 2024-01-15T18:16:05.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.166+0000 7fd461200640 1 -- start start 2024-01-15T18:16:05.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 0x7fd45c14ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:05.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c0712a0 0x7fd45c14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:05.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd45c15d1f0 0x7fd45c1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:05.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c147b50 con 0x7fd45c06bb70 2024-01-15T18:16:05.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c150000 con 0x7fd45c15d1f0 2024-01-15T18:16:05.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c1502e0 con 0x7fd45c0712a0 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 0x7fd45c14ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 0x7fd45c14ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33646/0 (socket says 172.21.15.143:33646) 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 -- 172.21.15.143:0/1339145006 learned_addr learned my addr 172.21.15.143:0/1339145006 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459400640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c0712a0 0x7fd45c14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd45a800640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd45c15d1f0 0x7fd45c1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 -- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c0712a0 msgr2=0x7fd45c14f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c0712a0 0x7fd45c14f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 -- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd45c15d1f0 msgr2=0x7fd45c1487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd45c15d1f0 0x7fd45c1487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 -- 172.21.15.143:0/1339145006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0094d0 con 0x7fd45c06bb70 2024-01-15T18:16:05.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd459e00640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 0x7fd45c14ef60 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0ac040 tx=0x7fd44c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0b3800 con 0x7fd45c06bb70 2024-01-15T18:16:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0b3980 con 0x7fd45c06bb70 2024-01-15T18:16:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c147ce0 con 0x7fd45c06bb70 2024-01-15T18:16:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0b3d30 con 0x7fd45c06bb70 2024-01-15T18:16:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c1482a0 con 0x7fd45c06bb70 2024-01-15T18:16:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c074b00 con 0x7fd45c06bb70 2024-01-15T18:16:05.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.170+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fd44c0c6390 con 0x7fd45c06bb70 2024-01-15T18:16:05.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.174+0000 7fd452000640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4340675e0 0x7fd434069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:05.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.174+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd44c0a7070 con 0x7fd45c06bb70 2024-01-15T18:16:05.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.174+0000 7fd459400640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4340675e0 0x7fd434069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:05.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.174+0000 7fd459400640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4340675e0 0x7fd434069aa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd45c14c830 tx=0x7fd444008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:05.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.178+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd44c0ec600 con 0x7fd45c06bb70 2024-01-15T18:16:05.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.358+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd45c148490 con 0x7fd45c06bb70 2024-01-15T18:16:05.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.994+0000 7fd452000640 1 -- 172.21.15.143:0/1339145006 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd44c0efc20 con 0x7fd45c06bb70 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4340675e0 msgr2=0x7fd434069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4340675e0 0x7fd434069aa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd45c14c830 tx=0x7fd444008040 comp rx=0 tx=0).stop 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 msgr2=0x7fd45c14ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 0x7fd45c14ef60 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0ac040 tx=0x7fd44c01ad10 comp rx=0 tx=0).stop 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 shutdown_connections 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4340675e0 0x7fd434069aa0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd45c15d1f0 0x7fd45c1487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd45c0712a0 0x7fd45c14f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 --2- 172.21.15.143:0/1339145006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd45c06bb70 0x7fd45c14ef60 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 >> 172.21.15.143:0/1339145006 conn(0x7fd45c0b1610 msgr2=0x7fd45c0cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:05.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 shutdown_connections 2024-01-15T18:16:05.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:05.998+0000 7fd461200640 1 -- 172.21.15.143:0/1339145006 wait complete. 2024-01-15T18:16:06.019 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:16:06.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:16:06.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 Processor -- start 2024-01-15T18:16:06.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 -- start start 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50f40ce560 0x7f50f415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f50f4071d00 0x7f50f4072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f4072650 con 0x7f50f4141660 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f415fbb0 con 0x7f50f40ce560 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50fbc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f415f310 con 0x7f50f4071d00 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50f40ce560 0x7f50f415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f9200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f50f4141660 0x7f50f4141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51932/0 (socket says 172.21.15.143:51932) 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f50f40ce560 0x7f50f415ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33656/0 (socket says 172.21.15.143:33656) 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f9200640 1 -- 172.21.15.143:0/1214476447 learned_addr learned my addr 172.21.15.143:0/1214476447 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f9c00640 1 --2- 172.21.15.143:0/1214476447 >> v2:172.21.15.204:3300/0 conn(0x7f50f4071d00 0x7f50f4072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3e00640 1 -- 172.21.15.143:0/1214476447 >> v2:172.21.15.204:3300/0 conn(0x7f50f4071d00 msgr2=0x7f50f4072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3e00640 1 --2- 172.21.15.143:0/1214476447 >> v2:172.21.15.204:3300/0 conn(0x7f50f4071d00 0x7f50f4072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3e00640 1 -- 172.21.15.143:0/1214476447 >> v1:172.21.15.143:6789/0 conn(0x7f50f4141660 legacy=0x7f50f4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3400640 1 -- 172.21.15.143:0/1214476447 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712965231 0 0) 0x7f50e4001120 con 0x7f50f4141660 2024-01-15T18:16:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f3e00640 1 -- 172.21.15.143:0/1214476447 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f415f5f0 con 0x7f50f40ce560 2024-01-15T18:16:06.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.162+0000 7f50f9c00640 1 --2- 172.21.15.143:0/1214476447 >> v2:172.21.15.204:3300/0 conn(0x7f50f4071d00 0x7f50f4072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50f3e00640 1 --2- 172.21.15.143:0/1214476447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f40ce560 0x7f50f415ec80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f50f406e250 tx=0x7f50dc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34554cec4705fad8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50f3400640 1 -- 172.21.15.143:0/1214476447 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc004230 con 0x7f50f40ce560 2024-01-15T18:16:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50f3400640 1 -- 172.21.15.143:0/1214476447 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc0a7050 con 0x7f50f40ce560 2024-01-15T18:16:06.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50f3400640 1 -- 172.21.15.143:0/1214476447 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc003620 con 0x7f50f40ce560 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 -- 172.21.15.143:0/1214476447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f40ce560 msgr2=0x7f50f415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 --2- 172.21.15.143:0/1214476447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f40ce560 0x7f50f415ec80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f50f406e250 tx=0x7f50dc01c460 comp rx=0 tx=0).stop 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 -- 172.21.15.143:0/1214476447 shutdown_connections 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 --2- 172.21.15.143:0/1214476447 >> v2:172.21.15.204:3300/0 conn(0x7f50f4071d00 0x7f50f4072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 --2- 172.21.15.143:0/1214476447 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f40ce560 0x7f50f415ec80 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 -- 172.21.15.143:0/1214476447 >> 172.21.15.143:0/1214476447 conn(0x7f50f406c3f0 msgr2=0x7f50f406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 -- 172.21.15.143:0/1214476447 shutdown_connections 2024-01-15T18:16:06.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.166+0000 7f50fbc00640 1 -- 172.21.15.143:0/1214476447 wait complete. 2024-01-15T18:16:06.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 Processor -- start 2024-01-15T18:16:06.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- start start 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 0x7f50f415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f40ce560 0x7f50f41349c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50f4141660 0x7f50f4134f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f40b34c0 con 0x7f50f4071d00 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4136030 con 0x7f50f40ce560 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f415a680 con 0x7f50f4141660 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 0x7f50f415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 0x7f50f415a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33666/0 (socket says 172.21.15.143:33666) 2024-01-15T18:16:06.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 -- 172.21.15.143:0/3125103431 learned_addr learned my addr 172.21.15.143:0/3125103431 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f3e00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f40ce560 0x7f50f41349c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9c00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50f4141660 0x7f50f4134f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 -- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50f4141660 msgr2=0x7f50f4134f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50f4141660 0x7f50f4134f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 -- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f40ce560 msgr2=0x7f50f41349c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f40ce560 0x7f50f41349c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 -- 172.21.15.143:0/3125103431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc009420 con 0x7f50f4071d00 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f9200640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 0x7f50f415a140 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f50e400c480 tx=0x7f50e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e40074e0 con 0x7f50f4071d00 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4007660 con 0x7f50f4071d00 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4004a30 con 0x7f50f4071d00 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f40cd150 con 0x7f50f4071d00 2024-01-15T18:16:06.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f4135a70 con 0x7f50f4071d00 2024-01-15T18:16:06.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.170+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f4067500 con 0x7f50f4071d00 2024-01-15T18:16:06.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.174+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f50e4004b90 con 0x7f50f4071d00 2024-01-15T18:16:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.174+0000 7f50f0c00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50cc0675e0 0x7f50cc069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.174+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f50e4081e50 con 0x7f50f4071d00 2024-01-15T18:16:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.174+0000 7f50f3e00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50cc0675e0 0x7f50cc069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.174+0000 7f50f3e00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50cc0675e0 0x7f50cc069aa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0ad000 tx=0x7f50dc0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.178+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50e4050b90 con 0x7f50f4071d00 2024-01-15T18:16:06.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.358+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50f4072650 con 0x7f50f4071d00 2024-01-15T18:16:06.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.358+0000 7f50f0c00640 1 -- 172.21.15.143:0/3125103431 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f50f4072650 con 0x7f50f4071d00 2024-01-15T18:16:06.355 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:06.355 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1584800432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1584800432},{"type":"v1","addr":"172.21.15.204:6834","nonce":1584800432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7843,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3526494826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3526494826},{"type":"v1","addr":"172.21.15.204:6835","nonce":3526494826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1307769037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1307769037},{"type":"v1","addr":"172.21.15.143:6835","nonce":1307769037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:16:02.335736+0000","modified":"2024-01-15T18:16:05.360719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/1230123305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1230123305},{"type":"v1","addr":"172.21.15.143:6837","nonce":1230123305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-01-15T18:16:06.356 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 52 2024-01-15T18:16:06.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.358+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50cc0675e0 msgr2=0x7f50cc069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.358+0000 7f50fbc00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50cc0675e0 0x7f50cc069aa0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0ad000 tx=0x7f50dc0a6000 comp rx=0 tx=0).stop 2024-01-15T18:16:06.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.358+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 msgr2=0x7f50f415a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.358+0000 7f50fbc00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 0x7f50f415a140 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f50e400c480 tx=0x7f50e400c940 comp rx=0 tx=0).stop 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 shutdown_connections 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f50cc0675e0 0x7f50cc069aa0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50f4141660 0x7f50f4134f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50f40ce560 0x7f50f41349c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 --2- 172.21.15.143:0/3125103431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50f4071d00 0x7f50f415a140 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 >> 172.21.15.143:0/3125103431 conn(0x7f50f406c3f0 msgr2=0x7f50f40af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 shutdown_connections 2024-01-15T18:16:06.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.362+0000 7f50fbc00640 1 -- 172.21.15.143:0/3125103431 wait complete. 2024-01-15T18:16:06.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:16:06.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.514+0000 7f7ee7800640 1 Processor -- start 2024-01-15T18:16:06.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee7800640 1 -- start start 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee7800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7ee0141660 0x7f7ee00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee7800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7ee0071d00 0x7f7ee0072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee7800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee0072650 con 0x7f7ee015ec80 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee7800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee015fbb0 con 0x7f7ee0071d00 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee7800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee015f130 con 0x7f7ee0141660 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7ee0071d00 0x7f7ee0072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7ee0071d00 0x7f7ee0072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33668/0 (socket says 172.21.15.143:33668) 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 -- 172.21.15.143:0/858240477 learned_addr learned my addr 172.21.15.143:0/858240477 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:06.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee4e00640 1 --2- 172.21.15.143:0/858240477 >> v2:172.21.15.204:3300/0 conn(0x7f7ee0141660 0x7f7ee00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 -- 172.21.15.143:0/858240477 >> v2:172.21.15.204:3300/0 conn(0x7f7ee0141660 msgr2=0x7f7ee00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:06.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 --2- 172.21.15.143:0/858240477 >> v2:172.21.15.204:3300/0 conn(0x7f7ee0141660 0x7f7ee00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 -- 172.21.15.143:0/858240477 >> v1:172.21.15.204:6789/0 conn(0x7f7ee015ec80 legacy=0x7f7ee0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.518+0000 7f7ee5800640 1 -- 172.21.15.143:0/858240477 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee015f310 con 0x7f7ee0071d00 2024-01-15T18:16:06.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee5800640 1 --2- 172.21.15.143:0/858240477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee0071d00 0x7f7ee0072110 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00b1aa0 tx=0x7f7ed801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d56621570c46b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7edf400640 1 -- 172.21.15.143:0/858240477 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80a5860 con 0x7f7ee0071d00 2024-01-15T18:16:06.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7edf400640 1 -- 172.21.15.143:0/858240477 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed80a59c0 con 0x7f7ee0071d00 2024-01-15T18:16:06.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7edf400640 1 -- 172.21.15.143:0/858240477 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80a5d50 con 0x7f7ee0071d00 2024-01-15T18:16:06.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 -- 172.21.15.143:0/858240477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee0071d00 msgr2=0x7f7ee0072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 --2- 172.21.15.143:0/858240477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee0071d00 0x7f7ee0072110 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00b1aa0 tx=0x7f7ed801c9a0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 -- 172.21.15.143:0/858240477 shutdown_connections 2024-01-15T18:16:06.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 --2- 172.21.15.143:0/858240477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee0071d00 0x7f7ee0072110 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 --2- 172.21.15.143:0/858240477 >> v2:172.21.15.204:3300/0 conn(0x7f7ee0141660 0x7f7ee00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 -- 172.21.15.143:0/858240477 >> 172.21.15.143:0/858240477 conn(0x7f7ee006c3f0 msgr2=0x7f7ee006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:06.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 -- 172.21.15.143:0/858240477 shutdown_connections 2024-01-15T18:16:06.520 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 -- 172.21.15.143:0/858240477 wait complete. 2024-01-15T18:16:06.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 Processor -- start 2024-01-15T18:16:06.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.522+0000 7f7ee7800640 1 -- start start 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ee0071d00 0x7f7ee015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee0141660 0x7f7ee013d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 0x7f7ee015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0159300 con 0x7f7ee015ec80 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee015aa80 con 0x7f7ee0141660 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee015a590 con 0x7f7ee0071d00 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 0x7f7ee015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 0x7f7ee015bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33678/0 (socket says 172.21.15.143:33678) 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 -- 172.21.15.143:0/1836456622 learned_addr learned my addr 172.21.15.143:0/1836456622 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee4e00640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ee0071d00 0x7f7ee015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.523 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7edfe00640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee0141660 0x7f7ee013d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 -- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ee0071d00 msgr2=0x7f7ee015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ee0071d00 0x7f7ee015a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 -- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee0141660 msgr2=0x7f7ee013d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee0141660 0x7f7ee013d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 -- 172.21.15.143:0/1836456622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed8009b30 con 0x7f7ee015ec80 2024-01-15T18:16:06.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee5800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 0x7f7ee015bb20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80af040 tx=0x7f7ed8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80b8670 con 0x7f7ee015ec80 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed80b87f0 con 0x7f7ee015ec80 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee0159570 con 0x7f7ee015ec80 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80b8ba0 con 0x7f7ee015ec80 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee0159b30 con 0x7f7ee015ec80 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.526+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee0159d20 con 0x7f7ee015ec80 2024-01-15T18:16:06.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.530+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f7ed80b8d00 con 0x7f7ee015ec80 2024-01-15T18:16:06.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.530+0000 7f7edcc00640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7eb8067610 0x7f7eb8069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.530+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7ed811e420 con 0x7f7ee015ec80 2024-01-15T18:16:06.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.530+0000 7f7ee4e00640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7eb8067610 0x7f7eb8069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.530+0000 7f7ee4e00640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7eb8067610 0x7f7eb8069ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0165f80 tx=0x7f7ed0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.534+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7ed80ecff0 con 0x7f7ee015ec80 2024-01-15T18:16:06.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.682+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ee0067500 con 0x7f7ee015ec80 2024-01-15T18:16:06.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.682+0000 7f7edcc00640 1 -- 172.21.15.143:0/1836456622 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16684 (secure 0 0 0) 0x7f7ed80bc020 con 0x7f7ee015ec80 2024-01-15T18:16:06.681 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:06.681 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":87,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:16:03.978037+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:16:00.088960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-15T18:16:01.336678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:16:06.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7eb8067610 msgr2=0x7f7eb8069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7eb8067610 0x7f7eb8069ad0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0165f80 tx=0x7f7ed0009190 comp rx=0 tx=0).stop 2024-01-15T18:16:06.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 msgr2=0x7f7ee015bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 0x7f7ee015bb20 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80af040 tx=0x7f7ed8019740 comp rx=0 tx=0).stop 2024-01-15T18:16:06.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 shutdown_connections 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7eb8067610 0x7f7eb8069ad0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ee015ec80 0x7f7ee015bb20 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ee0141660 0x7f7ee013d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 --2- 172.21.15.143:0/1836456622 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ee0071d00 0x7f7ee015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 >> 172.21.15.143:0/1836456622 conn(0x7f7ee006c3f0 msgr2=0x7f7ee00b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 shutdown_connections 2024-01-15T18:16:06.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.686+0000 7f7ee7800640 1 -- 172.21.15.143:0/1836456622 wait complete. 2024-01-15T18:16:06.704 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:16:06.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:16:06.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.842+0000 7f9441a00640 1 Processor -- start 2024-01-15T18:16:06.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.842+0000 7f9441a00640 1 -- start start 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f9441a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f943c1607c0 0x7f943c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f9441a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0cffd0 con 0x7f943c0cfb30 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f9441a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c0d02b0 con 0x7f943c0cbf50 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f9441a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c0d1ce0 con 0x7f943c1607c0 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f943aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f943c1607c0 0x7f943c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f943aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f943c1607c0 0x7f943c06b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33686/0 (socket says 172.21.15.143:33686) 2024-01-15T18:16:06.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f943aa00640 1 -- 172.21.15.143:0/1852502646 learned_addr learned my addr 172.21.15.143:0/1852502646 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:06.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f943aa00640 1 -- 172.21.15.143:0/1852502646 >> v1:172.21.15.204:6789/0 conn(0x7f943c0cbf50 legacy=0x7f943c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f943aa00640 1 -- 172.21.15.143:0/1852502646 >> v1:172.21.15.143:6789/0 conn(0x7f943c0cfb30 legacy=0x7f943c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.846+0000 7f943aa00640 1 -- 172.21.15.143:0/1852502646 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c0cb470 con 0x7f943c1607c0 2024-01-15T18:16:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f943aa00640 1 --2- 172.21.15.143:0/1852502646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c1607c0 0x7f943c06b7b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f942c009870 tx=0x7f942c01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6130efa5a2c8a247 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9439600640 1 -- 172.21.15.143:0/1852502646 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0a8a30 con 0x7f943c1607c0 2024-01-15T18:16:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9439600640 1 -- 172.21.15.143:0/1852502646 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0a8b90 con 0x7f943c1607c0 2024-01-15T18:16:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9439600640 1 -- 172.21.15.143:0/1852502646 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0b0440 con 0x7f943c1607c0 2024-01-15T18:16:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 -- 172.21.15.143:0/1852502646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c1607c0 msgr2=0x7f943c06b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 --2- 172.21.15.143:0/1852502646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c1607c0 0x7f943c06b7b0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f942c009870 tx=0x7f942c01ed30 comp rx=0 tx=0).stop 2024-01-15T18:16:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 -- 172.21.15.143:0/1852502646 shutdown_connections 2024-01-15T18:16:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 --2- 172.21.15.143:0/1852502646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c1607c0 0x7f943c06b7b0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 -- 172.21.15.143:0/1852502646 >> 172.21.15.143:0/1852502646 conn(0x7f943c0d0e20 msgr2=0x7f943c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 -- 172.21.15.143:0/1852502646 shutdown_connections 2024-01-15T18:16:06.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 -- 172.21.15.143:0/1852502646 wait complete. 2024-01-15T18:16:06.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 Processor -- start 2024-01-15T18:16:06.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.850+0000 7f9441a00640 1 -- start start 2024-01-15T18:16:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f943c0cbf50 0x7f943c06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 0x7f943c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943c1607c0 0x7f943c1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c150d10 con 0x7f943c0cfb30 2024-01-15T18:16:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c150fc0 con 0x7f943c0cbf50 2024-01-15T18:16:06.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c1512a0 con 0x7f943c1607c0 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 0x7f943c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 0x7f943c150290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33688/0 (socket says 172.21.15.143:33688) 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 -- 172.21.15.143:0/2404123131 learned_addr learned my addr 172.21.15.143:0/2404123131 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f943c0cbf50 0x7f943c06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943b400640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943c1607c0 0x7f943c1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 -- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943c1607c0 msgr2=0x7f943c1507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943c1607c0 0x7f943c1507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 -- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f943c0cbf50 msgr2=0x7f943c06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f943c0cbf50 0x7f943c06bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 -- 172.21.15.143:0/2404123131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0094d0 con 0x7f943c0cfb30 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943aa00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f943c0cbf50 0x7f943c06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:06.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f943a000640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 0x7f943c150290 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f942400c500 tx=0x7f942400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424005220 con 0x7f943c0cfb30 2024-01-15T18:16:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424005380 con 0x7f943c0cfb30 2024-01-15T18:16:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424007270 con 0x7f943c0cfb30 2024-01-15T18:16:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c151580 con 0x7f943c0cfb30 2024-01-15T18:16:06.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c151b40 con 0x7f943c0cfb30 2024-01-15T18:16:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c06d4b0 con 0x7f943c0cfb30 2024-01-15T18:16:06.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.854+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f94240073d0 con 0x7f943c0cfb30 2024-01-15T18:16:06.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.858+0000 7f9432a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94140675e0 0x7f9414069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:06.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.858+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9424082660 con 0x7f943c0cfb30 2024-01-15T18:16:06.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.858+0000 7f943aa00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94140675e0 0x7f9414069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:06.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.858+0000 7f943aa00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94140675e0 0x7f9414069aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f942c0b2000 tx=0x7f942c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:06.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:06.862+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9424051380 con 0x7f943c0cfb30 2024-01-15T18:16:07.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.014+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f943c0b6d90 con 0x7f94140675e0 2024-01-15T18:16:07.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.022+0000 7f9432a00640 1 -- 172.21.15.143:0/2404123131 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285130 (secure 0 0 0) 0x7f943c0b6d90 con 0x7f94140675e0 2024-01-15T18:16:07.024 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:07.026 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:16:07.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.026+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94140675e0 msgr2=0x7f9414069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.026+0000 7f9441a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94140675e0 0x7f9414069aa0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f942c0b2000 tx=0x7f942c009040 comp rx=0 tx=0).stop 2024-01-15T18:16:07.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.026+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 msgr2=0x7f943c150290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.026+0000 7f9441a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 0x7f943c150290 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f942400c500 tx=0x7f942400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 shutdown_connections 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94140675e0 0x7f9414069aa0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f943c1607c0 0x7f943c1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f943c0cfb30 0x7f943c150290 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 --2- 172.21.15.143:0/2404123131 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f943c0cbf50 0x7f943c06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 >> 172.21.15.143:0/2404123131 conn(0x7f943c0d0e20 msgr2=0x7f943c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 shutdown_connections 2024-01-15T18:16:07.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.030+0000 7f9441a00640 1 -- 172.21.15.143:0/2404123131 wait complete. 2024-01-15T18:16:07.045 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-01-15T18:16:06.858046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":591524,"kb_used_data":9400,"kb_used_omap":12,"kb_used_meta":582003,"kb_avail":754383196,"statfs":{"total":773094113280,"available":772488392704,"internally_reserved":0,"allocated":9625600,"data_stored":4781846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":595971632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913200"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172275+0000","last_change":"2024-01-15T18:16:02.977971+0000","last_active":"2024-01-15T18:16:04.172275+0000","last_peered":"2024-01-15T18:16:04.172275+0000","last_clean":"2024-01-15T18:16:04.172275+0000","last_became_active":"2024-01-15T18:16:02.977706+0000","last_became_peered":"2024-01-15T18:16:02.977706+0000","last_unstale":"2024-01-15T18:16:04.172275+0000","last_undegraded":"2024-01-15T18:16:04.172275+0000","last_fullsized":"2024-01-15T18:16:04.172275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958822+0000","last_change":"2024-01-15T18:16:01.958822+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958822+0000","last_undegraded":"2024-01-15T18:16:01.958822+0000","last_fullsized":"2024-01-15T18:16:01.958822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983752+0000","last_change":"2024-01-15T18:16:02.982423+0000","last_active":"2024-01-15T18:16:03.983752+0000","last_peered":"2024-01-15T18:16:03.983752+0000","last_clean":"2024-01-15T18:16:03.983752+0000","last_became_active":"2024-01-15T18:16:02.981906+0000","last_became_peered":"2024-01-15T18:16:02.981906+0000","last_unstale":"2024-01-15T18:16:03.983752+0000","last_undegraded":"2024-01-15T18:16:03.983752+0000","last_fullsized":"2024-01-15T18:16:03.983752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:02.961035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.988011+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.987583+0000","last_became_peered":"2024-01-15T18:16:02.987583+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672319+0000","last_change":"2024-01-15T18:16:02.984894+0000","last_active":"2024-01-15T18:16:04.672319+0000","last_peered":"2024-01-15T18:16:04.672319+0000","last_clean":"2024-01-15T18:16:04.672319+0000","last_became_active":"2024-01-15T18:16:02.984537+0000","last_became_peered":"2024-01-15T18:16:02.984537+0000","last_unstale":"2024-01-15T18:16:04.672319+0000","last_undegraded":"2024-01-15T18:16:04.672319+0000","last_fullsized":"2024-01-15T18:16:04.672319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672358+0000","last_change":"2024-01-15T18:16:02.988234+0000","last_active":"2024-01-15T18:16:04.672358+0000","last_peered":"2024-01-15T18:16:04.672358+0000","last_clean":"2024-01-15T18:16:04.672358+0000","last_became_active":"2024-01-15T18:16:02.987822+0000","last_became_peered":"2024-01-15T18:16:02.987822+0000","last_unstale":"2024-01-15T18:16:04.672358+0000","last_undegraded":"2024-01-15T18:16:04.672358+0000","last_fullsized":"2024-01-15T18:16:04.672358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672449+0000","last_change":"2024-01-15T18:16:02.978613+0000","last_active":"2024-01-15T18:16:04.672449+0000","last_peered":"2024-01-15T18:16:04.672449+0000","last_clean":"2024-01-15T18:16:04.672449+0000","last_became_active":"2024-01-15T18:16:02.978331+0000","last_became_peered":"2024-01-15T18:16:02.978331+0000","last_unstale":"2024-01-15T18:16:04.672449+0000","last_undegraded":"2024-01-15T18:16:04.672449+0000","last_fullsized":"2024-01-15T18:16:04.672449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.985584+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.984527+0000","last_became_peered":"2024-01-15T18:16:02.984527+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.960055+0000","last_change":"2024-01-15T18:16:01.960055+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.960055+0000","last_undegraded":"2024-01-15T18:16:01.960055+0000","last_fullsized":"2024-01-15T18:16:01.960055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.171943+0000","last_change":"2024-01-15T18:16:02.976922+0000","last_active":"2024-01-15T18:16:04.171943+0000","last_peered":"2024-01-15T18:16:04.171943+0000","last_clean":"2024-01-15T18:16:04.171943+0000","last_became_active":"2024-01-15T18:16:02.976638+0000","last_became_peered":"2024-01-15T18:16:02.976638+0000","last_unstale":"2024-01-15T18:16:04.171943+0000","last_undegraded":"2024-01-15T18:16:04.171943+0000","last_fullsized":"2024-01-15T18:16:04.171943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:28.008839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.980605+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.978977+0000","last_became_peered":"2024-01-15T18:16:02.978977+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:17.866154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954608+0000","last_change":"2024-01-15T18:16:01.954608+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954608+0000","last_undegraded":"2024-01-15T18:16:01.954608+0000","last_fullsized":"2024-01-15T18:16:01.954608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982727+0000","last_change":"2024-01-15T18:16:02.979861+0000","last_active":"2024-01-15T18:16:03.982727+0000","last_peered":"2024-01-15T18:16:03.982727+0000","last_clean":"2024-01-15T18:16:03.982727+0000","last_became_active":"2024-01-15T18:16:02.979569+0000","last_became_peered":"2024-01-15T18:16:02.979569+0000","last_unstale":"2024-01-15T18:16:03.982727+0000","last_undegraded":"2024-01-15T18:16:03.982727+0000","last_fullsized":"2024-01-15T18:16:03.982727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984023+0000","last_change":"2024-01-15T18:16:02.977093+0000","last_active":"2024-01-15T18:16:03.984023+0000","last_peered":"2024-01-15T18:16:03.984023+0000","last_clean":"2024-01-15T18:16:03.984023+0000","last_became_active":"2024-01-15T18:16:02.974914+0000","last_became_peered":"2024-01-15T18:16:02.974914+0000","last_unstale":"2024-01-15T18:16:03.984023+0000","last_undegraded":"2024-01-15T18:16:03.984023+0000","last_fullsized":"2024-01-15T18:16:03.984023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.961445+0000","last_change":"2024-01-15T18:16:01.961445+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.961445+0000","last_undegraded":"2024-01-15T18:16:01.961445+0000","last_fullsized":"2024-01-15T18:16:01.961445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957359+0000","last_change":"2024-01-15T18:16:01.957359+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957359+0000","last_undegraded":"2024-01-15T18:16:01.957359+0000","last_fullsized":"2024-01-15T18:16:01.957359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172017+0000","last_change":"2024-01-15T18:16:02.970218+0000","last_active":"2024-01-15T18:16:04.172017+0000","last_peered":"2024-01-15T18:16:04.172017+0000","last_clean":"2024-01-15T18:16:04.172017+0000","last_became_active":"2024-01-15T18:16:02.969901+0000","last_became_peered":"2024-01-15T18:16:02.969901+0000","last_unstale":"2024-01-15T18:16:04.172017+0000","last_undegraded":"2024-01-15T18:16:04.172017+0000","last_fullsized":"2024-01-15T18:16:04.172017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:31.551846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958084+0000","last_change":"2024-01-15T18:16:01.958084+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958084+0000","last_undegraded":"2024-01-15T18:16:01.958084+0000","last_fullsized":"2024-01-15T18:16:01.958084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983932+0000","last_change":"2024-01-15T18:16:02.991391+0000","last_active":"2024-01-15T18:16:03.983932+0000","last_peered":"2024-01-15T18:16:03.983932+0000","last_clean":"2024-01-15T18:16:03.983932+0000","last_became_active":"2024-01-15T18:16:02.991138+0000","last_became_peered":"2024-01-15T18:16:02.991138+0000","last_unstale":"2024-01-15T18:16:03.983932+0000","last_undegraded":"2024-01-15T18:16:03.983932+0000","last_fullsized":"2024-01-15T18:16:03.983932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:06.968039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983406+0000","last_change":"2024-01-15T18:16:02.976507+0000","last_active":"2024-01-15T18:16:03.983406+0000","last_peered":"2024-01-15T18:16:03.983406+0000","last_clean":"2024-01-15T18:16:03.983406+0000","last_became_active":"2024-01-15T18:16:02.975765+0000","last_became_peered":"2024-01-15T18:16:02.975765+0000","last_unstale":"2024-01-15T18:16:03.983406+0000","last_undegraded":"2024-01-15T18:16:03.983406+0000","last_fullsized":"2024-01-15T18:16:03.983406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172094+0000","last_change":"2024-01-15T18:16:02.976103+0000","last_active":"2024-01-15T18:16:04.172094+0000","last_peered":"2024-01-15T18:16:04.172094+0000","last_clean":"2024-01-15T18:16:04.172094+0000","last_became_active":"2024-01-15T18:16:02.975829+0000","last_became_peered":"2024-01-15T18:16:02.975829+0000","last_unstale":"2024-01-15T18:16:04.172094+0000","last_undegraded":"2024-01-15T18:16:04.172094+0000","last_fullsized":"2024-01-15T18:16:04.172094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956707+0000","last_change":"2024-01-15T18:16:01.956707+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956707+0000","last_undegraded":"2024-01-15T18:16:01.956707+0000","last_fullsized":"2024-01-15T18:16:01.956707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956925+0000","last_change":"2024-01-15T18:16:01.956925+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956925+0000","last_undegraded":"2024-01-15T18:16:01.956925+0000","last_fullsized":"2024-01-15T18:16:01.956925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982746+0000","last_change":"2024-01-15T18:16:02.974470+0000","last_active":"2024-01-15T18:16:03.982746+0000","last_peered":"2024-01-15T18:16:03.982746+0000","last_clean":"2024-01-15T18:16:03.982746+0000","last_became_active":"2024-01-15T18:16:02.974162+0000","last_became_peered":"2024-01-15T18:16:02.974162+0000","last_unstale":"2024-01-15T18:16:03.982746+0000","last_undegraded":"2024-01-15T18:16:03.982746+0000","last_fullsized":"2024-01-15T18:16:03.982746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172002+0000","last_change":"2024-01-15T18:16:02.986920+0000","last_active":"2024-01-15T18:16:04.172002+0000","last_peered":"2024-01-15T18:16:04.172002+0000","last_clean":"2024-01-15T18:16:04.172002+0000","last_became_active":"2024-01-15T18:16:02.986544+0000","last_became_peered":"2024-01-15T18:16:02.986544+0000","last_unstale":"2024-01-15T18:16:04.172002+0000","last_undegraded":"2024-01-15T18:16:04.172002+0000","last_fullsized":"2024-01-15T18:16:04.172002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:06.358800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983627+0000","last_change":"2024-01-15T18:16:02.981015+0000","last_active":"2024-01-15T18:16:03.983627+0000","last_peered":"2024-01-15T18:16:03.983627+0000","last_clean":"2024-01-15T18:16:03.983627+0000","last_became_active":"2024-01-15T18:16:02.979886+0000","last_became_peered":"2024-01-15T18:16:02.979886+0000","last_unstale":"2024-01-15T18:16:03.983627+0000","last_undegraded":"2024-01-15T18:16:03.983627+0000","last_fullsized":"2024-01-15T18:16:03.983627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:23.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004361+0000","last_change":"2024-01-15T18:16:01.968674+0000","last_active":"2024-01-15T18:16:03.004361+0000","last_peered":"2024-01-15T18:16:03.004361+0000","last_clean":"2024-01-15T18:16:03.004361+0000","last_became_active":"2024-01-15T18:16:01.967874+0000","last_became_peered":"2024-01-15T18:16:01.967874+0000","last_unstale":"2024-01-15T18:16:03.004361+0000","last_undegraded":"2024-01-15T18:16:03.004361+0000","last_fullsized":"2024-01-15T18:16:03.004361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172367+0000","last_change":"2024-01-15T18:16:02.971695+0000","last_active":"2024-01-15T18:16:04.172367+0000","last_peered":"2024-01-15T18:16:04.172367+0000","last_clean":"2024-01-15T18:16:04.172367+0000","last_became_active":"2024-01-15T18:16:02.971425+0000","last_became_peered":"2024-01-15T18:16:02.971425+0000","last_unstale":"2024-01-15T18:16:04.172367+0000","last_undegraded":"2024-01-15T18:16:04.172367+0000","last_fullsized":"2024-01-15T18:16:04.172367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002366+0000","last_change":"2024-01-15T18:16:01.960880+0000","last_active":"2024-01-15T18:16:03.002366+0000","last_peered":"2024-01-15T18:16:03.002366+0000","last_clean":"2024-01-15T18:16:03.002366+0000","last_became_active":"2024-01-15T18:16:01.960424+0000","last_became_peered":"2024-01-15T18:16:01.960424+0000","last_unstale":"2024-01-15T18:16:03.002366+0000","last_undegraded":"2024-01-15T18:16:03.002366+0000","last_fullsized":"2024-01-15T18:16:03.002366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:05.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172293+0000","last_change":"2024-01-15T18:16:02.987087+0000","last_active":"2024-01-15T18:16:04.172293+0000","last_peered":"2024-01-15T18:16:04.172293+0000","last_clean":"2024-01-15T18:16:04.172293+0000","last_became_active":"2024-01-15T18:16:02.986595+0000","last_became_peered":"2024-01-15T18:16:02.986595+0000","last_unstale":"2024-01-15T18:16:04.172293+0000","last_undegraded":"2024-01-15T18:16:04.172293+0000","last_fullsized":"2024-01-15T18:16:04.172293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973629+0000","last_change":"2024-01-15T18:16:01.972650+0000","last_active":"2024-01-15T18:16:01.973629+0000","last_peered":"2024-01-15T18:16:01.973629+0000","last_clean":"2024-01-15T18:16:01.973629+0000","last_became_active":"2024-01-15T18:16:01.972282+0000","last_became_peered":"2024-01-15T18:16:01.972282+0000","last_unstale":"2024-01-15T18:16:01.973629+0000","last_undegraded":"2024-01-15T18:16:01.973629+0000","last_fullsized":"2024-01-15T18:16:01.973629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984011+0000","last_change":"2024-01-15T18:16:02.980841+0000","last_active":"2024-01-15T18:16:03.984011+0000","last_peered":"2024-01-15T18:16:03.984011+0000","last_clean":"2024-01-15T18:16:03.984011+0000","last_became_active":"2024-01-15T18:16:02.980529+0000","last_became_peered":"2024-01-15T18:16:02.980529+0000","last_unstale":"2024-01-15T18:16:03.984011+0000","last_undegraded":"2024-01-15T18:16:03.984011+0000","last_fullsized":"2024-01-15T18:16:03.984011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:16.698507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.974689+0000","last_change":"2024-01-15T18:16:01.974052+0000","last_active":"2024-01-15T18:16:01.974689+0000","last_peered":"2024-01-15T18:16:01.974689+0000","last_clean":"2024-01-15T18:16:01.974689+0000","last_became_active":"2024-01-15T18:16:01.973571+0000","last_became_peered":"2024-01-15T18:16:01.973571+0000","last_unstale":"2024-01-15T18:16:01.974689+0000","last_undegraded":"2024-01-15T18:16:01.974689+0000","last_fullsized":"2024-01-15T18:16:01.974689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956466+0000","last_change":"2024-01-15T18:16:01.956466+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956466+0000","last_undegraded":"2024-01-15T18:16:01.956466+0000","last_fullsized":"2024-01-15T18:16:01.956466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002241+0000","last_change":"2024-01-15T18:16:01.961667+0000","last_active":"2024-01-15T18:16:03.002241+0000","last_peered":"2024-01-15T18:16:03.002241+0000","last_clean":"2024-01-15T18:16:03.002241+0000","last_became_active":"2024-01-15T18:16:01.961336+0000","last_became_peered":"2024-01-15T18:16:01.961336+0000","last_unstale":"2024-01-15T18:16:03.002241+0000","last_undegraded":"2024-01-15T18:16:03.002241+0000","last_fullsized":"2024-01-15T18:16:03.002241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.970823+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.970523+0000","last_became_peered":"2024-01-15T18:16:02.970523+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000246+0000","last_change":"2024-01-15T18:16:01.958649+0000","last_active":"2024-01-15T18:16:03.000246+0000","last_peered":"2024-01-15T18:16:03.000246+0000","last_clean":"2024-01-15T18:16:03.000246+0000","last_became_active":"2024-01-15T18:16:01.958043+0000","last_became_peered":"2024-01-15T18:16:01.958043+0000","last_unstale":"2024-01-15T18:16:03.000246+0000","last_undegraded":"2024-01-15T18:16:03.000246+0000","last_fullsized":"2024-01-15T18:16:03.000246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983181+0000","last_change":"2024-01-15T18:16:02.983728+0000","last_active":"2024-01-15T18:16:03.983181+0000","last_peered":"2024-01-15T18:16:03.983181+0000","last_clean":"2024-01-15T18:16:03.983181+0000","last_became_active":"2024-01-15T18:16:02.981960+0000","last_became_peered":"2024-01-15T18:16:02.981960+0000","last_unstale":"2024-01-15T18:16:03.983181+0000","last_undegraded":"2024-01-15T18:16:03.983181+0000","last_fullsized":"2024-01-15T18:16:03.983181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.971514+0000","last_change":"2024-01-15T18:16:01.968471+0000","last_active":"2024-01-15T18:16:01.971514+0000","last_peered":"2024-01-15T18:16:01.971514+0000","last_clean":"2024-01-15T18:16:01.971514+0000","last_became_active":"2024-01-15T18:16:01.967508+0000","last_became_peered":"2024-01-15T18:16:01.967508+0000","last_unstale":"2024-01-15T18:16:01.971514+0000","last_undegraded":"2024-01-15T18:16:01.971514+0000","last_fullsized":"2024-01-15T18:16:01.971514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:33.150205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954100+0000","last_change":"2024-01-15T18:16:01.954100+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954100+0000","last_undegraded":"2024-01-15T18:16:01.954100+0000","last_fullsized":"2024-01-15T18:16:01.954100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.954226+0000","last_change":"2024-01-15T18:16:01.954226+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954226+0000","last_undegraded":"2024-01-15T18:16:01.954226+0000","last_fullsized":"2024-01-15T18:16:01.954226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:51.202598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955399+0000","last_change":"2024-01-15T18:16:01.955399+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955399+0000","last_undegraded":"2024-01-15T18:16:01.955399+0000","last_fullsized":"2024-01-15T18:16:01.955399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.967235+0000","last_change":"2024-01-15T18:16:01.967948+0000","last_active":"2024-01-15T18:16:02.967235+0000","last_peered":"2024-01-15T18:16:02.967235+0000","last_clean":"2024-01-15T18:16:02.967235+0000","last_became_active":"2024-01-15T18:16:01.967662+0000","last_became_peered":"2024-01-15T18:16:01.967662+0000","last_unstale":"2024-01-15T18:16:02.967235+0000","last_undegraded":"2024-01-15T18:16:02.967235+0000","last_fullsized":"2024-01-15T18:16:02.967235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:28.859462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672459+0000","last_change":"2024-01-15T18:16:02.977816+0000","last_active":"2024-01-15T18:16:04.672459+0000","last_peered":"2024-01-15T18:16:04.672459+0000","last_clean":"2024-01-15T18:16:04.672459+0000","last_became_active":"2024-01-15T18:16:02.977488+0000","last_became_peered":"2024-01-15T18:16:02.977488+0000","last_unstale":"2024-01-15T18:16:04.672459+0000","last_undegraded":"2024-01-15T18:16:04.672459+0000","last_fullsized":"2024-01-15T18:16:04.672459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.966903+0000","last_change":"2024-01-15T18:16:01.970089+0000","last_active":"2024-01-15T18:16:02.966903+0000","last_peered":"2024-01-15T18:16:02.966903+0000","last_clean":"2024-01-15T18:16:02.966903+0000","last_became_active":"2024-01-15T18:16:01.969791+0000","last_became_peered":"2024-01-15T18:16:01.969791+0000","last_unstale":"2024-01-15T18:16:02.966903+0000","last_undegraded":"2024-01-15T18:16:02.966903+0000","last_fullsized":"2024-01-15T18:16:02.966903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:25.108362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956190+0000","last_change":"2024-01-15T18:16:01.956190+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956190+0000","last_undegraded":"2024-01-15T18:16:01.956190+0000","last_fullsized":"2024-01-15T18:16:01.956190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002343+0000","last_change":"2024-01-15T18:16:01.967367+0000","last_active":"2024-01-15T18:16:03.002343+0000","last_peered":"2024-01-15T18:16:03.002343+0000","last_clean":"2024-01-15T18:16:03.002343+0000","last_became_active":"2024-01-15T18:16:01.966610+0000","last_became_peered":"2024-01-15T18:16:01.966610+0000","last_unstale":"2024-01-15T18:16:03.002343+0000","last_undegraded":"2024-01-15T18:16:03.002343+0000","last_fullsized":"2024-01-15T18:16:03.002343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983502+0000","last_change":"2024-01-15T18:16:02.976324+0000","last_active":"2024-01-15T18:16:03.983502+0000","last_peered":"2024-01-15T18:16:03.983502+0000","last_clean":"2024-01-15T18:16:03.983502+0000","last_became_active":"2024-01-15T18:16:02.975452+0000","last_became_peered":"2024-01-15T18:16:02.975452+0000","last_unstale":"2024-01-15T18:16:03.983502+0000","last_undegraded":"2024-01-15T18:16:03.983502+0000","last_fullsized":"2024-01-15T18:16:03.983502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:31.636033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004546+0000","last_change":"2024-01-15T18:16:01.971530+0000","last_active":"2024-01-15T18:16:03.004546+0000","last_peered":"2024-01-15T18:16:03.004546+0000","last_clean":"2024-01-15T18:16:03.004546+0000","last_became_active":"2024-01-15T18:16:01.971080+0000","last_became_peered":"2024-01-15T18:16:01.971080+0000","last_unstale":"2024-01-15T18:16:03.004546+0000","last_undegraded":"2024-01-15T18:16:03.004546+0000","last_fullsized":"2024-01-15T18:16:03.004546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955769+0000","last_change":"2024-01-15T18:16:01.955769+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955769+0000","last_undegraded":"2024-01-15T18:16:01.955769+0000","last_fullsized":"2024-01-15T18:16:01.955769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968495+0000","last_change":"2024-01-15T18:16:01.968495+0000","last_active":"2024-01-15T18:16:01.968495+0000","last_peered":"2024-01-15T18:16:01.968495+0000","last_clean":"2024-01-15T18:16:01.968495+0000","last_became_active":"2024-01-15T18:16:01.967100+0000","last_became_peered":"2024-01-15T18:16:01.967100+0000","last_unstale":"2024-01-15T18:16:01.968495+0000","last_undegraded":"2024-01-15T18:16:01.968495+0000","last_fullsized":"2024-01-15T18:16:01.968495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:29.198637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983817+0000","last_change":"2024-01-15T18:16:02.985570+0000","last_active":"2024-01-15T18:16:03.983817+0000","last_peered":"2024-01-15T18:16:03.983817+0000","last_clean":"2024-01-15T18:16:03.983817+0000","last_became_active":"2024-01-15T18:16:02.985303+0000","last_became_peered":"2024-01-15T18:16:02.985303+0000","last_unstale":"2024-01-15T18:16:03.983817+0000","last_undegraded":"2024-01-15T18:16:03.983817+0000","last_fullsized":"2024-01-15T18:16:03.983817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:55.037442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968815+0000","last_change":"2024-01-15T18:16:01.965271+0000","last_active":"2024-01-15T18:16:01.968815+0000","last_peered":"2024-01-15T18:16:01.968815+0000","last_clean":"2024-01-15T18:16:01.968815+0000","last_became_active":"2024-01-15T18:16:01.964982+0000","last_became_peered":"2024-01-15T18:16:01.964982+0000","last_unstale":"2024-01-15T18:16:01.968815+0000","last_undegraded":"2024-01-15T18:16:01.968815+0000","last_fullsized":"2024-01-15T18:16:01.968815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957808+0000","last_change":"2024-01-15T18:16:01.957808+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957808+0000","last_undegraded":"2024-01-15T18:16:01.957808+0000","last_fullsized":"2024-01-15T18:16:01.957808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971256+0000","last_change":"2024-01-15T18:16:01.971126+0000","last_active":"2024-01-15T18:16:02.971256+0000","last_peered":"2024-01-15T18:16:02.971256+0000","last_clean":"2024-01-15T18:16:02.971256+0000","last_became_active":"2024-01-15T18:16:01.969238+0000","last_became_peered":"2024-01-15T18:16:01.969238+0000","last_unstale":"2024-01-15T18:16:02.971256+0000","last_undegraded":"2024-01-15T18:16:02.971256+0000","last_fullsized":"2024-01-15T18:16:02.971256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:30.543482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954474+0000","last_change":"2024-01-15T18:16:01.954474+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954474+0000","last_undegraded":"2024-01-15T18:16:01.954474+0000","last_fullsized":"2024-01-15T18:16:01.954474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980185+0000","last_change":"2024-01-15T18:16:01.979247+0000","last_active":"2024-01-15T18:16:01.980185+0000","last_peered":"2024-01-15T18:16:01.980185+0000","last_clean":"2024-01-15T18:16:01.980185+0000","last_became_active":"2024-01-15T18:16:01.978885+0000","last_became_peered":"2024-01-15T18:16:01.978885+0000","last_unstale":"2024-01-15T18:16:01.980185+0000","last_undegraded":"2024-01-15T18:16:01.980185+0000","last_fullsized":"2024-01-15T18:16:01.980185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:42.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984081+0000","last_change":"2024-01-15T18:16:02.983891+0000","last_active":"2024-01-15T18:16:03.984081+0000","last_peered":"2024-01-15T18:16:03.984081+0000","last_clean":"2024-01-15T18:16:03.984081+0000","last_became_active":"2024-01-15T18:16:02.983591+0000","last_became_peered":"2024-01-15T18:16:02.983591+0000","last_unstale":"2024-01-15T18:16:03.984081+0000","last_undegraded":"2024-01-15T18:16:03.984081+0000","last_fullsized":"2024-01-15T18:16:03.984081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:32.572896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.998942+0000","last_change":"2024-01-15T18:16:01.971805+0000","last_active":"2024-01-15T18:16:02.998942+0000","last_peered":"2024-01-15T18:16:02.998942+0000","last_clean":"2024-01-15T18:16:02.998942+0000","last_became_active":"2024-01-15T18:16:01.971492+0000","last_became_peered":"2024-01-15T18:16:01.971492+0000","last_unstale":"2024-01-15T18:16:02.998942+0000","last_undegraded":"2024-01-15T18:16:02.998942+0000","last_fullsized":"2024-01-15T18:16:02.998942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.974741+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.974455+0000","last_became_peered":"2024-01-15T18:16:02.974455+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973667+0000","last_change":"2024-01-15T18:16:01.970998+0000","last_active":"2024-01-15T18:16:01.973667+0000","last_peered":"2024-01-15T18:16:01.973667+0000","last_clean":"2024-01-15T18:16:01.973667+0000","last_became_active":"2024-01-15T18:16:01.970730+0000","last_became_peered":"2024-01-15T18:16:01.970730+0000","last_unstale":"2024-01-15T18:16:01.973667+0000","last_undegraded":"2024-01-15T18:16:01.973667+0000","last_fullsized":"2024-01-15T18:16:01.973667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983546+0000","last_change":"2024-01-15T18:16:02.989297+0000","last_active":"2024-01-15T18:16:03.983546+0000","last_peered":"2024-01-15T18:16:03.983546+0000","last_clean":"2024-01-15T18:16:03.983546+0000","last_became_active":"2024-01-15T18:16:02.988832+0000","last_became_peered":"2024-01-15T18:16:02.988832+0000","last_unstale":"2024-01-15T18:16:03.983546+0000","last_undegraded":"2024-01-15T18:16:03.983546+0000","last_fullsized":"2024-01-15T18:16:03.983546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:07.504686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.977256+0000","last_change":"2024-01-15T18:16:01.975857+0000","last_active":"2024-01-15T18:16:01.977256+0000","last_peered":"2024-01-15T18:16:01.977256+0000","last_clean":"2024-01-15T18:16:01.977256+0000","last_became_active":"2024-01-15T18:16:01.975585+0000","last_became_peered":"2024-01-15T18:16:01.975585+0000","last_unstale":"2024-01-15T18:16:01.977256+0000","last_undegraded":"2024-01-15T18:16:01.977256+0000","last_fullsized":"2024-01-15T18:16:01.977256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975212+0000","last_change":"2024-01-15T18:16:01.972470+0000","last_active":"2024-01-15T18:16:01.975212+0000","last_peered":"2024-01-15T18:16:01.975212+0000","last_clean":"2024-01-15T18:16:01.975212+0000","last_became_active":"2024-01-15T18:16:01.972202+0000","last_became_peered":"2024-01-15T18:16:01.972202+0000","last_unstale":"2024-01-15T18:16:01.975212+0000","last_undegraded":"2024-01-15T18:16:01.975212+0000","last_fullsized":"2024-01-15T18:16:01.975212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672268+0000","last_change":"2024-01-15T18:16:02.984567+0000","last_active":"2024-01-15T18:16:04.672268+0000","last_peered":"2024-01-15T18:16:04.672268+0000","last_clean":"2024-01-15T18:16:04.672268+0000","last_became_active":"2024-01-15T18:16:02.984223+0000","last_became_peered":"2024-01-15T18:16:02.984223+0000","last_unstale":"2024-01-15T18:16:04.672268+0000","last_undegraded":"2024-01-15T18:16:04.672268+0000","last_fullsized":"2024-01-15T18:16:04.672268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.952954+0000","last_change":"2024-01-15T18:16:00.952954+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.952954+0000","last_undegraded":"2024-01-15T18:16:00.952954+0000","last_fullsized":"2024-01-15T18:16:00.952954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.953595+0000","last_change":"2024-01-15T18:16:00.953595+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.953595+0000","last_undegraded":"2024-01-15T18:16:00.953595+0000","last_fullsized":"2024-01-15T18:16:00.953595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672166+0000","last_change":"2024-01-15T18:16:02.984528+0000","last_active":"2024-01-15T18:16:04.672166+0000","last_peered":"2024-01-15T18:16:04.672166+0000","last_clean":"2024-01-15T18:16:04.672166+0000","last_became_active":"2024-01-15T18:16:02.984179+0000","last_became_peered":"2024-01-15T18:16:02.984179+0000","last_unstale":"2024-01-15T18:16:04.672166+0000","last_undegraded":"2024-01-15T18:16:04.672166+0000","last_fullsized":"2024-01-15T18:16:04.672166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975902+0000","last_change":"2024-01-15T18:16:01.974336+0000","last_active":"2024-01-15T18:16:01.975902+0000","last_peered":"2024-01-15T18:16:01.975902+0000","last_clean":"2024-01-15T18:16:01.975902+0000","last_became_active":"2024-01-15T18:16:01.974035+0000","last_became_peered":"2024-01-15T18:16:01.974035+0000","last_unstale":"2024-01-15T18:16:01.975902+0000","last_undegraded":"2024-01-15T18:16:01.975902+0000","last_fullsized":"2024-01-15T18:16:01.975902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983695+0000","last_change":"2024-01-15T18:16:02.981820+0000","last_active":"2024-01-15T18:16:03.983695+0000","last_peered":"2024-01-15T18:16:03.983695+0000","last_clean":"2024-01-15T18:16:03.983695+0000","last_became_active":"2024-01-15T18:16:02.981478+0000","last_became_peered":"2024-01-15T18:16:02.981478+0000","last_unstale":"2024-01-15T18:16:03.983695+0000","last_undegraded":"2024-01-15T18:16:03.983695+0000","last_fullsized":"2024-01-15T18:16:03.983695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.957714+0000","last_change":"2024-01-15T18:16:01.957714+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957714+0000","last_undegraded":"2024-01-15T18:16:01.957714+0000","last_fullsized":"2024-01-15T18:16:01.957714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:50.381475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984005+0000","last_change":"2024-01-15T18:16:02.981174+0000","last_active":"2024-01-15T18:16:03.984005+0000","last_peered":"2024-01-15T18:16:03.984005+0000","last_clean":"2024-01-15T18:16:03.984005+0000","last_became_active":"2024-01-15T18:16:02.980913+0000","last_became_peered":"2024-01-15T18:16:02.980913+0000","last_unstale":"2024-01-15T18:16:03.984005+0000","last_undegraded":"2024-01-15T18:16:03.984005+0000","last_fullsized":"2024-01-15T18:16:03.984005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:12.319403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.969433+0000","last_change":"2024-01-15T18:16:01.967331+0000","last_active":"2024-01-15T18:16:01.969433+0000","last_peered":"2024-01-15T18:16:01.969433+0000","last_clean":"2024-01-15T18:16:01.969433+0000","last_became_active":"2024-01-15T18:16:01.967031+0000","last_became_peered":"2024-01-15T18:16:01.967031+0000","last_unstale":"2024-01-15T18:16:01.969433+0000","last_undegraded":"2024-01-15T18:16:01.969433+0000","last_fullsized":"2024-01-15T18:16:01.969433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:37.438291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172397+0000","last_change":"2024-01-15T18:16:02.969429+0000","last_active":"2024-01-15T18:16:04.172397+0000","last_peered":"2024-01-15T18:16:04.172397+0000","last_clean":"2024-01-15T18:16:04.172397+0000","last_became_active":"2024-01-15T18:16:02.969095+0000","last_became_peered":"2024-01-15T18:16:02.969095+0000","last_unstale":"2024-01-15T18:16:04.172397+0000","last_undegraded":"2024-01-15T18:16:04.172397+0000","last_fullsized":"2024-01-15T18:16:04.172397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964906+0000","last_change":"2024-01-15T18:16:01.968688+0000","last_active":"2024-01-15T18:16:02.964906+0000","last_peered":"2024-01-15T18:16:02.964906+0000","last_clean":"2024-01-15T18:16:02.964906+0000","last_became_active":"2024-01-15T18:16:01.968389+0000","last_became_peered":"2024-01-15T18:16:01.968389+0000","last_unstale":"2024-01-15T18:16:02.964906+0000","last_undegraded":"2024-01-15T18:16:02.964906+0000","last_fullsized":"2024-01-15T18:16:02.964906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.976712+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.976054+0000","last_became_peered":"2024-01-15T18:16:02.976054+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:26:19.031570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964799+0000","last_change":"2024-01-15T18:16:01.966697+0000","last_active":"2024-01-15T18:16:02.964799+0000","last_peered":"2024-01-15T18:16:02.964799+0000","last_clean":"2024-01-15T18:16:02.964799+0000","last_became_active":"2024-01-15T18:16:01.966397+0000","last_became_peered":"2024-01-15T18:16:01.966397+0000","last_unstale":"2024-01-15T18:16:02.964799+0000","last_undegraded":"2024-01-15T18:16:02.964799+0000","last_fullsized":"2024-01-15T18:16:02.964799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672238+0000","last_change":"2024-01-15T18:16:02.984048+0000","last_active":"2024-01-15T18:16:04.672238+0000","last_peered":"2024-01-15T18:16:04.672238+0000","last_clean":"2024-01-15T18:16:04.672238+0000","last_became_active":"2024-01-15T18:16:02.983546+0000","last_became_peered":"2024-01-15T18:16:02.983546+0000","last_unstale":"2024-01-15T18:16:04.672238+0000","last_undegraded":"2024-01-15T18:16:04.672238+0000","last_fullsized":"2024-01-15T18:16:04.672238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980104+0000","last_change":"2024-01-15T18:16:01.979160+0000","last_active":"2024-01-15T18:16:01.980104+0000","last_peered":"2024-01-15T18:16:01.980104+0000","last_clean":"2024-01-15T18:16:01.980104+0000","last_became_active":"2024-01-15T18:16:01.978901+0000","last_became_peered":"2024-01-15T18:16:01.978901+0000","last_unstale":"2024-01-15T18:16:01.980104+0000","last_undegraded":"2024-01-15T18:16:01.980104+0000","last_fullsized":"2024-01-15T18:16:01.980104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954987+0000","last_change":"2024-01-15T18:16:01.954987+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954987+0000","last_undegraded":"2024-01-15T18:16:01.954987+0000","last_fullsized":"2024-01-15T18:16:01.954987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.961378+0000","last_change":"2024-01-15T18:16:01.958840+0000","last_active":"2024-01-15T18:16:02.961378+0000","last_peered":"2024-01-15T18:16:02.961378+0000","last_clean":"2024-01-15T18:16:02.961378+0000","last_became_active":"2024-01-15T18:16:01.958367+0000","last_became_peered":"2024-01-15T18:16:01.958367+0000","last_unstale":"2024-01-15T18:16:02.961378+0000","last_undegraded":"2024-01-15T18:16:02.961378+0000","last_fullsized":"2024-01-15T18:16:02.961378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983304+0000","last_change":"2024-01-15T18:16:02.991930+0000","last_active":"2024-01-15T18:16:03.983304+0000","last_peered":"2024-01-15T18:16:03.983304+0000","last_clean":"2024-01-15T18:16:03.983304+0000","last_became_active":"2024-01-15T18:16:02.991630+0000","last_became_peered":"2024-01-15T18:16:02.991630+0000","last_unstale":"2024-01-15T18:16:03.983304+0000","last_undegraded":"2024-01-15T18:16:03.983304+0000","last_fullsized":"2024-01-15T18:16:03.983304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964174+0000","last_change":"2024-01-15T18:16:01.958054+0000","last_active":"2024-01-15T18:16:02.964174+0000","last_peered":"2024-01-15T18:16:02.964174+0000","last_clean":"2024-01-15T18:16:02.964174+0000","last_became_active":"2024-01-15T18:16:01.957368+0000","last_became_peered":"2024-01-15T18:16:01.957368+0000","last_unstale":"2024-01-15T18:16:02.964174+0000","last_undegraded":"2024-01-15T18:16:02.964174+0000","last_fullsized":"2024-01-15T18:16:02.964174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172054+0000","last_change":"2024-01-15T18:16:02.970069+0000","last_active":"2024-01-15T18:16:04.172054+0000","last_peered":"2024-01-15T18:16:04.172054+0000","last_clean":"2024-01-15T18:16:04.172054+0000","last_became_active":"2024-01-15T18:16:02.969737+0000","last_became_peered":"2024-01-15T18:16:02.969737+0000","last_unstale":"2024-01-15T18:16:04.172054+0000","last_undegraded":"2024-01-15T18:16:04.172054+0000","last_fullsized":"2024-01-15T18:16:04.172054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:06.396100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.963102+0000","last_change":"2024-01-15T18:16:01.968677+0000","last_active":"2024-01-15T18:16:02.963102+0000","last_peered":"2024-01-15T18:16:02.963102+0000","last_clean":"2024-01-15T18:16:02.963102+0000","last_became_active":"2024-01-15T18:16:01.967462+0000","last_became_peered":"2024-01-15T18:16:01.967462+0000","last_unstale":"2024-01-15T18:16:02.963102+0000","last_undegraded":"2024-01-15T18:16:02.963102+0000","last_fullsized":"2024-01-15T18:16:02.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.959339+0000","last_change":"2024-01-15T18:16:01.959339+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.959339+0000","last_undegraded":"2024-01-15T18:16:01.959339+0000","last_fullsized":"2024-01-15T18:16:01.959339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000937+0000","last_change":"2024-01-15T18:16:01.973544+0000","last_active":"2024-01-15T18:16:03.000937+0000","last_peered":"2024-01-15T18:16:03.000937+0000","last_clean":"2024-01-15T18:16:03.000937+0000","last_became_active":"2024-01-15T18:16:01.973240+0000","last_became_peered":"2024-01-15T18:16:01.973240+0000","last_unstale":"2024-01-15T18:16:03.000937+0000","last_undegraded":"2024-01-15T18:16:03.000937+0000","last_fullsized":"2024-01-15T18:16:03.000937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:06.496321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971217+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:16:02.971217+0000","last_peered":"2024-01-15T18:16:02.971217+0000","last_clean":"2024-01-15T18:16:02.971217+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:16:02.971217+0000","last_undegraded":"2024-01-15T18:16:02.971217+0000","last_fullsized":"2024-01-15T18:16:02.971217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77164,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294676,"statfs":{"total":96636764160,"available":96557748224,"internally_reserved":0,"allocated":1667072,"data_stored":1050036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.3839999999999999,"15min":0.92200000000000004},"min":{"1min":0.621,"5min":0.36899999999999999,"15min":0.29599999999999999},"max":{"1min":16.815999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.044,"15min":0.87},"min":{"1min":0.57399999999999995,"5min":0.33400000000000002,"15min":0.28399999999999997},"max":{"1min":4.6100000000000003,"5min":8.5419999999999998,"15min":13.744},"last":0.441}]},{"osd":1,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.885,"5min":1.6519999999999999,"15min":1.0700000000000001},"min":{"1min":0.60599999999999998,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":16.920000000000002,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.637,"15min":1.0680000000000001},"min":{"1min":0.60299999999999998,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":4.7679999999999998,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.496,"15min":0.99199999999999999},"min":{"1min":0.52000000000000002,"5min":0.433,"15min":0.38400000000000001},"max":{"1min":2.1680000000000001,"5min":30.846,"15min":30.846},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.887,"5min":1.3140000000000001,"15min":1.097},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.39800000000000002},"max":{"1min":17.039999999999999,"5min":17.039999999999999,"15min":39.695},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.6299999999999999,"15min":1.163},"min":{"1min":0.55300000000000005,"5min":0.47499999999999998,"15min":0.38800000000000001},"max":{"1min":2.1160000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.831,"5min":2.0049999999999999,"15min":1.228},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":16.879999999999999,"5min":42.692,"15min":42.692},"last":0.70199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.89100000000000001,"15min":0.72199999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":14.087,"5min":14.087,"15min":14.087},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.36699999999999999,"15min":0.36299999999999999},"max":{"1min":2.2450000000000001,"5min":2.952,"15min":4.3710000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":0.80500000000000005,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":11.334,"5min":11.334,"15min":11.334},"last":0.505},{"interface":"front","average":{"1min":1.012,"5min":0.91000000000000003,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.8179999999999996,"5min":8.423,"15min":8.423},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.91300000000000003,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.7140000000000004,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.376},"max":{"1min":1.1970000000000001,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76588,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295252,"statfs":{"total":96636764160,"available":96558338048,"internally_reserved":0,"allocated":1081344,"data_stored":460292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.83799999999999997,"15min":0.94799999999999995},"min":{"1min":0.41399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78200000000000003,"15min":0.872},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.266,"5min":3.4940000000000002,"15min":12.449},"last":0.81599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.1379999999999999,"15min":1.0109999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.379},"max":{"1min":1.615,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.087,"15min":1.1759999999999999},"min":{"1min":0.55600000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":14.743,"15min":42.825000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":1.147},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1890000000000001,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78100000000000003,"15min":1.0269999999999999},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":1.3640000000000001,"5min":3.016,"15min":24.199000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.0489999999999999,"15min":1.0629999999999999},"min":{"1min":0.60699999999999998,"5min":0.47799999999999998,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":18.791,"15min":18.791},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.585,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.66100000000000003,"15min":0.75800000000000001},"min":{"1min":0.50800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":12.491},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.443},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.4379999999999999,"5min":2.5259999999999998,"15min":12.945},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.98599999999999999,"15min":0.84599999999999997},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":5.1829999999999998,"5min":11.907,"15min":11.907},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80800000000000005,"15min":0.88900000000000001},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5569999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.84199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54456,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317384,"statfs":{"total":96636764160,"available":96581001216,"internally_reserved":0,"allocated":1085440,"data_stored":461232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":0.82899999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.4770000000000003,"5min":6.6820000000000004,"15min":11.519},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.95699999999999996,"15min":0.90300000000000002},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.34999999999999998},"max":{"1min":1.714,"5min":7.6260000000000003,"15min":12.952},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95499999999999996,"15min":0.88500000000000001},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":4.3140000000000001,"5min":4.609,"15min":13.358000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.859,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.149,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":2.867,"5min":18.314,"15min":18.314},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.341,"15min":1.006},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.434},"max":{"1min":2.855,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.89500000000000002,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":1.839,"5min":4.0330000000000004,"15min":13.317},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.034,"15min":1},"min":{"1min":0.443,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.1440000000000001,"5min":15.205,"15min":19.638000000000002},"last":0.74399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.28799999999999998},"max":{"1min":2.4289999999999998,"5min":5.899,"15min":11.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32100000000000001},"max":{"1min":2.516,"5min":3.097,"15min":3.3889999999999998},"last":0.623}]},{"osd":6,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.80000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.238,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.9470000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.76200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":2.3900000000000001,"5min":2.9870000000000001,"15min":4.1799999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.40400000000000003},"max":{"1min":2.3239999999999998,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":1069056,"data_stored":459598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.097,"15min":0.92600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":9.4179999999999993,"5min":11.374000000000001,"15min":34.802},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.135,"15min":0.89300000000000002},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.34699999999999998},"max":{"1min":3.3450000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.97399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2130000000000001,"5min":7.625,"15min":38.198},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.93999999999999995,"15min":0.95799999999999996},"min":{"1min":0.55700000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0589999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0149999999999999,"15min":0.85499999999999998},"min":{"1min":0.61499999999999999,"5min":0.40799999999999997,"15min":0.38700000000000001},"max":{"1min":4.4089999999999998,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.621}]},{"osd":3,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.2729999999999999,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":3.415,"5min":14.529,"15min":14.529},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.2430000000000001,"15min":0.93300000000000005},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78100000000000003,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.6070000000000002,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.96899999999999997,"15min":0.83099999999999996},"min":{"1min":0.47499999999999998,"5min":0.39200000000000002,"15min":0.34100000000000003},"max":{"1min":3.0209999999999999,"5min":25.887,"15min":25.887},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":7.5439999999999996,"15min":10.284000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85999999999999999,"15min":0.79300000000000004},"min":{"1min":0.45000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1680000000000001,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":1.232,"5min":11.67,"15min":11.67},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.595,"15min":0.95199999999999996},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":7.2839999999999998,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.37},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.4670000000000001,"15min":0.94799999999999995},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":7.5170000000000003,"5min":23.074999999999999,"15min":28.332000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.4690000000000001,"15min":0.96299999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.2970000000000002,"5min":20.321000000000002,"15min":40.457999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.514,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.372,"15min":0.30299999999999999},"max":{"1min":7.3170000000000002,"5min":26.260999999999999,"15min":27.427},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.615,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.28399999999999997},"max":{"1min":9.1180000000000003,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.8240000000000001,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.34300000000000003},"max":{"1min":7.2409999999999997,"5min":26.404,"15min":26.404},"last":0.53100000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3819999999999999,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":4.1040000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.069},"min":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.377},"max":{"1min":2.1150000000000002,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.6259999999999999,"15min":1.097},"min":{"1min":0.58399999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":6.5140000000000002,"5min":26.314,"15min":26.314},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.5649999999999999,"15min":1.131},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.44500000000000001},"max":{"1min":4.165,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.94399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.635,"15min":1.1140000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.7619999999999996,"5min":20.445,"15min":20.445},"last":1},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.7270000000000001,"15min":1.1699999999999999},"min":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.495},"max":{"1min":6.7149999999999999,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.6839999999999999,"15min":1.155},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.2030000000000003,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":2.1429999999999998,"15min":1.3089999999999999},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46000000000000002},"max":{"1min":2.2490000000000001,"5min":44.817999999999998,"15min":44.817999999999998},"last":1.0900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.153,"15min":1.0449999999999999},"min":{"1min":0.33800000000000002,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9950000000000001,"5min":12.695,"15min":29.527999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.591,"15min":1.3180000000000001},"min":{"1min":0.47499999999999998,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.288,"15min":1.5329999999999999},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":7.476,"5min":34.029000000000003,"15min":113.149},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.099},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.30199999999999999},"max":{"1min":14.728,"5min":20.619,"15min":41.302},"last":0.499}]},{"osd":3,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":2.1030000000000002,"15min":1.3620000000000001},"min":{"1min":0.36799999999999999,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":31.57,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":1.258},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":7.3609999999999998,"5min":7.3609999999999998,"15min":115.693},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1279999999999999,"15min":1.2090000000000001},"min":{"1min":0.627,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":7.4169999999999998,"5min":7.4169999999999998,"15min":24.366},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.3049999999999999,"15min":1.2090000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0580000000000001,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":2.548,"5min":2.222,"15min":1.482},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.43099999999999999},"max":{"1min":41.683999999999997,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.103,"5min":1.5169999999999999,"15min":1.3129999999999999},"min":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":7.5099999999999998,"5min":26.084,"15min":26.084},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.468,"15min":1.302},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.208,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.746},{"interface":"front","average":{"1min":2.589,"5min":2.3210000000000002,"15min":1.556},"min":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.47699999999999998},"max":{"1min":41.738999999999997,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.3979999999999999,"15min":1.242},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":1.298,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":2.6000000000000001,"5min":2.383,"15min":1.629},"min":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.48799999999999999},"max":{"1min":41.402000000000001,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.97399999999999998,"15min":0.92500000000000004},"min":{"1min":0.33100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.3330000000000002,"5min":7.1580000000000004,"15min":39.948},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.5660000000000001,"15min":1.1200000000000001},"min":{"1min":0.39600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.577,"5min":35.790999999999997,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.034,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":6.516,"5min":11.346,"15min":49.229999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.1870000000000001,"15min":0.86499999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.931,"5min":7.9489999999999998,"15min":10.349},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.655,"15min":1.2190000000000001},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4669999999999996,"5min":27.643999999999998,"15min":27.643999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.343,"15min":1.016},"min":{"1min":0.48499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":15.608000000000001,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.6819999999999999,"15min":1.276},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.40200000000000002},"max":{"1min":4.633,"5min":27.489999999999998,"15min":27.489999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2969999999999999,"15min":1.054},"min":{"1min":0.61099999999999999,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":5.9139999999999997,"5min":10.038,"15min":12.997},"last":0.70699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.7050000000000001,"15min":1.2390000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.50800000000000001},"max":{"1min":4.673,"5min":27.547000000000001,"15min":27.547000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.1419999999999999},"min":{"1min":0.68400000000000005,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":5.3399999999999999,"5min":9.9610000000000003,"15min":13.054},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.698,"15min":1.2490000000000001},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":4.7060000000000004,"5min":27.579999999999998,"15min":27.579999999999998},"last":0.746},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.198,"15min":1.0760000000000001},"min":{"1min":0.60199999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0840000000000001,"5min":11.643000000000001,"15min":11.712999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3600000000000001,"15min":1.145},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46500000000000002},"max":{"1min":9.5660000000000007,"5min":11.833,"15min":11.833},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.226,"5min":1.4450000000000001,"15min":1.153},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":5.9550000000000001,"5min":14.606,"15min":14.606},"last":0.83799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77120,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294720,"statfs":{"total":96636764160,"available":96557793280,"internally_reserved":0,"allocated":1626112,"data_stored":1030448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.4470000000000001,"15min":1.175},"min":{"1min":0.32500000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":1.137,"5min":16.149000000000001,"15min":31.977},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.254,"15min":0.94699999999999995},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70999999999999996,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.63700000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":1.6719999999999999,"15min":1.0820000000000001},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.30199999999999999},"max":{"1min":1.7450000000000001,"5min":36.386000000000003,"15min":41.387999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2689999999999999,"15min":1.0429999999999999},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.25},"max":{"1min":11.539999999999999,"5min":16.556000000000001,"15min":33.219999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.3260000000000001,"15min":1.1140000000000001},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":1.2230000000000001,"5min":9.8740000000000006,"15min":31.943999999999999},"last":0.623},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.7689999999999999,"15min":1.0600000000000001},"min":{"1min":0.499,"5min":0.432,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":45.128,"15min":45.128},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.2709999999999999,"15min":1.109},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":11.772,"5min":11.772,"15min":32.932000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.379,"15min":1.1870000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":14.831,"15min":31.782},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.306,"15min":1.1850000000000001},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.43099999999999999},"max":{"1min":1.383,"5min":11.247999999999999,"15min":31.858000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.2929999999999999,"15min":1.1080000000000001},"min":{"1min":0.58399999999999996,"5min":0.52200000000000002,"15min":0.45800000000000002},"max":{"1min":11.958,"5min":11.958,"15min":33.151000000000003},"last":0.748}]},{"osd":6,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3520000000000001,"15min":1.052},"min":{"1min":0.622,"5min":0.49399999999999999,"15min":0.46400000000000002},"max":{"1min":1.1479999999999999,"5min":13.055,"15min":13.055},"last":0.871},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.3460000000000001,"15min":1.137},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.498},"max":{"1min":11.833,"5min":11.833,"15min":33.098999999999997},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.4279999999999999,"15min":1.0580000000000001},"min":{"1min":0.64700000000000002,"5min":0.52000000000000002,"15min":0.501},"max":{"1min":1.091,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.4630000000000001,"15min":1.2430000000000001},"min":{"1min":0.60299999999999998,"5min":0.53700000000000003,"15min":0.504},"max":{"1min":1.335,"5min":10.377000000000001,"15min":31.905999999999999},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:16:07.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:16:07.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.230+0000 7f3c2f400640 1 Processor -- start 2024-01-15T18:16:07.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.230+0000 7f3c2f400640 1 -- start start 2024-01-15T18:16:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c281607c0 0x7f3c2806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2f400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3c280cfb30 0x7f3c2806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c280cffd0 con 0x7f3c2806c230 2024-01-15T18:16:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2f400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2806c670 con 0x7f3c280cfb30 2024-01-15T18:16:07.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280d1ce0 con 0x7f3c281607c0 2024-01-15T18:16:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c281607c0 0x7f3c2806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c27e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3c280cfb30 0x7f3c2806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2d400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f3c2806c230 0x7f3c280cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58212/0 (socket says 172.21.15.143:58212) 2024-01-15T18:16:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2d400640 1 -- 172.21.15.143:0/2975982782 learned_addr learned my addr 172.21.15.143:0/2975982782 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2ca00640 1 -- 172.21.15.143:0/2975982782 >> v2:172.21.15.204:3300/0 conn(0x7f3c280cfb30 msgr2=0x7f3c2806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2ca00640 1 --2- 172.21.15.143:0/2975982782 >> v2:172.21.15.204:3300/0 conn(0x7f3c280cfb30 0x7f3c2806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2ca00640 1 -- 172.21.15.143:0/2975982782 >> v1:172.21.15.204:6789/0 conn(0x7f3c2806c230 legacy=0x7f3c280cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c27400640 1 -- 172.21.15.143:0/2975982782 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057138840 0 0) 0x7f3c280cffd0 con 0x7f3c2806c230 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c2ca00640 1 -- 172.21.15.143:0/2975982782 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c280d02b0 con 0x7f3c281607c0 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.234+0000 7f3c27e00640 1 --2- 172.21.15.143:0/2975982782 >> v2:172.21.15.204:3300/0 conn(0x7f3c280cfb30 0x7f3c2806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:07.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2ca00640 1 --2- 172.21.15.143:0/2975982782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c281607c0 0x7f3c2806b7b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3c180097c0 tx=0x7f3c1801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5583f98942267e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c27400640 1 -- 172.21.15.143:0/2975982782 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180a8970 con 0x7f3c281607c0 2024-01-15T18:16:07.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c27400640 1 -- 172.21.15.143:0/2975982782 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c180a8ad0 con 0x7f3c281607c0 2024-01-15T18:16:07.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c27400640 1 -- 172.21.15.143:0/2975982782 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180a8e60 con 0x7f3c281607c0 2024-01-15T18:16:07.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 -- 172.21.15.143:0/2975982782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c281607c0 msgr2=0x7f3c2806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 --2- 172.21.15.143:0/2975982782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c281607c0 0x7f3c2806b7b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3c180097c0 tx=0x7f3c1801ec70 comp rx=0 tx=0).stop 2024-01-15T18:16:07.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 -- 172.21.15.143:0/2975982782 shutdown_connections 2024-01-15T18:16:07.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 --2- 172.21.15.143:0/2975982782 >> v2:172.21.15.204:3300/0 conn(0x7f3c280cfb30 0x7f3c2806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 --2- 172.21.15.143:0/2975982782 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c281607c0 0x7f3c2806b7b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 -- 172.21.15.143:0/2975982782 >> 172.21.15.143:0/2975982782 conn(0x7f3c280d0e20 msgr2=0x7f3c280d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 -- 172.21.15.143:0/2975982782 shutdown_connections 2024-01-15T18:16:07.236 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.238+0000 7f3c2f400640 1 -- 172.21.15.143:0/2975982782 wait complete. 2024-01-15T18:16:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 Processor -- start 2024-01-15T18:16:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- start start 2024-01-15T18:16:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 0x7f3c281f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c280cfb30 0x7f3c281574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c281607c0 0x7f3c28145f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280c62e0 con 0x7f3c2806c230 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280c6b80 con 0x7f3c281607c0 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c281579f0 con 0x7f3c280cfb30 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 0x7f3c281f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 0x7f3c281f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55020/0 (socket says 172.21.15.143:55020) 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 -- 172.21.15.143:0/1086415198 learned_addr learned my addr 172.21.15.143:0/1086415198 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c27e00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c280cfb30 0x7f3c281574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2d400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c281607c0 0x7f3c28145f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 -- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c280cfb30 msgr2=0x7f3c281574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c280cfb30 0x7f3c281574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 -- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c281607c0 msgr2=0x7f3c28145f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c281607c0 0x7f3c28145f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 -- 172.21.15.143:0/1086415198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10009420 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2ca00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 0x7f3c281f25f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3c1801d320 tx=0x7f3c1801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180bb640 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c180bb7c0 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180bbb70 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c18009420 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c280c6530 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.242+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c280b6d70 con 0x7f3c2806c230 2024-01-15T18:16:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.246+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f3c180b7040 con 0x7f3c2806c230 2024-01-15T18:16:07.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.246+0000 7f3c24c00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c00067610 0x7f3c00069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.246+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3c181213c0 con 0x7f3c2806c230 2024-01-15T18:16:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.246+0000 7f3c27e00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c00067610 0x7f3c00069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.246+0000 7f3c27e00640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c00067610 0x7f3c00069ad0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3c280cd120 tx=0x7f3c10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.250+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3c180efd90 con 0x7f3c2806c230 2024-01-15T18:16:07.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.398+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c280cba30 con 0x7f3c00067610 2024-01-15T18:16:07.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.406+0000 7f3c24c00640 1 -- 172.21.15.143:0/1086415198 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285130 (secure 0 0 0) 0x7f3c10011e60 con 0x7f3c00067610 2024-01-15T18:16:07.406 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:07.407 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c00067610 msgr2=0x7f3c00069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c00067610 0x7f3c00069ad0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3c280cd120 tx=0x7f3c10008040 comp rx=0 tx=0).stop 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 msgr2=0x7f3c281f25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 0x7f3c281f25f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3c1801d320 tx=0x7f3c1801ba10 comp rx=0 tx=0).stop 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 shutdown_connections 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3c00067610 0x7f3c00069ad0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c281607c0 0x7f3c28145f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3c280cfb30 0x7f3c281574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 --2- 172.21.15.143:0/1086415198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3c2806c230 0x7f3c281f25f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 >> 172.21.15.143:0/1086415198 conn(0x7f3c280d0e20 msgr2=0x7f3c2806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 shutdown_connections 2024-01-15T18:16:07.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.410+0000 7f3c2f400640 1 -- 172.21.15.143:0/1086415198 wait complete. 2024-01-15T18:16:07.426 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-01-15T18:16:06.858046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":591524,"kb_used_data":9400,"kb_used_omap":12,"kb_used_meta":582003,"kb_avail":754383196,"statfs":{"total":773094113280,"available":772488392704,"internally_reserved":0,"allocated":9625600,"data_stored":4781846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":595971632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913200"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172275+0000","last_change":"2024-01-15T18:16:02.977971+0000","last_active":"2024-01-15T18:16:04.172275+0000","last_peered":"2024-01-15T18:16:04.172275+0000","last_clean":"2024-01-15T18:16:04.172275+0000","last_became_active":"2024-01-15T18:16:02.977706+0000","last_became_peered":"2024-01-15T18:16:02.977706+0000","last_unstale":"2024-01-15T18:16:04.172275+0000","last_undegraded":"2024-01-15T18:16:04.172275+0000","last_fullsized":"2024-01-15T18:16:04.172275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958822+0000","last_change":"2024-01-15T18:16:01.958822+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958822+0000","last_undegraded":"2024-01-15T18:16:01.958822+0000","last_fullsized":"2024-01-15T18:16:01.958822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983752+0000","last_change":"2024-01-15T18:16:02.982423+0000","last_active":"2024-01-15T18:16:03.983752+0000","last_peered":"2024-01-15T18:16:03.983752+0000","last_clean":"2024-01-15T18:16:03.983752+0000","last_became_active":"2024-01-15T18:16:02.981906+0000","last_became_peered":"2024-01-15T18:16:02.981906+0000","last_unstale":"2024-01-15T18:16:03.983752+0000","last_undegraded":"2024-01-15T18:16:03.983752+0000","last_fullsized":"2024-01-15T18:16:03.983752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:02.961035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.988011+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.987583+0000","last_became_peered":"2024-01-15T18:16:02.987583+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672319+0000","last_change":"2024-01-15T18:16:02.984894+0000","last_active":"2024-01-15T18:16:04.672319+0000","last_peered":"2024-01-15T18:16:04.672319+0000","last_clean":"2024-01-15T18:16:04.672319+0000","last_became_active":"2024-01-15T18:16:02.984537+0000","last_became_peered":"2024-01-15T18:16:02.984537+0000","last_unstale":"2024-01-15T18:16:04.672319+0000","last_undegraded":"2024-01-15T18:16:04.672319+0000","last_fullsized":"2024-01-15T18:16:04.672319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672358+0000","last_change":"2024-01-15T18:16:02.988234+0000","last_active":"2024-01-15T18:16:04.672358+0000","last_peered":"2024-01-15T18:16:04.672358+0000","last_clean":"2024-01-15T18:16:04.672358+0000","last_became_active":"2024-01-15T18:16:02.987822+0000","last_became_peered":"2024-01-15T18:16:02.987822+0000","last_unstale":"2024-01-15T18:16:04.672358+0000","last_undegraded":"2024-01-15T18:16:04.672358+0000","last_fullsized":"2024-01-15T18:16:04.672358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672449+0000","last_change":"2024-01-15T18:16:02.978613+0000","last_active":"2024-01-15T18:16:04.672449+0000","last_peered":"2024-01-15T18:16:04.672449+0000","last_clean":"2024-01-15T18:16:04.672449+0000","last_became_active":"2024-01-15T18:16:02.978331+0000","last_became_peered":"2024-01-15T18:16:02.978331+0000","last_unstale":"2024-01-15T18:16:04.672449+0000","last_undegraded":"2024-01-15T18:16:04.672449+0000","last_fullsized":"2024-01-15T18:16:04.672449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.985584+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.984527+0000","last_became_peered":"2024-01-15T18:16:02.984527+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.960055+0000","last_change":"2024-01-15T18:16:01.960055+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.960055+0000","last_undegraded":"2024-01-15T18:16:01.960055+0000","last_fullsized":"2024-01-15T18:16:01.960055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.171943+0000","last_change":"2024-01-15T18:16:02.976922+0000","last_active":"2024-01-15T18:16:04.171943+0000","last_peered":"2024-01-15T18:16:04.171943+0000","last_clean":"2024-01-15T18:16:04.171943+0000","last_became_active":"2024-01-15T18:16:02.976638+0000","last_became_peered":"2024-01-15T18:16:02.976638+0000","last_unstale":"2024-01-15T18:16:04.171943+0000","last_undegraded":"2024-01-15T18:16:04.171943+0000","last_fullsized":"2024-01-15T18:16:04.171943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:28.008839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.980605+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.978977+0000","last_became_peered":"2024-01-15T18:16:02.978977+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:17.866154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954608+0000","last_change":"2024-01-15T18:16:01.954608+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954608+0000","last_undegraded":"2024-01-15T18:16:01.954608+0000","last_fullsized":"2024-01-15T18:16:01.954608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982727+0000","last_change":"2024-01-15T18:16:02.979861+0000","last_active":"2024-01-15T18:16:03.982727+0000","last_peered":"2024-01-15T18:16:03.982727+0000","last_clean":"2024-01-15T18:16:03.982727+0000","last_became_active":"2024-01-15T18:16:02.979569+0000","last_became_peered":"2024-01-15T18:16:02.979569+0000","last_unstale":"2024-01-15T18:16:03.982727+0000","last_undegraded":"2024-01-15T18:16:03.982727+0000","last_fullsized":"2024-01-15T18:16:03.982727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984023+0000","last_change":"2024-01-15T18:16:02.977093+0000","last_active":"2024-01-15T18:16:03.984023+0000","last_peered":"2024-01-15T18:16:03.984023+0000","last_clean":"2024-01-15T18:16:03.984023+0000","last_became_active":"2024-01-15T18:16:02.974914+0000","last_became_peered":"2024-01-15T18:16:02.974914+0000","last_unstale":"2024-01-15T18:16:03.984023+0000","last_undegraded":"2024-01-15T18:16:03.984023+0000","last_fullsized":"2024-01-15T18:16:03.984023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.961445+0000","last_change":"2024-01-15T18:16:01.961445+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.961445+0000","last_undegraded":"2024-01-15T18:16:01.961445+0000","last_fullsized":"2024-01-15T18:16:01.961445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957359+0000","last_change":"2024-01-15T18:16:01.957359+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957359+0000","last_undegraded":"2024-01-15T18:16:01.957359+0000","last_fullsized":"2024-01-15T18:16:01.957359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172017+0000","last_change":"2024-01-15T18:16:02.970218+0000","last_active":"2024-01-15T18:16:04.172017+0000","last_peered":"2024-01-15T18:16:04.172017+0000","last_clean":"2024-01-15T18:16:04.172017+0000","last_became_active":"2024-01-15T18:16:02.969901+0000","last_became_peered":"2024-01-15T18:16:02.969901+0000","last_unstale":"2024-01-15T18:16:04.172017+0000","last_undegraded":"2024-01-15T18:16:04.172017+0000","last_fullsized":"2024-01-15T18:16:04.172017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:31.551846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958084+0000","last_change":"2024-01-15T18:16:01.958084+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958084+0000","last_undegraded":"2024-01-15T18:16:01.958084+0000","last_fullsized":"2024-01-15T18:16:01.958084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983932+0000","last_change":"2024-01-15T18:16:02.991391+0000","last_active":"2024-01-15T18:16:03.983932+0000","last_peered":"2024-01-15T18:16:03.983932+0000","last_clean":"2024-01-15T18:16:03.983932+0000","last_became_active":"2024-01-15T18:16:02.991138+0000","last_became_peered":"2024-01-15T18:16:02.991138+0000","last_unstale":"2024-01-15T18:16:03.983932+0000","last_undegraded":"2024-01-15T18:16:03.983932+0000","last_fullsized":"2024-01-15T18:16:03.983932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:06.968039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983406+0000","last_change":"2024-01-15T18:16:02.976507+0000","last_active":"2024-01-15T18:16:03.983406+0000","last_peered":"2024-01-15T18:16:03.983406+0000","last_clean":"2024-01-15T18:16:03.983406+0000","last_became_active":"2024-01-15T18:16:02.975765+0000","last_became_peered":"2024-01-15T18:16:02.975765+0000","last_unstale":"2024-01-15T18:16:03.983406+0000","last_undegraded":"2024-01-15T18:16:03.983406+0000","last_fullsized":"2024-01-15T18:16:03.983406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172094+0000","last_change":"2024-01-15T18:16:02.976103+0000","last_active":"2024-01-15T18:16:04.172094+0000","last_peered":"2024-01-15T18:16:04.172094+0000","last_clean":"2024-01-15T18:16:04.172094+0000","last_became_active":"2024-01-15T18:16:02.975829+0000","last_became_peered":"2024-01-15T18:16:02.975829+0000","last_unstale":"2024-01-15T18:16:04.172094+0000","last_undegraded":"2024-01-15T18:16:04.172094+0000","last_fullsized":"2024-01-15T18:16:04.172094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956707+0000","last_change":"2024-01-15T18:16:01.956707+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956707+0000","last_undegraded":"2024-01-15T18:16:01.956707+0000","last_fullsized":"2024-01-15T18:16:01.956707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956925+0000","last_change":"2024-01-15T18:16:01.956925+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956925+0000","last_undegraded":"2024-01-15T18:16:01.956925+0000","last_fullsized":"2024-01-15T18:16:01.956925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982746+0000","last_change":"2024-01-15T18:16:02.974470+0000","last_active":"2024-01-15T18:16:03.982746+0000","last_peered":"2024-01-15T18:16:03.982746+0000","last_clean":"2024-01-15T18:16:03.982746+0000","last_became_active":"2024-01-15T18:16:02.974162+0000","last_became_peered":"2024-01-15T18:16:02.974162+0000","last_unstale":"2024-01-15T18:16:03.982746+0000","last_undegraded":"2024-01-15T18:16:03.982746+0000","last_fullsized":"2024-01-15T18:16:03.982746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172002+0000","last_change":"2024-01-15T18:16:02.986920+0000","last_active":"2024-01-15T18:16:04.172002+0000","last_peered":"2024-01-15T18:16:04.172002+0000","last_clean":"2024-01-15T18:16:04.172002+0000","last_became_active":"2024-01-15T18:16:02.986544+0000","last_became_peered":"2024-01-15T18:16:02.986544+0000","last_unstale":"2024-01-15T18:16:04.172002+0000","last_undegraded":"2024-01-15T18:16:04.172002+0000","last_fullsized":"2024-01-15T18:16:04.172002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:06.358800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983627+0000","last_change":"2024-01-15T18:16:02.981015+0000","last_active":"2024-01-15T18:16:03.983627+0000","last_peered":"2024-01-15T18:16:03.983627+0000","last_clean":"2024-01-15T18:16:03.983627+0000","last_became_active":"2024-01-15T18:16:02.979886+0000","last_became_peered":"2024-01-15T18:16:02.979886+0000","last_unstale":"2024-01-15T18:16:03.983627+0000","last_undegraded":"2024-01-15T18:16:03.983627+0000","last_fullsized":"2024-01-15T18:16:03.983627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:23.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004361+0000","last_change":"2024-01-15T18:16:01.968674+0000","last_active":"2024-01-15T18:16:03.004361+0000","last_peered":"2024-01-15T18:16:03.004361+0000","last_clean":"2024-01-15T18:16:03.004361+0000","last_became_active":"2024-01-15T18:16:01.967874+0000","last_became_peered":"2024-01-15T18:16:01.967874+0000","last_unstale":"2024-01-15T18:16:03.004361+0000","last_undegraded":"2024-01-15T18:16:03.004361+0000","last_fullsized":"2024-01-15T18:16:03.004361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172367+0000","last_change":"2024-01-15T18:16:02.971695+0000","last_active":"2024-01-15T18:16:04.172367+0000","last_peered":"2024-01-15T18:16:04.172367+0000","last_clean":"2024-01-15T18:16:04.172367+0000","last_became_active":"2024-01-15T18:16:02.971425+0000","last_became_peered":"2024-01-15T18:16:02.971425+0000","last_unstale":"2024-01-15T18:16:04.172367+0000","last_undegraded":"2024-01-15T18:16:04.172367+0000","last_fullsized":"2024-01-15T18:16:04.172367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002366+0000","last_change":"2024-01-15T18:16:01.960880+0000","last_active":"2024-01-15T18:16:03.002366+0000","last_peered":"2024-01-15T18:16:03.002366+0000","last_clean":"2024-01-15T18:16:03.002366+0000","last_became_active":"2024-01-15T18:16:01.960424+0000","last_became_peered":"2024-01-15T18:16:01.960424+0000","last_unstale":"2024-01-15T18:16:03.002366+0000","last_undegraded":"2024-01-15T18:16:03.002366+0000","last_fullsized":"2024-01-15T18:16:03.002366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:05.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172293+0000","last_change":"2024-01-15T18:16:02.987087+0000","last_active":"2024-01-15T18:16:04.172293+0000","last_peered":"2024-01-15T18:16:04.172293+0000","last_clean":"2024-01-15T18:16:04.172293+0000","last_became_active":"2024-01-15T18:16:02.986595+0000","last_became_peered":"2024-01-15T18:16:02.986595+0000","last_unstale":"2024-01-15T18:16:04.172293+0000","last_undegraded":"2024-01-15T18:16:04.172293+0000","last_fullsized":"2024-01-15T18:16:04.172293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973629+0000","last_change":"2024-01-15T18:16:01.972650+0000","last_active":"2024-01-15T18:16:01.973629+0000","last_peered":"2024-01-15T18:16:01.973629+0000","last_clean":"2024-01-15T18:16:01.973629+0000","last_became_active":"2024-01-15T18:16:01.972282+0000","last_became_peered":"2024-01-15T18:16:01.972282+0000","last_unstale":"2024-01-15T18:16:01.973629+0000","last_undegraded":"2024-01-15T18:16:01.973629+0000","last_fullsized":"2024-01-15T18:16:01.973629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984011+0000","last_change":"2024-01-15T18:16:02.980841+0000","last_active":"2024-01-15T18:16:03.984011+0000","last_peered":"2024-01-15T18:16:03.984011+0000","last_clean":"2024-01-15T18:16:03.984011+0000","last_became_active":"2024-01-15T18:16:02.980529+0000","last_became_peered":"2024-01-15T18:16:02.980529+0000","last_unstale":"2024-01-15T18:16:03.984011+0000","last_undegraded":"2024-01-15T18:16:03.984011+0000","last_fullsized":"2024-01-15T18:16:03.984011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:16.698507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.974689+0000","last_change":"2024-01-15T18:16:01.974052+0000","last_active":"2024-01-15T18:16:01.974689+0000","last_peered":"2024-01-15T18:16:01.974689+0000","last_clean":"2024-01-15T18:16:01.974689+0000","last_became_active":"2024-01-15T18:16:01.973571+0000","last_became_peered":"2024-01-15T18:16:01.973571+0000","last_unstale":"2024-01-15T18:16:01.974689+0000","last_undegraded":"2024-01-15T18:16:01.974689+0000","last_fullsized":"2024-01-15T18:16:01.974689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956466+0000","last_change":"2024-01-15T18:16:01.956466+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956466+0000","last_undegraded":"2024-01-15T18:16:01.956466+0000","last_fullsized":"2024-01-15T18:16:01.956466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002241+0000","last_change":"2024-01-15T18:16:01.961667+0000","last_active":"2024-01-15T18:16:03.002241+0000","last_peered":"2024-01-15T18:16:03.002241+0000","last_clean":"2024-01-15T18:16:03.002241+0000","last_became_active":"2024-01-15T18:16:01.961336+0000","last_became_peered":"2024-01-15T18:16:01.961336+0000","last_unstale":"2024-01-15T18:16:03.002241+0000","last_undegraded":"2024-01-15T18:16:03.002241+0000","last_fullsized":"2024-01-15T18:16:03.002241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.970823+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.970523+0000","last_became_peered":"2024-01-15T18:16:02.970523+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000246+0000","last_change":"2024-01-15T18:16:01.958649+0000","last_active":"2024-01-15T18:16:03.000246+0000","last_peered":"2024-01-15T18:16:03.000246+0000","last_clean":"2024-01-15T18:16:03.000246+0000","last_became_active":"2024-01-15T18:16:01.958043+0000","last_became_peered":"2024-01-15T18:16:01.958043+0000","last_unstale":"2024-01-15T18:16:03.000246+0000","last_undegraded":"2024-01-15T18:16:03.000246+0000","last_fullsized":"2024-01-15T18:16:03.000246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983181+0000","last_change":"2024-01-15T18:16:02.983728+0000","last_active":"2024-01-15T18:16:03.983181+0000","last_peered":"2024-01-15T18:16:03.983181+0000","last_clean":"2024-01-15T18:16:03.983181+0000","last_became_active":"2024-01-15T18:16:02.981960+0000","last_became_peered":"2024-01-15T18:16:02.981960+0000","last_unstale":"2024-01-15T18:16:03.983181+0000","last_undegraded":"2024-01-15T18:16:03.983181+0000","last_fullsized":"2024-01-15T18:16:03.983181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.971514+0000","last_change":"2024-01-15T18:16:01.968471+0000","last_active":"2024-01-15T18:16:01.971514+0000","last_peered":"2024-01-15T18:16:01.971514+0000","last_clean":"2024-01-15T18:16:01.971514+0000","last_became_active":"2024-01-15T18:16:01.967508+0000","last_became_peered":"2024-01-15T18:16:01.967508+0000","last_unstale":"2024-01-15T18:16:01.971514+0000","last_undegraded":"2024-01-15T18:16:01.971514+0000","last_fullsized":"2024-01-15T18:16:01.971514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:33.150205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954100+0000","last_change":"2024-01-15T18:16:01.954100+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954100+0000","last_undegraded":"2024-01-15T18:16:01.954100+0000","last_fullsized":"2024-01-15T18:16:01.954100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.954226+0000","last_change":"2024-01-15T18:16:01.954226+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954226+0000","last_undegraded":"2024-01-15T18:16:01.954226+0000","last_fullsized":"2024-01-15T18:16:01.954226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:51.202598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955399+0000","last_change":"2024-01-15T18:16:01.955399+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955399+0000","last_undegraded":"2024-01-15T18:16:01.955399+0000","last_fullsized":"2024-01-15T18:16:01.955399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.967235+0000","last_change":"2024-01-15T18:16:01.967948+0000","last_active":"2024-01-15T18:16:02.967235+0000","last_peered":"2024-01-15T18:16:02.967235+0000","last_clean":"2024-01-15T18:16:02.967235+0000","last_became_active":"2024-01-15T18:16:01.967662+0000","last_became_peered":"2024-01-15T18:16:01.967662+0000","last_unstale":"2024-01-15T18:16:02.967235+0000","last_undegraded":"2024-01-15T18:16:02.967235+0000","last_fullsized":"2024-01-15T18:16:02.967235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:28.859462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672459+0000","last_change":"2024-01-15T18:16:02.977816+0000","last_active":"2024-01-15T18:16:04.672459+0000","last_peered":"2024-01-15T18:16:04.672459+0000","last_clean":"2024-01-15T18:16:04.672459+0000","last_became_active":"2024-01-15T18:16:02.977488+0000","last_became_peered":"2024-01-15T18:16:02.977488+0000","last_unstale":"2024-01-15T18:16:04.672459+0000","last_undegraded":"2024-01-15T18:16:04.672459+0000","last_fullsized":"2024-01-15T18:16:04.672459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.966903+0000","last_change":"2024-01-15T18:16:01.970089+0000","last_active":"2024-01-15T18:16:02.966903+0000","last_peered":"2024-01-15T18:16:02.966903+0000","last_clean":"2024-01-15T18:16:02.966903+0000","last_became_active":"2024-01-15T18:16:01.969791+0000","last_became_peered":"2024-01-15T18:16:01.969791+0000","last_unstale":"2024-01-15T18:16:02.966903+0000","last_undegraded":"2024-01-15T18:16:02.966903+0000","last_fullsized":"2024-01-15T18:16:02.966903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:25.108362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956190+0000","last_change":"2024-01-15T18:16:01.956190+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956190+0000","last_undegraded":"2024-01-15T18:16:01.956190+0000","last_fullsized":"2024-01-15T18:16:01.956190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002343+0000","last_change":"2024-01-15T18:16:01.967367+0000","last_active":"2024-01-15T18:16:03.002343+0000","last_peered":"2024-01-15T18:16:03.002343+0000","last_clean":"2024-01-15T18:16:03.002343+0000","last_became_active":"2024-01-15T18:16:01.966610+0000","last_became_peered":"2024-01-15T18:16:01.966610+0000","last_unstale":"2024-01-15T18:16:03.002343+0000","last_undegraded":"2024-01-15T18:16:03.002343+0000","last_fullsized":"2024-01-15T18:16:03.002343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983502+0000","last_change":"2024-01-15T18:16:02.976324+0000","last_active":"2024-01-15T18:16:03.983502+0000","last_peered":"2024-01-15T18:16:03.983502+0000","last_clean":"2024-01-15T18:16:03.983502+0000","last_became_active":"2024-01-15T18:16:02.975452+0000","last_became_peered":"2024-01-15T18:16:02.975452+0000","last_unstale":"2024-01-15T18:16:03.983502+0000","last_undegraded":"2024-01-15T18:16:03.983502+0000","last_fullsized":"2024-01-15T18:16:03.983502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:31.636033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004546+0000","last_change":"2024-01-15T18:16:01.971530+0000","last_active":"2024-01-15T18:16:03.004546+0000","last_peered":"2024-01-15T18:16:03.004546+0000","last_clean":"2024-01-15T18:16:03.004546+0000","last_became_active":"2024-01-15T18:16:01.971080+0000","last_became_peered":"2024-01-15T18:16:01.971080+0000","last_unstale":"2024-01-15T18:16:03.004546+0000","last_undegraded":"2024-01-15T18:16:03.004546+0000","last_fullsized":"2024-01-15T18:16:03.004546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955769+0000","last_change":"2024-01-15T18:16:01.955769+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955769+0000","last_undegraded":"2024-01-15T18:16:01.955769+0000","last_fullsized":"2024-01-15T18:16:01.955769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968495+0000","last_change":"2024-01-15T18:16:01.968495+0000","last_active":"2024-01-15T18:16:01.968495+0000","last_peered":"2024-01-15T18:16:01.968495+0000","last_clean":"2024-01-15T18:16:01.968495+0000","last_became_active":"2024-01-15T18:16:01.967100+0000","last_became_peered":"2024-01-15T18:16:01.967100+0000","last_unstale":"2024-01-15T18:16:01.968495+0000","last_undegraded":"2024-01-15T18:16:01.968495+0000","last_fullsized":"2024-01-15T18:16:01.968495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:29.198637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983817+0000","last_change":"2024-01-15T18:16:02.985570+0000","last_active":"2024-01-15T18:16:03.983817+0000","last_peered":"2024-01-15T18:16:03.983817+0000","last_clean":"2024-01-15T18:16:03.983817+0000","last_became_active":"2024-01-15T18:16:02.985303+0000","last_became_peered":"2024-01-15T18:16:02.985303+0000","last_unstale":"2024-01-15T18:16:03.983817+0000","last_undegraded":"2024-01-15T18:16:03.983817+0000","last_fullsized":"2024-01-15T18:16:03.983817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:55.037442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968815+0000","last_change":"2024-01-15T18:16:01.965271+0000","last_active":"2024-01-15T18:16:01.968815+0000","last_peered":"2024-01-15T18:16:01.968815+0000","last_clean":"2024-01-15T18:16:01.968815+0000","last_became_active":"2024-01-15T18:16:01.964982+0000","last_became_peered":"2024-01-15T18:16:01.964982+0000","last_unstale":"2024-01-15T18:16:01.968815+0000","last_undegraded":"2024-01-15T18:16:01.968815+0000","last_fullsized":"2024-01-15T18:16:01.968815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957808+0000","last_change":"2024-01-15T18:16:01.957808+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957808+0000","last_undegraded":"2024-01-15T18:16:01.957808+0000","last_fullsized":"2024-01-15T18:16:01.957808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971256+0000","last_change":"2024-01-15T18:16:01.971126+0000","last_active":"2024-01-15T18:16:02.971256+0000","last_peered":"2024-01-15T18:16:02.971256+0000","last_clean":"2024-01-15T18:16:02.971256+0000","last_became_active":"2024-01-15T18:16:01.969238+0000","last_became_peered":"2024-01-15T18:16:01.969238+0000","last_unstale":"2024-01-15T18:16:02.971256+0000","last_undegraded":"2024-01-15T18:16:02.971256+0000","last_fullsized":"2024-01-15T18:16:02.971256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:30.543482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954474+0000","last_change":"2024-01-15T18:16:01.954474+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954474+0000","last_undegraded":"2024-01-15T18:16:01.954474+0000","last_fullsized":"2024-01-15T18:16:01.954474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980185+0000","last_change":"2024-01-15T18:16:01.979247+0000","last_active":"2024-01-15T18:16:01.980185+0000","last_peered":"2024-01-15T18:16:01.980185+0000","last_clean":"2024-01-15T18:16:01.980185+0000","last_became_active":"2024-01-15T18:16:01.978885+0000","last_became_peered":"2024-01-15T18:16:01.978885+0000","last_unstale":"2024-01-15T18:16:01.980185+0000","last_undegraded":"2024-01-15T18:16:01.980185+0000","last_fullsized":"2024-01-15T18:16:01.980185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:42.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984081+0000","last_change":"2024-01-15T18:16:02.983891+0000","last_active":"2024-01-15T18:16:03.984081+0000","last_peered":"2024-01-15T18:16:03.984081+0000","last_clean":"2024-01-15T18:16:03.984081+0000","last_became_active":"2024-01-15T18:16:02.983591+0000","last_became_peered":"2024-01-15T18:16:02.983591+0000","last_unstale":"2024-01-15T18:16:03.984081+0000","last_undegraded":"2024-01-15T18:16:03.984081+0000","last_fullsized":"2024-01-15T18:16:03.984081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:32.572896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.998942+0000","last_change":"2024-01-15T18:16:01.971805+0000","last_active":"2024-01-15T18:16:02.998942+0000","last_peered":"2024-01-15T18:16:02.998942+0000","last_clean":"2024-01-15T18:16:02.998942+0000","last_became_active":"2024-01-15T18:16:01.971492+0000","last_became_peered":"2024-01-15T18:16:01.971492+0000","last_unstale":"2024-01-15T18:16:02.998942+0000","last_undegraded":"2024-01-15T18:16:02.998942+0000","last_fullsized":"2024-01-15T18:16:02.998942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.974741+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.974455+0000","last_became_peered":"2024-01-15T18:16:02.974455+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973667+0000","last_change":"2024-01-15T18:16:01.970998+0000","last_active":"2024-01-15T18:16:01.973667+0000","last_peered":"2024-01-15T18:16:01.973667+0000","last_clean":"2024-01-15T18:16:01.973667+0000","last_became_active":"2024-01-15T18:16:01.970730+0000","last_became_peered":"2024-01-15T18:16:01.970730+0000","last_unstale":"2024-01-15T18:16:01.973667+0000","last_undegraded":"2024-01-15T18:16:01.973667+0000","last_fullsized":"2024-01-15T18:16:01.973667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983546+0000","last_change":"2024-01-15T18:16:02.989297+0000","last_active":"2024-01-15T18:16:03.983546+0000","last_peered":"2024-01-15T18:16:03.983546+0000","last_clean":"2024-01-15T18:16:03.983546+0000","last_became_active":"2024-01-15T18:16:02.988832+0000","last_became_peered":"2024-01-15T18:16:02.988832+0000","last_unstale":"2024-01-15T18:16:03.983546+0000","last_undegraded":"2024-01-15T18:16:03.983546+0000","last_fullsized":"2024-01-15T18:16:03.983546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:07.504686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.977256+0000","last_change":"2024-01-15T18:16:01.975857+0000","last_active":"2024-01-15T18:16:01.977256+0000","last_peered":"2024-01-15T18:16:01.977256+0000","last_clean":"2024-01-15T18:16:01.977256+0000","last_became_active":"2024-01-15T18:16:01.975585+0000","last_became_peered":"2024-01-15T18:16:01.975585+0000","last_unstale":"2024-01-15T18:16:01.977256+0000","last_undegraded":"2024-01-15T18:16:01.977256+0000","last_fullsized":"2024-01-15T18:16:01.977256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975212+0000","last_change":"2024-01-15T18:16:01.972470+0000","last_active":"2024-01-15T18:16:01.975212+0000","last_peered":"2024-01-15T18:16:01.975212+0000","last_clean":"2024-01-15T18:16:01.975212+0000","last_became_active":"2024-01-15T18:16:01.972202+0000","last_became_peered":"2024-01-15T18:16:01.972202+0000","last_unstale":"2024-01-15T18:16:01.975212+0000","last_undegraded":"2024-01-15T18:16:01.975212+0000","last_fullsized":"2024-01-15T18:16:01.975212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672268+0000","last_change":"2024-01-15T18:16:02.984567+0000","last_active":"2024-01-15T18:16:04.672268+0000","last_peered":"2024-01-15T18:16:04.672268+0000","last_clean":"2024-01-15T18:16:04.672268+0000","last_became_active":"2024-01-15T18:16:02.984223+0000","last_became_peered":"2024-01-15T18:16:02.984223+0000","last_unstale":"2024-01-15T18:16:04.672268+0000","last_undegraded":"2024-01-15T18:16:04.672268+0000","last_fullsized":"2024-01-15T18:16:04.672268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.952954+0000","last_change":"2024-01-15T18:16:00.952954+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.952954+0000","last_undegraded":"2024-01-15T18:16:00.952954+0000","last_fullsized":"2024-01-15T18:16:00.952954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.953595+0000","last_change":"2024-01-15T18:16:00.953595+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.953595+0000","last_undegraded":"2024-01-15T18:16:00.953595+0000","last_fullsized":"2024-01-15T18:16:00.953595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672166+0000","last_change":"2024-01-15T18:16:02.984528+0000","last_active":"2024-01-15T18:16:04.672166+0000","last_peered":"2024-01-15T18:16:04.672166+0000","last_clean":"2024-01-15T18:16:04.672166+0000","last_became_active":"2024-01-15T18:16:02.984179+0000","last_became_peered":"2024-01-15T18:16:02.984179+0000","last_unstale":"2024-01-15T18:16:04.672166+0000","last_undegraded":"2024-01-15T18:16:04.672166+0000","last_fullsized":"2024-01-15T18:16:04.672166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975902+0000","last_change":"2024-01-15T18:16:01.974336+0000","last_active":"2024-01-15T18:16:01.975902+0000","last_peered":"2024-01-15T18:16:01.975902+0000","last_clean":"2024-01-15T18:16:01.975902+0000","last_became_active":"2024-01-15T18:16:01.974035+0000","last_became_peered":"2024-01-15T18:16:01.974035+0000","last_unstale":"2024-01-15T18:16:01.975902+0000","last_undegraded":"2024-01-15T18:16:01.975902+0000","last_fullsized":"2024-01-15T18:16:01.975902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983695+0000","last_change":"2024-01-15T18:16:02.981820+0000","last_active":"2024-01-15T18:16:03.983695+0000","last_peered":"2024-01-15T18:16:03.983695+0000","last_clean":"2024-01-15T18:16:03.983695+0000","last_became_active":"2024-01-15T18:16:02.981478+0000","last_became_peered":"2024-01-15T18:16:02.981478+0000","last_unstale":"2024-01-15T18:16:03.983695+0000","last_undegraded":"2024-01-15T18:16:03.983695+0000","last_fullsized":"2024-01-15T18:16:03.983695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.957714+0000","last_change":"2024-01-15T18:16:01.957714+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957714+0000","last_undegraded":"2024-01-15T18:16:01.957714+0000","last_fullsized":"2024-01-15T18:16:01.957714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:50.381475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984005+0000","last_change":"2024-01-15T18:16:02.981174+0000","last_active":"2024-01-15T18:16:03.984005+0000","last_peered":"2024-01-15T18:16:03.984005+0000","last_clean":"2024-01-15T18:16:03.984005+0000","last_became_active":"2024-01-15T18:16:02.980913+0000","last_became_peered":"2024-01-15T18:16:02.980913+0000","last_unstale":"2024-01-15T18:16:03.984005+0000","last_undegraded":"2024-01-15T18:16:03.984005+0000","last_fullsized":"2024-01-15T18:16:03.984005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:12.319403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.969433+0000","last_change":"2024-01-15T18:16:01.967331+0000","last_active":"2024-01-15T18:16:01.969433+0000","last_peered":"2024-01-15T18:16:01.969433+0000","last_clean":"2024-01-15T18:16:01.969433+0000","last_became_active":"2024-01-15T18:16:01.967031+0000","last_became_peered":"2024-01-15T18:16:01.967031+0000","last_unstale":"2024-01-15T18:16:01.969433+0000","last_undegraded":"2024-01-15T18:16:01.969433+0000","last_fullsized":"2024-01-15T18:16:01.969433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:37.438291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172397+0000","last_change":"2024-01-15T18:16:02.969429+0000","last_active":"2024-01-15T18:16:04.172397+0000","last_peered":"2024-01-15T18:16:04.172397+0000","last_clean":"2024-01-15T18:16:04.172397+0000","last_became_active":"2024-01-15T18:16:02.969095+0000","last_became_peered":"2024-01-15T18:16:02.969095+0000","last_unstale":"2024-01-15T18:16:04.172397+0000","last_undegraded":"2024-01-15T18:16:04.172397+0000","last_fullsized":"2024-01-15T18:16:04.172397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964906+0000","last_change":"2024-01-15T18:16:01.968688+0000","last_active":"2024-01-15T18:16:02.964906+0000","last_peered":"2024-01-15T18:16:02.964906+0000","last_clean":"2024-01-15T18:16:02.964906+0000","last_became_active":"2024-01-15T18:16:01.968389+0000","last_became_peered":"2024-01-15T18:16:01.968389+0000","last_unstale":"2024-01-15T18:16:02.964906+0000","last_undegraded":"2024-01-15T18:16:02.964906+0000","last_fullsized":"2024-01-15T18:16:02.964906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.976712+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.976054+0000","last_became_peered":"2024-01-15T18:16:02.976054+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:26:19.031570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964799+0000","last_change":"2024-01-15T18:16:01.966697+0000","last_active":"2024-01-15T18:16:02.964799+0000","last_peered":"2024-01-15T18:16:02.964799+0000","last_clean":"2024-01-15T18:16:02.964799+0000","last_became_active":"2024-01-15T18:16:01.966397+0000","last_became_peered":"2024-01-15T18:16:01.966397+0000","last_unstale":"2024-01-15T18:16:02.964799+0000","last_undegraded":"2024-01-15T18:16:02.964799+0000","last_fullsized":"2024-01-15T18:16:02.964799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672238+0000","last_change":"2024-01-15T18:16:02.984048+0000","last_active":"2024-01-15T18:16:04.672238+0000","last_peered":"2024-01-15T18:16:04.672238+0000","last_clean":"2024-01-15T18:16:04.672238+0000","last_became_active":"2024-01-15T18:16:02.983546+0000","last_became_peered":"2024-01-15T18:16:02.983546+0000","last_unstale":"2024-01-15T18:16:04.672238+0000","last_undegraded":"2024-01-15T18:16:04.672238+0000","last_fullsized":"2024-01-15T18:16:04.672238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980104+0000","last_change":"2024-01-15T18:16:01.979160+0000","last_active":"2024-01-15T18:16:01.980104+0000","last_peered":"2024-01-15T18:16:01.980104+0000","last_clean":"2024-01-15T18:16:01.980104+0000","last_became_active":"2024-01-15T18:16:01.978901+0000","last_became_peered":"2024-01-15T18:16:01.978901+0000","last_unstale":"2024-01-15T18:16:01.980104+0000","last_undegraded":"2024-01-15T18:16:01.980104+0000","last_fullsized":"2024-01-15T18:16:01.980104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954987+0000","last_change":"2024-01-15T18:16:01.954987+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954987+0000","last_undegraded":"2024-01-15T18:16:01.954987+0000","last_fullsized":"2024-01-15T18:16:01.954987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.961378+0000","last_change":"2024-01-15T18:16:01.958840+0000","last_active":"2024-01-15T18:16:02.961378+0000","last_peered":"2024-01-15T18:16:02.961378+0000","last_clean":"2024-01-15T18:16:02.961378+0000","last_became_active":"2024-01-15T18:16:01.958367+0000","last_became_peered":"2024-01-15T18:16:01.958367+0000","last_unstale":"2024-01-15T18:16:02.961378+0000","last_undegraded":"2024-01-15T18:16:02.961378+0000","last_fullsized":"2024-01-15T18:16:02.961378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983304+0000","last_change":"2024-01-15T18:16:02.991930+0000","last_active":"2024-01-15T18:16:03.983304+0000","last_peered":"2024-01-15T18:16:03.983304+0000","last_clean":"2024-01-15T18:16:03.983304+0000","last_became_active":"2024-01-15T18:16:02.991630+0000","last_became_peered":"2024-01-15T18:16:02.991630+0000","last_unstale":"2024-01-15T18:16:03.983304+0000","last_undegraded":"2024-01-15T18:16:03.983304+0000","last_fullsized":"2024-01-15T18:16:03.983304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964174+0000","last_change":"2024-01-15T18:16:01.958054+0000","last_active":"2024-01-15T18:16:02.964174+0000","last_peered":"2024-01-15T18:16:02.964174+0000","last_clean":"2024-01-15T18:16:02.964174+0000","last_became_active":"2024-01-15T18:16:01.957368+0000","last_became_peered":"2024-01-15T18:16:01.957368+0000","last_unstale":"2024-01-15T18:16:02.964174+0000","last_undegraded":"2024-01-15T18:16:02.964174+0000","last_fullsized":"2024-01-15T18:16:02.964174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172054+0000","last_change":"2024-01-15T18:16:02.970069+0000","last_active":"2024-01-15T18:16:04.172054+0000","last_peered":"2024-01-15T18:16:04.172054+0000","last_clean":"2024-01-15T18:16:04.172054+0000","last_became_active":"2024-01-15T18:16:02.969737+0000","last_became_peered":"2024-01-15T18:16:02.969737+0000","last_unstale":"2024-01-15T18:16:04.172054+0000","last_undegraded":"2024-01-15T18:16:04.172054+0000","last_fullsized":"2024-01-15T18:16:04.172054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:06.396100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.963102+0000","last_change":"2024-01-15T18:16:01.968677+0000","last_active":"2024-01-15T18:16:02.963102+0000","last_peered":"2024-01-15T18:16:02.963102+0000","last_clean":"2024-01-15T18:16:02.963102+0000","last_became_active":"2024-01-15T18:16:01.967462+0000","last_became_peered":"2024-01-15T18:16:01.967462+0000","last_unstale":"2024-01-15T18:16:02.963102+0000","last_undegraded":"2024-01-15T18:16:02.963102+0000","last_fullsized":"2024-01-15T18:16:02.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.959339+0000","last_change":"2024-01-15T18:16:01.959339+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.959339+0000","last_undegraded":"2024-01-15T18:16:01.959339+0000","last_fullsized":"2024-01-15T18:16:01.959339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000937+0000","last_change":"2024-01-15T18:16:01.973544+0000","last_active":"2024-01-15T18:16:03.000937+0000","last_peered":"2024-01-15T18:16:03.000937+0000","last_clean":"2024-01-15T18:16:03.000937+0000","last_became_active":"2024-01-15T18:16:01.973240+0000","last_became_peered":"2024-01-15T18:16:01.973240+0000","last_unstale":"2024-01-15T18:16:03.000937+0000","last_undegraded":"2024-01-15T18:16:03.000937+0000","last_fullsized":"2024-01-15T18:16:03.000937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:06.496321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971217+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:16:02.971217+0000","last_peered":"2024-01-15T18:16:02.971217+0000","last_clean":"2024-01-15T18:16:02.971217+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:16:02.971217+0000","last_undegraded":"2024-01-15T18:16:02.971217+0000","last_fullsized":"2024-01-15T18:16:02.971217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77164,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294676,"statfs":{"total":96636764160,"available":96557748224,"internally_reserved":0,"allocated":1667072,"data_stored":1050036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.3839999999999999,"15min":0.92200000000000004},"min":{"1min":0.621,"5min":0.36899999999999999,"15min":0.29599999999999999},"max":{"1min":16.815999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.044,"15min":0.87},"min":{"1min":0.57399999999999995,"5min":0.33400000000000002,"15min":0.28399999999999997},"max":{"1min":4.6100000000000003,"5min":8.5419999999999998,"15min":13.744},"last":0.441}]},{"osd":1,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.885,"5min":1.6519999999999999,"15min":1.0700000000000001},"min":{"1min":0.60599999999999998,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":16.920000000000002,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.637,"15min":1.0680000000000001},"min":{"1min":0.60299999999999998,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":4.7679999999999998,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.496,"15min":0.99199999999999999},"min":{"1min":0.52000000000000002,"5min":0.433,"15min":0.38400000000000001},"max":{"1min":2.1680000000000001,"5min":30.846,"15min":30.846},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.887,"5min":1.3140000000000001,"15min":1.097},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.39800000000000002},"max":{"1min":17.039999999999999,"5min":17.039999999999999,"15min":39.695},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.6299999999999999,"15min":1.163},"min":{"1min":0.55300000000000005,"5min":0.47499999999999998,"15min":0.38800000000000001},"max":{"1min":2.1160000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.831,"5min":2.0049999999999999,"15min":1.228},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":16.879999999999999,"5min":42.692,"15min":42.692},"last":0.70199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.89100000000000001,"15min":0.72199999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":14.087,"5min":14.087,"15min":14.087},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.36699999999999999,"15min":0.36299999999999999},"max":{"1min":2.2450000000000001,"5min":2.952,"15min":4.3710000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":0.80500000000000005,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":11.334,"5min":11.334,"15min":11.334},"last":0.505},{"interface":"front","average":{"1min":1.012,"5min":0.91000000000000003,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.8179999999999996,"5min":8.423,"15min":8.423},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.91300000000000003,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.7140000000000004,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.376},"max":{"1min":1.1970000000000001,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76588,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295252,"statfs":{"total":96636764160,"available":96558338048,"internally_reserved":0,"allocated":1081344,"data_stored":460292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.83799999999999997,"15min":0.94799999999999995},"min":{"1min":0.41399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78200000000000003,"15min":0.872},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.266,"5min":3.4940000000000002,"15min":12.449},"last":0.81599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.1379999999999999,"15min":1.0109999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.379},"max":{"1min":1.615,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.087,"15min":1.1759999999999999},"min":{"1min":0.55600000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":14.743,"15min":42.825000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":1.147},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1890000000000001,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78100000000000003,"15min":1.0269999999999999},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":1.3640000000000001,"5min":3.016,"15min":24.199000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.0489999999999999,"15min":1.0629999999999999},"min":{"1min":0.60699999999999998,"5min":0.47799999999999998,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":18.791,"15min":18.791},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.585,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.66100000000000003,"15min":0.75800000000000001},"min":{"1min":0.50800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":12.491},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.443},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.4379999999999999,"5min":2.5259999999999998,"15min":12.945},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.98599999999999999,"15min":0.84599999999999997},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":5.1829999999999998,"5min":11.907,"15min":11.907},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80800000000000005,"15min":0.88900000000000001},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5569999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.84199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54456,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317384,"statfs":{"total":96636764160,"available":96581001216,"internally_reserved":0,"allocated":1085440,"data_stored":461232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":0.82899999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.4770000000000003,"5min":6.6820000000000004,"15min":11.519},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.95699999999999996,"15min":0.90300000000000002},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.34999999999999998},"max":{"1min":1.714,"5min":7.6260000000000003,"15min":12.952},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95499999999999996,"15min":0.88500000000000001},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":4.3140000000000001,"5min":4.609,"15min":13.358000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.859,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.149,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":2.867,"5min":18.314,"15min":18.314},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.341,"15min":1.006},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.434},"max":{"1min":2.855,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.89500000000000002,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":1.839,"5min":4.0330000000000004,"15min":13.317},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.034,"15min":1},"min":{"1min":0.443,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.1440000000000001,"5min":15.205,"15min":19.638000000000002},"last":0.74399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.28799999999999998},"max":{"1min":2.4289999999999998,"5min":5.899,"15min":11.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32100000000000001},"max":{"1min":2.516,"5min":3.097,"15min":3.3889999999999998},"last":0.623}]},{"osd":6,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.80000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.238,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.9470000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.76200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":2.3900000000000001,"5min":2.9870000000000001,"15min":4.1799999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.40400000000000003},"max":{"1min":2.3239999999999998,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":1069056,"data_stored":459598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.097,"15min":0.92600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":9.4179999999999993,"5min":11.374000000000001,"15min":34.802},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.135,"15min":0.89300000000000002},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.34699999999999998},"max":{"1min":3.3450000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.97399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2130000000000001,"5min":7.625,"15min":38.198},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.93999999999999995,"15min":0.95799999999999996},"min":{"1min":0.55700000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0589999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0149999999999999,"15min":0.85499999999999998},"min":{"1min":0.61499999999999999,"5min":0.40799999999999997,"15min":0.38700000000000001},"max":{"1min":4.4089999999999998,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.621}]},{"osd":3,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.2729999999999999,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":3.415,"5min":14.529,"15min":14.529},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.2430000000000001,"15min":0.93300000000000005},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78100000000000003,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.6070000000000002,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.96899999999999997,"15min":0.83099999999999996},"min":{"1min":0.47499999999999998,"5min":0.39200000000000002,"15min":0.34100000000000003},"max":{"1min":3.0209999999999999,"5min":25.887,"15min":25.887},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":7.5439999999999996,"15min":10.284000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85999999999999999,"15min":0.79300000000000004},"min":{"1min":0.45000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1680000000000001,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":1.232,"5min":11.67,"15min":11.67},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.595,"15min":0.95199999999999996},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":7.2839999999999998,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.37},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.4670000000000001,"15min":0.94799999999999995},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":7.5170000000000003,"5min":23.074999999999999,"15min":28.332000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.4690000000000001,"15min":0.96299999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.2970000000000002,"5min":20.321000000000002,"15min":40.457999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.514,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.372,"15min":0.30299999999999999},"max":{"1min":7.3170000000000002,"5min":26.260999999999999,"15min":27.427},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.615,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.28399999999999997},"max":{"1min":9.1180000000000003,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.8240000000000001,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.34300000000000003},"max":{"1min":7.2409999999999997,"5min":26.404,"15min":26.404},"last":0.53100000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3819999999999999,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":4.1040000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.069},"min":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.377},"max":{"1min":2.1150000000000002,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.6259999999999999,"15min":1.097},"min":{"1min":0.58399999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":6.5140000000000002,"5min":26.314,"15min":26.314},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.5649999999999999,"15min":1.131},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.44500000000000001},"max":{"1min":4.165,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.94399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.635,"15min":1.1140000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.7619999999999996,"5min":20.445,"15min":20.445},"last":1},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.7270000000000001,"15min":1.1699999999999999},"min":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.495},"max":{"1min":6.7149999999999999,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.6839999999999999,"15min":1.155},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.2030000000000003,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":2.1429999999999998,"15min":1.3089999999999999},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46000000000000002},"max":{"1min":2.2490000000000001,"5min":44.817999999999998,"15min":44.817999999999998},"last":1.0900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.153,"15min":1.0449999999999999},"min":{"1min":0.33800000000000002,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9950000000000001,"5min":12.695,"15min":29.527999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.591,"15min":1.3180000000000001},"min":{"1min":0.47499999999999998,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.288,"15min":1.5329999999999999},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":7.476,"5min":34.029000000000003,"15min":113.149},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.099},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.30199999999999999},"max":{"1min":14.728,"5min":20.619,"15min":41.302},"last":0.499}]},{"osd":3,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":2.1030000000000002,"15min":1.3620000000000001},"min":{"1min":0.36799999999999999,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":31.57,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":1.258},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":7.3609999999999998,"5min":7.3609999999999998,"15min":115.693},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1279999999999999,"15min":1.2090000000000001},"min":{"1min":0.627,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":7.4169999999999998,"5min":7.4169999999999998,"15min":24.366},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.3049999999999999,"15min":1.2090000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0580000000000001,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":2.548,"5min":2.222,"15min":1.482},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.43099999999999999},"max":{"1min":41.683999999999997,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.103,"5min":1.5169999999999999,"15min":1.3129999999999999},"min":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":7.5099999999999998,"5min":26.084,"15min":26.084},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.468,"15min":1.302},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.208,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.746},{"interface":"front","average":{"1min":2.589,"5min":2.3210000000000002,"15min":1.556},"min":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.47699999999999998},"max":{"1min":41.738999999999997,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.3979999999999999,"15min":1.242},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":1.298,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":2.6000000000000001,"5min":2.383,"15min":1.629},"min":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.48799999999999999},"max":{"1min":41.402000000000001,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.97399999999999998,"15min":0.92500000000000004},"min":{"1min":0.33100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.3330000000000002,"5min":7.1580000000000004,"15min":39.948},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.5660000000000001,"15min":1.1200000000000001},"min":{"1min":0.39600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.577,"5min":35.790999999999997,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.034,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":6.516,"5min":11.346,"15min":49.229999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.1870000000000001,"15min":0.86499999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.931,"5min":7.9489999999999998,"15min":10.349},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.655,"15min":1.2190000000000001},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4669999999999996,"5min":27.643999999999998,"15min":27.643999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.343,"15min":1.016},"min":{"1min":0.48499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":15.608000000000001,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.6819999999999999,"15min":1.276},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.40200000000000002},"max":{"1min":4.633,"5min":27.489999999999998,"15min":27.489999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2969999999999999,"15min":1.054},"min":{"1min":0.61099999999999999,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":5.9139999999999997,"5min":10.038,"15min":12.997},"last":0.70699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.7050000000000001,"15min":1.2390000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.50800000000000001},"max":{"1min":4.673,"5min":27.547000000000001,"15min":27.547000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.1419999999999999},"min":{"1min":0.68400000000000005,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":5.3399999999999999,"5min":9.9610000000000003,"15min":13.054},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.698,"15min":1.2490000000000001},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":4.7060000000000004,"5min":27.579999999999998,"15min":27.579999999999998},"last":0.746},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.198,"15min":1.0760000000000001},"min":{"1min":0.60199999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0840000000000001,"5min":11.643000000000001,"15min":11.712999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3600000000000001,"15min":1.145},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46500000000000002},"max":{"1min":9.5660000000000007,"5min":11.833,"15min":11.833},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.226,"5min":1.4450000000000001,"15min":1.153},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":5.9550000000000001,"5min":14.606,"15min":14.606},"last":0.83799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77120,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294720,"statfs":{"total":96636764160,"available":96557793280,"internally_reserved":0,"allocated":1626112,"data_stored":1030448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.4470000000000001,"15min":1.175},"min":{"1min":0.32500000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":1.137,"5min":16.149000000000001,"15min":31.977},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.254,"15min":0.94699999999999995},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70999999999999996,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.63700000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":1.6719999999999999,"15min":1.0820000000000001},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.30199999999999999},"max":{"1min":1.7450000000000001,"5min":36.386000000000003,"15min":41.387999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2689999999999999,"15min":1.0429999999999999},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.25},"max":{"1min":11.539999999999999,"5min":16.556000000000001,"15min":33.219999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.3260000000000001,"15min":1.1140000000000001},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":1.2230000000000001,"5min":9.8740000000000006,"15min":31.943999999999999},"last":0.623},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.7689999999999999,"15min":1.0600000000000001},"min":{"1min":0.499,"5min":0.432,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":45.128,"15min":45.128},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.2709999999999999,"15min":1.109},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":11.772,"5min":11.772,"15min":32.932000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.379,"15min":1.1870000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":14.831,"15min":31.782},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.306,"15min":1.1850000000000001},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.43099999999999999},"max":{"1min":1.383,"5min":11.247999999999999,"15min":31.858000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.2929999999999999,"15min":1.1080000000000001},"min":{"1min":0.58399999999999996,"5min":0.52200000000000002,"15min":0.45800000000000002},"max":{"1min":11.958,"5min":11.958,"15min":33.151000000000003},"last":0.748}]},{"osd":6,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3520000000000001,"15min":1.052},"min":{"1min":0.622,"5min":0.49399999999999999,"15min":0.46400000000000002},"max":{"1min":1.1479999999999999,"5min":13.055,"15min":13.055},"last":0.871},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.3460000000000001,"15min":1.137},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.498},"max":{"1min":11.833,"5min":11.833,"15min":33.098999999999997},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.4279999999999999,"15min":1.0580000000000001},"min":{"1min":0.64700000000000002,"5min":0.52000000000000002,"15min":0.501},"max":{"1min":1.091,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.4630000000000001,"15min":1.2430000000000001},"min":{"1min":0.60299999999999998,"5min":0.53700000000000003,"15min":0.504},"max":{"1min":1.335,"5min":10.377000000000001,"15min":31.905999999999999},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:16:07.431 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:16:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.610+0000 7f393ce00640 1 Processor -- start 2024-01-15T18:16:07.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.610+0000 7f393ce00640 1 -- start start 2024-01-15T18:16:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f393ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f393806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f393ce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f39380cbf50 0x7f39380cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f393ce00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39380cffd0 con 0x7f39381607c0 2024-01-15T18:16:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f393ce00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39380d1ce0 con 0x7f39380cbf50 2024-01-15T18:16:07.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f393ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39380d02b0 con 0x7f39380cfb30 2024-01-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f393806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f39381607c0 0x7f393806b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58216/0 (socket says 172.21.15.143:58216) 2024-01-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935e00640 1 -- 172.21.15.143:0/3263476678 learned_addr learned my addr 172.21.15.143:0/3263476678 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3936800640 1 --2- 172.21.15.143:0/3263476678 >> v2:172.21.15.204:3300/0 conn(0x7f39380cbf50 0x7f39380cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3934a00640 1 -- 172.21.15.143:0/3263476678 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24920738 0 0) 0x7f39380cffd0 con 0x7f39381607c0 2024-01-15T18:16:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935400640 1 -- 172.21.15.143:0/3263476678 >> v2:172.21.15.204:3300/0 conn(0x7f39380cbf50 msgr2=0x7f39380cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935400640 1 --2- 172.21.15.143:0/3263476678 >> v2:172.21.15.204:3300/0 conn(0x7f39380cbf50 0x7f39380cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935400640 1 -- 172.21.15.143:0/3263476678 >> v1:172.21.15.204:6789/0 conn(0x7f39381607c0 legacy=0x7f393806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.614+0000 7f3935400640 1 -- 172.21.15.143:0/3263476678 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39380cb470 con 0x7f39380cfb30 2024-01-15T18:16:07.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f3935400640 1 --2- 172.21.15.143:0/3263476678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f393806bec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39200097c0 tx=0x7f392001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19d7a2bcd0c460a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f3934a00640 1 -- 172.21.15.143:0/3263476678 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200a8970 con 0x7f39380cfb30 2024-01-15T18:16:07.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f3934a00640 1 -- 172.21.15.143:0/3263476678 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39200a8ad0 con 0x7f39380cfb30 2024-01-15T18:16:07.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f3934a00640 1 -- 172.21.15.143:0/3263476678 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200a8e60 con 0x7f39380cfb30 2024-01-15T18:16:07.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 -- 172.21.15.143:0/3263476678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 msgr2=0x7f393806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 --2- 172.21.15.143:0/3263476678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f393806bec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39200097c0 tx=0x7f392001ec70 comp rx=0 tx=0).stop 2024-01-15T18:16:07.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 -- 172.21.15.143:0/3263476678 shutdown_connections 2024-01-15T18:16:07.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 --2- 172.21.15.143:0/3263476678 >> v2:172.21.15.204:3300/0 conn(0x7f39380cbf50 0x7f39380cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 --2- 172.21.15.143:0/3263476678 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f393806bec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 -- 172.21.15.143:0/3263476678 >> 172.21.15.143:0/3263476678 conn(0x7f39380d0e20 msgr2=0x7f39380d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 -- 172.21.15.143:0/3263476678 shutdown_connections 2024-01-15T18:16:07.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 -- 172.21.15.143:0/3263476678 wait complete. 2024-01-15T18:16:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 Processor -- start 2024-01-15T18:16:07.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.618+0000 7f393ce00640 1 -- start start 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39380cbf50 0x7f39381f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f3938157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 0x7f3938158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39380ce3e0 con 0x7f39381607c0 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39380ce100 con 0x7f39380cbf50 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39380c5420 con 0x7f39380cfb30 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 0x7f3938158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3935e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39380cbf50 0x7f39381f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 0x7f3938158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55026/0 (socket says 172.21.15.143:55026) 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 -- 172.21.15.143:0/1050872722 learned_addr learned my addr 172.21.15.143:0/1050872722 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3935400640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f3938157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 -- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 msgr2=0x7f3938157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f3938157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 -- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39380cbf50 msgr2=0x7f39381f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39380cbf50 0x7f39381f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 -- 172.21.15.143:0/1050872722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3920009420 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f3936800640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 0x7f3938158200 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f393000efc0 tx=0x7f393000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930014480 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393000f040 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39380c5700 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930014820 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39381f2870 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.622+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393806d4b0 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.626+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f3930014980 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.626+0000 7f392e000640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39100675e0 0x7f3910069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.626+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3930083bd0 con 0x7f39381607c0 2024-01-15T18:16:07.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.626+0000 7f3935e00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39100675e0 0x7f3910069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.626+0000 7f3935e00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39100675e0 0x7f3910069aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3928000f70 tx=0x7f3928007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.630+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f39300525c0 con 0x7f39381607c0 2024-01-15T18:16:07.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.778+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39380ce6b0 con 0x7f39100675e0 2024-01-15T18:16:07.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.786+0000 7f392e000640 1 -- 172.21.15.143:0/1050872722 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285130 (secure 0 0 0) 0x7f39280072a0 con 0x7f39100675e0 2024-01-15T18:16:07.784 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:07.785 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39100675e0 msgr2=0x7f3910069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39100675e0 0x7f3910069aa0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3928000f70 tx=0x7f3928007df0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 msgr2=0x7f3938158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 0x7f3938158200 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f393000efc0 tx=0x7f393000c490 comp rx=0 tx=0).stop 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 shutdown_connections 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39100675e0 0x7f3910069aa0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39381607c0 0x7f3938158200 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39380cfb30 0x7f3938157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 --2- 172.21.15.143:0/1050872722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39380cbf50 0x7f39381f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 >> 172.21.15.143:0/1050872722 conn(0x7f39380d0e20 msgr2=0x7f393806f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 shutdown_connections 2024-01-15T18:16:07.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.790+0000 7f393ce00640 1 -- 172.21.15.143:0/1050872722 wait complete. 2024-01-15T18:16:07.805 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-01-15T18:16:06.858046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":591524,"kb_used_data":9400,"kb_used_omap":12,"kb_used_meta":582003,"kb_avail":754383196,"statfs":{"total":773094113280,"available":772488392704,"internally_reserved":0,"allocated":9625600,"data_stored":4781846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":595971632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913200"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172275+0000","last_change":"2024-01-15T18:16:02.977971+0000","last_active":"2024-01-15T18:16:04.172275+0000","last_peered":"2024-01-15T18:16:04.172275+0000","last_clean":"2024-01-15T18:16:04.172275+0000","last_became_active":"2024-01-15T18:16:02.977706+0000","last_became_peered":"2024-01-15T18:16:02.977706+0000","last_unstale":"2024-01-15T18:16:04.172275+0000","last_undegraded":"2024-01-15T18:16:04.172275+0000","last_fullsized":"2024-01-15T18:16:04.172275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958822+0000","last_change":"2024-01-15T18:16:01.958822+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958822+0000","last_undegraded":"2024-01-15T18:16:01.958822+0000","last_fullsized":"2024-01-15T18:16:01.958822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983752+0000","last_change":"2024-01-15T18:16:02.982423+0000","last_active":"2024-01-15T18:16:03.983752+0000","last_peered":"2024-01-15T18:16:03.983752+0000","last_clean":"2024-01-15T18:16:03.983752+0000","last_became_active":"2024-01-15T18:16:02.981906+0000","last_became_peered":"2024-01-15T18:16:02.981906+0000","last_unstale":"2024-01-15T18:16:03.983752+0000","last_undegraded":"2024-01-15T18:16:03.983752+0000","last_fullsized":"2024-01-15T18:16:03.983752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:02.961035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.988011+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.987583+0000","last_became_peered":"2024-01-15T18:16:02.987583+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672319+0000","last_change":"2024-01-15T18:16:02.984894+0000","last_active":"2024-01-15T18:16:04.672319+0000","last_peered":"2024-01-15T18:16:04.672319+0000","last_clean":"2024-01-15T18:16:04.672319+0000","last_became_active":"2024-01-15T18:16:02.984537+0000","last_became_peered":"2024-01-15T18:16:02.984537+0000","last_unstale":"2024-01-15T18:16:04.672319+0000","last_undegraded":"2024-01-15T18:16:04.672319+0000","last_fullsized":"2024-01-15T18:16:04.672319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672358+0000","last_change":"2024-01-15T18:16:02.988234+0000","last_active":"2024-01-15T18:16:04.672358+0000","last_peered":"2024-01-15T18:16:04.672358+0000","last_clean":"2024-01-15T18:16:04.672358+0000","last_became_active":"2024-01-15T18:16:02.987822+0000","last_became_peered":"2024-01-15T18:16:02.987822+0000","last_unstale":"2024-01-15T18:16:04.672358+0000","last_undegraded":"2024-01-15T18:16:04.672358+0000","last_fullsized":"2024-01-15T18:16:04.672358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672449+0000","last_change":"2024-01-15T18:16:02.978613+0000","last_active":"2024-01-15T18:16:04.672449+0000","last_peered":"2024-01-15T18:16:04.672449+0000","last_clean":"2024-01-15T18:16:04.672449+0000","last_became_active":"2024-01-15T18:16:02.978331+0000","last_became_peered":"2024-01-15T18:16:02.978331+0000","last_unstale":"2024-01-15T18:16:04.672449+0000","last_undegraded":"2024-01-15T18:16:04.672449+0000","last_fullsized":"2024-01-15T18:16:04.672449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.985584+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.984527+0000","last_became_peered":"2024-01-15T18:16:02.984527+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.960055+0000","last_change":"2024-01-15T18:16:01.960055+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.960055+0000","last_undegraded":"2024-01-15T18:16:01.960055+0000","last_fullsized":"2024-01-15T18:16:01.960055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.171943+0000","last_change":"2024-01-15T18:16:02.976922+0000","last_active":"2024-01-15T18:16:04.171943+0000","last_peered":"2024-01-15T18:16:04.171943+0000","last_clean":"2024-01-15T18:16:04.171943+0000","last_became_active":"2024-01-15T18:16:02.976638+0000","last_became_peered":"2024-01-15T18:16:02.976638+0000","last_unstale":"2024-01-15T18:16:04.171943+0000","last_undegraded":"2024-01-15T18:16:04.171943+0000","last_fullsized":"2024-01-15T18:16:04.171943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:28.008839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.980605+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.978977+0000","last_became_peered":"2024-01-15T18:16:02.978977+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:17.866154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954608+0000","last_change":"2024-01-15T18:16:01.954608+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954608+0000","last_undegraded":"2024-01-15T18:16:01.954608+0000","last_fullsized":"2024-01-15T18:16:01.954608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982727+0000","last_change":"2024-01-15T18:16:02.979861+0000","last_active":"2024-01-15T18:16:03.982727+0000","last_peered":"2024-01-15T18:16:03.982727+0000","last_clean":"2024-01-15T18:16:03.982727+0000","last_became_active":"2024-01-15T18:16:02.979569+0000","last_became_peered":"2024-01-15T18:16:02.979569+0000","last_unstale":"2024-01-15T18:16:03.982727+0000","last_undegraded":"2024-01-15T18:16:03.982727+0000","last_fullsized":"2024-01-15T18:16:03.982727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984023+0000","last_change":"2024-01-15T18:16:02.977093+0000","last_active":"2024-01-15T18:16:03.984023+0000","last_peered":"2024-01-15T18:16:03.984023+0000","last_clean":"2024-01-15T18:16:03.984023+0000","last_became_active":"2024-01-15T18:16:02.974914+0000","last_became_peered":"2024-01-15T18:16:02.974914+0000","last_unstale":"2024-01-15T18:16:03.984023+0000","last_undegraded":"2024-01-15T18:16:03.984023+0000","last_fullsized":"2024-01-15T18:16:03.984023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.961445+0000","last_change":"2024-01-15T18:16:01.961445+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.961445+0000","last_undegraded":"2024-01-15T18:16:01.961445+0000","last_fullsized":"2024-01-15T18:16:01.961445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957359+0000","last_change":"2024-01-15T18:16:01.957359+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957359+0000","last_undegraded":"2024-01-15T18:16:01.957359+0000","last_fullsized":"2024-01-15T18:16:01.957359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172017+0000","last_change":"2024-01-15T18:16:02.970218+0000","last_active":"2024-01-15T18:16:04.172017+0000","last_peered":"2024-01-15T18:16:04.172017+0000","last_clean":"2024-01-15T18:16:04.172017+0000","last_became_active":"2024-01-15T18:16:02.969901+0000","last_became_peered":"2024-01-15T18:16:02.969901+0000","last_unstale":"2024-01-15T18:16:04.172017+0000","last_undegraded":"2024-01-15T18:16:04.172017+0000","last_fullsized":"2024-01-15T18:16:04.172017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:31.551846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958084+0000","last_change":"2024-01-15T18:16:01.958084+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958084+0000","last_undegraded":"2024-01-15T18:16:01.958084+0000","last_fullsized":"2024-01-15T18:16:01.958084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983932+0000","last_change":"2024-01-15T18:16:02.991391+0000","last_active":"2024-01-15T18:16:03.983932+0000","last_peered":"2024-01-15T18:16:03.983932+0000","last_clean":"2024-01-15T18:16:03.983932+0000","last_became_active":"2024-01-15T18:16:02.991138+0000","last_became_peered":"2024-01-15T18:16:02.991138+0000","last_unstale":"2024-01-15T18:16:03.983932+0000","last_undegraded":"2024-01-15T18:16:03.983932+0000","last_fullsized":"2024-01-15T18:16:03.983932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:06.968039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983406+0000","last_change":"2024-01-15T18:16:02.976507+0000","last_active":"2024-01-15T18:16:03.983406+0000","last_peered":"2024-01-15T18:16:03.983406+0000","last_clean":"2024-01-15T18:16:03.983406+0000","last_became_active":"2024-01-15T18:16:02.975765+0000","last_became_peered":"2024-01-15T18:16:02.975765+0000","last_unstale":"2024-01-15T18:16:03.983406+0000","last_undegraded":"2024-01-15T18:16:03.983406+0000","last_fullsized":"2024-01-15T18:16:03.983406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172094+0000","last_change":"2024-01-15T18:16:02.976103+0000","last_active":"2024-01-15T18:16:04.172094+0000","last_peered":"2024-01-15T18:16:04.172094+0000","last_clean":"2024-01-15T18:16:04.172094+0000","last_became_active":"2024-01-15T18:16:02.975829+0000","last_became_peered":"2024-01-15T18:16:02.975829+0000","last_unstale":"2024-01-15T18:16:04.172094+0000","last_undegraded":"2024-01-15T18:16:04.172094+0000","last_fullsized":"2024-01-15T18:16:04.172094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956707+0000","last_change":"2024-01-15T18:16:01.956707+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956707+0000","last_undegraded":"2024-01-15T18:16:01.956707+0000","last_fullsized":"2024-01-15T18:16:01.956707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956925+0000","last_change":"2024-01-15T18:16:01.956925+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956925+0000","last_undegraded":"2024-01-15T18:16:01.956925+0000","last_fullsized":"2024-01-15T18:16:01.956925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982746+0000","last_change":"2024-01-15T18:16:02.974470+0000","last_active":"2024-01-15T18:16:03.982746+0000","last_peered":"2024-01-15T18:16:03.982746+0000","last_clean":"2024-01-15T18:16:03.982746+0000","last_became_active":"2024-01-15T18:16:02.974162+0000","last_became_peered":"2024-01-15T18:16:02.974162+0000","last_unstale":"2024-01-15T18:16:03.982746+0000","last_undegraded":"2024-01-15T18:16:03.982746+0000","last_fullsized":"2024-01-15T18:16:03.982746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172002+0000","last_change":"2024-01-15T18:16:02.986920+0000","last_active":"2024-01-15T18:16:04.172002+0000","last_peered":"2024-01-15T18:16:04.172002+0000","last_clean":"2024-01-15T18:16:04.172002+0000","last_became_active":"2024-01-15T18:16:02.986544+0000","last_became_peered":"2024-01-15T18:16:02.986544+0000","last_unstale":"2024-01-15T18:16:04.172002+0000","last_undegraded":"2024-01-15T18:16:04.172002+0000","last_fullsized":"2024-01-15T18:16:04.172002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:06.358800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983627+0000","last_change":"2024-01-15T18:16:02.981015+0000","last_active":"2024-01-15T18:16:03.983627+0000","last_peered":"2024-01-15T18:16:03.983627+0000","last_clean":"2024-01-15T18:16:03.983627+0000","last_became_active":"2024-01-15T18:16:02.979886+0000","last_became_peered":"2024-01-15T18:16:02.979886+0000","last_unstale":"2024-01-15T18:16:03.983627+0000","last_undegraded":"2024-01-15T18:16:03.983627+0000","last_fullsized":"2024-01-15T18:16:03.983627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:23.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004361+0000","last_change":"2024-01-15T18:16:01.968674+0000","last_active":"2024-01-15T18:16:03.004361+0000","last_peered":"2024-01-15T18:16:03.004361+0000","last_clean":"2024-01-15T18:16:03.004361+0000","last_became_active":"2024-01-15T18:16:01.967874+0000","last_became_peered":"2024-01-15T18:16:01.967874+0000","last_unstale":"2024-01-15T18:16:03.004361+0000","last_undegraded":"2024-01-15T18:16:03.004361+0000","last_fullsized":"2024-01-15T18:16:03.004361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172367+0000","last_change":"2024-01-15T18:16:02.971695+0000","last_active":"2024-01-15T18:16:04.172367+0000","last_peered":"2024-01-15T18:16:04.172367+0000","last_clean":"2024-01-15T18:16:04.172367+0000","last_became_active":"2024-01-15T18:16:02.971425+0000","last_became_peered":"2024-01-15T18:16:02.971425+0000","last_unstale":"2024-01-15T18:16:04.172367+0000","last_undegraded":"2024-01-15T18:16:04.172367+0000","last_fullsized":"2024-01-15T18:16:04.172367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002366+0000","last_change":"2024-01-15T18:16:01.960880+0000","last_active":"2024-01-15T18:16:03.002366+0000","last_peered":"2024-01-15T18:16:03.002366+0000","last_clean":"2024-01-15T18:16:03.002366+0000","last_became_active":"2024-01-15T18:16:01.960424+0000","last_became_peered":"2024-01-15T18:16:01.960424+0000","last_unstale":"2024-01-15T18:16:03.002366+0000","last_undegraded":"2024-01-15T18:16:03.002366+0000","last_fullsized":"2024-01-15T18:16:03.002366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:05.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172293+0000","last_change":"2024-01-15T18:16:02.987087+0000","last_active":"2024-01-15T18:16:04.172293+0000","last_peered":"2024-01-15T18:16:04.172293+0000","last_clean":"2024-01-15T18:16:04.172293+0000","last_became_active":"2024-01-15T18:16:02.986595+0000","last_became_peered":"2024-01-15T18:16:02.986595+0000","last_unstale":"2024-01-15T18:16:04.172293+0000","last_undegraded":"2024-01-15T18:16:04.172293+0000","last_fullsized":"2024-01-15T18:16:04.172293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973629+0000","last_change":"2024-01-15T18:16:01.972650+0000","last_active":"2024-01-15T18:16:01.973629+0000","last_peered":"2024-01-15T18:16:01.973629+0000","last_clean":"2024-01-15T18:16:01.973629+0000","last_became_active":"2024-01-15T18:16:01.972282+0000","last_became_peered":"2024-01-15T18:16:01.972282+0000","last_unstale":"2024-01-15T18:16:01.973629+0000","last_undegraded":"2024-01-15T18:16:01.973629+0000","last_fullsized":"2024-01-15T18:16:01.973629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984011+0000","last_change":"2024-01-15T18:16:02.980841+0000","last_active":"2024-01-15T18:16:03.984011+0000","last_peered":"2024-01-15T18:16:03.984011+0000","last_clean":"2024-01-15T18:16:03.984011+0000","last_became_active":"2024-01-15T18:16:02.980529+0000","last_became_peered":"2024-01-15T18:16:02.980529+0000","last_unstale":"2024-01-15T18:16:03.984011+0000","last_undegraded":"2024-01-15T18:16:03.984011+0000","last_fullsized":"2024-01-15T18:16:03.984011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:16.698507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.974689+0000","last_change":"2024-01-15T18:16:01.974052+0000","last_active":"2024-01-15T18:16:01.974689+0000","last_peered":"2024-01-15T18:16:01.974689+0000","last_clean":"2024-01-15T18:16:01.974689+0000","last_became_active":"2024-01-15T18:16:01.973571+0000","last_became_peered":"2024-01-15T18:16:01.973571+0000","last_unstale":"2024-01-15T18:16:01.974689+0000","last_undegraded":"2024-01-15T18:16:01.974689+0000","last_fullsized":"2024-01-15T18:16:01.974689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956466+0000","last_change":"2024-01-15T18:16:01.956466+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956466+0000","last_undegraded":"2024-01-15T18:16:01.956466+0000","last_fullsized":"2024-01-15T18:16:01.956466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002241+0000","last_change":"2024-01-15T18:16:01.961667+0000","last_active":"2024-01-15T18:16:03.002241+0000","last_peered":"2024-01-15T18:16:03.002241+0000","last_clean":"2024-01-15T18:16:03.002241+0000","last_became_active":"2024-01-15T18:16:01.961336+0000","last_became_peered":"2024-01-15T18:16:01.961336+0000","last_unstale":"2024-01-15T18:16:03.002241+0000","last_undegraded":"2024-01-15T18:16:03.002241+0000","last_fullsized":"2024-01-15T18:16:03.002241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.970823+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.970523+0000","last_became_peered":"2024-01-15T18:16:02.970523+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000246+0000","last_change":"2024-01-15T18:16:01.958649+0000","last_active":"2024-01-15T18:16:03.000246+0000","last_peered":"2024-01-15T18:16:03.000246+0000","last_clean":"2024-01-15T18:16:03.000246+0000","last_became_active":"2024-01-15T18:16:01.958043+0000","last_became_peered":"2024-01-15T18:16:01.958043+0000","last_unstale":"2024-01-15T18:16:03.000246+0000","last_undegraded":"2024-01-15T18:16:03.000246+0000","last_fullsized":"2024-01-15T18:16:03.000246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983181+0000","last_change":"2024-01-15T18:16:02.983728+0000","last_active":"2024-01-15T18:16:03.983181+0000","last_peered":"2024-01-15T18:16:03.983181+0000","last_clean":"2024-01-15T18:16:03.983181+0000","last_became_active":"2024-01-15T18:16:02.981960+0000","last_became_peered":"2024-01-15T18:16:02.981960+0000","last_unstale":"2024-01-15T18:16:03.983181+0000","last_undegraded":"2024-01-15T18:16:03.983181+0000","last_fullsized":"2024-01-15T18:16:03.983181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.971514+0000","last_change":"2024-01-15T18:16:01.968471+0000","last_active":"2024-01-15T18:16:01.971514+0000","last_peered":"2024-01-15T18:16:01.971514+0000","last_clean":"2024-01-15T18:16:01.971514+0000","last_became_active":"2024-01-15T18:16:01.967508+0000","last_became_peered":"2024-01-15T18:16:01.967508+0000","last_unstale":"2024-01-15T18:16:01.971514+0000","last_undegraded":"2024-01-15T18:16:01.971514+0000","last_fullsized":"2024-01-15T18:16:01.971514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:33.150205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954100+0000","last_change":"2024-01-15T18:16:01.954100+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954100+0000","last_undegraded":"2024-01-15T18:16:01.954100+0000","last_fullsized":"2024-01-15T18:16:01.954100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.954226+0000","last_change":"2024-01-15T18:16:01.954226+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954226+0000","last_undegraded":"2024-01-15T18:16:01.954226+0000","last_fullsized":"2024-01-15T18:16:01.954226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:51.202598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955399+0000","last_change":"2024-01-15T18:16:01.955399+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955399+0000","last_undegraded":"2024-01-15T18:16:01.955399+0000","last_fullsized":"2024-01-15T18:16:01.955399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.967235+0000","last_change":"2024-01-15T18:16:01.967948+0000","last_active":"2024-01-15T18:16:02.967235+0000","last_peered":"2024-01-15T18:16:02.967235+0000","last_clean":"2024-01-15T18:16:02.967235+0000","last_became_active":"2024-01-15T18:16:01.967662+0000","last_became_peered":"2024-01-15T18:16:01.967662+0000","last_unstale":"2024-01-15T18:16:02.967235+0000","last_undegraded":"2024-01-15T18:16:02.967235+0000","last_fullsized":"2024-01-15T18:16:02.967235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:28.859462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672459+0000","last_change":"2024-01-15T18:16:02.977816+0000","last_active":"2024-01-15T18:16:04.672459+0000","last_peered":"2024-01-15T18:16:04.672459+0000","last_clean":"2024-01-15T18:16:04.672459+0000","last_became_active":"2024-01-15T18:16:02.977488+0000","last_became_peered":"2024-01-15T18:16:02.977488+0000","last_unstale":"2024-01-15T18:16:04.672459+0000","last_undegraded":"2024-01-15T18:16:04.672459+0000","last_fullsized":"2024-01-15T18:16:04.672459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.966903+0000","last_change":"2024-01-15T18:16:01.970089+0000","last_active":"2024-01-15T18:16:02.966903+0000","last_peered":"2024-01-15T18:16:02.966903+0000","last_clean":"2024-01-15T18:16:02.966903+0000","last_became_active":"2024-01-15T18:16:01.969791+0000","last_became_peered":"2024-01-15T18:16:01.969791+0000","last_unstale":"2024-01-15T18:16:02.966903+0000","last_undegraded":"2024-01-15T18:16:02.966903+0000","last_fullsized":"2024-01-15T18:16:02.966903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:25.108362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956190+0000","last_change":"2024-01-15T18:16:01.956190+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956190+0000","last_undegraded":"2024-01-15T18:16:01.956190+0000","last_fullsized":"2024-01-15T18:16:01.956190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002343+0000","last_change":"2024-01-15T18:16:01.967367+0000","last_active":"2024-01-15T18:16:03.002343+0000","last_peered":"2024-01-15T18:16:03.002343+0000","last_clean":"2024-01-15T18:16:03.002343+0000","last_became_active":"2024-01-15T18:16:01.966610+0000","last_became_peered":"2024-01-15T18:16:01.966610+0000","last_unstale":"2024-01-15T18:16:03.002343+0000","last_undegraded":"2024-01-15T18:16:03.002343+0000","last_fullsized":"2024-01-15T18:16:03.002343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983502+0000","last_change":"2024-01-15T18:16:02.976324+0000","last_active":"2024-01-15T18:16:03.983502+0000","last_peered":"2024-01-15T18:16:03.983502+0000","last_clean":"2024-01-15T18:16:03.983502+0000","last_became_active":"2024-01-15T18:16:02.975452+0000","last_became_peered":"2024-01-15T18:16:02.975452+0000","last_unstale":"2024-01-15T18:16:03.983502+0000","last_undegraded":"2024-01-15T18:16:03.983502+0000","last_fullsized":"2024-01-15T18:16:03.983502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:31.636033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004546+0000","last_change":"2024-01-15T18:16:01.971530+0000","last_active":"2024-01-15T18:16:03.004546+0000","last_peered":"2024-01-15T18:16:03.004546+0000","last_clean":"2024-01-15T18:16:03.004546+0000","last_became_active":"2024-01-15T18:16:01.971080+0000","last_became_peered":"2024-01-15T18:16:01.971080+0000","last_unstale":"2024-01-15T18:16:03.004546+0000","last_undegraded":"2024-01-15T18:16:03.004546+0000","last_fullsized":"2024-01-15T18:16:03.004546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955769+0000","last_change":"2024-01-15T18:16:01.955769+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955769+0000","last_undegraded":"2024-01-15T18:16:01.955769+0000","last_fullsized":"2024-01-15T18:16:01.955769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968495+0000","last_change":"2024-01-15T18:16:01.968495+0000","last_active":"2024-01-15T18:16:01.968495+0000","last_peered":"2024-01-15T18:16:01.968495+0000","last_clean":"2024-01-15T18:16:01.968495+0000","last_became_active":"2024-01-15T18:16:01.967100+0000","last_became_peered":"2024-01-15T18:16:01.967100+0000","last_unstale":"2024-01-15T18:16:01.968495+0000","last_undegraded":"2024-01-15T18:16:01.968495+0000","last_fullsized":"2024-01-15T18:16:01.968495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:29.198637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983817+0000","last_change":"2024-01-15T18:16:02.985570+0000","last_active":"2024-01-15T18:16:03.983817+0000","last_peered":"2024-01-15T18:16:03.983817+0000","last_clean":"2024-01-15T18:16:03.983817+0000","last_became_active":"2024-01-15T18:16:02.985303+0000","last_became_peered":"2024-01-15T18:16:02.985303+0000","last_unstale":"2024-01-15T18:16:03.983817+0000","last_undegraded":"2024-01-15T18:16:03.983817+0000","last_fullsized":"2024-01-15T18:16:03.983817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:55.037442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968815+0000","last_change":"2024-01-15T18:16:01.965271+0000","last_active":"2024-01-15T18:16:01.968815+0000","last_peered":"2024-01-15T18:16:01.968815+0000","last_clean":"2024-01-15T18:16:01.968815+0000","last_became_active":"2024-01-15T18:16:01.964982+0000","last_became_peered":"2024-01-15T18:16:01.964982+0000","last_unstale":"2024-01-15T18:16:01.968815+0000","last_undegraded":"2024-01-15T18:16:01.968815+0000","last_fullsized":"2024-01-15T18:16:01.968815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957808+0000","last_change":"2024-01-15T18:16:01.957808+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957808+0000","last_undegraded":"2024-01-15T18:16:01.957808+0000","last_fullsized":"2024-01-15T18:16:01.957808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971256+0000","last_change":"2024-01-15T18:16:01.971126+0000","last_active":"2024-01-15T18:16:02.971256+0000","last_peered":"2024-01-15T18:16:02.971256+0000","last_clean":"2024-01-15T18:16:02.971256+0000","last_became_active":"2024-01-15T18:16:01.969238+0000","last_became_peered":"2024-01-15T18:16:01.969238+0000","last_unstale":"2024-01-15T18:16:02.971256+0000","last_undegraded":"2024-01-15T18:16:02.971256+0000","last_fullsized":"2024-01-15T18:16:02.971256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:30.543482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954474+0000","last_change":"2024-01-15T18:16:01.954474+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954474+0000","last_undegraded":"2024-01-15T18:16:01.954474+0000","last_fullsized":"2024-01-15T18:16:01.954474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980185+0000","last_change":"2024-01-15T18:16:01.979247+0000","last_active":"2024-01-15T18:16:01.980185+0000","last_peered":"2024-01-15T18:16:01.980185+0000","last_clean":"2024-01-15T18:16:01.980185+0000","last_became_active":"2024-01-15T18:16:01.978885+0000","last_became_peered":"2024-01-15T18:16:01.978885+0000","last_unstale":"2024-01-15T18:16:01.980185+0000","last_undegraded":"2024-01-15T18:16:01.980185+0000","last_fullsized":"2024-01-15T18:16:01.980185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:42.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984081+0000","last_change":"2024-01-15T18:16:02.983891+0000","last_active":"2024-01-15T18:16:03.984081+0000","last_peered":"2024-01-15T18:16:03.984081+0000","last_clean":"2024-01-15T18:16:03.984081+0000","last_became_active":"2024-01-15T18:16:02.983591+0000","last_became_peered":"2024-01-15T18:16:02.983591+0000","last_unstale":"2024-01-15T18:16:03.984081+0000","last_undegraded":"2024-01-15T18:16:03.984081+0000","last_fullsized":"2024-01-15T18:16:03.984081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:32.572896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.998942+0000","last_change":"2024-01-15T18:16:01.971805+0000","last_active":"2024-01-15T18:16:02.998942+0000","last_peered":"2024-01-15T18:16:02.998942+0000","last_clean":"2024-01-15T18:16:02.998942+0000","last_became_active":"2024-01-15T18:16:01.971492+0000","last_became_peered":"2024-01-15T18:16:01.971492+0000","last_unstale":"2024-01-15T18:16:02.998942+0000","last_undegraded":"2024-01-15T18:16:02.998942+0000","last_fullsized":"2024-01-15T18:16:02.998942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.974741+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.974455+0000","last_became_peered":"2024-01-15T18:16:02.974455+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973667+0000","last_change":"2024-01-15T18:16:01.970998+0000","last_active":"2024-01-15T18:16:01.973667+0000","last_peered":"2024-01-15T18:16:01.973667+0000","last_clean":"2024-01-15T18:16:01.973667+0000","last_became_active":"2024-01-15T18:16:01.970730+0000","last_became_peered":"2024-01-15T18:16:01.970730+0000","last_unstale":"2024-01-15T18:16:01.973667+0000","last_undegraded":"2024-01-15T18:16:01.973667+0000","last_fullsized":"2024-01-15T18:16:01.973667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983546+0000","last_change":"2024-01-15T18:16:02.989297+0000","last_active":"2024-01-15T18:16:03.983546+0000","last_peered":"2024-01-15T18:16:03.983546+0000","last_clean":"2024-01-15T18:16:03.983546+0000","last_became_active":"2024-01-15T18:16:02.988832+0000","last_became_peered":"2024-01-15T18:16:02.988832+0000","last_unstale":"2024-01-15T18:16:03.983546+0000","last_undegraded":"2024-01-15T18:16:03.983546+0000","last_fullsized":"2024-01-15T18:16:03.983546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:07.504686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.977256+0000","last_change":"2024-01-15T18:16:01.975857+0000","last_active":"2024-01-15T18:16:01.977256+0000","last_peered":"2024-01-15T18:16:01.977256+0000","last_clean":"2024-01-15T18:16:01.977256+0000","last_became_active":"2024-01-15T18:16:01.975585+0000","last_became_peered":"2024-01-15T18:16:01.975585+0000","last_unstale":"2024-01-15T18:16:01.977256+0000","last_undegraded":"2024-01-15T18:16:01.977256+0000","last_fullsized":"2024-01-15T18:16:01.977256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975212+0000","last_change":"2024-01-15T18:16:01.972470+0000","last_active":"2024-01-15T18:16:01.975212+0000","last_peered":"2024-01-15T18:16:01.975212+0000","last_clean":"2024-01-15T18:16:01.975212+0000","last_became_active":"2024-01-15T18:16:01.972202+0000","last_became_peered":"2024-01-15T18:16:01.972202+0000","last_unstale":"2024-01-15T18:16:01.975212+0000","last_undegraded":"2024-01-15T18:16:01.975212+0000","last_fullsized":"2024-01-15T18:16:01.975212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672268+0000","last_change":"2024-01-15T18:16:02.984567+0000","last_active":"2024-01-15T18:16:04.672268+0000","last_peered":"2024-01-15T18:16:04.672268+0000","last_clean":"2024-01-15T18:16:04.672268+0000","last_became_active":"2024-01-15T18:16:02.984223+0000","last_became_peered":"2024-01-15T18:16:02.984223+0000","last_unstale":"2024-01-15T18:16:04.672268+0000","last_undegraded":"2024-01-15T18:16:04.672268+0000","last_fullsized":"2024-01-15T18:16:04.672268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.952954+0000","last_change":"2024-01-15T18:16:00.952954+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.952954+0000","last_undegraded":"2024-01-15T18:16:00.952954+0000","last_fullsized":"2024-01-15T18:16:00.952954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.953595+0000","last_change":"2024-01-15T18:16:00.953595+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.953595+0000","last_undegraded":"2024-01-15T18:16:00.953595+0000","last_fullsized":"2024-01-15T18:16:00.953595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672166+0000","last_change":"2024-01-15T18:16:02.984528+0000","last_active":"2024-01-15T18:16:04.672166+0000","last_peered":"2024-01-15T18:16:04.672166+0000","last_clean":"2024-01-15T18:16:04.672166+0000","last_became_active":"2024-01-15T18:16:02.984179+0000","last_became_peered":"2024-01-15T18:16:02.984179+0000","last_unstale":"2024-01-15T18:16:04.672166+0000","last_undegraded":"2024-01-15T18:16:04.672166+0000","last_fullsized":"2024-01-15T18:16:04.672166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975902+0000","last_change":"2024-01-15T18:16:01.974336+0000","last_active":"2024-01-15T18:16:01.975902+0000","last_peered":"2024-01-15T18:16:01.975902+0000","last_clean":"2024-01-15T18:16:01.975902+0000","last_became_active":"2024-01-15T18:16:01.974035+0000","last_became_peered":"2024-01-15T18:16:01.974035+0000","last_unstale":"2024-01-15T18:16:01.975902+0000","last_undegraded":"2024-01-15T18:16:01.975902+0000","last_fullsized":"2024-01-15T18:16:01.975902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983695+0000","last_change":"2024-01-15T18:16:02.981820+0000","last_active":"2024-01-15T18:16:03.983695+0000","last_peered":"2024-01-15T18:16:03.983695+0000","last_clean":"2024-01-15T18:16:03.983695+0000","last_became_active":"2024-01-15T18:16:02.981478+0000","last_became_peered":"2024-01-15T18:16:02.981478+0000","last_unstale":"2024-01-15T18:16:03.983695+0000","last_undegraded":"2024-01-15T18:16:03.983695+0000","last_fullsized":"2024-01-15T18:16:03.983695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.957714+0000","last_change":"2024-01-15T18:16:01.957714+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957714+0000","last_undegraded":"2024-01-15T18:16:01.957714+0000","last_fullsized":"2024-01-15T18:16:01.957714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:50.381475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984005+0000","last_change":"2024-01-15T18:16:02.981174+0000","last_active":"2024-01-15T18:16:03.984005+0000","last_peered":"2024-01-15T18:16:03.984005+0000","last_clean":"2024-01-15T18:16:03.984005+0000","last_became_active":"2024-01-15T18:16:02.980913+0000","last_became_peered":"2024-01-15T18:16:02.980913+0000","last_unstale":"2024-01-15T18:16:03.984005+0000","last_undegraded":"2024-01-15T18:16:03.984005+0000","last_fullsized":"2024-01-15T18:16:03.984005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:12.319403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.969433+0000","last_change":"2024-01-15T18:16:01.967331+0000","last_active":"2024-01-15T18:16:01.969433+0000","last_peered":"2024-01-15T18:16:01.969433+0000","last_clean":"2024-01-15T18:16:01.969433+0000","last_became_active":"2024-01-15T18:16:01.967031+0000","last_became_peered":"2024-01-15T18:16:01.967031+0000","last_unstale":"2024-01-15T18:16:01.969433+0000","last_undegraded":"2024-01-15T18:16:01.969433+0000","last_fullsized":"2024-01-15T18:16:01.969433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:37.438291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172397+0000","last_change":"2024-01-15T18:16:02.969429+0000","last_active":"2024-01-15T18:16:04.172397+0000","last_peered":"2024-01-15T18:16:04.172397+0000","last_clean":"2024-01-15T18:16:04.172397+0000","last_became_active":"2024-01-15T18:16:02.969095+0000","last_became_peered":"2024-01-15T18:16:02.969095+0000","last_unstale":"2024-01-15T18:16:04.172397+0000","last_undegraded":"2024-01-15T18:16:04.172397+0000","last_fullsized":"2024-01-15T18:16:04.172397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964906+0000","last_change":"2024-01-15T18:16:01.968688+0000","last_active":"2024-01-15T18:16:02.964906+0000","last_peered":"2024-01-15T18:16:02.964906+0000","last_clean":"2024-01-15T18:16:02.964906+0000","last_became_active":"2024-01-15T18:16:01.968389+0000","last_became_peered":"2024-01-15T18:16:01.968389+0000","last_unstale":"2024-01-15T18:16:02.964906+0000","last_undegraded":"2024-01-15T18:16:02.964906+0000","last_fullsized":"2024-01-15T18:16:02.964906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.976712+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.976054+0000","last_became_peered":"2024-01-15T18:16:02.976054+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:26:19.031570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964799+0000","last_change":"2024-01-15T18:16:01.966697+0000","last_active":"2024-01-15T18:16:02.964799+0000","last_peered":"2024-01-15T18:16:02.964799+0000","last_clean":"2024-01-15T18:16:02.964799+0000","last_became_active":"2024-01-15T18:16:01.966397+0000","last_became_peered":"2024-01-15T18:16:01.966397+0000","last_unstale":"2024-01-15T18:16:02.964799+0000","last_undegraded":"2024-01-15T18:16:02.964799+0000","last_fullsized":"2024-01-15T18:16:02.964799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672238+0000","last_change":"2024-01-15T18:16:02.984048+0000","last_active":"2024-01-15T18:16:04.672238+0000","last_peered":"2024-01-15T18:16:04.672238+0000","last_clean":"2024-01-15T18:16:04.672238+0000","last_became_active":"2024-01-15T18:16:02.983546+0000","last_became_peered":"2024-01-15T18:16:02.983546+0000","last_unstale":"2024-01-15T18:16:04.672238+0000","last_undegraded":"2024-01-15T18:16:04.672238+0000","last_fullsized":"2024-01-15T18:16:04.672238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980104+0000","last_change":"2024-01-15T18:16:01.979160+0000","last_active":"2024-01-15T18:16:01.980104+0000","last_peered":"2024-01-15T18:16:01.980104+0000","last_clean":"2024-01-15T18:16:01.980104+0000","last_became_active":"2024-01-15T18:16:01.978901+0000","last_became_peered":"2024-01-15T18:16:01.978901+0000","last_unstale":"2024-01-15T18:16:01.980104+0000","last_undegraded":"2024-01-15T18:16:01.980104+0000","last_fullsized":"2024-01-15T18:16:01.980104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954987+0000","last_change":"2024-01-15T18:16:01.954987+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954987+0000","last_undegraded":"2024-01-15T18:16:01.954987+0000","last_fullsized":"2024-01-15T18:16:01.954987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.961378+0000","last_change":"2024-01-15T18:16:01.958840+0000","last_active":"2024-01-15T18:16:02.961378+0000","last_peered":"2024-01-15T18:16:02.961378+0000","last_clean":"2024-01-15T18:16:02.961378+0000","last_became_active":"2024-01-15T18:16:01.958367+0000","last_became_peered":"2024-01-15T18:16:01.958367+0000","last_unstale":"2024-01-15T18:16:02.961378+0000","last_undegraded":"2024-01-15T18:16:02.961378+0000","last_fullsized":"2024-01-15T18:16:02.961378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983304+0000","last_change":"2024-01-15T18:16:02.991930+0000","last_active":"2024-01-15T18:16:03.983304+0000","last_peered":"2024-01-15T18:16:03.983304+0000","last_clean":"2024-01-15T18:16:03.983304+0000","last_became_active":"2024-01-15T18:16:02.991630+0000","last_became_peered":"2024-01-15T18:16:02.991630+0000","last_unstale":"2024-01-15T18:16:03.983304+0000","last_undegraded":"2024-01-15T18:16:03.983304+0000","last_fullsized":"2024-01-15T18:16:03.983304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964174+0000","last_change":"2024-01-15T18:16:01.958054+0000","last_active":"2024-01-15T18:16:02.964174+0000","last_peered":"2024-01-15T18:16:02.964174+0000","last_clean":"2024-01-15T18:16:02.964174+0000","last_became_active":"2024-01-15T18:16:01.957368+0000","last_became_peered":"2024-01-15T18:16:01.957368+0000","last_unstale":"2024-01-15T18:16:02.964174+0000","last_undegraded":"2024-01-15T18:16:02.964174+0000","last_fullsized":"2024-01-15T18:16:02.964174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172054+0000","last_change":"2024-01-15T18:16:02.970069+0000","last_active":"2024-01-15T18:16:04.172054+0000","last_peered":"2024-01-15T18:16:04.172054+0000","last_clean":"2024-01-15T18:16:04.172054+0000","last_became_active":"2024-01-15T18:16:02.969737+0000","last_became_peered":"2024-01-15T18:16:02.969737+0000","last_unstale":"2024-01-15T18:16:04.172054+0000","last_undegraded":"2024-01-15T18:16:04.172054+0000","last_fullsized":"2024-01-15T18:16:04.172054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:06.396100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.963102+0000","last_change":"2024-01-15T18:16:01.968677+0000","last_active":"2024-01-15T18:16:02.963102+0000","last_peered":"2024-01-15T18:16:02.963102+0000","last_clean":"2024-01-15T18:16:02.963102+0000","last_became_active":"2024-01-15T18:16:01.967462+0000","last_became_peered":"2024-01-15T18:16:01.967462+0000","last_unstale":"2024-01-15T18:16:02.963102+0000","last_undegraded":"2024-01-15T18:16:02.963102+0000","last_fullsized":"2024-01-15T18:16:02.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.959339+0000","last_change":"2024-01-15T18:16:01.959339+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.959339+0000","last_undegraded":"2024-01-15T18:16:01.959339+0000","last_fullsized":"2024-01-15T18:16:01.959339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000937+0000","last_change":"2024-01-15T18:16:01.973544+0000","last_active":"2024-01-15T18:16:03.000937+0000","last_peered":"2024-01-15T18:16:03.000937+0000","last_clean":"2024-01-15T18:16:03.000937+0000","last_became_active":"2024-01-15T18:16:01.973240+0000","last_became_peered":"2024-01-15T18:16:01.973240+0000","last_unstale":"2024-01-15T18:16:03.000937+0000","last_undegraded":"2024-01-15T18:16:03.000937+0000","last_fullsized":"2024-01-15T18:16:03.000937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:06.496321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971217+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:16:02.971217+0000","last_peered":"2024-01-15T18:16:02.971217+0000","last_clean":"2024-01-15T18:16:02.971217+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:16:02.971217+0000","last_undegraded":"2024-01-15T18:16:02.971217+0000","last_fullsized":"2024-01-15T18:16:02.971217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77164,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294676,"statfs":{"total":96636764160,"available":96557748224,"internally_reserved":0,"allocated":1667072,"data_stored":1050036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.3839999999999999,"15min":0.92200000000000004},"min":{"1min":0.621,"5min":0.36899999999999999,"15min":0.29599999999999999},"max":{"1min":16.815999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.044,"15min":0.87},"min":{"1min":0.57399999999999995,"5min":0.33400000000000002,"15min":0.28399999999999997},"max":{"1min":4.6100000000000003,"5min":8.5419999999999998,"15min":13.744},"last":0.441}]},{"osd":1,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.885,"5min":1.6519999999999999,"15min":1.0700000000000001},"min":{"1min":0.60599999999999998,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":16.920000000000002,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.637,"15min":1.0680000000000001},"min":{"1min":0.60299999999999998,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":4.7679999999999998,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.496,"15min":0.99199999999999999},"min":{"1min":0.52000000000000002,"5min":0.433,"15min":0.38400000000000001},"max":{"1min":2.1680000000000001,"5min":30.846,"15min":30.846},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.887,"5min":1.3140000000000001,"15min":1.097},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.39800000000000002},"max":{"1min":17.039999999999999,"5min":17.039999999999999,"15min":39.695},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.6299999999999999,"15min":1.163},"min":{"1min":0.55300000000000005,"5min":0.47499999999999998,"15min":0.38800000000000001},"max":{"1min":2.1160000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.831,"5min":2.0049999999999999,"15min":1.228},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":16.879999999999999,"5min":42.692,"15min":42.692},"last":0.70199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.89100000000000001,"15min":0.72199999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":14.087,"5min":14.087,"15min":14.087},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.36699999999999999,"15min":0.36299999999999999},"max":{"1min":2.2450000000000001,"5min":2.952,"15min":4.3710000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":0.80500000000000005,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":11.334,"5min":11.334,"15min":11.334},"last":0.505},{"interface":"front","average":{"1min":1.012,"5min":0.91000000000000003,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.8179999999999996,"5min":8.423,"15min":8.423},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.91300000000000003,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.7140000000000004,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.376},"max":{"1min":1.1970000000000001,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76588,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295252,"statfs":{"total":96636764160,"available":96558338048,"internally_reserved":0,"allocated":1081344,"data_stored":460292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.83799999999999997,"15min":0.94799999999999995},"min":{"1min":0.41399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78200000000000003,"15min":0.872},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.266,"5min":3.4940000000000002,"15min":12.449},"last":0.81599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.1379999999999999,"15min":1.0109999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.379},"max":{"1min":1.615,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.087,"15min":1.1759999999999999},"min":{"1min":0.55600000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":14.743,"15min":42.825000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":1.147},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1890000000000001,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78100000000000003,"15min":1.0269999999999999},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":1.3640000000000001,"5min":3.016,"15min":24.199000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.0489999999999999,"15min":1.0629999999999999},"min":{"1min":0.60699999999999998,"5min":0.47799999999999998,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":18.791,"15min":18.791},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.585,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.66100000000000003,"15min":0.75800000000000001},"min":{"1min":0.50800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":12.491},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.443},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.4379999999999999,"5min":2.5259999999999998,"15min":12.945},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.98599999999999999,"15min":0.84599999999999997},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":5.1829999999999998,"5min":11.907,"15min":11.907},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80800000000000005,"15min":0.88900000000000001},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5569999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.84199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54456,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317384,"statfs":{"total":96636764160,"available":96581001216,"internally_reserved":0,"allocated":1085440,"data_stored":461232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":0.82899999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.4770000000000003,"5min":6.6820000000000004,"15min":11.519},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.95699999999999996,"15min":0.90300000000000002},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.34999999999999998},"max":{"1min":1.714,"5min":7.6260000000000003,"15min":12.952},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95499999999999996,"15min":0.88500000000000001},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":4.3140000000000001,"5min":4.609,"15min":13.358000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.859,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.149,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":2.867,"5min":18.314,"15min":18.314},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.341,"15min":1.006},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.434},"max":{"1min":2.855,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.89500000000000002,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":1.839,"5min":4.0330000000000004,"15min":13.317},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.034,"15min":1},"min":{"1min":0.443,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.1440000000000001,"5min":15.205,"15min":19.638000000000002},"last":0.74399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.28799999999999998},"max":{"1min":2.4289999999999998,"5min":5.899,"15min":11.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32100000000000001},"max":{"1min":2.516,"5min":3.097,"15min":3.3889999999999998},"last":0.623}]},{"osd":6,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.80000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.238,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.9470000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.76200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":2.3900000000000001,"5min":2.9870000000000001,"15min":4.1799999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.40400000000000003},"max":{"1min":2.3239999999999998,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":1069056,"data_stored":459598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.097,"15min":0.92600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":9.4179999999999993,"5min":11.374000000000001,"15min":34.802},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.135,"15min":0.89300000000000002},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.34699999999999998},"max":{"1min":3.3450000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.97399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2130000000000001,"5min":7.625,"15min":38.198},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.93999999999999995,"15min":0.95799999999999996},"min":{"1min":0.55700000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0589999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0149999999999999,"15min":0.85499999999999998},"min":{"1min":0.61499999999999999,"5min":0.40799999999999997,"15min":0.38700000000000001},"max":{"1min":4.4089999999999998,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.621}]},{"osd":3,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.2729999999999999,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":3.415,"5min":14.529,"15min":14.529},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.2430000000000001,"15min":0.93300000000000005},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78100000000000003,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.6070000000000002,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.96899999999999997,"15min":0.83099999999999996},"min":{"1min":0.47499999999999998,"5min":0.39200000000000002,"15min":0.34100000000000003},"max":{"1min":3.0209999999999999,"5min":25.887,"15min":25.887},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":7.5439999999999996,"15min":10.284000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85999999999999999,"15min":0.79300000000000004},"min":{"1min":0.45000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1680000000000001,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":1.232,"5min":11.67,"15min":11.67},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.595,"15min":0.95199999999999996},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":7.2839999999999998,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.37},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.4670000000000001,"15min":0.94799999999999995},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":7.5170000000000003,"5min":23.074999999999999,"15min":28.332000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.4690000000000001,"15min":0.96299999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.2970000000000002,"5min":20.321000000000002,"15min":40.457999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.514,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.372,"15min":0.30299999999999999},"max":{"1min":7.3170000000000002,"5min":26.260999999999999,"15min":27.427},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.615,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.28399999999999997},"max":{"1min":9.1180000000000003,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.8240000000000001,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.34300000000000003},"max":{"1min":7.2409999999999997,"5min":26.404,"15min":26.404},"last":0.53100000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3819999999999999,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":4.1040000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.069},"min":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.377},"max":{"1min":2.1150000000000002,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.6259999999999999,"15min":1.097},"min":{"1min":0.58399999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":6.5140000000000002,"5min":26.314,"15min":26.314},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.5649999999999999,"15min":1.131},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.44500000000000001},"max":{"1min":4.165,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.94399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.635,"15min":1.1140000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.7619999999999996,"5min":20.445,"15min":20.445},"last":1},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.7270000000000001,"15min":1.1699999999999999},"min":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.495},"max":{"1min":6.7149999999999999,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.6839999999999999,"15min":1.155},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.2030000000000003,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":2.1429999999999998,"15min":1.3089999999999999},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46000000000000002},"max":{"1min":2.2490000000000001,"5min":44.817999999999998,"15min":44.817999999999998},"last":1.0900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.153,"15min":1.0449999999999999},"min":{"1min":0.33800000000000002,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9950000000000001,"5min":12.695,"15min":29.527999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.591,"15min":1.3180000000000001},"min":{"1min":0.47499999999999998,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.288,"15min":1.5329999999999999},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":7.476,"5min":34.029000000000003,"15min":113.149},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.099},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.30199999999999999},"max":{"1min":14.728,"5min":20.619,"15min":41.302},"last":0.499}]},{"osd":3,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":2.1030000000000002,"15min":1.3620000000000001},"min":{"1min":0.36799999999999999,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":31.57,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":1.258},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":7.3609999999999998,"5min":7.3609999999999998,"15min":115.693},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1279999999999999,"15min":1.2090000000000001},"min":{"1min":0.627,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":7.4169999999999998,"5min":7.4169999999999998,"15min":24.366},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.3049999999999999,"15min":1.2090000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0580000000000001,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":2.548,"5min":2.222,"15min":1.482},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.43099999999999999},"max":{"1min":41.683999999999997,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.103,"5min":1.5169999999999999,"15min":1.3129999999999999},"min":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":7.5099999999999998,"5min":26.084,"15min":26.084},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.468,"15min":1.302},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.208,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.746},{"interface":"front","average":{"1min":2.589,"5min":2.3210000000000002,"15min":1.556},"min":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.47699999999999998},"max":{"1min":41.738999999999997,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.3979999999999999,"15min":1.242},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":1.298,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":2.6000000000000001,"5min":2.383,"15min":1.629},"min":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.48799999999999999},"max":{"1min":41.402000000000001,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.97399999999999998,"15min":0.92500000000000004},"min":{"1min":0.33100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.3330000000000002,"5min":7.1580000000000004,"15min":39.948},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.5660000000000001,"15min":1.1200000000000001},"min":{"1min":0.39600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.577,"5min":35.790999999999997,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.034,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":6.516,"5min":11.346,"15min":49.229999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.1870000000000001,"15min":0.86499999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.931,"5min":7.9489999999999998,"15min":10.349},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.655,"15min":1.2190000000000001},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4669999999999996,"5min":27.643999999999998,"15min":27.643999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.343,"15min":1.016},"min":{"1min":0.48499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":15.608000000000001,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.6819999999999999,"15min":1.276},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.40200000000000002},"max":{"1min":4.633,"5min":27.489999999999998,"15min":27.489999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2969999999999999,"15min":1.054},"min":{"1min":0.61099999999999999,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":5.9139999999999997,"5min":10.038,"15min":12.997},"last":0.70699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.7050000000000001,"15min":1.2390000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.50800000000000001},"max":{"1min":4.673,"5min":27.547000000000001,"15min":27.547000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.1419999999999999},"min":{"1min":0.68400000000000005,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":5.3399999999999999,"5min":9.9610000000000003,"15min":13.054},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.698,"15min":1.2490000000000001},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":4.7060000000000004,"5min":27.579999999999998,"15min":27.579999999999998},"last":0.746},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.198,"15min":1.0760000000000001},"min":{"1min":0.60199999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0840000000000001,"5min":11.643000000000001,"15min":11.712999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3600000000000001,"15min":1.145},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46500000000000002},"max":{"1min":9.5660000000000007,"5min":11.833,"15min":11.833},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.226,"5min":1.4450000000000001,"15min":1.153},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":5.9550000000000001,"5min":14.606,"15min":14.606},"last":0.83799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77120,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294720,"statfs":{"total":96636764160,"available":96557793280,"internally_reserved":0,"allocated":1626112,"data_stored":1030448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.4470000000000001,"15min":1.175},"min":{"1min":0.32500000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":1.137,"5min":16.149000000000001,"15min":31.977},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.254,"15min":0.94699999999999995},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70999999999999996,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.63700000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":1.6719999999999999,"15min":1.0820000000000001},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.30199999999999999},"max":{"1min":1.7450000000000001,"5min":36.386000000000003,"15min":41.387999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2689999999999999,"15min":1.0429999999999999},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.25},"max":{"1min":11.539999999999999,"5min":16.556000000000001,"15min":33.219999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.3260000000000001,"15min":1.1140000000000001},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":1.2230000000000001,"5min":9.8740000000000006,"15min":31.943999999999999},"last":0.623},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.7689999999999999,"15min":1.0600000000000001},"min":{"1min":0.499,"5min":0.432,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":45.128,"15min":45.128},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.2709999999999999,"15min":1.109},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":11.772,"5min":11.772,"15min":32.932000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.379,"15min":1.1870000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":14.831,"15min":31.782},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.306,"15min":1.1850000000000001},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.43099999999999999},"max":{"1min":1.383,"5min":11.247999999999999,"15min":31.858000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.2929999999999999,"15min":1.1080000000000001},"min":{"1min":0.58399999999999996,"5min":0.52200000000000002,"15min":0.45800000000000002},"max":{"1min":11.958,"5min":11.958,"15min":33.151000000000003},"last":0.748}]},{"osd":6,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3520000000000001,"15min":1.052},"min":{"1min":0.622,"5min":0.49399999999999999,"15min":0.46400000000000002},"max":{"1min":1.1479999999999999,"5min":13.055,"15min":13.055},"last":0.871},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.3460000000000001,"15min":1.137},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.498},"max":{"1min":11.833,"5min":11.833,"15min":33.098999999999997},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.4279999999999999,"15min":1.0580000000000001},"min":{"1min":0.64700000000000002,"5min":0.52000000000000002,"15min":0.501},"max":{"1min":1.091,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.4630000000000001,"15min":1.2430000000000001},"min":{"1min":0.60299999999999998,"5min":0.53700000000000003,"15min":0.504},"max":{"1min":1.335,"5min":10.377000000000001,"15min":31.905999999999999},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:16:07.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3e is not active+clean 2024-01-15T18:16:07.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.958822+0000', 'last_change': '2024-01-15T18:16:01.958822+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.958822+0000', 'last_undegraded': '2024-01-15T18:16:01.958822+0000', 'last_fullsized': '2024-01-15T18:16:01.958822+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3c is not active+clean 2024-01-15T18:16:07.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.39 is not active+clean 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.35 is not active+clean 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.960055+0000', 'last_change': '2024-01-15T18:16:01.960055+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.960055+0000', 'last_undegraded': '2024-01-15T18:16:01.960055+0000', 'last_fullsized': '2024-01-15T18:16:01.960055+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.32 is not active+clean 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.31 is not active+clean 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.954608+0000', 'last_change': '2024-01-15T18:16:01.954608+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.954608+0000', 'last_undegraded': '2024-01-15T18:16:01.954608+0000', 'last_fullsized': '2024-01-15T18:16:01.954608+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2e is not active+clean 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2d is not active+clean 2024-01-15T18:16:07.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.961445+0000', 'last_change': '2024-01-15T18:16:01.961445+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.961445+0000', 'last_undegraded': '2024-01-15T18:16:01.961445+0000', 'last_fullsized': '2024-01-15T18:16:01.961445+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2c is not active+clean 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2b is not active+clean 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.2a is not active+clean 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.957359+0000', 'last_change': '2024-01-15T18:16:01.957359+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.957359+0000', 'last_undegraded': '2024-01-15T18:16:01.957359+0000', 'last_fullsized': '2024-01-15T18:16:01.957359+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.28 is not active+clean 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.27 is not active+clean 2024-01-15T18:16:07.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.958084+0000', 'last_change': '2024-01-15T18:16:01.958084+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.958084+0000', 'last_undegraded': '2024-01-15T18:16:01.958084+0000', 'last_fullsized': '2024-01-15T18:16:01.958084+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.23 is not active+clean 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.956707+0000', 'last_change': '2024-01-15T18:16:01.956707+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.956707+0000', 'last_undegraded': '2024-01-15T18:16:01.956707+0000', 'last_fullsized': '2024-01-15T18:16:01.956707+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.22 is not active+clean 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.956925+0000', 'last_change': '2024-01-15T18:16:01.956925+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.956925+0000', 'last_undegraded': '2024-01-15T18:16:01.956925+0000', 'last_fullsized': '2024-01-15T18:16:01.956925+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.1a is not active+clean 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.956466+0000', 'last_change': '2024-01-15T18:16:01.956466+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.956466+0000', 'last_undegraded': '2024-01-15T18:16:01.956466+0000', 'last_fullsized': '2024-01-15T18:16:01.956466+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.19 is not active+clean 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.954100+0000', 'last_change': '2024-01-15T18:16:01.954100+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.954100+0000', 'last_undegraded': '2024-01-15T18:16:01.954100+0000', 'last_fullsized': '2024-01-15T18:16:01.954100+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 85, 'state': 'creating+activating', 'last_fresh': '2024-01-15T18:16:01.954226+0000', 'last_change': '2024-01-15T18:16:01.954226+0000', 'last_active': '2024-01-15T18:16:00.935298+0000', 'last_peered': '2024-01-15T18:16:00.935298+0000', 'last_clean': '2024-01-15T18:16:00.935298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.954226+0000', 'last_undegraded': '2024-01-15T18:16:01.954226+0000', 'last_fullsized': '2024-01-15T18:16:01.954226+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-17T02:29:51.202598+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': ['1', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.16 is not active+clean 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.955399+0000', 'last_change': '2024-01-15T18:16:01.955399+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.955399+0000', 'last_undegraded': '2024-01-15T18:16:01.955399+0000', 'last_fullsized': '2024-01-15T18:16:01.955399+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:16:07.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.14 is not active+clean 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.956190+0000', 'last_change': '2024-01-15T18:16:01.956190+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.956190+0000', 'last_undegraded': '2024-01-15T18:16:01.956190+0000', 'last_fullsized': '2024-01-15T18:16:01.956190+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.12 is not active+clean 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.955769+0000', 'last_change': '2024-01-15T18:16:01.955769+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.955769+0000', 'last_undegraded': '2024-01-15T18:16:01.955769+0000', 'last_fullsized': '2024-01-15T18:16:01.955769+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.10 is not active+clean 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.957808+0000', 'last_change': '2024-01-15T18:16:01.957808+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.957808+0000', 'last_undegraded': '2024-01-15T18:16:01.957808+0000', 'last_fullsized': '2024-01-15T18:16:01.957808+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.11 is not active+clean 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.954474+0000', 'last_change': '2024-01-15T18:16:01.954474+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.954474+0000', 'last_undegraded': '2024-01-15T18:16:01.954474+0000', 'last_fullsized': '2024-01-15T18:16:01.954474+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.a is not active+clean 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:00.952954+0000', 'last_change': '2024-01-15T18:16:00.952954+0000', 'last_active': '2024-01-15T18:16:00.935298+0000', 'last_peered': '2024-01-15T18:16:00.935298+0000', 'last_clean': '2024-01-15T18:16:00.935298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:00.952954+0000', 'last_undegraded': '2024-01-15T18:16:00.952954+0000', 'last_fullsized': '2024-01-15T18:16:00.952954+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:16:07.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.8 is not active+clean 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-01-15T18:16:01.943662+0000', 'last_change': '2024-01-15T18:16:01.943662+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.943662+0000', 'last_undegraded': '2024-01-15T18:16:01.943662+0000', 'last_fullsized': '2024-01-15T18:16:01.943662+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 84, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:00.953595+0000', 'last_change': '2024-01-15T18:16:00.953595+0000', 'last_active': '2024-01-15T18:16:00.935298+0000', 'last_peered': '2024-01-15T18:16:00.935298+0000', 'last_clean': '2024-01-15T18:16:00.935298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:00.953595+0000', 'last_undegraded': '2024-01-15T18:16:00.953595+0000', 'last_fullsized': '2024-01-15T18:16:00.953595+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 85, 'state': 'creating+activating', 'last_fresh': '2024-01-15T18:16:01.957714+0000', 'last_change': '2024-01-15T18:16:01.957714+0000', 'last_active': '2024-01-15T18:16:00.935298+0000', 'last_peered': '2024-01-15T18:16:00.935298+0000', 'last_clean': '2024-01-15T18:16:00.935298+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.957714+0000', 'last_undegraded': '2024-01-15T18:16:01.957714+0000', 'last_fullsized': '2024-01-15T18:16:01.957714+0000', 'mapping_epoch': 84, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 84, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:00.935298+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-16T23:46:50.381475+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': ['1', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.3 is not active+clean 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.954987+0000', 'last_change': '2024-01-15T18:16:01.954987+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.954987+0000', 'last_undegraded': '2024-01-15T18:16:01.954987+0000', 'last_fullsized': '2024-01-15T18:16:01.954987+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 13.c is not active+clean 2024-01-15T18:16:07.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '13.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 85, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:16:01.959339+0000', 'last_change': '2024-01-15T18:16:01.959339+0000', 'last_active': '2024-01-15T18:16:01.943662+0000', 'last_peered': '2024-01-15T18:16:01.943662+0000', 'last_clean': '2024-01-15T18:16:01.943662+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:16:01.959339+0000', 'last_undegraded': '2024-01-15T18:16:01.959339+0000', 'last_fullsized': '2024-01-15T18:16:01.959339+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'last_clean_scrub_stamp': '2024-01-15T18:16:01.943662+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:16:07.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T18:16:07.932 INFO:tasks.ceph.mds.c.smithi143.stderr:2024-01-15T18:16:07.934+0000 7f20a4e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:16:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.990+0000 7f3777c00640 1 Processor -- start 2024-01-15T18:16:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.990+0000 7f3777c00640 1 -- start start 2024-01-15T18:16:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3777c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3770143780 0x7f37700cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3777c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37700ce520 0x7f3770071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3777c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37700ceb50 con 0x7f37700720f0 2024-01-15T18:16:07.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3777c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377015c110 con 0x7f3770143780 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3777c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3770072530 con 0x7f37700ce520 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3770143780 0x7f37700cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3770143780 0x7f37700cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55038/0 (socket says 172.21.15.143:55038) 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 -- 172.21.15.143:0/1614587450 learned_addr learned my addr 172.21.15.143:0/1614587450 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f37700720f0 0x7f377015a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34718/0 (socket says 172.21.15.143:34718) 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f376fe00640 1 --2- 172.21.15.143:0/1614587450 >> v2:172.21.15.204:3300/0 conn(0x7f37700ce520 0x7f3770071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 -- 172.21.15.143:0/1614587450 >> v2:172.21.15.204:3300/0 conn(0x7f37700ce520 msgr2=0x7f3770071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 --2- 172.21.15.143:0/1614587450 >> v2:172.21.15.204:3300/0 conn(0x7f37700ce520 0x7f3770071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 -- 172.21.15.143:0/1614587450 >> v1:172.21.15.143:6789/0 conn(0x7f37700720f0 legacy=0x7f377015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f3775200640 1 -- 172.21.15.143:0/1614587450 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377015b7e0 con 0x7f3770143780 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f376f400640 1 -- 172.21.15.143:0/1614587450 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886761957 0 0) 0x7f37680014a0 con 0x7f37700720f0 2024-01-15T18:16:07.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.994+0000 7f376fe00640 1 --2- 172.21.15.143:0/1614587450 >> v2:172.21.15.204:3300/0 conn(0x7f37700ce520 0x7f3770071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3775200640 1 --2- 172.21.15.143:0/1614587450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f37700cdfe0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f377006beb0 tx=0x7f376001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5097c798fc530 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f376f400640 1 -- 172.21.15.143:0/1614587450 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600ab610 con 0x7f3770143780 2024-01-15T18:16:07.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f376f400640 1 -- 172.21.15.143:0/1614587450 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37600ab770 con 0x7f3770143780 2024-01-15T18:16:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f376f400640 1 -- 172.21.15.143:0/1614587450 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600abb00 con 0x7f3770143780 2024-01-15T18:16:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 -- 172.21.15.143:0/1614587450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 msgr2=0x7f37700cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 --2- 172.21.15.143:0/1614587450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f37700cdfe0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f377006beb0 tx=0x7f376001c500 comp rx=0 tx=0).stop 2024-01-15T18:16:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 -- 172.21.15.143:0/1614587450 shutdown_connections 2024-01-15T18:16:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 --2- 172.21.15.143:0/1614587450 >> v2:172.21.15.204:3300/0 conn(0x7f37700ce520 0x7f3770071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 --2- 172.21.15.143:0/1614587450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f37700cdfe0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 -- 172.21.15.143:0/1614587450 >> 172.21.15.143:0/1614587450 conn(0x7f377006c3f0 msgr2=0x7f377006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 -- 172.21.15.143:0/1614587450 shutdown_connections 2024-01-15T18:16:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 -- 172.21.15.143:0/1614587450 wait complete. 2024-01-15T18:16:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 Processor -- start 2024-01-15T18:16:07.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:07.998+0000 7f3777c00640 1 -- start start 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37700720f0 0x7f3770137550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37700ce520 0x7f37701f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f3770137da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770156dd0 con 0x7f3770143780 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37701fa090 con 0x7f37700ce520 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37701fa370 con 0x7f37700720f0 2024-01-15T18:16:07.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f3770137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f3770137da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55050/0 (socket says 172.21.15.143:55050) 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 -- 172.21.15.143:0/2223133643 learned_addr learned my addr 172.21.15.143:0/2223133643 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775200640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37700720f0 0x7f3770137550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f376fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37700ce520 0x7f37701f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 -- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37700720f0 msgr2=0x7f3770137550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37700720f0 0x7f3770137550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 -- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37700ce520 msgr2=0x7f37701f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37700ce520 0x7f37701f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 -- 172.21.15.143:0/2223133643 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37600094d0 con 0x7f3770143780 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3775c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f3770137da0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f376800c730 tx=0x7f376800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376800ce40 con 0x7f3770143780 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376800a640 con 0x7f3770143780 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376800a9f0 con 0x7f3770143780 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3770156a60 con 0x7f3770143780 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3770071b70 con 0x7f3770143780 2024-01-15T18:16:08.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.002+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37701568c0 con 0x7f3770143780 2024-01-15T18:16:08.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.006+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f376800ab50 con 0x7f3770143780 2024-01-15T18:16:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.006+0000 7f376cc00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3748067610 0x7f3748069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.006+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3768081fa0 con 0x7f3770143780 2024-01-15T18:16:08.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.006+0000 7f3775200640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3748067610 0x7f3748069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.006+0000 7f3775200640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3748067610 0x7f3748069ad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f37600af000 tx=0x7f37600b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:08.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.010+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37701568c0 con 0x7f3770143780 2024-01-15T18:16:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.214+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f37700ceb50 con 0x7f3770143780 2024-01-15T18:16:08.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.214+0000 7f376cc00640 1 -- 172.21.15.143:0/2223133643 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1221 (secure 0 0 0) 0x7f37700ceb50 con 0x7f3770143780 2024-01-15T18:16:08.212 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:08.212 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":87,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341886,"num_in_osds":8,"osd_in_since":1705341876,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":67},{"state_name":"creating+peering","count":19},{"state_name":"unknown","count":9},{"state_name":"creating+activating","count":2}],"num_pgs":97,"num_pools":3,"num_objects":12,"data_bytes":592744,"bytes_used":605720576,"bytes_avail":772488392704,"bytes_total":773094113280,"unknown_pgs_ratio":0.092783503234386444,"inactive_pgs_ratio":0.2164948433637619,"write_bytes_sec":1212,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":52,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":8025}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-01-15T18:12:40.782196+0000","services":{}},"progress_events":{}} 2024-01-15T18:16:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3748067610 msgr2=0x7f3748069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3748067610 0x7f3748069ad0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f37600af000 tx=0x7f37600b6040 comp rx=0 tx=0).stop 2024-01-15T18:16:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 msgr2=0x7f3770137da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f3770137da0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f376800c730 tx=0x7f376800cbf0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 shutdown_connections 2024-01-15T18:16:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3748067610 0x7f3748069ad0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3770143780 0x7f3770137da0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37700ce520 0x7f37701f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 --2- 172.21.15.143:0/2223133643 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37700720f0 0x7f3770137550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 >> 172.21.15.143:0/2223133643 conn(0x7f377006c3f0 msgr2=0x7f37700ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 shutdown_connections 2024-01-15T18:16:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.218+0000 7f3777c00640 1 -- 172.21.15.143:0/2223133643 wait complete. 2024-01-15T18:16:08.235 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T18:16:08.236 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:16:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 Processor -- start 2024-01-15T18:16:08.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 -- start start 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5ff80cbf50 0x7f5ff80cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff80cffd0 con 0x7f5ff81607c0 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff80d1ce0 con 0x7f5ff80cbf50 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5fff200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80d02b0 con 0x7f5ff80cfb30 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5ff7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5ff7e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f5ff81607c0 0x7f5ff806b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58222/0 (socket says 172.21.15.143:58222) 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5ff7e00640 1 -- 172.21.15.143:0/1018413534 learned_addr learned my addr 172.21.15.143:0/1018413534 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:08.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.374+0000 7f5ffd200640 1 --2- 172.21.15.143:0/1018413534 >> v2:172.21.15.204:3300/0 conn(0x7f5ff80cbf50 0x7f5ff80cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ff6a00640 1 -- 172.21.15.143:0/1018413534 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428504962 0 0) 0x7f5ff80cffd0 con 0x7f5ff81607c0 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ff6a00640 1 -- 172.21.15.143:0/1018413534 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd8003530 con 0x7f5ff81607c0 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ffd200640 1 -- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 msgr2=0x7f5ff806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ffd200640 1 --2- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff806bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ffd200640 1 -- 172.21.15.143:0/1018413534 >> v1:172.21.15.204:6789/0 conn(0x7f5ff81607c0 legacy=0x7f5ff806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ffd200640 1 -- 172.21.15.143:0/1018413534 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80cb470 con 0x7f5ff80cbf50 2024-01-15T18:16:08.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ff7400640 1 --2- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff806bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ffd200640 1 --2- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff80cc390 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009d90 tx=0x7f5ff001f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e2001c6562f3c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:08.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ff6a00640 1 -- 172.21.15.143:0/1018413534 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00af600 con 0x7f5ff80cbf50 2024-01-15T18:16:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ff6a00640 1 -- 172.21.15.143:0/1018413534 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00ab050 con 0x7f5ff80cbf50 2024-01-15T18:16:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.378+0000 7f5ff6a00640 1 -- 172.21.15.143:0/1018413534 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00af980 con 0x7f5ff80cbf50 2024-01-15T18:16:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 msgr2=0x7f5ff80cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 --2- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff80cc390 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009d90 tx=0x7f5ff001f130 comp rx=0 tx=0).stop 2024-01-15T18:16:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- 172.21.15.143:0/1018413534 shutdown_connections 2024-01-15T18:16:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 --2- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff80cc390 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 --2- 172.21.15.143:0/1018413534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff806bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- 172.21.15.143:0/1018413534 >> 172.21.15.143:0/1018413534 conn(0x7f5ff80d0e20 msgr2=0x7f5ff80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- 172.21.15.143:0/1018413534 shutdown_connections 2024-01-15T18:16:08.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- 172.21.15.143:0/1018413534 wait complete. 2024-01-15T18:16:08.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 Processor -- start 2024-01-15T18:16:08.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- start start 2024-01-15T18:16:08.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff8153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 0x7f5ff80ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8155960 con 0x7f5ff81607c0 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8155c00 con 0x7f5ff80cbf50 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5fff200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8155ee0 con 0x7f5ff80cfb30 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5ffd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 0x7f5ff80ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5ffd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 0x7f5ff80ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55064/0 (socket says 172.21.15.143:55064) 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5ffd200640 1 -- 172.21.15.143:0/3571229311 learned_addr learned my addr 172.21.15.143:0/3571229311 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5ff7400640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff8153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.382+0000 7f5ff7e00640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ffd200640 1 -- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 msgr2=0x7f5ff8153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ffd200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff8153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ffd200640 1 -- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 msgr2=0x7f5ff8156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ffd200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff8156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ffd200640 1 -- 172.21.15.143:0/3571229311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009420 con 0x7f5ff81607c0 2024-01-15T18:16:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ffd200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 0x7f5ff80ca600 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00b2040 tx=0x7f5ff001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0009450 con 0x7f5ff81607c0 2024-01-15T18:16:08.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00b9410 con 0x7f5ff81607c0 2024-01-15T18:16:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00b97c0 con 0x7f5ff81607c0 2024-01-15T18:16:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff0009a70 con 0x7f5ff81607c0 2024-01-15T18:16:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81564a0 con 0x7f5ff81607c0 2024-01-15T18:16:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff806d4b0 con 0x7f5ff81607c0 2024-01-15T18:16:08.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f5ff00b6050 con 0x7f5ff81607c0 2024-01-15T18:16:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fefe00640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fd00675e0 0x7f5fd0069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5ff0121170 con 0x7f5ff81607c0 2024-01-15T18:16:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.386+0000 7f5ff7e00640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fd00675e0 0x7f5fd0069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:08.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.390+0000 7f5ff7e00640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fd00675e0 0x7f5fd0069aa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80011f0 tx=0x7f5fe8007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:08.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.390+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5ff00efeb0 con 0x7f5ff81607c0 2024-01-15T18:16:08.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.538+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ff80ce6b0 con 0x7f5fd00675e0 2024-01-15T18:16:08.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.546+0000 7f5fefe00640 1 -- 172.21.15.143:0/3571229311 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285130 (secure 0 0 0) 0x7f5fe8005860 con 0x7f5fd00675e0 2024-01-15T18:16:08.545 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:08.547 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fd00675e0 msgr2=0x7f5fd0069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fd00675e0 0x7f5fd0069aa0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80011f0 tx=0x7f5fe8007d40 comp rx=0 tx=0).stop 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 msgr2=0x7f5ff80ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 0x7f5ff80ca600 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00b2040 tx=0x7f5ff001bed0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 shutdown_connections 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5fd00675e0 0x7f5fd0069aa0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5ff81607c0 0x7f5ff80ca600 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5ff80cfb30 0x7f5ff8153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 --2- 172.21.15.143:0/3571229311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ff80cbf50 0x7f5ff8156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:08.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 >> 172.21.15.143:0/3571229311 conn(0x7f5ff80d0e20 msgr2=0x7f5ff815f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:08.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 shutdown_connections 2024-01-15T18:16:08.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:08.550+0000 7f5fff200640 1 -- 172.21.15.143:0/3571229311 wait complete. 2024-01-15T18:16:08.566 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-01-15T18:16:06.858046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592744,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":176,"acting":176,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":591524,"kb_used_data":9400,"kb_used_omap":12,"kb_used_meta":582003,"kb_avail":754383196,"statfs":{"total":773094113280,"available":772488392704,"internally_reserved":0,"allocated":9625600,"data_stored":4781846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":595971632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.913200"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172275+0000","last_change":"2024-01-15T18:16:02.977971+0000","last_active":"2024-01-15T18:16:04.172275+0000","last_peered":"2024-01-15T18:16:04.172275+0000","last_clean":"2024-01-15T18:16:04.172275+0000","last_became_active":"2024-01-15T18:16:02.977706+0000","last_became_peered":"2024-01-15T18:16:02.977706+0000","last_unstale":"2024-01-15T18:16:04.172275+0000","last_undegraded":"2024-01-15T18:16:04.172275+0000","last_fullsized":"2024-01-15T18:16:04.172275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958822+0000","last_change":"2024-01-15T18:16:01.958822+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958822+0000","last_undegraded":"2024-01-15T18:16:01.958822+0000","last_fullsized":"2024-01-15T18:16:01.958822+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983752+0000","last_change":"2024-01-15T18:16:02.982423+0000","last_active":"2024-01-15T18:16:03.983752+0000","last_peered":"2024-01-15T18:16:03.983752+0000","last_clean":"2024-01-15T18:16:03.983752+0000","last_became_active":"2024-01-15T18:16:02.981906+0000","last_became_peered":"2024-01-15T18:16:02.981906+0000","last_unstale":"2024-01-15T18:16:03.983752+0000","last_undegraded":"2024-01-15T18:16:03.983752+0000","last_fullsized":"2024-01-15T18:16:03.983752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:02.961035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.988011+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.987583+0000","last_became_peered":"2024-01-15T18:16:02.987583+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672319+0000","last_change":"2024-01-15T18:16:02.984894+0000","last_active":"2024-01-15T18:16:04.672319+0000","last_peered":"2024-01-15T18:16:04.672319+0000","last_clean":"2024-01-15T18:16:04.672319+0000","last_became_active":"2024-01-15T18:16:02.984537+0000","last_became_peered":"2024-01-15T18:16:02.984537+0000","last_unstale":"2024-01-15T18:16:04.672319+0000","last_undegraded":"2024-01-15T18:16:04.672319+0000","last_fullsized":"2024-01-15T18:16:04.672319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672358+0000","last_change":"2024-01-15T18:16:02.988234+0000","last_active":"2024-01-15T18:16:04.672358+0000","last_peered":"2024-01-15T18:16:04.672358+0000","last_clean":"2024-01-15T18:16:04.672358+0000","last_became_active":"2024-01-15T18:16:02.987822+0000","last_became_peered":"2024-01-15T18:16:02.987822+0000","last_unstale":"2024-01-15T18:16:04.672358+0000","last_undegraded":"2024-01-15T18:16:04.672358+0000","last_fullsized":"2024-01-15T18:16:04.672358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672449+0000","last_change":"2024-01-15T18:16:02.978613+0000","last_active":"2024-01-15T18:16:04.672449+0000","last_peered":"2024-01-15T18:16:04.672449+0000","last_clean":"2024-01-15T18:16:04.672449+0000","last_became_active":"2024-01-15T18:16:02.978331+0000","last_became_peered":"2024-01-15T18:16:02.978331+0000","last_unstale":"2024-01-15T18:16:04.672449+0000","last_undegraded":"2024-01-15T18:16:04.672449+0000","last_fullsized":"2024-01-15T18:16:04.672449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.985584+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.984527+0000","last_became_peered":"2024-01-15T18:16:02.984527+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.960055+0000","last_change":"2024-01-15T18:16:01.960055+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.960055+0000","last_undegraded":"2024-01-15T18:16:01.960055+0000","last_fullsized":"2024-01-15T18:16:01.960055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.171943+0000","last_change":"2024-01-15T18:16:02.976922+0000","last_active":"2024-01-15T18:16:04.171943+0000","last_peered":"2024-01-15T18:16:04.171943+0000","last_clean":"2024-01-15T18:16:04.171943+0000","last_became_active":"2024-01-15T18:16:02.976638+0000","last_became_peered":"2024-01-15T18:16:02.976638+0000","last_unstale":"2024-01-15T18:16:04.171943+0000","last_undegraded":"2024-01-15T18:16:04.171943+0000","last_fullsized":"2024-01-15T18:16:04.171943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:28.008839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.980605+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.978977+0000","last_became_peered":"2024-01-15T18:16:02.978977+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:17.866154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954608+0000","last_change":"2024-01-15T18:16:01.954608+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954608+0000","last_undegraded":"2024-01-15T18:16:01.954608+0000","last_fullsized":"2024-01-15T18:16:01.954608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982727+0000","last_change":"2024-01-15T18:16:02.979861+0000","last_active":"2024-01-15T18:16:03.982727+0000","last_peered":"2024-01-15T18:16:03.982727+0000","last_clean":"2024-01-15T18:16:03.982727+0000","last_became_active":"2024-01-15T18:16:02.979569+0000","last_became_peered":"2024-01-15T18:16:02.979569+0000","last_unstale":"2024-01-15T18:16:03.982727+0000","last_undegraded":"2024-01-15T18:16:03.982727+0000","last_fullsized":"2024-01-15T18:16:03.982727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984023+0000","last_change":"2024-01-15T18:16:02.977093+0000","last_active":"2024-01-15T18:16:03.984023+0000","last_peered":"2024-01-15T18:16:03.984023+0000","last_clean":"2024-01-15T18:16:03.984023+0000","last_became_active":"2024-01-15T18:16:02.974914+0000","last_became_peered":"2024-01-15T18:16:02.974914+0000","last_unstale":"2024-01-15T18:16:03.984023+0000","last_undegraded":"2024-01-15T18:16:03.984023+0000","last_fullsized":"2024-01-15T18:16:03.984023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.961445+0000","last_change":"2024-01-15T18:16:01.961445+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.961445+0000","last_undegraded":"2024-01-15T18:16:01.961445+0000","last_fullsized":"2024-01-15T18:16:01.961445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957359+0000","last_change":"2024-01-15T18:16:01.957359+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957359+0000","last_undegraded":"2024-01-15T18:16:01.957359+0000","last_fullsized":"2024-01-15T18:16:01.957359+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172017+0000","last_change":"2024-01-15T18:16:02.970218+0000","last_active":"2024-01-15T18:16:04.172017+0000","last_peered":"2024-01-15T18:16:04.172017+0000","last_clean":"2024-01-15T18:16:04.172017+0000","last_became_active":"2024-01-15T18:16:02.969901+0000","last_became_peered":"2024-01-15T18:16:02.969901+0000","last_unstale":"2024-01-15T18:16:04.172017+0000","last_undegraded":"2024-01-15T18:16:04.172017+0000","last_fullsized":"2024-01-15T18:16:04.172017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:31.551846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.958084+0000","last_change":"2024-01-15T18:16:01.958084+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.958084+0000","last_undegraded":"2024-01-15T18:16:01.958084+0000","last_fullsized":"2024-01-15T18:16:01.958084+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983932+0000","last_change":"2024-01-15T18:16:02.991391+0000","last_active":"2024-01-15T18:16:03.983932+0000","last_peered":"2024-01-15T18:16:03.983932+0000","last_clean":"2024-01-15T18:16:03.983932+0000","last_became_active":"2024-01-15T18:16:02.991138+0000","last_became_peered":"2024-01-15T18:16:02.991138+0000","last_unstale":"2024-01-15T18:16:03.983932+0000","last_undegraded":"2024-01-15T18:16:03.983932+0000","last_fullsized":"2024-01-15T18:16:03.983932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:06.968039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983406+0000","last_change":"2024-01-15T18:16:02.976507+0000","last_active":"2024-01-15T18:16:03.983406+0000","last_peered":"2024-01-15T18:16:03.983406+0000","last_clean":"2024-01-15T18:16:03.983406+0000","last_became_active":"2024-01-15T18:16:02.975765+0000","last_became_peered":"2024-01-15T18:16:02.975765+0000","last_unstale":"2024-01-15T18:16:03.983406+0000","last_undegraded":"2024-01-15T18:16:03.983406+0000","last_fullsized":"2024-01-15T18:16:03.983406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172094+0000","last_change":"2024-01-15T18:16:02.976103+0000","last_active":"2024-01-15T18:16:04.172094+0000","last_peered":"2024-01-15T18:16:04.172094+0000","last_clean":"2024-01-15T18:16:04.172094+0000","last_became_active":"2024-01-15T18:16:02.975829+0000","last_became_peered":"2024-01-15T18:16:02.975829+0000","last_unstale":"2024-01-15T18:16:04.172094+0000","last_undegraded":"2024-01-15T18:16:04.172094+0000","last_fullsized":"2024-01-15T18:16:04.172094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956707+0000","last_change":"2024-01-15T18:16:01.956707+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956707+0000","last_undegraded":"2024-01-15T18:16:01.956707+0000","last_fullsized":"2024-01-15T18:16:01.956707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956925+0000","last_change":"2024-01-15T18:16:01.956925+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956925+0000","last_undegraded":"2024-01-15T18:16:01.956925+0000","last_fullsized":"2024-01-15T18:16:01.956925+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982746+0000","last_change":"2024-01-15T18:16:02.974470+0000","last_active":"2024-01-15T18:16:03.982746+0000","last_peered":"2024-01-15T18:16:03.982746+0000","last_clean":"2024-01-15T18:16:03.982746+0000","last_became_active":"2024-01-15T18:16:02.974162+0000","last_became_peered":"2024-01-15T18:16:02.974162+0000","last_unstale":"2024-01-15T18:16:03.982746+0000","last_undegraded":"2024-01-15T18:16:03.982746+0000","last_fullsized":"2024-01-15T18:16:03.982746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172002+0000","last_change":"2024-01-15T18:16:02.986920+0000","last_active":"2024-01-15T18:16:04.172002+0000","last_peered":"2024-01-15T18:16:04.172002+0000","last_clean":"2024-01-15T18:16:04.172002+0000","last_became_active":"2024-01-15T18:16:02.986544+0000","last_became_peered":"2024-01-15T18:16:02.986544+0000","last_unstale":"2024-01-15T18:16:04.172002+0000","last_undegraded":"2024-01-15T18:16:04.172002+0000","last_fullsized":"2024-01-15T18:16:04.172002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:06.358800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983627+0000","last_change":"2024-01-15T18:16:02.981015+0000","last_active":"2024-01-15T18:16:03.983627+0000","last_peered":"2024-01-15T18:16:03.983627+0000","last_clean":"2024-01-15T18:16:03.983627+0000","last_became_active":"2024-01-15T18:16:02.979886+0000","last_became_peered":"2024-01-15T18:16:02.979886+0000","last_unstale":"2024-01-15T18:16:03.983627+0000","last_undegraded":"2024-01-15T18:16:03.983627+0000","last_fullsized":"2024-01-15T18:16:03.983627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:23.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004361+0000","last_change":"2024-01-15T18:16:01.968674+0000","last_active":"2024-01-15T18:16:03.004361+0000","last_peered":"2024-01-15T18:16:03.004361+0000","last_clean":"2024-01-15T18:16:03.004361+0000","last_became_active":"2024-01-15T18:16:01.967874+0000","last_became_peered":"2024-01-15T18:16:01.967874+0000","last_unstale":"2024-01-15T18:16:03.004361+0000","last_undegraded":"2024-01-15T18:16:03.004361+0000","last_fullsized":"2024-01-15T18:16:03.004361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172367+0000","last_change":"2024-01-15T18:16:02.971695+0000","last_active":"2024-01-15T18:16:04.172367+0000","last_peered":"2024-01-15T18:16:04.172367+0000","last_clean":"2024-01-15T18:16:04.172367+0000","last_became_active":"2024-01-15T18:16:02.971425+0000","last_became_peered":"2024-01-15T18:16:02.971425+0000","last_unstale":"2024-01-15T18:16:04.172367+0000","last_undegraded":"2024-01-15T18:16:04.172367+0000","last_fullsized":"2024-01-15T18:16:04.172367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002366+0000","last_change":"2024-01-15T18:16:01.960880+0000","last_active":"2024-01-15T18:16:03.002366+0000","last_peered":"2024-01-15T18:16:03.002366+0000","last_clean":"2024-01-15T18:16:03.002366+0000","last_became_active":"2024-01-15T18:16:01.960424+0000","last_became_peered":"2024-01-15T18:16:01.960424+0000","last_unstale":"2024-01-15T18:16:03.002366+0000","last_undegraded":"2024-01-15T18:16:03.002366+0000","last_fullsized":"2024-01-15T18:16:03.002366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:05.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172293+0000","last_change":"2024-01-15T18:16:02.987087+0000","last_active":"2024-01-15T18:16:04.172293+0000","last_peered":"2024-01-15T18:16:04.172293+0000","last_clean":"2024-01-15T18:16:04.172293+0000","last_became_active":"2024-01-15T18:16:02.986595+0000","last_became_peered":"2024-01-15T18:16:02.986595+0000","last_unstale":"2024-01-15T18:16:04.172293+0000","last_undegraded":"2024-01-15T18:16:04.172293+0000","last_fullsized":"2024-01-15T18:16:04.172293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973629+0000","last_change":"2024-01-15T18:16:01.972650+0000","last_active":"2024-01-15T18:16:01.973629+0000","last_peered":"2024-01-15T18:16:01.973629+0000","last_clean":"2024-01-15T18:16:01.973629+0000","last_became_active":"2024-01-15T18:16:01.972282+0000","last_became_peered":"2024-01-15T18:16:01.972282+0000","last_unstale":"2024-01-15T18:16:01.973629+0000","last_undegraded":"2024-01-15T18:16:01.973629+0000","last_fullsized":"2024-01-15T18:16:01.973629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984011+0000","last_change":"2024-01-15T18:16:02.980841+0000","last_active":"2024-01-15T18:16:03.984011+0000","last_peered":"2024-01-15T18:16:03.984011+0000","last_clean":"2024-01-15T18:16:03.984011+0000","last_became_active":"2024-01-15T18:16:02.980529+0000","last_became_peered":"2024-01-15T18:16:02.980529+0000","last_unstale":"2024-01-15T18:16:03.984011+0000","last_undegraded":"2024-01-15T18:16:03.984011+0000","last_fullsized":"2024-01-15T18:16:03.984011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:16.698507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.974689+0000","last_change":"2024-01-15T18:16:01.974052+0000","last_active":"2024-01-15T18:16:01.974689+0000","last_peered":"2024-01-15T18:16:01.974689+0000","last_clean":"2024-01-15T18:16:01.974689+0000","last_became_active":"2024-01-15T18:16:01.973571+0000","last_became_peered":"2024-01-15T18:16:01.973571+0000","last_unstale":"2024-01-15T18:16:01.974689+0000","last_undegraded":"2024-01-15T18:16:01.974689+0000","last_fullsized":"2024-01-15T18:16:01.974689+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956466+0000","last_change":"2024-01-15T18:16:01.956466+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956466+0000","last_undegraded":"2024-01-15T18:16:01.956466+0000","last_fullsized":"2024-01-15T18:16:01.956466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002241+0000","last_change":"2024-01-15T18:16:01.961667+0000","last_active":"2024-01-15T18:16:03.002241+0000","last_peered":"2024-01-15T18:16:03.002241+0000","last_clean":"2024-01-15T18:16:03.002241+0000","last_became_active":"2024-01-15T18:16:01.961336+0000","last_became_peered":"2024-01-15T18:16:01.961336+0000","last_unstale":"2024-01-15T18:16:03.002241+0000","last_undegraded":"2024-01-15T18:16:03.002241+0000","last_fullsized":"2024-01-15T18:16:03.002241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.970823+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.970523+0000","last_became_peered":"2024-01-15T18:16:02.970523+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000246+0000","last_change":"2024-01-15T18:16:01.958649+0000","last_active":"2024-01-15T18:16:03.000246+0000","last_peered":"2024-01-15T18:16:03.000246+0000","last_clean":"2024-01-15T18:16:03.000246+0000","last_became_active":"2024-01-15T18:16:01.958043+0000","last_became_peered":"2024-01-15T18:16:01.958043+0000","last_unstale":"2024-01-15T18:16:03.000246+0000","last_undegraded":"2024-01-15T18:16:03.000246+0000","last_fullsized":"2024-01-15T18:16:03.000246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983181+0000","last_change":"2024-01-15T18:16:02.983728+0000","last_active":"2024-01-15T18:16:03.983181+0000","last_peered":"2024-01-15T18:16:03.983181+0000","last_clean":"2024-01-15T18:16:03.983181+0000","last_became_active":"2024-01-15T18:16:02.981960+0000","last_became_peered":"2024-01-15T18:16:02.981960+0000","last_unstale":"2024-01-15T18:16:03.983181+0000","last_undegraded":"2024-01-15T18:16:03.983181+0000","last_fullsized":"2024-01-15T18:16:03.983181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.971514+0000","last_change":"2024-01-15T18:16:01.968471+0000","last_active":"2024-01-15T18:16:01.971514+0000","last_peered":"2024-01-15T18:16:01.971514+0000","last_clean":"2024-01-15T18:16:01.971514+0000","last_became_active":"2024-01-15T18:16:01.967508+0000","last_became_peered":"2024-01-15T18:16:01.967508+0000","last_unstale":"2024-01-15T18:16:01.971514+0000","last_undegraded":"2024-01-15T18:16:01.971514+0000","last_fullsized":"2024-01-15T18:16:01.971514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:33.150205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954100+0000","last_change":"2024-01-15T18:16:01.954100+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954100+0000","last_undegraded":"2024-01-15T18:16:01.954100+0000","last_fullsized":"2024-01-15T18:16:01.954100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.954226+0000","last_change":"2024-01-15T18:16:01.954226+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954226+0000","last_undegraded":"2024-01-15T18:16:01.954226+0000","last_fullsized":"2024-01-15T18:16:01.954226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:51.202598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955399+0000","last_change":"2024-01-15T18:16:01.955399+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955399+0000","last_undegraded":"2024-01-15T18:16:01.955399+0000","last_fullsized":"2024-01-15T18:16:01.955399+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.967235+0000","last_change":"2024-01-15T18:16:01.967948+0000","last_active":"2024-01-15T18:16:02.967235+0000","last_peered":"2024-01-15T18:16:02.967235+0000","last_clean":"2024-01-15T18:16:02.967235+0000","last_became_active":"2024-01-15T18:16:01.967662+0000","last_became_peered":"2024-01-15T18:16:01.967662+0000","last_unstale":"2024-01-15T18:16:02.967235+0000","last_undegraded":"2024-01-15T18:16:02.967235+0000","last_fullsized":"2024-01-15T18:16:02.967235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:28.859462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672459+0000","last_change":"2024-01-15T18:16:02.977816+0000","last_active":"2024-01-15T18:16:04.672459+0000","last_peered":"2024-01-15T18:16:04.672459+0000","last_clean":"2024-01-15T18:16:04.672459+0000","last_became_active":"2024-01-15T18:16:02.977488+0000","last_became_peered":"2024-01-15T18:16:02.977488+0000","last_unstale":"2024-01-15T18:16:04.672459+0000","last_undegraded":"2024-01-15T18:16:04.672459+0000","last_fullsized":"2024-01-15T18:16:04.672459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.966903+0000","last_change":"2024-01-15T18:16:01.970089+0000","last_active":"2024-01-15T18:16:02.966903+0000","last_peered":"2024-01-15T18:16:02.966903+0000","last_clean":"2024-01-15T18:16:02.966903+0000","last_became_active":"2024-01-15T18:16:01.969791+0000","last_became_peered":"2024-01-15T18:16:01.969791+0000","last_unstale":"2024-01-15T18:16:02.966903+0000","last_undegraded":"2024-01-15T18:16:02.966903+0000","last_fullsized":"2024-01-15T18:16:02.966903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:25.108362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.956190+0000","last_change":"2024-01-15T18:16:01.956190+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.956190+0000","last_undegraded":"2024-01-15T18:16:01.956190+0000","last_fullsized":"2024-01-15T18:16:01.956190+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002343+0000","last_change":"2024-01-15T18:16:01.967367+0000","last_active":"2024-01-15T18:16:03.002343+0000","last_peered":"2024-01-15T18:16:03.002343+0000","last_clean":"2024-01-15T18:16:03.002343+0000","last_became_active":"2024-01-15T18:16:01.966610+0000","last_became_peered":"2024-01-15T18:16:01.966610+0000","last_unstale":"2024-01-15T18:16:03.002343+0000","last_undegraded":"2024-01-15T18:16:03.002343+0000","last_fullsized":"2024-01-15T18:16:03.002343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983502+0000","last_change":"2024-01-15T18:16:02.976324+0000","last_active":"2024-01-15T18:16:03.983502+0000","last_peered":"2024-01-15T18:16:03.983502+0000","last_clean":"2024-01-15T18:16:03.983502+0000","last_became_active":"2024-01-15T18:16:02.975452+0000","last_became_peered":"2024-01-15T18:16:02.975452+0000","last_unstale":"2024-01-15T18:16:03.983502+0000","last_undegraded":"2024-01-15T18:16:03.983502+0000","last_fullsized":"2024-01-15T18:16:03.983502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:31.636033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004546+0000","last_change":"2024-01-15T18:16:01.971530+0000","last_active":"2024-01-15T18:16:03.004546+0000","last_peered":"2024-01-15T18:16:03.004546+0000","last_clean":"2024-01-15T18:16:03.004546+0000","last_became_active":"2024-01-15T18:16:01.971080+0000","last_became_peered":"2024-01-15T18:16:01.971080+0000","last_unstale":"2024-01-15T18:16:03.004546+0000","last_undegraded":"2024-01-15T18:16:03.004546+0000","last_fullsized":"2024-01-15T18:16:03.004546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.955769+0000","last_change":"2024-01-15T18:16:01.955769+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.955769+0000","last_undegraded":"2024-01-15T18:16:01.955769+0000","last_fullsized":"2024-01-15T18:16:01.955769+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968495+0000","last_change":"2024-01-15T18:16:01.968495+0000","last_active":"2024-01-15T18:16:01.968495+0000","last_peered":"2024-01-15T18:16:01.968495+0000","last_clean":"2024-01-15T18:16:01.968495+0000","last_became_active":"2024-01-15T18:16:01.967100+0000","last_became_peered":"2024-01-15T18:16:01.967100+0000","last_unstale":"2024-01-15T18:16:01.968495+0000","last_undegraded":"2024-01-15T18:16:01.968495+0000","last_fullsized":"2024-01-15T18:16:01.968495+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:29.198637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983817+0000","last_change":"2024-01-15T18:16:02.985570+0000","last_active":"2024-01-15T18:16:03.983817+0000","last_peered":"2024-01-15T18:16:03.983817+0000","last_clean":"2024-01-15T18:16:03.983817+0000","last_became_active":"2024-01-15T18:16:02.985303+0000","last_became_peered":"2024-01-15T18:16:02.985303+0000","last_unstale":"2024-01-15T18:16:03.983817+0000","last_undegraded":"2024-01-15T18:16:03.983817+0000","last_fullsized":"2024-01-15T18:16:03.983817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:55.037442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.968815+0000","last_change":"2024-01-15T18:16:01.965271+0000","last_active":"2024-01-15T18:16:01.968815+0000","last_peered":"2024-01-15T18:16:01.968815+0000","last_clean":"2024-01-15T18:16:01.968815+0000","last_became_active":"2024-01-15T18:16:01.964982+0000","last_became_peered":"2024-01-15T18:16:01.964982+0000","last_unstale":"2024-01-15T18:16:01.968815+0000","last_undegraded":"2024-01-15T18:16:01.968815+0000","last_fullsized":"2024-01-15T18:16:01.968815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.957808+0000","last_change":"2024-01-15T18:16:01.957808+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957808+0000","last_undegraded":"2024-01-15T18:16:01.957808+0000","last_fullsized":"2024-01-15T18:16:01.957808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971256+0000","last_change":"2024-01-15T18:16:01.971126+0000","last_active":"2024-01-15T18:16:02.971256+0000","last_peered":"2024-01-15T18:16:02.971256+0000","last_clean":"2024-01-15T18:16:02.971256+0000","last_became_active":"2024-01-15T18:16:01.969238+0000","last_became_peered":"2024-01-15T18:16:01.969238+0000","last_unstale":"2024-01-15T18:16:02.971256+0000","last_undegraded":"2024-01-15T18:16:02.971256+0000","last_fullsized":"2024-01-15T18:16:02.971256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:30.543482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954474+0000","last_change":"2024-01-15T18:16:01.954474+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954474+0000","last_undegraded":"2024-01-15T18:16:01.954474+0000","last_fullsized":"2024-01-15T18:16:01.954474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980185+0000","last_change":"2024-01-15T18:16:01.979247+0000","last_active":"2024-01-15T18:16:01.980185+0000","last_peered":"2024-01-15T18:16:01.980185+0000","last_clean":"2024-01-15T18:16:01.980185+0000","last_became_active":"2024-01-15T18:16:01.978885+0000","last_became_peered":"2024-01-15T18:16:01.978885+0000","last_unstale":"2024-01-15T18:16:01.980185+0000","last_undegraded":"2024-01-15T18:16:01.980185+0000","last_fullsized":"2024-01-15T18:16:01.980185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:42.030347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984081+0000","last_change":"2024-01-15T18:16:02.983891+0000","last_active":"2024-01-15T18:16:03.984081+0000","last_peered":"2024-01-15T18:16:03.984081+0000","last_clean":"2024-01-15T18:16:03.984081+0000","last_became_active":"2024-01-15T18:16:02.983591+0000","last_became_peered":"2024-01-15T18:16:02.983591+0000","last_unstale":"2024-01-15T18:16:03.984081+0000","last_undegraded":"2024-01-15T18:16:03.984081+0000","last_fullsized":"2024-01-15T18:16:03.984081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:32.572896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.998942+0000","last_change":"2024-01-15T18:16:01.971805+0000","last_active":"2024-01-15T18:16:02.998942+0000","last_peered":"2024-01-15T18:16:02.998942+0000","last_clean":"2024-01-15T18:16:02.998942+0000","last_became_active":"2024-01-15T18:16:01.971492+0000","last_became_peered":"2024-01-15T18:16:01.971492+0000","last_unstale":"2024-01-15T18:16:02.998942+0000","last_undegraded":"2024-01-15T18:16:02.998942+0000","last_fullsized":"2024-01-15T18:16:02.998942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.974741+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.974455+0000","last_became_peered":"2024-01-15T18:16:02.974455+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.973667+0000","last_change":"2024-01-15T18:16:01.970998+0000","last_active":"2024-01-15T18:16:01.973667+0000","last_peered":"2024-01-15T18:16:01.973667+0000","last_clean":"2024-01-15T18:16:01.973667+0000","last_became_active":"2024-01-15T18:16:01.970730+0000","last_became_peered":"2024-01-15T18:16:01.970730+0000","last_unstale":"2024-01-15T18:16:01.973667+0000","last_undegraded":"2024-01-15T18:16:01.973667+0000","last_fullsized":"2024-01-15T18:16:01.973667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983546+0000","last_change":"2024-01-15T18:16:02.989297+0000","last_active":"2024-01-15T18:16:03.983546+0000","last_peered":"2024-01-15T18:16:03.983546+0000","last_clean":"2024-01-15T18:16:03.983546+0000","last_became_active":"2024-01-15T18:16:02.988832+0000","last_became_peered":"2024-01-15T18:16:02.988832+0000","last_unstale":"2024-01-15T18:16:03.983546+0000","last_undegraded":"2024-01-15T18:16:03.983546+0000","last_fullsized":"2024-01-15T18:16:03.983546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:07.504686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.977256+0000","last_change":"2024-01-15T18:16:01.975857+0000","last_active":"2024-01-15T18:16:01.977256+0000","last_peered":"2024-01-15T18:16:01.977256+0000","last_clean":"2024-01-15T18:16:01.977256+0000","last_became_active":"2024-01-15T18:16:01.975585+0000","last_became_peered":"2024-01-15T18:16:01.975585+0000","last_unstale":"2024-01-15T18:16:01.977256+0000","last_undegraded":"2024-01-15T18:16:01.977256+0000","last_fullsized":"2024-01-15T18:16:01.977256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975212+0000","last_change":"2024-01-15T18:16:01.972470+0000","last_active":"2024-01-15T18:16:01.975212+0000","last_peered":"2024-01-15T18:16:01.975212+0000","last_clean":"2024-01-15T18:16:01.975212+0000","last_became_active":"2024-01-15T18:16:01.972202+0000","last_became_peered":"2024-01-15T18:16:01.972202+0000","last_unstale":"2024-01-15T18:16:01.975212+0000","last_undegraded":"2024-01-15T18:16:01.975212+0000","last_fullsized":"2024-01-15T18:16:01.975212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672268+0000","last_change":"2024-01-15T18:16:02.984567+0000","last_active":"2024-01-15T18:16:04.672268+0000","last_peered":"2024-01-15T18:16:04.672268+0000","last_clean":"2024-01-15T18:16:04.672268+0000","last_became_active":"2024-01-15T18:16:02.984223+0000","last_became_peered":"2024-01-15T18:16:02.984223+0000","last_unstale":"2024-01-15T18:16:04.672268+0000","last_undegraded":"2024-01-15T18:16:04.672268+0000","last_fullsized":"2024-01-15T18:16:04.672268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.952954+0000","last_change":"2024-01-15T18:16:00.952954+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.952954+0000","last_undegraded":"2024-01-15T18:16:00.952954+0000","last_fullsized":"2024-01-15T18:16:00.952954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T18:16:01.943662+0000","last_change":"2024-01-15T18:16:01.943662+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.943662+0000","last_undegraded":"2024-01-15T18:16:01.943662+0000","last_fullsized":"2024-01-15T18:16:01.943662+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":84,"state":"creating+peering","last_fresh":"2024-01-15T18:16:00.953595+0000","last_change":"2024-01-15T18:16:00.953595+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:00.953595+0000","last_undegraded":"2024-01-15T18:16:00.953595+0000","last_fullsized":"2024-01-15T18:16:00.953595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672166+0000","last_change":"2024-01-15T18:16:02.984528+0000","last_active":"2024-01-15T18:16:04.672166+0000","last_peered":"2024-01-15T18:16:04.672166+0000","last_clean":"2024-01-15T18:16:04.672166+0000","last_became_active":"2024-01-15T18:16:02.984179+0000","last_became_peered":"2024-01-15T18:16:02.984179+0000","last_unstale":"2024-01-15T18:16:04.672166+0000","last_undegraded":"2024-01-15T18:16:04.672166+0000","last_fullsized":"2024-01-15T18:16:04.672166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.975902+0000","last_change":"2024-01-15T18:16:01.974336+0000","last_active":"2024-01-15T18:16:01.975902+0000","last_peered":"2024-01-15T18:16:01.975902+0000","last_clean":"2024-01-15T18:16:01.975902+0000","last_became_active":"2024-01-15T18:16:01.974035+0000","last_became_peered":"2024-01-15T18:16:01.974035+0000","last_unstale":"2024-01-15T18:16:01.975902+0000","last_undegraded":"2024-01-15T18:16:01.975902+0000","last_fullsized":"2024-01-15T18:16:01.975902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983695+0000","last_change":"2024-01-15T18:16:02.981820+0000","last_active":"2024-01-15T18:16:03.983695+0000","last_peered":"2024-01-15T18:16:03.983695+0000","last_clean":"2024-01-15T18:16:03.983695+0000","last_became_active":"2024-01-15T18:16:02.981478+0000","last_became_peered":"2024-01-15T18:16:02.981478+0000","last_unstale":"2024-01-15T18:16:03.983695+0000","last_undegraded":"2024-01-15T18:16:03.983695+0000","last_fullsized":"2024-01-15T18:16:03.983695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":3,"reported_epoch":85,"state":"creating+activating","last_fresh":"2024-01-15T18:16:01.957714+0000","last_change":"2024-01-15T18:16:01.957714+0000","last_active":"2024-01-15T18:16:00.935298+0000","last_peered":"2024-01-15T18:16:00.935298+0000","last_clean":"2024-01-15T18:16:00.935298+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.957714+0000","last_undegraded":"2024-01-15T18:16:01.957714+0000","last_fullsized":"2024-01-15T18:16:01.957714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:50.381475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984005+0000","last_change":"2024-01-15T18:16:02.981174+0000","last_active":"2024-01-15T18:16:03.984005+0000","last_peered":"2024-01-15T18:16:03.984005+0000","last_clean":"2024-01-15T18:16:03.984005+0000","last_became_active":"2024-01-15T18:16:02.980913+0000","last_became_peered":"2024-01-15T18:16:02.980913+0000","last_unstale":"2024-01-15T18:16:03.984005+0000","last_undegraded":"2024-01-15T18:16:03.984005+0000","last_fullsized":"2024-01-15T18:16:03.984005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:12.319403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.969433+0000","last_change":"2024-01-15T18:16:01.967331+0000","last_active":"2024-01-15T18:16:01.969433+0000","last_peered":"2024-01-15T18:16:01.969433+0000","last_clean":"2024-01-15T18:16:01.969433+0000","last_became_active":"2024-01-15T18:16:01.967031+0000","last_became_peered":"2024-01-15T18:16:01.967031+0000","last_unstale":"2024-01-15T18:16:01.969433+0000","last_undegraded":"2024-01-15T18:16:01.969433+0000","last_fullsized":"2024-01-15T18:16:01.969433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:37.438291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172397+0000","last_change":"2024-01-15T18:16:02.969429+0000","last_active":"2024-01-15T18:16:04.172397+0000","last_peered":"2024-01-15T18:16:04.172397+0000","last_clean":"2024-01-15T18:16:04.172397+0000","last_became_active":"2024-01-15T18:16:02.969095+0000","last_became_peered":"2024-01-15T18:16:02.969095+0000","last_unstale":"2024-01-15T18:16:04.172397+0000","last_undegraded":"2024-01-15T18:16:04.172397+0000","last_fullsized":"2024-01-15T18:16:04.172397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964906+0000","last_change":"2024-01-15T18:16:01.968688+0000","last_active":"2024-01-15T18:16:02.964906+0000","last_peered":"2024-01-15T18:16:02.964906+0000","last_clean":"2024-01-15T18:16:02.964906+0000","last_became_active":"2024-01-15T18:16:01.968389+0000","last_became_peered":"2024-01-15T18:16:01.968389+0000","last_unstale":"2024-01-15T18:16:02.964906+0000","last_undegraded":"2024-01-15T18:16:02.964906+0000","last_fullsized":"2024-01-15T18:16:02.964906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.976712+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.976054+0000","last_became_peered":"2024-01-15T18:16:02.976054+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:26:19.031570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964799+0000","last_change":"2024-01-15T18:16:01.966697+0000","last_active":"2024-01-15T18:16:02.964799+0000","last_peered":"2024-01-15T18:16:02.964799+0000","last_clean":"2024-01-15T18:16:02.964799+0000","last_became_active":"2024-01-15T18:16:01.966397+0000","last_became_peered":"2024-01-15T18:16:01.966397+0000","last_unstale":"2024-01-15T18:16:02.964799+0000","last_undegraded":"2024-01-15T18:16:02.964799+0000","last_fullsized":"2024-01-15T18:16:02.964799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672238+0000","last_change":"2024-01-15T18:16:02.984048+0000","last_active":"2024-01-15T18:16:04.672238+0000","last_peered":"2024-01-15T18:16:04.672238+0000","last_clean":"2024-01-15T18:16:04.672238+0000","last_became_active":"2024-01-15T18:16:02.983546+0000","last_became_peered":"2024-01-15T18:16:02.983546+0000","last_unstale":"2024-01-15T18:16:04.672238+0000","last_undegraded":"2024-01-15T18:16:04.672238+0000","last_fullsized":"2024-01-15T18:16:04.672238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-15T18:16:01.980104+0000","last_change":"2024-01-15T18:16:01.979160+0000","last_active":"2024-01-15T18:16:01.980104+0000","last_peered":"2024-01-15T18:16:01.980104+0000","last_clean":"2024-01-15T18:16:01.980104+0000","last_became_active":"2024-01-15T18:16:01.978901+0000","last_became_peered":"2024-01-15T18:16:01.978901+0000","last_unstale":"2024-01-15T18:16:01.980104+0000","last_undegraded":"2024-01-15T18:16:01.980104+0000","last_fullsized":"2024-01-15T18:16:01.980104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.954987+0000","last_change":"2024-01-15T18:16:01.954987+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.954987+0000","last_undegraded":"2024-01-15T18:16:01.954987+0000","last_fullsized":"2024-01-15T18:16:01.954987+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.961378+0000","last_change":"2024-01-15T18:16:01.958840+0000","last_active":"2024-01-15T18:16:02.961378+0000","last_peered":"2024-01-15T18:16:02.961378+0000","last_clean":"2024-01-15T18:16:02.961378+0000","last_became_active":"2024-01-15T18:16:01.958367+0000","last_became_peered":"2024-01-15T18:16:01.958367+0000","last_unstale":"2024-01-15T18:16:02.961378+0000","last_undegraded":"2024-01-15T18:16:02.961378+0000","last_fullsized":"2024-01-15T18:16:02.961378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983304+0000","last_change":"2024-01-15T18:16:02.991930+0000","last_active":"2024-01-15T18:16:03.983304+0000","last_peered":"2024-01-15T18:16:03.983304+0000","last_clean":"2024-01-15T18:16:03.983304+0000","last_became_active":"2024-01-15T18:16:02.991630+0000","last_became_peered":"2024-01-15T18:16:02.991630+0000","last_unstale":"2024-01-15T18:16:03.983304+0000","last_undegraded":"2024-01-15T18:16:03.983304+0000","last_fullsized":"2024-01-15T18:16:03.983304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964174+0000","last_change":"2024-01-15T18:16:01.958054+0000","last_active":"2024-01-15T18:16:02.964174+0000","last_peered":"2024-01-15T18:16:02.964174+0000","last_clean":"2024-01-15T18:16:02.964174+0000","last_became_active":"2024-01-15T18:16:01.957368+0000","last_became_peered":"2024-01-15T18:16:01.957368+0000","last_unstale":"2024-01-15T18:16:02.964174+0000","last_undegraded":"2024-01-15T18:16:02.964174+0000","last_fullsized":"2024-01-15T18:16:02.964174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172054+0000","last_change":"2024-01-15T18:16:02.970069+0000","last_active":"2024-01-15T18:16:04.172054+0000","last_peered":"2024-01-15T18:16:04.172054+0000","last_clean":"2024-01-15T18:16:04.172054+0000","last_became_active":"2024-01-15T18:16:02.969737+0000","last_became_peered":"2024-01-15T18:16:02.969737+0000","last_unstale":"2024-01-15T18:16:04.172054+0000","last_undegraded":"2024-01-15T18:16:04.172054+0000","last_fullsized":"2024-01-15T18:16:04.172054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:06.396100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.963102+0000","last_change":"2024-01-15T18:16:01.968677+0000","last_active":"2024-01-15T18:16:02.963102+0000","last_peered":"2024-01-15T18:16:02.963102+0000","last_clean":"2024-01-15T18:16:02.963102+0000","last_became_active":"2024-01-15T18:16:01.967462+0000","last_became_peered":"2024-01-15T18:16:01.967462+0000","last_unstale":"2024-01-15T18:16:02.963102+0000","last_undegraded":"2024-01-15T18:16:02.963102+0000","last_fullsized":"2024-01-15T18:16:02.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":85,"state":"creating+peering","last_fresh":"2024-01-15T18:16:01.959339+0000","last_change":"2024-01-15T18:16:01.959339+0000","last_active":"2024-01-15T18:16:01.943662+0000","last_peered":"2024-01-15T18:16:01.943662+0000","last_clean":"2024-01-15T18:16:01.943662+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:16:01.959339+0000","last_undegraded":"2024-01-15T18:16:01.959339+0000","last_fullsized":"2024-01-15T18:16:01.959339+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000937+0000","last_change":"2024-01-15T18:16:01.973544+0000","last_active":"2024-01-15T18:16:03.000937+0000","last_peered":"2024-01-15T18:16:03.000937+0000","last_clean":"2024-01-15T18:16:03.000937+0000","last_became_active":"2024-01-15T18:16:01.973240+0000","last_became_peered":"2024-01-15T18:16:01.973240+0000","last_unstale":"2024-01-15T18:16:03.000937+0000","last_undegraded":"2024-01-15T18:16:03.000937+0000","last_fullsized":"2024-01-15T18:16:03.000937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:06.496321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971217+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:16:02.971217+0000","last_peered":"2024-01-15T18:16:02.971217+0000","last_clean":"2024-01-15T18:16:02.971217+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:16:02.971217+0000","last_undegraded":"2024-01-15T18:16:02.971217+0000","last_fullsized":"2024-01-15T18:16:02.971217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":110,"acting":110,"num_store_stats":7},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2376,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77164,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294676,"statfs":{"total":96636764160,"available":96557748224,"internally_reserved":0,"allocated":1667072,"data_stored":1050036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.3839999999999999,"15min":0.92200000000000004},"min":{"1min":0.621,"5min":0.36899999999999999,"15min":0.29599999999999999},"max":{"1min":16.815999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.044,"15min":0.87},"min":{"1min":0.57399999999999995,"5min":0.33400000000000002,"15min":0.28399999999999997},"max":{"1min":4.6100000000000003,"5min":8.5419999999999998,"15min":13.744},"last":0.441}]},{"osd":1,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.885,"5min":1.6519999999999999,"15min":1.0700000000000001},"min":{"1min":0.60599999999999998,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":16.920000000000002,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.637,"15min":1.0680000000000001},"min":{"1min":0.60299999999999998,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":4.7679999999999998,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.496,"15min":0.99199999999999999},"min":{"1min":0.52000000000000002,"5min":0.433,"15min":0.38400000000000001},"max":{"1min":2.1680000000000001,"5min":30.846,"15min":30.846},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.887,"5min":1.3140000000000001,"15min":1.097},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.39800000000000002},"max":{"1min":17.039999999999999,"5min":17.039999999999999,"15min":39.695},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.6299999999999999,"15min":1.163},"min":{"1min":0.55300000000000005,"5min":0.47499999999999998,"15min":0.38800000000000001},"max":{"1min":2.1160000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.831,"5min":2.0049999999999999,"15min":1.228},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":16.879999999999999,"5min":42.692,"15min":42.692},"last":0.70199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.89100000000000001,"15min":0.72199999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":14.087,"5min":14.087,"15min":14.087},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.36699999999999999,"15min":0.36299999999999999},"max":{"1min":2.2450000000000001,"5min":2.952,"15min":4.3710000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":0.80500000000000005,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":11.334,"5min":11.334,"15min":11.334},"last":0.505},{"interface":"front","average":{"1min":1.012,"5min":0.91000000000000003,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.8179999999999996,"5min":8.423,"15min":8.423},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.91300000000000003,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.7140000000000004,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.376},"max":{"1min":1.1970000000000001,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76588,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295252,"statfs":{"total":96636764160,"available":96558338048,"internally_reserved":0,"allocated":1081344,"data_stored":460292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.83799999999999997,"15min":0.94799999999999995},"min":{"1min":0.41399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78200000000000003,"15min":0.872},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.266,"5min":3.4940000000000002,"15min":12.449},"last":0.81599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.1379999999999999,"15min":1.0109999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.379},"max":{"1min":1.615,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.087,"15min":1.1759999999999999},"min":{"1min":0.55600000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":14.743,"15min":42.825000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":1.147},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1890000000000001,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78100000000000003,"15min":1.0269999999999999},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":1.3640000000000001,"5min":3.016,"15min":24.199000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.0489999999999999,"15min":1.0629999999999999},"min":{"1min":0.60699999999999998,"5min":0.47799999999999998,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":18.791,"15min":18.791},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.585,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.66100000000000003,"15min":0.75800000000000001},"min":{"1min":0.50800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":12.491},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.443},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.4379999999999999,"5min":2.5259999999999998,"15min":12.945},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.98599999999999999,"15min":0.84599999999999997},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":5.1829999999999998,"5min":11.907,"15min":11.907},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80800000000000005,"15min":0.88900000000000001},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5569999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.84199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54456,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317384,"statfs":{"total":96636764160,"available":96581001216,"internally_reserved":0,"allocated":1085440,"data_stored":461232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":0.82899999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.4770000000000003,"5min":6.6820000000000004,"15min":11.519},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.95699999999999996,"15min":0.90300000000000002},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.34999999999999998},"max":{"1min":1.714,"5min":7.6260000000000003,"15min":12.952},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95499999999999996,"15min":0.88500000000000001},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":4.3140000000000001,"5min":4.609,"15min":13.358000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.859,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.149,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":2.867,"5min":18.314,"15min":18.314},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.341,"15min":1.006},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.434},"max":{"1min":2.855,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.89500000000000002,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":1.839,"5min":4.0330000000000004,"15min":13.317},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.034,"15min":1},"min":{"1min":0.443,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.1440000000000001,"5min":15.205,"15min":19.638000000000002},"last":0.74399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.28799999999999998},"max":{"1min":2.4289999999999998,"5min":5.899,"15min":11.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32100000000000001},"max":{"1min":2.516,"5min":3.097,"15min":3.3889999999999998},"last":0.623}]},{"osd":6,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.80000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.238,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.9470000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.76200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":2.3900000000000001,"5min":2.9870000000000001,"15min":4.1799999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.40400000000000003},"max":{"1min":2.3239999999999998,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":1069056,"data_stored":459598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.097,"15min":0.92600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":9.4179999999999993,"5min":11.374000000000001,"15min":34.802},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.135,"15min":0.89300000000000002},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.34699999999999998},"max":{"1min":3.3450000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.97399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2130000000000001,"5min":7.625,"15min":38.198},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.93999999999999995,"15min":0.95799999999999996},"min":{"1min":0.55700000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0589999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0149999999999999,"15min":0.85499999999999998},"min":{"1min":0.61499999999999999,"5min":0.40799999999999997,"15min":0.38700000000000001},"max":{"1min":4.4089999999999998,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.621}]},{"osd":3,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.2729999999999999,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":3.415,"5min":14.529,"15min":14.529},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.2430000000000001,"15min":0.93300000000000005},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78100000000000003,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.6070000000000002,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.96899999999999997,"15min":0.83099999999999996},"min":{"1min":0.47499999999999998,"5min":0.39200000000000002,"15min":0.34100000000000003},"max":{"1min":3.0209999999999999,"5min":25.887,"15min":25.887},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":7.5439999999999996,"15min":10.284000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85999999999999999,"15min":0.79300000000000004},"min":{"1min":0.45000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1680000000000001,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":1.232,"5min":11.67,"15min":11.67},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.595,"15min":0.95199999999999996},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":7.2839999999999998,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.37},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.4670000000000001,"15min":0.94799999999999995},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":7.5170000000000003,"5min":23.074999999999999,"15min":28.332000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.4690000000000001,"15min":0.96299999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.2970000000000002,"5min":20.321000000000002,"15min":40.457999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.514,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.372,"15min":0.30299999999999999},"max":{"1min":7.3170000000000002,"5min":26.260999999999999,"15min":27.427},"last":0.57199999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.615,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.28399999999999997},"max":{"1min":9.1180000000000003,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.8240000000000001,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.34300000000000003},"max":{"1min":7.2409999999999997,"5min":26.404,"15min":26.404},"last":0.53100000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3819999999999999,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":4.1040000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.069},"min":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.377},"max":{"1min":2.1150000000000002,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.79600000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.6259999999999999,"15min":1.097},"min":{"1min":0.58399999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":6.5140000000000002,"5min":26.314,"15min":26.314},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.204,"5min":1.5649999999999999,"15min":1.131},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.44500000000000001},"max":{"1min":4.165,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.94399999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.635,"15min":1.1140000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.7619999999999996,"5min":20.445,"15min":20.445},"last":1},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.7270000000000001,"15min":1.1699999999999999},"min":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.495},"max":{"1min":6.7149999999999999,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.6839999999999999,"15min":1.155},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.2030000000000003,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":2.1429999999999998,"15min":1.3089999999999999},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46000000000000002},"max":{"1min":2.2490000000000001,"5min":44.817999999999998,"15min":44.817999999999998},"last":1.0900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":1.153,"15min":1.0449999999999999},"min":{"1min":0.33800000000000002,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":5.9950000000000001,"5min":12.695,"15min":29.527999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.591,"15min":1.3180000000000001},"min":{"1min":0.47499999999999998,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.80300000000000005,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.55400000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.288,"15min":1.5329999999999999},"min":{"1min":0.371,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":7.476,"5min":34.029000000000003,"15min":113.149},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.361,"5min":1.3340000000000001,"15min":1.099},"min":{"1min":0.36799999999999999,"5min":0.308,"15min":0.30199999999999999},"max":{"1min":14.728,"5min":20.619,"15min":41.302},"last":0.499}]},{"osd":3,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.944,"5min":2.1030000000000002,"15min":1.3620000000000001},"min":{"1min":0.36799999999999999,"5min":0.32300000000000001,"15min":0.28899999999999998},"max":{"1min":31.57,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.92900000000000005,"15min":1.258},"min":{"1min":0.41899999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":7.3609999999999998,"5min":7.3609999999999998,"15min":115.693},"last":0.52600000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.1279999999999999,"15min":1.2090000000000001},"min":{"1min":0.627,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":7.4169999999999998,"5min":7.4169999999999998,"15min":24.366},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.79600000000000004,"5min":1.3049999999999999,"15min":1.2090000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0580000000000001,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":2.548,"5min":2.222,"15min":1.482},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.43099999999999999},"max":{"1min":41.683999999999997,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.103,"5min":1.5169999999999999,"15min":1.3129999999999999},"min":{"1min":0.59399999999999997,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":7.5099999999999998,"5min":26.084,"15min":26.084},"last":0.81699999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":1.468,"15min":1.302},"min":{"1min":0.46899999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.208,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.746},{"interface":"front","average":{"1min":2.589,"5min":2.3210000000000002,"15min":1.556},"min":{"1min":0.65900000000000003,"5min":0.50700000000000001,"15min":0.47699999999999998},"max":{"1min":41.738999999999997,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:00 2024","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":1.3979999999999999,"15min":1.242},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":1.298,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":2.6000000000000001,"5min":2.383,"15min":1.629},"min":{"1min":0.68899999999999995,"5min":0.57699999999999996,"15min":0.48799999999999999},"max":{"1min":41.402000000000001,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.84199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76540,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295300,"statfs":{"total":96636764160,"available":96558387200,"internally_reserved":0,"allocated":1032192,"data_stored":440080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.97399999999999998,"15min":0.92500000000000004},"min":{"1min":0.33100000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.3330000000000002,"5min":7.1580000000000004,"15min":39.948},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.5660000000000001,"15min":1.1200000000000001},"min":{"1min":0.39600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":4.577,"5min":35.790999999999997,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":1.034,"15min":0.97399999999999998},"min":{"1min":0.39400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":6.516,"5min":11.346,"15min":49.229999999999997},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.1870000000000001,"15min":0.86499999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":6.931,"5min":7.9489999999999998,"15min":10.349},"last":0.40200000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":1.655,"15min":1.2190000000000001},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.4669999999999996,"5min":27.643999999999998,"15min":27.643999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.343,"15min":1.016},"min":{"1min":0.48499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":15.608000000000001,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.6819999999999999,"15min":1.276},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.40200000000000002},"max":{"1min":4.633,"5min":27.489999999999998,"15min":27.489999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2969999999999999,"15min":1.054},"min":{"1min":0.61099999999999999,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":5.9139999999999997,"5min":10.038,"15min":12.997},"last":0.70699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.7050000000000001,"15min":1.2390000000000001},"min":{"1min":0.65900000000000003,"5min":0.53800000000000003,"15min":0.50800000000000001},"max":{"1min":4.673,"5min":27.547000000000001,"15min":27.547000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.228,"5min":1.359,"15min":1.1419999999999999},"min":{"1min":0.68400000000000005,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":5.3399999999999999,"5min":9.9610000000000003,"15min":13.054},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.0780000000000001,"5min":1.698,"15min":1.2490000000000001},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":4.7060000000000004,"5min":27.579999999999998,"15min":27.579999999999998},"last":0.746},{"interface":"front","average":{"1min":0.97799999999999998,"5min":1.198,"15min":1.0760000000000001},"min":{"1min":0.60199999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.0840000000000001,"5min":11.643000000000001,"15min":11.712999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:15:02 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3600000000000001,"15min":1.145},"min":{"1min":0.59799999999999998,"5min":0.46899999999999997,"15min":0.46500000000000002},"max":{"1min":9.5660000000000007,"5min":11.833,"15min":11.833},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.226,"5min":1.4450000000000001,"15min":1.153},"min":{"1min":0.56299999999999994,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":5.9550000000000001,"5min":14.606,"15min":14.606},"last":0.83799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77120,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294720,"statfs":{"total":96636764160,"available":96557793280,"internally_reserved":0,"allocated":1626112,"data_stored":1030448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":1.4470000000000001,"15min":1.175},"min":{"1min":0.32500000000000001,"5min":0.26600000000000001,"15min":0.23599999999999999},"max":{"1min":1.137,"5min":16.149000000000001,"15min":31.977},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":1.254,"15min":0.94699999999999995},"min":{"1min":0.376,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.70999999999999996,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.63700000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":1.6719999999999999,"15min":1.0820000000000001},"min":{"1min":0.44900000000000001,"5min":0.35299999999999998,"15min":0.30199999999999999},"max":{"1min":1.7450000000000001,"5min":36.386000000000003,"15min":41.387999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2689999999999999,"15min":1.0429999999999999},"min":{"1min":0.33900000000000002,"5min":0.314,"15min":0.25},"max":{"1min":11.539999999999999,"5min":16.556000000000001,"15min":33.219999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":1.3260000000000001,"15min":1.1140000000000001},"min":{"1min":0.39300000000000002,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":1.2230000000000001,"5min":9.8740000000000006,"15min":31.943999999999999},"last":0.623},{"interface":"front","average":{"1min":0.59399999999999997,"5min":1.7689999999999999,"15min":1.0600000000000001},"min":{"1min":0.499,"5min":0.432,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":45.128,"15min":45.128},"last":0.69599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.2709999999999999,"15min":1.109},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":11.772,"5min":11.772,"15min":32.932000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.379,"15min":1.1870000000000001},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.44},"max":{"1min":1.2849999999999999,"5min":14.831,"15min":31.782},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":1.306,"15min":1.1850000000000001},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.43099999999999999},"max":{"1min":1.383,"5min":11.247999999999999,"15min":31.858000000000001},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.2929999999999999,"15min":1.1080000000000001},"min":{"1min":0.58399999999999996,"5min":0.52200000000000002,"15min":0.45800000000000002},"max":{"1min":11.958,"5min":11.958,"15min":33.151000000000003},"last":0.748}]},{"osd":6,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.3520000000000001,"15min":1.052},"min":{"1min":0.622,"5min":0.49399999999999999,"15min":0.46400000000000002},"max":{"1min":1.1479999999999999,"5min":13.055,"15min":13.055},"last":0.871},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.3460000000000001,"15min":1.137},"min":{"1min":0.64600000000000002,"5min":0.50800000000000001,"15min":0.498},"max":{"1min":11.833,"5min":11.833,"15min":33.098999999999997},"last":0.70699999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:14:59 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":1.4279999999999999,"15min":1.0580000000000001},"min":{"1min":0.64700000000000002,"5min":0.52000000000000002,"15min":0.501},"max":{"1min":1.091,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.4630000000000001,"15min":1.2430000000000001},"min":{"1min":0.60299999999999998,"5min":0.53700000000000003,"15min":0.504},"max":{"1min":1.335,"5min":10.377000000000001,"15min":31.905999999999999},"last":0.88400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:16:11.572 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:16:11.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 Processor -- start 2024-01-15T18:16:11.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 -- start start 2024-01-15T18:16:11.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3a0015b260 0x7f3a000cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a000ccaf0 0x7f3a000cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a000c4d90 con 0x7f3a000d19f0 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0015b640 con 0x7f3a000ccaf0 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f3a05600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a000cf800 con 0x7f3a0015b260 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a000ccaf0 0x7f3a000cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3a000ccaf0 0x7f3a000cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55076/0 (socket says 172.21.15.143:55076) 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 -- 172.21.15.143:0/4217221812 learned_addr learned my addr 172.21.15.143:0/4217221812 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39fea00640 1 --2- 172.21.15.143:0/4217221812 >> v2:172.21.15.204:3300/0 conn(0x7f3a0015b260 0x7f3a000cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 -- 172.21.15.143:0/4217221812 >> v2:172.21.15.204:3300/0 conn(0x7f3a0015b260 msgr2=0x7f3a000cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 --2- 172.21.15.143:0/4217221812 >> v2:172.21.15.204:3300/0 conn(0x7f3a0015b260 0x7f3a000cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 -- 172.21.15.143:0/4217221812 >> v1:172.21.15.204:6789/0 conn(0x7f3a000d19f0 legacy=0x7f3a000c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:11.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.722+0000 7f39ff400640 1 -- 172.21.15.143:0/4217221812 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a000cfae0 con 0x7f3a000ccaf0 2024-01-15T18:16:11.723 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f39ff400640 1 --2- 172.21.15.143:0/4217221812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a000ccaf0 0x7f3a000cdd50 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3a000681e0 tx=0x7f39f801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204fd78bb2615544 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:11.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f39fd600640 1 -- 172.21.15.143:0/4217221812 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80a5740 con 0x7f3a000ccaf0 2024-01-15T18:16:11.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f39fd600640 1 -- 172.21.15.143:0/4217221812 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f80a58a0 con 0x7f3a000ccaf0 2024-01-15T18:16:11.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f39fd600640 1 -- 172.21.15.143:0/4217221812 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80a5c30 con 0x7f3a000ccaf0 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 -- 172.21.15.143:0/4217221812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a000ccaf0 msgr2=0x7f3a000cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 --2- 172.21.15.143:0/4217221812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a000ccaf0 0x7f3a000cdd50 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3a000681e0 tx=0x7f39f801c990 comp rx=0 tx=0).stop 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 -- 172.21.15.143:0/4217221812 shutdown_connections 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 --2- 172.21.15.143:0/4217221812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a000ccaf0 0x7f3a000cdd50 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 --2- 172.21.15.143:0/4217221812 >> v2:172.21.15.204:3300/0 conn(0x7f3a0015b260 0x7f3a000cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 -- 172.21.15.143:0/4217221812 >> 172.21.15.143:0/4217221812 conn(0x7f3a000d0ed0 msgr2=0x7f3a000d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 -- 172.21.15.143:0/4217221812 shutdown_connections 2024-01-15T18:16:11.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.726+0000 7f3a05600640 1 -- 172.21.15.143:0/4217221812 wait complete. 2024-01-15T18:16:11.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 Processor -- start 2024-01-15T18:16:11.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 -- start start 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a000ccaf0 0x7f3a00159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a000d19f0 0x7f3a000b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 0x7f3a000b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a000b2b30 con 0x7f3a0015b260 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a000b5a00 con 0x7f3a000ccaf0 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a000b5ce0 con 0x7f3a000d19f0 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 0x7f3a000b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:11.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 0x7f3a000b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55082/0 (socket says 172.21.15.143:55082) 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 -- 172.21.15.143:0/4147285958 learned_addr learned my addr 172.21.15.143:0/4147285958 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39fea00640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a000ccaf0 0x7f3a00159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39fe000640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a000d19f0 0x7f3a000b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 -- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a000d19f0 msgr2=0x7f3a000b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a000d19f0 0x7f3a000b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 -- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a000ccaf0 msgr2=0x7f3a00159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a000ccaf0 0x7f3a00159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 -- 172.21.15.143:0/4147285958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f8009b30 con 0x7f3a0015b260 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39fea00640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a000ccaf0 0x7f3a00159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39ff400640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 0x7f3a000b5330 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f39f801a5c0 tx=0x7f39f8019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80b8670 con 0x7f3a0015b260 2024-01-15T18:16:11.730 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f80b87f0 con 0x7f3a0015b260 2024-01-15T18:16:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a000b5fc0 con 0x7f3a0015b260 2024-01-15T18:16:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80b8ba0 con 0x7f3a0015b260 2024-01-15T18:16:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.730+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a00159cd0 con 0x7f3a0015b260 2024-01-15T18:16:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.734+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a00159e60 con 0x7f3a0015b260 2024-01-15T18:16:11.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.734+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f39f80b8d00 con 0x7f3a0015b260 2024-01-15T18:16:11.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.734+0000 7f39f6a00640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39d80675e0 0x7f39d8069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:11.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.734+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f39f811e670 con 0x7f3a0015b260 2024-01-15T18:16:11.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.734+0000 7f39fea00640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39d80675e0 0x7f39d8069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:11.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.738+0000 7f39fea00640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39d80675e0 0x7f39d8069aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f39f000b9f0 tx=0x7f39f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:11.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.738+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f39f80ed1f0 con 0x7f3a0015b260 2024-01-15T18:16:11.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.890+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a000b6470 con 0x7f39d80675e0 2024-01-15T18:16:11.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.898+0000 7f39f6a00640 1 -- 172.21.15.143:0/4147285958 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288260 (secure 0 0 0) 0x7f3a000b6470 con 0x7f39d80675e0 2024-01-15T18:16:11.898 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:11.900 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:16:11.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39d80675e0 msgr2=0x7f39d8069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:11.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39d80675e0 0x7f39d8069aa0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f39f000b9f0 tx=0x7f39f0008040 comp rx=0 tx=0).stop 2024-01-15T18:16:11.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 msgr2=0x7f3a000b5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:11.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 0x7f3a000b5330 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f39f801a5c0 tx=0x7f39f8019730 comp rx=0 tx=0).stop 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 shutdown_connections 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f39d80675e0 0x7f39d8069aa0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a0015b260 0x7f3a000b5330 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3a000d19f0 0x7f3a000b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 --2- 172.21.15.143:0/4147285958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a000ccaf0 0x7f3a00159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 >> 172.21.15.143:0/4147285958 conn(0x7f3a000d0ed0 msgr2=0x7f3a000c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 shutdown_connections 2024-01-15T18:16:11.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:11.902+0000 7f3a05600640 1 -- 172.21.15.143:0/4147285958 wait complete. 2024-01-15T18:16:11.919 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-01-15T18:16:10.859188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":591708,"kb_used_data":9576,"kb_used_omap":12,"kb_used_meta":582003,"kb_avail":754383012,"statfs":{"total":773094113280,"available":772488204288,"internally_reserved":0,"allocated":9805824,"data_stored":4862340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":595971632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002195"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172275+0000","last_change":"2024-01-15T18:16:02.977971+0000","last_active":"2024-01-15T18:16:04.172275+0000","last_peered":"2024-01-15T18:16:04.172275+0000","last_clean":"2024-01-15T18:16:04.172275+0000","last_became_active":"2024-01-15T18:16:02.977706+0000","last_became_peered":"2024-01-15T18:16:02.977706+0000","last_unstale":"2024-01-15T18:16:04.172275+0000","last_undegraded":"2024-01-15T18:16:04.172275+0000","last_fullsized":"2024-01-15T18:16:04.172275+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.679053+0000","last_change":"2024-01-15T18:16:02.986453+0000","last_active":"2024-01-15T18:16:04.679053+0000","last_peered":"2024-01-15T18:16:04.679053+0000","last_clean":"2024-01-15T18:16:04.679053+0000","last_became_active":"2024-01-15T18:16:02.986201+0000","last_became_peered":"2024-01-15T18:16:02.986201+0000","last_unstale":"2024-01-15T18:16:04.679053+0000","last_undegraded":"2024-01-15T18:16:04.679053+0000","last_fullsized":"2024-01-15T18:16:04.679053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:08:16.304982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983752+0000","last_change":"2024-01-15T18:16:02.982423+0000","last_active":"2024-01-15T18:16:03.983752+0000","last_peered":"2024-01-15T18:16:03.983752+0000","last_clean":"2024-01-15T18:16:03.983752+0000","last_became_active":"2024-01-15T18:16:02.981906+0000","last_became_peered":"2024-01-15T18:16:02.981906+0000","last_unstale":"2024-01-15T18:16:03.983752+0000","last_undegraded":"2024-01-15T18:16:03.983752+0000","last_fullsized":"2024-01-15T18:16:03.983752+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:02.961035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549892+0000","last_change":"2024-01-15T18:16:02.996209+0000","last_active":"2024-01-15T18:16:04.549892+0000","last_peered":"2024-01-15T18:16:04.549892+0000","last_clean":"2024-01-15T18:16:04.549892+0000","last_became_active":"2024-01-15T18:16:02.995949+0000","last_became_peered":"2024-01-15T18:16:02.995949+0000","last_unstale":"2024-01-15T18:16:04.549892+0000","last_undegraded":"2024-01-15T18:16:04.549892+0000","last_fullsized":"2024-01-15T18:16:04.549892+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:48.291910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.988011+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.987583+0000","last_became_peered":"2024-01-15T18:16:02.987583+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672319+0000","last_change":"2024-01-15T18:16:02.984894+0000","last_active":"2024-01-15T18:16:04.672319+0000","last_peered":"2024-01-15T18:16:04.672319+0000","last_clean":"2024-01-15T18:16:04.672319+0000","last_became_active":"2024-01-15T18:16:02.984537+0000","last_became_peered":"2024-01-15T18:16:02.984537+0000","last_unstale":"2024-01-15T18:16:04.672319+0000","last_undegraded":"2024-01-15T18:16:04.672319+0000","last_fullsized":"2024-01-15T18:16:04.672319+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549674+0000","last_change":"2024-01-15T18:16:02.982352+0000","last_active":"2024-01-15T18:16:04.549674+0000","last_peered":"2024-01-15T18:16:04.549674+0000","last_clean":"2024-01-15T18:16:04.549674+0000","last_became_active":"2024-01-15T18:16:02.981834+0000","last_became_peered":"2024-01-15T18:16:02.981834+0000","last_unstale":"2024-01-15T18:16:04.549674+0000","last_undegraded":"2024-01-15T18:16:04.549674+0000","last_fullsized":"2024-01-15T18:16:04.549674+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:23.654645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672358+0000","last_change":"2024-01-15T18:16:02.988234+0000","last_active":"2024-01-15T18:16:04.672358+0000","last_peered":"2024-01-15T18:16:04.672358+0000","last_clean":"2024-01-15T18:16:04.672358+0000","last_became_active":"2024-01-15T18:16:02.987822+0000","last_became_peered":"2024-01-15T18:16:02.987822+0000","last_unstale":"2024-01-15T18:16:04.672358+0000","last_undegraded":"2024-01-15T18:16:04.672358+0000","last_fullsized":"2024-01-15T18:16:04.672358+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672449+0000","last_change":"2024-01-15T18:16:02.978613+0000","last_active":"2024-01-15T18:16:04.672449+0000","last_peered":"2024-01-15T18:16:04.672449+0000","last_clean":"2024-01-15T18:16:04.672449+0000","last_became_active":"2024-01-15T18:16:02.978331+0000","last_became_peered":"2024-01-15T18:16:02.978331+0000","last_unstale":"2024-01-15T18:16:04.672449+0000","last_undegraded":"2024-01-15T18:16:04.672449+0000","last_fullsized":"2024-01-15T18:16:04.672449+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.985584+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.984527+0000","last_became_peered":"2024-01-15T18:16:02.984527+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678951+0000","last_change":"2024-01-15T18:16:02.985416+0000","last_active":"2024-01-15T18:16:04.678951+0000","last_peered":"2024-01-15T18:16:04.678951+0000","last_clean":"2024-01-15T18:16:04.678951+0000","last_became_active":"2024-01-15T18:16:02.985187+0000","last_became_peered":"2024-01-15T18:16:02.985187+0000","last_unstale":"2024-01-15T18:16:04.678951+0000","last_undegraded":"2024-01-15T18:16:04.678951+0000","last_fullsized":"2024-01-15T18:16:04.678951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:01:49.339199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.171943+0000","last_change":"2024-01-15T18:16:02.976922+0000","last_active":"2024-01-15T18:16:04.171943+0000","last_peered":"2024-01-15T18:16:04.171943+0000","last_clean":"2024-01-15T18:16:04.171943+0000","last_became_active":"2024-01-15T18:16:02.976638+0000","last_became_peered":"2024-01-15T18:16:02.976638+0000","last_unstale":"2024-01-15T18:16:04.171943+0000","last_undegraded":"2024-01-15T18:16:04.171943+0000","last_fullsized":"2024-01-15T18:16:04.171943+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:28.008839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.980605+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.978977+0000","last_became_peered":"2024-01-15T18:16:02.978977+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:17.866154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549980+0000","last_change":"2024-01-15T18:16:02.991158+0000","last_active":"2024-01-15T18:16:04.549980+0000","last_peered":"2024-01-15T18:16:04.549980+0000","last_clean":"2024-01-15T18:16:04.549980+0000","last_became_active":"2024-01-15T18:16:02.990890+0000","last_became_peered":"2024-01-15T18:16:02.990890+0000","last_unstale":"2024-01-15T18:16:04.549980+0000","last_undegraded":"2024-01-15T18:16:04.549980+0000","last_fullsized":"2024-01-15T18:16:04.549980+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:40.275555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.987068+0000","last_change":"2024-01-15T18:16:02.982167+0000","last_active":"2024-01-15T18:16:03.987068+0000","last_peered":"2024-01-15T18:16:03.987068+0000","last_clean":"2024-01-15T18:16:03.987068+0000","last_became_active":"2024-01-15T18:16:02.980915+0000","last_became_peered":"2024-01-15T18:16:02.980915+0000","last_unstale":"2024-01-15T18:16:03.987068+0000","last_undegraded":"2024-01-15T18:16:03.987068+0000","last_fullsized":"2024-01-15T18:16:03.987068+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982727+0000","last_change":"2024-01-15T18:16:02.979861+0000","last_active":"2024-01-15T18:16:03.982727+0000","last_peered":"2024-01-15T18:16:03.982727+0000","last_clean":"2024-01-15T18:16:03.982727+0000","last_became_active":"2024-01-15T18:16:02.979569+0000","last_became_peered":"2024-01-15T18:16:02.979569+0000","last_unstale":"2024-01-15T18:16:03.982727+0000","last_undegraded":"2024-01-15T18:16:03.982727+0000","last_fullsized":"2024-01-15T18:16:03.982727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984023+0000","last_change":"2024-01-15T18:16:02.977093+0000","last_active":"2024-01-15T18:16:03.984023+0000","last_peered":"2024-01-15T18:16:03.984023+0000","last_clean":"2024-01-15T18:16:03.984023+0000","last_became_active":"2024-01-15T18:16:02.974914+0000","last_became_peered":"2024-01-15T18:16:02.974914+0000","last_unstale":"2024-01-15T18:16:03.984023+0000","last_undegraded":"2024-01-15T18:16:03.984023+0000","last_fullsized":"2024-01-15T18:16:03.984023+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549927+0000","last_change":"2024-01-15T18:16:02.986735+0000","last_active":"2024-01-15T18:16:04.549927+0000","last_peered":"2024-01-15T18:16:04.549927+0000","last_clean":"2024-01-15T18:16:04.549927+0000","last_became_active":"2024-01-15T18:16:02.986480+0000","last_became_peered":"2024-01-15T18:16:02.986480+0000","last_unstale":"2024-01-15T18:16:04.549927+0000","last_undegraded":"2024-01-15T18:16:04.549927+0000","last_fullsized":"2024-01-15T18:16:04.549927+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:35.436527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.384007+0000","last_change":"2024-01-15T18:16:02.980583+0000","last_active":"2024-01-15T18:16:04.384007+0000","last_peered":"2024-01-15T18:16:04.384007+0000","last_clean":"2024-01-15T18:16:04.384007+0000","last_became_active":"2024-01-15T18:16:02.980302+0000","last_became_peered":"2024-01-15T18:16:02.980302+0000","last_unstale":"2024-01-15T18:16:04.384007+0000","last_undegraded":"2024-01-15T18:16:04.384007+0000","last_fullsized":"2024-01-15T18:16:04.384007+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.550037+0000","last_change":"2024-01-15T18:16:02.978130+0000","last_active":"2024-01-15T18:16:04.550037+0000","last_peered":"2024-01-15T18:16:04.550037+0000","last_clean":"2024-01-15T18:16:04.550037+0000","last_became_active":"2024-01-15T18:16:02.977864+0000","last_became_peered":"2024-01-15T18:16:02.977864+0000","last_unstale":"2024-01-15T18:16:04.550037+0000","last_undegraded":"2024-01-15T18:16:04.550037+0000","last_fullsized":"2024-01-15T18:16:04.550037+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:33:40.330923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.550013+0000","last_change":"2024-01-15T18:16:02.978040+0000","last_active":"2024-01-15T18:16:04.550013+0000","last_peered":"2024-01-15T18:16:04.550013+0000","last_clean":"2024-01-15T18:16:04.550013+0000","last_became_active":"2024-01-15T18:16:02.976113+0000","last_became_peered":"2024-01-15T18:16:02.976113+0000","last_unstale":"2024-01-15T18:16:04.550013+0000","last_undegraded":"2024-01-15T18:16:04.550013+0000","last_fullsized":"2024-01-15T18:16:04.550013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:55.837509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678571+0000","last_change":"2024-01-15T18:16:02.988763+0000","last_active":"2024-01-15T18:16:04.678571+0000","last_peered":"2024-01-15T18:16:04.678571+0000","last_clean":"2024-01-15T18:16:04.678571+0000","last_became_active":"2024-01-15T18:16:02.988490+0000","last_became_peered":"2024-01-15T18:16:02.988490+0000","last_unstale":"2024-01-15T18:16:04.678571+0000","last_undegraded":"2024-01-15T18:16:04.678571+0000","last_fullsized":"2024-01-15T18:16:04.678571+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:12:56.527247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172017+0000","last_change":"2024-01-15T18:16:02.970218+0000","last_active":"2024-01-15T18:16:04.172017+0000","last_peered":"2024-01-15T18:16:04.172017+0000","last_clean":"2024-01-15T18:16:04.172017+0000","last_became_active":"2024-01-15T18:16:02.969901+0000","last_became_peered":"2024-01-15T18:16:02.969901+0000","last_unstale":"2024-01-15T18:16:04.172017+0000","last_undegraded":"2024-01-15T18:16:04.172017+0000","last_fullsized":"2024-01-15T18:16:04.172017+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:31.551846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549548+0000","last_change":"2024-01-15T18:16:02.976433+0000","last_active":"2024-01-15T18:16:04.549548+0000","last_peered":"2024-01-15T18:16:04.549548+0000","last_clean":"2024-01-15T18:16:04.549548+0000","last_became_active":"2024-01-15T18:16:02.976144+0000","last_became_peered":"2024-01-15T18:16:02.976144+0000","last_unstale":"2024-01-15T18:16:04.549548+0000","last_undegraded":"2024-01-15T18:16:04.549548+0000","last_fullsized":"2024-01-15T18:16:04.549548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:04.455622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.679058+0000","last_change":"2024-01-15T18:16:02.989770+0000","last_active":"2024-01-15T18:16:04.679058+0000","last_peered":"2024-01-15T18:16:04.679058+0000","last_clean":"2024-01-15T18:16:04.679058+0000","last_became_active":"2024-01-15T18:16:02.989490+0000","last_became_peered":"2024-01-15T18:16:02.989490+0000","last_unstale":"2024-01-15T18:16:04.679058+0000","last_undegraded":"2024-01-15T18:16:04.679058+0000","last_fullsized":"2024-01-15T18:16:04.679058+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:58:01.698966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983932+0000","last_change":"2024-01-15T18:16:02.991391+0000","last_active":"2024-01-15T18:16:03.983932+0000","last_peered":"2024-01-15T18:16:03.983932+0000","last_clean":"2024-01-15T18:16:03.983932+0000","last_became_active":"2024-01-15T18:16:02.991138+0000","last_became_peered":"2024-01-15T18:16:02.991138+0000","last_unstale":"2024-01-15T18:16:03.983932+0000","last_undegraded":"2024-01-15T18:16:03.983932+0000","last_fullsized":"2024-01-15T18:16:03.983932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:06.968039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983406+0000","last_change":"2024-01-15T18:16:02.976507+0000","last_active":"2024-01-15T18:16:03.983406+0000","last_peered":"2024-01-15T18:16:03.983406+0000","last_clean":"2024-01-15T18:16:03.983406+0000","last_became_active":"2024-01-15T18:16:02.975765+0000","last_became_peered":"2024-01-15T18:16:02.975765+0000","last_unstale":"2024-01-15T18:16:03.983406+0000","last_undegraded":"2024-01-15T18:16:03.983406+0000","last_fullsized":"2024-01-15T18:16:03.983406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172094+0000","last_change":"2024-01-15T18:16:02.976103+0000","last_active":"2024-01-15T18:16:04.172094+0000","last_peered":"2024-01-15T18:16:04.172094+0000","last_clean":"2024-01-15T18:16:04.172094+0000","last_became_active":"2024-01-15T18:16:02.975829+0000","last_became_peered":"2024-01-15T18:16:02.975829+0000","last_unstale":"2024-01-15T18:16:04.172094+0000","last_undegraded":"2024-01-15T18:16:04.172094+0000","last_fullsized":"2024-01-15T18:16:04.172094+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678757+0000","last_change":"2024-01-15T18:16:02.983604+0000","last_active":"2024-01-15T18:16:04.678757+0000","last_peered":"2024-01-15T18:16:04.678757+0000","last_clean":"2024-01-15T18:16:04.678757+0000","last_became_active":"2024-01-15T18:16:02.983328+0000","last_became_peered":"2024-01-15T18:16:02.983328+0000","last_unstale":"2024-01-15T18:16:04.678757+0000","last_undegraded":"2024-01-15T18:16:04.678757+0000","last_fullsized":"2024-01-15T18:16:04.678757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:50.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678672+0000","last_change":"2024-01-15T18:16:02.985998+0000","last_active":"2024-01-15T18:16:04.678672+0000","last_peered":"2024-01-15T18:16:04.678672+0000","last_clean":"2024-01-15T18:16:04.678672+0000","last_became_active":"2024-01-15T18:16:02.984389+0000","last_became_peered":"2024-01-15T18:16:02.984389+0000","last_unstale":"2024-01-15T18:16:04.678672+0000","last_undegraded":"2024-01-15T18:16:04.678672+0000","last_fullsized":"2024-01-15T18:16:04.678672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:48.886152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982746+0000","last_change":"2024-01-15T18:16:02.974470+0000","last_active":"2024-01-15T18:16:03.982746+0000","last_peered":"2024-01-15T18:16:03.982746+0000","last_clean":"2024-01-15T18:16:03.982746+0000","last_became_active":"2024-01-15T18:16:02.974162+0000","last_became_peered":"2024-01-15T18:16:02.974162+0000","last_unstale":"2024-01-15T18:16:03.982746+0000","last_undegraded":"2024-01-15T18:16:03.982746+0000","last_fullsized":"2024-01-15T18:16:03.982746+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172002+0000","last_change":"2024-01-15T18:16:02.986920+0000","last_active":"2024-01-15T18:16:04.172002+0000","last_peered":"2024-01-15T18:16:04.172002+0000","last_clean":"2024-01-15T18:16:04.172002+0000","last_became_active":"2024-01-15T18:16:02.986544+0000","last_became_peered":"2024-01-15T18:16:02.986544+0000","last_unstale":"2024-01-15T18:16:04.172002+0000","last_undegraded":"2024-01-15T18:16:04.172002+0000","last_fullsized":"2024-01-15T18:16:04.172002+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:06.358800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983627+0000","last_change":"2024-01-15T18:16:02.981015+0000","last_active":"2024-01-15T18:16:03.983627+0000","last_peered":"2024-01-15T18:16:03.983627+0000","last_clean":"2024-01-15T18:16:03.983627+0000","last_became_active":"2024-01-15T18:16:02.979886+0000","last_became_peered":"2024-01-15T18:16:02.979886+0000","last_unstale":"2024-01-15T18:16:03.983627+0000","last_undegraded":"2024-01-15T18:16:03.983627+0000","last_fullsized":"2024-01-15T18:16:03.983627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:23.067211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004361+0000","last_change":"2024-01-15T18:16:01.968674+0000","last_active":"2024-01-15T18:16:03.004361+0000","last_peered":"2024-01-15T18:16:03.004361+0000","last_clean":"2024-01-15T18:16:03.004361+0000","last_became_active":"2024-01-15T18:16:01.967874+0000","last_became_peered":"2024-01-15T18:16:01.967874+0000","last_unstale":"2024-01-15T18:16:03.004361+0000","last_undegraded":"2024-01-15T18:16:03.004361+0000","last_fullsized":"2024-01-15T18:16:03.004361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172367+0000","last_change":"2024-01-15T18:16:02.971695+0000","last_active":"2024-01-15T18:16:04.172367+0000","last_peered":"2024-01-15T18:16:04.172367+0000","last_clean":"2024-01-15T18:16:04.172367+0000","last_became_active":"2024-01-15T18:16:02.971425+0000","last_became_peered":"2024-01-15T18:16:02.971425+0000","last_unstale":"2024-01-15T18:16:04.172367+0000","last_undegraded":"2024-01-15T18:16:04.172367+0000","last_fullsized":"2024-01-15T18:16:04.172367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:14.360727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002366+0000","last_change":"2024-01-15T18:16:01.960880+0000","last_active":"2024-01-15T18:16:03.002366+0000","last_peered":"2024-01-15T18:16:03.002366+0000","last_clean":"2024-01-15T18:16:03.002366+0000","last_became_active":"2024-01-15T18:16:01.960424+0000","last_became_peered":"2024-01-15T18:16:01.960424+0000","last_unstale":"2024-01-15T18:16:03.002366+0000","last_undegraded":"2024-01-15T18:16:03.002366+0000","last_fullsized":"2024-01-15T18:16:03.002366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:05.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172293+0000","last_change":"2024-01-15T18:16:02.987087+0000","last_active":"2024-01-15T18:16:04.172293+0000","last_peered":"2024-01-15T18:16:04.172293+0000","last_clean":"2024-01-15T18:16:04.172293+0000","last_became_active":"2024-01-15T18:16:02.986595+0000","last_became_peered":"2024-01-15T18:16:02.986595+0000","last_unstale":"2024-01-15T18:16:04.172293+0000","last_undegraded":"2024-01-15T18:16:04.172293+0000","last_fullsized":"2024-01-15T18:16:04.172293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.006815+0000","last_change":"2024-01-15T18:16:01.972650+0000","last_active":"2024-01-15T18:16:03.006815+0000","last_peered":"2024-01-15T18:16:03.006815+0000","last_clean":"2024-01-15T18:16:03.006815+0000","last_became_active":"2024-01-15T18:16:01.972282+0000","last_became_peered":"2024-01-15T18:16:01.972282+0000","last_unstale":"2024-01-15T18:16:03.006815+0000","last_undegraded":"2024-01-15T18:16:03.006815+0000","last_fullsized":"2024-01-15T18:16:03.006815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:05.387736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984011+0000","last_change":"2024-01-15T18:16:02.980841+0000","last_active":"2024-01-15T18:16:03.984011+0000","last_peered":"2024-01-15T18:16:03.984011+0000","last_clean":"2024-01-15T18:16:03.984011+0000","last_became_active":"2024-01-15T18:16:02.980529+0000","last_became_peered":"2024-01-15T18:16:02.980529+0000","last_unstale":"2024-01-15T18:16:03.984011+0000","last_undegraded":"2024-01-15T18:16:03.984011+0000","last_fullsized":"2024-01-15T18:16:03.984011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:16.698507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.008874+0000","last_change":"2024-01-15T18:16:01.974052+0000","last_active":"2024-01-15T18:16:03.008874+0000","last_peered":"2024-01-15T18:16:03.008874+0000","last_clean":"2024-01-15T18:16:03.008874+0000","last_became_active":"2024-01-15T18:16:01.973571+0000","last_became_peered":"2024-01-15T18:16:01.973571+0000","last_unstale":"2024-01-15T18:16:03.008874+0000","last_undegraded":"2024-01-15T18:16:03.008874+0000","last_fullsized":"2024-01-15T18:16:03.008874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:03.224372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678800+0000","last_change":"2024-01-15T18:16:02.986208+0000","last_active":"2024-01-15T18:16:04.678800+0000","last_peered":"2024-01-15T18:16:04.678800+0000","last_clean":"2024-01-15T18:16:04.678800+0000","last_became_active":"2024-01-15T18:16:02.984734+0000","last_became_peered":"2024-01-15T18:16:02.984734+0000","last_unstale":"2024-01-15T18:16:04.678800+0000","last_undegraded":"2024-01-15T18:16:04.678800+0000","last_fullsized":"2024-01-15T18:16:04.678800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:07.488202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002241+0000","last_change":"2024-01-15T18:16:01.961667+0000","last_active":"2024-01-15T18:16:03.002241+0000","last_peered":"2024-01-15T18:16:03.002241+0000","last_clean":"2024-01-15T18:16:03.002241+0000","last_became_active":"2024-01-15T18:16:01.961336+0000","last_became_peered":"2024-01-15T18:16:01.961336+0000","last_unstale":"2024-01-15T18:16:03.002241+0000","last_undegraded":"2024-01-15T18:16:03.002241+0000","last_fullsized":"2024-01-15T18:16:03.002241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:18.718744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.982899+0000","last_change":"2024-01-15T18:16:02.970823+0000","last_active":"2024-01-15T18:16:03.982899+0000","last_peered":"2024-01-15T18:16:03.982899+0000","last_clean":"2024-01-15T18:16:03.982899+0000","last_became_active":"2024-01-15T18:16:02.970523+0000","last_became_peered":"2024-01-15T18:16:02.970523+0000","last_unstale":"2024-01-15T18:16:03.982899+0000","last_undegraded":"2024-01-15T18:16:03.982899+0000","last_fullsized":"2024-01-15T18:16:03.982899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000246+0000","last_change":"2024-01-15T18:16:01.958649+0000","last_active":"2024-01-15T18:16:03.000246+0000","last_peered":"2024-01-15T18:16:03.000246+0000","last_clean":"2024-01-15T18:16:03.000246+0000","last_became_active":"2024-01-15T18:16:01.958043+0000","last_became_peered":"2024-01-15T18:16:01.958043+0000","last_unstale":"2024-01-15T18:16:03.000246+0000","last_undegraded":"2024-01-15T18:16:03.000246+0000","last_fullsized":"2024-01-15T18:16:03.000246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:48:10.293667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983181+0000","last_change":"2024-01-15T18:16:02.983728+0000","last_active":"2024-01-15T18:16:03.983181+0000","last_peered":"2024-01-15T18:16:03.983181+0000","last_clean":"2024-01-15T18:16:03.983181+0000","last_became_active":"2024-01-15T18:16:02.981960+0000","last_became_peered":"2024-01-15T18:16:02.981960+0000","last_unstale":"2024-01-15T18:16:03.983181+0000","last_undegraded":"2024-01-15T18:16:03.983181+0000","last_fullsized":"2024-01-15T18:16:03.983181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971999+0000","last_change":"2024-01-15T18:16:01.968471+0000","last_active":"2024-01-15T18:16:02.971999+0000","last_peered":"2024-01-15T18:16:02.971999+0000","last_clean":"2024-01-15T18:16:02.971999+0000","last_became_active":"2024-01-15T18:16:01.967508+0000","last_became_peered":"2024-01-15T18:16:01.967508+0000","last_unstale":"2024-01-15T18:16:02.971999+0000","last_undegraded":"2024-01-15T18:16:02.971999+0000","last_fullsized":"2024-01-15T18:16:02.971999+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:05.350436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.987141+0000","last_change":"2024-01-15T18:16:02.980575+0000","last_active":"2024-01-15T18:16:03.987141+0000","last_peered":"2024-01-15T18:16:03.987141+0000","last_clean":"2024-01-15T18:16:03.987141+0000","last_became_active":"2024-01-15T18:16:02.979705+0000","last_became_peered":"2024-01-15T18:16:02.979705+0000","last_unstale":"2024-01-15T18:16:03.987141+0000","last_undegraded":"2024-01-15T18:16:03.987141+0000","last_fullsized":"2024-01-15T18:16:03.987141+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.970364+0000","last_change":"2024-01-15T18:16:01.971878+0000","last_active":"2024-01-15T18:16:02.970364+0000","last_peered":"2024-01-15T18:16:02.970364+0000","last_clean":"2024-01-15T18:16:02.970364+0000","last_became_active":"2024-01-15T18:16:01.971510+0000","last_became_peered":"2024-01-15T18:16:01.971510+0000","last_unstale":"2024-01-15T18:16:02.970364+0000","last_undegraded":"2024-01-15T18:16:02.970364+0000","last_fullsized":"2024-01-15T18:16:02.970364+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:34.428163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.987469+0000","last_change":"2024-01-15T18:16:02.994947+0000","last_active":"2024-01-15T18:16:03.987469+0000","last_peered":"2024-01-15T18:16:03.987469+0000","last_clean":"2024-01-15T18:16:03.987469+0000","last_became_active":"2024-01-15T18:16:02.994189+0000","last_became_peered":"2024-01-15T18:16:02.994189+0000","last_unstale":"2024-01-15T18:16:03.987469+0000","last_undegraded":"2024-01-15T18:16:03.987469+0000","last_fullsized":"2024-01-15T18:16:03.987469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:34.328758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.967235+0000","last_change":"2024-01-15T18:16:01.967948+0000","last_active":"2024-01-15T18:16:02.967235+0000","last_peered":"2024-01-15T18:16:02.967235+0000","last_clean":"2024-01-15T18:16:02.967235+0000","last_became_active":"2024-01-15T18:16:01.967662+0000","last_became_peered":"2024-01-15T18:16:01.967662+0000","last_unstale":"2024-01-15T18:16:02.967235+0000","last_undegraded":"2024-01-15T18:16:02.967235+0000","last_fullsized":"2024-01-15T18:16:02.967235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:28.859462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672459+0000","last_change":"2024-01-15T18:16:02.977816+0000","last_active":"2024-01-15T18:16:04.672459+0000","last_peered":"2024-01-15T18:16:04.672459+0000","last_clean":"2024-01-15T18:16:04.672459+0000","last_became_active":"2024-01-15T18:16:02.977488+0000","last_became_peered":"2024-01-15T18:16:02.977488+0000","last_unstale":"2024-01-15T18:16:04.672459+0000","last_undegraded":"2024-01-15T18:16:04.672459+0000","last_fullsized":"2024-01-15T18:16:04.672459+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.966903+0000","last_change":"2024-01-15T18:16:01.970089+0000","last_active":"2024-01-15T18:16:02.966903+0000","last_peered":"2024-01-15T18:16:02.966903+0000","last_clean":"2024-01-15T18:16:02.966903+0000","last_became_active":"2024-01-15T18:16:01.969791+0000","last_became_peered":"2024-01-15T18:16:01.969791+0000","last_unstale":"2024-01-15T18:16:02.966903+0000","last_undegraded":"2024-01-15T18:16:02.966903+0000","last_fullsized":"2024-01-15T18:16:02.966903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:25.108362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.383856+0000","last_change":"2024-01-15T18:16:02.982703+0000","last_active":"2024-01-15T18:16:04.383856+0000","last_peered":"2024-01-15T18:16:04.383856+0000","last_clean":"2024-01-15T18:16:04.383856+0000","last_became_active":"2024-01-15T18:16:02.982469+0000","last_became_peered":"2024-01-15T18:16:02.982469+0000","last_unstale":"2024-01-15T18:16:04.383856+0000","last_undegraded":"2024-01-15T18:16:04.383856+0000","last_fullsized":"2024-01-15T18:16:04.383856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:05:43.464086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.002343+0000","last_change":"2024-01-15T18:16:01.967367+0000","last_active":"2024-01-15T18:16:03.002343+0000","last_peered":"2024-01-15T18:16:03.002343+0000","last_clean":"2024-01-15T18:16:03.002343+0000","last_became_active":"2024-01-15T18:16:01.966610+0000","last_became_peered":"2024-01-15T18:16:01.966610+0000","last_unstale":"2024-01-15T18:16:03.002343+0000","last_undegraded":"2024-01-15T18:16:03.002343+0000","last_fullsized":"2024-01-15T18:16:03.002343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:41:22.596382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983502+0000","last_change":"2024-01-15T18:16:02.976324+0000","last_active":"2024-01-15T18:16:03.983502+0000","last_peered":"2024-01-15T18:16:03.983502+0000","last_clean":"2024-01-15T18:16:03.983502+0000","last_became_active":"2024-01-15T18:16:02.975452+0000","last_became_peered":"2024-01-15T18:16:02.975452+0000","last_unstale":"2024-01-15T18:16:03.983502+0000","last_undegraded":"2024-01-15T18:16:03.983502+0000","last_fullsized":"2024-01-15T18:16:03.983502+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:31.636033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.004546+0000","last_change":"2024-01-15T18:16:01.971530+0000","last_active":"2024-01-15T18:16:03.004546+0000","last_peered":"2024-01-15T18:16:03.004546+0000","last_clean":"2024-01-15T18:16:03.004546+0000","last_became_active":"2024-01-15T18:16:01.971080+0000","last_became_peered":"2024-01-15T18:16:01.971080+0000","last_unstale":"2024-01-15T18:16:03.004546+0000","last_undegraded":"2024-01-15T18:16:03.004546+0000","last_fullsized":"2024-01-15T18:16:03.004546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.383731+0000","last_change":"2024-01-15T18:16:02.987392+0000","last_active":"2024-01-15T18:16:04.383731+0000","last_peered":"2024-01-15T18:16:04.383731+0000","last_clean":"2024-01-15T18:16:04.383731+0000","last_became_active":"2024-01-15T18:16:02.987137+0000","last_became_peered":"2024-01-15T18:16:02.987137+0000","last_unstale":"2024-01-15T18:16:04.383731+0000","last_undegraded":"2024-01-15T18:16:04.383731+0000","last_fullsized":"2024-01-15T18:16:04.383731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.007728+0000","last_change":"2024-01-15T18:16:01.968495+0000","last_active":"2024-01-15T18:16:03.007728+0000","last_peered":"2024-01-15T18:16:03.007728+0000","last_clean":"2024-01-15T18:16:03.007728+0000","last_became_active":"2024-01-15T18:16:01.967100+0000","last_became_peered":"2024-01-15T18:16:01.967100+0000","last_unstale":"2024-01-15T18:16:03.007728+0000","last_undegraded":"2024-01-15T18:16:03.007728+0000","last_fullsized":"2024-01-15T18:16:03.007728+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:47.283546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983817+0000","last_change":"2024-01-15T18:16:02.985570+0000","last_active":"2024-01-15T18:16:03.983817+0000","last_peered":"2024-01-15T18:16:03.983817+0000","last_clean":"2024-01-15T18:16:03.983817+0000","last_became_active":"2024-01-15T18:16:02.985303+0000","last_became_peered":"2024-01-15T18:16:02.985303+0000","last_unstale":"2024-01-15T18:16:03.983817+0000","last_undegraded":"2024-01-15T18:16:03.983817+0000","last_fullsized":"2024-01-15T18:16:03.983817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:55.037442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.978629+0000","last_change":"2024-01-15T18:16:01.965271+0000","last_active":"2024-01-15T18:16:02.978629+0000","last_peered":"2024-01-15T18:16:02.978629+0000","last_clean":"2024-01-15T18:16:02.978629+0000","last_became_active":"2024-01-15T18:16:01.964982+0000","last_became_peered":"2024-01-15T18:16:01.964982+0000","last_unstale":"2024-01-15T18:16:02.978629+0000","last_undegraded":"2024-01-15T18:16:02.978629+0000","last_fullsized":"2024-01-15T18:16:02.978629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:47.872157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678642+0000","last_change":"2024-01-15T18:16:02.984811+0000","last_active":"2024-01-15T18:16:04.678642+0000","last_peered":"2024-01-15T18:16:04.678642+0000","last_clean":"2024-01-15T18:16:04.678642+0000","last_became_active":"2024-01-15T18:16:02.984555+0000","last_became_peered":"2024-01-15T18:16:02.984555+0000","last_unstale":"2024-01-15T18:16:04.678642+0000","last_undegraded":"2024-01-15T18:16:04.678642+0000","last_fullsized":"2024-01-15T18:16:04.678642+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:49.543082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971256+0000","last_change":"2024-01-15T18:16:01.971126+0000","last_active":"2024-01-15T18:16:02.971256+0000","last_peered":"2024-01-15T18:16:02.971256+0000","last_clean":"2024-01-15T18:16:02.971256+0000","last_became_active":"2024-01-15T18:16:01.969238+0000","last_became_peered":"2024-01-15T18:16:01.969238+0000","last_unstale":"2024-01-15T18:16:02.971256+0000","last_undegraded":"2024-01-15T18:16:02.971256+0000","last_fullsized":"2024-01-15T18:16:02.971256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:30.543482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678404+0000","last_change":"2024-01-15T18:16:02.985951+0000","last_active":"2024-01-15T18:16:04.678404+0000","last_peered":"2024-01-15T18:16:04.678404+0000","last_clean":"2024-01-15T18:16:04.678404+0000","last_became_active":"2024-01-15T18:16:02.985648+0000","last_became_peered":"2024-01-15T18:16:02.985648+0000","last_unstale":"2024-01-15T18:16:04.678404+0000","last_undegraded":"2024-01-15T18:16:04.678404+0000","last_fullsized":"2024-01-15T18:16:04.678404+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:27:45.097215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.008984+0000","last_change":"2024-01-15T18:16:01.979247+0000","last_active":"2024-01-15T18:16:03.008984+0000","last_peered":"2024-01-15T18:16:03.008984+0000","last_clean":"2024-01-15T18:16:03.008984+0000","last_became_active":"2024-01-15T18:16:01.978885+0000","last_became_peered":"2024-01-15T18:16:01.978885+0000","last_unstale":"2024-01-15T18:16:03.008984+0000","last_undegraded":"2024-01-15T18:16:03.008984+0000","last_fullsized":"2024-01-15T18:16:03.008984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984081+0000","last_change":"2024-01-15T18:16:02.983891+0000","last_active":"2024-01-15T18:16:03.984081+0000","last_peered":"2024-01-15T18:16:03.984081+0000","last_clean":"2024-01-15T18:16:03.984081+0000","last_became_active":"2024-01-15T18:16:02.983591+0000","last_became_peered":"2024-01-15T18:16:02.983591+0000","last_unstale":"2024-01-15T18:16:03.984081+0000","last_undegraded":"2024-01-15T18:16:03.984081+0000","last_fullsized":"2024-01-15T18:16:03.984081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:32.572896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.998942+0000","last_change":"2024-01-15T18:16:01.971805+0000","last_active":"2024-01-15T18:16:02.998942+0000","last_peered":"2024-01-15T18:16:02.998942+0000","last_clean":"2024-01-15T18:16:02.998942+0000","last_became_active":"2024-01-15T18:16:01.971492+0000","last_became_peered":"2024-01-15T18:16:01.971492+0000","last_unstale":"2024-01-15T18:16:02.998942+0000","last_undegraded":"2024-01-15T18:16:02.998942+0000","last_fullsized":"2024-01-15T18:16:02.998942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:11.456590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172473+0000","last_change":"2024-01-15T18:16:02.974741+0000","last_active":"2024-01-15T18:16:04.172473+0000","last_peered":"2024-01-15T18:16:04.172473+0000","last_clean":"2024-01-15T18:16:04.172473+0000","last_became_active":"2024-01-15T18:16:02.974455+0000","last_became_peered":"2024-01-15T18:16:02.974455+0000","last_unstale":"2024-01-15T18:16:04.172473+0000","last_undegraded":"2024-01-15T18:16:04.172473+0000","last_fullsized":"2024-01-15T18:16:04.172473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.003442+0000","last_change":"2024-01-15T18:16:01.970998+0000","last_active":"2024-01-15T18:16:03.003442+0000","last_peered":"2024-01-15T18:16:03.003442+0000","last_clean":"2024-01-15T18:16:03.003442+0000","last_became_active":"2024-01-15T18:16:01.970730+0000","last_became_peered":"2024-01-15T18:16:01.970730+0000","last_unstale":"2024-01-15T18:16:03.003442+0000","last_undegraded":"2024-01-15T18:16:03.003442+0000","last_fullsized":"2024-01-15T18:16:03.003442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:01:30.543482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983546+0000","last_change":"2024-01-15T18:16:02.989297+0000","last_active":"2024-01-15T18:16:03.983546+0000","last_peered":"2024-01-15T18:16:03.983546+0000","last_clean":"2024-01-15T18:16:03.983546+0000","last_became_active":"2024-01-15T18:16:02.988832+0000","last_became_peered":"2024-01-15T18:16:02.988832+0000","last_unstale":"2024-01-15T18:16:03.983546+0000","last_undegraded":"2024-01-15T18:16:03.983546+0000","last_fullsized":"2024-01-15T18:16:03.983546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:07.504686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.973281+0000","last_change":"2024-01-15T18:16:01.975857+0000","last_active":"2024-01-15T18:16:02.973281+0000","last_peered":"2024-01-15T18:16:02.973281+0000","last_clean":"2024-01-15T18:16:02.973281+0000","last_became_active":"2024-01-15T18:16:01.975585+0000","last_became_peered":"2024-01-15T18:16:01.975585+0000","last_unstale":"2024-01-15T18:16:02.973281+0000","last_undegraded":"2024-01-15T18:16:02.973281+0000","last_fullsized":"2024-01-15T18:16:02.973281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:47.872157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549945+0000","last_change":"2024-01-15T18:16:02.990538+0000","last_active":"2024-01-15T18:16:04.549945+0000","last_peered":"2024-01-15T18:16:04.549945+0000","last_clean":"2024-01-15T18:16:04.549945+0000","last_became_active":"2024-01-15T18:16:02.989578+0000","last_became_peered":"2024-01-15T18:16:02.989578+0000","last_unstale":"2024-01-15T18:16:04.549945+0000","last_undegraded":"2024-01-15T18:16:04.549945+0000","last_fullsized":"2024-01-15T18:16:04.549945+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:00.084696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.009890+0000","last_change":"2024-01-15T18:16:01.972470+0000","last_active":"2024-01-15T18:16:03.009890+0000","last_peered":"2024-01-15T18:16:03.009890+0000","last_clean":"2024-01-15T18:16:03.009890+0000","last_became_active":"2024-01-15T18:16:01.972202+0000","last_became_peered":"2024-01-15T18:16:01.972202+0000","last_unstale":"2024-01-15T18:16:03.009890+0000","last_undegraded":"2024-01-15T18:16:03.009890+0000","last_fullsized":"2024-01-15T18:16:03.009890+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:13.352362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672268+0000","last_change":"2024-01-15T18:16:02.984567+0000","last_active":"2024-01-15T18:16:04.672268+0000","last_peered":"2024-01-15T18:16:04.672268+0000","last_clean":"2024-01-15T18:16:04.672268+0000","last_became_active":"2024-01-15T18:16:02.984223+0000","last_became_peered":"2024-01-15T18:16:02.984223+0000","last_unstale":"2024-01-15T18:16:04.672268+0000","last_undegraded":"2024-01-15T18:16:04.672268+0000","last_fullsized":"2024-01-15T18:16:04.672268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:48.880521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.965895+0000","last_change":"2024-01-15T18:16:01.973466+0000","last_active":"2024-01-15T18:16:02.965895+0000","last_peered":"2024-01-15T18:16:02.965895+0000","last_clean":"2024-01-15T18:16:02.965895+0000","last_became_active":"2024-01-15T18:16:01.973199+0000","last_became_peered":"2024-01-15T18:16:01.973199+0000","last_unstale":"2024-01-15T18:16:02.965895+0000","last_undegraded":"2024-01-15T18:16:02.965895+0000","last_fullsized":"2024-01-15T18:16:02.965895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:10:29.198637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.549638+0000","last_change":"2024-01-15T18:16:02.988743+0000","last_active":"2024-01-15T18:16:04.549638+0000","last_peered":"2024-01-15T18:16:04.549638+0000","last_clean":"2024-01-15T18:16:04.549638+0000","last_became_active":"2024-01-15T18:16:02.988460+0000","last_became_peered":"2024-01-15T18:16:02.988460+0000","last_unstale":"2024-01-15T18:16:04.549638+0000","last_undegraded":"2024-01-15T18:16:04.549638+0000","last_fullsized":"2024-01-15T18:16:04.549638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:53:54.122251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.965369+0000","last_change":"2024-01-15T18:16:01.976107+0000","last_active":"2024-01-15T18:16:02.965369+0000","last_peered":"2024-01-15T18:16:02.965369+0000","last_clean":"2024-01-15T18:16:02.965369+0000","last_became_active":"2024-01-15T18:16:01.975849+0000","last_became_peered":"2024-01-15T18:16:01.975849+0000","last_unstale":"2024-01-15T18:16:02.965369+0000","last_undegraded":"2024-01-15T18:16:02.965369+0000","last_fullsized":"2024-01-15T18:16:02.965369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:01:27.669007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672166+0000","last_change":"2024-01-15T18:16:02.984528+0000","last_active":"2024-01-15T18:16:04.672166+0000","last_peered":"2024-01-15T18:16:04.672166+0000","last_clean":"2024-01-15T18:16:04.672166+0000","last_became_active":"2024-01-15T18:16:02.984179+0000","last_became_peered":"2024-01-15T18:16:02.984179+0000","last_unstale":"2024-01-15T18:16:04.672166+0000","last_undegraded":"2024-01-15T18:16:04.672166+0000","last_fullsized":"2024-01-15T18:16:04.672166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:00.439825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.976027+0000","last_change":"2024-01-15T18:16:01.974336+0000","last_active":"2024-01-15T18:16:02.976027+0000","last_peered":"2024-01-15T18:16:02.976027+0000","last_clean":"2024-01-15T18:16:02.976027+0000","last_became_active":"2024-01-15T18:16:01.974035+0000","last_became_peered":"2024-01-15T18:16:01.974035+0000","last_unstale":"2024-01-15T18:16:02.976027+0000","last_undegraded":"2024-01-15T18:16:02.976027+0000","last_fullsized":"2024-01-15T18:16:02.976027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:03.224372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983695+0000","last_change":"2024-01-15T18:16:02.981820+0000","last_active":"2024-01-15T18:16:03.983695+0000","last_peered":"2024-01-15T18:16:03.983695+0000","last_clean":"2024-01-15T18:16:03.983695+0000","last_became_active":"2024-01-15T18:16:02.981478+0000","last_became_peered":"2024-01-15T18:16:02.981478+0000","last_unstale":"2024-01-15T18:16:03.983695+0000","last_undegraded":"2024-01-15T18:16:03.983695+0000","last_fullsized":"2024-01-15T18:16:03.983695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:21.866006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.007823+0000","last_change":"2024-01-15T18:16:01.976896+0000","last_active":"2024-01-15T18:16:03.007823+0000","last_peered":"2024-01-15T18:16:03.007823+0000","last_clean":"2024-01-15T18:16:03.007823+0000","last_became_active":"2024-01-15T18:16:01.976494+0000","last_became_peered":"2024-01-15T18:16:01.976494+0000","last_unstale":"2024-01-15T18:16:03.007823+0000","last_undegraded":"2024-01-15T18:16:03.007823+0000","last_fullsized":"2024-01-15T18:16:03.007823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:43.419719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.984005+0000","last_change":"2024-01-15T18:16:02.981174+0000","last_active":"2024-01-15T18:16:03.984005+0000","last_peered":"2024-01-15T18:16:03.984005+0000","last_clean":"2024-01-15T18:16:03.984005+0000","last_became_active":"2024-01-15T18:16:02.980913+0000","last_became_peered":"2024-01-15T18:16:02.980913+0000","last_unstale":"2024-01-15T18:16:03.984005+0000","last_undegraded":"2024-01-15T18:16:03.984005+0000","last_fullsized":"2024-01-15T18:16:03.984005+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:12.319403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.970802+0000","last_change":"2024-01-15T18:16:01.967331+0000","last_active":"2024-01-15T18:16:02.970802+0000","last_peered":"2024-01-15T18:16:02.970802+0000","last_clean":"2024-01-15T18:16:02.970802+0000","last_became_active":"2024-01-15T18:16:01.967031+0000","last_became_peered":"2024-01-15T18:16:01.967031+0000","last_unstale":"2024-01-15T18:16:02.970802+0000","last_undegraded":"2024-01-15T18:16:02.970802+0000","last_fullsized":"2024-01-15T18:16:02.970802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:39.267191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172397+0000","last_change":"2024-01-15T18:16:02.969429+0000","last_active":"2024-01-15T18:16:04.172397+0000","last_peered":"2024-01-15T18:16:04.172397+0000","last_clean":"2024-01-15T18:16:04.172397+0000","last_became_active":"2024-01-15T18:16:02.969095+0000","last_became_peered":"2024-01-15T18:16:02.969095+0000","last_unstale":"2024-01-15T18:16:04.172397+0000","last_undegraded":"2024-01-15T18:16:04.172397+0000","last_fullsized":"2024-01-15T18:16:04.172397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:47.386289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964906+0000","last_change":"2024-01-15T18:16:01.968688+0000","last_active":"2024-01-15T18:16:02.964906+0000","last_peered":"2024-01-15T18:16:02.964906+0000","last_clean":"2024-01-15T18:16:02.964906+0000","last_became_active":"2024-01-15T18:16:01.968389+0000","last_became_peered":"2024-01-15T18:16:01.968389+0000","last_unstale":"2024-01-15T18:16:02.964906+0000","last_undegraded":"2024-01-15T18:16:02.964906+0000","last_fullsized":"2024-01-15T18:16:02.964906+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.148479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983643+0000","last_change":"2024-01-15T18:16:02.976712+0000","last_active":"2024-01-15T18:16:03.983643+0000","last_peered":"2024-01-15T18:16:03.983643+0000","last_clean":"2024-01-15T18:16:03.983643+0000","last_became_active":"2024-01-15T18:16:02.976054+0000","last_became_peered":"2024-01-15T18:16:02.976054+0000","last_unstale":"2024-01-15T18:16:03.983643+0000","last_undegraded":"2024-01-15T18:16:03.983643+0000","last_fullsized":"2024-01-15T18:16:03.983643+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:26:19.031570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964799+0000","last_change":"2024-01-15T18:16:01.966697+0000","last_active":"2024-01-15T18:16:02.964799+0000","last_peered":"2024-01-15T18:16:02.964799+0000","last_clean":"2024-01-15T18:16:02.964799+0000","last_became_active":"2024-01-15T18:16:01.966397+0000","last_became_peered":"2024-01-15T18:16:01.966397+0000","last_unstale":"2024-01-15T18:16:02.964799+0000","last_undegraded":"2024-01-15T18:16:02.964799+0000","last_fullsized":"2024-01-15T18:16:02.964799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.672238+0000","last_change":"2024-01-15T18:16:02.984048+0000","last_active":"2024-01-15T18:16:04.672238+0000","last_peered":"2024-01-15T18:16:04.672238+0000","last_clean":"2024-01-15T18:16:04.672238+0000","last_became_active":"2024-01-15T18:16:02.983546+0000","last_became_peered":"2024-01-15T18:16:02.983546+0000","last_unstale":"2024-01-15T18:16:04.672238+0000","last_undegraded":"2024-01-15T18:16:04.672238+0000","last_fullsized":"2024-01-15T18:16:04.672238+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:52:04.232736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.973089+0000","last_change":"2024-01-15T18:16:01.979160+0000","last_active":"2024-01-15T18:16:02.973089+0000","last_peered":"2024-01-15T18:16:02.973089+0000","last_clean":"2024-01-15T18:16:02.973089+0000","last_became_active":"2024-01-15T18:16:01.978901+0000","last_became_peered":"2024-01-15T18:16:01.978901+0000","last_unstale":"2024-01-15T18:16:02.973089+0000","last_undegraded":"2024-01-15T18:16:02.973089+0000","last_fullsized":"2024-01-15T18:16:02.973089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:59.431461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.383863+0000","last_change":"2024-01-15T18:16:02.987865+0000","last_active":"2024-01-15T18:16:04.383863+0000","last_peered":"2024-01-15T18:16:04.383863+0000","last_clean":"2024-01-15T18:16:04.383863+0000","last_became_active":"2024-01-15T18:16:02.987466+0000","last_became_peered":"2024-01-15T18:16:02.987466+0000","last_unstale":"2024-01-15T18:16:04.383863+0000","last_undegraded":"2024-01-15T18:16:04.383863+0000","last_fullsized":"2024-01-15T18:16:04.383863+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:42:01.960146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.961378+0000","last_change":"2024-01-15T18:16:01.958840+0000","last_active":"2024-01-15T18:16:02.961378+0000","last_peered":"2024-01-15T18:16:02.961378+0000","last_clean":"2024-01-15T18:16:02.961378+0000","last_became_active":"2024-01-15T18:16:01.958367+0000","last_became_peered":"2024-01-15T18:16:01.958367+0000","last_unstale":"2024-01-15T18:16:02.961378+0000","last_undegraded":"2024-01-15T18:16:02.961378+0000","last_fullsized":"2024-01-15T18:16:02.961378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:26.540155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.983304+0000","last_change":"2024-01-15T18:16:02.991930+0000","last_active":"2024-01-15T18:16:03.983304+0000","last_peered":"2024-01-15T18:16:03.983304+0000","last_clean":"2024-01-15T18:16:03.983304+0000","last_became_active":"2024-01-15T18:16:02.991630+0000","last_became_peered":"2024-01-15T18:16:02.991630+0000","last_unstale":"2024-01-15T18:16:03.983304+0000","last_undegraded":"2024-01-15T18:16:03.983304+0000","last_fullsized":"2024-01-15T18:16:03.983304+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:22.822515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.964174+0000","last_change":"2024-01-15T18:16:01.958054+0000","last_active":"2024-01-15T18:16:02.964174+0000","last_peered":"2024-01-15T18:16:02.964174+0000","last_clean":"2024-01-15T18:16:02.964174+0000","last_became_active":"2024-01-15T18:16:01.957368+0000","last_became_peered":"2024-01-15T18:16:01.957368+0000","last_unstale":"2024-01-15T18:16:02.964174+0000","last_undegraded":"2024-01-15T18:16:02.964174+0000","last_fullsized":"2024-01-15T18:16:02.964174+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:50.037183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.172054+0000","last_change":"2024-01-15T18:16:02.970069+0000","last_active":"2024-01-15T18:16:04.172054+0000","last_peered":"2024-01-15T18:16:04.172054+0000","last_clean":"2024-01-15T18:16:04.172054+0000","last_became_active":"2024-01-15T18:16:02.969737+0000","last_became_peered":"2024-01-15T18:16:02.969737+0000","last_unstale":"2024-01-15T18:16:04.172054+0000","last_undegraded":"2024-01-15T18:16:04.172054+0000","last_fullsized":"2024-01-15T18:16:04.172054+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:06.396100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.963102+0000","last_change":"2024-01-15T18:16:01.968677+0000","last_active":"2024-01-15T18:16:02.963102+0000","last_peered":"2024-01-15T18:16:02.963102+0000","last_clean":"2024-01-15T18:16:02.963102+0000","last_became_active":"2024-01-15T18:16:01.967462+0000","last_became_peered":"2024-01-15T18:16:01.967462+0000","last_unstale":"2024-01-15T18:16:02.963102+0000","last_undegraded":"2024-01-15T18:16:02.963102+0000","last_fullsized":"2024-01-15T18:16:02.963102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:10:27.000475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:16:04.678834+0000","last_change":"2024-01-15T18:16:02.988020+0000","last_active":"2024-01-15T18:16:04.678834+0000","last_peered":"2024-01-15T18:16:04.678834+0000","last_clean":"2024-01-15T18:16:04.678834+0000","last_became_active":"2024-01-15T18:16:02.987772+0000","last_became_peered":"2024-01-15T18:16:02.987772+0000","last_unstale":"2024-01-15T18:16:04.678834+0000","last_undegraded":"2024-01-15T18:16:04.678834+0000","last_fullsized":"2024-01-15T18:16:04.678834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:01.943662+0000","last_clean_scrub_stamp":"2024-01-15T18:16:01.943662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:50.682116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:03.000937+0000","last_change":"2024-01-15T18:16:01.973544+0000","last_active":"2024-01-15T18:16:03.000937+0000","last_peered":"2024-01-15T18:16:03.000937+0000","last_clean":"2024-01-15T18:16:03.000937+0000","last_became_active":"2024-01-15T18:16:01.973240+0000","last_became_peered":"2024-01-15T18:16:01.973240+0000","last_unstale":"2024-01-15T18:16:03.000937+0000","last_undegraded":"2024-01-15T18:16:03.000937+0000","last_fullsized":"2024-01-15T18:16:03.000937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:16:00.935298+0000","last_clean_scrub_stamp":"2024-01-15T18:16:00.935298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:06.496321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:16:02.971217+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:16:02.971217+0000","last_peered":"2024-01-15T18:16:02.971217+0000","last_clean":"2024-01-15T18:16:02.971217+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:16:02.971217+0000","last_undegraded":"2024-01-15T18:16:02.971217+0000","last_fullsized":"2024-01-15T18:16:02.971217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77164,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294676,"statfs":{"total":96636764160,"available":96557748224,"internally_reserved":0,"allocated":1667072,"data_stored":1050036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.875,"5min":1.3839999999999999,"15min":0.92200000000000004},"min":{"1min":0.621,"5min":0.36899999999999999,"15min":0.29599999999999999},"max":{"1min":16.815999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.044,"15min":0.87},"min":{"1min":0.57399999999999995,"5min":0.33400000000000002,"15min":0.28399999999999997},"max":{"1min":4.6100000000000003,"5min":8.5419999999999998,"15min":13.744},"last":0.441}]},{"osd":1,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.885,"5min":1.6519999999999999,"15min":1.0700000000000001},"min":{"1min":0.60599999999999998,"5min":0.39900000000000002,"15min":0.36699999999999999},"max":{"1min":16.920000000000002,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.637,"15min":1.0680000000000001},"min":{"1min":0.60299999999999998,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":4.7679999999999998,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":1.496,"15min":0.99199999999999999},"min":{"1min":0.52000000000000002,"5min":0.433,"15min":0.38400000000000001},"max":{"1min":2.1680000000000001,"5min":30.846,"15min":30.846},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.887,"5min":1.3140000000000001,"15min":1.097},"min":{"1min":0.60899999999999999,"5min":0.40999999999999998,"15min":0.39800000000000002},"max":{"1min":17.039999999999999,"5min":17.039999999999999,"15min":39.695},"last":0.73299999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":1.6299999999999999,"15min":1.163},"min":{"1min":0.55300000000000005,"5min":0.47499999999999998,"15min":0.38800000000000001},"max":{"1min":2.1160000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.831,"5min":2.0049999999999999,"15min":1.228},"min":{"1min":0.51300000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"max":{"1min":16.879999999999999,"5min":42.692,"15min":42.692},"last":0.70199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":0.89100000000000001,"15min":0.72199999999999998},"min":{"1min":0.432,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":14.087,"5min":14.087,"15min":14.087},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68500000000000005,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.36699999999999999,"15min":0.36299999999999999},"max":{"1min":2.2450000000000001,"5min":2.952,"15min":4.3710000000000004},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":0.80500000000000005,"15min":0.74199999999999999},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":11.334,"5min":11.334,"15min":11.334},"last":0.505},{"interface":"front","average":{"1min":1.012,"5min":0.91000000000000003,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":4.8179999999999996,"5min":8.423,"15min":8.423},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:15:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":0.91300000000000003,"15min":0.82499999999999996},"min":{"1min":0.53700000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":4.7140000000000004,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.439,"15min":0.376},"max":{"1min":1.1970000000000001,"5min":3.3450000000000002,"15min":3.3450000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76588,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295252,"statfs":{"total":96636764160,"available":96558338048,"internally_reserved":0,"allocated":1081344,"data_stored":460292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.83799999999999997,"15min":0.94799999999999995},"min":{"1min":0.41399999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.78200000000000003,"15min":0.872},"min":{"1min":0.39700000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.266,"5min":3.4940000000000002,"15min":12.449},"last":0.81599999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.46999999999999997,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":0.85999999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.1379999999999999,"15min":1.0109999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.379},"max":{"1min":1.615,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":1.087,"15min":1.1759999999999999},"min":{"1min":0.55600000000000005,"5min":0.435,"15min":0.435},"max":{"1min":0.94499999999999995,"5min":14.743,"15min":42.825000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.91400000000000003,"15min":1.147},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1890000000000001,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.78100000000000003,"15min":1.0269999999999999},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":1.3640000000000001,"5min":3.016,"15min":24.199000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":1.0489999999999999,"15min":1.0629999999999999},"min":{"1min":0.60699999999999998,"5min":0.47799999999999998,"15min":0.38300000000000001},"max":{"1min":0.91300000000000003,"5min":18.791,"15min":18.791},"last":0.83199999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.93200000000000005,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.585,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.66100000000000003,"15min":0.75800000000000001},"min":{"1min":0.50800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":12.491},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.53700000000000003,"5min":0.36799999999999999,"15min":0.34799999999999998},"max":{"1min":0.92300000000000004,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.443},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.77400000000000002,"15min":0.90900000000000003},"min":{"1min":0.628,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.4379999999999999,"5min":2.5259999999999998,"15min":12.945},"last":0.73599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:16:04 2024","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":0.98599999999999999,"15min":0.84599999999999997},"min":{"1min":0.501,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":5.1829999999999998,"5min":11.907,"15min":11.907},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80800000000000005,"15min":0.88900000000000001},"min":{"1min":0.45200000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5569999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.84199999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54456,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317384,"statfs":{"total":96636764160,"available":96581001216,"internally_reserved":0,"allocated":1085440,"data_stored":461232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.94999999999999996,"15min":0.82899999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":4.4770000000000003,"5min":6.6820000000000004,"15min":11.519},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.95699999999999996,"15min":0.90300000000000002},"min":{"1min":0.46999999999999997,"5min":0.39800000000000002,"15min":0.34999999999999998},"max":{"1min":1.714,"5min":7.6260000000000003,"15min":12.952},"last":0.88200000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.95499999999999996,"15min":0.88500000000000001},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":4.3140000000000001,"5min":4.609,"15min":13.358000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.91500000000000004,"15min":0.94099999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.859,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.149,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.435,"15min":0.38200000000000001},"max":{"1min":2.867,"5min":18.314,"15min":18.314},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.341,"15min":1.006},"min":{"1min":0.55700000000000005,"5min":0.442,"15min":0.434},"max":{"1min":2.855,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.89500000000000002,"15min":0.93300000000000005},"min":{"1min":0.51700000000000002,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":1.839,"5min":4.0330000000000004,"15min":13.317},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":1.034,"15min":1},"min":{"1min":0.443,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.1440000000000001,"5min":15.205,"15min":19.638000000000002},"last":0.74399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.41099999999999998,"5min":0.35099999999999998,"15min":0.28799999999999998},"max":{"1min":2.4289999999999998,"5min":5.899,"15min":11.093},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32100000000000001},"max":{"1min":2.516,"5min":3.097,"15min":3.3889999999999998},"last":0.623}]},{"osd":6,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.80000000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.238,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.9470000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.82799999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.77100000000000002,"15min":0.76200000000000001},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":2.3900000000000001,"5min":2.9870000000000001,"15min":4.1799999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.83299999999999996},"min":{"1min":0.52200000000000002,"5min":0.46800000000000003,"15min":0.40400000000000003},"max":{"1min":2.3239999999999998,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":1069056,"data_stored":459598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.097,"15min":0.92600000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":9.4179999999999993,"5min":11.374000000000001,"15min":34.802},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":1.135,"15min":0.89300000000000002},"min":{"1min":0.41099999999999998,"5min":0.35399999999999998,"15min":0.34699999999999998},"max":{"1min":3.3450000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.57499999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.97399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2130000000000001,"5min":7.625,"15min":38.198},"last":0.52400000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":0.93999999999999995,"15min":0.95799999999999996},"min":{"1min":0.55700000000000005,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.4550000000000001,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.76500000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":1.0589999999999999,"15min":0.86399999999999999},"min":{"1min":0.57999999999999996,"5min":0.46300000000000002,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":7.6989999999999998,"15min":7.6989999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0149999999999999,"15min":0.85499999999999998},"min":{"1min":0.61499999999999999,"5min":0.40799999999999997,"15min":0.38700000000000001},"max":{"1min":4.4089999999999998,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.621}]},{"osd":3,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.2729999999999999,"15min":0.94499999999999995},"min":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.40300000000000002},"max":{"1min":3.415,"5min":14.529,"15min":14.529},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":1.2430000000000001,"15min":0.93300000000000005},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.284,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.78100000000000003,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":3.6070000000000002,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.96899999999999997,"15min":0.83099999999999996},"min":{"1min":0.47499999999999998,"5min":0.39200000000000002,"15min":0.34100000000000003},"max":{"1min":3.0209999999999999,"5min":25.887,"15min":25.887},"last":0.74299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.45000000000000001,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.96399999999999997,"5min":7.5439999999999996,"15min":10.284000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.68200000000000005,"15min":0.70599999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":3.6949999999999998,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55000000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:15:06 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.85999999999999999,"15min":0.79300000000000004},"min":{"1min":0.45000000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.1680000000000001,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81100000000000005,"15min":0.79400000000000004},"min":{"1min":0.46500000000000002,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":1.232,"5min":11.67,"15min":11.67},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574987,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76588,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295252,"statfs":{"total":96636764160,"available":96558338048,"internally_reserved":0,"allocated":1081344,"data_stored":461210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.595,"15min":0.95199999999999996},"min":{"1min":0.32000000000000001,"5min":0.25900000000000001,"15min":0.255},"max":{"1min":7.2839999999999998,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.436},{"interface":"front","average":{"1min":0.99199999999999999,"5min":1.4670000000000001,"15min":0.94799999999999995},"min":{"1min":0.32200000000000001,"5min":0.27500000000000002,"15min":0.246},"max":{"1min":7.5170000000000003,"5min":23.074999999999999,"15min":28.332000000000001},"last":0.38900000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":1.4690000000000001,"15min":0.96299999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.2970000000000002,"5min":20.321000000000002,"15min":40.457999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.514,"15min":1.1399999999999999},"min":{"1min":0.40500000000000003,"5min":0.372,"15min":0.30299999999999999},"max":{"1min":7.3170000000000002,"5min":26.260999999999999,"15min":27.427},"last":0.51400000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.615,"15min":0.97899999999999998},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.28399999999999997},"max":{"1min":9.1180000000000003,"5min":23.036999999999999,"15min":23.036999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.8240000000000001,"15min":1.0780000000000001},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.34300000000000003},"max":{"1min":7.2409999999999997,"5min":26.404,"15min":26.404},"last":0.55600000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3819999999999999,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.45400000000000001,"15min":0.41199999999999998},"max":{"1min":4.1040000000000001,"5min":22.998000000000001,"15min":22.998000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.069},"min":{"1min":0.51100000000000001,"5min":0.49399999999999999,"15min":0.377},"max":{"1min":2.1150000000000002,"5min":20.399000000000001,"15min":20.399000000000001},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.6259999999999999,"15min":1.097},"min":{"1min":0.58399999999999996,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":6.5140000000000002,"5min":26.314,"15min":26.314},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.204,"5min":1.5649999999999999,"15min":1.131},"min":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.44500000000000001},"max":{"1min":4.165,"5min":22.908999999999999,"15min":22.908999999999999},"last":0.69199999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.635,"15min":1.1140000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.7619999999999996,"5min":20.445,"15min":20.445},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.7270000000000001,"15min":1.1699999999999999},"min":{"1min":0.61399999999999999,"5min":0.55000000000000004,"15min":0.495},"max":{"1min":6.7149999999999999,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Jan 15 18:15:07 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.6839999999999999,"15min":1.155},"min":{"1min":0.498,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":4.2030000000000003,"5min":23.111999999999998,"15min":23.111999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":2.1429999999999998,"15min":1.3089999999999999},"min":{"1min":0.56699999999999995,"5min":0.46400000000000002,"15min":0.46000000000000002},"max":{"1min":2.2490000000000001,"5min":44.817999999999998,"15min":44.817999999999998},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76584,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295256,"statfs":{"total":96636764160,"available":96558342144,"internally_reserved":0,"allocated":1073152,"data_stored":460180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.76800000000000002,"15min":1.0509999999999999},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":1.1040000000000001,"5min":11.727,"15min":29.527999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":1.0860000000000001,"15min":1.321},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.33800000000000002},"max":{"1min":0.92400000000000004,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.75,"15min":1.5369999999999999},"min":{"1min":0.313,"5min":0.309,"15min":0.30199999999999999},"max":{"1min":1.137,"5min":7.476,"15min":113.149},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":1.077,"15min":1.103},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.82999999999999996,"5min":20.619,"15min":41.302},"last":0.47099999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.6679999999999999,"15min":1.3680000000000001},"min":{"1min":0.35799999999999998,"5min":0.33000000000000002,"15min":0.28899999999999998},"max":{"1min":0.88200000000000001,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.78000000000000003,"15min":1.262},"min":{"1min":0.40500000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.80700000000000005,"5min":7.3609999999999998,"15min":115.693},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":1.0089999999999999,"15min":1.214},"min":{"1min":0.51800000000000002,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":2.7530000000000001,"5min":7.4169999999999998,"15min":24.366},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":1.167,"15min":1.212},"min":{"1min":0.46000000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.095,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":1.8240000000000001,"15min":1.4950000000000001},"min":{"1min":0.59599999999999997,"5min":0.504,"15min":0.43099999999999999},"max":{"1min":2.7709999999999999,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":1.1020000000000001,"15min":1.3129999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.496},"max":{"1min":1.2130000000000001,"5min":8.5239999999999991,"15min":26.084},"last":0.67100000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.333,"15min":1.3029999999999999},"min":{"1min":0.56100000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.169,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":1.8480000000000001,"15min":1.5609999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":1.784,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.85399999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":1.292,"15min":1.248},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":2.383,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":1.9910000000000001,"15min":1.6319999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.48799999999999999},"max":{"1min":2.3929999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.74199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574988,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76592,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295248,"statfs":{"total":96636764160,"available":96558333952,"internally_reserved":0,"allocated":1081344,"data_stored":459792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.61799999999999999,"15min":0.91600000000000004},"min":{"1min":0.29499999999999998,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.6000000000000001,"5min":4.3330000000000002,"15min":39.948},"last":0.434},{"interface":"front","average":{"1min":0.90600000000000003,"5min":1.0600000000000001,"15min":1.1299999999999999},"min":{"1min":0.35199999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":8.141,"5min":27.608000000000001,"15min":35.790999999999997},"last":0.62}]},{"osd":2,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.69399999999999995,"15min":0.96299999999999997},"min":{"1min":0.314,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.3250000000000002,"5min":6.516,"15min":49.229999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.79700000000000004,"15min":0.84999999999999998},"min":{"1min":0.34100000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2889999999999999,"5min":7.9489999999999998,"15min":10.349},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":1.0880000000000001,"15min":1.2050000000000001},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3640000000000001,"5min":27.643999999999998,"15min":27.643999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.89000000000000001,"15min":1.0009999999999999},"min":{"1min":0.40699999999999997,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.54,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.40200000000000002},"max":{"1min":2.5539999999999998,"5min":27.489999999999998,"15min":27.489999999999998},"last":1.028},{"interface":"front","average":{"1min":1.157,"5min":1.014,"15min":1.0669999999999999},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":8.4870000000000001,"5min":8.4870000000000001,"15min":12.997},"last":0.89800000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":1.27,"15min":1.2330000000000001},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.734,"5min":27.547000000000001,"15min":27.547000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.876,"5min":1.0169999999999999,"15min":1.127},"min":{"1min":0.65300000000000002,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":1.613,"5min":5.3399999999999999,"15min":13.054},"last":0.93400000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":0.55300000000000005,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":1.7749999999999999,"5min":27.579999999999998,"15min":27.579999999999998},"last":0.995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.89500000000000002,"15min":1.0600000000000001},"min":{"1min":0.59799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6919999999999999,"5min":5.0860000000000003,"15min":11.712999999999999},"last":0.97899999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:16:05 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":1.0309999999999999,"15min":1.1299999999999999},"min":{"1min":0.63700000000000001,"5min":0.46899999999999997,"15min":0.46500000000000002},"max":{"1min":1.9650000000000001,"5min":9.5660000000000007,"15min":11.833},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.995,"15min":1.139},"min":{"1min":0.505,"5min":0.505,"15min":0.502},"max":{"1min":1.5800000000000001,"5min":5.9550000000000001,"15min":14.606},"last":1.0429999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77160,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294680,"statfs":{"total":96636764160,"available":96557752320,"internally_reserved":0,"allocated":1667072,"data_stored":1050000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":0.95199999999999996,"15min":1.198},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":11.356,"5min":11.356,"15min":31.977},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.357,"5min":1.04,"15min":1.0069999999999999},"min":{"1min":0.34499999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":11.629,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.37}]},{"osd":2,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0049999999999999,"15min":1.109},"min":{"1min":0.38700000000000001,"5min":0.35299999999999998,"15min":0.30199999999999999},"max":{"1min":4.9349999999999996,"5min":13.166,"15min":41.387999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.90800000000000003,"15min":1.05},"min":{"1min":0.40699999999999997,"5min":0.314,"15min":0.25},"max":{"1min":4.1520000000000001,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.45600000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.0700000000000001,"15min":1.1379999999999999},"min":{"1min":0.34699999999999998,"5min":0.33700000000000002,"15min":0.29899999999999999},"max":{"1min":11.391999999999999,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.032,"15min":1.0760000000000001},"min":{"1min":0.433,"5min":0.432,"15min":0.33300000000000002},"max":{"1min":2.5470000000000002,"5min":12.954000000000001,"15min":45.128},"last":0.48399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":1.0920000000000001,"15min":1.095},"min":{"1min":0.501,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.093,"5min":11.772,"15min":32.932000000000002},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.167,"15min":1.2210000000000001},"min":{"1min":0.45500000000000002,"5min":0.45100000000000001,"15min":0.44},"max":{"1min":11.220000000000001,"5min":11.220000000000001,"15min":31.782},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.123,"15min":1.218},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":11.287000000000001,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.872},{"interface":"front","average":{"1min":0.73999999999999999,"5min":1.111,"15min":1.0920000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.45800000000000002},"max":{"1min":1.143,"5min":11.958,"15min":33.151000000000003},"last":0.75600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.1870000000000001,"15min":1.056},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.46400000000000002},"max":{"1min":1.708,"5min":13.055,"15min":13.055},"last":1.008},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.1579999999999999,"15min":1.123},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1830000000000001,"5min":11.833,"15min":33.098999999999997},"last":0.85799999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:16:02 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":1.1850000000000001,"15min":1.0669999999999999},"min":{"1min":0.53700000000000003,"5min":0.52000000000000002,"15min":0.501},"max":{"1min":1.669,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.1499999999999999,"15min":1.2729999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.504},"max":{"1min":11.321999999999999,"5min":11.321999999999999,"15min":31.905999999999999},"last":0.80900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:16:11.924 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:16:11.924 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:16:12.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.106+0000 7f545d400640 1 Processor -- start 2024-01-15T18:16:12.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.106+0000 7f545d400640 1 -- start start 2024-01-15T18:16:12.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f545d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f54580b3d80 0x7f54580b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f545d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f545806d080 0x7f545806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f545d400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54580cd5b0 con 0x7f54581594b0 2024-01-15T18:16:12.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f545d400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545806da00 con 0x7f545806d080 2024-01-15T18:16:12.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f545d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580b2ec0 con 0x7f54580b3d80 2024-01-15T18:16:12.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f545806d080 0x7f545806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f545806d080 0x7f545806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55096/0 (socket says 172.21.15.143:55096) 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 -- 172.21.15.143:0/224517898 learned_addr learned my addr 172.21.15.143:0/224517898 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5455e00640 1 --2- 172.21.15.143:0/224517898 >> v2:172.21.15.204:3300/0 conn(0x7f54580b3d80 0x7f54580b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 -- 172.21.15.143:0/224517898 >> v2:172.21.15.204:3300/0 conn(0x7f54580b3d80 msgr2=0x7f54580b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 --2- 172.21.15.143:0/224517898 >> v2:172.21.15.204:3300/0 conn(0x7f54580b3d80 0x7f54580b27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 -- 172.21.15.143:0/224517898 >> v1:172.21.15.143:6789/0 conn(0x7f54581594b0 legacy=0x7f54580cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5457200640 1 -- 172.21.15.143:0/224517898 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580ccf60 con 0x7f545806d080 2024-01-15T18:16:12.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.110+0000 7f5455e00640 1 --2- 172.21.15.143:0/224517898 >> v2:172.21.15.204:3300/0 conn(0x7f54580b3d80 0x7f54580b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:12.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f5457200640 1 --2- 172.21.15.143:0/224517898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f545806d080 0x7f545806d4c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f54580b43f0 tx=0x7f545001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8989971d2a4296f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f5455400640 1 -- 172.21.15.143:0/224517898 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500a5720 con 0x7f545806d080 2024-01-15T18:16:12.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f5455400640 1 -- 172.21.15.143:0/224517898 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54500a5880 con 0x7f545806d080 2024-01-15T18:16:12.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f5455400640 1 -- 172.21.15.143:0/224517898 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500a5c10 con 0x7f545806d080 2024-01-15T18:16:12.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 -- 172.21.15.143:0/224517898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f545806d080 msgr2=0x7f545806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 --2- 172.21.15.143:0/224517898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f545806d080 0x7f545806d4c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f54580b43f0 tx=0x7f545001c9d0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 -- 172.21.15.143:0/224517898 shutdown_connections 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 --2- 172.21.15.143:0/224517898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f545806d080 0x7f545806d4c0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 --2- 172.21.15.143:0/224517898 >> v2:172.21.15.204:3300/0 conn(0x7f54580b3d80 0x7f54580b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 -- 172.21.15.143:0/224517898 >> 172.21.15.143:0/224517898 conn(0x7f545806a890 msgr2=0x7f545806aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 -- 172.21.15.143:0/224517898 shutdown_connections 2024-01-15T18:16:12.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 -- 172.21.15.143:0/224517898 wait complete. 2024-01-15T18:16:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 Processor -- start 2024-01-15T18:16:12.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.114+0000 7f545d400640 1 -- start start 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f545806d080 0x7f545813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54580b3d80 0x7f545814df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 0x7f5458157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545814fa80 con 0x7f54581594b0 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545814e9e0 con 0x7f54580b3d80 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545814ecc0 con 0x7f545806d080 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 0x7f5458157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 0x7f5458157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55098/0 (socket says 172.21.15.143:55098) 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 -- 172.21.15.143:0/1935390193 learned_addr learned my addr 172.21.15.143:0/1935390193 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5455e00640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54580b3d80 0x7f545814df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5456800640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f545806d080 0x7f545813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 -- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f545806d080 msgr2=0x7f545813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f545806d080 0x7f545813f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 -- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54580b3d80 msgr2=0x7f545814df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54580b3d80 0x7f545814df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 -- 172.21.15.143:0/1935390193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5450009b30 con 0x7f54581594b0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f5457200640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 0x7f5458157aa0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f54500af040 tx=0x7f5450019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500b8670 con 0x7f54581594b0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54500b87d0 con 0x7f54581594b0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545814efa0 con 0x7f54581594b0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54500b8b80 con 0x7f54581594b0 2024-01-15T18:16:12.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545814f560 con 0x7f54581594b0 2024-01-15T18:16:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545814f750 con 0x7f54581594b0 2024-01-15T18:16:12.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.118+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f54500b8ce0 con 0x7f54581594b0 2024-01-15T18:16:12.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.122+0000 7f544ea00640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54300675e0 0x7f5430069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.122+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f545011e580 con 0x7f54581594b0 2024-01-15T18:16:12.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.122+0000 7f5456800640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54300675e0 0x7f5430069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.122+0000 7f5456800640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54300675e0 0x7f5430069aa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5448000f70 tx=0x7f5448009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.126+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54500ecfc0 con 0x7f54581594b0 2024-01-15T18:16:12.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.326+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5458067500 con 0x7f54581594b0 2024-01-15T18:16:12.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f544ea00640 1 -- 172.21.15.143:0/1935390193 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f54500f05e0 con 0x7f54581594b0 2024-01-15T18:16:12.325 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:16:12.325 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:16:12.326 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:16:12.326 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:16:12.326 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:16:12.326 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54300675e0 msgr2=0x7f5430069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54300675e0 0x7f5430069aa0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5448000f70 tx=0x7f5448009040 comp rx=0 tx=0).stop 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 msgr2=0x7f5458157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 0x7f5458157aa0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f54500af040 tx=0x7f5450019770 comp rx=0 tx=0).stop 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 shutdown_connections 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54300675e0 0x7f5430069aa0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54581594b0 0x7f5458157aa0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54580b3d80 0x7f545814df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 --2- 172.21.15.143:0/1935390193 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f545806d080 0x7f545813f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 >> 172.21.15.143:0/1935390193 conn(0x7f545806a890 msgr2=0x7f545806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:12.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 shutdown_connections 2024-01-15T18:16:12.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.330+0000 7f545d400640 1 -- 172.21.15.143:0/1935390193 wait complete. 2024-01-15T18:16:12.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:16:12.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.486+0000 7fcc9be00640 1 Processor -- start 2024-01-15T18:16:12.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.486+0000 7fcc9be00640 1 -- start start 2024-01-15T18:16:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9406bfb0 0x7fcc9406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc9be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcc940b0310 0x7fcc940cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc9be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc940cbff0 con 0x7fcc940cc9c0 2024-01-15T18:16:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc9be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc940d0c50 con 0x7fcc940b0310 2024-01-15T18:16:12.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc9be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9406c8d0 con 0x7fcc9406bfb0 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9406bfb0 0x7fcc9406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fcc940cc9c0 0x7fcc940cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58242/0 (socket says 172.21.15.143:58242) 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc98a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcc940b0310 0x7fcc940cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99e00640 1 -- 172.21.15.143:0/238835049 learned_addr learned my addr 172.21.15.143:0/238835049 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99400640 1 -- 172.21.15.143:0/238835049 >> v2:172.21.15.204:3300/0 conn(0x7fcc940b0310 msgr2=0x7fcc940cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99400640 1 --2- 172.21.15.143:0/238835049 >> v2:172.21.15.204:3300/0 conn(0x7fcc940b0310 0x7fcc940cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99400640 1 -- 172.21.15.143:0/238835049 >> v1:172.21.15.204:6789/0 conn(0x7fcc940cc9c0 legacy=0x7fcc940cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.490+0000 7fcc99400640 1 -- 172.21.15.143:0/238835049 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc940cb6c0 con 0x7fcc9406bfb0 2024-01-15T18:16:12.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc99400640 1 --2- 172.21.15.143:0/238835049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9406bfb0 0x7fcc9406c390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc840097c0 tx=0x7fcc8401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c8fa33cc20fd2c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc93e00640 1 -- 172.21.15.143:0/238835049 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840ac620 con 0x7fcc9406bfb0 2024-01-15T18:16:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc93e00640 1 -- 172.21.15.143:0/238835049 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840ac780 con 0x7fcc9406bfb0 2024-01-15T18:16:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc93e00640 1 -- 172.21.15.143:0/238835049 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840acb10 con 0x7fcc9406bfb0 2024-01-15T18:16:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 -- 172.21.15.143:0/238835049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9406bfb0 msgr2=0x7fcc9406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 --2- 172.21.15.143:0/238835049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9406bfb0 0x7fcc9406c390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcc840097c0 tx=0x7fcc8401c3c0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 -- 172.21.15.143:0/238835049 shutdown_connections 2024-01-15T18:16:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 --2- 172.21.15.143:0/238835049 >> v2:172.21.15.204:3300/0 conn(0x7fcc940b0310 0x7fcc940cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 --2- 172.21.15.143:0/238835049 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc9406bfb0 0x7fcc9406c390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 -- 172.21.15.143:0/238835049 >> 172.21.15.143:0/238835049 conn(0x7fcc940d0ed0 msgr2=0x7fcc940d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 -- 172.21.15.143:0/238835049 shutdown_connections 2024-01-15T18:16:12.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 -- 172.21.15.143:0/238835049 wait complete. 2024-01-15T18:16:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 Processor -- start 2024-01-15T18:16:12.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.494+0000 7fcc9be00640 1 -- start start 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9406bfb0 0x7fcc941588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc940b0310 0x7fcc9415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 0x7fcc94152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94159a00 con 0x7fcc940cc9c0 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94159160 con 0x7fcc9406bfb0 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94159440 con 0x7fcc940b0310 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 0x7fcc94152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 0x7fcc94152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55106/0 (socket says 172.21.15.143:55106) 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 -- 172.21.15.143:0/3131084706 learned_addr learned my addr 172.21.15.143:0/3131084706 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99400640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9406bfb0 0x7fcc941588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc98a00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc940b0310 0x7fcc9415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 -- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc940b0310 msgr2=0x7fcc9415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc940b0310 0x7fcc9415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 -- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9406bfb0 msgr2=0x7fcc941588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9406bfb0 0x7fcc941588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 -- 172.21.15.143:0/3131084706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc84009420 con 0x7fcc940cc9c0 2024-01-15T18:16:12.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc99e00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 0x7fcc94152d30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00cbc0 tx=0x7fcc8c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00ad40 con 0x7fcc940cc9c0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c012400 con 0x7fcc940cc9c0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00aea0 con 0x7fcc940cc9c0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc94159720 con 0x7fcc940cc9c0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc94158de0 con 0x7fcc940cc9c0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.498+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc94067500 con 0x7fcc940cc9c0 2024-01-15T18:16:12.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.502+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fcc8c007070 con 0x7fcc940cc9c0 2024-01-15T18:16:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.502+0000 7fcc91600640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc6c0675e0 0x7fcc6c069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.502+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcc8c082d50 con 0x7fcc940cc9c0 2024-01-15T18:16:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.502+0000 7fcc99400640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc6c0675e0 0x7fcc6c069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.502+0000 7fcc99400640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc6c0675e0 0x7fcc6c069aa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcc840ae000 tx=0x7fcc84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.506+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcc8c051a90 con 0x7fcc940cc9c0 2024-01-15T18:16:12.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.686+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc94153b80 con 0x7fcc940cc9c0 2024-01-15T18:16:12.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.686+0000 7fcc91600640 1 -- 172.21.15.143:0/3131084706 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fcc8c017070 con 0x7fcc940cc9c0 2024-01-15T18:16:12.684 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:12.684 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":52,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1584800432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1584800432},{"type":"v1","addr":"172.21.15.204:6834","nonce":1584800432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7843,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3526494826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3526494826},{"type":"v1","addr":"172.21.15.204:6835","nonce":3526494826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1307769037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1307769037},{"type":"v1","addr":"172.21.15.143:6835","nonce":1307769037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:16:02.335736+0000","modified":"2024-01-15T18:16:05.360719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8025},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8025":{"gid":8025,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6837/1230123305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":1230123305},{"type":"v1","addr":"172.21.15.143:6837","nonce":1230123305}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-01-15T18:16:12.685 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 52 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc6c0675e0 msgr2=0x7fcc6c069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc6c0675e0 0x7fcc6c069aa0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcc840ae000 tx=0x7fcc84008040 comp rx=0 tx=0).stop 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 msgr2=0x7fcc94152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 0x7fcc94152d30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00cbc0 tx=0x7fcc8c00a430 comp rx=0 tx=0).stop 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 shutdown_connections 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcc6c0675e0 0x7fcc6c069aa0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc940cc9c0 0x7fcc94152d30 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc940b0310 0x7fcc9415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 --2- 172.21.15.143:0/3131084706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc9406bfb0 0x7fcc941588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 >> 172.21.15.143:0/3131084706 conn(0x7fcc940d0ed0 msgr2=0x7fcc940adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 shutdown_connections 2024-01-15T18:16:12.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:12.690+0000 7fcc9be00640 1 -- 172.21.15.143:0/3131084706 wait complete. 2024-01-15T18:16:13.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:16:02.335736+0000', 'modified': '2024-01-15T18:16:05.360719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8025}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8025': {'gid': 8025, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6837/1230123305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 1230123305}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 1230123305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:16:13.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:16:13.108 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-15T18:16:13.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 Processor -- start 2024-01-15T18:16:13.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 -- start start 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd8901607c0 0x7fd89006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd8900cbf50 0x7fd8900cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8900cffd0 con 0x7fd8900cfb30 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8900d1ce0 con 0x7fd8900cbf50 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd895000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8900d02b0 con 0x7fd8901607c0 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd8900cbf50 0x7fd8900cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd8900cbf50 0x7fd8900cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55112/0 (socket says 172.21.15.143:55112) 2024-01-15T18:16:13.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 -- 172.21.15.143:0/530423715 learned_addr learned my addr 172.21.15.143:0/530423715 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 -- 172.21.15.143:0/530423715 >> v2:172.21.15.204:3300/0 conn(0x7fd8901607c0 msgr2=0x7fd89006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 --2- 172.21.15.143:0/530423715 >> v2:172.21.15.204:3300/0 conn(0x7fd8901607c0 0x7fd89006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 -- 172.21.15.143:0/530423715 >> v1:172.21.15.143:6789/0 conn(0x7fd8900cfb30 legacy=0x7fd89006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.254+0000 7fd88e800640 1 -- 172.21.15.143:0/530423715 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8900cb470 con 0x7fd8900cbf50 2024-01-15T18:16:13.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd88e800640 1 --2- 172.21.15.143:0/530423715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cbf50 0x7fd8900cc3c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd890071a00 tx=0x7fd88801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9790be5f61444d1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd88ca00640 1 -- 172.21.15.143:0/530423715 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880af400 con 0x7fd8900cbf50 2024-01-15T18:16:13.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd88ca00640 1 -- 172.21.15.143:0/530423715 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8880ab050 con 0x7fd8900cbf50 2024-01-15T18:16:13.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd88ca00640 1 -- 172.21.15.143:0/530423715 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8880af780 con 0x7fd8900cbf50 2024-01-15T18:16:13.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 -- 172.21.15.143:0/530423715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cbf50 msgr2=0x7fd8900cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 --2- 172.21.15.143:0/530423715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cbf50 0x7fd8900cc3c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fd890071a00 tx=0x7fd88801f1c0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 -- 172.21.15.143:0/530423715 shutdown_connections 2024-01-15T18:16:13.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 --2- 172.21.15.143:0/530423715 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cbf50 0x7fd8900cc3c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 --2- 172.21.15.143:0/530423715 >> v2:172.21.15.204:3300/0 conn(0x7fd8901607c0 0x7fd89006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 -- 172.21.15.143:0/530423715 >> 172.21.15.143:0/530423715 conn(0x7fd8900d0e20 msgr2=0x7fd8900d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 -- 172.21.15.143:0/530423715 shutdown_connections 2024-01-15T18:16:13.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 -- 172.21.15.143:0/530423715 wait complete. 2024-01-15T18:16:13.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.258+0000 7fd895000640 1 Processor -- start 2024-01-15T18:16:13.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 -- start start 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8900cbf50 0x7fd8900c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 0x7fd8900c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8901607c0 0x7fd8901585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900ca920 con 0x7fd8900cfb30 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900c4f40 con 0x7fd8901607c0 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8900c5220 con 0x7fd8900cbf50 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 0x7fd8900c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 0x7fd8900c4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55114/0 (socket says 172.21.15.143:55114) 2024-01-15T18:16:13.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 -- 172.21.15.143:0/2731628370 learned_addr learned my addr 172.21.15.143:0/2731628370 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88e800640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8901607c0 0x7fd8901585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88de00640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8900cbf50 0x7fd8900c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 -- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8900cbf50 msgr2=0x7fd8900c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8900cbf50 0x7fd8900c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 -- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8901607c0 msgr2=0x7fd8901585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8901607c0 0x7fd8901585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 -- 172.21.15.143:0/2731628370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888009b30 con 0x7fd8900cfb30 2024-01-15T18:16:13.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd88d400640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 0x7fd8900c4880 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c500 tx=0x7fd87800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd886000640 1 -- 172.21.15.143:0/2731628370 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878005220 con 0x7fd8900cfb30 2024-01-15T18:16:13.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd886000640 1 -- 172.21.15.143:0/2731628370 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878005380 con 0x7fd8900cfb30 2024-01-15T18:16:13.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd886000640 1 -- 172.21.15.143:0/2731628370 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878007270 con 0x7fd8900cfb30 2024-01-15T18:16:13.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8900c5500 con 0x7fd8900cfb30 2024-01-15T18:16:13.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.262+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8900c5ac0 con 0x7fd8900cfb30 2024-01-15T18:16:13.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd886000640 1 -- 172.21.15.143:0/2731628370 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fd8780073d0 con 0x7fd8900cfb30 2024-01-15T18:16:13.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd886000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8680675e0 0x7fd868069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd886000640 1 -- 172.21.15.143:0/2731628370 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd878082770 con 0x7fd8900cfb30 2024-01-15T18:16:13.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88de00640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8680675e0 0x7fd868069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 Processor -- start 2024-01-15T18:16:13.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 -- start start 2024-01-15T18:16:13.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88de00640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8680675e0 0x7fd868069aa0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fd0 tx=0x7fd880008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d1b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 0x5643d3cd0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3cd1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3b3b130 con 0x5643d3cd1320 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3b289a0 con 0x5643d3d1ac90 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643d3b25ed0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3cd1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 0x5643d3cd0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d1b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 0x5643d3cd0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:39720/0 (socket says 172.21.15.143:39720) 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 -- 172.21.15.143:0/440484495 learned_addr learned my addr 172.21.15.143:0/440484495 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 -- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 msgr2=0x5643d3d1b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d1b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 -- 172.21.15.143:0/440484495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 msgr2=0x5643d3cd1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.266+0000 7fd88d400640 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3cd1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd88d400640 1 -- 172.21.15.143:0/440484495 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643d3819890 con 0x5643d3d1b5b0 2024-01-15T18:16:13.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd88e800640 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3cd1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:13.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd88d400640 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 0x5643d3cd0de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd878054d40 tx=0x7fd8780549a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd782e02044bc63d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd87ea00640 1 -- 172.21.15.143:0/440484495 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878055040 con 0x5643d3d1b5b0 2024-01-15T18:16:13.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd87ea00640 1 -- 172.21.15.143:0/440484495 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878059190 con 0x5643d3d1b5b0 2024-01-15T18:16:13.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd87ea00640 1 -- 172.21.15.143:0/440484495 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87805d340 con 0x5643d3d1b5b0 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 msgr2=0x5643d3cd0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 0x5643d3cd0de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd878054d40 tx=0x7fd8780549a0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- 172.21.15.143:0/440484495 shutdown_connections 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3cd1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1b5b0 0x5643d3cd0de0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- 172.21.15.143:0/440484495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d1b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- 172.21.15.143:0/440484495 >> 172.21.15.143:0/440484495 conn(0x5643d3ccd2d0 msgr2=0x5643d3ccf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- 172.21.15.143:0/440484495 shutdown_connections 2024-01-15T18:16:13.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- 172.21.15.143:0/440484495 wait complete. 2024-01-15T18:16:13.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- start start 2024-01-15T18:16:13.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 Processor -- start 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3cd1320 0x5643d3cd5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1ac90 0x5643d3cd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 0x5643d3ce9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3b217c0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3ad9bf0 con 0x5643d3cd1320 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643d3adfa90 con 0x5643d3d1ac90 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd89846f000 10 client.0 _ll_register_callbacks cb 0x5643d3cbc3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.270+0000 7fd88e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 0x5643d3ce9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 0x5643d3ce9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55136/0 (socket says 172.21.15.143:55136) 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 -- 172.21.15.143:0/2695480374 learned_addr learned my addr 172.21.15.143:0/2695480374 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88d400640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1ac90 0x5643d3cd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88de00640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3cd1320 0x5643d3cd5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 -- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1ac90 msgr2=0x5643d3cd6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1ac90 0x5643d3cd6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 -- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3cd1320 msgr2=0x5643d3cd5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3cd1320 0x5643d3cd5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 -- 172.21.15.143:0/2695480374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8900cb470 con 0x5643d3d1b5b0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88e800640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 0x5643d3ce9910 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd88801bdd0 tx=0x7fd888004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509bd5600b3d2708 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 1 -- 172.21.15.143:0/2695480374 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888009190 con 0x5643d3d1b5b0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 1 -- 172.21.15.143:0/2695480374 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd8880c5b10 con 0x5643d3d1b5b0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643d3ae3470 con 0x5643d3d1b5b0 2024-01-15T18:16:13.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 1 -- 172.21.15.143:0/2695480374 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8880c5cd0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 10 client.8166 fetch_fsmap learned FSMap version 52 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x7fd8900c5ac0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 1 -- 172.21.15.143:0/2695480374 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5643d3ae3470 con 0x5643d3d1b5b0 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 1 -- 172.21.15.143:0/2695480374 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8880b9ac0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 10 client.8166 fetch_fsmap finished waiting for FSMap version 52 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 10 client.8166 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 20 client.8166 populate_metadata read hostname 'smithi143' 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3cea320 0x5643d3d2cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 4 client.8166 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 --> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643d3ae6b50 con 0x5643d3cea320 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88de00640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3cea320 0x5643d3d2cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 10 client.8166 ms_handle_connect on v2:172.21.15.143:6836/1230123305 2024-01-15T18:16:13.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd88de00640 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3cea320 0x5643d3d2cf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efbc27c214bf8c1f server_cookie=fd6d468a2cb319a0 in_seq=0 out_seq=0 2024-01-15T18:16:13.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 1 -- 172.21.15.143:0/2695480374 <== mds.0 v2:172.21.15.143:6836/1230123305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643d3ae6b50 con 0x5643d3cea320 2024-01-15T18:16:13.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd877400640 10 client.8166 handle_command_reply: tid=0 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 client.8166 shutdown 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 msgr2=0x5643d3ce9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 0x5643d3ce9910 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd88801bdd0 tx=0x7fd888004a50 comp rx=0 tx=0).stop 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 shutdown_connections 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3cea320 0x5643d3d2cf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.274+0000 7fd89846f000 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3d1b5b0 0x5643d3ce9910 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd89846f000 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3d1ac90 0x5643d3cd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd877400640 0 client.8166 ms_handle_reset on v2:172.21.15.143:6836/1230123305 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd89846f000 1 --2- 172.21.15.143:0/2695480374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3cd1320 0x5643d3cd5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 >> 172.21.15.143:0/2695480374 conn(0x5643d3ccd2d0 msgr2=0x5643d3ccf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 shutdown_connections 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd89846f000 1 -- 172.21.15.143:0/2695480374 wait complete. 2024-01-15T18:16:13.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.278+0000 7fd89846f000 20 client.8166 trim_cache size 0 max 16384 2024-01-15T18:16:13.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.290+0000 7fd89846f000 1 Processor -- start 2024-01-15T18:16:13.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.290+0000 7fd89846f000 1 -- start start 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3ccd2d0 0x5643d3d18b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3d19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3cfd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d37da940 con 0x5643d3cd1320 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3ce98c0 con 0x5643d3d1ac90 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643d3cccc70 con 0x5643d3ccd2d0 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3d19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3d19070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55144/0 (socket says 172.21.15.143:55144) 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 -- 172.21.15.143:0/1342886007 learned_addr learned my addr 172.21.15.143:0/1342886007 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88de00640 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3ccd2d0 0x5643d3d18b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 -- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3ccd2d0 msgr2=0x5643d3d18b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3ccd2d0 0x5643d3d18b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 -- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 msgr2=0x5643d3cfd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88e800640 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3cfd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3cfd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 -- 172.21.15.143:0/1342886007 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643d3ae2190 con 0x5643d3cd1320 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88e800640 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3cfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd88d400640 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3d19070 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd878054f80 tx=0x7fd878073000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5b784ccd034bab server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd87f400640 1 -- 172.21.15.143:0/1342886007 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878055040 con 0x5643d3cd1320 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd87f400640 1 -- 172.21.15.143:0/1342886007 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878059190 con 0x5643d3cd1320 2024-01-15T18:16:13.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd87f400640 1 -- 172.21.15.143:0/1342886007 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87805d340 con 0x5643d3cd1320 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- 172.21.15.143:0/1342886007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 msgr2=0x5643d3d19070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3d19070 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd878054f80 tx=0x7fd878073000 comp rx=0 tx=0).stop 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- 172.21.15.143:0/1342886007 shutdown_connections 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3cfd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3cd1320 0x5643d3d19070 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 --2- 172.21.15.143:0/1342886007 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3ccd2d0 0x5643d3d18b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- 172.21.15.143:0/1342886007 >> 172.21.15.143:0/1342886007 conn(0x5643d3cd5ee0 msgr2=0x5643d3ccf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- 172.21.15.143:0/1342886007 shutdown_connections 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- 172.21.15.143:0/1342886007 wait complete. 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 -- start start 2024-01-15T18:16:13.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.294+0000 7fd89846f000 1 Processor -- start 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 0x5643d3cfdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3cd1320 0x5643d3d2cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d2d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3cd6820 con 0x5643d3ccd2d0 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643d3acc3b0 con 0x5643d3d1ac90 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643d3b1d5c0 con 0x5643d3cd1320 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 10 client.0 _ll_register_callbacks cb 0x5643d3b6fc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 0x5643d3cfdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 0x5643d3cfdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55158/0 (socket says 172.21.15.143:55158) 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 -- 172.21.15.143:0/359848388 learned_addr learned my addr 172.21.15.143:0/359848388 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88d400640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3cd1320 0x5643d3d2cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88e800640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d2d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 -- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3cd1320 msgr2=0x5643d3d2cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3cd1320 0x5643d3d2cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 -- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 msgr2=0x5643d3d2d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d2d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 -- 172.21.15.143:0/359848388 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643d3819890 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88e800640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d2d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88de00640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 0x5643d3cfdbd0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fa0 tx=0x7fd8800046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a09f6215fd60f9c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 1 -- 172.21.15.143:0/359848388 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880027e90 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 1 -- 172.21.15.143:0/359848388 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd880021880 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643d3b20110 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 1 -- 172.21.15.143:0/359848388 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd880021a40 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 10 client.8172 fetch_fsmap learned FSMap version 52 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x5643d3ac8490 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 1 -- 172.21.15.143:0/359848388 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5643d3b20110 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 1 -- 172.21.15.143:0/359848388 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd88002d020 con 0x5643d3ccd2d0 2024-01-15T18:16:13.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 10 client.8172 fetch_fsmap finished waiting for FSMap version 52 2024-01-15T18:16:13.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 10 client.8172 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:16:13.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 20 client.8172 populate_metadata read hostname 'smithi143' 2024-01-15T18:16:13.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3d1b5b0 0x5643d3d7be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 4 client.8172 mds_command: new command op to 8025 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:16:13.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 --> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5643d3b4f9a0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88d400640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3d1b5b0 0x5643d3d7be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd88d400640 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3d1b5b0 0x5643d3d7be00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9434aa59ca68f7e server_cookie=d78341bf76996d68 in_seq=0 out_seq=0 2024-01-15T18:16:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 10 client.8172 ms_handle_connect on v2:172.21.15.143:6836/1230123305 2024-01-15T18:16:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 1 -- 172.21.15.143:0/359848388 <== mds.0 v2:172.21.15.143:6836/1230123305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5643d3b4f9a0 con 0x5643d3d1b5b0 2024-01-15T18:16:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 10 client.8172 handle_command_reply: tid=0 2024-01-15T18:16:13.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 client.8172 shutdown 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 msgr2=0x5643d3cfdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 0x5643d3cfdbd0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fa0 tx=0x7fd8800046d0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 shutdown_connections 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:6836/1230123305,v1:172.21.15.143:6837/1230123305] conn(0x5643d3d1b5b0 0x5643d3d7be00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5643d3d1ac90 0x5643d3d2d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x5643d3cd1320 0x5643d3d2cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd877400640 0 client.8172 ms_handle_reset on v2:172.21.15.143:6836/1230123305 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 --2- 172.21.15.143:0/359848388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643d3ccd2d0 0x5643d3cfdbd0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.298+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 >> 172.21.15.143:0/359848388 conn(0x5643d3cd5ee0 msgr2=0x5643d3ccff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 shutdown_connections 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd89846f000 1 -- 172.21.15.143:0/359848388 wait complete. 2024-01-15T18:16:13.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd89846f000 20 client.8172 trim_cache size 0 max 16384 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 8025, 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.322159106999999, 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 52, 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 87, 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 87, 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 16.239733189999999 2024-01-15T18:16:13.302 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8680675e0 msgr2=0x7fd868069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8680675e0 0x7fd868069aa0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd880006fd0 tx=0x7fd880008040 comp rx=0 tx=0).stop 2024-01-15T18:16:13.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 msgr2=0x7fd8900c4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 0x7fd8900c4880 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c500 tx=0x7fd87800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 shutdown_connections 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd8680675e0 0x7fd868069aa0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd8901607c0 0x7fd8901585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd8900cfb30 0x7fd8900c4880 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 --2- 172.21.15.143:0/2731628370 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd8900cbf50 0x7fd8900c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 >> 172.21.15.143:0/2731628370 conn(0x7fd8900d0e20 msgr2=0x7fd8900cc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 shutdown_connections 2024-01-15T18:16:13.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.302+0000 7fd895000640 1 -- 172.21.15.143:0/2731628370 wait complete. 2024-01-15T18:16:13.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:16:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 Processor -- start 2024-01-15T18:16:13.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 -- start start 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3aa00cb180 0x7f3aa015a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa015da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa015e190 con 0x7f3aa00cb560 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa0070130 con 0x7f3aa00cb180 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3aa6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015df80 con 0x7f3aa00b13e0 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3a9f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3aa00cb180 0x7f3aa015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3a9f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3aa00cb180 0x7f3aa015a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55170/0 (socket says 172.21.15.143:55170) 2024-01-15T18:16:13.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.458+0000 7f3a9f400640 1 -- 172.21.15.143:0/473783564 learned_addr learned my addr 172.21.15.143:0/473783564 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9f400640 1 -- 172.21.15.143:0/473783564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 msgr2=0x7f3aa015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9f400640 1 --2- 172.21.15.143:0/473783564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9f400640 1 -- 172.21.15.143:0/473783564 >> v1:172.21.15.204:6789/0 conn(0x7f3aa00cb560 legacy=0x7f3aa00b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:13.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9f400640 1 -- 172.21.15.143:0/473783564 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa015e470 con 0x7f3aa00cb180 2024-01-15T18:16:13.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9f400640 1 --2- 172.21.15.143:0/473783564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa015a450 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009870 tx=0x7f3a9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b86d8a275fad02 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9e000640 1 -- 172.21.15.143:0/473783564 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900aa640 con 0x7f3aa00cb180 2024-01-15T18:16:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9e000640 1 -- 172.21.15.143:0/473783564 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a900aa7a0 con 0x7f3aa00cb180 2024-01-15T18:16:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.462+0000 7f3a9e000640 1 -- 172.21.15.143:0/473783564 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900aab30 con 0x7f3aa00cb180 2024-01-15T18:16:13.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- 172.21.15.143:0/473783564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 msgr2=0x7f3aa015a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 --2- 172.21.15.143:0/473783564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa015a450 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009870 tx=0x7f3a9001c520 comp rx=0 tx=0).stop 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- 172.21.15.143:0/473783564 shutdown_connections 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 --2- 172.21.15.143:0/473783564 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 --2- 172.21.15.143:0/473783564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa015a450 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- 172.21.15.143:0/473783564 >> 172.21.15.143:0/473783564 conn(0x7f3aa006a660 msgr2=0x7f3aa006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- 172.21.15.143:0/473783564 shutdown_connections 2024-01-15T18:16:13.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- 172.21.15.143:0/473783564 wait complete. 2024-01-15T18:16:13.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 Processor -- start 2024-01-15T18:16:13.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- start start 2024-01-15T18:16:13.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa00cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa00ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa00cb560 0x7f3aa00cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa01558a0 con 0x7f3aa00cb180 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa01475e0 con 0x7f3aa00cb560 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3aa6200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa0154c20 con 0x7f3aa00b13e0 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3a9ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa00ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3a9ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa00ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55186/0 (socket says 172.21.15.143:55186) 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.466+0000 7f3a9ea00640 1 -- 172.21.15.143:0/4184278771 learned_addr learned my addr 172.21.15.143:0/4184278771 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9fe00640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa00cb560 0x7f3aa00cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9f400640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa00cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9ea00640 1 -- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 msgr2=0x7f3aa00cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9ea00640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa00cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9ea00640 1 -- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa00cb560 msgr2=0x7f3aa00cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9ea00640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa00cb560 0x7f3aa00cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9ea00640 1 -- 172.21.15.143:0/4184278771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900094d0 con 0x7f3aa00cb180 2024-01-15T18:16:13.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9fe00640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa00cb560 0x7f3aa00cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:13.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9ea00640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa00ce190 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b3e0 tx=0x7f3a8800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88011830 con 0x7f3aa00cb180 2024-01-15T18:16:13.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a880119b0 con 0x7f3aa00cb180 2024-01-15T18:16:13.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88011d60 con 0x7f3aa00cb180 2024-01-15T18:16:13.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa0154f00 con 0x7f3aa00cb180 2024-01-15T18:16:13.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa01554c0 con 0x7f3aa00cb180 2024-01-15T18:16:13.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa01559e0 con 0x7f3aa00cb180 2024-01-15T18:16:13.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f3a8801a400 con 0x7f3aa00cb180 2024-01-15T18:16:13.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a97400640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a78067610 0x7f3a78069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:13.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3a88082ed0 con 0x7f3aa00cb180 2024-01-15T18:16:13.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.470+0000 7f3a9f400640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a78067610 0x7f3a78069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:13.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.474+0000 7f3a9f400640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a78067610 0x7f3a78069ad0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3a900ae000 tx=0x7f3a900b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:13.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.474+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3aa01559e0 con 0x7f3aa00cb180 2024-01-15T18:16:13.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3aa01559e0 con 0x7f3aa00cb180 2024-01-15T18:16:13.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3a97400640 1 -- 172.21.15.143:0/4184278771 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3aa01559e0 con 0x7f3aa00cb180 2024-01-15T18:16:13.658 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:16:13.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a78067610 msgr2=0x7f3a78069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a78067610 0x7f3a78069ad0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3a900ae000 tx=0x7f3a900b5040 comp rx=0 tx=0).stop 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 msgr2=0x7f3aa00ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa00ce190 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800b3e0 tx=0x7f3a8800b8a0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 shutdown_connections 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3a78067610 0x7f3a78069ad0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3aa00cb560 0x7f3aa00cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3aa00cb180 0x7f3aa00ce190 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 --2- 172.21.15.143:0/4184278771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3aa00b13e0 0x7f3aa00cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 >> 172.21.15.143:0/4184278771 conn(0x7f3aa006a660 msgr2=0x7f3aa00b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 shutdown_connections 2024-01-15T18:16:13.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:13.662+0000 7f3aa6200640 1 -- 172.21.15.143:0/4184278771 wait complete. 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:16:13.680 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:16:13.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:13.681 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.690 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 41769sec preferred_lft 41769sec 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.691 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.692 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:16:13.693 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:16:13.693 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:16:13.693 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:13.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:13.693 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:16:13.751 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:16:13.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:13.753 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:16:13.807 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:16:13.807 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:16:13.807 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:16:13.807 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:16:13.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:13.808 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:16:13.808 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:16:13.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:16:13.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:16:13.809 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:16:13.809 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:16:13.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:13.897 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:16:13.906 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:16:13.906 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:16:13.907 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:16:13.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:13.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:16:13.968 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:16:13.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:16:13.968 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:16:14.017 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:16:14.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:16:14.017 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:16:14.068 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:16:14.129 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:16:14.182 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:16:14.183 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:16:14.184 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:16:14.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:14.185 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:16:14.236 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:16:14.237 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:16:14.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:16:14.300 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:16:14.350 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:16:14.350 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:16:14.350 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:16:14.351 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:16:14.352 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:16:14.353 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:16:14.353 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:16:14.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:14.354 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:16:14.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:16:14.378+0000 7f510b43a580 -1 init, newargv = 0x55fc7d2f7c40 newargc=15 2024-01-15T18:16:14.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[49175]: starting ceph client 2024-01-15T18:16:14.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[49175]: starting fuse 2024-01-15T18:16:14.383 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:16:14.383 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:16:14.383 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:16:14.383 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:16:14.384 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:16:14.385 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:16:14.504 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.49175.asok 2024-01-15T18:16:14.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:16:14.515 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49175.asok 2024-01-15T18:16:14.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:14.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49175.asok status 2024-01-15T18:16:14.684 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "49175", 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 8184, 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:16:14.685 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 8184 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3110544767 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3110544767 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:16:14.686 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.8184 192.168.0.1:0/3110544767", 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/3110544767", 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 52, 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 87, 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:16:14.687 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:14.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:14.700 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:16:14.700 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:16:14.709 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:16:14.709 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:16:14.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:14.709 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:16:14.772 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:16:14.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.950+0000 7f5df1400640 1 Processor -- start 2024-01-15T18:16:14.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.950+0000 7f5df1400640 1 -- start start 2024-01-15T18:16:14.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5df1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5df1400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5dec15c5a0 0x7f5dec15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5df1400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dec0ceb50 con 0x7f5dec143780 2024-01-15T18:16:14.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5df1400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec072570 con 0x7f5dec15c5a0 2024-01-15T18:16:14.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5df1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec15c110 con 0x7f5dec071bb0 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5dea800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5dec143780 0x7f5dec0cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34756/0 (socket says 172.21.15.143:34756) 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5dea800640 1 -- 172.21.15.143:0/743456274 learned_addr learned my addr 172.21.15.143:0/743456274 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9e00640 1 --2- 172.21.15.143:0/743456274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5deb200640 1 --2- 172.21.15.143:0/743456274 >> v2:172.21.15.204:3300/0 conn(0x7f5dec15c5a0 0x7f5dec15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9400640 1 -- 172.21.15.143:0/743456274 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221917752 0 0) 0x7f5ddc001120 con 0x7f5dec143780 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9400640 1 -- 172.21.15.143:0/743456274 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dcc003530 con 0x7f5dec143780 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9e00640 1 -- 172.21.15.143:0/743456274 >> v2:172.21.15.204:3300/0 conn(0x7f5dec15c5a0 msgr2=0x7f5dec15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9e00640 1 --2- 172.21.15.143:0/743456274 >> v2:172.21.15.204:3300/0 conn(0x7f5dec15c5a0 0x7f5dec15ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9e00640 1 -- 172.21.15.143:0/743456274 >> v1:172.21.15.143:6789/0 conn(0x7f5dec143780 legacy=0x7f5dec0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.954+0000 7f5de9e00640 1 -- 172.21.15.143:0/743456274 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec15b7e0 con 0x7f5dec071bb0 2024-01-15T18:16:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5de9e00640 1 --2- 172.21.15.143:0/743456274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec072030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40097c0 tx=0x7f5dd401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d88b919203d938d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5de9400640 1 -- 172.21.15.143:0/743456274 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40ab680 con 0x7f5dec071bb0 2024-01-15T18:16:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5de9400640 1 -- 172.21.15.143:0/743456274 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd40ab7e0 con 0x7f5dec071bb0 2024-01-15T18:16:14.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5de9400640 1 -- 172.21.15.143:0/743456274 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40abb70 con 0x7f5dec071bb0 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 -- 172.21.15.143:0/743456274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 msgr2=0x7f5dec072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 --2- 172.21.15.143:0/743456274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec072030 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40097c0 tx=0x7f5dd401c480 comp rx=0 tx=0).stop 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 -- 172.21.15.143:0/743456274 shutdown_connections 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 --2- 172.21.15.143:0/743456274 >> v2:172.21.15.204:3300/0 conn(0x7f5dec15c5a0 0x7f5dec15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 --2- 172.21.15.143:0/743456274 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec072030 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 -- 172.21.15.143:0/743456274 >> 172.21.15.143:0/743456274 conn(0x7f5dec06c3f0 msgr2=0x7f5dec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:14.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 -- 172.21.15.143:0/743456274 shutdown_connections 2024-01-15T18:16:14.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 -- 172.21.15.143:0/743456274 wait complete. 2024-01-15T18:16:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 Processor -- start 2024-01-15T18:16:14.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.958+0000 7f5df1400640 1 -- start start 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec12f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 0x7f5dec1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec15c5a0 0x7f5dec162250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec164910 con 0x7f5dec143780 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec12d450 con 0x7f5dec15c5a0 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec130230 con 0x7f5dec071bb0 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 0x7f5dec1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 0x7f5dec1643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55202/0 (socket says 172.21.15.143:55202) 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 -- 172.21.15.143:0/1354675038 learned_addr learned my addr 172.21.15.143:0/1354675038 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5dea800640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec12f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5deb200640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec15c5a0 0x7f5dec162250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 -- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 msgr2=0x7f5dec12f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec12f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 -- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec15c5a0 msgr2=0x7f5dec162250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec15c5a0 0x7f5dec162250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 -- 172.21.15.143:0/1354675038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4009420 con 0x7f5dec143780 2024-01-15T18:16:14.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de9e00640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 0x7f5dec1643d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5dd401a0b0 tx=0x7f5dd401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40bf610 con 0x7f5dec143780 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd40bf790 con 0x7f5dec143780 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec162790 con 0x7f5dec143780 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40bfb40 con 0x7f5dec143780 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec12fc70 con 0x7f5dec143780 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.962+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec067500 con 0x7f5dec143780 2024-01-15T18:16:14.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.966+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f5dd40bfca0 con 0x7f5dec143780 2024-01-15T18:16:14.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.966+0000 7f5de2a00640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5dc40675e0 0x7f5dc4069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.966+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5dd411df90 con 0x7f5dec143780 2024-01-15T18:16:14.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.966+0000 7f5dea800640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5dc40675e0 0x7f5dc4069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.966+0000 7f5dea800640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5dc40675e0 0x7f5dc4069aa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc001280 tx=0x7f5ddc007bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:14.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:14.970+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5dd40eca40 con 0x7f5dec143780 2024-01-15T18:16:15.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5dec12d590 con 0x7f5dec143780 2024-01-15T18:16:15.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5de2a00640 1 -- 172.21.15.143:0/1354675038 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5dd40c4020 con 0x7f5dec143780 2024-01-15T18:16:15.150 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5dc40675e0 msgr2=0x7f5dc4069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5dc40675e0 0x7f5dc4069aa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc001280 tx=0x7f5ddc007bc0 comp rx=0 tx=0).stop 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 msgr2=0x7f5dec1643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 0x7f5dec1643d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5dd401a0b0 tx=0x7f5dd401acc0 comp rx=0 tx=0).stop 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 shutdown_connections 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5dc40675e0 0x7f5dc4069aa0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5dec15c5a0 0x7f5dec162250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5dec143780 0x7f5dec1643d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 --2- 172.21.15.143:0/1354675038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5dec071bb0 0x7f5dec12f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 >> 172.21.15.143:0/1354675038 conn(0x7f5dec06c3f0 msgr2=0x7f5dec0ad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 shutdown_connections 2024-01-15T18:16:15.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:15.154+0000 7f5df1400640 1 -- 172.21.15.143:0/1354675038 wait complete. 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:16:15.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:16:15.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.177 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:16:15.186 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:16:15.186 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:15.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 41762sec preferred_lft 41762sec 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.188 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:16:15.189 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:16:15.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.190 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:16:15.253 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:16:15.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.255 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:16:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:16:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:16:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:16:15.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.309 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:16:15.309 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:16:15.309 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:16:15.310 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:16:15.310 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:16:15.310 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:16:15.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.402 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:16:15.411 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:16:15.411 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:16:15.411 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:16:15.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.412 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:16:15.474 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:16:15.474 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:16:15.474 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:15.525 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:16:15.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:16:15.525 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:15.576 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:16:15.638 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:16:15.690 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:16:15.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:16:15.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:16:15.693 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:16:15.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:16:15.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:15.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:15.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:15.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:15.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.694 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:16:15.745 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:16:15.745 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:16:15.792 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:16:15.813 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:16:15.863 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:16:15.864 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:15.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:15.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:15.866 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:16:15.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:16:15.886+0000 7fab34445580 -1 init, newargv = 0x55975262c150 newargc=15 2024-01-15T18:16:15.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34160]: starting ceph client 2024-01-15T18:16:15.897 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34160]: starting fuse 2024-01-15T18:16:16.998 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:16:17.022 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:16:17.079 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:16:17.079 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:16:17.080 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:16:17.081 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:17.082 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:16:17.082 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:17.082 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:16:17.082 INFO:teuthology.orchestra.run.smithi204.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:17.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:17.083 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:16:17.137 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:16:17.137 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:16:17.137 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:16:17.138 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:16:17.139 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:16:17.268 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.34160.asok 2024-01-15T18:16:17.272 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:16:17.281 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34160.asok 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:17.281 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34160.asok status 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "34160", 2024-01-15T18:16:17.454 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 7010, 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 7010 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 4122362872 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:16:17.455 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 4122362872 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.7010 192.168.0.1:0/4122362872", 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/4122362872", 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 52, 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 87, 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:16:17.456 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:16:17.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:17.469 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:16:17.469 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:17.478 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:16:17.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:17.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:17.478 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:16:17.540 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:16:17.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 Processor -- start 2024-01-15T18:16:17.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 -- start start 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f664c141660 0x7f664c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f664c071d00 0x7f664c072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c072650 con 0x7f664c15ec80 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c15fbb0 con 0x7f664c071d00 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6654200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c15f130 con 0x7f664c141660 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f664c071d00 0x7f664c072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f664c071d00 0x7f664c072110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43206/0 (socket says 172.21.15.143:43206) 2024-01-15T18:16:17.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 -- 172.21.15.143:0/9794080 learned_addr learned my addr 172.21.15.143:0/9794080 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:17.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6651800640 1 --2- 172.21.15.143:0/9794080 >> v2:172.21.15.204:3300/0 conn(0x7f664c141660 0x7f664c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 -- 172.21.15.143:0/9794080 >> v2:172.21.15.204:3300/0 conn(0x7f664c141660 msgr2=0x7f664c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:17.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 --2- 172.21.15.143:0/9794080 >> v2:172.21.15.204:3300/0 conn(0x7f664c141660 0x7f664c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 -- 172.21.15.143:0/9794080 >> v1:172.21.15.204:6789/0 conn(0x7f664c15ec80 legacy=0x7f664c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.694+0000 7f6652200640 1 -- 172.21.15.143:0/9794080 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c15f310 con 0x7f664c071d00 2024-01-15T18:16:17.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6652200640 1 --2- 172.21.15.143:0/9794080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c071d00 0x7f664c072110 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f664c0b1aa0 tx=0x7f664401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63558c2efd7840b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:17.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f664be00640 1 -- 172.21.15.143:0/9794080 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440a5860 con 0x7f664c071d00 2024-01-15T18:16:17.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f664be00640 1 -- 172.21.15.143:0/9794080 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66440a59c0 con 0x7f664c071d00 2024-01-15T18:16:17.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f664be00640 1 -- 172.21.15.143:0/9794080 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440a5d50 con 0x7f664c071d00 2024-01-15T18:16:17.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 -- 172.21.15.143:0/9794080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c071d00 msgr2=0x7f664c072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 --2- 172.21.15.143:0/9794080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c071d00 0x7f664c072110 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f664c0b1aa0 tx=0x7f664401c9a0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 -- 172.21.15.143:0/9794080 shutdown_connections 2024-01-15T18:16:17.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 --2- 172.21.15.143:0/9794080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c071d00 0x7f664c072110 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 --2- 172.21.15.143:0/9794080 >> v2:172.21.15.204:3300/0 conn(0x7f664c141660 0x7f664c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 -- 172.21.15.143:0/9794080 >> 172.21.15.143:0/9794080 conn(0x7f664c06c3f0 msgr2=0x7f664c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:17.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 -- 172.21.15.143:0/9794080 shutdown_connections 2024-01-15T18:16:17.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.698+0000 7f6654200640 1 -- 172.21.15.143:0/9794080 wait complete. 2024-01-15T18:16:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 Processor -- start 2024-01-15T18:16:17.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 -- start start 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f664c071d00 0x7f664c15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f664c141660 0x7f664c139f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 0x7f664c13bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c13a710 con 0x7f664c15ec80 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b2ef0 con 0x7f664c071d00 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b3380 con 0x7f664c141660 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 0x7f664c13bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6651800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f664c071d00 0x7f664c15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 0x7f664c13bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43208/0 (socket says 172.21.15.143:43208) 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 -- 172.21.15.143:0/897542318 learned_addr learned my addr 172.21.15.143:0/897542318 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6650e00640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f664c141660 0x7f664c139f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 -- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f664c141660 msgr2=0x7f664c139f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f664c141660 0x7f664c139f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 -- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f664c071d00 msgr2=0x7f664c15e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f664c071d00 0x7f664c15e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 -- 172.21.15.143:0/897542318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644009b30 con 0x7f664c15ec80 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6652200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 0x7f664c13bc10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f66440af040 tx=0x7f6644019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440b8670 con 0x7f664c15ec80 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66440b87f0 con 0x7f664c15ec80 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66440b8ba0 con 0x7f664c15ec80 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c13a470 con 0x7f664c15ec80 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.702+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c0b28a0 con 0x7f664c15ec80 2024-01-15T18:16:17.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.706+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c06dd80 con 0x7f664c15ec80 2024-01-15T18:16:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.706+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f66440b8d00 con 0x7f664c15ec80 2024-01-15T18:16:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.706+0000 7f6649600640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6624067610 0x7f6624069ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.706+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f664411e4a0 con 0x7f664c15ec80 2024-01-15T18:16:17.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.706+0000 7f6651800640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6624067610 0x7f6624069ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.706+0000 7f6651800640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6624067610 0x7f6624069ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f663c009b60 tx=0x7f663c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:17.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.710+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f66440ed070 con 0x7f664c15ec80 2024-01-15T18:16:17.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.890+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f664c067500 con 0x7f664c15ec80 2024-01-15T18:16:17.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.966+0000 7f6649600640 1 -- 172.21.15.143:0/897542318 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 136+0+0 (secure 0 0 0) 0x7f66440bc0c0 con 0x7f664c15ec80 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6624067610 msgr2=0x7f6624069ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6624067610 0x7f6624069ad0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f663c009b60 tx=0x7f663c009190 comp rx=0 tx=0).stop 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 msgr2=0x7f664c13bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 0x7f664c13bc10 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f66440af040 tx=0x7f6644019740 comp rx=0 tx=0).stop 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 shutdown_connections 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6624067610 0x7f6624069ad0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f664c15ec80 0x7f664c13bc10 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f664c141660 0x7f664c139f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 --2- 172.21.15.143:0/897542318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f664c071d00 0x7f664c15e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 >> 172.21.15.143:0/897542318 conn(0x7f664c06c3f0 msgr2=0x7f664c140190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 shutdown_connections 2024-01-15T18:16:17.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:17.970+0000 7f6654200640 1 -- 172.21.15.143:0/897542318 wait complete. 2024-01-15T18:16:17.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:16:18.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 Processor -- start 2024-01-15T18:16:18.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 -- start start 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f015b2d0 0x7fc7f00d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7f00cebf0 0x7fc7f00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc7f00cf930 0x7fc7f00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f015b6b0 con 0x7fc7f00cebf0 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00b0620 con 0x7fc7f00cf930 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b0320 con 0x7fc7f015b2d0 2024-01-15T18:16:18.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7f00cebf0 0x7fc7f00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc7f00cebf0 0x7fc7f00cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43214/0 (socket says 172.21.15.143:43214) 2024-01-15T18:16:18.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 -- 172.21.15.143:0/2313513100 learned_addr learned my addr 172.21.15.143:0/2313513100 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:18.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5c00640 1 --2- 172.21.15.143:0/2313513100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f015b2d0 0x7fc7f00d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f6600640 1 --2- 172.21.15.143:0/2313513100 >> v2:172.21.15.204:3300/0 conn(0x7fc7f00cf930 0x7fc7f00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 -- 172.21.15.143:0/2313513100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f015b2d0 msgr2=0x7fc7f00d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:18.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 --2- 172.21.15.143:0/2313513100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f015b2d0 0x7fc7f00d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 -- 172.21.15.143:0/2313513100 >> v2:172.21.15.204:3300/0 conn(0x7fc7f00cf930 msgr2=0x7fc7f00cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:18.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 --2- 172.21.15.143:0/2313513100 >> v2:172.21.15.204:3300/0 conn(0x7fc7f00cf930 0x7fc7f00cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.130+0000 7fc7f5200640 1 -- 172.21.15.143:0/2313513100 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00cc3c0 con 0x7fc7f00cebf0 2024-01-15T18:16:18.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f5200640 1 --2- 172.21.15.143:0/2313513100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cebf0 0x7fc7f00cf3f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00ce910 tx=0x7fc7d801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9c4974d2c58277 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7efe00640 1 -- 172.21.15.143:0/2313513100 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80ab600 con 0x7fc7f00cebf0 2024-01-15T18:16:18.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7efe00640 1 -- 172.21.15.143:0/2313513100 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d80a7050 con 0x7fc7f00cebf0 2024-01-15T18:16:18.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7efe00640 1 -- 172.21.15.143:0/2313513100 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80ab980 con 0x7fc7f00cebf0 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 -- 172.21.15.143:0/2313513100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cebf0 msgr2=0x7fc7f00cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 --2- 172.21.15.143:0/2313513100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cebf0 0x7fc7f00cf3f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00ce910 tx=0x7fc7d801c240 comp rx=0 tx=0).stop 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 -- 172.21.15.143:0/2313513100 shutdown_connections 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 --2- 172.21.15.143:0/2313513100 >> v2:172.21.15.204:3300/0 conn(0x7fc7f00cf930 0x7fc7f00cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 --2- 172.21.15.143:0/2313513100 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cebf0 0x7fc7f00cf3f0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 --2- 172.21.15.143:0/2313513100 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f015b2d0 0x7fc7f00d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 -- 172.21.15.143:0/2313513100 >> 172.21.15.143:0/2313513100 conn(0x7fc7f00d0e10 msgr2=0x7fc7f00d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 -- 172.21.15.143:0/2313513100 shutdown_connections 2024-01-15T18:16:18.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.134+0000 7fc7f8600640 1 -- 172.21.15.143:0/2313513100 wait complete. 2024-01-15T18:16:18.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 Processor -- start 2024-01-15T18:16:18.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 -- start start 2024-01-15T18:16:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f00cebf0 0x7fc7f00b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 0x7fc7f006bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f015b2d0 0x7fc7f006c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f006ca30 con 0x7fc7f00cf930 2024-01-15T18:16:18.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f01f9a00 con 0x7fc7f015b2d0 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f01f9060 con 0x7fc7f00cebf0 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 0x7fc7f006bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 0x7fc7f006bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43216/0 (socket says 172.21.15.143:43216) 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 -- 172.21.15.143:0/83092313 learned_addr learned my addr 172.21.15.143:0/83092313 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5c00640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f00cebf0 0x7fc7f00b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f6600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f015b2d0 0x7fc7f006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 -- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f00cebf0 msgr2=0x7fc7f00b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f00cebf0 0x7fc7f00b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 -- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f015b2d0 msgr2=0x7fc7f006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f015b2d0 0x7fc7f006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 -- 172.21.15.143:0/83092313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80094d0 con 0x7fc7f00cf930 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f5200640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 0x7fc7f006bee0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80ae040 tx=0x7fc7d8018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80abe90 con 0x7fc7f00cf930 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d80b4440 con 0x7fc7f00cf930 2024-01-15T18:16:18.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f01f9340 con 0x7fc7f00cf930 2024-01-15T18:16:18.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80b47d0 con 0x7fc7f00cf930 2024-01-15T18:16:18.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.138+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f00b5cd0 con 0x7fc7f00cf930 2024-01-15T18:16:18.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.142+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f0067500 con 0x7fc7f00cf930 2024-01-15T18:16:18.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.142+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fc7d80b2070 con 0x7fc7f00cf930 2024-01-15T18:16:18.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.142+0000 7fc7ed600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7c80675e0 0x7fc7c8069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.142+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc7d811cb80 con 0x7fc7f00cf930 2024-01-15T18:16:18.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.142+0000 7fc7f5c00640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7c80675e0 0x7fc7c8069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.142+0000 7fc7f5c00640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7c80675e0 0x7fc7c8069aa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0156bf0 tx=0x7fc7e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.146+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7d80eb8c0 con 0x7fc7f00cf930 2024-01-15T18:16:18.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.314+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7f00d0860 con 0x7fc7c80675e0 2024-01-15T18:16:18.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7ed600640 1 -- 172.21.15.143:0/83092313 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc7f00d0860 con 0x7fc7c80675e0 2024-01-15T18:16:18.315 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:16:18.315 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:16:18.315 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:16:18.315 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:18.315 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7c80675e0 msgr2=0x7fc7c8069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7c80675e0 0x7fc7c8069aa0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0156bf0 tx=0x7fc7e0009190 comp rx=0 tx=0).stop 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 msgr2=0x7fc7f006bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 0x7fc7f006bee0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80ae040 tx=0x7fc7d8018fe0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 shutdown_connections 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc7c80675e0 0x7fc7c8069aa0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc7f015b2d0 0x7fc7f006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7f00cf930 0x7fc7f006bee0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 --2- 172.21.15.143:0/83092313 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7f00cebf0 0x7fc7f00b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 >> 172.21.15.143:0/83092313 conn(0x7fc7f00d0e10 msgr2=0x7fc7f00735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 shutdown_connections 2024-01-15T18:16:18.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.318+0000 7fc7f8600640 1 -- 172.21.15.143:0/83092313 wait complete. 2024-01-15T18:16:18.337 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:16:18.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.478+0000 7f5e7a800640 1 Processor -- start 2024-01-15T18:16:18.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.478+0000 7f5e7a800640 1 -- start start 2024-01-15T18:16:18.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e7a800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5e74072030 0x7f5e7415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e7a800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e740ceb50 con 0x7f5e74071bb0 2024-01-15T18:16:18.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e7a800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7415c110 con 0x7f5e74143780 2024-01-15T18:16:18.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e7a800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7415b7e0 con 0x7f5e74072030 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f5e74071bb0 0x7f5e7415c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45624/0 (socket says 172.21.15.143:45624) 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72a00640 1 -- 172.21.15.143:0/711245369 learned_addr learned my addr 172.21.15.143:0/711245369 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73e00640 1 --2- 172.21.15.143:0/711245369 >> v2:172.21.15.204:3300/0 conn(0x7f5e74072030 0x7f5e7415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552452832 0 0) 0x7f5e5c001120 con 0x7f5e74071bb0 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e54003530 con 0x7f5e74071bb0 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205941140 0 0) 0x7f5e7415c110 con 0x7f5e74143780 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e5c001120 con 0x7f5e74143780 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544550567 0 0) 0x7f5e54003530 con 0x7f5e74071bb0 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 >> v2:172.21.15.204:3300/0 conn(0x7f5e74072030 msgr2=0x7f5e7415ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 --2- 172.21.15.143:0/711245369 >> v2:172.21.15.204:3300/0 conn(0x7f5e74072030 0x7f5e7415ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 >> v1:172.21.15.204:6789/0 conn(0x7f5e74143780 legacy=0x7f5e740cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7415bac0 con 0x7f5e74071bb0 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f5e5c001810 con 0x7f5e74071bb0 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73e00640 1 --2- 172.21.15.143:0/711245369 >> v2:172.21.15.204:3300/0 conn(0x7f5e74072030 0x7f5e7415ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:18.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 >> v1:172.21.15.143:6789/0 conn(0x7f5e74071bb0 legacy=0x7f5e7415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54003b10 0x7f5e54003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e540047d0 0x7f5e54004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54005190 0x7f5e5400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7415b7e0 con 0x7f5e54003b10 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5400e220 con 0x7f5e54005190 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5400e3e0 con 0x7f5e540047d0 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73400640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54003b10 0x7f5e54003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73e00640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54005190 0x7f5e5400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e72a00640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e540047d0 0x7f5e54004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73400640 1 -- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e540047d0 msgr2=0x7f5e54004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73400640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e540047d0 0x7f5e54004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73400640 1 -- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54005190 msgr2=0x7f5e5400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73400640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54005190 0x7f5e5400dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73400640 1 -- 172.21.15.143:0/711245369 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e54003530 con 0x7f5e54003b10 2024-01-15T18:16:18.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.482+0000 7f5e73e00640 1 -- 172.21.15.143:0/711245369 reap_dead start 2024-01-15T18:16:18.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e73400640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54003b10 0x7f5e54003f10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f5e640018d0 tx=0x7f5e6401c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c291d0e5dc0f2ae8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e640052d0 con 0x7f5e54003b10 2024-01-15T18:16:18.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e640a7050 con 0x7f5e54003b10 2024-01-15T18:16:18.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e72000640 1 -- 172.21.15.143:0/711245369 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64003600 con 0x7f5e54003b10 2024-01-15T18:16:18.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 -- 172.21.15.143:0/711245369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54003b10 msgr2=0x7f5e54003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54003b10 0x7f5e54003f10 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f5e640018d0 tx=0x7f5e6401c240 comp rx=0 tx=0).stop 2024-01-15T18:16:18.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 -- 172.21.15.143:0/711245369 shutdown_connections 2024-01-15T18:16:18.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 --2- 172.21.15.143:0/711245369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e54003b10 0x7f5e54003f10 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 -- 172.21.15.143:0/711245369 >> 172.21.15.143:0/711245369 conn(0x7f5e7406c3f0 msgr2=0x7f5e7406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:18.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 -- 172.21.15.143:0/711245369 shutdown_connections 2024-01-15T18:16:18.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 -- 172.21.15.143:0/711245369 wait complete. 2024-01-15T18:16:18.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.486+0000 7f5e7a800640 1 Processor -- start 2024-01-15T18:16:18.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- start start 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54003b10 0x7f5e741375e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e74141510 0x7f5e74141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 0x7f5e740b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c4a80 con 0x7f5e741f5930 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c3e40 con 0x7f5e54003b10 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e740c40e0 con 0x7f5e74141510 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 0x7f5e740b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 0x7f5e740b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43236/0 (socket says 172.21.15.143:43236) 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 -- 172.21.15.143:0/2963744570 learned_addr learned my addr 172.21.15.143:0/2963744570 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73400640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54003b10 0x7f5e741375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e72a00640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e74141510 0x7f5e74141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 -- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e74141510 msgr2=0x7f5e74141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e74141510 0x7f5e74141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 -- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54003b10 msgr2=0x7f5e741375e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54003b10 0x7f5e741375e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 -- 172.21.15.143:0/2963744570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740c43c0 con 0x7f5e741f5930 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e73e00640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 0x7f5e740b2bc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5e74074700 tx=0x7f5e6c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00e040 con 0x7f5e741f5930 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c00f040 con 0x7f5e741f5930 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c003be0 con 0x7f5e741f5930 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e740c46a0 con 0x7f5e741f5930 2024-01-15T18:16:18.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e741f5dc0 con 0x7f5e741f5930 2024-01-15T18:16:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.490+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e741f6040 con 0x7f5e741f5930 2024-01-15T18:16:18.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.494+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f5e6c015430 con 0x7f5e741f5930 2024-01-15T18:16:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.494+0000 7f5e6b400640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e4c0675e0 0x7f5e4c069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.494+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5e6c083880 con 0x7f5e741f5930 2024-01-15T18:16:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.494+0000 7f5e73400640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e4c0675e0 0x7f5e4c069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.494+0000 7f5e73400640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e4c0675e0 0x7f5e4c069aa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5e640ac000 tx=0x7f5e640a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.498+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5e741f6040 con 0x7f5e741f5930 2024-01-15T18:16:18.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.646+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5e740c4bc0 con 0x7f5e741f5930 2024-01-15T18:16:18.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e6b400640 1 -- 172.21.15.143:0/2963744570 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e740c4bc0 con 0x7f5e741f5930 2024-01-15T18:16:18.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e4c0675e0 msgr2=0x7f5e4c069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e4c0675e0 0x7f5e4c069aa0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5e640ac000 tx=0x7f5e640a6000 comp rx=0 tx=0).stop 2024-01-15T18:16:18.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 msgr2=0x7f5e740b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 0x7f5e740b2bc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5e74074700 tx=0x7f5e6c009f90 comp rx=0 tx=0).stop 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 shutdown_connections 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5e4c0675e0 0x7f5e4c069aa0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5e741f5930 0x7f5e740b2bc0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5e74141510 0x7f5e74141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 --2- 172.21.15.143:0/2963744570 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5e54003b10 0x7f5e741375e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 >> 172.21.15.143:0/2963744570 conn(0x7f5e7406c3f0 msgr2=0x7f5e740ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 shutdown_connections 2024-01-15T18:16:18.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.650+0000 7f5e7a800640 1 -- 172.21.15.143:0/2963744570 wait complete. 2024-01-15T18:16:18.668 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:16:18.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.814+0000 7f54b2200640 1 Processor -- start 2024-01-15T18:16:18.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.814+0000 7f54b2200640 1 -- start start 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54b2200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54ac0ce560 0x7f54ac15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54b2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac071d00 0x7f54ac072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54b2200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ac15f310 con 0x7f54ac141660 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54b2200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ac15fbb0 con 0x7f54ac0ce560 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54b2200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac072690 con 0x7f54ac071d00 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54ac0ce560 0x7f54ac15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f54ac0ce560 0x7f54ac15ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43238/0 (socket says 172.21.15.143:43238) 2024-01-15T18:16:18.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 -- 172.21.15.143:0/1146682681 learned_addr learned my addr 172.21.15.143:0/1146682681 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:18.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54abe00640 1 --2- 172.21.15.143:0/1146682681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac071d00 0x7f54ac072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 -- 172.21.15.143:0/1146682681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac071d00 msgr2=0x7f54ac072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:18.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 --2- 172.21.15.143:0/1146682681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac071d00 0x7f54ac072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 -- 172.21.15.143:0/1146682681 >> v1:172.21.15.204:6789/0 conn(0x7f54ac141660 legacy=0x7f54ac141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.818+0000 7f54aaa00640 1 -- 172.21.15.143:0/1146682681 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ac15f5f0 con 0x7f54ac0ce560 2024-01-15T18:16:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54aaa00640 1 --2- 172.21.15.143:0/1146682681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac0ce560 0x7f54ac15ec80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5494009870 tx=0x7f549401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e905cfc955753a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54aa000640 1 -- 172.21.15.143:0/1146682681 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54940a7070 con 0x7f54ac0ce560 2024-01-15T18:16:18.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54aa000640 1 -- 172.21.15.143:0/1146682681 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54940ab430 con 0x7f54ac0ce560 2024-01-15T18:16:18.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54aa000640 1 -- 172.21.15.143:0/1146682681 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54940ab7c0 con 0x7f54ac0ce560 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 -- 172.21.15.143:0/1146682681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac0ce560 msgr2=0x7f54ac15ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 --2- 172.21.15.143:0/1146682681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac0ce560 0x7f54ac15ec80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f5494009870 tx=0x7f549401c540 comp rx=0 tx=0).stop 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 -- 172.21.15.143:0/1146682681 shutdown_connections 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 --2- 172.21.15.143:0/1146682681 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac071d00 0x7f54ac072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 --2- 172.21.15.143:0/1146682681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac0ce560 0x7f54ac15ec80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 -- 172.21.15.143:0/1146682681 >> 172.21.15.143:0/1146682681 conn(0x7f54ac06c3f0 msgr2=0x7f54ac06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 -- 172.21.15.143:0/1146682681 shutdown_connections 2024-01-15T18:16:18.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 -- 172.21.15.143:0/1146682681 wait complete. 2024-01-15T18:16:18.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 Processor -- start 2024-01-15T18:16:18.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.822+0000 7f54b2200640 1 -- start start 2024-01-15T18:16:18.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 0x7f54ac0ac770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac0ce560 0x7f54ac13d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54ac141660 0x7f54ac13dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac13ed20 con 0x7f54ac071d00 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac15afd0 con 0x7f54ac141660 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ac15d640 con 0x7f54ac0ce560 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 0x7f54ac0ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 0x7f54ac0ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43242/0 (socket says 172.21.15.143:43242) 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54aaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac0ce560 0x7f54ac13d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 -- 172.21.15.143:0/2311586373 learned_addr learned my addr 172.21.15.143:0/2311586373 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:18.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54abe00640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54ac141660 0x7f54ac13dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 -- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac0ce560 msgr2=0x7f54ac13d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac0ce560 0x7f54ac13d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 -- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54ac141660 msgr2=0x7f54ac13dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54ac141660 0x7f54ac13dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 -- 172.21.15.143:0/2311586373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54940094d0 con 0x7f54ac071d00 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54aaa00640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac0ce560 0x7f54ac13d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54ab400640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 0x7f54ac0ac770 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c3f0 tx=0x7f549c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c0052d0 con 0x7f54ac071d00 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c005430 con 0x7f54ac071d00 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ac0cd150 con 0x7f54ac071d00 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c003840 con 0x7f54ac071d00 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ac15d080 con 0x7f54ac071d00 2024-01-15T18:16:18.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ac13ee60 con 0x7f54ac071d00 2024-01-15T18:16:18.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.826+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7f549c0039a0 con 0x7f54ac071d00 2024-01-15T18:16:18.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.830+0000 7f54a3400640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54840675e0 0x7f5484069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:18.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.830+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f549c082340 con 0x7f54ac071d00 2024-01-15T18:16:18.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.830+0000 7f54aaa00640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54840675e0 0x7f5484069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:18.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.830+0000 7f54aaa00640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54840675e0 0x7f5484069aa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f54940af000 tx=0x7f5494008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:18.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:18.834+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f54ac13ee60 con 0x7f54ac071d00 2024-01-15T18:16:19.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.010+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f54ac13eaa0 con 0x7f54840675e0 2024-01-15T18:16:19.014 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:19.018+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:19.014 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:19.018+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:19.014 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:19.018+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:19.014 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:19.018+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:19.014 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:19.018+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:19.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.046+0000 7f54a3400640 1 -- 172.21.15.143:0/2311586373 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f54940bc0c0 con 0x7f54840675e0 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54840675e0 msgr2=0x7f5484069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54840675e0 0x7f5484069aa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f54940af000 tx=0x7f5494008040 comp rx=0 tx=0).stop 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 msgr2=0x7f54ac0ac770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 0x7f54ac0ac770 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c3f0 tx=0x7f549c00c8b0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 shutdown_connections 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f54840675e0 0x7f5484069aa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54ac141660 0x7f54ac13dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f54ac0ce560 0x7f54ac13d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 --2- 172.21.15.143:0/2311586373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54ac071d00 0x7f54ac0ac770 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 >> 172.21.15.143:0/2311586373 conn(0x7f54ac06c3f0 msgr2=0x7f54ac142f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 shutdown_connections 2024-01-15T18:16:19.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.050+0000 7f54b2200640 1 -- 172.21.15.143:0/2311586373 wait complete. 2024-01-15T18:16:19.066 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:16:19.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd21800640 1 Processor -- start 2024-01-15T18:16:19.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd21800640 1 -- start start 2024-01-15T18:16:19.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd21800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c143780 0x7fbd1c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd21800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1c0ceb50 con 0x7fbd1c0722c0 2024-01-15T18:16:19.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd21800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1c15c110 con 0x7fbd1c0ce520 2024-01-15T18:16:19.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd21800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c15b7e0 con 0x7fbd1c143780 2024-01-15T18:16:19.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd1b400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbd1c0722c0 0x7fbd1c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45626/0 (socket says 172.21.15.143:45626) 2024-01-15T18:16:19.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd1b400640 1 -- 172.21.15.143:0/3964614610 learned_addr learned my addr 172.21.15.143:0/3964614610 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd1aa00640 1 --2- 172.21.15.143:0/3964614610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c143780 0x7fbd1c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379659262 0 0) 0x7fbd140014a0 con 0x7fbd1c0722c0 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc003530 con 0x7fbd1c0722c0 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848434963 0 0) 0x7fbd1c15c110 con 0x7fbd1c0ce520 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd140014a0 con 0x7fbd1c0ce520 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd1aa00640 1 -- 172.21.15.143:0/3964614610 >> v1:172.21.15.204:6789/0 conn(0x7fbd1c0ce520 legacy=0x7fbd1c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3786452803 0 0) 0x7fbcfc003530 con 0x7fbd1c0722c0 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd1aa00640 1 -- 172.21.15.143:0/3964614610 >> v1:172.21.15.143:6789/0 conn(0x7fbd1c0722c0 legacy=0x7fbd1c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.210+0000 7fbd1aa00640 1 -- 172.21.15.143:0/3964614610 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c15bac0 con 0x7fbd1c143780 2024-01-15T18:16:19.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd1aa00640 1 --2- 172.21.15.143:0/3964614610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c143780 0x7fbd1c0cdfe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0097f0 tx=0x7fbd0c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349742ba9adf72a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:19.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0ac630 con 0x7fbd1c143780 2024-01-15T18:16:19.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c0ac790 con 0x7fbd1c143780 2024-01-15T18:16:19.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd19600640 1 -- 172.21.15.143:0/3964614610 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0acb20 con 0x7fbd1c143780 2024-01-15T18:16:19.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 -- 172.21.15.143:0/3964614610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c143780 msgr2=0x7fbd1c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 --2- 172.21.15.143:0/3964614610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c143780 0x7fbd1c0cdfe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0097f0 tx=0x7fbd0c01c4c0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 -- 172.21.15.143:0/3964614610 shutdown_connections 2024-01-15T18:16:19.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 --2- 172.21.15.143:0/3964614610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c143780 0x7fbd1c0cdfe0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 -- 172.21.15.143:0/3964614610 >> 172.21.15.143:0/3964614610 conn(0x7fbd1c06c3f0 msgr2=0x7fbd1c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:19.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 -- 172.21.15.143:0/3964614610 shutdown_connections 2024-01-15T18:16:19.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.214+0000 7fbd21800640 1 -- 172.21.15.143:0/3964614610 wait complete. 2024-01-15T18:16:19.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 Processor -- start 2024-01-15T18:16:19.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 -- start start 2024-01-15T18:16:19.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd1c0722c0 0x7fbd1c159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c0ce520 0x7fbd1c141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 0x7fbd1c13e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c159e90 con 0x7fbd1c143780 2024-01-15T18:16:19.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c141930 con 0x7fbd1c0722c0 2024-01-15T18:16:19.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c141000 con 0x7fbd1c0ce520 2024-01-15T18:16:19.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 0x7fbd1c13e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 0x7fbd1c13e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43272/0 (socket says 172.21.15.143:43272) 2024-01-15T18:16:19.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 -- 172.21.15.143:0/3127868899 learned_addr learned my addr 172.21.15.143:0/3127868899 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:19.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1aa00640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd1c0722c0 0x7fbd1c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1a000640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c0ce520 0x7fbd1c141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 -- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c0ce520 msgr2=0x7fbd1c141dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c0ce520 0x7fbd1c141dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 -- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd1c0722c0 msgr2=0x7fbd1c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd1c0722c0 0x7fbd1c159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 -- 172.21.15.143:0/3127868899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c009420 con 0x7fbd1c143780 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1aa00640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd1c0722c0 0x7fbd1c159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd1b400640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 0x7fbd1c13e650 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400cc20 tx=0x7fbd1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400ce30 con 0x7fbd1c143780 2024-01-15T18:16:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14012400 con 0x7fbd1c143780 2024-01-15T18:16:19.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14012770 con 0x7fbd1c143780 2024-01-15T18:16:19.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1c1412e0 con 0x7fbd1c143780 2024-01-15T18:16:19.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.218+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c13f0c0 con 0x7fbd1c143780 2024-01-15T18:16:19.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.222+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1c067500 con 0x7fbd1c143780 2024-01-15T18:16:19.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.222+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 30) v1 ==== 87333+0+0 (secure 0 0 0) 0x7fbd140128d0 con 0x7fbd1c143780 2024-01-15T18:16:19.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.222+0000 7fbd12a00640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf40675e0 0x7fbcf4069aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.222+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbd140832a0 con 0x7fbd1c143780 2024-01-15T18:16:19.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.222+0000 7fbd1aa00640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf40675e0 0x7fbcf4069aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.222+0000 7fbd1aa00640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf40675e0 0x7fbcf4069aa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0ae000 tx=0x7fbd0c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:19.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.226+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbd14051fe0 con 0x7fbd1c143780 2024-01-15T18:16:19.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.398+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd1c15d710 con 0x7fbcf40675e0 2024-01-15T18:16:19.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd12a00640 1 -- 172.21.15.143:0/3127868899 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fbd0c0acdb0 con 0x7fbcf40675e0 2024-01-15T18:16:19.402 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c 2024-01-15T18:16:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf40675e0 msgr2=0x7fbcf4069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf40675e0 0x7fbcf4069aa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0ae000 tx=0x7fbd0c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:16:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 msgr2=0x7fbd1c13e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 0x7fbd1c13e650 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400cc20 tx=0x7fbd1400a430 comp rx=0 tx=0).stop 2024-01-15T18:16:19.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 shutdown_connections 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf40675e0 0x7fbcf4069aa0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd1c143780 0x7fbd1c13e650 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd1c0ce520 0x7fbd1c141dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 --2- 172.21.15.143:0/3127868899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd1c0722c0 0x7fbd1c159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 >> 172.21.15.143:0/3127868899 conn(0x7fbd1c06c3f0 msgr2=0x7fbd1c13fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 shutdown_connections 2024-01-15T18:16:19.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:19.406+0000 7fbd21800640 1 -- 172.21.15.143:0/3127868899 wait complete. 2024-01-15T18:16:19.424 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c 2024-01-15T18:16:19.424 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:16:19.424 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:16:19.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:19.433 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:16:26.603 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:16:26.604 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:16:26.604 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.10447 s, 151 MB/s 2024-01-15T18:16:26.610 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:16:26.610 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:16:26.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:26.619 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:16:29.483 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:29.488 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:16:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:16:33.529 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.85624 s, 157 MB/s 2024-01-15T18:16:33.530 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:16:33.530 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:16:33.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:33.539 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:16:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:16:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:16:40.642 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.04877 s, 152 MB/s 2024-01-15T18:16:40.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T18:16:40.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 Processor -- start 2024-01-15T18:16:40.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 -- start start 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9fb0143780 0x7f9fb00cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9fb00720f0 0x7f9fb015a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb0072580 con 0x7f9fb0143780 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00ceb50 con 0x7f9fb00720f0 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb7600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb015c110 con 0x7f9fb00ce520 2024-01-15T18:16:40.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9fb0143780 0x7f9fb00cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9fb0143780 0x7f9fb00cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43826/0 (socket says 172.21.15.143:43826) 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 -- 172.21.15.143:0/1660037700 learned_addr learned my addr 172.21.15.143:0/1660037700 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fafe00640 1 --2- 172.21.15.143:0/1660037700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb5600640 1 --2- 172.21.15.143:0/1660037700 >> v2:172.21.15.204:3300/0 conn(0x7f9fb00720f0 0x7f9fb015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 -- 172.21.15.143:0/1660037700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 msgr2=0x7f9fb0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 --2- 172.21.15.143:0/1660037700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 -- 172.21.15.143:0/1660037700 >> v2:172.21.15.204:3300/0 conn(0x7f9fb00720f0 msgr2=0x7f9fb015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:40.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 --2- 172.21.15.143:0/1660037700 >> v2:172.21.15.204:3300/0 conn(0x7f9fb00720f0 0x7f9fb015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.785+0000 7f9fb4c00640 1 -- 172.21.15.143:0/1660037700 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb015b7e0 con 0x7f9fb0143780 2024-01-15T18:16:40.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb4c00640 1 --2- 172.21.15.143:0/1660037700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00cdfe0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00b1da0 tx=0x7f9fa001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1cf3657cef0e1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:40.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9faf400640 1 -- 172.21.15.143:0/1660037700 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00ac630 con 0x7f9fb0143780 2024-01-15T18:16:40.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9faf400640 1 -- 172.21.15.143:0/1660037700 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa00ac790 con 0x7f9fb0143780 2024-01-15T18:16:40.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9faf400640 1 -- 172.21.15.143:0/1660037700 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa00acb20 con 0x7f9fb0143780 2024-01-15T18:16:40.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 -- 172.21.15.143:0/1660037700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 msgr2=0x7f9fb00cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:40.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 --2- 172.21.15.143:0/1660037700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00cdfe0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00b1da0 tx=0x7f9fa001c500 comp rx=0 tx=0).stop 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 -- 172.21.15.143:0/1660037700 shutdown_connections 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 --2- 172.21.15.143:0/1660037700 >> v2:172.21.15.204:3300/0 conn(0x7f9fb00720f0 0x7f9fb015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 --2- 172.21.15.143:0/1660037700 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 --2- 172.21.15.143:0/1660037700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00cdfe0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 -- 172.21.15.143:0/1660037700 >> 172.21.15.143:0/1660037700 conn(0x7f9fb006c3f0 msgr2=0x7f9fb006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 -- 172.21.15.143:0/1660037700 shutdown_connections 2024-01-15T18:16:40.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.789+0000 7f9fb7600640 1 -- 172.21.15.143:0/1660037700 wait complete. 2024-01-15T18:16:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 Processor -- start 2024-01-15T18:16:40.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 -- start start 2024-01-15T18:16:40.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fb00720f0 0x7f9fb013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00b3da0 con 0x7f9fb0143780 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00b4080 con 0x7f9fb00720f0 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb7600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb0140b80 con 0x7f9fb00ce520 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fb00720f0 0x7f9fb013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43832/0 (socket says 172.21.15.143:43832) 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 -- 172.21.15.143:0/3930586740 learned_addr learned my addr 172.21.15.143:0/3930586740 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fafe00640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 -- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 msgr2=0x7f9fb0141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 -- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fb00720f0 msgr2=0x7f9fb013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fb00720f0 0x7f9fb013a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 -- 172.21.15.143:0/3930586740 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa00094d0 con 0x7f9fb0143780 2024-01-15T18:16:40.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.793+0000 7f9fb5600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00b33e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8007d10 tx=0x7f9fa800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.797+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8014420 con 0x7f9fb0143780 2024-01-15T18:16:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.797+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa80145a0 con 0x7f9fb0143780 2024-01-15T18:16:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.797+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8014950 con 0x7f9fb0143780 2024-01-15T18:16:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.797+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb0140e60 con 0x7f9fb0143780 2024-01-15T18:16:40.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.797+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb0159630 con 0x7f9fb0143780 2024-01-15T18:16:40.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.797+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb01409c0 con 0x7f9fb0143780 2024-01-15T18:16:40.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.801+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 31) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f9fa8014ab0 con 0x7f9fb0143780 2024-01-15T18:16:40.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.801+0000 7f9facc00640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f88067670 0x7f9f88069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:40.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.801+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9fa8083900 con 0x7f9fb0143780 2024-01-15T18:16:40.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.801+0000 7f9fb4c00640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f88067670 0x7f9f88069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:40.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.801+0000 7f9fb4c00640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f88067670 0x7f9f88069b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ae000 tx=0x7f9fa00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.809+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9fa8052340 con 0x7f9fb0143780 2024-01-15T18:16:40.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.961+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f9fb0067500 con 0x7f9fb0143780 2024-01-15T18:16:40.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.965+0000 7f9facc00640 1 -- 172.21.15.143:0/3930586740 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v31)=0 v31) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9fa8055960 con 0x7f9fb0143780 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f88067670 msgr2=0x7f9f88069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f88067670 0x7f9f88069b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ae000 tx=0x7f9fa00a7000 comp rx=0 tx=0).stop 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 msgr2=0x7f9fb00b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00b33e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8007d10 tx=0x7f9fa800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 shutdown_connections 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f88067670 0x7f9f88069b30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9fb0143780 0x7f9fb00b33e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9fb00ce520 0x7f9fb0141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 --2- 172.21.15.143:0/3930586740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9fb00720f0 0x7f9fb013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 >> 172.21.15.143:0/3930586740 conn(0x7f9fb006c3f0 msgr2=0x7f9fb013d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 shutdown_connections 2024-01-15T18:16:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:40.969+0000 7f9fb7600640 1 -- 172.21.15.143:0/3930586740 wait complete. 2024-01-15T18:16:40.987 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:16:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 Processor -- start 2024-01-15T18:16:41.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 -- start start 2024-01-15T18:16:41.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff50c157300 0x7ff50c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b21d0 0x7ff50c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff50c0b48f0 0x7ff50c15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c0b35a0 con 0x7ff50c157300 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c0b4d80 con 0x7ff50c0b48f0 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff513c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0b2a20 con 0x7ff50c0b21d0 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff50c157300 0x7ff50c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff50c157300 0x7ff50c159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43848/0 (socket says 172.21.15.143:43848) 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 -- 172.21.15.143:0/4270988456 learned_addr learned my addr 172.21.15.143:0/4270988456 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff50be00640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b21d0 0x7ff50c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511c00640 1 --2- 172.21.15.143:0/4270988456 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0b48f0 0x7ff50c15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 -- 172.21.15.143:0/4270988456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b21d0 msgr2=0x7ff50c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b21d0 0x7ff50c0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 -- 172.21.15.143:0/4270988456 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0b48f0 msgr2=0x7ff50c15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 --2- 172.21.15.143:0/4270988456 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0b48f0 0x7ff50c15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511200640 1 -- 172.21.15.143:0/4270988456 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0b2d00 con 0x7ff50c157300 2024-01-15T18:16:41.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff511c00640 1 --2- 172.21.15.143:0/4270988456 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0b48f0 0x7ff50c15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.133+0000 7ff50be00640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b21d0 0x7ff50c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff511200640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c159b70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff50c15a0b0 tx=0x7ff4fc01e980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42005b1dc2bbb953 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:41.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff50b400640 1 -- 172.21.15.143:0/4270988456 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc004230 con 0x7ff50c157300 2024-01-15T18:16:41.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff50b400640 1 -- 172.21.15.143:0/4270988456 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0ab070 con 0x7ff50c157300 2024-01-15T18:16:41.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff50b400640 1 -- 172.21.15.143:0/4270988456 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0a7810 con 0x7ff50c157300 2024-01-15T18:16:41.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 -- 172.21.15.143:0/4270988456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 msgr2=0x7ff50c159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c159b70 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff50c15a0b0 tx=0x7ff4fc01e980 comp rx=0 tx=0).stop 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 -- 172.21.15.143:0/4270988456 shutdown_connections 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 --2- 172.21.15.143:0/4270988456 >> v2:172.21.15.204:3300/0 conn(0x7ff50c0b48f0 0x7ff50c15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b21d0 0x7ff50c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 --2- 172.21.15.143:0/4270988456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c159b70 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 -- 172.21.15.143:0/4270988456 >> 172.21.15.143:0/4270988456 conn(0x7ff50c06c3f0 msgr2=0x7ff50c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 -- 172.21.15.143:0/4270988456 shutdown_connections 2024-01-15T18:16:41.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.137+0000 7ff513c00640 1 -- 172.21.15.143:0/4270988456 wait complete. 2024-01-15T18:16:41.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 Processor -- start 2024-01-15T18:16:41.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- start start 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c0b21d0 0x7ff50c152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b48f0 0x7ff50c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1592f0 con 0x7ff50c157300 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1595d0 con 0x7ff50c0b21d0 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0af420 con 0x7ff50c0b48f0 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c14e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43850/0 (socket says 172.21.15.143:43850) 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 -- 172.21.15.143:0/210260637 learned_addr learned my addr 172.21.15.143:0/210260637 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff50be00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b48f0 0x7ff50c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511200640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c0b21d0 0x7ff50c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 -- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b48f0 msgr2=0x7ff50c14b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b48f0 0x7ff50c14b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 -- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c0b21d0 msgr2=0x7ff50c152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c0b21d0 0x7ff50c152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 -- 172.21.15.143:0/210260637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc00b780 con 0x7ff50c157300 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511200640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c0b21d0 0x7ff50c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff50be00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b48f0 0x7ff50c14b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff511c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c14e1c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff504007e60 tx=0x7ff50400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504003ba0 con 0x7ff50c157300 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50400f040 con 0x7ff50c157300 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c0af700 con 0x7ff50c157300 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400de80 con 0x7ff50c157300 2024-01-15T18:16:41.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c152ed0 con 0x7ff50c157300 2024-01-15T18:16:41.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.141+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c0afc20 con 0x7ff50c157300 2024-01-15T18:16:41.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.145+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 31) v1 ==== 87384+0+0 (secure 0 0 0) 0x7ff504004030 con 0x7ff50c157300 2024-01-15T18:16:41.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.145+0000 7ff508c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40676a0 0x7ff4e4069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.145+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5794+0+0 (secure 0 0 0) 0x7ff5040834b0 con 0x7ff50c157300 2024-01-15T18:16:41.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.145+0000 7ff511200640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40676a0 0x7ff4e4069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.145+0000 7ff511200640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40676a0 0x7ff4e4069b60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0b1000 tx=0x7ff4fc0a9070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:41.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.149+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff50c0afc20 con 0x7ff50c157300 2024-01-15T18:16:41.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.321+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff50c159740 con 0x7ff4e40676a0 2024-01-15T18:16:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff508c00640 1 -- 172.21.15.143:0/210260637 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff50c159740 con 0x7ff4e40676a0 2024-01-15T18:16:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40676a0 msgr2=0x7ff4e4069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40676a0 0x7ff4e4069b60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0b1000 tx=0x7ff4fc0a9070 comp rx=0 tx=0).stop 2024-01-15T18:16:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 msgr2=0x7ff50c14e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c14e1c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff504007e60 tx=0x7ff50400d4a0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 shutdown_connections 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff4e40676a0 0x7ff4e4069b60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff50c157300 0x7ff50c14e1c0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff50c0b48f0 0x7ff50c14b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 --2- 172.21.15.143:0/210260637 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff50c0b21d0 0x7ff50c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.337+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 >> 172.21.15.143:0/210260637 conn(0x7ff50c06c3f0 msgr2=0x7ff50c0ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.341+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 shutdown_connections 2024-01-15T18:16:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.341+0000 7ff513c00640 1 -- 172.21.15.143:0/210260637 wait complete. 2024-01-15T18:16:41.358 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:16:41.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.497+0000 7f7194000640 1 Processor -- start 2024-01-15T18:16:41.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7194000640 1 -- start start 2024-01-15T18:16:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7194000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f718c143780 0x7f718c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7194000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f718c0722c0 0x7f718c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7194000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718c0ceb50 con 0x7f718c0ce520 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7194000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c15c110 con 0x7f718c143780 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7194000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c15b7e0 con 0x7f718c0722c0 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f718c143780 0x7f718c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f718c143780 0x7f718c0cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43862/0 (socket says 172.21.15.143:43862) 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 -- 172.21.15.143:0/1998252501 learned_addr learned my addr 172.21.15.143:0/1998252501 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7192000640 1 --2- 172.21.15.143:0/1998252501 >> v2:172.21.15.204:3300/0 conn(0x7f718c0722c0 0x7f718c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 -- 172.21.15.143:0/1998252501 >> v2:172.21.15.204:3300/0 conn(0x7f718c0722c0 msgr2=0x7f718c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 --2- 172.21.15.143:0/1998252501 >> v2:172.21.15.204:3300/0 conn(0x7f718c0722c0 0x7f718c15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 -- 172.21.15.143:0/1998252501 >> v1:172.21.15.143:6789/0 conn(0x7f718c0ce520 legacy=0x7f718c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7191600640 1 -- 172.21.15.143:0/1998252501 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c15bac0 con 0x7f718c143780 2024-01-15T18:16:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.501+0000 7f7192000640 1 --2- 172.21.15.143:0/1998252501 >> v2:172.21.15.204:3300/0 conn(0x7f718c0722c0 0x7f718c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7191600640 1 --2- 172.21.15.143:0/1998252501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c0cdfe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f718c070a20 tx=0x7f717c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d6ece4cb5c0e70 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:41.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f718be00640 1 -- 172.21.15.143:0/1998252501 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0ab610 con 0x7f718c143780 2024-01-15T18:16:41.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f718be00640 1 -- 172.21.15.143:0/1998252501 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c0ab770 con 0x7f718c143780 2024-01-15T18:16:41.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f718be00640 1 -- 172.21.15.143:0/1998252501 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0abb00 con 0x7f718c143780 2024-01-15T18:16:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 -- 172.21.15.143:0/1998252501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 msgr2=0x7f718c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 --2- 172.21.15.143:0/1998252501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c0cdfe0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f718c070a20 tx=0x7f717c01c500 comp rx=0 tx=0).stop 2024-01-15T18:16:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 -- 172.21.15.143:0/1998252501 shutdown_connections 2024-01-15T18:16:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 --2- 172.21.15.143:0/1998252501 >> v2:172.21.15.204:3300/0 conn(0x7f718c0722c0 0x7f718c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 --2- 172.21.15.143:0/1998252501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c0cdfe0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 -- 172.21.15.143:0/1998252501 >> 172.21.15.143:0/1998252501 conn(0x7f718c06c3f0 msgr2=0x7f718c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 -- 172.21.15.143:0/1998252501 shutdown_connections 2024-01-15T18:16:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.505+0000 7f7194000640 1 -- 172.21.15.143:0/1998252501 wait complete. 2024-01-15T18:16:41.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 Processor -- start 2024-01-15T18:16:41.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- start start 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f718c0722c0 0x7f718c0b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f718c0ce520 0x7f718c160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c142cb0 con 0x7f718c143780 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c142f90 con 0x7f718c0ce520 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c1407d0 con 0x7f718c0722c0 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43876/0 (socket says 172.21.15.143:43876) 2024-01-15T18:16:41.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 -- 172.21.15.143:0/2339749221 learned_addr learned my addr 172.21.15.143:0/2339749221 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7190c00640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f718c0ce520 0x7f718c160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7191600640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f718c0722c0 0x7f718c0b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 -- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f718c0722c0 msgr2=0x7f718c0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f718c0722c0 0x7f718c0b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 -- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f718c0ce520 msgr2=0x7f718c160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f718c0ce520 0x7f718c160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 -- 172.21.15.143:0/2339749221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0094d0 con 0x7f718c143780 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7190c00640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f718c0ce520 0x7f718c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7191600640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f718c0722c0 0x7f718c0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:41.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7192000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c159200 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f7184007de0 tx=0x7f718400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:41.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840039a0 con 0x7f718c143780 2024-01-15T18:16:41.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718400f040 con 0x7f718c143780 2024-01-15T18:16:41.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c140ab0 con 0x7f718c143780 2024-01-15T18:16:41.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184003d40 con 0x7f718c143780 2024-01-15T18:16:41.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c0b35d0 con 0x7f718c143780 2024-01-15T18:16:41.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.509+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c067500 con 0x7f718c143780 2024-01-15T18:16:41.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.513+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 31) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f7184004030 con 0x7f718c143780 2024-01-15T18:16:41.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.513+0000 7f7189600640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71640676a0 0x7f7164069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:41.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.513+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f71840834f0 con 0x7f718c143780 2024-01-15T18:16:41.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.513+0000 7f7191600640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71640676a0 0x7f7164069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:41.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.513+0000 7f7191600640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71640676a0 0x7f7164069b60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f717c0af000 tx=0x7f717c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:41.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.517+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7184051ed0 con 0x7f718c143780 2024-01-15T18:16:41.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.689+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f718c140f60 con 0x7f71640676a0 2024-01-15T18:16:41.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.693+0000 7f7189600640 1 -- 172.21.15.143:0/2339749221 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f718c140f60 con 0x7f71640676a0 2024-01-15T18:16:41.693 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c 2024-01-15T18:16:41.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71640676a0 msgr2=0x7f7164069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71640676a0 0x7f7164069b60 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f717c0af000 tx=0x7f717c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 msgr2=0x7f718c159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c159200 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f7184007de0 tx=0x7f718400d4a0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 shutdown_connections 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f71640676a0 0x7f7164069b60 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f718c143780 0x7f718c159200 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f718c0ce520 0x7f718c160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 --2- 172.21.15.143:0/2339749221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f718c0722c0 0x7f718c0b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 >> 172.21.15.143:0/2339749221 conn(0x7f718c06c3f0 msgr2=0x7f718c06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 shutdown_connections 2024-01-15T18:16:41.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:41.697+0000 7f7194000640 1 -- 172.21.15.143:0/2339749221 wait complete. 2024-01-15T18:16:41.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:41.715 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c) 2024-01-15T18:16:41.727 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c 2024-01-15T18:16:41.727 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:16:41.728 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:41.728 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:16:43.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:16:43.730 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c) 2024-01-15T18:16:43.744 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/100f9e1f-4eff-4865-be3d-99f4692a8d2c 2024-01-15T18:16:43.744 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:16:43.745 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:16:43.745 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:16:43.746 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-15T18:16:43.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.941+0000 7fd224e00640 1 Processor -- start 2024-01-15T18:16:43.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.941+0000 7fd224e00640 1 -- start start 2024-01-15T18:16:43.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd224e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd220141660 0x7fd2200ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd224e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd22015ec80 0x7fd220141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd224e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd220142470 0x7fd220071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd224e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22015fbb0 con 0x7fd220142470 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd224e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22015f100 con 0x7fd220141660 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd224e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22015f310 con 0x7fd22015ec80 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd220142470 0x7fd220071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd220142470 0x7fd220071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43880/0 (socket says 172.21.15.143:43880) 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 -- 172.21.15.143:0/2222309750 learned_addr learned my addr 172.21.15.143:0/2222309750 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21d400640 1 --2- 172.21.15.143:0/2222309750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd22015ec80 0x7fd220141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21de00640 1 --2- 172.21.15.143:0/2222309750 >> v2:172.21.15.204:3300/0 conn(0x7fd220141660 0x7fd2200ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 -- 172.21.15.143:0/2222309750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd22015ec80 msgr2=0x7fd220141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 --2- 172.21.15.143:0/2222309750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd22015ec80 0x7fd220141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 -- 172.21.15.143:0/2222309750 >> v2:172.21.15.204:3300/0 conn(0x7fd220141660 msgr2=0x7fd2200ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 --2- 172.21.15.143:0/2222309750 >> v2:172.21.15.204:3300/0 conn(0x7fd220141660 0x7fd2200ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 -- 172.21.15.143:0/2222309750 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22015f5f0 con 0x7fd220142470 2024-01-15T18:16:43.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.945+0000 7fd21e800640 1 --2- 172.21.15.143:0/2222309750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd220142470 0x7fd220071d00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd2200b1aa0 tx=0x7fd21801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69ceb2455b312ae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:43.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd21ca00640 1 -- 172.21.15.143:0/2222309750 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180a5730 con 0x7fd220142470 2024-01-15T18:16:43.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd21ca00640 1 -- 172.21.15.143:0/2222309750 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2180a5890 con 0x7fd220142470 2024-01-15T18:16:43.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd21ca00640 1 -- 172.21.15.143:0/2222309750 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180a5c20 con 0x7fd220142470 2024-01-15T18:16:43.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- 172.21.15.143:0/2222309750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd220142470 msgr2=0x7fd220071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:43.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- 172.21.15.143:0/2222309750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd220142470 0x7fd220071d00 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fd2200b1aa0 tx=0x7fd21801c9a0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- 172.21.15.143:0/2222309750 shutdown_connections 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- 172.21.15.143:0/2222309750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd220142470 0x7fd220071d00 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- 172.21.15.143:0/2222309750 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd22015ec80 0x7fd220141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- 172.21.15.143:0/2222309750 >> v2:172.21.15.204:3300/0 conn(0x7fd220141660 0x7fd2200ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- 172.21.15.143:0/2222309750 >> 172.21.15.143:0/2222309750 conn(0x7fd22006c3f0 msgr2=0x7fd22006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- 172.21.15.143:0/2222309750 shutdown_connections 2024-01-15T18:16:43.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- 172.21.15.143:0/2222309750 wait complete. 2024-01-15T18:16:43.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 Processor -- start 2024-01-15T18:16:43.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- start start 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220141660 0x7fd2201581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd220142470 0x7fd2200ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 0x7fd2200ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd220140950 con 0x7fd22015ec80 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd220158e10 con 0x7fd220141660 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.949+0000 7fd224e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd220139170 con 0x7fd220142470 2024-01-15T18:16:43.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 0x7fd2200ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 0x7fd2200ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43888/0 (socket says 172.21.15.143:43888) 2024-01-15T18:16:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 -- 172.21.15.143:0/2383407321 learned_addr learned my addr 172.21.15.143:0/2383407321 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21de00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220141660 0x7fd2201581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21d400640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd220142470 0x7fd2200ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 -- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd220142470 msgr2=0x7fd2200ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd220142470 0x7fd2200ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 -- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220141660 msgr2=0x7fd2201581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220141660 0x7fd2201581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 -- 172.21.15.143:0/2383407321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd218009b30 con 0x7fd22015ec80 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd21e800640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 0x7fd2200ace20 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd2180af040 tx=0x7fd218019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180b8670 con 0x7fd22015ec80 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2180b87d0 con 0x7fd22015ec80 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180b8b80 con 0x7fd22015ec80 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2200cd150 con 0x7fd22015ec80 2024-01-15T18:16:43.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2201405e0 con 0x7fd22015ec80 2024-01-15T18:16:43.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd220140ac0 con 0x7fd22015ec80 2024-01-15T18:16:43.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.953+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 31) v1 ==== 87384+0+0 (secure 0 0 0) 0x7fd2180b8ce0 con 0x7fd22015ec80 2024-01-15T18:16:43.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.957+0000 7fd216000640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f8067670 0x7fd1f8069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:43.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.957+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fd21811e6b0 con 0x7fd22015ec80 2024-01-15T18:16:43.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.957+0000 7fd21de00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f8067670 0x7fd1f8069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:43.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.957+0000 7fd21de00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f8067670 0x7fd1f8069b30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd2100045f0 tx=0x7fd210008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:43.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:43.961+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2180ed320 con 0x7fd22015ec80 2024-01-15T18:16:44.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.137+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2200ccfd0 con 0x7fd1f8067670 2024-01-15T18:16:44.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.193+0000 7fd216000640 1 -- 172.21.15.143:0/2383407321 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd210004240 con 0x7fd1f8067670 2024-01-15T18:16:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f8067670 msgr2=0x7fd1f8069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f8067670 0x7fd1f8069b30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd2100045f0 tx=0x7fd210008040 comp rx=0 tx=0).stop 2024-01-15T18:16:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 msgr2=0x7fd2200ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 0x7fd2200ace20 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fd2180af040 tx=0x7fd218019740 comp rx=0 tx=0).stop 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 shutdown_connections 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f8067670 0x7fd1f8069b30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd22015ec80 0x7fd2200ace20 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd220142470 0x7fd2200ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 --2- 172.21.15.143:0/2383407321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd220141660 0x7fd2201581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 >> 172.21.15.143:0/2383407321 conn(0x7fd22006c3f0 msgr2=0x7fd220142850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 shutdown_connections 2024-01-15T18:16:44.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.197+0000 7fd224e00640 1 -- 172.21.15.143:0/2383407321 wait complete. 2024-01-15T18:16:44.196 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.197+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.197 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.197+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.197 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.197+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.197 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.197+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.197 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.197+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.212 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.213+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.212 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.213+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.212 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.213+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.212 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.213+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.212 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:16:44.213+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:16:44.219 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:16:44.219 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:44.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f9042000640 1 Processor -- start 2024-01-15T18:16:44.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f9042000640 1 -- start start 2024-01-15T18:16:44.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f9042000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f9042000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c0ceb50 con 0x7f903c071bb0 2024-01-15T18:16:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f9042000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c15c110 con 0x7f903c143780 2024-01-15T18:16:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f9042000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c0724c0 con 0x7f903c072030 2024-01-15T18:16:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f903aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f903c071bb0 0x7f903c15c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58660/0 (socket says 172.21.15.143:58660) 2024-01-15T18:16:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f903aa00640 1 -- 172.21.15.143:0/1403173712 learned_addr learned my addr 172.21.15.143:0/1403173712 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:44.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.361+0000 7f903be00640 1 --2- 172.21.15.143:0/1403173712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658460679 0 0) 0x7f9024001120 con 0x7f903c071bb0 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901c005740 con 0x7f903c071bb0 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607533464 0 0) 0x7f903c15c110 con 0x7f903c143780 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9024001120 con 0x7f903c143780 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903be00640 1 -- 172.21.15.143:0/1403173712 >> v1:172.21.15.204:6789/0 conn(0x7f903c143780 legacy=0x7f903c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903be00640 1 -- 172.21.15.143:0/1403173712 >> v1:172.21.15.143:6789/0 conn(0x7f903c071bb0 legacy=0x7f903c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903be00640 1 -- 172.21.15.143:0/1403173712 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c15b7e0 con 0x7f903c072030 2024-01-15T18:16:44.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903be00640 1 --2- 172.21.15.143:0/1403173712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c15ccb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9034009dc0 tx=0x7f903401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f843241822947c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:44.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340a5a30 con 0x7f903c072030 2024-01-15T18:16:44.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90340a5b90 con 0x7f903c072030 2024-01-15T18:16:44.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.365+0000 7f903a000640 1 -- 172.21.15.143:0/1403173712 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340ad440 con 0x7f903c072030 2024-01-15T18:16:44.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- 172.21.15.143:0/1403173712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 msgr2=0x7f903c15ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 --2- 172.21.15.143:0/1403173712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c15ccb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9034009dc0 tx=0x7f903401c970 comp rx=0 tx=0).stop 2024-01-15T18:16:44.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- 172.21.15.143:0/1403173712 shutdown_connections 2024-01-15T18:16:44.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 --2- 172.21.15.143:0/1403173712 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c15ccb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- 172.21.15.143:0/1403173712 >> 172.21.15.143:0/1403173712 conn(0x7f903c06c3f0 msgr2=0x7f903c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:44.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- 172.21.15.143:0/1403173712 shutdown_connections 2024-01-15T18:16:44.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- 172.21.15.143:0/1403173712 wait complete. 2024-01-15T18:16:44.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 Processor -- start 2024-01-15T18:16:44.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- start start 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f903c071bb0 0x7f903c13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 0x7f903c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c13ebf0 con 0x7f903c143780 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c140f60 con 0x7f903c071bb0 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f9042000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c139400 con 0x7f903c072030 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f903be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 0x7f903c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f903be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 0x7f903c158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43942/0 (socket says 172.21.15.143:43942) 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f903be00640 1 -- 172.21.15.143:0/371270411 learned_addr learned my addr 172.21.15.143:0/371270411 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f903aa00640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.369+0000 7f903b400640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f903c071bb0 0x7f903c13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f903be00640 1 -- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 msgr2=0x7f903c0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f903be00640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c0acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f903be00640 1 -- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f903c071bb0 msgr2=0x7f903c13e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f903be00640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f903c071bb0 0x7f903c13e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f903be00640 1 -- 172.21.15.143:0/371270411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9034009a70 con 0x7f903c143780 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f903be00640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 0x7f903c158720 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9034019580 tx=0x7f9034019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340aed30 con 0x7f903c143780 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90340aee90 con 0x7f903c143780 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c1403e0 con 0x7f903c143780 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90340b7610 con 0x7f903c143780 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c1409a0 con 0x7f903c143780 2024-01-15T18:16:44.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c067500 con 0x7f903c143780 2024-01-15T18:16:44.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 31) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f90340b4050 con 0x7f903c143780 2024-01-15T18:16:44.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9033400640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9014067670 0x7f9014069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.373+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f903411e220 con 0x7f903c143780 2024-01-15T18:16:44.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.377+0000 7f903b400640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9014067670 0x7f9014069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.377+0000 7f903b400640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9014067670 0x7f9014069b30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f903c0ad930 tx=0x7f902c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:44.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.381+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90340eccb0 con 0x7f903c143780 2024-01-15T18:16:44.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.557+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f903c070aa0 con 0x7f9014067670 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9033400640 1 -- 172.21.15.143:0/371270411 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f902c0013e0 con 0x7f9014067670 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:44.562 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:44.563 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:44.563 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9014067670 msgr2=0x7f9014069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9014067670 0x7f9014069b30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f903c0ad930 tx=0x7f902c009040 comp rx=0 tx=0).stop 2024-01-15T18:16:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 msgr2=0x7f903c158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 0x7f903c158720 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f9034019580 tx=0x7f9034019710 comp rx=0 tx=0).stop 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 shutdown_connections 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9014067670 0x7f9014069b30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f903c143780 0x7f903c158720 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f903c072030 0x7f903c0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 --2- 172.21.15.143:0/371270411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f903c071bb0 0x7f903c13e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 >> 172.21.15.143:0/371270411 conn(0x7f903c06c3f0 msgr2=0x7f903c0cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 shutdown_connections 2024-01-15T18:16:44.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:44.565+0000 7f9042000640 1 -- 172.21.15.143:0/371270411 wait complete. 2024-01-15T18:16:44.584 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:45.585 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:45.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.737+0000 7f0e7a000640 1 Processor -- start 2024-01-15T18:16:45.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.737+0000 7f0e7a000640 1 -- start start 2024-01-15T18:16:45.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.737+0000 7f0e7a000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0e74143780 0x7f0e740cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.737+0000 7f0e7a000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e740ceb50 con 0x7f0e740ce520 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.737+0000 7f0e7a000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7415c110 con 0x7f0e740722c0 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.737+0000 7f0e7a000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7415b7e0 con 0x7f0e74143780 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0e740ce520 0x7f0e74071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58672/0 (socket says 172.21.15.143:58672) 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72a00640 1 -- 172.21.15.143:0/1429958575 learned_addr learned my addr 172.21.15.143:0/1429958575 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206805787 0 0) 0x7f0e5c001120 con 0x7f0e740ce520 2024-01-15T18:16:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e54005740 con 0x7f0e740ce520 2024-01-15T18:16:45.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73400640 1 --2- 172.21.15.143:0/1429958575 >> v2:172.21.15.204:3300/0 conn(0x7f0e74143780 0x7f0e740cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2559295705 0 0) 0x7f0e54005740 con 0x7f0e740ce520 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 >> v2:172.21.15.204:3300/0 conn(0x7f0e74143780 msgr2=0x7f0e740cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 --2- 172.21.15.143:0/1429958575 >> v2:172.21.15.204:3300/0 conn(0x7f0e74143780 0x7f0e740cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 >> v1:172.21.15.204:6789/0 conn(0x7f0e740722c0 legacy=0x7f0e7415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7415bac0 con 0x7f0e740ce520 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f0e5c001810 con 0x7f0e740ce520 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73400640 1 --2- 172.21.15.143:0/1429958575 >> v2:172.21.15.204:3300/0 conn(0x7f0e74143780 0x7f0e740cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 >> v1:172.21.15.143:6789/0 conn(0x7f0e740ce520 legacy=0x7f0e74071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e54005be0 0x7f0e54006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e540069e0 0x7f0e54006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e540073f0 0x7f0e5400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7415b7e0 con 0x7f0e540073f0 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5400e340 con 0x7f0e540069e0 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5400e500 con 0x7f0e54005be0 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73e00640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e540073f0 0x7f0e5400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e72a00640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e540069e0 0x7f0e54006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73400640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e54005be0 0x7f0e54006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73e00640 1 -- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e54005be0 msgr2=0x7f0e54006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73e00640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e54005be0 0x7f0e54006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73e00640 1 -- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e540069e0 msgr2=0x7f0e54006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73e00640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e540069e0 0x7f0e54006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.741+0000 7f0e73e00640 1 -- 172.21.15.143:0/1429958575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e54005740 con 0x7f0e540073f0 2024-01-15T18:16:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e73e00640 1 -- 172.21.15.143:0/1429958575 reap_dead start 2024-01-15T18:16:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e73e00640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e540073f0 0x7f0e5400dba0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c001e70 tx=0x7f0e6c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae000b7e523582 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c003b80 con 0x7f0e540073f0 2024-01-15T18:16:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c003d20 con 0x7f0e540073f0 2024-01-15T18:16:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e72000640 1 -- 172.21.15.143:0/1429958575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c005260 con 0x7f0e540073f0 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 -- 172.21.15.143:0/1429958575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e540073f0 msgr2=0x7f0e5400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e540073f0 0x7f0e5400dba0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c001e70 tx=0x7f0e6c01c760 comp rx=0 tx=0).stop 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 -- 172.21.15.143:0/1429958575 shutdown_connections 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 --2- 172.21.15.143:0/1429958575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e540073f0 0x7f0e5400dba0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 -- 172.21.15.143:0/1429958575 >> 172.21.15.143:0/1429958575 conn(0x7f0e7406c3f0 msgr2=0x7f0e7406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 -- 172.21.15.143:0/1429958575 shutdown_connections 2024-01-15T18:16:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 -- 172.21.15.143:0/1429958575 wait complete. 2024-01-15T18:16:45.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 Processor -- start 2024-01-15T18:16:45.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.745+0000 7f0e7a000640 1 -- start start 2024-01-15T18:16:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e540073f0 0x7f0e74141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 0x7f0e74142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e741f28c0 0x7f0e741f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e741f4510 con 0x7f0e7414c770 2024-01-15T18:16:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e741f37c0 con 0x7f0e741f28c0 2024-01-15T18:16:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e741f3aa0 con 0x7f0e540073f0 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 0x7f0e74142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 0x7f0e74142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43970/0 (socket says 172.21.15.143:43970) 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 -- 172.21.15.143:0/2073516575 learned_addr learned my addr 172.21.15.143:0/2073516575 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e73e00640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e741f28c0 0x7f0e741f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e73400640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e540073f0 0x7f0e74141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 -- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e540073f0 msgr2=0x7f0e74141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e540073f0 0x7f0e74141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 -- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e741f28c0 msgr2=0x7f0e741f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e741f28c0 0x7f0e741f2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 -- 172.21.15.143:0/2073516575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e741f3d80 con 0x7f0e7414c770 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e72a00640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 0x7f0e74142050 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c001940 tx=0x7f0e5c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00f730 con 0x7f0e7414c770 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c00f890 con 0x7f0e7414c770 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e741f4060 con 0x7f0e7414c770 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c00fc40 con 0x7f0e7414c770 2024-01-15T18:16:45.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e741f3450 con 0x7f0e7414c770 2024-01-15T18:16:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.749+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e741f4680 con 0x7f0e7414c770 2024-01-15T18:16:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.753+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 31) v1 ==== 87384+0+0 (secure 0 0 0) 0x7f0e5c01d460 con 0x7f0e7414c770 2024-01-15T18:16:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.753+0000 7f0e6b400640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e4c067670 0x7f0e4c069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.753+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f0e5c082e80 con 0x7f0e7414c770 2024-01-15T18:16:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.753+0000 7f0e73400640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e4c067670 0x7f0e4c069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.753+0000 7f0e73400640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e4c067670 0x7f0e4c069b30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0e64006f90 tx=0x7f0e64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.757+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e741f4680 con 0x7f0e7414c770 2024-01-15T18:16:45.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.925+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f0e5c082a20 con 0x7f0e7414c770 2024-01-15T18:16:45.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.941+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e740b4ce0 con 0x7f0e4c067670 2024-01-15T18:16:45.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.945+0000 7f0e6b400640 1 -- 172.21.15.143:0/2073516575 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f0e64009e80 con 0x7f0e4c067670 2024-01-15T18:16:45.943 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:45.943 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:45.943 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:45.943 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:45.944 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:45.944 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:45.944 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:45.944 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:45.944 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:45.944 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:45.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e4c067670 msgr2=0x7f0e4c069b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e4c067670 0x7f0e4c069b30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0e64006f90 tx=0x7f0e64008040 comp rx=0 tx=0).stop 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 msgr2=0x7f0e74142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 0x7f0e74142050 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c001940 tx=0x7f0e5c0074f0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 shutdown_connections 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e4c067670 0x7f0e4c069b30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e741f28c0 0x7f0e741f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e7414c770 0x7f0e74142050 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 --2- 172.21.15.143:0/2073516575 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e540073f0 0x7f0e74141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 >> 172.21.15.143:0/2073516575 conn(0x7f0e7406c3f0 msgr2=0x7f0e740b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 shutdown_connections 2024-01-15T18:16:45.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:45.949+0000 7f0e7a000640 1 -- 172.21.15.143:0/2073516575 wait complete. 2024-01-15T18:16:45.965 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:46.966 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.125+0000 7f92a9a00640 1 Processor -- start 2024-01-15T18:16:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.125+0000 7f92a9a00640 1 -- start start 2024-01-15T18:16:47.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.125+0000 7f92a9a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f92a415d3f0 0x7f92a40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.125+0000 7f92a9a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40cb480 con 0x7f92a406c630 2024-01-15T18:16:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.125+0000 7f92a9a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40cb6d0 con 0x7f92a40d0050 2024-01-15T18:16:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.125+0000 7f92a9a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40cb9b0 con 0x7f92a415d3f0 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f92a406c630 0x7f92a40b1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:58674/0 (socket says 172.21.15.143:58674) 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 -- 172.21.15.143:0/1106228588 learned_addr learned my addr 172.21.15.143:0/1106228588 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084300072 0 0) 0x7f92a40cb480 con 0x7f92a406c630 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a2a00640 1 --2- 172.21.15.143:0/1106228588 >> v2:172.21.15.204:3300/0 conn(0x7f92a415d3f0 0x7f92a40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9284003530 con 0x7f92a406c630 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1064707189 0 0) 0x7f9284003530 con 0x7f92a406c630 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 >> v2:172.21.15.204:3300/0 conn(0x7f92a415d3f0 msgr2=0x7f92a40cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 --2- 172.21.15.143:0/1106228588 >> v2:172.21.15.204:3300/0 conn(0x7f92a415d3f0 0x7f92a40cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 >> v1:172.21.15.204:6789/0 conn(0x7f92a40d0050 legacy=0x7f92a406bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40cbc90 con 0x7f92a406c630 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f929c001a00 con 0x7f92a406c630 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 >> v1:172.21.15.143:6789/0 conn(0x7f92a406c630 legacy=0x7f92a40b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284003980 0x7f9284003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92840047d0 0x7f9284004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9284005190 0x7f928400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40cb9b0 con 0x7f9284005190 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928400c030 con 0x7f92840047d0 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928400c1f0 con 0x7f9284003980 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9284005190 0x7f928400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a2a00640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284003980 0x7f9284003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a2000640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92840047d0 0x7f9284004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 -- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284003980 msgr2=0x7f9284003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284003980 0x7f9284003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 -- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92840047d0 msgr2=0x7f9284004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92840047d0 0x7f9284004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.129+0000 7f92a3400640 1 -- 172.21.15.143:0/1106228588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284003530 con 0x7f9284005190 2024-01-15T18:16:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a3400640 1 -- 172.21.15.143:0/1106228588 reap_dead start 2024-01-15T18:16:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a3400640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9284005190 0x7f928400b990 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f92a40712e0 tx=0x7f929c01d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426bf4857156082c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c0072d0 con 0x7f9284005190 2024-01-15T18:16:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c0a8070 con 0x7f9284005190 2024-01-15T18:16:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a1600640 1 -- 172.21.15.143:0/1106228588 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c007670 con 0x7f9284005190 2024-01-15T18:16:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 -- 172.21.15.143:0/1106228588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9284005190 msgr2=0x7f928400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9284005190 0x7f928400b990 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f92a40712e0 tx=0x7f929c01d2e0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 -- 172.21.15.143:0/1106228588 shutdown_connections 2024-01-15T18:16:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 --2- 172.21.15.143:0/1106228588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9284005190 0x7f928400b990 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 -- 172.21.15.143:0/1106228588 >> 172.21.15.143:0/1106228588 conn(0x7f92a40d0e30 msgr2=0x7f92a40d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 -- 172.21.15.143:0/1106228588 shutdown_connections 2024-01-15T18:16:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 -- 172.21.15.143:0/1106228588 wait complete. 2024-01-15T18:16:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 Processor -- start 2024-01-15T18:16:47.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.133+0000 7f92a9a00640 1 -- start start 2024-01-15T18:16:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284005190 0x7f92a415f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a40b63c0 0x7f92a40ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 0x7f92a4151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a415e750 con 0x7f92a40b6840 2024-01-15T18:16:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a415ea30 con 0x7f92a40b63c0 2024-01-15T18:16:47.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a415ed10 con 0x7f9284005190 2024-01-15T18:16:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 0x7f92a4151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 0x7f92a4151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43994/0 (socket says 172.21.15.143:43994) 2024-01-15T18:16:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a3400640 1 -- 172.21.15.143:0/2364472809 learned_addr learned my addr 172.21.15.143:0/2364472809 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a40b63c0 0x7f92a40ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284005190 0x7f92a415f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 -- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284005190 msgr2=0x7f92a415f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284005190 0x7f92a415f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 -- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 msgr2=0x7f92a4151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 0x7f92a4151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 -- 172.21.15.143:0/2364472809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a415eff0 con 0x7f92a40b63c0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a3400640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 0x7f92a4151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a2000640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a40b63c0 0x7f92a40ca0b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f928c00f4f0 tx=0x7f928c00f8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c0053a0 con 0x7f92a40b63c0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c003600 con 0x7f92a40b63c0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a415f2d0 con 0x7f92a40b63c0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c005500 con 0x7f92a40b63c0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a415fd30 con 0x7f92a40b63c0 2024-01-15T18:16:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.137+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a4067500 con 0x7f92a40b63c0 2024-01-15T18:16:47.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.141+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f928c003990 con 0x7f92a40b63c0 2024-01-15T18:16:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.141+0000 7f929aa00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f927c0677c0 0x7f927c069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.141+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f928c082c20 con 0x7f92a40b63c0 2024-01-15T18:16:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.141+0000 7f92a2a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f927c0677c0 0x7f927c069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.141+0000 7f92a2a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f927c0677c0 0x7f927c069c80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9294009b60 tx=0x7f9294009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:47.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.145+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f928c0518b0 con 0x7f92a40b63c0 2024-01-15T18:16:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.329+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f92a415d7d0 con 0x7f927c0677c0 2024-01-15T18:16:47.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.333+0000 7f929aa00640 1 -- 172.21.15.143:0/2364472809 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f9294019070 con 0x7f927c0677c0 2024-01-15T18:16:47.332 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:47.332 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:47.332 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:47.332 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:47.332 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:47.333 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:47.333 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:47.333 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:47.333 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:47.333 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:47.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f927c0677c0 msgr2=0x7f927c069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f927c0677c0 0x7f927c069c80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9294009b60 tx=0x7f9294009190 comp rx=0 tx=0).stop 2024-01-15T18:16:47.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a40b63c0 msgr2=0x7f92a40ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a40b63c0 0x7f92a40ca0b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f928c00f4f0 tx=0x7f928c00f8f0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 shutdown_connections 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f927c0677c0 0x7f927c069c80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f92a40b6840 0x7f92a4151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f92a40b63c0 0x7f92a40ca0b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 --2- 172.21.15.143:0/2364472809 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9284005190 0x7f92a415f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 >> 172.21.15.143:0/2364472809 conn(0x7f92a40d0e30 msgr2=0x7f92a415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 shutdown_connections 2024-01-15T18:16:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:47.337+0000 7f92a9a00640 1 -- 172.21.15.143:0/2364472809 wait complete. 2024-01-15T18:16:47.354 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:48.355 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.505+0000 7f061dc00640 1 Processor -- start 2024-01-15T18:16:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.505+0000 7f061dc00640 1 -- start start 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f061dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0618157300 0x7f0618159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f061dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b4ac0 0x7f0618070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f061dc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06180b2a20 con 0x7f06180b21d0 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f061dc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06180b35a0 con 0x7f0618157300 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f061dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06180b2d00 con 0x7f06180b4ac0 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0618157300 0x7f0618159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0618157300 0x7f0618159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60754/0 (socket says 172.21.15.143:60754) 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 -- 172.21.15.143:0/780281852 learned_addr learned my addr 172.21.15.143:0/780281852 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0617400640 1 --2- 172.21.15.143:0/780281852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b4ac0 0x7f0618070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 -- 172.21.15.143:0/780281852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b4ac0 msgr2=0x7f0618070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 --2- 172.21.15.143:0/780281852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b4ac0 0x7f0618070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 -- 172.21.15.143:0/780281852 >> v1:172.21.15.204:6789/0 conn(0x7f06180b21d0 legacy=0x7f06180b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 -- 172.21.15.143:0/780281852 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06180b2fe0 con 0x7f0618157300 2024-01-15T18:16:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.509+0000 7f0616a00640 1 --2- 172.21.15.143:0/780281852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f0618159b70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f0608009870 tx=0x7f060801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3bf1bb160793c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f0615600640 1 -- 172.21.15.143:0/780281852 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080a95f0 con 0x7f0618157300 2024-01-15T18:16:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f0615600640 1 -- 172.21.15.143:0/780281852 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06080a9750 con 0x7f0618157300 2024-01-15T18:16:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f0615600640 1 -- 172.21.15.143:0/780281852 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080a9ae0 con 0x7f0618157300 2024-01-15T18:16:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 -- 172.21.15.143:0/780281852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 msgr2=0x7f0618159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 --2- 172.21.15.143:0/780281852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f0618159b70 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f0608009870 tx=0x7f060801c510 comp rx=0 tx=0).stop 2024-01-15T18:16:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 -- 172.21.15.143:0/780281852 shutdown_connections 2024-01-15T18:16:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 --2- 172.21.15.143:0/780281852 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b4ac0 0x7f0618070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 --2- 172.21.15.143:0/780281852 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f0618159b70 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 -- 172.21.15.143:0/780281852 >> 172.21.15.143:0/780281852 conn(0x7f061806c3f0 msgr2=0x7f061806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 -- 172.21.15.143:0/780281852 shutdown_connections 2024-01-15T18:16:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 -- 172.21.15.143:0/780281852 wait complete. 2024-01-15T18:16:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 Processor -- start 2024-01-15T18:16:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 -- start start 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b21d0 0x7f06180ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06180b4ac0 0x7f06180ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.513+0000 7f061dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f06180adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f061dc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06180c49e0 con 0x7f0618157300 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f061dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06180c4cc0 con 0x7f06180b4ac0 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f061dc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06180c4fa0 con 0x7f06180b21d0 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f06180adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f06180adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60770/0 (socket says 172.21.15.143:60770) 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 -- 172.21.15.143:0/3542844015 learned_addr learned my addr 172.21.15.143:0/3542844015 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0616a00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b21d0 0x7f06180ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0616000640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06180b4ac0 0x7f06180ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 -- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b21d0 msgr2=0x7f06180ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b21d0 0x7f06180ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 -- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06180b4ac0 msgr2=0x7f06180ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06180b4ac0 0x7f06180ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 -- 172.21.15.143:0/3542844015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06080094d0 con 0x7f0618157300 2024-01-15T18:16:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f0617400640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f06180adda0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0610007d40 tx=0x7f061000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610014420 con 0x7f0618157300 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06100145a0 con 0x7f0618157300 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610014950 con 0x7f0618157300 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06180c5280 con 0x7f0618157300 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0618153db0 con 0x7f0618157300 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06180c5470 con 0x7f0618157300 2024-01-15T18:16:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.517+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f0610014ab0 con 0x7f0618157300 2024-01-15T18:16:48.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.521+0000 7f060ea00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f0067790 0x7f05f0069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:48.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.521+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f06100839d0 con 0x7f0618157300 2024-01-15T18:16:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.521+0000 7f0616a00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f0067790 0x7f05f0069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.521+0000 7f0616a00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f0067790 0x7f05f0069c50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f06080ae000 tx=0x7f06080b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:48.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.525+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f06100523b0 con 0x7f0618157300 2024-01-15T18:16:48.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.701+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f06180b4180 con 0x7f05f0067790 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f060ea00640 1 -- 172.21.15.143:0/3542844015 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f06180b4180 con 0x7f05f0067790 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:48.705 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:48.706 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:48.706 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:48.706 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:48.706 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:48.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f0067790 msgr2=0x7f05f0069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:48.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f0067790 0x7f05f0069c50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f06080ae000 tx=0x7f06080b3040 comp rx=0 tx=0).stop 2024-01-15T18:16:48.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 msgr2=0x7f06180adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:48.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f06180adda0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0610007d40 tx=0x7f061000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 shutdown_connections 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05f0067790 0x7f05f0069c50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0618157300 0x7f06180adda0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06180b4ac0 0x7f06180ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 --2- 172.21.15.143:0/3542844015 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f06180b21d0 0x7f06180ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 >> 172.21.15.143:0/3542844015 conn(0x7f061806c3f0 msgr2=0x7f06180701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 shutdown_connections 2024-01-15T18:16:48.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:48.709+0000 7f061dc00640 1 -- 172.21.15.143:0/3542844015 wait complete. 2024-01-15T18:16:48.728 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:49.729 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:49.883 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.885+0000 7f14e9400640 1 Processor -- start 2024-01-15T18:16:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.885+0000 7f14e9400640 1 -- start start 2024-01-15T18:16:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.885+0000 7f14e9400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 0x7f14e4167ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.885+0000 7f14e9400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e415e6c0 con 0x7f14e4071300 2024-01-15T18:16:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.885+0000 7f14e9400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e406bd40 con 0x7f14e4070f20 2024-01-15T18:16:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.885+0000 7f14e9400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e406a490 con 0x7f14e41578d0 2024-01-15T18:16:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f14e4071300 0x7f14e41571c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39118/0 (socket says 172.21.15.143:39118) 2024-01-15T18:16:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1e00640 1 -- 172.21.15.143:0/2330625377 learned_addr learned my addr 172.21.15.143:0/2330625377 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 0x7f14e4167ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469577838 0 0) 0x7f14e415e6c0 con 0x7f14e4071300 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c4003530 con 0x7f14e4071300 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389762034 0 0) 0x7f14e406bd40 con 0x7f14e4070f20 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e415e6c0 con 0x7f14e4070f20 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1879270316 0 0) 0x7f14c4003530 con 0x7f14e4071300 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 msgr2=0x7f14e4167ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 0x7f14e4167ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 >> v1:172.21.15.204:6789/0 conn(0x7f14e4070f20 legacy=0x7f14e406b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e406a770 con 0x7f14e4071300 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f14cc001680 con 0x7f14e4071300 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 >> v1:172.21.15.143:6789/0 conn(0x7f14e4071300 legacy=0x7f14e41571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 0x7f14c4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c40040f0 0x7f14c40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004d10 0x7f14c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e406a490 con 0x7f14c4004d10 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c400bad0 con 0x7f14c40040f0 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c400bc90 con 0x7f14e41578d0 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004d10 0x7f14c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e1e00640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c40040f0 0x7f14c40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e2800640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 0x7f14c4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 -- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 msgr2=0x7f14c4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14e41578d0 0x7f14c4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 -- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c40040f0 msgr2=0x7f14c40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14c40040f0 0x7f14c40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.889+0000 7f14e3200640 1 -- 172.21.15.143:0/2330625377 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4003530 con 0x7f14c4004d10 2024-01-15T18:16:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e3200640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004d10 0x7f14c4005190 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009f90 tx=0x7f14dc01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad0a98c271455af server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0a5b20 con 0x7f14c4004d10 2024-01-15T18:16:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc00f040 con 0x7f14c4004d10 2024-01-15T18:16:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e1400640 1 -- 172.21.15.143:0/2330625377 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0ad440 con 0x7f14c4004d10 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 -- 172.21.15.143:0/2330625377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004d10 msgr2=0x7f14c4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 --2- 172.21.15.143:0/2330625377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14c4004d10 0x7f14c4005190 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009f90 tx=0x7f14dc01cc20 comp rx=0 tx=0).stop 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e3200640 1 -- 172.21.15.143:0/2330625377 reap_dead start 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 -- 172.21.15.143:0/2330625377 shutdown_connections 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 -- 172.21.15.143:0/2330625377 >> 172.21.15.143:0/2330625377 conn(0x7f14e40b1610 msgr2=0x7f14e40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 -- 172.21.15.143:0/2330625377 shutdown_connections 2024-01-15T18:16:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 -- 172.21.15.143:0/2330625377 wait complete. 2024-01-15T18:16:49.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 Processor -- start 2024-01-15T18:16:49.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.893+0000 7f14e9400640 1 -- start start 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14c4004d10 0x7f14e413e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 0x7f14e40afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4151ba0 0x7f14e4152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e413ee10 con 0x7f14e40af910 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e413f990 con 0x7f14e4151ba0 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e40b02d0 con 0x7f14c4004d10 2024-01-15T18:16:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 0x7f14e40afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 0x7f14e40afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60788/0 (socket says 172.21.15.143:60788) 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 -- 172.21.15.143:0/1033907986 learned_addr learned my addr 172.21.15.143:0/1033907986 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e2800640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14c4004d10 0x7f14e413e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e3200640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4151ba0 0x7f14e4152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 -- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14c4004d10 msgr2=0x7f14e413e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14c4004d10 0x7f14e413e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 -- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4151ba0 msgr2=0x7f14e4152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4151ba0 0x7f14e4152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 -- 172.21.15.143:0/1033907986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e413f0a0 con 0x7f14e40af910 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e1e00640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 0x7f14e40afd90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0124d0 tx=0x7f14cc012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc014810 con 0x7f14e40af910 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc014970 con 0x7f14e40af910 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc014ce0 con 0x7f14e40af910 2024-01-15T18:16:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e4152a20 con 0x7f14e40af910 2024-01-15T18:16:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e413f620 con 0x7f14e40af910 2024-01-15T18:16:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.897+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e4074b00 con 0x7f14e40af910 2024-01-15T18:16:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.901+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f14cc016480 con 0x7f14e40af910 2024-01-15T18:16:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.901+0000 7f14daa00640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14bc0677c0 0x7f14bc069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.901+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f14cc086530 con 0x7f14e40af910 2024-01-15T18:16:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.901+0000 7f14e2800640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14bc0677c0 0x7f14bc069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.901+0000 7f14e2800640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14bc0677c0 0x7f14bc069c80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007fd0 tx=0x7f14d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:49.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:49.905+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14cc058a80 con 0x7f14e40af910 2024-01-15T18:16:50.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.089+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f14e406b010 con 0x7f14bc0677c0 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14daa00640 1 -- 172.21.15.143:0/1033907986 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f14d4001750 con 0x7f14bc0677c0 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:50.094 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:50.095 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:50.095 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:50.095 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:50.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14bc0677c0 msgr2=0x7f14bc069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14bc0677c0 0x7f14bc069c80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f14d4007fd0 tx=0x7f14d4009040 comp rx=0 tx=0).stop 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 msgr2=0x7f14e40afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 0x7f14e40afd90 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0124d0 tx=0x7f14cc012990 comp rx=0 tx=0).stop 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 shutdown_connections 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14bc0677c0 0x7f14bc069c80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14e4151ba0 0x7f14e4152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14e40af910 0x7f14e40afd90 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 --2- 172.21.15.143:0/1033907986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f14c4004d10 0x7f14e413e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 >> 172.21.15.143:0/1033907986 conn(0x7f14e40b1610 msgr2=0x7f14e406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 shutdown_connections 2024-01-15T18:16:50.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:50.097+0000 7f14e9400640 1 -- 172.21.15.143:0/1033907986 wait complete. 2024-01-15T18:16:50.117 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:51.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 Processor -- start 2024-01-15T18:16:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 -- start start 2024-01-15T18:16:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 0x7f102806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 0x7f10280cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280cffd0 con 0x7f10281607c0 2024-01-15T18:16:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280d1ce0 con 0x7f10280cbf50 2024-01-15T18:16:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.269+0000 7f102d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280d02b0 con 0x7f10280cfb30 2024-01-15T18:16:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 0x7f102806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1026800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 0x7f10280cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 0x7f102806bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:44592/0 (socket says 172.21.15.143:44592) 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1026800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 0x7f10280cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:43916/0 (socket says 172.21.15.143:43916) 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 -- 172.21.15.143:0/2928647804 learned_addr learned my addr 172.21.15.143:0/2928647804 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 -- 172.21.15.143:0/2928647804 >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 msgr2=0x7f10280cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 --2- 172.21.15.143:0/2928647804 >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 0x7f10280cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 -- 172.21.15.143:0/2928647804 >> v1:172.21.15.204:6789/0 conn(0x7f10281607c0 legacy=0x7f102806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1025400640 1 -- 172.21.15.143:0/2928647804 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280cb470 con 0x7f10280cfb30 2024-01-15T18:16:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1026800640 1 --2- 172.21.15.143:0/2928647804 >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 0x7f10280cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:16:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.273+0000 7f1024a00640 1 -- 172.21.15.143:0/2928647804 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606860059 0 0) 0x7f10280cffd0 con 0x7f10281607c0 2024-01-15T18:16:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f1025400640 1 --2- 172.21.15.143:0/2928647804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 0x7f102806bec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1010009870 tx=0x7f101001ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1699f70b51eca4a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f1024a00640 1 -- 172.21.15.143:0/2928647804 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100a8a30 con 0x7f10280cfb30 2024-01-15T18:16:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f1024a00640 1 -- 172.21.15.143:0/2928647804 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10100a8b90 con 0x7f10280cfb30 2024-01-15T18:16:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f1024a00640 1 -- 172.21.15.143:0/2928647804 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100b0440 con 0x7f10280cfb30 2024-01-15T18:16:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 -- 172.21.15.143:0/2928647804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 msgr2=0x7f102806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 --2- 172.21.15.143:0/2928647804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 0x7f102806bec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1010009870 tx=0x7f101001ed30 comp rx=0 tx=0).stop 2024-01-15T18:16:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 -- 172.21.15.143:0/2928647804 shutdown_connections 2024-01-15T18:16:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 --2- 172.21.15.143:0/2928647804 >> v2:172.21.15.204:3300/0 conn(0x7f10280cbf50 0x7f10280cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 --2- 172.21.15.143:0/2928647804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cfb30 0x7f102806bec0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 -- 172.21.15.143:0/2928647804 >> 172.21.15.143:0/2928647804 conn(0x7f10280d0e20 msgr2=0x7f10280d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:51.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 -- 172.21.15.143:0/2928647804 shutdown_connections 2024-01-15T18:16:51.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 -- 172.21.15.143:0/2928647804 wait complete. 2024-01-15T18:16:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 Processor -- start 2024-01-15T18:16:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.277+0000 7f102d000640 1 -- start start 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cbf50 0x7f1028154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280cfb30 0x7f102815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 0x7f102815b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102815b710 con 0x7f10281607c0 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028154e90 con 0x7f10280cfb30 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1028155170 con 0x7f10280cbf50 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 0x7f102815b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 0x7f102815b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60800/0 (socket says 172.21.15.143:60800) 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 -- 172.21.15.143:0/2728121479 learned_addr learned my addr 172.21.15.143:0/2728121479 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1025e00640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cbf50 0x7f1028154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1025400640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280cfb30 0x7f102815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 -- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cbf50 msgr2=0x7f1028154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cbf50 0x7f1028154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 -- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280cfb30 msgr2=0x7f102815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280cfb30 0x7f102815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 -- 172.21.15.143:0/2728121479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1020009a70 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f1026800640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 0x7f102815b1d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f102000efc0 tx=0x7f1020009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020009d70 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102000f040 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10100094d0 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10200039a0 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1028155730 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102806d4b0 con 0x7f10281607c0 2024-01-15T18:16:51.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.281+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f10200144a0 con 0x7f10281607c0 2024-01-15T18:16:51.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.285+0000 7f101e000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1000067790 0x7f1000069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:51.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.285+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f1020083a60 con 0x7f10281607c0 2024-01-15T18:16:51.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.285+0000 7f1025e00640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1000067790 0x7f1000069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:51.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.285+0000 7f1025e00640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1000067790 0x7f1000069c50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1018007fd0 tx=0x7f1018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:51.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.289+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f10200527a0 con 0x7f10281607c0 2024-01-15T18:16:51.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.469+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f10280ce6b0 con 0x7f1000067790 2024-01-15T18:16:51.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.473+0000 7f101e000640 1 -- 172.21.15.143:0/2728121479 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f1018001470 con 0x7f1000067790 2024-01-15T18:16:51.472 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:51.472 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:51.473 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1000067790 msgr2=0x7f1000069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1000067790 0x7f1000069c50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1018007fd0 tx=0x7f1018009040 comp rx=0 tx=0).stop 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 msgr2=0x7f102815b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 0x7f102815b1d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f102000efc0 tx=0x7f1020009f90 comp rx=0 tx=0).stop 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 shutdown_connections 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1000067790 0x7f1000069c50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10281607c0 0x7f102815b1d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10280cfb30 0x7f102815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 --2- 172.21.15.143:0/2728121479 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f10280cbf50 0x7f1028154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 >> 172.21.15.143:0/2728121479 conn(0x7f10280d0e20 msgr2=0x7f10280cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 shutdown_connections 2024-01-15T18:16:51.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:51.477+0000 7f102d000640 1 -- 172.21.15.143:0/2728121479 wait complete. 2024-01-15T18:16:51.494 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:52.496 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:52.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.645+0000 7f49bc800640 1 Processor -- start 2024-01-15T18:16:52.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.645+0000 7f49bc800640 1 -- start start 2024-01-15T18:16:52.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49bc800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49b40ce690 0x7f49b40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49bc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b40b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49bc800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49b4070bf0 con 0x7f49b40b1da0 2024-01-15T18:16:52.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49bc800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b415dba0 con 0x7f49b40ce690 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49bc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b4920 con 0x7f49b40b3bf0 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49b40ce690 0x7f49b40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f49b40b1da0 0x7f49b40b2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39122/0 (socket says 172.21.15.143:39122) 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9400640 1 -- 172.21.15.143:0/235388325 learned_addr learned my addr 172.21.15.143:0/235388325 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49ba800640 1 --2- 172.21.15.143:0/235388325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b40b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9e00640 1 -- 172.21.15.143:0/235388325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 msgr2=0x7f49b40b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9e00640 1 --2- 172.21.15.143:0/235388325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b40b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9e00640 1 -- 172.21.15.143:0/235388325 >> v1:172.21.15.143:6789/0 conn(0x7f49b40b1da0 legacy=0x7f49b40b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.649+0000 7f49b9e00640 1 -- 172.21.15.143:0/235388325 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b40b4c00 con 0x7f49b40ce690 2024-01-15T18:16:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49b9e00640 1 --2- 172.21.15.143:0/235388325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b40ac8e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f49a40097c0 tx=0x7f49a401ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f059b6d262805906 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:52.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49b8a00640 1 -- 172.21.15.143:0/235388325 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40ab670 con 0x7f49b40ce690 2024-01-15T18:16:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49b8a00640 1 -- 172.21.15.143:0/235388325 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a40ab7d0 con 0x7f49b40ce690 2024-01-15T18:16:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49b8a00640 1 -- 172.21.15.143:0/235388325 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40abb60 con 0x7f49b40ce690 2024-01-15T18:16:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 -- 172.21.15.143:0/235388325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 msgr2=0x7f49b40ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 --2- 172.21.15.143:0/235388325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b40ac8e0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f49a40097c0 tx=0x7f49a401ceb0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 -- 172.21.15.143:0/235388325 shutdown_connections 2024-01-15T18:16:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 --2- 172.21.15.143:0/235388325 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b40b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 --2- 172.21.15.143:0/235388325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b40ac8e0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 -- 172.21.15.143:0/235388325 >> 172.21.15.143:0/235388325 conn(0x7f49b406c3f0 msgr2=0x7f49b406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 -- 172.21.15.143:0/235388325 shutdown_connections 2024-01-15T18:16:52.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 -- 172.21.15.143:0/235388325 wait complete. 2024-01-15T18:16:52.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 Processor -- start 2024-01-15T18:16:52.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.653+0000 7f49bc800640 1 -- start start 2024-01-15T18:16:52.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b40b1da0 0x7f49b4159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b4144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b41448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b413a1a0 con 0x7f49b40ce690 2024-01-15T18:16:52.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4146310 con 0x7f49b40b1da0 2024-01-15T18:16:52.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4139ec0 con 0x7f49b40b3bf0 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b41448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b41448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60824/0 (socket says 172.21.15.143:60824) 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 -- 172.21.15.143:0/3524801391 learned_addr learned my addr 172.21.15.143:0/3524801391 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49b9400640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b4144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49b9e00640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b40b1da0 0x7f49b4159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 -- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 msgr2=0x7f49b4144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b4144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 -- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b40b1da0 msgr2=0x7f49b4159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b40b1da0 0x7f49b4159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 -- 172.21.15.143:0/3524801391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4009420 con 0x7f49b40ce690 2024-01-15T18:16:52.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49ba800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b41448d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f49ac007d90 tx=0x7f49ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:52.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac014420 con 0x7f49b40ce690 2024-01-15T18:16:52.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac0145a0 con 0x7f49b40ce690 2024-01-15T18:16:52.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b4145cc0 con 0x7f49b40ce690 2024-01-15T18:16:52.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac014950 con 0x7f49b40ce690 2024-01-15T18:16:52.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b415a3c0 con 0x7f49b40ce690 2024-01-15T18:16:52.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.657+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b4145b20 con 0x7f49b40ce690 2024-01-15T18:16:52.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.661+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f49ac014ab0 con 0x7f49b40ce690 2024-01-15T18:16:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.661+0000 7f49b2000640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f498c067790 0x7f498c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.661+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f49ac0839e0 con 0x7f49b40ce690 2024-01-15T18:16:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.661+0000 7f49b9e00640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f498c067790 0x7f498c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.661+0000 7f49b9e00640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f498c067790 0x7f498c069c50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f49a40af000 tx=0x7f49a4004740 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:52.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.665+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49b4145b20 con 0x7f49b40ce690 2024-01-15T18:16:52.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.845+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f49b40acb70 con 0x7f498c067790 2024-01-15T18:16:52.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.849+0000 7f49b2000640 1 -- 172.21.15.143:0/3524801391 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f49b40acb70 con 0x7f498c067790 2024-01-15T18:16:52.847 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:52.847 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:52.847 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:52.847 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:52.848 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:52.848 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f498c067790 msgr2=0x7f498c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f498c067790 0x7f498c069c50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f49a40af000 tx=0x7f49a4004740 comp rx=0 tx=0).stop 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 msgr2=0x7f49b41448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b41448d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f49ac007d90 tx=0x7f49ac00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 shutdown_connections 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f498c067790 0x7f498c069c50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49b40ce690 0x7f49b41448d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49b40b3bf0 0x7f49b4144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 --2- 172.21.15.143:0/3524801391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49b40b1da0 0x7f49b4159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 >> 172.21.15.143:0/3524801391 conn(0x7f49b406c3f0 msgr2=0x7f49b415e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:52.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 shutdown_connections 2024-01-15T18:16:52.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:52.853+0000 7f49bc800640 1 -- 172.21.15.143:0/3524801391 wait complete. 2024-01-15T18:16:52.869 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:53.871 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 Processor -- start 2024-01-15T18:16:54.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 -- start start 2024-01-15T18:16:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d130 0x7fb4680aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb46815d510 0x7fb46806e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb468160eb0 con 0x7fb46806a490 2024-01-15T18:16:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb468161190 con 0x7fb46815d510 2024-01-15T18:16:54.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb46d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb468161470 con 0x7fb46815d130 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb46815d510 0x7fb46806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb46815d510 0x7fb46806e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60830/0 (socket says 172.21.15.143:60830) 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 -- 172.21.15.143:0/2572370980 learned_addr learned my addr 172.21.15.143:0/2572370980 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 -- 172.21.15.143:0/2572370980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d130 msgr2=0x7fb4680aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 --2- 172.21.15.143:0/2572370980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d130 0x7fb4680aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 -- 172.21.15.143:0/2572370980 >> v1:172.21.15.204:6789/0 conn(0x7fb46806a490 legacy=0x7fb468071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.025+0000 7fb466000640 1 -- 172.21.15.143:0/2572370980 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468161750 con 0x7fb46815d510 2024-01-15T18:16:54.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb466000640 1 --2- 172.21.15.143:0/2572370980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46815d510 0x7fb46806e060 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb450009870 tx=0x7fb45001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1abb143375949cd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:54.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb465600640 1 -- 172.21.15.143:0/2572370980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4500aa640 con 0x7fb46815d510 2024-01-15T18:16:54.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb465600640 1 -- 172.21.15.143:0/2572370980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4500aa7a0 con 0x7fb46815d510 2024-01-15T18:16:54.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb465600640 1 -- 172.21.15.143:0/2572370980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4500aab30 con 0x7fb46815d510 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 -- 172.21.15.143:0/2572370980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46815d510 msgr2=0x7fb46806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 --2- 172.21.15.143:0/2572370980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46815d510 0x7fb46806e060 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fb450009870 tx=0x7fb45001c500 comp rx=0 tx=0).stop 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 -- 172.21.15.143:0/2572370980 shutdown_connections 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 --2- 172.21.15.143:0/2572370980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46815d510 0x7fb46806e060 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 --2- 172.21.15.143:0/2572370980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d130 0x7fb4680aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 -- 172.21.15.143:0/2572370980 >> 172.21.15.143:0/2572370980 conn(0x7fb4680b1b90 msgr2=0x7fb4680b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 -- 172.21.15.143:0/2572370980 shutdown_connections 2024-01-15T18:16:54.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.029+0000 7fb46d600640 1 -- 172.21.15.143:0/2572370980 wait complete. 2024-01-15T18:16:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 Processor -- start 2024-01-15T18:16:54.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 -- start start 2024-01-15T18:16:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 0x7fb468158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46815d130 0x7fb468139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d510 0x7fb46815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46815e6e0 con 0x7fb46806a490 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46814f850 con 0x7fb46815d130 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46815da60 con 0x7fb46815d510 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 0x7fb468158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 0x7fb468158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60836/0 (socket says 172.21.15.143:60836) 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 -- 172.21.15.143:0/3991777227 learned_addr learned my addr 172.21.15.143:0/3991777227 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466000640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46815d130 0x7fb468139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb467400640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d510 0x7fb46815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 -- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d510 msgr2=0x7fb46815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d510 0x7fb46815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 -- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46815d130 msgr2=0x7fb468139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46815d130 0x7fb468139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 -- 172.21.15.143:0/3991777227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500094d0 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb466a00640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 0x7fb468158460 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b580 tx=0x7fb45800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458010870 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580109f0 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458010da0 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46815dd40 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.033+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46815e300 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.037+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb468067500 con 0x7fb46806a490 2024-01-15T18:16:54.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.037+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7fb45800f3c0 con 0x7fb46806a490 2024-01-15T18:16:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.037+0000 7fb45ea00640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067790 0x7fb440069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.037+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fb458083200 con 0x7fb46806a490 2024-01-15T18:16:54.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.037+0000 7fb466000640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067790 0x7fb440069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.037+0000 7fb466000640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067790 0x7fb440069c50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb4500ae000 tx=0x7fb4500b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:54.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.041+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4580b7280 con 0x7fb46806a490 2024-01-15T18:16:54.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.221+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4680b4660 con 0x7fb440067790 2024-01-15T18:16:54.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb45ea00640 1 -- 172.21.15.143:0/3991777227 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fb4500aa910 con 0x7fb440067790 2024-01-15T18:16:54.225 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:54.226 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067790 msgr2=0x7fb440069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067790 0x7fb440069c50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb4500ae000 tx=0x7fb4500b5040 comp rx=0 tx=0).stop 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 msgr2=0x7fb468158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 0x7fb468158460 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb45800b580 tx=0x7fb45800ba40 comp rx=0 tx=0).stop 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 shutdown_connections 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb440067790 0x7fb440069c50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb46815d510 0x7fb46815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46815d130 0x7fb468139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 --2- 172.21.15.143:0/3991777227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46806a490 0x7fb468158460 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 >> 172.21.15.143:0/3991777227 conn(0x7fb4680b1b90 msgr2=0x7fb4680aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 shutdown_connections 2024-01-15T18:16:54.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:54.229+0000 7fb46d600640 1 -- 172.21.15.143:0/3991777227 wait complete. 2024-01-15T18:16:54.250 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:16:55.251 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.401+0000 7f8b2de00640 1 Processor -- start 2024-01-15T18:16:55.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.401+0000 7f8b2de00640 1 -- start start 2024-01-15T18:16:55.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b2de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 0x7f8b280b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b2de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b280b48f0 0x7f8b28070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b2de00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b280b2a20 con 0x7f8b28157300 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b2de00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b280b35a0 con 0x7f8b280b48f0 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b2de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28159fd0 con 0x7f8b28159b70 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b26a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8b28157300 0x7f8b280b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39136/0 (socket says 172.21.15.143:39136) 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b26a00640 1 -- 172.21.15.143:0/1583592986 learned_addr learned my addr 172.21.15.143:0/1583592986 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b26000640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 0x7f8b280b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 --2- 172.21.15.143:0/1583592986 >> v2:172.21.15.204:3300/0 conn(0x7f8b280b48f0 0x7f8b28070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793758998 0 0) 0x7f8b280b2a20 con 0x7f8b28157300 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b08005740 con 0x7f8b28157300 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963628639 0 0) 0x7f8b08005740 con 0x7f8b28157300 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 msgr2=0x7f8b280b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 0x7f8b280b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 >> v2:172.21.15.204:3300/0 conn(0x7f8b280b48f0 msgr2=0x7f8b28070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:55.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 --2- 172.21.15.143:0/1583592986 >> v2:172.21.15.204:3300/0 conn(0x7f8b280b48f0 0x7f8b28070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b280b2d00 con 0x7f8b28157300 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8b18001680 con 0x7f8b28157300 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 >> v1:172.21.15.143:6789/0 conn(0x7f8b28157300 legacy=0x7f8b280b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b08005b90 0x7f8b08005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 0x7f8b080069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b080073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280b35a0 con 0x7f8b08006f20 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28159fd0 con 0x7f8b08005b90 2024-01-15T18:16:55.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0800dba0 con 0x7f8b28159b70 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b080073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b26a00640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b08005b90 0x7f8b08005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b26000640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 0x7f8b080069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 -- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 msgr2=0x7f8b080069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b28159b70 0x7f8b080069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 -- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b08005b90 msgr2=0x7f8b08005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b08005b90 0x7f8b08005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b27400640 1 -- 172.21.15.143:0/1583592986 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08005740 con 0x7f8b08006f20 2024-01-15T18:16:55.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.405+0000 7f8b26a00640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b08005b90 0x7f8b08005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b27400640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b080073a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009fc0 tx=0x7f8b2001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1587f993ce053885 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b200a5cb0 con 0x7f8b08006f20 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2000f040 con 0x7f8b08006f20 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b25600640 1 -- 172.21.15.143:0/1583592986 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b200ad680 con 0x7f8b08006f20 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b2de00640 1 -- 172.21.15.143:0/1583592986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 msgr2=0x7f8b080073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1583592986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b080073a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8b20009fc0 tx=0x7f8b2001cca0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b27400640 1 -- 172.21.15.143:0/1583592986 reap_dead start 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b2de00640 1 -- 172.21.15.143:0/1583592986 shutdown_connections 2024-01-15T18:16:55.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b2de00640 1 -- 172.21.15.143:0/1583592986 >> 172.21.15.143:0/1583592986 conn(0x7f8b2806c3f0 msgr2=0x7f8b2806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b2de00640 1 -- 172.21.15.143:0/1583592986 shutdown_connections 2024-01-15T18:16:55.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.409+0000 7f8b2de00640 1 -- 172.21.15.143:0/1583592986 wait complete. 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 Processor -- start 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 -- start start 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b28147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2815b360 0x7f8b280ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b280ace20 0x7f8b281f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b28148a00 con 0x7f8b08006f20 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b280ad2a0 con 0x7f8b280ace20 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b281505a0 con 0x7f8b2815b360 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b28147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b28147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60846/0 (socket says 172.21.15.143:60846) 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 -- 172.21.15.143:0/1838829098 learned_addr learned my addr 172.21.15.143:0/1838829098 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26000640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2815b360 0x7f8b280ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b27400640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b280ace20 0x7f8b281f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 -- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2815b360 msgr2=0x7f8b280ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2815b360 0x7f8b280ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 -- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b280ace20 msgr2=0x7f8b281f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b280ace20 0x7f8b281f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 -- 172.21.15.143:0/1838829098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28148050 con 0x7f8b08006f20 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b26a00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b28147b10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800e7e0 tx=0x7f8b1800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800c870 con 0x7f8b08006f20 2024-01-15T18:16:55.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1800c9d0 con 0x7f8b08006f20 2024-01-15T18:16:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1800cd60 con 0x7f8b08006f20 2024-01-15T18:16:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2815c7d0 con 0x7f8b08006f20 2024-01-15T18:16:55.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.413+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b28148340 con 0x7f8b08006f20 2024-01-15T18:16:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.417+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b28072650 con 0x7f8b08006f20 2024-01-15T18:16:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.417+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f8b1801d460 con 0x7f8b08006f20 2024-01-15T18:16:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.417+0000 7f8b1f400640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b00067790 0x7f8b00069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.417+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f8b18082ff0 con 0x7f8b08006f20 2024-01-15T18:16:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.417+0000 7f8b26000640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b00067790 0x7f8b00069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.417+0000 7f8b26000640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b00067790 0x7f8b00069c50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b28070ef0 tx=0x7f8b10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:55.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.421+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b18051a90 con 0x7f8b08006f20 2024-01-15T18:16:55.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:55.601+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b280b32c0 con 0x7f8b00067790 2024-01-15T18:16:57.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b1f400640 1 -- 172.21.15.143:0/1838829098 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f8b10014a90 con 0x7f8b00067790 2024-01-15T18:16:57.074 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:57.074 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:57.075 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b00067790 msgr2=0x7f8b00069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b00067790 0x7f8b00069c50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b28070ef0 tx=0x7f8b10008040 comp rx=0 tx=0).stop 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 msgr2=0x7f8b28147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b28147b10 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800e7e0 tx=0x7f8b1800eca0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 shutdown_connections 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b00067790 0x7f8b00069c50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b280ace20 0x7f8b281f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2815b360 0x7f8b280ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 --2- 172.21.15.143:0/1838829098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b08006f20 0x7f8b28147b10 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.077+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 >> 172.21.15.143:0/1838829098 conn(0x7f8b2806c3f0 msgr2=0x7f8b2814fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.081+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 shutdown_connections 2024-01-15T18:16:57.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:57.081+0000 7f8b2de00640 1 -- 172.21.15.143:0/1838829098 wait complete. 2024-01-15T18:16:59.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:16:59.343 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.345+0000 7f56d6e00640 1 Processor -- start 2024-01-15T18:16:59.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.345+0000 7f56d6e00640 1 -- start start 2024-01-15T18:16:59.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56d6e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f56d01614e0 0x7f56d006d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:59.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56d6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d0165ec0 0x7f56d00ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:59.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56d6e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d00cda20 con 0x7f56d00caf20 2024-01-15T18:16:59.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56d6e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00cdd00 con 0x7f56d01614e0 2024-01-15T18:16:59.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56d6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00cdfe0 con 0x7f56d0165ec0 2024-01-15T18:16:59.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f56d01614e0 0x7f56d006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:59.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f56d01614e0 0x7f56d006d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34922/0 (socket says 172.21.15.143:34922) 2024-01-15T18:16:59.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 -- 172.21.15.143:0/638805366 learned_addr learned my addr 172.21.15.143:0/638805366 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 -- 172.21.15.143:0/638805366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d0165ec0 msgr2=0x7f56d00ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 --2- 172.21.15.143:0/638805366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d0165ec0 0x7f56d00ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 -- 172.21.15.143:0/638805366 >> v1:172.21.15.143:6789/0 conn(0x7f56d00caf20 legacy=0x7f56d006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:59.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.349+0000 7f56cfe00640 1 -- 172.21.15.143:0/638805366 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00ce2c0 con 0x7f56d01614e0 2024-01-15T18:16:59.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.353+0000 7f56cfe00640 1 --2- 172.21.15.143:0/638805366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d01614e0 0x7f56d006d140 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009870 tx=0x7f56b801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed9d2f301ea0699 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:59.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.353+0000 7f56cea00640 1 -- 172.21.15.143:0/638805366 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80ab620 con 0x7f56d01614e0 2024-01-15T18:16:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.353+0000 7f56cea00640 1 -- 172.21.15.143:0/638805366 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80ab780 con 0x7f56d01614e0 2024-01-15T18:16:59.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.353+0000 7f56cea00640 1 -- 172.21.15.143:0/638805366 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80abb10 con 0x7f56d01614e0 2024-01-15T18:16:59.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.357+0000 7f56d6e00640 1 -- 172.21.15.143:0/638805366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d01614e0 msgr2=0x7f56d006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:59.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.357+0000 7f56d6e00640 1 --2- 172.21.15.143:0/638805366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d01614e0 0x7f56d006d140 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009870 tx=0x7f56b801c440 comp rx=0 tx=0).stop 2024-01-15T18:16:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.357+0000 7f56d6e00640 1 -- 172.21.15.143:0/638805366 shutdown_connections 2024-01-15T18:16:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.357+0000 7f56d6e00640 1 --2- 172.21.15.143:0/638805366 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d0165ec0 0x7f56d00ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.357+0000 7f56d6e00640 1 --2- 172.21.15.143:0/638805366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d01614e0 0x7f56d006d140 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.357+0000 7f56d6e00640 1 -- 172.21.15.143:0/638805366 >> 172.21.15.143:0/638805366 conn(0x7f56d00cfd20 msgr2=0x7f56d00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:59.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.361+0000 7f56d6e00640 1 -- 172.21.15.143:0/638805366 shutdown_connections 2024-01-15T18:16:59.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.361+0000 7f56d6e00640 1 -- 172.21.15.143:0/638805366 wait complete. 2024-01-15T18:16:59.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.365+0000 7f56d6e00640 1 Processor -- start 2024-01-15T18:16:59.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.365+0000 7f56d6e00640 1 -- start start 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.365+0000 7f56d6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d00caf20 0x7f56d014fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.365+0000 7f56d6e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d01614e0 0x7f56d0162e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56d6e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d0165ec0 0x7f56d015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56d6e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d015f340 con 0x7f56d0165ec0 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56d6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0163850 con 0x7f56d00caf20 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56d6e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d0163b30 con 0x7f56d01614e0 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d01614e0 0x7f56d0162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:59.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cf400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d01614e0 0x7f56d0162e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46272/0 (socket says 172.21.15.143:46272) 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cf400640 1 -- 172.21.15.143:0/1231532347 learned_addr learned my addr 172.21.15.143:0/1231532347 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d00caf20 0x7f56d014fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56d4e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d0165ec0 0x7f56d015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 -- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d01614e0 msgr2=0x7f56d0162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d01614e0 0x7f56d0162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 -- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d0165ec0 msgr2=0x7f56d015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d0165ec0 0x7f56d015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 -- 172.21.15.143:0/1231532347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80094d0 con 0x7f56d00caf20 2024-01-15T18:16:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56cfe00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d00caf20 0x7f56d014fe60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f56b80ac000 tx=0x7f56b801ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80be3f0 con 0x7f56d00caf20 2024-01-15T18:16:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80be590 con 0x7f56d00caf20 2024-01-15T18:16:59.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80be940 con 0x7f56d00caf20 2024-01-15T18:16:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.369+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d006ae30 con 0x7f56d00caf20 2024-01-15T18:16:59.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.373+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d015f800 con 0x7f56d00caf20 2024-01-15T18:16:59.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.373+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d0067500 con 0x7f56d00caf20 2024-01-15T18:16:59.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.377+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f56b80b43f0 con 0x7f56d00caf20 2024-01-15T18:16:59.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.381+0000 7f56c7e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f56a8067860 0x7f56a8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.381+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f56b811d950 con 0x7f56d00caf20 2024-01-15T18:16:59.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.381+0000 7f56cf400640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f56a8067860 0x7f56a8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:59.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.385+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f56b80ec3b0 con 0x7f56d00caf20 2024-01-15T18:16:59.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.385+0000 7f56cf400640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f56a8067860 0x7f56a8069d20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f56c00099b0 tx=0x7f56c0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:59.541 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:59.546 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:59.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.713+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f56d00d0bc0 con 0x7f56a8067860 2024-01-15T18:16:59.720 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.721+0000 7f56c7e00640 1 -- 172.21.15.143:0/1231532347 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f56d00d0bc0 con 0x7f56a8067860 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:16:59.721 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 0.0, 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "0/3", 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:16:59.722 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f56a8067860 msgr2=0x7f56a8069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f56a8067860 0x7f56a8069d20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f56c00099b0 tx=0x7f56c0009190 comp rx=0 tx=0).stop 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d00caf20 msgr2=0x7f56d014fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d00caf20 0x7f56d014fe60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f56b80ac000 tx=0x7f56b801ac80 comp rx=0 tx=0).stop 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 shutdown_connections 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f56a8067860 0x7f56a8069d20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f56d0165ec0 0x7f56d015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.724 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f56d01614e0 0x7f56d0162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 --2- 172.21.15.143:0/1231532347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56d00caf20 0x7f56d014fe60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:59.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 >> 172.21.15.143:0/1231532347 conn(0x7f56d00cfd20 msgr2=0x7f56d00c2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:59.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 shutdown_connections 2024-01-15T18:16:59.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:16:59.725+0000 7f56d6e00640 1 -- 172.21.15.143:0/1231532347 wait complete. 2024-01-15T18:16:59.767 INFO:tasks.cephfs.test_volumes:percetange completed = 0.0 2024-01-15T18:17:01.768 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:17:01.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3b6600640 1 Processor -- start 2024-01-15T18:17:01.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3b6600640 1 -- start start 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3b6600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc3b00cf910 0x7fc3b00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3b6600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00cbff0 con 0x7fc3b006bfb0 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3b6600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b00d0c50 con 0x7fc3b00b0310 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3b6600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b00cb6c0 con 0x7fc3b00cf910 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3af400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc3b006bfb0 0x7fc3b006c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:40040/0 (socket says 172.21.15.143:40040) 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3af400640 1 -- 172.21.15.143:0/802137664 learned_addr learned my addr 172.21.15.143:0/802137664 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:01.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3afe00640 1 --2- 172.21.15.143:0/802137664 >> v2:172.21.15.204:3300/0 conn(0x7fc3b00cf910 0x7fc3b00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:01.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3ae000640 1 -- 172.21.15.143:0/802137664 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649142956 0 0) 0x7fc3b00cbff0 con 0x7fc3b006bfb0 2024-01-15T18:17:01.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3ae000640 1 -- 172.21.15.143:0/802137664 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc390003530 con 0x7fc3b006bfb0 2024-01-15T18:17:01.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3afe00640 1 -- 172.21.15.143:0/802137664 >> v1:172.21.15.204:6789/0 conn(0x7fc3b00b0310 legacy=0x7fc3b00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:01.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3afe00640 1 -- 172.21.15.143:0/802137664 >> v1:172.21.15.143:6789/0 conn(0x7fc3b006bfb0 legacy=0x7fc3b006c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:01.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.917+0000 7fc3afe00640 1 -- 172.21.15.143:0/802137664 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b00cb9a0 con 0x7fc3b00cf910 2024-01-15T18:17:01.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3afe00640 1 --2- 172.21.15.143:0/802137664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00cf910 0x7fc3b00cfdc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009d90 tx=0x7fc3a801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b842c7f173011d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:01.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3ae000640 1 -- 172.21.15.143:0/802137664 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80a5880 con 0x7fc3b00cf910 2024-01-15T18:17:01.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3ae000640 1 -- 172.21.15.143:0/802137664 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a80a59e0 con 0x7fc3b00cf910 2024-01-15T18:17:01.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3ae000640 1 -- 172.21.15.143:0/802137664 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80a5d70 con 0x7fc3b00cf910 2024-01-15T18:17:01.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 -- 172.21.15.143:0/802137664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00cf910 msgr2=0x7fc3b00cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:01.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 --2- 172.21.15.143:0/802137664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00cf910 0x7fc3b00cfdc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009d90 tx=0x7fc3a801c890 comp rx=0 tx=0).stop 2024-01-15T18:17:01.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 -- 172.21.15.143:0/802137664 shutdown_connections 2024-01-15T18:17:01.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 --2- 172.21.15.143:0/802137664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00cf910 0x7fc3b00cfdc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:01.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 -- 172.21.15.143:0/802137664 >> 172.21.15.143:0/802137664 conn(0x7fc3b00d0ed0 msgr2=0x7fc3b00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:01.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 -- 172.21.15.143:0/802137664 shutdown_connections 2024-01-15T18:17:01.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.921+0000 7fc3b6600640 1 -- 172.21.15.143:0/802137664 wait complete. 2024-01-15T18:17:01.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 Processor -- start 2024-01-15T18:17:01.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 -- start start 2024-01-15T18:17:01.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 0x7fc3b015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:01.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00b0310 0x7fc3b00ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:01.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3b00cf910 0x7fc3b01512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:01.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00c9380 con 0x7fc3b006bfb0 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00cb280 con 0x7fc3b00b0310 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00c8a50 con 0x7fc3b00cf910 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 0x7fc3b015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 0x7fc3b015c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34940/0 (socket says 172.21.15.143:34940) 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 -- 172.21.15.143:0/3832561112 learned_addr learned my addr 172.21.15.143:0/3832561112 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3afe00640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3b00cf910 0x7fc3b01512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3aea00640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00b0310 0x7fc3b00ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 -- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3b00cf910 msgr2=0x7fc3b01512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3b00cf910 0x7fc3b01512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 -- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00b0310 msgr2=0x7fc3b00ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00b0310 0x7fc3b00ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:01.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 -- 172.21.15.143:0/3832561112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009a70 con 0x7fc3b006bfb0 2024-01-15T18:17:01.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3af400640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 0x7fc3b015c3f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c420 tx=0x7fc3a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:01.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0005cf0 con 0x7fc3b006bfb0 2024-01-15T18:17:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0005e50 con 0x7fc3b006bfb0 2024-01-15T18:17:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00072d0 con 0x7fc3b006bfb0 2024-01-15T18:17:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b00c8d30 con 0x7fc3b006bfb0 2024-01-15T18:17:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.925+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b00caf20 con 0x7fc3b006bfb0 2024-01-15T18:17:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.929+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b0067500 con 0x7fc3b006bfb0 2024-01-15T18:17:01.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.929+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7fc3a0007430 con 0x7fc3b006bfb0 2024-01-15T18:17:01.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.929+0000 7fc3a7400640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc388067790 0x7fc388069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:01.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.929+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fc3a0082770 con 0x7fc3b006bfb0 2024-01-15T18:17:01.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.929+0000 7fc3aea00640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc388067790 0x7fc388069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:01.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.929+0000 7fc3aea00640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc388067790 0x7fc388069c50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0155ed0 tx=0x7fc398009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:01.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:01.933+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc3a00511d0 con 0x7fc3b006bfb0 2024-01-15T18:17:02.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.117+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3b00c8bc0 con 0x7fc388067790 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3a7400640 1 -- 172.21.15.143:0/3832561112 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fc3b00c8bc0 con 0x7fc388067790 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:17:02.122 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:02.123 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:02.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc388067790 msgr2=0x7fc388069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:02.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc388067790 0x7fc388069c50 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0155ed0 tx=0x7fc398009040 comp rx=0 tx=0).stop 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 msgr2=0x7fc3b015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 0x7fc3b015c3f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000c420 tx=0x7fc3a000c8e0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 shutdown_connections 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc388067790 0x7fc388069c50 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc3b00cf910 0x7fc3b01512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3b00b0310 0x7fc3b00ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 --2- 172.21.15.143:0/3832561112 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc3b006bfb0 0x7fc3b015c3f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 >> 172.21.15.143:0/3832561112 conn(0x7fc3b00d0ed0 msgr2=0x7fc3b0154ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 shutdown_connections 2024-01-15T18:17:02.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:02.125+0000 7fc3b6600640 1 -- 172.21.15.143:0/3832561112 wait complete. 2024-01-15T18:17:02.145 INFO:tasks.cephfs.test_volumes:percetange completed = 33.33333333333333 2024-01-15T18:17:04.145 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:17:04.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 Processor -- start 2024-01-15T18:17:04.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 -- start start 2024-01-15T18:17:04.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7d006db20 0x7fa7d00b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:04.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7d015a440 0x7fa7d00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:04.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00cc790 con 0x7fa7d00cd9e0 2024-01-15T18:17:04.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00b4ae0 con 0x7fa7d006db20 2024-01-15T18:17:04.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.441+0000 7fa7d5e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00b4800 con 0x7fa7d015a440 2024-01-15T18:17:04.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7d015a440 0x7fa7d00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:04.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7d015a440 0x7fa7d00cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:44822/0 (socket says 172.21.15.143:44822) 2024-01-15T18:17:04.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 -- 172.21.15.143:0/995184542 learned_addr learned my addr 172.21.15.143:0/995184542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:04.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7cea00640 1 --2- 172.21.15.143:0/995184542 >> v2:172.21.15.143:3300/0 conn(0x7fa7d006db20 0x7fa7d00b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:04.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 -- 172.21.15.143:0/995184542 >> v2:172.21.15.143:3300/0 conn(0x7fa7d006db20 msgr2=0x7fa7d00b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:04.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 --2- 172.21.15.143:0/995184542 >> v2:172.21.15.143:3300/0 conn(0x7fa7d006db20 0x7fa7d00b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 -- 172.21.15.143:0/995184542 >> v1:172.21.15.204:6789/0 conn(0x7fa7d00cd9e0 legacy=0x7fa7d00b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:04.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.445+0000 7fa7ce000640 1 -- 172.21.15.143:0/995184542 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00cc9c0 con 0x7fa7d015a440 2024-01-15T18:17:04.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7ce000640 1 --2- 172.21.15.143:0/995184542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d015a440 0x7fa7d00cd4a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa7d006fb00 tx=0x7fa7b8020bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f79622a7652e431 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:04.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7cd600640 1 -- 172.21.15.143:0/995184542 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80a9c90 con 0x7fa7d015a440 2024-01-15T18:17:04.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7cd600640 1 -- 172.21.15.143:0/995184542 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b80ad070 con 0x7fa7d015a440 2024-01-15T18:17:04.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7cd600640 1 -- 172.21.15.143:0/995184542 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80b3400 con 0x7fa7d015a440 2024-01-15T18:17:04.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 -- 172.21.15.143:0/995184542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d015a440 msgr2=0x7fa7d00cd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:04.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/995184542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d015a440 0x7fa7d00cd4a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa7d006fb00 tx=0x7fa7b8020bc0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 -- 172.21.15.143:0/995184542 shutdown_connections 2024-01-15T18:17:04.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/995184542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d015a440 0x7fa7d00cd4a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/995184542 >> v2:172.21.15.143:3300/0 conn(0x7fa7d006db20 0x7fa7d00b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 -- 172.21.15.143:0/995184542 >> 172.21.15.143:0/995184542 conn(0x7fa7d00aea10 msgr2=0x7fa7d00aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:04.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 -- 172.21.15.143:0/995184542 shutdown_connections 2024-01-15T18:17:04.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.453+0000 7fa7d5e00640 1 -- 172.21.15.143:0/995184542 wait complete. 2024-01-15T18:17:04.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 Processor -- start 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 -- start start 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 0x7fa7d014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7d00cd9e0 0x7fa7d015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7d015a440 0x7fa7d015c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d015aab0 con 0x7fa7d00cd9e0 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d015ad50 con 0x7fa7d006db20 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.457+0000 7fa7d5e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d015b030 con 0x7fa7d015a440 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 0x7fa7d014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 0x7fa7d014dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:44830/0 (socket says 172.21.15.143:44830) 2024-01-15T18:17:04.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 -- 172.21.15.143:0/354606108 learned_addr learned my addr 172.21.15.143:0/354606108 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:04.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7ce000640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7d00cd9e0 0x7fa7d015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 -- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7d015a440 msgr2=0x7fa7d015c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7d015a440 0x7fa7d015c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 -- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7d00cd9e0 msgr2=0x7fa7d015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7d00cd9e0 0x7fa7d015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 -- 172.21.15.143:0/354606108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b800a570 con 0x7fa7d006db20 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7ce000640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7d00cd9e0 0x7fa7d015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7cea00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 0x7fa7d014dc40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000d800 tx=0x7fa7c000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:04.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000b6c0 con 0x7fa7d006db20 2024-01-15T18:17:04.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c000b820 con 0x7fa7d006db20 2024-01-15T18:17:04.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000bb90 con 0x7fa7d006db20 2024-01-15T18:17:04.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d015b310 con 0x7fa7d006db20 2024-01-15T18:17:04.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.461+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d015d580 con 0x7fa7d006db20 2024-01-15T18:17:04.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.465+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d00b0230 con 0x7fa7d006db20 2024-01-15T18:17:04.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.465+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7fa7c000bcf0 con 0x7fa7d006db20 2024-01-15T18:17:04.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.469+0000 7fa7c7400640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7a80677c0 0x7fa7a8069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:04.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.469+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fa7c00836e0 con 0x7fa7d006db20 2024-01-15T18:17:04.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.469+0000 7fa7ce000640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7a80677c0 0x7fa7a8069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:04.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.473+0000 7fa7ce000640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7a80677c0 0x7fa7a8069c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80b4000 tx=0x7fa7b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:04.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.473+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7c00520f0 con 0x7fa7d006db20 2024-01-15T18:17:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.837+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d00b43a0 con 0x7fa7a80677c0 2024-01-15T18:17:04.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.881+0000 7fa7c7400640 1 -- 172.21.15.143:0/354606108 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fa7b8010050 con 0x7fa7a80677c0 2024-01-15T18:17:04.879 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:04.879 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "1/3", 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:04.880 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7a80677c0 msgr2=0x7fa7a8069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7a80677c0 0x7fa7a8069c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80b4000 tx=0x7fa7b8008040 comp rx=0 tx=0).stop 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 msgr2=0x7fa7d014dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 0x7fa7d014dc40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000d800 tx=0x7fa7c000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 shutdown_connections 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7a80677c0 0x7fa7a8069c80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7d015a440 0x7fa7d015c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7d00cd9e0 0x7fa7d015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 --2- 172.21.15.143:0/354606108 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7d006db20 0x7fa7d014dc40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:04.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 >> 172.21.15.143:0/354606108 conn(0x7fa7d00aea10 msgr2=0x7fa7d00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 shutdown_connections 2024-01-15T18:17:04.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:04.885+0000 7fa7d5e00640 1 -- 172.21.15.143:0/354606108 wait complete. 2024-01-15T18:17:04.936 INFO:tasks.cephfs.test_volumes:percetange completed = 33.33333333333333 2024-01-15T18:17:06.937 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:17:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.209+0000 7f8e99000640 1 Processor -- start 2024-01-15T18:17:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.209+0000 7f8e99000640 1 -- start start 2024-01-15T18:17:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.209+0000 7f8e99000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8e94070370 0x7f8e940707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.213+0000 7f8e99000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9406cf30 con 0x7f8e940b1710 2024-01-15T18:17:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.213+0000 7f8e99000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e940cb5d0 con 0x7f8e940ce310 2024-01-15T18:17:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.213+0000 7f8e99000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e940cb0e0 con 0x7f8e94070370 2024-01-15T18:17:07.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.213+0000 7f8e92800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8e94070370 0x7f8e940707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.213+0000 7f8e91e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8e940b1710 0x7f8e9406d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42202/0 (socket says 172.21.15.143:42202) 2024-01-15T18:17:07.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.213+0000 7f8e91e00640 1 -- 172.21.15.143:0/3706896411 learned_addr learned my addr 172.21.15.143:0/3706896411 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.217+0000 7f8e90a00640 1 -- 172.21.15.143:0/3706896411 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011185408 0 0) 0x7f8e940cb5d0 con 0x7f8e940ce310 2024-01-15T18:17:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.217+0000 7f8e92800640 1 -- 172.21.15.143:0/3706896411 >> v1:172.21.15.204:6789/0 conn(0x7f8e940ce310 legacy=0x7f8e94158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.217+0000 7f8e92800640 1 -- 172.21.15.143:0/3706896411 >> v1:172.21.15.143:6789/0 conn(0x7f8e940b1710 legacy=0x7f8e9406d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.217+0000 7f8e92800640 1 -- 172.21.15.143:0/3706896411 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9406d180 con 0x7f8e94070370 2024-01-15T18:17:07.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.225+0000 7f8e92800640 1 --2- 172.21.15.143:0/3706896411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e940707b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097c0 tx=0x7f8e7c01c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ad931384720f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:07.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.229+0000 7f8e90a00640 1 -- 172.21.15.143:0/3706896411 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940cebe0 con 0x7f8e94070370 2024-01-15T18:17:07.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.229+0000 7f8e90a00640 1 -- 172.21.15.143:0/3706896411 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0a7050 con 0x7f8e94070370 2024-01-15T18:17:07.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.229+0000 7f8e90a00640 1 -- 172.21.15.143:0/3706896411 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0ab820 con 0x7f8e94070370 2024-01-15T18:17:07.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.229+0000 7f8e99000640 1 -- 172.21.15.143:0/3706896411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 msgr2=0x7f8e940707b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.233+0000 7f8e99000640 1 --2- 172.21.15.143:0/3706896411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e940707b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097c0 tx=0x7f8e7c01c280 comp rx=0 tx=0).stop 2024-01-15T18:17:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.233+0000 7f8e99000640 1 -- 172.21.15.143:0/3706896411 shutdown_connections 2024-01-15T18:17:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.233+0000 7f8e99000640 1 --2- 172.21.15.143:0/3706896411 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e940707b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.233+0000 7f8e99000640 1 -- 172.21.15.143:0/3706896411 >> 172.21.15.143:0/3706896411 conn(0x7f8e940b0bf0 msgr2=0x7f8e940b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:07.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.237+0000 7f8e99000640 1 -- 172.21.15.143:0/3706896411 shutdown_connections 2024-01-15T18:17:07.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.237+0000 7f8e99000640 1 -- 172.21.15.143:0/3706896411 wait complete. 2024-01-15T18:17:07.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.237+0000 7f8e99000640 1 Processor -- start 2024-01-15T18:17:07.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.237+0000 7f8e99000640 1 -- start start 2024-01-15T18:17:07.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e99000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e9414c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e99000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e940b1710 0x7f8e94160b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e99000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 0x7f8e9414bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e99000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94161300 con 0x7f8e94070370 2024-01-15T18:17:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e99000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e94151360 con 0x7f8e940b1710 2024-01-15T18:17:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e99000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e941517f0 con 0x7f8e940ce310 2024-01-15T18:17:07.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 0x7f8e9414bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 0x7f8e9414bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:46216/0 (socket says 172.21.15.143:46216) 2024-01-15T18:17:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 -- 172.21.15.143:0/637119293 learned_addr learned my addr 172.21.15.143:0/637119293 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e91e00640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e9414c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 -- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e940b1710 msgr2=0x7f8e94160b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e91400640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e940b1710 0x7f8e94160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e940b1710 0x7f8e94160b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 -- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 msgr2=0x7f8e9414c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e9414c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e92800640 1 -- 172.21.15.143:0/637119293 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c009420 con 0x7f8e940ce310 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.241+0000 7f8e91e00640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e9414c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e91400640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e940b1710 0x7f8e94160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:07.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e92800640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 0x7f8e9414bd40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097c0 tx=0x7f8e7c01aac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0b7450 con 0x7f8e940ce310 2024-01-15T18:17:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0b75b0 con 0x7f8e940ce310 2024-01-15T18:17:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0b7940 con 0x7f8e940ce310 2024-01-15T18:17:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9414cd60 con 0x7f8e940ce310 2024-01-15T18:17:07.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.245+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e94150a30 con 0x7f8e940ce310 2024-01-15T18:17:07.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.249+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e941514f0 con 0x7f8e940ce310 2024-01-15T18:17:07.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.249+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f8e7c0b7aa0 con 0x7f8e940ce310 2024-01-15T18:17:07.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.253+0000 7f8e8a000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8e6c0677e0 0x7f8e6c069ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.253+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f8e7c11dca0 con 0x7f8e940ce310 2024-01-15T18:17:07.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.253+0000 7f8e91e00640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8e6c0677e0 0x7f8e6c069ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.257+0000 7f8e91e00640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8e6c0677e0 0x7f8e6c069ca0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0012f0 tx=0x7f8e8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:07.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.257+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8e7c0eca50 con 0x7f8e940ce310 2024-01-15T18:17:07.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.641+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e94067680 con 0x7f8e6c0677e0 2024-01-15T18:17:07.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.709+0000 7f8e8a000640 1 -- 172.21.15.143:0/637119293 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f8e94067680 con 0x7f8e6c0677e0 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "in-progress", 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_report": { 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:17:07.707 INFO:teuthology.orchestra.run.smithi143.stdout: "regfiles copied": "2/3", 2024-01-15T18:17:07.708 INFO:teuthology.orchestra.run.smithi143.stdout: "subdirs copied": "0/1", 2024-01-15T18:17:07.708 INFO:teuthology.orchestra.run.smithi143.stdout: "symlinks copied": "0/0" 2024-01-15T18:17:07.708 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:07.708 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:07.708 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:07.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.713+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8e6c0677e0 msgr2=0x7f8e6c069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.713+0000 7f8e99000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8e6c0677e0 0x7f8e6c069ca0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0012f0 tx=0x7f8e8c009040 comp rx=0 tx=0).stop 2024-01-15T18:17:07.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.713+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 msgr2=0x7f8e9414bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.713+0000 7f8e99000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 0x7f8e9414bd40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0097c0 tx=0x7f8e7c01aac0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 shutdown_connections 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8e6c0677e0 0x7f8e6c069ca0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e940ce310 0x7f8e9414bd40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e940b1710 0x7f8e94160b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 --2- 172.21.15.143:0/637119293 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e94070370 0x7f8e9414c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 >> 172.21.15.143:0/637119293 conn(0x7f8e940b0bf0 msgr2=0x7f8e94166c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:07.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 shutdown_connections 2024-01-15T18:17:07.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:07.717+0000 7f8e99000640 1 -- 172.21.15.143:0/637119293 wait complete. 2024-01-15T18:17:07.770 INFO:tasks.cephfs.test_volumes:percetange completed = 66.66666666666666 2024-01-15T18:17:09.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:17:09.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 Processor -- start 2024-01-15T18:17:09.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 -- start start 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff9180ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff918071d00 0x7ff918072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91815f310 con 0x7ff91815ec80 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91815fbb0 con 0x7ff918071d00 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff91da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff918072690 con 0x7ff918141660 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff917400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff918071d00 0x7ff918072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff917400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff918071d00 0x7ff918072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56752/0 (socket says 172.21.15.143:56752) 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff917400640 1 -- 172.21.15.143:0/1449842388 learned_addr learned my addr 172.21.15.143:0/1449842388 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:09.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.925+0000 7ff916a00640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff9180ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:09.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.929+0000 7ff917400640 1 -- 172.21.15.143:0/1449842388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 msgr2=0x7ff9180ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:09.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.929+0000 7ff917400640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff9180ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:09.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.929+0000 7ff917400640 1 -- 172.21.15.143:0/1449842388 >> v1:172.21.15.204:6789/0 conn(0x7ff91815ec80 legacy=0x7ff918141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:09.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.929+0000 7ff917400640 1 -- 172.21.15.143:0/1449842388 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91815f5f0 con 0x7ff918071d00 2024-01-15T18:17:09.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.929+0000 7ff916a00640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff9180ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:09.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff917400640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918072150 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff910009e50 tx=0x7ff91001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4b69daaefa695 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:09.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff915600640 1 -- 172.21.15.143:0/1449842388 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a57d0 con 0x7ff918071d00 2024-01-15T18:17:09.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff915600640 1 -- 172.21.15.143:0/1449842388 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9100a5930 con 0x7ff918071d00 2024-01-15T18:17:09.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff915600640 1 -- 172.21.15.143:0/1449842388 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9100a5cc0 con 0x7ff918071d00 2024-01-15T18:17:09.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 -- 172.21.15.143:0/1449842388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 msgr2=0x7ff918072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:09.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918072150 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff910009e50 tx=0x7ff91001ca10 comp rx=0 tx=0).stop 2024-01-15T18:17:09.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 -- 172.21.15.143:0/1449842388 shutdown_connections 2024-01-15T18:17:09.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918072150 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:09.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 --2- 172.21.15.143:0/1449842388 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff9180ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:09.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 -- 172.21.15.143:0/1449842388 >> 172.21.15.143:0/1449842388 conn(0x7ff91806c3f0 msgr2=0x7ff91806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:09.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 -- 172.21.15.143:0/1449842388 shutdown_connections 2024-01-15T18:17:09.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 -- 172.21.15.143:0/1449842388 wait complete. 2024-01-15T18:17:09.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 Processor -- start 2024-01-15T18:17:09.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.933+0000 7ff91da00640 1 -- start start 2024-01-15T18:17:09.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918165480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff91815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff91815ec80 0x7ff9180ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180b2280 con 0x7ff918071d00 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9181659c0 con 0x7ff91815ec80 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff918140810 con 0x7ff918141660 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918165480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918165480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56768/0 (socket says 172.21.15.143:56768) 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 -- 172.21.15.143:0/4211539829 learned_addr learned my addr 172.21.15.143:0/4211539829 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916000640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff91815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff917400640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff91815ec80 0x7ff9180ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 -- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 msgr2=0x7ff91815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff91815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 -- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff91815ec80 msgr2=0x7ff9180ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:09.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff91815ec80 0x7ff9180ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 -- 172.21.15.143:0/4211539829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910009b30 con 0x7ff918071d00 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff916a00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918165480 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff90800d800 tx=0x7ff90800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080096c0 con 0x7ff918071d00 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908009820 con 0x7ff918071d00 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180cd150 con 0x7ff918071d00 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908009bd0 con 0x7ff918071d00 2024-01-15T18:17:09.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff918140dd0 con 0x7ff918071d00 2024-01-15T18:17:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9180b24f0 con 0x7ff918071d00 2024-01-15T18:17:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.937+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7ff908009d30 con 0x7ff918071d00 2024-01-15T18:17:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.941+0000 7ff90ea00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8f0067790 0x7ff8f0069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.941+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7ff908083760 con 0x7ff918071d00 2024-01-15T18:17:09.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.941+0000 7ff916000640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8f0067790 0x7ff8f0069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:09.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.941+0000 7ff916000640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8f0067790 0x7ff8f0069c50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff9181f9700 tx=0x7ff900009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:09.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:09.945+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9080521c0 con 0x7ff918071d00 2024-01-15T18:17:10.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:10.125+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9180ccfd0 con 0x7ff8f0067790 2024-01-15T18:17:12.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.073+0000 7ff90ea00640 1 -- 172.21.15.143:0/4211539829 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7ff9000014f0 con 0x7ff8f0067790 2024-01-15T18:17:12.071 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:12.071 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:17:12.071 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:17:12.072 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:12.072 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8f0067790 msgr2=0x7ff8f0069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8f0067790 0x7ff8f0069c50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff9181f9700 tx=0x7ff900009040 comp rx=0 tx=0).stop 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 msgr2=0x7ff918165480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918165480 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff90800d800 tx=0x7ff90800dcc0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 shutdown_connections 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8f0067790 0x7ff8f0069c50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff91815ec80 0x7ff9180ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff918141660 0x7ff91815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 --2- 172.21.15.143:0/4211539829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff918071d00 0x7ff918165480 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 >> 172.21.15.143:0/4211539829 conn(0x7ff91806c3f0 msgr2=0x7ff9180affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 shutdown_connections 2024-01-15T18:17:12.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.077+0000 7ff91da00640 1 -- 172.21.15.143:0/4211539829 wait complete. 2024-01-15T18:17:12.097 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:17:12.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:17:12.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f502c000640 1 Processor -- start 2024-01-15T18:17:12.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f502c000640 1 -- start start 2024-01-15T18:17:12.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f502c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f502c000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5024070bf0 con 0x7f50240b1da0 2024-01-15T18:17:12.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f502c000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50240b4920 con 0x7f50240b3bf0 2024-01-15T18:17:12.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f502c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502415dba0 con 0x7f50240ce690 2024-01-15T18:17:12.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5028c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f50240b1da0 0x7f50240b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42218/0 (socket says 172.21.15.143:42218) 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5028c00640 1 -- 172.21.15.143:0/2606226661 learned_addr learned my addr 172.21.15.143:0/2606226661 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5029600640 1 --2- 172.21.15.143:0/2606226661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512563466 0 0) 0x7f5024070bf0 con 0x7f50240b1da0 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5004003530 con 0x7f50240b1da0 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763254490 0 0) 0x7f50240b4920 con 0x7f50240b3bf0 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5024070bf0 con 0x7f50240b3bf0 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5029600640 1 -- 172.21.15.143:0/2606226661 >> v1:172.21.15.204:6789/0 conn(0x7f50240b3bf0 legacy=0x7f50240b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 734684065 0 0) 0x7f5004003530 con 0x7f50240b1da0 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5029600640 1 -- 172.21.15.143:0/2606226661 >> v1:172.21.15.143:6789/0 conn(0x7f50240b1da0 legacy=0x7f50240b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.245+0000 7f5029600640 1 -- 172.21.15.143:0/2606226661 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240b4c00 con 0x7f50240ce690 2024-01-15T18:17:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f5029600640 1 --2- 172.21.15.143:0/2606226661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240ac8e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f50140097f0 tx=0x7f501401ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df0c611dcf5de29 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c001680 con 0x7f50240ce690 2024-01-15T18:17:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50140a7050 con 0x7f50240ce690 2024-01-15T18:17:12.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f5023e00640 1 -- 172.21.15.143:0/2606226661 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014004250 con 0x7f50240ce690 2024-01-15T18:17:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 -- 172.21.15.143:0/2606226661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 msgr2=0x7f50240ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 --2- 172.21.15.143:0/2606226661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240ac8e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f50140097f0 tx=0x7f501401ce10 comp rx=0 tx=0).stop 2024-01-15T18:17:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 -- 172.21.15.143:0/2606226661 shutdown_connections 2024-01-15T18:17:12.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 --2- 172.21.15.143:0/2606226661 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240ac8e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 -- 172.21.15.143:0/2606226661 >> 172.21.15.143:0/2606226661 conn(0x7f502406c3f0 msgr2=0x7f502406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 -- 172.21.15.143:0/2606226661 shutdown_connections 2024-01-15T18:17:12.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.249+0000 7f502c000640 1 -- 172.21.15.143:0/2606226661 wait complete. 2024-01-15T18:17:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 Processor -- start 2024-01-15T18:17:12.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 -- start start 2024-01-15T18:17:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 0x7f5024141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50240b3bf0 0x7f502413ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50241423d0 con 0x7f50240b1da0 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5024144fa0 con 0x7f50240b3bf0 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5024142f70 con 0x7f50240ce690 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 0x7f5024141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 0x7f5024141e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56770/0 (socket says 172.21.15.143:56770) 2024-01-15T18:17:12.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 -- 172.21.15.143:0/1355866789 learned_addr learned my addr 172.21.15.143:0/1355866789 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5028c00640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50240b3bf0 0x7f502413ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502a000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 -- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 msgr2=0x7f50240aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 -- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50240b3bf0 msgr2=0x7f502413ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50240b3bf0 0x7f502413ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 -- 172.21.15.143:0/1355866789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5014009420 con 0x7f50240b1da0 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5029600640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 0x7f5024141e90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f50140097f0 tx=0x7f5014019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014003af0 con 0x7f50240b1da0 2024-01-15T18:17:12.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014003c50 con 0x7f50240b1da0 2024-01-15T18:17:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5024143250 con 0x7f50240b1da0 2024-01-15T18:17:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50140b6480 con 0x7f50240b1da0 2024-01-15T18:17:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.253+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5024146600 con 0x7f50240b1da0 2024-01-15T18:17:12.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.257+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5024067500 con 0x7f50240b1da0 2024-01-15T18:17:12.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.257+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f50140b2040 con 0x7f50240b1da0 2024-01-15T18:17:12.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.257+0000 7f5021600640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ffc067790 0x7f4ffc069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.257+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f501411d720 con 0x7f50240b1da0 2024-01-15T18:17:12.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.257+0000 7f5028c00640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ffc067790 0x7f4ffc069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.257+0000 7f5028c00640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ffc067790 0x7f4ffc069c50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5024142e80 tx=0x7f500c0016c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.261+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f50140ec2e0 con 0x7f50240b1da0 2024-01-15T18:17:12.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.441+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f502413fc50 con 0x7f4ffc067790 2024-01-15T18:17:12.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.445+0000 7f5021600640 1 -- 172.21.15.143:0/1355866789 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f502413fc50 con 0x7f4ffc067790 2024-01-15T18:17:12.445 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:12.445 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:17:12.445 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "complete" 2024-01-15T18:17:12.445 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:12.446 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ffc067790 msgr2=0x7f4ffc069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ffc067790 0x7f4ffc069c50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5024142e80 tx=0x7f500c0016c0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 msgr2=0x7f5024141e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 0x7f5024141e90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f50140097f0 tx=0x7f5014019150 comp rx=0 tx=0).stop 2024-01-15T18:17:12.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 shutdown_connections 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ffc067790 0x7f4ffc069c50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f50240ce690 0x7f50240aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f50240b3bf0 0x7f502413ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 --2- 172.21.15.143:0/1355866789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f50240b1da0 0x7f5024141e90 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 >> 172.21.15.143:0/1355866789 conn(0x7f502406c3f0 msgr2=0x7f5024161130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 shutdown_connections 2024-01-15T18:17:12.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.449+0000 7f502c000640 1 -- 172.21.15.143:0/1355866789 wait complete. 2024-01-15T18:17:12.467 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:17:12.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.605+0000 7f8338c00640 1 Processor -- start 2024-01-15T18:17:12.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.605+0000 7f8338c00640 1 -- start start 2024-01-15T18:17:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8338c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8334141660 0x7f83340ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8338c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8334071d00 0x7f8334072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8338c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833415f310 con 0x7f833415ec80 2024-01-15T18:17:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8338c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833415fbb0 con 0x7f8334141660 2024-01-15T18:17:12.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8338c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833415f130 con 0x7f8334071d00 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8334141660 0x7f83340ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8334141660 0x7f83340ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56784/0 (socket says 172.21.15.143:56784) 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 -- 172.21.15.143:0/3020916663 learned_addr learned my addr 172.21.15.143:0/3020916663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8332800640 1 --2- 172.21.15.143:0/3020916663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8334071d00 0x7f8334072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 -- 172.21.15.143:0/3020916663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8334071d00 msgr2=0x7f8334072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 --2- 172.21.15.143:0/3020916663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8334071d00 0x7f8334072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 -- 172.21.15.143:0/3020916663 >> v1:172.21.15.204:6789/0 conn(0x7f833415ec80 legacy=0x7f8334141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.609+0000 7f8331e00640 1 -- 172.21.15.143:0/3020916663 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833415f5f0 con 0x7f8334141660 2024-01-15T18:17:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8331e00640 1 --2- 172.21.15.143:0/3020916663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334141660 0x7f83340ce560 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8324009870 tx=0x7f832401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42afe09846c5fef0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8330a00640 1 -- 172.21.15.143:0/3020916663 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83240a7070 con 0x7f8334141660 2024-01-15T18:17:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8330a00640 1 -- 172.21.15.143:0/3020916663 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83240ab430 con 0x7f8334141660 2024-01-15T18:17:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8330a00640 1 -- 172.21.15.143:0/3020916663 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83240ab7c0 con 0x7f8334141660 2024-01-15T18:17:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 -- 172.21.15.143:0/3020916663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334141660 msgr2=0x7f83340ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 --2- 172.21.15.143:0/3020916663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334141660 0x7f83340ce560 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f8324009870 tx=0x7f832401c540 comp rx=0 tx=0).stop 2024-01-15T18:17:12.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 -- 172.21.15.143:0/3020916663 shutdown_connections 2024-01-15T18:17:12.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 --2- 172.21.15.143:0/3020916663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8334071d00 0x7f8334072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 --2- 172.21.15.143:0/3020916663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334141660 0x7f83340ce560 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 -- 172.21.15.143:0/3020916663 >> 172.21.15.143:0/3020916663 conn(0x7f833406c3f0 msgr2=0x7f833406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 -- 172.21.15.143:0/3020916663 shutdown_connections 2024-01-15T18:17:12.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 -- 172.21.15.143:0/3020916663 wait complete. 2024-01-15T18:17:12.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 Processor -- start 2024-01-15T18:17:12.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.613+0000 7f8338c00640 1 -- start start 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 0x7f833415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8334141660 0x7f8334138b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f833415ec80 0x7f8334139050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340b3b80 con 0x7f8334071d00 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83340b3000 con 0x7f8334141660 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83340b32e0 con 0x7f833415ec80 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 0x7f833415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 0x7f833415aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56788/0 (socket says 172.21.15.143:56788) 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 -- 172.21.15.143:0/2584805051 learned_addr learned my addr 172.21.15.143:0/2584805051 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331400640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8334141660 0x7f8334138b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8332800640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f833415ec80 0x7f8334139050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 -- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f833415ec80 msgr2=0x7f8334139050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f833415ec80 0x7f8334139050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 -- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8334141660 msgr2=0x7f8334138b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8334141660 0x7f8334138b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 -- 172.21.15.143:0/2584805051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240094d0 con 0x7f8334071d00 2024-01-15T18:17:12.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331400640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8334141660 0x7f8334138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8331e00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 0x7f833415aae0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f83240ad040 tx=0x7f83240192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83240bc070 con 0x7f8334071d00 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83240abe10 con 0x7f8334071d00 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83340cd150 con 0x7f8334071d00 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83240b4870 con 0x7f8334071d00 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83340b38a0 con 0x7f8334071d00 2024-01-15T18:17:12.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.617+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8334067500 con 0x7f8334071d00 2024-01-15T18:17:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.621+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f83240b49d0 con 0x7f8334071d00 2024-01-15T18:17:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.621+0000 7f832a000640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f830c067790 0x7f830c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.621+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f832411cb40 con 0x7f8334071d00 2024-01-15T18:17:12.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.621+0000 7f8331400640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f830c067790 0x7f830c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.621+0000 7f8331400640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f830c067790 0x7f830c069c50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f833413c2c0 tx=0x7f831c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.625+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f83240eb5a0 con 0x7f8334071d00 2024-01-15T18:17:12.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.793+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f83340ccfd0 con 0x7f830c067790 2024-01-15T18:17:12.793 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f832a000640 1 -- 172.21.15.143:0/2584805051 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f831c0018e0 con 0x7f830c067790 2024-01-15T18:17:12.793 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:17:12.794 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:12.794 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:17:12.794 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:12.794 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f830c067790 msgr2=0x7f830c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f830c067790 0x7f830c069c50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f833413c2c0 tx=0x7f831c009040 comp rx=0 tx=0).stop 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 msgr2=0x7f833415aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 0x7f833415aae0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f83240ad040 tx=0x7f83240192e0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 shutdown_connections 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f830c067790 0x7f830c069c50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f833415ec80 0x7f8334139050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8334141660 0x7f8334138b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 --2- 172.21.15.143:0/2584805051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8334071d00 0x7f833415aae0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 >> 172.21.15.143:0/2584805051 conn(0x7f833406c3f0 msgr2=0x7f83340afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 shutdown_connections 2024-01-15T18:17:12.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.797+0000 7f8338c00640 1 -- 172.21.15.143:0/2584805051 wait complete. 2024-01-15T18:17:12.816 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:17:12.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab5f400640 1 Processor -- start 2024-01-15T18:17:12.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab5f400640 1 -- start start 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab5f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab580cfb30 0x7fab5806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab5f400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580cffd0 con 0x7fab581607c0 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab5f400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab580d02b0 con 0x7fab580cbf50 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab5f400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab580d1ce0 con 0x7fab580cfb30 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab57e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab580cfb30 0x7fab5806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab57e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab580cfb30 0x7fab5806bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56790/0 (socket says 172.21.15.143:56790) 2024-01-15T18:17:12.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab57e00640 1 -- 172.21.15.143:0/261836065 learned_addr learned my addr 172.21.15.143:0/261836065 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:12.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab57e00640 1 -- 172.21.15.143:0/261836065 >> v1:172.21.15.204:6789/0 conn(0x7fab580cbf50 legacy=0x7fab580cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab57e00640 1 -- 172.21.15.143:0/261836065 >> v1:172.21.15.143:6789/0 conn(0x7fab581607c0 legacy=0x7fab5806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.961+0000 7fab57e00640 1 -- 172.21.15.143:0/261836065 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580cb470 con 0x7fab580cfb30 2024-01-15T18:17:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab57e00640 1 --2- 172.21.15.143:0/261836065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab580cfb30 0x7fab5806bec0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fab40009870 tx=0x7fab4001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3850d8b326433b74 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab57400640 1 -- 172.21.15.143:0/261836065 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab400a8a30 con 0x7fab580cfb30 2024-01-15T18:17:12.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab57400640 1 -- 172.21.15.143:0/261836065 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab400a8b90 con 0x7fab580cfb30 2024-01-15T18:17:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab57400640 1 -- 172.21.15.143:0/261836065 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab400b0440 con 0x7fab580cfb30 2024-01-15T18:17:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 -- 172.21.15.143:0/261836065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab580cfb30 msgr2=0x7fab5806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 --2- 172.21.15.143:0/261836065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab580cfb30 0x7fab5806bec0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fab40009870 tx=0x7fab4001ed30 comp rx=0 tx=0).stop 2024-01-15T18:17:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 -- 172.21.15.143:0/261836065 shutdown_connections 2024-01-15T18:17:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 --2- 172.21.15.143:0/261836065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab580cfb30 0x7fab5806bec0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 -- 172.21.15.143:0/261836065 >> 172.21.15.143:0/261836065 conn(0x7fab580d0e20 msgr2=0x7fab580d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 -- 172.21.15.143:0/261836065 shutdown_connections 2024-01-15T18:17:12.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.965+0000 7fab5f400640 1 -- 172.21.15.143:0/261836065 wait complete. 2024-01-15T18:17:12.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 Processor -- start 2024-01-15T18:17:12.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 -- start start 2024-01-15T18:17:12.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab580cbf50 0x7fab580c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab580cfb30 0x7fab580c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 0x7fab58155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580c4910 con 0x7fab581607c0 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580c8ee0 con 0x7fab580cbf50 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5f400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580c91c0 con 0x7fab580cfb30 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 0x7fab58155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 0x7fab58155480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56792/0 (socket says 172.21.15.143:56792) 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 -- 172.21.15.143:0/1027738549 learned_addr learned my addr 172.21.15.143:0/1027738549 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5ca00640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab580cbf50 0x7fab580c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab57e00640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab580cfb30 0x7fab580c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 -- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab580cfb30 msgr2=0x7fab580c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab580cfb30 0x7fab580c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 -- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab580cbf50 msgr2=0x7fab580c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab580cbf50 0x7fab580c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 -- 172.21.15.143:0/1027738549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400094d0 con 0x7fab581607c0 2024-01-15T18:17:12.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.969+0000 7fab5d400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 0x7fab58155480 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c770 tx=0x7fab5000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.973+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50003b80 con 0x7fab581607c0 2024-01-15T18:17:12.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.973+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50003ce0 con 0x7fab581607c0 2024-01-15T18:17:12.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.973+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000aa90 con 0x7fab581607c0 2024-01-15T18:17:12.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.973+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab580c94a0 con 0x7fab581607c0 2024-01-15T18:17:12.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.973+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581f3b80 con 0x7fab581607c0 2024-01-15T18:17:12.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.973+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5806d4b0 con 0x7fab581607c0 2024-01-15T18:17:12.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.977+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7fab5000abf0 con 0x7fab581607c0 2024-01-15T18:17:12.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.977+0000 7fab54c00640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab30067790 0x7fab30069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.977+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fab50082c00 con 0x7fab581607c0 2024-01-15T18:17:12.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.977+0000 7fab5ca00640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab30067790 0x7fab30069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.977+0000 7fab5ca00640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab30067790 0x7fab30069c50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fab48007fd0 tx=0x7fab48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:12.981+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab50051940 con 0x7fab581607c0 2024-01-15T18:17:13.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.157+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab580c9330 con 0x7fab30067790 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.157+0000 7fab54c00640 1 -- 172.21.15.143:0/1027738549 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7fab580c9330 con 0x7fab30067790 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1clone1" 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:13.157 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:17:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab30067790 msgr2=0x7fab30069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab30067790 0x7fab30069c50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fab48007fd0 tx=0x7fab48009040 comp rx=0 tx=0).stop 2024-01-15T18:17:13.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 msgr2=0x7fab58155480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 0x7fab58155480 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fab5000c770 tx=0x7fab5000cc30 comp rx=0 tx=0).stop 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 shutdown_connections 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab30067790 0x7fab30069c50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab581607c0 0x7fab58155480 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab580cfb30 0x7fab580c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 --2- 172.21.15.143:0/1027738549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab580cbf50 0x7fab580c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 >> 172.21.15.143:0/1027738549 conn(0x7fab580d0e20 msgr2=0x7fab5815dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 shutdown_connections 2024-01-15T18:17:13.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.161+0000 7fab5f400640 1 -- 172.21.15.143:0/1027738549 wait complete. 2024-01-15T18:17:13.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:17:13.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.321+0000 7f8f90000640 1 Processor -- start 2024-01-15T18:17:13.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.321+0000 7f8f90000640 1 -- start start 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f90000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f880ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f90000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f88070bf0 con 0x7f8f880ce690 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f90000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f880b4920 con 0x7f8f880b24b0 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f90000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8815dba0 con 0x7f8f880b4300 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f880ce690 0x7f8f880b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42246/0 (socket says 172.21.15.143:42246) 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 -- 172.21.15.143:0/2791900487 learned_addr learned my addr 172.21.15.143:0/2791900487 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:13.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8e000640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f880ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411554663 0 0) 0x7f8f88070bf0 con 0x7f8f880ce690 2024-01-15T18:17:13.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f68003530 con 0x7f8f880ce690 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267452337 0 0) 0x7f8f880b4920 con 0x7f8f880b24b0 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f88070bf0 con 0x7f8f880b24b0 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874420599 0 0) 0x7f8f68003530 con 0x7f8f880ce690 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 msgr2=0x7f8f880ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f880ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 >> v1:172.21.15.204:6789/0 conn(0x7f8f880b24b0 legacy=0x7f8f880b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f880b4c00 con 0x7f8f880ce690 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8f78001680 con 0x7f8f880ce690 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8e000640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f880ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 >> v1:172.21.15.143:6789/0 conn(0x7f8f880ce690 legacy=0x7f8f880b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f68003b40 0x7f8f68003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f680047d0 0x7f8f68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f68005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8815dba0 con 0x7f8f68003b40 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6800b9f0 con 0x7f8f680047d0 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6800bbb0 con 0x7f8f880b4300 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f68003b40 0x7f8f68003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8cc00640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f680047d0 0x7f8f68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8e000640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f68005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 -- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 msgr2=0x7f8f68005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f880b4300 0x7f8f68005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 -- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f680047d0 msgr2=0x7f8f68004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f680047d0 0x7f8f68004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.325+0000 7f8f8d600640 1 -- 172.21.15.143:0/2791900487 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80009a70 con 0x7f8f68003b40 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f68003b40 0x7f8f68003f40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c160 tx=0x7f8f7801cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b22d110490d54c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f780058e0 con 0x7f8f68003b40 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78005a40 con 0x7f8f68003b40 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f87e00640 1 -- 172.21.15.143:0/2791900487 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78005df0 con 0x7f8f68003b40 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 -- 172.21.15.143:0/2791900487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f68003b40 msgr2=0x7f8f68003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 --2- 172.21.15.143:0/2791900487 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f68003b40 0x7f8f68003f40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c160 tx=0x7f8f7801cb30 comp rx=0 tx=0).stop 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f8e000640 1 -- 172.21.15.143:0/2791900487 reap_dead start 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 -- 172.21.15.143:0/2791900487 shutdown_connections 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 -- 172.21.15.143:0/2791900487 >> 172.21.15.143:0/2791900487 conn(0x7f8f8806c3f0 msgr2=0x7f8f8806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 -- 172.21.15.143:0/2791900487 shutdown_connections 2024-01-15T18:17:13.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 -- 172.21.15.143:0/2791900487 wait complete. 2024-01-15T18:17:13.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 Processor -- start 2024-01-15T18:17:13.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.329+0000 7f8f90000640 1 -- start start 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f68003b40 0x7f8f880ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 0x7f8f8815a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f88145a80 0x7f8f88145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f880af650 con 0x7f8f880b01d0 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f88146440 con 0x7f8f88145a80 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f880ae9d0 con 0x7f8f68003b40 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 0x7f8f8815a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 0x7f8f8815a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56806/0 (socket says 172.21.15.143:56806) 2024-01-15T18:17:13.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 -- 172.21.15.143:0/2848727189 learned_addr learned my addr 172.21.15.143:0/2848727189 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f68003b40 0x7f8f880ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8e000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f88145a80 0x7f8f88145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 -- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f68003b40 msgr2=0x7f8f880ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f68003b40 0x7f8f880ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 -- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f88145a80 msgr2=0x7f8f88145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f88145a80 0x7f8f88145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 -- 172.21.15.143:0/2848727189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f88071e40 con 0x7f8f880b01d0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f8cc00640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 0x7f8f8815a020 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000f7a0 tx=0x7f8f7000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70005110 con 0x7f8f880b01d0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70005270 con 0x7f8f880b01d0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f880aecb0 con 0x7f8f880b01d0 2024-01-15T18:17:13.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700067e0 con 0x7f8f880b01d0 2024-01-15T18:17:13.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f880af270 con 0x7f8f880b01d0 2024-01-15T18:17:13.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.333+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f88067500 con 0x7f8f880b01d0 2024-01-15T18:17:13.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.337+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f8f70006940 con 0x7f8f880b01d0 2024-01-15T18:17:13.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.337+0000 7f8f85600640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f60067790 0x7f8f60069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.337+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f8f70082460 con 0x7f8f880b01d0 2024-01-15T18:17:13.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.337+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f60067790 0x7f8f60069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.337+0000 7f8f8d600640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f60067790 0x7f8f60069c50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8f780aa000 tx=0x7f8f78001990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:13.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.341+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8f70051180 con 0x7f8f880b01d0 2024-01-15T18:17:13.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.517+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f881f61e0 con 0x7f8f60067790 2024-01-15T18:17:13.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f85600640 1 -- 172.21.15.143:0/2848727189 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8f881f61e0 con 0x7f8f60067790 2024-01-15T18:17:13.525 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:13.526 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:17:13.526 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:13.526 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:17:13.526 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:13.526 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:17:13.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f60067790 msgr2=0x7f8f60069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f60067790 0x7f8f60069c50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8f780aa000 tx=0x7f8f78001990 comp rx=0 tx=0).stop 2024-01-15T18:17:13.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 msgr2=0x7f8f8815a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 0x7f8f8815a020 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000f7a0 tx=0x7f8f7000fc60 comp rx=0 tx=0).stop 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 shutdown_connections 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8f60067790 0x7f8f60069c50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8f88145a80 0x7f8f88145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f880b01d0 0x7f8f8815a020 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 --2- 172.21.15.143:0/2848727189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8f68003b40 0x7f8f880ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 >> 172.21.15.143:0/2848727189 conn(0x7f8f8806c3f0 msgr2=0x7f8f8815ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 shutdown_connections 2024-01-15T18:17:13.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.529+0000 7f8f90000640 1 -- 172.21.15.143:0/2848727189 wait complete. 2024-01-15T18:17:13.548 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:17:13.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.693+0000 7f8b54200640 1 Processor -- start 2024-01-15T18:17:13.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.693+0000 7f8b54200640 1 -- start start 2024-01-15T18:17:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.693+0000 7f8b54200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c141660 0x7f8b4c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.693+0000 7f8b54200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c15f310 con 0x7f8b4c15ec80 2024-01-15T18:17:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.693+0000 7f8b54200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c15f5f0 con 0x7f8b4c071d00 2024-01-15T18:17:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.693+0000 7f8b54200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c15fbb0 con 0x7f8b4c141660 2024-01-15T18:17:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b50e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8b4c15ec80 0x7f8b4c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42248/0 (socket says 172.21.15.143:42248) 2024-01-15T18:17:13.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b50e00640 1 -- 172.21.15.143:0/925115617 learned_addr learned my addr 172.21.15.143:0/925115617 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b51800640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c141660 0x7f8b4c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855297170 0 0) 0x7f8b4c15f310 con 0x7f8b4c15ec80 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b2c003530 con 0x7f8b4c15ec80 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587829521 0 0) 0x7f8b4c15f5f0 con 0x7f8b4c071d00 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b4c15f310 con 0x7f8b4c071d00 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489706099 0 0) 0x7f8b2c003530 con 0x7f8b4c15ec80 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c141660 msgr2=0x7f8b4c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c141660 0x7f8b4c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 >> v1:172.21.15.204:6789/0 conn(0x7f8b4c071d00 legacy=0x7f8b4c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c15f8d0 con 0x7f8b4c15ec80 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8b34001680 con 0x7f8b4c15ec80 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 >> v1:172.21.15.143:6789/0 conn(0x7f8b4c15ec80 legacy=0x7f8b4c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c003b40 0x7f8b2c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2c0047d0 0x7f8b2c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b2c005190 0x7f8b2c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c15fbb0 con 0x7f8b2c005190 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c00bf80 con 0x7f8b2c0047d0 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c00c140 con 0x7f8b2c003b40 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b51800640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b4c141660 0x7f8b4c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b52200640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b2c005190 0x7f8b2c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b50e00640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2c0047d0 0x7f8b2c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b51800640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c003b40 0x7f8b2c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b52200640 1 -- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c003b40 msgr2=0x7f8b2c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b52200640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c003b40 0x7f8b2c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b52200640 1 -- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2c0047d0 msgr2=0x7f8b2c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b52200640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b2c0047d0 0x7f8b2c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.697+0000 7f8b52200640 1 -- 172.21.15.143:0/925115617 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c003530 con 0x7f8b2c005190 2024-01-15T18:17:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b52200640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b2c005190 0x7f8b2c00b990 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f8b44001fd0 tx=0x7f8b4401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dc28e47293311f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34007ea0 con 0x7f8b2c005190 2024-01-15T18:17:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44003ba0 con 0x7f8b2c005190 2024-01-15T18:17:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b4be00640 1 -- 172.21.15.143:0/925115617 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440050d0 con 0x7f8b2c005190 2024-01-15T18:17:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 -- 172.21.15.143:0/925115617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b2c005190 msgr2=0x7f8b2c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 --2- 172.21.15.143:0/925115617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b2c005190 0x7f8b2c00b990 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f8b44001fd0 tx=0x7f8b4401c8e0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b52200640 1 -- 172.21.15.143:0/925115617 reap_dead start 2024-01-15T18:17:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 -- 172.21.15.143:0/925115617 shutdown_connections 2024-01-15T18:17:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 -- 172.21.15.143:0/925115617 >> 172.21.15.143:0/925115617 conn(0x7f8b4c06c3f0 msgr2=0x7f8b4c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 -- 172.21.15.143:0/925115617 shutdown_connections 2024-01-15T18:17:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 -- 172.21.15.143:0/925115617 wait complete. 2024-01-15T18:17:13.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 Processor -- start 2024-01-15T18:17:13.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.701+0000 7f8b54200640 1 -- start start 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c005190 0x7f8b4c13b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 0x7f8b4c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0a8f00 0x7f8b4c0a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0a98c0 con 0x7f8b4c15c940 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c15e550 con 0x7f8b4c0a8f00 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c15d4b0 con 0x7f8b2c005190 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 0x7f8b4c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 0x7f8b4c15cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56834/0 (socket says 172.21.15.143:56834) 2024-01-15T18:17:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 -- 172.21.15.143:0/729567442 learned_addr learned my addr 172.21.15.143:0/729567442 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b52200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0a8f00 0x7f8b4c0a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b51800640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c005190 0x7f8b4c13b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 -- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c005190 msgr2=0x7f8b4c13b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c005190 0x7f8b4c13b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 -- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0a8f00 msgr2=0x7f8b4c0a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0a8f00 0x7f8b4c0a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 -- 172.21.15.143:0/729567442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c15d790 con 0x7f8b4c15c940 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b50e00640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 0x7f8b4c15cda0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8b34012510 tx=0x7f8b34012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34014750 con 0x7f8b4c15c940 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b340148b0 con 0x7f8b4c15c940 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c15da70 con 0x7f8b4c15c940 2024-01-15T18:17:13.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34014c40 con 0x7f8b4c15c940 2024-01-15T18:17:13.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c15e030 con 0x7f8b4c15c940 2024-01-15T18:17:13.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.705+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c1425c0 con 0x7f8b4c15c940 2024-01-15T18:17:13.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.709+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f8b340153e0 con 0x7f8b4c15c940 2024-01-15T18:17:13.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.709+0000 7f8b49600640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b24067790 0x7f8b24069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:13.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.709+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f8b34085a90 con 0x7f8b4c15c940 2024-01-15T18:17:13.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.709+0000 7f8b51800640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b24067790 0x7f8b24069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:13.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.709+0000 7f8b51800640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b24067790 0x7f8b24069c50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0b1aa0 tx=0x7f8b3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:13.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.713+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b34054460 con 0x7f8b4c15c940 2024-01-15T18:17:13.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.893+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b4c141a40 con 0x7f8b24067790 2024-01-15T18:17:13.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.917+0000 7f8b49600640 1 -- 172.21.15.143:0/729567442 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b3c014b00 con 0x7f8b24067790 2024-01-15T18:17:13.916 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b24067790 msgr2=0x7f8b24069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b24067790 0x7f8b24069c50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0b1aa0 tx=0x7f8b3c008040 comp rx=0 tx=0).stop 2024-01-15T18:17:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 msgr2=0x7f8b4c15cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 0x7f8b4c15cda0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f8b34012510 tx=0x7f8b34012910 comp rx=0 tx=0).stop 2024-01-15T18:17:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 shutdown_connections 2024-01-15T18:17:13.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b24067790 0x7f8b24069c50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b4c0a8f00 0x7f8b4c0a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b4c15c940 0x7f8b4c15cda0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 --2- 172.21.15.143:0/729567442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b2c005190 0x7f8b4c13b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 >> 172.21.15.143:0/729567442 conn(0x7f8b4c06c3f0 msgr2=0x7f8b4c0af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 shutdown_connections 2024-01-15T18:17:13.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:13.921+0000 7f8b54200640 1 -- 172.21.15.143:0/729567442 wait complete. 2024-01-15T18:17:13.939 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:17:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 Processor -- start 2024-01-15T18:17:14.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 -- start start 2024-01-15T18:17:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fd415d3f0 0x7f0fd40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0fd40d0050 0x7f0fd406bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd406c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40cb480 con 0x7f0fd415d3f0 2024-01-15T18:17:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40cb6d0 con 0x7f0fd40d0050 2024-01-15T18:17:14.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fdc600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40cb9b0 con 0x7f0fd40b0490 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fd415d3f0 0x7f0fd40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fd415d3f0 0x7f0fd40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56844/0 (socket says 172.21.15.143:56844) 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 -- 172.21.15.143:0/977854464 learned_addr learned my addr 172.21.15.143:0/977854464 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9200640 1 --2- 172.21.15.143:0/977854464 >> v2:172.21.15.204:3300/0 conn(0x7f0fd40d0050 0x7f0fd406bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fda600640 1 --2- 172.21.15.143:0/977854464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 -- 172.21.15.143:0/977854464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 msgr2=0x7f0fd406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 --2- 172.21.15.143:0/977854464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd406c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 -- 172.21.15.143:0/977854464 >> v2:172.21.15.204:3300/0 conn(0x7f0fd40d0050 msgr2=0x7f0fd406bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 --2- 172.21.15.143:0/977854464 >> v2:172.21.15.204:3300/0 conn(0x7f0fd40d0050 0x7f0fd406bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9c00640 1 -- 172.21.15.143:0/977854464 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40cbc90 con 0x7f0fd415d3f0 2024-01-15T18:17:14.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.085+0000 7f0fd9200640 1 --2- 172.21.15.143:0/977854464 >> v2:172.21.15.204:3300/0 conn(0x7f0fd40d0050 0x7f0fd406bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:14.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fd9c00640 1 --2- 172.21.15.143:0/977854464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd415d3f0 0x7f0fd40cfb10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4070fe0 tx=0x7f0fc401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c82a77019aed56 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fd3e00640 1 -- 172.21.15.143:0/977854464 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40aa610 con 0x7f0fd415d3f0 2024-01-15T18:17:14.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fd3e00640 1 -- 172.21.15.143:0/977854464 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc40a6040 con 0x7f0fd415d3f0 2024-01-15T18:17:14.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fd3e00640 1 -- 172.21.15.143:0/977854464 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40aa990 con 0x7f0fd415d3f0 2024-01-15T18:17:14.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 -- 172.21.15.143:0/977854464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd415d3f0 msgr2=0x7f0fd40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 --2- 172.21.15.143:0/977854464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd415d3f0 0x7f0fd40cfb10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4070fe0 tx=0x7f0fc401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 -- 172.21.15.143:0/977854464 shutdown_connections 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 --2- 172.21.15.143:0/977854464 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd406c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 --2- 172.21.15.143:0/977854464 >> v2:172.21.15.204:3300/0 conn(0x7f0fd40d0050 0x7f0fd406bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 --2- 172.21.15.143:0/977854464 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd415d3f0 0x7f0fd40cfb10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 -- 172.21.15.143:0/977854464 >> 172.21.15.143:0/977854464 conn(0x7f0fd40d0e30 msgr2=0x7f0fd40d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 -- 172.21.15.143:0/977854464 shutdown_connections 2024-01-15T18:17:14.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.089+0000 7f0fdc600640 1 -- 172.21.15.143:0/977854464 wait complete. 2024-01-15T18:17:14.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 Processor -- start 2024-01-15T18:17:14.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 -- start start 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 0x7f0fd41603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd415d3f0 0x7f0fd406bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4160910 con 0x7f0fd40d0050 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4157890 con 0x7f0fd415d3f0 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40cade0 con 0x7f0fd40b0490 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 0x7f0fd41603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 0x7f0fd41603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56850/0 (socket says 172.21.15.143:56850) 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 -- 172.21.15.143:0/3810749028 learned_addr learned my addr 172.21.15.143:0/3810749028 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9c00640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fda600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd415d3f0 0x7f0fd406bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 -- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 msgr2=0x7f0fd4156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd4156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 -- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd415d3f0 msgr2=0x7f0fd406bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd415d3f0 0x7f0fd406bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 -- 172.21.15.143:0/3810749028 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40094d0 con 0x7f0fd40d0050 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd9200640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 0x7f0fd41603d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e810 tx=0x7f0fbc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00c8d0 con 0x7f0fd40d0050 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc00ca30 con 0x7f0fd40d0050 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd41575b0 con 0x7f0fd40d0050 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00cde0 con 0x7f0fd40d0050 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.093+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd40cab00 con 0x7f0fd40d0050 2024-01-15T18:17:14.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.097+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd4067500 con 0x7f0fd40d0050 2024-01-15T18:17:14.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.097+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f0fbc013410 con 0x7f0fd40d0050 2024-01-15T18:17:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.097+0000 7f0fd1600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0fac067790 0x7f0fac069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.097+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f0fbc082ed0 con 0x7f0fd40d0050 2024-01-15T18:17:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.097+0000 7f0fd9c00640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0fac067790 0x7f0fac069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.097+0000 7f0fd9c00640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0fac067790 0x7f0fac069c50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40ae000 tx=0x7f0fc40b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.101+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0fbc051930 con 0x7f0fd40d0050 2024-01-15T18:17:14.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.273+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fd406cc10 con 0x7f0fac067790 2024-01-15T18:17:14.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.289+0000 7f0fd1600640 1 -- 172.21.15.143:0/3810749028 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0fd406cc10 con 0x7f0fac067790 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0fac067790 msgr2=0x7f0fac069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0fac067790 0x7f0fac069c50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40ae000 tx=0x7f0fc40b5040 comp rx=0 tx=0).stop 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 msgr2=0x7f0fd41603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 0x7f0fd41603d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00e810 tx=0x7f0fbc00ecd0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 shutdown_connections 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0fac067790 0x7f0fac069c50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fd415d3f0 0x7f0fd406bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fd40d0050 0x7f0fd41603d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 --2- 172.21.15.143:0/3810749028 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fd40b0490 0x7f0fd4156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 >> 172.21.15.143:0/3810749028 conn(0x7f0fd40d0e30 msgr2=0x7f0fd40afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 shutdown_connections 2024-01-15T18:17:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f0fdc600640 1 -- 172.21.15.143:0/3810749028 wait complete. 2024-01-15T18:17:14.292 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.292 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.292 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.292 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.293 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.293+0000 7f3caca00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.310 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.313+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.310 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.313+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.310 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.313+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.310 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.313+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.310 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:14.313+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:14.315 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-15T18:17:14.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9898c00640 1 Processor -- start 2024-01-15T18:17:14.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9898c00640 1 -- start start 2024-01-15T18:17:14.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9898c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98940c6230 0x7f98940cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9898c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98940c4d90 con 0x7f989415b260 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9898c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989415b640 con 0x7f98940cdf20 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9898c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98940cf800 con 0x7f98940c6230 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9892800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98940c6230 0x7f98940cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9892800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98940c6230 0x7f98940cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56890/0 (socket says 172.21.15.143:56890) 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9892800640 1 -- 172.21.15.143:0/1737104332 learned_addr learned my addr 172.21.15.143:0/1737104332 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9892800640 1 -- 172.21.15.143:0/1737104332 >> v1:172.21.15.204:6789/0 conn(0x7f98940cdf20 legacy=0x7f98940cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9892800640 1 -- 172.21.15.143:0/1737104332 >> v1:172.21.15.143:6789/0 conn(0x7f989415b260 legacy=0x7f98940cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.465+0000 7f9892800640 1 -- 172.21.15.143:0/1737104332 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940cfae0 con 0x7f98940c6230 2024-01-15T18:17:14.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9892800640 1 --2- 172.21.15.143:0/1737104332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940c6230 0x7f98940cc1f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f988c009e50 tx=0x7f988c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44af60c89939e082 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9890a00640 1 -- 172.21.15.143:0/1737104332 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0a5860 con 0x7f98940c6230 2024-01-15T18:17:14.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9890a00640 1 -- 172.21.15.143:0/1737104332 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98940ccf40 con 0x7f98940c6230 2024-01-15T18:17:14.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9890a00640 1 -- 172.21.15.143:0/1737104332 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0a5be0 con 0x7f98940c6230 2024-01-15T18:17:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 -- 172.21.15.143:0/1737104332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940c6230 msgr2=0x7f98940cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 --2- 172.21.15.143:0/1737104332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940c6230 0x7f98940cc1f0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f988c009e50 tx=0x7f988c01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 -- 172.21.15.143:0/1737104332 shutdown_connections 2024-01-15T18:17:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 --2- 172.21.15.143:0/1737104332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940c6230 0x7f98940cc1f0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 -- 172.21.15.143:0/1737104332 >> 172.21.15.143:0/1737104332 conn(0x7f98940d0ed0 msgr2=0x7f98940d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:14.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 -- 172.21.15.143:0/1737104332 shutdown_connections 2024-01-15T18:17:14.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 -- 172.21.15.143:0/1737104332 wait complete. 2024-01-15T18:17:14.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 Processor -- start 2024-01-15T18:17:14.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.473+0000 7f9898c00640 1 -- start start 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98940c6230 0x7f989406b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 0x7f989406bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989415b260 0x7f98940b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941538f0 con 0x7f98940cdf20 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9894157fe0 con 0x7f989415b260 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9894157740 con 0x7f98940c6230 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 0x7f989406bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 0x7f989406bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56898/0 (socket says 172.21.15.143:56898) 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 -- 172.21.15.143:0/4056110114 learned_addr learned my addr 172.21.15.143:0/4056110114 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9892800640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989415b260 0x7f98940b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891e00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98940c6230 0x7f989406b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 -- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98940c6230 msgr2=0x7f989406b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98940c6230 0x7f989406b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 -- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989415b260 msgr2=0x7f98940b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989415b260 0x7f98940b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 -- 172.21.15.143:0/4056110114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009b30 con 0x7f98940cdf20 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9891400640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 0x7f989406bae0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b440 tx=0x7f987c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c011890 con 0x7f98940cdf20 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0119f0 con 0x7f98940cdf20 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c011da0 con 0x7f98940cdf20 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9894157a20 con 0x7f98940cdf20 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98940b3c20 con 0x7f98940cdf20 2024-01-15T18:17:14.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.477+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9894067500 con 0x7f98940cdf20 2024-01-15T18:17:14.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.481+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7f987c004020 con 0x7f98940cdf20 2024-01-15T18:17:14.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.481+0000 7f988a000640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f986c067790 0x7f986c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.481+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f987c082da0 con 0x7f98940cdf20 2024-01-15T18:17:14.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.481+0000 7f9891e00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f986c067790 0x7f986c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.481+0000 7f9891e00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f986c067790 0x7f986c069c50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9884007fd0 tx=0x7f9884009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.485+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f987c051ae0 con 0x7f98940cdf20 2024-01-15T18:17:14.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.661+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f98940c4770 con 0x7f986c067790 2024-01-15T18:17:14.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.673+0000 7f988a000640 1 -- 172.21.15.143:0/4056110114 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f98940c4770 con 0x7f986c067790 2024-01-15T18:17:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f986c067790 msgr2=0x7f986c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f986c067790 0x7f986c069c50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9884007fd0 tx=0x7f9884009040 comp rx=0 tx=0).stop 2024-01-15T18:17:14.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 msgr2=0x7f989406bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 0x7f989406bae0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f987c00b440 tx=0x7f987c00b900 comp rx=0 tx=0).stop 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 shutdown_connections 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f986c067790 0x7f986c069c50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f989415b260 0x7f98940b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98940cdf20 0x7f989406bae0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 --2- 172.21.15.143:0/4056110114 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98940c6230 0x7f989406b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 >> 172.21.15.143:0/4056110114 conn(0x7f98940d0ed0 msgr2=0x7f989406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 shutdown_connections 2024-01-15T18:17:14.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.677+0000 7f9898c00640 1 -- 172.21.15.143:0/4056110114 wait complete. 2024-01-15T18:17:14.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:14.700 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:17:14.718 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-15T18:17:14.719 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:17:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 Processor -- start 2024-01-15T18:17:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 -- start start 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff04015b260 0x7ff0400cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff0400cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0400c4d90 con 0x7ff0400d19f0 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04015b640 con 0x7ff04015b260 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff044c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400cf800 con 0x7ff0400ccaf0 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff04015b260 0x7ff0400cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff04015b260 0x7ff0400cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56914/0 (socket says 172.21.15.143:56914) 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03d400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff0400d19f0 0x7ff0400c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42256/0 (socket says 172.21.15.143:42256) 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 -- 172.21.15.143:0/3902204993 learned_addr learned my addr 172.21.15.143:0/3902204993 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:14.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03e800640 1 --2- 172.21.15.143:0/3902204993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff0400cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 -- 172.21.15.143:0/3902204993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 msgr2=0x7ff0400cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 --2- 172.21.15.143:0/3902204993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff0400cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 -- 172.21.15.143:0/3902204993 >> v1:172.21.15.143:6789/0 conn(0x7ff0400d19f0 legacy=0x7ff0400c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03de00640 1 -- 172.21.15.143:0/3902204993 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0400cfae0 con 0x7ff04015b260 2024-01-15T18:17:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.905+0000 7ff03ca00640 1 -- 172.21.15.143:0/3902204993 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229548767 0 0) 0x7ff0400c4d90 con 0x7ff0400d19f0 2024-01-15T18:17:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff03de00640 1 --2- 172.21.15.143:0/3902204993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff0400cd810 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff030009870 tx=0x7ff03001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6099de26a95427a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff03ca00640 1 -- 172.21.15.143:0/3902204993 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300a7070 con 0x7ff04015b260 2024-01-15T18:17:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff03ca00640 1 -- 172.21.15.143:0/3902204993 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0300ab430 con 0x7ff04015b260 2024-01-15T18:17:14.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff03ca00640 1 -- 172.21.15.143:0/3902204993 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0300ab7c0 con 0x7ff04015b260 2024-01-15T18:17:14.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 -- 172.21.15.143:0/3902204993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 msgr2=0x7ff0400cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 --2- 172.21.15.143:0/3902204993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff0400cd810 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7ff030009870 tx=0x7ff03001c540 comp rx=0 tx=0).stop 2024-01-15T18:17:14.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 -- 172.21.15.143:0/3902204993 shutdown_connections 2024-01-15T18:17:14.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 --2- 172.21.15.143:0/3902204993 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff0400cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 --2- 172.21.15.143:0/3902204993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff0400cd810 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 -- 172.21.15.143:0/3902204993 >> 172.21.15.143:0/3902204993 conn(0x7ff0400d0ed0 msgr2=0x7ff0400d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:14.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 -- 172.21.15.143:0/3902204993 shutdown_connections 2024-01-15T18:17:14.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.909+0000 7ff044c00640 1 -- 172.21.15.143:0/3902204993 wait complete. 2024-01-15T18:17:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 Processor -- start 2024-01-15T18:17:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 -- start start 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff04009d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0400d19f0 0x7ff04009b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff040153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04009df00 con 0x7ff04015b260 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400b6530 con 0x7ff0400d19f0 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff044c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04009bc50 con 0x7ff0400ccaf0 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff040153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff040153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56926/0 (socket says 172.21.15.143:56926) 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 -- 172.21.15.143:0/1589173147 learned_addr learned my addr 172.21.15.143:0/1589173147 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03de00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff04009d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03d400640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0400d19f0 0x7ff04009b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 -- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 msgr2=0x7ff04009d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff04009d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 -- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0400d19f0 msgr2=0x7ff04009b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0400d19f0 0x7ff04009b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 -- 172.21.15.143:0/1589173147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0300094d0 con 0x7ff04015b260 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03de00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff04009d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.913+0000 7ff03e800640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff040153360 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800efc0 tx=0x7ff03800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038009ba0 con 0x7ff04015b260 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03800f040 con 0x7ff04015b260 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380146c0 con 0x7ff04015b260 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04009bed0 con 0x7ff04015b260 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04009c490 con 0x7ff04015b260 2024-01-15T18:17:14.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff040067500 con 0x7ff04015b260 2024-01-15T18:17:14.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.917+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 32) v1 ==== 87486+0+0 (secure 0 0 0) 0x7ff038014820 con 0x7ff04015b260 2024-01-15T18:17:14.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.921+0000 7ff036000640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff01806bba0 0x7ff01806e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:14.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.921+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7ff038084100 con 0x7ff04015b260 2024-01-15T18:17:14.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.921+0000 7ff03de00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff01806bba0 0x7ff01806e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:14.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.921+0000 7ff03de00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff01806bba0 0x7ff01806e060 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff0300af000 tx=0x7ff030008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:14.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:14.921+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff038052dd0 con 0x7ff04015b260 2024-01-15T18:17:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:15.097+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0400b6d50 con 0x7ff01806bba0 2024-01-15T18:17:15.129 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:17:15.129+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:17:15.219 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:17:15.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:15.837+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 33) v1 ==== 87588+0+0 (secure 0 0 0) 0x7ff0380531a0 con 0x7ff04015b260 2024-01-15T18:17:17.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:17.852+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7ff038056cf0 con 0x7ff04015b260 2024-01-15T18:17:18.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.156+0000 7ff036000640 1 -- 172.21.15.143:0/1589173147 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7ff0400b6d50 con 0x7ff01806bba0 2024-01-15T18:17:18.154 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:17:18.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff01806bba0 msgr2=0x7ff01806e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff01806bba0 0x7ff01806e060 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff0300af000 tx=0x7ff030008040 comp rx=0 tx=0).stop 2024-01-15T18:17:18.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 msgr2=0x7ff040153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff040153360 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff03800efc0 tx=0x7ff03800c490 comp rx=0 tx=0).stop 2024-01-15T18:17:18.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 shutdown_connections 2024-01-15T18:17:18.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff01806bba0 0x7ff01806e060 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff04015b260 0x7ff040153360 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff0400d19f0 0x7ff04009b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 --2- 172.21.15.143:0/1589173147 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff0400ccaf0 0x7ff04009d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.160+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 >> 172.21.15.143:0/1589173147 conn(0x7ff0400d0ed0 msgr2=0x7ff0400c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:18.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.164+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 shutdown_connections 2024-01-15T18:17:18.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.164+0000 7ff044c00640 1 -- 172.21.15.143:0/1589173147 wait complete. 2024-01-15T18:17:18.203 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:17:18.203 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:17:18.295 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:17:18.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:17:18.386 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:18.386 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:18.395 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:18.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.576+0000 7fe0a4800640 1 Processor -- start 2024-01-15T18:17:18.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.576+0000 7fe0a4800640 1 -- start start 2024-01-15T18:17:18.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0cdf20 0x7fe09c0cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a4800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe09c0c6230 0x7fe09c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a4800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09c0c4d90 con 0x7fe09c15b260 2024-01-15T18:17:18.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a4800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c15b640 con 0x7fe09c0c6230 2024-01-15T18:17:18.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0cf800 con 0x7fe09c0cdf20 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0cdf20 0x7fe09c0cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a2800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe09c0c6230 0x7fe09c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fe09c15b260 0x7fe09c0cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:42994/0 (socket says 172.21.15.143:42994) 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1e00640 1 -- 172.21.15.143:0/3830847848 learned_addr learned my addr 172.21.15.143:0/3830847848 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1400640 1 -- 172.21.15.143:0/3830847848 >> v2:172.21.15.204:3300/0 conn(0x7fe09c0c6230 msgr2=0x7fe09c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1400640 1 --2- 172.21.15.143:0/3830847848 >> v2:172.21.15.204:3300/0 conn(0x7fe09c0c6230 0x7fe09c0cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1400640 1 -- 172.21.15.143:0/3830847848 >> v1:172.21.15.204:6789/0 conn(0x7fe09c15b260 legacy=0x7fe09c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.580+0000 7fe0a1400640 1 -- 172.21.15.143:0/3830847848 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c0cfae0 con 0x7fe09c0cdf20 2024-01-15T18:17:18.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a1400640 1 --2- 172.21.15.143:0/3830847848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0cdf20 0x7fe09c0cbae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097c0 tx=0x7fe08401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74a3361b3c16da7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:18.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a0a00640 1 -- 172.21.15.143:0/3830847848 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840ab650 con 0x7fe09c0cdf20 2024-01-15T18:17:18.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a0a00640 1 -- 172.21.15.143:0/3830847848 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0840ab7b0 con 0x7fe09c0cdf20 2024-01-15T18:17:18.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a0a00640 1 -- 172.21.15.143:0/3830847848 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0840abb40 con 0x7fe09c0cdf20 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 -- 172.21.15.143:0/3830847848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0cdf20 msgr2=0x7fe09c0cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 --2- 172.21.15.143:0/3830847848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0cdf20 0x7fe09c0cbae0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe0840097c0 tx=0x7fe08401c450 comp rx=0 tx=0).stop 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 -- 172.21.15.143:0/3830847848 shutdown_connections 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 --2- 172.21.15.143:0/3830847848 >> v2:172.21.15.204:3300/0 conn(0x7fe09c0c6230 0x7fe09c0cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 --2- 172.21.15.143:0/3830847848 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0cdf20 0x7fe09c0cbae0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 -- 172.21.15.143:0/3830847848 >> 172.21.15.143:0/3830847848 conn(0x7fe09c0d0ed0 msgr2=0x7fe09c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 -- 172.21.15.143:0/3830847848 shutdown_connections 2024-01-15T18:17:18.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 -- 172.21.15.143:0/3830847848 wait complete. 2024-01-15T18:17:18.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 Processor -- start 2024-01-15T18:17:18.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.584+0000 7fe0a4800640 1 -- start start 2024-01-15T18:17:18.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0c6230 0x7fe09c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0cdf20 0x7fe09c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 0x7fe09c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c8e90 con 0x7fe09c15b260 2024-01-15T18:17:18.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0af1b0 con 0x7fe09c0cdf20 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c154ef0 con 0x7fe09c0c6230 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 0x7fe09c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a1e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0c6230 0x7fe09c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0cdf20 0x7fe09c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 0x7fe09c0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56142/0 (socket says 172.21.15.143:56142) 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 -- 172.21.15.143:0/69871012 learned_addr learned my addr 172.21.15.143:0/69871012 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 -- 172.21.15.143:0/69871012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0c6230 msgr2=0x7fe09c153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0c6230 0x7fe09c153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 -- 172.21.15.143:0/69871012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0cdf20 msgr2=0x7fe09c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0cdf20 0x7fe09c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 -- 172.21.15.143:0/69871012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084009420 con 0x7fe09c15b260 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a2800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 0x7fe09c0c84e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe094009cb0 tx=0x7fe09400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094009b20 con 0x7fe09c15b260 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09400f040 con 0x7fe09c15b260 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940146c0 con 0x7fe09c15b260 2024-01-15T18:17:18.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c1551d0 con 0x7fe09c15b260 2024-01-15T18:17:18.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c1538a0 con 0x7fe09c15b260 2024-01-15T18:17:18.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c0af380 con 0x7fe09c15b260 2024-01-15T18:17:18.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.588+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fe094014820 con 0x7fe09c15b260 2024-01-15T18:17:18.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.592+0000 7fe09a000640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe074067820 0x7fe074069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.592+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe094083b80 con 0x7fe09c15b260 2024-01-15T18:17:18.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.592+0000 7fe0a1e00640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe074067820 0x7fe074069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.592+0000 7fe0a1e00640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe074067820 0x7fe074069ce0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe08c005890 tx=0x7fe08c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:18.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.596+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe09c0af380 con 0x7fe09c15b260 2024-01-15T18:17:18.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.744+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe09c0af380 con 0x7fe09c15b260 2024-01-15T18:17:18.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe09a000640 1 -- 172.21.15.143:0/69871012 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13113 (secure 0 0 0) 0x7fe09c0af380 con 0x7fe09c15b260 2024-01-15T18:17:18.746 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:18.746 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":92,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:18.153310+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/1230123305":"2024-01-16T18:17:15.135584+0000","172.21.15.143:6836/1230123305":"2024-01-16T18:17:15.135584+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe074067820 msgr2=0x7fe074069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe074067820 0x7fe074069ce0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe08c005890 tx=0x7fe08c009040 comp rx=0 tx=0).stop 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 msgr2=0x7fe09c0c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 0x7fe09c0c84e0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe094009cb0 tx=0x7fe09400c490 comp rx=0 tx=0).stop 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 shutdown_connections 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe074067820 0x7fe074069ce0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe09c15b260 0x7fe09c0c84e0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe09c0cdf20 0x7fe09c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 --2- 172.21.15.143:0/69871012 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe09c0c6230 0x7fe09c153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 >> 172.21.15.143:0/69871012 conn(0x7fe09c0d0ed0 msgr2=0x7fe09c0c6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 shutdown_connections 2024-01-15T18:17:18.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.748+0000 7fe0a4800640 1 -- 172.21.15.143:0/69871012 wait complete. 2024-01-15T18:17:18.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:18.768 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:18.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.916+0000 7f1900200640 1 Processor -- start 2024-01-15T18:17:18.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f1900200640 1 -- start start 2024-01-15T18:17:18.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f1900200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f18f8071bb0 0x7f18f8072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f1900200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f18f815c5a0 0x7f18f815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f1900200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f80ceb50 con 0x7f18f8143780 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f1900200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f8072570 con 0x7f18f815c5a0 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f1900200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f815c110 con 0x7f18f8071bb0 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fe200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f18f815c5a0 0x7f18f815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fd800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f18f8143780 0x7f18f80cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47746/0 (socket says 172.21.15.143:47746) 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fd800640 1 -- 172.21.15.143:0/2819199815 learned_addr learned my addr 172.21.15.143:0/2819199815 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fe200640 1 --2- 172.21.15.143:0/2819199815 >> v2:172.21.15.143:3300/0 conn(0x7f18f815c5a0 0x7f18f815c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56158/0 (socket says 172.21.15.143:56158) 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fce00640 1 --2- 172.21.15.143:0/2819199815 >> v2:172.21.15.204:3300/0 conn(0x7f18f8071bb0 0x7f18f8072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fe200640 1 -- 172.21.15.143:0/2819199815 >> v2:172.21.15.204:3300/0 conn(0x7f18f8071bb0 msgr2=0x7f18f8072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fe200640 1 --2- 172.21.15.143:0/2819199815 >> v2:172.21.15.204:3300/0 conn(0x7f18f8071bb0 0x7f18f8072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fe200640 1 -- 172.21.15.143:0/2819199815 >> v1:172.21.15.143:6789/0 conn(0x7f18f8143780 legacy=0x7f18f80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.920+0000 7f18fe200640 1 -- 172.21.15.143:0/2819199815 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f815b7e0 con 0x7f18f815c5a0 2024-01-15T18:17:18.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f18fe200640 1 --2- 172.21.15.143:0/2819199815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f815c5a0 0x7f18f815c9e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f18f806beb0 tx=0x7f18f001eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdf39cf131f5788 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:18.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f18f7e00640 1 -- 172.21.15.143:0/2819199815 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f000b2d0 con 0x7f18f815c5a0 2024-01-15T18:17:18.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f18f7e00640 1 -- 172.21.15.143:0/2819199815 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f00a79e0 con 0x7f18f815c5a0 2024-01-15T18:17:18.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f18f7e00640 1 -- 172.21.15.143:0/2819199815 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00a7d70 con 0x7f18f815c5a0 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 -- 172.21.15.143:0/2819199815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f815c5a0 msgr2=0x7f18f815c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 --2- 172.21.15.143:0/2819199815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f815c5a0 0x7f18f815c9e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f18f806beb0 tx=0x7f18f001eb10 comp rx=0 tx=0).stop 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 -- 172.21.15.143:0/2819199815 shutdown_connections 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 --2- 172.21.15.143:0/2819199815 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f815c5a0 0x7f18f815c9e0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 --2- 172.21.15.143:0/2819199815 >> v2:172.21.15.204:3300/0 conn(0x7f18f8071bb0 0x7f18f8072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 -- 172.21.15.143:0/2819199815 >> 172.21.15.143:0/2819199815 conn(0x7f18f806c3f0 msgr2=0x7f18f806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 -- 172.21.15.143:0/2819199815 shutdown_connections 2024-01-15T18:17:18.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 -- 172.21.15.143:0/2819199815 wait complete. 2024-01-15T18:17:18.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 Processor -- start 2024-01-15T18:17:18.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.924+0000 7f1900200640 1 -- start start 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 0x7f18f8159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18f8143780 0x7f18f813b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18f815c5a0 0x7f18f80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8159ab0 con 0x7f18f8071bb0 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8130290 con 0x7f18f815c5a0 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f8130570 con 0x7f18f8143780 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 0x7f18f8159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 0x7f18f8159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56162/0 (socket says 172.21.15.143:56162) 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 -- 172.21.15.143:0/549789785 learned_addr learned my addr 172.21.15.143:0/549789785 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fe200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18f815c5a0 0x7f18f80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fce00640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18f8143780 0x7f18f813b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 -- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18f8143780 msgr2=0x7f18f813b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18f8143780 0x7f18f813b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 -- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18f815c5a0 msgr2=0x7f18f80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18f815c5a0 0x7f18f80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 -- 172.21.15.143:0/549789785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f000bc80 con 0x7f18f8071bb0 2024-01-15T18:17:18.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fce00640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18f8143780 0x7f18f813b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18fd800640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 0x7f18f8159570 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f18e800c500 tx=0x7f18e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e8005cf0 con 0x7f18f8071bb0 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e8005e50 con 0x7f18f8071bb0 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f8130850 con 0x7f18f8071bb0 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80072f0 con 0x7f18f8071bb0 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f8130e10 con 0x7f18f8071bb0 2024-01-15T18:17:18.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.928+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f8067500 con 0x7f18f8071bb0 2024-01-15T18:17:18.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.932+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f18e8007450 con 0x7f18f8071bb0 2024-01-15T18:17:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.932+0000 7f18f5600640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18d0067820 0x7f18d0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.932+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f18e8082250 con 0x7f18f8071bb0 2024-01-15T18:17:18.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.932+0000 7f18fce00640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18d0067820 0x7f18d0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:18.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.932+0000 7f18fce00640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18d0067820 0x7f18d0069ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f18f81f5160 tx=0x7f18e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:18.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:18.936+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18e8050ff0 con 0x7f18f8071bb0 2024-01-15T18:17:19.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.088+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18f80ceb50 con 0x7f18f8071bb0 2024-01-15T18:17:19.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.088+0000 7f18f5600640 1 -- 172.21.15.143:0/549789785 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18f80ceb50 con 0x7f18f8071bb0 2024-01-15T18:17:19.088 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:17:19.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18d0067820 msgr2=0x7f18d0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18d0067820 0x7f18d0069ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f18f81f5160 tx=0x7f18e0008040 comp rx=0 tx=0).stop 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 msgr2=0x7f18f8159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 0x7f18f8159570 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f18e800c500 tx=0x7f18e800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 shutdown_connections 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18d0067820 0x7f18d0069ce0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f18f815c5a0 0x7f18f80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18f8143780 0x7f18f813b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 --2- 172.21.15.143:0/549789785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18f8071bb0 0x7f18f8159570 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 >> 172.21.15.143:0/549789785 conn(0x7f18f806c3f0 msgr2=0x7f18f80af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 shutdown_connections 2024-01-15T18:17:19.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.092+0000 7f1900200640 1 -- 172.21.15.143:0/549789785 wait complete. 2024-01-15T18:17:19.110 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:17:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 Processor -- start 2024-01-15T18:17:19.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 -- start start 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5414b110 0x7fdf540cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdf540cc8f0 0x7fdf540b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf540b30a0 con 0x7fdf54071f70 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf54070340 con 0x7fdf540cc8f0 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf59c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54070620 con 0x7fdf5414b110 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf52000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fdf540cc8f0 0x7fdf540b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf53400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdf54071f70 0x7fdf54071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47748/0 (socket says 172.21.15.143:47748) 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.248+0000 7fdf53400640 1 -- 172.21.15.143:0/2041391416 learned_addr learned my addr 172.21.15.143:0/2041391416 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf52a00640 1 --2- 172.21.15.143:0/2041391416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5414b110 0x7fdf540cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf52000640 1 -- 172.21.15.143:0/2041391416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5414b110 msgr2=0x7fdf540cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:19.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf52000640 1 --2- 172.21.15.143:0/2041391416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5414b110 0x7fdf540cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf52000640 1 -- 172.21.15.143:0/2041391416 >> v1:172.21.15.143:6789/0 conn(0x7fdf54071f70 legacy=0x7fdf54071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf52000640 1 -- 172.21.15.143:0/2041391416 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54070900 con 0x7fdf540cc8f0 2024-01-15T18:17:19.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf51600640 1 -- 172.21.15.143:0/2041391416 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052388200 0 0) 0x7fdf540b30a0 con 0x7fdf54071f70 2024-01-15T18:17:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf52000640 1 --2- 172.21.15.143:0/2041391416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf540b3a40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0097c0 tx=0x7fdf3c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55427275a5c89484 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf51600640 1 -- 172.21.15.143:0/2041391416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0ac640 con 0x7fdf540cc8f0 2024-01-15T18:17:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf51600640 1 -- 172.21.15.143:0/2041391416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540b2e90 con 0x7fdf540cc8f0 2024-01-15T18:17:19.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.252+0000 7fdf51600640 1 -- 172.21.15.143:0/2041391416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0ac9c0 con 0x7fdf540cc8f0 2024-01-15T18:17:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- 172.21.15.143:0/2041391416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 msgr2=0x7fdf540b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2041391416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf540b3a40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0097c0 tx=0x7fdf3c01c3c0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- 172.21.15.143:0/2041391416 shutdown_connections 2024-01-15T18:17:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2041391416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf540b3a40 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2041391416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf5414b110 0x7fdf540cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- 172.21.15.143:0/2041391416 >> 172.21.15.143:0/2041391416 conn(0x7fdf5406c3f0 msgr2=0x7fdf5406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- 172.21.15.143:0/2041391416 shutdown_connections 2024-01-15T18:17:19.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- 172.21.15.143:0/2041391416 wait complete. 2024-01-15T18:17:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 Processor -- start 2024-01-15T18:17:19.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- start start 2024-01-15T18:17:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf54071f70 0x7fdf541440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf541643c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5414b110 0x7fdf540aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf541610f0 con 0x7fdf540cc8f0 2024-01-15T18:17:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54160470 con 0x7fdf5414b110 2024-01-15T18:17:19.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf59c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54160750 con 0x7fdf54071f70 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf541643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf541643c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56182/0 (socket says 172.21.15.143:56182) 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 -- 172.21.15.143:0/2842447699 learned_addr learned my addr 172.21.15.143:0/2842447699 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52a00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf54071f70 0x7fdf541440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf53400640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5414b110 0x7fdf540aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 -- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf54071f70 msgr2=0x7fdf541440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf54071f70 0x7fdf541440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 -- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5414b110 msgr2=0x7fdf540aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5414b110 0x7fdf540aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52000640 1 -- 172.21.15.143:0/2842447699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c009420 con 0x7fdf540cc8f0 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf52a00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf54071f70 0x7fdf541440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.256+0000 7fdf53400640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5414b110 0x7fdf540aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf52000640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf541643c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0af040 tx=0x7fdf3c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:19.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0b5610 con 0x7fdf540cc8f0 2024-01-15T18:17:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c0b5790 con 0x7fdf540cc8f0 2024-01-15T18:17:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf54160a30 con 0x7fdf540cc8f0 2024-01-15T18:17:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0b5b40 con 0x7fdf540cc8f0 2024-01-15T18:17:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf54144600 con 0x7fdf540cc8f0 2024-01-15T18:17:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf54161230 con 0x7fdf540cc8f0 2024-01-15T18:17:19.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fdf3c0b3040 con 0x7fdf540cc8f0 2024-01-15T18:17:19.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf4aa00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdf2c067820 0x7fdf2c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fdf3c11d570 con 0x7fdf540cc8f0 2024-01-15T18:17:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.260+0000 7fdf52a00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdf2c067820 0x7fdf2c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.261 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.264+0000 7fdf52a00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdf2c067820 0x7fdf2c069ce0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b9f0 tx=0x7fdf44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:19.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.264+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdf54161230 con 0x7fdf540cc8f0 2024-01-15T18:17:19.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.540+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdf54161230 con 0x7fdf540cc8f0 2024-01-15T18:17:19.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.544+0000 7fdf4aa00640 1 -- 172.21.15.143:0/2842447699 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v92) v1 ==== 80+0+124 (secure 0 0 0) 0x7fdf54161230 con 0x7fdf540cc8f0 2024-01-15T18:17:19.544 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/1230123305 2024-01-16T18:17:15.135584+0000 2024-01-15T18:17:19.544 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/1230123305 2024-01-16T18:17:15.135584+0000 2024-01-15T18:17:19.544 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:17:19.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdf2c067820 msgr2=0x7fdf2c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdf2c067820 0x7fdf2c069ce0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400b9f0 tx=0x7fdf44008040 comp rx=0 tx=0).stop 2024-01-15T18:17:19.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 msgr2=0x7fdf541643c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.547 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf541643c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0af040 tx=0x7fdf3c019160 comp rx=0 tx=0).stop 2024-01-15T18:17:19.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 shutdown_connections 2024-01-15T18:17:19.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdf2c067820 0x7fdf2c069ce0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdf5414b110 0x7fdf540aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf540cc8f0 0x7fdf541643c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.548 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 --2- 172.21.15.143:0/2842447699 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdf54071f70 0x7fdf541440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.549 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.548+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 >> 172.21.15.143:0/2842447699 conn(0x7fdf5406c3f0 msgr2=0x7fdf540734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:19.550 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.552+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 shutdown_connections 2024-01-15T18:17:19.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.552+0000 7fdf59c00640 1 -- 172.21.15.143:0/2842447699 wait complete. 2024-01-15T18:17:19.597 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:17:19.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:19.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:19.634 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[49175]: fuse finished with error 0 and tester_r 0 2024-01-15T18:17:19.635 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:17:19.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:19.636 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:17:19.830 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:17:19.831 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:19.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:19.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.976+0000 7fa05b800640 1 Processor -- start 2024-01-15T18:17:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.976+0000 7fa05b800640 1 -- start start 2024-01-15T18:17:19.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa05b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa05b800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa05415c5a0 0x7fa05415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa05b800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540ceb50 con 0x7fa054143780 2024-01-15T18:17:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa05b800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa054072570 con 0x7fa05415c5a0 2024-01-15T18:17:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa05b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05415c110 con 0x7fa054071bb0 2024-01-15T18:17:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa058e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa054143780 0x7fa0540cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43010/0 (socket says 172.21.15.143:43010) 2024-01-15T18:17:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa059800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa05415c5a0 0x7fa05415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa058e00640 1 -- 172.21.15.143:0/329646970 learned_addr learned my addr 172.21.15.143:0/329646970 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053400640 1 -- 172.21.15.143:0/329646970 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1699162428 0 0) 0x7fa044001120 con 0x7fa054143780 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053400640 1 -- 172.21.15.143:0/329646970 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa034003530 con 0x7fa054143780 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053e00640 1 -- 172.21.15.143:0/329646970 >> v2:172.21.15.204:3300/0 conn(0x7fa05415c5a0 msgr2=0x7fa05415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053e00640 1 --2- 172.21.15.143:0/329646970 >> v2:172.21.15.204:3300/0 conn(0x7fa05415c5a0 0x7fa05415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053e00640 1 -- 172.21.15.143:0/329646970 >> v1:172.21.15.204:6789/0 conn(0x7fa054143780 legacy=0x7fa0540cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa053e00640 1 -- 172.21.15.143:0/329646970 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05415b7e0 con 0x7fa054071bb0 2024-01-15T18:17:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.980+0000 7fa059800640 1 --2- 172.21.15.143:0/329646970 >> v2:172.21.15.204:3300/0 conn(0x7fa05415c5a0 0x7fa05415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa053e00640 1 --2- 172.21.15.143:0/329646970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054072030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0097c0 tx=0x7fa03c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fdc036bcb59a38d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa053400640 1 -- 172.21.15.143:0/329646970 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0ab680 con 0x7fa054071bb0 2024-01-15T18:17:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa053400640 1 -- 172.21.15.143:0/329646970 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c0ab7e0 con 0x7fa054071bb0 2024-01-15T18:17:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa053400640 1 -- 172.21.15.143:0/329646970 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0abb70 con 0x7fa054071bb0 2024-01-15T18:17:19.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 -- 172.21.15.143:0/329646970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 msgr2=0x7fa054072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 --2- 172.21.15.143:0/329646970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054072030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0097c0 tx=0x7fa03c01c480 comp rx=0 tx=0).stop 2024-01-15T18:17:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 -- 172.21.15.143:0/329646970 shutdown_connections 2024-01-15T18:17:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 --2- 172.21.15.143:0/329646970 >> v2:172.21.15.204:3300/0 conn(0x7fa05415c5a0 0x7fa05415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 --2- 172.21.15.143:0/329646970 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054072030 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 -- 172.21.15.143:0/329646970 >> 172.21.15.143:0/329646970 conn(0x7fa05406c3f0 msgr2=0x7fa05406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 -- 172.21.15.143:0/329646970 shutdown_connections 2024-01-15T18:17:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.984+0000 7fa05b800640 1 -- 172.21.15.143:0/329646970 wait complete. 2024-01-15T18:17:19.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 Processor -- start 2024-01-15T18:17:19.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- start start 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 0x7fa054166680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05415c5a0 0x7fa054157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054155580 con 0x7fa054143780 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054139120 con 0x7fa05415c5a0 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa054139400 con 0x7fa054071bb0 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 0x7fa054166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa058e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa059800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05415c5a0 0x7fa054157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 0x7fa054166680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56196/0 (socket says 172.21.15.143:56196) 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 -- 172.21.15.143:0/331995098 learned_addr learned my addr 172.21.15.143:0/331995098 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 -- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 msgr2=0x7fa054156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 -- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05415c5a0 msgr2=0x7fa054157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05415c5a0 0x7fa054157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 -- 172.21.15.143:0/331995098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c009a70 con 0x7fa054143780 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa058e00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa059800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05415c5a0 0x7fa054157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa053e00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 0x7fa054166680 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fa03c01a0b0 tx=0x7fa03c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0bf610 con 0x7fa054143780 2024-01-15T18:17:19.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c0bf790 con 0x7fa054143780 2024-01-15T18:17:19.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c0bfb40 con 0x7fa054143780 2024-01-15T18:17:19.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c009420 con 0x7fa054143780 2024-01-15T18:17:19.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0541399c0 con 0x7fa054143780 2024-01-15T18:17:19.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.988+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa054067500 con 0x7fa054143780 2024-01-15T18:17:19.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.992+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fa03c0b5400 con 0x7fa054143780 2024-01-15T18:17:19.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.992+0000 7fa050c00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c067850 0x7fa02c069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:19.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.992+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fa03c11d960 con 0x7fa054143780 2024-01-15T18:17:19.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.992+0000 7fa058e00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c067850 0x7fa02c069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:19.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.992+0000 7fa058e00640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c067850 0x7fa02c069d10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa044001280 tx=0x7fa044007bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:19.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:19.996+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa03c0ec6a0 con 0x7fa054143780 2024-01-15T18:17:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.144+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0540008e0 con 0x7fa054143780 2024-01-15T18:17:20.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa050c00640 1 -- 172.21.15.143:0/331995098 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13113 (secure 0 0 0) 0x7fa03c0efcc0 con 0x7fa054143780 2024-01-15T18:17:20.145 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:20.146 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":92,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:18.153310+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/1230123305":"2024-01-16T18:17:15.135584+0000","172.21.15.143:6836/1230123305":"2024-01-16T18:17:15.135584+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c067850 msgr2=0x7fa02c069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c067850 0x7fa02c069d10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa044001280 tx=0x7fa044007bc0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 msgr2=0x7fa054166680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 0x7fa054166680 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fa03c01a0b0 tx=0x7fa03c01acc0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 shutdown_connections 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa02c067850 0x7fa02c069d10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa05415c5a0 0x7fa054157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa054143780 0x7fa054166680 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 --2- 172.21.15.143:0/331995098 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa054071bb0 0x7fa054156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 >> 172.21.15.143:0/331995098 conn(0x7fa05406c3f0 msgr2=0x7fa05413afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 shutdown_connections 2024-01-15T18:17:20.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.148+0000 7fa05b800640 1 -- 172.21.15.143:0/331995098 wait complete. 2024-01-15T18:17:20.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:20.168 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:20.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 Processor -- start 2024-01-15T18:17:20.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 -- start start 2024-01-15T18:17:20.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8541607c0 0x7fe85406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8540cbf50 0x7fe8540cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8540cffd0 con 0x7fe8540cfb30 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8540d1ce0 con 0x7fe8540cbf50 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540d02b0 con 0x7fe8541607c0 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8540cbf50 0x7fe8540cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85a400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8540cbf50 0x7fe8540cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56206/0 (socket says 172.21.15.143:56206) 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe85a400640 1 -- 172.21.15.143:0/1018218052 learned_addr learned my addr 172.21.15.143:0/1018218052 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.308+0000 7fe859a00640 1 --2- 172.21.15.143:0/1018218052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8541607c0 0x7fe85406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe85a400640 1 -- 172.21.15.143:0/1018218052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8541607c0 msgr2=0x7fe85406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe85a400640 1 --2- 172.21.15.143:0/1018218052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8541607c0 0x7fe85406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe85a400640 1 -- 172.21.15.143:0/1018218052 >> v1:172.21.15.143:6789/0 conn(0x7fe8540cfb30 legacy=0x7fe85406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe85a400640 1 -- 172.21.15.143:0/1018218052 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540cb470 con 0x7fe8540cbf50 2024-01-15T18:17:20.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe85a400640 1 --2- 172.21.15.143:0/1018218052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8540cbf50 0x7fe8540cc390 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009e50 tx=0x7fe84c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea3cac0cc1a11ee server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe853e00640 1 -- 172.21.15.143:0/1018218052 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0af400 con 0x7fe8540cbf50 2024-01-15T18:17:20.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe853e00640 1 -- 172.21.15.143:0/1018218052 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c0ab050 con 0x7fe8540cbf50 2024-01-15T18:17:20.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.312+0000 7fe853e00640 1 -- 172.21.15.143:0/1018218052 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0af780 con 0x7fe8540cbf50 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- 172.21.15.143:0/1018218052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8540cbf50 msgr2=0x7fe8540cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 --2- 172.21.15.143:0/1018218052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8540cbf50 0x7fe8540cc390 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009e50 tx=0x7fe84c01f1f0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- 172.21.15.143:0/1018218052 shutdown_connections 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 --2- 172.21.15.143:0/1018218052 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8540cbf50 0x7fe8540cc390 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 --2- 172.21.15.143:0/1018218052 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8541607c0 0x7fe85406b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- 172.21.15.143:0/1018218052 >> 172.21.15.143:0/1018218052 conn(0x7fe8540d0e20 msgr2=0x7fe8540d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:20.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- 172.21.15.143:0/1018218052 shutdown_connections 2024-01-15T18:17:20.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- 172.21.15.143:0/1018218052 wait complete. 2024-01-15T18:17:20.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 Processor -- start 2024-01-15T18:17:20.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- start start 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8540cbf50 0x7fe8541583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8540cfb30 0x7fe8540c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 0x7fe85414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540c7840 con 0x7fe8541607c0 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85414e810 con 0x7fe8540cbf50 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe854158a70 con 0x7fe8540cfb30 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 0x7fe85414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85a400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 0x7fe85414e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56218/0 (socket says 172.21.15.143:56218) 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe85a400640 1 -- 172.21.15.143:0/957372257 learned_addr learned my addr 172.21.15.143:0/957372257 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe859a00640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8540cbf50 0x7fe8541583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.316+0000 7fe859000640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8540cfb30 0x7fe8540c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85a400640 1 -- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8540cfb30 msgr2=0x7fe8540c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85a400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8540cfb30 0x7fe8540c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85a400640 1 -- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8540cbf50 msgr2=0x7fe8541583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85a400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8540cbf50 0x7fe8541583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85a400640 1 -- 172.21.15.143:0/957372257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c009b30 con 0x7fe8541607c0 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85a400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 0x7fe85414e2d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe84c01ce20 tx=0x7fe84c01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0afd30 con 0x7fe8541607c0 2024-01-15T18:17:20.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c0afe90 con 0x7fe8541607c0 2024-01-15T18:17:20.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe854158d50 con 0x7fe8541607c0 2024-01-15T18:17:20.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0ba660 con 0x7fe8541607c0 2024-01-15T18:17:20.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8540c4c00 con 0x7fe8541607c0 2024-01-15T18:17:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85406d4b0 con 0x7fe8541607c0 2024-01-15T18:17:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fe84c0b6040 con 0x7fe8541607c0 2024-01-15T18:17:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe851600640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe82c067850 0x7fe82c069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.320+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe84c121210 con 0x7fe8541607c0 2024-01-15T18:17:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.324+0000 7fe859a00640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe82c067850 0x7fe82c069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.324+0000 7fe859a00640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe82c067850 0x7fe82c069d10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe85415b120 tx=0x7fe844009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.324+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe84c0efe40 con 0x7fe8541607c0 2024-01-15T18:17:20.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.476+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe854067500 con 0x7fe8541607c0 2024-01-15T18:17:20.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.476+0000 7fe851600640 1 -- 172.21.15.143:0/957372257 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe84c009510 con 0x7fe8541607c0 2024-01-15T18:17:20.475 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.476+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe82c067850 msgr2=0x7fe82c069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.476+0000 7fe85c400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe82c067850 0x7fe82c069d10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe85415b120 tx=0x7fe844009190 comp rx=0 tx=0).stop 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.476+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 msgr2=0x7fe85414e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.476+0000 7fe85c400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 0x7fe85414e2d0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe84c01ce20 tx=0x7fe84c01bf90 comp rx=0 tx=0).stop 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 shutdown_connections 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe82c067850 0x7fe82c069d10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8541607c0 0x7fe85414e2d0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8540cfb30 0x7fe8540c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 --2- 172.21.15.143:0/957372257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8540cbf50 0x7fe8541583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 >> 172.21.15.143:0/957372257 conn(0x7fe8540d0e20 msgr2=0x7fe8540d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:20.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 shutdown_connections 2024-01-15T18:17:20.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.480+0000 7fe85c400640 1 -- 172.21.15.143:0/957372257 wait complete. 2024-01-15T18:17:20.497 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:17:20.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 Processor -- start 2024-01-15T18:17:20.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 -- start start 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2fa4071bb0 0x7f2fa4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fa415c5a0 0x7f2fa415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa40ceb50 con 0x7f2fa4143780 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa4072570 con 0x7f2fa415c5a0 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faca00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa415c110 con 0x7f2fa4071bb0 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fa415c5a0 0x7f2fa415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faaa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2fa415c5a0 0x7f2fa415c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56230/0 (socket says 172.21.15.143:56230) 2024-01-15T18:17:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2faaa00640 1 -- 172.21.15.143:0/4200996980 learned_addr learned my addr 172.21.15.143:0/4200996980 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.636+0000 7f2fa9600640 1 --2- 172.21.15.143:0/4200996980 >> v2:172.21.15.204:3300/0 conn(0x7f2fa4071bb0 0x7f2fa4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faaa00640 1 -- 172.21.15.143:0/4200996980 >> v2:172.21.15.204:3300/0 conn(0x7f2fa4071bb0 msgr2=0x7f2fa4072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faaa00640 1 --2- 172.21.15.143:0/4200996980 >> v2:172.21.15.204:3300/0 conn(0x7f2fa4071bb0 0x7f2fa4072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faaa00640 1 -- 172.21.15.143:0/4200996980 >> v1:172.21.15.204:6789/0 conn(0x7f2fa4143780 legacy=0x7f2fa40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faaa00640 1 -- 172.21.15.143:0/4200996980 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa415b7e0 con 0x7f2fa415c5a0 2024-01-15T18:17:20.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faaa00640 1 --2- 172.21.15.143:0/4200996980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa415c5a0 0x7f2fa415c9e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2fa406beb0 tx=0x7f2f9c01ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6db48baf1dab96 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2fa8c00640 1 -- 172.21.15.143:0/4200996980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00b2d0 con 0x7f2fa415c5a0 2024-01-15T18:17:20.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2fa8c00640 1 -- 172.21.15.143:0/4200996980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c00b430 con 0x7f2fa415c5a0 2024-01-15T18:17:20.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2fa8c00640 1 -- 172.21.15.143:0/4200996980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0a7cc0 con 0x7f2fa415c5a0 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faca00640 1 -- 172.21.15.143:0/4200996980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa415c5a0 msgr2=0x7f2fa415c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.640+0000 7f2faca00640 1 --2- 172.21.15.143:0/4200996980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa415c5a0 0x7f2fa415c9e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f2fa406beb0 tx=0x7f2f9c01ebd0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- 172.21.15.143:0/4200996980 shutdown_connections 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 --2- 172.21.15.143:0/4200996980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa415c5a0 0x7f2fa415c9e0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 --2- 172.21.15.143:0/4200996980 >> v2:172.21.15.204:3300/0 conn(0x7f2fa4071bb0 0x7f2fa4072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- 172.21.15.143:0/4200996980 >> 172.21.15.143:0/4200996980 conn(0x7f2fa406c3f0 msgr2=0x7f2fa406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- 172.21.15.143:0/4200996980 shutdown_connections 2024-01-15T18:17:20.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- 172.21.15.143:0/4200996980 wait complete. 2024-01-15T18:17:20.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 Processor -- start 2024-01-15T18:17:20.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- start start 2024-01-15T18:17:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 0x7f2fa41385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fa4143780 0x7f2fa41344b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fa415c5a0 0x7f2fa41f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41fa360 con 0x7f2fa4071bb0 2024-01-15T18:17:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4138b90 con 0x7f2fa415c5a0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4138e70 con 0x7f2fa4143780 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 0x7f2fa41385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 0x7f2fa41385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56244/0 (socket says 172.21.15.143:56244) 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 -- 172.21.15.143:0/1268819819 learned_addr learned my addr 172.21.15.143:0/1268819819 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faaa00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fa415c5a0 0x7f2fa41f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2fa9600640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fa4143780 0x7f2fa41344b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 -- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fa4143780 msgr2=0x7f2fa41344b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fa4143780 0x7f2fa41344b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 -- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fa415c5a0 msgr2=0x7f2fa41f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fa415c5a0 0x7f2fa41f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.644+0000 7f2faa000640 1 -- 172.21.15.143:0/1268819819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c00bd40 con 0x7f2fa4071bb0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2faa000640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 0x7f2fa41385b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c500 tx=0x7f2f9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94005220 con 0x7f2fa4071bb0 2024-01-15T18:17:20.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f94005380 con 0x7f2fa4071bb0 2024-01-15T18:17:20.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007270 con 0x7f2fa4071bb0 2024-01-15T18:17:20.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa4139150 con 0x7f2fa4071bb0 2024-01-15T18:17:20.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa4139710 con 0x7f2fa4071bb0 2024-01-15T18:17:20.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa4139930 con 0x7f2fa4071bb0 2024-01-15T18:17:20.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f2f940073d0 con 0x7f2fa4071bb0 2024-01-15T18:17:20.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa2000640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f7c067820 0x7f2f7c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f2f94082940 con 0x7f2fa4071bb0 2024-01-15T18:17:20.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.648+0000 7f2fa9600640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f7c067820 0x7f2f7c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.652+0000 7f2fa9600640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f7c067820 0x7f2f7c069ce0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2fa413cc50 tx=0x7f2f8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.652+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f94051310 con 0x7f2fa4071bb0 2024-01-15T18:17:20.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2fa40ceb50 con 0x7f2fa4071bb0 2024-01-15T18:17:20.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2fa2000640 1 -- 172.21.15.143:0/1268819819 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v92) v1 ==== 80+0+124 (secure 0 0 0) 0x7f2fa40ceb50 con 0x7f2fa4071bb0 2024-01-15T18:17:20.815 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6837/1230123305 2024-01-16T18:17:15.135584+0000 2024-01-15T18:17:20.815 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6836/1230123305 2024-01-16T18:17:15.135584+0000 2024-01-15T18:17:20.815 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f7c067820 msgr2=0x7f2f7c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2faca00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f7c067820 0x7f2f7c069ce0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2fa413cc50 tx=0x7f2f8c008040 comp rx=0 tx=0).stop 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 msgr2=0x7f2fa41385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2faca00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 0x7f2fa41385b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c500 tx=0x7f2f9400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.816+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 shutdown_connections 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f7c067820 0x7f2f7c069ce0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2fa415c5a0 0x7f2fa41f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2fa4143780 0x7f2fa41344b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 --2- 172.21.15.143:0/1268819819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa4071bb0 0x7f2fa41385b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 >> 172.21.15.143:0/1268819819 conn(0x7f2fa406c3f0 msgr2=0x7f2fa40aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:20.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 shutdown_connections 2024-01-15T18:17:20.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:20.820+0000 7f2faca00640 1 -- 172.21.15.143:0/1268819819 wait complete. 2024-01-15T18:17:20.838 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:17:20.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:20.838 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:20.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34160]: fuse finished with error 0 and tester_r 0 2024-01-15T18:17:20.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:17:20.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:20.862 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:17:21.034 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:21.035 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:21.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:17:21.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:17:21.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:21.044 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:17:21.096 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:17:21.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:17:21.097 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:17:21.098 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:21.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:17:21.107 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:17:21.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:21.108 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:17:21.158 INFO:teuthology.orchestra.run.smithi204.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:17:21.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:17:21.159 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:17:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 Processor -- start 2024-01-15T18:17:21.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 -- start start 2024-01-15T18:17:21.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba9006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fba900cbf50 0x7fba900cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba900cffd0 con 0x7fba901607c0 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba900d1ce0 con 0x7fba900cbf50 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.296+0000 7fba97a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba900d02b0 con 0x7fba900cfb30 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fba900cbf50 0x7fba900cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fba900cbf50 0x7fba900cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56248/0 (socket says 172.21.15.143:56248) 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 -- 172.21.15.143:0/3260823506 learned_addr learned my addr 172.21.15.143:0/3260823506 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba8fe00640 1 --2- 172.21.15.143:0/3260823506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba9006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 -- 172.21.15.143:0/3260823506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 msgr2=0x7fba9006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 --2- 172.21.15.143:0/3260823506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba9006bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 -- 172.21.15.143:0/3260823506 >> v1:172.21.15.204:6789/0 conn(0x7fba901607c0 legacy=0x7fba9006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 -- 172.21.15.143:0/3260823506 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900cb470 con 0x7fba900cbf50 2024-01-15T18:17:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba95a00640 1 --2- 172.21.15.143:0/3260823506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba900cbf50 0x7fba900cc390 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fba88009e50 tx=0x7fba8801f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf6e51ccc9f3171 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba8f400640 1 -- 172.21.15.143:0/3260823506 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880af400 con 0x7fba900cbf50 2024-01-15T18:17:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba8f400640 1 -- 172.21.15.143:0/3260823506 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba880ab050 con 0x7fba900cbf50 2024-01-15T18:17:21.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.300+0000 7fba8f400640 1 -- 172.21.15.143:0/3260823506 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880af780 con 0x7fba900cbf50 2024-01-15T18:17:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- 172.21.15.143:0/3260823506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba900cbf50 msgr2=0x7fba900cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 --2- 172.21.15.143:0/3260823506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba900cbf50 0x7fba900cc390 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fba88009e50 tx=0x7fba8801f1f0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- 172.21.15.143:0/3260823506 shutdown_connections 2024-01-15T18:17:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 --2- 172.21.15.143:0/3260823506 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba900cbf50 0x7fba900cc390 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 --2- 172.21.15.143:0/3260823506 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba9006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- 172.21.15.143:0/3260823506 >> 172.21.15.143:0/3260823506 conn(0x7fba900d0e20 msgr2=0x7fba900d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- 172.21.15.143:0/3260823506 shutdown_connections 2024-01-15T18:17:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- 172.21.15.143:0/3260823506 wait complete. 2024-01-15T18:17:21.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 Processor -- start 2024-01-15T18:17:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- start start 2024-01-15T18:17:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba900cbf50 0x7fba901f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba901541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 0x7fba90157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90159090 con 0x7fba901607c0 2024-01-15T18:17:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90158880 con 0x7fba900cbf50 2024-01-15T18:17:21.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.304+0000 7fba97a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba901f30e0 con 0x7fba900cfb30 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 0x7fba90157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 0x7fba90157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56262/0 (socket says 172.21.15.143:56262) 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 -- 172.21.15.143:0/1542207021 learned_addr learned my addr 172.21.15.143:0/1542207021 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95000640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba900cbf50 0x7fba901f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8fe00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba901541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 -- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 msgr2=0x7fba901541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba901541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 -- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba900cbf50 msgr2=0x7fba901f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba900cbf50 0x7fba901f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 -- 172.21.15.143:0/1542207021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88009b30 con 0x7fba901607c0 2024-01-15T18:17:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8fe00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba901541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba95a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 0x7fba90157e70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fba8801ce20 tx=0x7fba8801bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880afde0 con 0x7fba901607c0 2024-01-15T18:17:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88009510 con 0x7fba901607c0 2024-01-15T18:17:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba901f2af0 con 0x7fba901607c0 2024-01-15T18:17:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880ba680 con 0x7fba901607c0 2024-01-15T18:17:21.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba901546f0 con 0x7fba901607c0 2024-01-15T18:17:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9006d4b0 con 0x7fba901607c0 2024-01-15T18:17:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fba880b6040 con 0x7fba901607c0 2024-01-15T18:17:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.308+0000 7fba8cc00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fba68067820 0x7fba68069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.312+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fba88121440 con 0x7fba901607c0 2024-01-15T18:17:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.312+0000 7fba95000640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fba68067820 0x7fba68069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.312+0000 7fba95000640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fba68067820 0x7fba68069ce0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fba80000f70 tx=0x7fba80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.312+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fba880efea0 con 0x7fba901607c0 2024-01-15T18:17:21.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba901f3270 con 0x7fba901607c0 2024-01-15T18:17:21.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba8cc00640 1 -- 172.21.15.143:0/1542207021 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fba880f34c0 con 0x7fba901607c0 2024-01-15T18:17:21.494 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:21.494 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1584800432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1584800432},{"type":"v1","addr":"172.21.15.204:6834","nonce":1584800432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7843,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3526494826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3526494826},{"type":"v1","addr":"172.21.15.204:6835","nonce":3526494826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1307769037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1307769037},{"type":"v1","addr":"172.21.15.143:6835","nonce":1307769037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/606420289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":606420289},{"type":"v1","addr":"172.21.15.143:6837","nonce":606420289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2024-01-15T18:17:21.494 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fba68067820 msgr2=0x7fba68069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fba68067820 0x7fba68069ce0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fba80000f70 tx=0x7fba80009040 comp rx=0 tx=0).stop 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 msgr2=0x7fba90157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 0x7fba90157e70 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fba8801ce20 tx=0x7fba8801bf90 comp rx=0 tx=0).stop 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 shutdown_connections 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fba68067820 0x7fba68069ce0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba901607c0 0x7fba90157e70 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fba900cfb30 0x7fba901541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 --2- 172.21.15.143:0/1542207021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba900cbf50 0x7fba901f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 >> 172.21.15.143:0/1542207021 conn(0x7fba900d0e20 msgr2=0x7fba9015d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:21.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 shutdown_connections 2024-01-15T18:17:21.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.496+0000 7fba97a00640 1 -- 172.21.15.143:0/1542207021 wait complete. 2024-01-15T18:17:21.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:17:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 Processor -- start 2024-01-15T18:17:21.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 -- start start 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4490157300 0x7f4490159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f44900b4ac0 0x7f4490070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900b2a20 con 0x7f44900b21d0 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900b35a0 con 0x7f4490157300 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4498000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44900b2d00 con 0x7f44900b4ac0 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4490157300 0x7f4490159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4490157300 0x7f4490159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56268/0 (socket says 172.21.15.143:56268) 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 -- 172.21.15.143:0/3115046151 learned_addr learned my addr 172.21.15.143:0/3115046151 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 -- 172.21.15.143:0/3115046151 >> v2:172.21.15.204:3300/0 conn(0x7f44900b4ac0 msgr2=0x7f4490070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:21.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 --2- 172.21.15.143:0/3115046151 >> v2:172.21.15.204:3300/0 conn(0x7f44900b4ac0 0x7f4490070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 -- 172.21.15.143:0/3115046151 >> v1:172.21.15.143:6789/0 conn(0x7f44900b21d0 legacy=0x7f44900b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.656+0000 7f4495600640 1 -- 172.21.15.143:0/3115046151 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900b2fe0 con 0x7f4490157300 2024-01-15T18:17:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4495600640 1 --2- 172.21.15.143:0/3115046151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f4490159b70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f44900b1f70 tx=0x7f448001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce9d19761da7ef9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f448fe00640 1 -- 172.21.15.143:0/3115046151 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800aa650 con 0x7f4490157300 2024-01-15T18:17:21.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f448fe00640 1 -- 172.21.15.143:0/3115046151 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44800aa7b0 con 0x7f4490157300 2024-01-15T18:17:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f448fe00640 1 -- 172.21.15.143:0/3115046151 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800aab40 con 0x7f4490157300 2024-01-15T18:17:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 -- 172.21.15.143:0/3115046151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 msgr2=0x7f4490159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 --2- 172.21.15.143:0/3115046151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f4490159b70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f44900b1f70 tx=0x7f448001c4d0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 -- 172.21.15.143:0/3115046151 shutdown_connections 2024-01-15T18:17:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 --2- 172.21.15.143:0/3115046151 >> v2:172.21.15.204:3300/0 conn(0x7f44900b4ac0 0x7f4490070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 --2- 172.21.15.143:0/3115046151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f4490159b70 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 -- 172.21.15.143:0/3115046151 >> 172.21.15.143:0/3115046151 conn(0x7f449006c3f0 msgr2=0x7f449006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 -- 172.21.15.143:0/3115046151 shutdown_connections 2024-01-15T18:17:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.660+0000 7f4498000640 1 -- 172.21.15.143:0/3115046151 wait complete. 2024-01-15T18:17:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 Processor -- start 2024-01-15T18:17:21.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 -- start start 2024-01-15T18:17:21.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44900b21d0 0x7f449013a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44900b4ac0 0x7f4490150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f44900ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490151820 con 0x7f4490157300 2024-01-15T18:17:21.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449013abc0 con 0x7f44900b21d0 2024-01-15T18:17:21.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4490151540 con 0x7f44900b4ac0 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f44900ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f44900ad0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56278/0 (socket says 172.21.15.143:56278) 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 -- 172.21.15.143:0/669879950 learned_addr learned my addr 172.21.15.143:0/669879950 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4495600640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44900b21d0 0x7f449013a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4494c00640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44900b4ac0 0x7f4490150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 -- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44900b4ac0 msgr2=0x7f4490150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44900b4ac0 0x7f4490150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 -- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44900b21d0 msgr2=0x7f449013a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44900b21d0 0x7f449013a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 -- 172.21.15.143:0/669879950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44800094d0 con 0x7f4490157300 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4495600640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44900b21d0 0x7f449013a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4496000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f44900ad0d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f448800b9e0 tx=0x7f448800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448800caa0 con 0x7f4490157300 2024-01-15T18:17:21.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448800cc20 con 0x7f4490157300 2024-01-15T18:17:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488014440 con 0x7f4490157300 2024-01-15T18:17:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4490152eb0 con 0x7f4490157300 2024-01-15T18:17:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.664+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4490153470 con 0x7f4490157300 2024-01-15T18:17:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.668+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4490152d10 con 0x7f4490157300 2024-01-15T18:17:21.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.668+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f448801d3b0 con 0x7f4490157300 2024-01-15T18:17:21.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.668+0000 7f448d600640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4468067820 0x7f4468069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.668+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f4488082d70 con 0x7f4490157300 2024-01-15T18:17:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.668+0000 7f4495600640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4468067820 0x7f4468069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.668+0000 7f4495600640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4468067820 0x7f4468069ce0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f44800ae000 tx=0x7f44800a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.672+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4490152d10 con 0x7f4490157300 2024-01-15T18:17:21.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.824+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f4490152d10 con 0x7f4490157300 2024-01-15T18:17:21.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.828+0000 7f448d600640 1 -- 172.21.15.143:0/669879950 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4490152d10 con 0x7f4490157300 2024-01-15T18:17:21.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4468067820 msgr2=0x7f4468069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4468067820 0x7f4468069ce0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f44800ae000 tx=0x7f44800a7000 comp rx=0 tx=0).stop 2024-01-15T18:17:21.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 msgr2=0x7f44900ad0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f44900ad0d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f448800b9e0 tx=0x7f448800bea0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 shutdown_connections 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4468067820 0x7f4468069ce0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4490157300 0x7f44900ad0d0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44900b4ac0 0x7f4490150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 --2- 172.21.15.143:0/669879950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44900b21d0 0x7f449013a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 >> 172.21.15.143:0/669879950 conn(0x7f449006c3f0 msgr2=0x7f44900b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 shutdown_connections 2024-01-15T18:17:21.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.832+0000 7f4498000640 1 -- 172.21.15.143:0/669879950 wait complete. 2024-01-15T18:17:21.850 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:17:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 Processor -- start 2024-01-15T18:17:21.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 -- start start 2024-01-15T18:17:21.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5c60143780 0x7f5c600cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 0x7f5c6015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c600ceb50 con 0x7f5c600ce520 2024-01-15T18:17:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6015c110 con 0x7f5c60143780 2024-01-15T18:17:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c64e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6015b7e0 con 0x7f5c600722c0 2024-01-15T18:17:21.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 0x7f5c6015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5c60143780 0x7f5c600cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 0x7f5c6015c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:39146/0 (socket says 172.21.15.143:39146) 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 -- 172.21.15.143:0/738547535 learned_addr learned my addr 172.21.15.143:0/738547535 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5d400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f5c600ce520 0x7f5c60071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43036/0 (socket says 172.21.15.143:43036) 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 -- 172.21.15.143:0/738547535 >> v2:172.21.15.204:3300/0 conn(0x7f5c60143780 msgr2=0x7f5c600cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 --2- 172.21.15.143:0/738547535 >> v2:172.21.15.204:3300/0 conn(0x7f5c60143780 0x7f5c600cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 -- 172.21.15.143:0/738547535 >> v1:172.21.15.204:6789/0 conn(0x7f5c600ce520 legacy=0x7f5c60071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5e800640 1 -- 172.21.15.143:0/738547535 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6015bac0 con 0x7f5c600722c0 2024-01-15T18:17:21.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.992+0000 7f5c5de00640 1 --2- 172.21.15.143:0/738547535 >> v2:172.21.15.204:3300/0 conn(0x7f5c60143780 0x7f5c600cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:21.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c5e800640 1 --2- 172.21.15.143:0/738547535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 0x7f5c6015c5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009e50 tx=0x7f5c5801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=905b52a875b68b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c5ca00640 1 -- 172.21.15.143:0/738547535 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c580a57f0 con 0x7f5c600722c0 2024-01-15T18:17:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c5ca00640 1 -- 172.21.15.143:0/738547535 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c580a5950 con 0x7f5c600722c0 2024-01-15T18:17:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c5ca00640 1 -- 172.21.15.143:0/738547535 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c580a5ce0 con 0x7f5c600722c0 2024-01-15T18:17:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c64e00640 1 -- 172.21.15.143:0/738547535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 msgr2=0x7f5c6015c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c64e00640 1 --2- 172.21.15.143:0/738547535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 0x7f5c6015c5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009e50 tx=0x7f5c5801ca10 comp rx=0 tx=0).stop 2024-01-15T18:17:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c64e00640 1 -- 172.21.15.143:0/738547535 shutdown_connections 2024-01-15T18:17:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c64e00640 1 --2- 172.21.15.143:0/738547535 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c600722c0 0x7f5c6015c5a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c64e00640 1 --2- 172.21.15.143:0/738547535 >> v2:172.21.15.204:3300/0 conn(0x7f5c60143780 0x7f5c600cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:21.996+0000 7f5c64e00640 1 -- 172.21.15.143:0/738547535 >> 172.21.15.143:0/738547535 conn(0x7f5c6006c3f0 msgr2=0x7f5c6006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 -- 172.21.15.143:0/738547535 shutdown_connections 2024-01-15T18:17:21.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 -- 172.21.15.143:0/738547535 wait complete. 2024-01-15T18:17:21.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 Processor -- start 2024-01-15T18:17:21.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 -- start start 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c600722c0 0x7f5c6013c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 0x7f5c60141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c60143780 0x7f5c6013e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6013e220 con 0x7f5c600ce520 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6015db60 con 0x7f5c600722c0 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c64e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6013d5a0 con 0x7f5c60143780 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 0x7f5c60141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 0x7f5c60141c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56290/0 (socket says 172.21.15.143:56290) 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 -- 172.21.15.143:0/3353398952 learned_addr learned my addr 172.21.15.143:0/3353398952 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5de00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c600722c0 0x7f5c6013c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5e800640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c60143780 0x7f5c6013e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 -- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c60143780 msgr2=0x7f5c6013e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c60143780 0x7f5c6013e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 -- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c600722c0 msgr2=0x7f5c6013c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c600722c0 0x7f5c6013c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 -- 172.21.15.143:0/3353398952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c50009420 con 0x7f5c600ce520 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.000+0000 7f5c5d400640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 0x7f5c60141c70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c490 tx=0x7f5c4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48007480 con 0x7f5c600ce520 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480075e0 con 0x7f5c600ce520 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004a10 con 0x7f5c600ce520 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c58009b30 con 0x7f5c600ce520 2024-01-15T18:17:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6013db60 con 0x7f5c600ce520 2024-01-15T18:17:22.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6013e360 con 0x7f5c600ce520 2024-01-15T18:17:22.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f5c48004b70 con 0x7f5c600ce520 2024-01-15T18:17:22.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c56000640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c38067820 0x7f5c38069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f5c48082190 con 0x7f5c600ce520 2024-01-15T18:17:22.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.004+0000 7f5c5de00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c38067820 0x7f5c38069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.008+0000 7f5c5de00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c38067820 0x7f5c38069ce0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009c50 tx=0x7f5c50009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.008+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5c6013e360 con 0x7f5c600ce520 2024-01-15T18:17:22.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.160+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5c600ceb50 con 0x7f5c600ce520 2024-01-15T18:17:22.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.164+0000 7f5c56000640 1 -- 172.21.15.143:0/3353398952 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5c600ceb50 con 0x7f5c600ce520 2024-01-15T18:17:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c38067820 msgr2=0x7f5c38069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c38067820 0x7f5c38069ce0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009c50 tx=0x7f5c50009190 comp rx=0 tx=0).stop 2024-01-15T18:17:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 msgr2=0x7f5c60141c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 0x7f5c60141c70 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f5c4800c490 tx=0x7f5c4800c950 comp rx=0 tx=0).stop 2024-01-15T18:17:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 shutdown_connections 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5c38067820 0x7f5c38069ce0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5c60143780 0x7f5c6013e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c600ce520 0x7f5c60141c70 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 --2- 172.21.15.143:0/3353398952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5c600722c0 0x7f5c6013c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 >> 172.21.15.143:0/3353398952 conn(0x7f5c6006c3f0 msgr2=0x7f5c600ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 shutdown_connections 2024-01-15T18:17:22.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.168+0000 7f5c64e00640 1 -- 172.21.15.143:0/3353398952 wait complete. 2024-01-15T18:17:22.186 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed' 2024-01-15T18:17:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe27bc00640 1 Processor -- start 2024-01-15T18:17:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe27bc00640 1 -- start start 2024-01-15T18:17:22.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe27bc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe2740b0310 0x7fe2740cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe27bc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740cbff0 con 0x7fe2740cf740 2024-01-15T18:17:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe27bc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2740d0c50 con 0x7fe27406bfb0 2024-01-15T18:17:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe27bc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2740cb6c0 con 0x7fe2740b0310 2024-01-15T18:17:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe279c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe2740cf740 0x7fe2740cfbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47778/0 (socket says 172.21.15.143:47778) 2024-01-15T18:17:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe279c00640 1 -- 172.21.15.143:0/4104386803 learned_addr learned my addr 172.21.15.143:0/4104386803 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe273e00640 1 --2- 172.21.15.143:0/4104386803 >> v2:172.21.15.204:3300/0 conn(0x7fe2740b0310 0x7fe2740cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe273400640 1 -- 172.21.15.143:0/4104386803 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379403993 0 0) 0x7fe2740cbff0 con 0x7fe2740cf740 2024-01-15T18:17:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe273400640 1 -- 172.21.15.143:0/4104386803 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe254003530 con 0x7fe2740cf740 2024-01-15T18:17:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe273e00640 1 -- 172.21.15.143:0/4104386803 >> v1:172.21.15.204:6789/0 conn(0x7fe27406bfb0 legacy=0x7fe27406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe273e00640 1 -- 172.21.15.143:0/4104386803 >> v1:172.21.15.143:6789/0 conn(0x7fe2740cf740 legacy=0x7fe2740cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.324+0000 7fe273e00640 1 -- 172.21.15.143:0/4104386803 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2740cb9a0 con 0x7fe2740b0310 2024-01-15T18:17:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.328+0000 7fe273e00640 1 --2- 172.21.15.143:0/4104386803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740b0310 0x7fe2740cf200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097c0 tx=0x7fe25c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4fc704abe7241b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.328+0000 7fe273400640 1 -- 172.21.15.143:0/4104386803 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0ab600 con 0x7fe2740b0310 2024-01-15T18:17:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.328+0000 7fe273400640 1 -- 172.21.15.143:0/4104386803 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c0a7050 con 0x7fe2740b0310 2024-01-15T18:17:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.328+0000 7fe273400640 1 -- 172.21.15.143:0/4104386803 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0ab980 con 0x7fe2740b0310 2024-01-15T18:17:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.328+0000 7fe27bc00640 1 -- 172.21.15.143:0/4104386803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740b0310 msgr2=0x7fe2740cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.328+0000 7fe27bc00640 1 --2- 172.21.15.143:0/4104386803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740b0310 0x7fe2740cf200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe25c0097c0 tx=0x7fe25c01c3c0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- 172.21.15.143:0/4104386803 shutdown_connections 2024-01-15T18:17:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 --2- 172.21.15.143:0/4104386803 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740b0310 0x7fe2740cf200 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- 172.21.15.143:0/4104386803 >> 172.21.15.143:0/4104386803 conn(0x7fe2740d0ed0 msgr2=0x7fe2740d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- 172.21.15.143:0/4104386803 shutdown_connections 2024-01-15T18:17:22.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- 172.21.15.143:0/4104386803 wait complete. 2024-01-15T18:17:22.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 Processor -- start 2024-01-15T18:17:22.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- start start 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe27406bfb0 0x7fe2741596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 0x7fe274154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740cf740 0x7fe2740c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe274159bf0 con 0x7fe2740b0310 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe274154b60 con 0x7fe2740cf740 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe27bc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2740c7750 con 0x7fe27406bfb0 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe273e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 0x7fe274154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe273e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 0x7fe274154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56300/0 (socket says 172.21.15.143:56300) 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe273e00640 1 -- 172.21.15.143:0/2968249383 learned_addr learned my addr 172.21.15.143:0/2968249383 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe279c00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740cf740 0x7fe2740c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe279200640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe27406bfb0 0x7fe2741596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe273e00640 1 -- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe27406bfb0 msgr2=0x7fe2741596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.332+0000 7fe273e00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe27406bfb0 0x7fe2741596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe273e00640 1 -- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740cf740 msgr2=0x7fe2740c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe273e00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740cf740 0x7fe2740c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe273e00640 1 -- 172.21.15.143:0/2968249383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c009420 con 0x7fe2740b0310 2024-01-15T18:17:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe273e00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 0x7fe274154620 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe25c018fd0 tx=0x7fe25c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0b7450 con 0x7fe2740b0310 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c0b75b0 con 0x7fe2740b0310 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0b7960 con 0x7fe2740b0310 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2740c7a30 con 0x7fe2740b0310 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2740c5ab0 con 0x7fe2740b0310 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe274067500 con 0x7fe2740b0310 2024-01-15T18:17:22.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fe25c0b7ac0 con 0x7fe2740b0310 2024-01-15T18:17:22.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.336+0000 7fe270c00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe24c067820 0x7fe24c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.340+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe25c11d4f0 con 0x7fe2740b0310 2024-01-15T18:17:22.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.340+0000 7fe279200640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe24c067820 0x7fe24c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.337 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.340+0000 7fe279200640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe24c067820 0x7fe24c069ce0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe2741f92b0 tx=0x7fe264009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.344+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe25c0ec230 con 0x7fe2740b0310 2024-01-15T18:17:22.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.548+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]} v 0) v1 -- 0x7fe2740d0a30 con 0x7fe2740b0310 2024-01-15T18:17:22.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe270c00640 1 -- 172.21.15.143:0/2968249383 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]}]=0 v580) v1 ==== 145+0+0 (secure 0 0 0) 0x7fe25c0ef850 con 0x7fe2740b0310 2024-01-15T18:17:22.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe24c067820 msgr2=0x7fe24c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe24c067820 0x7fe24c069ce0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe2741f92b0 tx=0x7fe264009040 comp rx=0 tx=0).stop 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 msgr2=0x7fe274154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 0x7fe274154620 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fe25c018fd0 tx=0x7fe25c01ac00 comp rx=0 tx=0).stop 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 shutdown_connections 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe24c067820 0x7fe24c069ce0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe2740cf740 0x7fe2740c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe2740b0310 0x7fe274154620 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 --2- 172.21.15.143:0/2968249383 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe27406bfb0 0x7fe2741596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 >> 172.21.15.143:0/2968249383 conn(0x7fe2740d0ed0 msgr2=0x7fe27406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 shutdown_connections 2024-01-15T18:17:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:22.864+0000 7fe27bc00640 1 -- 172.21.15.143:0/2968249383 wait complete. 2024-01-15T18:17:22.885 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-01-15T18:17:22.885 INFO:tasks.cephfs_test_runner:Starting test: test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:17:22.885 INFO:tasks.cephfs_test_runner:test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:22.886 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:23.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.024+0000 7f871ca00640 1 Processor -- start 2024-01-15T18:17:23.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.024+0000 7f871ca00640 1 -- start start 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.024+0000 7f871ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 0x7f8714072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.024+0000 7f871ca00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87140ceb50 con 0x7f8714143780 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.024+0000 7f871ca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871415c110 con 0x7f871415c5a0 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.024+0000 7f871ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8714072570 con 0x7f8714071bb0 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f871a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8714143780 0x7f87140cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47792/0 (socket says 172.21.15.143:47792) 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f871a000640 1 -- 172.21.15.143:0/3749901180 learned_addr learned my addr 172.21.15.143:0/3749901180 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 0x7f8714072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28238360 0 0) 0x7f8704001120 con 0x7f8714143780 2024-01-15T18:17:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f4003530 con 0x7f8714143780 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1835922424 0 0) 0x7f871415c110 con 0x7f871415c5a0 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8704001120 con 0x7f871415c5a0 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2405337156 0 0) 0x7f86f4003530 con 0x7f8714143780 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 msgr2=0x7f8714072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 0x7f8714072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 >> v1:172.21.15.204:6789/0 conn(0x7f871415c5a0 legacy=0x7f871415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871415b7e0 con 0x7f8714143780 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f8704001810 con 0x7f8714143780 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 >> v1:172.21.15.143:6789/0 conn(0x7f8714143780 legacy=0x7f87140cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f4003b40 0x7f86f4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f86f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 0x7f86f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8714072570 con 0x7f86f40047d0 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f400ba40 con 0x7f86f4003b40 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f400bc00 con 0x7f8714071bb0 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f86f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f871a000640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f4003b40 0x7f86f4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f871aa00640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 0x7f86f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 -- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 msgr2=0x7f86f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8714071bb0 0x7f86f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 -- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f4003b40 msgr2=0x7f86f4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86f4003b40 0x7f86f4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.028+0000 7f8719600640 1 -- 172.21.15.143:0/3749901180 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4003530 con 0x7f86f40047d0 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f8719600640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f86f4004c50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009ae0 tx=0x7f86fc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8e34867bfecff4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0a5640 con 0x7f86f40047d0 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc0a57a0 con 0x7f86f40047d0 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f8718c00640 1 -- 172.21.15.143:0/3749901180 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0a5b50 con 0x7f86f40047d0 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 -- 172.21.15.143:0/3749901180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 msgr2=0x7f86f4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 --2- 172.21.15.143:0/3749901180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f86f4004c50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009ae0 tx=0x7f86fc01ca50 comp rx=0 tx=0).stop 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871aa00640 1 -- 172.21.15.143:0/3749901180 reap_dead start 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 -- 172.21.15.143:0/3749901180 shutdown_connections 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 -- 172.21.15.143:0/3749901180 >> 172.21.15.143:0/3749901180 conn(0x7f871406c3f0 msgr2=0x7f871406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 -- 172.21.15.143:0/3749901180 shutdown_connections 2024-01-15T18:17:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 -- 172.21.15.143:0/3749901180 wait complete. 2024-01-15T18:17:23.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 Processor -- start 2024-01-15T18:17:23.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.032+0000 7f871ca00640 1 -- start start 2024-01-15T18:17:23.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f87141591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f871413c800 0x7f87140c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f871406cd70 0x7f871406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871406d760 con 0x7f86f40047d0 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871413c370 con 0x7f871413c800 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87141596f0 con 0x7f871406cd70 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f87141591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f87141591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56316/0 (socket says 172.21.15.143:56316) 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 -- 172.21.15.143:0/51964882 learned_addr learned my addr 172.21.15.143:0/51964882 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871aa00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f871406cd70 0x7f871406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f8719600640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f871413c800 0x7f87140c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 -- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f871406cd70 msgr2=0x7f871406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f871406cd70 0x7f871406d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 -- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f871413c800 msgr2=0x7f87140c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f871413c800 0x7f87140c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 -- 172.21.15.143:0/51964882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871413b760 con 0x7f86f40047d0 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871a000640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f87141591b0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f870400e8f0 tx=0x7f870400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400c8d0 con 0x7f86f40047d0 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870400ca30 con 0x7f86f40047d0 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871413ba40 con 0x7f86f40047d0 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870400cde0 con 0x7f86f40047d0 2024-01-15T18:17:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871413c000 con 0x7f86f40047d0 2024-01-15T18:17:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8714067500 con 0x7f86f40047d0 2024-01-15T18:17:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.036+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f8704013420 con 0x7f86f40047d0 2024-01-15T18:17:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.040+0000 7f8712000640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86ec067850 0x7f86ec069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.040+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f8704082c80 con 0x7f86f40047d0 2024-01-15T18:17:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.040+0000 7f8719600640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86ec067850 0x7f86ec069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.040+0000 7f8719600640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86ec067850 0x7f86ec069d10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0af000 tx=0x7f86fc0046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.044+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f87040516b0 con 0x7f86f40047d0 2024-01-15T18:17:23.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.192+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87140ceb50 con 0x7f86f40047d0 2024-01-15T18:17:23.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.192+0000 7f8712000640 1 -- 172.21.15.143:0/51964882 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13113 (secure 0 0 0) 0x7f87140ceb50 con 0x7f86f40047d0 2024-01-15T18:17:23.193 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:23.193 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":92,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:18.153310+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6837/1230123305":"2024-01-16T18:17:15.135584+0000","172.21.15.143:6836/1230123305":"2024-01-16T18:17:15.135584+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:23.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86ec067850 msgr2=0x7f86ec069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86ec067850 0x7f86ec069d10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0af000 tx=0x7f86fc0046d0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 msgr2=0x7f87141591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f87141591b0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f870400e8f0 tx=0x7f870400edb0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 shutdown_connections 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86ec067850 0x7f86ec069d10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f871406cd70 0x7f871406d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f871413c800 0x7f87140c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 --2- 172.21.15.143:0/51964882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86f40047d0 0x7f87141591b0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 >> 172.21.15.143:0/51964882 conn(0x7f871406c3f0 msgr2=0x7f87140ad9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 shutdown_connections 2024-01-15T18:17:23.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.196+0000 7f871ca00640 1 -- 172.21.15.143:0/51964882 wait complete. 2024-01-15T18:17:23.215 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:23.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:23.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe846c00640 1 Processor -- start 2024-01-15T18:17:23.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe846c00640 1 -- start start 2024-01-15T18:17:23.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe846c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8401607c0 0x7fe84006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe846c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400cffd0 con 0x7fe8400cbf50 2024-01-15T18:17:23.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe846c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400d02b0 con 0x7fe8400cfb30 2024-01-15T18:17:23.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe846c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400d1ce0 con 0x7fe8401607c0 2024-01-15T18:17:23.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe844c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe8400cbf50 0x7fe8400cc390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47802/0 (socket says 172.21.15.143:47802) 2024-01-15T18:17:23.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe844c00640 1 -- 172.21.15.143:0/2755306944 learned_addr learned my addr 172.21.15.143:0/2755306944 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:23.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83fe00640 1 --2- 172.21.15.143:0/2755306944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8401607c0 0x7fe84006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987784157 0 0) 0x7fe8400cffd0 con 0x7fe8400cbf50 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe820003530 con 0x7fe8400cbf50 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736905483 0 0) 0x7fe8400d02b0 con 0x7fe8400cfb30 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8400cffd0 con 0x7fe8400cfb30 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83fe00640 1 -- 172.21.15.143:0/2755306944 >> v1:172.21.15.204:6789/0 conn(0x7fe8400cfb30 legacy=0x7fe84006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376057623 0 0) 0x7fe820003530 con 0x7fe8400cbf50 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83fe00640 1 -- 172.21.15.143:0/2755306944 >> v1:172.21.15.143:6789/0 conn(0x7fe8400cbf50 legacy=0x7fe8400cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.356+0000 7fe83fe00640 1 -- 172.21.15.143:0/2755306944 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400cb470 con 0x7fe8401607c0 2024-01-15T18:17:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe83fe00640 1 --2- 172.21.15.143:0/2755306944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8401607c0 0x7fe84006b7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097f0 tx=0x7fe83001ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f48de19f4b7ba398 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838001a00 con 0x7fe8401607c0 2024-01-15T18:17:23.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8300a8650 con 0x7fe8401607c0 2024-01-15T18:17:23.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe83ea00640 1 -- 172.21.15.143:0/2755306944 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300a89c0 con 0x7fe8401607c0 2024-01-15T18:17:23.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 -- 172.21.15.143:0/2755306944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8401607c0 msgr2=0x7fe84006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 --2- 172.21.15.143:0/2755306944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8401607c0 0x7fe84006b7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097f0 tx=0x7fe83001ebd0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 -- 172.21.15.143:0/2755306944 shutdown_connections 2024-01-15T18:17:23.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 --2- 172.21.15.143:0/2755306944 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8401607c0 0x7fe84006b7b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 -- 172.21.15.143:0/2755306944 >> 172.21.15.143:0/2755306944 conn(0x7fe8400d0e20 msgr2=0x7fe8400d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:23.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 -- 172.21.15.143:0/2755306944 shutdown_connections 2024-01-15T18:17:23.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.360+0000 7fe846c00640 1 -- 172.21.15.143:0/2755306944 wait complete. 2024-01-15T18:17:23.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 Processor -- start 2024-01-15T18:17:23.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 -- start start 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8400cbf50 0x7fe8400c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8400cfb30 0x7fe8400c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 0x7fe840154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8401f3f90 con 0x7fe8401607c0 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400c5de0 con 0x7fe8400cfb30 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe846c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400c60c0 con 0x7fe8400cbf50 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 0x7fe840154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 0x7fe840154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56328/0 (socket says 172.21.15.143:56328) 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 -- 172.21.15.143:0/1409010684 learned_addr learned my addr 172.21.15.143:0/1409010684 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:23.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe83fe00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8400cbf50 0x7fe8400c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe83f400640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8400cfb30 0x7fe8400c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 -- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8400cbf50 msgr2=0x7fe8400c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8400cbf50 0x7fe8400c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 -- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8400cfb30 msgr2=0x7fe8400c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8400cfb30 0x7fe8400c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 -- 172.21.15.143:0/1409010684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830009420 con 0x7fe8401607c0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe844c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 0x7fe840154f20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe838001fb0 tx=0x7fe83800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.364+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838012400 con 0x7fe8401607c0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838012560 con 0x7fe8401607c0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8401f4250 con 0x7fe8401607c0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838012910 con 0x7fe8401607c0 2024-01-15T18:17:23.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8401561e0 con 0x7fe8401607c0 2024-01-15T18:17:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84006d4b0 con 0x7fe8401607c0 2024-01-15T18:17:23.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fe838012a70 con 0x7fe8401607c0 2024-01-15T18:17:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe837e00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe818067820 0x7fe818069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe8380838d0 con 0x7fe8401607c0 2024-01-15T18:17:23.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.368+0000 7fe83fe00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe818067820 0x7fe818069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.372+0000 7fe83fe00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe818067820 0x7fe818069ce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe8300b2000 tx=0x7fe8300aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.372+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe838052320 con 0x7fe8401607c0 2024-01-15T18:17:23.521 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe840067500 con 0x7fe8401607c0 2024-01-15T18:17:23.522 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe837e00640 1 -- 172.21.15.143:0/1409010684 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe838055940 con 0x7fe8401607c0 2024-01-15T18:17:23.522 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe818067820 msgr2=0x7fe818069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe818067820 0x7fe818069ce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe8300b2000 tx=0x7fe8300aa070 comp rx=0 tx=0).stop 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 msgr2=0x7fe840154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 0x7fe840154f20 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe838001fb0 tx=0x7fe83800a430 comp rx=0 tx=0).stop 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 shutdown_connections 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe818067820 0x7fe818069ce0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8401607c0 0x7fe840154f20 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8400cfb30 0x7fe8400c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 --2- 172.21.15.143:0/1409010684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8400cbf50 0x7fe8400c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 >> 172.21.15.143:0/1409010684 conn(0x7fe8400d0e20 msgr2=0x7fe84015f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:23.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 shutdown_connections 2024-01-15T18:17:23.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.524+0000 7fe846c00640 1 -- 172.21.15.143:0/1409010684 wait complete. 2024-01-15T18:17:23.545 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled' 2024-01-15T18:17:23.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.680+0000 7fbe64c00640 1 Processor -- start 2024-01-15T18:17:23.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe64c00640 1 -- start start 2024-01-15T18:17:23.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe64c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbe60072410 0x7fbe6015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe64c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6015f310 con 0x7fbe600ce560 2024-01-15T18:17:23.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe64c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6015f5f0 con 0x7fbe60141660 2024-01-15T18:17:23.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe64c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6015fbb0 con 0x7fbe60072410 2024-01-15T18:17:23.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe5e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbe60072410 0x7fbe6015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe5e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbe60072410 0x7fbe6015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56342/0 (socket says 172.21.15.143:56342) 2024-01-15T18:17:23.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe5e800640 1 -- 172.21.15.143:0/1180754240 learned_addr learned my addr 172.21.15.143:0/1180754240 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:23.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe5e800640 1 -- 172.21.15.143:0/1180754240 >> v1:172.21.15.204:6789/0 conn(0x7fbe60141660 legacy=0x7fbe60141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe5e800640 1 -- 172.21.15.143:0/1180754240 >> v1:172.21.15.143:6789/0 conn(0x7fbe600ce560 legacy=0x7fbe60071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.684+0000 7fbe5e800640 1 -- 172.21.15.143:0/1180754240 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6015f8d0 con 0x7fbe60072410 2024-01-15T18:17:23.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe5e800640 1 --2- 172.21.15.143:0/1180754240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6015ec80 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009e50 tx=0x7fbe5801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971549df90f71f9f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe5ca00640 1 -- 172.21.15.143:0/1180754240 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580a57f0 con 0x7fbe60072410 2024-01-15T18:17:23.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe5ca00640 1 -- 172.21.15.143:0/1180754240 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe580a5950 con 0x7fbe60072410 2024-01-15T18:17:23.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe5ca00640 1 -- 172.21.15.143:0/1180754240 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580a5ce0 con 0x7fbe60072410 2024-01-15T18:17:23.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 -- 172.21.15.143:0/1180754240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 msgr2=0x7fbe6015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 --2- 172.21.15.143:0/1180754240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6015ec80 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009e50 tx=0x7fbe5801ca10 comp rx=0 tx=0).stop 2024-01-15T18:17:23.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 -- 172.21.15.143:0/1180754240 shutdown_connections 2024-01-15T18:17:23.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 --2- 172.21.15.143:0/1180754240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6015ec80 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 -- 172.21.15.143:0/1180754240 >> 172.21.15.143:0/1180754240 conn(0x7fbe6006c3f0 msgr2=0x7fbe6006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:23.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 -- 172.21.15.143:0/1180754240 shutdown_connections 2024-01-15T18:17:23.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.688+0000 7fbe64c00640 1 -- 172.21.15.143:0/1180754240 wait complete. 2024-01-15T18:17:23.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 Processor -- start 2024-01-15T18:17:23.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 -- start start 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6012a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe600ce560 0x7fbe60154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe60141660 0x7fbe600ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6012a9c0 con 0x7fbe60072410 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe60139d70 con 0x7fbe600ce560 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe60139160 con 0x7fbe60141660 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6012a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6012a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56352/0 (socket says 172.21.15.143:56352) 2024-01-15T18:17:23.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 -- 172.21.15.143:0/3514771565 learned_addr learned my addr 172.21.15.143:0/3514771565 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5e800640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe60141660 0x7fbe600ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5d400640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe600ce560 0x7fbe60154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 -- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe60141660 msgr2=0x7fbe600ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe60141660 0x7fbe600ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 -- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe600ce560 msgr2=0x7fbe60154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe600ce560 0x7fbe60154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 -- 172.21.15.143:0/3514771565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58009b30 con 0x7fbe60072410 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe5de00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6012a480 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c3c0 tx=0x7fbe5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500052d0 con 0x7fbe60072410 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50005430 con 0x7fbe60072410 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50003840 con 0x7fbe60072410 2024-01-15T18:17:23.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe600cd150 con 0x7fbe60072410 2024-01-15T18:17:23.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.692+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe60139720 con 0x7fbe60072410 2024-01-15T18:17:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.696+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe60067500 con 0x7fbe60072410 2024-01-15T18:17:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.696+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fbe500039a0 con 0x7fbe60072410 2024-01-15T18:17:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.696+0000 7fbe56000640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe38067850 0x7fbe38069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.696+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fbe500823e0 con 0x7fbe60072410 2024-01-15T18:17:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.696+0000 7fbe5d400640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe38067850 0x7fbe38069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:23.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.696+0000 7fbe5d400640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe38067850 0x7fbe38069d10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbe601f9880 tx=0x7fbe48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:23.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.700+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbe50051120 con 0x7fbe60072410 2024-01-15T18:17:23.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:23.904+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]} v 0) v1 -- 0x7fbe6006e2a0 con 0x7fbe60072410 2024-01-15T18:17:24.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.884+0000 7fbe56000640 1 -- 172.21.15.143:0/3514771565 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]}]=0 v582) v1 ==== 156+0+0 (secure 0 0 0) 0x7fbe50054740 con 0x7fbe60072410 2024-01-15T18:17:24.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe38067850 msgr2=0x7fbe38069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:24.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe38067850 0x7fbe38069d10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbe601f9880 tx=0x7fbe48009040 comp rx=0 tx=0).stop 2024-01-15T18:17:24.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 msgr2=0x7fbe6012a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:24.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6012a480 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c3c0 tx=0x7fbe5000c880 comp rx=0 tx=0).stop 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 shutdown_connections 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe38067850 0x7fbe38069d10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbe60141660 0x7fbe600ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbe600ce560 0x7fbe60154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 --2- 172.21.15.143:0/3514771565 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe60072410 0x7fbe6012a480 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 >> 172.21.15.143:0/3514771565 conn(0x7fbe6006c3f0 msgr2=0x7fbe600af290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 shutdown_connections 2024-01-15T18:17:24.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:24.888+0000 7fbe64c00640 1 -- 172.21.15.143:0/3514771565 wait complete. 2024-01-15T18:17:24.909 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:17:25.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.052+0000 7fb088600640 1 Processor -- start 2024-01-15T18:17:25.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 -- start start 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb080143780 0x7fb0800cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb0800ce520 0x7fb080071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0800720f0 0x7fb08015a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb080072580 con 0x7fb0800720f0 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0800ceb50 con 0x7fb0800ce520 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb088600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08015c110 con 0x7fb080143780 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0800720f0 0x7fb08015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb0800720f0 0x7fb08015a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56354/0 (socket says 172.21.15.143:56354) 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 -- 172.21.15.143:0/558309547 learned_addr learned my addr 172.21.15.143:0/558309547 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb085c00640 1 --2- 172.21.15.143:0/558309547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb080143780 0x7fb0800cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb085200640 1 --2- 172.21.15.143:0/558309547 >> v2:172.21.15.204:3300/0 conn(0x7fb0800ce520 0x7fb080071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 -- 172.21.15.143:0/558309547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb080143780 msgr2=0x7fb0800cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 --2- 172.21.15.143:0/558309547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb080143780 0x7fb0800cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 -- 172.21.15.143:0/558309547 >> v2:172.21.15.204:3300/0 conn(0x7fb0800ce520 msgr2=0x7fb080071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 --2- 172.21.15.143:0/558309547 >> v2:172.21.15.204:3300/0 conn(0x7fb0800ce520 0x7fb080071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.056+0000 7fb086600640 1 -- 172.21.15.143:0/558309547 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08015b7e0 con 0x7fb0800720f0 2024-01-15T18:17:25.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb086600640 1 --2- 172.21.15.143:0/558309547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0800720f0 0x7fb08015a450 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb08006bee0 tx=0x7fb07801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb436c2d820afe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb07fe00640 1 -- 172.21.15.143:0/558309547 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780a5760 con 0x7fb0800720f0 2024-01-15T18:17:25.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb07fe00640 1 -- 172.21.15.143:0/558309547 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0780a58c0 con 0x7fb0800720f0 2024-01-15T18:17:25.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb07fe00640 1 -- 172.21.15.143:0/558309547 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780a5c50 con 0x7fb0800720f0 2024-01-15T18:17:25.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 -- 172.21.15.143:0/558309547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0800720f0 msgr2=0x7fb08015a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 --2- 172.21.15.143:0/558309547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0800720f0 0x7fb08015a450 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb08006bee0 tx=0x7fb07801c9d0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 -- 172.21.15.143:0/558309547 shutdown_connections 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 --2- 172.21.15.143:0/558309547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb0800720f0 0x7fb08015a450 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 --2- 172.21.15.143:0/558309547 >> v2:172.21.15.204:3300/0 conn(0x7fb0800ce520 0x7fb080071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 --2- 172.21.15.143:0/558309547 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb080143780 0x7fb0800cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 -- 172.21.15.143:0/558309547 >> 172.21.15.143:0/558309547 conn(0x7fb08006c3f0 msgr2=0x7fb08006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 -- 172.21.15.143:0/558309547 shutdown_connections 2024-01-15T18:17:25.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 -- 172.21.15.143:0/558309547 wait complete. 2024-01-15T18:17:25.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.060+0000 7fb088600640 1 Processor -- start 2024-01-15T18:17:25.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- start start 2024-01-15T18:17:25.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0800720f0 0x7fb08013a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0800ce520 0x7fb08013f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 0x7fb080158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08013a900 con 0x7fb080143780 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08013e5f0 con 0x7fb0800ce520 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08013e8d0 con 0x7fb0800720f0 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 0x7fb080158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 0x7fb080158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56360/0 (socket says 172.21.15.143:56360) 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 -- 172.21.15.143:0/3821288724 learned_addr learned my addr 172.21.15.143:0/3821288724 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb085200640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0800ce520 0x7fb08013f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb085c00640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0800720f0 0x7fb08013a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 -- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0800720f0 msgr2=0x7fb08013a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0800720f0 0x7fb08013a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 -- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0800ce520 msgr2=0x7fb08013f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0800ce520 0x7fb08013f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 -- 172.21.15.143:0/3821288724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009b30 con 0x7fb080143780 2024-01-15T18:17:25.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb086600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 0x7fb080158a60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb07801b080 tx=0x7fb078019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780b8670 con 0x7fb080143780 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0780b87d0 con 0x7fb080143780 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780b8b60 con 0x7fb080143780 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08013ebb0 con 0x7fb080143780 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08013f170 con 0x7fb080143780 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.064+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0800b4f50 con 0x7fb080143780 2024-01-15T18:17:25.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.068+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fb0780b8cc0 con 0x7fb080143780 2024-01-15T18:17:25.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.068+0000 7fb07d600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb058067820 0x7fb058069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.068+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fb07811e330 con 0x7fb080143780 2024-01-15T18:17:25.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.068+0000 7fb085c00640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb058067820 0x7fb058069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.068+0000 7fb085c00640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb058067820 0x7fb058069ce0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb0801f93f0 tx=0x7fb070008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.072+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb0780ecfc0 con 0x7fb080143780 2024-01-15T18:17:25.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.224+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb080067500 con 0x7fb080143780 2024-01-15T18:17:25.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb07d600640 1 -- 172.21.15.143:0/3821288724 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb0780bc020 con 0x7fb080143780 2024-01-15T18:17:25.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb058067820 msgr2=0x7fb058069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb058067820 0x7fb058069ce0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb0801f93f0 tx=0x7fb070008040 comp rx=0 tx=0).stop 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 msgr2=0x7fb080158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 0x7fb080158a60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fb07801b080 tx=0x7fb078019770 comp rx=0 tx=0).stop 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 shutdown_connections 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb058067820 0x7fb058069ce0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb080143780 0x7fb080158a60 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0800ce520 0x7fb08013f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 --2- 172.21.15.143:0/3821288724 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb0800720f0 0x7fb08013a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.228+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 >> 172.21.15.143:0/3821288724 conn(0x7fb08006c3f0 msgr2=0x7fb0800ada60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:25.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.232+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 shutdown_connections 2024-01-15T18:17:25.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.232+0000 7fb088600640 1 -- 172.21.15.143:0/3821288724 wait complete. 2024-01-15T18:17:25.248 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:17:25.248 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:17:25.308 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:17:25.377 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:17:25.377 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:25.377 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:25.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:17:25.386 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:17:25.386 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:25.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:17:25.395 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:25.395 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:25.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:17:25.437 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:17:25.437 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:25.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:17:25.446 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:17:25.446 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:17:25.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 Processor -- start 2024-01-15T18:17:25.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 -- start start 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc06bfb0 0x7f3ddc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ddc0d0210 0x7f3ddc0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc0cbff0 con 0x7f3ddc0b0310 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc0d0c50 con 0x7f3ddc0d0210 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc06c8d0 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ddc0d0210 0x7f3ddc0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3ddc0d0210 0x7f3ddc0ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56366/0 (socket says 172.21.15.143:56366) 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 -- 172.21.15.143:0/1526852011 learned_addr learned my addr 172.21.15.143:0/1526852011 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de1c00640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc06bfb0 0x7f3ddc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 -- 172.21.15.143:0/1526852011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc06bfb0 msgr2=0x7f3ddc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc06bfb0 0x7f3ddc06c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 -- 172.21.15.143:0/1526852011 >> v1:172.21.15.143:6789/0 conn(0x7f3ddc0b0310 legacy=0x7f3ddc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de2600640 1 -- 172.21.15.143:0/1526852011 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc0cb6c0 con 0x7f3ddc0d0210 2024-01-15T18:17:25.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.620+0000 7f3de1c00640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc06bfb0 0x7f3ddc06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:25.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de2600640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc0ccb90 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009e50 tx=0x7f3dd401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd602fe7967d78b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3ddbe00640 1 -- 172.21.15.143:0/1526852011 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40a5810 con 0x7f3ddc0d0210 2024-01-15T18:17:25.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3ddbe00640 1 -- 172.21.15.143:0/1526852011 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd40a5970 con 0x7f3ddc0d0210 2024-01-15T18:17:25.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3ddbe00640 1 -- 172.21.15.143:0/1526852011 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd40a5d00 con 0x7f3ddc0d0210 2024-01-15T18:17:25.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 -- 172.21.15.143:0/1526852011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc0d0210 msgr2=0x7f3ddc0ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc0ccb90 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009e50 tx=0x7f3dd401c950 comp rx=0 tx=0).stop 2024-01-15T18:17:25.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 -- 172.21.15.143:0/1526852011 shutdown_connections 2024-01-15T18:17:25.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc0ccb90 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 --2- 172.21.15.143:0/1526852011 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc06bfb0 0x7f3ddc06c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 -- 172.21.15.143:0/1526852011 >> 172.21.15.143:0/1526852011 conn(0x7f3ddc0d0ed0 msgr2=0x7f3ddc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:25.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 -- 172.21.15.143:0/1526852011 shutdown_connections 2024-01-15T18:17:25.624 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.624+0000 7f3de4600640 1 -- 172.21.15.143:0/1526852011 wait complete. 2024-01-15T18:17:25.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 Processor -- start 2024-01-15T18:17:25.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- start start 2024-01-15T18:17:25.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 0x7f3ddc0c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc0b0310 0x7f3ddc154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0c6b30 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0c5f20 con 0x7f3ddc0d0210 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0c6200 con 0x7f3ddc0b0310 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 0x7f3ddc0c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 0x7f3ddc0c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56378/0 (socket says 172.21.15.143:56378) 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 -- 172.21.15.143:0/3063497442 learned_addr learned my addr 172.21.15.143:0/3063497442 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de2600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1200640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc0b0310 0x7f3ddc154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 -- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc0b0310 msgr2=0x7f3ddc154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.628 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc0b0310 0x7f3ddc154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 -- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ddc0d0210 msgr2=0x7f3ddc154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 -- 172.21.15.143:0/3063497442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4009b30 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1200640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc0b0310 0x7f3ddc154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de1c00640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 0x7f3ddc0c5840 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00d800 tx=0x7f3dcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00b6c0 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc00b820 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00bbd0 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc0c64e0 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.629 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc0c7730 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.628+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc067500 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.632+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f3dcc00bd30 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.632+0000 7f3dd9600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3db4067820 0x7f3db4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.630 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.632+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f3dcc083680 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.632+0000 7f3de1200640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3db4067820 0x7f3db4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.632+0000 7f3de1200640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3db4067820 0x7f3db4069ce0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc155350 tx=0x7f3dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.636+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3dcc052020 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.816+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ddc0ce660 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.816+0000 7f3dd9600640 1 -- 172.21.15.143:0/3063497442 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f3dcc055640 con 0x7f3ddc06bfb0 2024-01-15T18:17:25.815 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:25.815 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6875,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/1584800432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":1584800432},{"type":"v1","addr":"172.21.15.204:6834","nonce":1584800432}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7843,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/3526494826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3526494826},{"type":"v1","addr":"172.21.15.204:6835","nonce":3526494826}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8016,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1307769037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1307769037},{"type":"v1","addr":"172.21.15.143:6835","nonce":1307769037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/606420289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":606420289},{"type":"v1","addr":"172.21.15.143:6837","nonce":606420289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2024-01-15T18:17:25.815 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 55 2024-01-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3db4067820 msgr2=0x7f3db4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3db4067820 0x7f3db4069ce0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc155350 tx=0x7f3dc4009040 comp rx=0 tx=0).stop 2024-01-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 msgr2=0x7f3ddc0c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 0x7f3ddc0c5840 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00d800 tx=0x7f3dcc00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 shutdown_connections 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3db4067820 0x7f3db4069ce0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3ddc0d0210 0x7f3ddc154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3ddc0b0310 0x7f3ddc154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 --2- 172.21.15.143:0/3063497442 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ddc06bfb0 0x7f3ddc0c5840 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 >> 172.21.15.143:0/3063497442 conn(0x7f3ddc0d0ed0 msgr2=0x7f3ddc06f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 shutdown_connections 2024-01-15T18:17:25.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:25.820+0000 7f3de4600640 1 -- 172.21.15.143:0/3063497442 wait complete. 2024-01-15T18:17:25.837 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:17:25.837 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:17:25.837 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:17:25.837 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:17:25.838 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:17:25.838 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:17:25.838 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:17:25.838 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:17:25.838 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:17:25.838 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:17:25.839 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:17:25.839 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:17:25.839 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:17:25.839 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:17:25.839 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:17:25.839 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:17:25.852 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:17:25.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:17:25.853 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:17:25.853 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:17:25.855 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:17:25.855 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:17:25.856 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:17:25.856 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:17:25.903 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:17:25.903 DEBUG:teuthology.parallel:result is None 2024-01-15T18:17:25.904 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:17:25.904 DEBUG:teuthology.parallel:result is None 2024-01-15T18:17:25.905 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:17:25.905 DEBUG:teuthology.parallel:result is None 2024-01-15T18:17:25.906 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:17:25.906 DEBUG:teuthology.parallel:result is None 2024-01-15T18:17:25.906 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:17:25.991 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:17:25.993 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:17:25.993 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:17:25.994 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:17:26.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 Processor -- start 2024-01-15T18:17:26.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 -- start start 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea1c15d130 0x7fea1c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 0x7fea1c06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c06a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c06ae10 con 0x7fea1c15d510 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c160eb0 con 0x7fea1c15d130 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea22e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c161190 con 0x7fea1c06a490 2024-01-15T18:17:26.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 0x7fea1c06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea1c15d130 0x7fea1c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea20e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c06a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fea1c15d130 0x7fea1c0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:51372/0 (socket says 172.21.15.143:51372) 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 0x7fea1c06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56416/0 (socket says 172.21.15.143:56416) 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 -- 172.21.15.143:0/736557110 learned_addr learned my addr 172.21.15.143:0/736557110 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 -- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 msgr2=0x7fea1c06a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 --2- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c06a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 -- 172.21.15.143:0/736557110 >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 msgr2=0x7fea1c06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 --2- 172.21.15.143:0/736557110 >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 0x7fea1c06e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1be00640 1 -- 172.21.15.143:0/736557110 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c161470 con 0x7fea1c15d130 2024-01-15T18:17:26.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.052+0000 7fea1b400640 1 --2- 172.21.15.143:0/736557110 >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 0x7fea1c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:26.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea1be00640 1 --2- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c0aeed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fea1c06b550 tx=0x7fea0c01c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d809f3558a8d311 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:26.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea1aa00640 1 -- 172.21.15.143:0/736557110 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0aa640 con 0x7fea1c15d130 2024-01-15T18:17:26.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea1aa00640 1 -- 172.21.15.143:0/736557110 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c0aa7a0 con 0x7fea1c15d130 2024-01-15T18:17:26.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea1aa00640 1 -- 172.21.15.143:0/736557110 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0aab30 con 0x7fea1c15d130 2024-01-15T18:17:26.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 -- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 msgr2=0x7fea1c0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:26.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 --2- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c0aeed0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fea1c06b550 tx=0x7fea0c01c500 comp rx=0 tx=0).stop 2024-01-15T18:17:26.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 -- 172.21.15.143:0/736557110 shutdown_connections 2024-01-15T18:17:26.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 --2- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c06a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 --2- 172.21.15.143:0/736557110 >> v2:172.21.15.143:3300/0 conn(0x7fea1c15d510 0x7fea1c06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 --2- 172.21.15.143:0/736557110 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c0aeed0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 -- 172.21.15.143:0/736557110 >> 172.21.15.143:0/736557110 conn(0x7fea1c0b1b90 msgr2=0x7fea1c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:26.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 -- 172.21.15.143:0/736557110 shutdown_connections 2024-01-15T18:17:26.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.056+0000 7fea22e00640 1 -- 172.21.15.143:0/736557110 wait complete. 2024-01-15T18:17:26.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 Processor -- start 2024-01-15T18:17:26.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 -- start start 2024-01-15T18:17:26.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c1479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 0x7fea1c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c1535a0 con 0x7fea1c15d510 2024-01-15T18:17:26.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c153810 con 0x7fea1c15d130 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c148de0 con 0x7fea1c06a490 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 0x7fea1c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 0x7fea1c151220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56422/0 (socket says 172.21.15.143:56422) 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 -- 172.21.15.143:0/3490641826 learned_addr learned my addr 172.21.15.143:0/3490641826 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea1b400640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea1be00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c1479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 -- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 msgr2=0x7fea1c1479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c1479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 -- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 msgr2=0x7fea1c150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 -- 172.21.15.143:0/3490641826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009420 con 0x7fea1c15d510 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea1be00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c1479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:26.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea20e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 0x7fea1c151220 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fea1400eed0 tx=0x7fea1400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14010040 con 0x7fea1c15d510 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1400bd20 con 0x7fea1c15d510 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c0094d0 con 0x7fea1c15d510 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14014640 con 0x7fea1c15d510 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.060+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c149360 con 0x7fea1c15d510 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.064+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c149580 con 0x7fea1c15d510 2024-01-15T18:17:26.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.064+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fea140147a0 con 0x7fea1c15d510 2024-01-15T18:17:26.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.064+0000 7fea13e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9f4067850 0x7fe9f4069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:26.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.064+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fea14083710 con 0x7fea1c15d510 2024-01-15T18:17:26.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.064+0000 7fea1be00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9f4067850 0x7fe9f4069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:26.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.064+0000 7fea1be00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9f4067850 0x7fe9f4069d10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0ae000 tx=0x7fea0c0b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:26.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.068+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fea14052450 con 0x7fea1c15d510 2024-01-15T18:17:26.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.228+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fea1c153980 con 0x7fea1c15d510 2024-01-15T18:17:26.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.900+0000 7fea13e00640 1 -- 172.21.15.143:0/3490641826 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v94) v1 ==== 97+0+0 (secure 0 0 0) 0x7fea14018080 con 0x7fea1c15d510 2024-01-15T18:17:26.898 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:17:26.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9f4067850 msgr2=0x7fe9f4069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:26.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9f4067850 0x7fe9f4069d10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0ae000 tx=0x7fea0c0b5040 comp rx=0 tx=0).stop 2024-01-15T18:17:26.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 msgr2=0x7fea1c151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 0x7fea1c151220 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fea1400eed0 tx=0x7fea1400b480 comp rx=0 tx=0).stop 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 shutdown_connections 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9f4067850 0x7fe9f4069d10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea1c15d510 0x7fea1c151220 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea1c15d130 0x7fea1c150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 --2- 172.21.15.143:0/3490641826 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fea1c06a490 0x7fea1c1479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 >> 172.21.15.143:0/3490641826 conn(0x7fea1c0b1b90 msgr2=0x7fea1c0b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 shutdown_connections 2024-01-15T18:17:26.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:26.904+0000 7fea22e00640 1 -- 172.21.15.143:0/3490641826 wait complete. 2024-01-15T18:17:26.927 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:17:27.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 Processor -- start 2024-01-15T18:17:27.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 -- start start 2024-01-15T18:17:27.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f885c15b260 0x7f885c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f885c0d19f0 0x7f885c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0c5220 0x7f885c0c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0c4d90 con 0x7f885c0d19f0 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c15b640 con 0x7f885c15b260 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f8862600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cf800 con 0x7f885c0c5220 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f885c0d19f0 0x7f885c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f885c0d19f0 0x7f885c0cdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56424/0 (socket says 172.21.15.143:56424) 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 -- 172.21.15.143:0/4159052948 learned_addr learned my addr 172.21.15.143:0/4159052948 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885b400640 1 --2- 172.21.15.143:0/4159052948 >> v2:172.21.15.204:3300/0 conn(0x7f885c15b260 0x7f885c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885be00640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0c5220 0x7f885c0c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 -- 172.21.15.143:0/4159052948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0c5220 msgr2=0x7f885c0c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0c5220 0x7f885c0c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 -- 172.21.15.143:0/4159052948 >> v2:172.21.15.204:3300/0 conn(0x7f885c15b260 msgr2=0x7f885c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 --2- 172.21.15.143:0/4159052948 >> v2:172.21.15.204:3300/0 conn(0x7f885c15b260 0x7f885c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885aa00640 1 -- 172.21.15.143:0/4159052948 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0cfae0 con 0x7f885c0d19f0 2024-01-15T18:17:27.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.068+0000 7f885be00640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0c5220 0x7f885c0c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:27.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f885aa00640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cdd50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f885c0681e0 tx=0x7f884401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dd73d5ba4b0973 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f885a000640 1 -- 172.21.15.143:0/4159052948 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440aa5f0 con 0x7f885c0d19f0 2024-01-15T18:17:27.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f885a000640 1 -- 172.21.15.143:0/4159052948 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88440aa750 con 0x7f885c0d19f0 2024-01-15T18:17:27.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f885a000640 1 -- 172.21.15.143:0/4159052948 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440aaae0 con 0x7f885c0d19f0 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 -- 172.21.15.143:0/4159052948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 msgr2=0x7f885c0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cdd50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f885c0681e0 tx=0x7f884401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 -- 172.21.15.143:0/4159052948 shutdown_connections 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c0c5220 0x7f885c0c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 --2- 172.21.15.143:0/4159052948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cdd50 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 --2- 172.21.15.143:0/4159052948 >> v2:172.21.15.204:3300/0 conn(0x7f885c15b260 0x7f885c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 -- 172.21.15.143:0/4159052948 >> 172.21.15.143:0/4159052948 conn(0x7f885c0d0ed0 msgr2=0x7f885c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 -- 172.21.15.143:0/4159052948 shutdown_connections 2024-01-15T18:17:27.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.072+0000 7f8862600640 1 -- 172.21.15.143:0/4159052948 wait complete. 2024-01-15T18:17:27.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 Processor -- start 2024-01-15T18:17:27.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 -- start start 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0c5220 0x7f885c0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c15b260 0x7f885c0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0c9d50 con 0x7f885c0d19f0 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0c9790 con 0x7f885c0c5220 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0c9a70 con 0x7f885c15b260 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56430/0 (socket says 172.21.15.143:56430) 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 -- 172.21.15.143:0/3816768829 learned_addr learned my addr 172.21.15.143:0/3816768829 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:27.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885b400640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0c5220 0x7f885c0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885be00640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c15b260 0x7f885c0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 -- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c15b260 msgr2=0x7f885c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c15b260 0x7f885c0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 -- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0c5220 msgr2=0x7f885c0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0c5220 0x7f885c0c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 -- 172.21.15.143:0/3816768829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440094d0 con 0x7f885c0d19f0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885b400640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0c5220 0x7f885c0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f885aa00640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cd430 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f88440ac040 tx=0x7f884401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440b3800 con 0x7f885c0d19f0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88440b3960 con 0x7f885c0d19f0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c0b3c00 con 0x7f885c0d19f0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88440b3d10 con 0x7f885c0d19f0 2024-01-15T18:17:27.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.076+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c0c8ef0 con 0x7f885c0d19f0 2024-01-15T18:17:27.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.080+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c0b3a60 con 0x7f885c0d19f0 2024-01-15T18:17:27.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.080+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f88440b2380 con 0x7f885c0d19f0 2024-01-15T18:17:27.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.080+0000 7f8853400640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8834067820 0x7f8834069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.080+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f88440a7070 con 0x7f885c0d19f0 2024-01-15T18:17:27.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.080+0000 7f885b400640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8834067820 0x7f8834069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.080+0000 7f885b400640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8834067820 0x7f8834069ce0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b9f0 tx=0x7f884c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.084+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f885c0b3a60 con 0x7f885c0d19f0 2024-01-15T18:17:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f885c0b3a60 con 0x7f885c0d19f0 2024-01-15T18:17:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8853400640 1 -- 172.21.15.143:0/3816768829 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f885c0b3a60 con 0x7f885c0d19f0 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.286 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:17:27.288 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:17:27.289 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.290 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.291 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.292 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:17:27.293 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.294 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:17:27.295 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.296 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:17:27.297 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:17:27.298 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:27.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8834067820 msgr2=0x7f8834069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8834067820 0x7f8834069ce0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f884c00b9f0 tx=0x7f884c008040 comp rx=0 tx=0).stop 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 msgr2=0x7f885c0cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cd430 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f88440ac040 tx=0x7f884401ad10 comp rx=0 tx=0).stop 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 shutdown_connections 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8834067820 0x7f8834069ce0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f885c15b260 0x7f885c0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f885c0d19f0 0x7f885c0cd430 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 --2- 172.21.15.143:0/3816768829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f885c0c5220 0x7f885c0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 >> 172.21.15.143:0/3816768829 conn(0x7f885c0d0ed0 msgr2=0x7f885c15a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 shutdown_connections 2024-01-15T18:17:27.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.288+0000 7f8862600640 1 -- 172.21.15.143:0/3816768829 wait complete. 2024-01-15T18:17:27.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff198200640 1 Processor -- start 2024-01-15T18:17:27.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff198200640 1 -- start start 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff198200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff1900cdf50 0x7ff1900cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff198200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1900b0b30 con 0x7ff19015b2d0 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff198200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1900cf960 con 0x7ff1900cf410 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff198200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1900cfc40 con 0x7ff1900cdf50 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff195800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff19015b2d0 0x7ff1900cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41876/0 (socket says 172.21.15.143:41876) 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff195800640 1 -- 172.21.15.143:0/819606268 learned_addr learned my addr 172.21.15.143:0/819606268 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599604871 0 0) 0x7ff1900b0b30 con 0x7ff19015b2d0 2024-01-15T18:17:27.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170003530 con 0x7ff19015b2d0 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff194e00640 1 --2- 172.21.15.143:0/819606268 >> v2:172.21.15.204:3300/0 conn(0x7ff1900cdf50 0x7ff1900cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011825890 0 0) 0x7ff170003530 con 0x7ff19015b2d0 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 >> v2:172.21.15.204:3300/0 conn(0x7ff1900cdf50 msgr2=0x7ff1900cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 --2- 172.21.15.143:0/819606268 >> v2:172.21.15.204:3300/0 conn(0x7ff1900cdf50 0x7ff1900cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 >> v1:172.21.15.204:6789/0 conn(0x7ff1900cf410 legacy=0x7ff1900cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900b07c0 con 0x7ff19015b2d0 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff180001800 con 0x7ff19015b2d0 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.448+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 >> v1:172.21.15.143:6789/0 conn(0x7ff19015b2d0 legacy=0x7ff1900cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170003980 0x7ff170003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1700047d0 0x7ff170004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff170005190 0x7ff17000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900cfc40 con 0x7ff170005190 2024-01-15T18:17:27.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000c030 con 0x7ff170003980 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17000c1f0 con 0x7ff1700047d0 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff170005190 0x7ff17000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff195800640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170003980 0x7ff170003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff194e00640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1700047d0 0x7ff170004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 -- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1700047d0 msgr2=0x7ff170004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1700047d0 0x7ff170004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 -- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170003980 msgr2=0x7ff170003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170003980 0x7ff170003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 -- 172.21.15.143:0/819606268 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170003530 con 0x7ff170005190 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 -- 172.21.15.143:0/819606268 reap_dead start 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff196200640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff170005190 0x7ff17000b990 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff188001e70 tx=0x7ff18801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563ff4bc28ba930f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188003b80 con 0x7ff170005190 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188003d20 con 0x7ff170005190 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.452+0000 7ff18fe00640 1 -- 172.21.15.143:0/819606268 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188005260 con 0x7ff170005190 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- 172.21.15.143:0/819606268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff170005190 msgr2=0x7ff17000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff170005190 0x7ff17000b990 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff188001e70 tx=0x7ff18801c760 comp rx=0 tx=0).stop 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- 172.21.15.143:0/819606268 shutdown_connections 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 --2- 172.21.15.143:0/819606268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff170005190 0x7ff17000b990 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- 172.21.15.143:0/819606268 >> 172.21.15.143:0/819606268 conn(0x7ff1900d0e30 msgr2=0x7ff1900d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- 172.21.15.143:0/819606268 shutdown_connections 2024-01-15T18:17:27.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- 172.21.15.143:0/819606268 wait complete. 2024-01-15T18:17:27.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 Processor -- start 2024-01-15T18:17:27.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- start start 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170005190 0x7ff1900b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1900c4960 0x7ff190156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 0x7ff1900ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900ae410 con 0x7ff1900ca270 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1900b4960 con 0x7ff170005190 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff198200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1901572f0 con 0x7ff1900c4960 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff196200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 0x7ff1900ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff195800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170005190 0x7ff1900b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff196200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 0x7ff1900ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39958/0 (socket says 172.21.15.143:39958) 2024-01-15T18:17:27.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff196200640 1 -- 172.21.15.143:0/3637833236 learned_addr learned my addr 172.21.15.143:0/3637833236 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.456+0000 7ff194e00640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1900c4960 0x7ff190156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff196200640 1 -- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1900c4960 msgr2=0x7ff190156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff196200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1900c4960 0x7ff190156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff196200640 1 -- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170005190 msgr2=0x7ff1900b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff196200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170005190 0x7ff1900b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff196200640 1 -- 172.21.15.143:0/3637833236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1900b4680 con 0x7ff1900ca270 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff194e00640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1900c4960 0x7ff190156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff196200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 0x7ff1900ca6f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff18801cc40 tx=0x7ff170005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1880ac070 con 0x7ff1900ca270 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff188003e90 con 0x7ff1900ca270 2024-01-15T18:17:27.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1900add50 con 0x7ff1900ca270 2024-01-15T18:17:27.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188006d60 con 0x7ff1900ca270 2024-01-15T18:17:27.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1900cac30 con 0x7ff1900ca270 2024-01-15T18:17:27.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1900ae550 con 0x7ff1900ca270 2024-01-15T18:17:27.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.460+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7ff188007030 con 0x7ff1900ca270 2024-01-15T18:17:27.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.464+0000 7ff18d600640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff168067820 0x7ff168069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.464+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4913+0+0 (secure 0 0 0) 0x7ff18811c8d0 con 0x7ff1900ca270 2024-01-15T18:17:27.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.464+0000 7ff195800640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff168067820 0x7ff168069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.464+0000 7ff195800640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff168067820 0x7ff168069ce0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f00 tx=0x7ff180009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.468+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff1900ae550 con 0x7ff1900ca270 2024-01-15T18:17:27.618 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.620+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff190067500 con 0x7ff1900ca270 2024-01-15T18:17:27.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.620+0000 7ff18d600640 1 -- 172.21.15.143:0/3637833236 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+12982 (secure 0 0 0) 0x7ff1880eb1b0 con 0x7ff1900ca270 2024-01-15T18:17:27.619 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:27.620 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":96,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:26.914750+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff168067820 msgr2=0x7ff168069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff168067820 0x7ff168069ce0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff180000f00 tx=0x7ff180009040 comp rx=0 tx=0).stop 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 msgr2=0x7ff1900ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 0x7ff1900ca6f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff18801cc40 tx=0x7ff170005800 comp rx=0 tx=0).stop 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 shutdown_connections 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff168067820 0x7ff168069ce0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1900ca270 0x7ff1900ca6f0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff1900c4960 0x7ff190156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 --2- 172.21.15.143:0/3637833236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff170005190 0x7ff1900b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 >> 172.21.15.143:0/3637833236 conn(0x7ff1900d0e30 msgr2=0x7ff19006fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 shutdown_connections 2024-01-15T18:17:27.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.624+0000 7ff198200640 1 -- 172.21.15.143:0/3637833236 wait complete. 2024-01-15T18:17:27.642 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:27.643 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:27.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.784+0000 7f8709200640 1 Processor -- start 2024-01-15T18:17:27.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.784+0000 7f8709200640 1 -- start start 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.784+0000 7f8709200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87041607c0 0x7f870406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.784+0000 7f8709200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87040cffd0 con 0x7f87040cfb30 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8709200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87040d02b0 con 0x7f87040cbf50 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8709200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87040d1ce0 con 0x7f87041607c0 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f87041607c0 0x7f870406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f87040cfb30 0x7f870406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41890/0 (socket says 172.21.15.143:41890) 2024-01-15T18:17:27.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701400640 1 -- 172.21.15.143:0/3028368347 learned_addr learned my addr 172.21.15.143:0/3028368347 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:27.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701e00640 1 -- 172.21.15.143:0/3028368347 >> v1:172.21.15.204:6789/0 conn(0x7f87040cbf50 legacy=0x7f87040cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701e00640 1 -- 172.21.15.143:0/3028368347 >> v1:172.21.15.143:6789/0 conn(0x7f87040cfb30 legacy=0x7f870406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701e00640 1 -- 172.21.15.143:0/3028368347 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87040cb470 con 0x7f87041607c0 2024-01-15T18:17:27.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8701e00640 1 --2- 172.21.15.143:0/3028368347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87041607c0 0x7f870406b7b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f86f40097c0 tx=0x7f86f401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df9653e88e74415 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8700a00640 1 -- 172.21.15.143:0/3028368347 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40ab600 con 0x7f87041607c0 2024-01-15T18:17:27.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8700a00640 1 -- 172.21.15.143:0/3028368347 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f40a7050 con 0x7f87041607c0 2024-01-15T18:17:27.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.788+0000 7f8700a00640 1 -- 172.21.15.143:0/3028368347 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40ab980 con 0x7f87041607c0 2024-01-15T18:17:27.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- 172.21.15.143:0/3028368347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87041607c0 msgr2=0x7f870406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 --2- 172.21.15.143:0/3028368347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87041607c0 0x7f870406b7b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f86f40097c0 tx=0x7f86f401c3a0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- 172.21.15.143:0/3028368347 shutdown_connections 2024-01-15T18:17:27.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 --2- 172.21.15.143:0/3028368347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87041607c0 0x7f870406b7b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- 172.21.15.143:0/3028368347 >> 172.21.15.143:0/3028368347 conn(0x7f87040d0e20 msgr2=0x7f87040d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- 172.21.15.143:0/3028368347 shutdown_connections 2024-01-15T18:17:27.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- 172.21.15.143:0/3028368347 wait complete. 2024-01-15T18:17:27.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 Processor -- start 2024-01-15T18:17:27.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- start start 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87040cbf50 0x7f870409c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 0x7f870415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87041607c0 0x7f87041f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870409b7d0 con 0x7f87040cfb30 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87041f6f80 con 0x7f87041607c0 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.792+0000 7f8709200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87041f7260 con 0x7f87040cbf50 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 0x7f870415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 0x7f870415a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39980/0 (socket says 172.21.15.143:39980) 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 -- 172.21.15.143:0/8851863 learned_addr learned my addr 172.21.15.143:0/8851863 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8702800640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87041607c0 0x7f87041f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701e00640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87040cbf50 0x7f870409c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 -- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87040cbf50 msgr2=0x7f870409c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87040cbf50 0x7f870409c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 -- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87041607c0 msgr2=0x7f87041f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87041607c0 0x7f87041f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 -- 172.21.15.143:0/8851863 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4009420 con 0x7f87040cfb30 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8701400640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 0x7f870415a340 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009f30 tx=0x7f86ec00c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec007af0 con 0x7f87040cfb30 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ec007c50 con 0x7f87040cfb30 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00ce70 con 0x7f87040cfb30 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870409b9e0 con 0x7f87040cfb30 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870409b510 con 0x7f87040cfb30 2024-01-15T18:17:27.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870406d4b0 con 0x7f87040cfb30 2024-01-15T18:17:27.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.796+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f86ec009070 con 0x7f87040cfb30 2024-01-15T18:17:27.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.800+0000 7f86fa000640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86dc067820 0x7f86dc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.800+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f86ec084e50 con 0x7f87040cfb30 2024-01-15T18:17:27.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.800+0000 7f8701e00640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86dc067820 0x7f86dc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.800+0000 7f8701e00640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86dc067820 0x7f86dc069ce0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f870409a980 tx=0x7f86f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.804+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f86ec053bf0 con 0x7f87040cfb30 2024-01-15T18:17:27.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8704067500 con 0x7f87040cfb30 2024-01-15T18:17:27.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f86fa000640 1 -- 172.21.15.143:0/8851863 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86ec019020 con 0x7f87040cfb30 2024-01-15T18:17:27.958 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:17:27.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86dc067820 msgr2=0x7f86dc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86dc067820 0x7f86dc069ce0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f870409a980 tx=0x7f86f4008040 comp rx=0 tx=0).stop 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 msgr2=0x7f870415a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 0x7f870415a340 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f86ec009f30 tx=0x7f86ec00c4f0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 shutdown_connections 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f86dc067820 0x7f86dc069ce0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f87041607c0 0x7f87041f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87040cfb30 0x7f870415a340 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 --2- 172.21.15.143:0/8851863 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f87040cbf50 0x7f870409c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 >> 172.21.15.143:0/8851863 conn(0x7f87040d0e20 msgr2=0x7f870406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 shutdown_connections 2024-01-15T18:17:27.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:27.960+0000 7f8709200640 1 -- 172.21.15.143:0/8851863 wait complete. 2024-01-15T18:17:27.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:17:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.116+0000 7f51dbe00640 1 Processor -- start 2024-01-15T18:17:28.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.116+0000 7f51dbe00640 1 -- start start 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.116+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.116+0000 7f51dbe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d40ceb50 con 0x7f51d4071bb0 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.116+0000 7f51dbe00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d415c110 con 0x7f51d4143780 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.116+0000 7f51dbe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40724c0 con 0x7f51d4072030 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d8a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f51d4071bb0 0x7f51d415c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41904/0 (socket says 172.21.15.143:41904) 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d8a00640 1 -- 172.21.15.143:0/4191164439 learned_addr learned my addr 172.21.15.143:0/4191164439 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:28.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d9e00640 1 --2- 172.21.15.143:0/4191164439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261044210 0 0) 0x7f51bc001120 con 0x7f51d4071bb0 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b4005740 con 0x7f51d4071bb0 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764018737 0 0) 0x7f51d415c110 con 0x7f51d4143780 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51bc001120 con 0x7f51d4143780 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d9e00640 1 -- 172.21.15.143:0/4191164439 >> v1:172.21.15.204:6789/0 conn(0x7f51d4143780 legacy=0x7f51d40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d9e00640 1 -- 172.21.15.143:0/4191164439 >> v1:172.21.15.143:6789/0 conn(0x7f51d4071bb0 legacy=0x7f51d415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4235540612 0 0) 0x7f51b4005740 con 0x7f51d4071bb0 2024-01-15T18:17:28.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d9e00640 1 -- 172.21.15.143:0/4191164439 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d415b7e0 con 0x7f51d4072030 2024-01-15T18:17:28.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.120+0000 7f51d9e00640 1 --2- 172.21.15.143:0/4191164439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d415ccb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009dc0 tx=0x7f51cc01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6dfc22a9d42e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:28.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc001830 con 0x7f51d4072030 2024-01-15T18:17:28.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0a56d0 con 0x7f51d4072030 2024-01-15T18:17:28.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51d3e00640 1 -- 172.21.15.143:0/4191164439 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0a5a60 con 0x7f51d4072030 2024-01-15T18:17:28.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 -- 172.21.15.143:0/4191164439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 msgr2=0x7f51d415ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 --2- 172.21.15.143:0/4191164439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d415ccb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009dc0 tx=0x7f51cc01c890 comp rx=0 tx=0).stop 2024-01-15T18:17:28.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 -- 172.21.15.143:0/4191164439 shutdown_connections 2024-01-15T18:17:28.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 --2- 172.21.15.143:0/4191164439 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d415ccb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 -- 172.21.15.143:0/4191164439 >> 172.21.15.143:0/4191164439 conn(0x7f51d406c3f0 msgr2=0x7f51d406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:28.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 -- 172.21.15.143:0/4191164439 shutdown_connections 2024-01-15T18:17:28.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 -- 172.21.15.143:0/4191164439 wait complete. 2024-01-15T18:17:28.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 Processor -- start 2024-01-15T18:17:28.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.124+0000 7f51dbe00640 1 -- start start 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d4071bb0 0x7f51d4158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d4135560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 0x7f51d40b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d4137370 con 0x7f51d4143780 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d4137650 con 0x7f51d4071bb0 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d4137930 con 0x7f51d4072030 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 0x7f51d40b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 0x7f51d40b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39986/0 (socket says 172.21.15.143:39986) 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 -- 172.21.15.143:0/2701788062 learned_addr learned my addr 172.21.15.143:0/2701788062 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d8a00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d4135560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9400640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d4071bb0 0x7f51d4158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 -- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 msgr2=0x7f51d4135560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d4135560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 -- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d4071bb0 msgr2=0x7f51d4158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d4071bb0 0x7f51d4158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 -- 172.21.15.143:0/2701788062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc009a70 con 0x7f51d4143780 2024-01-15T18:17:28.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d9e00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 0x7f51d40b2f90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f51cc01cd70 tx=0x7f51cc019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0b9860 con 0x7f51d4143780 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0b99c0 con 0x7f51d4143780 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0b9d30 con 0x7f51d4143780 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d4137c10 con 0x7f51d4143780 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d4159340 con 0x7f51d4143780 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d41594d0 con 0x7f51d4143780 2024-01-15T18:17:28.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.128+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f51cc0ad440 con 0x7f51d4143780 2024-01-15T18:17:28.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.132+0000 7f51d1600640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51ac067820 0x7f51ac069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.132+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f51cc11df50 con 0x7f51d4143780 2024-01-15T18:17:28.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.132+0000 7f51d9400640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51ac067820 0x7f51ac069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.132+0000 7f51d9400640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51ac067820 0x7f51ac069ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f51c40011f0 tx=0x7f51c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:28.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.136+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f51cc0eccf0 con 0x7f51d4143780 2024-01-15T18:17:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.284+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f51d4067500 con 0x7f51d4143780 2024-01-15T18:17:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51d1600640 1 -- 172.21.15.143:0/2701788062 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v96) v1 ==== 84+0+65 (secure 0 0 0) 0x7f51cc0f0310 con 0x7f51d4143780 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:28.285 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:17:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51ac067820 msgr2=0x7f51ac069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51ac067820 0x7f51ac069ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f51c40011f0 tx=0x7f51c4009040 comp rx=0 tx=0).stop 2024-01-15T18:17:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 msgr2=0x7f51d40b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 0x7f51d40b2f90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f51cc01cd70 tx=0x7f51cc019630 comp rx=0 tx=0).stop 2024-01-15T18:17:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 shutdown_connections 2024-01-15T18:17:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f51ac067820 0x7f51ac069ce0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51d4143780 0x7f51d40b2f90 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f51d4072030 0x7f51d4135560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 --2- 172.21.15.143:0/2701788062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f51d4071bb0 0x7f51d4158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 >> 172.21.15.143:0/2701788062 conn(0x7f51d406c3f0 msgr2=0x7f51d40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 shutdown_connections 2024-01-15T18:17:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.288+0000 7f51dbe00640 1 -- 172.21.15.143:0/2701788062 wait complete. 2024-01-15T18:17:28.307 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:17:28.308 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:17:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.448+0000 7fbd25800640 1 Processor -- start 2024-01-15T18:17:28.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.448+0000 7fbd25800640 1 -- start start 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd25800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbd200ce690 0x7fbd200ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd25800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd200b1da0 0x7fbd200b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd25800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200b3bf0 0x7fbd200b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd25800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2015dba0 con 0x7fbd200b1da0 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd25800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd20070bf0 con 0x7fbd200ce690 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd25800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200b2770 con 0x7fbd200b3bf0 2024-01-15T18:17:28.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd200b1da0 0x7fbd200b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbd200b1da0 0x7fbd200b2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39992/0 (socket says 172.21.15.143:39992) 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 -- 172.21.15.143:0/2515789432 learned_addr learned my addr 172.21.15.143:0/2515789432 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1f400640 1 --2- 172.21.15.143:0/2515789432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200b3bf0 0x7fbd200b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1ea00640 1 --2- 172.21.15.143:0/2515789432 >> v2:172.21.15.204:3300/0 conn(0x7fbd200ce690 0x7fbd200ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 -- 172.21.15.143:0/2515789432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200b3bf0 msgr2=0x7fbd200b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 --2- 172.21.15.143:0/2515789432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200b3bf0 0x7fbd200b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 -- 172.21.15.143:0/2515789432 >> v2:172.21.15.204:3300/0 conn(0x7fbd200ce690 msgr2=0x7fbd200ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 --2- 172.21.15.143:0/2515789432 >> v2:172.21.15.204:3300/0 conn(0x7fbd200ce690 0x7fbd200ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.452+0000 7fbd1e000640 1 -- 172.21.15.143:0/2515789432 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd200b4920 con 0x7fbd200b1da0 2024-01-15T18:17:28.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd1e000640 1 --2- 172.21.15.143:0/2515789432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b1da0 0x7fbd200b2230 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbd200b1aa0 tx=0x7fbd0801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c08bf531471b5f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:28.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd1d600640 1 -- 172.21.15.143:0/2515789432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080aa5f0 con 0x7fbd200b1da0 2024-01-15T18:17:28.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd1d600640 1 -- 172.21.15.143:0/2515789432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd080aa750 con 0x7fbd200b1da0 2024-01-15T18:17:28.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd1d600640 1 -- 172.21.15.143:0/2515789432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080aaae0 con 0x7fbd200b1da0 2024-01-15T18:17:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 -- 172.21.15.143:0/2515789432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b1da0 msgr2=0x7fbd200b2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 --2- 172.21.15.143:0/2515789432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b1da0 0x7fbd200b2230 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbd200b1aa0 tx=0x7fbd0801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 -- 172.21.15.143:0/2515789432 shutdown_connections 2024-01-15T18:17:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 --2- 172.21.15.143:0/2515789432 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200b3bf0 0x7fbd200b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 --2- 172.21.15.143:0/2515789432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b1da0 0x7fbd200b2230 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 --2- 172.21.15.143:0/2515789432 >> v2:172.21.15.204:3300/0 conn(0x7fbd200ce690 0x7fbd200ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 -- 172.21.15.143:0/2515789432 >> 172.21.15.143:0/2515789432 conn(0x7fbd2006c3f0 msgr2=0x7fbd2006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 -- 172.21.15.143:0/2515789432 shutdown_connections 2024-01-15T18:17:28.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 -- 172.21.15.143:0/2515789432 wait complete. 2024-01-15T18:17:28.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 Processor -- start 2024-01-15T18:17:28.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.456+0000 7fbd25800640 1 -- start start 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200b1da0 0x7fbd201579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 0x7fbd200ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200ce690 0x7fbd2014a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201584b0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20158750 con 0x7fbd200b1da0 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200add40 con 0x7fbd200ce690 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 0x7fbd200ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 0x7fbd200ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40004/0 (socket says 172.21.15.143:40004) 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 -- 172.21.15.143:0/1686132036 learned_addr learned my addr 172.21.15.143:0/1686132036 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1ea00640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200b1da0 0x7fbd201579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1f400640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200ce690 0x7fbd2014a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 -- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200ce690 msgr2=0x7fbd2014a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200ce690 0x7fbd2014a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 -- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200b1da0 msgr2=0x7fbd201579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200b1da0 0x7fbd201579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 -- 172.21.15.143:0/1686132036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd080094d0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd1e000640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 0x7fbd200ad800 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ac040 tx=0x7fbd0801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:28.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080aa5f0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd080b3820 con 0x7fbd200b3bf0 2024-01-15T18:17:28.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd080b3bd0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd200ace60 con 0x7fbd200b3bf0 2024-01-15T18:17:28.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd200ad400 con 0x7fbd200b3bf0 2024-01-15T18:17:28.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd201588e0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.460+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fbd080b3d30 con 0x7fbd200b3bf0 2024-01-15T18:17:28.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.464+0000 7fbd16a00640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf8067820 0x7fbcf8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:28.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.464+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fbd080a7070 con 0x7fbd200b3bf0 2024-01-15T18:17:28.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.464+0000 7fbd1ea00640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf8067820 0x7fbcf8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:28.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.464+0000 7fbd1ea00640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf8067820 0x7fbcf8069ce0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbd100079c0 tx=0x7fbd10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:28.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.468+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbd080ec200 con 0x7fbd200b3bf0 2024-01-15T18:17:28.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.616+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbd200ad5f0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.908+0000 7fbd16a00640 1 -- 172.21.15.143:0/1686132036 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v97) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd200ad5f0 con 0x7fbd200b3bf0 2024-01-15T18:17:28.908 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:17:28.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf8067820 msgr2=0x7fbcf8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf8067820 0x7fbcf8069ce0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbd100079c0 tx=0x7fbd10008040 comp rx=0 tx=0).stop 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 msgr2=0x7fbd200ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 0x7fbd200ad800 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ac040 tx=0x7fbd0801ad10 comp rx=0 tx=0).stop 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 shutdown_connections 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbcf8067820 0x7fbcf8069ce0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbd200ce690 0x7fbd2014a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbd200b3bf0 0x7fbd200ad800 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 --2- 172.21.15.143:0/1686132036 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd200b1da0 0x7fbd201579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 >> 172.21.15.143:0/1686132036 conn(0x7fbd2006c3f0 msgr2=0x7fbd200b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 shutdown_connections 2024-01-15T18:17:28.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:28.912+0000 7fbd25800640 1 -- 172.21.15.143:0/1686132036 wait complete. 2024-01-15T18:17:28.938 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:17:29.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 Processor -- start 2024-01-15T18:17:29.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 -- start start 2024-01-15T18:17:29.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:29.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f099c06bf50 0x7f099c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:29.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099c06c490 con 0x7f099c15d4f0 2024-01-15T18:17:29.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c0b0e80 con 0x7f099c06bf50 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a2e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c06ca50 con 0x7f099c15d110 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f099c06bf50 0x7f099c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f099c06bf50 0x7f099c0b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40008/0 (socket says 172.21.15.143:40008) 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 -- 172.21.15.143:0/2787350738 learned_addr learned my addr 172.21.15.143:0/2787350738 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f099be00640 1 --2- 172.21.15.143:0/2787350738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 -- 172.21.15.143:0/2787350738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 msgr2=0x7f099c0cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:29.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 --2- 172.21.15.143:0/2787350738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c0cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 -- 172.21.15.143:0/2787350738 >> v1:172.21.15.204:6789/0 conn(0x7f099c15d4f0 legacy=0x7f099c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:29.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.084+0000 7f09a0e00640 1 -- 172.21.15.143:0/2787350738 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c06c770 con 0x7f099c06bf50 2024-01-15T18:17:29.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a0e00640 1 --2- 172.21.15.143:0/2787350738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0b12a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0994009e50 tx=0x7f099401d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32839efcf468233f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:29.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f099aa00640 1 -- 172.21.15.143:0/2787350738 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940a5750 con 0x7f099c06bf50 2024-01-15T18:17:29.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f099aa00640 1 -- 172.21.15.143:0/2787350738 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09940a58b0 con 0x7f099c06bf50 2024-01-15T18:17:29.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f099aa00640 1 -- 172.21.15.143:0/2787350738 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940a5c40 con 0x7f099c06bf50 2024-01-15T18:17:29.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 -- 172.21.15.143:0/2787350738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 msgr2=0x7f099c0b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:29.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 --2- 172.21.15.143:0/2787350738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0b12a0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f0994009e50 tx=0x7f099401d3e0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 -- 172.21.15.143:0/2787350738 shutdown_connections 2024-01-15T18:17:29.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 --2- 172.21.15.143:0/2787350738 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0b12a0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 --2- 172.21.15.143:0/2787350738 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c0cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 -- 172.21.15.143:0/2787350738 >> 172.21.15.143:0/2787350738 conn(0x7f099c0b2a40 msgr2=0x7f099c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:29.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 -- 172.21.15.143:0/2787350738 shutdown_connections 2024-01-15T18:17:29.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.088+0000 7f09a2e00640 1 -- 172.21.15.143:0/2787350738 wait complete. 2024-01-15T18:17:29.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 Processor -- start 2024-01-15T18:17:29.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 -- start start 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c14b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099c15d4f0 0x7f099c14bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0c74e0 con 0x7f099c06bf50 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0c5500 con 0x7f099c15d4f0 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a2e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0c6f20 con 0x7f099c15d110 2024-01-15T18:17:29.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40018/0 (socket says 172.21.15.143:40018) 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 -- 172.21.15.143:0/1886734332 learned_addr learned my addr 172.21.15.143:0/1886734332 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f09a0e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099c15d4f0 0x7f099c14bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099b400640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c14b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 -- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 msgr2=0x7f099c14b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c14b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 -- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099c15d4f0 msgr2=0x7f099c14bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099c15d4f0 0x7f099c14bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 -- 172.21.15.143:0/1886734332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009b30 con 0x7f099c06bf50 2024-01-15T18:17:29.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f099be00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0c5cc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f098c00e870 tx=0x7f098c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:29.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.092+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c009510 con 0x7f099c06bf50 2024-01-15T18:17:29.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c00eec0 con 0x7f099c06bf50 2024-01-15T18:17:29.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c0c7200 con 0x7f099c06bf50 2024-01-15T18:17:29.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c00cb40 con 0x7f099c06bf50 2024-01-15T18:17:29.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c0c6200 con 0x7f099c06bf50 2024-01-15T18:17:29.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c067500 con 0x7f099c06bf50 2024-01-15T18:17:29.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f098c00cca0 con 0x7f099c06bf50 2024-01-15T18:17:29.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f0993e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0974067820 0x7f0974069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:29.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f098c0837a0 con 0x7f099c06bf50 2024-01-15T18:17:29.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.096+0000 7f099b400640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0974067820 0x7f0974069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:29.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.100+0000 7f099b400640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0974067820 0x7f0974069ce0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f099c06f5d0 tx=0x7f0984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:29.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.100+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f098c0524e0 con 0x7f099c06bf50 2024-01-15T18:17:29.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.252+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f099c066760 con 0x7f099c06bf50 2024-01-15T18:17:29.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:29.747 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:29.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.912+0000 7f0993e00640 1 -- 172.21.15.143:0/1886734332 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v98) v1 ==== 170+0+0 (secure 0 0 0) 0x7f098c0170d0 con 0x7f099c06bf50 2024-01-15T18:17:29.912 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:17:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0974067820 msgr2=0x7f0974069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0974067820 0x7f0974069ce0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f099c06f5d0 tx=0x7f0984009040 comp rx=0 tx=0).stop 2024-01-15T18:17:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 msgr2=0x7f099c0c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:29.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0c5cc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f098c00e870 tx=0x7f098c00ec70 comp rx=0 tx=0).stop 2024-01-15T18:17:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 shutdown_connections 2024-01-15T18:17:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0974067820 0x7f0974069ce0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f099c15d4f0 0x7f099c14bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f099c15d110 0x7f099c14b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 --2- 172.21.15.143:0/1886734332 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f099c06bf50 0x7f099c0c5cc0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 >> 172.21.15.143:0/1886734332 conn(0x7f099c0b2a40 msgr2=0x7f099c0afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 shutdown_connections 2024-01-15T18:17:29.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:29.916+0000 7f09a2e00640 1 -- 172.21.15.143:0/1886734332 wait complete. 2024-01-15T18:17:29.960 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:17:30.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 Processor -- start 2024-01-15T18:17:30.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 -- start start 2024-01-15T18:17:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc15806bfb0 0x7fc15806c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc1580b0310 0x7fc1580cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1580cc9c0 0x7fc1580cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580cd3b0 con 0x7fc15806bfb0 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1580cbff0 con 0x7fc1580b0310 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc15e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580d0c50 con 0x7fc1580cc9c0 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc15806bfb0 0x7fc15806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc15806bfb0 0x7fc15806c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40022/0 (socket says 172.21.15.143:40022) 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 -- 172.21.15.143:0/3473436948 learned_addr learned my addr 172.21.15.143:0/3473436948 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc156a00640 1 --2- 172.21.15.143:0/3473436948 >> v2:172.21.15.204:3300/0 conn(0x7fc1580b0310 0x7fc1580cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157e00640 1 --2- 172.21.15.143:0/3473436948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1580cc9c0 0x7fc1580cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 -- 172.21.15.143:0/3473436948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1580cc9c0 msgr2=0x7fc1580cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 --2- 172.21.15.143:0/3473436948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1580cc9c0 0x7fc1580cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 -- 172.21.15.143:0/3473436948 >> v2:172.21.15.204:3300/0 conn(0x7fc1580b0310 msgr2=0x7fc1580cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 --2- 172.21.15.143:0/3473436948 >> v2:172.21.15.204:3300/0 conn(0x7fc1580b0310 0x7fc1580cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc157400640 1 -- 172.21.15.143:0/3473436948 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15806c8d0 con 0x7fc15806bfb0 2024-01-15T18:17:30.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.104+0000 7fc156a00640 1 --2- 172.21.15.143:0/3473436948 >> v2:172.21.15.204:3300/0 conn(0x7fc1580b0310 0x7fc1580cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc157400640 1 --2- 172.21.15.143:0/3473436948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15806bfb0 0x7fc15806c390 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc15806bf00 tx=0x7fc14801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295bd483f24ba7e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc156000640 1 -- 172.21.15.143:0/3473436948 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480ab600 con 0x7fc15806bfb0 2024-01-15T18:17:30.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc156000640 1 -- 172.21.15.143:0/3473436948 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1480ab760 con 0x7fc15806bfb0 2024-01-15T18:17:30.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc156000640 1 -- 172.21.15.143:0/3473436948 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480abaf0 con 0x7fc15806bfb0 2024-01-15T18:17:30.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 -- 172.21.15.143:0/3473436948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15806bfb0 msgr2=0x7fc15806c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 --2- 172.21.15.143:0/3473436948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15806bfb0 0x7fc15806c390 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc15806bf00 tx=0x7fc14801c440 comp rx=0 tx=0).stop 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 -- 172.21.15.143:0/3473436948 shutdown_connections 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 --2- 172.21.15.143:0/3473436948 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1580cc9c0 0x7fc1580cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 --2- 172.21.15.143:0/3473436948 >> v2:172.21.15.204:3300/0 conn(0x7fc1580b0310 0x7fc1580cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 --2- 172.21.15.143:0/3473436948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc15806bfb0 0x7fc15806c390 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.108+0000 7fc15e200640 1 -- 172.21.15.143:0/3473436948 >> 172.21.15.143:0/3473436948 conn(0x7fc1580d0ed0 msgr2=0x7fc1580d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 -- 172.21.15.143:0/3473436948 shutdown_connections 2024-01-15T18:17:30.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 -- 172.21.15.143:0/3473436948 wait complete. 2024-01-15T18:17:30.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 Processor -- start 2024-01-15T18:17:30.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 -- start start 2024-01-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15806bfb0 0x7fc1580ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1580b0310 0x7fc1580c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 0x7fc158154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158156fa0 con 0x7fc1580cc9c0 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc158155a90 con 0x7fc1580b0310 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc15e200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc158155d70 con 0x7fc15806bfb0 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 0x7fc158154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 0x7fc158154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40024/0 (socket says 172.21.15.143:40024) 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157e00640 1 -- 172.21.15.143:0/1372668034 learned_addr learned my addr 172.21.15.143:0/1372668034 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc156a00640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1580b0310 0x7fc1580c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15806bfb0 0x7fc1580ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 -- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1580b0310 msgr2=0x7fc1580c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1580b0310 0x7fc1580c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 -- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 msgr2=0x7fc158154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 0x7fc158154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 -- 172.21.15.143:0/1372668034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480094d0 con 0x7fc15806bfb0 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc156a00640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1580b0310 0x7fc1580c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157e00640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 0x7fc158154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:30.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.112+0000 7fc157400640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15806bfb0 0x7fc1580ca7b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc1480ac000 tx=0x7fc14801ac80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:30.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480b4800 con 0x7fc15806bfb0 2024-01-15T18:17:30.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1480b4960 con 0x7fc15806bfb0 2024-01-15T18:17:30.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc158156050 con 0x7fc15806bfb0 2024-01-15T18:17:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480b4d10 con 0x7fc15806bfb0 2024-01-15T18:17:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc158156610 con 0x7fc15806bfb0 2024-01-15T18:17:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc158067500 con 0x7fc15806bfb0 2024-01-15T18:17:30.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fc1480c83a0 con 0x7fc15806bfb0 2024-01-15T18:17:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc14f400640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc130067820 0x7fc130069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5665+0+0 (secure 0 0 0) 0x7fc14811d7c0 con 0x7fc15806bfb0 2024-01-15T18:17:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc156a00640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc130067820 0x7fc130069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.116+0000 7fc156a00640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc130067820 0x7fc130069ce0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc1580cb250 tx=0x7fc140008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:30.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.120+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1480ec4c0 con 0x7fc15806bfb0 2024-01-15T18:17:30.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.304+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc158155eb0 con 0x7fc15806bfb0 2024-01-15T18:17:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.932+0000 7fc14f400640 1 -- 172.21.15.143:0/1372668034 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc158155eb0 con 0x7fc15806bfb0 2024-01-15T18:17:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 14 and data pool 15 2024-01-15T18:17:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc130067820 msgr2=0x7fc130069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc130067820 0x7fc130069ce0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc1580cb250 tx=0x7fc140008040 comp rx=0 tx=0).stop 2024-01-15T18:17:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15806bfb0 msgr2=0x7fc1580ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15806bfb0 0x7fc1580ca7b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc1480ac000 tx=0x7fc14801ac80 comp rx=0 tx=0).stop 2024-01-15T18:17:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 shutdown_connections 2024-01-15T18:17:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc130067820 0x7fc130069ce0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1580cc9c0 0x7fc158154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1580b0310 0x7fc1580c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 --2- 172.21.15.143:0/1372668034 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc15806bfb0 0x7fc1580ca7b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.944+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 >> 172.21.15.143:0/1372668034 conn(0x7fc1580d0ed0 msgr2=0x7fc15806ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:30.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.948+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 shutdown_connections 2024-01-15T18:17:30.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:30.952+0000 7fc15e200640 1 -- 172.21.15.143:0/1372668034 wait complete. 2024-01-15T18:17:30.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:31.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 Processor -- start 2024-01-15T18:17:31.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 -- start start 2024-01-15T18:17:31.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fab7806bfb0 0x7fab7806c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab780b0310 0x7fab780cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab780cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab780cd3b0 con 0x7fab780b0310 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab780cbff0 con 0x7fab7806bfb0 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab780d0c50 con 0x7fab780cc9c0 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab780b0310 0x7fab780cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fab780b0310 0x7fab780cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40038/0 (socket says 172.21.15.143:40038) 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 -- 172.21.15.143:0/540982261 learned_addr learned my addr 172.21.15.143:0/540982261 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77e00640 1 --2- 172.21.15.143:0/540982261 >> v2:172.21.15.204:3300/0 conn(0x7fab7806bfb0 0x7fab7806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab7d200640 1 --2- 172.21.15.143:0/540982261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab780cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 -- 172.21.15.143:0/540982261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 msgr2=0x7fab780cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:31.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 --2- 172.21.15.143:0/540982261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab780cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 -- 172.21.15.143:0/540982261 >> v2:172.21.15.204:3300/0 conn(0x7fab7806bfb0 msgr2=0x7fab7806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:31.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 --2- 172.21.15.143:0/540982261 >> v2:172.21.15.204:3300/0 conn(0x7fab7806bfb0 0x7fab7806c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.128+0000 7fab77400640 1 -- 172.21.15.143:0/540982261 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7806c8d0 con 0x7fab780b0310 2024-01-15T18:17:31.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab77400640 1 --2- 172.21.15.143:0/540982261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab780b0310 0x7fab780cc480 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fab780716c0 tx=0x7fab6001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3307c4c2b9ef6103 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab76a00640 1 -- 172.21.15.143:0/540982261 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab600ab600 con 0x7fab780b0310 2024-01-15T18:17:31.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab76a00640 1 -- 172.21.15.143:0/540982261 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab600ab760 con 0x7fab780b0310 2024-01-15T18:17:31.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab76a00640 1 -- 172.21.15.143:0/540982261 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab600abaf0 con 0x7fab780b0310 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 -- 172.21.15.143:0/540982261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab780b0310 msgr2=0x7fab780cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 --2- 172.21.15.143:0/540982261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab780b0310 0x7fab780cc480 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fab780716c0 tx=0x7fab6001c440 comp rx=0 tx=0).stop 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 -- 172.21.15.143:0/540982261 shutdown_connections 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 --2- 172.21.15.143:0/540982261 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab780cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 --2- 172.21.15.143:0/540982261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab780b0310 0x7fab780cc480 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 --2- 172.21.15.143:0/540982261 >> v2:172.21.15.204:3300/0 conn(0x7fab7806bfb0 0x7fab7806c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 -- 172.21.15.143:0/540982261 >> 172.21.15.143:0/540982261 conn(0x7fab780d0ed0 msgr2=0x7fab780d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 -- 172.21.15.143:0/540982261 shutdown_connections 2024-01-15T18:17:31.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.132+0000 7fab7f200640 1 -- 172.21.15.143:0/540982261 wait complete. 2024-01-15T18:17:31.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 Processor -- start 2024-01-15T18:17:31.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 -- start start 2024-01-15T18:17:31.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 0x7fab78150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab780b0310 0x7fab781528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab781566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab781514e0 con 0x7fab7806bfb0 2024-01-15T18:17:31.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78157140 con 0x7fab780b0310 2024-01-15T18:17:31.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab78157420 con 0x7fab780cc9c0 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 0x7fab78150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 0x7fab78150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40042/0 (socket says 172.21.15.143:40042) 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 -- 172.21.15.143:0/1623465444 learned_addr learned my addr 172.21.15.143:0/1623465444 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77400640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab780b0310 0x7fab781528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7d200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab781566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 -- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 msgr2=0x7fab781566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab781566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 -- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab780b0310 msgr2=0x7fab781528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab780b0310 0x7fab781528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 -- 172.21.15.143:0/1623465444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab600094d0 con 0x7fab7806bfb0 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab77e00640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 0x7fab78150fa0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e870 tx=0x7fab6800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68009510 con 0x7fab7806bfb0 2024-01-15T18:17:31.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6800eec0 con 0x7fab7806bfb0 2024-01-15T18:17:31.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6800cb40 con 0x7fab7806bfb0 2024-01-15T18:17:31.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab78151dc0 con 0x7fab7806bfb0 2024-01-15T18:17:31.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.136+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab78152380 con 0x7fab7806bfb0 2024-01-15T18:17:31.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.140+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab78067500 con 0x7fab7806bfb0 2024-01-15T18:17:31.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.140+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fab6800cca0 con 0x7fab7806bfb0 2024-01-15T18:17:31.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.140+0000 7fab6fe00640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab50067820 0x7fab50069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.140+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fab680837b0 con 0x7fab7806bfb0 2024-01-15T18:17:31.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.140+0000 7fab77400640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab50067820 0x7fab50069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.140+0000 7fab77400640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab50067820 0x7fab50069ce0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fab600af000 tx=0x7fab600b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.144+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fab680524f0 con 0x7fab7806bfb0 2024-01-15T18:17:31.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.296+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab780d1db0 con 0x7fab7806bfb0 2024-01-15T18:17:31.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.296+0000 7fab6fe00640 1 -- 172.21.15.143:0/1623465444 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fab68017090 con 0x7fab7806bfb0 2024-01-15T18:17:31.296 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:31.296 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":99,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:30.310189+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:17:28.667297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-15T18:17:29.300772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab50067820 msgr2=0x7fab50069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab50067820 0x7fab50069ce0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fab600af000 tx=0x7fab600b6040 comp rx=0 tx=0).stop 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 msgr2=0x7fab78150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 0x7fab78150fa0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fab6800e870 tx=0x7fab6800ec70 comp rx=0 tx=0).stop 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 shutdown_connections 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab50067820 0x7fab50069ce0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fab780cc9c0 0x7fab781566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fab780b0310 0x7fab781528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 --2- 172.21.15.143:0/1623465444 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fab7806bfb0 0x7fab78150fa0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 >> 172.21.15.143:0/1623465444 conn(0x7fab780d0ed0 msgr2=0x7fab780b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:31.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 shutdown_connections 2024-01-15T18:17:31.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.300+0000 7fab7f200640 1 -- 172.21.15.143:0/1623465444 wait complete. 2024-01-15T18:17:31.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:31.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 Processor -- start 2024-01-15T18:17:31.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 -- start start 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f85d0141660 0x7f85d00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d015ec80 0x7f85d0141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d0071d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d015fbb0 con 0x7f85d015ec80 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d015f130 con 0x7f85d0141660 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85d4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015f310 con 0x7f85d0142470 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d015ec80 0x7f85d0141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d015ec80 0x7f85d0141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40050/0 (socket says 172.21.15.143:40050) 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 -- 172.21.15.143:0/1233158297 learned_addr learned my addr 172.21.15.143:0/1233158297 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:31.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85ce800640 1 --2- 172.21.15.143:0/1233158297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d0071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cde00640 1 --2- 172.21.15.143:0/1233158297 >> v2:172.21.15.204:3300/0 conn(0x7f85d0141660 0x7f85d00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 -- 172.21.15.143:0/1233158297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 msgr2=0x7f85d0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 --2- 172.21.15.143:0/1233158297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d0071d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 -- 172.21.15.143:0/1233158297 >> v2:172.21.15.204:3300/0 conn(0x7f85d0141660 msgr2=0x7f85d00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 --2- 172.21.15.143:0/1233158297 >> v2:172.21.15.204:3300/0 conn(0x7f85d0141660 0x7f85d00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cd400640 1 -- 172.21.15.143:0/1233158297 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d015f5f0 con 0x7f85d015ec80 2024-01-15T18:17:31.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.456+0000 7f85cde00640 1 --2- 172.21.15.143:0/1233158297 >> v2:172.21.15.204:3300/0 conn(0x7f85d0141660 0x7f85d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:31.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85cd400640 1 --2- 172.21.15.143:0/1233158297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d015ec80 0x7f85d0141f30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f85d00b1aa0 tx=0x7f85b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0df8420013b53de server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85cca00640 1 -- 172.21.15.143:0/1233158297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80ac600 con 0x7f85d015ec80 2024-01-15T18:17:31.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85cca00640 1 -- 172.21.15.143:0/1233158297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b80a8050 con 0x7f85d015ec80 2024-01-15T18:17:31.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85cca00640 1 -- 172.21.15.143:0/1233158297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80ac980 con 0x7f85d015ec80 2024-01-15T18:17:31.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 -- 172.21.15.143:0/1233158297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d015ec80 msgr2=0x7f85d0141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 --2- 172.21.15.143:0/1233158297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d015ec80 0x7f85d0141f30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f85d00b1aa0 tx=0x7f85b801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 -- 172.21.15.143:0/1233158297 shutdown_connections 2024-01-15T18:17:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 --2- 172.21.15.143:0/1233158297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d0071d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 --2- 172.21.15.143:0/1233158297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d015ec80 0x7f85d0141f30 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 --2- 172.21.15.143:0/1233158297 >> v2:172.21.15.204:3300/0 conn(0x7f85d0141660 0x7f85d00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 -- 172.21.15.143:0/1233158297 >> 172.21.15.143:0/1233158297 conn(0x7f85d006c3f0 msgr2=0x7f85d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:31.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 -- 172.21.15.143:0/1233158297 shutdown_connections 2024-01-15T18:17:31.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.460+0000 7f85d4e00640 1 -- 172.21.15.143:0/1233158297 wait complete. 2024-01-15T18:17:31.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 Processor -- start 2024-01-15T18:17:31.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 -- start start 2024-01-15T18:17:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 0x7f85d01636b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d015ec80 0x7f85d015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d012d3e0 con 0x7f85d0141660 2024-01-15T18:17:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015b2a0 con 0x7f85d015ec80 2024-01-15T18:17:31.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015d530 con 0x7f85d0142470 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85cde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 0x7f85d01636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85cde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 0x7f85d01636b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40062/0 (socket says 172.21.15.143:40062) 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d015ec80 0x7f85d015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85cd400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85cde00640 1 -- 172.21.15.143:0/3888991608 learned_addr learned my addr 172.21.15.143:0/3888991608 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 -- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 msgr2=0x7f85d015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d015ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 -- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 msgr2=0x7f85d01636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 0x7f85d01636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 -- 172.21.15.143:0/3888991608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80094d0 con 0x7f85d015ec80 2024-01-15T18:17:31.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85cde00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 0x7f85d01636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85ce800640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d015ec80 0x7f85d015cf50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f85c800efc0 tx=0x7f85c800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8009b10 con 0x7f85d015ec80 2024-01-15T18:17:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c800f040 con 0x7f85d015ec80 2024-01-15T18:17:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80146a0 con 0x7f85d015ec80 2024-01-15T18:17:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d00cd150 con 0x7f85d015ec80 2024-01-15T18:17:31.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.464+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d015daf0 con 0x7f85d015ec80 2024-01-15T18:17:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.468+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d012d520 con 0x7f85d015ec80 2024-01-15T18:17:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.468+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f85c8014800 con 0x7f85d015ec80 2024-01-15T18:17:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.468+0000 7f85c6000640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85a8067820 0x7f85a8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.468+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f85c8083c10 con 0x7f85d015ec80 2024-01-15T18:17:31.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.468+0000 7f85cde00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85a8067820 0x7f85a8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.468+0000 7f85cde00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85a8067820 0x7f85a8069ce0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009d80 tx=0x7f85c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.472+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85d012d520 con 0x7f85d015ec80 2024-01-15T18:17:31.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.624+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85d012d520 con 0x7f85d015ec80 2024-01-15T18:17:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.624+0000 7f85c6000640 1 -- 172.21.15.143:0/3888991608 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f85d012d520 con 0x7f85d015ec80 2024-01-15T18:17:31.623 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:31.623 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":99,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:30.310189+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:17:28.667297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-15T18:17:29.300772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:31.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85a8067820 msgr2=0x7f85a8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85a8067820 0x7f85a8069ce0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009d80 tx=0x7f85c0008040 comp rx=0 tx=0).stop 2024-01-15T18:17:31.625 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d015ec80 msgr2=0x7f85d015cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d015ec80 0x7f85d015cf50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f85c800efc0 tx=0x7f85c800c490 comp rx=0 tx=0).stop 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 shutdown_connections 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85a8067820 0x7f85a8069ce0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d015ec80 0x7f85d015cf50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d0142470 0x7f85d015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 --2- 172.21.15.143:0/3888991608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d0141660 0x7f85d01636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 >> 172.21.15.143:0/3888991608 conn(0x7f85d006c3f0 msgr2=0x7f85d00afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 shutdown_connections 2024-01-15T18:17:31.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.628+0000 7f85d4e00640 1 -- 172.21.15.143:0/3888991608 wait complete. 2024-01-15T18:17:31.645 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:17:31.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.784+0000 7fcc11400640 1 Processor -- start 2024-01-15T18:17:31.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.784+0000 7fcc11400640 1 -- start start 2024-01-15T18:17:31.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc11400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcc0c141660 0x7fcc0c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc11400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c15f310 con 0x7fcc0c15ec80 2024-01-15T18:17:31.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc11400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c15f5f0 con 0x7fcc0c071d00 2024-01-15T18:17:31.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc11400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c15fbb0 con 0x7fcc0c141660 2024-01-15T18:17:31.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc09e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcc0c15ec80 0x7fcc0c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41918/0 (socket says 172.21.15.143:41918) 2024-01-15T18:17:31.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc09e00640 1 -- 172.21.15.143:0/542330247 learned_addr learned my addr 172.21.15.143:0/542330247 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:31.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc0a800640 1 --2- 172.21.15.143:0/542330247 >> v2:172.21.15.204:3300/0 conn(0x7fcc0c141660 0x7fcc0c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc09400640 1 -- 172.21.15.143:0/542330247 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822573896 0 0) 0x7fcc0c15f310 con 0x7fcc0c15ec80 2024-01-15T18:17:31.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc09400640 1 -- 172.21.15.143:0/542330247 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbec003530 con 0x7fcc0c15ec80 2024-01-15T18:17:31.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc0a800640 1 -- 172.21.15.143:0/542330247 >> v1:172.21.15.204:6789/0 conn(0x7fcc0c071d00 legacy=0x7fcc0c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc09400640 1 -- 172.21.15.143:0/542330247 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3999610015 0 0) 0x7fcbec003530 con 0x7fcc0c15ec80 2024-01-15T18:17:31.786 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc0a800640 1 -- 172.21.15.143:0/542330247 >> v1:172.21.15.143:6789/0 conn(0x7fcc0c15ec80 legacy=0x7fcc0c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.788+0000 7fcc0a800640 1 -- 172.21.15.143:0/542330247 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c15f8d0 con 0x7fcc0c141660 2024-01-15T18:17:31.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc0a800640 1 --2- 172.21.15.143:0/542330247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c141660 0x7fcc0c0ce560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0097c0 tx=0x7fcbfc01c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=352ad65c4a154e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc09400640 1 -- 172.21.15.143:0/542330247 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4001680 con 0x7fcc0c141660 2024-01-15T18:17:31.789 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc09400640 1 -- 172.21.15.143:0/542330247 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc0ac440 con 0x7fcc0c141660 2024-01-15T18:17:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc09400640 1 -- 172.21.15.143:0/542330247 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0ac7d0 con 0x7fcc0c141660 2024-01-15T18:17:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 -- 172.21.15.143:0/542330247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c141660 msgr2=0x7fcc0c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 --2- 172.21.15.143:0/542330247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c141660 0x7fcc0c0ce560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0097c0 tx=0x7fcbfc01c370 comp rx=0 tx=0).stop 2024-01-15T18:17:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 -- 172.21.15.143:0/542330247 shutdown_connections 2024-01-15T18:17:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 --2- 172.21.15.143:0/542330247 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c141660 0x7fcc0c0ce560 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.790 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 -- 172.21.15.143:0/542330247 >> 172.21.15.143:0/542330247 conn(0x7fcc0c06c3f0 msgr2=0x7fcc0c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:31.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 -- 172.21.15.143:0/542330247 shutdown_connections 2024-01-15T18:17:31.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 -- 172.21.15.143:0/542330247 wait complete. 2024-01-15T18:17:31.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 Processor -- start 2024-01-15T18:17:31.792 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.792+0000 7fcc11400640 1 -- start start 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c071d00 0x7fcc0c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 0x7fcc0c143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15ec80 0x7fcc0c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c15f130 con 0x7fcc0c141660 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b3290 con 0x7fcc0c15ec80 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0b3570 con 0x7fcc0c071d00 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 0x7fcc0c143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 0x7fcc0c143360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40070/0 (socket says 172.21.15.143:40070) 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 -- 172.21.15.143:0/3144976351 learned_addr learned my addr 172.21.15.143:0/3144976351 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc0b200640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15ec80 0x7fcc0c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc0a800640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c071d00 0x7fcc0c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 -- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c071d00 msgr2=0x7fcc0c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c071d00 0x7fcc0c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 -- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15ec80 msgr2=0x7fcc0c1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15ec80 0x7fcc0c1438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 -- 172.21.15.143:0/3144976351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009420 con 0x7fcc0c141660 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc09e00640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 0x7fcc0c143360 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c750 tx=0x7fcbf400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4005ae0 con 0x7fcc0c141660 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf4005c40 con 0x7fcc0c141660 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c0cd150 con 0x7fcc0c141660 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf40072b0 con 0x7fcc0c141660 2024-01-15T18:17:31.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c0b3b30 con 0x7fcc0c141660 2024-01-15T18:17:31.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.796+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c067500 con 0x7fcc0c141660 2024-01-15T18:17:31.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.800+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fcbf4007410 con 0x7fcc0c141660 2024-01-15T18:17:31.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.800+0000 7fcc02a00640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbe4067820 0x7fcbe4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:31.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.800+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcbf4082620 con 0x7fcc0c141660 2024-01-15T18:17:31.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.800+0000 7fcc0a800640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbe4067820 0x7fcbe4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:31.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.800+0000 7fcc0a800640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbe4067820 0x7fcbe4069ce0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0ae000 tx=0x7fcbfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:31.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.804+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcbf40513d0 con 0x7fcc0c141660 2024-01-15T18:17:31.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:31.988+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcc0c06e2a0 con 0x7fcc0c141660 2024-01-15T18:17:32.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.952+0000 7fcc02a00640 1 -- 172.21.15.143:0/3144976351 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcbf40549f0 con 0x7fcc0c141660 2024-01-15T18:17:32.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbe4067820 msgr2=0x7fcbe4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbe4067820 0x7fcbe4069ce0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc0ae000 tx=0x7fcbfc008040 comp rx=0 tx=0).stop 2024-01-15T18:17:32.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 msgr2=0x7fcc0c143360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 0x7fcc0c143360 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400c750 tx=0x7fcbf400cc10 comp rx=0 tx=0).stop 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 shutdown_connections 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcbe4067820 0x7fcbe4069ce0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc0c15ec80 0x7fcc0c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc0c141660 0x7fcc0c143360 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 --2- 172.21.15.143:0/3144976351 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcc0c071d00 0x7fcc0c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 >> 172.21.15.143:0/3144976351 conn(0x7fcc0c06c3f0 msgr2=0x7fcc0c070110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 shutdown_connections 2024-01-15T18:17:32.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:32.956+0000 7fcc11400640 1 -- 172.21.15.143:0/3144976351 wait complete. 2024-01-15T18:17:32.980 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:17:32.980 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:17:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a7600640 1 Processor -- start 2024-01-15T18:17:33.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a7600640 1 -- start start 2024-01-15T18:17:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a7600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 0x7fe0a00af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a7600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a00725b0 con 0x7fe0a00adf10 2024-01-15T18:17:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a7600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a0072890 con 0x7fe0a015d570 2024-01-15T18:17:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a7600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00b3f30 con 0x7fe0a00ac060 2024-01-15T18:17:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a5600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe0a00adf10 0x7fe0a00ad450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41928/0 (socket says 172.21.15.143:41928) 2024-01-15T18:17:33.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a5600640 1 -- 172.21.15.143:0/184888607 learned_addr learned my addr 172.21.15.143:0/184888607 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 0x7fe0a00af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063999836 0 0) 0x7fe0a00725b0 con 0x7fe0a00adf10 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe080003530 con 0x7fe0a00adf10 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 910082315 0 0) 0x7fe080003530 con 0x7fe0a00adf10 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 msgr2=0x7fe0a00af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 0x7fe0a00af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 >> v1:172.21.15.204:6789/0 conn(0x7fe0a015d570 legacy=0x7fe0a00af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00b4210 con 0x7fe0a00adf10 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fe098001a00 con 0x7fe0a00adf10 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 >> v1:172.21.15.143:6789/0 conn(0x7fe0a00adf10 legacy=0x7fe0a00ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe080003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0800047d0 0x7fe080004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 0x7fe080005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00b3f30 con 0x7fe0800039b0 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08000ba40 con 0x7fe0800047d0 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08000bc00 con 0x7fe0a00ac060 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe080003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe09fe00640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0800047d0 0x7fe080004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a5600640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 0x7fe080005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 -- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 msgr2=0x7fe080005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a00ac060 0x7fe080005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 -- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0800047d0 msgr2=0x7fe080004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0800047d0 0x7fe080004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.128+0000 7fe0a4c00640 1 -- 172.21.15.143:0/184888607 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe080003530 con 0x7fe0800039b0 2024-01-15T18:17:33.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe080003dd0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ab0 tx=0x7fe09001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686dd14361d5bb60 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:33.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900a55d0 con 0x7fe0800039b0 2024-01-15T18:17:33.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0900a5730 con 0x7fe0800039b0 2024-01-15T18:17:33.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe09f400640 1 -- 172.21.15.143:0/184888607 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900a5ae0 con 0x7fe0800039b0 2024-01-15T18:17:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a7600640 1 -- 172.21.15.143:0/184888607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 msgr2=0x7fe080003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a7600640 1 --2- 172.21.15.143:0/184888607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe080003dd0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ab0 tx=0x7fe09001c9e0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a5600640 1 -- 172.21.15.143:0/184888607 reap_dead start 2024-01-15T18:17:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a7600640 1 -- 172.21.15.143:0/184888607 shutdown_connections 2024-01-15T18:17:33.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a7600640 1 -- 172.21.15.143:0/184888607 >> 172.21.15.143:0/184888607 conn(0x7fe0a0071a90 msgr2=0x7fe0a0071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:33.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a7600640 1 -- 172.21.15.143:0/184888607 shutdown_connections 2024-01-15T18:17:33.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.132+0000 7fe0a7600640 1 -- 172.21.15.143:0/184888607 wait complete. 2024-01-15T18:17:33.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 Processor -- start 2024-01-15T18:17:33.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 -- start start 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe0a0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a0157b10 0x7fe0a01587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a01f63f0 0x7fe0a01f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0099280 con 0x7fe0800039b0 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0099560 con 0x7fe0a01f63f0 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a0099840 con 0x7fe0a0157b10 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe0a0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe0a0157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40094/0 (socket says 172.21.15.143:40094) 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 -- 172.21.15.143:0/1898271104 learned_addr learned my addr 172.21.15.143:0/1898271104 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a5600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a01f63f0 0x7fe0a01f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe09fe00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a0157b10 0x7fe0a01587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 -- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a0157b10 msgr2=0x7fe0a01587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a0157b10 0x7fe0a01587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 -- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a01f63f0 msgr2=0x7fe0a01f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a01f63f0 0x7fe0a01f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 -- 172.21.15.143:0/1898271104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0099b20 con 0x7fe0800039b0 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a5600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a01f63f0 0x7fe0a01f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe09fe00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a0157b10 0x7fe0a01587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:33.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a4c00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe0a0157550 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b0040 tx=0x7fe080005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900a5ea0 con 0x7fe0800039b0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe090004020 con 0x7fe0800039b0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a015c210 con 0x7fe0800039b0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.136+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900043b0 con 0x7fe0800039b0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a0158cf0 con 0x7fe0800039b0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a0158ee0 con 0x7fe0800039b0 2024-01-15T18:17:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fe0900b4040 con 0x7fe0800039b0 2024-01-15T18:17:33.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe09cc00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe078067820 0x7fe078069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe09011d1a0 con 0x7fe0800039b0 2024-01-15T18:17:33.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe09fe00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe078067820 0x7fe078069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.140+0000 7fe09fe00640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe078067820 0x7fe078069ce0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe0a006df70 tx=0x7fe088009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.144+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe0900ebc50 con 0x7fe0800039b0 2024-01-15T18:17:33.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.324+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe0a0067500 con 0x7fe0800039b0 2024-01-15T18:17:33.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.956+0000 7fe09cc00640 1 -- 172.21.15.143:0/1898271104 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0900ef270 con 0x7fe0800039b0 2024-01-15T18:17:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe078067820 msgr2=0x7fe078069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe078067820 0x7fe078069ce0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe0a006df70 tx=0x7fe088009040 comp rx=0 tx=0).stop 2024-01-15T18:17:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 msgr2=0x7fe0a0157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe0a0157550 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b0040 tx=0x7fe080005800 comp rx=0 tx=0).stop 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 shutdown_connections 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe078067820 0x7fe078069ce0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe0a01f63f0 0x7fe0a01f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe0a0157b10 0x7fe0a01587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 --2- 172.21.15.143:0/1898271104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0800039b0 0x7fe0a0157550 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 >> 172.21.15.143:0/1898271104 conn(0x7fe0a0071a90 msgr2=0x7fe0a00cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 shutdown_connections 2024-01-15T18:17:33.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:33.960+0000 7fe0a7600640 1 -- 172.21.15.143:0/1898271104 wait complete. 2024-01-15T18:17:33.982 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:17:33.982 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:17:34.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 Processor -- start 2024-01-15T18:17:34.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 -- start start 2024-01-15T18:17:34.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9880143780 0x7f98800cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98800722c0 0x7f988015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98800ceb50 con 0x7f98800ce520 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988015c110 con 0x7f98800722c0 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f9885600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988015b7e0 con 0x7f9880143780 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f987f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98800722c0 0x7f988015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f987f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f98800722c0 0x7f988015c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40110/0 (socket says 172.21.15.143:40110) 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f987f400640 1 -- 172.21.15.143:0/1672497519 learned_addr learned my addr 172.21.15.143:0/1672497519 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f987f400640 1 -- 172.21.15.143:0/1672497519 >> v2:172.21.15.204:3300/0 conn(0x7f9880143780 msgr2=0x7f98800cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.124+0000 7f987ea00640 1 --2- 172.21.15.143:0/1672497519 >> v2:172.21.15.204:3300/0 conn(0x7f9880143780 0x7f98800cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987f400640 1 --2- 172.21.15.143:0/1672497519 >> v2:172.21.15.204:3300/0 conn(0x7f9880143780 0x7f98800cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987f400640 1 -- 172.21.15.143:0/1672497519 >> v1:172.21.15.204:6789/0 conn(0x7f98800ce520 legacy=0x7f9880071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987f400640 1 -- 172.21.15.143:0/1672497519 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988015bac0 con 0x7f98800722c0 2024-01-15T18:17:34.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987ea00640 1 --2- 172.21.15.143:0/1672497519 >> v2:172.21.15.204:3300/0 conn(0x7f9880143780 0x7f98800cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:34.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987f400640 1 --2- 172.21.15.143:0/1672497519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800722c0 0x7f988015c5a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f988006be30 tx=0x7f987801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e237b921e7e72e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987d600640 1 -- 172.21.15.143:0/1672497519 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780a5760 con 0x7f98800722c0 2024-01-15T18:17:34.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987d600640 1 -- 172.21.15.143:0/1672497519 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98780a58c0 con 0x7f98800722c0 2024-01-15T18:17:34.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f987d600640 1 -- 172.21.15.143:0/1672497519 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780a5c50 con 0x7f98800722c0 2024-01-15T18:17:34.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f9885600640 1 -- 172.21.15.143:0/1672497519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800722c0 msgr2=0x7f988015c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.128+0000 7f9885600640 1 --2- 172.21.15.143:0/1672497519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800722c0 0x7f988015c5a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f988006be30 tx=0x7f987801c990 comp rx=0 tx=0).stop 2024-01-15T18:17:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- 172.21.15.143:0/1672497519 shutdown_connections 2024-01-15T18:17:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 --2- 172.21.15.143:0/1672497519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800722c0 0x7f988015c5a0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 --2- 172.21.15.143:0/1672497519 >> v2:172.21.15.204:3300/0 conn(0x7f9880143780 0x7f98800cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- 172.21.15.143:0/1672497519 >> 172.21.15.143:0/1672497519 conn(0x7f988006c3f0 msgr2=0x7f988006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- 172.21.15.143:0/1672497519 shutdown_connections 2024-01-15T18:17:34.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- 172.21.15.143:0/1672497519 wait complete. 2024-01-15T18:17:34.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 Processor -- start 2024-01-15T18:17:34.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- start start 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98800722c0 0x7f9880154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 0x7f9880070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9880143780 0x7f9880070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98801562b0 con 0x7f98800ce520 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b2190 con 0x7f9880143780 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f9885600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9880152010 con 0x7f98800722c0 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 0x7f9880070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 0x7f9880070760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40116/0 (socket says 172.21.15.143:40116) 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 -- 172.21.15.143:0/2439357176 learned_addr learned my addr 172.21.15.143:0/2439357176 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987ea00640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98800722c0 0x7f9880154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987f400640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9880143780 0x7f9880070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 -- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98800722c0 msgr2=0x7f9880154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98800722c0 0x7f9880154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 -- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9880143780 msgr2=0x7f9880070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9880143780 0x7f9880070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.132+0000 7f987e000640 1 -- 172.21.15.143:0/2439357176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9878009b30 con 0x7f98800ce520 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f987e000640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 0x7f9880070760 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f986800c450 tx=0x7f986800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868012830 con 0x7f98800ce520 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98680129b0 con 0x7f98800ce520 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868012d60 con 0x7f98800ce520 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9880155630 con 0x7f98800ce520 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9880155bf0 con 0x7f98800ce520 2024-01-15T18:17:34.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98801563f0 con 0x7f98800ce520 2024-01-15T18:17:34.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f986801c460 con 0x7f98800ce520 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9876a00640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9858067820 0x7f9858069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9868082f50 con 0x7f98800ce520 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.136+0000 7f987ea00640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9858067820 0x7f9858069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.140+0000 7f987ea00640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9858067820 0x7f9858069ce0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f98800afde0 tx=0x7f9870008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.144+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98801563f0 con 0x7f98800ce520 2024-01-15T18:17:34.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.324+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98801563f0 con 0x7f98800ce520 2024-01-15T18:17:34.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.324+0000 7f9876a00640 1 -- 172.21.15.143:0/2439357176 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f98801563f0 con 0x7f98800ce520 2024-01-15T18:17:34.324 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:34.324 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1910039233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1910039233},{"type":"v1","addr":"172.21.15.204:6835","nonce":1910039233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2986890926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2986890926},{"type":"v1","addr":"172.21.15.143:6837","nonce":2986890926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2769510067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2769510067},{"type":"v1","addr":"172.21.15.204:6834","nonce":2769510067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:17:30.310521+0000","modified":"2024-01-15T18:17:33.332084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/4145140990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4145140990},{"type":"v1","addr":"172.21.15.143:6835","nonce":4145140990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-01-15T18:17:34.325 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 62 2024-01-15T18:17:34.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9858067820 msgr2=0x7f9858069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9858067820 0x7f9858069ce0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f98800afde0 tx=0x7f9870008040 comp rx=0 tx=0).stop 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 msgr2=0x7f9880070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 0x7f9880070760 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f986800c450 tx=0x7f986800c910 comp rx=0 tx=0).stop 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 shutdown_connections 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9858067820 0x7f9858069ce0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9880143780 0x7f9880070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98800ce520 0x7f9880070760 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 --2- 172.21.15.143:0/2439357176 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98800722c0 0x7f9880154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 >> 172.21.15.143:0/2439357176 conn(0x7f988006c3f0 msgr2=0x7f98800aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 shutdown_connections 2024-01-15T18:17:34.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.328+0000 7f9885600640 1 -- 172.21.15.143:0/2439357176 wait complete. 2024-01-15T18:17:34.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:34.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 Processor -- start 2024-01-15T18:17:34.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 -- start start 2024-01-15T18:17:34.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faef806bfb0 0x7faef806c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faef80d0210 0x7faef80ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef80cbff0 con 0x7faef80b0310 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef80d0c50 con 0x7faef806bfb0 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faefe400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef806c8d0 con 0x7faef80d0210 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faef806bfb0 0x7faef806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faef806bfb0 0x7faef806c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40122/0 (socket says 172.21.15.143:40122) 2024-01-15T18:17:34.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 -- 172.21.15.143:0/2723623396 learned_addr learned my addr 172.21.15.143:0/2723623396 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 -- 172.21.15.143:0/2723623396 >> v2:172.21.15.204:3300/0 conn(0x7faef80d0210 msgr2=0x7faef80ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7e00640 1 --2- 172.21.15.143:0/2723623396 >> v2:172.21.15.204:3300/0 conn(0x7faef80d0210 0x7faef80ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 --2- 172.21.15.143:0/2723623396 >> v2:172.21.15.204:3300/0 conn(0x7faef80d0210 0x7faef80ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 -- 172.21.15.143:0/2723623396 >> v1:172.21.15.204:6789/0 conn(0x7faef80b0310 legacy=0x7faef80cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7400640 1 -- 172.21.15.143:0/2723623396 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80cb6c0 con 0x7faef806bfb0 2024-01-15T18:17:34.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.484+0000 7faef7e00640 1 --2- 172.21.15.143:0/2723623396 >> v2:172.21.15.204:3300/0 conn(0x7faef80d0210 0x7faef80ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faef7400640 1 --2- 172.21.15.143:0/2723623396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef806bfb0 0x7faef806c390 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7faef806b3d0 tx=0x7faee801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc3892a1fba3b32 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faef6000640 1 -- 172.21.15.143:0/2723623396 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80ac640 con 0x7faef806bfb0 2024-01-15T18:17:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faef6000640 1 -- 172.21.15.143:0/2723623396 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee80ac7a0 con 0x7faef806bfb0 2024-01-15T18:17:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faef6000640 1 -- 172.21.15.143:0/2723623396 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80acb30 con 0x7faef806bfb0 2024-01-15T18:17:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 -- 172.21.15.143:0/2723623396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef806bfb0 msgr2=0x7faef806c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 --2- 172.21.15.143:0/2723623396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef806bfb0 0x7faef806c390 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7faef806b3d0 tx=0x7faee801c400 comp rx=0 tx=0).stop 2024-01-15T18:17:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 -- 172.21.15.143:0/2723623396 shutdown_connections 2024-01-15T18:17:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 --2- 172.21.15.143:0/2723623396 >> v2:172.21.15.204:3300/0 conn(0x7faef80d0210 0x7faef80ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 --2- 172.21.15.143:0/2723623396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef806bfb0 0x7faef806c390 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 -- 172.21.15.143:0/2723623396 >> 172.21.15.143:0/2723623396 conn(0x7faef80d0ed0 msgr2=0x7faef80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.488+0000 7faefe400640 1 -- 172.21.15.143:0/2723623396 shutdown_connections 2024-01-15T18:17:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 -- 172.21.15.143:0/2723623396 wait complete. 2024-01-15T18:17:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 Processor -- start 2024-01-15T18:17:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 -- start start 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faef806bfb0 0x7faef8153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef80b0310 0x7faef8154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 0x7faef80c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef80c94d0 con 0x7faef80d0210 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef8153a50 con 0x7faef80b0310 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faefe400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef80c98f0 con 0x7faef806bfb0 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 0x7faef80c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 0x7faef80c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40136/0 (socket says 172.21.15.143:40136) 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 -- 172.21.15.143:0/3495390413 learned_addr learned my addr 172.21.15.143:0/3495390413 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faef806bfb0 0x7faef8153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef6a00640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef80b0310 0x7faef8154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 -- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faef806bfb0 msgr2=0x7faef8153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faef806bfb0 0x7faef8153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 -- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef80b0310 msgr2=0x7faef8154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef80b0310 0x7faef8154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 -- 172.21.15.143:0/3495390413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee80094d0 con 0x7faef80d0210 2024-01-15T18:17:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.492+0000 7faef7e00640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 0x7faef80c9b90 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7faef000cd50 tx=0x7faef000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000f680 con 0x7faef80d0210 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef000f800 con 0x7faef80d0210 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef80c8b30 con 0x7faef80d0210 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000fbb0 con 0x7faef80d0210 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef80c90f0 con 0x7faef80d0210 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef80c9610 con 0x7faef80d0210 2024-01-15T18:17:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7faef000b040 con 0x7faef80d0210 2024-01-15T18:17:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faeef400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faed0067820 0x7faed0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7faef0083b10 con 0x7faef80d0210 2024-01-15T18:17:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.496+0000 7faef7400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faed0067820 0x7faed0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.500+0000 7faef7400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faed0067820 0x7faed0069ce0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faee80ae000 tx=0x7faee80a6070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.500+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faef80c9610 con 0x7faef80d0210 2024-01-15T18:17:34.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.652+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faef80c9610 con 0x7faef80d0210 2024-01-15T18:17:34.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.652+0000 7faeef400640 1 -- 172.21.15.143:0/3495390413 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16634 (secure 0 0 0) 0x7faef80c9610 con 0x7faef80d0210 2024-01-15T18:17:34.652 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:34.653 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":100,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:17:31.945696+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:17:28.667297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-15T18:17:29.300772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faed0067820 msgr2=0x7faed0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faed0067820 0x7faed0069ce0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faee80ae000 tx=0x7faee80a6070 comp rx=0 tx=0).stop 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 msgr2=0x7faef80c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 0x7faef80c9b90 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7faef000cd50 tx=0x7faef000a650 comp rx=0 tx=0).stop 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 shutdown_connections 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faed0067820 0x7faed0069ce0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef80d0210 0x7faef80c9b90 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faef80b0310 0x7faef8154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 --2- 172.21.15.143:0/3495390413 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faef806bfb0 0x7faef8153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 >> 172.21.15.143:0/3495390413 conn(0x7faef80d0ed0 msgr2=0x7faef80ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 shutdown_connections 2024-01-15T18:17:34.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.656+0000 7faefe400640 1 -- 172.21.15.143:0/3495390413 wait complete. 2024-01-15T18:17:34.675 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:17:34.675 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:17:34.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.812+0000 7ff713a00640 1 Processor -- start 2024-01-15T18:17:34.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.812+0000 7ff713a00640 1 -- start start 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff713a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff70c070f20 0x7ff70c06b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff713a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff70c071300 0x7ff70c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff713a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c15e6c0 con 0x7ff70c157700 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff713a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c157b90 con 0x7ff70c070f20 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff713a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c06a490 con 0x7ff70c071300 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff70c070f20 0x7ff70c06b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff70c157700 0x7ff70c06cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41944/0 (socket says 172.21.15.143:41944) 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711a00640 1 -- 172.21.15.143:0/2377396088 learned_addr learned my addr 172.21.15.143:0/2377396088 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:34.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff70be00640 1 --2- 172.21.15.143:0/2377396088 >> v2:172.21.15.204:3300/0 conn(0x7ff70c071300 0x7ff70c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711000640 1 -- 172.21.15.143:0/2377396088 >> v2:172.21.15.204:3300/0 conn(0x7ff70c071300 msgr2=0x7ff70c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711000640 1 --2- 172.21.15.143:0/2377396088 >> v2:172.21.15.204:3300/0 conn(0x7ff70c071300 0x7ff70c1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711000640 1 -- 172.21.15.143:0/2377396088 >> v1:172.21.15.143:6789/0 conn(0x7ff70c157700 legacy=0x7ff70c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff711000640 1 -- 172.21.15.143:0/2377396088 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c06a770 con 0x7ff70c070f20 2024-01-15T18:17:34.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.816+0000 7ff70b400640 1 -- 172.21.15.143:0/2377396088 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333943670 0 0) 0x7ff70c15e6c0 con 0x7ff70c157700 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff711000640 1 --2- 172.21.15.143:0/2377396088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c070f20 0x7ff70c06b630 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0b42c0 tx=0x7ff6fc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68eac9a4794c334 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff70b400640 1 -- 172.21.15.143:0/2377396088 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc004230 con 0x7ff70c070f20 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff70b400640 1 -- 172.21.15.143:0/2377396088 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0a7050 con 0x7ff70c070f20 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff70b400640 1 -- 172.21.15.143:0/2377396088 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc003620 con 0x7ff70c070f20 2024-01-15T18:17:34.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 -- 172.21.15.143:0/2377396088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c070f20 msgr2=0x7ff70c06b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 --2- 172.21.15.143:0/2377396088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c070f20 0x7ff70c06b630 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff70c0b42c0 tx=0x7ff6fc01c430 comp rx=0 tx=0).stop 2024-01-15T18:17:34.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 -- 172.21.15.143:0/2377396088 shutdown_connections 2024-01-15T18:17:34.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 --2- 172.21.15.143:0/2377396088 >> v2:172.21.15.204:3300/0 conn(0x7ff70c071300 0x7ff70c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 --2- 172.21.15.143:0/2377396088 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c070f20 0x7ff70c06b630 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 -- 172.21.15.143:0/2377396088 >> 172.21.15.143:0/2377396088 conn(0x7ff70c0b1610 msgr2=0x7ff70c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 -- 172.21.15.143:0/2377396088 shutdown_connections 2024-01-15T18:17:34.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 -- 172.21.15.143:0/2377396088 wait complete. 2024-01-15T18:17:34.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 Processor -- start 2024-01-15T18:17:34.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.820+0000 7ff713a00640 1 -- start start 2024-01-15T18:17:34.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff70c070f20 0x7ff70c147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff70c071300 0x7ff70c135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 0x7ff70c1370c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c14ff50 con 0x7ff70c157700 2024-01-15T18:17:34.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c150230 con 0x7ff70c070f20 2024-01-15T18:17:34.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c150510 con 0x7ff70c071300 2024-01-15T18:17:34.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 0x7ff70c1370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 0x7ff70c1370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40156/0 (socket says 172.21.15.143:40156) 2024-01-15T18:17:34.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 -- 172.21.15.143:0/3178226534 learned_addr learned my addr 172.21.15.143:0/3178226534 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:34.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711000640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff70c070f20 0x7ff70c147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff70be00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff70c071300 0x7ff70c135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 -- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff70c071300 msgr2=0x7ff70c135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff70c071300 0x7ff70c135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 -- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff70c070f20 msgr2=0x7ff70c147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff70c070f20 0x7ff70c147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 -- 172.21.15.143:0/3178226534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc009420 con 0x7ff70c157700 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711000640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff70c070f20 0x7ff70c147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff711a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 0x7ff70c1370c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff70400c780 tx=0x7ff70400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70400ce90 con 0x7ff70c157700 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70400a640 con 0x7ff70c157700 2024-01-15T18:17:34.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c15e9f0 con 0x7ff70c157700 2024-01-15T18:17:34.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70400a9d0 con 0x7ff70c157700 2024-01-15T18:17:34.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c147ac0 con 0x7ff70c157700 2024-01-15T18:17:34.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c074b00 con 0x7ff70c157700 2024-01-15T18:17:34.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.824+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7ff70400ab30 con 0x7ff70c157700 2024-01-15T18:17:34.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.828+0000 7ff708c00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6e4067820 0x7ff6e4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.828+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff704082720 con 0x7ff70c157700 2024-01-15T18:17:34.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.828+0000 7ff711000640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6e4067820 0x7ff6e4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.826 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.828+0000 7ff711000640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6e4067820 0x7ff6e4069ce0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0ad000 tx=0x7ff6fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.832+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff704051120 con 0x7ff70c157700 2024-01-15T18:17:34.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.980+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff70c06bb70 con 0x7ff6e4067820 2024-01-15T18:17:34.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.988+0000 7ff708c00640 1 -- 172.21.15.143:0/3178226534 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288296 (secure 0 0 0) 0x7ff6fc0044d0 con 0x7ff6e4067820 2024-01-15T18:17:34.991 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:34.993 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6e4067820 msgr2=0x7ff6e4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6e4067820 0x7ff6e4069ce0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0ad000 tx=0x7ff6fc008040 comp rx=0 tx=0).stop 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 msgr2=0x7ff70c1370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 0x7ff70c1370c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff70400c780 tx=0x7ff70400cc40 comp rx=0 tx=0).stop 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 shutdown_connections 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff6e4067820 0x7ff6e4069ce0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff70c157700 0x7ff70c1370c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff70c071300 0x7ff70c135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 --2- 172.21.15.143:0/3178226534 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff70c070f20 0x7ff70c147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 >> 172.21.15.143:0/3178226534 conn(0x7ff70c0b1610 msgr2=0x7ff70c0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 shutdown_connections 2024-01-15T18:17:34.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:34.992+0000 7ff713a00640 1 -- 172.21.15.143:0/3178226534 wait complete. 2024-01-15T18:17:35.012 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-01-15T18:17:34.879462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":530496,"kb_used_data":10756,"kb_used_omap":12,"kb_used_meta":519603,"kb_avail":754444224,"statfs":{"total":773094113280,"available":772550885376,"internally_reserved":0,"allocated":11014144,"data_stored":5404436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":532074032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.961479"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.962544+0000","last_change":"2024-01-15T18:17:30.954624+0000","last_active":"2024-01-15T18:17:30.962544+0000","last_peered":"2024-01-15T18:17:30.962544+0000","last_clean":"2024-01-15T18:17:30.962544+0000","last_became_active":"2024-01-15T18:17:30.954352+0000","last_became_peered":"2024-01-15T18:17:30.954352+0000","last_unstale":"2024-01-15T18:17:30.962544+0000","last_undegraded":"2024-01-15T18:17:30.962544+0000","last_fullsized":"2024-01-15T18:17:30.962544+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:00.544507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975605+0000","last_change":"2024-01-15T18:17:30.963931+0000","last_active":"2024-01-15T18:17:30.975605+0000","last_peered":"2024-01-15T18:17:30.975605+0000","last_clean":"2024-01-15T18:17:30.975605+0000","last_became_active":"2024-01-15T18:17:30.963002+0000","last_became_peered":"2024-01-15T18:17:30.963002+0000","last_unstale":"2024-01-15T18:17:30.975605+0000","last_undegraded":"2024-01-15T18:17:30.975605+0000","last_fullsized":"2024-01-15T18:17:30.975605+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:11.421513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968018+0000","last_change":"2024-01-15T18:17:30.957766+0000","last_active":"2024-01-15T18:17:30.968018+0000","last_peered":"2024-01-15T18:17:30.968018+0000","last_clean":"2024-01-15T18:17:30.968018+0000","last_became_active":"2024-01-15T18:17:30.957492+0000","last_became_peered":"2024-01-15T18:17:30.957492+0000","last_unstale":"2024-01-15T18:17:30.968018+0000","last_undegraded":"2024-01-15T18:17:30.968018+0000","last_fullsized":"2024-01-15T18:17:30.968018+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:45.837765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.976097+0000","last_change":"2024-01-15T18:17:30.972054+0000","last_active":"2024-01-15T18:17:30.976097+0000","last_peered":"2024-01-15T18:17:30.976097+0000","last_clean":"2024-01-15T18:17:30.976097+0000","last_became_active":"2024-01-15T18:17:30.971356+0000","last_became_peered":"2024-01-15T18:17:30.971356+0000","last_unstale":"2024-01-15T18:17:30.976097+0000","last_undegraded":"2024-01-15T18:17:30.976097+0000","last_fullsized":"2024-01-15T18:17:30.976097+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964087+0000","last_change":"2024-01-15T18:17:30.953830+0000","last_active":"2024-01-15T18:17:30.964087+0000","last_peered":"2024-01-15T18:17:30.964087+0000","last_clean":"2024-01-15T18:17:30.964087+0000","last_became_active":"2024-01-15T18:17:30.953494+0000","last_became_peered":"2024-01-15T18:17:30.953494+0000","last_unstale":"2024-01-15T18:17:30.964087+0000","last_undegraded":"2024-01-15T18:17:30.964087+0000","last_fullsized":"2024-01-15T18:17:30.964087+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.329164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964361+0000","last_change":"2024-01-15T18:17:30.956596+0000","last_active":"2024-01-15T18:17:30.964361+0000","last_peered":"2024-01-15T18:17:30.964361+0000","last_clean":"2024-01-15T18:17:30.964361+0000","last_became_active":"2024-01-15T18:17:30.954556+0000","last_became_peered":"2024-01-15T18:17:30.954556+0000","last_unstale":"2024-01-15T18:17:30.964361+0000","last_undegraded":"2024-01-15T18:17:30.964361+0000","last_fullsized":"2024-01-15T18:17:30.964361+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:51.759595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964212+0000","last_change":"2024-01-15T18:17:30.955196+0000","last_active":"2024-01-15T18:17:30.964212+0000","last_peered":"2024-01-15T18:17:30.964212+0000","last_clean":"2024-01-15T18:17:30.964212+0000","last_became_active":"2024-01-15T18:17:30.954775+0000","last_became_peered":"2024-01-15T18:17:30.954775+0000","last_unstale":"2024-01-15T18:17:30.964212+0000","last_undegraded":"2024-01-15T18:17:30.964212+0000","last_fullsized":"2024-01-15T18:17:30.964212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:16:49.180169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977325+0000","last_change":"2024-01-15T18:17:30.969882+0000","last_active":"2024-01-15T18:17:30.977325+0000","last_peered":"2024-01-15T18:17:30.977325+0000","last_clean":"2024-01-15T18:17:30.977325+0000","last_became_active":"2024-01-15T18:17:30.969568+0000","last_became_peered":"2024-01-15T18:17:30.969568+0000","last_unstale":"2024-01-15T18:17:30.977325+0000","last_undegraded":"2024-01-15T18:17:30.977325+0000","last_fullsized":"2024-01-15T18:17:30.977325+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:16:49.180169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.961112+0000","last_change":"2024-01-15T18:17:30.951953+0000","last_active":"2024-01-15T18:17:30.961112+0000","last_peered":"2024-01-15T18:17:30.961112+0000","last_clean":"2024-01-15T18:17:30.961112+0000","last_became_active":"2024-01-15T18:17:30.951643+0000","last_became_peered":"2024-01-15T18:17:30.951643+0000","last_unstale":"2024-01-15T18:17:30.961112+0000","last_undegraded":"2024-01-15T18:17:30.961112+0000","last_fullsized":"2024-01-15T18:17:30.961112+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:34.425305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.945943+0000","last_change":"2024-01-15T18:17:30.940607+0000","last_active":"2024-01-15T18:17:30.945943+0000","last_peered":"2024-01-15T18:17:30.945943+0000","last_clean":"2024-01-15T18:17:30.945943+0000","last_became_active":"2024-01-15T18:17:30.940333+0000","last_became_peered":"2024-01-15T18:17:30.940333+0000","last_unstale":"2024-01-15T18:17:30.945943+0000","last_undegraded":"2024-01-15T18:17:30.945943+0000","last_fullsized":"2024-01-15T18:17:30.945943+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:01.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.965784+0000","last_change":"2024-01-15T18:17:30.962459+0000","last_active":"2024-01-15T18:17:30.965784+0000","last_peered":"2024-01-15T18:17:30.965784+0000","last_clean":"2024-01-15T18:17:30.965784+0000","last_became_active":"2024-01-15T18:17:30.961797+0000","last_became_peered":"2024-01-15T18:17:30.961797+0000","last_unstale":"2024-01-15T18:17:30.965784+0000","last_undegraded":"2024-01-15T18:17:30.965784+0000","last_fullsized":"2024-01-15T18:17:30.965784+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:17.557216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949272+0000","last_change":"2024-01-15T18:17:30.942808+0000","last_active":"2024-01-15T18:17:30.949272+0000","last_peered":"2024-01-15T18:17:30.949272+0000","last_clean":"2024-01-15T18:17:30.949272+0000","last_became_active":"2024-01-15T18:17:30.942511+0000","last_became_peered":"2024-01-15T18:17:30.942511+0000","last_unstale":"2024-01-15T18:17:30.949272+0000","last_undegraded":"2024-01-15T18:17:30.949272+0000","last_fullsized":"2024-01-15T18:17:30.949272+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.970148+0000","last_change":"2024-01-15T18:17:30.966690+0000","last_active":"2024-01-15T18:17:30.970148+0000","last_peered":"2024-01-15T18:17:30.970148+0000","last_clean":"2024-01-15T18:17:30.970148+0000","last_became_active":"2024-01-15T18:17:30.966411+0000","last_became_peered":"2024-01-15T18:17:30.966411+0000","last_unstale":"2024-01-15T18:17:30.970148+0000","last_undegraded":"2024-01-15T18:17:30.970148+0000","last_fullsized":"2024-01-15T18:17:30.970148+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.329164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978429+0000","last_change":"2024-01-15T18:17:30.975937+0000","last_active":"2024-01-15T18:17:30.978429+0000","last_peered":"2024-01-15T18:17:30.978429+0000","last_clean":"2024-01-15T18:17:30.978429+0000","last_became_active":"2024-01-15T18:17:30.974548+0000","last_became_peered":"2024-01-15T18:17:30.974548+0000","last_unstale":"2024-01-15T18:17:30.978429+0000","last_undegraded":"2024-01-15T18:17:30.978429+0000","last_fullsized":"2024-01-15T18:17:30.978429+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:16.852132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958545+0000","last_change":"2024-01-15T18:17:30.956101+0000","last_active":"2024-01-15T18:17:30.958545+0000","last_peered":"2024-01-15T18:17:30.958545+0000","last_clean":"2024-01-15T18:17:30.958545+0000","last_became_active":"2024-01-15T18:17:30.955832+0000","last_became_peered":"2024-01-15T18:17:30.955832+0000","last_unstale":"2024-01-15T18:17:30.958545+0000","last_undegraded":"2024-01-15T18:17:30.958545+0000","last_fullsized":"2024-01-15T18:17:30.958545+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:13.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964579+0000","last_change":"2024-01-15T18:17:30.958977+0000","last_active":"2024-01-15T18:17:30.964579+0000","last_peered":"2024-01-15T18:17:30.964579+0000","last_clean":"2024-01-15T18:17:30.964579+0000","last_became_active":"2024-01-15T18:17:30.958666+0000","last_became_peered":"2024-01-15T18:17:30.958666+0000","last_unstale":"2024-01-15T18:17:30.964579+0000","last_undegraded":"2024-01-15T18:17:30.964579+0000","last_fullsized":"2024-01-15T18:17:30.964579+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:40.172255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.969175+0000","last_change":"2024-01-15T18:17:30.962228+0000","last_active":"2024-01-15T18:17:30.969175+0000","last_peered":"2024-01-15T18:17:30.969175+0000","last_clean":"2024-01-15T18:17:30.969175+0000","last_became_active":"2024-01-15T18:17:30.961930+0000","last_became_peered":"2024-01-15T18:17:30.961930+0000","last_unstale":"2024-01-15T18:17:30.969175+0000","last_undegraded":"2024-01-15T18:17:30.969175+0000","last_fullsized":"2024-01-15T18:17:30.969175+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:13:41.331545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.976366+0000","last_change":"2024-01-15T18:17:30.975553+0000","last_active":"2024-01-15T18:17:30.976366+0000","last_peered":"2024-01-15T18:17:30.976366+0000","last_clean":"2024-01-15T18:17:30.976366+0000","last_became_active":"2024-01-15T18:17:30.974646+0000","last_became_peered":"2024-01-15T18:17:30.974646+0000","last_unstale":"2024-01-15T18:17:30.976366+0000","last_undegraded":"2024-01-15T18:17:30.976366+0000","last_fullsized":"2024-01-15T18:17:30.976366+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:20.452712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978083+0000","last_change":"2024-01-15T18:17:30.973968+0000","last_active":"2024-01-15T18:17:30.978083+0000","last_peered":"2024-01-15T18:17:30.978083+0000","last_clean":"2024-01-15T18:17:30.978083+0000","last_became_active":"2024-01-15T18:17:30.970746+0000","last_became_peered":"2024-01-15T18:17:30.970746+0000","last_unstale":"2024-01-15T18:17:30.978083+0000","last_undegraded":"2024-01-15T18:17:30.978083+0000","last_fullsized":"2024-01-15T18:17:30.978083+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:13:42.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964673+0000","last_change":"2024-01-15T18:17:30.953059+0000","last_active":"2024-01-15T18:17:30.964673+0000","last_peered":"2024-01-15T18:17:30.964673+0000","last_clean":"2024-01-15T18:17:30.964673+0000","last_became_active":"2024-01-15T18:17:30.952775+0000","last_became_peered":"2024-01-15T18:17:30.952775+0000","last_unstale":"2024-01-15T18:17:30.964673+0000","last_undegraded":"2024-01-15T18:17:30.964673+0000","last_fullsized":"2024-01-15T18:17:30.964673+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:20.452712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977964+0000","last_change":"2024-01-15T18:17:30.971220+0000","last_active":"2024-01-15T18:17:30.977964+0000","last_peered":"2024-01-15T18:17:30.977964+0000","last_clean":"2024-01-15T18:17:30.977964+0000","last_became_active":"2024-01-15T18:17:30.970836+0000","last_became_peered":"2024-01-15T18:17:30.970836+0000","last_unstale":"2024-01-15T18:17:30.977964+0000","last_undegraded":"2024-01-15T18:17:30.977964+0000","last_fullsized":"2024-01-15T18:17:30.977964+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:55.980450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963256+0000","last_change":"2024-01-15T18:17:30.960737+0000","last_active":"2024-01-15T18:17:30.963256+0000","last_peered":"2024-01-15T18:17:30.963256+0000","last_clean":"2024-01-15T18:17:30.963256+0000","last_became_active":"2024-01-15T18:17:30.960448+0000","last_became_peered":"2024-01-15T18:17:30.960448+0000","last_unstale":"2024-01-15T18:17:30.963256+0000","last_undegraded":"2024-01-15T18:17:30.963256+0000","last_fullsized":"2024-01-15T18:17:30.963256+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963819+0000","last_change":"2024-01-15T18:17:30.961193+0000","last_active":"2024-01-15T18:17:30.963819+0000","last_peered":"2024-01-15T18:17:30.963819+0000","last_clean":"2024-01-15T18:17:30.963819+0000","last_became_active":"2024-01-15T18:17:30.960904+0000","last_became_peered":"2024-01-15T18:17:30.960904+0000","last_unstale":"2024-01-15T18:17:30.963819+0000","last_undegraded":"2024-01-15T18:17:30.963819+0000","last_fullsized":"2024-01-15T18:17:30.963819+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:19.861186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963997+0000","last_change":"2024-01-15T18:17:30.950164+0000","last_active":"2024-01-15T18:17:30.963997+0000","last_peered":"2024-01-15T18:17:30.963997+0000","last_clean":"2024-01-15T18:17:30.963997+0000","last_became_active":"2024-01-15T18:17:30.949893+0000","last_became_peered":"2024-01-15T18:17:30.949893+0000","last_unstale":"2024-01-15T18:17:30.963997+0000","last_undegraded":"2024-01-15T18:17:30.963997+0000","last_fullsized":"2024-01-15T18:17:30.963997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:31:12.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.948848+0000","last_change":"2024-01-15T18:17:30.941972+0000","last_active":"2024-01-15T18:17:30.948848+0000","last_peered":"2024-01-15T18:17:30.948848+0000","last_clean":"2024-01-15T18:17:30.948848+0000","last_became_active":"2024-01-15T18:17:30.941658+0000","last_became_peered":"2024-01-15T18:17:30.941658+0000","last_unstale":"2024-01-15T18:17:30.948848+0000","last_undegraded":"2024-01-15T18:17:30.948848+0000","last_fullsized":"2024-01-15T18:17:30.948848+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:21.130381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.967931+0000","last_change":"2024-01-15T18:17:30.959819+0000","last_active":"2024-01-15T18:17:30.967931+0000","last_peered":"2024-01-15T18:17:30.967931+0000","last_clean":"2024-01-15T18:17:30.967931+0000","last_became_active":"2024-01-15T18:17:30.956562+0000","last_became_peered":"2024-01-15T18:17:30.956562+0000","last_unstale":"2024-01-15T18:17:30.967931+0000","last_undegraded":"2024-01-15T18:17:30.967931+0000","last_fullsized":"2024-01-15T18:17:30.967931+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:51.759595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968265+0000","last_change":"2024-01-15T18:17:30.967418+0000","last_active":"2024-01-15T18:17:30.968265+0000","last_peered":"2024-01-15T18:17:30.968265+0000","last_clean":"2024-01-15T18:17:30.968265+0000","last_became_active":"2024-01-15T18:17:30.967115+0000","last_became_peered":"2024-01-15T18:17:30.967115+0000","last_unstale":"2024-01-15T18:17:30.968265+0000","last_undegraded":"2024-01-15T18:17:30.968265+0000","last_fullsized":"2024-01-15T18:17:30.968265+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:19.861186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963631+0000","last_change":"2024-01-15T18:17:30.955557+0000","last_active":"2024-01-15T18:17:30.963631+0000","last_peered":"2024-01-15T18:17:30.963631+0000","last_clean":"2024-01-15T18:17:30.963631+0000","last_became_active":"2024-01-15T18:17:30.955265+0000","last_became_peered":"2024-01-15T18:17:30.955265+0000","last_unstale":"2024-01-15T18:17:30.963631+0000","last_undegraded":"2024-01-15T18:17:30.963631+0000","last_fullsized":"2024-01-15T18:17:30.963631+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:43.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.960005+0000","last_change":"2024-01-15T18:17:30.954073+0000","last_active":"2024-01-15T18:17:30.960005+0000","last_peered":"2024-01-15T18:17:30.960005+0000","last_clean":"2024-01-15T18:17:30.960005+0000","last_became_active":"2024-01-15T18:17:30.953811+0000","last_became_peered":"2024-01-15T18:17:30.953811+0000","last_unstale":"2024-01-15T18:17:30.960005+0000","last_undegraded":"2024-01-15T18:17:30.960005+0000","last_fullsized":"2024-01-15T18:17:30.960005+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:28.174662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978407+0000","last_change":"2024-01-15T18:17:30.975772+0000","last_active":"2024-01-15T18:17:30.978407+0000","last_peered":"2024-01-15T18:17:30.978407+0000","last_clean":"2024-01-15T18:17:30.978407+0000","last_became_active":"2024-01-15T18:17:30.973202+0000","last_became_peered":"2024-01-15T18:17:30.973202+0000","last_unstale":"2024-01-15T18:17:30.978407+0000","last_undegraded":"2024-01-15T18:17:30.978407+0000","last_fullsized":"2024-01-15T18:17:30.978407+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:53:32.204347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958527+0000","last_change":"2024-01-15T18:17:30.955625+0000","last_active":"2024-01-15T18:17:30.958527+0000","last_peered":"2024-01-15T18:17:30.958527+0000","last_clean":"2024-01-15T18:17:30.958527+0000","last_became_active":"2024-01-15T18:17:30.955351+0000","last_became_peered":"2024-01-15T18:17:30.955351+0000","last_unstale":"2024-01-15T18:17:30.958527+0000","last_undegraded":"2024-01-15T18:17:30.958527+0000","last_fullsized":"2024-01-15T18:17:30.958527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:26.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.966041+0000","last_change":"2024-01-15T18:17:30.957564+0000","last_active":"2024-01-15T18:17:30.966041+0000","last_peered":"2024-01-15T18:17:30.966041+0000","last_clean":"2024-01-15T18:17:30.966041+0000","last_became_active":"2024-01-15T18:17:30.957078+0000","last_became_peered":"2024-01-15T18:17:30.957078+0000","last_unstale":"2024-01-15T18:17:30.966041+0000","last_undegraded":"2024-01-15T18:17:30.966041+0000","last_fullsized":"2024-01-15T18:17:30.966041+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:18.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978055+0000","last_change":"2024-01-15T18:17:30.973606+0000","last_active":"2024-01-15T18:17:30.978055+0000","last_peered":"2024-01-15T18:17:30.978055+0000","last_clean":"2024-01-15T18:17:30.978055+0000","last_became_active":"2024-01-15T18:17:30.971413+0000","last_became_peered":"2024-01-15T18:17:30.971413+0000","last_unstale":"2024-01-15T18:17:30.978055+0000","last_undegraded":"2024-01-15T18:17:30.978055+0000","last_fullsized":"2024-01-15T18:17:30.978055+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:28.411435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.973053+0000","last_change":"2024-01-15T18:17:29.942253+0000","last_active":"2024-01-15T18:17:30.973053+0000","last_peered":"2024-01-15T18:17:30.973053+0000","last_clean":"2024-01-15T18:17:30.973053+0000","last_became_active":"2024-01-15T18:17:29.941981+0000","last_became_peered":"2024-01-15T18:17:29.941981+0000","last_unstale":"2024-01-15T18:17:30.973053+0000","last_undegraded":"2024-01-15T18:17:30.973053+0000","last_fullsized":"2024-01-15T18:17:30.973053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:00.199422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968710+0000","last_change":"2024-01-15T18:17:30.965909+0000","last_active":"2024-01-15T18:17:30.968710+0000","last_peered":"2024-01-15T18:17:30.968710+0000","last_clean":"2024-01-15T18:17:30.968710+0000","last_became_active":"2024-01-15T18:17:30.965640+0000","last_became_peered":"2024-01-15T18:17:30.965640+0000","last_unstale":"2024-01-15T18:17:30.968710+0000","last_undegraded":"2024-01-15T18:17:30.968710+0000","last_fullsized":"2024-01-15T18:17:30.968710+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:43.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964404+0000","last_change":"2024-01-15T18:17:29.941599+0000","last_active":"2024-01-15T18:17:30.964404+0000","last_peered":"2024-01-15T18:17:30.964404+0000","last_clean":"2024-01-15T18:17:30.964404+0000","last_became_active":"2024-01-15T18:17:29.941311+0000","last_became_peered":"2024-01-15T18:17:29.941311+0000","last_unstale":"2024-01-15T18:17:30.964404+0000","last_undegraded":"2024-01-15T18:17:30.964404+0000","last_fullsized":"2024-01-15T18:17:30.964404+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:37:27.097439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.962866+0000","last_change":"2024-01-15T18:17:30.958196+0000","last_active":"2024-01-15T18:17:30.962866+0000","last_peered":"2024-01-15T18:17:30.962866+0000","last_clean":"2024-01-15T18:17:30.962866+0000","last_became_active":"2024-01-15T18:17:30.957916+0000","last_became_peered":"2024-01-15T18:17:30.957916+0000","last_unstale":"2024-01-15T18:17:30.962866+0000","last_undegraded":"2024-01-15T18:17:30.962866+0000","last_fullsized":"2024-01-15T18:17:30.962866+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:05:40.291013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.971835+0000","last_change":"2024-01-15T18:17:29.950920+0000","last_active":"2024-01-15T18:17:30.971835+0000","last_peered":"2024-01-15T18:17:30.971835+0000","last_clean":"2024-01-15T18:17:30.971835+0000","last_became_active":"2024-01-15T18:17:29.950548+0000","last_became_peered":"2024-01-15T18:17:29.950548+0000","last_unstale":"2024-01-15T18:17:30.971835+0000","last_undegraded":"2024-01-15T18:17:30.971835+0000","last_fullsized":"2024-01-15T18:17:30.971835+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:33.414707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949399+0000","last_change":"2024-01-15T18:17:30.942088+0000","last_active":"2024-01-15T18:17:30.949399+0000","last_peered":"2024-01-15T18:17:30.949399+0000","last_clean":"2024-01-15T18:17:30.949399+0000","last_became_active":"2024-01-15T18:17:30.941775+0000","last_became_peered":"2024-01-15T18:17:30.941775+0000","last_unstale":"2024-01-15T18:17:30.949399+0000","last_undegraded":"2024-01-15T18:17:30.949399+0000","last_fullsized":"2024-01-15T18:17:30.949399+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:00.192627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.972869+0000","last_change":"2024-01-15T18:17:29.943225+0000","last_active":"2024-01-15T18:17:30.972869+0000","last_peered":"2024-01-15T18:17:30.972869+0000","last_clean":"2024-01-15T18:17:30.972869+0000","last_became_active":"2024-01-15T18:17:29.940937+0000","last_became_peered":"2024-01-15T18:17:29.940937+0000","last_unstale":"2024-01-15T18:17:30.972869+0000","last_undegraded":"2024-01-15T18:17:30.972869+0000","last_fullsized":"2024-01-15T18:17:30.972869+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:16.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.962563+0000","last_change":"2024-01-15T18:17:30.954274+0000","last_active":"2024-01-15T18:17:30.962563+0000","last_peered":"2024-01-15T18:17:30.962563+0000","last_clean":"2024-01-15T18:17:30.962563+0000","last_became_active":"2024-01-15T18:17:30.953891+0000","last_became_peered":"2024-01-15T18:17:30.953891+0000","last_unstale":"2024-01-15T18:17:30.962563+0000","last_undegraded":"2024-01-15T18:17:30.962563+0000","last_fullsized":"2024-01-15T18:17:30.962563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.329164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.971078+0000","last_change":"2024-01-15T18:17:29.938531+0000","last_active":"2024-01-15T18:17:30.971078+0000","last_peered":"2024-01-15T18:17:30.971078+0000","last_clean":"2024-01-15T18:17:30.971078+0000","last_became_active":"2024-01-15T18:17:29.938134+0000","last_became_peered":"2024-01-15T18:17:29.938134+0000","last_unstale":"2024-01-15T18:17:30.971078+0000","last_undegraded":"2024-01-15T18:17:30.971078+0000","last_fullsized":"2024-01-15T18:17:30.971078+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:41:33.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963949+0000","last_change":"2024-01-15T18:17:30.950683+0000","last_active":"2024-01-15T18:17:30.963949+0000","last_peered":"2024-01-15T18:17:30.963949+0000","last_clean":"2024-01-15T18:17:30.963949+0000","last_became_active":"2024-01-15T18:17:30.950259+0000","last_became_peered":"2024-01-15T18:17:30.950259+0000","last_unstale":"2024-01-15T18:17:30.963949+0000","last_undegraded":"2024-01-15T18:17:30.963949+0000","last_fullsized":"2024-01-15T18:17:30.963949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:28.174662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.966846+0000","last_change":"2024-01-15T18:17:29.945416+0000","last_active":"2024-01-15T18:17:30.966846+0000","last_peered":"2024-01-15T18:17:30.966846+0000","last_clean":"2024-01-15T18:17:30.966846+0000","last_became_active":"2024-01-15T18:17:29.945114+0000","last_became_peered":"2024-01-15T18:17:29.945114+0000","last_unstale":"2024-01-15T18:17:30.966846+0000","last_undegraded":"2024-01-15T18:17:30.966846+0000","last_fullsized":"2024-01-15T18:17:30.966846+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:14:59.182029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.959200+0000","last_change":"2024-01-15T18:17:30.953160+0000","last_active":"2024-01-15T18:17:30.959200+0000","last_peered":"2024-01-15T18:17:30.959200+0000","last_clean":"2024-01-15T18:17:30.959200+0000","last_became_active":"2024-01-15T18:17:30.952788+0000","last_became_peered":"2024-01-15T18:17:30.952788+0000","last_unstale":"2024-01-15T18:17:30.959200+0000","last_undegraded":"2024-01-15T18:17:30.959200+0000","last_fullsized":"2024-01-15T18:17:30.959200+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.692797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.932814+0000","last_change":"2024-01-15T18:17:29.935434+0000","last_active":"2024-01-15T18:17:30.932814+0000","last_peered":"2024-01-15T18:17:30.932814+0000","last_clean":"2024-01-15T18:17:30.932814+0000","last_became_active":"2024-01-15T18:17:29.934593+0000","last_became_peered":"2024-01-15T18:17:29.934593+0000","last_unstale":"2024-01-15T18:17:30.932814+0000","last_undegraded":"2024-01-15T18:17:30.932814+0000","last_fullsized":"2024-01-15T18:17:30.932814+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:33.414707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.957863+0000","last_change":"2024-01-15T18:17:30.949748+0000","last_active":"2024-01-15T18:17:30.957863+0000","last_peered":"2024-01-15T18:17:30.957863+0000","last_clean":"2024-01-15T18:17:30.957863+0000","last_became_active":"2024-01-15T18:17:30.949302+0000","last_became_peered":"2024-01-15T18:17:30.949302+0000","last_unstale":"2024-01-15T18:17:30.957863+0000","last_undegraded":"2024-01-15T18:17:30.957863+0000","last_fullsized":"2024-01-15T18:17:30.957863+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:36:40.270304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.943297+0000","last_change":"2024-01-15T18:17:29.938253+0000","last_active":"2024-01-15T18:17:30.943297+0000","last_peered":"2024-01-15T18:17:30.943297+0000","last_clean":"2024-01-15T18:17:30.943297+0000","last_became_active":"2024-01-15T18:17:29.937847+0000","last_became_peered":"2024-01-15T18:17:29.937847+0000","last_unstale":"2024-01-15T18:17:30.943297+0000","last_undegraded":"2024-01-15T18:17:30.943297+0000","last_fullsized":"2024-01-15T18:17:30.943297+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:45.493577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975627+0000","last_change":"2024-01-15T18:17:30.964507+0000","last_active":"2024-01-15T18:17:30.975627+0000","last_peered":"2024-01-15T18:17:30.975627+0000","last_clean":"2024-01-15T18:17:30.975627+0000","last_became_active":"2024-01-15T18:17:30.964235+0000","last_became_peered":"2024-01-15T18:17:30.964235+0000","last_unstale":"2024-01-15T18:17:30.975627+0000","last_undegraded":"2024-01-15T18:17:30.975627+0000","last_fullsized":"2024-01-15T18:17:30.975627+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:21.130381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.942546+0000","last_change":"2024-01-15T18:17:29.945885+0000","last_active":"2024-01-15T18:17:30.942546+0000","last_peered":"2024-01-15T18:17:30.942546+0000","last_clean":"2024-01-15T18:17:30.942546+0000","last_became_active":"2024-01-15T18:17:29.945535+0000","last_became_peered":"2024-01-15T18:17:29.945535+0000","last_unstale":"2024-01-15T18:17:30.942546+0000","last_undegraded":"2024-01-15T18:17:30.942546+0000","last_fullsized":"2024-01-15T18:17:30.942546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:31:43.659519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.961092+0000","last_change":"2024-01-15T18:17:30.950971+0000","last_active":"2024-01-15T18:17:30.961092+0000","last_peered":"2024-01-15T18:17:30.961092+0000","last_clean":"2024-01-15T18:17:30.961092+0000","last_became_active":"2024-01-15T18:17:30.948559+0000","last_became_peered":"2024-01-15T18:17:30.948559+0000","last_unstale":"2024-01-15T18:17:30.961092+0000","last_undegraded":"2024-01-15T18:17:30.961092+0000","last_fullsized":"2024-01-15T18:17:30.961092+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:16.106025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.950596+0000","last_change":"2024-01-15T18:17:29.943048+0000","last_active":"2024-01-15T18:17:30.950596+0000","last_peered":"2024-01-15T18:17:30.950596+0000","last_clean":"2024-01-15T18:17:30.950596+0000","last_became_active":"2024-01-15T18:17:29.942602+0000","last_became_peered":"2024-01-15T18:17:29.942602+0000","last_unstale":"2024-01-15T18:17:30.950596+0000","last_undegraded":"2024-01-15T18:17:30.950596+0000","last_fullsized":"2024-01-15T18:17:30.950596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:14:59.182029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.950902+0000","last_change":"2024-01-15T18:17:30.945486+0000","last_active":"2024-01-15T18:17:30.950902+0000","last_peered":"2024-01-15T18:17:30.950902+0000","last_clean":"2024-01-15T18:17:30.950902+0000","last_became_active":"2024-01-15T18:17:30.945067+0000","last_became_peered":"2024-01-15T18:17:30.945067+0000","last_unstale":"2024-01-15T18:17:30.950902+0000","last_undegraded":"2024-01-15T18:17:30.950902+0000","last_fullsized":"2024-01-15T18:17:30.950902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:34.709281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.967978+0000","last_change":"2024-01-15T18:17:29.933095+0000","last_active":"2024-01-15T18:17:30.967978+0000","last_peered":"2024-01-15T18:17:30.967978+0000","last_clean":"2024-01-15T18:17:30.967978+0000","last_became_active":"2024-01-15T18:17:29.932824+0000","last_became_peered":"2024-01-15T18:17:29.932824+0000","last_unstale":"2024-01-15T18:17:30.967978+0000","last_undegraded":"2024-01-15T18:17:30.967978+0000","last_fullsized":"2024-01-15T18:17:30.967978+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:37.286051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964306+0000","last_change":"2024-01-15T18:17:30.955749+0000","last_active":"2024-01-15T18:17:30.964306+0000","last_peered":"2024-01-15T18:17:30.964306+0000","last_clean":"2024-01-15T18:17:30.964306+0000","last_became_active":"2024-01-15T18:17:30.955428+0000","last_became_peered":"2024-01-15T18:17:30.955428+0000","last_unstale":"2024-01-15T18:17:30.964306+0000","last_undegraded":"2024-01-15T18:17:30.964306+0000","last_fullsized":"2024-01-15T18:17:30.964306+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:11.421513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.972210+0000","last_change":"2024-01-15T18:17:29.929962+0000","last_active":"2024-01-15T18:17:30.972210+0000","last_peered":"2024-01-15T18:17:30.972210+0000","last_clean":"2024-01-15T18:17:30.972210+0000","last_became_active":"2024-01-15T18:17:29.929647+0000","last_became_peered":"2024-01-15T18:17:29.929647+0000","last_unstale":"2024-01-15T18:17:30.972210+0000","last_undegraded":"2024-01-15T18:17:30.972210+0000","last_fullsized":"2024-01-15T18:17:30.972210+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:19:10.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.961235+0000","last_change":"2024-01-15T18:17:30.956453+0000","last_active":"2024-01-15T18:17:30.961235+0000","last_peered":"2024-01-15T18:17:30.961235+0000","last_clean":"2024-01-15T18:17:30.961235+0000","last_became_active":"2024-01-15T18:17:30.956174+0000","last_became_peered":"2024-01-15T18:17:30.956174+0000","last_unstale":"2024-01-15T18:17:30.961235+0000","last_undegraded":"2024-01-15T18:17:30.961235+0000","last_fullsized":"2024-01-15T18:17:30.961235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:34.709281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.976514+0000","last_change":"2024-01-15T18:17:29.948422+0000","last_active":"2024-01-15T18:17:30.976514+0000","last_peered":"2024-01-15T18:17:30.976514+0000","last_clean":"2024-01-15T18:17:30.976514+0000","last_became_active":"2024-01-15T18:17:29.948121+0000","last_became_peered":"2024-01-15T18:17:29.948121+0000","last_unstale":"2024-01-15T18:17:30.976514+0000","last_undegraded":"2024-01-15T18:17:30.976514+0000","last_fullsized":"2024-01-15T18:17:30.976514+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:28.921159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.974239+0000","last_change":"2024-01-15T18:17:30.972425+0000","last_active":"2024-01-15T18:17:30.974239+0000","last_peered":"2024-01-15T18:17:30.974239+0000","last_clean":"2024-01-15T18:17:30.974239+0000","last_became_active":"2024-01-15T18:17:30.972016+0000","last_became_peered":"2024-01-15T18:17:30.972016+0000","last_unstale":"2024-01-15T18:17:30.974239+0000","last_undegraded":"2024-01-15T18:17:30.974239+0000","last_fullsized":"2024-01-15T18:17:30.974239+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:30.932645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958514+0000","last_change":"2024-01-15T18:17:29.950742+0000","last_active":"2024-01-15T18:17:30.958514+0000","last_peered":"2024-01-15T18:17:30.958514+0000","last_clean":"2024-01-15T18:17:30.958514+0000","last_became_active":"2024-01-15T18:17:29.950468+0000","last_became_peered":"2024-01-15T18:17:29.950468+0000","last_unstale":"2024-01-15T18:17:30.958514+0000","last_undegraded":"2024-01-15T18:17:30.958514+0000","last_fullsized":"2024-01-15T18:17:30.958514+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:07:10.425098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949462+0000","last_change":"2024-01-15T18:17:30.941430+0000","last_active":"2024-01-15T18:17:30.949462+0000","last_peered":"2024-01-15T18:17:30.949462+0000","last_clean":"2024-01-15T18:17:30.949462+0000","last_became_active":"2024-01-15T18:17:30.941040+0000","last_became_peered":"2024-01-15T18:17:30.941040+0000","last_unstale":"2024-01-15T18:17:30.949462+0000","last_undegraded":"2024-01-15T18:17:30.949462+0000","last_fullsized":"2024-01-15T18:17:30.949462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:43.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.935679+0000","last_change":"2024-01-15T18:17:29.926806+0000","last_active":"2024-01-15T18:17:30.935679+0000","last_peered":"2024-01-15T18:17:30.935679+0000","last_clean":"2024-01-15T18:17:30.935679+0000","last_became_active":"2024-01-15T18:17:29.926515+0000","last_became_peered":"2024-01-15T18:17:29.926515+0000","last_unstale":"2024-01-15T18:17:30.935679+0000","last_undegraded":"2024-01-15T18:17:30.935679+0000","last_fullsized":"2024-01-15T18:17:30.935679+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:00.199422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977998+0000","last_change":"2024-01-15T18:17:30.970932+0000","last_active":"2024-01-15T18:17:30.977998+0000","last_peered":"2024-01-15T18:17:30.977998+0000","last_clean":"2024-01-15T18:17:30.977998+0000","last_became_active":"2024-01-15T18:17:30.970588+0000","last_became_peered":"2024-01-15T18:17:30.970588+0000","last_unstale":"2024-01-15T18:17:30.977998+0000","last_undegraded":"2024-01-15T18:17:30.977998+0000","last_fullsized":"2024-01-15T18:17:30.977998+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:44:40.436564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.973403+0000","last_change":"2024-01-15T18:17:29.943639+0000","last_active":"2024-01-15T18:17:30.973403+0000","last_peered":"2024-01-15T18:17:30.973403+0000","last_clean":"2024-01-15T18:17:30.973403+0000","last_became_active":"2024-01-15T18:17:29.940051+0000","last_became_peered":"2024-01-15T18:17:29.940051+0000","last_unstale":"2024-01-15T18:17:30.973403+0000","last_undegraded":"2024-01-15T18:17:30.973403+0000","last_fullsized":"2024-01-15T18:17:30.973403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:38:07.832563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958158+0000","last_change":"2024-01-15T18:17:30.951848+0000","last_active":"2024-01-15T18:17:30.958158+0000","last_peered":"2024-01-15T18:17:30.958158+0000","last_clean":"2024-01-15T18:17:30.958158+0000","last_became_active":"2024-01-15T18:17:30.951562+0000","last_became_peered":"2024-01-15T18:17:30.951562+0000","last_unstale":"2024-01-15T18:17:30.958158+0000","last_undegraded":"2024-01-15T18:17:30.958158+0000","last_fullsized":"2024-01-15T18:17:30.958158+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:00.192627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.970270+0000","last_change":"2024-01-15T18:17:29.944762+0000","last_active":"2024-01-15T18:17:30.970270+0000","last_peered":"2024-01-15T18:17:30.970270+0000","last_clean":"2024-01-15T18:17:30.970270+0000","last_became_active":"2024-01-15T18:17:29.944486+0000","last_became_peered":"2024-01-15T18:17:29.944486+0000","last_unstale":"2024-01-15T18:17:30.970270+0000","last_undegraded":"2024-01-15T18:17:30.970270+0000","last_fullsized":"2024-01-15T18:17:30.970270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:20.119784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.951164+0000","last_change":"2024-01-15T18:17:30.940311+0000","last_active":"2024-01-15T18:17:30.951164+0000","last_peered":"2024-01-15T18:17:30.951164+0000","last_clean":"2024-01-15T18:17:30.951164+0000","last_became_active":"2024-01-15T18:17:30.939949+0000","last_became_peered":"2024-01-15T18:17:30.939949+0000","last_unstale":"2024-01-15T18:17:30.951164+0000","last_undegraded":"2024-01-15T18:17:30.951164+0000","last_fullsized":"2024-01-15T18:17:30.951164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:13.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964273+0000","last_change":"2024-01-15T18:17:29.934554+0000","last_active":"2024-01-15T18:17:30.964273+0000","last_peered":"2024-01-15T18:17:30.964273+0000","last_clean":"2024-01-15T18:17:30.964273+0000","last_became_active":"2024-01-15T18:17:29.933273+0000","last_became_peered":"2024-01-15T18:17:29.933273+0000","last_unstale":"2024-01-15T18:17:30.964273+0000","last_undegraded":"2024-01-15T18:17:30.964273+0000","last_fullsized":"2024-01-15T18:17:30.964273+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:13:40.320947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.965363+0000","last_change":"2024-01-15T18:17:30.962718+0000","last_active":"2024-01-15T18:17:30.965363+0000","last_peered":"2024-01-15T18:17:30.965363+0000","last_clean":"2024-01-15T18:17:30.965363+0000","last_became_active":"2024-01-15T18:17:30.962161+0000","last_became_peered":"2024-01-15T18:17:30.962161+0000","last_unstale":"2024-01-15T18:17:30.965363+0000","last_undegraded":"2024-01-15T18:17:30.965363+0000","last_fullsized":"2024-01-15T18:17:30.965363+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:29.158281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.943111+0000","last_change":"2024-01-15T18:17:29.943737+0000","last_active":"2024-01-15T18:17:30.943111+0000","last_peered":"2024-01-15T18:17:30.943111+0000","last_clean":"2024-01-15T18:17:30.943111+0000","last_became_active":"2024-01-15T18:17:29.943450+0000","last_became_peered":"2024-01-15T18:17:29.943450+0000","last_unstale":"2024-01-15T18:17:30.943111+0000","last_undegraded":"2024-01-15T18:17:30.943111+0000","last_fullsized":"2024-01-15T18:17:30.943111+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:37:27.097439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978459+0000","last_change":"2024-01-15T18:17:30.976128+0000","last_active":"2024-01-15T18:17:30.978459+0000","last_peered":"2024-01-15T18:17:30.978459+0000","last_clean":"2024-01-15T18:17:30.978459+0000","last_became_active":"2024-01-15T18:17:30.971307+0000","last_became_peered":"2024-01-15T18:17:30.971307+0000","last_unstale":"2024-01-15T18:17:30.978459+0000","last_undegraded":"2024-01-15T18:17:30.978459+0000","last_fullsized":"2024-01-15T18:17:30.978459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:34.367711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.971476+0000","last_change":"2024-01-15T18:17:29.947014+0000","last_active":"2024-01-15T18:17:30.971476+0000","last_peered":"2024-01-15T18:17:30.971476+0000","last_clean":"2024-01-15T18:17:30.971476+0000","last_became_active":"2024-01-15T18:17:29.946704+0000","last_became_peered":"2024-01-15T18:17:29.946704+0000","last_unstale":"2024-01-15T18:17:30.971476+0000","last_undegraded":"2024-01-15T18:17:30.971476+0000","last_fullsized":"2024-01-15T18:17:30.971476+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:39.161657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977666+0000","last_change":"2024-01-15T18:17:30.970718+0000","last_active":"2024-01-15T18:17:30.977666+0000","last_peered":"2024-01-15T18:17:30.977666+0000","last_clean":"2024-01-15T18:17:30.977666+0000","last_became_active":"2024-01-15T18:17:30.970409+0000","last_became_peered":"2024-01-15T18:17:30.970409+0000","last_unstale":"2024-01-15T18:17:30.977666+0000","last_undegraded":"2024-01-15T18:17:30.977666+0000","last_fullsized":"2024-01-15T18:17:30.977666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:21.191564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.932437+0000","last_change":"2024-01-15T18:17:29.939275+0000","last_active":"2024-01-15T18:17:30.932437+0000","last_peered":"2024-01-15T18:17:30.932437+0000","last_clean":"2024-01-15T18:17:30.932437+0000","last_became_active":"2024-01-15T18:17:29.938950+0000","last_became_peered":"2024-01-15T18:17:29.938950+0000","last_unstale":"2024-01-15T18:17:30.932437+0000","last_undegraded":"2024-01-15T18:17:30.932437+0000","last_fullsized":"2024-01-15T18:17:30.932437+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:42.085028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975811+0000","last_change":"2024-01-15T18:17:30.969018+0000","last_active":"2024-01-15T18:17:30.975811+0000","last_peered":"2024-01-15T18:17:30.975811+0000","last_clean":"2024-01-15T18:17:30.975811+0000","last_became_active":"2024-01-15T18:17:30.968689+0000","last_became_peered":"2024-01-15T18:17:30.968689+0000","last_unstale":"2024-01-15T18:17:30.975811+0000","last_undegraded":"2024-01-15T18:17:30.975811+0000","last_fullsized":"2024-01-15T18:17:30.975811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:57.839436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.931626+0000","last_change":"2024-01-15T18:17:29.936346+0000","last_active":"2024-01-15T18:17:30.931626+0000","last_peered":"2024-01-15T18:17:30.931626+0000","last_clean":"2024-01-15T18:17:30.931626+0000","last_became_active":"2024-01-15T18:17:29.935851+0000","last_became_peered":"2024-01-15T18:17:29.935851+0000","last_unstale":"2024-01-15T18:17:30.931626+0000","last_undegraded":"2024-01-15T18:17:30.931626+0000","last_fullsized":"2024-01-15T18:17:30.931626+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:25.933470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.956365+0000","last_change":"2024-01-15T18:17:30.941874+0000","last_active":"2024-01-15T18:17:30.956365+0000","last_peered":"2024-01-15T18:17:30.956365+0000","last_clean":"2024-01-15T18:17:30.956365+0000","last_became_active":"2024-01-15T18:17:30.941528+0000","last_became_peered":"2024-01-15T18:17:30.941528+0000","last_unstale":"2024-01-15T18:17:30.956365+0000","last_undegraded":"2024-01-15T18:17:30.956365+0000","last_fullsized":"2024-01-15T18:17:30.956365+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:16.106025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.944799+0000","last_change":"2024-01-15T18:17:29.948061+0000","last_active":"2024-01-15T18:17:30.944799+0000","last_peered":"2024-01-15T18:17:30.944799+0000","last_clean":"2024-01-15T18:17:30.944799+0000","last_became_active":"2024-01-15T18:17:29.945780+0000","last_became_peered":"2024-01-15T18:17:29.945780+0000","last_unstale":"2024-01-15T18:17:30.944799+0000","last_undegraded":"2024-01-15T18:17:30.944799+0000","last_fullsized":"2024-01-15T18:17:30.944799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.255371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.967950+0000","last_change":"2024-01-15T18:17:30.960933+0000","last_active":"2024-01-15T18:17:30.967950+0000","last_peered":"2024-01-15T18:17:30.967950+0000","last_clean":"2024-01-15T18:17:30.967950+0000","last_became_active":"2024-01-15T18:17:30.960654+0000","last_became_peered":"2024-01-15T18:17:30.960654+0000","last_unstale":"2024-01-15T18:17:30.967950+0000","last_undegraded":"2024-01-15T18:17:30.967950+0000","last_fullsized":"2024-01-15T18:17:30.967950+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:16.106025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968075+0000","last_change":"2024-01-15T18:17:29.932519+0000","last_active":"2024-01-15T18:17:30.968075+0000","last_peered":"2024-01-15T18:17:30.968075+0000","last_clean":"2024-01-15T18:17:30.968075+0000","last_became_active":"2024-01-15T18:17:29.931627+0000","last_became_peered":"2024-01-15T18:17:29.931627+0000","last_unstale":"2024-01-15T18:17:30.968075+0000","last_undegraded":"2024-01-15T18:17:30.968075+0000","last_fullsized":"2024-01-15T18:17:30.968075+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:33.698683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977580+0000","last_change":"2024-01-15T18:17:30.975437+0000","last_active":"2024-01-15T18:17:30.977580+0000","last_peered":"2024-01-15T18:17:30.977580+0000","last_clean":"2024-01-15T18:17:30.977580+0000","last_became_active":"2024-01-15T18:17:30.975018+0000","last_became_peered":"2024-01-15T18:17:30.975018+0000","last_unstale":"2024-01-15T18:17:30.977580+0000","last_undegraded":"2024-01-15T18:17:30.977580+0000","last_fullsized":"2024-01-15T18:17:30.977580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:11:50.794125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.946360+0000","last_change":"2024-01-15T18:17:29.951765+0000","last_active":"2024-01-15T18:17:30.946360+0000","last_peered":"2024-01-15T18:17:30.946360+0000","last_clean":"2024-01-15T18:17:30.946360+0000","last_became_active":"2024-01-15T18:17:29.951464+0000","last_became_peered":"2024-01-15T18:17:29.951464+0000","last_unstale":"2024-01-15T18:17:30.946360+0000","last_undegraded":"2024-01-15T18:17:30.946360+0000","last_fullsized":"2024-01-15T18:17:30.946360+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:28.147683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.952852+0000","last_change":"2024-01-15T18:17:30.949656+0000","last_active":"2024-01-15T18:17:30.952852+0000","last_peered":"2024-01-15T18:17:30.952852+0000","last_clean":"2024-01-15T18:17:30.952852+0000","last_became_active":"2024-01-15T18:17:30.949383+0000","last_became_peered":"2024-01-15T18:17:30.949383+0000","last_unstale":"2024-01-15T18:17:30.952852+0000","last_undegraded":"2024-01-15T18:17:30.952852+0000","last_fullsized":"2024-01-15T18:17:30.952852+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:23.009053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.942371+0000","last_change":"2024-01-15T18:17:29.940472+0000","last_active":"2024-01-15T18:17:30.942371+0000","last_peered":"2024-01-15T18:17:30.942371+0000","last_clean":"2024-01-15T18:17:30.942371+0000","last_became_active":"2024-01-15T18:17:29.940126+0000","last_became_peered":"2024-01-15T18:17:29.940126+0000","last_unstale":"2024-01-15T18:17:30.942371+0000","last_undegraded":"2024-01-15T18:17:30.942371+0000","last_fullsized":"2024-01-15T18:17:30.942371+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:19:10.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963680+0000","last_change":"2024-01-15T18:17:30.956401+0000","last_active":"2024-01-15T18:17:30.963680+0000","last_peered":"2024-01-15T18:17:30.963680+0000","last_clean":"2024-01-15T18:17:30.963680+0000","last_became_active":"2024-01-15T18:17:30.956126+0000","last_became_peered":"2024-01-15T18:17:30.956126+0000","last_unstale":"2024-01-15T18:17:30.963680+0000","last_undegraded":"2024-01-15T18:17:30.963680+0000","last_fullsized":"2024-01-15T18:17:30.963680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.931732+0000","last_change":"2024-01-15T18:17:29.931951+0000","last_active":"2024-01-15T18:17:30.931732+0000","last_peered":"2024-01-15T18:17:30.931732+0000","last_clean":"2024-01-15T18:17:30.931732+0000","last_became_active":"2024-01-15T18:17:29.931090+0000","last_became_peered":"2024-01-15T18:17:29.931090+0000","last_unstale":"2024-01-15T18:17:30.931732+0000","last_undegraded":"2024-01-15T18:17:30.931732+0000","last_fullsized":"2024-01-15T18:17:30.931732+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:19.442115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977851+0000","last_change":"2024-01-15T18:17:30.970766+0000","last_active":"2024-01-15T18:17:30.977851+0000","last_peered":"2024-01-15T18:17:30.977851+0000","last_clean":"2024-01-15T18:17:30.977851+0000","last_became_active":"2024-01-15T18:17:30.970475+0000","last_became_peered":"2024-01-15T18:17:30.970475+0000","last_unstale":"2024-01-15T18:17:30.977851+0000","last_undegraded":"2024-01-15T18:17:30.977851+0000","last_fullsized":"2024-01-15T18:17:30.977851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:34.330410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.943540+0000","last_change":"2024-01-15T18:17:29.941702+0000","last_active":"2024-01-15T18:17:30.943540+0000","last_peered":"2024-01-15T18:17:30.943540+0000","last_clean":"2024-01-15T18:17:30.943540+0000","last_became_active":"2024-01-15T18:17:29.941419+0000","last_became_peered":"2024-01-15T18:17:29.941419+0000","last_unstale":"2024-01-15T18:17:30.943540+0000","last_undegraded":"2024-01-15T18:17:30.943540+0000","last_fullsized":"2024-01-15T18:17:30.943540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:31:11.677129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949433+0000","last_change":"2024-01-15T18:17:30.940722+0000","last_active":"2024-01-15T18:17:30.949433+0000","last_peered":"2024-01-15T18:17:30.949433+0000","last_clean":"2024-01-15T18:17:30.949433+0000","last_became_active":"2024-01-15T18:17:30.940442+0000","last_became_peered":"2024-01-15T18:17:30.940442+0000","last_unstale":"2024-01-15T18:17:30.949433+0000","last_undegraded":"2024-01-15T18:17:30.949433+0000","last_fullsized":"2024-01-15T18:17:30.949433+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:13:41.331545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.931150+0000","last_change":"2024-01-15T18:17:29.935189+0000","last_active":"2024-01-15T18:17:30.931150+0000","last_peered":"2024-01-15T18:17:30.931150+0000","last_clean":"2024-01-15T18:17:30.931150+0000","last_became_active":"2024-01-15T18:17:29.934834+0000","last_became_peered":"2024-01-15T18:17:29.934834+0000","last_unstale":"2024-01-15T18:17:30.931150+0000","last_undegraded":"2024-01-15T18:17:30.931150+0000","last_fullsized":"2024-01-15T18:17:30.931150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:12.392637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975791+0000","last_change":"2024-01-15T18:17:30.968893+0000","last_active":"2024-01-15T18:17:30.975791+0000","last_peered":"2024-01-15T18:17:30.975791+0000","last_clean":"2024-01-15T18:17:30.975791+0000","last_became_active":"2024-01-15T18:17:30.968571+0000","last_became_peered":"2024-01-15T18:17:30.968571+0000","last_unstale":"2024-01-15T18:17:30.975791+0000","last_undegraded":"2024-01-15T18:17:30.975791+0000","last_fullsized":"2024-01-15T18:17:30.975791+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:43:47.698719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.948605+0000","last_change":"2024-01-15T18:17:29.947332+0000","last_active":"2024-01-15T18:17:30.948605+0000","last_peered":"2024-01-15T18:17:30.948605+0000","last_clean":"2024-01-15T18:17:30.948605+0000","last_became_active":"2024-01-15T18:17:29.947063+0000","last_became_peered":"2024-01-15T18:17:29.947063+0000","last_unstale":"2024-01-15T18:17:30.948605+0000","last_undegraded":"2024-01-15T18:17:30.948605+0000","last_fullsized":"2024-01-15T18:17:30.948605+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:36:39.259706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963566+0000","last_change":"2024-01-15T18:17:30.954209+0000","last_active":"2024-01-15T18:17:30.963566+0000","last_peered":"2024-01-15T18:17:30.963566+0000","last_clean":"2024-01-15T18:17:30.963566+0000","last_became_active":"2024-01-15T18:17:30.950464+0000","last_became_peered":"2024-01-15T18:17:30.950464+0000","last_unstale":"2024-01-15T18:17:30.963566+0000","last_undegraded":"2024-01-15T18:17:30.963566+0000","last_fullsized":"2024-01-15T18:17:30.963566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:36:40.270304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.954094+0000","last_change":"2024-01-15T18:17:29.951425+0000","last_active":"2024-01-15T18:17:30.954094+0000","last_peered":"2024-01-15T18:17:30.954094+0000","last_clean":"2024-01-15T18:17:30.954094+0000","last_became_active":"2024-01-15T18:17:29.951157+0000","last_became_peered":"2024-01-15T18:17:29.951157+0000","last_unstale":"2024-01-15T18:17:30.954094+0000","last_undegraded":"2024-01-15T18:17:30.954094+0000","last_fullsized":"2024-01-15T18:17:30.954094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:14.347302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978035+0000","last_change":"2024-01-15T18:17:30.973568+0000","last_active":"2024-01-15T18:17:30.978035+0000","last_peered":"2024-01-15T18:17:30.978035+0000","last_clean":"2024-01-15T18:17:30.978035+0000","last_became_active":"2024-01-15T18:17:30.971112+0000","last_became_peered":"2024-01-15T18:17:30.971112+0000","last_unstale":"2024-01-15T18:17:30.978035+0000","last_undegraded":"2024-01-15T18:17:30.978035+0000","last_fullsized":"2024-01-15T18:17:30.978035+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:02:59.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964288+0000","last_change":"2024-01-15T18:17:29.936213+0000","last_active":"2024-01-15T18:17:30.964288+0000","last_peered":"2024-01-15T18:17:30.964288+0000","last_clean":"2024-01-15T18:17:30.964288+0000","last_became_active":"2024-01-15T18:17:29.935918+0000","last_became_peered":"2024-01-15T18:17:29.935918+0000","last_unstale":"2024-01-15T18:17:30.964288+0000","last_undegraded":"2024-01-15T18:17:30.964288+0000","last_fullsized":"2024-01-15T18:17:30.964288+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.255371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-01-15T18:17:29.925909+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:17:29.925909+0000","last_peered":"2024-01-15T18:17:29.925909+0000","last_clean":"2024-01-15T18:17:29.925909+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:17:29.925909+0000","last_undegraded":"2024-01-15T18:17:29.925909+0000","last_fullsized":"2024-01-15T18:17:29.925909+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64836,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307004,"statfs":{"total":96636764160,"available":96570372096,"internally_reserved":0,"allocated":1826816,"data_stored":1119638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":1.1180000000000001,"15min":0.92300000000000004},"min":{"1min":0.46100000000000002,"5min":0.39600000000000002,"15min":0.29599999999999999},"max":{"1min":0.98999999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.96799999999999997,"15min":0.86499999999999999},"min":{"1min":0.495,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.88800000000000001,"5min":8.5419999999999998,"15min":13.744},"last":0.59799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":1.288,"15min":1.0629999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.90500000000000003,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":1.264,"15min":1.0609999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92700000000000005,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.628}]},{"osd":2,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.76700000000000002,"15min":0.99299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.143,"5min":2.1680000000000001,"15min":30.846},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.0940000000000001,"15min":1.0800000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":0.98099999999999998,"5min":17.039999999999999,"15min":39.695},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":1.1919999999999999,"15min":1.1699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":1.6020000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.0700000000000001,"15min":1.2130000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94499999999999995,"5min":16.879999999999999,"15min":42.692},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.84899999999999998,"15min":0.71199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":14.087,"15min":14.087},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.64200000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.48299999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81599999999999995,"15min":0.72399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":0.90300000000000002,"5min":11.334,"15min":11.334},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.83899999999999997,"15min":0.77100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.745,"5min":8.423,"15min":8.423},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.872,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.70699999999999996,"15min":0.73099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86499999999999999,"5min":1.7909999999999999,"15min":3.3450000000000002},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64240,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307600,"statfs":{"total":96636764160,"available":96570982400,"internally_reserved":0,"allocated":1212416,"data_stored":522327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.85099999999999998,"15min":0.95399999999999996},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4670000000000001,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.75900000000000001,"15min":0.876},"min":{"1min":0.375,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96499999999999997,"5min":3.4940000000000002,"15min":12.449},"last":0.627}]},{"osd":1,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.91300000000000003,"15min":0.89900000000000002},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":2.0710000000000002,"5min":6.8239999999999998,"15min":14.246},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.1439999999999999,"15min":1.0109999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.5389999999999999,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.73399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":1.107,"15min":1.1830000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.016,"5min":14.743,"15min":42.825000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.91900000000000004,"15min":1.153},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.994,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.70699999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78700000000000003,"15min":1.03},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":1.911,"5min":3.016,"15min":24.199000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.087,"15min":1.081},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":5.0609999999999999,"5min":18.791,"15min":18.791},"last":0.72199999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.88900000000000001,"15min":0.751},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.417,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.65200000000000002,"15min":0.753},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":1.7450000000000001,"15min":12.491},"last":0.47799999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.34799999999999998},"max":{"1min":1.9410000000000001,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.434},{"interface":"front","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.1040000000000001,"5min":2.5259999999999998,"15min":12.945},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":0.84799999999999998},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.9830000000000001,"5min":11.907,"15min":11.907},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.89300000000000002},"min":{"1min":0.55300000000000005,"5min":0.39600000000000002,"15min":0.34200000000000003},"max":{"1min":1.7929999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.46600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575005,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54580,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317260,"statfs":{"total":96636764160,"available":96580874240,"internally_reserved":0,"allocated":1212416,"data_stored":522327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.91700000000000004,"15min":0.83199999999999996},"min":{"1min":0.42799999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.7450000000000001,"5min":6.6820000000000004,"15min":11.519},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.012,"15min":0.92400000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":3.1699999999999999,"5min":7.6260000000000003,"15min":12.952},"last":0.84499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":1.0089999999999999,"15min":0.90300000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.37},"max":{"1min":3.2080000000000002,"5min":4.609,"15min":13.358000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.94199999999999995,"15min":0.95099999999999996},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.1580000000000004,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.167,"15min":0.93500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.5499999999999998,"5min":18.314,"15min":18.314},"last":0.755},{"interface":"front","average":{"1min":0.77900000000000003,"5min":1.361,"15min":1.0109999999999999},"min":{"1min":0.51600000000000001,"5min":0.47899999999999998,"15min":0.434},"max":{"1min":2.02,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92600000000000005,"15min":0.94899999999999995},"min":{"1min":0.503,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":3.2400000000000002,"5min":4.0330000000000004,"15min":13.317},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":1.0600000000000001,"15min":1.0089999999999999},"min":{"1min":0.56599999999999995,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.5099999999999998,"5min":15.205,"15min":19.638000000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.77300000000000002,"15min":0.72299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":3.1240000000000001,"5min":5.899,"15min":11.093},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73799999999999999,"15min":0.69599999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":2.3580000000000001,"5min":3.097,"15min":3.3889999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86799999999999999,"15min":0.80200000000000005},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.4129999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.73499999999999999},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.635,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.88100000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81899999999999995,"15min":0.76900000000000002},"min":{"1min":0.44500000000000001,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.90800000000000003,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":2.4660000000000002,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76756,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295084,"statfs":{"total":96636764160,"available":96558166016,"internally_reserved":0,"allocated":1245184,"data_stored":530894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.98699999999999999,"15min":0.93600000000000005},"min":{"1min":0.438,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":1.151,"5min":11.374000000000001,"15min":34.802},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1279999999999999,"15min":0.91100000000000003},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":2.6760000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.81599999999999995,"15min":0.96499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.873,"5min":3.9380000000000002,"15min":38.198},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.96099999999999997},"min":{"1min":0.52200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98199999999999998,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89100000000000001,"15min":0.86799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.39600000000000002},"max":{"1min":1.6259999999999999,"5min":7.4530000000000003,"15min":7.6989999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90700000000000003,"15min":0.85499999999999998},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.38700000000000001},"max":{"1min":1.595,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.65900000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":1.2030000000000001,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.40300000000000002},"max":{"1min":1.9339999999999999,"5min":14.529,"15min":14.529},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.995,"15min":0.93000000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.69599999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.77400000000000002,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":0.875,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.97999999999999998,"15min":0.82899999999999996},"min":{"1min":0.51900000000000002,"5min":0.40400000000000003,"15min":0.34100000000000003},"max":{"1min":0.82199999999999995,"5min":25.887,"15min":25.887},"last":0.53600000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.78300000000000003,"5min":7.4930000000000003,"15min":10.284000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.50800000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.83299999999999996,"15min":0.78700000000000003},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.30399999999999999},"max":{"1min":0.89600000000000002,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.70599999999999996,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.84199999999999997,"5min":1.232,"15min":11.67},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64264,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307576,"statfs":{"total":96636764160,"available":96570957824,"internally_reserved":0,"allocated":1241088,"data_stored":530792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":1.234,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":0.77900000000000003,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.91400000000000003,"15min":0.95299999999999996},"min":{"1min":0.315,"5min":0.25,"15min":0.246},"max":{"1min":0.63800000000000001,"5min":12.247999999999999,"15min":28.332000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.88300000000000001,"15min":0.98299999999999998},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.6869999999999998,"5min":12.545,"15min":40.457999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.3029999999999999,"15min":1.1419999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":1.2769999999999999,"5min":26.260999999999999,"15min":27.427},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":1.0169999999999999,"15min":0.98599999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":12.209,"15min":23.036999999999999},"last":0.377},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.5329999999999999,"15min":1.0860000000000001},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":0.81399999999999995,"5min":26.404,"15min":26.404},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.054,"15min":1.0249999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.41199999999999998},"max":{"1min":1.304,"5min":12.073,"15min":22.998000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.042,"15min":1.0780000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.827,"5min":7.149,"15min":20.399000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.4350000000000001,"15min":1.099},"min":{"1min":0.52400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.093,"5min":26.314,"15min":26.314},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":1.0429999999999999,"15min":1.1259999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91600000000000004,"5min":12.173999999999999,"15min":22.908999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.2450000000000001,"15min":1.1539999999999999},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.843,"5min":7.2809999999999997,"15min":20.445},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.6080000000000001,"15min":1.1899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":6.7640000000000002,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":1.204,"15min":1.1599999999999999},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.1879999999999999,"5min":12.129,"15min":23.111999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.2190000000000001,"15min":1.3240000000000001},"min":{"1min":0.58199999999999996,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":2.9340000000000002,"5min":10.920999999999999,"15min":44.817999999999998},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":1228800,"data_stored":529192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.63700000000000001,"15min":1.0509999999999999},"min":{"1min":0.27300000000000002,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.73699999999999999,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.94299999999999995,"15min":1.3240000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33800000000000002},"max":{"1min":0.875,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.496}]},{"osd":1,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.746,"15min":1.5429999999999999},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":1.954,"5min":7.476,"15min":113.149},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":1.046,"15min":1.107},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.82699999999999996,"5min":20.619,"15min":41.302},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.5660000000000001,"15min":1.377},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28899999999999998},"max":{"1min":1.506,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.74399999999999999,"15min":1.2669999999999999},"min":{"1min":0.437,"5min":0.374,"15min":0.34200000000000003},"max":{"1min":1.0960000000000001,"5min":7.3609999999999998,"15min":115.693},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.98799999999999999,"15min":1.212},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":0.98999999999999999,"5min":7.4169999999999998,"15min":24.366},"last":0.746},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.202,"15min":1.2350000000000001},"min":{"1min":0.51700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":6.2469999999999999,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":1.8049999999999999,"15min":1.5089999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":4.1699999999999999,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.085,"15min":1.3100000000000001},"min":{"1min":0.54900000000000004,"5min":0.497,"15min":0.496},"max":{"1min":1.032,"5min":8.5239999999999991,"15min":26.084},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.373,"15min":1.3240000000000001},"min":{"1min":0.51800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3049999999999997,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.849,"15min":1.573},"min":{"1min":0.59199999999999997,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":4.0919999999999996,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.3160000000000001,"15min":1.268},"min":{"1min":0.60599999999999998,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":6.1210000000000004,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.962,"15min":1.639},"min":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.48799999999999999},"max":{"1min":3.4489999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":1224704,"data_stored":529158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.68799999999999994,"15min":0.93400000000000005},"min":{"1min":0.27700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":9.452,"5min":9.452,"15min":39.948},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.75900000000000001,"15min":1.1200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":2.6989999999999998,"5min":8.141,"15min":35.790999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":0.76400000000000001,"15min":0.998},"min":{"1min":0.34300000000000003,"5min":0.30499999999999999,"15min":0.27600000000000002},"max":{"1min":15.137,"5min":15.137,"15min":49.229999999999997},"last":0.498},{"interface":"front","average":{"1min":1.113,"5min":0.81299999999999994,"15min":0.875},"min":{"1min":0.36799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":9.8469999999999995,"5min":9.8469999999999995,"15min":10.349},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.77000000000000002,"15min":1.1899999999999999},"min":{"1min":0.41999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.258,"5min":11.901,"15min":27.643999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.95299999999999996,"15min":1.0189999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":9.6519999999999992,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.97399999999999998,"15min":1.244},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.40200000000000002},"max":{"1min":1.1739999999999999,"5min":7.4450000000000003,"15min":27.489999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.304,"5min":1.091,"15min":1.089},"min":{"1min":0.56799999999999995,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":9.6989999999999998,"5min":9.6989999999999998,"15min":12.997},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.99099999999999999,"15min":1.2230000000000001},"min":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.298,"5min":7.3380000000000001,"15min":27.547000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.1020000000000001,"15min":1.1439999999999999},"min":{"1min":0.63,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":9.7650000000000006,"5min":9.7650000000000006,"15min":13.054},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.96899999999999997,"15min":1.22},"min":{"1min":0.59399999999999997,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":1.226,"5min":7.5,"15min":27.579999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.02,"5min":0.93300000000000005,"15min":1.0589999999999999},"min":{"1min":0.56299999999999994,"5min":0.50900000000000001,"15min":0.42399999999999999},"max":{"1min":5.9020000000000001,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0669999999999999,"15min":1.1279999999999999},"min":{"1min":0.60799999999999998,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":5.9779999999999998,"5min":9.5660000000000007,"15min":11.833},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.351,"5min":1.081,"15min":1.1579999999999999},"min":{"1min":0.64400000000000002,"5min":0.505,"15min":0.502},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":14.606},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77312,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294528,"statfs":{"total":96636764160,"available":96557596672,"internally_reserved":0,"allocated":1822720,"data_stored":1120108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.86199999999999999,"15min":1.1679999999999999},"min":{"1min":0.33300000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.60799999999999998,"5min":11.356,"15min":31.977},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.97799999999999998,"15min":1.0209999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.0250000000000004,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.495}]},{"osd":2,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.96099999999999997,"15min":1.1299999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":5.8929999999999998,"5min":13.166,"15min":41.387999999999998},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.91700000000000004,"15min":1.0489999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.25},"max":{"1min":3.484,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.51900000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.038,"15min":1.113},"min":{"1min":0.39400000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.841,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.97499999999999998,"15min":1.0960000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.33300000000000002},"max":{"1min":5.9800000000000004,"5min":12.954000000000001,"15min":45.128},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":1.079,"15min":1.083},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.372},"max":{"1min":1.877,"5min":11.772,"15min":32.932000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.119,"15min":1.206},"min":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.93999999999999995,"5min":11.220000000000001,"15min":31.782},"last":1.0309999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":1.1200000000000001,"15min":1.2},"min":{"1min":0.47799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.103,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":1.1060000000000001,"15min":1.079},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.794,"5min":11.958,"15min":33.151000000000003},"last":0.872}]},{"osd":6,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.099,"15min":1.0660000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":4.0970000000000004,"5min":13.055,"15min":13.055},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.49399999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5960000000000001,"5min":11.833,"15min":33.098999999999997},"last":0.91900000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.115,"15min":1.0780000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.501},"max":{"1min":4.1349999999999998,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.135,"15min":1.256},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.0069999999999999,"5min":11.321999999999999,"15min":31.905999999999999},"last":1.0129999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:17:35.048 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:17:35.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 Processor -- start 2024-01-15T18:17:35.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 -- start start 2024-01-15T18:17:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f00cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc1f00d19f0 0x7fc1f00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00c4d90 con 0x7fc1f00c5220 2024-01-15T18:17:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f015b640 con 0x7fc1f00d19f0 2024-01-15T18:17:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1f5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00cf800 con 0x7fc1f015b260 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ede00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f00cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ee800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fc1f00c5220 0x7fc1f00c56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:48414/0 (socket says 172.21.15.143:48414) 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ee800640 1 -- 172.21.15.143:0/572805166 learned_addr learned my addr 172.21.15.143:0/572805166 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ed400640 1 --2- 172.21.15.143:0/572805166 >> v2:172.21.15.204:3300/0 conn(0x7fc1f00d19f0 0x7fc1f00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1eca00640 1 -- 172.21.15.143:0/572805166 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057554599 0 0) 0x7fc1f00c4d90 con 0x7fc1f00c5220 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1eca00640 1 -- 172.21.15.143:0/572805166 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d0003530 con 0x7fc1f00c5220 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ed400640 1 -- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 msgr2=0x7fc1f00cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ed400640 1 --2- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f00cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ed400640 1 -- 172.21.15.143:0/572805166 >> v1:172.21.15.204:6789/0 conn(0x7fc1f00c5220 legacy=0x7fc1f00c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ed400640 1 -- 172.21.15.143:0/572805166 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00cfae0 con 0x7fc1f00d19f0 2024-01-15T18:17:35.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.196+0000 7fc1ede00640 1 --2- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1ed400640 1 --2- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00d19f0 0x7fc1f00cdd50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80097c0 tx=0x7fc1d801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2259d385e535e0a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1eca00640 1 -- 172.21.15.143:0/572805166 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80ad660 con 0x7fc1f00d19f0 2024-01-15T18:17:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1eca00640 1 -- 172.21.15.143:0/572805166 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d80ad7c0 con 0x7fc1f00d19f0 2024-01-15T18:17:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1eca00640 1 -- 172.21.15.143:0/572805166 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80adb50 con 0x7fc1f00d19f0 2024-01-15T18:17:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 -- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00d19f0 msgr2=0x7fc1f00cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 --2- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00d19f0 0x7fc1f00cdd50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80097c0 tx=0x7fc1d801c450 comp rx=0 tx=0).stop 2024-01-15T18:17:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 -- 172.21.15.143:0/572805166 shutdown_connections 2024-01-15T18:17:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 --2- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00d19f0 0x7fc1f00cdd50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 --2- 172.21.15.143:0/572805166 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f00cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 -- 172.21.15.143:0/572805166 >> 172.21.15.143:0/572805166 conn(0x7fc1f00d0ed0 msgr2=0x7fc1f00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 -- 172.21.15.143:0/572805166 shutdown_connections 2024-01-15T18:17:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.200+0000 7fc1f5000640 1 -- 172.21.15.143:0/572805166 wait complete. 2024-01-15T18:17:35.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 Processor -- start 2024-01-15T18:17:35.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 -- start start 2024-01-15T18:17:35.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00c5220 0x7fc1f014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 0x7fc1f0157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f0159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0158070 con 0x7fc1f00d19f0 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f01509f0 con 0x7fc1f00c5220 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0157e40 con 0x7fc1f015b260 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 0x7fc1f0157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 0x7fc1f0157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40166/0 (socket says 172.21.15.143:40166) 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 -- 172.21.15.143:0/3301899202 learned_addr learned my addr 172.21.15.143:0/3301899202 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ee800640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f0159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ede00640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00c5220 0x7fc1f014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 -- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 msgr2=0x7fc1f0159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f0159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 -- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00c5220 msgr2=0x7fc1f014f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00c5220 0x7fc1f014f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 -- 172.21.15.143:0/3301899202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0009420 con 0x7fc1f00d19f0 2024-01-15T18:17:35.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ede00640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00c5220 0x7fc1f014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1ed400640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 0x7fc1f0157900 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80b0040 tx=0x7fc1d80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80b6610 con 0x7fc1f00d19f0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d80b6790 con 0x7fc1f00d19f0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d8009420 con 0x7fc1f00d19f0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d80b6b40 con 0x7fc1f00d19f0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.204+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f014e650 con 0x7fc1f00d19f0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.208+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f0067500 con 0x7fc1f00d19f0 2024-01-15T18:17:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.208+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fc1d80b4040 con 0x7fc1f00d19f0 2024-01-15T18:17:35.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.208+0000 7fc1e6000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1c8067820 0x7fc1c8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.208+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc1d811cf30 con 0x7fc1f00d19f0 2024-01-15T18:17:35.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.208+0000 7fc1ede00640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1c8067820 0x7fc1c8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.208+0000 7fc1ede00640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1c8067820 0x7fc1c8069ce0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009ca0 tx=0x7fc1e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.212+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc1d80ebbc0 con 0x7fc1f00d19f0 2024-01-15T18:17:35.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.360+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1f00b6d50 con 0x7fc1c8067820 2024-01-15T18:17:35.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.368+0000 7fc1e6000640 1 -- 172.21.15.143:0/3301899202 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288296 (secure 0 0 0) 0x7fc1f00b6d50 con 0x7fc1c8067820 2024-01-15T18:17:35.370 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:35.371 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1c8067820 msgr2=0x7fc1c8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1c8067820 0x7fc1c8069ce0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009ca0 tx=0x7fc1e0008040 comp rx=0 tx=0).stop 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 msgr2=0x7fc1f0157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 0x7fc1f0157900 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc1d80b0040 tx=0x7fc1d80191f0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 shutdown_connections 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc1c8067820 0x7fc1c8069ce0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc1f015b260 0x7fc1f0159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc1f00d19f0 0x7fc1f0157900 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 --2- 172.21.15.143:0/3301899202 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc1f00c5220 0x7fc1f014f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 >> 172.21.15.143:0/3301899202 conn(0x7fc1f00d0ed0 msgr2=0x7fc1f00cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 shutdown_connections 2024-01-15T18:17:35.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.372+0000 7fc1f5000640 1 -- 172.21.15.143:0/3301899202 wait complete. 2024-01-15T18:17:35.392 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-01-15T18:17:34.879462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":79,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":530496,"kb_used_data":10756,"kb_used_omap":12,"kb_used_meta":519603,"kb_avail":754444224,"statfs":{"total":773094113280,"available":772550885376,"internally_reserved":0,"allocated":11014144,"data_stored":5404436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":532074032},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.961479"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.962544+0000","last_change":"2024-01-15T18:17:30.954624+0000","last_active":"2024-01-15T18:17:30.962544+0000","last_peered":"2024-01-15T18:17:30.962544+0000","last_clean":"2024-01-15T18:17:30.962544+0000","last_became_active":"2024-01-15T18:17:30.954352+0000","last_became_peered":"2024-01-15T18:17:30.954352+0000","last_unstale":"2024-01-15T18:17:30.962544+0000","last_undegraded":"2024-01-15T18:17:30.962544+0000","last_fullsized":"2024-01-15T18:17:30.962544+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:00.544507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975605+0000","last_change":"2024-01-15T18:17:30.963931+0000","last_active":"2024-01-15T18:17:30.975605+0000","last_peered":"2024-01-15T18:17:30.975605+0000","last_clean":"2024-01-15T18:17:30.975605+0000","last_became_active":"2024-01-15T18:17:30.963002+0000","last_became_peered":"2024-01-15T18:17:30.963002+0000","last_unstale":"2024-01-15T18:17:30.975605+0000","last_undegraded":"2024-01-15T18:17:30.975605+0000","last_fullsized":"2024-01-15T18:17:30.975605+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:11.421513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968018+0000","last_change":"2024-01-15T18:17:30.957766+0000","last_active":"2024-01-15T18:17:30.968018+0000","last_peered":"2024-01-15T18:17:30.968018+0000","last_clean":"2024-01-15T18:17:30.968018+0000","last_became_active":"2024-01-15T18:17:30.957492+0000","last_became_peered":"2024-01-15T18:17:30.957492+0000","last_unstale":"2024-01-15T18:17:30.968018+0000","last_undegraded":"2024-01-15T18:17:30.968018+0000","last_fullsized":"2024-01-15T18:17:30.968018+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:45.837765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.976097+0000","last_change":"2024-01-15T18:17:30.972054+0000","last_active":"2024-01-15T18:17:30.976097+0000","last_peered":"2024-01-15T18:17:30.976097+0000","last_clean":"2024-01-15T18:17:30.976097+0000","last_became_active":"2024-01-15T18:17:30.971356+0000","last_became_peered":"2024-01-15T18:17:30.971356+0000","last_unstale":"2024-01-15T18:17:30.976097+0000","last_undegraded":"2024-01-15T18:17:30.976097+0000","last_fullsized":"2024-01-15T18:17:30.976097+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964087+0000","last_change":"2024-01-15T18:17:30.953830+0000","last_active":"2024-01-15T18:17:30.964087+0000","last_peered":"2024-01-15T18:17:30.964087+0000","last_clean":"2024-01-15T18:17:30.964087+0000","last_became_active":"2024-01-15T18:17:30.953494+0000","last_became_peered":"2024-01-15T18:17:30.953494+0000","last_unstale":"2024-01-15T18:17:30.964087+0000","last_undegraded":"2024-01-15T18:17:30.964087+0000","last_fullsized":"2024-01-15T18:17:30.964087+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.329164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964361+0000","last_change":"2024-01-15T18:17:30.956596+0000","last_active":"2024-01-15T18:17:30.964361+0000","last_peered":"2024-01-15T18:17:30.964361+0000","last_clean":"2024-01-15T18:17:30.964361+0000","last_became_active":"2024-01-15T18:17:30.954556+0000","last_became_peered":"2024-01-15T18:17:30.954556+0000","last_unstale":"2024-01-15T18:17:30.964361+0000","last_undegraded":"2024-01-15T18:17:30.964361+0000","last_fullsized":"2024-01-15T18:17:30.964361+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:51.759595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964212+0000","last_change":"2024-01-15T18:17:30.955196+0000","last_active":"2024-01-15T18:17:30.964212+0000","last_peered":"2024-01-15T18:17:30.964212+0000","last_clean":"2024-01-15T18:17:30.964212+0000","last_became_active":"2024-01-15T18:17:30.954775+0000","last_became_peered":"2024-01-15T18:17:30.954775+0000","last_unstale":"2024-01-15T18:17:30.964212+0000","last_undegraded":"2024-01-15T18:17:30.964212+0000","last_fullsized":"2024-01-15T18:17:30.964212+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:16:49.180169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977325+0000","last_change":"2024-01-15T18:17:30.969882+0000","last_active":"2024-01-15T18:17:30.977325+0000","last_peered":"2024-01-15T18:17:30.977325+0000","last_clean":"2024-01-15T18:17:30.977325+0000","last_became_active":"2024-01-15T18:17:30.969568+0000","last_became_peered":"2024-01-15T18:17:30.969568+0000","last_unstale":"2024-01-15T18:17:30.977325+0000","last_undegraded":"2024-01-15T18:17:30.977325+0000","last_fullsized":"2024-01-15T18:17:30.977325+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:16:49.180169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.961112+0000","last_change":"2024-01-15T18:17:30.951953+0000","last_active":"2024-01-15T18:17:30.961112+0000","last_peered":"2024-01-15T18:17:30.961112+0000","last_clean":"2024-01-15T18:17:30.961112+0000","last_became_active":"2024-01-15T18:17:30.951643+0000","last_became_peered":"2024-01-15T18:17:30.951643+0000","last_unstale":"2024-01-15T18:17:30.961112+0000","last_undegraded":"2024-01-15T18:17:30.961112+0000","last_fullsized":"2024-01-15T18:17:30.961112+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:34.425305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.945943+0000","last_change":"2024-01-15T18:17:30.940607+0000","last_active":"2024-01-15T18:17:30.945943+0000","last_peered":"2024-01-15T18:17:30.945943+0000","last_clean":"2024-01-15T18:17:30.945943+0000","last_became_active":"2024-01-15T18:17:30.940333+0000","last_became_peered":"2024-01-15T18:17:30.940333+0000","last_unstale":"2024-01-15T18:17:30.945943+0000","last_undegraded":"2024-01-15T18:17:30.945943+0000","last_fullsized":"2024-01-15T18:17:30.945943+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:01.210019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.965784+0000","last_change":"2024-01-15T18:17:30.962459+0000","last_active":"2024-01-15T18:17:30.965784+0000","last_peered":"2024-01-15T18:17:30.965784+0000","last_clean":"2024-01-15T18:17:30.965784+0000","last_became_active":"2024-01-15T18:17:30.961797+0000","last_became_peered":"2024-01-15T18:17:30.961797+0000","last_unstale":"2024-01-15T18:17:30.965784+0000","last_undegraded":"2024-01-15T18:17:30.965784+0000","last_fullsized":"2024-01-15T18:17:30.965784+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:17.557216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949272+0000","last_change":"2024-01-15T18:17:30.942808+0000","last_active":"2024-01-15T18:17:30.949272+0000","last_peered":"2024-01-15T18:17:30.949272+0000","last_clean":"2024-01-15T18:17:30.949272+0000","last_became_active":"2024-01-15T18:17:30.942511+0000","last_became_peered":"2024-01-15T18:17:30.942511+0000","last_unstale":"2024-01-15T18:17:30.949272+0000","last_undegraded":"2024-01-15T18:17:30.949272+0000","last_fullsized":"2024-01-15T18:17:30.949272+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.970148+0000","last_change":"2024-01-15T18:17:30.966690+0000","last_active":"2024-01-15T18:17:30.970148+0000","last_peered":"2024-01-15T18:17:30.970148+0000","last_clean":"2024-01-15T18:17:30.970148+0000","last_became_active":"2024-01-15T18:17:30.966411+0000","last_became_peered":"2024-01-15T18:17:30.966411+0000","last_unstale":"2024-01-15T18:17:30.970148+0000","last_undegraded":"2024-01-15T18:17:30.970148+0000","last_fullsized":"2024-01-15T18:17:30.970148+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.329164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978429+0000","last_change":"2024-01-15T18:17:30.975937+0000","last_active":"2024-01-15T18:17:30.978429+0000","last_peered":"2024-01-15T18:17:30.978429+0000","last_clean":"2024-01-15T18:17:30.978429+0000","last_became_active":"2024-01-15T18:17:30.974548+0000","last_became_peered":"2024-01-15T18:17:30.974548+0000","last_unstale":"2024-01-15T18:17:30.978429+0000","last_undegraded":"2024-01-15T18:17:30.978429+0000","last_fullsized":"2024-01-15T18:17:30.978429+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:16.852132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958545+0000","last_change":"2024-01-15T18:17:30.956101+0000","last_active":"2024-01-15T18:17:30.958545+0000","last_peered":"2024-01-15T18:17:30.958545+0000","last_clean":"2024-01-15T18:17:30.958545+0000","last_became_active":"2024-01-15T18:17:30.955832+0000","last_became_peered":"2024-01-15T18:17:30.955832+0000","last_unstale":"2024-01-15T18:17:30.958545+0000","last_undegraded":"2024-01-15T18:17:30.958545+0000","last_fullsized":"2024-01-15T18:17:30.958545+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:13.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964579+0000","last_change":"2024-01-15T18:17:30.958977+0000","last_active":"2024-01-15T18:17:30.964579+0000","last_peered":"2024-01-15T18:17:30.964579+0000","last_clean":"2024-01-15T18:17:30.964579+0000","last_became_active":"2024-01-15T18:17:30.958666+0000","last_became_peered":"2024-01-15T18:17:30.958666+0000","last_unstale":"2024-01-15T18:17:30.964579+0000","last_undegraded":"2024-01-15T18:17:30.964579+0000","last_fullsized":"2024-01-15T18:17:30.964579+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:40.172255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.969175+0000","last_change":"2024-01-15T18:17:30.962228+0000","last_active":"2024-01-15T18:17:30.969175+0000","last_peered":"2024-01-15T18:17:30.969175+0000","last_clean":"2024-01-15T18:17:30.969175+0000","last_became_active":"2024-01-15T18:17:30.961930+0000","last_became_peered":"2024-01-15T18:17:30.961930+0000","last_unstale":"2024-01-15T18:17:30.969175+0000","last_undegraded":"2024-01-15T18:17:30.969175+0000","last_fullsized":"2024-01-15T18:17:30.969175+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:13:41.331545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.976366+0000","last_change":"2024-01-15T18:17:30.975553+0000","last_active":"2024-01-15T18:17:30.976366+0000","last_peered":"2024-01-15T18:17:30.976366+0000","last_clean":"2024-01-15T18:17:30.976366+0000","last_became_active":"2024-01-15T18:17:30.974646+0000","last_became_peered":"2024-01-15T18:17:30.974646+0000","last_unstale":"2024-01-15T18:17:30.976366+0000","last_undegraded":"2024-01-15T18:17:30.976366+0000","last_fullsized":"2024-01-15T18:17:30.976366+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:20.452712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978083+0000","last_change":"2024-01-15T18:17:30.973968+0000","last_active":"2024-01-15T18:17:30.978083+0000","last_peered":"2024-01-15T18:17:30.978083+0000","last_clean":"2024-01-15T18:17:30.978083+0000","last_became_active":"2024-01-15T18:17:30.970746+0000","last_became_peered":"2024-01-15T18:17:30.970746+0000","last_unstale":"2024-01-15T18:17:30.978083+0000","last_undegraded":"2024-01-15T18:17:30.978083+0000","last_fullsized":"2024-01-15T18:17:30.978083+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:13:42.332337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964673+0000","last_change":"2024-01-15T18:17:30.953059+0000","last_active":"2024-01-15T18:17:30.964673+0000","last_peered":"2024-01-15T18:17:30.964673+0000","last_clean":"2024-01-15T18:17:30.964673+0000","last_became_active":"2024-01-15T18:17:30.952775+0000","last_became_peered":"2024-01-15T18:17:30.952775+0000","last_unstale":"2024-01-15T18:17:30.964673+0000","last_undegraded":"2024-01-15T18:17:30.964673+0000","last_fullsized":"2024-01-15T18:17:30.964673+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:20.452712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977964+0000","last_change":"2024-01-15T18:17:30.971220+0000","last_active":"2024-01-15T18:17:30.977964+0000","last_peered":"2024-01-15T18:17:30.977964+0000","last_clean":"2024-01-15T18:17:30.977964+0000","last_became_active":"2024-01-15T18:17:30.970836+0000","last_became_peered":"2024-01-15T18:17:30.970836+0000","last_unstale":"2024-01-15T18:17:30.977964+0000","last_undegraded":"2024-01-15T18:17:30.977964+0000","last_fullsized":"2024-01-15T18:17:30.977964+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:55.980450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963256+0000","last_change":"2024-01-15T18:17:30.960737+0000","last_active":"2024-01-15T18:17:30.963256+0000","last_peered":"2024-01-15T18:17:30.963256+0000","last_clean":"2024-01-15T18:17:30.963256+0000","last_became_active":"2024-01-15T18:17:30.960448+0000","last_became_peered":"2024-01-15T18:17:30.960448+0000","last_unstale":"2024-01-15T18:17:30.963256+0000","last_undegraded":"2024-01-15T18:17:30.963256+0000","last_fullsized":"2024-01-15T18:17:30.963256+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963819+0000","last_change":"2024-01-15T18:17:30.961193+0000","last_active":"2024-01-15T18:17:30.963819+0000","last_peered":"2024-01-15T18:17:30.963819+0000","last_clean":"2024-01-15T18:17:30.963819+0000","last_became_active":"2024-01-15T18:17:30.960904+0000","last_became_peered":"2024-01-15T18:17:30.960904+0000","last_unstale":"2024-01-15T18:17:30.963819+0000","last_undegraded":"2024-01-15T18:17:30.963819+0000","last_fullsized":"2024-01-15T18:17:30.963819+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:19.861186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963997+0000","last_change":"2024-01-15T18:17:30.950164+0000","last_active":"2024-01-15T18:17:30.963997+0000","last_peered":"2024-01-15T18:17:30.963997+0000","last_clean":"2024-01-15T18:17:30.963997+0000","last_became_active":"2024-01-15T18:17:30.949893+0000","last_became_peered":"2024-01-15T18:17:30.949893+0000","last_unstale":"2024-01-15T18:17:30.963997+0000","last_undegraded":"2024-01-15T18:17:30.963997+0000","last_fullsized":"2024-01-15T18:17:30.963997+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:31:12.687727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.948848+0000","last_change":"2024-01-15T18:17:30.941972+0000","last_active":"2024-01-15T18:17:30.948848+0000","last_peered":"2024-01-15T18:17:30.948848+0000","last_clean":"2024-01-15T18:17:30.948848+0000","last_became_active":"2024-01-15T18:17:30.941658+0000","last_became_peered":"2024-01-15T18:17:30.941658+0000","last_unstale":"2024-01-15T18:17:30.948848+0000","last_undegraded":"2024-01-15T18:17:30.948848+0000","last_fullsized":"2024-01-15T18:17:30.948848+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:21.130381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.967931+0000","last_change":"2024-01-15T18:17:30.959819+0000","last_active":"2024-01-15T18:17:30.967931+0000","last_peered":"2024-01-15T18:17:30.967931+0000","last_clean":"2024-01-15T18:17:30.967931+0000","last_became_active":"2024-01-15T18:17:30.956562+0000","last_became_peered":"2024-01-15T18:17:30.956562+0000","last_unstale":"2024-01-15T18:17:30.967931+0000","last_undegraded":"2024-01-15T18:17:30.967931+0000","last_fullsized":"2024-01-15T18:17:30.967931+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:51.759595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968265+0000","last_change":"2024-01-15T18:17:30.967418+0000","last_active":"2024-01-15T18:17:30.968265+0000","last_peered":"2024-01-15T18:17:30.968265+0000","last_clean":"2024-01-15T18:17:30.968265+0000","last_became_active":"2024-01-15T18:17:30.967115+0000","last_became_peered":"2024-01-15T18:17:30.967115+0000","last_unstale":"2024-01-15T18:17:30.968265+0000","last_undegraded":"2024-01-15T18:17:30.968265+0000","last_fullsized":"2024-01-15T18:17:30.968265+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:19.861186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963631+0000","last_change":"2024-01-15T18:17:30.955557+0000","last_active":"2024-01-15T18:17:30.963631+0000","last_peered":"2024-01-15T18:17:30.963631+0000","last_clean":"2024-01-15T18:17:30.963631+0000","last_became_active":"2024-01-15T18:17:30.955265+0000","last_became_peered":"2024-01-15T18:17:30.955265+0000","last_unstale":"2024-01-15T18:17:30.963631+0000","last_undegraded":"2024-01-15T18:17:30.963631+0000","last_fullsized":"2024-01-15T18:17:30.963631+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:43.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.960005+0000","last_change":"2024-01-15T18:17:30.954073+0000","last_active":"2024-01-15T18:17:30.960005+0000","last_peered":"2024-01-15T18:17:30.960005+0000","last_clean":"2024-01-15T18:17:30.960005+0000","last_became_active":"2024-01-15T18:17:30.953811+0000","last_became_peered":"2024-01-15T18:17:30.953811+0000","last_unstale":"2024-01-15T18:17:30.960005+0000","last_undegraded":"2024-01-15T18:17:30.960005+0000","last_fullsized":"2024-01-15T18:17:30.960005+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:28.174662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978407+0000","last_change":"2024-01-15T18:17:30.975772+0000","last_active":"2024-01-15T18:17:30.978407+0000","last_peered":"2024-01-15T18:17:30.978407+0000","last_clean":"2024-01-15T18:17:30.978407+0000","last_became_active":"2024-01-15T18:17:30.973202+0000","last_became_peered":"2024-01-15T18:17:30.973202+0000","last_unstale":"2024-01-15T18:17:30.978407+0000","last_undegraded":"2024-01-15T18:17:30.978407+0000","last_fullsized":"2024-01-15T18:17:30.978407+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:53:32.204347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958527+0000","last_change":"2024-01-15T18:17:30.955625+0000","last_active":"2024-01-15T18:17:30.958527+0000","last_peered":"2024-01-15T18:17:30.958527+0000","last_clean":"2024-01-15T18:17:30.958527+0000","last_became_active":"2024-01-15T18:17:30.955351+0000","last_became_peered":"2024-01-15T18:17:30.955351+0000","last_unstale":"2024-01-15T18:17:30.958527+0000","last_undegraded":"2024-01-15T18:17:30.958527+0000","last_fullsized":"2024-01-15T18:17:30.958527+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:26.944068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.966041+0000","last_change":"2024-01-15T18:17:30.957564+0000","last_active":"2024-01-15T18:17:30.966041+0000","last_peered":"2024-01-15T18:17:30.966041+0000","last_clean":"2024-01-15T18:17:30.966041+0000","last_became_active":"2024-01-15T18:17:30.957078+0000","last_became_peered":"2024-01-15T18:17:30.957078+0000","last_unstale":"2024-01-15T18:17:30.966041+0000","last_undegraded":"2024-01-15T18:17:30.966041+0000","last_fullsized":"2024-01-15T18:17:30.966041+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:18.327933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978055+0000","last_change":"2024-01-15T18:17:30.973606+0000","last_active":"2024-01-15T18:17:30.978055+0000","last_peered":"2024-01-15T18:17:30.978055+0000","last_clean":"2024-01-15T18:17:30.978055+0000","last_became_active":"2024-01-15T18:17:30.971413+0000","last_became_peered":"2024-01-15T18:17:30.971413+0000","last_unstale":"2024-01-15T18:17:30.978055+0000","last_undegraded":"2024-01-15T18:17:30.978055+0000","last_fullsized":"2024-01-15T18:17:30.978055+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:28.411435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"99'4","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.973053+0000","last_change":"2024-01-15T18:17:29.942253+0000","last_active":"2024-01-15T18:17:30.973053+0000","last_peered":"2024-01-15T18:17:30.973053+0000","last_clean":"2024-01-15T18:17:30.973053+0000","last_became_active":"2024-01-15T18:17:29.941981+0000","last_became_peered":"2024-01-15T18:17:29.941981+0000","last_unstale":"2024-01-15T18:17:30.973053+0000","last_undegraded":"2024-01-15T18:17:30.973053+0000","last_fullsized":"2024-01-15T18:17:30.973053+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:00.199422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968710+0000","last_change":"2024-01-15T18:17:30.965909+0000","last_active":"2024-01-15T18:17:30.968710+0000","last_peered":"2024-01-15T18:17:30.968710+0000","last_clean":"2024-01-15T18:17:30.968710+0000","last_became_active":"2024-01-15T18:17:30.965640+0000","last_became_peered":"2024-01-15T18:17:30.965640+0000","last_unstale":"2024-01-15T18:17:30.968710+0000","last_undegraded":"2024-01-15T18:17:30.968710+0000","last_fullsized":"2024-01-15T18:17:30.968710+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:43.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964404+0000","last_change":"2024-01-15T18:17:29.941599+0000","last_active":"2024-01-15T18:17:30.964404+0000","last_peered":"2024-01-15T18:17:30.964404+0000","last_clean":"2024-01-15T18:17:30.964404+0000","last_became_active":"2024-01-15T18:17:29.941311+0000","last_became_peered":"2024-01-15T18:17:29.941311+0000","last_unstale":"2024-01-15T18:17:30.964404+0000","last_undegraded":"2024-01-15T18:17:30.964404+0000","last_fullsized":"2024-01-15T18:17:30.964404+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:37:27.097439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.962866+0000","last_change":"2024-01-15T18:17:30.958196+0000","last_active":"2024-01-15T18:17:30.962866+0000","last_peered":"2024-01-15T18:17:30.962866+0000","last_clean":"2024-01-15T18:17:30.962866+0000","last_became_active":"2024-01-15T18:17:30.957916+0000","last_became_peered":"2024-01-15T18:17:30.957916+0000","last_unstale":"2024-01-15T18:17:30.962866+0000","last_undegraded":"2024-01-15T18:17:30.962866+0000","last_fullsized":"2024-01-15T18:17:30.962866+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:05:40.291013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.971835+0000","last_change":"2024-01-15T18:17:29.950920+0000","last_active":"2024-01-15T18:17:30.971835+0000","last_peered":"2024-01-15T18:17:30.971835+0000","last_clean":"2024-01-15T18:17:30.971835+0000","last_became_active":"2024-01-15T18:17:29.950548+0000","last_became_peered":"2024-01-15T18:17:29.950548+0000","last_unstale":"2024-01-15T18:17:30.971835+0000","last_undegraded":"2024-01-15T18:17:30.971835+0000","last_fullsized":"2024-01-15T18:17:30.971835+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:33.414707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949399+0000","last_change":"2024-01-15T18:17:30.942088+0000","last_active":"2024-01-15T18:17:30.949399+0000","last_peered":"2024-01-15T18:17:30.949399+0000","last_clean":"2024-01-15T18:17:30.949399+0000","last_became_active":"2024-01-15T18:17:30.941775+0000","last_became_peered":"2024-01-15T18:17:30.941775+0000","last_unstale":"2024-01-15T18:17:30.949399+0000","last_undegraded":"2024-01-15T18:17:30.949399+0000","last_fullsized":"2024-01-15T18:17:30.949399+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:00.192627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.972869+0000","last_change":"2024-01-15T18:17:29.943225+0000","last_active":"2024-01-15T18:17:30.972869+0000","last_peered":"2024-01-15T18:17:30.972869+0000","last_clean":"2024-01-15T18:17:30.972869+0000","last_became_active":"2024-01-15T18:17:29.940937+0000","last_became_peered":"2024-01-15T18:17:29.940937+0000","last_unstale":"2024-01-15T18:17:30.972869+0000","last_undegraded":"2024-01-15T18:17:30.972869+0000","last_fullsized":"2024-01-15T18:17:30.972869+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:16.546618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.962563+0000","last_change":"2024-01-15T18:17:30.954274+0000","last_active":"2024-01-15T18:17:30.962563+0000","last_peered":"2024-01-15T18:17:30.962563+0000","last_clean":"2024-01-15T18:17:30.962563+0000","last_became_active":"2024-01-15T18:17:30.953891+0000","last_became_peered":"2024-01-15T18:17:30.953891+0000","last_unstale":"2024-01-15T18:17:30.962563+0000","last_undegraded":"2024-01-15T18:17:30.962563+0000","last_fullsized":"2024-01-15T18:17:30.962563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:06:59.329164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.971078+0000","last_change":"2024-01-15T18:17:29.938531+0000","last_active":"2024-01-15T18:17:30.971078+0000","last_peered":"2024-01-15T18:17:30.971078+0000","last_clean":"2024-01-15T18:17:30.971078+0000","last_became_active":"2024-01-15T18:17:29.938134+0000","last_became_peered":"2024-01-15T18:17:29.938134+0000","last_unstale":"2024-01-15T18:17:30.971078+0000","last_undegraded":"2024-01-15T18:17:30.971078+0000","last_fullsized":"2024-01-15T18:17:30.971078+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:41:33.929051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963949+0000","last_change":"2024-01-15T18:17:30.950683+0000","last_active":"2024-01-15T18:17:30.963949+0000","last_peered":"2024-01-15T18:17:30.963949+0000","last_clean":"2024-01-15T18:17:30.963949+0000","last_became_active":"2024-01-15T18:17:30.950259+0000","last_became_peered":"2024-01-15T18:17:30.950259+0000","last_unstale":"2024-01-15T18:17:30.963949+0000","last_undegraded":"2024-01-15T18:17:30.963949+0000","last_fullsized":"2024-01-15T18:17:30.963949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:28.174662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.966846+0000","last_change":"2024-01-15T18:17:29.945416+0000","last_active":"2024-01-15T18:17:30.966846+0000","last_peered":"2024-01-15T18:17:30.966846+0000","last_clean":"2024-01-15T18:17:30.966846+0000","last_became_active":"2024-01-15T18:17:29.945114+0000","last_became_peered":"2024-01-15T18:17:29.945114+0000","last_unstale":"2024-01-15T18:17:30.966846+0000","last_undegraded":"2024-01-15T18:17:30.966846+0000","last_fullsized":"2024-01-15T18:17:30.966846+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:14:59.182029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.959200+0000","last_change":"2024-01-15T18:17:30.953160+0000","last_active":"2024-01-15T18:17:30.959200+0000","last_peered":"2024-01-15T18:17:30.959200+0000","last_clean":"2024-01-15T18:17:30.959200+0000","last_became_active":"2024-01-15T18:17:30.952788+0000","last_became_peered":"2024-01-15T18:17:30.952788+0000","last_unstale":"2024-01-15T18:17:30.959200+0000","last_undegraded":"2024-01-15T18:17:30.959200+0000","last_fullsized":"2024-01-15T18:17:30.959200+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:34.692797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.932814+0000","last_change":"2024-01-15T18:17:29.935434+0000","last_active":"2024-01-15T18:17:30.932814+0000","last_peered":"2024-01-15T18:17:30.932814+0000","last_clean":"2024-01-15T18:17:30.932814+0000","last_became_active":"2024-01-15T18:17:29.934593+0000","last_became_peered":"2024-01-15T18:17:29.934593+0000","last_unstale":"2024-01-15T18:17:30.932814+0000","last_undegraded":"2024-01-15T18:17:30.932814+0000","last_fullsized":"2024-01-15T18:17:30.932814+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:33.414707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.957863+0000","last_change":"2024-01-15T18:17:30.949748+0000","last_active":"2024-01-15T18:17:30.957863+0000","last_peered":"2024-01-15T18:17:30.957863+0000","last_clean":"2024-01-15T18:17:30.957863+0000","last_became_active":"2024-01-15T18:17:30.949302+0000","last_became_peered":"2024-01-15T18:17:30.949302+0000","last_unstale":"2024-01-15T18:17:30.957863+0000","last_undegraded":"2024-01-15T18:17:30.957863+0000","last_fullsized":"2024-01-15T18:17:30.957863+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:36:40.270304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.943297+0000","last_change":"2024-01-15T18:17:29.938253+0000","last_active":"2024-01-15T18:17:30.943297+0000","last_peered":"2024-01-15T18:17:30.943297+0000","last_clean":"2024-01-15T18:17:30.943297+0000","last_became_active":"2024-01-15T18:17:29.937847+0000","last_became_peered":"2024-01-15T18:17:29.937847+0000","last_unstale":"2024-01-15T18:17:30.943297+0000","last_undegraded":"2024-01-15T18:17:30.943297+0000","last_fullsized":"2024-01-15T18:17:30.943297+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:45.493577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975627+0000","last_change":"2024-01-15T18:17:30.964507+0000","last_active":"2024-01-15T18:17:30.975627+0000","last_peered":"2024-01-15T18:17:30.975627+0000","last_clean":"2024-01-15T18:17:30.975627+0000","last_became_active":"2024-01-15T18:17:30.964235+0000","last_became_peered":"2024-01-15T18:17:30.964235+0000","last_unstale":"2024-01-15T18:17:30.975627+0000","last_undegraded":"2024-01-15T18:17:30.975627+0000","last_fullsized":"2024-01-15T18:17:30.975627+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:21.130381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.942546+0000","last_change":"2024-01-15T18:17:29.945885+0000","last_active":"2024-01-15T18:17:30.942546+0000","last_peered":"2024-01-15T18:17:30.942546+0000","last_clean":"2024-01-15T18:17:30.942546+0000","last_became_active":"2024-01-15T18:17:29.945535+0000","last_became_peered":"2024-01-15T18:17:29.945535+0000","last_unstale":"2024-01-15T18:17:30.942546+0000","last_undegraded":"2024-01-15T18:17:30.942546+0000","last_fullsized":"2024-01-15T18:17:30.942546+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:31:43.659519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.961092+0000","last_change":"2024-01-15T18:17:30.950971+0000","last_active":"2024-01-15T18:17:30.961092+0000","last_peered":"2024-01-15T18:17:30.961092+0000","last_clean":"2024-01-15T18:17:30.961092+0000","last_became_active":"2024-01-15T18:17:30.948559+0000","last_became_peered":"2024-01-15T18:17:30.948559+0000","last_unstale":"2024-01-15T18:17:30.961092+0000","last_undegraded":"2024-01-15T18:17:30.961092+0000","last_fullsized":"2024-01-15T18:17:30.961092+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:16.106025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.950596+0000","last_change":"2024-01-15T18:17:29.943048+0000","last_active":"2024-01-15T18:17:30.950596+0000","last_peered":"2024-01-15T18:17:30.950596+0000","last_clean":"2024-01-15T18:17:30.950596+0000","last_became_active":"2024-01-15T18:17:29.942602+0000","last_became_peered":"2024-01-15T18:17:29.942602+0000","last_unstale":"2024-01-15T18:17:30.950596+0000","last_undegraded":"2024-01-15T18:17:30.950596+0000","last_fullsized":"2024-01-15T18:17:30.950596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:14:59.182029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.950902+0000","last_change":"2024-01-15T18:17:30.945486+0000","last_active":"2024-01-15T18:17:30.950902+0000","last_peered":"2024-01-15T18:17:30.950902+0000","last_clean":"2024-01-15T18:17:30.950902+0000","last_became_active":"2024-01-15T18:17:30.945067+0000","last_became_peered":"2024-01-15T18:17:30.945067+0000","last_unstale":"2024-01-15T18:17:30.950902+0000","last_undegraded":"2024-01-15T18:17:30.950902+0000","last_fullsized":"2024-01-15T18:17:30.950902+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:34.709281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"99'3","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.967978+0000","last_change":"2024-01-15T18:17:29.933095+0000","last_active":"2024-01-15T18:17:30.967978+0000","last_peered":"2024-01-15T18:17:30.967978+0000","last_clean":"2024-01-15T18:17:30.967978+0000","last_became_active":"2024-01-15T18:17:29.932824+0000","last_became_peered":"2024-01-15T18:17:29.932824+0000","last_unstale":"2024-01-15T18:17:30.967978+0000","last_undegraded":"2024-01-15T18:17:30.967978+0000","last_fullsized":"2024-01-15T18:17:30.967978+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:37.286051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964306+0000","last_change":"2024-01-15T18:17:30.955749+0000","last_active":"2024-01-15T18:17:30.964306+0000","last_peered":"2024-01-15T18:17:30.964306+0000","last_clean":"2024-01-15T18:17:30.964306+0000","last_became_active":"2024-01-15T18:17:30.955428+0000","last_became_peered":"2024-01-15T18:17:30.955428+0000","last_unstale":"2024-01-15T18:17:30.964306+0000","last_undegraded":"2024-01-15T18:17:30.964306+0000","last_fullsized":"2024-01-15T18:17:30.964306+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:11.421513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"99'6","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.972210+0000","last_change":"2024-01-15T18:17:29.929962+0000","last_active":"2024-01-15T18:17:30.972210+0000","last_peered":"2024-01-15T18:17:30.972210+0000","last_clean":"2024-01-15T18:17:30.972210+0000","last_became_active":"2024-01-15T18:17:29.929647+0000","last_became_peered":"2024-01-15T18:17:29.929647+0000","last_unstale":"2024-01-15T18:17:30.972210+0000","last_undegraded":"2024-01-15T18:17:30.972210+0000","last_fullsized":"2024-01-15T18:17:30.972210+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:19:10.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.961235+0000","last_change":"2024-01-15T18:17:30.956453+0000","last_active":"2024-01-15T18:17:30.961235+0000","last_peered":"2024-01-15T18:17:30.961235+0000","last_clean":"2024-01-15T18:17:30.961235+0000","last_became_active":"2024-01-15T18:17:30.956174+0000","last_became_peered":"2024-01-15T18:17:30.956174+0000","last_unstale":"2024-01-15T18:17:30.961235+0000","last_undegraded":"2024-01-15T18:17:30.961235+0000","last_fullsized":"2024-01-15T18:17:30.961235+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:34.709281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.976514+0000","last_change":"2024-01-15T18:17:29.948422+0000","last_active":"2024-01-15T18:17:30.976514+0000","last_peered":"2024-01-15T18:17:30.976514+0000","last_clean":"2024-01-15T18:17:30.976514+0000","last_became_active":"2024-01-15T18:17:29.948121+0000","last_became_peered":"2024-01-15T18:17:29.948121+0000","last_unstale":"2024-01-15T18:17:30.976514+0000","last_undegraded":"2024-01-15T18:17:30.976514+0000","last_fullsized":"2024-01-15T18:17:30.976514+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:28.921159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.974239+0000","last_change":"2024-01-15T18:17:30.972425+0000","last_active":"2024-01-15T18:17:30.974239+0000","last_peered":"2024-01-15T18:17:30.974239+0000","last_clean":"2024-01-15T18:17:30.974239+0000","last_became_active":"2024-01-15T18:17:30.972016+0000","last_became_peered":"2024-01-15T18:17:30.972016+0000","last_unstale":"2024-01-15T18:17:30.974239+0000","last_undegraded":"2024-01-15T18:17:30.974239+0000","last_fullsized":"2024-01-15T18:17:30.974239+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:30.932645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958514+0000","last_change":"2024-01-15T18:17:29.950742+0000","last_active":"2024-01-15T18:17:30.958514+0000","last_peered":"2024-01-15T18:17:30.958514+0000","last_clean":"2024-01-15T18:17:30.958514+0000","last_became_active":"2024-01-15T18:17:29.950468+0000","last_became_peered":"2024-01-15T18:17:29.950468+0000","last_unstale":"2024-01-15T18:17:30.958514+0000","last_undegraded":"2024-01-15T18:17:30.958514+0000","last_fullsized":"2024-01-15T18:17:30.958514+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:07:10.425098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949462+0000","last_change":"2024-01-15T18:17:30.941430+0000","last_active":"2024-01-15T18:17:30.949462+0000","last_peered":"2024-01-15T18:17:30.949462+0000","last_clean":"2024-01-15T18:17:30.949462+0000","last_became_active":"2024-01-15T18:17:30.941040+0000","last_became_peered":"2024-01-15T18:17:30.941040+0000","last_unstale":"2024-01-15T18:17:30.949462+0000","last_undegraded":"2024-01-15T18:17:30.949462+0000","last_fullsized":"2024-01-15T18:17:30.949462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:43.095626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.935679+0000","last_change":"2024-01-15T18:17:29.926806+0000","last_active":"2024-01-15T18:17:30.935679+0000","last_peered":"2024-01-15T18:17:30.935679+0000","last_clean":"2024-01-15T18:17:30.935679+0000","last_became_active":"2024-01-15T18:17:29.926515+0000","last_became_peered":"2024-01-15T18:17:29.926515+0000","last_unstale":"2024-01-15T18:17:30.935679+0000","last_undegraded":"2024-01-15T18:17:30.935679+0000","last_fullsized":"2024-01-15T18:17:30.935679+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:00.199422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977998+0000","last_change":"2024-01-15T18:17:30.970932+0000","last_active":"2024-01-15T18:17:30.977998+0000","last_peered":"2024-01-15T18:17:30.977998+0000","last_clean":"2024-01-15T18:17:30.977998+0000","last_became_active":"2024-01-15T18:17:30.970588+0000","last_became_peered":"2024-01-15T18:17:30.970588+0000","last_unstale":"2024-01-15T18:17:30.977998+0000","last_undegraded":"2024-01-15T18:17:30.977998+0000","last_fullsized":"2024-01-15T18:17:30.977998+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:44:40.436564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"99'8","reported_seq":19,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.973403+0000","last_change":"2024-01-15T18:17:29.943639+0000","last_active":"2024-01-15T18:17:30.973403+0000","last_peered":"2024-01-15T18:17:30.973403+0000","last_clean":"2024-01-15T18:17:30.973403+0000","last_became_active":"2024-01-15T18:17:29.940051+0000","last_became_peered":"2024-01-15T18:17:29.940051+0000","last_unstale":"2024-01-15T18:17:30.973403+0000","last_undegraded":"2024-01-15T18:17:30.973403+0000","last_fullsized":"2024-01-15T18:17:30.973403+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:38:07.832563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.958158+0000","last_change":"2024-01-15T18:17:30.951848+0000","last_active":"2024-01-15T18:17:30.958158+0000","last_peered":"2024-01-15T18:17:30.958158+0000","last_clean":"2024-01-15T18:17:30.958158+0000","last_became_active":"2024-01-15T18:17:30.951562+0000","last_became_peered":"2024-01-15T18:17:30.951562+0000","last_unstale":"2024-01-15T18:17:30.958158+0000","last_undegraded":"2024-01-15T18:17:30.958158+0000","last_fullsized":"2024-01-15T18:17:30.958158+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:00.192627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.970270+0000","last_change":"2024-01-15T18:17:29.944762+0000","last_active":"2024-01-15T18:17:30.970270+0000","last_peered":"2024-01-15T18:17:30.970270+0000","last_clean":"2024-01-15T18:17:30.970270+0000","last_became_active":"2024-01-15T18:17:29.944486+0000","last_became_peered":"2024-01-15T18:17:29.944486+0000","last_unstale":"2024-01-15T18:17:30.970270+0000","last_undegraded":"2024-01-15T18:17:30.970270+0000","last_fullsized":"2024-01-15T18:17:30.970270+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:20.119784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.951164+0000","last_change":"2024-01-15T18:17:30.940311+0000","last_active":"2024-01-15T18:17:30.951164+0000","last_peered":"2024-01-15T18:17:30.951164+0000","last_clean":"2024-01-15T18:17:30.951164+0000","last_became_active":"2024-01-15T18:17:30.939949+0000","last_became_peered":"2024-01-15T18:17:30.939949+0000","last_unstale":"2024-01-15T18:17:30.951164+0000","last_undegraded":"2024-01-15T18:17:30.951164+0000","last_fullsized":"2024-01-15T18:17:30.951164+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:13.403235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964273+0000","last_change":"2024-01-15T18:17:29.934554+0000","last_active":"2024-01-15T18:17:30.964273+0000","last_peered":"2024-01-15T18:17:30.964273+0000","last_clean":"2024-01-15T18:17:30.964273+0000","last_became_active":"2024-01-15T18:17:29.933273+0000","last_became_peered":"2024-01-15T18:17:29.933273+0000","last_unstale":"2024-01-15T18:17:30.964273+0000","last_undegraded":"2024-01-15T18:17:30.964273+0000","last_fullsized":"2024-01-15T18:17:30.964273+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:13:40.320947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.965363+0000","last_change":"2024-01-15T18:17:30.962718+0000","last_active":"2024-01-15T18:17:30.965363+0000","last_peered":"2024-01-15T18:17:30.965363+0000","last_clean":"2024-01-15T18:17:30.965363+0000","last_became_active":"2024-01-15T18:17:30.962161+0000","last_became_peered":"2024-01-15T18:17:30.962161+0000","last_unstale":"2024-01-15T18:17:30.965363+0000","last_undegraded":"2024-01-15T18:17:30.965363+0000","last_fullsized":"2024-01-15T18:17:30.965363+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:29.158281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.943111+0000","last_change":"2024-01-15T18:17:29.943737+0000","last_active":"2024-01-15T18:17:30.943111+0000","last_peered":"2024-01-15T18:17:30.943111+0000","last_clean":"2024-01-15T18:17:30.943111+0000","last_became_active":"2024-01-15T18:17:29.943450+0000","last_became_peered":"2024-01-15T18:17:29.943450+0000","last_unstale":"2024-01-15T18:17:30.943111+0000","last_undegraded":"2024-01-15T18:17:30.943111+0000","last_fullsized":"2024-01-15T18:17:30.943111+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:37:27.097439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978459+0000","last_change":"2024-01-15T18:17:30.976128+0000","last_active":"2024-01-15T18:17:30.978459+0000","last_peered":"2024-01-15T18:17:30.978459+0000","last_clean":"2024-01-15T18:17:30.978459+0000","last_became_active":"2024-01-15T18:17:30.971307+0000","last_became_peered":"2024-01-15T18:17:30.971307+0000","last_unstale":"2024-01-15T18:17:30.978459+0000","last_undegraded":"2024-01-15T18:17:30.978459+0000","last_fullsized":"2024-01-15T18:17:30.978459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:34.367711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.971476+0000","last_change":"2024-01-15T18:17:29.947014+0000","last_active":"2024-01-15T18:17:30.971476+0000","last_peered":"2024-01-15T18:17:30.971476+0000","last_clean":"2024-01-15T18:17:30.971476+0000","last_became_active":"2024-01-15T18:17:29.946704+0000","last_became_peered":"2024-01-15T18:17:29.946704+0000","last_unstale":"2024-01-15T18:17:30.971476+0000","last_undegraded":"2024-01-15T18:17:30.971476+0000","last_fullsized":"2024-01-15T18:17:30.971476+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:41:39.161657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977666+0000","last_change":"2024-01-15T18:17:30.970718+0000","last_active":"2024-01-15T18:17:30.977666+0000","last_peered":"2024-01-15T18:17:30.977666+0000","last_clean":"2024-01-15T18:17:30.977666+0000","last_became_active":"2024-01-15T18:17:30.970409+0000","last_became_peered":"2024-01-15T18:17:30.970409+0000","last_unstale":"2024-01-15T18:17:30.977666+0000","last_undegraded":"2024-01-15T18:17:30.977666+0000","last_fullsized":"2024-01-15T18:17:30.977666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:21.191564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.932437+0000","last_change":"2024-01-15T18:17:29.939275+0000","last_active":"2024-01-15T18:17:30.932437+0000","last_peered":"2024-01-15T18:17:30.932437+0000","last_clean":"2024-01-15T18:17:30.932437+0000","last_became_active":"2024-01-15T18:17:29.938950+0000","last_became_peered":"2024-01-15T18:17:29.938950+0000","last_unstale":"2024-01-15T18:17:30.932437+0000","last_undegraded":"2024-01-15T18:17:30.932437+0000","last_fullsized":"2024-01-15T18:17:30.932437+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:42.085028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975811+0000","last_change":"2024-01-15T18:17:30.969018+0000","last_active":"2024-01-15T18:17:30.975811+0000","last_peered":"2024-01-15T18:17:30.975811+0000","last_clean":"2024-01-15T18:17:30.975811+0000","last_became_active":"2024-01-15T18:17:30.968689+0000","last_became_peered":"2024-01-15T18:17:30.968689+0000","last_unstale":"2024-01-15T18:17:30.975811+0000","last_undegraded":"2024-01-15T18:17:30.975811+0000","last_fullsized":"2024-01-15T18:17:30.975811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:57.839436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.931626+0000","last_change":"2024-01-15T18:17:29.936346+0000","last_active":"2024-01-15T18:17:30.931626+0000","last_peered":"2024-01-15T18:17:30.931626+0000","last_clean":"2024-01-15T18:17:30.931626+0000","last_became_active":"2024-01-15T18:17:29.935851+0000","last_became_peered":"2024-01-15T18:17:29.935851+0000","last_unstale":"2024-01-15T18:17:30.931626+0000","last_undegraded":"2024-01-15T18:17:30.931626+0000","last_fullsized":"2024-01-15T18:17:30.931626+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:25.933470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.956365+0000","last_change":"2024-01-15T18:17:30.941874+0000","last_active":"2024-01-15T18:17:30.956365+0000","last_peered":"2024-01-15T18:17:30.956365+0000","last_clean":"2024-01-15T18:17:30.956365+0000","last_became_active":"2024-01-15T18:17:30.941528+0000","last_became_peered":"2024-01-15T18:17:30.941528+0000","last_unstale":"2024-01-15T18:17:30.956365+0000","last_undegraded":"2024-01-15T18:17:30.956365+0000","last_fullsized":"2024-01-15T18:17:30.956365+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:16.106025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.944799+0000","last_change":"2024-01-15T18:17:29.948061+0000","last_active":"2024-01-15T18:17:30.944799+0000","last_peered":"2024-01-15T18:17:30.944799+0000","last_clean":"2024-01-15T18:17:30.944799+0000","last_became_active":"2024-01-15T18:17:29.945780+0000","last_became_peered":"2024-01-15T18:17:29.945780+0000","last_unstale":"2024-01-15T18:17:30.944799+0000","last_undegraded":"2024-01-15T18:17:30.944799+0000","last_fullsized":"2024-01-15T18:17:30.944799+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.255371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.967950+0000","last_change":"2024-01-15T18:17:30.960933+0000","last_active":"2024-01-15T18:17:30.967950+0000","last_peered":"2024-01-15T18:17:30.967950+0000","last_clean":"2024-01-15T18:17:30.967950+0000","last_became_active":"2024-01-15T18:17:30.960654+0000","last_became_peered":"2024-01-15T18:17:30.960654+0000","last_unstale":"2024-01-15T18:17:30.967950+0000","last_undegraded":"2024-01-15T18:17:30.967950+0000","last_fullsized":"2024-01-15T18:17:30.967950+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:16.106025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"99'2","reported_seq":13,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.968075+0000","last_change":"2024-01-15T18:17:29.932519+0000","last_active":"2024-01-15T18:17:30.968075+0000","last_peered":"2024-01-15T18:17:30.968075+0000","last_clean":"2024-01-15T18:17:30.968075+0000","last_became_active":"2024-01-15T18:17:29.931627+0000","last_became_peered":"2024-01-15T18:17:29.931627+0000","last_unstale":"2024-01-15T18:17:30.968075+0000","last_undegraded":"2024-01-15T18:17:30.968075+0000","last_fullsized":"2024-01-15T18:17:30.968075+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:33.698683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977580+0000","last_change":"2024-01-15T18:17:30.975437+0000","last_active":"2024-01-15T18:17:30.977580+0000","last_peered":"2024-01-15T18:17:30.977580+0000","last_clean":"2024-01-15T18:17:30.977580+0000","last_became_active":"2024-01-15T18:17:30.975018+0000","last_became_peered":"2024-01-15T18:17:30.975018+0000","last_unstale":"2024-01-15T18:17:30.977580+0000","last_undegraded":"2024-01-15T18:17:30.977580+0000","last_fullsized":"2024-01-15T18:17:30.977580+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:11:50.794125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.946360+0000","last_change":"2024-01-15T18:17:29.951765+0000","last_active":"2024-01-15T18:17:30.946360+0000","last_peered":"2024-01-15T18:17:30.946360+0000","last_clean":"2024-01-15T18:17:30.946360+0000","last_became_active":"2024-01-15T18:17:29.951464+0000","last_became_peered":"2024-01-15T18:17:29.951464+0000","last_unstale":"2024-01-15T18:17:30.946360+0000","last_undegraded":"2024-01-15T18:17:30.946360+0000","last_fullsized":"2024-01-15T18:17:30.946360+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:28.147683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.952852+0000","last_change":"2024-01-15T18:17:30.949656+0000","last_active":"2024-01-15T18:17:30.952852+0000","last_peered":"2024-01-15T18:17:30.952852+0000","last_clean":"2024-01-15T18:17:30.952852+0000","last_became_active":"2024-01-15T18:17:30.949383+0000","last_became_peered":"2024-01-15T18:17:30.949383+0000","last_unstale":"2024-01-15T18:17:30.952852+0000","last_undegraded":"2024-01-15T18:17:30.952852+0000","last_fullsized":"2024-01-15T18:17:30.952852+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:23.009053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.942371+0000","last_change":"2024-01-15T18:17:29.940472+0000","last_active":"2024-01-15T18:17:30.942371+0000","last_peered":"2024-01-15T18:17:30.942371+0000","last_clean":"2024-01-15T18:17:30.942371+0000","last_became_active":"2024-01-15T18:17:29.940126+0000","last_became_peered":"2024-01-15T18:17:29.940126+0000","last_unstale":"2024-01-15T18:17:30.942371+0000","last_undegraded":"2024-01-15T18:17:30.942371+0000","last_fullsized":"2024-01-15T18:17:30.942371+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:19:10.575142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963680+0000","last_change":"2024-01-15T18:17:30.956401+0000","last_active":"2024-01-15T18:17:30.963680+0000","last_peered":"2024-01-15T18:17:30.963680+0000","last_clean":"2024-01-15T18:17:30.963680+0000","last_became_active":"2024-01-15T18:17:30.956126+0000","last_became_peered":"2024-01-15T18:17:30.956126+0000","last_unstale":"2024-01-15T18:17:30.963680+0000","last_undegraded":"2024-01-15T18:17:30.963680+0000","last_fullsized":"2024-01-15T18:17:30.963680+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:00.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.931732+0000","last_change":"2024-01-15T18:17:29.931951+0000","last_active":"2024-01-15T18:17:30.931732+0000","last_peered":"2024-01-15T18:17:30.931732+0000","last_clean":"2024-01-15T18:17:30.931732+0000","last_became_active":"2024-01-15T18:17:29.931090+0000","last_became_peered":"2024-01-15T18:17:29.931090+0000","last_unstale":"2024-01-15T18:17:30.931732+0000","last_undegraded":"2024-01-15T18:17:30.931732+0000","last_fullsized":"2024-01-15T18:17:30.931732+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:19.442115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.977851+0000","last_change":"2024-01-15T18:17:30.970766+0000","last_active":"2024-01-15T18:17:30.977851+0000","last_peered":"2024-01-15T18:17:30.977851+0000","last_clean":"2024-01-15T18:17:30.977851+0000","last_became_active":"2024-01-15T18:17:30.970475+0000","last_became_peered":"2024-01-15T18:17:30.970475+0000","last_unstale":"2024-01-15T18:17:30.977851+0000","last_undegraded":"2024-01-15T18:17:30.977851+0000","last_fullsized":"2024-01-15T18:17:30.977851+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:34.330410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.943540+0000","last_change":"2024-01-15T18:17:29.941702+0000","last_active":"2024-01-15T18:17:30.943540+0000","last_peered":"2024-01-15T18:17:30.943540+0000","last_clean":"2024-01-15T18:17:30.943540+0000","last_became_active":"2024-01-15T18:17:29.941419+0000","last_became_peered":"2024-01-15T18:17:29.941419+0000","last_unstale":"2024-01-15T18:17:30.943540+0000","last_undegraded":"2024-01-15T18:17:30.943540+0000","last_fullsized":"2024-01-15T18:17:30.943540+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:31:11.677129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.949433+0000","last_change":"2024-01-15T18:17:30.940722+0000","last_active":"2024-01-15T18:17:30.949433+0000","last_peered":"2024-01-15T18:17:30.949433+0000","last_clean":"2024-01-15T18:17:30.949433+0000","last_became_active":"2024-01-15T18:17:30.940442+0000","last_became_peered":"2024-01-15T18:17:30.940442+0000","last_unstale":"2024-01-15T18:17:30.949433+0000","last_undegraded":"2024-01-15T18:17:30.949433+0000","last_fullsized":"2024-01-15T18:17:30.949433+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:13:41.331545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.931150+0000","last_change":"2024-01-15T18:17:29.935189+0000","last_active":"2024-01-15T18:17:30.931150+0000","last_peered":"2024-01-15T18:17:30.931150+0000","last_clean":"2024-01-15T18:17:30.931150+0000","last_became_active":"2024-01-15T18:17:29.934834+0000","last_became_peered":"2024-01-15T18:17:29.934834+0000","last_unstale":"2024-01-15T18:17:30.931150+0000","last_undegraded":"2024-01-15T18:17:30.931150+0000","last_fullsized":"2024-01-15T18:17:30.931150+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:12.392637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.975791+0000","last_change":"2024-01-15T18:17:30.968893+0000","last_active":"2024-01-15T18:17:30.975791+0000","last_peered":"2024-01-15T18:17:30.975791+0000","last_clean":"2024-01-15T18:17:30.975791+0000","last_became_active":"2024-01-15T18:17:30.968571+0000","last_became_peered":"2024-01-15T18:17:30.968571+0000","last_unstale":"2024-01-15T18:17:30.975791+0000","last_undegraded":"2024-01-15T18:17:30.975791+0000","last_fullsized":"2024-01-15T18:17:30.975791+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:43:47.698719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.948605+0000","last_change":"2024-01-15T18:17:29.947332+0000","last_active":"2024-01-15T18:17:30.948605+0000","last_peered":"2024-01-15T18:17:30.948605+0000","last_clean":"2024-01-15T18:17:30.948605+0000","last_became_active":"2024-01-15T18:17:29.947063+0000","last_became_peered":"2024-01-15T18:17:29.947063+0000","last_unstale":"2024-01-15T18:17:30.948605+0000","last_undegraded":"2024-01-15T18:17:30.948605+0000","last_fullsized":"2024-01-15T18:17:30.948605+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:36:39.259706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.963566+0000","last_change":"2024-01-15T18:17:30.954209+0000","last_active":"2024-01-15T18:17:30.963566+0000","last_peered":"2024-01-15T18:17:30.963566+0000","last_clean":"2024-01-15T18:17:30.963566+0000","last_became_active":"2024-01-15T18:17:30.950464+0000","last_became_peered":"2024-01-15T18:17:30.950464+0000","last_unstale":"2024-01-15T18:17:30.963566+0000","last_undegraded":"2024-01-15T18:17:30.963566+0000","last_fullsized":"2024-01-15T18:17:30.963566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:36:40.270304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.954094+0000","last_change":"2024-01-15T18:17:29.951425+0000","last_active":"2024-01-15T18:17:30.954094+0000","last_peered":"2024-01-15T18:17:30.954094+0000","last_clean":"2024-01-15T18:17:30.954094+0000","last_became_active":"2024-01-15T18:17:29.951157+0000","last_became_peered":"2024-01-15T18:17:29.951157+0000","last_unstale":"2024-01-15T18:17:30.954094+0000","last_undegraded":"2024-01-15T18:17:30.954094+0000","last_fullsized":"2024-01-15T18:17:30.954094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:14.347302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.978035+0000","last_change":"2024-01-15T18:17:30.973568+0000","last_active":"2024-01-15T18:17:30.978035+0000","last_peered":"2024-01-15T18:17:30.978035+0000","last_clean":"2024-01-15T18:17:30.978035+0000","last_became_active":"2024-01-15T18:17:30.971112+0000","last_became_peered":"2024-01-15T18:17:30.971112+0000","last_unstale":"2024-01-15T18:17:30.978035+0000","last_undegraded":"2024-01-15T18:17:30.978035+0000","last_fullsized":"2024-01-15T18:17:30.978035+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:29.915273+0000","last_clean_scrub_stamp":"2024-01-15T18:17:29.915273+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:02:59.523457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"99'1","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:17:30.964288+0000","last_change":"2024-01-15T18:17:29.936213+0000","last_active":"2024-01-15T18:17:30.964288+0000","last_peered":"2024-01-15T18:17:30.964288+0000","last_clean":"2024-01-15T18:17:30.964288+0000","last_became_active":"2024-01-15T18:17:29.935918+0000","last_became_peered":"2024-01-15T18:17:29.935918+0000","last_unstale":"2024-01-15T18:17:30.964288+0000","last_undegraded":"2024-01-15T18:17:30.964288+0000","last_fullsized":"2024-01-15T18:17:30.964288+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:28.904675+0000","last_clean_scrub_stamp":"2024-01-15T18:17:28.904675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.255371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":232,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-01-15T18:17:29.925909+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:17:29.925909+0000","last_peered":"2024-01-15T18:17:29.925909+0000","last_clean":"2024-01-15T18:17:29.925909+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:17:29.925909+0000","last_undegraded":"2024-01-15T18:17:29.925909+0000","last_fullsized":"2024-01-15T18:17:29.925909+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64836,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307004,"statfs":{"total":96636764160,"available":96570372096,"internally_reserved":0,"allocated":1826816,"data_stored":1119638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":1.1180000000000001,"15min":0.92300000000000004},"min":{"1min":0.46100000000000002,"5min":0.39600000000000002,"15min":0.29599999999999999},"max":{"1min":0.98999999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.96799999999999997,"15min":0.86499999999999999},"min":{"1min":0.495,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.88800000000000001,"5min":8.5419999999999998,"15min":13.744},"last":0.59799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":1.288,"15min":1.0629999999999999},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":0.90500000000000003,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":1.264,"15min":1.0609999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92700000000000005,"5min":31.774999999999999,"15min":31.774999999999999},"last":0.628}]},{"osd":2,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.76700000000000002,"15min":0.99299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.143,"5min":2.1680000000000001,"15min":30.846},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":1.0940000000000001,"15min":1.0800000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":0.98099999999999998,"5min":17.039999999999999,"15min":39.695},"last":0.64100000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":1.1919999999999999,"15min":1.1699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":1.6020000000000001,"5min":19.131,"15min":26.463999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.0700000000000001,"15min":1.2130000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94499999999999995,"5min":16.879999999999999,"15min":42.692},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.84899999999999998,"15min":0.71199999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":14.087,"15min":14.087},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.64200000000000002,"15min":0.68799999999999994},"min":{"1min":0.39600000000000002,"5min":0.37,"15min":0.36299999999999999},"max":{"1min":0.77000000000000002,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.48299999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.81599999999999995,"15min":0.72399999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":0.90300000000000002,"5min":11.334,"15min":11.334},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.83899999999999997,"15min":0.77100000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.38800000000000001},"max":{"1min":0.745,"5min":8.423,"15min":8.423},"last":0.39900000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.872,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.88500000000000001,"5min":8.5030000000000001,"15min":8.5030000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.70699999999999996,"15min":0.73099999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86499999999999999,"5min":1.7909999999999999,"15min":3.3450000000000002},"last":0.55200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64240,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307600,"statfs":{"total":96636764160,"available":96570982400,"internally_reserved":0,"allocated":1212416,"data_stored":522327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.85099999999999998,"15min":0.95399999999999996},"min":{"1min":0.44900000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.4670000000000001,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.75900000000000001,"15min":0.876},"min":{"1min":0.375,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.96499999999999997,"5min":3.4940000000000002,"15min":12.449},"last":0.627}]},{"osd":1,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.91300000000000003,"15min":0.89900000000000002},"min":{"1min":0.41199999999999998,"5min":0.34300000000000003,"15min":0.33900000000000002},"max":{"1min":2.0710000000000002,"5min":6.8239999999999998,"15min":14.246},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":1.1439999999999999,"15min":1.0109999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.5389999999999999,"5min":19.475000000000001,"15min":19.475000000000001},"last":0.73399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":1.107,"15min":1.1830000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.016,"5min":14.743,"15min":42.825000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.91900000000000004,"15min":1.153},"min":{"1min":0.47499999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.994,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.70699999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78700000000000003,"15min":1.03},"min":{"1min":0.441,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":1.911,"5min":3.016,"15min":24.199000000000002},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.98499999999999999,"5min":1.087,"15min":1.081},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":5.0609999999999999,"5min":18.791,"15min":18.791},"last":0.72199999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.88900000000000001,"15min":0.751},"min":{"1min":0.35499999999999998,"5min":0.35399999999999998,"15min":0.33300000000000002},"max":{"1min":1.417,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.65200000000000002,"15min":0.753},"min":{"1min":0.374,"5min":0.374,"15min":0.33700000000000002},"max":{"1min":0.85899999999999999,"5min":1.7450000000000001,"15min":12.491},"last":0.47799999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.442,"5min":0.373,"15min":0.34799999999999998},"max":{"1min":1.9410000000000001,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.434},{"interface":"front","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.90500000000000003},"min":{"1min":0.53700000000000003,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":1.1040000000000001,"5min":2.5259999999999998,"15min":12.945},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:17:05 2024","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":1.0029999999999999,"15min":0.84799999999999998},"min":{"1min":0.52100000000000002,"5min":0.40300000000000002,"15min":0.38600000000000001},"max":{"1min":1.9830000000000001,"5min":11.907,"15min":11.907},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82499999999999996,"15min":0.89300000000000002},"min":{"1min":0.55300000000000005,"5min":0.39600000000000002,"15min":0.34200000000000003},"max":{"1min":1.7929999999999999,"5min":9.9440000000000008,"15min":27.651},"last":0.46600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575005,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54580,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317260,"statfs":{"total":96636764160,"available":96580874240,"internally_reserved":0,"allocated":1212416,"data_stored":522327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.91700000000000004,"15min":0.83199999999999996},"min":{"1min":0.42799999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.7450000000000001,"5min":6.6820000000000004,"15min":11.519},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.012,"15min":0.92400000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":3.1699999999999999,"5min":7.6260000000000003,"15min":12.952},"last":0.84499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":1.0089999999999999,"15min":0.90300000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.37},"max":{"1min":3.2080000000000002,"5min":4.609,"15min":13.358000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.94199999999999995,"15min":0.95099999999999996},"min":{"1min":0.40000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":4.1580000000000004,"5min":12.428000000000001,"15min":21.856999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":1.167,"15min":0.93500000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":2.5499999999999998,"5min":18.314,"15min":18.314},"last":0.755},{"interface":"front","average":{"1min":0.77900000000000003,"5min":1.361,"15min":1.0109999999999999},"min":{"1min":0.51600000000000001,"5min":0.47899999999999998,"15min":0.434},"max":{"1min":2.02,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.79500000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.92600000000000005,"15min":0.94899999999999995},"min":{"1min":0.503,"5min":0.45500000000000002,"15min":0.42499999999999999},"max":{"1min":3.2400000000000002,"5min":4.0330000000000004,"15min":13.317},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":1.0600000000000001,"15min":1.0089999999999999},"min":{"1min":0.56599999999999995,"5min":0.443,"15min":0.42899999999999999},"max":{"1min":2.5099999999999998,"5min":15.205,"15min":19.638000000000002},"last":0.82099999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":0.77300000000000002,"15min":0.72299999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":3.1240000000000001,"5min":5.899,"15min":11.093},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73799999999999999,"15min":0.69599999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":2.3580000000000001,"5min":3.097,"15min":3.3889999999999998},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.86799999999999999,"15min":0.80200000000000005},"min":{"1min":0.55000000000000004,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":2.4129999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77000000000000002,"15min":0.73499999999999999},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.308},"max":{"1min":1.635,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.88100000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.81899999999999995,"15min":0.76900000000000002},"min":{"1min":0.44500000000000001,"5min":0.42399999999999999,"15min":0.33200000000000002},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.90800000000000003,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":2.4660000000000002,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76756,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295084,"statfs":{"total":96636764160,"available":96558166016,"internally_reserved":0,"allocated":1245184,"data_stored":530894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.98699999999999999,"15min":0.93600000000000005},"min":{"1min":0.438,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":1.151,"5min":11.374000000000001,"15min":34.802},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":1.1279999999999999,"15min":0.91100000000000003},"min":{"1min":0.45700000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":2.6760000000000002,"5min":14.488,"15min":22.335999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.81599999999999995,"15min":0.96499999999999997},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.873,"5min":3.9380000000000002,"15min":38.198},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.89000000000000001,"15min":0.96099999999999997},"min":{"1min":0.52200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.98199999999999998,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.89100000000000001,"15min":0.86799999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.39600000000000002},"max":{"1min":1.6259999999999999,"5min":7.4530000000000003,"15min":7.6989999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.90700000000000003,"15min":0.85499999999999998},"min":{"1min":0.48099999999999998,"5min":0.45300000000000001,"15min":0.38700000000000001},"max":{"1min":1.595,"5min":7.4829999999999997,"15min":7.4829999999999997},"last":0.65900000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":1.2030000000000001,"15min":0.95699999999999996},"min":{"1min":0.51900000000000002,"5min":0.48499999999999999,"15min":0.40300000000000002},"max":{"1min":1.9339999999999999,"5min":14.529,"15min":14.529},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.995,"15min":0.93000000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.41999999999999998},"max":{"1min":1.008,"5min":25.774000000000001,"15min":25.774000000000001},"last":0.69599999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.77400000000000002,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":0.875,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.97999999999999998,"15min":0.82899999999999996},"min":{"1min":0.51900000000000002,"5min":0.40400000000000003,"15min":0.34100000000000003},"max":{"1min":0.82199999999999995,"5min":25.887,"15min":25.887},"last":0.53600000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.38100000000000001},"max":{"1min":0.78300000000000003,"5min":7.4930000000000003,"15min":10.284000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.70399999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.50800000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.83299999999999996,"15min":0.78700000000000003},"min":{"1min":0.46899999999999997,"5min":0.40999999999999998,"15min":0.30399999999999999},"max":{"1min":0.89600000000000002,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.70599999999999996,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.36199999999999999},"max":{"1min":0.84199999999999997,"5min":1.232,"15min":11.67},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64264,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307576,"statfs":{"total":96636764160,"available":96570957824,"internally_reserved":0,"allocated":1241088,"data_stored":530792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":1.234,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.26600000000000001,"15min":0.255},"max":{"1min":0.77900000000000003,"5min":26.364000000000001,"15min":26.364000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.91400000000000003,"15min":0.95299999999999996},"min":{"1min":0.315,"5min":0.25,"15min":0.246},"max":{"1min":0.63800000000000001,"5min":12.247999999999999,"15min":28.332000000000001},"last":0.42199999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.88300000000000001,"15min":0.98299999999999998},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":2.6869999999999998,"5min":12.545,"15min":40.457999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":1.3029999999999999,"15min":1.1419999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":1.2769999999999999,"5min":26.260999999999999,"15min":27.427},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":1.0169999999999999,"15min":0.98599999999999999},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.72299999999999998,"5min":12.209,"15min":23.036999999999999},"last":0.377},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.5329999999999999,"15min":1.0860000000000001},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":0.81399999999999995,"5min":26.404,"15min":26.404},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.054,"15min":1.0249999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.41199999999999998},"max":{"1min":1.304,"5min":12.073,"15min":22.998000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.042,"15min":1.0780000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.827,"5min":7.149,"15min":20.399000000000001},"last":0.53900000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.4350000000000001,"15min":1.099},"min":{"1min":0.52400000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.093,"5min":26.314,"15min":26.314},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":1.0429999999999999,"15min":1.1259999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91600000000000004,"5min":12.173999999999999,"15min":22.908999999999999},"last":0.63400000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.2450000000000001,"15min":1.1539999999999999},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.42799999999999999},"max":{"1min":6.843,"5min":7.2809999999999997,"15min":20.445},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.6080000000000001,"15min":1.1899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":6.7640000000000002,"5min":26.091999999999999,"15min":26.091999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Mon Jan 15 18:17:09 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":1.204,"15min":1.1599999999999999},"min":{"1min":0.55500000000000005,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.1879999999999999,"5min":12.129,"15min":23.111999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.2190000000000001,"15min":1.3240000000000001},"min":{"1min":0.58199999999999996,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":2.9340000000000002,"5min":10.920999999999999,"15min":44.817999999999998},"last":0.66900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64256,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307584,"statfs":{"total":96636764160,"available":96570966016,"internally_reserved":0,"allocated":1228800,"data_stored":529192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.63700000000000001,"15min":1.0509999999999999},"min":{"1min":0.27300000000000002,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.73699999999999999,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.94299999999999995,"15min":1.3240000000000001},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.33800000000000002},"max":{"1min":0.875,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.496}]},{"osd":1,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.746,"15min":1.5429999999999999},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":1.954,"5min":7.476,"15min":113.149},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":1.046,"15min":1.107},"min":{"1min":0.37,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.82699999999999996,"5min":20.619,"15min":41.302},"last":0.40400000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":1.5660000000000001,"15min":1.377},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.28899999999999998},"max":{"1min":1.506,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.74399999999999999,"15min":1.2669999999999999},"min":{"1min":0.437,"5min":0.374,"15min":0.34200000000000003},"max":{"1min":1.0960000000000001,"5min":7.3609999999999998,"15min":115.693},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.98799999999999999,"15min":1.212},"min":{"1min":0.497,"5min":0.47899999999999998,"15min":0.47199999999999998},"max":{"1min":0.98999999999999999,"5min":7.4169999999999998,"15min":24.366},"last":0.746},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.202,"15min":1.2350000000000001},"min":{"1min":0.51700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":6.2469999999999999,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":1.8049999999999999,"15min":1.5089999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":4.1699999999999999,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":1.085,"15min":1.3100000000000001},"min":{"1min":0.54900000000000004,"5min":0.497,"15min":0.496},"max":{"1min":1.032,"5min":8.5239999999999991,"15min":26.084},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.373,"15min":1.3240000000000001},"min":{"1min":0.51800000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3049999999999997,"5min":22.199000000000002,"15min":22.199000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.849,"15min":1.573},"min":{"1min":0.59199999999999997,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":4.0919999999999996,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.84199999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:17:06 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.3160000000000001,"15min":1.268},"min":{"1min":0.60599999999999998,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":6.1210000000000004,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.962,"15min":1.639},"min":{"1min":0.56499999999999995,"5min":0.55900000000000005,"15min":0.48799999999999999},"max":{"1min":3.4489999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64252,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307588,"statfs":{"total":96636764160,"available":96570970112,"internally_reserved":0,"allocated":1224704,"data_stored":529158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.68799999999999994,"15min":0.93400000000000005},"min":{"1min":0.27700000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":9.452,"5min":9.452,"15min":39.948},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.75900000000000001,"15min":1.1200000000000001},"min":{"1min":0.30499999999999999,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":2.6989999999999998,"5min":8.141,"15min":35.790999999999997},"last":0.45300000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":0.76400000000000001,"15min":0.998},"min":{"1min":0.34300000000000003,"5min":0.30499999999999999,"15min":0.27600000000000002},"max":{"1min":15.137,"5min":15.137,"15min":49.229999999999997},"last":0.498},{"interface":"front","average":{"1min":1.113,"5min":0.81299999999999994,"15min":0.875},"min":{"1min":0.36799999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":9.8469999999999995,"5min":9.8469999999999995,"15min":10.349},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.77000000000000002,"15min":1.1899999999999999},"min":{"1min":0.41999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.258,"5min":11.901,"15min":27.643999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.95299999999999996,"15min":1.0189999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":9.6519999999999992,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.97399999999999998,"15min":1.244},"min":{"1min":0.48299999999999998,"5min":0.47199999999999998,"15min":0.40200000000000002},"max":{"1min":1.1739999999999999,"5min":7.4450000000000003,"15min":27.489999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.304,"5min":1.091,"15min":1.089},"min":{"1min":0.56799999999999995,"5min":0.495,"15min":0.47699999999999998},"max":{"1min":9.6989999999999998,"5min":9.6989999999999998,"15min":12.997},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.99099999999999999,"15min":1.2230000000000001},"min":{"1min":0.52700000000000002,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.298,"5min":7.3380000000000001,"15min":27.547000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.1020000000000001,"15min":1.1439999999999999},"min":{"1min":0.63,"5min":0.55400000000000005,"15min":0.496},"max":{"1min":9.7650000000000006,"5min":9.7650000000000006,"15min":13.054},"last":0.68700000000000006}]},{"osd":6,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.96899999999999997,"15min":1.22},"min":{"1min":0.59399999999999997,"5min":0.50800000000000001,"15min":0.46500000000000002},"max":{"1min":1.226,"5min":7.5,"15min":27.579999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.02,"5min":0.93300000000000005,"15min":1.0589999999999999},"min":{"1min":0.56299999999999994,"5min":0.50900000000000001,"15min":0.42399999999999999},"max":{"1min":5.9020000000000001,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:17:08 2024","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0669999999999999,"15min":1.1279999999999999},"min":{"1min":0.60799999999999998,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":5.9779999999999998,"5min":9.5660000000000007,"15min":11.833},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.351,"5min":1.081,"15min":1.1579999999999999},"min":{"1min":0.64400000000000002,"5min":0.505,"15min":0.502},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":14.606},"last":0.65000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77312,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294528,"statfs":{"total":96636764160,"available":96557596672,"internally_reserved":0,"allocated":1822720,"data_stored":1120108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.86199999999999999,"15min":1.1679999999999999},"min":{"1min":0.33300000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.60799999999999998,"5min":11.356,"15min":31.977},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.97799999999999998,"15min":1.0209999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":4.0250000000000004,"5min":13.018000000000001,"15min":36.145000000000003},"last":0.495}]},{"osd":2,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.96099999999999997,"15min":1.1299999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":5.8929999999999998,"5min":13.166,"15min":41.387999999999998},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.91700000000000004,"15min":1.0489999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.25},"max":{"1min":3.484,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.51900000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":1.038,"15min":1.113},"min":{"1min":0.39400000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.841,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.97499999999999998,"15min":1.0960000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.33300000000000002},"max":{"1min":5.9800000000000004,"5min":12.954000000000001,"15min":45.128},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":1.079,"15min":1.083},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.372},"max":{"1min":1.877,"5min":11.772,"15min":32.932000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":1.119,"15min":1.206},"min":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":0.93999999999999995,"5min":11.220000000000001,"15min":31.782},"last":1.0309999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":1.1200000000000001,"15min":1.2},"min":{"1min":0.47799999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.103,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":1.1060000000000001,"15min":1.079},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.794,"5min":11.958,"15min":33.151000000000003},"last":0.872}]},{"osd":6,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":1.099,"15min":1.0660000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":4.0970000000000004,"5min":13.055,"15min":13.055},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.117,"15min":1.1080000000000001},"min":{"1min":0.49399999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.5960000000000001,"5min":11.833,"15min":33.098999999999997},"last":0.91900000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:17:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.115,"15min":1.0780000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.501},"max":{"1min":4.1349999999999998,"5min":13.090999999999999,"15min":13.090999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":1.135,"15min":1.256},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.0069999999999999,"5min":11.321999999999999,"15min":31.905999999999999},"last":1.0129999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:17:35.396 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:17:35.396 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:17:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 Processor -- start 2024-01-15T18:17:35.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 -- start start 2024-01-15T18:17:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f622015d110 0x7f62200cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f622006bf50 0x7f62200b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622006c490 con 0x7f622015d4f0 2024-01-15T18:17:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62200b0e80 con 0x7f622015d110 2024-01-15T18:17:35.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f6225600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622006ca50 con 0x7f622006bf50 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f621ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f622015d110 0x7f62200cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f621e000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f622015d4f0 0x7f62200cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41946/0 (socket says 172.21.15.143:41946) 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.580+0000 7f621e000640 1 -- 172.21.15.143:0/346234836 learned_addr learned my addr 172.21.15.143:0/346234836 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621f400640 1 --2- 172.21.15.143:0/346234836 >> v2:172.21.15.204:3300/0 conn(0x7f622006bf50 0x7f62200b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621ea00640 1 -- 172.21.15.143:0/346234836 >> v2:172.21.15.204:3300/0 conn(0x7f622006bf50 msgr2=0x7f62200b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621ea00640 1 --2- 172.21.15.143:0/346234836 >> v2:172.21.15.204:3300/0 conn(0x7f622006bf50 0x7f62200b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621ea00640 1 -- 172.21.15.143:0/346234836 >> v1:172.21.15.143:6789/0 conn(0x7f622015d4f0 legacy=0x7f62200cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621ea00640 1 -- 172.21.15.143:0/346234836 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622006c770 con 0x7f622015d110 2024-01-15T18:17:35.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621f400640 1 --2- 172.21.15.143:0/346234836 >> v2:172.21.15.204:3300/0 conn(0x7f622006bf50 0x7f62200b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621ea00640 1 --2- 172.21.15.143:0/346234836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622015d110 0x7f62200cf130 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f622006df10 tx=0x7f621001ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8d445648d169e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621d600640 1 -- 172.21.15.143:0/346234836 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100ab600 con 0x7f622015d110 2024-01-15T18:17:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621d600640 1 -- 172.21.15.143:0/346234836 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62100a7050 con 0x7f622015d110 2024-01-15T18:17:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.584+0000 7f621d600640 1 -- 172.21.15.143:0/346234836 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100ab980 con 0x7f622015d110 2024-01-15T18:17:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- 172.21.15.143:0/346234836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622015d110 msgr2=0x7f62200cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 --2- 172.21.15.143:0/346234836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622015d110 0x7f62200cf130 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f622006df10 tx=0x7f621001ce30 comp rx=0 tx=0).stop 2024-01-15T18:17:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- 172.21.15.143:0/346234836 shutdown_connections 2024-01-15T18:17:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 --2- 172.21.15.143:0/346234836 >> v2:172.21.15.204:3300/0 conn(0x7f622006bf50 0x7f62200b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 --2- 172.21.15.143:0/346234836 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622015d110 0x7f62200cf130 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- 172.21.15.143:0/346234836 >> 172.21.15.143:0/346234836 conn(0x7f62200b2a40 msgr2=0x7f62200b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- 172.21.15.143:0/346234836 shutdown_connections 2024-01-15T18:17:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- 172.21.15.143:0/346234836 wait complete. 2024-01-15T18:17:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 Processor -- start 2024-01-15T18:17:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- start start 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 0x7f622015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f622015d110 0x7f622006ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f622015d4f0 0x7f622006f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622015bd30 con 0x7f622006bf50 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62200b3cd0 con 0x7f622015d4f0 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f6225600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622015b120 con 0x7f622015d110 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 0x7f622015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 0x7f622015a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40198/0 (socket says 172.21.15.143:40198) 2024-01-15T18:17:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 -- 172.21.15.143:0/22498314 learned_addr learned my addr 172.21.15.143:0/22498314 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621f400640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f622015d4f0 0x7f622006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 -- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f622015d110 msgr2=0x7f622006ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621e000640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f622015d110 0x7f622006ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f622015d110 0x7f622006ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 -- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f622015d4f0 msgr2=0x7f622006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f622015d4f0 0x7f622006f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621ea00640 1 -- 172.21.15.143:0/22498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210009420 con 0x7f622006bf50 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.588+0000 7f621f400640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f622015d4f0 0x7f622006f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f621ea00640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 0x7f622015a740 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f62100ae040 tx=0x7f621001ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100b7450 con 0x7f622006bf50 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62100b75b0 con 0x7f622006bf50 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62100b7940 con 0x7f622006bf50 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622015b400 con 0x7f622006bf50 2024-01-15T18:17:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622015b9c0 con 0x7f622006bf50 2024-01-15T18:17:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6220067500 con 0x7f622006bf50 2024-01-15T18:17:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.592+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f62100b7aa0 con 0x7f622006bf50 2024-01-15T18:17:35.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.596+0000 7f6216a00640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61f8067820 0x7f61f8069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.596+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f621011d840 con 0x7f622006bf50 2024-01-15T18:17:35.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.596+0000 7f621e000640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61f8067820 0x7f61f8069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.596+0000 7f621e000640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61f8067820 0x7f61f8069ce0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6220156d80 tx=0x7f6208009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.600+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f62100ec410 con 0x7f622006bf50 2024-01-15T18:17:35.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.800+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f62200b6e70 con 0x7f622006bf50 2024-01-15T18:17:35.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6216a00640 1 -- 172.21.15.143:0/22498314 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+147 (secure 0 0 0) 0x7f62100efa30 con 0x7f622006bf50 2024-01-15T18:17:35.801 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:17:35.801 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:17:35.801 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:17:35.801 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:17:35.801 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:17:35.801 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61f8067820 msgr2=0x7f61f8069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61f8067820 0x7f61f8069ce0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6220156d80 tx=0x7f6208009040 comp rx=0 tx=0).stop 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 msgr2=0x7f622015a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 0x7f622015a740 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f62100ae040 tx=0x7f621001ac10 comp rx=0 tx=0).stop 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 shutdown_connections 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f61f8067820 0x7f61f8069ce0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f622015d4f0 0x7f622006f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f622015d110 0x7f622006ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 --2- 172.21.15.143:0/22498314 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f622006bf50 0x7f622015a740 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 >> 172.21.15.143:0/22498314 conn(0x7f62200b2a40 msgr2=0x7f62200cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 shutdown_connections 2024-01-15T18:17:35.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.804+0000 7f6225600640 1 -- 172.21.15.143:0/22498314 wait complete. 2024-01-15T18:17:35.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:17:35.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f189ba00640 1 Processor -- start 2024-01-15T18:17:35.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f189ba00640 1 -- start start 2024-01-15T18:17:35.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f189ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18940aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f189ba00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1894160eb0 con 0x7f189415d510 2024-01-15T18:17:35.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f189ba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1894161190 con 0x7f189406aba0 2024-01-15T18:17:35.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f189ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1894161470 con 0x7f189415d130 2024-01-15T18:17:35.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f1893e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f189415d510 0x7f189406a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41960/0 (socket says 172.21.15.143:41960) 2024-01-15T18:17:35.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.960+0000 7f1893e00640 1 -- 172.21.15.143:0/2506564590 learned_addr learned my addr 172.21.15.143:0/2506564590 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899000640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18940aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560187335 0 0) 0x7f1894160eb0 con 0x7f189415d510 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1874003530 con 0x7f189415d510 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971845972 0 0) 0x7f1894161190 con 0x7f189406aba0 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1894160eb0 con 0x7f189406aba0 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702101427 0 0) 0x7f1874003530 con 0x7f189415d510 2024-01-15T18:17:35.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 msgr2=0x7f18940aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18940aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 >> v1:172.21.15.204:6789/0 conn(0x7f189406aba0 legacy=0x7f18940b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894161750 con 0x7f189415d510 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f187c001600 con 0x7f189415d510 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 >> v1:172.21.15.143:6789/0 conn(0x7f189415d510 legacy=0x7f189406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1874003b40 0x7f1874003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f1874005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894161470 con 0x7f1874004d10 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187400ba40 con 0x7f1874003b40 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187400bc00 con 0x7f189415d130 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899a00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f1874005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893e00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899000640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1874003b40 0x7f1874003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899a00640 1 -- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 msgr2=0x7f18740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899a00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899a00640 1 -- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1874003b40 msgr2=0x7f1874003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899a00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1874003b40 0x7f1874003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1899a00640 1 -- 172.21.15.143:0/2506564590 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874003530 con 0x7f1874004d10 2024-01-15T18:17:35.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.964+0000 7f1893e00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f189415d130 0x7f18740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f1899a00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f1874005190 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f188c001fd0 tx=0x7f188c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147c60f991940b49 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c003b80 con 0x7f1874004d10 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c003d00 con 0x7f1874004d10 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f1893400640 1 -- 172.21.15.143:0/2506564590 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c005310 con 0x7f1874004d10 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 -- 172.21.15.143:0/2506564590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 msgr2=0x7f1874005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 --2- 172.21.15.143:0/2506564590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f1874005190 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f188c001fd0 tx=0x7f188c01c820 comp rx=0 tx=0).stop 2024-01-15T18:17:35.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f1899a00640 1 -- 172.21.15.143:0/2506564590 reap_dead start 2024-01-15T18:17:35.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 -- 172.21.15.143:0/2506564590 shutdown_connections 2024-01-15T18:17:35.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 -- 172.21.15.143:0/2506564590 >> 172.21.15.143:0/2506564590 conn(0x7f18940b1b90 msgr2=0x7f18940b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 -- 172.21.15.143:0/2506564590 shutdown_connections 2024-01-15T18:17:35.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 -- 172.21.15.143:0/2506564590 wait complete. 2024-01-15T18:17:35.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 Processor -- start 2024-01-15T18:17:35.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.968+0000 7f189ba00640 1 -- start start 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f189ba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f18941437a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f189ba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18941563f0 0x7f1894156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f189ba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f189415dbd0 0x7f189415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f189ba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189415e900 con 0x7f1874004d10 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f189ba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1894156db0 con 0x7f189415dbd0 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f189ba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1894159810 con 0x7f18941563f0 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f18941437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f18941437a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40224/0 (socket says 172.21.15.143:40224) 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 -- 172.21.15.143:0/2830282138 learned_addr learned my addr 172.21.15.143:0/2830282138 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899a00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f189415dbd0 0x7f189415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1893e00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18941563f0 0x7f1894156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 -- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18941563f0 msgr2=0x7f1894156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18941563f0 0x7f1894156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 -- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f189415dbd0 msgr2=0x7f189415e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f189415dbd0 0x7f189415e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 -- 172.21.15.143:0/2830282138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894159ca0 con 0x7f1874004d10 2024-01-15T18:17:35.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.972+0000 7f1899000640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f18941437a0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f1884011960 tx=0x7f1884011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400faf0 con 0x7f1874004d10 2024-01-15T18:17:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188400fc50 con 0x7f1874004d10 2024-01-15T18:17:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1894158ee0 con 0x7f1874004d10 2024-01-15T18:17:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884014440 con 0x7f1874004d10 2024-01-15T18:17:35.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941594a0 con 0x7f1874004d10 2024-01-15T18:17:35.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1894067500 con 0x7f1874004d10 2024-01-15T18:17:35.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f18840145a0 con 0x7f1874004d10 2024-01-15T18:17:35.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1890c00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f186c067820 0x7f186c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1884083640 con 0x7f1874004d10 2024-01-15T18:17:35.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1893e00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f186c067820 0x7f186c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.976+0000 7f1893e00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f186c067820 0x7f186c069ce0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f187c001730 tx=0x7f187c001480 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:35.980+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f18840523e0 con 0x7f1874004d10 2024-01-15T18:17:36.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18940b2ff0 con 0x7f1874004d10 2024-01-15T18:17:36.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f1890c00640 1 -- 172.21.15.143:0/2830282138 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f1884055a00 con 0x7f1874004d10 2024-01-15T18:17:36.162 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:17:36.162 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":62,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1910039233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1910039233},{"type":"v1","addr":"172.21.15.204:6835","nonce":1910039233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2986890926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2986890926},{"type":"v1","addr":"172.21.15.143:6837","nonce":2986890926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2769510067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2769510067},{"type":"v1","addr":"172.21.15.204:6834","nonce":2769510067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:17:30.310521+0000","modified":"2024-01-15T18:17:33.332084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8535},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8535":{"gid":8535,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6835/4145140990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":4145140990},{"type":"v1","addr":"172.21.15.143:6835","nonce":4145140990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-01-15T18:17:36.162 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 62 2024-01-15T18:17:36.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f186c067820 msgr2=0x7f186c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f186c067820 0x7f186c069ce0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f187c001730 tx=0x7f187c001480 comp rx=0 tx=0).stop 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 msgr2=0x7f18941437a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f18941437a0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f1884011960 tx=0x7f1884011e20 comp rx=0 tx=0).stop 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 shutdown_connections 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f186c067820 0x7f186c069ce0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f189415dbd0 0x7f189415e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f18941563f0 0x7f1894156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 --2- 172.21.15.143:0/2830282138 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1874004d10 0x7f18941437a0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 >> 172.21.15.143:0/2830282138 conn(0x7f18940b1b90 msgr2=0x7f18940cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 shutdown_connections 2024-01-15T18:17:36.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.164+0000 7f189ba00640 1 -- 172.21.15.143:0/2830282138 wait complete. 2024-01-15T18:17:36.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:17:30.310521+0000', 'modified': '2024-01-15T18:17:33.332084+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8535}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8535': {'gid': 8535, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.143:6835/4145140990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 4145140990}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 4145140990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:17:36.587 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:17:36.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-15T18:17:36.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f24a00640 1 Processor -- start 2024-01-15T18:17:36.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f24a00640 1 -- start start 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f24a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1f1c0c6230 0x7f1f1c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f24a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c0c4d90 con 0x7f1f1c0cdf20 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f24a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1c15b640 con 0x7f1f1c15b260 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f24a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1c0cf800 con 0x7f1f1c0c6230 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f21600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1f1c0cdf20 0x7f1f1c0cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41974/0 (socket says 172.21.15.143:41974) 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f21600640 1 -- 172.21.15.143:0/2763636049 learned_addr learned my addr 172.21.15.143:0/2763636049 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:36.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f22a00640 1 --2- 172.21.15.143:0/2763636049 >> v2:172.21.15.204:3300/0 conn(0x7f1f1c0c6230 0x7f1f1c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690169118 0 0) 0x7f1f1c0c4d90 con 0x7f1f1c0cdf20 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.728+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1efc003530 con 0x7f1f1c0cdf20 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 83761231 0 0) 0x7f1f1c15b640 con 0x7f1f1c15b260 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f1c0c4d90 con 0x7f1f1c15b260 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f22a00640 1 -- 172.21.15.143:0/2763636049 >> v1:172.21.15.204:6789/0 conn(0x7f1f1c15b260 legacy=0x7f1f1c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f22a00640 1 -- 172.21.15.143:0/2763636049 >> v1:172.21.15.143:6789/0 conn(0x7f1f1c0cdf20 legacy=0x7f1f1c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f22a00640 1 -- 172.21.15.143:0/2763636049 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1c0cfae0 con 0x7f1f1c0c6230 2024-01-15T18:17:36.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f22a00640 1 --2- 172.21.15.143:0/2763636049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0c6230 0x7f1f1c0cc1f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009d90 tx=0x7f1f1401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c0e0c698c6a53e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140a59e0 con 0x7f1f1c0c6230 2024-01-15T18:17:36.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c0ccf40 con 0x7f1f1c0c6230 2024-01-15T18:17:36.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.732+0000 7f1f20c00640 1 -- 172.21.15.143:0/2763636049 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140a5d60 con 0x7f1f1c0c6230 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- 172.21.15.143:0/2763636049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0c6230 msgr2=0x7f1f1c0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 --2- 172.21.15.143:0/2763636049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0c6230 0x7f1f1c0cc1f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009d90 tx=0x7f1f1401c920 comp rx=0 tx=0).stop 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- 172.21.15.143:0/2763636049 shutdown_connections 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 --2- 172.21.15.143:0/2763636049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0c6230 0x7f1f1c0cc1f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- 172.21.15.143:0/2763636049 >> 172.21.15.143:0/2763636049 conn(0x7f1f1c0d0ed0 msgr2=0x7f1f1c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- 172.21.15.143:0/2763636049 shutdown_connections 2024-01-15T18:17:36.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- 172.21.15.143:0/2763636049 wait complete. 2024-01-15T18:17:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 Processor -- start 2024-01-15T18:17:36.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- start start 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f1c0c6230 0x7f1f1c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0cdf20 0x7f1f1c0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 0x7f1f1c1f9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0c9150 con 0x7f1f1c15b260 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b4d60 con 0x7f1f1c0cdf20 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f24a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c0b47a0 con 0x7f1f1c0c6230 2024-01-15T18:17:36.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 0x7f1f1c1f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 0x7f1f1c1f9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40230/0 (socket says 172.21.15.143:40230) 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 -- 172.21.15.143:0/3700636155 learned_addr learned my addr 172.21.15.143:0/3700636155 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f21600640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0cdf20 0x7f1f1c0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22000640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f1c0c6230 0x7f1f1c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 -- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f1c0c6230 msgr2=0x7f1f1c06ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f1c0c6230 0x7f1f1c06ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 -- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0cdf20 msgr2=0x7f1f1c0b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0cdf20 0x7f1f1c0b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22a00640 1 -- 172.21.15.143:0/3700636155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14009a70 con 0x7f1f1c15b260 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.736+0000 7f1f22000640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f1c0c6230 0x7f1f1c06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f22a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 0x7f1f1c1f9910 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1f140b0040 tx=0x7f1f140196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f1a000640 1 -- 172.21.15.143:0/3700636155 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140b9860 con 0x7f1f1c15b260 2024-01-15T18:17:36.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f1a000640 1 -- 172.21.15.143:0/3700636155 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f140b99c0 con 0x7f1f1c15b260 2024-01-15T18:17:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c0b4a80 con 0x7f1f1c15b260 2024-01-15T18:17:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f1a000640 1 -- 172.21.15.143:0/3700636155 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140b9d70 con 0x7f1f1c15b260 2024-01-15T18:17:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c0b4400 con 0x7f1f1c15b260 2024-01-15T18:17:36.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f1a000640 1 -- 172.21.15.143:0/3700636155 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f1f140c4460 con 0x7f1f1c15b260 2024-01-15T18:17:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f1a000640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ef4067820 0x7f1ef4069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f1a000640 1 -- 172.21.15.143:0/3700636155 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1f1411e3f0 con 0x7f1f1c15b260 2024-01-15T18:17:36.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f22000640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ef4067820 0x7f1ef4069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.740+0000 7f1f27d95000 1 Processor -- start 2024-01-15T18:17:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- start start 2024-01-15T18:17:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22000640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ef4067820 0x7f1ef4069ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0011f0 tx=0x7f1f0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239ed9c90 0x55c239eda070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239e8fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239e90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239cfa130 con 0x55c239e90320 2024-01-15T18:17:36.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ce79a0 con 0x55c239eda5b0 2024-01-15T18:17:36.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c239ce4ed0 con 0x55c239ed9c90 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239e90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239e90700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40238/0 (socket says 172.21.15.143:40238) 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 -- 172.21.15.143:0/2672580057 learned_addr learned my addr 172.21.15.143:0/2672580057 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22000640 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239ed9c90 0x55c239eda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f21600640 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239e8fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 -- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239ed9c90 msgr2=0x55c239eda070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239ed9c90 0x55c239eda070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 -- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 msgr2=0x55c239e8fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239e8fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 -- 172.21.15.143:0/2672580057 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2399d8890 con 0x55c239e90320 2024-01-15T18:17:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f22a00640 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239e90700 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009d90 tx=0x7f1f140f06c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d7d68b592f78fc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f12a00640 1 -- 172.21.15.143:0/2672580057 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140f0a50 con 0x55c239e90320 2024-01-15T18:17:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f12a00640 1 -- 172.21.15.143:0/2672580057 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f140f4ba0 con 0x55c239e90320 2024-01-15T18:17:36.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f12a00640 1 -- 172.21.15.143:0/2672580057 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140f8d50 con 0x55c239e90320 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- 172.21.15.143:0/2672580057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 msgr2=0x55c239e90700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239e90700 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f1f14009d90 tx=0x7f1f140f06c0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- 172.21.15.143:0/2672580057 shutdown_connections 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239e90700 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239e8fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 --2- 172.21.15.143:0/2672580057 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239ed9c90 0x55c239eda070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- 172.21.15.143:0/2672580057 >> 172.21.15.143:0/2672580057 conn(0x55c239e8c2d0 msgr2=0x55c239e8e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- 172.21.15.143:0/2672580057 shutdown_connections 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.744+0000 7f1f27d95000 1 -- 172.21.15.143:0/2672580057 wait complete. 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- start start 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 Processor -- start 2024-01-15T18:17:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e90320 0x55c239e94d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 0x55c239e95250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239ea8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ce07c0 con 0x55c239ed9c90 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239c98bf0 con 0x55c239eda5b0 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c239c9ea90 con 0x55c239e90320 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 10 client.0 _ll_register_callbacks cb 0x55c239e7b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 0x55c239e95250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 0x55c239e95250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40246/0 (socket says 172.21.15.143:40246) 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 -- 172.21.15.143:0/3709617875 learned_addr learned my addr 172.21.15.143:0/3709617875 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f22000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e90320 0x55c239e94d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f22a00640 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239ea8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 -- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e90320 msgr2=0x55c239e94d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e90320 0x55c239e94d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 -- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 msgr2=0x55c239ea8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239ea8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 -- 172.21.15.143:0/3709617875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c0b4400 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f21600640 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 0x55c239e95250 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001e60 tx=0x7f1f040116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f610acb9ac1fd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f0b400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f0b400640 1 -- 172.21.15.143:0/3709617875 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04001280 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f0b400640 1 -- 172.21.15.143:0/3709617875 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1f04005a70 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c239ca2470 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f0b400640 1 -- 172.21.15.143:0/3709617875 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f04011eb0 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 10 client.8670 fetch_fsmap learned FSMap version 62 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55c239ca1190 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f0b400640 1 -- 172.21.15.143:0/3709617875 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c239ca2470 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f0b400640 1 -- 172.21.15.143:0/3709617875 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f04005c30 con 0x55c239ed9c90 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 10 client.8670 fetch_fsmap finished waiting for FSMap version 62 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 10 client.8670 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 20 client.8670 populate_metadata read hostname 'smithi143' 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239ea9320 0x55c239eebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 4 client.8670 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:17:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 --> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c239ca5b50 con 0x55c239ea9320 2024-01-15T18:17:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.748+0000 7f1f22000640 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239ea9320 0x55c239eebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f22000640 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239ea9320 0x55c239eebf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a3e43ebd2cf49c server_cookie=823ae290c6f7c065 in_seq=0 out_seq=0 2024-01-15T18:17:36.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f0b400640 10 client.8670 ms_handle_connect on v2:172.21.15.143:6834/4145140990 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f0b400640 1 -- 172.21.15.143:0/3709617875 <== mds.0 v2:172.21.15.143:6834/4145140990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c239ca5b50 con 0x55c239ea9320 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f0b400640 10 client.8670 handle_command_reply: tid=0 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 client.8670 shutdown 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 msgr2=0x55c239e95250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 0x55c239e95250 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f1f04001e60 tx=0x7f1f040116e0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 shutdown_connections 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239ea9320 0x55c239eebf50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239eda5b0 0x55c239ea8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239ed9c90 0x55c239e95250 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f0b400640 0 client.8670 ms_handle_reset on v2:172.21.15.143:6834/4145140990 2024-01-15T18:17:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3709617875 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e90320 0x55c239e94d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 >> 172.21.15.143:0/3709617875 conn(0x55c239e8c2d0 msgr2=0x55c239e8e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 shutdown_connections 2024-01-15T18:17:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 1 -- 172.21.15.143:0/3709617875 wait complete. 2024-01-15T18:17:36.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.752+0000 7f1f27d95000 20 client.8670 trim_cache size 0 max 16384 2024-01-15T18:17:36.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 Processor -- start 2024-01-15T18:17:36.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 -- start start 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ed7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239ed8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239ebc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239999940 con 0x55c239e90320 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ea88c0 con 0x55c239ed9c90 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c239e97890 con 0x55c239e8c2d0 2024-01-15T18:17:36.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239ed8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239ed8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40254/0 (socket says 172.21.15.143:40254) 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 -- 172.21.15.143:0/843990137 learned_addr learned my addr 172.21.15.143:0/843990137 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f22000640 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ed7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f22a00640 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239ebc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 -- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 msgr2=0x55c239ed7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ed7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 -- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 msgr2=0x55c239ebc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239ebc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 -- 172.21.15.143:0/843990137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c239c87490 con 0x55c239e90320 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f22a00640 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239ebc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f21600640 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239ed8070 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1f04000eb0 tx=0x7f1f0400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ce6e19a4049298 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f13400640 1 -- 172.21.15.143:0/843990137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04011eb0 con 0x55c239e90320 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f13400640 1 -- 172.21.15.143:0/843990137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04011be0 con 0x55c239e90320 2024-01-15T18:17:36.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.768+0000 7f1f13400640 1 -- 172.21.15.143:0/843990137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040011d0 con 0x55c239e90320 2024-01-15T18:17:36.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- 172.21.15.143:0/843990137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 msgr2=0x55c239ed8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239ed8070 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f1f04000eb0 tx=0x7f1f0400c000 comp rx=0 tx=0).stop 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- 172.21.15.143:0/843990137 shutdown_connections 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239ebc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239ed8070 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- 172.21.15.143:0/843990137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ed7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- 172.21.15.143:0/843990137 >> 172.21.15.143:0/843990137 conn(0x55c239e94ee0 msgr2=0x55c239e8e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- 172.21.15.143:0/843990137 shutdown_connections 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- 172.21.15.143:0/843990137 wait complete. 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- start start 2024-01-15T18:17:36.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 Processor -- start 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ebcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239eebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239eec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239cddbc0 con 0x55c239e90320 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c239ca9910 con 0x55c239ed9c90 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c239e8bc70 con 0x55c239e8c2d0 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 10 client.0 _ll_register_callbacks cb 0x55c239d2ec00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239eebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239eebf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40270/0 (socket says 172.21.15.143:40270) 2024-01-15T18:17:36.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 -- 172.21.15.143:0/3954080073 learned_addr learned my addr 172.21.15.143:0/3954080073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f22000640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ebcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f22a00640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239eec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 -- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 msgr2=0x55c239ebcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ebcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 -- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 msgr2=0x55c239eec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239eec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 -- 172.21.15.143:0/3954080073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c239ca1190 con 0x55c239e90320 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f21600640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239eebf50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011720 tx=0x7f1f04001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6c95105baae921 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f0b400640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f0b400640 1 -- 172.21.15.143:0/3954080073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f040074a0 con 0x55c239e90320 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f0b400640 1 -- 172.21.15.143:0/3954080073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1f04007600 con 0x55c239e90320 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.772+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c239cdf110 con 0x55c239e90320 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 1 -- 172.21.15.143:0/3954080073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f04005a70 con 0x55c239e90320 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 10 client.8676 fetch_fsmap learned FSMap version 62 2024-01-15T18:17:36.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55c2399d8cc0 con 0x55c239e90320 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 1 -- 172.21.15.143:0/3954080073 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c239cdf110 con 0x55c239e90320 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 1 -- 172.21.15.143:0/3954080073 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1f04005bd0 con 0x55c239e90320 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 10 client.8676 fetch_fsmap finished waiting for FSMap version 62 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 10 client.8676 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 20 client.8676 populate_metadata read hostname 'smithi143' 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239eda5b0 0x55c239f42e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 4 client.8676 mds_command: new command op to 8535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 --> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c239d0e9a0 con 0x55c239eda5b0 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f22000640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239eda5b0 0x55c239f42e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f22000640 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239eda5b0 0x55c239f42e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=240e4310b8717c8d server_cookie=ce588968f3ebd780 in_seq=0 out_seq=0 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 10 client.8676 ms_handle_connect on v2:172.21.15.143:6834/4145140990 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 1 -- 172.21.15.143:0/3954080073 <== mds.0 v2:172.21.15.143:6834/4145140990 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c239d0e9a0 con 0x55c239eda5b0 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 10 client.8676 handle_command_reply: tid=0 2024-01-15T18:17:36.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 client.8676 shutdown 2024-01-15T18:17:36.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 msgr2=0x55c239eebf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239eebf50 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f1f04011720 tx=0x7f1f04001160 comp rx=0 tx=0).stop 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 shutdown_connections 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:6834/4145140990,v1:172.21.15.143:6835/4145140990] conn(0x55c239eda5b0 0x55c239f42e40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c239ed9c90 0x55c239eec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c239e90320 0x55c239eebf50 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f0b400640 0 client.8676 ms_handle_reset on v2:172.21.15.143:6834/4145140990 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 --2- 172.21.15.143:0/3954080073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55c239e8c2d0 0x55c239ebcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 >> 172.21.15.143:0/3954080073 conn(0x55c239e94ee0 msgr2=0x55c239e8ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 shutdown_connections 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 1 -- 172.21.15.143:0/3954080073 wait complete. 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f27d95000 20 client.8676 trim_cache size 0 max 16384 2024-01-15T18:17:36.778 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 8535, 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 5.8475249490000003, 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 62, 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 100, 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 100, 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 10.778584821000001 2024-01-15T18:17:36.779 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ef4067820 msgr2=0x7f1ef4069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ef4067820 0x7f1ef4069ce0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0011f0 tx=0x7f1f0c009040 comp rx=0 tx=0).stop 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 msgr2=0x7f1f1c1f9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 0x7f1f1c1f9910 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1f140b0040 tx=0x7f1f140196c0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 shutdown_connections 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ef4067820 0x7f1ef4069ce0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1f1c15b260 0x7f1f1c1f9910 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f1c0cdf20 0x7f1f1c0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 --2- 172.21.15.143:0/3700636155 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1f1c0c6230 0x7f1f1c06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 >> 172.21.15.143:0/3700636155 conn(0x7f1f1c0d0ed0 msgr2=0x7f1f1c158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 shutdown_connections 2024-01-15T18:17:36.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.776+0000 7f1f24a00640 1 -- 172.21.15.143:0/3700636155 wait complete. 2024-01-15T18:17:36.796 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:17:36.903 INFO:tasks.ceph.mds.a.smithi143.stderr:2024-01-15T18:17:36.904+0000 7fd51cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:17:36.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 Processor -- start 2024-01-15T18:17:36.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 -- start start 2024-01-15T18:17:36.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc8406bfb0 0x7fbc8406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc840d0210 0x7fbc840ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840cbff0 con 0x7fbc840b0310 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc840d0c50 con 0x7fbc8406bfb0 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc8b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8406c8d0 con 0x7fbc840d0210 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc8406bfb0 0x7fbc8406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc8406bfb0 0x7fbc8406c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40272/0 (socket says 172.21.15.143:40272) 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 -- 172.21.15.143:0/3622666613 learned_addr learned my addr 172.21.15.143:0/3622666613 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc89000640 1 --2- 172.21.15.143:0/3622666613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc840d0210 0x7fbc840ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 -- 172.21.15.143:0/3622666613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc840d0210 msgr2=0x7fbc840ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 --2- 172.21.15.143:0/3622666613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc840d0210 0x7fbc840ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 -- 172.21.15.143:0/3622666613 >> v1:172.21.15.204:6789/0 conn(0x7fbc840b0310 legacy=0x7fbc840cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.936+0000 7fbc83e00640 1 -- 172.21.15.143:0/3622666613 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840cb6c0 con 0x7fbc8406bfb0 2024-01-15T18:17:36.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc83e00640 1 --2- 172.21.15.143:0/3622666613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc8406bfb0 0x7fbc8406c390 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009870 tx=0x7fbc7401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161b5c40641e0479 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc82a00640 1 -- 172.21.15.143:0/3622666613 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740ad640 con 0x7fbc8406bfb0 2024-01-15T18:17:36.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc82a00640 1 -- 172.21.15.143:0/3622666613 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc740ad7a0 con 0x7fbc8406bfb0 2024-01-15T18:17:36.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc82a00640 1 -- 172.21.15.143:0/3622666613 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740adb30 con 0x7fbc8406bfb0 2024-01-15T18:17:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc8b000640 1 -- 172.21.15.143:0/3622666613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc8406bfb0 msgr2=0x7fbc8406c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc8b000640 1 --2- 172.21.15.143:0/3622666613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc8406bfb0 0x7fbc8406c390 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009870 tx=0x7fbc7401c480 comp rx=0 tx=0).stop 2024-01-15T18:17:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc8b000640 1 -- 172.21.15.143:0/3622666613 shutdown_connections 2024-01-15T18:17:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc8b000640 1 --2- 172.21.15.143:0/3622666613 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc840d0210 0x7fbc840ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc8b000640 1 --2- 172.21.15.143:0/3622666613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc8406bfb0 0x7fbc8406c390 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.940+0000 7fbc8b000640 1 -- 172.21.15.143:0/3622666613 >> 172.21.15.143:0/3622666613 conn(0x7fbc840d0ed0 msgr2=0x7fbc840d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 -- 172.21.15.143:0/3622666613 shutdown_connections 2024-01-15T18:17:36.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 -- 172.21.15.143:0/3622666613 wait complete. 2024-01-15T18:17:36.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 Processor -- start 2024-01-15T18:17:36.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 -- start start 2024-01-15T18:17:36.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc8406bfb0 0x7fbc84156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 0x7fbc8415a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc840d0210 0x7fbc8415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84159dc0 con 0x7fbc840b0310 2024-01-15T18:17:36.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84156740 con 0x7fbc840d0210 2024-01-15T18:17:36.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc8b000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc841591d0 con 0x7fbc8406bfb0 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 0x7fbc8415a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 0x7fbc8415a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40288/0 (socket says 172.21.15.143:40288) 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 -- 172.21.15.143:0/2119512493 learned_addr learned my addr 172.21.15.143:0/2119512493 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc89000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc840d0210 0x7fbc8415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83e00640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc8406bfb0 0x7fbc84156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 -- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc8406bfb0 msgr2=0x7fbc84156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc8406bfb0 0x7fbc84156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 -- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc840d0210 msgr2=0x7fbc8415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc840d0210 0x7fbc8415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 -- 172.21.15.143:0/2119512493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc740094d0 con 0x7fbc840b0310 2024-01-15T18:17:36.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.944+0000 7fbc83400640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 0x7fbc8415a250 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c390 tx=0x7fbc6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0052d0 con 0x7fbc840b0310 2024-01-15T18:17:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c005430 con 0x7fbc840b0310 2024-01-15T18:17:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc84159490 con 0x7fbc840b0310 2024-01-15T18:17:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c003820 con 0x7fbc840b0310 2024-01-15T18:17:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc84159a50 con 0x7fbc840b0310 2024-01-15T18:17:36.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc84159f70 con 0x7fbc840b0310 2024-01-15T18:17:36.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fbc6c003980 con 0x7fbc840b0310 2024-01-15T18:17:36.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc7be00640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc5c067850 0x7fbc5c069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbc6c082310 con 0x7fbc840b0310 2024-01-15T18:17:36.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.948+0000 7fbc83e00640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc5c067850 0x7fbc5c069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.952+0000 7fbc83e00640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc5c067850 0x7fbc5c069d10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbc740af000 tx=0x7fbc74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:36.952+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc84159f70 con 0x7fbc840b0310 2024-01-15T18:17:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.140+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbc840ce660 con 0x7fbc840b0310 2024-01-15T18:17:37.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc7be00640 1 -- 172.21.15.143:0/2119512493 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbc6c0510b0 con 0x7fbc840b0310 2024-01-15T18:17:37.141 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc5c067850 msgr2=0x7fbc5c069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc5c067850 0x7fbc5c069d10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbc740af000 tx=0x7fbc74008040 comp rx=0 tx=0).stop 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 msgr2=0x7fbc8415a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 0x7fbc8415a250 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c390 tx=0x7fbc6c00c850 comp rx=0 tx=0).stop 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 shutdown_connections 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc5c067850 0x7fbc5c069d10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc840d0210 0x7fbc8415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc840b0310 0x7fbc8415a250 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 --2- 172.21.15.143:0/2119512493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc8406bfb0 0x7fbc84156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 >> 172.21.15.143:0/2119512493 conn(0x7fbc840d0ed0 msgr2=0x7fbc840afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:37.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 shutdown_connections 2024-01-15T18:17:37.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:37.144+0000 7fbc8b000640 1 -- 172.21.15.143:0/2119512493 wait complete. 2024-01-15T18:17:37.163 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:17:37.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:17:37.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:17:37.163 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:17:37.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:37.163 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:17:37.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:17:37.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:17:37.164 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:17:37.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.164 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:17:37.173 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:17:37.173 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:17:37.174 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 41686sec preferred_lft 41686sec 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:17:37.175 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:17:37.176 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.176 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:17:37.176 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:17:37.176 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:17:37.176 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:37.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.176 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:17:37.238 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:17:37.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.240 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:17:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:17:37.300 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:17:37.300 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:17:37.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.300 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:17:37.300 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:17:37.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:17:37.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:17:37.300 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:17:37.300 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:17:37.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.389 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:17:37.398 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:17:37.398 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:17:37.398 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:17:37.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:17:37.459 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:17:37.459 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:17:37.459 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:37.512 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:17:37.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:17:37.512 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:37.563 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:17:37.624 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:17:37.678 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:17:37.679 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:17:37.680 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:17:37.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.681 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:17:37.732 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:17:37.732 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:17:37.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:17:37.799 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:17:37.850 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:17:37.851 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:17:37.852 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:17:37.852 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:17:37.852 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:17:37.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:37.853 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:17:37.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:17:37.872+0000 7f4b322ec580 -1 init, newargv = 0x55583a429090 newargc=15 2024-01-15T18:17:37.872 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[51204]: starting ceph client 2024-01-15T18:17:37.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[51204]: starting fuse 2024-01-15T18:17:37.881 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:17:37.881 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:17:37.881 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:17:37.882 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:17:37.883 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:17:37.883 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:17:38.007 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.51204.asok 2024-01-15T18:17:38.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:17:38.019 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51204.asok 2024-01-15T18:17:38.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51204.asok status 2024-01-15T18:17:38.187 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:17:38.187 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:17:38.187 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:17:38.187 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "51204", 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 8691, 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 8691 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:38.188 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2726899 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 2726899 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.8691 192.168.0.1:0/2726899", 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/2726899", 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 62, 2024-01-15T18:17:38.189 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 100, 2024-01-15T18:17:38.190 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:17:38.190 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:17:38.190 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:17:38.190 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:17:38.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.203 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:17:38.203 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:38.212 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:17:38.212 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:38.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:38.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:17:38.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9eb000640 1 Processor -- start 2024-01-15T18:17:38.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9eb000640 1 -- start start 2024-01-15T18:17:38.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9eb000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e415f310 con 0x7fe9e40ce560 2024-01-15T18:17:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9eb000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e415f5f0 con 0x7fe9e4141660 2024-01-15T18:17:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9eb000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e415fbb0 con 0x7fe9e4072410 2024-01-15T18:17:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9e3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe9e40ce560 0x7fe9e4071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51368/0 (socket says 172.21.15.143:51368) 2024-01-15T18:17:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9e3400640 1 -- 172.21.15.143:0/125005060 learned_addr learned my addr 172.21.15.143:0/125005060 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9e9000640 1 --2- 172.21.15.143:0/125005060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394824269 0 0) 0x7fe9e415f310 con 0x7fe9e40ce560 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c4003530 con 0x7fe9e40ce560 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.452+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948818213 0 0) 0x7fe9e415f5f0 con 0x7fe9e4141660 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e415f310 con 0x7fe9e4141660 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e9000640 1 -- 172.21.15.143:0/125005060 >> v1:172.21.15.204:6789/0 conn(0x7fe9e4141660 legacy=0x7fe9e4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e9000640 1 -- 172.21.15.143:0/125005060 >> v1:172.21.15.143:6789/0 conn(0x7fe9e40ce560 legacy=0x7fe9e4071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963604264 0 0) 0x7fe9c4003530 con 0x7fe9e40ce560 2024-01-15T18:17:38.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e9000640 1 -- 172.21.15.143:0/125005060 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e415f8d0 con 0x7fe9e4072410 2024-01-15T18:17:38.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e9000640 1 --2- 172.21.15.143:0/125005060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415ec80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009dc0 tx=0x7fe9dc01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37e95d1494172213 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:38.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc001680 con 0x7fe9e4072410 2024-01-15T18:17:38.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc0a55e0 con 0x7fe9e4072410 2024-01-15T18:17:38.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.456+0000 7fe9e2a00640 1 -- 172.21.15.143:0/125005060 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0a5970 con 0x7fe9e4072410 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- 172.21.15.143:0/125005060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 msgr2=0x7fe9e415ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 --2- 172.21.15.143:0/125005060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415ec80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009dc0 tx=0x7fe9dc01c890 comp rx=0 tx=0).stop 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- 172.21.15.143:0/125005060 shutdown_connections 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 --2- 172.21.15.143:0/125005060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415ec80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- 172.21.15.143:0/125005060 >> 172.21.15.143:0/125005060 conn(0x7fe9e406c3f0 msgr2=0x7fe9e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- 172.21.15.143:0/125005060 shutdown_connections 2024-01-15T18:17:38.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- 172.21.15.143:0/125005060 wait complete. 2024-01-15T18:17:38.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 Processor -- start 2024-01-15T18:17:38.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- start start 2024-01-15T18:17:38.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:38.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e40ce560 0x7fe9e40acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:38.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 0x7fe9e414c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:38.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e415e700 con 0x7fe9e4141660 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e415da80 con 0x7fe9e40ce560 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9eb000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e415dd60 con 0x7fe9e4072410 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9e9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 0x7fe9e414c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9e9000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 0x7fe9e414c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48336/0 (socket says 172.21.15.143:48336) 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.460+0000 7fe9e9000640 1 -- 172.21.15.143:0/2273830746 learned_addr learned my addr 172.21.15.143:0/2273830746 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e3400640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e40ce560 0x7fe9e40acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e3e00640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e9000640 1 -- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 msgr2=0x7fe9e415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e9000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e9000640 1 -- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e40ce560 msgr2=0x7fe9e40acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e9000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e40ce560 0x7fe9e40acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e9000640 1 -- 172.21.15.143:0/2273830746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9dc009a70 con 0x7fe9e4141660 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e3e00640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:38.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9e9000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 0x7fe9e414c350 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc01cd70 tx=0x7fe9dc019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0b9650 con 0x7fe9e4141660 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc0b97b0 con 0x7fe9e4141660 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0b9b20 con 0x7fe9e4141660 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e40cd150 con 0x7fe9e4141660 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e415e320 con 0x7fe9e4141660 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e415e840 con 0x7fe9e4141660 2024-01-15T18:17:38.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.464+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fe9dc0b9c80 con 0x7fe9e4141660 2024-01-15T18:17:38.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.468+0000 7fe9dbe00640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9bc067820 0x7fe9bc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:38.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.468+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe9dc11e520 con 0x7fe9e4141660 2024-01-15T18:17:38.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.468+0000 7fe9e3e00640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9bc067820 0x7fe9bc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:38.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.468+0000 7fe9e3e00640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9bc067820 0x7fe9bc069ce0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40011f0 tx=0x7fe9d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:38.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.472+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe9dc0ecf50 con 0x7fe9e4141660 2024-01-15T18:17:38.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.652+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe9e4067500 con 0x7fe9e4141660 2024-01-15T18:17:38.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.652+0000 7fe9dbe00640 1 -- 172.21.15.143:0/2273830746 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe9dc0b2020 con 0x7fe9e4141660 2024-01-15T18:17:38.652 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9bc067820 msgr2=0x7fe9bc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9bc067820 0x7fe9bc069ce0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40011f0 tx=0x7fe9d4009040 comp rx=0 tx=0).stop 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 msgr2=0x7fe9e414c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 0x7fe9e414c350 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc01cd70 tx=0x7fe9dc019630 comp rx=0 tx=0).stop 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 shutdown_connections 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe9bc067820 0x7fe9bc069ce0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe9e4141660 0x7fe9e414c350 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9e40ce560 0x7fe9e40acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 --2- 172.21.15.143:0/2273830746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe9e4072410 0x7fe9e415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 >> 172.21.15.143:0/2273830746 conn(0x7fe9e406c3f0 msgr2=0x7fe9e40af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:38.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 shutdown_connections 2024-01-15T18:17:38.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:38.656+0000 7fe9eb000640 1 -- 172.21.15.143:0/2273830746 wait complete. 2024-01-15T18:17:38.674 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:17:38.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:17:38.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:17:38.675 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:17:38.675 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:38.675 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:17:38.675 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:17:38.675 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:17:38.675 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:17:38.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.675 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:17:38.685 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:17:38.685 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:17:38.685 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:38.686 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 41679sec preferred_lft 41679sec 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:17:38.687 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.688 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:17:38.688 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:17:38.688 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:17:38.688 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:17:38.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.689 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:17:38.753 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:17:38.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.755 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:17:38.809 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:17:38.810 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:17:38.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.810 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:17:38.810 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:17:38.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:17:38.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:17:38.810 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:17:38.810 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:17:38.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.904 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:17:38.912 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:17:38.912 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:17:38.913 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:17:38.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.913 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:17:38.983 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:17:38.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:17:38.983 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:39.033 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:17:39.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:17:39.034 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:39.085 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:17:39.146 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:17:39.199 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:17:39.200 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:17:39.201 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:17:39.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:39.202 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:17:39.253 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:17:39.253 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:17:39.296 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:17:39.316 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:17:39.367 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:17:39.368 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:17:39.369 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:17:39.369 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:17:39.369 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:17:39.369 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:17:39.369 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:17:39.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:39.370 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:17:39.391 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:17:39.388+0000 7f99467b4580 -1 init, newargv = 0x55c87c61b410 newargc=15 2024-01-15T18:17:39.391 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34321]: starting ceph client 2024-01-15T18:17:39.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34321]: starting fuse 2024-01-15T18:17:39.399 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:17:39.400 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:17:39.400 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:17:39.401 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:17:39.402 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:17:39.402 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:17:39.402 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:17:39.524 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.34321.asok 2024-01-15T18:17:39.528 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:17:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:17:39.535 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34321.asok 2024-01-15T18:17:39.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:39.535 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34321.asok status 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "34321", 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:17:39.704 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 8509, 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 8509 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 3701958606 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:17:39.705 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 3701958606 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.8509 192.168.0.1:0/3701958606", 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/3701958606", 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 62, 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 100, 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:17:39.706 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:17:39.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:39.719 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:17:39.719 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:39.728 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:17:39.728 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:39.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:39.728 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:39.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:17:39.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.940+0000 7fb448c00640 1 Processor -- start 2024-01-15T18:17:39.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.940+0000 7fb448c00640 1 -- start start 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb448c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4441607c0 0x7fb44406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb448c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4440cfb30 0x7fb44406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb448c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44406c230 0x7fb4440cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb448c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44406c670 con 0x7fb4441607c0 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb448c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440d1ce0 con 0x7fb4440cfb30 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb448c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440cffd0 con 0x7fb44406c230 2024-01-15T18:17:39.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4441607c0 0x7fb44406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4441607c0 0x7fb44406b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48348/0 (socket says 172.21.15.143:48348) 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 -- 172.21.15.143:0/3194500955 learned_addr learned my addr 172.21.15.143:0/3194500955 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb442800640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44406c230 0x7fb4440cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 -- 172.21.15.143:0/3194500955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44406c230 msgr2=0x7fb4440cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441400640 1 --2- 172.21.15.143:0/3194500955 >> v2:172.21.15.204:3300/0 conn(0x7fb4440cfb30 0x7fb44406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44406c230 0x7fb4440cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 -- 172.21.15.143:0/3194500955 >> v2:172.21.15.204:3300/0 conn(0x7fb4440cfb30 msgr2=0x7fb44406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 --2- 172.21.15.143:0/3194500955 >> v2:172.21.15.204:3300/0 conn(0x7fb4440cfb30 0x7fb44406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441e00640 1 -- 172.21.15.143:0/3194500955 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4440d02b0 con 0x7fb4441607c0 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb442800640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44406c230 0x7fb4440cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:39.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.944+0000 7fb441400640 1 --2- 172.21.15.143:0/3194500955 >> v2:172.21.15.204:3300/0 conn(0x7fb4440cfb30 0x7fb44406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb441e00640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4441607c0 0x7fb44406b7b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb44409a980 tx=0x7fb43401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b834f53a58d4828 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb440a00640 1 -- 172.21.15.143:0/3194500955 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340a89b0 con 0x7fb4441607c0 2024-01-15T18:17:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb440a00640 1 -- 172.21.15.143:0/3194500955 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340a8b10 con 0x7fb4441607c0 2024-01-15T18:17:39.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb440a00640 1 -- 172.21.15.143:0/3194500955 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340a8ea0 con 0x7fb4441607c0 2024-01-15T18:17:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 -- 172.21.15.143:0/3194500955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4441607c0 msgr2=0x7fb44406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4441607c0 0x7fb44406b7b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb44409a980 tx=0x7fb43401ecf0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 -- 172.21.15.143:0/3194500955 shutdown_connections 2024-01-15T18:17:39.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb44406c230 0x7fb4440cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 --2- 172.21.15.143:0/3194500955 >> v2:172.21.15.204:3300/0 conn(0x7fb4440cfb30 0x7fb44406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 --2- 172.21.15.143:0/3194500955 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4441607c0 0x7fb44406b7b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 -- 172.21.15.143:0/3194500955 >> 172.21.15.143:0/3194500955 conn(0x7fb4440d0e20 msgr2=0x7fb4440d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 -- 172.21.15.143:0/3194500955 shutdown_connections 2024-01-15T18:17:39.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 -- 172.21.15.143:0/3194500955 wait complete. 2024-01-15T18:17:39.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 Processor -- start 2024-01-15T18:17:39.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.948+0000 7fb448c00640 1 -- start start 2024-01-15T18:17:39.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 0x7fb44415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4440cfb30 0x7fb4440c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4441607c0 0x7fb4440c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b4c00 con 0x7fb44406c230 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44415c200 con 0x7fb4441607c0 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440ca640 con 0x7fb4440cfb30 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 0x7fb44415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 0x7fb44415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48350/0 (socket says 172.21.15.143:48350) 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 -- 172.21.15.143:0/93324912 learned_addr learned my addr 172.21.15.143:0/93324912 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441400640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4440cfb30 0x7fb4440c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb442800640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4441607c0 0x7fb4440c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 -- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4440cfb30 msgr2=0x7fb4440c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4440cfb30 0x7fb4440c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 -- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4441607c0 msgr2=0x7fb4440c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4441607c0 0x7fb4440c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 -- 172.21.15.143:0/93324912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340094d0 con 0x7fb44406c230 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb441e00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 0x7fb44415bcc0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b3040 tx=0x7fb43401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340be070 con 0x7fb44406c230 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4340ba400 con 0x7fb44406c230 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340ba7b0 con 0x7fb44406c230 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4440b3b60 con 0x7fb44406c230 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4440b4120 con 0x7fb44406c230 2024-01-15T18:17:39.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.952+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4440b6d70 con 0x7fb44406c230 2024-01-15T18:17:39.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.956+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fb4340ba910 con 0x7fb44406c230 2024-01-15T18:17:39.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.956+0000 7fb43a000640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb41c067850 0x7fb41c069d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.956+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb434120bc0 con 0x7fb44406c230 2024-01-15T18:17:39.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.956+0000 7fb441400640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb41c067850 0x7fb41c069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.956+0000 7fb441400640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb41c067850 0x7fb41c069d10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb44415eeb0 tx=0x7fb42c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:39.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:39.960+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb4340ef960 con 0x7fb44406c230 2024-01-15T18:17:40.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.144+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb44406d4b0 con 0x7fb44406c230 2024-01-15T18:17:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.812+0000 7fb43a000640 1 -- 172.21.15.143:0/93324912 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb4340f2f80 con 0x7fb44406c230 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb41c067850 msgr2=0x7fb41c069d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb41c067850 0x7fb41c069d10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb44415eeb0 tx=0x7fb42c008040 comp rx=0 tx=0).stop 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 msgr2=0x7fb44415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 0x7fb44415bcc0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb4340b3040 tx=0x7fb43401ba90 comp rx=0 tx=0).stop 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 shutdown_connections 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb41c067850 0x7fb41c069d10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4441607c0 0x7fb4440c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4440cfb30 0x7fb4440c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 --2- 172.21.15.143:0/93324912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb44406c230 0x7fb44415bcc0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 >> 172.21.15.143:0/93324912 conn(0x7fb4440d0e20 msgr2=0x7fb4440cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 shutdown_connections 2024-01-15T18:17:40.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.816+0000 7fb448c00640 1 -- 172.21.15.143:0/93324912 wait complete. 2024-01-15T18:17:40.837 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:17:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 Processor -- start 2024-01-15T18:17:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 -- start start 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cde50 0x7f96f40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f96f40c5600 0x7f96f40c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f40ce7c0 con 0x7f96f40cf410 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f40cef50 con 0x7f96f40c5600 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f40ce290 con 0x7f96f40cde50 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f96f40c5600 0x7f96f40c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f96f40c5600 0x7f96f40c5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48360/0 (socket says 172.21.15.143:48360) 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 -- 172.21.15.143:0/3240773913 learned_addr learned my addr 172.21.15.143:0/3240773913 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f2000640 1 --2- 172.21.15.143:0/3240773913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cde50 0x7f96f40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 -- 172.21.15.143:0/3240773913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cde50 msgr2=0x7f96f40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 --2- 172.21.15.143:0/3240773913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cde50 0x7f96f40d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 -- 172.21.15.143:0/3240773913 >> v1:172.21.15.204:6789/0 conn(0x7f96f40cf410 legacy=0x7f96f40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.980+0000 7f96f3400640 1 -- 172.21.15.143:0/3240773913 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40ce4e0 con 0x7f96f40c5600 2024-01-15T18:17:40.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f3400640 1 --2- 172.21.15.143:0/3240773913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40c5600 0x7f96f40c5a90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009e50 tx=0x7f96ec01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91888b9ff0ed9d13 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f1600640 1 -- 172.21.15.143:0/3240773913 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0a57d0 con 0x7f96f40c5600 2024-01-15T18:17:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f1600640 1 -- 172.21.15.143:0/3240773913 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec0a5930 con 0x7f96f40c5600 2024-01-15T18:17:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f1600640 1 -- 172.21.15.143:0/3240773913 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0a5cc0 con 0x7f96f40c5600 2024-01-15T18:17:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 -- 172.21.15.143:0/3240773913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40c5600 msgr2=0x7f96f40c5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 --2- 172.21.15.143:0/3240773913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40c5600 0x7f96f40c5a90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009e50 tx=0x7f96ec01ca10 comp rx=0 tx=0).stop 2024-01-15T18:17:40.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 -- 172.21.15.143:0/3240773913 shutdown_connections 2024-01-15T18:17:40.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 --2- 172.21.15.143:0/3240773913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40c5600 0x7f96f40c5a90 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 --2- 172.21.15.143:0/3240773913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cde50 0x7f96f40d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 -- 172.21.15.143:0/3240773913 >> 172.21.15.143:0/3240773913 conn(0x7f96f40d0e30 msgr2=0x7f96f40d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 -- 172.21.15.143:0/3240773913 shutdown_connections 2024-01-15T18:17:40.984 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.984+0000 7f96f9600640 1 -- 172.21.15.143:0/3240773913 wait complete. 2024-01-15T18:17:40.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 Processor -- start 2024-01-15T18:17:40.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 -- start start 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f40c5600 0x7f96f4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 0x7f96f4149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cf410 0x7f96f4158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4154fb0 con 0x7f96f40cde50 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4155f60 con 0x7f96f40c5600 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f4155440 con 0x7f96f40cf410 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 0x7f96f4149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 0x7f96f4149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48374/0 (socket says 172.21.15.143:48374) 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 -- 172.21.15.143:0/3015896684 learned_addr learned my addr 172.21.15.143:0/3015896684 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f3400640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cf410 0x7f96f4158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2a00640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f40c5600 0x7f96f4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 -- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cf410 msgr2=0x7f96f4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cf410 0x7f96f4158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 -- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f40c5600 msgr2=0x7f96f4153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f40c5600 0x7f96f4153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 -- 172.21.15.143:0/3015896684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec009b30 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f2000640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 0x7f96f4149e90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00d800 tx=0x7f96dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00b6c0 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc00b820 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00bb90 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f41543a0 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.988+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f4154960 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.992+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f4155120 con 0x7f96f40cde50 2024-01-15T18:17:40.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.992+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f96dc00bcf0 con 0x7f96f40cde50 2024-01-15T18:17:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.992+0000 7f96eaa00640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96cc067820 0x7f96cc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.992+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f96dc083610 con 0x7f96f40cde50 2024-01-15T18:17:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.992+0000 7f96f2a00640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96cc067820 0x7f96cc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.992+0000 7f96f2a00640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96cc067820 0x7f96cc069ce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f96e4007fd0 tx=0x7f96e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:40.996+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f96dc052060 con 0x7f96f40cde50 2024-01-15T18:17:41.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.164+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f96f4066880 con 0x7f96cc067820 2024-01-15T18:17:41.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96eaa00640 1 -- 172.21.15.143:0/3015896684 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f96f4066880 con 0x7f96cc067820 2024-01-15T18:17:41.166 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:17:41.166 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:17:41.166 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:17:41.166 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:17:41.166 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96cc067820 msgr2=0x7f96cc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96cc067820 0x7f96cc069ce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f96e4007fd0 tx=0x7f96e4009040 comp rx=0 tx=0).stop 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 msgr2=0x7f96f4149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 0x7f96f4149e90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00d800 tx=0x7f96dc00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 shutdown_connections 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f96cc067820 0x7f96cc069ce0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f96f40cf410 0x7f96f4158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f40cde50 0x7f96f4149e90 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 --2- 172.21.15.143:0/3015896684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f96f40c5600 0x7f96f4153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 >> 172.21.15.143:0/3015896684 conn(0x7f96f40d0e30 msgr2=0x7f96f406e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 shutdown_connections 2024-01-15T18:17:41.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.168+0000 7f96f9600640 1 -- 172.21.15.143:0/3015896684 wait complete. 2024-01-15T18:17:41.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:17:41.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.328+0000 7f4d3b600640 1 Processor -- start 2024-01-15T18:17:41.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.328+0000 7f4d3b600640 1 -- start start 2024-01-15T18:17:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d3b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d340cdf20 0x7f4d340cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d3b600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d340c6230 0x7f4d340cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d3b600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d340c4d90 con 0x7f4d3415b260 2024-01-15T18:17:41.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d3b600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3415b640 con 0x7f4d340c6230 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d3b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340cf800 con 0x7f4d340cdf20 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d39600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d340c6230 0x7f4d340cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d38c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4d3415b260 0x7f4d340cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51378/0 (socket says 172.21.15.143:51378) 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d38c00640 1 -- 172.21.15.143:0/473710221 learned_addr learned my addr 172.21.15.143:0/473710221 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d33e00640 1 --2- 172.21.15.143:0/473710221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d340cdf20 0x7f4d340cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d39600640 1 -- 172.21.15.143:0/473710221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d340cdf20 msgr2=0x7f4d340cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d39600640 1 --2- 172.21.15.143:0/473710221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d340cdf20 0x7f4d340cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d39600640 1 -- 172.21.15.143:0/473710221 >> v1:172.21.15.143:6789/0 conn(0x7f4d3415b260 legacy=0x7f4d340cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d39600640 1 -- 172.21.15.143:0/473710221 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340cfae0 con 0x7f4d340c6230 2024-01-15T18:17:41.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d33400640 1 -- 172.21.15.143:0/473710221 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275750499 0 0) 0x7f4d340c4d90 con 0x7f4d3415b260 2024-01-15T18:17:41.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.332+0000 7f4d39600640 1 --2- 172.21.15.143:0/473710221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d340cc020 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009d90 tx=0x7f4d2c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1e4a0ac0a807a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d33400640 1 -- 172.21.15.143:0/473710221 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0a59e0 con 0x7f4d340c6230 2024-01-15T18:17:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d33400640 1 -- 172.21.15.143:0/473710221 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c0a5b40 con 0x7f4d340c6230 2024-01-15T18:17:41.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d33400640 1 -- 172.21.15.143:0/473710221 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0ad440 con 0x7f4d340c6230 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- 172.21.15.143:0/473710221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 msgr2=0x7f4d340cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 --2- 172.21.15.143:0/473710221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d340cc020 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009d90 tx=0x7f4d2c01c920 comp rx=0 tx=0).stop 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- 172.21.15.143:0/473710221 shutdown_connections 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 --2- 172.21.15.143:0/473710221 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d340cc020 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 --2- 172.21.15.143:0/473710221 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d340cdf20 0x7f4d340cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- 172.21.15.143:0/473710221 >> 172.21.15.143:0/473710221 conn(0x7f4d340d0ed0 msgr2=0x7f4d340d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- 172.21.15.143:0/473710221 shutdown_connections 2024-01-15T18:17:41.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- 172.21.15.143:0/473710221 wait complete. 2024-01-15T18:17:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 Processor -- start 2024-01-15T18:17:41.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- start start 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d3406d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d340cdf20 0x7f4d340b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d3415b260 0x7f4d34159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3406b640 con 0x7f4d340c6230 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340ca1e0 con 0x7f4d340cdf20 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.336+0000 7f4d3b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340ca480 con 0x7f4d3415b260 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d3406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d3406d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48392/0 (socket says 172.21.15.143:48392) 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 -- 172.21.15.143:0/3553172236 learned_addr learned my addr 172.21.15.143:0/3553172236 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d33e00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d340cdf20 0x7f4d340b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d39600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d3415b260 0x7f4d34159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 -- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d3415b260 msgr2=0x7f4d34159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.338 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d3415b260 0x7f4d34159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 -- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d340cdf20 msgr2=0x7f4d340b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d340cdf20 0x7f4d340b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 -- 172.21.15.143:0/3553172236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c009a70 con 0x7f4d340c6230 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d38c00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d3406d150 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4d240011f0 tx=0x7f4d2400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240074e0 con 0x7f4d340c6230 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d24007640 con 0x7f4d340c6230 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3406b8b0 con 0x7f4d340c6230 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24004a10 con 0x7f4d340c6230 2024-01-15T18:17:41.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d340b24b0 con 0x7f4d340c6230 2024-01-15T18:17:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3406bdd0 con 0x7f4d340c6230 2024-01-15T18:17:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.340+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f4d24004b70 con 0x7f4d340c6230 2024-01-15T18:17:41.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.344+0000 7f4d30c00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4d0c067820 0x7f4d0c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.344+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4d24082430 con 0x7f4d340c6230 2024-01-15T18:17:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.344+0000 7f4d33e00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4d0c067820 0x7f4d0c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.344+0000 7f4d33e00640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4d0c067820 0x7f4d0c069ce0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4d34155000 tx=0x7f4d1c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.348+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4d3406bdd0 con 0x7f4d340c6230 2024-01-15T18:17:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.500+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4d3406bdd0 con 0x7f4d340c6230 2024-01-15T18:17:41.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d30c00640 1 -- 172.21.15.143:0/3553172236 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4d3406bdd0 con 0x7f4d340c6230 2024-01-15T18:17:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4d0c067820 msgr2=0x7f4d0c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4d0c067820 0x7f4d0c069ce0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4d34155000 tx=0x7f4d1c009190 comp rx=0 tx=0).stop 2024-01-15T18:17:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 msgr2=0x7f4d3406d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d3406d150 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4d240011f0 tx=0x7f4d2400c9b0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 shutdown_connections 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4d0c067820 0x7f4d0c069ce0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4d3415b260 0x7f4d34159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4d340cdf20 0x7f4d340b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 --2- 172.21.15.143:0/3553172236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d340c6230 0x7f4d3406d150 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 >> 172.21.15.143:0/3553172236 conn(0x7f4d340d0ed0 msgr2=0x7f4d3406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 shutdown_connections 2024-01-15T18:17:41.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.504+0000 7f4d3b600640 1 -- 172.21.15.143:0/3553172236 wait complete. 2024-01-15T18:17:41.523 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:17:41.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 Processor -- start 2024-01-15T18:17:41.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 -- start start 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c015d150 0x7f39c00d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39c015d530 0x7f39c00d3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39c00b0b20 con 0x7f39c00afed0 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39c00d0bf0 con 0x7f39c015d530 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00d1a20 con 0x7f39c015d150 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c5600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39c015d530 0x7f39c00d3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c6a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f39c00afed0 0x7f39c00b0330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51390/0 (socket says 172.21.15.143:51390) 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c6a00640 1 -- 172.21.15.143:0/1629697502 learned_addr learned my addr 172.21.15.143:0/1629697502 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c6000640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c015d150 0x7f39c00d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c5600640 1 -- 172.21.15.143:0/1629697502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c015d150 msgr2=0x7f39c00d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c5600640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c015d150 0x7f39c00d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c5600640 1 -- 172.21.15.143:0/1629697502 >> v1:172.21.15.143:6789/0 conn(0x7f39c00afed0 legacy=0x7f39c00b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c5600640 1 -- 172.21.15.143:0/1629697502 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c00b0e00 con 0x7f39c015d530 2024-01-15T18:17:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c4c00640 1 -- 172.21.15.143:0/1629697502 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555465063 0 0) 0x7f39c00b0b20 con 0x7f39c00afed0 2024-01-15T18:17:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.664+0000 7f39c6000640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c015d150 0x7f39c00d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c5600640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c00d3680 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f39a80097f0 tx=0x7f39a801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64304ad82c43d973 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c4c00640 1 -- 172.21.15.143:0/1629697502 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a80a8970 con 0x7f39c015d530 2024-01-15T18:17:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c4c00640 1 -- 172.21.15.143:0/1629697502 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a80a8ad0 con 0x7f39c015d530 2024-01-15T18:17:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c4c00640 1 -- 172.21.15.143:0/1629697502 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a80a8e60 con 0x7f39c015d530 2024-01-15T18:17:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c8a00640 1 -- 172.21.15.143:0/1629697502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 msgr2=0x7f39c00d3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c8a00640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c00d3680 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f39a80097f0 tx=0x7f39a801ec70 comp rx=0 tx=0).stop 2024-01-15T18:17:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c8a00640 1 -- 172.21.15.143:0/1629697502 shutdown_connections 2024-01-15T18:17:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c8a00640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c00d3680 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c8a00640 1 --2- 172.21.15.143:0/1629697502 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c015d150 0x7f39c00d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.668+0000 7f39c8a00640 1 -- 172.21.15.143:0/1629697502 >> 172.21.15.143:0/1629697502 conn(0x7f39c008ccc0 msgr2=0x7f39c009c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 -- 172.21.15.143:0/1629697502 shutdown_connections 2024-01-15T18:17:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 -- 172.21.15.143:0/1629697502 wait complete. 2024-01-15T18:17:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 Processor -- start 2024-01-15T18:17:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 -- start start 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c00afed0 0x7f39c01603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c015d150 0x7f39c015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c0152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c00cb280 con 0x7f39c015d530 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c015ea30 con 0x7f39c015d150 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c8a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c015ec80 con 0x7f39c00afed0 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c0152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c0152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48408/0 (socket says 172.21.15.143:48408) 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 -- 172.21.15.143:0/694808641 learned_addr learned my addr 172.21.15.143:0/694808641 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6000640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c00afed0 0x7f39c01603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c5600640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c015d150 0x7f39c015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 -- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c00afed0 msgr2=0x7f39c01603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c00afed0 0x7f39c01603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 -- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c015d150 msgr2=0x7f39c015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c015d150 0x7f39c015afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c6a00640 1 -- 172.21.15.143:0/694808641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a8009420 con 0x7f39c015d530 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.672+0000 7f39c5600640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c015d150 0x7f39c015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39c6a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c0152390 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f39b8001570 tx=0x7f39b800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800cea0 con 0x7f39c015d530 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b800a5e0 con 0x7f39c015d530 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b800a990 con 0x7f39c015d530 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c015ef60 con 0x7f39c015d530 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c015f520 con 0x7f39c015d530 2024-01-15T18:17:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c00b1680 con 0x7f39c015d530 2024-01-15T18:17:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f39b800aaf0 con 0x7f39c015d530 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39be000640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3998067820 0x7f3998069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f39b80826b0 con 0x7f39c015d530 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.676+0000 7f39c6000640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3998067820 0x7f3998069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.680+0000 7f39c6000640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3998067820 0x7f3998069ce0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f39b0006e90 tx=0x7f39b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.680+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f39c00b1680 con 0x7f39c015d530 2024-01-15T18:17:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.836+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7f39c00b1680 con 0x7f39c015d530 2024-01-15T18:17:41.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39be000640 1 -- 172.21.15.143:0/694808641 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v35)=0 v35) v1 ==== 131+0+0 (secure 0 0 0) 0x7f39c00b1680 con 0x7f39c015d530 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3998067820 msgr2=0x7f3998069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3998067820 0x7f3998069ce0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f39b0006e90 tx=0x7f39b0008040 comp rx=0 tx=0).stop 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 msgr2=0x7f39c0152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c0152390 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f39b8001570 tx=0x7f39b800cc50 comp rx=0 tx=0).stop 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 shutdown_connections 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3998067820 0x7f3998069ce0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39c015d530 0x7f39c0152390 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f39c015d150 0x7f39c015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 --2- 172.21.15.143:0/694808641 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f39c00afed0 0x7f39c01603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 >> 172.21.15.143:0/694808641 conn(0x7f39c008ccc0 msgr2=0x7f39c00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 shutdown_connections 2024-01-15T18:17:41.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:41.840+0000 7f39c8a00640 1 -- 172.21.15.143:0/694808641 wait complete. 2024-01-15T18:17:41.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 120 2024-01-15T18:17:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 Processor -- start 2024-01-15T18:17:42.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 -- start start 2024-01-15T18:17:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2815aef0 0x7fec2806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fec280b29a0 0x7fec280ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2815f7c0 con 0x7fec2806c560 2024-01-15T18:17:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2806c310 con 0x7fec280b29a0 2024-01-15T18:17:42.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec2de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2815ebb0 con 0x7fec2815aef0 2024-01-15T18:17:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec27e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fec280b29a0 0x7fec280ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec26a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2815aef0 0x7fec2806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec26a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2815aef0 0x7fec2806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:48808/0 (socket says 172.21.15.143:48808) 2024-01-15T18:17:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec26a00640 1 -- 172.21.15.143:0/163797706 learned_addr learned my addr 172.21.15.143:0/163797706 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:42.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec27e00640 1 -- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2815aef0 msgr2=0x7fec2806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec27e00640 1 --2- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2815aef0 0x7fec2806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec27e00640 1 -- 172.21.15.143:0/163797706 >> v1:172.21.15.204:6789/0 conn(0x7fec2806c560 legacy=0x7fec280b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec27e00640 1 -- 172.21.15.143:0/163797706 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2815ee90 con 0x7fec280b29a0 2024-01-15T18:17:42.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.004+0000 7fec25600640 1 -- 172.21.15.143:0/163797706 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807255349 0 0) 0x7fec2815f7c0 con 0x7fec2806c560 2024-01-15T18:17:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.008+0000 7fec27e00640 1 --2- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280ceca0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fec20009d90 tx=0x7fec2001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26b69faaa48160b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.008+0000 7fec25600640 1 -- 172.21.15.143:0/163797706 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200a5960 con 0x7fec280b29a0 2024-01-15T18:17:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.008+0000 7fec25600640 1 -- 172.21.15.143:0/163797706 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec200a5ac0 con 0x7fec280b29a0 2024-01-15T18:17:42.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.008+0000 7fec25600640 1 -- 172.21.15.143:0/163797706 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200a5e50 con 0x7fec280b29a0 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.008+0000 7fec2de00640 1 -- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 msgr2=0x7fec280ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.008+0000 7fec2de00640 1 --2- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280ceca0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fec20009d90 tx=0x7fec2001c920 comp rx=0 tx=0).stop 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- 172.21.15.143:0/163797706 shutdown_connections 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 --2- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280ceca0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 --2- 172.21.15.143:0/163797706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2815aef0 0x7fec2806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- 172.21.15.143:0/163797706 >> 172.21.15.143:0/163797706 conn(0x7fec28066720 msgr2=0x7fec2806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- 172.21.15.143:0/163797706 shutdown_connections 2024-01-15T18:17:42.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- 172.21.15.143:0/163797706 wait complete. 2024-01-15T18:17:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 Processor -- start 2024-01-15T18:17:42.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- start start 2024-01-15T18:17:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2806c560 0x7fec2814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 0x7fec2814c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec280b4a90 con 0x7fec2815aef0 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec280b3c40 con 0x7fec280b29a0 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec2de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec280b3f10 con 0x7fec2806c560 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 0x7fec2814c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 0x7fec2814c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48422/0 (socket says 172.21.15.143:48422) 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 -- 172.21.15.143:0/2373568115 learned_addr learned my addr 172.21.15.143:0/2373568115 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec26a00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2806c560 0x7fec2814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec26000640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 -- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2806c560 msgr2=0x7fec2814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2806c560 0x7fec2814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 -- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 msgr2=0x7fec280b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.012+0000 7fec27e00640 1 -- 172.21.15.143:0/2373568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec20009a70 con 0x7fec2815aef0 2024-01-15T18:17:42.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec27e00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 0x7fec2814c470 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fec2001a550 tx=0x7fec200196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20009450 con 0x7fec2815aef0 2024-01-15T18:17:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec200adb50 con 0x7fec2815aef0 2024-01-15T18:17:42.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec280b41f0 con 0x7fec2815aef0 2024-01-15T18:17:42.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200b8420 con 0x7fec2815aef0 2024-01-15T18:17:42.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec280b47b0 con 0x7fec2815aef0 2024-01-15T18:17:42.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec28067680 con 0x7fec2815aef0 2024-01-15T18:17:42.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7fec200b8580 con 0x7fec2815aef0 2024-01-15T18:17:42.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec1f400640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec00067820 0x7fec00069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fec2011e5b0 con 0x7fec2815aef0 2024-01-15T18:17:42.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.016+0000 7fec26a00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec00067820 0x7fec00069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.020+0000 7fec26a00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec00067820 0x7fec00069ce0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec18009ab0 tx=0x7fec18009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.020+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fec200ed410 con 0x7fec2815aef0 2024-01-15T18:17:42.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.172+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7fec2806e790 con 0x7fec2815aef0 2024-01-15T18:17:42.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.176+0000 7fec1f400640 1 -- 172.21.15.143:0/2373568115 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v36)=0 v36) v1 ==== 137+0+0 (secure 0 0 0) 0x7fec200bc0d0 con 0x7fec2815aef0 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec00067820 msgr2=0x7fec00069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec00067820 0x7fec00069ce0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fec18009ab0 tx=0x7fec18009240 comp rx=0 tx=0).stop 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 msgr2=0x7fec2814c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 0x7fec2814c470 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fec2001a550 tx=0x7fec200196c0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 shutdown_connections 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fec00067820 0x7fec00069ce0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec2815aef0 0x7fec2814c470 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fec280b29a0 0x7fec280b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 --2- 172.21.15.143:0/2373568115 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fec2806c560 0x7fec2814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 >> 172.21.15.143:0/2373568115 conn(0x7fec28066720 msgr2=0x7fec2806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 shutdown_connections 2024-01-15T18:17:42.180 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.180+0000 7fec2de00640 1 -- 172.21.15.143:0/2373568115 wait complete. 2024-01-15T18:17:42.200 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:17:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.340+0000 7f404c400640 1 Processor -- start 2024-01-15T18:17:42.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404c400640 1 -- start start 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404c400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f404415d1f0 0x7f404406b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f40440acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404c400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40440b3370 con 0x7f404406bb70 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404c400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404415d5d0 con 0x7f404415d1f0 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404406a800 con 0x7f4044071470 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4049000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f404406bb70 0x7f4044070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:51396/0 (socket says 172.21.15.143:51396) 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4049000640 1 -- 172.21.15.143:0/4063822318 learned_addr learned my addr 172.21.15.143:0/4063822318 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:17:42.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f40440acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4049a00640 1 --2- 172.21.15.143:0/4063822318 >> v2:172.21.15.204:3300/0 conn(0x7f404415d1f0 0x7f404406b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340946807 0 0) 0x7f40440b3370 con 0x7f404406bb70 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4024003530 con 0x7f404406bb70 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 109691181 0 0) 0x7f4024003530 con 0x7f404406bb70 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 msgr2=0x7f40440acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f40440acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 >> v2:172.21.15.204:3300/0 conn(0x7f404415d1f0 msgr2=0x7f404406b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 --2- 172.21.15.143:0/4063822318 >> v2:172.21.15.204:3300/0 conn(0x7f404415d1f0 0x7f404406b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404406aae0 con 0x7f404406bb70 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f402c001680 con 0x7f404406bb70 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f40440acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4049a00640 1 --2- 172.21.15.143:0/4063822318 >> v2:172.21.15.204:3300/0 conn(0x7f404415d1f0 0x7f404406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 >> v1:172.21.15.143:6789/0 conn(0x7f404406bb70 legacy=0x7f4044070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f4024003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4024003ec0 0x7f40240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d10 0x7f4024005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404415d5d0 con 0x7f4024004d10 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404406a800 con 0x7f4024003ec0 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402400b990 con 0x7f4044071470 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d10 0x7f4024005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4049a00640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f4024003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f4049000640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4024003ec0 0x7f40240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 -- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 msgr2=0x7f4024003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4044071470 0x7f4024003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 -- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4024003ec0 msgr2=0x7f40240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4024003ec0 0x7f40240047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.344+0000 7f404a400640 1 -- 172.21.15.143:0/4063822318 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c009a70 con 0x7f4024004d10 2024-01-15T18:17:42.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404a400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d10 0x7f4024005190 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f403c00e320 tx=0x7f403c01cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cead71247f9162 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0a5db0 con 0x7f4024004d10 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c00f040 con 0x7f4024004d10 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f4043e00640 1 -- 172.21.15.143:0/4063822318 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0ad640 con 0x7f4024004d10 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404c400640 1 -- 172.21.15.143:0/4063822318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d10 msgr2=0x7f4024005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404c400640 1 --2- 172.21.15.143:0/4063822318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4024004d10 0x7f4024005190 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f403c00e320 tx=0x7f403c01cda0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404a400640 1 -- 172.21.15.143:0/4063822318 reap_dead start 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404c400640 1 -- 172.21.15.143:0/4063822318 shutdown_connections 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404c400640 1 -- 172.21.15.143:0/4063822318 >> 172.21.15.143:0/4063822318 conn(0x7f40440b1610 msgr2=0x7f40440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404c400640 1 -- 172.21.15.143:0/4063822318 shutdown_connections 2024-01-15T18:17:42.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.348+0000 7f404c400640 1 -- 172.21.15.143:0/4063822318 wait complete. 2024-01-15T18:17:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 Processor -- start 2024-01-15T18:17:42.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 -- start start 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4024004d10 0x7f40441572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 0x7f40440c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40440b0450 0x7f40440b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044157830 con 0x7f40440c63e0 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440b0e10 con 0x7f40440b0450 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40440ae940 con 0x7f4024004d10 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 0x7f40440c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 0x7f40440c6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48438/0 (socket says 172.21.15.143:48438) 2024-01-15T18:17:42.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 -- 172.21.15.143:0/1510259297 learned_addr learned my addr 172.21.15.143:0/1510259297 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404a400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40440b0450 0x7f40440b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049a00640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4024004d10 0x7f40441572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 -- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4024004d10 msgr2=0x7f40441572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4024004d10 0x7f40441572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 -- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40440b0450 msgr2=0x7f40440b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40440b0450 0x7f40440b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 -- 172.21.15.143:0/1510259297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440aec20 con 0x7f40440c63e0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404a400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40440b0450 0x7f40440b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4049000640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 0x7f40440c6860 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f402c007310 tx=0x7f402c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c00f9c0 con 0x7f40440c63e0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00fb20 con 0x7f40440c63e0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c00fed0 con 0x7f40440c63e0 2024-01-15T18:17:42.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440aef00 con 0x7f40440c63e0 2024-01-15T18:17:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.352+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40440ae520 con 0x7f40440c63e0 2024-01-15T18:17:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.356+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4044074b00 con 0x7f40440c63e0 2024-01-15T18:17:42.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.356+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f402c01d460 con 0x7f40440c63e0 2024-01-15T18:17:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.356+0000 7f4041600640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f401c067820 0x7f401c069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.356+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f402c083530 con 0x7f40440c63e0 2024-01-15T18:17:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.356+0000 7f4049a00640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f401c067820 0x7f401c069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.356+0000 7f4049a00640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f401c067820 0x7f401c069ce0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d30 tx=0x7f4034008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.360+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f402c051e90 con 0x7f40440c63e0 2024-01-15T18:17:42.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.532+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f404406b380 con 0x7f401c067820 2024-01-15T18:17:42.536 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:42.536+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:42.536 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:42.536+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:42.536 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:42.536+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:42.536 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:42.536+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:42.536 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:17:42.536+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:17:42.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f4041600640 1 -- 172.21.15.143:0/1510259297 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4034009980 con 0x7f401c067820 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f401c067820 msgr2=0x7f401c069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f401c067820 0x7f401c069ce0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d30 tx=0x7f4034008040 comp rx=0 tx=0).stop 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 msgr2=0x7f40440c6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 0x7f40440c6860 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f402c007310 tx=0x7f402c0077d0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 shutdown_connections 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f401c067820 0x7f401c069ce0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40440b0450 0x7f40440b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40440c63e0 0x7f40440c6860 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 --2- 172.21.15.143:0/1510259297 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4024004d10 0x7f40441572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 >> 172.21.15.143:0/1510259297 conn(0x7f40440b1610 msgr2=0x7f4044159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 shutdown_connections 2024-01-15T18:17:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.568+0000 7f404c400640 1 -- 172.21.15.143:0/1510259297 wait complete. 2024-01-15T18:17:42.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:17:42.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 Processor -- start 2024-01-15T18:17:42.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 -- start start 2024-01-15T18:17:42.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e8071bb0 0x7f78e8072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78e815c5a0 0x7f78e815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e80ceb50 con 0x7f78e8143780 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e8072570 con 0x7f78e815c5a0 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78ed200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e815c110 con 0x7f78e8071bb0 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78e6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78e815c5a0 0x7f78e815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78e6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f78e815c5a0 0x7f78e815ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48452/0 (socket says 172.21.15.143:48452) 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78e6800640 1 -- 172.21.15.143:0/4046530255 learned_addr learned my addr 172.21.15.143:0/4046530255 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.728+0000 7f78e5400640 1 --2- 172.21.15.143:0/4046530255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e8071bb0 0x7f78e8072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e6800640 1 -- 172.21.15.143:0/4046530255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e8071bb0 msgr2=0x7f78e8072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e6800640 1 --2- 172.21.15.143:0/4046530255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e8071bb0 0x7f78e8072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e6800640 1 -- 172.21.15.143:0/4046530255 >> v1:172.21.15.143:6789/0 conn(0x7f78e8143780 legacy=0x7f78e80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e6800640 1 -- 172.21.15.143:0/4046530255 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e815b7e0 con 0x7f78e815c5a0 2024-01-15T18:17:42.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e6800640 1 --2- 172.21.15.143:0/4046530255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e815c5a0 0x7f78e815ca30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009e50 tx=0x7f78e001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e185e0fab59dcb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e4a00640 1 -- 172.21.15.143:0/4046530255 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00a58e0 con 0x7f78e815c5a0 2024-01-15T18:17:42.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e4a00640 1 -- 172.21.15.143:0/4046530255 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e00a5a40 con 0x7f78e815c5a0 2024-01-15T18:17:42.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78e4a00640 1 -- 172.21.15.143:0/4046530255 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00a5dd0 con 0x7f78e815c5a0 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78ed200640 1 -- 172.21.15.143:0/4046530255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e815c5a0 msgr2=0x7f78e815ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.732+0000 7f78ed200640 1 --2- 172.21.15.143:0/4046530255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e815c5a0 0x7f78e815ca30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f78e0009e50 tx=0x7f78e001ca10 comp rx=0 tx=0).stop 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- 172.21.15.143:0/4046530255 shutdown_connections 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 --2- 172.21.15.143:0/4046530255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e815c5a0 0x7f78e815ca30 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 --2- 172.21.15.143:0/4046530255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e8071bb0 0x7f78e8072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- 172.21.15.143:0/4046530255 >> 172.21.15.143:0/4046530255 conn(0x7f78e806c3f0 msgr2=0x7f78e806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- 172.21.15.143:0/4046530255 shutdown_connections 2024-01-15T18:17:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- 172.21.15.143:0/4046530255 wait complete. 2024-01-15T18:17:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 Processor -- start 2024-01-15T18:17:42.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- start start 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78e8071bb0 0x7f78e813cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 0x7f78e80b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e815c5a0 0x7f78e80b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8142e40 con 0x7f78e8143780 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8140530 con 0x7f78e8071bb0 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78ed200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8159900 con 0x7f78e815c5a0 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 0x7f78e80b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 0x7f78e80b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48468/0 (socket says 172.21.15.143:48468) 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 -- 172.21.15.143:0/1267046113 learned_addr learned my addr 172.21.15.143:0/1267046113 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e6800640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e815c5a0 0x7f78e80b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5e00640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78e8071bb0 0x7f78e813cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 -- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e815c5a0 msgr2=0x7f78e80b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e815c5a0 0x7f78e80b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 -- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78e8071bb0 msgr2=0x7f78e813cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78e8071bb0 0x7f78e813cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5400640 1 -- 172.21.15.143:0/1267046113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e0009b30 con 0x7f78e8143780 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.736+0000 7f78e5e00640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78e8071bb0 0x7f78e813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78e5400640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 0x7f78e80b2d10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f78d000e960 tx=0x7f78d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0009690 con 0x7f78e8143780 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d000ca00 con 0x7f78e8143780 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000cd70 con 0x7f78e8143780 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e8140740 con 0x7f78e8143780 2024-01-15T18:17:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8140d00 con 0x7f78e8143780 2024-01-15T18:17:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e80705d0 con 0x7f78e8143780 2024-01-15T18:17:42.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f78d0013470 con 0x7f78e8143780 2024-01-15T18:17:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78de000640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78c0067820 0x7f78c0069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f78d0083060 con 0x7f78e8143780 2024-01-15T18:17:42.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.740+0000 7f78e5e00640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78c0067820 0x7f78c0069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.744+0000 7f78e5e00640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78c0067820 0x7f78c0069ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78d8001710 tx=0x7f78d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.744+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f78d0051e00 con 0x7f78e8143780 2024-01-15T18:17:42.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.924+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78e8166ac0 con 0x7f78c0067820 2024-01-15T18:17:42.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78de000640 1 -- 172.21.15.143:0/1267046113 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f78d8001350 con 0x7f78c0067820 2024-01-15T18:17:42.925 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78c0067820 msgr2=0x7f78c0069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78c0067820 0x7f78c0069ce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78d8001710 tx=0x7f78d8009040 comp rx=0 tx=0).stop 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 msgr2=0x7f78e80b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 0x7f78e80b2d10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f78d000e960 tx=0x7f78d000ee20 comp rx=0 tx=0).stop 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 shutdown_connections 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f78c0067820 0x7f78c0069ce0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f78e815c5a0 0x7f78e80b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78e8143780 0x7f78e80b2d10 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 --2- 172.21.15.143:0/1267046113 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f78e8071bb0 0x7f78e813cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 >> 172.21.15.143:0/1267046113 conn(0x7f78e806c3f0 msgr2=0x7f78e80ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 shutdown_connections 2024-01-15T18:17:42.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:42.928+0000 7f78ed200640 1 -- 172.21.15.143:0/1267046113 wait complete. 2024-01-15T18:17:42.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:17:43.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 Processor -- start 2024-01-15T18:17:43.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 -- start start 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f562c15b120 0x7f562c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f562c0cf400 0x7f562c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0cf970 con 0x7f562c0d1940 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c0cfc50 con 0x7f562c15b120 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f5631600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c0c4b10 con 0x7f562c0cf400 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.088+0000 7f562aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f562c15b120 0x7f562c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f562c15b120 0x7f562c0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48470/0 (socket says 172.21.15.143:48470) 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 -- 172.21.15.143:0/3828459720 learned_addr learned my addr 172.21.15.143:0/3828459720 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562a000640 1 --2- 172.21.15.143:0/3828459720 >> v2:172.21.15.204:3300/0 conn(0x7f562c0cf400 0x7f562c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 -- 172.21.15.143:0/3828459720 >> v2:172.21.15.204:3300/0 conn(0x7f562c0cf400 msgr2=0x7f562c0cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 --2- 172.21.15.143:0/3828459720 >> v2:172.21.15.204:3300/0 conn(0x7f562c0cf400 0x7f562c0cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 -- 172.21.15.143:0/3828459720 >> v1:172.21.15.204:6789/0 conn(0x7f562c0d1940 legacy=0x7f562c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:43.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 -- 172.21.15.143:0/3828459720 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0c4df0 con 0x7f562c15b120 2024-01-15T18:17:43.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f562aa00640 1 --2- 172.21.15.143:0/3828459720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0cd810 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f562c06b3f0 tx=0x7f561c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f432b505094daf11 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f5629600640 1 -- 172.21.15.143:0/3828459720 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0ab600 con 0x7f562c15b120 2024-01-15T18:17:43.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f5629600640 1 -- 172.21.15.143:0/3828459720 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c0a7050 con 0x7f562c15b120 2024-01-15T18:17:43.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.092+0000 7f5629600640 1 -- 172.21.15.143:0/3828459720 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0ab980 con 0x7f562c15b120 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- 172.21.15.143:0/3828459720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 msgr2=0x7f562c0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 --2- 172.21.15.143:0/3828459720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0cd810 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f562c06b3f0 tx=0x7f561c01c430 comp rx=0 tx=0).stop 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- 172.21.15.143:0/3828459720 shutdown_connections 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 --2- 172.21.15.143:0/3828459720 >> v2:172.21.15.204:3300/0 conn(0x7f562c0cf400 0x7f562c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 --2- 172.21.15.143:0/3828459720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0cd810 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- 172.21.15.143:0/3828459720 >> 172.21.15.143:0/3828459720 conn(0x7f562c0d0e20 msgr2=0x7f562c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- 172.21.15.143:0/3828459720 shutdown_connections 2024-01-15T18:17:43.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- 172.21.15.143:0/3828459720 wait complete. 2024-01-15T18:17:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 Processor -- start 2024-01-15T18:17:43.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- start start 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f562c0cf400 0x7f562c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f562c0d1940 0x7f562c155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c1536e0 con 0x7f562c15b120 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c156970 con 0x7f562c0cf400 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f5631600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c156c50 con 0x7f562c0d1940 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f562b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f562b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48476/0 (socket says 172.21.15.143:48476) 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.096+0000 7f562b400640 1 -- 172.21.15.143:0/94255429 learned_addr learned my addr 172.21.15.143:0/94255429 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562aa00640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f562c0cf400 0x7f562c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562a000640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f562c0d1940 0x7f562c155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562b400640 1 -- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f562c0d1940 msgr2=0x7f562c155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562b400640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f562c0d1940 0x7f562c155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562b400640 1 -- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f562c0cf400 msgr2=0x7f562c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562b400640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f562c0cf400 0x7f562c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562b400640 1 -- 172.21.15.143:0/94255429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c0094d0 con 0x7f562c15b120 2024-01-15T18:17:43.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f562b400640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0b1ee0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f562400c780 tx=0x7f562400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624004b70 con 0x7f562c15b120 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562400ce90 con 0x7f562c15b120 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562400a850 con 0x7f562c15b120 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c156f30 con 0x7f562c15b120 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c1574f0 con 0x7f562c15b120 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c067500 con 0x7f562c15b120 2024-01-15T18:17:43.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 34) v1 ==== 87537+0+0 (secure 0 0 0) 0x7f562400a9b0 con 0x7f562c15b120 2024-01-15T18:17:43.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.100+0000 7f5622a00640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5604067820 0x7f5604069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.104+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5624083240 con 0x7f562c15b120 2024-01-15T18:17:43.100 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.104+0000 7f562aa00640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5604067820 0x7f5604069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.104+0000 7f562aa00640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5604067820 0x7f5604069ce0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ad000 tx=0x7f561c0a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.108+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5624051cd0 con 0x7f562c15b120 2024-01-15T18:17:43.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.280+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f562c156dc0 con 0x7f5604067820 2024-01-15T18:17:43.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5622a00640 1 -- 172.21.15.143:0/94255429 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f562c156dc0 con 0x7f5604067820 2024-01-15T18:17:43.282 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab 2024-01-15T18:17:43.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5604067820 msgr2=0x7f5604069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5604067820 0x7f5604069ce0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f561c0ad000 tx=0x7f561c0a6000 comp rx=0 tx=0).stop 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 msgr2=0x7f562c0b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0b1ee0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f562400c780 tx=0x7f562400cc40 comp rx=0 tx=0).stop 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 shutdown_connections 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5604067820 0x7f5604069ce0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f562c15b120 0x7f562c0b1ee0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f562c0d1940 0x7f562c155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 --2- 172.21.15.143:0/94255429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f562c0cf400 0x7f562c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.284+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 >> 172.21.15.143:0/94255429 conn(0x7f562c0d0e20 msgr2=0x7f562c0cd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.288+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 shutdown_connections 2024-01-15T18:17:43.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:17:43.288+0000 7f5631600640 1 -- 172.21.15.143:0/94255429 wait complete. 2024-01-15T18:17:43.304 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab 2024-01-15T18:17:43.304 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:43.304 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:43.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:43.313 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:17:50.574 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:17:50.574 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:17:50.575 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.19451 s, 149 MB/s 2024-01-15T18:17:50.576 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:50.576 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:50.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.585 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:17:57.523 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:17:57.524 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:17:57.524 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.8836 s, 156 MB/s 2024-01-15T18:17:57.525 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:17:57.525 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:57.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:57.536 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:17:59.813 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:59.817 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:04.459 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:18:04.459 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:18:04.459 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.87086 s, 156 MB/s 2024-01-15T18:18:04.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:18:04.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 Processor -- start 2024-01-15T18:18:04.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 -- start start 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1908143780 0x7f19080cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19080ce520 0x7f1908071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19080ceb50 con 0x7f19080720f0 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190815c110 con 0x7f19080ce520 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f190ce00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1908072530 con 0x7f1908143780 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19080ce520 0x7f1908071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19080ce520 0x7f1908071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56356/0 (socket says 172.21.15.143:56356) 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1906800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f19080720f0 0x7f190815a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36162/0 (socket says 172.21.15.143:36162) 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 -- 172.21.15.143:0/1997053070 learned_addr learned my addr 172.21.15.143:0/1997053070 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:04.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905e00640 1 --2- 172.21.15.143:0/1997053070 >> v2:172.21.15.204:3300/0 conn(0x7f1908143780 0x7f19080cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 -- 172.21.15.143:0/1997053070 >> v2:172.21.15.204:3300/0 conn(0x7f1908143780 msgr2=0x7f19080cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:04.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 --2- 172.21.15.143:0/1997053070 >> v2:172.21.15.204:3300/0 conn(0x7f1908143780 0x7f19080cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 -- 172.21.15.143:0/1997053070 >> v1:172.21.15.143:6789/0 conn(0x7f19080720f0 legacy=0x7f190815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.608+0000 7f1905400640 1 -- 172.21.15.143:0/1997053070 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190815b7e0 con 0x7f19080ce520 2024-01-15T18:18:04.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f1905400640 1 --2- 172.21.15.143:0/1997053070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908071bb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f190806be30 tx=0x7f18f001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a949b211a91c09 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:04.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f1904a00640 1 -- 172.21.15.143:0/1997053070 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00ab610 con 0x7f19080ce520 2024-01-15T18:18:04.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f1904a00640 1 -- 172.21.15.143:0/1997053070 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f00ab770 con 0x7f19080ce520 2024-01-15T18:18:04.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f1904a00640 1 -- 172.21.15.143:0/1997053070 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00abb00 con 0x7f19080ce520 2024-01-15T18:18:04.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 -- 172.21.15.143:0/1997053070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 msgr2=0x7f1908071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 --2- 172.21.15.143:0/1997053070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908071bb0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f190806be30 tx=0x7f18f001c500 comp rx=0 tx=0).stop 2024-01-15T18:18:04.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 -- 172.21.15.143:0/1997053070 shutdown_connections 2024-01-15T18:18:04.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 --2- 172.21.15.143:0/1997053070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908071bb0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 --2- 172.21.15.143:0/1997053070 >> v2:172.21.15.204:3300/0 conn(0x7f1908143780 0x7f19080cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 -- 172.21.15.143:0/1997053070 >> 172.21.15.143:0/1997053070 conn(0x7f190806c3f0 msgr2=0x7f190806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:04.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 -- 172.21.15.143:0/1997053070 shutdown_connections 2024-01-15T18:18:04.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.612+0000 7f190ce00640 1 -- 172.21.15.143:0/1997053070 wait complete. 2024-01-15T18:18:04.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 Processor -- start 2024-01-15T18:18:04.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 -- start start 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19080720f0 0x7f19080b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1908143780 0x7f19080ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081585a0 con 0x7f19080ce520 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908159380 con 0x7f1908143780 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f190ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1908158800 con 0x7f19080720f0 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56358/0 (socket says 172.21.15.143:56358) 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 -- 172.21.15.143:0/1807917142 learned_addr learned my addr 172.21.15.143:0/1807917142 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1906800640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1908143780 0x7f19080ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905e00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19080720f0 0x7f19080b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 -- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19080720f0 msgr2=0x7f19080b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19080720f0 0x7f19080b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 -- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1908143780 msgr2=0x7f19080ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1908143780 0x7f19080ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 -- 172.21.15.143:0/1807917142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00094d0 con 0x7f19080ce520 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.616+0000 7f1905400640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908158060 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f18f0019110 tx=0x7f18f001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00b4800 con 0x7f19080ce520 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f00b4960 con 0x7f19080ce520 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1908158ae0 con 0x7f19080ce520 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f00b4cd0 con 0x7f19080ce520 2024-01-15T18:18:04.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081590a0 con 0x7f19080ce520 2024-01-15T18:18:04.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190813f670 con 0x7f19080ce520 2024-01-15T18:18:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 35) v1 ==== 87588+0+0 (secure 0 0 0) 0x7f18f00b3380 con 0x7f19080ce520 2024-01-15T18:18:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f18fe000640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18e00678b0 0x7f18e0069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f18f011d810 con 0x7f19080ce520 2024-01-15T18:18:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f1905e00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18e00678b0 0x7f18e0069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.620+0000 7f1905e00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18e00678b0 0x7f18e0069d70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009ac0 tx=0x7f18f8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:04.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.624+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f190813f670 con 0x7f19080ce520 2024-01-15T18:18:04.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.800+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f190813f670 con 0x7f18e00678b0 2024-01-15T18:18:04.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f18fe000640 1 -- 172.21.15.143:0/1807917142 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f190813f670 con 0x7f18e00678b0 2024-01-15T18:18:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18e00678b0 msgr2=0x7f18e0069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18e00678b0 0x7f18e0069d70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009ac0 tx=0x7f18f8009190 comp rx=0 tx=0).stop 2024-01-15T18:18:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 msgr2=0x7f1908158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908158060 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f18f0019110 tx=0x7f18f001ad40 comp rx=0 tx=0).stop 2024-01-15T18:18:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 shutdown_connections 2024-01-15T18:18:04.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f18e00678b0 0x7f18e0069d70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1908143780 0x7f19080ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19080ce520 0x7f1908158060 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 --2- 172.21.15.143:0/1807917142 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f19080720f0 0x7f19080b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 >> 172.21.15.143:0/1807917142 conn(0x7f190806c3f0 msgr2=0x7f190813b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:04.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 shutdown_connections 2024-01-15T18:18:04.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.816+0000 7f190ce00640 1 -- 172.21.15.143:0/1807917142 wait complete. 2024-01-15T18:18:04.831 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:18:04.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 Processor -- start 2024-01-15T18:18:04.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 -- start start 2024-01-15T18:18:04.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0110143c80 0x7f01100b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01100b27d0 0x7f01100b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01100b37a0 con 0x7f0110143430 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011006ca70 con 0x7f0110143c80 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.980+0000 7f0115400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0110143810 con 0x7f01100b27d0 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0110143c80 0x7f01100b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010e800640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f0110143430 0x7f01101462c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:47816/0 (socket says 172.21.15.143:47816) 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010e800640 1 -- 172.21.15.143:0/2597962740 learned_addr learned my addr 172.21.15.143:0/2597962740 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f0114a00640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01100b27d0 0x7f01100b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010d400640 1 -- 172.21.15.143:0/2597962740 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525977277 0 0) 0x7f01100b37a0 con 0x7f0110143430 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010d400640 1 -- 172.21.15.143:0/2597962740 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f0003530 con 0x7f0110143430 2024-01-15T18:18:04.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010de00640 1 -- 172.21.15.143:0/2597962740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01100b27d0 msgr2=0x7f01100b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010de00640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01100b27d0 0x7f01100b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010de00640 1 -- 172.21.15.143:0/2597962740 >> v1:172.21.15.204:6789/0 conn(0x7f0110143430 legacy=0x7f01101462c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010de00640 1 -- 172.21.15.143:0/2597962740 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01100b3a80 con 0x7f0110143c80 2024-01-15T18:18:04.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f0114a00640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01100b27d0 0x7f01100b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:18:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010de00640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143c80 0x7f01100b2290 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f00f80097f0 tx=0x7f00f801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfe9f2037096a61 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010d400640 1 -- 172.21.15.143:0/2597962740 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80ab600 con 0x7f0110143c80 2024-01-15T18:18:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010d400640 1 -- 172.21.15.143:0/2597962740 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80a7050 con 0x7f0110143c80 2024-01-15T18:18:04.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.984+0000 7f010d400640 1 -- 172.21.15.143:0/2597962740 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80ab980 con 0x7f0110143c80 2024-01-15T18:18:04.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- 172.21.15.143:0/2597962740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143c80 msgr2=0x7f01100b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143c80 0x7f01100b2290 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f00f80097f0 tx=0x7f00f801c480 comp rx=0 tx=0).stop 2024-01-15T18:18:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- 172.21.15.143:0/2597962740 shutdown_connections 2024-01-15T18:18:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f01100b27d0 0x7f01100b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 --2- 172.21.15.143:0/2597962740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143c80 0x7f01100b2290 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- 172.21.15.143:0/2597962740 >> 172.21.15.143:0/2597962740 conn(0x7f0110066720 msgr2=0x7f011006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- 172.21.15.143:0/2597962740 shutdown_connections 2024-01-15T18:18:04.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- 172.21.15.143:0/2597962740 wait complete. 2024-01-15T18:18:04.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 Processor -- start 2024-01-15T18:18:04.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- start start 2024-01-15T18:18:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01100b27d0 0x7f011015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 0x7f011013aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0110143c80 0x7f011013b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011013c410 con 0x7f0110143430 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011013b550 con 0x7f01100b27d0 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0115400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011013b790 con 0x7f0110143c80 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f010de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 0x7f011013aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f010de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 0x7f011013aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56368/0 (socket says 172.21.15.143:56368) 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f010de00640 1 -- 172.21.15.143:0/351218301 learned_addr learned my addr 172.21.15.143:0/351218301 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f010e800640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01100b27d0 0x7f011015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.988+0000 7f0114a00640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0110143c80 0x7f011013b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010de00640 1 -- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0110143c80 msgr2=0x7f011013b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010de00640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0110143c80 0x7f011013b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010de00640 1 -- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01100b27d0 msgr2=0x7f011015fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010de00640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01100b27d0 0x7f011015fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010de00640 1 -- 172.21.15.143:0/351218301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0108009a70 con 0x7f0110143430 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010de00640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 0x7f011013aad0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f00f801a0b0 tx=0x7f00f8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8006e20 con 0x7f0110143430 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f80b4400 con 0x7f0110143430 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f80b47d0 con 0x7f0110143430 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f8009420 con 0x7f0110143430 2024-01-15T18:18:04.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011013bd50 con 0x7f0110143430 2024-01-15T18:18:04.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011013c550 con 0x7f0110143430 2024-01-15T18:18:04.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 35) v1 ==== 87588+0+0 (secure 0 0 0) 0x7f00f80b2040 con 0x7f0110143430 2024-01-15T18:18:04.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0106a00640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00e80678b0 0x7f00e8069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:04.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f00f811d010 con 0x7f0110143430 2024-01-15T18:18:04.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.992+0000 7f010e800640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00e80678b0 0x7f00e8069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:04.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.996+0000 7f010e800640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00e80678b0 0x7f00e8069d70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f01000011f0 tx=0x7f0100007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:04.992 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:04.996+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f011013c550 con 0x7f0110143430 2024-01-15T18:18:05.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.172+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f01100d0200 con 0x7f00e80678b0 2024-01-15T18:18:05.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0106a00640 1 -- 172.21.15.143:0/351218301 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f01100d0200 con 0x7f00e80678b0 2024-01-15T18:18:05.173 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00e80678b0 msgr2=0x7f00e8069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00e80678b0 0x7f00e8069d70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f01000011f0 tx=0x7f0100007d40 comp rx=0 tx=0).stop 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 msgr2=0x7f011013aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 0x7f011013aad0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f00f801a0b0 tx=0x7f00f8019220 comp rx=0 tx=0).stop 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 shutdown_connections 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00e80678b0 0x7f00e8069d70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0110143c80 0x7f011013b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0110143430 0x7f011013aad0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 --2- 172.21.15.143:0/351218301 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f01100b27d0 0x7f011015fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 >> 172.21.15.143:0/351218301 conn(0x7f0110066720 msgr2=0x7f01101460a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:05.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 shutdown_connections 2024-01-15T18:18:05.177 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:05.180+0000 7f0115400640 1 -- 172.21.15.143:0/351218301 wait complete. 2024-01-15T18:18:05.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:05.195 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab) 2024-01-15T18:18:05.209 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab 2024-01-15T18:18:05.209 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:18:05.209 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:05.209 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:18:07.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:07.210 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab) 2024-01-15T18:18:07.225 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/8c031f94-d4b5-49ab-aba2-1d38188c9eab 2024-01-15T18:18:07.225 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:18:07.226 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:07.227 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:18:07.227 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-15T18:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.424+0000 7fa528c00640 1 Processor -- start 2024-01-15T18:18:07.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.424+0000 7fa528c00640 1 -- start start 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.424+0000 7fa528c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa524141660 0x7fa5240ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.424+0000 7fa528c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52415f310 con 0x7fa524071d00 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.424+0000 7fa528c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52415f5f0 con 0x7fa52415ec80 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.424+0000 7fa528c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52415fbb0 con 0x7fa524141660 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa522800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa524071d00 0x7fa5240b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41806/0 (socket says 172.21.15.143:41806) 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa522800640 1 -- 172.21.15.143:0/2265461531 learned_addr learned my addr 172.21.15.143:0/2265461531 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:07.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa521e00640 1 --2- 172.21.15.143:0/2265461531 >> v2:172.21.15.204:3300/0 conn(0x7fa524141660 0x7fa5240ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa520a00640 1 -- 172.21.15.143:0/2265461531 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080688272 0 0) 0x7fa52415f310 con 0x7fa524071d00 2024-01-15T18:18:07.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa520a00640 1 -- 172.21.15.143:0/2265461531 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa504003530 con 0x7fa524071d00 2024-01-15T18:18:07.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa521e00640 1 -- 172.21.15.143:0/2265461531 >> v1:172.21.15.204:6789/0 conn(0x7fa52415ec80 legacy=0x7fa524141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa521e00640 1 -- 172.21.15.143:0/2265461531 >> v1:172.21.15.143:6789/0 conn(0x7fa524071d00 legacy=0x7fa5240b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa520a00640 1 -- 172.21.15.143:0/2265461531 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210054646 0 0) 0x7fa504003530 con 0x7fa524071d00 2024-01-15T18:18:07.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa521e00640 1 -- 172.21.15.143:0/2265461531 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52415f8d0 con 0x7fa524141660 2024-01-15T18:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa521e00640 1 --2- 172.21.15.143:0/2265461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa524141660 0x7fa5240ce560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa5140097c0 tx=0x7fa51401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cefd2ade6221a4a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.428+0000 7fa520a00640 1 -- 172.21.15.143:0/2265461531 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140ab670 con 0x7fa524141660 2024-01-15T18:18:07.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa520a00640 1 -- 172.21.15.143:0/2265461531 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5140ab7d0 con 0x7fa524141660 2024-01-15T18:18:07.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa520a00640 1 -- 172.21.15.143:0/2265461531 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140abb60 con 0x7fa524141660 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 -- 172.21.15.143:0/2265461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa524141660 msgr2=0x7fa5240ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 --2- 172.21.15.143:0/2265461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa524141660 0x7fa5240ce560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fa5140097c0 tx=0x7fa51401c450 comp rx=0 tx=0).stop 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 -- 172.21.15.143:0/2265461531 shutdown_connections 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 --2- 172.21.15.143:0/2265461531 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa524141660 0x7fa5240ce560 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 -- 172.21.15.143:0/2265461531 >> 172.21.15.143:0/2265461531 conn(0x7fa52406c3f0 msgr2=0x7fa52406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 -- 172.21.15.143:0/2265461531 shutdown_connections 2024-01-15T18:18:07.426 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 -- 172.21.15.143:0/2265461531 wait complete. 2024-01-15T18:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 Processor -- start 2024-01-15T18:18:07.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 -- start start 2024-01-15T18:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa524071d00 0x7fa524159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.432+0000 7fa528c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 0x7fa52413d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52415ec80 0x7fa5241f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52415f130 con 0x7fa524141660 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52415ab60 con 0x7fa52415ec80 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52415aff0 con 0x7fa524071d00 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 0x7fa52413d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 0x7fa52413d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53448/0 (socket says 172.21.15.143:53448) 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 -- 172.21.15.143:0/3784877653 learned_addr learned my addr 172.21.15.143:0/3784877653 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa522800640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52415ec80 0x7fa5241f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521e00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa524071d00 0x7fa524159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 -- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa524071d00 msgr2=0x7fa524159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa524071d00 0x7fa524159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 -- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52415ec80 msgr2=0x7fa5241f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52415ec80 0x7fa5241f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 -- 172.21.15.143:0/3784877653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514009420 con 0x7fa524141660 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa521400640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 0x7fa52413d050 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c3c0 tx=0x7fa50c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0052d0 con 0x7fa524141660 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c005430 con 0x7fa524141660 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c003840 con 0x7fa524141660 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5240cd150 con 0x7fa524141660 2024-01-15T18:18:07.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52415a230 con 0x7fa524141660 2024-01-15T18:18:07.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52415acf0 con 0x7fa524141660 2024-01-15T18:18:07.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.436+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 35) v1 ==== 87588+0+0 (secure 0 0 0) 0x7fa50c0039a0 con 0x7fa524141660 2024-01-15T18:18:07.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.440+0000 7fa51a000640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4fc0678e0 0x7fa4fc069da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.440+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa50c0823b0 con 0x7fa524141660 2024-01-15T18:18:07.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.440+0000 7fa521e00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4fc0678e0 0x7fa4fc069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.440+0000 7fa521e00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4fc0678e0 0x7fa4fc069da0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa5140af000 tx=0x7fa5140a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:07.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.444+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa50c0510f0 con 0x7fa524141660 2024-01-15T18:18:07.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.620+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa524072270 con 0x7fa4fc0678e0 2024-01-15T18:18:07.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.672+0000 7fa51a000640 1 -- 172.21.15.143:0/3784877653 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa524072270 con 0x7fa4fc0678e0 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4fc0678e0 msgr2=0x7fa4fc069da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4fc0678e0 0x7fa4fc069da0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa5140af000 tx=0x7fa5140a7000 comp rx=0 tx=0).stop 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 msgr2=0x7fa52413d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 0x7fa52413d050 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c3c0 tx=0x7fa50c00c880 comp rx=0 tx=0).stop 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 shutdown_connections 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa4fc0678e0 0x7fa4fc069da0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa52415ec80 0x7fa5241f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa524141660 0x7fa52413d050 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 --2- 172.21.15.143:0/3784877653 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa524071d00 0x7fa524159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 >> 172.21.15.143:0/3784877653 conn(0x7fa52406c3f0 msgr2=0x7fa5240aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 shutdown_connections 2024-01-15T18:18:07.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.676+0000 7fa528c00640 1 -- 172.21.15.143:0/3784877653 wait complete. 2024-01-15T18:18:07.674 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.680+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.674 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.680+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.674 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.680+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.674 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.680+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.674 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.680+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.692 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.696+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.692 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.696+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.692 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.696+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.692 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.696+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.692 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:07.696+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:07.693 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:18:07.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:18:07.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.840+0000 7f7675400640 1 Processor -- start 2024-01-15T18:18:07.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.840+0000 7f7675400640 1 -- start start 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f7675400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76700ce560 0x7f767015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f7675400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7670071d00 0x7f7670072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f7675400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7670072650 con 0x7f7670141660 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f7675400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767015fbb0 con 0x7f76700ce560 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f7675400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767015f310 con 0x7f7670071d00 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76700ce560 0x7f767015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766e800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7670141660 0x7f7670141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41812/0 (socket says 172.21.15.143:41812) 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f76700ce560 0x7f767015ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53486/0 (socket says 172.21.15.143:53486) 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766e800640 1 -- 172.21.15.143:0/4000063302 learned_addr learned my addr 172.21.15.143:0/4000063302 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766f200640 1 --2- 172.21.15.143:0/4000063302 >> v2:172.21.15.204:3300/0 conn(0x7f7670071d00 0x7f7670072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766de00640 1 -- 172.21.15.143:0/4000063302 >> v2:172.21.15.204:3300/0 conn(0x7f7670071d00 msgr2=0x7f7670072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:07.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766de00640 1 --2- 172.21.15.143:0/4000063302 >> v2:172.21.15.204:3300/0 conn(0x7f7670071d00 0x7f7670072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766de00640 1 -- 172.21.15.143:0/4000063302 >> v1:172.21.15.143:6789/0 conn(0x7f7670141660 legacy=0x7f7670141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.844+0000 7f766de00640 1 -- 172.21.15.143:0/4000063302 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767015f5f0 con 0x7f76700ce560 2024-01-15T18:18:07.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f766de00640 1 --2- 172.21.15.143:0/4000063302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76700ce560 0x7f767015ec80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f767006e250 tx=0x7f765801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f452b38471c87e8d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:07.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f766d400640 1 -- 172.21.15.143:0/4000063302 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658004230 con 0x7f76700ce560 2024-01-15T18:18:07.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f766d400640 1 -- 172.21.15.143:0/4000063302 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76580a7050 con 0x7f76700ce560 2024-01-15T18:18:07.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f766d400640 1 -- 172.21.15.143:0/4000063302 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658003620 con 0x7f76700ce560 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 -- 172.21.15.143:0/4000063302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76700ce560 msgr2=0x7f767015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 --2- 172.21.15.143:0/4000063302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76700ce560 0x7f767015ec80 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f767006e250 tx=0x7f765801c460 comp rx=0 tx=0).stop 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 -- 172.21.15.143:0/4000063302 shutdown_connections 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 --2- 172.21.15.143:0/4000063302 >> v2:172.21.15.204:3300/0 conn(0x7f7670071d00 0x7f7670072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 --2- 172.21.15.143:0/4000063302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76700ce560 0x7f767015ec80 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 -- 172.21.15.143:0/4000063302 >> 172.21.15.143:0/4000063302 conn(0x7f767006c3f0 msgr2=0x7f767006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 -- 172.21.15.143:0/4000063302 shutdown_connections 2024-01-15T18:18:07.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 -- 172.21.15.143:0/4000063302 wait complete. 2024-01-15T18:18:07.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 Processor -- start 2024-01-15T18:18:07.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.848+0000 7f7675400640 1 -- start start 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 0x7f767013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76700ce560 0x7f767015b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670141660 0x7f7670158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700b2da0 con 0x7f7670071d00 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767015bca0 con 0x7f7670141660 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767015bf40 con 0x7f76700ce560 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 0x7f767013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 0x7f767013cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53502/0 (socket says 172.21.15.143:53502) 2024-01-15T18:18:07.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 -- 172.21.15.143:0/1758791787 learned_addr learned my addr 172.21.15.143:0/1758791787 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766de00640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76700ce560 0x7f767015b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766f200640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670141660 0x7f7670158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 -- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76700ce560 msgr2=0x7f767015b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76700ce560 0x7f767015b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 -- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670141660 msgr2=0x7f7670158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670141660 0x7f7670158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 -- 172.21.15.143:0/1758791787 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658009420 con 0x7f7670071d00 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f766e800640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 0x7f767013cd50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f766000c400 tx=0x7f766000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660005cf0 con 0x7f7670071d00 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660005e50 con 0x7f7670071d00 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76700cd150 con 0x7f7670071d00 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600072d0 con 0x7f7670071d00 2024-01-15T18:18:07.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767015c500 con 0x7f7670071d00 2024-01-15T18:18:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7670067500 con 0x7f7670071d00 2024-01-15T18:18:07.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.852+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 35) v1 ==== 87588+0+0 (secure 0 0 0) 0x7f7660007430 con 0x7f7670071d00 2024-01-15T18:18:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.856+0000 7f7666a00640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f76480678b0 0x7f7648069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.856+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7660082550 con 0x7f7670071d00 2024-01-15T18:18:07.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.856+0000 7f766de00640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f76480678b0 0x7f7648069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:07.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.856+0000 7f766de00640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f76480678b0 0x7f7648069d70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f76580ad000 tx=0x7f76580a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:07.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:07.860+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7660050f80 con 0x7f7670071d00 2024-01-15T18:18:08.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.044+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f76700ccfd0 con 0x7f76480678b0 2024-01-15T18:18:08.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.048+0000 7f7666a00640 1 -- 172.21.15.143:0/1758791787 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f7658004350 con 0x7f76480678b0 2024-01-15T18:18:08.043 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:18:08.043 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:08.044 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f76480678b0 msgr2=0x7f7648069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f76480678b0 0x7f7648069d70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f76580ad000 tx=0x7f76580a6000 comp rx=0 tx=0).stop 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 msgr2=0x7f767013cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 0x7f767013cd50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f766000c400 tx=0x7f766000c8c0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 shutdown_connections 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f76480678b0 0x7f7648069d70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7670141660 0x7f7670158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f76700ce560 0x7f767015b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 --2- 172.21.15.143:0/1758791787 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7670071d00 0x7f767013cd50 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 >> 172.21.15.143:0/1758791787 conn(0x7f767006c3f0 msgr2=0x7f76700ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:08.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 shutdown_connections 2024-01-15T18:18:08.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.052+0000 7f7675400640 1 -- 172.21.15.143:0/1758791787 wait complete. 2024-01-15T18:18:08.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-01-15T18:18:08.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.208+0000 7fef2b600640 1 Processor -- start 2024-01-15T18:18:08.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.208+0000 7fef2b600640 1 -- start start 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef2b600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fef240b24b0 0x7fef240ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef2b600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef240b3bf0 0x7fef240b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef2b600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef24070bf0 con 0x7fef240ce690 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef2b600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2415dba0 con 0x7fef240b3bf0 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef2b600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef240b4920 con 0x7fef240b24b0 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef29600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef240b3bf0 0x7fef240b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef28c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fef240ce690 0x7fef240b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41826/0 (socket says 172.21.15.143:41826) 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef28c00640 1 -- 172.21.15.143:0/1839864358 learned_addr learned my addr 172.21.15.143:0/1839864358 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef23e00640 1 --2- 172.21.15.143:0/1839864358 >> v2:172.21.15.204:3300/0 conn(0x7fef240b24b0 0x7fef240ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef29600640 1 -- 172.21.15.143:0/1839864358 >> v2:172.21.15.204:3300/0 conn(0x7fef240b24b0 msgr2=0x7fef240ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:08.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef29600640 1 --2- 172.21.15.143:0/1839864358 >> v2:172.21.15.204:3300/0 conn(0x7fef240b24b0 0x7fef240ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef29600640 1 -- 172.21.15.143:0/1839864358 >> v1:172.21.15.143:6789/0 conn(0x7fef240ce690 legacy=0x7fef240b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:08.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef29600640 1 -- 172.21.15.143:0/1839864358 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef240b4c00 con 0x7fef240b3bf0 2024-01-15T18:18:08.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef23400640 1 -- 172.21.15.143:0/1839864358 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300399846 0 0) 0x7fef24070bf0 con 0x7fef240ce690 2024-01-15T18:18:08.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.212+0000 7fef23e00640 1 --2- 172.21.15.143:0/1839864358 >> v2:172.21.15.204:3300/0 conn(0x7fef240b24b0 0x7fef240ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:08.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef29600640 1 --2- 172.21.15.143:0/1839864358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240b3bf0 0x7fef240b40a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fef2406f940 tx=0x7fef1c01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b35673c917763e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:08.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef23400640 1 -- 172.21.15.143:0/1839864358 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0a59c0 con 0x7fef240b3bf0 2024-01-15T18:18:08.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef23400640 1 -- 172.21.15.143:0/1839864358 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c0a5b20 con 0x7fef240b3bf0 2024-01-15T18:18:08.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef23400640 1 -- 172.21.15.143:0/1839864358 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0ad440 con 0x7fef240b3bf0 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 -- 172.21.15.143:0/1839864358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240b3bf0 msgr2=0x7fef240b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 --2- 172.21.15.143:0/1839864358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240b3bf0 0x7fef240b40a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fef2406f940 tx=0x7fef1c01d340 comp rx=0 tx=0).stop 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 -- 172.21.15.143:0/1839864358 shutdown_connections 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 --2- 172.21.15.143:0/1839864358 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240b3bf0 0x7fef240b40a0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 --2- 172.21.15.143:0/1839864358 >> v2:172.21.15.204:3300/0 conn(0x7fef240b24b0 0x7fef240ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 -- 172.21.15.143:0/1839864358 >> 172.21.15.143:0/1839864358 conn(0x7fef2406c3f0 msgr2=0x7fef2406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 -- 172.21.15.143:0/1839864358 shutdown_connections 2024-01-15T18:18:08.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 -- 172.21.15.143:0/1839864358 wait complete. 2024-01-15T18:18:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 Processor -- start 2024-01-15T18:18:08.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.216+0000 7fef2b600640 1 -- start start 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240b24b0 0x7fef24145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b3bf0 0x7fef24146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 0x7fef240afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2415c410 con 0x7fef240ce690 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2415c6b0 con 0x7fef240b24b0 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2415c990 con 0x7fef240b3bf0 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 0x7fef240afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 0x7fef240afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53520/0 (socket says 172.21.15.143:53520) 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 -- 172.21.15.143:0/3720538416 learned_addr learned my addr 172.21.15.143:0/3720538416 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef28c00640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240b24b0 0x7fef24145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef23e00640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b3bf0 0x7fef24146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:08.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 -- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b3bf0 msgr2=0x7fef24146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b3bf0 0x7fef24146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 -- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240b24b0 msgr2=0x7fef24145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240b24b0 0x7fef24145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 -- 172.21.15.143:0/3720538416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c009a70 con 0x7fef240ce690 2024-01-15T18:18:08.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef29600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 0x7fef240afd10 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0b0040 tx=0x7fef1c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0adb10 con 0x7fef240ce690 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c0adc90 con 0x7fef240ce690 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0b6400 con 0x7fef240ce690 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef24145130 con 0x7fef240ce690 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef24161e90 con 0x7fef240ce690 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.220+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef24145320 con 0x7fef240ce690 2024-01-15T18:18:08.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.224+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 35) v1 ==== 87588+0+0 (secure 0 0 0) 0x7fef1c0b4040 con 0x7fef240ce690 2024-01-15T18:18:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.224+0000 7fef20c00640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feefc0678b0 0x7feefc069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.224+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fef1c11dad0 con 0x7fef240ce690 2024-01-15T18:18:08.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.224+0000 7fef28c00640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feefc0678b0 0x7feefc069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:08.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.224+0000 7fef28c00640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feefc0678b0 0x7feefc069d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fef14000f70 tx=0x7fef14007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:08.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.228+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fef1c0ec620 con 0x7fef240ce690 2024-01-15T18:18:08.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:08.408+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fef24067500 con 0x7feefc0678b0 2024-01-15T18:18:09.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:09.832+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7fef1c0ec9f0 con 0x7fef240ce690 2024-01-15T18:18:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.071+0000 7fef20c00640 1 -- 172.21.15.143:0/3720538416 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fef24067500 con 0x7feefc0678b0 2024-01-15T18:18:21.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feefc0678b0 msgr2=0x7feefc069d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feefc0678b0 0x7feefc069d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fef14000f70 tx=0x7fef14007df0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 msgr2=0x7fef240afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 0x7fef240afd10 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0b0040 tx=0x7fef1c019680 comp rx=0 tx=0).stop 2024-01-15T18:18:21.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 shutdown_connections 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7feefc0678b0 0x7feefc069d70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef240ce690 0x7fef240afd10 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fef240b3bf0 0x7fef24146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 --2- 172.21.15.143:0/3720538416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef240b24b0 0x7fef24145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 >> 172.21.15.143:0/3720538416 conn(0x7fef2406c3f0 msgr2=0x7fef2415a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 shutdown_connections 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.075+0000 7fef2b600640 1 -- 172.21.15.143:0/3720538416 wait complete. 2024-01-15T18:18:21.091 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:18:21.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2436a00640 1 Processor -- start 2024-01-15T18:18:21.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2436a00640 1 -- start start 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2436a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f24300c6230 0x7f24300cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2436a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24300c4d90 con 0x7f243015b260 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2436a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243015b640 con 0x7f24300cdf20 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2436a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24300cf800 con 0x7f24300c6230 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f242fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f243015b260 0x7f24300cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43416/0 (socket says 172.21.15.143:43416) 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f242fe00640 1 -- 172.21.15.143:0/1631973296 learned_addr learned my addr 172.21.15.143:0/1631973296 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:21.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2434a00640 1 --2- 172.21.15.143:0/1631973296 >> v2:172.21.15.204:3300/0 conn(0x7f24300c6230 0x7f24300cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f242ea00640 1 -- 172.21.15.143:0/1631973296 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143068088 0 0) 0x7f24300c4d90 con 0x7f243015b260 2024-01-15T18:18:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f242ea00640 1 -- 172.21.15.143:0/1631973296 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2410003530 con 0x7f243015b260 2024-01-15T18:18:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f242ea00640 1 -- 172.21.15.143:0/1631973296 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1178300710 0 0) 0x7f2410003530 con 0x7f243015b260 2024-01-15T18:18:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2434a00640 1 -- 172.21.15.143:0/1631973296 >> v1:172.21.15.204:6789/0 conn(0x7f24300cdf20 legacy=0x7f24300cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2434a00640 1 -- 172.21.15.143:0/1631973296 >> v1:172.21.15.143:6789/0 conn(0x7f243015b260 legacy=0x7f24300cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.239+0000 7f2434a00640 1 -- 172.21.15.143:0/1631973296 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300cfae0 con 0x7f24300c6230 2024-01-15T18:18:21.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2434a00640 1 --2- 172.21.15.143:0/1631973296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24300cc1f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2428009d90 tx=0x7f242801c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b608f8da35f28693 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:21.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f242ea00640 1 -- 172.21.15.143:0/1631973296 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2420001680 con 0x7f24300c6230 2024-01-15T18:18:21.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f242ea00640 1 -- 172.21.15.143:0/1631973296 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24300ccf40 con 0x7f24300c6230 2024-01-15T18:18:21.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f242ea00640 1 -- 172.21.15.143:0/1631973296 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24280a5790 con 0x7f24300c6230 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 -- 172.21.15.143:0/1631973296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 msgr2=0x7f24300cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 --2- 172.21.15.143:0/1631973296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24300cc1f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2428009d90 tx=0x7f242801c6f0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 -- 172.21.15.143:0/1631973296 shutdown_connections 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 --2- 172.21.15.143:0/1631973296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24300cc1f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 -- 172.21.15.143:0/1631973296 >> 172.21.15.143:0/1631973296 conn(0x7f24300d0ed0 msgr2=0x7f24300d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 -- 172.21.15.143:0/1631973296 shutdown_connections 2024-01-15T18:18:21.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.243+0000 7f2436a00640 1 -- 172.21.15.143:0/1631973296 wait complete. 2024-01-15T18:18:21.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 Processor -- start 2024-01-15T18:18:21.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 -- start start 2024-01-15T18:18:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24301512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 0x7f2430153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243015b260 0x7f2430153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2430155560 con 0x7f24300cdf20 2024-01-15T18:18:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24301541d0 con 0x7f24300c6230 2024-01-15T18:18:21.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24301557b0 con 0x7f243015b260 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 0x7f2430153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 0x7f2430153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57092/0 (socket says 172.21.15.143:57092) 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 -- 172.21.15.143:0/1725912212 learned_addr learned my addr 172.21.15.143:0/1725912212 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242fe00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24301512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2434a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243015b260 0x7f2430153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 -- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243015b260 msgr2=0x7f2430153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243015b260 0x7f2430153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 -- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 msgr2=0x7f24301512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24301512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 -- 172.21.15.143:0/1725912212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009a70 con 0x7f24300cdf20 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242fe00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24301512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f242f400640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 0x7f2430153750 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f241800c500 tx=0x7f241800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:21.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418003600 con 0x7f24300cdf20 2024-01-15T18:18:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418003760 con 0x7f24300cdf20 2024-01-15T18:18:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2430155a90 con 0x7f24300cdf20 2024-01-15T18:18:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418003b10 con 0x7f24300cdf20 2024-01-15T18:18:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.247+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2430156050 con 0x7f24300cdf20 2024-01-15T18:18:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.251+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2430067500 con 0x7f24300cdf20 2024-01-15T18:18:21.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.251+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7f2418003c70 con 0x7f24300cdf20 2024-01-15T18:18:21.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.251+0000 7f2427e00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24080679d0 0x7f2408069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.251+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f24180829e0 con 0x7f24300cdf20 2024-01-15T18:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.251+0000 7f242fe00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24080679d0 0x7f2408069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.251+0000 7f242fe00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24080679d0 0x7f2408069e90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2420001ec0 tx=0x7f2420001cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:21.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.255+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2418051330 con 0x7f24300cdf20 2024-01-15T18:18:21.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:21.435+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f24300c4770 con 0x7f24080679d0 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2427e00640 1 -- 172.21.15.143:0/1725912212 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f24300c4770 con 0x7f24080679d0 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "canceled", 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.078 INFO:teuthology.orchestra.run.smithi143.stdout: "failure": { 2024-01-15T18:18:22.079 INFO:teuthology.orchestra.run.smithi143.stdout: "errno": "4", 2024-01-15T18:18:22.079 INFO:teuthology.orchestra.run.smithi143.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:18:22.079 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:22.079 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:22.079 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24080679d0 msgr2=0x7f2408069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24080679d0 0x7f2408069e90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2420001ec0 tx=0x7f2420001cf0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 msgr2=0x7f2430153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 0x7f2430153750 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f241800c500 tx=0x7f241800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 shutdown_connections 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24080679d0 0x7f2408069e90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f243015b260 0x7f2430153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24300cdf20 0x7f2430153750 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 --2- 172.21.15.143:0/1725912212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24300c6230 0x7f24301512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 >> 172.21.15.143:0/1725912212 conn(0x7f24300d0ed0 msgr2=0x7f24300c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 shutdown_connections 2024-01-15T18:18:22.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.083+0000 7f2436a00640 1 -- 172.21.15.143:0/1725912212 wait complete. 2024-01-15T18:18:22.101 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:18:22.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 Processor -- start 2024-01-15T18:18:22.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 -- start start 2024-01-15T18:18:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9838143430 0x7f98381462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9838143c80 0x7f9838146800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98380b2290 0x7f98380b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f983806ca70 con 0x7f9838143430 2024-01-15T18:18:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9838143810 con 0x7f9838143c80 2024-01-15T18:18:22.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380b2c40 con 0x7f98380b2290 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9838143430 0x7f98381462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9838143430 0x7f98381462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57104/0 (socket says 172.21.15.143:57104) 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 -- 172.21.15.143:0/284083451 learned_addr learned my addr 172.21.15.143:0/284083451 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9836a00640 1 --2- 172.21.15.143:0/284083451 >> v2:172.21.15.204:3300/0 conn(0x7f9838143c80 0x7f9838146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f983ce00640 1 --2- 172.21.15.143:0/284083451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98380b2290 0x7f98380b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 -- 172.21.15.143:0/284083451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98380b2290 msgr2=0x7f98380b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 --2- 172.21.15.143:0/284083451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98380b2290 0x7f98380b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 -- 172.21.15.143:0/284083451 >> v2:172.21.15.204:3300/0 conn(0x7f9838143c80 msgr2=0x7f9838146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 --2- 172.21.15.143:0/284083451 >> v2:172.21.15.204:3300/0 conn(0x7f9838143c80 0x7f9838146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9837400640 1 -- 172.21.15.143:0/284083451 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380b37a0 con 0x7f9838143430 2024-01-15T18:18:22.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.243+0000 7f9836a00640 1 --2- 172.21.15.143:0/284083451 >> v2:172.21.15.204:3300/0 conn(0x7f9838143c80 0x7f9838146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f9837400640 1 --2- 172.21.15.143:0/284083451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143430 0x7f98381462c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f983806fc50 tx=0x7f982801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d116da7524d3fb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f9836000640 1 -- 172.21.15.143:0/284083451 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280aa5f0 con 0x7f9838143430 2024-01-15T18:18:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f9836000640 1 -- 172.21.15.143:0/284083451 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98280aa750 con 0x7f9838143430 2024-01-15T18:18:22.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f9836000640 1 -- 172.21.15.143:0/284083451 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280aaae0 con 0x7f9838143430 2024-01-15T18:18:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 -- 172.21.15.143:0/284083451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143430 msgr2=0x7f98381462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 --2- 172.21.15.143:0/284083451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143430 0x7f98381462c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f983806fc50 tx=0x7f982801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 -- 172.21.15.143:0/284083451 shutdown_connections 2024-01-15T18:18:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 --2- 172.21.15.143:0/284083451 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f98380b2290 0x7f98380b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 --2- 172.21.15.143:0/284083451 >> v2:172.21.15.204:3300/0 conn(0x7f9838143c80 0x7f9838146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 --2- 172.21.15.143:0/284083451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143430 0x7f98381462c0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.247+0000 7f983e400640 1 -- 172.21.15.143:0/284083451 >> 172.21.15.143:0/284083451 conn(0x7f9838066720 msgr2=0x7f983806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 -- 172.21.15.143:0/284083451 shutdown_connections 2024-01-15T18:18:22.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 -- 172.21.15.143:0/284083451 wait complete. 2024-01-15T18:18:22.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 Processor -- start 2024-01-15T18:18:22.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 -- start start 2024-01-15T18:18:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98380b2290 0x7f983815fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9838143430 0x7f9838146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 0x7f983815b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983815eb30 con 0x7f9838143c80 2024-01-15T18:18:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983815ed80 con 0x7f98380b2290 2024-01-15T18:18:22.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983e400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983815f060 con 0x7f9838143430 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 0x7f983815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 0x7f983815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57108/0 (socket says 172.21.15.143:57108) 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 -- 172.21.15.143:0/1986516907 learned_addr learned my addr 172.21.15.143:0/1986516907 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f9837400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98380b2290 0x7f983815fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f9836a00640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9838143430 0x7f9838146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 -- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9838143430 msgr2=0x7f9838146490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9838143430 0x7f9838146490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 -- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98380b2290 msgr2=0x7f983815fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98380b2290 0x7f983815fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f983ce00640 1 -- 172.21.15.143:0/1986516907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98280094d0 con 0x7f9838143c80 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.251+0000 7f9836a00640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9838143430 0x7f9838146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:22.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f983ce00640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 0x7f983815b700 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9830007e60 tx=0x7f983000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:22.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830014620 con 0x7f9838143c80 2024-01-15T18:18:22.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983000ee90 con 0x7f9838143c80 2024-01-15T18:18:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300149e0 con 0x7f9838143c80 2024-01-15T18:18:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983815f340 con 0x7f9838143c80 2024-01-15T18:18:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983815c470 con 0x7f9838143c80 2024-01-15T18:18:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9838067680 con 0x7f9838143c80 2024-01-15T18:18:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7f9830014b40 con 0x7f9838143c80 2024-01-15T18:18:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f982f400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9810067a00 0x7f9810069ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9830083830 con 0x7f9838143c80 2024-01-15T18:18:22.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.255+0000 7f9837400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9810067a00 0x7f9810069ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.259+0000 7f9837400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9810067a00 0x7f9810069ec0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f98280ae000 tx=0x7f98280b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:22.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.259+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f98300524c0 con 0x7f9838143c80 2024-01-15T18:18:22.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.463+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f983815f7f0 con 0x7f9838143c80 2024-01-15T18:18:22.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.463+0000 7f982f400640 1 -- 172.21.15.143:0/1986516907 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1853 (secure 0 0 0) 0x7f9830055ae0 con 0x7f9838143c80 2024-01-15T18:18:22.458 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:22.458 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:18:22.458 INFO:teuthology.orchestra.run.smithi143.stdout: "fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "health": { 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "HEALTH_OK", 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "checks": {}, 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "mutes": [] 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "election_epoch": 4, 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum": [ 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: 0, 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: 1, 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: 2 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_names": [ 2024-01-15T18:18:22.459 INFO:teuthology.orchestra.run.smithi143.stdout: "a", 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "b", 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "c" 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "quorum_age": 828, 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "monmap": { 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 1, 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "min_mon_release_name": "reef", 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "num_mons": 3 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap": { 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 100, 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "num_osds": 8, 2024-01-15T18:18:22.460 INFO:teuthology.orchestra.run.smithi143.stdout: "num_up_osds": 8, 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_up_since": 1705341886, 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_in_osds": 8, 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_in_since": 1705341876, 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_remapped_pgs": 0 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap": { 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "pgs_by_state": [ 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "state_name": "active+clean", 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "count": 97 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:18:22.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pgs": 97, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "num_pools": 3, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "num_objects": 797, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "data_bytes": 3222080367, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_used": 7013244928, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_avail": 766080868352, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "bytes_total": 773094113280, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "read_bytes_sec": 170, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "write_bytes_sec": 7336, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "read_op_per_sec": 0, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "write_op_per_sec": 2 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "fsmap": { 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 63, 2024-01-15T18:18:22.462 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 7, 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "up": 1, 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "in": 1, 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1, 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "by_rank": [ 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "filesystem_id": 7, 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "rank": 0, 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "a", 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "status": "up:active", 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: "gid": 8535 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:22.463 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "up:standby": 3 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "mgrmap": { 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "available": true, 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "num_standbys": 1, 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "modules": [ 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "iostat", 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "nfs", 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "restful" 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: ], 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.464 INFO:teuthology.orchestra.run.smithi143.stdout: "servicemap": { 2024-01-15T18:18:22.465 INFO:teuthology.orchestra.run.smithi143.stdout: "epoch": 13, 2024-01-15T18:18:22.465 INFO:teuthology.orchestra.run.smithi143.stdout: "modified": "2024-01-15T18:17:08.873691+0000", 2024-01-15T18:18:22.465 INFO:teuthology.orchestra.run.smithi143.stdout: "services": {} 2024-01-15T18:18:22.465 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:22.465 INFO:teuthology.orchestra.run.smithi143.stdout: "progress_events": {} 2024-01-15T18:18:22.465 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.463+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9810067a00 msgr2=0x7f9810069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.463+0000 7f983e400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9810067a00 0x7f9810069ec0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f98280ae000 tx=0x7f98280b5040 comp rx=0 tx=0).stop 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.463+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 msgr2=0x7f983815b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.463+0000 7f983e400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 0x7f983815b700 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f9830007e60 tx=0x7f983000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 shutdown_connections 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9810067a00 0x7f9810069ec0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9838143c80 0x7f983815b700 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9838143430 0x7f9838146490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 --2- 172.21.15.143:0/1986516907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98380b2290 0x7f983815fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 >> 172.21.15.143:0/1986516907 conn(0x7f9838066720 msgr2=0x7f9838139860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 shutdown_connections 2024-01-15T18:18:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.467+0000 7f983e400640 1 -- 172.21.15.143:0/1986516907 wait complete. 2024-01-15T18:18:22.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:22.481 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/8c031f94-d4b5-49ab-aba2-1d38188c9eab) 2024-01-15T18:18:22.493 INFO:teuthology.orchestra.run.smithi143.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/8c031f94-d4b5-49ab-aba2-1d38188c9eab': No such file or directory 2024-01-15T18:18:22.494 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-01-15T18:18:22.494 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:18:22.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 Processor -- start 2024-01-15T18:18:22.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 -- start start 2024-01-15T18:18:22.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7b815b610 0x7fa7b8140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b8071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa7b806a490 0x7fa7b806a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b813e420 con 0x7fa7b815b610 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b81416a0 con 0x7fa7b806a490 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7be200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b806ae60 con 0x7fa7b8071920 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7b815b610 0x7fa7b8140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa7b815b610 0x7fa7b8140700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57118/0 (socket says 172.21.15.143:57118) 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 -- 172.21.15.143:0/4149382268 learned_addr learned my addr 172.21.15.143:0/4149382268 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b6a00640 1 --2- 172.21.15.143:0/4149382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b8071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7e00640 1 --2- 172.21.15.143:0/4149382268 >> v2:172.21.15.204:3300/0 conn(0x7fa7b806a490 0x7fa7b806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 -- 172.21.15.143:0/4149382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 msgr2=0x7fa7b8071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 --2- 172.21.15.143:0/4149382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b8071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 -- 172.21.15.143:0/4149382268 >> v2:172.21.15.204:3300/0 conn(0x7fa7b806a490 msgr2=0x7fa7b806a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 --2- 172.21.15.143:0/4149382268 >> v2:172.21.15.204:3300/0 conn(0x7fa7b806a490 0x7fa7b806a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.679+0000 7fa7b7400640 1 -- 172.21.15.143:0/4149382268 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b8140e00 con 0x7fa7b815b610 2024-01-15T18:18:22.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7b7400640 1 --2- 172.21.15.143:0/4149382268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b815b610 0x7fa7b8140700 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa7b806fe10 tx=0x7fa7a801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0f84766346a3d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:22.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7b6000640 1 -- 172.21.15.143:0/4149382268 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80aa620 con 0x7fa7b815b610 2024-01-15T18:18:22.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7b6000640 1 -- 172.21.15.143:0/4149382268 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80aa780 con 0x7fa7b815b610 2024-01-15T18:18:22.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7b6000640 1 -- 172.21.15.143:0/4149382268 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80aab10 con 0x7fa7b815b610 2024-01-15T18:18:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 -- 172.21.15.143:0/4149382268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b815b610 msgr2=0x7fa7b8140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 --2- 172.21.15.143:0/4149382268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b815b610 0x7fa7b8140700 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa7b806fe10 tx=0x7fa7a801c500 comp rx=0 tx=0).stop 2024-01-15T18:18:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 -- 172.21.15.143:0/4149382268 shutdown_connections 2024-01-15T18:18:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 --2- 172.21.15.143:0/4149382268 >> v2:172.21.15.204:3300/0 conn(0x7fa7b806a490 0x7fa7b806a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 --2- 172.21.15.143:0/4149382268 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b8071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 --2- 172.21.15.143:0/4149382268 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b815b610 0x7fa7b8140700 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 -- 172.21.15.143:0/4149382268 >> 172.21.15.143:0/4149382268 conn(0x7fa7b80b1b90 msgr2=0x7fa7b80b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:22.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 -- 172.21.15.143:0/4149382268 shutdown_connections 2024-01-15T18:18:22.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.683+0000 7fa7be200640 1 -- 172.21.15.143:0/4149382268 wait complete. 2024-01-15T18:18:22.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 Processor -- start 2024-01-15T18:18:22.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 -- start start 2024-01-15T18:18:22.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 0x7fa7b8139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b815b610 0x7fa7b8135c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8136ca0 con 0x7fa7b806a490 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8139ed0 con 0x7fa7b815b610 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b813a1b0 con 0x7fa7b8071920 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 0x7fa7b8139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 0x7fa7b8139810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57130/0 (socket says 172.21.15.143:57130) 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 -- 172.21.15.143:0/929457888 learned_addr learned my addr 172.21.15.143:0/929457888 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:22.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b6a00640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7e00640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b815b610 0x7fa7b8135c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 -- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 msgr2=0x7fa7b815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b815a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 -- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b815b610 msgr2=0x7fa7b8135c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b815b610 0x7fa7b8135c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 -- 172.21.15.143:0/929457888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80094d0 con 0x7fa7b806a490 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7b7400640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 0x7fa7b8139810 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80ac040 tx=0x7fa7a801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80bd3f0 con 0x7fa7b806a490 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a80bd550 con 0x7fa7b806a490 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a80bd900 con 0x7fa7b806a490 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b8136930 con 0x7fa7b806a490 2024-01-15T18:18:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.687+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b80b1090 con 0x7fa7b806a490 2024-01-15T18:18:22.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.691+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b8136ac0 con 0x7fa7b806a490 2024-01-15T18:18:22.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.691+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7fa7a80bda60 con 0x7fa7b806a490 2024-01-15T18:18:22.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.691+0000 7fa7af400640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7900679d0 0x7fa790069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:22.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.691+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa7a811d510 con 0x7fa7b806a490 2024-01-15T18:18:22.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.691+0000 7fa7b6a00640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7900679d0 0x7fa790069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:22.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.691+0000 7fa7b6a00640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7900679d0 0x7fa790069e90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80cc1f0 tx=0x7fa7a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:22.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.695+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa7a80ec340 con 0x7fa7b806a490 2024-01-15T18:18:22.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.867+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7b813a320 con 0x7fa7900679d0 2024-01-15T18:18:22.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.867+0000 7fa7af400640 1 -- 172.21.15.143:0/929457888 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa7b813a320 con 0x7fa7900679d0 2024-01-15T18:18:22.863 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:18:22.863 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:22.863 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:18:22.863 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:22.863 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:18:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7900679d0 msgr2=0x7fa790069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7900679d0 0x7fa790069e90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa7b80cc1f0 tx=0x7fa7a0008040 comp rx=0 tx=0).stop 2024-01-15T18:18:22.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 msgr2=0x7fa7b8139810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 0x7fa7b8139810 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa7a80ac040 tx=0x7fa7a801ad40 comp rx=0 tx=0).stop 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 shutdown_connections 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa7900679d0 0x7fa790069e90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7b815b610 0x7fa7b8135c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa7b8071920 0x7fa7b815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 --2- 172.21.15.143:0/929457888 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7b806a490 0x7fa7b8139810 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 >> 172.21.15.143:0/929457888 conn(0x7fa7b80b1b90 msgr2=0x7fa7b80afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 shutdown_connections 2024-01-15T18:18:22.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:22.871+0000 7fa7be200640 1 -- 172.21.15.143:0/929457888 wait complete. 2024-01-15T18:18:22.885 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:18:23.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 Processor -- start 2024-01-15T18:18:23.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 -- start start 2024-01-15T18:18:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f444c0ce520 0x7f444c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c0ceb50 con 0x7f444c0720f0 2024-01-15T18:18:23.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c072580 con 0x7f444c0ce520 2024-01-15T18:18:23.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4454600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c15c110 con 0x7f444c143780 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4452600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f444c0720f0 0x7f444c15a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43432/0 (socket says 172.21.15.143:43432) 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4452600640 1 -- 172.21.15.143:0/3924080925 learned_addr learned my addr 172.21.15.143:0/3924080925 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4451200640 1 --2- 172.21.15.143:0/3924080925 >> v2:172.21.15.204:3300/0 conn(0x7f444c0ce520 0x7f444c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.031+0000 7f4451c00640 1 --2- 172.21.15.143:0/3924080925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f444be00640 1 -- 172.21.15.143:0/3924080925 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113513904 0 0) 0x7f44440014a0 con 0x7f444c0720f0 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f444be00640 1 -- 172.21.15.143:0/3924080925 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f442c003530 con 0x7f444c0720f0 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f4451c00640 1 -- 172.21.15.143:0/3924080925 >> v2:172.21.15.204:3300/0 conn(0x7f444c0ce520 msgr2=0x7f444c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f4451c00640 1 --2- 172.21.15.143:0/3924080925 >> v2:172.21.15.204:3300/0 conn(0x7f444c0ce520 0x7f444c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f4451c00640 1 -- 172.21.15.143:0/3924080925 >> v1:172.21.15.143:6789/0 conn(0x7f444c0720f0 legacy=0x7f444c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f444be00640 1 -- 172.21.15.143:0/3924080925 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263178658 0 0) 0x7f442c003530 con 0x7f444c0720f0 2024-01-15T18:18:23.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f4451c00640 1 -- 172.21.15.143:0/3924080925 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c15b7e0 con 0x7f444c143780 2024-01-15T18:18:23.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f4451c00640 1 --2- 172.21.15.143:0/3924080925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c0cdfe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f443c0097c0 tx=0x7f443c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f65fa7002a3c0f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f444be00640 1 -- 172.21.15.143:0/3924080925 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0ad640 con 0x7f444c143780 2024-01-15T18:18:23.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f444be00640 1 -- 172.21.15.143:0/3924080925 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c0ad7a0 con 0x7f444c143780 2024-01-15T18:18:23.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.035+0000 7f444be00640 1 -- 172.21.15.143:0/3924080925 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c0adb30 con 0x7f444c143780 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- 172.21.15.143:0/3924080925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 msgr2=0x7f444c0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 --2- 172.21.15.143:0/3924080925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c0cdfe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f443c0097c0 tx=0x7f443c01c480 comp rx=0 tx=0).stop 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- 172.21.15.143:0/3924080925 shutdown_connections 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 --2- 172.21.15.143:0/3924080925 >> v2:172.21.15.204:3300/0 conn(0x7f444c0ce520 0x7f444c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 --2- 172.21.15.143:0/3924080925 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c0cdfe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- 172.21.15.143:0/3924080925 >> 172.21.15.143:0/3924080925 conn(0x7f444c06c3f0 msgr2=0x7f444c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- 172.21.15.143:0/3924080925 shutdown_connections 2024-01-15T18:18:23.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- 172.21.15.143:0/3924080925 wait complete. 2024-01-15T18:18:23.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 Processor -- start 2024-01-15T18:18:23.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- start start 2024-01-15T18:18:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0720f0 0x7f444c1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 0x7f444c131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c134f80 con 0x7f444c0ce520 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c132520 con 0x7f444c0720f0 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4454600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1428a0 con 0x7f444c143780 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4451200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 0x7f444c131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4451200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 0x7f444c131fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57134/0 (socket says 172.21.15.143:57134) 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.039+0000 7f4451200640 1 -- 172.21.15.143:0/2930825054 learned_addr learned my addr 172.21.15.143:0/2930825054 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451c00640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0720f0 0x7f444c1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4452600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451200640 1 -- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 msgr2=0x7f444c158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451200640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451200640 1 -- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0720f0 msgr2=0x7f444c1595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451200640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0720f0 0x7f444c1595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451200640 1 -- 172.21.15.143:0/2930825054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009420 con 0x7f444c0ce520 2024-01-15T18:18:23.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451c00640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0720f0 0x7f444c1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4451200640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 0x7f444c131fe0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f443400e930 tx=0x7f443400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434009690 con 0x7f444c0ce520 2024-01-15T18:18:23.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443400ca00 con 0x7f444c0ce520 2024-01-15T18:18:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c142b80 con 0x7f444c0ce520 2024-01-15T18:18:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443400cd70 con 0x7f444c0ce520 2024-01-15T18:18:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c134a60 con 0x7f444c0ce520 2024-01-15T18:18:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c067500 con 0x7f444c0ce520 2024-01-15T18:18:23.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7f4434013470 con 0x7f444c0ce520 2024-01-15T18:18:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.043+0000 7f4449600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44240679d0 0x7f4424069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.047+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4434083510 con 0x7f444c0ce520 2024-01-15T18:18:23.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.047+0000 7f4451c00640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44240679d0 0x7f4424069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.047+0000 7f4451c00640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44240679d0 0x7f4424069e90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f443c0af000 tx=0x7f443c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.051+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4434051f20 con 0x7f444c0ce520 2024-01-15T18:18:23.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.219+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f444c071bd0 con 0x7f44240679d0 2024-01-15T18:18:23.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.223+0000 7f4449600640 1 -- 172.21.15.143:0/2930825054 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7f444c071bd0 con 0x7f44240679d0 2024-01-15T18:18:23.218 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:23.218 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:18:23.218 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:23.218 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:18:23.219 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:23.219 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:23.219 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1clone1" 2024-01-15T18:18:23.219 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:23.219 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.223+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44240679d0 msgr2=0x7f4424069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.223+0000 7f4454600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44240679d0 0x7f4424069e90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f443c0af000 tx=0x7f443c008040 comp rx=0 tx=0).stop 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.223+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 msgr2=0x7f444c131fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.223+0000 7f4454600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 0x7f444c131fe0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f443400e930 tx=0x7f443400edf0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 shutdown_connections 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44240679d0 0x7f4424069e90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f444c143780 0x7f444c158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f444c0ce520 0x7f444c131fe0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 --2- 172.21.15.143:0/2930825054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f444c0720f0 0x7f444c1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 >> 172.21.15.143:0/2930825054 conn(0x7f444c06c3f0 msgr2=0x7f444c0af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 shutdown_connections 2024-01-15T18:18:23.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.227+0000 7f4454600640 1 -- 172.21.15.143:0/2930825054 wait complete. 2024-01-15T18:18:23.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:18:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 Processor -- start 2024-01-15T18:18:23.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 -- start start 2024-01-15T18:18:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd2040b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2040b48f0 0x7fd204070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040b2a20 con 0x7fd204157300 2024-01-15T18:18:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2040b35a0 con 0x7fd2040b48f0 2024-01-15T18:18:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd204159fd0 con 0x7fd204159b70 2024-01-15T18:18:23.384 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2040b48f0 0x7fd204070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2040b48f0 0x7fd204070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57148/0 (socket says 172.21.15.143:57148) 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 -- 172.21.15.143:0/423251036 learned_addr learned my addr 172.21.15.143:0/423251036 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd209600640 1 --2- 172.21.15.143:0/423251036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd2040b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 -- 172.21.15.143:0/423251036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 msgr2=0x7fd2040b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 --2- 172.21.15.143:0/423251036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd2040b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 -- 172.21.15.143:0/423251036 >> v1:172.21.15.204:6789/0 conn(0x7fd204157300 legacy=0x7fd2040b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.387+0000 7fd20aa00640 1 -- 172.21.15.143:0/423251036 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040b2d00 con 0x7fd2040b48f0 2024-01-15T18:18:23.386 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20aa00640 1 --2- 172.21.15.143:0/423251036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd204070e80 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009e50 tx=0x7fd1fc01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1f104fd0550d3e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd208c00640 1 -- 172.21.15.143:0/423251036 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0a57d0 con 0x7fd2040b48f0 2024-01-15T18:18:23.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd208c00640 1 -- 172.21.15.143:0/423251036 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc0a5930 con 0x7fd2040b48f0 2024-01-15T18:18:23.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd208c00640 1 -- 172.21.15.143:0/423251036 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc0a5cc0 con 0x7fd2040b48f0 2024-01-15T18:18:23.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 -- 172.21.15.143:0/423251036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 msgr2=0x7fd204070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 --2- 172.21.15.143:0/423251036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd204070e80 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009e50 tx=0x7fd1fc01ca10 comp rx=0 tx=0).stop 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 -- 172.21.15.143:0/423251036 shutdown_connections 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 --2- 172.21.15.143:0/423251036 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd204070e80 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 --2- 172.21.15.143:0/423251036 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd2040b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 -- 172.21.15.143:0/423251036 >> 172.21.15.143:0/423251036 conn(0x7fd20406c3f0 msgr2=0x7fd20406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 -- 172.21.15.143:0/423251036 shutdown_connections 2024-01-15T18:18:23.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.391+0000 7fd20ca00640 1 -- 172.21.15.143:0/423251036 wait complete. 2024-01-15T18:18:23.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 Processor -- start 2024-01-15T18:18:23.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 -- start start 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd20414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd204157300 0x7fd20415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd20415b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041500f0 con 0x7fd2040b48f0 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204154da0 con 0x7fd204157300 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd204155080 con 0x7fd204159b70 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd20414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd20414b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57162/0 (socket says 172.21.15.143:57162) 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 -- 172.21.15.143:0/1440008663 learned_addr learned my addr 172.21.15.143:0/1440008663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd209600640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd204157300 0x7fd20415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20aa00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd20415b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 -- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 msgr2=0x7fd20415b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd20415b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 -- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd204157300 msgr2=0x7fd20415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd204157300 0x7fd20415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 -- 172.21.15.143:0/1440008663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009b30 con 0x7fd2040b48f0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd209600640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd204157300 0x7fd20415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20a000640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd20414b010 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c500 tx=0x7fd1f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4005220 con 0x7fd2040b48f0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4005380 con 0x7fd2040b48f0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4007270 con 0x7fd2040b48f0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20414faa0 con 0x7fd2040b48f0 2024-01-15T18:18:23.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.395+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20414b550 con 0x7fd2040b48f0 2024-01-15T18:18:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.399+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd204067500 con 0x7fd2040b48f0 2024-01-15T18:18:23.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.399+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7fd1f40073d0 con 0x7fd2040b48f0 2024-01-15T18:18:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.399+0000 7fd202000640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1dc0679d0 0x7fd1dc069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.399+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd1f4082870 con 0x7fd2040b48f0 2024-01-15T18:18:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.399+0000 7fd209600640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1dc0679d0 0x7fd1dc069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.399+0000 7fd209600640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1dc0679d0 0x7fd1dc069e90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd204156db0 tx=0x7fd1ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.403+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd1f40514e0 con 0x7fd2040b48f0 2024-01-15T18:18:23.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.583+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd20414ff50 con 0x7fd1dc0679d0 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd202000640 1 -- 172.21.15.143:0/1440008663 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd1ec004030 con 0x7fd1dc0679d0 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:23.589 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:18:23.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1dc0679d0 msgr2=0x7fd1dc069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1dc0679d0 0x7fd1dc069e90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd204156db0 tx=0x7fd1ec008040 comp rx=0 tx=0).stop 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 msgr2=0x7fd20414b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd20414b010 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c500 tx=0x7fd1f400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 shutdown_connections 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1dc0679d0 0x7fd1dc069e90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd204159b70 0x7fd20415b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd204157300 0x7fd20415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 --2- 172.21.15.143:0/1440008663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2040b48f0 0x7fd20414b010 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 >> 172.21.15.143:0/1440008663 conn(0x7fd20406c3f0 msgr2=0x7fd20406c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 shutdown_connections 2024-01-15T18:18:23.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.595+0000 7fd20ca00640 1 -- 172.21.15.143:0/1440008663 wait complete. 2024-01-15T18:18:23.610 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:18:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.755+0000 7f17ccc00640 1 Processor -- start 2024-01-15T18:18:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.755+0000 7f17ccc00640 1 -- start start 2024-01-15T18:18:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17ccc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80ce690 0x7f17c80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17ccc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f17c80b1da0 0x7f17c80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17ccc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17c80b3bf0 0x7f17c80b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17ccc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c815dba0 con 0x7f17c80b3bf0 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17ccc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c8070bf0 con 0x7f17c80b1da0 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17ccc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80b2740 con 0x7f17c80ce690 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17c80b3bf0 0x7f17c80b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17c80b3bf0 0x7f17c80b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57178/0 (socket says 172.21.15.143:57178) 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 -- 172.21.15.143:0/393484834 learned_addr learned my addr 172.21.15.143:0/393484834 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c5400640 1 --2- 172.21.15.143:0/393484834 >> v2:172.21.15.204:3300/0 conn(0x7f17c80b1da0 0x7f17c80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c5e00640 1 --2- 172.21.15.143:0/393484834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80ce690 0x7f17c80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 -- 172.21.15.143:0/393484834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80ce690 msgr2=0x7f17c80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 --2- 172.21.15.143:0/393484834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80ce690 0x7f17c80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 -- 172.21.15.143:0/393484834 >> v2:172.21.15.204:3300/0 conn(0x7f17c80b1da0 msgr2=0x7f17c80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 --2- 172.21.15.143:0/393484834 >> v2:172.21.15.204:3300/0 conn(0x7f17c80b1da0 0x7f17c80b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.759+0000 7f17c6800640 1 -- 172.21.15.143:0/393484834 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c80b4920 con 0x7f17c80b3bf0 2024-01-15T18:18:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17c6800640 1 --2- 172.21.15.143:0/393484834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80b3bf0 0x7f17c80b4080 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f17c806bee0 tx=0x7f17c001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8daf8675c6c43c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17c4a00640 1 -- 172.21.15.143:0/393484834 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00a5820 con 0x7f17c80b3bf0 2024-01-15T18:18:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17c4a00640 1 -- 172.21.15.143:0/393484834 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c00a5980 con 0x7f17c80b3bf0 2024-01-15T18:18:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17c4a00640 1 -- 172.21.15.143:0/393484834 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00a5d10 con 0x7f17c80b3bf0 2024-01-15T18:18:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 -- 172.21.15.143:0/393484834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80b3bf0 msgr2=0x7f17c80b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 --2- 172.21.15.143:0/393484834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80b3bf0 0x7f17c80b4080 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f17c806bee0 tx=0x7f17c001c9a0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 -- 172.21.15.143:0/393484834 shutdown_connections 2024-01-15T18:18:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 --2- 172.21.15.143:0/393484834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80b3bf0 0x7f17c80b4080 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 --2- 172.21.15.143:0/393484834 >> v2:172.21.15.204:3300/0 conn(0x7f17c80b1da0 0x7f17c80b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 --2- 172.21.15.143:0/393484834 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80ce690 0x7f17c80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 -- 172.21.15.143:0/393484834 >> 172.21.15.143:0/393484834 conn(0x7f17c806c3f0 msgr2=0x7f17c806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 -- 172.21.15.143:0/393484834 shutdown_connections 2024-01-15T18:18:23.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 -- 172.21.15.143:0/393484834 wait complete. 2024-01-15T18:18:23.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.763+0000 7f17ccc00640 1 Processor -- start 2024-01-15T18:18:23.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- start start 2024-01-15T18:18:23.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80b1da0 0x7f17c8159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c80b3bf0 0x7f17c8126b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 0x7f17c80acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8140a60 con 0x7f17c80ce690 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8140d40 con 0x7f17c80b3bf0 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8141020 con 0x7f17c80b1da0 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 0x7f17c80acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 0x7f17c80acf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57186/0 (socket says 172.21.15.143:57186) 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 -- 172.21.15.143:0/2710593640 learned_addr learned my addr 172.21.15.143:0/2710593640 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c5400640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c80b3bf0 0x7f17c8126b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c5e00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80b1da0 0x7f17c8159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 -- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80b1da0 msgr2=0x7f17c8159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80b1da0 0x7f17c8159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 -- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c80b3bf0 msgr2=0x7f17c8126b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c80b3bf0 0x7f17c8126b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 -- 172.21.15.143:0/2710593640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0009b30 con 0x7f17c80ce690 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17c6800640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 0x7f17c80acf60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f17c001b050 tx=0x7f17c0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00b8670 con 0x7f17c80ce690 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c00b87d0 con 0x7f17c80ce690 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00b8b80 con 0x7f17c80ce690 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c8141300 con 0x7f17c80ce690 2024-01-15T18:18:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c815a230 con 0x7f17c80ce690 2024-01-15T18:18:23.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.767+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c8067500 con 0x7f17c80ce690 2024-01-15T18:18:23.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.771+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7f17c00b8ce0 con 0x7f17c80ce690 2024-01-15T18:18:23.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.771+0000 7f17be000640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f17a00679d0 0x7f17a0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:23.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.771+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f17c011e4b0 con 0x7f17c80ce690 2024-01-15T18:18:23.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.771+0000 7f17c5e00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f17a00679d0 0x7f17a0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:23.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.771+0000 7f17c5e00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f17a00679d0 0x7f17a0069e90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17b80079c0 tx=0x7f17b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:23.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.775+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f17c00ed1f0 con 0x7f17c80ce690 2024-01-15T18:18:23.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.959+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17c815a3c0 con 0x7f17a00679d0 2024-01-15T18:18:23.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.983+0000 7f17be000640 1 -- 172.21.15.143:0/2710593640 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f17c815a3c0 con 0x7f17a00679d0 2024-01-15T18:18:23.979 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:23.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f17a00679d0 msgr2=0x7f17a0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f17a00679d0 0x7f17a0069e90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17b80079c0 tx=0x7f17b8008040 comp rx=0 tx=0).stop 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 msgr2=0x7f17c80acf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 0x7f17c80acf60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f17c001b050 tx=0x7f17c0019740 comp rx=0 tx=0).stop 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 shutdown_connections 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f17a00679d0 0x7f17a0069e90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17c80ce690 0x7f17c80acf60 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f17c80b3bf0 0x7f17c8126b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 --2- 172.21.15.143:0/2710593640 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f17c80b1da0 0x7f17c8159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 >> 172.21.15.143:0/2710593640 conn(0x7f17c806c3f0 msgr2=0x7f17c815dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 shutdown_connections 2024-01-15T18:18:23.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:23.987+0000 7f17ccc00640 1 -- 172.21.15.143:0/2710593640 wait complete. 2024-01-15T18:18:24.001 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:18:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 Processor -- start 2024-01-15T18:18:24.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 -- start start 2024-01-15T18:18:24.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f7ca80b0310 0x7f7ca80cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7ca80cf740 0x7f7ca80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca80cbff0 con 0x7f7ca806bfb0 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80d0c50 con 0x7f7ca80cf740 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7cade00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca80cb6c0 con 0x7f7ca80b0310 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca6a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7ca806bfb0 0x7f7ca806c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43436/0 (socket says 172.21.15.143:43436) 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca6a00640 1 -- 172.21.15.143:0/3128894594 learned_addr learned my addr 172.21.15.143:0/3128894594 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca7400640 1 --2- 172.21.15.143:0/3128894594 >> v2:172.21.15.143:3300/0 conn(0x7f7ca80cf740 0x7f7ca80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca6000640 1 --2- 172.21.15.143:0/3128894594 >> v2:172.21.15.204:3300/0 conn(0x7f7ca80b0310 0x7f7ca80cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca7400640 1 -- 172.21.15.143:0/3128894594 >> v2:172.21.15.204:3300/0 conn(0x7f7ca80b0310 msgr2=0x7f7ca80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca7400640 1 --2- 172.21.15.143:0/3128894594 >> v2:172.21.15.204:3300/0 conn(0x7f7ca80b0310 0x7f7ca80cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca7400640 1 -- 172.21.15.143:0/3128894594 >> v1:172.21.15.143:6789/0 conn(0x7f7ca806bfb0 legacy=0x7f7ca806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca5600640 1 -- 172.21.15.143:0/3128894594 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725366748 0 0) 0x7f7ca80cbff0 con 0x7f7ca806bfb0 2024-01-15T18:18:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca7400640 1 -- 172.21.15.143:0/3128894594 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca80cb9a0 con 0x7f7ca80cf740 2024-01-15T18:18:24.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.147+0000 7f7ca6000640 1 --2- 172.21.15.143:0/3128894594 >> v2:172.21.15.204:3300/0 conn(0x7f7ca80b0310 0x7f7ca80cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:24.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7ca7400640 1 --2- 172.21.15.143:0/3128894594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca80cf740 0x7f7ca80cfbf0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7ca806bf00 tx=0x7f7ca001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b653d27f990ec61 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7ca5600640 1 -- 172.21.15.143:0/3128894594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00a5910 con 0x7f7ca80cf740 2024-01-15T18:18:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7ca5600640 1 -- 172.21.15.143:0/3128894594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca00a5a70 con 0x7f7ca80cf740 2024-01-15T18:18:24.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7ca5600640 1 -- 172.21.15.143:0/3128894594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00a5e00 con 0x7f7ca80cf740 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 -- 172.21.15.143:0/3128894594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca80cf740 msgr2=0x7f7ca80cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 --2- 172.21.15.143:0/3128894594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca80cf740 0x7f7ca80cfbf0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f7ca806bf00 tx=0x7f7ca001c850 comp rx=0 tx=0).stop 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 -- 172.21.15.143:0/3128894594 shutdown_connections 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 --2- 172.21.15.143:0/3128894594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca80cf740 0x7f7ca80cfbf0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 --2- 172.21.15.143:0/3128894594 >> v2:172.21.15.204:3300/0 conn(0x7f7ca80b0310 0x7f7ca80cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 -- 172.21.15.143:0/3128894594 >> 172.21.15.143:0/3128894594 conn(0x7f7ca80d0ed0 msgr2=0x7f7ca80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 -- 172.21.15.143:0/3128894594 shutdown_connections 2024-01-15T18:18:24.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.151+0000 7f7cade00640 1 -- 172.21.15.143:0/3128894594 wait complete. 2024-01-15T18:18:24.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 Processor -- start 2024-01-15T18:18:24.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 -- start start 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 0x7f7ca814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca80b0310 0x7f7ca81558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ca80cf740 0x7f7ca8156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca814ee50 con 0x7f7ca806bfb0 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca80b13c0 con 0x7f7ca80b0310 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8154df0 con 0x7f7ca80cf740 2024-01-15T18:18:24.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 0x7f7ca814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 0x7f7ca814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57212/0 (socket says 172.21.15.143:57212) 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 -- 172.21.15.143:0/3273689412 learned_addr learned my addr 172.21.15.143:0/3273689412 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 -- 172.21.15.143:0/3273689412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ca80cf740 msgr2=0x7f7ca8156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ca80cf740 0x7f7ca8156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 -- 172.21.15.143:0/3273689412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca80b0310 msgr2=0x7f7ca81558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca80b0310 0x7f7ca81558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 -- 172.21.15.143:0/3273689412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0009a70 con 0x7f7ca806bfb0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7ca6a00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 0x7f7ca814e910 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c480 tx=0x7f7c9800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980074e0 con 0x7f7ca806bfb0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c98007660 con 0x7f7ca806bfb0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca81550d0 con 0x7f7ca806bfb0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98004a30 con 0x7f7ca806bfb0 2024-01-15T18:18:24.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.155+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca80b16a0 con 0x7f7ca806bfb0 2024-01-15T18:18:24.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.159+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca8067500 con 0x7f7ca806bfb0 2024-01-15T18:18:24.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.159+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7f7c98004b90 con 0x7f7ca806bfb0 2024-01-15T18:18:24.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.159+0000 7f7c9f400640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c800679d0 0x7f7c80069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.159+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7c98082860 con 0x7f7ca806bfb0 2024-01-15T18:18:24.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.159+0000 7f7ca6000640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c800679d0 0x7f7c80069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.159+0000 7f7ca6000640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c800679d0 0x7f7c80069e90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7c90006e90 tx=0x7f7c90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.163+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f7c980b7280 con 0x7f7ca806bfb0 2024-01-15T18:18:24.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.339+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ca80ce690 con 0x7f7c800679d0 2024-01-15T18:18:24.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.351+0000 7f7c9f400640 1 -- 172.21.15.143:0/3273689412 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7c9000be40 con 0x7f7c800679d0 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c800679d0 msgr2=0x7f7c80069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c800679d0 0x7f7c80069e90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7c90006e90 tx=0x7f7c90008040 comp rx=0 tx=0).stop 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 msgr2=0x7f7ca814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 0x7f7ca814e910 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c480 tx=0x7f7c9800c940 comp rx=0 tx=0).stop 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 shutdown_connections 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7c800679d0 0x7f7c80069e90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7ca80cf740 0x7f7ca8156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7ca80b0310 0x7f7ca81558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 --2- 172.21.15.143:0/3273689412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ca806bfb0 0x7f7ca814e910 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 >> 172.21.15.143:0/3273689412 conn(0x7f7ca80d0ed0 msgr2=0x7f7ca806e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 shutdown_connections 2024-01-15T18:18:24.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.355+0000 7f7cade00640 1 -- 172.21.15.143:0/3273689412 wait complete. 2024-01-15T18:18:24.354 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.359+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.354 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.359+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.354 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.359+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.355 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.359+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.355 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.359+0000 7f3cad400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.371 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.375+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.371 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.375+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.371 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.375+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.371 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.375+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.371 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:24.375+0000 7f3cac000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:24.377 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-15T18:18:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.527+0000 7f55f8200640 1 Processor -- start 2024-01-15T18:18:24.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.527+0000 7f55f8200640 1 -- start start 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f8200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55f0141660 0x7f55f00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f8200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f55f015ec80 0x7f55f0141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f8200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f015f310 con 0x7f55f0142470 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f8200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f015fbb0 con 0x7f55f0141660 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f8200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f015f130 con 0x7f55f015ec80 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55f0141660 0x7f55f00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f55f0141660 0x7f55f00ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57266/0 (socket says 172.21.15.143:57266) 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 -- 172.21.15.143:0/3887765586 learned_addr learned my addr 172.21.15.143:0/3887765586 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f4e00640 1 --2- 172.21.15.143:0/3887765586 >> v2:172.21.15.204:3300/0 conn(0x7f55f015ec80 0x7f55f0141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 -- 172.21.15.143:0/3887765586 >> v2:172.21.15.204:3300/0 conn(0x7f55f015ec80 msgr2=0x7f55f0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:24.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 --2- 172.21.15.143:0/3887765586 >> v2:172.21.15.204:3300/0 conn(0x7f55f015ec80 0x7f55f0141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 -- 172.21.15.143:0/3887765586 >> v1:172.21.15.204:6789/0 conn(0x7f55f0142470 legacy=0x7f55f0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:24.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.531+0000 7f55f5800640 1 -- 172.21.15.143:0/3887765586 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f015f5f0 con 0x7f55f0141660 2024-01-15T18:18:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f5800640 1 --2- 172.21.15.143:0/3887765586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f0141660 0x7f55f00ce560 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f55f006f940 tx=0x7f55e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4354697ea86385ba server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55efe00640 1 -- 172.21.15.143:0/3887765586 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00aa5f0 con 0x7f55f0141660 2024-01-15T18:18:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55efe00640 1 -- 172.21.15.143:0/3887765586 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00aa750 con 0x7f55f0141660 2024-01-15T18:18:24.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55efe00640 1 -- 172.21.15.143:0/3887765586 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00aaae0 con 0x7f55f0141660 2024-01-15T18:18:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 -- 172.21.15.143:0/3887765586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f0141660 msgr2=0x7f55f00ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 --2- 172.21.15.143:0/3887765586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f0141660 0x7f55f00ce560 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f55f006f940 tx=0x7f55e001c490 comp rx=0 tx=0).stop 2024-01-15T18:18:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 -- 172.21.15.143:0/3887765586 shutdown_connections 2024-01-15T18:18:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 --2- 172.21.15.143:0/3887765586 >> v2:172.21.15.204:3300/0 conn(0x7f55f015ec80 0x7f55f0141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 --2- 172.21.15.143:0/3887765586 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f0141660 0x7f55f00ce560 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 -- 172.21.15.143:0/3887765586 >> 172.21.15.143:0/3887765586 conn(0x7f55f006c3f0 msgr2=0x7f55f006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 -- 172.21.15.143:0/3887765586 shutdown_connections 2024-01-15T18:18:24.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.535+0000 7f55f8200640 1 -- 172.21.15.143:0/3887765586 wait complete. 2024-01-15T18:18:24.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 Processor -- start 2024-01-15T18:18:24.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 -- start start 2024-01-15T18:18:24.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55f0141660 0x7f55f015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f0142470 0x7f55f013db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 0x7f55f013f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015ff10 con 0x7f55f015ec80 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015e5e0 con 0x7f55f0142470 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f01f6d30 con 0x7f55f0141660 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 0x7f55f013f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f0142470 0x7f55f013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f5800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55f0141660 0x7f55f015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 0x7f55f013f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57270/0 (socket says 172.21.15.143:57270) 2024-01-15T18:18:24.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 -- 172.21.15.143:0/345805631 learned_addr learned my addr 172.21.15.143:0/345805631 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 -- 172.21.15.143:0/345805631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55f0141660 msgr2=0x7f55f015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55f0141660 0x7f55f015e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 -- 172.21.15.143:0/345805631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f0142470 msgr2=0x7f55f013db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f0142470 0x7f55f013db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 -- 172.21.15.143:0/345805631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00094d0 con 0x7f55f015ec80 2024-01-15T18:18:24.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f6200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 0x7f55f013f7f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c780 tx=0x7f55e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8004b20 con 0x7f55f015ec80 2024-01-15T18:18:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8004c80 con 0x7f55f015ec80 2024-01-15T18:18:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e800aa90 con 0x7f55f015ec80 2024-01-15T18:18:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f00cd150 con 0x7f55f015ec80 2024-01-15T18:18:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.539+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f01f66e0 con 0x7f55f015ec80 2024-01-15T18:18:24.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.543+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f01f6ea0 con 0x7f55f015ec80 2024-01-15T18:18:24.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.543+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7f55e800abf0 con 0x7f55f015ec80 2024-01-15T18:18:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.543+0000 7f55ed600640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55c80679d0 0x7f55c8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.543+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f55e8083360 con 0x7f55f015ec80 2024-01-15T18:18:24.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.543+0000 7f55f5800640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55c80679d0 0x7f55c8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.543+0000 7f55f5800640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55c80679d0 0x7f55c8069e90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f55e00ae000 tx=0x7f55e00b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.547+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f55e8051cd0 con 0x7f55f015ec80 2024-01-15T18:18:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.727+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f00ccfd0 con 0x7f55c80679d0 2024-01-15T18:18:24.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.743+0000 7f55ed600640 1 -- 172.21.15.143:0/345805631 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f55e00aa8c0 con 0x7f55c80679d0 2024-01-15T18:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.743+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55c80679d0 msgr2=0x7f55c8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.743+0000 7f55f8200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55c80679d0 0x7f55c8069e90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f55e00ae000 tx=0x7f55e00b5040 comp rx=0 tx=0).stop 2024-01-15T18:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.743+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 msgr2=0x7f55f013f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.743+0000 7f55f8200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 0x7f55f013f7f0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c780 tx=0x7f55e800cc40 comp rx=0 tx=0).stop 2024-01-15T18:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 shutdown_connections 2024-01-15T18:18:24.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f55c80679d0 0x7f55c8069e90 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f55f015ec80 0x7f55f013f7f0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55f0142470 0x7f55f013db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 --2- 172.21.15.143:0/345805631 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f55f0141660 0x7f55f015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 >> 172.21.15.143:0/345805631 conn(0x7f55f006c3f0 msgr2=0x7f55f00af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 shutdown_connections 2024-01-15T18:18:24.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.747+0000 7f55f8200640 1 -- 172.21.15.143:0/345805631 wait complete. 2024-01-15T18:18:24.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:24.765 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:18:24.782 INFO:teuthology.orchestra.run.smithi143.stdout:2 2024-01-15T18:18:24.783 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:18:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.975+0000 7fbc1be00640 1 Processor -- start 2024-01-15T18:18:24.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.975+0000 7fbc1be00640 1 -- start start 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.975+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1406c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.975+0000 7fbc1be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc140d0210 0x7fbc140ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.975+0000 7fbc1be00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc140cbff0 con 0x7fbc140b0310 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.975+0000 7fbc1be00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc140d0c50 con 0x7fbc140d0210 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc1be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1406c8d0 con 0x7fbc1406bfb0 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc140d0210 0x7fbc140ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbc140d0210 0x7fbc140ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57274/0 (socket says 172.21.15.143:57274) 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 -- 172.21.15.143:0/575540529 learned_addr learned my addr 172.21.15.143:0/575540529 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19400640 1 --2- 172.21.15.143:0/575540529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 -- 172.21.15.143:0/575540529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 msgr2=0x7fbc1406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 --2- 172.21.15.143:0/575540529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 -- 172.21.15.143:0/575540529 >> v1:172.21.15.204:6789/0 conn(0x7fbc140b0310 legacy=0x7fbc140cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.973 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 -- 172.21.15.143:0/575540529 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140cb6c0 con 0x7fbc140d0210 2024-01-15T18:18:24.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc19e00640 1 --2- 172.21.15.143:0/575540529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140ccb90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009e50 tx=0x7fbc0c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e73164ef1afa331 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.979+0000 7fbc13e00640 1 -- 172.21.15.143:0/575540529 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0a5810 con 0x7fbc140d0210 2024-01-15T18:18:24.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc13e00640 1 -- 172.21.15.143:0/575540529 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c0a5970 con 0x7fbc140d0210 2024-01-15T18:18:24.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc13e00640 1 -- 172.21.15.143:0/575540529 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0a5d00 con 0x7fbc140d0210 2024-01-15T18:18:24.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- 172.21.15.143:0/575540529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 msgr2=0x7fbc140ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 --2- 172.21.15.143:0/575540529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140ccb90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009e50 tx=0x7fbc0c01c950 comp rx=0 tx=0).stop 2024-01-15T18:18:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- 172.21.15.143:0/575540529 shutdown_connections 2024-01-15T18:18:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 --2- 172.21.15.143:0/575540529 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140ccb90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 --2- 172.21.15.143:0/575540529 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- 172.21.15.143:0/575540529 >> 172.21.15.143:0/575540529 conn(0x7fbc140d0ed0 msgr2=0x7fbc140d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:24.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- 172.21.15.143:0/575540529 shutdown_connections 2024-01-15T18:18:24.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- 172.21.15.143:0/575540529 wait complete. 2024-01-15T18:18:24.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 Processor -- start 2024-01-15T18:18:24.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- start start 2024-01-15T18:18:24.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc140b0310 0x7fbc1414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc140aec50 con 0x7fbc140d0210 2024-01-15T18:18:24.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc14154a70 con 0x7fbc140b0310 2024-01-15T18:18:24.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.983+0000 7fbc1be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1415e2d0 con 0x7fbc1406bfb0 2024-01-15T18:18:24.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57280/0 (socket says 172.21.15.143:57280) 2024-01-15T18:18:24.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 -- 172.21.15.143:0/585984624 learned_addr learned my addr 172.21.15.143:0/585984624 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:24.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc18a00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc140b0310 0x7fbc1414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19400640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 -- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 msgr2=0x7fbc1415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1415c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 -- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc140b0310 msgr2=0x7fbc1414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc140b0310 0x7fbc1414c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 -- 172.21.15.143:0/585984624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c009b30 con 0x7fbc140d0210 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc19e00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140aef80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c01b000 tx=0x7fbc0c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0b8640 con 0x7fbc140d0210 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c0b87c0 con 0x7fbc140d0210 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c0b8b70 con 0x7fbc140d0210 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc140ae0d0 con 0x7fbc140d0210 2024-01-15T18:18:24.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc140ae690 con 0x7fbc140d0210 2024-01-15T18:18:24.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc14067500 con 0x7fbc140d0210 2024-01-15T18:18:24.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.987+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 36) v1 ==== 87690+0+0 (secure 0 0 0) 0x7fbc0c0b8cd0 con 0x7fbc140d0210 2024-01-15T18:18:24.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.991+0000 7fbc11600640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbbec0679d0 0x7fbbec069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:24.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.991+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbc0c11e000 con 0x7fbc140d0210 2024-01-15T18:18:24.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.991+0000 7fbc19400640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbbec0679d0 0x7fbbec069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:24.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.991+0000 7fbc19400640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbbec0679d0 0x7fbbec069e90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbc040079c0 tx=0x7fbc04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:24.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:24.995+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbc0c0ecec0 con 0x7fbc140d0210 2024-01-15T18:18:25.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:25.167+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc1406ff20 con 0x7fbbec0679d0 2024-01-15T18:18:25.600 INFO:tasks.ceph.mon.a.smithi143.stderr:2024-01-15T18:18:25.607+0000 7f257e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:18:25.673 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:18:25.828 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:25.831+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 37) v1 ==== 87792+0+0 (secure 0 0 0) 0x7fbc0c0ed290 con 0x7fbc140d0210 2024-01-15T18:18:27.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:27.847+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fbc0c11aa90 con 0x7fbc140d0210 2024-01-15T18:18:28.617 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.623+0000 7fbc11600640 1 -- 172.21.15.143:0/585984624 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fbc1406ff20 con 0x7fbbec0679d0 2024-01-15T18:18:28.618 INFO:teuthology.orchestra.run.smithi143.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbbec0679d0 msgr2=0x7fbbec069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbbec0679d0 0x7fbbec069e90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbc040079c0 tx=0x7fbc04008040 comp rx=0 tx=0).stop 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 msgr2=0x7fbc140aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140aef80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c01b000 tx=0x7fbc0c0196f0 comp rx=0 tx=0).stop 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 shutdown_connections 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbbec0679d0 0x7fbbec069e90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbc140d0210 0x7fbc140aef80 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:28.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbc140b0310 0x7fbc1414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 --2- 172.21.15.143:0/585984624 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbc1406bfb0 0x7fbc1415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 >> 172.21.15.143:0/585984624 conn(0x7fbc140d0ed0 msgr2=0x7fbc1406f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 shutdown_connections 2024-01-15T18:18:28.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:28.627+0000 7fbc1be00640 1 -- 172.21.15.143:0/585984624 wait complete. 2024-01-15T18:18:28.660 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:18:28.661 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:18:28.762 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:18:28.817 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:18:28.817 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:18:28.817 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:28.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:29.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.007+0000 7fad14800640 1 Processor -- start 2024-01-15T18:18:29.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.007+0000 7fad14800640 1 -- start start 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad14800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad0c071bb0 0x7fad0c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad14800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c15c5a0 0x7fad0c15ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad14800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0ceb50 con 0x7fad0c143780 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad14800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c072570 con 0x7fad0c071bb0 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad14800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c15c110 con 0x7fad0c15c5a0 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad0c071bb0 0x7fad0c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fad0c071bb0 0x7fad0c072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48262/0 (socket says 172.21.15.143:48262) 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 -- 172.21.15.143:0/4051414522 learned_addr learned my addr 172.21.15.143:0/4051414522 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:29.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad12800640 1 --2- 172.21.15.143:0/4051414522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c15c5a0 0x7fad0c15ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 -- 172.21.15.143:0/4051414522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c15c5a0 msgr2=0x7fad0c15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 --2- 172.21.15.143:0/4051414522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c15c5a0 0x7fad0c15ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 -- 172.21.15.143:0/4051414522 >> v1:172.21.15.204:6789/0 conn(0x7fad0c143780 legacy=0x7fad0c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.011+0000 7fad11400640 1 -- 172.21.15.143:0/4051414522 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c15b7e0 con 0x7fad0c071bb0 2024-01-15T18:18:29.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad11400640 1 --2- 172.21.15.143:0/4051414522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c071bb0 0x7fad0c072030 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7facf4009870 tx=0x7facf401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5477cdf5b93f2c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad10a00640 1 -- 172.21.15.143:0/4051414522 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40a9400 con 0x7fad0c071bb0 2024-01-15T18:18:29.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad10a00640 1 -- 172.21.15.143:0/4051414522 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf40a9560 con 0x7fad0c071bb0 2024-01-15T18:18:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad10a00640 1 -- 172.21.15.143:0/4051414522 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf40a98f0 con 0x7fad0c071bb0 2024-01-15T18:18:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 -- 172.21.15.143:0/4051414522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c071bb0 msgr2=0x7fad0c072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 --2- 172.21.15.143:0/4051414522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c071bb0 0x7fad0c072030 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7facf4009870 tx=0x7facf401c540 comp rx=0 tx=0).stop 2024-01-15T18:18:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 -- 172.21.15.143:0/4051414522 shutdown_connections 2024-01-15T18:18:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 --2- 172.21.15.143:0/4051414522 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c15c5a0 0x7fad0c15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 --2- 172.21.15.143:0/4051414522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c071bb0 0x7fad0c072030 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 -- 172.21.15.143:0/4051414522 >> 172.21.15.143:0/4051414522 conn(0x7fad0c06c3f0 msgr2=0x7fad0c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:29.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 -- 172.21.15.143:0/4051414522 shutdown_connections 2024-01-15T18:18:29.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.015+0000 7fad14800640 1 -- 172.21.15.143:0/4051414522 wait complete. 2024-01-15T18:18:29.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 Processor -- start 2024-01-15T18:18:29.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 -- start start 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c071bb0 0x7fad0c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c143780 0x7fad0c06cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 0x7fad0c06d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c1412e0 con 0x7fad0c15c5a0 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c159450 con 0x7fad0c071bb0 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c140940 con 0x7fad0c143780 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 0x7fad0c06d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 0x7fad0c06d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48272/0 (socket says 172.21.15.143:48272) 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 -- 172.21.15.143:0/470353132 learned_addr learned my addr 172.21.15.143:0/470353132 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad11400640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c143780 0x7fad0c06cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad11e00640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c071bb0 0x7fad0c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 -- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c143780 msgr2=0x7fad0c06cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c143780 0x7fad0c06cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 -- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c071bb0 msgr2=0x7fad0c158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c071bb0 0x7fad0c158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 -- 172.21.15.143:0/470353132 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf40094d0 con 0x7fad0c15c5a0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad11400640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c143780 0x7fad0c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad12800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 0x7fad0c06d220 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fad0400ec10 tx=0x7fad0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0400cd20 con 0x7fad0c15c5a0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0400f040 con 0x7fad0c15c5a0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040146c0 con 0x7fad0c15c5a0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c140c20 con 0x7fad0c15c5a0 2024-01-15T18:18:29.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.019+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c141700 con 0x7fad0c15c5a0 2024-01-15T18:18:29.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.023+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c141420 con 0x7fad0c15c5a0 2024-01-15T18:18:29.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.023+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fad04014a30 con 0x7fad0c15c5a0 2024-01-15T18:18:29.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.023+0000 7fad0a000640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7face4067a60 0x7face4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.023+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fad04083e60 con 0x7fad0c15c5a0 2024-01-15T18:18:29.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.023+0000 7fad11e00640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7face4067a60 0x7face4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.023+0000 7fad11e00640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7face4067a60 0x7face4069f20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.027+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fad04052850 con 0x7fad0c15c5a0 2024-01-15T18:18:29.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.287+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad0c067500 con 0x7fad0c15c5a0 2024-01-15T18:18:29.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.287+0000 7fad0a000640 1 -- 172.21.15.143:0/470353132 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13114 (secure 0 0 0) 0x7fad04018070 con 0x7fad0c15c5a0 2024-01-15T18:18:29.283 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:29.283 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:28.617118+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4145140990":"2024-01-16T18:18:25.607861+0000","172.21.15.143:6834/4145140990":"2024-01-16T18:18:25.607861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7face4067a60 msgr2=0x7face4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7face4067a60 0x7face4069f20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc009040 comp rx=0 tx=0).stop 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 msgr2=0x7fad0c06d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 0x7fad0c06d220 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fad0400ec10 tx=0x7fad0400c490 comp rx=0 tx=0).stop 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 shutdown_connections 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7face4067a60 0x7face4069f20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fad0c15c5a0 0x7fad0c06d220 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fad0c143780 0x7fad0c06cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 --2- 172.21.15.143:0/470353132 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad0c071bb0 0x7fad0c158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 >> 172.21.15.143:0/470353132 conn(0x7fad0c06c3f0 msgr2=0x7fad0c0aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 shutdown_connections 2024-01-15T18:18:29.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.291+0000 7fad14800640 1 -- 172.21.15.143:0/470353132 wait complete. 2024-01-15T18:18:29.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:18:29.306 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:18:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 Processor -- start 2024-01-15T18:18:29.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 -- start start 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff364143780 0x7ff3640cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff3640722c0 0x7ff36415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3640ceb50 con 0x7ff3640ce520 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff36415c110 con 0x7ff364143780 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36c200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff36415b7e0 con 0x7ff3640722c0 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff364143780 0x7ff3640cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff364143780 0x7ff3640cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48284/0 (socket says 172.21.15.143:48284) 2024-01-15T18:18:29.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff368e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff3640ce520 0x7ff364071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49178/0 (socket says 172.21.15.143:49178) 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 -- 172.21.15.143:0/3592619739 learned_addr learned my addr 172.21.15.143:0/3592619739 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36a200640 1 --2- 172.21.15.143:0/3592619739 >> v2:172.21.15.204:3300/0 conn(0x7ff3640722c0 0x7ff36415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 -- 172.21.15.143:0/3592619739 >> v2:172.21.15.204:3300/0 conn(0x7ff3640722c0 msgr2=0x7ff36415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 --2- 172.21.15.143:0/3592619739 >> v2:172.21.15.204:3300/0 conn(0x7ff3640722c0 0x7ff36415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 -- 172.21.15.143:0/3592619739 >> v1:172.21.15.143:6789/0 conn(0x7ff3640ce520 legacy=0x7ff364071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff369800640 1 -- 172.21.15.143:0/3592619739 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36415bac0 con 0x7ff364143780 2024-01-15T18:18:29.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.451+0000 7ff36a200640 1 --2- 172.21.15.143:0/3592619739 >> v2:172.21.15.204:3300/0 conn(0x7ff3640722c0 0x7ff36415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:29.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff369800640 1 --2- 172.21.15.143:0/3592619739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff3640cdfe0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7ff364070a20 tx=0x7ff35401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582b341318242787 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff363e00640 1 -- 172.21.15.143:0/3592619739 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3540ab610 con 0x7ff364143780 2024-01-15T18:18:29.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff363e00640 1 -- 172.21.15.143:0/3592619739 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3540ab770 con 0x7ff364143780 2024-01-15T18:18:29.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff363e00640 1 -- 172.21.15.143:0/3592619739 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3540abb00 con 0x7ff364143780 2024-01-15T18:18:29.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 -- 172.21.15.143:0/3592619739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 msgr2=0x7ff3640cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 --2- 172.21.15.143:0/3592619739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff3640cdfe0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7ff364070a20 tx=0x7ff35401c500 comp rx=0 tx=0).stop 2024-01-15T18:18:29.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 -- 172.21.15.143:0/3592619739 shutdown_connections 2024-01-15T18:18:29.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 --2- 172.21.15.143:0/3592619739 >> v2:172.21.15.204:3300/0 conn(0x7ff3640722c0 0x7ff36415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 --2- 172.21.15.143:0/3592619739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff3640cdfe0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 -- 172.21.15.143:0/3592619739 >> 172.21.15.143:0/3592619739 conn(0x7ff36406c3f0 msgr2=0x7ff36406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:29.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 -- 172.21.15.143:0/3592619739 shutdown_connections 2024-01-15T18:18:29.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.455+0000 7ff36c200640 1 -- 172.21.15.143:0/3592619739 wait complete. 2024-01-15T18:18:29.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 Processor -- start 2024-01-15T18:18:29.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 -- start start 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3640722c0 0x7ff3640b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3640ce520 0x7ff364160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff364159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364142cb0 con 0x7ff364143780 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff364142f90 con 0x7ff3640722c0 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3641407d0 con 0x7ff3640ce520 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff364159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff364159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48294/0 (socket says 172.21.15.143:48294) 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 -- 172.21.15.143:0/310344070 learned_addr learned my addr 172.21.15.143:0/310344070 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff368e00640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3640ce520 0x7ff364160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff369800640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3640722c0 0x7ff3640b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 -- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3640ce520 msgr2=0x7ff364160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3640ce520 0x7ff364160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 -- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3640722c0 msgr2=0x7ff3640b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3640722c0 0x7ff3640b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 -- 172.21.15.143:0/310344070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540094d0 con 0x7ff364143780 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff369800640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3640722c0 0x7ff3640b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff368e00640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3640ce520 0x7ff364160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36a200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff364159200 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007de0 tx=0x7ff35c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0039a0 con 0x7ff364143780 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c00f040 con 0x7ff364143780 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c003d40 con 0x7ff364143780 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff364140ab0 con 0x7ff364143780 2024-01-15T18:18:29.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.459+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3640b35d0 con 0x7ff364143780 2024-01-15T18:18:29.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.463+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff364067500 con 0x7ff364143780 2024-01-15T18:18:29.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.463+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7ff35c004030 con 0x7ff364143780 2024-01-15T18:18:29.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.463+0000 7ff361600640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff33c067a90 0x7ff33c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.463+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7ff35c0836f0 con 0x7ff364143780 2024-01-15T18:18:29.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.463+0000 7ff369800640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff33c067a90 0x7ff33c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.463+0000 7ff369800640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff33c067a90 0x7ff33c069f50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff3540af000 tx=0x7ff3540a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.467+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff35c08b070 con 0x7ff364143780 2024-01-15T18:18:29.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff364140f60 con 0x7ff364143780 2024-01-15T18:18:29.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff361600640 1 -- 172.21.15.143:0/310344070 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff35c01a0a0 con 0x7ff364143780 2024-01-15T18:18:29.613 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:18:29.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff33c067a90 msgr2=0x7ff33c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff33c067a90 0x7ff33c069f50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff3540af000 tx=0x7ff3540a7000 comp rx=0 tx=0).stop 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 msgr2=0x7ff364159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff364159200 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7ff35c007de0 tx=0x7ff35c00d4a0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 shutdown_connections 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff33c067a90 0x7ff33c069f50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff364143780 0x7ff364159200 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff3640ce520 0x7ff364160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 --2- 172.21.15.143:0/310344070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff3640722c0 0x7ff3640b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 >> 172.21.15.143:0/310344070 conn(0x7ff36406c3f0 msgr2=0x7ff36406c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 shutdown_connections 2024-01-15T18:18:29.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.619+0000 7ff36c200640 1 -- 172.21.15.143:0/310344070 wait complete. 2024-01-15T18:18:29.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:18:29.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 Processor -- start 2024-01-15T18:18:29.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 -- start start 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c15d130 0x7f2d0c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d0c06aba0 0x7f2d0c06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c160eb0 con 0x7f2d0c15d510 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c161190 con 0x7f2d0c06aba0 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d13400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c161470 con 0x7f2d0c15d130 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d0c06aba0 0x7f2d0c06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2d0c06aba0 0x7f2d0c06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48308/0 (socket says 172.21.15.143:48308) 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 -- 172.21.15.143:0/4146004889 learned_addr learned my addr 172.21.15.143:0/4146004889 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d10a00640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c15d130 0x7f2d0c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 -- 172.21.15.143:0/4146004889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c15d130 msgr2=0x7f2d0c0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c15d130 0x7f2d0c0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 -- 172.21.15.143:0/4146004889 >> v1:172.21.15.143:6789/0 conn(0x7f2d0c15d510 legacy=0x7f2d0c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d11400640 1 -- 172.21.15.143:0/4146004889 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c161750 con 0x7f2d0c06aba0 2024-01-15T18:18:29.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.779+0000 7f2d10a00640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c15d130 0x7f2d0c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:29.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d11400640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c06aba0 0x7f2d0c06e060 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a46804a4c4196a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d0b400640 1 -- 172.21.15.143:0/4146004889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040a5900 con 0x7f2d0c06aba0 2024-01-15T18:18:29.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d0b400640 1 -- 172.21.15.143:0/4146004889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040a5a60 con 0x7f2d0c06aba0 2024-01-15T18:18:29.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d0b400640 1 -- 172.21.15.143:0/4146004889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040a5df0 con 0x7f2d0c06aba0 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 -- 172.21.15.143:0/4146004889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c06aba0 msgr2=0x7f2d0c06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c06aba0 0x7f2d0c06e060 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0401ca10 comp rx=0 tx=0).stop 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 -- 172.21.15.143:0/4146004889 shutdown_connections 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c06aba0 0x7f2d0c06e060 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 --2- 172.21.15.143:0/4146004889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c15d130 0x7f2d0c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 -- 172.21.15.143:0/4146004889 >> 172.21.15.143:0/4146004889 conn(0x7f2d0c0b1b90 msgr2=0x7f2d0c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:29.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 -- 172.21.15.143:0/4146004889 shutdown_connections 2024-01-15T18:18:29.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.783+0000 7f2d13400640 1 -- 172.21.15.143:0/4146004889 wait complete. 2024-01-15T18:18:29.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 Processor -- start 2024-01-15T18:18:29.781 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 -- start start 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c06aba0 0x7f2d0c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c15d130 0x7f2d0c0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 0x7f2d0c155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c0accf0 con 0x7f2d0c15d510 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c151b50 con 0x7f2d0c15d130 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c151e30 con 0x7f2d0c06aba0 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 0x7f2d0c155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 0x7f2d0c155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48320/0 (socket says 172.21.15.143:48320) 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 -- 172.21.15.143:0/899741425 learned_addr learned my addr 172.21.15.143:0/899741425 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:29.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d10a00640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c06aba0 0x7f2d0c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d0be00640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c15d130 0x7f2d0c0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 -- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c06aba0 msgr2=0x7f2d0c1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c06aba0 0x7f2d0c1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 -- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c15d130 msgr2=0x7f2d0c0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c15d130 0x7f2d0c0ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 -- 172.21.15.143:0/899741425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04009b30 con 0x7f2d0c15d510 2024-01-15T18:18:29.784 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d11400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 0x7f2d0c155830 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2d040af040 tx=0x7f2d040197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040b8880 con 0x7f2d0c15d510 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040b8a00 con 0x7f2d0c15d510 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040b8db0 con 0x7f2d0c15d510 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c152110 con 0x7f2d0c15d510 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.787+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c151720 con 0x7f2d0c15d510 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.791+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c151910 con 0x7f2d0c15d510 2024-01-15T18:18:29.785 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.791+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f2d040c3460 con 0x7f2d0c15d510 2024-01-15T18:18:29.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.791+0000 7f2d08c00640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ce4067a60 0x7f2ce4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:29.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.791+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f2d0411e9b0 con 0x7f2d0c15d510 2024-01-15T18:18:29.787 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.791+0000 7f2d10a00640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ce4067a60 0x7f2ce4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:29.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.791+0000 7f2d10a00640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ce4067a60 0x7f2ce4069f20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009950 tx=0x7f2cfc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:29.791 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.795+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2d0c151910 con 0x7f2d0c15d510 2024-01-15T18:18:29.863 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:29.867 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:29.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.959+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2d0c0acea0 con 0x7f2d0c15d510 2024-01-15T18:18:29.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.959+0000 7f2d08c00640 1 -- 172.21.15.143:0/899741425 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v103) v1 ==== 80+0+124 (secure 0 0 0) 0x7f2d0c0acea0 con 0x7f2d0c15d510 2024-01-15T18:18:29.955 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/4145140990 2024-01-16T18:18:25.607861+0000 2024-01-15T18:18:29.955 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6834/4145140990 2024-01-16T18:18:25.607861+0000 2024-01-15T18:18:29.956 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ce4067a60 msgr2=0x7f2ce4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ce4067a60 0x7f2ce4069f20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009950 tx=0x7f2cfc008040 comp rx=0 tx=0).stop 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 msgr2=0x7f2d0c155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 0x7f2d0c155830 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2d040af040 tx=0x7f2d040197b0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 shutdown_connections 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2ce4067a60 0x7f2ce4069f20 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d0c15d510 0x7f2d0c155830 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2d0c15d130 0x7f2d0c0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 --2- 172.21.15.143:0/899741425 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2d0c06aba0 0x7f2d0c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 >> 172.21.15.143:0/899741425 conn(0x7f2d0c0b1b90 msgr2=0x7f2d0c150090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:29.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 shutdown_connections 2024-01-15T18:18:29.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:29.963+0000 7f2d13400640 1 -- 172.21.15.143:0/899741425 wait complete. 2024-01-15T18:18:29.978 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:18:29.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:29.978 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:30.039 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:18:30.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:30.039 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:18:30.043 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[51204]: fuse finished with error 0 and tester_r 0 2024-01-15T18:18:30.179 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:18:30.179 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:30.188 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:30.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.367+0000 7f355da00640 1 Processor -- start 2024-01-15T18:18:30.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.367+0000 7f355da00640 1 -- start start 2024-01-15T18:18:30.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f355da00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f35581607c0 0x7f355806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f355da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35580cfb30 0x7f355806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f355da00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580cffd0 con 0x7f355806c230 2024-01-15T18:18:30.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f355da00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355806c670 con 0x7f35581607c0 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f355da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35580d1ce0 con 0x7f35580cfb30 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3557400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f355806c230 0x7f35580cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49194/0 (socket says 172.21.15.143:49194) 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3557400640 1 -- 172.21.15.143:0/326599411 learned_addr learned my addr 172.21.15.143:0/326599411 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3556a00640 1 --2- 172.21.15.143:0/326599411 >> v2:172.21.15.204:3300/0 conn(0x7f35581607c0 0x7f355806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3556000640 1 --2- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35580cfb30 0x7f355806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3555600640 1 -- 172.21.15.143:0/326599411 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481358197 0 0) 0x7f35580cffd0 con 0x7f355806c230 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3555600640 1 -- 172.21.15.143:0/326599411 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3538003530 con 0x7f355806c230 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3556a00640 1 -- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35580cfb30 msgr2=0x7f355806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3556a00640 1 --2- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35580cfb30 0x7f355806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3556a00640 1 -- 172.21.15.143:0/326599411 >> v1:172.21.15.143:6789/0 conn(0x7f355806c230 legacy=0x7f35580cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.371+0000 7f3556a00640 1 -- 172.21.15.143:0/326599411 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580d02b0 con 0x7f35581607c0 2024-01-15T18:18:30.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f3556a00640 1 --2- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35581607c0 0x7f355806b7b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f35480097c0 tx=0x7f354801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e782a43ac5f1fb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:30.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f3555600640 1 -- 172.21.15.143:0/326599411 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35480a8970 con 0x7f35581607c0 2024-01-15T18:18:30.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f3555600640 1 -- 172.21.15.143:0/326599411 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480a8ad0 con 0x7f35581607c0 2024-01-15T18:18:30.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f3555600640 1 -- 172.21.15.143:0/326599411 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35480a8e60 con 0x7f35581607c0 2024-01-15T18:18:30.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 -- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35581607c0 msgr2=0x7f355806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 --2- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35581607c0 0x7f355806b7b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f35480097c0 tx=0x7f354801ec70 comp rx=0 tx=0).stop 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 -- 172.21.15.143:0/326599411 shutdown_connections 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 --2- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35580cfb30 0x7f355806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 --2- 172.21.15.143:0/326599411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35581607c0 0x7f355806b7b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 -- 172.21.15.143:0/326599411 >> 172.21.15.143:0/326599411 conn(0x7f35580d0e20 msgr2=0x7f35580d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 -- 172.21.15.143:0/326599411 shutdown_connections 2024-01-15T18:18:30.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 -- 172.21.15.143:0/326599411 wait complete. 2024-01-15T18:18:30.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 Processor -- start 2024-01-15T18:18:30.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.375+0000 7f355da00640 1 -- start start 2024-01-15T18:18:30.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f355806c230 0x7f35581f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 0x7f35580c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35581607c0 0x7f355806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558143bc0 con 0x7f35580cfb30 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c6b50 con 0x7f355806c230 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35581f2870 con 0x7f35581607c0 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 0x7f35580c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 0x7f35580c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48334/0 (socket says 172.21.15.143:48334) 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 -- 172.21.15.143:0/905717895 learned_addr learned my addr 172.21.15.143:0/905717895 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556a00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f355806c230 0x7f35581f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3557400640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35581607c0 0x7f355806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 -- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35581607c0 msgr2=0x7f355806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35581607c0 0x7f355806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 -- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f355806c230 msgr2=0x7f35581f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f355806c230 0x7f35581f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 -- 172.21.15.143:0/905717895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009420 con 0x7f35580cfb30 2024-01-15T18:18:30.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f3556000640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 0x7f35580c58d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f354000e930 tx=0x7f354000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:30.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540009690 con 0x7f35580cfb30 2024-01-15T18:18:30.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354000ca00 con 0x7f35580cfb30 2024-01-15T18:18:30.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35580c62b0 con 0x7f35580cfb30 2024-01-15T18:18:30.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000cd70 con 0x7f35580cfb30 2024-01-15T18:18:30.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35580c6870 con 0x7f35580cfb30 2024-01-15T18:18:30.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355806d4b0 con 0x7f35580cfb30 2024-01-15T18:18:30.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.379+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f3540013470 con 0x7f35580cfb30 2024-01-15T18:18:30.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.383+0000 7f354ea00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3530067a60 0x7f3530069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.383+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f35400838e0 con 0x7f35580cfb30 2024-01-15T18:18:30.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.383+0000 7f3556a00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3530067a60 0x7f3530069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.383+0000 7f3556a00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3530067a60 0x7f3530069f20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f35480b2000 tx=0x7f3548008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:30.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.387+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3540052390 con 0x7f35580cfb30 2024-01-15T18:18:30.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.539+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3558067500 con 0x7f35580cfb30 2024-01-15T18:18:30.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.539+0000 7f354ea00640 1 -- 172.21.15.143:0/905717895 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13114 (secure 0 0 0) 0x7f3540017090 con 0x7f35580cfb30 2024-01-15T18:18:30.536 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:30.536 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:28.617118+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4145140990":"2024-01-16T18:18:25.607861+0000","172.21.15.143:6834/4145140990":"2024-01-16T18:18:25.607861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3530067a60 msgr2=0x7f3530069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3530067a60 0x7f3530069f20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f35480b2000 tx=0x7f3548008040 comp rx=0 tx=0).stop 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 msgr2=0x7f35580c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 0x7f35580c58d0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f354000e930 tx=0x7f354000edf0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 shutdown_connections 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3530067a60 0x7f3530069f20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35581607c0 0x7f355806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35580cfb30 0x7f35580c58d0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 --2- 172.21.15.143:0/905717895 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f355806c230 0x7f35581f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 >> 172.21.15.143:0/905717895 conn(0x7f35580d0e20 msgr2=0x7f35580cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 shutdown_connections 2024-01-15T18:18:30.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.543+0000 7f355da00640 1 -- 172.21.15.143:0/905717895 wait complete. 2024-01-15T18:18:30.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:18:30.558 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:18:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 Processor -- start 2024-01-15T18:18:30.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 -- start start 2024-01-15T18:18:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcd540cfb30 0x7fcd5406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd540cbf50 0x7fcd540cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd540cffd0 con 0x7fcd541607c0 2024-01-15T18:18:30.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd540d1ce0 con 0x7fcd540cbf50 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd5b400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd540d02b0 con 0x7fcd540cfb30 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd540cbf50 0x7fcd540cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd540cbf50 0x7fcd540cc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48338/0 (socket says 172.21.15.143:48338) 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 -- 172.21.15.143:0/1305068011 learned_addr learned my addr 172.21.15.143:0/1305068011 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd53e00640 1 --2- 172.21.15.143:0/1305068011 >> v2:172.21.15.204:3300/0 conn(0x7fcd540cfb30 0x7fcd5406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 -- 172.21.15.143:0/1305068011 >> v2:172.21.15.204:3300/0 conn(0x7fcd540cfb30 msgr2=0x7fcd5406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 --2- 172.21.15.143:0/1305068011 >> v2:172.21.15.204:3300/0 conn(0x7fcd540cfb30 0x7fcd5406bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 -- 172.21.15.143:0/1305068011 >> v1:172.21.15.143:6789/0 conn(0x7fcd541607c0 legacy=0x7fcd5406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.703+0000 7fcd59400640 1 -- 172.21.15.143:0/1305068011 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540cb470 con 0x7fcd540cbf50 2024-01-15T18:18:30.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd59400640 1 --2- 172.21.15.143:0/1305068011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd540cc3c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fcd5409a980 tx=0x7fcd4c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dbfd2a5a24a74e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:30.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd53400640 1 -- 172.21.15.143:0/1305068011 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0af400 con 0x7fcd540cbf50 2024-01-15T18:18:30.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd53400640 1 -- 172.21.15.143:0/1305068011 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c0ab050 con 0x7fcd540cbf50 2024-01-15T18:18:30.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd53400640 1 -- 172.21.15.143:0/1305068011 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0af780 con 0x7fcd540cbf50 2024-01-15T18:18:30.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 -- 172.21.15.143:0/1305068011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 msgr2=0x7fcd540cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 --2- 172.21.15.143:0/1305068011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd540cc3c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fcd5409a980 tx=0x7fcd4c01f1c0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 -- 172.21.15.143:0/1305068011 shutdown_connections 2024-01-15T18:18:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 --2- 172.21.15.143:0/1305068011 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd540cc3c0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 --2- 172.21.15.143:0/1305068011 >> v2:172.21.15.204:3300/0 conn(0x7fcd540cfb30 0x7fcd5406bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 -- 172.21.15.143:0/1305068011 >> 172.21.15.143:0/1305068011 conn(0x7fcd540d0e20 msgr2=0x7fcd540d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 -- 172.21.15.143:0/1305068011 shutdown_connections 2024-01-15T18:18:30.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.707+0000 7fcd5b400640 1 -- 172.21.15.143:0/1305068011 wait complete. 2024-01-15T18:18:30.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 Processor -- start 2024-01-15T18:18:30.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 -- start start 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd541584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd540cfb30 0x7fcd54157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd541607c0 0x7fcd541f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54157ad0 con 0x7fcd540cbf50 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5406e420 con 0x7fcd541607c0 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd54158a20 con 0x7fcd540cfb30 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd541584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd541584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48344/0 (socket says 172.21.15.143:48344) 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 -- 172.21.15.143:0/2919743811 learned_addr learned my addr 172.21.15.143:0/2919743811 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd59400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd541607c0 0x7fcd541f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd53e00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd540cfb30 0x7fcd54157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 -- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd540cfb30 msgr2=0x7fcd54157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd540cfb30 0x7fcd54157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 -- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd541607c0 msgr2=0x7fcd541f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd541607c0 0x7fcd541f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 -- 172.21.15.143:0/2919743811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009b30 con 0x7fcd540cbf50 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd58a00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd541584e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400c3c0 tx=0x7fcd4400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440052d0 con 0x7fcd540cbf50 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44005430 con 0x7fcd540cbf50 2024-01-15T18:18:30.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44003840 con 0x7fcd540cbf50 2024-01-15T18:18:30.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd541578a0 con 0x7fcd540cbf50 2024-01-15T18:18:30.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.711+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd54158000 con 0x7fcd540cbf50 2024-01-15T18:18:30.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.715+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd5406d4b0 con 0x7fcd540cbf50 2024-01-15T18:18:30.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.715+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fcd440039a0 con 0x7fcd540cbf50 2024-01-15T18:18:30.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.715+0000 7fcd50c00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd2c067a60 0x7fcd2c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:30.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.715+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fcd440828e0 con 0x7fcd540cbf50 2024-01-15T18:18:30.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.715+0000 7fcd53e00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd2c067a60 0x7fcd2c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:30.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.715+0000 7fcd53e00640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd2c067a60 0x7fcd2c069f20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcd541f7380 tx=0x7fcd3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:30.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.719+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcd44051220 con 0x7fcd540cbf50 2024-01-15T18:18:30.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.871+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd54067500 con 0x7fcd540cbf50 2024-01-15T18:18:30.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd50c00640 1 -- 172.21.15.143:0/2919743811 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd440170a0 con 0x7fcd540cbf50 2024-01-15T18:18:30.869 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd2c067a60 msgr2=0x7fcd2c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd2c067a60 0x7fcd2c069f20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fcd541f7380 tx=0x7fcd3c008040 comp rx=0 tx=0).stop 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 msgr2=0x7fcd541584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd541584e0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400c3c0 tx=0x7fcd4400c880 comp rx=0 tx=0).stop 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 shutdown_connections 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcd2c067a60 0x7fcd2c069f20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd541607c0 0x7fcd541f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcd540cfb30 0x7fcd54157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 --2- 172.21.15.143:0/2919743811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd540cbf50 0x7fcd541584e0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 >> 172.21.15.143:0/2919743811 conn(0x7fcd540d0e20 msgr2=0x7fcd540ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 shutdown_connections 2024-01-15T18:18:30.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:30.875+0000 7fcd5b400640 1 -- 172.21.15.143:0/2919743811 wait complete. 2024-01-15T18:18:30.892 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:18:31.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 Processor -- start 2024-01-15T18:18:31.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 -- start start 2024-01-15T18:18:31.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4af0141660 0x7f4af00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af015ec80 0x7f4af0141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af015f310 con 0x7f4af0142470 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af015fbb0 con 0x7f4af0141660 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4af5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015f100 con 0x7f4af015ec80 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4af0141660 0x7f4af00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4af0141660 0x7f4af00ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48358/0 (socket says 172.21.15.143:48358) 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 -- 172.21.15.143:0/3146409024 learned_addr learned my addr 172.21.15.143:0/3146409024 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aed400640 1 --2- 172.21.15.143:0/3146409024 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af015ec80 0x7f4af0141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 -- 172.21.15.143:0/3146409024 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af015ec80 msgr2=0x7f4af0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 --2- 172.21.15.143:0/3146409024 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af015ec80 0x7f4af0141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 -- 172.21.15.143:0/3146409024 >> v1:172.21.15.204:6789/0 conn(0x7f4af0142470 legacy=0x7f4af0071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.035+0000 7f4aede00640 1 -- 172.21.15.143:0/3146409024 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af015f5f0 con 0x7f4af0141660 2024-01-15T18:18:31.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4aede00640 1 --2- 172.21.15.143:0/3146409024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af0141660 0x7f4af00ce560 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0ca720eb2fe897 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:31.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4aeca00640 1 -- 172.21.15.143:0/3146409024 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00aa670 con 0x7f4af0141660 2024-01-15T18:18:31.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4aeca00640 1 -- 172.21.15.143:0/3146409024 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae00aa7d0 con 0x7f4af0141660 2024-01-15T18:18:31.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4aeca00640 1 -- 172.21.15.143:0/3146409024 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00aab60 con 0x7f4af0141660 2024-01-15T18:18:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 -- 172.21.15.143:0/3146409024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af0141660 msgr2=0x7f4af00ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 --2- 172.21.15.143:0/3146409024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af0141660 0x7f4af00ce560 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae001c510 comp rx=0 tx=0).stop 2024-01-15T18:18:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 -- 172.21.15.143:0/3146409024 shutdown_connections 2024-01-15T18:18:31.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 --2- 172.21.15.143:0/3146409024 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af015ec80 0x7f4af0141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 --2- 172.21.15.143:0/3146409024 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af0141660 0x7f4af00ce560 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 -- 172.21.15.143:0/3146409024 >> 172.21.15.143:0/3146409024 conn(0x7f4af006c3f0 msgr2=0x7f4af006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 -- 172.21.15.143:0/3146409024 shutdown_connections 2024-01-15T18:18:31.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.039+0000 7f4af5000640 1 -- 172.21.15.143:0/3146409024 wait complete. 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 Processor -- start 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 -- start start 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af0141660 0x7f4af00acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0142470 0x7f4af00b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 0x7f4af00b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015c070 con 0x7f4af015ec80 2024-01-15T18:18:31.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b2fb0 con 0x7f4af0142470 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00b3280 con 0x7f4af0141660 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 0x7f4af00b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 0x7f4af00b2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48368/0 (socket says 172.21.15.143:48368) 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 -- 172.21.15.143:0/6013391 learned_addr learned my addr 172.21.15.143:0/6013391 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aede00640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af0141660 0x7f4af00acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aed400640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0142470 0x7f4af00b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 -- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af0141660 msgr2=0x7f4af00acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af0141660 0x7f4af00acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 -- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0142470 msgr2=0x7f4af00b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0142470 0x7f4af00b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 -- 172.21.15.143:0/6013391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00094d0 con 0x7f4af015ec80 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4aee800640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 0x7f4af00b2a70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c6c0 tx=0x7f4ae800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8003b80 con 0x7f4af015ec80 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae8003ce0 con 0x7f4af015ec80 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80052f0 con 0x7f4af015ec80 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af00cd150 con 0x7f4af015ec80 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.043+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af00b3840 con 0x7f4af015ec80 2024-01-15T18:18:31.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.047+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af015c240 con 0x7f4af015ec80 2024-01-15T18:18:31.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.047+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f4ae8005450 con 0x7f4af015ec80 2024-01-15T18:18:31.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.047+0000 7f4ae6000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ac8067a60 0x7f4ac8069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.047+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f4ae8082fa0 con 0x7f4af015ec80 2024-01-15T18:18:31.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.047+0000 7f4aede00640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ac8067a60 0x7f4ac8069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.047+0000 7f4aede00640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ac8067a60 0x7f4ac8069f20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ae000 tx=0x7f4ae00b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.051+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4af015c240 con 0x7f4af015ec80 2024-01-15T18:18:31.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.211+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4af0067500 con 0x7f4af015ec80 2024-01-15T18:18:31.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.211+0000 7f4ae6000640 1 -- 172.21.15.143:0/6013391 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v103) v1 ==== 80+0+124 (secure 0 0 0) 0x7f4ae8051990 con 0x7f4af015ec80 2024-01-15T18:18:31.207 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6835/4145140990 2024-01-16T18:18:25.607861+0000 2024-01-15T18:18:31.207 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.15.143:6834/4145140990 2024-01-16T18:18:25.607861+0000 2024-01-15T18:18:31.207 INFO:teuthology.orchestra.run.smithi143.stderr:listed 2 entries 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.211+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ac8067a60 msgr2=0x7f4ac8069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.211+0000 7f4af5000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ac8067a60 0x7f4ac8069f20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ae000 tx=0x7f4ae00b5040 comp rx=0 tx=0).stop 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.211+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 msgr2=0x7f4af00b2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.211+0000 7f4af5000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 0x7f4af00b2a70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c6c0 tx=0x7f4ae800cb80 comp rx=0 tx=0).stop 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 shutdown_connections 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ac8067a60 0x7f4ac8069f20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4af015ec80 0x7f4af00b2a70 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4af0142470 0x7f4af00b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 --2- 172.21.15.143:0/6013391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4af0141660 0x7f4af00acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 >> 172.21.15.143:0/6013391 conn(0x7f4af006c3f0 msgr2=0x7f4af006c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 shutdown_connections 2024-01-15T18:18:31.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.215+0000 7f4af5000640 1 -- 172.21.15.143:0/6013391 wait complete. 2024-01-15T18:18:31.228 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:18:31.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:31.229 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:31.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34321]: fuse finished with error 0 and tester_r 0 2024-01-15T18:18:31.252 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:18:31.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:31.253 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:18:31.426 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:18:31.426 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:31.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:18:31.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:18:31.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:31.436 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:18:31.488 INFO:teuthology.orchestra.run.smithi143.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:18:31.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:18:31.489 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:18:31.489 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:31.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:18:31.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:18:31.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:31.500 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:18:31.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:18:31.551 INFO:teuthology.orchestra.run.smithi204.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:18:31.552 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:18:31.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.691+0000 7facce000640 1 Processor -- start 2024-01-15T18:18:31.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.691+0000 7facce000640 1 -- start start 2024-01-15T18:18:31.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facce000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7facc80ce560 0x7facc815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facce000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc815f310 con 0x7facc8141660 2024-01-15T18:18:31.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facce000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc815f5f0 con 0x7facc8071d00 2024-01-15T18:18:31.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facce000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc815fbb0 con 0x7facc80ce560 2024-01-15T18:18:31.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7facc8141660 0x7facc8141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49214/0 (socket says 172.21.15.143:49214) 2024-01-15T18:18:31.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7400640 1 -- 172.21.15.143:0/967144950 learned_addr learned my addr 172.21.15.143:0/967144950 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:31.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6a00640 1 --2- 172.21.15.143:0/967144950 >> v2:172.21.15.204:3300/0 conn(0x7facc80ce560 0x7facc815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336735377 0 0) 0x7facc815f310 con 0x7facc8141660 2024-01-15T18:18:31.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faca8003530 con 0x7facc8141660 2024-01-15T18:18:31.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1294878971 0 0) 0x7faca8003530 con 0x7facc8141660 2024-01-15T18:18:31.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 >> v2:172.21.15.204:3300/0 conn(0x7facc80ce560 msgr2=0x7facc815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 --2- 172.21.15.143:0/967144950 >> v2:172.21.15.204:3300/0 conn(0x7facc80ce560 0x7facc815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 >> v1:172.21.15.204:6789/0 conn(0x7facc8071d00 legacy=0x7facc80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc815f8d0 con 0x7facc8141660 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7facb8001680 con 0x7facc8141660 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 >> v1:172.21.15.143:6789/0 conn(0x7facc8141660 legacy=0x7facc8141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faca8003980 0x7faca8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca80047d0 0x7faca8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca8005190 0x7faca800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc815fbb0 con 0x7faca8005190 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca800c030 con 0x7faca8003980 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca800c1f0 con 0x7faca80047d0 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7e00640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca8005190 0x7faca800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc6a00640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca80047d0 0x7faca8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7400640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faca8003980 0x7faca8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7e00640 1 -- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca80047d0 msgr2=0x7faca8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7e00640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca80047d0 0x7faca8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7e00640 1 -- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faca8003980 msgr2=0x7faca8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:31.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7e00640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faca8003980 0x7faca8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.695+0000 7facc7e00640 1 -- 172.21.15.143:0/967144950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca8003530 con 0x7faca8005190 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facc7e00640 1 -- 172.21.15.143:0/967144950 reap_dead start 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facc7e00640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca8005190 0x7faca800b990 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7facc0001e70 tx=0x7facc001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a6c5c5f76be55c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0003b80 con 0x7faca8005190 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0003d20 con 0x7faca8005190 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facc6000640 1 -- 172.21.15.143:0/967144950 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0005260 con 0x7faca8005190 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 -- 172.21.15.143:0/967144950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca8005190 msgr2=0x7faca800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca8005190 0x7faca800b990 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7facc0001e70 tx=0x7facc001c760 comp rx=0 tx=0).stop 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 -- 172.21.15.143:0/967144950 shutdown_connections 2024-01-15T18:18:31.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 --2- 172.21.15.143:0/967144950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faca8005190 0x7faca800b990 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 -- 172.21.15.143:0/967144950 >> 172.21.15.143:0/967144950 conn(0x7facc806c3f0 msgr2=0x7facc806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:31.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 -- 172.21.15.143:0/967144950 shutdown_connections 2024-01-15T18:18:31.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.699+0000 7facce000640 1 -- 172.21.15.143:0/967144950 wait complete. 2024-01-15T18:18:31.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 Processor -- start 2024-01-15T18:18:31.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- start start 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca8005190 0x7facc815cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8158190 0x7facc815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 0x7facc80acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80ad2d0 con 0x7facc80ac8e0 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc815d910 con 0x7facc8158190 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc80b3dd0 con 0x7faca8005190 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 0x7facc80acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 0x7facc80acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48394/0 (socket says 172.21.15.143:48394) 2024-01-15T18:18:31.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 -- 172.21.15.143:0/2095030911 learned_addr learned my addr 172.21.15.143:0/2095030911 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7400640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca8005190 0x7facc815cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc6a00640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8158190 0x7facc815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 -- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca8005190 msgr2=0x7facc815cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca8005190 0x7facc815cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 -- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8158190 msgr2=0x7facc815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8158190 0x7facc815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 -- 172.21.15.143:0/2095030911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80cd150 con 0x7facc80ac8e0 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc6a00640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8158190 0x7facc815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:31.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facc7e00640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 0x7facc80acd90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7facc001cc40 tx=0x7faca8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00ac070 con 0x7facc80ac8e0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0007dc0 con 0x7facc80ac8e0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc815b8b0 con 0x7facc80ac8e0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0006d60 con 0x7facc80ac8e0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc815be70 con 0x7facc80ac8e0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.703+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc8067500 con 0x7facc80ac8e0 2024-01-15T18:18:31.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.707+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7facc0007030 con 0x7facc80ac8e0 2024-01-15T18:18:31.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.707+0000 7facbf400640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca0067a90 0x7faca0069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:31.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.707+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7facc011c4c0 con 0x7facc80ac8e0 2024-01-15T18:18:31.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.707+0000 7facc7400640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca0067a90 0x7faca0069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:31.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.707+0000 7facc7400640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca0067a90 0x7faca0069f50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7facb8007bb0 tx=0x7facb8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:31.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.711+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7facc00eaeb0 con 0x7facc80ac8e0 2024-01-15T18:18:31.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.895+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facc806e2a0 con 0x7facc80ac8e0 2024-01-15T18:18:31.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.895+0000 7facbf400640 1 -- 172.21.15.143:0/2095030911 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3711 (secure 0 0 0) 0x7facc00b5030 con 0x7facc80ac8e0 2024-01-15T18:18:31.891 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:31.891 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1910039233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1910039233},{"type":"v1","addr":"172.21.15.204:6835","nonce":1910039233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2986890926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2986890926},{"type":"v1","addr":"172.21.15.143:6837","nonce":2986890926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2769510067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2769510067},{"type":"v1","addr":"172.21.15.204:6834","nonce":2769510067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1268740903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1268740903},{"type":"v1","addr":"172.21.15.143:6835","nonce":1268740903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2024-01-15T18:18:31.892 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2024-01-15T18:18:31.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca0067a90 msgr2=0x7faca0069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca0067a90 0x7faca0069f50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7facb8007bb0 tx=0x7facb8009040 comp rx=0 tx=0).stop 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 msgr2=0x7facc80acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 0x7facc80acd90 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7facc001cc40 tx=0x7faca8005800 comp rx=0 tx=0).stop 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 shutdown_connections 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca0067a90 0x7faca0069f50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facc80ac8e0 0x7facc80acd90 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facc8158190 0x7facc815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 --2- 172.21.15.143:0/2095030911 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faca8005190 0x7facc815cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:31.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 >> 172.21.15.143:0/2095030911 conn(0x7facc806c3f0 msgr2=0x7facc80af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:31.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 shutdown_connections 2024-01-15T18:18:31.895 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:31.899+0000 7facce000640 1 -- 172.21.15.143:0/2095030911 wait complete. 2024-01-15T18:18:31.913 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:18:32.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.055+0000 7fa1ac000640 1 Processor -- start 2024-01-15T18:18:32.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.055+0000 7fa1ac000640 1 -- start start 2024-01-15T18:18:32.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1ac000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 0x7fa1a40711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1ac000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa1a40acbc0 0x7fa1a40ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1ac000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40b3370 con 0x7fa1a415d1f0 2024-01-15T18:18:32.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1ac000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a415d5d0 con 0x7fa1a40acbc0 2024-01-15T18:18:32.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1ac000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a406a800 con 0x7fa1a4070d60 2024-01-15T18:18:32.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 0x7fa1a40711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a9600640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fa1a415d1f0 0x7fa1a406b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:55016/0 (socket says 172.21.15.143:55016) 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 0x7fa1a40711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:55588/0 (socket says 172.21.15.143:55588) 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a9600640 1 -- 172.21.15.143:0/955754599 learned_addr learned my addr 172.21.15.143:0/955754599 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1aa000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa1a40acbc0 0x7fa1a40ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a3e00640 1 -- 172.21.15.143:0/955754599 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886407800 0 0) 0x7fa1a40b3370 con 0x7fa1a415d1f0 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a8c00640 1 -- 172.21.15.143:0/955754599 >> v2:172.21.15.204:3300/0 conn(0x7fa1a40acbc0 msgr2=0x7fa1a40ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a8c00640 1 --2- 172.21.15.143:0/955754599 >> v2:172.21.15.204:3300/0 conn(0x7fa1a40acbc0 0x7fa1a40ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a8c00640 1 -- 172.21.15.143:0/955754599 >> v1:172.21.15.204:6789/0 conn(0x7fa1a415d1f0 legacy=0x7fa1a406b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1a8c00640 1 -- 172.21.15.143:0/955754599 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4071700 con 0x7fa1a4070d60 2024-01-15T18:18:32.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.059+0000 7fa1aa000640 1 --2- 172.21.15.143:0/955754599 >> v2:172.21.15.204:3300/0 conn(0x7fa1a40acbc0 0x7fa1a40ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:18:32.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1a8c00640 1 --2- 172.21.15.143:0/955754599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 0x7fa1a40711c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0097c0 tx=0x7fa18c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3e8c56f5ea96d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1a3e00640 1 -- 172.21.15.143:0/955754599 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0ab650 con 0x7fa1a4070d60 2024-01-15T18:18:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1a3e00640 1 -- 172.21.15.143:0/955754599 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c0ab7b0 con 0x7fa1a4070d60 2024-01-15T18:18:32.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1a3e00640 1 -- 172.21.15.143:0/955754599 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0abb40 con 0x7fa1a4070d60 2024-01-15T18:18:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 -- 172.21.15.143:0/955754599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 msgr2=0x7fa1a40711c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 --2- 172.21.15.143:0/955754599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 0x7fa1a40711c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0097c0 tx=0x7fa18c01c450 comp rx=0 tx=0).stop 2024-01-15T18:18:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 -- 172.21.15.143:0/955754599 shutdown_connections 2024-01-15T18:18:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 --2- 172.21.15.143:0/955754599 >> v2:172.21.15.204:3300/0 conn(0x7fa1a40acbc0 0x7fa1a40ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 --2- 172.21.15.143:0/955754599 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a4070d60 0x7fa1a40711c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 -- 172.21.15.143:0/955754599 >> 172.21.15.143:0/955754599 conn(0x7fa1a40b1610 msgr2=0x7fa1a40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:32.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 -- 172.21.15.143:0/955754599 shutdown_connections 2024-01-15T18:18:32.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.063+0000 7fa1ac000640 1 -- 172.21.15.143:0/955754599 wait complete. 2024-01-15T18:18:32.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 Processor -- start 2024-01-15T18:18:32.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- start start 2024-01-15T18:18:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a4070d60 0x7fa1a41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a40acbc0 0x7fa1a4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 0x7fa1a4149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a414e450 con 0x7fa1a415d1f0 2024-01-15T18:18:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4153320 con 0x7fa1a4070d60 2024-01-15T18:18:32.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4153550 con 0x7fa1a40acbc0 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 0x7fa1a4149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 0x7fa1a4149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48406/0 (socket says 172.21.15.143:48406) 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 -- 172.21.15.143:0/4211590096 learned_addr learned my addr 172.21.15.143:0/4211590096 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1a9600640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a4070d60 0x7fa1a41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1a8c00640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a40acbc0 0x7fa1a4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 -- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a40acbc0 msgr2=0x7fa1a4152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a40acbc0 0x7fa1a4152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 -- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a4070d60 msgr2=0x7fa1a41525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a4070d60 0x7fa1a41525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 -- 172.21.15.143:0/4211590096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c009a70 con 0x7fa1a415d1f0 2024-01-15T18:18:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1aa000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 0x7fa1a4149540 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00efc0 tx=0x7fa19c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c009d70 con 0x7fa1a415d1f0 2024-01-15T18:18:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c00f040 con 0x7fa1a415d1f0 2024-01-15T18:18:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0039a0 con 0x7fa1a415d1f0 2024-01-15T18:18:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18c009420 con 0x7fa1a415d1f0 2024-01-15T18:18:32.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a414e0e0 con 0x7fa1a415d1f0 2024-01-15T18:18:32.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.067+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a4074b00 con 0x7fa1a415d1f0 2024-01-15T18:18:32.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.071+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fa19c0144a0 con 0x7fa1a415d1f0 2024-01-15T18:18:32.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.071+0000 7fa1a1600640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa17c067a60 0x7fa17c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.071+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fa19c083f10 con 0x7fa1a415d1f0 2024-01-15T18:18:32.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.071+0000 7fa1a9600640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa17c067a60 0x7fa17c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.071+0000 7fa1a9600640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa17c067a60 0x7fa17c069f20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa194000f70 tx=0x7fa194007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.075+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa19c052900 con 0x7fa1a415d1f0 2024-01-15T18:18:32.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.227+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7fa1a4067500 con 0x7fa1a415d1f0 2024-01-15T18:18:32.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1a1600640 1 -- 172.21.15.143:0/4211590096 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v37) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa19c0180b0 con 0x7fa1a415d1f0 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa17c067a60 msgr2=0x7fa17c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa17c067a60 0x7fa17c069f20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa194000f70 tx=0x7fa194007df0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 msgr2=0x7fa1a4149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 0x7fa1a4149540 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00efc0 tx=0x7fa19c009f90 comp rx=0 tx=0).stop 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 shutdown_connections 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa17c067a60 0x7fa17c069f20 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a415d1f0 0x7fa1a4149540 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa1a40acbc0 0x7fa1a4152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 --2- 172.21.15.143:0/4211590096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1a4070d60 0x7fa1a41525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.231+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 >> 172.21.15.143:0/4211590096 conn(0x7fa1a40b1610 msgr2=0x7fa1a40b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.235+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 shutdown_connections 2024-01-15T18:18:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.235+0000 7fa1ac000640 1 -- 172.21.15.143:0/4211590096 wait complete. 2024-01-15T18:18:32.249 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:18:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.391+0000 7f5767c00640 1 Processor -- start 2024-01-15T18:18:32.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.391+0000 7f5767c00640 1 -- start start 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5767c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5760141660 0x7f57600ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5767c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576015f310 con 0x7f576015ec80 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5767c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576015f5f0 con 0x7f5760071d00 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5767c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576015fbb0 con 0x7f5760141660 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f576015ec80 0x7f5760141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49224/0 (socket says 172.21.15.143:49224) 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 -- 172.21.15.143:0/3983964006 learned_addr learned my addr 172.21.15.143:0/3983964006 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:32.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5765200640 1 --2- 172.21.15.143:0/3983964006 >> v2:172.21.15.204:3300/0 conn(0x7f5760141660 0x7f57600ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771930275 0 0) 0x7f576015f310 con 0x7f576015ec80 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5740003530 con 0x7f576015ec80 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 797004491 0 0) 0x7f5740003530 con 0x7f576015ec80 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 >> v2:172.21.15.204:3300/0 conn(0x7f5760141660 msgr2=0x7f57600ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 --2- 172.21.15.143:0/3983964006 >> v2:172.21.15.204:3300/0 conn(0x7f5760141660 0x7f57600ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 >> v1:172.21.15.204:6789/0 conn(0x7f5760071d00 legacy=0x7f57600b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576015f8d0 con 0x7f576015ec80 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f5748001680 con 0x7f576015ec80 2024-01-15T18:18:32.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 >> v1:172.21.15.143:6789/0 conn(0x7f576015ec80 legacy=0x7f5760141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740003980 0x7f5740003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57400047d0 0x7f5740004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5740005190 0x7f574000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576015fbb0 con 0x7f57400047d0 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574000c030 con 0x7f5740003980 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574000c1f0 con 0x7f5740005190 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57400047d0 0x7f5740004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5765200640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740003980 0x7f5740003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5765c00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5740005190 0x7f574000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 -- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5740005190 msgr2=0x7f574000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5740005190 0x7f574000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 -- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740003980 msgr2=0x7f5740003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740003980 0x7f5740003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f575fe00640 1 -- 172.21.15.143:0/3983964006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740003530 con 0x7f57400047d0 2024-01-15T18:18:32.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5765c00640 1 -- 172.21.15.143:0/3983964006 reap_dead start 2024-01-15T18:18:32.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.395+0000 7f5765200640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740003980 0x7f5740003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:32.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f575fe00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57400047d0 0x7f5740004c50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f574800c240 tx=0x7f574801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3920a02f35675a21 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748005ae0 con 0x7f57400047d0 2024-01-15T18:18:32.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57480ab070 con 0x7f57400047d0 2024-01-15T18:18:32.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f575f400640 1 -- 172.21.15.143:0/3983964006 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748007070 con 0x7f57400047d0 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 -- 172.21.15.143:0/3983964006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57400047d0 msgr2=0x7f5740004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57400047d0 0x7f5740004c50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f574800c240 tx=0x7f574801f3b0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 -- 172.21.15.143:0/3983964006 shutdown_connections 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 --2- 172.21.15.143:0/3983964006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f57400047d0 0x7f5740004c50 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 -- 172.21.15.143:0/3983964006 >> 172.21.15.143:0/3983964006 conn(0x7f576006c3f0 msgr2=0x7f576006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 -- 172.21.15.143:0/3983964006 shutdown_connections 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.399+0000 7f5767c00640 1 -- 172.21.15.143:0/3983964006 wait complete. 2024-01-15T18:18:32.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 Processor -- start 2024-01-15T18:18:32.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 -- start start 2024-01-15T18:18:32.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57400047d0 0x7f57601388b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 0x7f57601f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f576014e600 0x7f576014ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576015f130 con 0x7f576013b5c0 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576013b1a0 con 0x7f576014e600 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576013a800 con 0x7f57400047d0 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 0x7f57601f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 0x7f57601f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48442/0 (socket says 172.21.15.143:48442) 2024-01-15T18:18:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 -- 172.21.15.143:0/2218276440 learned_addr learned my addr 172.21.15.143:0/2218276440 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5765c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f576014e600 0x7f576014ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5765200640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57400047d0 0x7f57601388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 -- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57400047d0 msgr2=0x7f57601388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57400047d0 0x7f57601388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 -- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f576014e600 msgr2=0x7f576014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f576014e600 0x7f576014ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 -- 172.21.15.143:0/2218276440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600cd150 con 0x7f576013b5c0 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5765200640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57400047d0 0x7f57601388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575fe00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 0x7f57601f6590 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f574801f1e0 tx=0x7f5740005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748004b30 con 0x7f576013b5c0 2024-01-15T18:18:32.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5748004c90 con 0x7f576013b5c0 2024-01-15T18:18:32.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576013aae0 con 0x7f576013b5c0 2024-01-15T18:18:32.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5748007070 con 0x7f576013b5c0 2024-01-15T18:18:32.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.403+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5760138df0 con 0x7f576013b5c0 2024-01-15T18:18:32.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.407+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5760067500 con 0x7f576013b5c0 2024-01-15T18:18:32.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.407+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f57480b9030 con 0x7f576013b5c0 2024-01-15T18:18:32.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.407+0000 7f575cc00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5738067a60 0x7f5738069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.407+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f57480b7070 con 0x7f576013b5c0 2024-01-15T18:18:32.405 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.407+0000 7f5765200640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5738067a60 0x7f5738069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.407+0000 7f5765200640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5738067a60 0x7f5738069f20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5760067bb0 tx=0x7f5750008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.411+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f57480eeb00 con 0x7f576013b5c0 2024-01-15T18:18:32.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.563+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f576006e2a0 con 0x7f576013b5c0 2024-01-15T18:18:32.561 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f575cc00640 1 -- 172.21.15.143:0/2218276440 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57480c1020 con 0x7f576013b5c0 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5738067a60 msgr2=0x7f5738069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5738067a60 0x7f5738069f20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5760067bb0 tx=0x7f5750008040 comp rx=0 tx=0).stop 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 msgr2=0x7f57601f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 0x7f57601f6590 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f574801f1e0 tx=0x7f5740005800 comp rx=0 tx=0).stop 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 shutdown_connections 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5738067a60 0x7f5738069f20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f576014e600 0x7f576014ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f576013b5c0 0x7f57601f6590 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 --2- 172.21.15.143:0/2218276440 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f57400047d0 0x7f57601388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 >> 172.21.15.143:0/2218276440 conn(0x7f576006c3f0 msgr2=0x7f57600ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:32.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 shutdown_connections 2024-01-15T18:18:32.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.567+0000 7f5767c00640 1 -- 172.21.15.143:0/2218276440 wait complete. 2024-01-15T18:18:32.583 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-01-15T18:18:32.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45e0c00640 1 Processor -- start 2024-01-15T18:18:32.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45e0c00640 1 -- start start 2024-01-15T18:18:32.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45e0c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f45dc0ce560 0x7f45dc15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45e0c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc15f310 con 0x7f45dc141660 2024-01-15T18:18:32.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45e0c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc15f5f0 con 0x7f45dc071d00 2024-01-15T18:18:32.726 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45e0c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45dc15fbb0 con 0x7f45dc0ce560 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d9e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f45dc141660 0x7f45dc141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49240/0 (socket says 172.21.15.143:49240) 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d9e00640 1 -- 172.21.15.143:0/191584694 learned_addr learned my addr 172.21.15.143:0/191584694 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781268025 0 0) 0x7f45dc15f310 con 0x7f45dc141660 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45bc003530 con 0x7f45dc141660 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d9400640 1 --2- 172.21.15.143:0/191584694 >> v2:172.21.15.204:3300/0 conn(0x7f45dc0ce560 0x7f45dc15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2891664326 0 0) 0x7f45bc003530 con 0x7f45dc141660 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 >> v2:172.21.15.204:3300/0 conn(0x7f45dc0ce560 msgr2=0x7f45dc15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 --2- 172.21.15.143:0/191584694 >> v2:172.21.15.204:3300/0 conn(0x7f45dc0ce560 0x7f45dc15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 >> v1:172.21.15.204:6789/0 conn(0x7f45dc071d00 legacy=0x7f45dc0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc15f8d0 con 0x7f45dc141660 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f45cc001680 con 0x7f45dc141660 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 >> v1:172.21.15.143:6789/0 conn(0x7f45dc141660 legacy=0x7f45dc141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc0039d0 0x7f45bc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.727+0000 7f45d8a00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc0047d0 0x7f45bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45bc005190 0x7f45bc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc15fbb0 con 0x7f45bc005190 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc00c090 con 0x7f45bc0039d0 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc00c250 con 0x7f45bc0047d0 2024-01-15T18:18:32.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d9e00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc0039d0 0x7f45bc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45bc005190 0x7f45bc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d9400640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc0047d0 0x7f45bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 -- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc0047d0 msgr2=0x7f45bc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45bc0047d0 0x7f45bc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 -- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc0039d0 msgr2=0x7f45bc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc0039d0 0x7f45bc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 -- 172.21.15.143:0/191584694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc003530 con 0x7f45bc005190 2024-01-15T18:18:32.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d9e00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc0039d0 0x7f45bc003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 -- 172.21.15.143:0/191584694 reap_dead start 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45da800640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45bc005190 0x7f45bc00b990 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f45d4001e70 tx=0x7f45d401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b73720a62fe58 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4003b80 con 0x7f45bc005190 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4003d20 con 0x7f45bc005190 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.731+0000 7f45d8a00640 1 -- 172.21.15.143:0/191584694 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4005260 con 0x7f45bc005190 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- 172.21.15.143:0/191584694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45bc005190 msgr2=0x7f45bc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45bc005190 0x7f45bc00b990 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f45d4001e70 tx=0x7f45d401c760 comp rx=0 tx=0).stop 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- 172.21.15.143:0/191584694 shutdown_connections 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 --2- 172.21.15.143:0/191584694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45bc005190 0x7f45bc00b990 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- 172.21.15.143:0/191584694 >> 172.21.15.143:0/191584694 conn(0x7f45dc06c3f0 msgr2=0x7f45dc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- 172.21.15.143:0/191584694 shutdown_connections 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- 172.21.15.143:0/191584694 wait complete. 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 Processor -- start 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- start start 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc005190 0x7f45dc1362b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45dc141bc0 0x7f45dc1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 0x7f45dc15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc15e4f0 con 0x7f45dc15db30 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc142410 con 0x7f45bc005190 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45e0c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1426f0 con 0x7f45dc141bc0 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45da800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 0x7f45dc15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45da800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 0x7f45dc15dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48462/0 (socket says 172.21.15.143:48462) 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.735+0000 7f45da800640 1 -- 172.21.15.143:0/1416223037 learned_addr learned my addr 172.21.15.143:0/1416223037 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d9e00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc005190 0x7f45dc1362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d9400640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45dc141bc0 0x7f45dc1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45da800640 1 -- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45dc141bc0 msgr2=0x7f45dc1f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45da800640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45dc141bc0 0x7f45dc1f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45da800640 1 -- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc005190 msgr2=0x7f45dc1362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45da800640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc005190 0x7f45dc1362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45da800640 1 -- 172.21.15.143:0/1416223037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0cd150 con 0x7f45dc15db30 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45da800640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 0x7f45dc15dfb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f45d401cc40 tx=0x7f45bc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d40ac070 con 0x7f45dc15db30 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4007dc0 con 0x7f45dc15db30 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4006d60 con 0x7f45dc15db30 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc1429d0 con 0x7f45dc15db30 2024-01-15T18:18:32.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc142fd0 con 0x7f45dc15db30 2024-01-15T18:18:32.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc067500 con 0x7f45dc15db30 2024-01-15T18:18:32.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f45d4007030 con 0x7f45dc15db30 2024-01-15T18:18:32.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.739+0000 7f45d2000640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45b4067a90 0x7f45b4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:32.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.743+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f45d411bdf0 con 0x7f45dc15db30 2024-01-15T18:18:32.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.743+0000 7f45d9e00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45b4067a90 0x7f45b4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:32.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.743+0000 7f45d9e00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45b4067a90 0x7f45b4069f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f45cc000c10 tx=0x7f45cc007bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:32.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.747+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f45d40ea7e0 con 0x7f45dc15db30 2024-01-15T18:18:32.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.899+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) v1 -- 0x7f45dc06e2a0 con 0x7f45dc15db30 2024-01-15T18:18:32.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45d2000640 1 -- 172.21.15.143:0/1416223037 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v39) v1 ==== 115+0+0 (secure 0 0 0) 0x7f45d40b5030 con 0x7f45dc15db30 2024-01-15T18:18:32.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45b4067a90 msgr2=0x7f45b4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45b4067a90 0x7f45b4069f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f45cc000c10 tx=0x7f45cc007bc0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 msgr2=0x7f45dc15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 0x7f45dc15dfb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f45d401cc40 tx=0x7f45bc005800 comp rx=0 tx=0).stop 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 shutdown_connections 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f45b4067a90 0x7f45b4069f50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f45dc15db30 0x7f45dc15dfb0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f45dc141bc0 0x7f45dc1f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 --2- 172.21.15.143:0/1416223037 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f45bc005190 0x7f45dc1362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 >> 172.21.15.143:0/1416223037 conn(0x7f45dc06c3f0 msgr2=0x7f45dc0afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 shutdown_connections 2024-01-15T18:18:32.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:32.903+0000 7f45e0c00640 1 -- 172.21.15.143:0/1416223037 wait complete. 2024-01-15T18:18:32.921 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled' 2024-01-15T18:18:33.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 Processor -- start 2024-01-15T18:18:33.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 -- start start 2024-01-15T18:18:33.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44c8159b70 0x7f44c80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:33.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f44c80b48f0 0x7f44c8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:33.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c80b4d20 con 0x7f44c8157300 2024-01-15T18:18:33.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c80b35a0 con 0x7f44c8159b70 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44cf600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44c8159fd0 con 0x7f44c80b48f0 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44c7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44c8159b70 0x7f44c80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44c7e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44c8159b70 0x7f44c80b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48472/0 (socket says 172.21.15.143:48472) 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.063+0000 7f44c7e00640 1 -- 172.21.15.143:0/3671239716 learned_addr learned my addr 172.21.15.143:0/3671239716 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44cd600640 1 --2- 172.21.15.143:0/3671239716 >> v2:172.21.15.204:3300/0 conn(0x7f44c80b48f0 0x7f44c8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7e00640 1 -- 172.21.15.143:0/3671239716 >> v2:172.21.15.204:3300/0 conn(0x7f44c80b48f0 msgr2=0x7f44c8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7e00640 1 --2- 172.21.15.143:0/3671239716 >> v2:172.21.15.204:3300/0 conn(0x7f44c80b48f0 0x7f44c8070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7e00640 1 -- 172.21.15.143:0/3671239716 >> v1:172.21.15.143:6789/0 conn(0x7f44c8157300 legacy=0x7f44c80b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:33.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7e00640 1 -- 172.21.15.143:0/3671239716 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c80b2a20 con 0x7f44c8159b70 2024-01-15T18:18:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7e00640 1 --2- 172.21.15.143:0/3671239716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c80b43b0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f44c80b1f70 tx=0x7f44b001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee25d21c319c059 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7400640 1 -- 172.21.15.143:0/3671239716 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00aa430 con 0x7f44c8159b70 2024-01-15T18:18:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7400640 1 -- 172.21.15.143:0/3671239716 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b00aa590 con 0x7f44c8159b70 2024-01-15T18:18:33.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.067+0000 7f44c7400640 1 -- 172.21.15.143:0/3671239716 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00aa920 con 0x7f44c8159b70 2024-01-15T18:18:33.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- 172.21.15.143:0/3671239716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 msgr2=0x7f44c80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:33.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 --2- 172.21.15.143:0/3671239716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c80b43b0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f44c80b1f70 tx=0x7f44b001c4d0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- 172.21.15.143:0/3671239716 shutdown_connections 2024-01-15T18:18:33.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 --2- 172.21.15.143:0/3671239716 >> v2:172.21.15.204:3300/0 conn(0x7f44c80b48f0 0x7f44c8070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 --2- 172.21.15.143:0/3671239716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c80b43b0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- 172.21.15.143:0/3671239716 >> 172.21.15.143:0/3671239716 conn(0x7f44c806c3f0 msgr2=0x7f44c806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:33.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- 172.21.15.143:0/3671239716 shutdown_connections 2024-01-15T18:18:33.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- 172.21.15.143:0/3671239716 wait complete. 2024-01-15T18:18:33.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 Processor -- start 2024-01-15T18:18:33.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- start start 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c80b48f0 0x7f44c8136410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c8157300 0x7f44c814bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c814a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c8150f50 con 0x7f44c8159b70 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c814ca00 con 0x7f44c8157300 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cf600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c814cce0 con 0x7f44c80b48f0 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.071+0000 7f44cd600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c814a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:33.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c814a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48486/0 (socket says 172.21.15.143:48486) 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 -- 172.21.15.143:0/3790157889 learned_addr learned my addr 172.21.15.143:0/3790157889 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44c7e00640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c8157300 0x7f44c814bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44ccc00640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c80b48f0 0x7f44c8136410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 -- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c80b48f0 msgr2=0x7f44c8136410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c80b48f0 0x7f44c8136410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 -- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c8157300 msgr2=0x7f44c814bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c8157300 0x7f44c814bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 -- 172.21.15.143:0/3790157889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b00094d0 con 0x7f44c8159b70 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cd600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c814a050 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f44c0007d90 tx=0x7f44c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:33.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0014420 con 0x7f44c8159b70 2024-01-15T18:18:33.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c00145a0 con 0x7f44c8159b70 2024-01-15T18:18:33.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0014950 con 0x7f44c8159b70 2024-01-15T18:18:33.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c814cfc0 con 0x7f44c8159b70 2024-01-15T18:18:33.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c8136950 con 0x7f44c8159b70 2024-01-15T18:18:33.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c81510c0 con 0x7f44c8159b70 2024-01-15T18:18:33.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f44c0014ab0 con 0x7f44c8159b70 2024-01-15T18:18:33.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.075+0000 7f44c4c00640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44a0067a90 0x7f44a0069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:33.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.079+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f44c0083aa0 con 0x7f44c8159b70 2024-01-15T18:18:33.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.079+0000 7f44ccc00640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44a0067a90 0x7f44a0069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:33.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.079+0000 7f44ccc00640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44a0067a90 0x7f44a0069f50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f44b8007fd0 tx=0x7f44b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:33.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.079+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f44c00524f0 con 0x7f44c8159b70 2024-01-15T18:18:33.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.283+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]} v 0) v1 -- 0x7f44c8067500 con 0x7f44c8159b70 2024-01-15T18:18:33.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.863+0000 7f44c4c00640 1 -- 172.21.15.143:0/3790157889 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]}]=0 v635) v1 ==== 153+0+0 (secure 0 0 0) 0x7f44c0055b10 con 0x7f44c8159b70 2024-01-15T18:18:33.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44a0067a90 msgr2=0x7f44a0069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:33.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44a0067a90 0x7f44a0069f50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f44b8007fd0 tx=0x7f44b8009040 comp rx=0 tx=0).stop 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 msgr2=0x7f44c814a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c814a050 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f44c0007d90 tx=0x7f44c000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 shutdown_connections 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f44a0067a90 0x7f44a0069f50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44c8159b70 0x7f44c814a050 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f44c8157300 0x7f44c814bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 --2- 172.21.15.143:0/3790157889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f44c80b48f0 0x7f44c8136410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 >> 172.21.15.143:0/3790157889 conn(0x7f44c806c3f0 msgr2=0x7f44c81568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 shutdown_connections 2024-01-15T18:18:33.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:33.867+0000 7f44cf600640 1 -- 172.21.15.143:0/3790157889 wait complete. 2024-01-15T18:18:33.888 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the ... ok 2024-01-15T18:18:33.888 INFO:tasks.cephfs_test_runner:Starting test: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:18:33.888 INFO:tasks.cephfs_test_runner:test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:18:33.889 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.035+0000 7f24f7000640 1 Processor -- start 2024-01-15T18:18:34.031 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.035+0000 7f24f7000640 1 -- start start 2024-01-15T18:18:34.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24f7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00cf400 0x7f24f00cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24f7000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f015b5b0 con 0x7f24f00ce390 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24f7000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f00cba70 con 0x7f24f00cda70 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24f7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f00cbd10 con 0x7f24f00cf400 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24f5000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f24f00ce390 0x7f24f00b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49254/0 (socket says 172.21.15.143:49254) 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24f5000640 1 -- 172.21.15.143:0/3087595855 learned_addr learned my addr 172.21.15.143:0/3087595855 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24ef400640 1 --2- 172.21.15.143:0/3087595855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00cf400 0x7f24f00cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997980365 0 0) 0x7f24f015b5b0 con 0x7f24f00ce390 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d0003530 con 0x7f24f00ce390 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380991926 0 0) 0x7f24f00cba70 con 0x7f24f00cda70 2024-01-15T18:18:34.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24f015b5b0 con 0x7f24f00cda70 2024-01-15T18:18:34.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349375062 0 0) 0x7f24d0003530 con 0x7f24f00ce390 2024-01-15T18:18:34.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24ef400640 1 -- 172.21.15.143:0/3087595855 >> v1:172.21.15.204:6789/0 conn(0x7f24f00cda70 legacy=0x7f24f00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24ef400640 1 -- 172.21.15.143:0/3087595855 >> v1:172.21.15.143:6789/0 conn(0x7f24f00ce390 legacy=0x7f24f00b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.039+0000 7f24ef400640 1 -- 172.21.15.143:0/3087595855 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f00cbff0 con 0x7f24f00cf400 2024-01-15T18:18:34.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24ef400640 1 --2- 172.21.15.143:0/3087595855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00cf400 0x7f24f00cde50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f24d80097f0 tx=0x7f24d801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33015c6670ade0bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8001a00 con 0x7f24f00cf400 2024-01-15T18:18:34.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d80a7050 con 0x7f24f00cf400 2024-01-15T18:18:34.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24eea00640 1 -- 172.21.15.143:0/3087595855 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8004250 con 0x7f24f00cf400 2024-01-15T18:18:34.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 -- 172.21.15.143:0/3087595855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00cf400 msgr2=0x7f24f00cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 --2- 172.21.15.143:0/3087595855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00cf400 0x7f24f00cde50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f24d80097f0 tx=0x7f24d801c3b0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 -- 172.21.15.143:0/3087595855 shutdown_connections 2024-01-15T18:18:34.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 --2- 172.21.15.143:0/3087595855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00cf400 0x7f24f00cde50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 -- 172.21.15.143:0/3087595855 >> 172.21.15.143:0/3087595855 conn(0x7f24f00d0e20 msgr2=0x7f24f00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:34.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 -- 172.21.15.143:0/3087595855 shutdown_connections 2024-01-15T18:18:34.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 -- 172.21.15.143:0/3087595855 wait complete. 2024-01-15T18:18:34.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 Processor -- start 2024-01-15T18:18:34.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.043+0000 7f24f7000640 1 -- start start 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24f00cda70 0x7f24f006d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00ce390 0x7f24f006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 0x7f24f01665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f006c500 con 0x7f24f00cf400 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f0166b30 con 0x7f24f00cda70 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f006b5a0 con 0x7f24f00ce390 2024-01-15T18:18:34.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 0x7f24f01665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 0x7f24f01665f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48496/0 (socket says 172.21.15.143:48496) 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24efe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24f00cda70 0x7f24f006d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 -- 172.21.15.143:0/2543144524 learned_addr learned my addr 172.21.15.143:0/2543144524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24ef400640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00ce390 0x7f24f006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 -- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00ce390 msgr2=0x7f24f006c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00ce390 0x7f24f006c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 -- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24f00cda70 msgr2=0x7f24f006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24f00cda70 0x7f24f006d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 -- 172.21.15.143:0/2543144524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8009420 con 0x7f24f00cf400 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f5000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 0x7f24f01665f0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f24e8001f80 tx=0x7f24e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8012400 con 0x7f24f00cf400 2024-01-15T18:18:34.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8012580 con 0x7f24f00cf400 2024-01-15T18:18:34.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8012930 con 0x7f24f00cf400 2024-01-15T18:18:34.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f006b880 con 0x7f24f00cf400 2024-01-15T18:18:34.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f006be40 con 0x7f24f00cf400 2024-01-15T18:18:34.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.047+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f006c640 con 0x7f24f00cf400 2024-01-15T18:18:34.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.051+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f24e8012a90 con 0x7f24f00cf400 2024-01-15T18:18:34.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.051+0000 7f24e7e00640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24c8067a90 0x7f24c8069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.051+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f24e80839b0 con 0x7f24f00cf400 2024-01-15T18:18:34.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.051+0000 7f24efe00640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24c8067a90 0x7f24c8069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.051+0000 7f24efe00640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24c8067a90 0x7f24c8069f50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f24e00011f0 tx=0x7f24e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.055+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f24f006c640 con 0x7f24f00cf400 2024-01-15T18:18:34.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.203+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24f006c640 con 0x7f24f00cf400 2024-01-15T18:18:34.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24e7e00640 1 -- 172.21.15.143:0/2543144524 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13114 (secure 0 0 0) 0x7f24f006c640 con 0x7f24f00cf400 2024-01-15T18:18:34.200 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:34.201 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":103,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:28.617118+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.143:6835/4145140990":"2024-01-16T18:18:25.607861+0000","172.21.15.143:6834/4145140990":"2024-01-16T18:18:25.607861+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24c8067a90 msgr2=0x7f24c8069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24c8067a90 0x7f24c8069f50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f24e00011f0 tx=0x7f24e0009040 comp rx=0 tx=0).stop 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 msgr2=0x7f24f01665f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 0x7f24f01665f0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f24e8001f80 tx=0x7f24e800a430 comp rx=0 tx=0).stop 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 shutdown_connections 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24c8067a90 0x7f24c8069f50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24f00cf400 0x7f24f01665f0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f24f00ce390 0x7f24f006c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 --2- 172.21.15.143:0/2543144524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f24f00cda70 0x7f24f006d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 >> 172.21.15.143:0/2543144524 conn(0x7f24f00d0e20 msgr2=0x7f24f006f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:34.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 shutdown_connections 2024-01-15T18:18:34.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.207+0000 7f24f7000640 1 -- 172.21.15.143:0/2543144524 wait complete. 2024-01-15T18:18:34.222 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:18:34.223 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:18:34.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 Processor -- start 2024-01-15T18:18:34.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 -- start start 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2eac06bfb0 0x7f2eac06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2eac0b0310 0x7f2eac0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0cbff0 con 0x7f2eac0cc9c0 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0d0c50 con 0x7f2eac06bfb0 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eb3200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac06c8d0 con 0x7f2eac0b0310 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2eac06bfb0 0x7f2eac06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2eac06bfb0 0x7f2eac06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48500/0 (socket says 172.21.15.143:48500) 2024-01-15T18:18:34.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 -- 172.21.15.143:0/1052295618 learned_addr learned my addr 172.21.15.143:0/1052295618 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:34.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 -- 172.21.15.143:0/1052295618 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0b0310 msgr2=0x7f2eac0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:34.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eab400640 1 --2- 172.21.15.143:0/1052295618 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0b0310 0x7f2eac0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 --2- 172.21.15.143:0/1052295618 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0b0310 0x7f2eac0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 -- 172.21.15.143:0/1052295618 >> v1:172.21.15.143:6789/0 conn(0x7f2eac0cc9c0 legacy=0x7f2eac0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eabe00640 1 -- 172.21.15.143:0/1052295618 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0cb6c0 con 0x7f2eac06bfb0 2024-01-15T18:18:34.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.367+0000 7f2eab400640 1 --2- 172.21.15.143:0/1052295618 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0b0310 0x7f2eac0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:34.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eabe00640 1 --2- 172.21.15.143:0/1052295618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac06bfb0 0x7f2eac06c390 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2eac06bf00 tx=0x7f2e9c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bc3f213cb8cc8b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eaaa00640 1 -- 172.21.15.143:0/1052295618 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0ab640 con 0x7f2eac06bfb0 2024-01-15T18:18:34.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eaaa00640 1 -- 172.21.15.143:0/1052295618 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0a7070 con 0x7f2eac06bfb0 2024-01-15T18:18:34.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eaaa00640 1 -- 172.21.15.143:0/1052295618 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0ab9c0 con 0x7f2eac06bfb0 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 -- 172.21.15.143:0/1052295618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac06bfb0 msgr2=0x7f2eac06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1052295618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac06bfb0 0x7f2eac06c390 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f2eac06bf00 tx=0x7f2e9c01c440 comp rx=0 tx=0).stop 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 -- 172.21.15.143:0/1052295618 shutdown_connections 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1052295618 >> v2:172.21.15.204:3300/0 conn(0x7f2eac0b0310 0x7f2eac0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1052295618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac06bfb0 0x7f2eac06c390 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 -- 172.21.15.143:0/1052295618 >> 172.21.15.143:0/1052295618 conn(0x7f2eac0d0ed0 msgr2=0x7f2eac0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 -- 172.21.15.143:0/1052295618 shutdown_connections 2024-01-15T18:18:34.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.371+0000 7f2eb3200640 1 -- 172.21.15.143:0/1052295618 wait complete. 2024-01-15T18:18:34.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 Processor -- start 2024-01-15T18:18:34.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 -- start start 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac06bfb0 0x7f2eac153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 0x7f2eac0c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eac0cc9c0 0x7f2eac157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac156aa0 con 0x7f2eac0b0310 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac1567c0 con 0x7f2eac0cc9c0 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac150a80 con 0x7f2eac06bfb0 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 0x7f2eac0c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 0x7f2eac0c7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48516/0 (socket says 172.21.15.143:48516) 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 -- 172.21.15.143:0/1562737359 learned_addr learned my addr 172.21.15.143:0/1562737359 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb1200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eac0cc9c0 0x7f2eac157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eabe00640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac06bfb0 0x7f2eac153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 -- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac06bfb0 msgr2=0x7f2eac153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac06bfb0 0x7f2eac153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 -- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eac0cc9c0 msgr2=0x7f2eac157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eac0cc9c0 0x7f2eac157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 -- 172.21.15.143:0/1562737359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0094d0 con 0x7f2eac0b0310 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb1200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eac0cc9c0 0x7f2eac157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eab400640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 0x7f2eac0c7500 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c7f0 tx=0x7f2e9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94004220 con 0x7f2eac0b0310 2024-01-15T18:18:34.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94004380 con 0x7f2eac0b0310 2024-01-15T18:18:34.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac150d60 con 0x7f2eac0b0310 2024-01-15T18:18:34.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94003840 con 0x7f2eac0b0310 2024-01-15T18:18:34.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.375+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac0c7a40 con 0x7f2eac0b0310 2024-01-15T18:18:34.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.379+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac067500 con 0x7f2eac0b0310 2024-01-15T18:18:34.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.379+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f2e94003bb0 con 0x7f2eac0b0310 2024-01-15T18:18:34.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.379+0000 7f2ea3e00640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2e84067a60 0x7f2e84069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.379+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f2e940832e0 con 0x7f2eac0b0310 2024-01-15T18:18:34.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.379+0000 7f2eabe00640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2e84067a60 0x7f2e84069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.383+0000 7f2eabe00640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2e84067a60 0x7f2e84069f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0af000 tx=0x7f2e9c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.383+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2e94051c20 con 0x7f2eac0b0310 2024-01-15T18:18:34.527 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.531+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2eac0d1db0 con 0x7f2eac0b0310 2024-01-15T18:18:34.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.531+0000 7f2ea3e00640 1 -- 172.21.15.143:0/1562737359 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e9401e020 con 0x7f2eac0b0310 2024-01-15T18:18:34.528 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2e84067a60 msgr2=0x7f2e84069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2e84067a60 0x7f2e84069f20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0af000 tx=0x7f2e9c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 msgr2=0x7f2eac0c7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 0x7f2eac0c7500 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f2e9400c7f0 tx=0x7f2e9400ccb0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 shutdown_connections 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2e84067a60 0x7f2e84069f20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2eac0cc9c0 0x7f2eac157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2eac0b0310 0x7f2eac0c7500 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 --2- 172.21.15.143:0/1562737359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2eac06bfb0 0x7f2eac153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 >> 172.21.15.143:0/1562737359 conn(0x7f2eac0d0ed0 msgr2=0x7f2eac0adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 shutdown_connections 2024-01-15T18:18:34.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.535+0000 7f2eb3200640 1 -- 172.21.15.143:0/1562737359 wait complete. 2024-01-15T18:18:34.551 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_3_clones_are_cancelled' 2024-01-15T18:18:34.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1fc600640 1 Processor -- start 2024-01-15T18:18:34.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1fc600640 1 -- start start 2024-01-15T18:18:34.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1fc600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe1f40b4ac0 0x7fe1f4164540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1fc600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40b2a20 con 0x7fe1f4157300 2024-01-15T18:18:34.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1fc600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f40b2d00 con 0x7fe1f4159b70 2024-01-15T18:18:34.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1fc600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f40b35a0 con 0x7fe1f40b4ac0 2024-01-15T18:18:34.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1f9c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe1f4157300 0x7fe1f40b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49282/0 (socket says 172.21.15.143:49282) 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.691+0000 7fe1f9c00640 1 -- 172.21.15.143:0/3580848457 learned_addr learned my addr 172.21.15.143:0/3580848457 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237316552 0 0) 0x7fe1f40b2a20 con 0x7fe1f4157300 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d4003530 con 0x7fe1f4157300 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1fa600640 1 --2- 172.21.15.143:0/3580848457 >> v2:172.21.15.204:3300/0 conn(0x7fe1f40b4ac0 0x7fe1f4164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3592359416 0 0) 0x7fe1d4003530 con 0x7fe1f4157300 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 >> v2:172.21.15.204:3300/0 conn(0x7fe1f40b4ac0 msgr2=0x7fe1f4164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 --2- 172.21.15.143:0/3580848457 >> v2:172.21.15.204:3300/0 conn(0x7fe1f40b4ac0 0x7fe1f4164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 >> v1:172.21.15.204:6789/0 conn(0x7fe1f4159b70 legacy=0x7fe1f40b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f40b2fe0 con 0x7fe1f4157300 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fe1e4001680 con 0x7fe1f4157300 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1fa600640 1 --2- 172.21.15.143:0/3580848457 >> v2:172.21.15.204:3300/0 conn(0x7fe1f40b4ac0 0x7fe1f4164540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 >> v1:172.21.15.143:6789/0 conn(0x7fe1f4157300 legacy=0x7fe1f40b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40047d0 0x7fe1d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4005190 0x7fe1d400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f40b35a0 con 0x7fe1d4003980 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400c0e0 con 0x7fe1d40047d0 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400c2a0 con 0x7fe1d4005190 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9200640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40047d0 0x7fe1d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1fa600640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4005190 0x7fe1d400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9c00640 1 -- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4005190 msgr2=0x7fe1d400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1d4005190 0x7fe1d400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9c00640 1 -- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40047d0 msgr2=0x7fe1d4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1d40047d0 0x7fe1d4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1f9c00640 1 -- 172.21.15.143:0/3580848457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4003530 con 0x7fe1d4003980 2024-01-15T18:18:34.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.695+0000 7fe1fa600640 1 -- 172.21.15.143:0/3580848457 reap_dead start 2024-01-15T18:18:34.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1d4003da0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c240 tx=0x7fe1e401cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef47cf58ea06fb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4005ae0 con 0x7fe1d4003980 2024-01-15T18:18:34.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4005c40 con 0x7fe1d4003980 2024-01-15T18:18:34.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1f3e00640 1 -- 172.21.15.143:0/3580848457 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40072b0 con 0x7fe1d4003980 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 -- 172.21.15.143:0/3580848457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 msgr2=0x7fe1d4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1d4003da0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c240 tx=0x7fe1e401cb90 comp rx=0 tx=0).stop 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 -- 172.21.15.143:0/3580848457 shutdown_connections 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 --2- 172.21.15.143:0/3580848457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1d4003da0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 -- 172.21.15.143:0/3580848457 >> 172.21.15.143:0/3580848457 conn(0x7fe1f406c3f0 msgr2=0x7fe1f406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 -- 172.21.15.143:0/3580848457 shutdown_connections 2024-01-15T18:18:34.693 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 -- 172.21.15.143:0/3580848457 wait complete. 2024-01-15T18:18:34.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 Processor -- start 2024-01-15T18:18:34.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.699+0000 7fe1fc600640 1 -- start start 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1f40b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f4152300 0x7fe1f41f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f414fc90 0x7fe1f40ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4159fd0 con 0x7fe1d4003980 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f4152760 con 0x7fe1f414fc90 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f414cf60 con 0x7fe1f4152300 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1f40b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1f40b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48538/0 (socket says 172.21.15.143:48538) 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 -- 172.21.15.143:0/33948978 learned_addr learned my addr 172.21.15.143:0/33948978 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:34.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9200640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f4152300 0x7fe1f41f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fa600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f414fc90 0x7fe1f40ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 -- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f4152300 msgr2=0x7fe1f41f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f4152300 0x7fe1f41f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 -- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f414fc90 msgr2=0x7fe1f40ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f414fc90 0x7fe1f40ad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 -- 172.21.15.143:0/33948978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f41576e0 con 0x7fe1d4003980 2024-01-15T18:18:34.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f9c00640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1f40b0560 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c240 tx=0x7fe1d4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40b4070 con 0x7fe1d4003980 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4001480 con 0x7fe1d4003980 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f4157930 con 0x7fe1d4003980 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40072b0 con 0x7fe1d4003980 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f4158090 con 0x7fe1d4003980 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f4067500 con 0x7fe1d4003980 2024-01-15T18:18:34.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.703+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fe1e4007410 con 0x7fe1d4003980 2024-01-15T18:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.707+0000 7fe1f1600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc067a90 0x7fe1cc069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:34.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.707+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe1e411ce70 con 0x7fe1d4003980 2024-01-15T18:18:34.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.707+0000 7fe1f9200640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc067a90 0x7fe1cc069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:34.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.707+0000 7fe1f9200640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc067a90 0x7fe1cc069f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc000c10 tx=0x7fe1dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:34.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.711+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe1e40eb860 con 0x7fe1d4003980 2024-01-15T18:18:34.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:34.915+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_3_clones_are_cancelled"]} v 0) v1 -- 0x7fe1f406e4e0 con 0x7fe1d4003980 2024-01-15T18:18:35.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1f1600640 1 -- 172.21.15.143:0/33948978 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_3_clones_are_cancelled"]}]=0 v637) v1 ==== 158+0+0 (secure 0 0 0) 0x7fe1e40eee80 con 0x7fe1d4003980 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc067a90 msgr2=0x7fe1cc069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc067a90 0x7fe1cc069f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc000c10 tx=0x7fe1dc009040 comp rx=0 tx=0).stop 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 msgr2=0x7fe1f40b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1f40b0560 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe1e400c240 tx=0x7fe1d4005800 comp rx=0 tx=0).stop 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 shutdown_connections 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe1cc067a90 0x7fe1cc069f50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe1f414fc90 0x7fe1f40ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe1f4152300 0x7fe1f41f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 --2- 172.21.15.143:0/33948978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe1d4003980 0x7fe1f40b0560 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 >> 172.21.15.143:0/33948978 conn(0x7fe1f406c3f0 msgr2=0x7fe1f406d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:35.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 shutdown_connections 2024-01-15T18:18:35.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:35.875+0000 7fe1fc600640 1 -- 172.21.15.143:0/33948978 wait complete. 2024-01-15T18:18:35.895 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:18:36.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6879200640 1 Processor -- start 2024-01-15T18:18:36.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6879200640 1 -- start start 2024-01-15T18:18:36.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6879200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 0x7f68740cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6879200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68740ceb50 con 0x7f68740722c0 2024-01-15T18:18:36.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6879200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687415c110 con 0x7f68740ce520 2024-01-15T18:18:36.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6879200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687415b7e0 con 0x7f6874143780 2024-01-15T18:18:36.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6872800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68740722c0 0x7f687415c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49288/0 (socket says 172.21.15.143:49288) 2024-01-15T18:18:36.039 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6872800640 1 -- 172.21.15.143:0/2646763263 learned_addr learned my addr 172.21.15.143:0/2646763263 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:36.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6871e00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 0x7f68740cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868217266 0 0) 0x7f686c0014a0 con 0x7f68740722c0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.043+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6854003530 con 0x7f68740722c0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880262015 0 0) 0x7f6854003530 con 0x7f68740722c0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 msgr2=0x7f68740cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 0x7f68740cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 >> v1:172.21.15.204:6789/0 conn(0x7f68740ce520 legacy=0x7f6874071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687415bac0 con 0x7f68740722c0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f686c001b90 con 0x7f68740722c0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 >> v1:172.21.15.143:6789/0 conn(0x7f68740722c0 legacy=0x7f687415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68540039b0 0x7f6854003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 0x7f68540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854004d10 0x7f6854005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687415b7e0 con 0x7f68540039b0 2024-01-15T18:18:36.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685400dc50 con 0x7f6854004d10 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685400ddc0 con 0x7f6874143780 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68540039b0 0x7f6854003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871400640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 0x7f68540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6872800640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854004d10 0x7f6854005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 -- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 msgr2=0x7f68540047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6874143780 0x7f68540047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 -- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854004d10 msgr2=0x7f6854005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6854004d10 0x7f6854005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 -- 172.21.15.143:0/2646763263 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854003530 con 0x7f68540039b0 2024-01-15T18:18:36.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6871e00640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68540039b0 0x7f6854003d90 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6864009ae0 tx=0x7f686401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f828c480185a1daf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:36.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68640a5640 con 0x7f68540039b0 2024-01-15T18:18:36.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68640a57a0 con 0x7f68540039b0 2024-01-15T18:18:36.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.047+0000 7f6870a00640 1 -- 172.21.15.143:0/2646763263 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68640a5b50 con 0x7f68540039b0 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 -- 172.21.15.143:0/2646763263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68540039b0 msgr2=0x7f6854003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 --2- 172.21.15.143:0/2646763263 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68540039b0 0x7f6854003d90 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f6864009ae0 tx=0x7f686401ca50 comp rx=0 tx=0).stop 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6872800640 1 -- 172.21.15.143:0/2646763263 reap_dead start 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 -- 172.21.15.143:0/2646763263 shutdown_connections 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 -- 172.21.15.143:0/2646763263 >> 172.21.15.143:0/2646763263 conn(0x7f687406c3f0 msgr2=0x7f687406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 -- 172.21.15.143:0/2646763263 shutdown_connections 2024-01-15T18:18:36.045 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 -- 172.21.15.143:0/2646763263 wait complete. 2024-01-15T18:18:36.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 Processor -- start 2024-01-15T18:18:36.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 -- start start 2024-01-15T18:18:36.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68540039b0 0x7f68741598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 0x7f68740c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.051+0000 7f6879200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68740c8b10 0x7f68740c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6879200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68740c5870 con 0x7f68740c4ed0 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6879200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68740c94d0 con 0x7f68540039b0 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6879200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68740c6120 con 0x7f68740c8b10 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 0x7f68740c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 0x7f68740c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48558/0 (socket says 172.21.15.143:48558) 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 -- 172.21.15.143:0/3343716323 learned_addr learned my addr 172.21.15.143:0/3343716323 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6872800640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68740c8b10 0x7f68740c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871e00640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68540039b0 0x7f68741598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 -- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68740c8b10 msgr2=0x7f68740c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68740c8b10 0x7f68740c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 -- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68540039b0 msgr2=0x7f68741598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68540039b0 0x7f68741598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 -- 172.21.15.143:0/3343716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68740c6f80 con 0x7f68740c4ed0 2024-01-15T18:18:36.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6871400640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 0x7f68740c5330 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f685c011bb0 tx=0x7f685c00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c00fc50 con 0x7f68740c4ed0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c00fdb0 con 0x7f68740c4ed0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c005310 con 0x7f68740c4ed0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68740c6370 con 0x7f68740c4ed0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68740c6930 con 0x7f68740c4ed0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68740c7110 con 0x7f68740c4ed0 2024-01-15T18:18:36.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.055+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f685c003600 con 0x7f68740c4ed0 2024-01-15T18:18:36.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.059+0000 7f686a000640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f684c067a60 0x7f684c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.059+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f685c086c40 con 0x7f68740c4ed0 2024-01-15T18:18:36.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.059+0000 7f6871e00640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f684c067a60 0x7f684c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.059+0000 7f6871e00640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f684c067a60 0x7f684c069f20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f68640af000 tx=0x7f68640046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:36.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.063+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f685c0555a0 con 0x7f68740c4ed0 2024-01-15T18:18:36.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.211+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f68740b4cb0 con 0x7f68740c4ed0 2024-01-15T18:18:36.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.215+0000 7f686a000640 1 -- 172.21.15.143:0/3343716323 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f685c019070 con 0x7f68740c4ed0 2024-01-15T18:18:36.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f684c067a60 msgr2=0x7f684c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f684c067a60 0x7f684c069f20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f68640af000 tx=0x7f68640046d0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 msgr2=0x7f68740c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 0x7f68740c5330 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f685c011bb0 tx=0x7f685c00f5a0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 shutdown_connections 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f684c067a60 0x7f684c069f20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68740c8b10 0x7f68740c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68740c4ed0 0x7f68740c5330 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 --2- 172.21.15.143:0/3343716323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68540039b0 0x7f68741598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 >> 172.21.15.143:0/3343716323 conn(0x7f687406c3f0 msgr2=0x7f68740af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 shutdown_connections 2024-01-15T18:18:36.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.219+0000 7f6879200640 1 -- 172.21.15.143:0/3343716323 wait complete. 2024-01-15T18:18:36.233 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:18:36.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:18:36.319 DEBUG:teuthology.orchestra.run.smithi204:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:18:36.396 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:18:36.397 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:18:36.397 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:36.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:18:36.406 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:18:36.406 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:36.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:18:36.416 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:18:36.416 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:36.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:18:36.460 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:18:36.460 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:36.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:18:36.470 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:18:36.470 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:18:36.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.647+0000 7f27cac00640 1 Processor -- start 2024-01-15T18:18:36.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.647+0000 7f27cac00640 1 -- start start 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27cac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27c4159b70 0x7f27c40b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27cac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c40b4d20 con 0x7f27c4157300 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27cac00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c40b2a20 con 0x7f27c40b48f0 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27cac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c40b35a0 con 0x7f27c4159b70 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27c3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27c4159b70 0x7f27c40b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27c3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27c4159b70 0x7f27c40b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48566/0 (socket says 172.21.15.143:48566) 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27c3400640 1 -- 172.21.15.143:0/2751897542 learned_addr learned my addr 172.21.15.143:0/2751897542 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:36.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27c3400640 1 -- 172.21.15.143:0/2751897542 >> v1:172.21.15.204:6789/0 conn(0x7f27c40b48f0 legacy=0x7f27c4070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:36.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27c3400640 1 -- 172.21.15.143:0/2751897542 >> v1:172.21.15.143:6789/0 conn(0x7f27c4157300 legacy=0x7f27c40b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.651+0000 7f27c3400640 1 -- 172.21.15.143:0/2751897542 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c40b2d00 con 0x7f27c4159b70 2024-01-15T18:18:36.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27c3400640 1 --2- 172.21.15.143:0/2751897542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4159b70 0x7f27c40b43b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009870 tx=0x7f27ac01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca633ac0c7c2dd3b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:36.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27c2a00640 1 -- 172.21.15.143:0/2751897542 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0aa670 con 0x7f27c4159b70 2024-01-15T18:18:36.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27c2a00640 1 -- 172.21.15.143:0/2751897542 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac0aa7d0 con 0x7f27c4159b70 2024-01-15T18:18:36.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27c2a00640 1 -- 172.21.15.143:0/2751897542 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0aab60 con 0x7f27c4159b70 2024-01-15T18:18:36.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 -- 172.21.15.143:0/2751897542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4159b70 msgr2=0x7f27c40b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 --2- 172.21.15.143:0/2751897542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4159b70 0x7f27c40b43b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009870 tx=0x7f27ac01c510 comp rx=0 tx=0).stop 2024-01-15T18:18:36.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 -- 172.21.15.143:0/2751897542 shutdown_connections 2024-01-15T18:18:36.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 --2- 172.21.15.143:0/2751897542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4159b70 0x7f27c40b43b0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 -- 172.21.15.143:0/2751897542 >> 172.21.15.143:0/2751897542 conn(0x7f27c406c3f0 msgr2=0x7f27c406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:36.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 -- 172.21.15.143:0/2751897542 shutdown_connections 2024-01-15T18:18:36.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 -- 172.21.15.143:0/2751897542 wait complete. 2024-01-15T18:18:36.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 Processor -- start 2024-01-15T18:18:36.651 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.655+0000 7f27cac00640 1 -- start start 2024-01-15T18:18:36.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40b48f0 0x7f27c4141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 0x7f27c40afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4159b70 0x7f27c41416f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c4159fd0 con 0x7f27c4157300 2024-01-15T18:18:36.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c40b0160 con 0x7f27c4159b70 2024-01-15T18:18:36.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c415c1d0 con 0x7f27c40b48f0 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 0x7f27c40afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 0x7f27c40afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48578/0 (socket says 172.21.15.143:48578) 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 -- 172.21.15.143:0/1296680727 learned_addr learned my addr 172.21.15.143:0/1296680727 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3e00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40b48f0 0x7f27c4141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 -- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40b48f0 msgr2=0x7f27c4141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40b48f0 0x7f27c4141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 -- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4159b70 msgr2=0x7f27c41416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c8c00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4159b70 0x7f27c41416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4159b70 0x7f27c41416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 -- 172.21.15.143:0/1296680727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0094d0 con 0x7f27c4157300 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3e00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40b48f0 0x7f27c4141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c8c00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4159b70 0x7f27c41416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27c3400640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 0x7f27c40afad0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f27ac019120 tx=0x7f27ac01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0b3800 con 0x7f27c4157300 2024-01-15T18:18:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac0b3980 con 0x7f27c4157300 2024-01-15T18:18:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c415b600 con 0x7f27c4157300 2024-01-15T18:18:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0b3d30 con 0x7f27c4157300 2024-01-15T18:18:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c415bb80 con 0x7f27c4157300 2024-01-15T18:18:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c4067500 con 0x7f27c4157300 2024-01-15T18:18:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.659+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f27ac0c6390 con 0x7f27c4157300 2024-01-15T18:18:36.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.663+0000 7f27bbe00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f279c067a90 0x7f279c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:36.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.663+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f27ac11dfb0 con 0x7f27c4157300 2024-01-15T18:18:36.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.663+0000 7f27c3e00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f279c067a90 0x7f279c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:36.658 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.663+0000 7f27c3e00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f279c067a90 0x7f279c069f50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f27c414c2b0 tx=0x7f27b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:36.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.667+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f27ac0ec950 con 0x7f27c4157300 2024-01-15T18:18:36.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27c406e4e0 con 0x7f27c4157300 2024-01-15T18:18:36.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27bbe00640 1 -- 172.21.15.143:0/1296680727 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f27ac0eff70 con 0x7f27c4157300 2024-01-15T18:18:36.841 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:36.842 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7280,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/1910039233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":1910039233},{"type":"v1","addr":"172.21.15.204:6835","nonce":1910039233}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/2986890926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":2986890926},{"type":"v1","addr":"172.21.15.143:6837","nonce":2986890926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2769510067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2769510067},{"type":"v1","addr":"172.21.15.204:6834","nonce":2769510067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/1268740903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":1268740903},{"type":"v1","addr":"172.21.15.143:6835","nonce":1268740903}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65}],"filesystems":[]} 2024-01-15T18:18:36.842 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 65 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f279c067a90 msgr2=0x7f279c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f279c067a90 0x7f279c069f50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f27c414c2b0 tx=0x7f27b4009040 comp rx=0 tx=0).stop 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 msgr2=0x7f27c40afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 0x7f27c40afad0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f27ac019120 tx=0x7f27ac01ad50 comp rx=0 tx=0).stop 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 shutdown_connections 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f279c067a90 0x7f279c069f50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27c4159b70 0x7f27c41416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27c4157300 0x7f27c40afad0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 --2- 172.21.15.143:0/1296680727 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27c40b48f0 0x7f27c4141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 >> 172.21.15.143:0/1296680727 conn(0x7f27c406c3f0 msgr2=0x7f27c4167570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 shutdown_connections 2024-01-15T18:18:36.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:36.847+0000 7f27cac00640 1 -- 172.21.15.143:0/1296680727 wait complete. 2024-01-15T18:18:36.864 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:18:36.865 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:18:36.865 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:18:36.865 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:18:36.865 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:18:36.865 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:18:36.865 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:18:36.865 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:18:36.866 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:18:36.866 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:18:36.866 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:18:36.866 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:18:36.866 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:18:36.866 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:18:36.866 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:18:36.866 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:18:36.877 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:18:36.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:18:36.878 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:18:36.878 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:18:36.882 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:18:36.883 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:18:36.883 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:18:36.883 DEBUG:teuthology.orchestra.run.smithi204:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:18:36.927 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:18:36.927 DEBUG:teuthology.parallel:result is None 2024-01-15T18:18:36.928 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:18:36.928 DEBUG:teuthology.parallel:result is None 2024-01-15T18:18:36.930 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:18:36.930 DEBUG:teuthology.parallel:result is None 2024-01-15T18:18:36.931 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:18:36.931 DEBUG:teuthology.parallel:result is None 2024-01-15T18:18:36.932 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:18:37.013 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2024-01-15T18:18:37.013 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2024-01-15T18:18:37.021 INFO:tasks.ceph.mds.d.smithi204.stdout:starting mds.d at 2024-01-15T18:18:37.021 INFO:tasks.ceph.mds.b.smithi204.stdout:starting mds.b at 2024-01-15T18:18:37.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 Processor -- start 2024-01-15T18:18:37.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 -- start start 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec157300 0x7f20ec159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f20ec0b21d0 0x7f20ec0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec0b2a20 con 0x7f20ec0b48f0 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0b35a0 con 0x7f20ec0b21d0 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f3000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec0b4d80 con 0x7f20ec157300 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f1000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f20ec0b48f0 0x7f20ec15d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:49320/0 (socket says 172.21.15.143:49320) 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20f1000640 1 -- 172.21.15.143:0/1965150980 learned_addr learned my addr 172.21.15.143:0/1965150980 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20ebe00640 1 --2- 172.21.15.143:0/1965150980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec157300 0x7f20ec159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:37.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20eb400640 1 --2- 172.21.15.143:0/1965150980 >> v2:172.21.15.204:3300/0 conn(0x7f20ec0b21d0 0x7f20ec0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:37.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20eaa00640 1 -- 172.21.15.143:0/1965150980 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881803776 0 0) 0x7f20ec0b2a20 con 0x7f20ec0b48f0 2024-01-15T18:18:37.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20eaa00640 1 -- 172.21.15.143:0/1965150980 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc003530 con 0x7f20ec0b48f0 2024-01-15T18:18:37.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20ebe00640 1 -- 172.21.15.143:0/1965150980 >> v2:172.21.15.204:3300/0 conn(0x7f20ec0b21d0 msgr2=0x7f20ec0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:37.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20ebe00640 1 --2- 172.21.15.143:0/1965150980 >> v2:172.21.15.204:3300/0 conn(0x7f20ec0b21d0 0x7f20ec0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20ebe00640 1 -- 172.21.15.143:0/1965150980 >> v1:172.21.15.143:6789/0 conn(0x7f20ec0b48f0 legacy=0x7f20ec15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:37.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.083+0000 7f20ebe00640 1 -- 172.21.15.143:0/1965150980 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0b2d00 con 0x7f20ec157300 2024-01-15T18:18:37.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20ebe00640 1 --2- 172.21.15.143:0/1965150980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec157300 0x7f20ec159b70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00a960 tx=0x7f20dc01d820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e7f944959c33d27 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:37.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20eaa00640 1 -- 172.21.15.143:0/1965150980 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0a68d0 con 0x7f20ec157300 2024-01-15T18:18:37.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20eaa00640 1 -- 172.21.15.143:0/1965150980 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc0a6a30 con 0x7f20ec157300 2024-01-15T18:18:37.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20eaa00640 1 -- 172.21.15.143:0/1965150980 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0a6da0 con 0x7f20ec157300 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 -- 172.21.15.143:0/1965150980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec157300 msgr2=0x7f20ec159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 --2- 172.21.15.143:0/1965150980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec157300 0x7f20ec159b70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00a960 tx=0x7f20dc01d820 comp rx=0 tx=0).stop 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 -- 172.21.15.143:0/1965150980 shutdown_connections 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 --2- 172.21.15.143:0/1965150980 >> v2:172.21.15.204:3300/0 conn(0x7f20ec0b21d0 0x7f20ec0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 --2- 172.21.15.143:0/1965150980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec157300 0x7f20ec159b70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 -- 172.21.15.143:0/1965150980 >> 172.21.15.143:0/1965150980 conn(0x7f20ec06c3f0 msgr2=0x7f20ec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 -- 172.21.15.143:0/1965150980 shutdown_connections 2024-01-15T18:18:37.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.087+0000 7f20f3000640 1 -- 172.21.15.143:0/1965150980 wait complete. 2024-01-15T18:18:37.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 Processor -- start 2024-01-15T18:18:37.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 -- start start 2024-01-15T18:18:37.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec0b21d0 0x7f20ec147a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec0b48f0 0x7f20ec158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 0x7f20ec158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec1507d0 con 0x7f20ec157300 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec150ab0 con 0x7f20ec0b48f0 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec150d90 con 0x7f20ec0b21d0 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 0x7f20ec158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:37.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec0b21d0 0x7f20ec147a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 0x7f20ec158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48632/0 (socket says 172.21.15.143:48632) 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f1000640 1 -- 172.21.15.143:0/199988181 learned_addr learned my addr 172.21.15.143:0/199988181 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20eb400640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec0b48f0 0x7f20ec158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 -- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec0b48f0 msgr2=0x7f20ec158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec0b48f0 0x7f20ec158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 -- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 msgr2=0x7f20ec158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 0x7f20ec158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 -- 172.21.15.143:0/199988181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc00a5c0 con 0x7f20ec0b21d0 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f1000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 0x7f20ec158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:18:37.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20ebe00640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec0b21d0 0x7f20ec147a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0ad000 tx=0x7f20dc01a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc0bd070 con 0x7f20ec0b21d0 2024-01-15T18:18:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc004250 con 0x7f20ec0b21d0 2024-01-15T18:18:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec151070 con 0x7f20ec0b21d0 2024-01-15T18:18:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc003620 con 0x7f20ec0b21d0 2024-01-15T18:18:37.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.091+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec151870 con 0x7f20ec0b21d0 2024-01-15T18:18:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.095+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec067500 con 0x7f20ec0b21d0 2024-01-15T18:18:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.095+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f20dc0b5040 con 0x7f20ec0b21d0 2024-01-15T18:18:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.095+0000 7f20e3e00640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20c4067a60 0x7f20c4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.095+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f20dc11d0f0 con 0x7f20ec0b21d0 2024-01-15T18:18:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.095+0000 7f20eb400640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20c4067a60 0x7f20c4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:37.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.095+0000 7f20eb400640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20c4067a60 0x7f20c4069f20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f20ec156170 tx=0x7f20d4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.099+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f20dc0ebae0 con 0x7f20ec0b21d0 2024-01-15T18:18:37.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.259+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f20ec06e4e0 con 0x7f20ec0b21d0 2024-01-15T18:18:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.899+0000 7f20e3e00640 1 -- 172.21.15.143:0/199988181 <== mon.2 v2:172.21.15.204:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7f20dc0b3020 con 0x7f20ec0b21d0 2024-01-15T18:18:37.894 INFO:teuthology.orchestra.run.smithi143.stderr: removed all blocklist entries 2024-01-15T18:18:37.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20c4067a60 msgr2=0x7f20c4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:37.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20c4067a60 0x7f20c4069f20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f20ec156170 tx=0x7f20d4009190 comp rx=0 tx=0).stop 2024-01-15T18:18:37.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec0b21d0 msgr2=0x7f20ec147a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:37.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec0b21d0 0x7f20ec147a70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0ad000 tx=0x7f20dc01a5c0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 shutdown_connections 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f20c4067a60 0x7f20c4069f20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ec157300 0x7f20ec158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f20ec0b48f0 0x7f20ec158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 --2- 172.21.15.143:0/199988181 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f20ec0b21d0 0x7f20ec147a70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 >> 172.21.15.143:0/199988181 conn(0x7f20ec06c3f0 msgr2=0x7f20ec14b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 shutdown_connections 2024-01-15T18:18:37.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:37.903+0000 7f20f3000640 1 -- 172.21.15.143:0/199988181 wait complete. 2024-01-15T18:18:37.921 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:18:38.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.063+0000 7f2918800640 1 Processor -- start 2024-01-15T18:18:38.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.063+0000 7f2918800640 1 -- start start 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2918800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2910143780 0x7f29100cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2918800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29100ceb50 con 0x7f29100ce520 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2918800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f291015c110 con 0x7f29100722c0 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2918800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291015b7e0 con 0x7f2910143780 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2915400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f29100ce520 0x7f2910071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37920/0 (socket says 172.21.15.143:37920) 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2915400640 1 -- 172.21.15.143:0/3914460831 learned_addr learned my addr 172.21.15.143:0/3914460831 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2915e00640 1 --2- 172.21.15.143:0/3914460831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2910143780 0x7f29100cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664959087 0 0) 0x7f28f8001120 con 0x7f29100ce520 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f0003530 con 0x7f29100ce520 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104419593 0 0) 0x7f291015c110 con 0x7f29100722c0 2024-01-15T18:18:38.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f8001120 con 0x7f29100722c0 2024-01-15T18:18:38.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2915e00640 1 -- 172.21.15.143:0/3914460831 >> v1:172.21.15.204:6789/0 conn(0x7f29100722c0 legacy=0x7f291015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2915e00640 1 -- 172.21.15.143:0/3914460831 >> v1:172.21.15.143:6789/0 conn(0x7f29100ce520 legacy=0x7f2910071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.063 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.067+0000 7f2915e00640 1 -- 172.21.15.143:0/3914460831 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291015bac0 con 0x7f2910143780 2024-01-15T18:18:38.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2915e00640 1 --2- 172.21.15.143:0/3914460831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2910143780 0x7f29100cdfe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f29000097f0 tx=0x7f290001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4283670277ed8f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000ac630 con 0x7f2910143780 2024-01-15T18:18:38.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29000ac790 con 0x7f2910143780 2024-01-15T18:18:38.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2914a00640 1 -- 172.21.15.143:0/3914460831 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000acb20 con 0x7f2910143780 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 -- 172.21.15.143:0/3914460831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2910143780 msgr2=0x7f29100cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 --2- 172.21.15.143:0/3914460831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2910143780 0x7f29100cdfe0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f29000097f0 tx=0x7f290001c4c0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 -- 172.21.15.143:0/3914460831 shutdown_connections 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 --2- 172.21.15.143:0/3914460831 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2910143780 0x7f29100cdfe0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 -- 172.21.15.143:0/3914460831 >> 172.21.15.143:0/3914460831 conn(0x7f291006c3f0 msgr2=0x7f291006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 -- 172.21.15.143:0/3914460831 shutdown_connections 2024-01-15T18:18:38.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 -- 172.21.15.143:0/3914460831 wait complete. 2024-01-15T18:18:38.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 Processor -- start 2024-01-15T18:18:38.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.071+0000 7f2918800640 1 -- start start 2024-01-15T18:18:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29100722c0 0x7f291013c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29100ce520 0x7f291013ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 0x7f2910070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291013f6c0 con 0x7f2910143780 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291013b140 con 0x7f29100722c0 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291013b3e0 con 0x7f29100ce520 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 0x7f2910070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 0x7f2910070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47800/0 (socket says 172.21.15.143:47800) 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 -- 172.21.15.143:0/3017582232 learned_addr learned my addr 172.21.15.143:0/3017582232 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2915400640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29100ce520 0x7f291013ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2915e00640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29100722c0 0x7f291013c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 -- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29100ce520 msgr2=0x7f291013ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29100ce520 0x7f291013ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 -- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29100722c0 msgr2=0x7f291013c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29100722c0 0x7f291013c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 -- 172.21.15.143:0/3017582232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2900009420 con 0x7f2910143780 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2916800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 0x7f2910070b70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f290800c6b0 tx=0x7f290800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908003b80 con 0x7f2910143780 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908003ce0 con 0x7f2910143780 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291013b6c0 con 0x7f2910143780 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908005310 con 0x7f2910143780 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291013f910 con 0x7f2910143780 2024-01-15T18:18:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.075+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2910067500 con 0x7f2910143780 2024-01-15T18:18:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.079+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f2908005470 con 0x7f2910143780 2024-01-15T18:18:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.079+0000 7f290e000640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f28e8067a60 0x7f28e8069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.079+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f2908083160 con 0x7f2910143780 2024-01-15T18:18:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.079+0000 7f2915e00640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f28e8067a60 0x7f28e8069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.079+0000 7f2915e00640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f28e8067a60 0x7f28e8069f20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f29000ae000 tx=0x7f29000a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.083+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2908051aa0 con 0x7f2910143780 2024-01-15T18:18:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.279+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f291013fad0 con 0x7f2910143780 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.279+0000 7f290e000640 1 -- 172.21.15.143:0/3017582232 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f29080550c0 con 0x7f2910143780 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: "auth_dump": [ 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.a", 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl57omBxAAQvmrICW4QKbU3Wi8YTa0cA==", 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.276 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.b", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlXYeZExAAHpvInxH9S9WW6y3DWLJSiQ==", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.277 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.c", 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlYqemDhAAZd1tpmnMqHeATUx8xbjgxg==", 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mds.d", 2024-01-15T18:18:38.278 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VljhgSGxAAXTfpeGMeabYT2/eO8RoYpw==", 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.0", 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCSc6VlgukBORAABw7uBOg+rOXpnefhQsTZMQ==", 2024-01-15T18:18:38.279 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.1", 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCWc6Vli1cDBhAApbPh1qYNkxWb2j9aH5lC0Q==", 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.280 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.2", 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCZc6VlrSXjDBAARElSUcYa3PA7Qgb76I0HMg==", 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.3", 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCcc6VlmhzoEhAAAtYhakfzVuFpdZmCHJ7/YQ==", 2024-01-15T18:18:38.281 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.4", 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCic6Vl9zrIFxAASYlaZC3ulwtJ3plA/4KzkA==", 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.282 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.5", 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQClc6VlADWoHRAA0zpYxc641zRpMDZnTpNPwg==", 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.6", 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCoc6VlRDbAIxAAIvP4Peok72F5wSvW8TneoQ==", 2024-01-15T18:18:38.283 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "osd.7", 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCrc6VlJgfXKhAA8P6qraWOKevJ4ElGzlDKaA==", 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow profile osd", 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile osd", 2024-01-15T18:18:38.284 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.0", 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A==", 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow r", 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.1", 2024-01-15T18:18:38.285 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlsv++KBAAIB5fncQ+ArTYE2UjdSlJAw==", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.2", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlP9RLJRAAPKC8gGTaRA5MCRnKu6rvVg==", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:18:38.286 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.3", 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6Vlp37DLhAAfHK6jNXd9DLcwHL9TK66TQ==", 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow", 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow r", 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow rw", 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow rwx" 2024-01-15T18:18:38.287 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "client.admin", 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vl9xWiJhAAYLwyGFqN9BckYAd2ROnvWA==", 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mgr": "allow *", 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow *", 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.288 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.x", 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCPc6VlPq9cAhAAbATaGLUZ1qLCI4EHwTA6BA==", 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:38.289 INFO:teuthology.orchestra.run.smithi143.stdout: "entity": "mgr.y", 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: "key": "AQCOc6Vlujo3OhAAvGgcWlWkOXqZtaQdEv1wGg==", 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": { 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": "allow *", 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: "mon": "allow profile mgr", 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: "osd": "allow *" 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-01-15T18:18:38.290 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:18:38.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f28e8067a60 msgr2=0x7f28e8069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f28e8067a60 0x7f28e8069f20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f29000ae000 tx=0x7f29000a7000 comp rx=0 tx=0).stop 2024-01-15T18:18:38.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 msgr2=0x7f2910070b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 0x7f2910070b70 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f290800c6b0 tx=0x7f290800cb70 comp rx=0 tx=0).stop 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 shutdown_connections 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f28e8067a60 0x7f28e8069f20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2910143780 0x7f2910070b70 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f29100ce520 0x7f291013ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 --2- 172.21.15.143:0/3017582232 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f29100722c0 0x7f291013c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 >> 172.21.15.143:0/3017582232 conn(0x7f291006c3f0 msgr2=0x7f29100b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 shutdown_connections 2024-01-15T18:18:38.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.283+0000 7f2918800640 1 -- 172.21.15.143:0/3017582232 wait complete. 2024-01-15T18:18:38.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.439+0000 7f142e800640 1 Processor -- start 2024-01-15T18:18:38.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.439+0000 7f142e800640 1 -- start start 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f142e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14280ce560 0x7f142815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f142e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f1428071d00 0x7f1428072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f142e800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1428072650 con 0x7f1428141660 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f142e800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142815fbb0 con 0x7f14280ce560 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f142e800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142815f310 con 0x7f1428071d00 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14280ce560 0x7f142815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f14280ce560 0x7f142815ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47816/0 (socket says 172.21.15.143:47816) 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1427400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1428141660 0x7f1428141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37934/0 (socket says 172.21.15.143:37934) 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 -- 172.21.15.143:0/885229385 learned_addr learned my addr 172.21.15.143:0/885229385 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1427e00640 1 --2- 172.21.15.143:0/885229385 >> v2:172.21.15.204:3300/0 conn(0x7f1428071d00 0x7f1428072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 -- 172.21.15.143:0/885229385 >> v2:172.21.15.204:3300/0 conn(0x7f1428071d00 msgr2=0x7f1428072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 --2- 172.21.15.143:0/885229385 >> v2:172.21.15.204:3300/0 conn(0x7f1428071d00 0x7f1428072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 -- 172.21.15.143:0/885229385 >> v1:172.21.15.143:6789/0 conn(0x7f1428141660 legacy=0x7f1428141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 -- 172.21.15.143:0/885229385 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142815f5f0 con 0x7f14280ce560 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426000640 1 -- 172.21.15.143:0/885229385 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791708349 0 0) 0x7f1418001120 con 0x7f1428141660 2024-01-15T18:18:38.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1427e00640 1 --2- 172.21.15.143:0/885229385 >> v2:172.21.15.204:3300/0 conn(0x7f1428071d00 0x7f1428072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.443+0000 7f1426a00640 1 --2- 172.21.15.143:0/885229385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f142815ec80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f142806e250 tx=0x7f141001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47d78e4412453d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f1426000640 1 -- 172.21.15.143:0/885229385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100ab5f0 con 0x7f14280ce560 2024-01-15T18:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f1426000640 1 -- 172.21.15.143:0/885229385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14100ab750 con 0x7f14280ce560 2024-01-15T18:18:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f1426000640 1 -- 172.21.15.143:0/885229385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100abae0 con 0x7f14280ce560 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- 172.21.15.143:0/885229385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 msgr2=0x7f142815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 --2- 172.21.15.143:0/885229385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f142815ec80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f142806e250 tx=0x7f141001c500 comp rx=0 tx=0).stop 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- 172.21.15.143:0/885229385 shutdown_connections 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 --2- 172.21.15.143:0/885229385 >> v2:172.21.15.204:3300/0 conn(0x7f1428071d00 0x7f1428072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 --2- 172.21.15.143:0/885229385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f142815ec80 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- 172.21.15.143:0/885229385 >> 172.21.15.143:0/885229385 conn(0x7f142806c3f0 msgr2=0x7f142806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:38.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- 172.21.15.143:0/885229385 shutdown_connections 2024-01-15T18:18:38.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- 172.21.15.143:0/885229385 wait complete. 2024-01-15T18:18:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 Processor -- start 2024-01-15T18:18:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- start start 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1428071d00 0x7f14280b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f1428136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1428141660 0x7f142815b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14280b2d50 con 0x7f14280ce560 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14280b2140 con 0x7f1428071d00 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.447+0000 7f142e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14280b2420 con 0x7f1428141660 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f1428136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f1428136ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47832/0 (socket says 172.21.15.143:47832) 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 -- 172.21.15.143:0/2439120907 learned_addr learned my addr 172.21.15.143:0/2439120907 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1427400640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1428071d00 0x7f14280b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1427e00640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1428141660 0x7f142815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 -- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1428141660 msgr2=0x7f142815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1428141660 0x7f142815b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 -- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1428071d00 msgr2=0x7f14280b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1428071d00 0x7f14280b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 -- 172.21.15.143:0/2439120907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14100094d0 con 0x7f14280ce560 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f1426a00640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f1428136ed0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1410019110 tx=0x7f141001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100be5f0 con 0x7f14280ce560 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14100be750 con 0x7f14280ce560 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14100beb00 con 0x7f14280ce560 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14280cd150 con 0x7f14280ce560 2024-01-15T18:18:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14280b29e0 con 0x7f14280ce560 2024-01-15T18:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1428067500 con 0x7f14280ce560 2024-01-15T18:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.451+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f14100b43f0 con 0x7f14280ce560 2024-01-15T18:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.455+0000 7f141f400640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1400067a60 0x7f1400069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.455+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f14100a8050 con 0x7f14280ce560 2024-01-15T18:18:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.455+0000 7f1427400640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1400067a60 0x7f1400069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.455+0000 7f1427400640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1400067a60 0x7f1400069f20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1418000f70 tx=0x7f1418007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.459+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f14100ec850 con 0x7f14280ce560 2024-01-15T18:18:38.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.607+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1428144e60 con 0x7f14280ce560 2024-01-15T18:18:38.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.607+0000 7f141f400640 1 -- 172.21.15.143:0/2439120907 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+12983 (secure 0 0 0) 0x7f14100c4070 con 0x7f14280ce560 2024-01-15T18:18:38.603 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:38.603 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":107,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:37.906350+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":98,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:38.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1400067a60 msgr2=0x7f1400069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1400067a60 0x7f1400069f20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1418000f70 tx=0x7f1418007df0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 msgr2=0x7f1428136ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f1428136ed0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1410019110 tx=0x7f141001ad40 comp rx=0 tx=0).stop 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 shutdown_connections 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1400067a60 0x7f1400069f20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1428141660 0x7f142815b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f14280ce560 0x7f1428136ed0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 --2- 172.21.15.143:0/2439120907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1428071d00 0x7f14280b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 >> 172.21.15.143:0/2439120907 conn(0x7f142806c3f0 msgr2=0x7f142806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 shutdown_connections 2024-01-15T18:18:38.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.611+0000 7f142e800640 1 -- 172.21.15.143:0/2439120907 wait complete. 2024-01-15T18:18:38.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:18:38.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:18:38.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c3600640 1 Processor -- start 2024-01-15T18:18:38.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c3600640 1 -- start start 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c3600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05bc06bfb0 0x7f05bc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c3600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0cbff0 con 0x7f05bc0b0310 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c3600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0d0c50 con 0x7f05bc0d0210 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c3600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc06c8d0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c0c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05bc06bfb0 0x7f05bc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05bbe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f05bc0b0310 0x7f05bc0cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37948/0 (socket says 172.21.15.143:37948) 2024-01-15T18:18:38.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05bbe00640 1 -- 172.21.15.143:0/3090691968 learned_addr learned my addr 172.21.15.143:0/3090691968 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:38.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c0c00640 1 -- 172.21.15.143:0/3090691968 >> v1:172.21.15.204:6789/0 conn(0x7f05bc0d0210 legacy=0x7f05bc0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:38.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c0c00640 1 -- 172.21.15.143:0/3090691968 >> v1:172.21.15.143:6789/0 conn(0x7f05bc0b0310 legacy=0x7f05bc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.771+0000 7f05c0c00640 1 -- 172.21.15.143:0/3090691968 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0cb6c0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c0c00640 1 --2- 172.21.15.143:0/3090691968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc06c390 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097c0 tx=0x7f05ac01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5497e013ab921d81 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05bb400640 1 -- 172.21.15.143:0/3090691968 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0ad680 con 0x7f05bc06bfb0 2024-01-15T18:18:38.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05bb400640 1 -- 172.21.15.143:0/3090691968 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0ad7e0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05bb400640 1 -- 172.21.15.143:0/3090691968 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0adb70 con 0x7f05bc06bfb0 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 -- 172.21.15.143:0/3090691968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 msgr2=0x7f05bc06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 --2- 172.21.15.143:0/3090691968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc06c390 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0097c0 tx=0x7f05ac01c380 comp rx=0 tx=0).stop 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 -- 172.21.15.143:0/3090691968 shutdown_connections 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 --2- 172.21.15.143:0/3090691968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc06c390 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 -- 172.21.15.143:0/3090691968 >> 172.21.15.143:0/3090691968 conn(0x7f05bc0d0ed0 msgr2=0x7f05bc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 -- 172.21.15.143:0/3090691968 shutdown_connections 2024-01-15T18:18:38.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.775+0000 7f05c3600640 1 -- 172.21.15.143:0/3090691968 wait complete. 2024-01-15T18:18:38.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 Processor -- start 2024-01-15T18:18:38.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 -- start start 2024-01-15T18:18:38.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc0c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05bc0b0310 0x7f05bc153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05bc0d0210 0x7f05bc15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c5710 con 0x7f05bc06bfb0 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c59f0 con 0x7f05bc0b0310 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c3600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0c5ee0 con 0x7f05bc0d0210 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc0c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc0c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47860/0 (socket says 172.21.15.143:47860) 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 -- 172.21.15.143:0/2665883524 learned_addr learned my addr 172.21.15.143:0/2665883524 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c1600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05bc0d0210 0x7f05bc15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05bbe00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05bc0b0310 0x7f05bc153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 -- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05bc0d0210 msgr2=0x7f05bc15c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05bc0d0210 0x7f05bc15c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 -- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05bc0b0310 msgr2=0x7f05bc153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05bc0b0310 0x7f05bc153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 -- 172.21.15.143:0/2665883524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009420 con 0x7f05bc06bfb0 2024-01-15T18:18:38.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05c0c00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc0c69c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f05ac018f90 tx=0x7f05ac01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0adea0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.779+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0b8450 con 0x7f05bc06bfb0 2024-01-15T18:18:38.777 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc0c61c0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0b8820 con 0x7f05bc06bfb0 2024-01-15T18:18:38.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0c6f00 con 0x7f05bc06bfb0 2024-01-15T18:18:38.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc067500 con 0x7f05bc06bfb0 2024-01-15T18:18:38.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f05ac0b4070 con 0x7f05bc06bfb0 2024-01-15T18:18:38.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05b8c00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0594067a60 0x7f0594069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:38.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f05ac11d650 con 0x7f05bc06bfb0 2024-01-15T18:18:38.779 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.783+0000 7f05bbe00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0594067a60 0x7f0594069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:38.780 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.787+0000 7f05bbe00640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0594067a60 0x7f0594069f20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0b0820 tx=0x7f05a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:38.783 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.787+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05ac0ec3f0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.947+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05bc0d1db0 con 0x7f05bc06bfb0 2024-01-15T18:18:38.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.947+0000 7f05b8c00640 1 -- 172.21.15.143:0/2665883524 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05ac0bc020 con 0x7f05bc06bfb0 2024-01-15T18:18:38.944 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0594067a60 msgr2=0x7f0594069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0594067a60 0x7f0594069f20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f05bc0b0820 tx=0x7f05a4009040 comp rx=0 tx=0).stop 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 msgr2=0x7f05bc0c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc0c69c0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f05ac018f90 tx=0x7f05ac01abc0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 shutdown_connections 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0594067a60 0x7f0594069f20 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05bc0d0210 0x7f05bc15c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05bc0b0310 0x7f05bc153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 --2- 172.21.15.143:0/2665883524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05bc06bfb0 0x7f05bc0c69c0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:38.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 >> 172.21.15.143:0/2665883524 conn(0x7f05bc0d0ed0 msgr2=0x7f05bc0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:38.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 shutdown_connections 2024-01-15T18:18:38.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:38.951+0000 7f05c3600640 1 -- 172.21.15.143:0/2665883524 wait complete. 2024-01-15T18:18:38.967 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:18:39.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 Processor -- start 2024-01-15T18:18:39.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 -- start start 2024-01-15T18:18:39.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f36c80722c0 0x7f36c815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c80ceb50 con 0x7f36c80ce520 2024-01-15T18:18:39.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c815c110 con 0x7f36c80722c0 2024-01-15T18:18:39.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cfa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c815b7e0 con 0x7f36c8143780 2024-01-15T18:18:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cda00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f36c80722c0 0x7f36c815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cd000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36c7e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f36c80ce520 0x7f36c8071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:53992/0 (socket says 172.21.15.143:53992) 2024-01-15T18:18:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36c7e00640 1 -- 172.21.15.143:0/1263306804 learned_addr learned my addr 172.21.15.143:0/1263306804 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cda00640 1 -- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 msgr2=0x7f36c80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cda00640 1 --2- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cda00640 1 -- 172.21.15.143:0/1263306804 >> v1:172.21.15.204:6789/0 conn(0x7f36c80ce520 legacy=0x7f36c8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.111+0000 7f36cda00640 1 -- 172.21.15.143:0/1263306804 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c815bac0 con 0x7f36c80722c0 2024-01-15T18:18:39.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cda00640 1 --2- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80722c0 0x7f36c815c5a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009d90 tx=0x7f36c001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66bdeaf36221f4df server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36c7400640 1 -- 172.21.15.143:0/1263306804 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00a5920 con 0x7f36c80722c0 2024-01-15T18:18:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36c7400640 1 -- 172.21.15.143:0/1263306804 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c00a5a80 con 0x7f36c80722c0 2024-01-15T18:18:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36c7400640 1 -- 172.21.15.143:0/1263306804 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00a5e10 con 0x7f36c80722c0 2024-01-15T18:18:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cfa00640 1 -- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80722c0 msgr2=0x7f36c815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80722c0 0x7f36c815c5a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009d90 tx=0x7f36c001c950 comp rx=0 tx=0).stop 2024-01-15T18:18:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cfa00640 1 -- 172.21.15.143:0/1263306804 shutdown_connections 2024-01-15T18:18:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80722c0 0x7f36c815c5a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1263306804 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.115+0000 7f36cfa00640 1 -- 172.21.15.143:0/1263306804 >> 172.21.15.143:0/1263306804 conn(0x7f36c806c3f0 msgr2=0x7f36c806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 -- 172.21.15.143:0/1263306804 shutdown_connections 2024-01-15T18:18:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 -- 172.21.15.143:0/1263306804 wait complete. 2024-01-15T18:18:39.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 Processor -- start 2024-01-15T18:18:39.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 -- start start 2024-01-15T18:18:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 0x7f36c80b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80ce520 0x7f36c80b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c814d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80b23b0 con 0x7f36c80722c0 2024-01-15T18:18:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80b2690 con 0x7f36c80ce520 2024-01-15T18:18:39.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cfa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80b2970 con 0x7f36c8143780 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 0x7f36c80b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 0x7f36c80b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47862/0 (socket says 172.21.15.143:47862) 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 -- 172.21.15.143:0/1814040647 learned_addr learned my addr 172.21.15.143:0/1814040647 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36c7e00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80ce520 0x7f36c80b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cda00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c814d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 -- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 msgr2=0x7f36c814d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c814d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 -- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80ce520 msgr2=0x7f36c80b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80ce520 0x7f36c80b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 -- 172.21.15.143:0/1814040647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009a70 con 0x7f36c80722c0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.119+0000 7f36cd000640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 0x7f36c80b1dd0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e870 tx=0x7f36b800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8009530 con 0x7f36c80722c0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b800c940 con 0x7f36c80722c0 2024-01-15T18:18:39.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b800ccd0 con 0x7f36c80722c0 2024-01-15T18:18:39.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c80b2c50 con 0x7f36c80722c0 2024-01-15T18:18:39.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c81419b0 con 0x7f36c80722c0 2024-01-15T18:18:39.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c8067500 con 0x7f36c80722c0 2024-01-15T18:18:39.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f36b8013470 con 0x7f36c80722c0 2024-01-15T18:18:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c4c00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36a0067a60 0x7f36a0069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f36b8083420 con 0x7f36c80722c0 2024-01-15T18:18:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c7e00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36a0067a60 0x7f36a0069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.123+0000 7f36c7e00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36a0067a60 0x7f36a0069f20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f36c81374f0 tx=0x7f36b0007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:39.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.127+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f36b8051e10 con 0x7f36c80722c0 2024-01-15T18:18:39.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.279+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f36c8142fb0 con 0x7f36c80722c0 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.279+0000 7f36c4c00640 1 -- 172.21.15.143:0/1814040647 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+65 (secure 0 0 0) 0x7f36b8021090 con 0x7f36c80722c0 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:39.276 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:18:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36a0067a60 msgr2=0x7f36a0069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36a0067a60 0x7f36a0069f20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f36c81374f0 tx=0x7f36b0007e90 comp rx=0 tx=0).stop 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 msgr2=0x7f36c80b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 0x7f36c80b1dd0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e870 tx=0x7f36b800ed30 comp rx=0 tx=0).stop 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 shutdown_connections 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f36a0067a60 0x7f36a0069f20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f36c8143780 0x7f36c814d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f36c80ce520 0x7f36c80b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 --2- 172.21.15.143:0/1814040647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36c80722c0 0x7f36c80b1dd0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 >> 172.21.15.143:0/1814040647 conn(0x7f36c806c3f0 msgr2=0x7f36c806d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 shutdown_connections 2024-01-15T18:18:39.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.283+0000 7f36cfa00640 1 -- 172.21.15.143:0/1814040647 wait complete. 2024-01-15T18:18:39.299 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:18:39.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:18:39.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.443+0000 7f6e01400640 1 Processor -- start 2024-01-15T18:18:39.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.443+0000 7f6e01400640 1 -- start start 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6e01400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 0x7f6dfc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6e01400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0cbff0 con 0x7f6dfc06bfb0 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6e01400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dfc0d0c50 con 0x7f6dfc0b0310 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6e01400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0cb6c0 con 0x7f6dfc0cf910 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6dfc06bfb0 0x7f6dfc06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37964/0 (socket says 172.21.15.143:37964) 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 -- 172.21.15.143:0/822892950 learned_addr learned my addr 172.21.15.143:0/822892950 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:39.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfb200640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 0x7f6dfc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459401846 0 0) 0x7f6dfc0cbff0 con 0x7f6dfc06bfb0 2024-01-15T18:18:39.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ddc003530 con 0x7f6dfc06bfb0 2024-01-15T18:18:39.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1716109713 0 0) 0x7f6ddc003530 con 0x7f6dfc06bfb0 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 msgr2=0x7f6dfc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 0x7f6dfc0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 >> v1:172.21.15.204:6789/0 conn(0x7f6dfc0b0310 legacy=0x7f6dfc0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0cb9a0 con 0x7f6dfc06bfb0 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f6dec001680 con 0x7f6dfc06bfb0 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 >> v1:172.21.15.143:6789/0 conn(0x7f6dfc06bfb0 legacy=0x7f6dfc06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6ddc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 0x7f6ddc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ddc004d10 0x7f6ddc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0cb6c0 con 0x7f6ddc0039b0 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc00ba70 con 0x7f6ddc004d10 2024-01-15T18:18:39.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc00bc30 con 0x7f6dfc0cf910 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6ddc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6df9e00640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 0x7f6ddc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfb200640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ddc004d10 0x7f6ddc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 -- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 msgr2=0x7f6ddc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc0cf910 0x7f6ddc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 -- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ddc004d10 msgr2=0x7f6ddc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ddc004d10 0x7f6ddc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.447+0000 7f6dfa800640 1 -- 172.21.15.143:0/822892950 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc003530 con 0x7f6ddc0039b0 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6dfa800640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6ddc003d90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00c210 tx=0x7f6dec01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4736effc32b0a27d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec005ae0 con 0x7f6ddc0039b0 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec005c60 con 0x7f6ddc0039b0 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6df9400640 1 -- 172.21.15.143:0/822892950 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0a5840 con 0x7f6ddc0039b0 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6e01400640 1 -- 172.21.15.143:0/822892950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 msgr2=0x7f6ddc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6e01400640 1 --2- 172.21.15.143:0/822892950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6ddc003d90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00c210 tx=0x7f6dec01cc30 comp rx=0 tx=0).stop 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6dfb200640 1 -- 172.21.15.143:0/822892950 reap_dead start 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6e01400640 1 -- 172.21.15.143:0/822892950 shutdown_connections 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6e01400640 1 -- 172.21.15.143:0/822892950 >> 172.21.15.143:0/822892950 conn(0x7f6dfc0d0ed0 msgr2=0x7f6dfc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6e01400640 1 -- 172.21.15.143:0/822892950 shutdown_connections 2024-01-15T18:18:39.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.451+0000 7f6e01400640 1 -- 172.21.15.143:0/822892950 wait complete. 2024-01-15T18:18:39.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 Processor -- start 2024-01-15T18:18:39.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- start start 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6dfc158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc1588d0 0x7f6dfc0b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0b6300 0x7f6dfc0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc158d50 con 0x7f6ddc0039b0 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0b6780 con 0x7f6dfc0b6300 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dfc0c72b0 con 0x7f6dfc1588d0 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6dfc158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6dfc158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47884/0 (socket says 172.21.15.143:47884) 2024-01-15T18:18:39.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 -- 172.21.15.143:0/235696840 learned_addr learned my addr 172.21.15.143:0/235696840 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6df9e00640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc1588d0 0x7f6dfc0b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfb200640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0b6300 0x7f6dfc0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 -- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc1588d0 msgr2=0x7f6dfc0b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc1588d0 0x7f6dfc0b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 -- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0b6300 msgr2=0x7f6dfc0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0b6300 0x7f6dfc0c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 -- 172.21.15.143:0/235696840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc0c6fd0 con 0x7f6ddc0039b0 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6dfa800640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6dfc158390 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00c210 tx=0x7f6ddc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0b4070 con 0x7f6ddc0039b0 2024-01-15T18:18:39.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec005dd0 con 0x7f6ddc0039b0 2024-01-15T18:18:39.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dfc0c4aa0 con 0x7f6ddc0039b0 2024-01-15T18:18:39.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec001640 con 0x7f6ddc0039b0 2024-01-15T18:18:39.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dfc0c5060 con 0x7f6ddc0039b0 2024-01-15T18:18:39.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.455+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dfc0c5250 con 0x7f6ddc0039b0 2024-01-15T18:18:39.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.459+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f6dec0a5840 con 0x7f6ddc0039b0 2024-01-15T18:18:39.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.459+0000 7f6df2a00640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6dd4067a90 0x7f6dd4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:39.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.459+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f6dec11cbe0 con 0x7f6ddc0039b0 2024-01-15T18:18:39.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.459+0000 7f6df9e00640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6dd4067a90 0x7f6dd4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:39.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.459+0000 7f6df9e00640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6dd4067a90 0x7f6dd4069f50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0ce060 tx=0x7f6de4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:39.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.463+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6dec0eb610 con 0x7f6ddc0039b0 2024-01-15T18:18:39.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.615+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6dfc0c5250 con 0x7f6ddc0039b0 2024-01-15T18:18:39.894 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.899+0000 7f6df2a00640 1 -- 172.21.15.143:0/235696840 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v108) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6dec0eec30 con 0x7f6ddc0039b0 2024-01-15T18:18:39.894 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6dd4067a90 msgr2=0x7f6dd4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6dd4067a90 0x7f6dd4069f50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0ce060 tx=0x7f6de4009040 comp rx=0 tx=0).stop 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 msgr2=0x7f6dfc158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6dfc158390 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00c210 tx=0x7f6ddc005800 comp rx=0 tx=0).stop 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 shutdown_connections 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6dd4067a90 0x7f6dd4069f50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6dfc0b6300 0x7f6dfc0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.897 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6dfc1588d0 0x7f6dfc0b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 --2- 172.21.15.143:0/235696840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6ddc0039b0 0x7f6dfc158390 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 >> 172.21.15.143:0/235696840 conn(0x7f6dfc0d0ed0 msgr2=0x7f6dfc06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 shutdown_connections 2024-01-15T18:18:39.898 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:39.903+0000 7f6e01400640 1 -- 172.21.15.143:0/235696840 wait complete. 2024-01-15T18:18:39.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:18:40.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.071+0000 7f8b5e000640 1 Processor -- start 2024-01-15T18:18:40.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.071+0000 7f8b5e000640 1 -- start start 2024-01-15T18:18:40.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b5e000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b5815d110 0x7f8b580cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b5e000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b5815d4f0 0x7f8b580cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b5e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b580cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b5e000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b580b0e80 con 0x7f8b5815d4f0 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b5e000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5806ca50 con 0x7f8b5815d110 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b5e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580d00f0 con 0x7f8b5806bf50 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b5815d4f0 0x7f8b580cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b5815d4f0 0x7f8b580cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47888/0 (socket says 172.21.15.143:47888) 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 -- 172.21.15.143:0/2336491035 learned_addr learned my addr 172.21.15.143:0/2336491035 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b57e00640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b580cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b57400640 1 --2- 172.21.15.143:0/2336491035 >> v2:172.21.15.204:3300/0 conn(0x7f8b5815d110 0x7f8b580cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 -- 172.21.15.143:0/2336491035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 msgr2=0x7f8b580cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b580cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 -- 172.21.15.143:0/2336491035 >> v2:172.21.15.204:3300/0 conn(0x7f8b5815d110 msgr2=0x7f8b580cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:40.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 --2- 172.21.15.143:0/2336491035 >> v2:172.21.15.204:3300/0 conn(0x7f8b5815d110 0x7f8b580cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b56a00640 1 -- 172.21.15.143:0/2336491035 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5806c490 con 0x7f8b5815d4f0 2024-01-15T18:18:40.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.075+0000 7f8b57e00640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b580cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:40.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b56a00640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d4f0 0x7f8b580cf670 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f8b58071640 tx=0x7f8b4001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d876b07fcea3f387 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:40.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b56000640 1 -- 172.21.15.143:0/2336491035 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400ab600 con 0x7f8b5815d4f0 2024-01-15T18:18:40.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b56000640 1 -- 172.21.15.143:0/2336491035 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400a7050 con 0x7f8b5815d4f0 2024-01-15T18:18:40.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b56000640 1 -- 172.21.15.143:0/2336491035 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400ab980 con 0x7f8b5815d4f0 2024-01-15T18:18:40.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 -- 172.21.15.143:0/2336491035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d4f0 msgr2=0x7f8b580cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d4f0 0x7f8b580cf670 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f8b58071640 tx=0x7f8b4001c470 comp rx=0 tx=0).stop 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 -- 172.21.15.143:0/2336491035 shutdown_connections 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b580cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2336491035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d4f0 0x7f8b580cf670 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2336491035 >> v2:172.21.15.204:3300/0 conn(0x7f8b5815d110 0x7f8b580cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 -- 172.21.15.143:0/2336491035 >> 172.21.15.143:0/2336491035 conn(0x7f8b580b2a40 msgr2=0x7f8b580b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 -- 172.21.15.143:0/2336491035 shutdown_connections 2024-01-15T18:18:40.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 -- 172.21.15.143:0/2336491035 wait complete. 2024-01-15T18:18:40.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 Processor -- start 2024-01-15T18:18:40.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.079+0000 7f8b5e000640 1 -- start start 2024-01-15T18:18:40.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b581554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 0x7f8b580b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b5815d4f0 0x7f8b580b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580b4120 con 0x7f8b5815d110 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580b3b30 con 0x7f8b5815d4f0 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581559e0 con 0x7f8b5806bf50 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 0x7f8b580b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 0x7f8b580b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47896/0 (socket says 172.21.15.143:47896) 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 -- 172.21.15.143:0/2089290768 learned_addr learned my addr 172.21.15.143:0/2089290768 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b57400640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b581554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b57e00640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b5815d4f0 0x7f8b580b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 -- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 msgr2=0x7f8b581554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b581554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 -- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b5815d4f0 msgr2=0x7f8b580b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b5815d4f0 0x7f8b580b5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 -- 172.21.15.143:0/2089290768 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400094d0 con 0x7f8b5815d110 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b56a00640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 0x7f8b580b5810 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8b4001ab20 tx=0x7f8b40019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b4840 con 0x7f8b5815d110 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b400b49a0 con 0x7f8b5815d110 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b580c71b0 con 0x7f8b5815d110 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400b4d30 con 0x7f8b5815d110 2024-01-15T18:18:40.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b580c76e0 con 0x7f8b5815d110 2024-01-15T18:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.083+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b580c78d0 con 0x7f8b5815d110 2024-01-15T18:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.087+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f8b400b2040 con 0x7f8b5815d110 2024-01-15T18:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.087+0000 7f8b4f400640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b30067a60 0x7f8b30069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.087+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f8b4011d5a0 con 0x7f8b5815d110 2024-01-15T18:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.087+0000 7f8b57400640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b30067a60 0x7f8b30069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.087+0000 7f8b57400640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b30067a60 0x7f8b30069f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800b9d0 tx=0x7f8b48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:40.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.091+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b400ebe60 con 0x7f8b5815d110 2024-01-15T18:18:40.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.239+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8b58067500 con 0x7f8b5815d110 2024-01-15T18:18:40.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.907+0000 7f8b4f400640 1 -- 172.21.15.143:0/2089290768 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v109) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8b400b00d0 con 0x7f8b5815d110 2024-01-15T18:18:40.903 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b30067a60 msgr2=0x7f8b30069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b30067a60 0x7f8b30069f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800b9d0 tx=0x7f8b48008040 comp rx=0 tx=0).stop 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 msgr2=0x7f8b580b5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 0x7f8b580b5810 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8b4001ab20 tx=0x7f8b40019210 comp rx=0 tx=0).stop 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 shutdown_connections 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b30067a60 0x7f8b30069f20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b5815d4f0 0x7f8b580b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b5815d110 0x7f8b580b5810 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 --2- 172.21.15.143:0/2089290768 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b5806bf50 0x7f8b581554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:40.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 >> 172.21.15.143:0/2089290768 conn(0x7f8b580b2a40 msgr2=0x7f8b580adde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:40.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 shutdown_connections 2024-01-15T18:18:40.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:40.911+0000 7f8b5e000640 1 -- 172.21.15.143:0/2089290768 wait complete. 2024-01-15T18:18:40.948 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:18:41.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.095+0000 7f8216800640 1 Processor -- start 2024-01-15T18:18:41.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.095+0000 7f8216800640 1 -- start start 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.095+0000 7f8216800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f82100ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.095+0000 7f8216800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821015f310 con 0x7f8210071d00 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.095+0000 7f8216800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821015f5f0 con 0x7f821015ec80 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.095+0000 7f8216800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821015fbb0 con 0x7f8210141660 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820fe00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8210071d00 0x7f82100b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37976/0 (socket says 172.21.15.143:37976) 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820fe00640 1 -- 172.21.15.143:0/3055535009 learned_addr learned my addr 172.21.15.143:0/3055535009 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820f400640 1 --2- 172.21.15.143:0/3055535009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f82100ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959650209 0 0) 0x7f821015f310 con 0x7f8210071d00 2024-01-15T18:18:41.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f0003530 con 0x7f8210071d00 2024-01-15T18:18:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162378464 0 0) 0x7f821015f5f0 con 0x7f821015ec80 2024-01-15T18:18:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821015f310 con 0x7f821015ec80 2024-01-15T18:18:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820f400640 1 -- 172.21.15.143:0/3055535009 >> v1:172.21.15.204:6789/0 conn(0x7f821015ec80 legacy=0x7f8210141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820f400640 1 -- 172.21.15.143:0/3055535009 >> v1:172.21.15.143:6789/0 conn(0x7f8210071d00 legacy=0x7f82100b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820f400640 1 -- 172.21.15.143:0/3055535009 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821015f8d0 con 0x7f8210141660 2024-01-15T18:18:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820f400640 1 --2- 172.21.15.143:0/3055535009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f82100ce560 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f82000097f0 tx=0x7f820001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b68a9b0fb190b1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000ac630 con 0x7f8210141660 2024-01-15T18:18:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000ac790 con 0x7f8210141660 2024-01-15T18:18:41.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.099+0000 7f820e000640 1 -- 172.21.15.143:0/3055535009 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000acb00 con 0x7f8210141660 2024-01-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- 172.21.15.143:0/3055535009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 msgr2=0x7f82100ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 --2- 172.21.15.143:0/3055535009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f82100ce560 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f82000097f0 tx=0x7f820001c4c0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- 172.21.15.143:0/3055535009 shutdown_connections 2024-01-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 --2- 172.21.15.143:0/3055535009 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f82100ce560 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- 172.21.15.143:0/3055535009 >> 172.21.15.143:0/3055535009 conn(0x7f821006c3f0 msgr2=0x7f821006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:41.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- 172.21.15.143:0/3055535009 shutdown_connections 2024-01-15T18:18:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- 172.21.15.143:0/3055535009 wait complete. 2024-01-15T18:18:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 Processor -- start 2024-01-15T18:18:41.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- start start 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 0x7f82100b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f8210159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821015ec80 0x7f8210139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821013fd30 con 0x7f8210071d00 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821013f120 con 0x7f821015ec80 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f8216800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821013f400 con 0x7f8210141660 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f820f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 0x7f82100b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f820f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 0x7f82100b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47912/0 (socket says 172.21.15.143:47912) 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.103+0000 7f820f400640 1 -- 172.21.15.143:0/3622445832 learned_addr learned my addr 172.21.15.143:0/3622445832 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820fe00640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821015ec80 0x7f8210139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820ea00640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f8210159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820f400640 1 -- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 msgr2=0x7f8210159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820f400640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f8210159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820f400640 1 -- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821015ec80 msgr2=0x7f8210139340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820f400640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821015ec80 0x7f8210139340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820f400640 1 -- 172.21.15.143:0/3622445832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200009420 con 0x7f8210071d00 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f820f400640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 0x7f82100b1a30 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8200000c10 tx=0x7f820001ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000b8650 con 0x7f8210071d00 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000b87b0 con 0x7f8210071d00 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82100cd150 con 0x7f8210071d00 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000b8b40 con 0x7f8210071d00 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821013f9c0 con 0x7f8210071d00 2024-01-15T18:18:41.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8210067500 con 0x7f8210071d00 2024-01-15T18:18:41.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.107+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f82000b8ca0 con 0x7f8210071d00 2024-01-15T18:18:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.111+0000 7f8207400640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e8067a90 0x7f81e8069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.111+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f820011e590 con 0x7f8210071d00 2024-01-15T18:18:41.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.111+0000 7f820ea00640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e8067a90 0x7f81e8069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.111+0000 7f820ea00640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e8067a90 0x7f81e8069f50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f82101376f0 tx=0x7f81f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:41.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.115+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82000ed0a0 con 0x7f8210071d00 2024-01-15T18:18:41.288 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.295+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f821006e2a0 con 0x7f8210071d00 2024-01-15T18:18:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.911+0000 7f8207400640 1 -- 172.21.15.143:0/3622445832 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7f82000a80c0 con 0x7f8210071d00 2024-01-15T18:18:41.906 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 16 and data pool 17 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e8067a90 msgr2=0x7f81e8069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e8067a90 0x7f81e8069f50 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f82101376f0 tx=0x7f81f8009040 comp rx=0 tx=0).stop 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 msgr2=0x7f82100b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 0x7f82100b1a30 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8200000c10 tx=0x7f820001ad00 comp rx=0 tx=0).stop 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 shutdown_connections 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f81e8067a90 0x7f81e8069f50 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f821015ec80 0x7f8210139340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8210141660 0x7f8210159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 --2- 172.21.15.143:0/3622445832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8210071d00 0x7f82100b1a30 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 >> 172.21.15.143:0/3622445832 conn(0x7f821006c3f0 msgr2=0x7f82100ae910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 shutdown_connections 2024-01-15T18:18:41.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:41.915+0000 7f8216800640 1 -- 172.21.15.143:0/3622445832 wait complete. 2024-01-15T18:18:41.960 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:42.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.103+0000 7fd4d1400640 1 Processor -- start 2024-01-15T18:18:42.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.103+0000 7fd4d1400640 1 -- start start 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4d1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4d1400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc0ceb50 con 0x7fd4cc0ce520 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4d1400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4cc15c110 con 0x7fd4cc0722c0 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4d1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc15b7e0 con 0x7fd4cc143780 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4c9e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd4cc0ce520 0x7fd4cc071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37986/0 (socket says 172.21.15.143:37986) 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4c9e00640 1 -- 172.21.15.143:0/1586503939 learned_addr learned my addr 172.21.15.143:0/1586503939 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:42.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4ca800640 1 --2- 172.21.15.143:0/1586503939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247189793 0 0) 0x7fd4b4001120 con 0x7fd4cc0ce520 2024-01-15T18:18:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4ac003530 con 0x7fd4cc0ce520 2024-01-15T18:18:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462210244 0 0) 0x7fd4cc15c110 con 0x7fd4cc0722c0 2024-01-15T18:18:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4b4001120 con 0x7fd4cc0722c0 2024-01-15T18:18:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4ca800640 1 -- 172.21.15.143:0/1586503939 >> v1:172.21.15.204:6789/0 conn(0x7fd4cc0722c0 legacy=0x7fd4cc15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4ca800640 1 -- 172.21.15.143:0/1586503939 >> v1:172.21.15.143:6789/0 conn(0x7fd4cc0ce520 legacy=0x7fd4cc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.107+0000 7fd4ca800640 1 -- 172.21.15.143:0/1586503939 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc15bac0 con 0x7fd4cc143780 2024-01-15T18:18:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4ca800640 1 --2- 172.21.15.143:0/1586503939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc0cdfe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097f0 tx=0x7fd4bc01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2efc40d6aa45923 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0ac630 con 0x7fd4cc143780 2024-01-15T18:18:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc0ac790 con 0x7fd4cc143780 2024-01-15T18:18:42.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4c9400640 1 -- 172.21.15.143:0/1586503939 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0acb20 con 0x7fd4cc143780 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 -- 172.21.15.143:0/1586503939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 msgr2=0x7fd4cc0cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 --2- 172.21.15.143:0/1586503939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc0cdfe0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0097f0 tx=0x7fd4bc01c4c0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 -- 172.21.15.143:0/1586503939 shutdown_connections 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 --2- 172.21.15.143:0/1586503939 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc0cdfe0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 -- 172.21.15.143:0/1586503939 >> 172.21.15.143:0/1586503939 conn(0x7fd4cc06c3f0 msgr2=0x7fd4cc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 -- 172.21.15.143:0/1586503939 shutdown_connections 2024-01-15T18:18:42.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 -- 172.21.15.143:0/1586503939 wait complete. 2024-01-15T18:18:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 Processor -- start 2024-01-15T18:18:42.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.111+0000 7fd4d1400640 1 -- start start 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 0x7fd4cc135f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4cc0ce520 0x7fd4cc142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc142c90 con 0x7fd4cc0722c0 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc136570 con 0x7fd4cc0ce520 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc136850 con 0x7fd4cc143780 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 0x7fd4cc135f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 0x7fd4cc135f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47918/0 (socket says 172.21.15.143:47918) 2024-01-15T18:18:42.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 -- 172.21.15.143:0/983874814 learned_addr learned my addr 172.21.15.143:0/983874814 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4c9e00640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4cc0ce520 0x7fd4cc142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4cb200640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 -- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 msgr2=0x7fd4cc146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 -- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4cc0ce520 msgr2=0x7fd4cc142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4cc0ce520 0x7fd4cc142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 -- 172.21.15.143:0/983874814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc009420 con 0x7fd4cc0722c0 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4c9e00640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4cc0ce520 0x7fd4cc142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:42.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4cb200640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4ca800640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 0x7fd4cc135f70 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0190d0 tx=0x7fd4bc019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0b6670 con 0x7fd4cc0722c0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc0b67f0 con 0x7fd4cc0722c0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc136b30 con 0x7fd4cc0722c0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0b6ba0 con 0x7fd4cc0722c0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc146640 con 0x7fd4cc0722c0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.115+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc067500 con 0x7fd4cc0722c0 2024-01-15T18:18:42.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.119+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fd4bc0b3040 con 0x7fd4cc0722c0 2024-01-15T18:18:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.119+0000 7fd4c2a00640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4a4067a60 0x7fd4a4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.119+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd4bc11d6b0 con 0x7fd4cc0722c0 2024-01-15T18:18:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.119+0000 7fd4c9e00640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4a4067a60 0x7fd4a4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.119+0000 7fd4c9e00640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4a4067a60 0x7fd4a4069f20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc071610 tx=0x7fd4b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.123+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd4bc0ebff0 con 0x7fd4cc0722c0 2024-01-15T18:18:42.268 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.275+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4cc15c640 con 0x7fd4cc0722c0 2024-01-15T18:18:42.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.275+0000 7fd4c2a00640 1 -- 172.21.15.143:0/983874814 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16664 (secure 0 0 0) 0x7fd4bc0a8020 con 0x7fd4cc0722c0 2024-01-15T18:18:42.270 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:42.270 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":110,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:41.296842+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:42.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.275+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4a4067a60 msgr2=0x7fd4a4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.275+0000 7fd4d1400640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4a4067a60 0x7fd4a4069f20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc071610 tx=0x7fd4b4009040 comp rx=0 tx=0).stop 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 msgr2=0x7fd4cc135f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 0x7fd4cc135f70 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc0190d0 tx=0x7fd4bc019260 comp rx=0 tx=0).stop 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 shutdown_connections 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd4a4067a60 0x7fd4a4069f20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4cc143780 0x7fd4cc146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4cc0ce520 0x7fd4cc142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 --2- 172.21.15.143:0/983874814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4cc0722c0 0x7fd4cc135f70 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 >> 172.21.15.143:0/983874814 conn(0x7fd4cc06c3f0 msgr2=0x7fd4cc0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 shutdown_connections 2024-01-15T18:18:42.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.279+0000 7fd4d1400640 1 -- 172.21.15.143:0/983874814 wait complete. 2024-01-15T18:18:42.292 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e39200640 1 Processor -- start 2024-01-15T18:18:42.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e39200640 1 -- start start 2024-01-15T18:18:42.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e39200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0e340c6230 0x7f0e340cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e39200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e340c4d90 con 0x7f0e3415b260 2024-01-15T18:18:42.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e39200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3415b640 con 0x7f0e340cdf20 2024-01-15T18:18:42.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e39200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e340cf800 con 0x7f0e340c6230 2024-01-15T18:18:42.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e31e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0e3415b260 0x7f0e340cd810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37990/0 (socket says 172.21.15.143:37990) 2024-01-15T18:18:42.431 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e31e00640 1 -- 172.21.15.143:0/1823444869 learned_addr learned my addr 172.21.15.143:0/1823444869 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e32800640 1 --2- 172.21.15.143:0/1823444869 >> v2:172.21.15.204:3300/0 conn(0x7f0e340c6230 0x7f0e340cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992010257 0 0) 0x7f0e340c4d90 con 0x7f0e3415b260 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e14003530 con 0x7f0e3415b260 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358486284 0 0) 0x7f0e3415b640 con 0x7f0e340cdf20 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e340c4d90 con 0x7f0e340cdf20 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e32800640 1 -- 172.21.15.143:0/1823444869 >> v1:172.21.15.204:6789/0 conn(0x7f0e340cdf20 legacy=0x7f0e340cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e32800640 1 -- 172.21.15.143:0/1823444869 >> v1:172.21.15.143:6789/0 conn(0x7f0e3415b260 legacy=0x7f0e340cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.432 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.435+0000 7f0e32800640 1 -- 172.21.15.143:0/1823444869 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340cfae0 con 0x7f0e340c6230 2024-01-15T18:18:42.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e32800640 1 --2- 172.21.15.143:0/1823444869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e340c6230 0x7f0e340cc1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009d90 tx=0x7f0e2c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdab0252f5a28872 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0a59e0 con 0x7f0e340c6230 2024-01-15T18:18:42.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340ccf40 con 0x7f0e340c6230 2024-01-15T18:18:42.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e30a00640 1 -- 172.21.15.143:0/1823444869 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0a5d40 con 0x7f0e340c6230 2024-01-15T18:18:42.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 -- 172.21.15.143:0/1823444869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e340c6230 msgr2=0x7f0e340cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 --2- 172.21.15.143:0/1823444869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e340c6230 0x7f0e340cc1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009d90 tx=0x7f0e2c01c920 comp rx=0 tx=0).stop 2024-01-15T18:18:42.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 -- 172.21.15.143:0/1823444869 shutdown_connections 2024-01-15T18:18:42.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 --2- 172.21.15.143:0/1823444869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e340c6230 0x7f0e340cc1f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 -- 172.21.15.143:0/1823444869 >> 172.21.15.143:0/1823444869 conn(0x7f0e340d0ed0 msgr2=0x7f0e340d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:42.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 -- 172.21.15.143:0/1823444869 shutdown_connections 2024-01-15T18:18:42.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.439+0000 7f0e39200640 1 -- 172.21.15.143:0/1823444869 wait complete. 2024-01-15T18:18:42.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 Processor -- start 2024-01-15T18:18:42.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 -- start start 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 0x7f0e3415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e340cdf20 0x7f0e340c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3415b260 0x7f0e341512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340b4900 con 0x7f0e340c6230 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340ca130 con 0x7f0e3415b260 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340ca3d0 con 0x7f0e340cdf20 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 0x7f0e3415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 0x7f0e3415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47924/0 (socket says 172.21.15.143:47924) 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 -- 172.21.15.143:0/503401339 learned_addr learned my addr 172.21.15.143:0/503401339 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:42.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e32800640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3415b260 0x7f0e341512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31400640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e340cdf20 0x7f0e340c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 -- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e340cdf20 msgr2=0x7f0e340c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e340cdf20 0x7f0e340c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 -- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3415b260 msgr2=0x7f0e341512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3415b260 0x7f0e341512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 -- 172.21.15.143:0/503401339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c009a70 con 0x7f0e340c6230 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31400640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e340cdf20 0x7f0e340c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e31e00640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 0x7f0e3415a650 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c4d0 tx=0x7f0e2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24005cf0 con 0x7f0e340c6230 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24005e50 con 0x7f0e340c6230 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e240072b0 con 0x7f0e340c6230 2024-01-15T18:18:42.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e340b3fd0 con 0x7f0e340c6230 2024-01-15T18:18:42.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.443+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e340b4590 con 0x7f0e340c6230 2024-01-15T18:18:42.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.447+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e34067500 con 0x7f0e340c6230 2024-01-15T18:18:42.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.447+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f0e24007410 con 0x7f0e340c6230 2024-01-15T18:18:42.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.447+0000 7f0e2a000640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e0c067a60 0x7f0e0c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.447+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0e240828c0 con 0x7f0e340c6230 2024-01-15T18:18:42.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.447+0000 7f0e31400640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e0c067a60 0x7f0e0c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.447+0000 7f0e31400640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e0c067a60 0x7f0e0c069f20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0e3406f220 tx=0x7f0e1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.451+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e240512b0 con 0x7f0e340c6230 2024-01-15T18:18:42.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.603+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e340d13f0 con 0x7f0e340c6230 2024-01-15T18:18:42.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.603+0000 7f0e2a000640 1 -- 172.21.15.143:0/503401339 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16664 (secure 0 0 0) 0x7f0e24017020 con 0x7f0e340c6230 2024-01-15T18:18:42.598 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:42.598 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":110,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:41.296842+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.603+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e0c067a60 msgr2=0x7f0e0c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.603+0000 7f0e39200640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e0c067a60 0x7f0e0c069f20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0e3406f220 tx=0x7f0e1c009040 comp rx=0 tx=0).stop 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.603+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 msgr2=0x7f0e3415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.603+0000 7f0e39200640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 0x7f0e3415a650 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c4d0 tx=0x7f0e2400c990 comp rx=0 tx=0).stop 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 shutdown_connections 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0e0c067a60 0x7f0e0c069f20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3415b260 0x7f0e341512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e340cdf20 0x7f0e340c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 --2- 172.21.15.143:0/503401339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340c6230 0x7f0e3415a650 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 >> 172.21.15.143:0/503401339 conn(0x7f0e340d0ed0 msgr2=0x7f0e340c6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:42.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 shutdown_connections 2024-01-15T18:18:42.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.607+0000 7f0e39200640 1 -- 172.21.15.143:0/503401339 wait complete. 2024-01-15T18:18:42.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:18:42.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc4e000640 1 Processor -- start 2024-01-15T18:18:42.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc4e000640 1 -- start start 2024-01-15T18:18:42.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cda70 0x7fdc480cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc4e000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4815b5b0 con 0x7fdc480cf400 2024-01-15T18:18:42.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc4e000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc480ccd30 con 0x7fdc480cf800 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc4e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480cba70 con 0x7fdc480cda70 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fdc480cf400 0x7fdc480cde50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37996/0 (socket says 172.21.15.143:37996) 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46a00640 1 -- 172.21.15.143:0/215202931 learned_addr learned my addr 172.21.15.143:0/215202931 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc47400640 1 --2- 172.21.15.143:0/215202931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cda70 0x7fdc480cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913176135 0 0) 0x7fdc4815b5b0 con 0x7fdc480cf400 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc28003530 con 0x7fdc480cf400 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084319051 0 0) 0x7fdc480ccd30 con 0x7fdc480cf800 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc4815b5b0 con 0x7fdc480cf800 2024-01-15T18:18:42.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc47400640 1 -- 172.21.15.143:0/215202931 >> v1:172.21.15.204:6789/0 conn(0x7fdc480cf800 legacy=0x7fdc480b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc47400640 1 -- 172.21.15.143:0/215202931 >> v1:172.21.15.143:6789/0 conn(0x7fdc480cf400 legacy=0x7fdc480cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692156128 0 0) 0x7fdc28003530 con 0x7fdc480cf400 2024-01-15T18:18:42.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.767+0000 7fdc47400640 1 -- 172.21.15.143:0/215202931 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480cbd10 con 0x7fdc480cda70 2024-01-15T18:18:42.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc47400640 1 --2- 172.21.15.143:0/215202931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cda70 0x7fdc480cc7f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdc380097f0 tx=0x7fdc3801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff84b5b6c57560c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc30001680 con 0x7fdc480cda70 2024-01-15T18:18:42.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc380a7050 con 0x7fdc480cda70 2024-01-15T18:18:42.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc46000640 1 -- 172.21.15.143:0/215202931 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380ab620 con 0x7fdc480cda70 2024-01-15T18:18:42.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 -- 172.21.15.143:0/215202931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cda70 msgr2=0x7fdc480cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 --2- 172.21.15.143:0/215202931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cda70 0x7fdc480cc7f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdc380097f0 tx=0x7fdc3801c3b0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 -- 172.21.15.143:0/215202931 shutdown_connections 2024-01-15T18:18:42.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 --2- 172.21.15.143:0/215202931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cda70 0x7fdc480cc7f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 -- 172.21.15.143:0/215202931 >> 172.21.15.143:0/215202931 conn(0x7fdc480d0e20 msgr2=0x7fdc480d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:42.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 -- 172.21.15.143:0/215202931 shutdown_connections 2024-01-15T18:18:42.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.771+0000 7fdc4e000640 1 -- 172.21.15.143:0/215202931 wait complete. 2024-01-15T18:18:42.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 Processor -- start 2024-01-15T18:18:42.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 -- start start 2024-01-15T18:18:42.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 0x7fdc480c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cf400 0x7fdc480c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc480cf800 0x7fdc480c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4814a7d0 con 0x7fdc480cda70 2024-01-15T18:18:42.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4814ac60 con 0x7fdc480cf800 2024-01-15T18:18:42.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4814b140 con 0x7fdc480cf400 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 0x7fdc480c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 0x7fdc480c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47930/0 (socket says 172.21.15.143:47930) 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 -- 172.21.15.143:0/3378271711 learned_addr learned my addr 172.21.15.143:0/3378271711 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc46a00640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cf400 0x7fdc480c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47e00640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc480cf800 0x7fdc480c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 -- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cf400 msgr2=0x7fdc480c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cf400 0x7fdc480c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 -- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc480cf800 msgr2=0x7fdc480c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc480cf800 0x7fdc480c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:42.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 -- 172.21.15.143:0/3378271711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009420 con 0x7fdc480cda70 2024-01-15T18:18:42.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc47400640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 0x7fdc480c88b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdc380ae040 tx=0x7fdc3801abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380bc070 con 0x7fdc480cda70 2024-01-15T18:18:42.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc380abcf0 con 0x7fdc480cda70 2024-01-15T18:18:42.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380b76b0 con 0x7fdc480cda70 2024-01-15T18:18:42.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480af290 con 0x7fdc480cda70 2024-01-15T18:18:42.774 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.775+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc480af850 con 0x7fdc480cda70 2024-01-15T18:18:42.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.779+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc480afa40 con 0x7fdc480cda70 2024-01-15T18:18:42.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.779+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fdc380b7810 con 0x7fdc480cda70 2024-01-15T18:18:42.775 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.779+0000 7fdc3f400640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdc20067a60 0x7fdc20069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:42.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.779+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fdc3811cc00 con 0x7fdc480cda70 2024-01-15T18:18:42.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.779+0000 7fdc46a00640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdc20067a60 0x7fdc20069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:42.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.779+0000 7fdc46a00640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdc20067a60 0x7fdc20069f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc480b27d0 tx=0x7fdc300016c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:42.778 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.783+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdc380eb940 con 0x7fdc480cda70 2024-01-15T18:18:42.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:42.967+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdc48067500 con 0x7fdc480cda70 2024-01-15T18:18:43.891 INFO:tasks.ceph.mds.d.smithi204.stderr:2024-01-15T18:18:43.887+0000 7fe670e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:18:43.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.943+0000 7fdc3f400640 1 -- 172.21.15.143:0/3378271711 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdc380b00b0 con 0x7fdc480cda70 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdc20067a60 msgr2=0x7fdc20069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdc20067a60 0x7fdc20069f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc480b27d0 tx=0x7fdc300016c0 comp rx=0 tx=0).stop 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 msgr2=0x7fdc480c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 0x7fdc480c88b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdc380ae040 tx=0x7fdc3801abf0 comp rx=0 tx=0).stop 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 shutdown_connections 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdc20067a60 0x7fdc20069f20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc480cf800 0x7fdc480c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fdc480cf400 0x7fdc480c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 --2- 172.21.15.143:0/3378271711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdc480cda70 0x7fdc480c88b0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:43.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 >> 172.21.15.143:0/3378271711 conn(0x7fdc480d0e20 msgr2=0x7fdc481534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:43.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 shutdown_connections 2024-01-15T18:18:43.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:43.947+0000 7fdc4e000640 1 -- 172.21.15.143:0/3378271711 wait complete. 2024-01-15T18:18:43.966 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:18:43.967 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:18:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.115+0000 7efe03800640 1 Processor -- start 2024-01-15T18:18:44.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.115+0000 7efe03800640 1 -- start start 2024-01-15T18:18:44.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.115+0000 7efe03800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7efdfc15b2d0 0x7efdfc0d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:44.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.115+0000 7efe03800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdfc0cebf0 0x7efdfc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:44.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efe03800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdfc15b6b0 con 0x7efdfc0cf930 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efe03800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0b0620 con 0x7efdfc0cebf0 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efe03800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdfc0b0320 con 0x7efdfc15b2d0 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdfc0cebf0 0x7efdfc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdfc0cebf0 0x7efdfc0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47936/0 (socket says 172.21.15.143:47936) 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efe01800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efdfc0cf930 0x7efdfc0cd470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37998/0 (socket says 172.21.15.143:37998) 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 -- 172.21.15.143:0/2173395662 learned_addr learned my addr 172.21.15.143:0/2173395662 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 -- 172.21.15.143:0/2173395662 >> v2:172.21.15.204:3300/0 conn(0x7efdfc15b2d0 msgr2=0x7efdfc0d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efe00e00640 1 --2- 172.21.15.143:0/2173395662 >> v2:172.21.15.204:3300/0 conn(0x7efdfc15b2d0 0x7efdfc0d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfb400640 1 -- 172.21.15.143:0/2173395662 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263496832 0 0) 0x7efdfc15b6b0 con 0x7efdfc0cf930 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 --2- 172.21.15.143:0/2173395662 >> v2:172.21.15.204:3300/0 conn(0x7efdfc15b2d0 0x7efdfc0d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 -- 172.21.15.143:0/2173395662 >> v1:172.21.15.143:6789/0 conn(0x7efdfc0cf930 legacy=0x7efdfc0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:44.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 -- 172.21.15.143:0/2173395662 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc0cc3c0 con 0x7efdfc0cebf0 2024-01-15T18:18:44.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efe00e00640 1 --2- 172.21.15.143:0/2173395662 >> v2:172.21.15.204:3300/0 conn(0x7efdfc15b2d0 0x7efdfc0d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:44.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfbe00640 1 --2- 172.21.15.143:0/2173395662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0cf3f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7efdfc070fe0 tx=0x7efde401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137b96375f98282e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:44.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfb400640 1 -- 172.21.15.143:0/2173395662 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde40aa660 con 0x7efdfc0cebf0 2024-01-15T18:18:44.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfb400640 1 -- 172.21.15.143:0/2173395662 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde40aa7c0 con 0x7efdfc0cebf0 2024-01-15T18:18:44.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.119+0000 7efdfb400640 1 -- 172.21.15.143:0/2173395662 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde40aab50 con 0x7efdfc0cebf0 2024-01-15T18:18:44.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- 172.21.15.143:0/2173395662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cebf0 msgr2=0x7efdfc0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 --2- 172.21.15.143:0/2173395662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0cf3f0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7efdfc070fe0 tx=0x7efde401c500 comp rx=0 tx=0).stop 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- 172.21.15.143:0/2173395662 shutdown_connections 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 --2- 172.21.15.143:0/2173395662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0cf3f0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 --2- 172.21.15.143:0/2173395662 >> v2:172.21.15.204:3300/0 conn(0x7efdfc15b2d0 0x7efdfc0d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- 172.21.15.143:0/2173395662 >> 172.21.15.143:0/2173395662 conn(0x7efdfc0d0e10 msgr2=0x7efdfc0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- 172.21.15.143:0/2173395662 shutdown_connections 2024-01-15T18:18:44.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- 172.21.15.143:0/2173395662 wait complete. 2024-01-15T18:18:44.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 Processor -- start 2024-01-15T18:18:44.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- start start 2024-01-15T18:18:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0c5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 0x7efdfc0c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdfc15b2d0 0x7efdfc151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc15a2f0 con 0x7efdfc0cf930 2024-01-15T18:18:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc159720 con 0x7efdfc0cebf0 2024-01-15T18:18:44.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efe03800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdfc1599c0 con 0x7efdfc15b2d0 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.123+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 0x7efdfc0c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 0x7efdfc0c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47948/0 (socket says 172.21.15.143:47948) 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 -- 172.21.15.143:0/377917543 learned_addr learned my addr 172.21.15.143:0/377917543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efe01800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdfc15b2d0 0x7efdfc151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efe00e00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 -- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdfc15b2d0 msgr2=0x7efdfc151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdfc15b2d0 0x7efdfc151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 -- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdfc0cebf0 msgr2=0x7efdfc0c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 -- 172.21.15.143:0/377917543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40094d0 con 0x7efdfc0cf930 2024-01-15T18:18:44.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efe01800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdfc15b2d0 0x7efdfc151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdfbe00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 0x7efdfc0c7030 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efde4019110 tx=0x7efde401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde40a9990 con 0x7efdfc0cf930 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde40a9b10 con 0x7efdfc0cf930 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdfc159ca0 con 0x7efdfc0cf930 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde40a9ec0 con 0x7efdfc0cf930 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdfc0c57d0 con 0x7efdfc0cf930 2024-01-15T18:18:44.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdfc15a4a0 con 0x7efdfc0cf930 2024-01-15T18:18:44.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.127+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7efde40b33f0 con 0x7efdfc0cf930 2024-01-15T18:18:44.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.131+0000 7efdf8c00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efdd4067a60 0x7efdd4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:44.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.131+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7efde40a6040 con 0x7efdfc0cf930 2024-01-15T18:18:44.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.131+0000 7efe00e00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efdd4067a60 0x7efdd4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:44.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.131+0000 7efe00e00640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efdd4067a60 0x7efdd4069f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efdec006fd0 tx=0x7efdec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:44.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.135+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efdfc15a4a0 con 0x7efdfc0cf930 2024-01-15T18:18:44.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.315+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efdfc15a4a0 con 0x7efdfc0cf930 2024-01-15T18:18:44.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efdf8c00640 1 -- 172.21.15.143:0/377917543 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7efdfc15a4a0 con 0x7efdfc0cf930 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efdd4067a60 msgr2=0x7efdd4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efdd4067a60 0x7efdd4069f20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efdec006fd0 tx=0x7efdec008040 comp rx=0 tx=0).stop 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 msgr2=0x7efdfc0c7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 0x7efdfc0c7030 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7efde4019110 tx=0x7efde401ad40 comp rx=0 tx=0).stop 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 shutdown_connections 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efdd4067a60 0x7efdd4069f20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efdfc15b2d0 0x7efdfc151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdfc0cf930 0x7efdfc0c7030 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 --2- 172.21.15.143:0/377917543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efdfc0cebf0 0x7efdfc0c5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:44.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 >> 172.21.15.143:0/377917543 conn(0x7efdfc0d0e10 msgr2=0x7efdfc0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:44.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 shutdown_connections 2024-01-15T18:18:44.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:44.955+0000 7efe03800640 1 -- 172.21.15.143:0/377917543 wait complete. 2024-01-15T18:18:44.975 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:18:44.975 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:18:45.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f359d200640 1 Processor -- start 2024-01-15T18:18:45.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f359d200640 1 -- start start 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f359d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f359815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f359d200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359815f310 con 0x7f3598141660 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f359d200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359815f5f0 con 0x7f35980ce560 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f359d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359815fbb0 con 0x7f3598072410 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3595e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f3598141660 0x7f3598141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38006/0 (socket says 172.21.15.143:38006) 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3595e00640 1 -- 172.21.15.143:0/3571377636 learned_addr learned my addr 172.21.15.143:0/3571377636 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3596800640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f359815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000339096 0 0) 0x7f359815f310 con 0x7f3598141660 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578003530 con 0x7f3598141660 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872591996 0 0) 0x7f3578003530 con 0x7f3598141660 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 msgr2=0x7f359815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f359815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 >> v1:172.21.15.204:6789/0 conn(0x7f35980ce560 legacy=0x7f3598071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359815f8d0 con 0x7f3598141660 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f3588001680 con 0x7f3598141660 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 >> v1:172.21.15.143:6789/0 conn(0x7f3598141660 legacy=0x7f3598141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35780039b0 0x7f3578003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f35780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3578005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359815fbb0 con 0x7f3578004d10 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357800ba70 con 0x7f35780039b0 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.119+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357800bc30 con 0x7f3598072410 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3578005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3595e00640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35780039b0 0x7f3578003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3595400640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f35780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 -- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 msgr2=0x7f35780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f35780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 -- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35780039b0 msgr2=0x7f3578003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35780039b0 0x7f3578003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 -- 172.21.15.143:0/3571377636 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3578003530 con 0x7f3578004d10 2024-01-15T18:18:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3595400640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598072410 0x7f35780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3596800640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3578005190 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f359806e270 tx=0x7f359001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550baea80fc1d265 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900a5c10 con 0x7f3578004d10 2024-01-15T18:18:45.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359000f040 con 0x7f3578004d10 2024-01-15T18:18:45.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.123+0000 7f3594a00640 1 -- 172.21.15.143:0/3571377636 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900ad440 con 0x7f3578004d10 2024-01-15T18:18:45.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- 172.21.15.143:0/3571377636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 msgr2=0x7f3578005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 --2- 172.21.15.143:0/3571377636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3578005190 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f359806e270 tx=0x7f359001cc00 comp rx=0 tx=0).stop 2024-01-15T18:18:45.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f3596800640 1 -- 172.21.15.143:0/3571377636 reap_dead start 2024-01-15T18:18:45.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- 172.21.15.143:0/3571377636 shutdown_connections 2024-01-15T18:18:45.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- 172.21.15.143:0/3571377636 >> 172.21.15.143:0/3571377636 conn(0x7f359806c3f0 msgr2=0x7f359806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:45.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- 172.21.15.143:0/3571377636 shutdown_connections 2024-01-15T18:18:45.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- 172.21.15.143:0/3571377636 wait complete. 2024-01-15T18:18:45.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 Processor -- start 2024-01-15T18:18:45.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- start start 2024-01-15T18:18:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3598139260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598136a60 0x7f35980ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35980ace20 0x7f359814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35981360b0 con 0x7f3578004d10 2024-01-15T18:18:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35980ad2a0 con 0x7f35980ace20 2024-01-15T18:18:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f359d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359814f060 con 0x7f3598136a60 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f3595e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3598139260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.127+0000 7f3595e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3598139260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47980/0 (socket says 172.21.15.143:47980) 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 -- 172.21.15.143:0/515892022 learned_addr learned my addr 172.21.15.143:0/515892022 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595400640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598136a60 0x7f35980ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3596800640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35980ace20 0x7f359814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 -- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598136a60 msgr2=0x7f35980ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598136a60 0x7f35980ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 -- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35980ace20 msgr2=0x7f359814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35980ace20 0x7f359814e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 -- 172.21.15.143:0/515892022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598138780 con 0x7f3578004d10 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3596800640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35980ace20 0x7f359814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595400640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598136a60 0x7f35980ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f3595e00640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3598139260 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f35880072c0 tx=0x7f3588007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358800f960 con 0x7f3578004d10 2024-01-15T18:18:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358800fac0 con 0x7f3578004d10 2024-01-15T18:18:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358800fe70 con 0x7f3578004d10 2024-01-15T18:18:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3598138a60 con 0x7f3578004d10 2024-01-15T18:18:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3598138430 con 0x7f3578004d10 2024-01-15T18:18:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3598067500 con 0x7f3578004d10 2024-01-15T18:18:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.131+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f358801d460 con 0x7f3578004d10 2024-01-15T18:18:45.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.135+0000 7f358e000640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3570067a60 0x7f3570069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.135+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3588083590 con 0x7f3578004d10 2024-01-15T18:18:45.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.135+0000 7f3595400640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3570067a60 0x7f3570069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.135+0000 7f3595400640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3570067a60 0x7f3570069f20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f359815aaf0 tx=0x7f3580009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.139+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3588051f30 con 0x7f3578004d10 2024-01-15T18:18:45.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.327+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3598138f10 con 0x7f3578004d10 2024-01-15T18:18:45.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.327+0000 7f358e000640 1 -- 172.21.15.143:0/515892022 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f3588055550 con 0x7f3578004d10 2024-01-15T18:18:45.324 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:45.324 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8776,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2963172683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2963172683},{"type":"v1","addr":"172.21.15.204:6834","nonce":2963172683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":9012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/813203633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":813203633},{"type":"v1","addr":"172.21.15.143:6836","nonce":813203633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":9015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/3497546933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3497546933},{"type":"v1","addr":"172.21.15.143:6837","nonce":3497546933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:18:41.297206+0000","modified":"2024-01-15T18:18:44.320349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3419861535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3419861535},{"type":"v1","addr":"172.21.15.204:6835","nonce":3419861535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-01-15T18:18:45.324 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 72 2024-01-15T18:18:45.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3570067a60 msgr2=0x7f3570069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3570067a60 0x7f3570069f20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f359815aaf0 tx=0x7f3580009040 comp rx=0 tx=0).stop 2024-01-15T18:18:45.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 msgr2=0x7f3598139260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3598139260 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f35880072c0 tx=0x7f3588007780 comp rx=0 tx=0).stop 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 shutdown_connections 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3570067a60 0x7f3570069f20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35980ace20 0x7f359814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3598136a60 0x7f35980ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 --2- 172.21.15.143:0/515892022 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3578004d10 0x7f3598139260 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 >> 172.21.15.143:0/515892022 conn(0x7f359806c3f0 msgr2=0x7f35980af510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 shutdown_connections 2024-01-15T18:18:45.327 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.331+0000 7f359d200640 1 -- 172.21.15.143:0/515892022 wait complete. 2024-01-15T18:18:45.347 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:18:45.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.487+0000 7f352f000640 1 Processor -- start 2024-01-15T18:18:45.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.487+0000 7f352f000640 1 -- start start 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f352f000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3528143780 0x7f35280cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f352f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35280ce520 0x7f3528071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f352f000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35280ceb50 con 0x7f35280720f0 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f352f000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3528072580 con 0x7f3528143780 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f352f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352815c110 con 0x7f35280ce520 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3528143780 0x7f35280cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3528143780 0x7f35280cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47984/0 (socket says 172.21.15.143:47984) 2024-01-15T18:18:45.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 -- 172.21.15.143:0/4198740269 learned_addr learned my addr 172.21.15.143:0/4198740269 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:45.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527400640 1 --2- 172.21.15.143:0/4198740269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35280ce520 0x7f3528071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 -- 172.21.15.143:0/4198740269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35280ce520 msgr2=0x7f3528071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:45.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 --2- 172.21.15.143:0/4198740269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35280ce520 0x7f3528071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 -- 172.21.15.143:0/4198740269 >> v1:172.21.15.204:6789/0 conn(0x7f35280720f0 legacy=0x7f352815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 -- 172.21.15.143:0/4198740269 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352815b7e0 con 0x7f3528143780 2024-01-15T18:18:45.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.491+0000 7f3527e00640 1 --2- 172.21.15.143:0/4198740269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3528143780 0x7f35280cdfe0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f3518009870 tx=0x7f351801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b91a572ffc510c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f3526a00640 1 -- 172.21.15.143:0/4198740269 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180ab650 con 0x7f3528143780 2024-01-15T18:18:45.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f3526a00640 1 -- 172.21.15.143:0/4198740269 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35180ab7b0 con 0x7f3528143780 2024-01-15T18:18:45.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f3526a00640 1 -- 172.21.15.143:0/4198740269 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180abb40 con 0x7f3528143780 2024-01-15T18:18:45.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 -- 172.21.15.143:0/4198740269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3528143780 msgr2=0x7f35280cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 --2- 172.21.15.143:0/4198740269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3528143780 0x7f35280cdfe0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f3518009870 tx=0x7f351801c540 comp rx=0 tx=0).stop 2024-01-15T18:18:45.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 -- 172.21.15.143:0/4198740269 shutdown_connections 2024-01-15T18:18:45.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 --2- 172.21.15.143:0/4198740269 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f35280ce520 0x7f3528071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 --2- 172.21.15.143:0/4198740269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3528143780 0x7f35280cdfe0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 -- 172.21.15.143:0/4198740269 >> 172.21.15.143:0/4198740269 conn(0x7f352806c3f0 msgr2=0x7f352806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:45.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 -- 172.21.15.143:0/4198740269 shutdown_connections 2024-01-15T18:18:45.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 -- 172.21.15.143:0/4198740269 wait complete. 2024-01-15T18:18:45.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 Processor -- start 2024-01-15T18:18:45.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.495+0000 7f352f000640 1 -- start start 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35280720f0 0x7f35281f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 0x7f3528138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3528143780 0x7f3528155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3528156150 con 0x7f35280ce520 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352813b440 con 0x7f35280720f0 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352813b720 con 0x7f3528143780 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 0x7f3528138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 0x7f3528138ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47986/0 (socket says 172.21.15.143:47986) 2024-01-15T18:18:45.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 -- 172.21.15.143:0/3182073040 learned_addr learned my addr 172.21.15.143:0/3182073040 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352d000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3528143780 0x7f3528155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527e00640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35280720f0 0x7f35281f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 -- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3528143780 msgr2=0x7f3528155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3528143780 0x7f3528155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 -- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35280720f0 msgr2=0x7f35281f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35280720f0 0x7f35281f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 -- 172.21.15.143:0/3182073040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180094d0 con 0x7f35280ce520 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f3527400640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 0x7f3528138ed0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f351000e930 tx=0x7f351000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510009690 con 0x7f35280ce520 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000ca00 con 0x7f35280ce520 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000cd70 con 0x7f35280ce520 2024-01-15T18:18:45.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352813ba00 con 0x7f35280ce520 2024-01-15T18:18:45.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3528139410 con 0x7f35280ce520 2024-01-15T18:18:45.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352813b280 con 0x7f35280ce520 2024-01-15T18:18:45.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.499+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f3510013470 con 0x7f35280ce520 2024-01-15T18:18:45.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.503+0000 7f351fe00640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3500067a60 0x7f3500069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.503+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f35100834a0 con 0x7f35280ce520 2024-01-15T18:18:45.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.503+0000 7f3527e00640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3500067a60 0x7f3500069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.503+0000 7f3527e00640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3500067a60 0x7f3500069f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f35180af000 tx=0x7f35180b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.507+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f351008b070 con 0x7f35280ce520 2024-01-15T18:18:45.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.655+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3528067500 con 0x7f35280ce520 2024-01-15T18:18:45.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.655+0000 7f351fe00640 1 -- 172.21.15.143:0/3182073040 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16651 (secure 0 0 0) 0x7f3510021090 con 0x7f35280ce520 2024-01-15T18:18:45.652 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:45.652 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":111,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:18:42.931152+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:18:45.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3500067a60 msgr2=0x7f3500069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3500067a60 0x7f3500069f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f35180af000 tx=0x7f35180b6040 comp rx=0 tx=0).stop 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 msgr2=0x7f3528138ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 0x7f3528138ed0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f351000e930 tx=0x7f351000edf0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 shutdown_connections 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3500067a60 0x7f3500069f20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3528143780 0x7f3528155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f35280ce520 0x7f3528138ed0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 --2- 172.21.15.143:0/3182073040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f35280720f0 0x7f35281f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 >> 172.21.15.143:0/3182073040 conn(0x7f352806c3f0 msgr2=0x7f352806c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:45.655 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 shutdown_connections 2024-01-15T18:18:45.656 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.659+0000 7f352f000640 1 -- 172.21.15.143:0/3182073040 wait complete. 2024-01-15T18:18:45.674 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:18:45.674 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:45.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc87600640 1 Processor -- start 2024-01-15T18:18:45.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc87600640 1 -- start start 2024-01-15T18:18:45.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc87600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800b24b0 0x7efc800ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc87600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc80070bf0 con 0x7efc800ce690 2024-01-15T18:18:45.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc87600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc800b4920 con 0x7efc800b3bf0 2024-01-15T18:18:45.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc87600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8015dba0 con 0x7efc800b24b0 2024-01-15T18:18:45.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc84c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efc800ce690 0x7efc800b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38010/0 (socket says 172.21.15.143:38010) 2024-01-15T18:18:45.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc84c00640 1 -- 172.21.15.143:0/2491406940 learned_addr learned my addr 172.21.15.143:0/2491406940 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:45.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.815+0000 7efc7fe00640 1 --2- 172.21.15.143:0/2491406940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800b24b0 0x7efc800ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7f400640 1 -- 172.21.15.143:0/2491406940 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731641798 0 0) 0x7efc80070bf0 con 0x7efc800ce690 2024-01-15T18:18:45.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7f400640 1 -- 172.21.15.143:0/2491406940 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc60003530 con 0x7efc800ce690 2024-01-15T18:18:45.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7fe00640 1 -- 172.21.15.143:0/2491406940 >> v1:172.21.15.204:6789/0 conn(0x7efc800b3bf0 legacy=0x7efc800b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7f400640 1 -- 172.21.15.143:0/2491406940 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807105302 0 0) 0x7efc800b4920 con 0x7efc800b3bf0 2024-01-15T18:18:45.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7fe00640 1 -- 172.21.15.143:0/2491406940 >> v1:172.21.15.143:6789/0 conn(0x7efc800ce690 legacy=0x7efc800b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7fe00640 1 -- 172.21.15.143:0/2491406940 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800b4c00 con 0x7efc800b24b0 2024-01-15T18:18:45.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7fe00640 1 --2- 172.21.15.143:0/2491406940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800b24b0 0x7efc800ac8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc680097f0 tx=0x7efc6801cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ad5b5de7df910f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7f400640 1 -- 172.21.15.143:0/2491406940 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680aa610 con 0x7efc800b24b0 2024-01-15T18:18:45.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7f400640 1 -- 172.21.15.143:0/2491406940 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc680a6040 con 0x7efc800b24b0 2024-01-15T18:18:45.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.819+0000 7efc7f400640 1 -- 172.21.15.143:0/2491406940 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc680aa970 con 0x7efc800b24b0 2024-01-15T18:18:45.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- 172.21.15.143:0/2491406940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800b24b0 msgr2=0x7efc800ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 --2- 172.21.15.143:0/2491406940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800b24b0 0x7efc800ac8e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc680097f0 tx=0x7efc6801cef0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- 172.21.15.143:0/2491406940 shutdown_connections 2024-01-15T18:18:45.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 --2- 172.21.15.143:0/2491406940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800b24b0 0x7efc800ac8e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- 172.21.15.143:0/2491406940 >> 172.21.15.143:0/2491406940 conn(0x7efc8006c3f0 msgr2=0x7efc8006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:45.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- 172.21.15.143:0/2491406940 shutdown_connections 2024-01-15T18:18:45.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- 172.21.15.143:0/2491406940 wait complete. 2024-01-15T18:18:45.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 Processor -- start 2024-01-15T18:18:45.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- start start 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc800b24b0 0x7efc8013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 0x7efc80168980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800ce690 0x7efc8015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc80168ec0 con 0x7efc800b3bf0 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc80146280 con 0x7efc800b24b0 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc87600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc80146560 con 0x7efc800ce690 2024-01-15T18:18:45.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc7fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 0x7efc80168980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc7fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 0x7efc80168980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47988/0 (socket says 172.21.15.143:47988) 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc7fe00640 1 -- 172.21.15.143:0/2791941760 learned_addr learned my addr 172.21.15.143:0/2791941760 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc85600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800ce690 0x7efc8015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.823+0000 7efc84c00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc800b24b0 0x7efc8013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7fe00640 1 -- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800ce690 msgr2=0x7efc8015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7fe00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800ce690 0x7efc8015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7fe00640 1 -- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc800b24b0 msgr2=0x7efc8013dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7fe00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc800b24b0 0x7efc8013dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7fe00640 1 -- 172.21.15.143:0/2791941760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc68009420 con 0x7efc800b3bf0 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc84c00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc800b24b0 0x7efc8013dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7fe00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 0x7efc80168980 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7efc680190a0 tx=0x7efc68019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68003820 con 0x7efc800b3bf0 2024-01-15T18:18:45.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc68003980 con 0x7efc800b3bf0 2024-01-15T18:18:45.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc68003cf0 con 0x7efc800b3bf0 2024-01-15T18:18:45.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc80146840 con 0x7efc800b3bf0 2024-01-15T18:18:45.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80146e00 con 0x7efc800b3bf0 2024-01-15T18:18:45.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc80067500 con 0x7efc800b3bf0 2024-01-15T18:18:45.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.827+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7efc680aa410 con 0x7efc800b3bf0 2024-01-15T18:18:45.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.831+0000 7efc7cc00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efc58067a60 0x7efc58069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:45.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.831+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7efc6811d2e0 con 0x7efc800b3bf0 2024-01-15T18:18:45.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.831+0000 7efc84c00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efc58067a60 0x7efc58069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:45.825 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.831+0000 7efc84c00640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efc58067a60 0x7efc58069f20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efc70005890 tx=0x7efc70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:45.829 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.835+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7efc680ebc20 con 0x7efc800b3bf0 2024-01-15T18:18:45.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.983+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efc801466a0 con 0x7efc58067a60 2024-01-15T18:18:45.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.991+0000 7efc7cc00640 1 -- 172.21.15.143:0/2791941760 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286013 (secure 0 0 0) 0x7efc801466a0 con 0x7efc58067a60 2024-01-15T18:18:45.989 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:45.992 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:18:45.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efc58067a60 msgr2=0x7efc58069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efc58067a60 0x7efc58069f20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efc70005890 tx=0x7efc70009040 comp rx=0 tx=0).stop 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 msgr2=0x7efc80168980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 0x7efc80168980 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7efc680190a0 tx=0x7efc68019230 comp rx=0 tx=0).stop 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 shutdown_connections 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7efc58067a60 0x7efc58069f20 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7efc800ce690 0x7efc8015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efc800b3bf0 0x7efc80168980 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 --2- 172.21.15.143:0/2791941760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc800b24b0 0x7efc8013dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:45.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 >> 172.21.15.143:0/2791941760 conn(0x7efc8006c3f0 msgr2=0x7efc800af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:45.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 shutdown_connections 2024-01-15T18:18:45.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:45.995+0000 7efc87600640 1 -- 172.21.15.143:0/2791941760 wait complete. 2024-01-15T18:18:46.009 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-01-15T18:18:44.895731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":557816,"kb_used_data":11772,"kb_used_omap":12,"kb_used_meta":545907,"kb_avail":754416904,"statfs":{"total":773094113280,"available":772522909696,"internally_reserved":0,"allocated":12054528,"data_stored":5876835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":559009328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992969"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955237+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:18:41.955237+0000","last_peered":"2024-01-15T18:18:41.955237+0000","last_clean":"2024-01-15T18:18:41.955237+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:18:41.955237+0000","last_undegraded":"2024-01-15T18:18:41.955237+0000","last_fullsized":"2024-01-15T18:18:41.955237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908026+0000","last_change":"2024-01-15T18:18:40.908026+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908026+0000","last_undegraded":"2024-01-15T18:18:40.908026+0000","last_fullsized":"2024-01-15T18:18:40.908026+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913598+0000","last_change":"2024-01-15T18:18:40.913598+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913598+0000","last_undegraded":"2024-01-15T18:18:40.913598+0000","last_fullsized":"2024-01-15T18:18:40.913598+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.945216+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:18:41.945216+0000","last_peered":"2024-01-15T18:18:41.945216+0000","last_clean":"2024-01-15T18:18:41.945216+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:18:41.945216+0000","last_undegraded":"2024-01-15T18:18:41.945216+0000","last_fullsized":"2024-01-15T18:18:41.945216+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.925271+0000","last_change":"2024-01-15T18:18:40.925271+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.925271+0000","last_undegraded":"2024-01-15T18:18:40.925271+0000","last_fullsized":"2024-01-15T18:18:40.925271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954702+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:18:41.954702+0000","last_peered":"2024-01-15T18:18:41.954702+0000","last_clean":"2024-01-15T18:18:41.954702+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:18:41.954702+0000","last_undegraded":"2024-01-15T18:18:41.954702+0000","last_fullsized":"2024-01-15T18:18:41.954702+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:04:30.848650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.919588+0000","last_change":"2024-01-15T18:18:40.919588+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.919588+0000","last_undegraded":"2024-01-15T18:18:40.919588+0000","last_fullsized":"2024-01-15T18:18:40.919588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913325+0000","last_change":"2024-01-15T18:18:40.913325+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913325+0000","last_undegraded":"2024-01-15T18:18:40.913325+0000","last_fullsized":"2024-01-15T18:18:40.913325+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.956014+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:18:41.956014+0000","last_peered":"2024-01-15T18:18:41.956014+0000","last_clean":"2024-01-15T18:18:41.956014+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:18:41.956014+0000","last_undegraded":"2024-01-15T18:18:41.956014+0000","last_fullsized":"2024-01-15T18:18:41.956014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954228+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:18:41.954228+0000","last_peered":"2024-01-15T18:18:41.954228+0000","last_clean":"2024-01-15T18:18:41.954228+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:18:41.954228+0000","last_undegraded":"2024-01-15T18:18:41.954228+0000","last_fullsized":"2024-01-15T18:18:41.954228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907781+0000","last_change":"2024-01-15T18:18:40.907781+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907781+0000","last_undegraded":"2024-01-15T18:18:40.907781+0000","last_fullsized":"2024-01-15T18:18:40.907781+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951965+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:18:41.951965+0000","last_peered":"2024-01-15T18:18:41.951965+0000","last_clean":"2024-01-15T18:18:41.951965+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:18:41.951965+0000","last_undegraded":"2024-01-15T18:18:41.951965+0000","last_fullsized":"2024-01-15T18:18:41.951965+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:27.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913469+0000","last_change":"2024-01-15T18:18:40.913469+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913469+0000","last_undegraded":"2024-01-15T18:18:40.913469+0000","last_fullsized":"2024-01-15T18:18:40.913469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.914537+0000","last_change":"2024-01-15T18:18:40.914537+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.914537+0000","last_undegraded":"2024-01-15T18:18:40.914537+0000","last_fullsized":"2024-01-15T18:18:40.914537+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955421+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:18:41.955421+0000","last_peered":"2024-01-15T18:18:41.955421+0000","last_clean":"2024-01-15T18:18:41.955421+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:18:41.955421+0000","last_undegraded":"2024-01-15T18:18:41.955421+0000","last_fullsized":"2024-01-15T18:18:41.955421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:34.253432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947043+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:18:41.947043+0000","last_peered":"2024-01-15T18:18:41.947043+0000","last_clean":"2024-01-15T18:18:41.947043+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:18:41.947043+0000","last_undegraded":"2024-01-15T18:18:41.947043+0000","last_fullsized":"2024-01-15T18:18:41.947043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.910456+0000","last_change":"2024-01-15T18:18:40.910456+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.910456+0000","last_undegraded":"2024-01-15T18:18:40.910456+0000","last_fullsized":"2024-01-15T18:18:40.910456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909936+0000","last_change":"2024-01-15T18:18:40.909936+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909936+0000","last_undegraded":"2024-01-15T18:18:40.909936+0000","last_fullsized":"2024-01-15T18:18:40.909936+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913060+0000","last_change":"2024-01-15T18:18:40.913060+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913060+0000","last_undegraded":"2024-01-15T18:18:40.913060+0000","last_fullsized":"2024-01-15T18:18:40.913060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907829+0000","last_change":"2024-01-15T18:18:40.907829+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907829+0000","last_undegraded":"2024-01-15T18:18:40.907829+0000","last_fullsized":"2024-01-15T18:18:40.907829+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907707+0000","last_change":"2024-01-15T18:18:40.907707+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907707+0000","last_undegraded":"2024-01-15T18:18:40.907707+0000","last_fullsized":"2024-01-15T18:18:40.907707+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955395+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:18:41.955395+0000","last_peered":"2024-01-15T18:18:41.955395+0000","last_clean":"2024-01-15T18:18:41.955395+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:18:41.955395+0000","last_undegraded":"2024-01-15T18:18:41.955395+0000","last_fullsized":"2024-01-15T18:18:41.955395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.906954+0000","last_change":"2024-01-15T18:18:40.906954+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.906954+0000","last_undegraded":"2024-01-15T18:18:40.906954+0000","last_fullsized":"2024-01-15T18:18:40.906954+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955258+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:18:41.955258+0000","last_peered":"2024-01-15T18:18:41.955258+0000","last_clean":"2024-01-15T18:18:41.955258+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:18:41.955258+0000","last_undegraded":"2024-01-15T18:18:41.955258+0000","last_fullsized":"2024-01-15T18:18:41.955258+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908132+0000","last_change":"2024-01-15T18:18:40.908132+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908132+0000","last_undegraded":"2024-01-15T18:18:40.908132+0000","last_fullsized":"2024-01-15T18:18:40.908132+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955709+0000","last_change":"2024-01-15T18:18:41.949864+0000","last_active":"2024-01-15T18:18:41.955709+0000","last_peered":"2024-01-15T18:18:41.955709+0000","last_clean":"2024-01-15T18:18:41.955709+0000","last_became_active":"2024-01-15T18:18:41.944900+0000","last_became_peered":"2024-01-15T18:18:41.944900+0000","last_unstale":"2024-01-15T18:18:41.955709+0000","last_undegraded":"2024-01-15T18:18:41.955709+0000","last_fullsized":"2024-01-15T18:18:41.955709+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933525+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:18:41.933525+0000","last_peered":"2024-01-15T18:18:41.933525+0000","last_clean":"2024-01-15T18:18:41.933525+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:18:41.933525+0000","last_undegraded":"2024-01-15T18:18:41.933525+0000","last_fullsized":"2024-01-15T18:18:41.933525+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:28.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957639+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:18:41.957639+0000","last_peered":"2024-01-15T18:18:41.957639+0000","last_clean":"2024-01-15T18:18:41.957639+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:18:41.957639+0000","last_undegraded":"2024-01-15T18:18:41.957639+0000","last_fullsized":"2024-01-15T18:18:41.957639+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.910848+0000","last_change":"2024-01-15T18:18:40.910848+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.910848+0000","last_undegraded":"2024-01-15T18:18:40.910848+0000","last_fullsized":"2024-01-15T18:18:40.910848+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.917233+0000","last_change":"2024-01-15T18:18:40.917233+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.917233+0000","last_undegraded":"2024-01-15T18:18:40.917233+0000","last_fullsized":"2024-01-15T18:18:40.917233+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955724+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:18:41.955724+0000","last_peered":"2024-01-15T18:18:41.955724+0000","last_clean":"2024-01-15T18:18:41.955724+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:18:41.955724+0000","last_undegraded":"2024-01-15T18:18:41.955724+0000","last_fullsized":"2024-01-15T18:18:41.955724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955368+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:18:41.955368+0000","last_peered":"2024-01-15T18:18:41.955368+0000","last_clean":"2024-01-15T18:18:41.955368+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:18:41.955368+0000","last_undegraded":"2024-01-15T18:18:41.955368+0000","last_fullsized":"2024-01-15T18:18:41.955368+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954348+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:18:41.954348+0000","last_peered":"2024-01-15T18:18:41.954348+0000","last_clean":"2024-01-15T18:18:41.954348+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:18:41.954348+0000","last_undegraded":"2024-01-15T18:18:41.954348+0000","last_fullsized":"2024-01-15T18:18:41.954348+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:14.769436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.927486+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:18:40.927486+0000","last_peered":"2024-01-15T18:18:40.927486+0000","last_clean":"2024-01-15T18:18:40.927486+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:18:40.927486+0000","last_undegraded":"2024-01-15T18:18:40.927486+0000","last_fullsized":"2024-01-15T18:18:40.927486+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:17.166792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908694+0000","last_change":"2024-01-15T18:18:40.908694+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908694+0000","last_undegraded":"2024-01-15T18:18:40.908694+0000","last_fullsized":"2024-01-15T18:18:40.908694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925246+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:18:40.925246+0000","last_peered":"2024-01-15T18:18:40.925246+0000","last_clean":"2024-01-15T18:18:40.925246+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:18:40.925246+0000","last_undegraded":"2024-01-15T18:18:40.925246+0000","last_fullsized":"2024-01-15T18:18:40.925246+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:34.563635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954412+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:18:41.954412+0000","last_peered":"2024-01-15T18:18:41.954412+0000","last_clean":"2024-01-15T18:18:41.954412+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:18:41.954412+0000","last_undegraded":"2024-01-15T18:18:41.954412+0000","last_fullsized":"2024-01-15T18:18:41.954412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:49.284112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946203+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:18:40.946203+0000","last_peered":"2024-01-15T18:18:40.946203+0000","last_clean":"2024-01-15T18:18:40.946203+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:18:40.946203+0000","last_undegraded":"2024-01-15T18:18:40.946203+0000","last_fullsized":"2024-01-15T18:18:40.946203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:09.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912545+0000","last_change":"2024-01-15T18:18:40.912545+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912545+0000","last_undegraded":"2024-01-15T18:18:40.912545+0000","last_fullsized":"2024-01-15T18:18:40.912545+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946121+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:18:40.946121+0000","last_peered":"2024-01-15T18:18:40.946121+0000","last_clean":"2024-01-15T18:18:40.946121+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:18:40.946121+0000","last_undegraded":"2024-01-15T18:18:40.946121+0000","last_fullsized":"2024-01-15T18:18:40.946121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:10.172303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955156+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:18:41.955156+0000","last_peered":"2024-01-15T18:18:41.955156+0000","last_clean":"2024-01-15T18:18:41.955156+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:18:41.955156+0000","last_undegraded":"2024-01-15T18:18:41.955156+0000","last_fullsized":"2024-01-15T18:18:41.955156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.932352+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:18:40.932352+0000","last_peered":"2024-01-15T18:18:40.932352+0000","last_clean":"2024-01-15T18:18:40.932352+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:18:40.932352+0000","last_undegraded":"2024-01-15T18:18:40.932352+0000","last_fullsized":"2024-01-15T18:18:40.932352+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:23.333161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909660+0000","last_change":"2024-01-15T18:18:40.909660+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909660+0000","last_undegraded":"2024-01-15T18:18:40.909660+0000","last_fullsized":"2024-01-15T18:18:40.909660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949383+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:18:41.949383+0000","last_peered":"2024-01-15T18:18:41.949383+0000","last_clean":"2024-01-15T18:18:41.949383+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:18:41.949383+0000","last_undegraded":"2024-01-15T18:18:41.949383+0000","last_fullsized":"2024-01-15T18:18:41.949383+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954520+0000","last_change":"2024-01-15T18:18:41.940718+0000","last_active":"2024-01-15T18:18:41.954520+0000","last_peered":"2024-01-15T18:18:41.954520+0000","last_clean":"2024-01-15T18:18:41.954520+0000","last_became_active":"2024-01-15T18:18:41.940432+0000","last_became_peered":"2024-01-15T18:18:41.940432+0000","last_unstale":"2024-01-15T18:18:41.954520+0000","last_undegraded":"2024-01-15T18:18:41.954520+0000","last_fullsized":"2024-01-15T18:18:41.954520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925490+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:18:40.925490+0000","last_peered":"2024-01-15T18:18:40.925490+0000","last_clean":"2024-01-15T18:18:40.925490+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:18:40.925490+0000","last_undegraded":"2024-01-15T18:18:40.925490+0000","last_fullsized":"2024-01-15T18:18:40.925490+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:41.088175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.918761+0000","last_change":"2024-01-15T18:18:40.918761+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.918761+0000","last_undegraded":"2024-01-15T18:18:40.918761+0000","last_fullsized":"2024-01-15T18:18:40.918761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918022+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:18:41.918022+0000","last_peered":"2024-01-15T18:18:41.918022+0000","last_clean":"2024-01-15T18:18:41.918022+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:18:41.918022+0000","last_undegraded":"2024-01-15T18:18:41.918022+0000","last_fullsized":"2024-01-15T18:18:41.918022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.916907+0000","last_change":"2024-01-15T18:18:40.916907+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.916907+0000","last_undegraded":"2024-01-15T18:18:40.916907+0000","last_fullsized":"2024-01-15T18:18:40.916907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946140+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:18:40.946140+0000","last_peered":"2024-01-15T18:18:40.946140+0000","last_clean":"2024-01-15T18:18:40.946140+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:18:40.946140+0000","last_undegraded":"2024-01-15T18:18:40.946140+0000","last_fullsized":"2024-01-15T18:18:40.946140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907383+0000","last_change":"2024-01-15T18:18:40.907383+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907383+0000","last_undegraded":"2024-01-15T18:18:40.907383+0000","last_fullsized":"2024-01-15T18:18:40.907383+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.926304+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:18:41.926304+0000","last_peered":"2024-01-15T18:18:41.926304+0000","last_clean":"2024-01-15T18:18:41.926304+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:18:41.926304+0000","last_undegraded":"2024-01-15T18:18:41.926304+0000","last_fullsized":"2024-01-15T18:18:41.926304+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955327+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:18:41.955327+0000","last_peered":"2024-01-15T18:18:41.955327+0000","last_clean":"2024-01-15T18:18:41.955327+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:18:41.955327+0000","last_undegraded":"2024-01-15T18:18:41.955327+0000","last_fullsized":"2024-01-15T18:18:41.955327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957957+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:18:41.957957+0000","last_peered":"2024-01-15T18:18:41.957957+0000","last_clean":"2024-01-15T18:18:41.957957+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:18:41.957957+0000","last_undegraded":"2024-01-15T18:18:41.957957+0000","last_fullsized":"2024-01-15T18:18:41.957957+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.916144+0000","last_change":"2024-01-15T18:18:40.916144+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.916144+0000","last_undegraded":"2024-01-15T18:18:40.916144+0000","last_fullsized":"2024-01-15T18:18:40.916144+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.960177+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:18:41.960177+0000","last_peered":"2024-01-15T18:18:41.960177+0000","last_clean":"2024-01-15T18:18:41.960177+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:18:41.960177+0000","last_undegraded":"2024-01-15T18:18:41.960177+0000","last_fullsized":"2024-01-15T18:18:41.960177+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955377+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:18:41.955377+0000","last_peered":"2024-01-15T18:18:41.955377+0000","last_clean":"2024-01-15T18:18:41.955377+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:18:41.955377+0000","last_undegraded":"2024-01-15T18:18:41.955377+0000","last_fullsized":"2024-01-15T18:18:41.955377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949761+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:18:41.949761+0000","last_peered":"2024-01-15T18:18:41.949761+0000","last_clean":"2024-01-15T18:18:41.949761+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:18:41.949761+0000","last_undegraded":"2024-01-15T18:18:41.949761+0000","last_fullsized":"2024-01-15T18:18:41.949761+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955653+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:18:41.955653+0000","last_peered":"2024-01-15T18:18:41.955653+0000","last_clean":"2024-01-15T18:18:41.955653+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:18:41.955653+0000","last_undegraded":"2024-01-15T18:18:41.955653+0000","last_fullsized":"2024-01-15T18:18:41.955653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.702381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.923005+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:18:40.923005+0000","last_peered":"2024-01-15T18:18:40.923005+0000","last_clean":"2024-01-15T18:18:40.923005+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:18:40.923005+0000","last_undegraded":"2024-01-15T18:18:40.923005+0000","last_fullsized":"2024-01-15T18:18:40.923005+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:16.430602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955766+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:18:41.955766+0000","last_peered":"2024-01-15T18:18:41.955766+0000","last_clean":"2024-01-15T18:18:41.955766+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:18:41.955766+0000","last_undegraded":"2024-01-15T18:18:41.955766+0000","last_fullsized":"2024-01-15T18:18:41.955766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:17.438389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.939931+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:18:40.939931+0000","last_peered":"2024-01-15T18:18:40.939931+0000","last_clean":"2024-01-15T18:18:40.939931+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:18:40.939931+0000","last_undegraded":"2024-01-15T18:18:40.939931+0000","last_fullsized":"2024-01-15T18:18:40.939931+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:30.432388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.921779+0000","last_change":"2024-01-15T18:18:40.921779+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.921779+0000","last_undegraded":"2024-01-15T18:18:40.921779+0000","last_fullsized":"2024-01-15T18:18:40.921779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.946443+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:18:41.946443+0000","last_peered":"2024-01-15T18:18:41.946443+0000","last_clean":"2024-01-15T18:18:41.946443+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:18:41.946443+0000","last_undegraded":"2024-01-15T18:18:41.946443+0000","last_fullsized":"2024-01-15T18:18:41.946443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933399+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:18:41.933399+0000","last_peered":"2024-01-15T18:18:41.933399+0000","last_clean":"2024-01-15T18:18:41.933399+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:18:41.933399+0000","last_undegraded":"2024-01-15T18:18:41.933399+0000","last_fullsized":"2024-01-15T18:18:41.933399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.408977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931086+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:18:40.931086+0000","last_peered":"2024-01-15T18:18:40.931086+0000","last_clean":"2024-01-15T18:18:40.931086+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:18:40.931086+0000","last_undegraded":"2024-01-15T18:18:40.931086+0000","last_fullsized":"2024-01-15T18:18:40.931086+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:51:44.688957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.911900+0000","last_change":"2024-01-15T18:18:40.911900+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.911900+0000","last_undegraded":"2024-01-15T18:18:40.911900+0000","last_fullsized":"2024-01-15T18:18:40.911900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.923118+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:18:40.923118+0000","last_peered":"2024-01-15T18:18:40.923118+0000","last_clean":"2024-01-15T18:18:40.923118+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:18:40.923118+0000","last_undegraded":"2024-01-15T18:18:40.923118+0000","last_fullsized":"2024-01-15T18:18:40.923118+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:15.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.911924+0000","last_change":"2024-01-15T18:18:40.911924+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.911924+0000","last_undegraded":"2024-01-15T18:18:40.911924+0000","last_fullsized":"2024-01-15T18:18:40.911924+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931943+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:18:40.931943+0000","last_peered":"2024-01-15T18:18:40.931943+0000","last_clean":"2024-01-15T18:18:40.931943+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:18:40.931943+0000","last_undegraded":"2024-01-15T18:18:40.931943+0000","last_fullsized":"2024-01-15T18:18:40.931943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:46.912158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.914035+0000","last_change":"2024-01-15T18:18:40.914035+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.914035+0000","last_undegraded":"2024-01-15T18:18:40.914035+0000","last_fullsized":"2024-01-15T18:18:40.914035+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958826+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:18:41.958826+0000","last_peered":"2024-01-15T18:18:41.958826+0000","last_clean":"2024-01-15T18:18:41.958826+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:18:41.958826+0000","last_undegraded":"2024-01-15T18:18:41.958826+0000","last_fullsized":"2024-01-15T18:18:41.958826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.940187+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:18:41.940187+0000","last_peered":"2024-01-15T18:18:41.940187+0000","last_clean":"2024-01-15T18:18:41.940187+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:18:41.940187+0000","last_undegraded":"2024-01-15T18:18:41.940187+0000","last_fullsized":"2024-01-15T18:18:41.940187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.947226+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:18:40.947226+0000","last_peered":"2024-01-15T18:18:40.947226+0000","last_clean":"2024-01-15T18:18:40.947226+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:18:40.947226+0000","last_undegraded":"2024-01-15T18:18:40.947226+0000","last_fullsized":"2024-01-15T18:18:40.947226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:53.075302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912833+0000","last_change":"2024-01-15T18:18:40.912833+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912833+0000","last_undegraded":"2024-01-15T18:18:40.912833+0000","last_fullsized":"2024-01-15T18:18:40.912833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.924617+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:18:41.924617+0000","last_peered":"2024-01-15T18:18:41.924617+0000","last_clean":"2024-01-15T18:18:41.924617+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:18:41.924617+0000","last_undegraded":"2024-01-15T18:18:41.924617+0000","last_fullsized":"2024-01-15T18:18:41.924617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947927+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:18:41.947927+0000","last_peered":"2024-01-15T18:18:41.947927+0000","last_clean":"2024-01-15T18:18:41.947927+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:18:41.947927+0000","last_undegraded":"2024-01-15T18:18:41.947927+0000","last_fullsized":"2024-01-15T18:18:41.947927+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:43.903199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918200+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:18:41.918200+0000","last_peered":"2024-01-15T18:18:41.918200+0000","last_clean":"2024-01-15T18:18:41.918200+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:18:41.918200+0000","last_undegraded":"2024-01-15T18:18:41.918200+0000","last_fullsized":"2024-01-15T18:18:41.918200+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908189+0000","last_change":"2024-01-15T18:18:40.908189+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908189+0000","last_undegraded":"2024-01-15T18:18:40.908189+0000","last_fullsized":"2024-01-15T18:18:40.908189+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957662+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:18:41.957662+0000","last_peered":"2024-01-15T18:18:41.957662+0000","last_clean":"2024-01-15T18:18:41.957662+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:18:41.957662+0000","last_undegraded":"2024-01-15T18:18:41.957662+0000","last_fullsized":"2024-01-15T18:18:41.957662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.944610+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:18:41.944610+0000","last_peered":"2024-01-15T18:18:41.944610+0000","last_clean":"2024-01-15T18:18:41.944610+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:18:41.944610+0000","last_undegraded":"2024-01-15T18:18:41.944610+0000","last_fullsized":"2024-01-15T18:18:41.944610+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925197+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:18:40.925197+0000","last_peered":"2024-01-15T18:18:40.925197+0000","last_clean":"2024-01-15T18:18:40.925197+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:18:40.925197+0000","last_undegraded":"2024-01-15T18:18:40.925197+0000","last_fullsized":"2024-01-15T18:18:40.925197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:41.088175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951944+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:18:41.951944+0000","last_peered":"2024-01-15T18:18:41.951944+0000","last_clean":"2024-01-15T18:18:41.951944+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:18:41.951944+0000","last_undegraded":"2024-01-15T18:18:41.951944+0000","last_fullsized":"2024-01-15T18:18:41.951944+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918797+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:18:41.918797+0000","last_peered":"2024-01-15T18:18:41.918797+0000","last_clean":"2024-01-15T18:18:41.918797+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:18:41.918797+0000","last_undegraded":"2024-01-15T18:18:41.918797+0000","last_fullsized":"2024-01-15T18:18:41.918797+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907378+0000","last_change":"2024-01-15T18:18:40.907378+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907378+0000","last_undegraded":"2024-01-15T18:18:40.907378+0000","last_fullsized":"2024-01-15T18:18:40.907378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.927483+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:18:41.927483+0000","last_peered":"2024-01-15T18:18:41.927483+0000","last_clean":"2024-01-15T18:18:41.927483+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:18:41.927483+0000","last_undegraded":"2024-01-15T18:18:41.927483+0000","last_fullsized":"2024-01-15T18:18:41.927483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908276+0000","last_change":"2024-01-15T18:18:40.908276+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908276+0000","last_undegraded":"2024-01-15T18:18:40.908276+0000","last_fullsized":"2024-01-15T18:18:40.908276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.927500+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:18:40.927500+0000","last_peered":"2024-01-15T18:18:40.927500+0000","last_clean":"2024-01-15T18:18:40.927500+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:18:40.927500+0000","last_undegraded":"2024-01-15T18:18:40.927500+0000","last_fullsized":"2024-01-15T18:18:40.927500+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:33.731825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909185+0000","last_change":"2024-01-15T18:18:40.909185+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909185+0000","last_undegraded":"2024-01-15T18:18:40.909185+0000","last_fullsized":"2024-01-15T18:18:40.909185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.936592+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:18:40.936592+0000","last_peered":"2024-01-15T18:18:40.936592+0000","last_clean":"2024-01-15T18:18:40.936592+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:18:40.936592+0000","last_undegraded":"2024-01-15T18:18:40.936592+0000","last_fullsized":"2024-01-15T18:18:40.936592+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:15.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912287+0000","last_change":"2024-01-15T18:18:40.912287+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912287+0000","last_undegraded":"2024-01-15T18:18:40.912287+0000","last_fullsized":"2024-01-15T18:18:40.912287+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946157+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:18:40.946157+0000","last_peered":"2024-01-15T18:18:40.946157+0000","last_clean":"2024-01-15T18:18:40.946157+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:18:40.946157+0000","last_undegraded":"2024-01-15T18:18:40.946157+0000","last_fullsized":"2024-01-15T18:18:40.946157+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:50.249980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955068+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:18:41.955068+0000","last_peered":"2024-01-15T18:18:41.955068+0000","last_clean":"2024-01-15T18:18:41.955068+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:18:41.955068+0000","last_undegraded":"2024-01-15T18:18:41.955068+0000","last_fullsized":"2024-01-15T18:18:41.955068+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918226+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:18:41.918226+0000","last_peered":"2024-01-15T18:18:41.918226+0000","last_clean":"2024-01-15T18:18:41.918226+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:18:41.918226+0000","last_undegraded":"2024-01-15T18:18:41.918226+0000","last_fullsized":"2024-01-15T18:18:41.918226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.953440+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:18:41.953440+0000","last_peered":"2024-01-15T18:18:41.953440+0000","last_clean":"2024-01-15T18:18:41.953440+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:18:41.953440+0000","last_undegraded":"2024-01-15T18:18:41.953440+0000","last_fullsized":"2024-01-15T18:18:41.953440+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:52.321006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.930839+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:18:40.930839+0000","last_peered":"2024-01-15T18:18:40.930839+0000","last_clean":"2024-01-15T18:18:40.930839+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:18:40.930839+0000","last_undegraded":"2024-01-15T18:18:40.930839+0000","last_fullsized":"2024-01-15T18:18:40.930839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:47.752921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":239,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-01-15T18:18:39.904797+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:18:39.904797+0000","last_peered":"2024-01-15T18:18:39.904797+0000","last_clean":"2024-01-15T18:18:39.904797+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:18:39.904797+0000","last_undegraded":"2024-01-15T18:18:39.904797+0000","last_fullsized":"2024-01-15T18:18:39.904797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575019,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64960,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306880,"statfs":{"total":96636764160,"available":96570245120,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.86399999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.622},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.77400000000000002,"15min":0.85999999999999999},"min":{"1min":0.48299999999999998,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.92500000000000004,"5min":4.6100000000000003,"15min":13.744},"last":0.60799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.96799999999999997,"15min":1.0640000000000001},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":3.016,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.81100000000000005,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":4.7679999999999998,"15min":31.774999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.74099999999999999,"15min":0.98699999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":2.1680000000000001,"15min":30.846},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":1.073},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":17.039999999999999,"15min":39.695},"last":0.65300000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.82599999999999996,"15min":1.1579999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.88400000000000001,"5min":8.2650000000000006,"15min":26.463999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.94699999999999995,"15min":1.202},"min":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":16.879999999999999,"15min":42.692},"last":0.68200000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78800000000000003,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":1.147,"5min":14.087,"15min":14.087},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.629,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.63500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.81299999999999994,"15min":0.71199999999999997},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":1.0209999999999999,"5min":11.334,"15min":11.334},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":4.8179999999999996,"15min":8.423},"last":0.69299999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5369999999999999,"5min":4.7140000000000004,"15min":8.5030000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1970000000000001,"15min":3.3450000000000002},"last":0.45800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.81699999999999995,"15min":0.95299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.754},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74199999999999999,"15min":0.873},"min":{"1min":0.375,"5min":0.375,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":3.4940000000000002,"15min":12.449},"last":0.82599999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.85599999999999998,"15min":0.89700000000000002},"min":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.33900000000000002},"max":{"1min":1.0489999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.84199999999999997,"15min":1.004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.81899999999999995,"5min":8.9320000000000004,"15min":19.475000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.069,"15min":1.179},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":14.743,"15min":42.825000000000003},"last":0.997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.89000000000000001,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":9.4000000000000004,"15min":42.851999999999997},"last":1.0720000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78300000000000003,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":0.82099999999999995,"5min":3.016,"15min":24.199000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0629999999999999,"15min":1.087},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":2.956,"5min":18.791,"15min":18.791},"last":0.872}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.89300000000000002,"15min":0.745},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":19.114999999999998,"15min":19.114999999999998},"last":1.044},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33700000000000002},"max":{"1min":0.875,"5min":1.7450000000000001,"15min":12.491},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81899999999999995,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.89800000000000002},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.5259999999999998,"15min":12.945},"last":1.1120000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":1.006,"15min":0.83899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":11.907,"15min":11.907},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82499999999999996,"15min":0.88400000000000001},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":9.9440000000000008,"15min":27.651},"last":1.1910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76860,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294980,"statfs":{"total":96636764160,"available":96558059520,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":4.4770000000000003,"15min":11.519},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.874,"15min":0.92400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.91700000000000004,"5min":7.6260000000000003,"15min":12.952},"last":0.68899999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.92700000000000005,"15min":0.90300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.95499999999999996,"5min":4.3140000000000001,"15min":13.358000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.94699999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0169999999999999,"5min":4.1580000000000004,"15min":21.856999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":1,"5min":6.7160000000000002,"15min":18.314},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.145,"15min":1.0109999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.0580000000000001,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.752}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.85499999999999998,"15min":0.94499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":3.2400000000000002,"15min":13.317},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":1.01},"min":{"1min":0.437,"5min":0.437,"15min":0.42899999999999999},"max":{"1min":0.998,"5min":3.931,"15min":19.638000000000002},"last":0.72799999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":0.96099999999999997,"5min":5.899,"15min":11.093},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":0.88700000000000001,"5min":2.516,"15min":3.3889999999999998},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84399999999999997,"15min":0.79500000000000004},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97899999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.308},"max":{"1min":1.0840000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.85899999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":1.071,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":1.034,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81020,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290820,"statfs":{"total":96636764160,"available":96553799680,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.79800000000000004,"15min":0.93300000000000005},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":9.4179999999999993,"15min":34.802},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":14.488,"15min":22.335999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.95699999999999996},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":2.2130000000000001,"15min":38.198},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.80500000000000005,"15min":0.95599999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.86599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95099999999999996,"5min":1.6259999999999999,"15min":7.6989999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.78500000000000003,"15min":0.84799999999999998},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":4.4089999999999998,"15min":7.4829999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.91300000000000003,"15min":0.95399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":14.529,"15min":14.529},"last":0.628},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72299999999999998,"15min":0.92900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":1.4279999999999999,"15min":25.774000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":0.68200000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.97799999999999998,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.47499999999999998,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":25.887,"15min":25.887},"last":0.36299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":10.284000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.46200000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.82799999999999996,"15min":0.78300000000000003},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.77100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"max":{"1min":1.03,"5min":1.232,"15min":11.67},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.625,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.255},"max":{"1min":0.81899999999999995,"5min":7.2839999999999998,"15min":26.364000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.66300000000000003,"15min":0.95599999999999996},"min":{"1min":0.312,"5min":0.25,"15min":0.246},"max":{"1min":0.71499999999999997,"5min":7.8520000000000003,"15min":28.332000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.60799999999999998,"15min":0.98799999999999999},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.84799999999999998,"5min":3.8100000000000001,"15min":40.457999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.76300000000000001,"15min":1.1419999999999999},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":9.282,"15min":27.427},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.71199999999999997,"15min":0.98899999999999999},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.90400000000000003,"5min":9.1180000000000003,"15min":23.036999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.81499999999999995,"15min":1.089},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":1.048,"5min":9.3510000000000009,"15min":26.404},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":1.024},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.41199999999999998},"max":{"1min":1.0780000000000001,"5min":4.1040000000000001,"15min":22.998000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84599999999999997,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.0229999999999999,"5min":3.8540000000000001,"15min":20.399000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":1.097},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":6.5140000000000002,"15min":26.314},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85599999999999998,"15min":1.1220000000000001},"min":{"1min":0.46100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":4.165,"15min":22.908999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":1.048,"15min":1.153},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.42799999999999999},"max":{"1min":1.0609999999999999,"5min":6.843,"15min":20.445},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.1850000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.123,"5min":6.7640000000000002,"15min":26.091999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.98999999999999999,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":7.0250000000000004,"15min":23.111999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.92000000000000004,"15min":1.321},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46000000000000002},"max":{"1min":1.1419999999999999,"5min":4.4470000000000001,"15min":44.817999999999998},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64380,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307460,"statfs":{"total":96636764160,"available":96570839040,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54400000000000004,"15min":1.0529999999999999},"min":{"1min":0.31,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.373},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.86199999999999999,"15min":1.321},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.66400000000000003,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.39200000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.64700000000000002,"15min":1.5409999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.56299999999999994,"5min":7.476,"15min":113.149},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.96899999999999997,"15min":1.1080000000000001},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":20.619,"15min":41.302},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.341,"15min":1.379},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28899999999999998},"max":{"1min":0.67400000000000004,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66000000000000003,"15min":1.27},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":0.77300000000000002,"5min":7.3609999999999998,"15min":115.693},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":1.206},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":7.4169999999999998,"15min":24.366},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0740000000000001,"15min":1.23},"min":{"1min":0.51600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.5800000000000001,"15min":1.5089999999999999},"min":{"1min":0.54700000000000004,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84999999999999998,"15min":1.304},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":7.5099999999999998,"15min":26.084},"last":0.54900000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":1.0980000000000001,"15min":1.3149999999999999},"min":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":20.120999999999999,"15min":22.199000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.748,"5min":1.6240000000000001,"15min":1.5700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":1.165,"15min":1.262},"min":{"1min":0.55200000000000005,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":0.92600000000000005,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":1.613,"15min":1.6299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64404,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307436,"statfs":{"total":96636764160,"available":96570814464,"internally_reserved":0,"allocated":1376256,"data_stored":593098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.64200000000000002,"15min":0.91500000000000004},"min":{"1min":0.245,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.68200000000000005,"5min":9.452,"15min":39.948},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.68799999999999994,"15min":1.103},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":8.141,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.71799999999999997,"15min":0.98199999999999998},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":0.63800000000000001,"5min":15.137,"15min":49.229999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61799999999999999,"5min":9.8469999999999995,"15min":10.349},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61099999999999999,"15min":1.173},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":4.4669999999999996,"15min":27.643999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.90000000000000002,"15min":0.997},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.66800000000000004,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84299999999999997,"15min":1.222},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":4.633,"15min":27.489999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67600000000000005,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98599999999999999,"5min":9.6989999999999998,"15min":12.997},"last":0.83599999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.86599999999999999,"15min":1.212},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":4.673,"15min":27.547000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.0289999999999999,"15min":1.1200000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.496},"max":{"1min":1.028,"5min":9.7650000000000006,"15min":13.054},"last":0.78100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.86499999999999999,"15min":1.204},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":4.7060000000000004,"15min":27.579999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.87,"15min":1.0369999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.1319999999999999},"min":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":6.6950000000000003,"5min":9.5660000000000007,"15min":11.833},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1.0229999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.502},"max":{"1min":0.93200000000000005,"5min":9.8079999999999998,"15min":14.606},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.78600000000000003,"15min":1.1379999999999999},"min":{"1min":0.27100000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.94299999999999995,"5min":11.356,"15min":31.977},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.81299999999999994,"15min":1.0229999999999999},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0289999999999999,"5min":11.629,"15min":36.145000000000003},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.76600000000000001,"15min":1.1319999999999999},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":0.91800000000000004,"5min":5.8929999999999998,"15min":41.387999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.877,"15min":1.038},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.25},"max":{"1min":0.97499999999999998,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.97599999999999998,"15min":1.0860000000000001},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.093,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.79400000000000004,"15min":1.097},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":6.3739999999999997,"15min":45.128},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.0289999999999999,"15min":1.0740000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.372},"max":{"1min":2.1469999999999998,"5min":11.772,"15min":32.932000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.0840000000000001,"15min":1.1919999999999999},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":1.4370000000000001,"5min":11.220000000000001,"15min":31.782},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1.083,"15min":1.1850000000000001},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3979999999999999,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.841,"5min":11.958,"15min":33.151000000000003},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":1.0640000000000001},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.379,"5min":4.0970000000000004,"15min":13.055},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.749,"5min":1.0449999999999999,"15min":1.0920000000000001},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.522,"5min":11.833,"15min":33.098999999999997},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.92300000000000004,"15min":1.0740000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3160000000000001,"5min":4.1349999999999998,"15min":13.090999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.1120000000000001,"15min":1.242},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.464,"5min":11.321999999999999,"15min":31.905999999999999},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:46.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:46.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.195+0000 7facd1600640 1 Processor -- start 2024-01-15T18:18:46.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.195+0000 7facd1600640 1 -- start start 2024-01-15T18:18:46.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facd1600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faccc0cf7e0 0x7faccc0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facd1600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc15b5b0 con 0x7faccc0cda70 2024-01-15T18:18:46.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facd1600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0cba70 con 0x7faccc0cf400 2024-01-15T18:18:46.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facd1600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc0cbd10 con 0x7faccc0cf7e0 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faccc0cda70 0x7faccc0cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38018/0 (socket says 172.21.15.143:38018) 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 -- 172.21.15.143:0/1733579435 learned_addr learned my addr 172.21.15.143:0/1733579435 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccb400640 1 --2- 172.21.15.143:0/1733579435 >> v2:172.21.15.204:3300/0 conn(0x7faccc0cf7e0 0x7faccc0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122951042 0 0) 0x7faccc15b5b0 con 0x7faccc0cda70 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facac003530 con 0x7faccc0cda70 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885576970 0 0) 0x7faccc0cba70 con 0x7faccc0cf400 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faccc15b5b0 con 0x7faccc0cf400 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 552300594 0 0) 0x7facac003530 con 0x7faccc0cda70 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 >> v2:172.21.15.204:3300/0 conn(0x7faccc0cf7e0 msgr2=0x7faccc0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:46.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 --2- 172.21.15.143:0/1733579435 >> v2:172.21.15.204:3300/0 conn(0x7faccc0cf7e0 0x7faccc0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 >> v1:172.21.15.204:6789/0 conn(0x7faccc0cf400 legacy=0x7faccc0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0cbff0 con 0x7faccc0cda70 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7facbc001680 con 0x7faccc0cda70 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 >> v1:172.21.15.143:6789/0 conn(0x7faccc0cda70 legacy=0x7faccc0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7facac003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facac0047d0 0x7facac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facac005190 0x7facac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0cbd10 con 0x7facac003b10 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac00c010 con 0x7facac0047d0 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac00c1d0 con 0x7facac005190 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7facac003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7facca000640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facac0047d0 0x7facac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccb400640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facac005190 0x7facac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 -- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facac005190 msgr2=0x7facac00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7facac005190 0x7facac00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 -- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facac0047d0 msgr2=0x7facac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:46.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7facac0047d0 0x7facac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccaa00640 1 -- 172.21.15.143:0/1733579435 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac003530 con 0x7facac003b10 2024-01-15T18:18:46.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.199+0000 7faccb400640 1 -- 172.21.15.143:0/1733579435 reap_dead start 2024-01-15T18:18:46.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7faccaa00640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7facac003f10 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7facbc00c160 tx=0x7facbc01cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c22ebd5e2eabe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0058e0 con 0x7facac003b10 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc005a40 con 0x7facac003b10 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facc9600640 1 -- 172.21.15.143:0/1733579435 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc005df0 con 0x7facac003b10 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 -- 172.21.15.143:0/1733579435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 msgr2=0x7facac003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7facac003f10 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7facbc00c160 tx=0x7facbc01cb90 comp rx=0 tx=0).stop 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 -- 172.21.15.143:0/1733579435 shutdown_connections 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 --2- 172.21.15.143:0/1733579435 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7facac003f10 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 -- 172.21.15.143:0/1733579435 >> 172.21.15.143:0/1733579435 conn(0x7faccc0d0e20 msgr2=0x7faccc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:46.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 -- 172.21.15.143:0/1733579435 shutdown_connections 2024-01-15T18:18:46.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.203+0000 7facd1600640 1 -- 172.21.15.143:0/1733579435 wait complete. 2024-01-15T18:18:46.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 Processor -- start 2024-01-15T18:18:46.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- start start 2024-01-15T18:18:46.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7faccc0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc150230 0x7faccc0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faccc150690 0x7faccc0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c9580 con 0x7facac003b10 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c8970 con 0x7faccc150230 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c8c50 con 0x7faccc150690 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7faccc0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7faccc0c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48018/0 (socket says 172.21.15.143:48018) 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 -- 172.21.15.143:0/1574987628 learned_addr learned my addr 172.21.15.143:0/1574987628 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccb400640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faccc150690 0x7faccc0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facca000640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc150230 0x7faccc0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 -- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faccc150690 msgr2=0x7faccc0ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faccc150690 0x7faccc0ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 -- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc150230 msgr2=0x7faccc0c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc150230 0x7faccc0c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 -- 172.21.15.143:0/1574987628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0c8f30 con 0x7facac003b10 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7faccaa00640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7faccc0c9a10 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7facbc01c9c0 tx=0x7facac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0a5810 con 0x7facac003b10 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc0a5970 con 0x7facac003b10 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0c9210 con 0x7facac003b10 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc0a5ce0 con 0x7facac003b10 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc0c5940 con 0x7facac003b10 2024-01-15T18:18:46.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.207+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc067500 con 0x7facac003b10 2024-01-15T18:18:46.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.211+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7facbc0047f0 con 0x7facac003b10 2024-01-15T18:18:46.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.211+0000 7facc2a00640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca4067a60 0x7faca4069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.211+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7faccc0ce660 con 0x7facac003b10 2024-01-15T18:18:46.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.211+0000 7facca000640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca4067a60 0x7faca4069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.211+0000 7facca000640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca4067a60 0x7faca4069f20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faccc06e2e0 tx=0x7facb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.215+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7facbc0eb150 con 0x7facac003b10 2024-01-15T18:18:46.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.359+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faccc0b0740 con 0x7faca4067a60 2024-01-15T18:18:46.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.367+0000 7facc2a00640 1 -- 172.21.15.143:0/1574987628 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286013 (secure 0 0 0) 0x7facb40127c0 con 0x7faca4067a60 2024-01-15T18:18:46.366 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:46.368 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca4067a60 msgr2=0x7faca4069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca4067a60 0x7faca4069f20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7faccc06e2e0 tx=0x7facb4009040 comp rx=0 tx=0).stop 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 msgr2=0x7faccc0c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7faccc0c9a10 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7facbc01c9c0 tx=0x7facac005800 comp rx=0 tx=0).stop 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 shutdown_connections 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faca4067a60 0x7faca4069f20 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faccc150690 0x7faccc0ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faccc150230 0x7faccc0c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 --2- 172.21.15.143:0/1574987628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7facac003b10 0x7faccc0c9a10 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 >> 172.21.15.143:0/1574987628 conn(0x7faccc0d0e20 msgr2=0x7faccc0b1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:46.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 shutdown_connections 2024-01-15T18:18:46.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.371+0000 7facd1600640 1 -- 172.21.15.143:0/1574987628 wait complete. 2024-01-15T18:18:46.387 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-01-15T18:18:44.895731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":557816,"kb_used_data":11772,"kb_used_omap":12,"kb_used_meta":545907,"kb_avail":754416904,"statfs":{"total":773094113280,"available":772522909696,"internally_reserved":0,"allocated":12054528,"data_stored":5876835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":559009328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992969"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955237+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:18:41.955237+0000","last_peered":"2024-01-15T18:18:41.955237+0000","last_clean":"2024-01-15T18:18:41.955237+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:18:41.955237+0000","last_undegraded":"2024-01-15T18:18:41.955237+0000","last_fullsized":"2024-01-15T18:18:41.955237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908026+0000","last_change":"2024-01-15T18:18:40.908026+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908026+0000","last_undegraded":"2024-01-15T18:18:40.908026+0000","last_fullsized":"2024-01-15T18:18:40.908026+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913598+0000","last_change":"2024-01-15T18:18:40.913598+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913598+0000","last_undegraded":"2024-01-15T18:18:40.913598+0000","last_fullsized":"2024-01-15T18:18:40.913598+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.945216+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:18:41.945216+0000","last_peered":"2024-01-15T18:18:41.945216+0000","last_clean":"2024-01-15T18:18:41.945216+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:18:41.945216+0000","last_undegraded":"2024-01-15T18:18:41.945216+0000","last_fullsized":"2024-01-15T18:18:41.945216+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.925271+0000","last_change":"2024-01-15T18:18:40.925271+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.925271+0000","last_undegraded":"2024-01-15T18:18:40.925271+0000","last_fullsized":"2024-01-15T18:18:40.925271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954702+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:18:41.954702+0000","last_peered":"2024-01-15T18:18:41.954702+0000","last_clean":"2024-01-15T18:18:41.954702+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:18:41.954702+0000","last_undegraded":"2024-01-15T18:18:41.954702+0000","last_fullsized":"2024-01-15T18:18:41.954702+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:04:30.848650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.919588+0000","last_change":"2024-01-15T18:18:40.919588+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.919588+0000","last_undegraded":"2024-01-15T18:18:40.919588+0000","last_fullsized":"2024-01-15T18:18:40.919588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913325+0000","last_change":"2024-01-15T18:18:40.913325+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913325+0000","last_undegraded":"2024-01-15T18:18:40.913325+0000","last_fullsized":"2024-01-15T18:18:40.913325+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.956014+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:18:41.956014+0000","last_peered":"2024-01-15T18:18:41.956014+0000","last_clean":"2024-01-15T18:18:41.956014+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:18:41.956014+0000","last_undegraded":"2024-01-15T18:18:41.956014+0000","last_fullsized":"2024-01-15T18:18:41.956014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954228+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:18:41.954228+0000","last_peered":"2024-01-15T18:18:41.954228+0000","last_clean":"2024-01-15T18:18:41.954228+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:18:41.954228+0000","last_undegraded":"2024-01-15T18:18:41.954228+0000","last_fullsized":"2024-01-15T18:18:41.954228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907781+0000","last_change":"2024-01-15T18:18:40.907781+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907781+0000","last_undegraded":"2024-01-15T18:18:40.907781+0000","last_fullsized":"2024-01-15T18:18:40.907781+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951965+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:18:41.951965+0000","last_peered":"2024-01-15T18:18:41.951965+0000","last_clean":"2024-01-15T18:18:41.951965+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:18:41.951965+0000","last_undegraded":"2024-01-15T18:18:41.951965+0000","last_fullsized":"2024-01-15T18:18:41.951965+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:27.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913469+0000","last_change":"2024-01-15T18:18:40.913469+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913469+0000","last_undegraded":"2024-01-15T18:18:40.913469+0000","last_fullsized":"2024-01-15T18:18:40.913469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.914537+0000","last_change":"2024-01-15T18:18:40.914537+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.914537+0000","last_undegraded":"2024-01-15T18:18:40.914537+0000","last_fullsized":"2024-01-15T18:18:40.914537+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955421+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:18:41.955421+0000","last_peered":"2024-01-15T18:18:41.955421+0000","last_clean":"2024-01-15T18:18:41.955421+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:18:41.955421+0000","last_undegraded":"2024-01-15T18:18:41.955421+0000","last_fullsized":"2024-01-15T18:18:41.955421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:34.253432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947043+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:18:41.947043+0000","last_peered":"2024-01-15T18:18:41.947043+0000","last_clean":"2024-01-15T18:18:41.947043+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:18:41.947043+0000","last_undegraded":"2024-01-15T18:18:41.947043+0000","last_fullsized":"2024-01-15T18:18:41.947043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.910456+0000","last_change":"2024-01-15T18:18:40.910456+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.910456+0000","last_undegraded":"2024-01-15T18:18:40.910456+0000","last_fullsized":"2024-01-15T18:18:40.910456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909936+0000","last_change":"2024-01-15T18:18:40.909936+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909936+0000","last_undegraded":"2024-01-15T18:18:40.909936+0000","last_fullsized":"2024-01-15T18:18:40.909936+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913060+0000","last_change":"2024-01-15T18:18:40.913060+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913060+0000","last_undegraded":"2024-01-15T18:18:40.913060+0000","last_fullsized":"2024-01-15T18:18:40.913060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907829+0000","last_change":"2024-01-15T18:18:40.907829+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907829+0000","last_undegraded":"2024-01-15T18:18:40.907829+0000","last_fullsized":"2024-01-15T18:18:40.907829+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907707+0000","last_change":"2024-01-15T18:18:40.907707+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907707+0000","last_undegraded":"2024-01-15T18:18:40.907707+0000","last_fullsized":"2024-01-15T18:18:40.907707+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955395+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:18:41.955395+0000","last_peered":"2024-01-15T18:18:41.955395+0000","last_clean":"2024-01-15T18:18:41.955395+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:18:41.955395+0000","last_undegraded":"2024-01-15T18:18:41.955395+0000","last_fullsized":"2024-01-15T18:18:41.955395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.906954+0000","last_change":"2024-01-15T18:18:40.906954+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.906954+0000","last_undegraded":"2024-01-15T18:18:40.906954+0000","last_fullsized":"2024-01-15T18:18:40.906954+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955258+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:18:41.955258+0000","last_peered":"2024-01-15T18:18:41.955258+0000","last_clean":"2024-01-15T18:18:41.955258+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:18:41.955258+0000","last_undegraded":"2024-01-15T18:18:41.955258+0000","last_fullsized":"2024-01-15T18:18:41.955258+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908132+0000","last_change":"2024-01-15T18:18:40.908132+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908132+0000","last_undegraded":"2024-01-15T18:18:40.908132+0000","last_fullsized":"2024-01-15T18:18:40.908132+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955709+0000","last_change":"2024-01-15T18:18:41.949864+0000","last_active":"2024-01-15T18:18:41.955709+0000","last_peered":"2024-01-15T18:18:41.955709+0000","last_clean":"2024-01-15T18:18:41.955709+0000","last_became_active":"2024-01-15T18:18:41.944900+0000","last_became_peered":"2024-01-15T18:18:41.944900+0000","last_unstale":"2024-01-15T18:18:41.955709+0000","last_undegraded":"2024-01-15T18:18:41.955709+0000","last_fullsized":"2024-01-15T18:18:41.955709+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933525+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:18:41.933525+0000","last_peered":"2024-01-15T18:18:41.933525+0000","last_clean":"2024-01-15T18:18:41.933525+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:18:41.933525+0000","last_undegraded":"2024-01-15T18:18:41.933525+0000","last_fullsized":"2024-01-15T18:18:41.933525+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:28.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957639+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:18:41.957639+0000","last_peered":"2024-01-15T18:18:41.957639+0000","last_clean":"2024-01-15T18:18:41.957639+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:18:41.957639+0000","last_undegraded":"2024-01-15T18:18:41.957639+0000","last_fullsized":"2024-01-15T18:18:41.957639+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.910848+0000","last_change":"2024-01-15T18:18:40.910848+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.910848+0000","last_undegraded":"2024-01-15T18:18:40.910848+0000","last_fullsized":"2024-01-15T18:18:40.910848+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.917233+0000","last_change":"2024-01-15T18:18:40.917233+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.917233+0000","last_undegraded":"2024-01-15T18:18:40.917233+0000","last_fullsized":"2024-01-15T18:18:40.917233+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955724+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:18:41.955724+0000","last_peered":"2024-01-15T18:18:41.955724+0000","last_clean":"2024-01-15T18:18:41.955724+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:18:41.955724+0000","last_undegraded":"2024-01-15T18:18:41.955724+0000","last_fullsized":"2024-01-15T18:18:41.955724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955368+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:18:41.955368+0000","last_peered":"2024-01-15T18:18:41.955368+0000","last_clean":"2024-01-15T18:18:41.955368+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:18:41.955368+0000","last_undegraded":"2024-01-15T18:18:41.955368+0000","last_fullsized":"2024-01-15T18:18:41.955368+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954348+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:18:41.954348+0000","last_peered":"2024-01-15T18:18:41.954348+0000","last_clean":"2024-01-15T18:18:41.954348+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:18:41.954348+0000","last_undegraded":"2024-01-15T18:18:41.954348+0000","last_fullsized":"2024-01-15T18:18:41.954348+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:14.769436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.927486+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:18:40.927486+0000","last_peered":"2024-01-15T18:18:40.927486+0000","last_clean":"2024-01-15T18:18:40.927486+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:18:40.927486+0000","last_undegraded":"2024-01-15T18:18:40.927486+0000","last_fullsized":"2024-01-15T18:18:40.927486+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:17.166792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908694+0000","last_change":"2024-01-15T18:18:40.908694+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908694+0000","last_undegraded":"2024-01-15T18:18:40.908694+0000","last_fullsized":"2024-01-15T18:18:40.908694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925246+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:18:40.925246+0000","last_peered":"2024-01-15T18:18:40.925246+0000","last_clean":"2024-01-15T18:18:40.925246+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:18:40.925246+0000","last_undegraded":"2024-01-15T18:18:40.925246+0000","last_fullsized":"2024-01-15T18:18:40.925246+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:34.563635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954412+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:18:41.954412+0000","last_peered":"2024-01-15T18:18:41.954412+0000","last_clean":"2024-01-15T18:18:41.954412+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:18:41.954412+0000","last_undegraded":"2024-01-15T18:18:41.954412+0000","last_fullsized":"2024-01-15T18:18:41.954412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:49.284112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946203+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:18:40.946203+0000","last_peered":"2024-01-15T18:18:40.946203+0000","last_clean":"2024-01-15T18:18:40.946203+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:18:40.946203+0000","last_undegraded":"2024-01-15T18:18:40.946203+0000","last_fullsized":"2024-01-15T18:18:40.946203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:09.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912545+0000","last_change":"2024-01-15T18:18:40.912545+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912545+0000","last_undegraded":"2024-01-15T18:18:40.912545+0000","last_fullsized":"2024-01-15T18:18:40.912545+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946121+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:18:40.946121+0000","last_peered":"2024-01-15T18:18:40.946121+0000","last_clean":"2024-01-15T18:18:40.946121+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:18:40.946121+0000","last_undegraded":"2024-01-15T18:18:40.946121+0000","last_fullsized":"2024-01-15T18:18:40.946121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:10.172303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955156+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:18:41.955156+0000","last_peered":"2024-01-15T18:18:41.955156+0000","last_clean":"2024-01-15T18:18:41.955156+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:18:41.955156+0000","last_undegraded":"2024-01-15T18:18:41.955156+0000","last_fullsized":"2024-01-15T18:18:41.955156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.932352+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:18:40.932352+0000","last_peered":"2024-01-15T18:18:40.932352+0000","last_clean":"2024-01-15T18:18:40.932352+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:18:40.932352+0000","last_undegraded":"2024-01-15T18:18:40.932352+0000","last_fullsized":"2024-01-15T18:18:40.932352+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:23.333161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909660+0000","last_change":"2024-01-15T18:18:40.909660+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909660+0000","last_undegraded":"2024-01-15T18:18:40.909660+0000","last_fullsized":"2024-01-15T18:18:40.909660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949383+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:18:41.949383+0000","last_peered":"2024-01-15T18:18:41.949383+0000","last_clean":"2024-01-15T18:18:41.949383+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:18:41.949383+0000","last_undegraded":"2024-01-15T18:18:41.949383+0000","last_fullsized":"2024-01-15T18:18:41.949383+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954520+0000","last_change":"2024-01-15T18:18:41.940718+0000","last_active":"2024-01-15T18:18:41.954520+0000","last_peered":"2024-01-15T18:18:41.954520+0000","last_clean":"2024-01-15T18:18:41.954520+0000","last_became_active":"2024-01-15T18:18:41.940432+0000","last_became_peered":"2024-01-15T18:18:41.940432+0000","last_unstale":"2024-01-15T18:18:41.954520+0000","last_undegraded":"2024-01-15T18:18:41.954520+0000","last_fullsized":"2024-01-15T18:18:41.954520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925490+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:18:40.925490+0000","last_peered":"2024-01-15T18:18:40.925490+0000","last_clean":"2024-01-15T18:18:40.925490+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:18:40.925490+0000","last_undegraded":"2024-01-15T18:18:40.925490+0000","last_fullsized":"2024-01-15T18:18:40.925490+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:41.088175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.918761+0000","last_change":"2024-01-15T18:18:40.918761+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.918761+0000","last_undegraded":"2024-01-15T18:18:40.918761+0000","last_fullsized":"2024-01-15T18:18:40.918761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918022+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:18:41.918022+0000","last_peered":"2024-01-15T18:18:41.918022+0000","last_clean":"2024-01-15T18:18:41.918022+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:18:41.918022+0000","last_undegraded":"2024-01-15T18:18:41.918022+0000","last_fullsized":"2024-01-15T18:18:41.918022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.916907+0000","last_change":"2024-01-15T18:18:40.916907+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.916907+0000","last_undegraded":"2024-01-15T18:18:40.916907+0000","last_fullsized":"2024-01-15T18:18:40.916907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946140+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:18:40.946140+0000","last_peered":"2024-01-15T18:18:40.946140+0000","last_clean":"2024-01-15T18:18:40.946140+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:18:40.946140+0000","last_undegraded":"2024-01-15T18:18:40.946140+0000","last_fullsized":"2024-01-15T18:18:40.946140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907383+0000","last_change":"2024-01-15T18:18:40.907383+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907383+0000","last_undegraded":"2024-01-15T18:18:40.907383+0000","last_fullsized":"2024-01-15T18:18:40.907383+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.926304+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:18:41.926304+0000","last_peered":"2024-01-15T18:18:41.926304+0000","last_clean":"2024-01-15T18:18:41.926304+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:18:41.926304+0000","last_undegraded":"2024-01-15T18:18:41.926304+0000","last_fullsized":"2024-01-15T18:18:41.926304+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955327+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:18:41.955327+0000","last_peered":"2024-01-15T18:18:41.955327+0000","last_clean":"2024-01-15T18:18:41.955327+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:18:41.955327+0000","last_undegraded":"2024-01-15T18:18:41.955327+0000","last_fullsized":"2024-01-15T18:18:41.955327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957957+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:18:41.957957+0000","last_peered":"2024-01-15T18:18:41.957957+0000","last_clean":"2024-01-15T18:18:41.957957+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:18:41.957957+0000","last_undegraded":"2024-01-15T18:18:41.957957+0000","last_fullsized":"2024-01-15T18:18:41.957957+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.916144+0000","last_change":"2024-01-15T18:18:40.916144+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.916144+0000","last_undegraded":"2024-01-15T18:18:40.916144+0000","last_fullsized":"2024-01-15T18:18:40.916144+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.960177+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:18:41.960177+0000","last_peered":"2024-01-15T18:18:41.960177+0000","last_clean":"2024-01-15T18:18:41.960177+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:18:41.960177+0000","last_undegraded":"2024-01-15T18:18:41.960177+0000","last_fullsized":"2024-01-15T18:18:41.960177+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955377+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:18:41.955377+0000","last_peered":"2024-01-15T18:18:41.955377+0000","last_clean":"2024-01-15T18:18:41.955377+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:18:41.955377+0000","last_undegraded":"2024-01-15T18:18:41.955377+0000","last_fullsized":"2024-01-15T18:18:41.955377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949761+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:18:41.949761+0000","last_peered":"2024-01-15T18:18:41.949761+0000","last_clean":"2024-01-15T18:18:41.949761+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:18:41.949761+0000","last_undegraded":"2024-01-15T18:18:41.949761+0000","last_fullsized":"2024-01-15T18:18:41.949761+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955653+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:18:41.955653+0000","last_peered":"2024-01-15T18:18:41.955653+0000","last_clean":"2024-01-15T18:18:41.955653+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:18:41.955653+0000","last_undegraded":"2024-01-15T18:18:41.955653+0000","last_fullsized":"2024-01-15T18:18:41.955653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.702381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.923005+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:18:40.923005+0000","last_peered":"2024-01-15T18:18:40.923005+0000","last_clean":"2024-01-15T18:18:40.923005+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:18:40.923005+0000","last_undegraded":"2024-01-15T18:18:40.923005+0000","last_fullsized":"2024-01-15T18:18:40.923005+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:16.430602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955766+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:18:41.955766+0000","last_peered":"2024-01-15T18:18:41.955766+0000","last_clean":"2024-01-15T18:18:41.955766+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:18:41.955766+0000","last_undegraded":"2024-01-15T18:18:41.955766+0000","last_fullsized":"2024-01-15T18:18:41.955766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:17.438389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.939931+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:18:40.939931+0000","last_peered":"2024-01-15T18:18:40.939931+0000","last_clean":"2024-01-15T18:18:40.939931+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:18:40.939931+0000","last_undegraded":"2024-01-15T18:18:40.939931+0000","last_fullsized":"2024-01-15T18:18:40.939931+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:30.432388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.921779+0000","last_change":"2024-01-15T18:18:40.921779+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.921779+0000","last_undegraded":"2024-01-15T18:18:40.921779+0000","last_fullsized":"2024-01-15T18:18:40.921779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.946443+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:18:41.946443+0000","last_peered":"2024-01-15T18:18:41.946443+0000","last_clean":"2024-01-15T18:18:41.946443+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:18:41.946443+0000","last_undegraded":"2024-01-15T18:18:41.946443+0000","last_fullsized":"2024-01-15T18:18:41.946443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933399+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:18:41.933399+0000","last_peered":"2024-01-15T18:18:41.933399+0000","last_clean":"2024-01-15T18:18:41.933399+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:18:41.933399+0000","last_undegraded":"2024-01-15T18:18:41.933399+0000","last_fullsized":"2024-01-15T18:18:41.933399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.408977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931086+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:18:40.931086+0000","last_peered":"2024-01-15T18:18:40.931086+0000","last_clean":"2024-01-15T18:18:40.931086+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:18:40.931086+0000","last_undegraded":"2024-01-15T18:18:40.931086+0000","last_fullsized":"2024-01-15T18:18:40.931086+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:51:44.688957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.911900+0000","last_change":"2024-01-15T18:18:40.911900+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.911900+0000","last_undegraded":"2024-01-15T18:18:40.911900+0000","last_fullsized":"2024-01-15T18:18:40.911900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.923118+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:18:40.923118+0000","last_peered":"2024-01-15T18:18:40.923118+0000","last_clean":"2024-01-15T18:18:40.923118+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:18:40.923118+0000","last_undegraded":"2024-01-15T18:18:40.923118+0000","last_fullsized":"2024-01-15T18:18:40.923118+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:15.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.911924+0000","last_change":"2024-01-15T18:18:40.911924+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.911924+0000","last_undegraded":"2024-01-15T18:18:40.911924+0000","last_fullsized":"2024-01-15T18:18:40.911924+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931943+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:18:40.931943+0000","last_peered":"2024-01-15T18:18:40.931943+0000","last_clean":"2024-01-15T18:18:40.931943+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:18:40.931943+0000","last_undegraded":"2024-01-15T18:18:40.931943+0000","last_fullsized":"2024-01-15T18:18:40.931943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:46.912158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.914035+0000","last_change":"2024-01-15T18:18:40.914035+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.914035+0000","last_undegraded":"2024-01-15T18:18:40.914035+0000","last_fullsized":"2024-01-15T18:18:40.914035+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958826+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:18:41.958826+0000","last_peered":"2024-01-15T18:18:41.958826+0000","last_clean":"2024-01-15T18:18:41.958826+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:18:41.958826+0000","last_undegraded":"2024-01-15T18:18:41.958826+0000","last_fullsized":"2024-01-15T18:18:41.958826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.940187+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:18:41.940187+0000","last_peered":"2024-01-15T18:18:41.940187+0000","last_clean":"2024-01-15T18:18:41.940187+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:18:41.940187+0000","last_undegraded":"2024-01-15T18:18:41.940187+0000","last_fullsized":"2024-01-15T18:18:41.940187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.947226+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:18:40.947226+0000","last_peered":"2024-01-15T18:18:40.947226+0000","last_clean":"2024-01-15T18:18:40.947226+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:18:40.947226+0000","last_undegraded":"2024-01-15T18:18:40.947226+0000","last_fullsized":"2024-01-15T18:18:40.947226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:53.075302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912833+0000","last_change":"2024-01-15T18:18:40.912833+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912833+0000","last_undegraded":"2024-01-15T18:18:40.912833+0000","last_fullsized":"2024-01-15T18:18:40.912833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.924617+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:18:41.924617+0000","last_peered":"2024-01-15T18:18:41.924617+0000","last_clean":"2024-01-15T18:18:41.924617+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:18:41.924617+0000","last_undegraded":"2024-01-15T18:18:41.924617+0000","last_fullsized":"2024-01-15T18:18:41.924617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947927+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:18:41.947927+0000","last_peered":"2024-01-15T18:18:41.947927+0000","last_clean":"2024-01-15T18:18:41.947927+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:18:41.947927+0000","last_undegraded":"2024-01-15T18:18:41.947927+0000","last_fullsized":"2024-01-15T18:18:41.947927+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:43.903199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918200+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:18:41.918200+0000","last_peered":"2024-01-15T18:18:41.918200+0000","last_clean":"2024-01-15T18:18:41.918200+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:18:41.918200+0000","last_undegraded":"2024-01-15T18:18:41.918200+0000","last_fullsized":"2024-01-15T18:18:41.918200+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908189+0000","last_change":"2024-01-15T18:18:40.908189+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908189+0000","last_undegraded":"2024-01-15T18:18:40.908189+0000","last_fullsized":"2024-01-15T18:18:40.908189+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957662+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:18:41.957662+0000","last_peered":"2024-01-15T18:18:41.957662+0000","last_clean":"2024-01-15T18:18:41.957662+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:18:41.957662+0000","last_undegraded":"2024-01-15T18:18:41.957662+0000","last_fullsized":"2024-01-15T18:18:41.957662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.944610+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:18:41.944610+0000","last_peered":"2024-01-15T18:18:41.944610+0000","last_clean":"2024-01-15T18:18:41.944610+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:18:41.944610+0000","last_undegraded":"2024-01-15T18:18:41.944610+0000","last_fullsized":"2024-01-15T18:18:41.944610+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925197+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:18:40.925197+0000","last_peered":"2024-01-15T18:18:40.925197+0000","last_clean":"2024-01-15T18:18:40.925197+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:18:40.925197+0000","last_undegraded":"2024-01-15T18:18:40.925197+0000","last_fullsized":"2024-01-15T18:18:40.925197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:41.088175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951944+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:18:41.951944+0000","last_peered":"2024-01-15T18:18:41.951944+0000","last_clean":"2024-01-15T18:18:41.951944+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:18:41.951944+0000","last_undegraded":"2024-01-15T18:18:41.951944+0000","last_fullsized":"2024-01-15T18:18:41.951944+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918797+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:18:41.918797+0000","last_peered":"2024-01-15T18:18:41.918797+0000","last_clean":"2024-01-15T18:18:41.918797+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:18:41.918797+0000","last_undegraded":"2024-01-15T18:18:41.918797+0000","last_fullsized":"2024-01-15T18:18:41.918797+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907378+0000","last_change":"2024-01-15T18:18:40.907378+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907378+0000","last_undegraded":"2024-01-15T18:18:40.907378+0000","last_fullsized":"2024-01-15T18:18:40.907378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.927483+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:18:41.927483+0000","last_peered":"2024-01-15T18:18:41.927483+0000","last_clean":"2024-01-15T18:18:41.927483+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:18:41.927483+0000","last_undegraded":"2024-01-15T18:18:41.927483+0000","last_fullsized":"2024-01-15T18:18:41.927483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908276+0000","last_change":"2024-01-15T18:18:40.908276+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908276+0000","last_undegraded":"2024-01-15T18:18:40.908276+0000","last_fullsized":"2024-01-15T18:18:40.908276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.927500+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:18:40.927500+0000","last_peered":"2024-01-15T18:18:40.927500+0000","last_clean":"2024-01-15T18:18:40.927500+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:18:40.927500+0000","last_undegraded":"2024-01-15T18:18:40.927500+0000","last_fullsized":"2024-01-15T18:18:40.927500+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:33.731825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909185+0000","last_change":"2024-01-15T18:18:40.909185+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909185+0000","last_undegraded":"2024-01-15T18:18:40.909185+0000","last_fullsized":"2024-01-15T18:18:40.909185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.936592+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:18:40.936592+0000","last_peered":"2024-01-15T18:18:40.936592+0000","last_clean":"2024-01-15T18:18:40.936592+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:18:40.936592+0000","last_undegraded":"2024-01-15T18:18:40.936592+0000","last_fullsized":"2024-01-15T18:18:40.936592+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:15.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912287+0000","last_change":"2024-01-15T18:18:40.912287+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912287+0000","last_undegraded":"2024-01-15T18:18:40.912287+0000","last_fullsized":"2024-01-15T18:18:40.912287+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946157+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:18:40.946157+0000","last_peered":"2024-01-15T18:18:40.946157+0000","last_clean":"2024-01-15T18:18:40.946157+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:18:40.946157+0000","last_undegraded":"2024-01-15T18:18:40.946157+0000","last_fullsized":"2024-01-15T18:18:40.946157+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:50.249980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955068+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:18:41.955068+0000","last_peered":"2024-01-15T18:18:41.955068+0000","last_clean":"2024-01-15T18:18:41.955068+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:18:41.955068+0000","last_undegraded":"2024-01-15T18:18:41.955068+0000","last_fullsized":"2024-01-15T18:18:41.955068+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918226+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:18:41.918226+0000","last_peered":"2024-01-15T18:18:41.918226+0000","last_clean":"2024-01-15T18:18:41.918226+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:18:41.918226+0000","last_undegraded":"2024-01-15T18:18:41.918226+0000","last_fullsized":"2024-01-15T18:18:41.918226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.953440+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:18:41.953440+0000","last_peered":"2024-01-15T18:18:41.953440+0000","last_clean":"2024-01-15T18:18:41.953440+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:18:41.953440+0000","last_undegraded":"2024-01-15T18:18:41.953440+0000","last_fullsized":"2024-01-15T18:18:41.953440+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:52.321006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.930839+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:18:40.930839+0000","last_peered":"2024-01-15T18:18:40.930839+0000","last_clean":"2024-01-15T18:18:40.930839+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:18:40.930839+0000","last_undegraded":"2024-01-15T18:18:40.930839+0000","last_fullsized":"2024-01-15T18:18:40.930839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:47.752921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":239,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-01-15T18:18:39.904797+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:18:39.904797+0000","last_peered":"2024-01-15T18:18:39.904797+0000","last_clean":"2024-01-15T18:18:39.904797+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:18:39.904797+0000","last_undegraded":"2024-01-15T18:18:39.904797+0000","last_fullsized":"2024-01-15T18:18:39.904797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575019,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64960,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306880,"statfs":{"total":96636764160,"available":96570245120,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.86399999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.622},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.77400000000000002,"15min":0.85999999999999999},"min":{"1min":0.48299999999999998,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.92500000000000004,"5min":4.6100000000000003,"15min":13.744},"last":0.60799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.96799999999999997,"15min":1.0640000000000001},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":3.016,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.81100000000000005,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":4.7679999999999998,"15min":31.774999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.74099999999999999,"15min":0.98699999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":2.1680000000000001,"15min":30.846},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":1.073},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":17.039999999999999,"15min":39.695},"last":0.65300000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.82599999999999996,"15min":1.1579999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.88400000000000001,"5min":8.2650000000000006,"15min":26.463999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.94699999999999995,"15min":1.202},"min":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":16.879999999999999,"15min":42.692},"last":0.68200000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78800000000000003,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":1.147,"5min":14.087,"15min":14.087},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.629,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.63500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.81299999999999994,"15min":0.71199999999999997},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":1.0209999999999999,"5min":11.334,"15min":11.334},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":4.8179999999999996,"15min":8.423},"last":0.69299999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5369999999999999,"5min":4.7140000000000004,"15min":8.5030000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1970000000000001,"15min":3.3450000000000002},"last":0.45800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.81699999999999995,"15min":0.95299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.754},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74199999999999999,"15min":0.873},"min":{"1min":0.375,"5min":0.375,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":3.4940000000000002,"15min":12.449},"last":0.82599999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.85599999999999998,"15min":0.89700000000000002},"min":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.33900000000000002},"max":{"1min":1.0489999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.84199999999999997,"15min":1.004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.81899999999999995,"5min":8.9320000000000004,"15min":19.475000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.069,"15min":1.179},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":14.743,"15min":42.825000000000003},"last":0.997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.89000000000000001,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":9.4000000000000004,"15min":42.851999999999997},"last":1.0720000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78300000000000003,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":0.82099999999999995,"5min":3.016,"15min":24.199000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0629999999999999,"15min":1.087},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":2.956,"5min":18.791,"15min":18.791},"last":0.872}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.89300000000000002,"15min":0.745},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":19.114999999999998,"15min":19.114999999999998},"last":1.044},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33700000000000002},"max":{"1min":0.875,"5min":1.7450000000000001,"15min":12.491},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81899999999999995,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.89800000000000002},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.5259999999999998,"15min":12.945},"last":1.1120000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":1.006,"15min":0.83899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":11.907,"15min":11.907},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82499999999999996,"15min":0.88400000000000001},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":9.9440000000000008,"15min":27.651},"last":1.1910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76860,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294980,"statfs":{"total":96636764160,"available":96558059520,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":4.4770000000000003,"15min":11.519},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.874,"15min":0.92400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.91700000000000004,"5min":7.6260000000000003,"15min":12.952},"last":0.68899999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.92700000000000005,"15min":0.90300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.95499999999999996,"5min":4.3140000000000001,"15min":13.358000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.94699999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0169999999999999,"5min":4.1580000000000004,"15min":21.856999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":1,"5min":6.7160000000000002,"15min":18.314},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.145,"15min":1.0109999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.0580000000000001,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.752}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.85499999999999998,"15min":0.94499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":3.2400000000000002,"15min":13.317},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":1.01},"min":{"1min":0.437,"5min":0.437,"15min":0.42899999999999999},"max":{"1min":0.998,"5min":3.931,"15min":19.638000000000002},"last":0.72799999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":0.96099999999999997,"5min":5.899,"15min":11.093},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":0.88700000000000001,"5min":2.516,"15min":3.3889999999999998},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84399999999999997,"15min":0.79500000000000004},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97899999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.308},"max":{"1min":1.0840000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.85899999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":1.071,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":1.034,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81020,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290820,"statfs":{"total":96636764160,"available":96553799680,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.79800000000000004,"15min":0.93300000000000005},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":9.4179999999999993,"15min":34.802},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":14.488,"15min":22.335999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.95699999999999996},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":2.2130000000000001,"15min":38.198},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.80500000000000005,"15min":0.95599999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.86599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95099999999999996,"5min":1.6259999999999999,"15min":7.6989999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.78500000000000003,"15min":0.84799999999999998},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":4.4089999999999998,"15min":7.4829999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.91300000000000003,"15min":0.95399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":14.529,"15min":14.529},"last":0.628},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72299999999999998,"15min":0.92900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":1.4279999999999999,"15min":25.774000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":0.68200000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.97799999999999998,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.47499999999999998,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":25.887,"15min":25.887},"last":0.36299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":10.284000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.46200000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.82799999999999996,"15min":0.78300000000000003},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.77100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"max":{"1min":1.03,"5min":1.232,"15min":11.67},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.625,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.255},"max":{"1min":0.81899999999999995,"5min":7.2839999999999998,"15min":26.364000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.66300000000000003,"15min":0.95599999999999996},"min":{"1min":0.312,"5min":0.25,"15min":0.246},"max":{"1min":0.71499999999999997,"5min":7.8520000000000003,"15min":28.332000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.60799999999999998,"15min":0.98799999999999999},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.84799999999999998,"5min":3.8100000000000001,"15min":40.457999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.76300000000000001,"15min":1.1419999999999999},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":9.282,"15min":27.427},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.71199999999999997,"15min":0.98899999999999999},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.90400000000000003,"5min":9.1180000000000003,"15min":23.036999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.81499999999999995,"15min":1.089},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":1.048,"5min":9.3510000000000009,"15min":26.404},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":1.024},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.41199999999999998},"max":{"1min":1.0780000000000001,"5min":4.1040000000000001,"15min":22.998000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84599999999999997,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.0229999999999999,"5min":3.8540000000000001,"15min":20.399000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":1.097},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":6.5140000000000002,"15min":26.314},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85599999999999998,"15min":1.1220000000000001},"min":{"1min":0.46100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":4.165,"15min":22.908999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":1.048,"15min":1.153},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.42799999999999999},"max":{"1min":1.0609999999999999,"5min":6.843,"15min":20.445},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.1850000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.123,"5min":6.7640000000000002,"15min":26.091999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.98999999999999999,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":7.0250000000000004,"15min":23.111999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.92000000000000004,"15min":1.321},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46000000000000002},"max":{"1min":1.1419999999999999,"5min":4.4470000000000001,"15min":44.817999999999998},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64380,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307460,"statfs":{"total":96636764160,"available":96570839040,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54400000000000004,"15min":1.0529999999999999},"min":{"1min":0.31,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.373},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.86199999999999999,"15min":1.321},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.66400000000000003,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.39200000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.64700000000000002,"15min":1.5409999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.56299999999999994,"5min":7.476,"15min":113.149},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.96899999999999997,"15min":1.1080000000000001},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":20.619,"15min":41.302},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.341,"15min":1.379},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28899999999999998},"max":{"1min":0.67400000000000004,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66000000000000003,"15min":1.27},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":0.77300000000000002,"5min":7.3609999999999998,"15min":115.693},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":1.206},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":7.4169999999999998,"15min":24.366},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0740000000000001,"15min":1.23},"min":{"1min":0.51600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.5800000000000001,"15min":1.5089999999999999},"min":{"1min":0.54700000000000004,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84999999999999998,"15min":1.304},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":7.5099999999999998,"15min":26.084},"last":0.54900000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":1.0980000000000001,"15min":1.3149999999999999},"min":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":20.120999999999999,"15min":22.199000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.748,"5min":1.6240000000000001,"15min":1.5700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":1.165,"15min":1.262},"min":{"1min":0.55200000000000005,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":0.92600000000000005,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":1.613,"15min":1.6299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64404,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307436,"statfs":{"total":96636764160,"available":96570814464,"internally_reserved":0,"allocated":1376256,"data_stored":593098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.64200000000000002,"15min":0.91500000000000004},"min":{"1min":0.245,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.68200000000000005,"5min":9.452,"15min":39.948},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.68799999999999994,"15min":1.103},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":8.141,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.71799999999999997,"15min":0.98199999999999998},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":0.63800000000000001,"5min":15.137,"15min":49.229999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61799999999999999,"5min":9.8469999999999995,"15min":10.349},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61099999999999999,"15min":1.173},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":4.4669999999999996,"15min":27.643999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.90000000000000002,"15min":0.997},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.66800000000000004,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84299999999999997,"15min":1.222},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":4.633,"15min":27.489999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67600000000000005,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98599999999999999,"5min":9.6989999999999998,"15min":12.997},"last":0.83599999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.86599999999999999,"15min":1.212},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":4.673,"15min":27.547000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.0289999999999999,"15min":1.1200000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.496},"max":{"1min":1.028,"5min":9.7650000000000006,"15min":13.054},"last":0.78100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.86499999999999999,"15min":1.204},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":4.7060000000000004,"15min":27.579999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.87,"15min":1.0369999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.1319999999999999},"min":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":6.6950000000000003,"5min":9.5660000000000007,"15min":11.833},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1.0229999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.502},"max":{"1min":0.93200000000000005,"5min":9.8079999999999998,"15min":14.606},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.78600000000000003,"15min":1.1379999999999999},"min":{"1min":0.27100000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.94299999999999995,"5min":11.356,"15min":31.977},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.81299999999999994,"15min":1.0229999999999999},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0289999999999999,"5min":11.629,"15min":36.145000000000003},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.76600000000000001,"15min":1.1319999999999999},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":0.91800000000000004,"5min":5.8929999999999998,"15min":41.387999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.877,"15min":1.038},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.25},"max":{"1min":0.97499999999999998,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.97599999999999998,"15min":1.0860000000000001},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.093,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.79400000000000004,"15min":1.097},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":6.3739999999999997,"15min":45.128},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.0289999999999999,"15min":1.0740000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.372},"max":{"1min":2.1469999999999998,"5min":11.772,"15min":32.932000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.0840000000000001,"15min":1.1919999999999999},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":1.4370000000000001,"5min":11.220000000000001,"15min":31.782},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1.083,"15min":1.1850000000000001},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3979999999999999,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.841,"5min":11.958,"15min":33.151000000000003},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":1.0640000000000001},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.379,"5min":4.0970000000000004,"15min":13.055},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.749,"5min":1.0449999999999999,"15min":1.0920000000000001},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.522,"5min":11.833,"15min":33.098999999999997},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.92300000000000004,"15min":1.0740000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3160000000000001,"5min":4.1349999999999998,"15min":13.090999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.1120000000000001,"15min":1.242},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.464,"5min":11.321999999999999,"15min":31.905999999999999},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:46.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.567+0000 7fcba2000640 1 Processor -- start 2024-01-15T18:18:46.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.567+0000 7fcba2000640 1 -- start start 2024-01-15T18:18:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcba2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c0cfb30 0x7fcb9c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcba2000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcb9c0cbf50 0x7fcb9c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcba2000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9c0cffd0 con 0x7fcb9c1607c0 2024-01-15T18:18:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcba2000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c0d1ce0 con 0x7fcb9c0cbf50 2024-01-15T18:18:46.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcba2000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0d02b0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c0cfb30 0x7fcb9c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcb9c0cbf50 0x7fcb9c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9b400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fcb9c1607c0 0x7fcb9c06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:54074/0 (socket says 172.21.15.143:54074) 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9b400640 1 -- 172.21.15.143:0/2429311847 learned_addr learned my addr 172.21.15.143:0/2429311847 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9a000640 1 -- 172.21.15.143:0/2429311847 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525975333 0 0) 0x7fcb9c0cffd0 con 0x7fcb9c1607c0 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9be00640 1 -- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c0cfb30 msgr2=0x7fcb9c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:46.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9be00640 1 --2- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c0cfb30 0x7fcb9c06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9be00640 1 -- 172.21.15.143:0/2429311847 >> v1:172.21.15.204:6789/0 conn(0x7fcb9c1607c0 legacy=0x7fcb9c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9be00640 1 -- 172.21.15.143:0/2429311847 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c0cb470 con 0x7fcb9c0cbf50 2024-01-15T18:18:46.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.571+0000 7fcb9aa00640 1 --2- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c0cfb30 0x7fcb9c06bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:18:46.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcb9be00640 1 --2- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c0cc390 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009d90 tx=0x7fcb9401f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f5176ff9da814e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcb9a000640 1 -- 172.21.15.143:0/2429311847 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940af600 con 0x7fcb9c0cbf50 2024-01-15T18:18:46.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcb9a000640 1 -- 172.21.15.143:0/2429311847 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb940ab050 con 0x7fcb9c0cbf50 2024-01-15T18:18:46.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcb9a000640 1 -- 172.21.15.143:0/2429311847 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940af980 con 0x7fcb9c0cbf50 2024-01-15T18:18:46.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 -- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 msgr2=0x7fcb9c0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 --2- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c0cc390 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009d90 tx=0x7fcb9401f130 comp rx=0 tx=0).stop 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 -- 172.21.15.143:0/2429311847 shutdown_connections 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 --2- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c0cc390 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 --2- 172.21.15.143:0/2429311847 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c0cfb30 0x7fcb9c06bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 -- 172.21.15.143:0/2429311847 >> 172.21.15.143:0/2429311847 conn(0x7fcb9c0d0e20 msgr2=0x7fcb9c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 -- 172.21.15.143:0/2429311847 shutdown_connections 2024-01-15T18:18:46.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.575+0000 7fcba2000640 1 -- 172.21.15.143:0/2429311847 wait complete. 2024-01-15T18:18:46.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 Processor -- start 2024-01-15T18:18:46.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- start start 2024-01-15T18:18:46.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 0x7fcb9c1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c1607c0 0x7fcb9c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1f6fb0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c06e550 con 0x7fcb9c0cbf50 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c146d30 con 0x7fcb9c1607c0 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 0x7fcb9c1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 0x7fcb9c1f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48024/0 (socket says 172.21.15.143:48024) 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 -- 172.21.15.143:0/1187532094 learned_addr learned my addr 172.21.15.143:0/1187532094 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:46.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9b400640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9be00640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c1607c0 0x7fcb9c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 -- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c1607c0 msgr2=0x7fcb9c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c1607c0 0x7fcb9c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 -- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 msgr2=0x7fcb9c146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 -- 172.21.15.143:0/1187532094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84009420 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb9aa00640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 0x7fcb9c1f6a70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ebe0 tx=0x7fcb8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400eee0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8400cc30 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84013470 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb94009a70 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c1472f0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.579+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c06d4b0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.583+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fcb840135d0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.583+0000 7fcb93400640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb74067a60 0x7fcb74069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.583+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcb84083660 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.583+0000 7fcb9b400640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb74067a60 0x7fcb74069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.583+0000 7fcb9b400640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb74067a60 0x7fcb74069f20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c000f70 tx=0x7fcb8c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.587+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcb84051fa0 con 0x7fcb9c0cfb30 2024-01-15T18:18:46.727 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.731+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb9c0ce6b0 con 0x7fcb74067a60 2024-01-15T18:18:46.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.739+0000 7fcb93400640 1 -- 172.21.15.143:0/1187532094 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286013 (secure 0 0 0) 0x7fcb8c007da0 con 0x7fcb74067a60 2024-01-15T18:18:46.737 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:46.739 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:18:46.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb74067a60 msgr2=0x7fcb74069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb74067a60 0x7fcb74069f20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c000f70 tx=0x7fcb8c007c10 comp rx=0 tx=0).stop 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 msgr2=0x7fcb9c1f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 0x7fcb9c1f6a70 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ebe0 tx=0x7fcb8400c520 comp rx=0 tx=0).stop 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 shutdown_connections 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb74067a60 0x7fcb74069f20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb9c1607c0 0x7fcb9c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb9c0cfb30 0x7fcb9c1f6a70 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 --2- 172.21.15.143:0/1187532094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb9c0cbf50 0x7fcb9c146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 >> 172.21.15.143:0/1187532094 conn(0x7fcb9c0d0e20 msgr2=0x7fcb9c15d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 shutdown_connections 2024-01-15T18:18:46.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.743+0000 7fcba2000640 1 -- 172.21.15.143:0/1187532094 wait complete. 2024-01-15T18:18:46.758 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-01-15T18:18:44.895731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592126,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":177,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":557816,"kb_used_data":11772,"kb_used_omap":12,"kb_used_meta":545907,"kb_avail":754416904,"statfs":{"total":773094113280,"available":772522909696,"internally_reserved":0,"allocated":12054528,"data_stored":5876835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":559009328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1758,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992969"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955237+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:18:41.955237+0000","last_peered":"2024-01-15T18:18:41.955237+0000","last_clean":"2024-01-15T18:18:41.955237+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:18:41.955237+0000","last_undegraded":"2024-01-15T18:18:41.955237+0000","last_fullsized":"2024-01-15T18:18:41.955237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908026+0000","last_change":"2024-01-15T18:18:40.908026+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908026+0000","last_undegraded":"2024-01-15T18:18:40.908026+0000","last_fullsized":"2024-01-15T18:18:40.908026+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913598+0000","last_change":"2024-01-15T18:18:40.913598+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913598+0000","last_undegraded":"2024-01-15T18:18:40.913598+0000","last_fullsized":"2024-01-15T18:18:40.913598+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.945216+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:18:41.945216+0000","last_peered":"2024-01-15T18:18:41.945216+0000","last_clean":"2024-01-15T18:18:41.945216+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:18:41.945216+0000","last_undegraded":"2024-01-15T18:18:41.945216+0000","last_fullsized":"2024-01-15T18:18:41.945216+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.925271+0000","last_change":"2024-01-15T18:18:40.925271+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.925271+0000","last_undegraded":"2024-01-15T18:18:40.925271+0000","last_fullsized":"2024-01-15T18:18:40.925271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954702+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:18:41.954702+0000","last_peered":"2024-01-15T18:18:41.954702+0000","last_clean":"2024-01-15T18:18:41.954702+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:18:41.954702+0000","last_undegraded":"2024-01-15T18:18:41.954702+0000","last_fullsized":"2024-01-15T18:18:41.954702+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:04:30.848650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.919588+0000","last_change":"2024-01-15T18:18:40.919588+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.919588+0000","last_undegraded":"2024-01-15T18:18:40.919588+0000","last_fullsized":"2024-01-15T18:18:40.919588+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913325+0000","last_change":"2024-01-15T18:18:40.913325+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913325+0000","last_undegraded":"2024-01-15T18:18:40.913325+0000","last_fullsized":"2024-01-15T18:18:40.913325+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.956014+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:18:41.956014+0000","last_peered":"2024-01-15T18:18:41.956014+0000","last_clean":"2024-01-15T18:18:41.956014+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:18:41.956014+0000","last_undegraded":"2024-01-15T18:18:41.956014+0000","last_fullsized":"2024-01-15T18:18:41.956014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954228+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:18:41.954228+0000","last_peered":"2024-01-15T18:18:41.954228+0000","last_clean":"2024-01-15T18:18:41.954228+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:18:41.954228+0000","last_undegraded":"2024-01-15T18:18:41.954228+0000","last_fullsized":"2024-01-15T18:18:41.954228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907781+0000","last_change":"2024-01-15T18:18:40.907781+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907781+0000","last_undegraded":"2024-01-15T18:18:40.907781+0000","last_fullsized":"2024-01-15T18:18:40.907781+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951965+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:18:41.951965+0000","last_peered":"2024-01-15T18:18:41.951965+0000","last_clean":"2024-01-15T18:18:41.951965+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:18:41.951965+0000","last_undegraded":"2024-01-15T18:18:41.951965+0000","last_fullsized":"2024-01-15T18:18:41.951965+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:27.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913469+0000","last_change":"2024-01-15T18:18:40.913469+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913469+0000","last_undegraded":"2024-01-15T18:18:40.913469+0000","last_fullsized":"2024-01-15T18:18:40.913469+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.914537+0000","last_change":"2024-01-15T18:18:40.914537+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.914537+0000","last_undegraded":"2024-01-15T18:18:40.914537+0000","last_fullsized":"2024-01-15T18:18:40.914537+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955421+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:18:41.955421+0000","last_peered":"2024-01-15T18:18:41.955421+0000","last_clean":"2024-01-15T18:18:41.955421+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:18:41.955421+0000","last_undegraded":"2024-01-15T18:18:41.955421+0000","last_fullsized":"2024-01-15T18:18:41.955421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:34.253432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947043+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:18:41.947043+0000","last_peered":"2024-01-15T18:18:41.947043+0000","last_clean":"2024-01-15T18:18:41.947043+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:18:41.947043+0000","last_undegraded":"2024-01-15T18:18:41.947043+0000","last_fullsized":"2024-01-15T18:18:41.947043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.910456+0000","last_change":"2024-01-15T18:18:40.910456+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.910456+0000","last_undegraded":"2024-01-15T18:18:40.910456+0000","last_fullsized":"2024-01-15T18:18:40.910456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909936+0000","last_change":"2024-01-15T18:18:40.909936+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909936+0000","last_undegraded":"2024-01-15T18:18:40.909936+0000","last_fullsized":"2024-01-15T18:18:40.909936+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.913060+0000","last_change":"2024-01-15T18:18:40.913060+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.913060+0000","last_undegraded":"2024-01-15T18:18:40.913060+0000","last_fullsized":"2024-01-15T18:18:40.913060+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907829+0000","last_change":"2024-01-15T18:18:40.907829+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907829+0000","last_undegraded":"2024-01-15T18:18:40.907829+0000","last_fullsized":"2024-01-15T18:18:40.907829+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907707+0000","last_change":"2024-01-15T18:18:40.907707+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907707+0000","last_undegraded":"2024-01-15T18:18:40.907707+0000","last_fullsized":"2024-01-15T18:18:40.907707+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955395+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:18:41.955395+0000","last_peered":"2024-01-15T18:18:41.955395+0000","last_clean":"2024-01-15T18:18:41.955395+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:18:41.955395+0000","last_undegraded":"2024-01-15T18:18:41.955395+0000","last_fullsized":"2024-01-15T18:18:41.955395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.906954+0000","last_change":"2024-01-15T18:18:40.906954+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.906954+0000","last_undegraded":"2024-01-15T18:18:40.906954+0000","last_fullsized":"2024-01-15T18:18:40.906954+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955258+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:18:41.955258+0000","last_peered":"2024-01-15T18:18:41.955258+0000","last_clean":"2024-01-15T18:18:41.955258+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:18:41.955258+0000","last_undegraded":"2024-01-15T18:18:41.955258+0000","last_fullsized":"2024-01-15T18:18:41.955258+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908132+0000","last_change":"2024-01-15T18:18:40.908132+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908132+0000","last_undegraded":"2024-01-15T18:18:40.908132+0000","last_fullsized":"2024-01-15T18:18:40.908132+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955709+0000","last_change":"2024-01-15T18:18:41.949864+0000","last_active":"2024-01-15T18:18:41.955709+0000","last_peered":"2024-01-15T18:18:41.955709+0000","last_clean":"2024-01-15T18:18:41.955709+0000","last_became_active":"2024-01-15T18:18:41.944900+0000","last_became_peered":"2024-01-15T18:18:41.944900+0000","last_unstale":"2024-01-15T18:18:41.955709+0000","last_undegraded":"2024-01-15T18:18:41.955709+0000","last_fullsized":"2024-01-15T18:18:41.955709+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933525+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:18:41.933525+0000","last_peered":"2024-01-15T18:18:41.933525+0000","last_clean":"2024-01-15T18:18:41.933525+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:18:41.933525+0000","last_undegraded":"2024-01-15T18:18:41.933525+0000","last_fullsized":"2024-01-15T18:18:41.933525+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:28.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957639+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:18:41.957639+0000","last_peered":"2024-01-15T18:18:41.957639+0000","last_clean":"2024-01-15T18:18:41.957639+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:18:41.957639+0000","last_undegraded":"2024-01-15T18:18:41.957639+0000","last_fullsized":"2024-01-15T18:18:41.957639+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.910848+0000","last_change":"2024-01-15T18:18:40.910848+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.910848+0000","last_undegraded":"2024-01-15T18:18:40.910848+0000","last_fullsized":"2024-01-15T18:18:40.910848+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.917233+0000","last_change":"2024-01-15T18:18:40.917233+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.917233+0000","last_undegraded":"2024-01-15T18:18:40.917233+0000","last_fullsized":"2024-01-15T18:18:40.917233+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955724+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:18:41.955724+0000","last_peered":"2024-01-15T18:18:41.955724+0000","last_clean":"2024-01-15T18:18:41.955724+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:18:41.955724+0000","last_undegraded":"2024-01-15T18:18:41.955724+0000","last_fullsized":"2024-01-15T18:18:41.955724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955368+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:18:41.955368+0000","last_peered":"2024-01-15T18:18:41.955368+0000","last_clean":"2024-01-15T18:18:41.955368+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:18:41.955368+0000","last_undegraded":"2024-01-15T18:18:41.955368+0000","last_fullsized":"2024-01-15T18:18:41.955368+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954348+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:18:41.954348+0000","last_peered":"2024-01-15T18:18:41.954348+0000","last_clean":"2024-01-15T18:18:41.954348+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:18:41.954348+0000","last_undegraded":"2024-01-15T18:18:41.954348+0000","last_fullsized":"2024-01-15T18:18:41.954348+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:14.769436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.927486+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:18:40.927486+0000","last_peered":"2024-01-15T18:18:40.927486+0000","last_clean":"2024-01-15T18:18:40.927486+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:18:40.927486+0000","last_undegraded":"2024-01-15T18:18:40.927486+0000","last_fullsized":"2024-01-15T18:18:40.927486+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:17.166792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908694+0000","last_change":"2024-01-15T18:18:40.908694+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908694+0000","last_undegraded":"2024-01-15T18:18:40.908694+0000","last_fullsized":"2024-01-15T18:18:40.908694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925246+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:18:40.925246+0000","last_peered":"2024-01-15T18:18:40.925246+0000","last_clean":"2024-01-15T18:18:40.925246+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:18:40.925246+0000","last_undegraded":"2024-01-15T18:18:40.925246+0000","last_fullsized":"2024-01-15T18:18:40.925246+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:34.563635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954412+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:18:41.954412+0000","last_peered":"2024-01-15T18:18:41.954412+0000","last_clean":"2024-01-15T18:18:41.954412+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:18:41.954412+0000","last_undegraded":"2024-01-15T18:18:41.954412+0000","last_fullsized":"2024-01-15T18:18:41.954412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:49.284112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946203+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:18:40.946203+0000","last_peered":"2024-01-15T18:18:40.946203+0000","last_clean":"2024-01-15T18:18:40.946203+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:18:40.946203+0000","last_undegraded":"2024-01-15T18:18:40.946203+0000","last_fullsized":"2024-01-15T18:18:40.946203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:09.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912545+0000","last_change":"2024-01-15T18:18:40.912545+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912545+0000","last_undegraded":"2024-01-15T18:18:40.912545+0000","last_fullsized":"2024-01-15T18:18:40.912545+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946121+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:18:40.946121+0000","last_peered":"2024-01-15T18:18:40.946121+0000","last_clean":"2024-01-15T18:18:40.946121+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:18:40.946121+0000","last_undegraded":"2024-01-15T18:18:40.946121+0000","last_fullsized":"2024-01-15T18:18:40.946121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:10.172303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955156+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:18:41.955156+0000","last_peered":"2024-01-15T18:18:41.955156+0000","last_clean":"2024-01-15T18:18:41.955156+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:18:41.955156+0000","last_undegraded":"2024-01-15T18:18:41.955156+0000","last_fullsized":"2024-01-15T18:18:41.955156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.932352+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:18:40.932352+0000","last_peered":"2024-01-15T18:18:40.932352+0000","last_clean":"2024-01-15T18:18:40.932352+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:18:40.932352+0000","last_undegraded":"2024-01-15T18:18:40.932352+0000","last_fullsized":"2024-01-15T18:18:40.932352+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:23.333161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909660+0000","last_change":"2024-01-15T18:18:40.909660+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909660+0000","last_undegraded":"2024-01-15T18:18:40.909660+0000","last_fullsized":"2024-01-15T18:18:40.909660+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949383+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:18:41.949383+0000","last_peered":"2024-01-15T18:18:41.949383+0000","last_clean":"2024-01-15T18:18:41.949383+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:18:41.949383+0000","last_undegraded":"2024-01-15T18:18:41.949383+0000","last_fullsized":"2024-01-15T18:18:41.949383+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954520+0000","last_change":"2024-01-15T18:18:41.940718+0000","last_active":"2024-01-15T18:18:41.954520+0000","last_peered":"2024-01-15T18:18:41.954520+0000","last_clean":"2024-01-15T18:18:41.954520+0000","last_became_active":"2024-01-15T18:18:41.940432+0000","last_became_peered":"2024-01-15T18:18:41.940432+0000","last_unstale":"2024-01-15T18:18:41.954520+0000","last_undegraded":"2024-01-15T18:18:41.954520+0000","last_fullsized":"2024-01-15T18:18:41.954520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925490+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:18:40.925490+0000","last_peered":"2024-01-15T18:18:40.925490+0000","last_clean":"2024-01-15T18:18:40.925490+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:18:40.925490+0000","last_undegraded":"2024-01-15T18:18:40.925490+0000","last_fullsized":"2024-01-15T18:18:40.925490+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:41.088175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.918761+0000","last_change":"2024-01-15T18:18:40.918761+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.918761+0000","last_undegraded":"2024-01-15T18:18:40.918761+0000","last_fullsized":"2024-01-15T18:18:40.918761+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918022+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:18:41.918022+0000","last_peered":"2024-01-15T18:18:41.918022+0000","last_clean":"2024-01-15T18:18:41.918022+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:18:41.918022+0000","last_undegraded":"2024-01-15T18:18:41.918022+0000","last_fullsized":"2024-01-15T18:18:41.918022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.916907+0000","last_change":"2024-01-15T18:18:40.916907+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.916907+0000","last_undegraded":"2024-01-15T18:18:40.916907+0000","last_fullsized":"2024-01-15T18:18:40.916907+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946140+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:18:40.946140+0000","last_peered":"2024-01-15T18:18:40.946140+0000","last_clean":"2024-01-15T18:18:40.946140+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:18:40.946140+0000","last_undegraded":"2024-01-15T18:18:40.946140+0000","last_fullsized":"2024-01-15T18:18:40.946140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907383+0000","last_change":"2024-01-15T18:18:40.907383+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907383+0000","last_undegraded":"2024-01-15T18:18:40.907383+0000","last_fullsized":"2024-01-15T18:18:40.907383+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.926304+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:18:41.926304+0000","last_peered":"2024-01-15T18:18:41.926304+0000","last_clean":"2024-01-15T18:18:41.926304+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:18:41.926304+0000","last_undegraded":"2024-01-15T18:18:41.926304+0000","last_fullsized":"2024-01-15T18:18:41.926304+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955327+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:18:41.955327+0000","last_peered":"2024-01-15T18:18:41.955327+0000","last_clean":"2024-01-15T18:18:41.955327+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:18:41.955327+0000","last_undegraded":"2024-01-15T18:18:41.955327+0000","last_fullsized":"2024-01-15T18:18:41.955327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957957+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:18:41.957957+0000","last_peered":"2024-01-15T18:18:41.957957+0000","last_clean":"2024-01-15T18:18:41.957957+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:18:41.957957+0000","last_undegraded":"2024-01-15T18:18:41.957957+0000","last_fullsized":"2024-01-15T18:18:41.957957+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.916144+0000","last_change":"2024-01-15T18:18:40.916144+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.916144+0000","last_undegraded":"2024-01-15T18:18:40.916144+0000","last_fullsized":"2024-01-15T18:18:40.916144+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.960177+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:18:41.960177+0000","last_peered":"2024-01-15T18:18:41.960177+0000","last_clean":"2024-01-15T18:18:41.960177+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:18:41.960177+0000","last_undegraded":"2024-01-15T18:18:41.960177+0000","last_fullsized":"2024-01-15T18:18:41.960177+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955377+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:18:41.955377+0000","last_peered":"2024-01-15T18:18:41.955377+0000","last_clean":"2024-01-15T18:18:41.955377+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:18:41.955377+0000","last_undegraded":"2024-01-15T18:18:41.955377+0000","last_fullsized":"2024-01-15T18:18:41.955377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949761+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:18:41.949761+0000","last_peered":"2024-01-15T18:18:41.949761+0000","last_clean":"2024-01-15T18:18:41.949761+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:18:41.949761+0000","last_undegraded":"2024-01-15T18:18:41.949761+0000","last_fullsized":"2024-01-15T18:18:41.949761+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955653+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:18:41.955653+0000","last_peered":"2024-01-15T18:18:41.955653+0000","last_clean":"2024-01-15T18:18:41.955653+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:18:41.955653+0000","last_undegraded":"2024-01-15T18:18:41.955653+0000","last_fullsized":"2024-01-15T18:18:41.955653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.702381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.923005+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:18:40.923005+0000","last_peered":"2024-01-15T18:18:40.923005+0000","last_clean":"2024-01-15T18:18:40.923005+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:18:40.923005+0000","last_undegraded":"2024-01-15T18:18:40.923005+0000","last_fullsized":"2024-01-15T18:18:40.923005+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:16.430602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955766+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:18:41.955766+0000","last_peered":"2024-01-15T18:18:41.955766+0000","last_clean":"2024-01-15T18:18:41.955766+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:18:41.955766+0000","last_undegraded":"2024-01-15T18:18:41.955766+0000","last_fullsized":"2024-01-15T18:18:41.955766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:17.438389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.939931+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:18:40.939931+0000","last_peered":"2024-01-15T18:18:40.939931+0000","last_clean":"2024-01-15T18:18:40.939931+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:18:40.939931+0000","last_undegraded":"2024-01-15T18:18:40.939931+0000","last_fullsized":"2024-01-15T18:18:40.939931+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:30.432388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.921779+0000","last_change":"2024-01-15T18:18:40.921779+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.921779+0000","last_undegraded":"2024-01-15T18:18:40.921779+0000","last_fullsized":"2024-01-15T18:18:40.921779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.946443+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:18:41.946443+0000","last_peered":"2024-01-15T18:18:41.946443+0000","last_clean":"2024-01-15T18:18:41.946443+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:18:41.946443+0000","last_undegraded":"2024-01-15T18:18:41.946443+0000","last_fullsized":"2024-01-15T18:18:41.946443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933399+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:18:41.933399+0000","last_peered":"2024-01-15T18:18:41.933399+0000","last_clean":"2024-01-15T18:18:41.933399+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:18:41.933399+0000","last_undegraded":"2024-01-15T18:18:41.933399+0000","last_fullsized":"2024-01-15T18:18:41.933399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.408977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931086+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:18:40.931086+0000","last_peered":"2024-01-15T18:18:40.931086+0000","last_clean":"2024-01-15T18:18:40.931086+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:18:40.931086+0000","last_undegraded":"2024-01-15T18:18:40.931086+0000","last_fullsized":"2024-01-15T18:18:40.931086+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:51:44.688957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.911900+0000","last_change":"2024-01-15T18:18:40.911900+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.911900+0000","last_undegraded":"2024-01-15T18:18:40.911900+0000","last_fullsized":"2024-01-15T18:18:40.911900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.923118+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:18:40.923118+0000","last_peered":"2024-01-15T18:18:40.923118+0000","last_clean":"2024-01-15T18:18:40.923118+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:18:40.923118+0000","last_undegraded":"2024-01-15T18:18:40.923118+0000","last_fullsized":"2024-01-15T18:18:40.923118+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:15.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.911924+0000","last_change":"2024-01-15T18:18:40.911924+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.911924+0000","last_undegraded":"2024-01-15T18:18:40.911924+0000","last_fullsized":"2024-01-15T18:18:40.911924+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931943+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:18:40.931943+0000","last_peered":"2024-01-15T18:18:40.931943+0000","last_clean":"2024-01-15T18:18:40.931943+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:18:40.931943+0000","last_undegraded":"2024-01-15T18:18:40.931943+0000","last_fullsized":"2024-01-15T18:18:40.931943+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:46.912158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.914035+0000","last_change":"2024-01-15T18:18:40.914035+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.914035+0000","last_undegraded":"2024-01-15T18:18:40.914035+0000","last_fullsized":"2024-01-15T18:18:40.914035+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958826+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:18:41.958826+0000","last_peered":"2024-01-15T18:18:41.958826+0000","last_clean":"2024-01-15T18:18:41.958826+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:18:41.958826+0000","last_undegraded":"2024-01-15T18:18:41.958826+0000","last_fullsized":"2024-01-15T18:18:41.958826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.940187+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:18:41.940187+0000","last_peered":"2024-01-15T18:18:41.940187+0000","last_clean":"2024-01-15T18:18:41.940187+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:18:41.940187+0000","last_undegraded":"2024-01-15T18:18:41.940187+0000","last_fullsized":"2024-01-15T18:18:41.940187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.947226+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:18:40.947226+0000","last_peered":"2024-01-15T18:18:40.947226+0000","last_clean":"2024-01-15T18:18:40.947226+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:18:40.947226+0000","last_undegraded":"2024-01-15T18:18:40.947226+0000","last_fullsized":"2024-01-15T18:18:40.947226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:53.075302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912833+0000","last_change":"2024-01-15T18:18:40.912833+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912833+0000","last_undegraded":"2024-01-15T18:18:40.912833+0000","last_fullsized":"2024-01-15T18:18:40.912833+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.924617+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:18:41.924617+0000","last_peered":"2024-01-15T18:18:41.924617+0000","last_clean":"2024-01-15T18:18:41.924617+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:18:41.924617+0000","last_undegraded":"2024-01-15T18:18:41.924617+0000","last_fullsized":"2024-01-15T18:18:41.924617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947927+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:18:41.947927+0000","last_peered":"2024-01-15T18:18:41.947927+0000","last_clean":"2024-01-15T18:18:41.947927+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:18:41.947927+0000","last_undegraded":"2024-01-15T18:18:41.947927+0000","last_fullsized":"2024-01-15T18:18:41.947927+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:43.903199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918200+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:18:41.918200+0000","last_peered":"2024-01-15T18:18:41.918200+0000","last_clean":"2024-01-15T18:18:41.918200+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:18:41.918200+0000","last_undegraded":"2024-01-15T18:18:41.918200+0000","last_fullsized":"2024-01-15T18:18:41.918200+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908189+0000","last_change":"2024-01-15T18:18:40.908189+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908189+0000","last_undegraded":"2024-01-15T18:18:40.908189+0000","last_fullsized":"2024-01-15T18:18:40.908189+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957662+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:18:41.957662+0000","last_peered":"2024-01-15T18:18:41.957662+0000","last_clean":"2024-01-15T18:18:41.957662+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:18:41.957662+0000","last_undegraded":"2024-01-15T18:18:41.957662+0000","last_fullsized":"2024-01-15T18:18:41.957662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.944610+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:18:41.944610+0000","last_peered":"2024-01-15T18:18:41.944610+0000","last_clean":"2024-01-15T18:18:41.944610+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:18:41.944610+0000","last_undegraded":"2024-01-15T18:18:41.944610+0000","last_fullsized":"2024-01-15T18:18:41.944610+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.925197+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:18:40.925197+0000","last_peered":"2024-01-15T18:18:40.925197+0000","last_clean":"2024-01-15T18:18:40.925197+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:18:40.925197+0000","last_undegraded":"2024-01-15T18:18:40.925197+0000","last_fullsized":"2024-01-15T18:18:40.925197+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:41.088175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951944+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:18:41.951944+0000","last_peered":"2024-01-15T18:18:41.951944+0000","last_clean":"2024-01-15T18:18:41.951944+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:18:41.951944+0000","last_undegraded":"2024-01-15T18:18:41.951944+0000","last_fullsized":"2024-01-15T18:18:41.951944+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918797+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:18:41.918797+0000","last_peered":"2024-01-15T18:18:41.918797+0000","last_clean":"2024-01-15T18:18:41.918797+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:18:41.918797+0000","last_undegraded":"2024-01-15T18:18:41.918797+0000","last_fullsized":"2024-01-15T18:18:41.918797+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.907378+0000","last_change":"2024-01-15T18:18:40.907378+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.907378+0000","last_undegraded":"2024-01-15T18:18:40.907378+0000","last_fullsized":"2024-01-15T18:18:40.907378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.927483+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:18:41.927483+0000","last_peered":"2024-01-15T18:18:41.927483+0000","last_clean":"2024-01-15T18:18:41.927483+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:18:41.927483+0000","last_undegraded":"2024-01-15T18:18:41.927483+0000","last_fullsized":"2024-01-15T18:18:41.927483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.908276+0000","last_change":"2024-01-15T18:18:40.908276+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.908276+0000","last_undegraded":"2024-01-15T18:18:40.908276+0000","last_fullsized":"2024-01-15T18:18:40.908276+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.927500+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:18:40.927500+0000","last_peered":"2024-01-15T18:18:40.927500+0000","last_clean":"2024-01-15T18:18:40.927500+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:18:40.927500+0000","last_undegraded":"2024-01-15T18:18:40.927500+0000","last_fullsized":"2024-01-15T18:18:40.927500+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:33.731825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.909185+0000","last_change":"2024-01-15T18:18:40.909185+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.909185+0000","last_undegraded":"2024-01-15T18:18:40.909185+0000","last_fullsized":"2024-01-15T18:18:40.909185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.936592+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:18:40.936592+0000","last_peered":"2024-01-15T18:18:40.936592+0000","last_clean":"2024-01-15T18:18:40.936592+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:18:40.936592+0000","last_undegraded":"2024-01-15T18:18:40.936592+0000","last_fullsized":"2024-01-15T18:18:40.936592+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:15.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.912287+0000","last_change":"2024-01-15T18:18:40.912287+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.912287+0000","last_undegraded":"2024-01-15T18:18:40.912287+0000","last_fullsized":"2024-01-15T18:18:40.912287+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946157+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:18:40.946157+0000","last_peered":"2024-01-15T18:18:40.946157+0000","last_clean":"2024-01-15T18:18:40.946157+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:18:40.946157+0000","last_undegraded":"2024-01-15T18:18:40.946157+0000","last_fullsized":"2024-01-15T18:18:40.946157+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:50.249980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955068+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:18:41.955068+0000","last_peered":"2024-01-15T18:18:41.955068+0000","last_clean":"2024-01-15T18:18:41.955068+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:18:41.955068+0000","last_undegraded":"2024-01-15T18:18:41.955068+0000","last_fullsized":"2024-01-15T18:18:41.955068+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918226+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:18:41.918226+0000","last_peered":"2024-01-15T18:18:41.918226+0000","last_clean":"2024-01-15T18:18:41.918226+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:18:41.918226+0000","last_undegraded":"2024-01-15T18:18:41.918226+0000","last_fullsized":"2024-01-15T18:18:41.918226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.953440+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:18:41.953440+0000","last_peered":"2024-01-15T18:18:41.953440+0000","last_clean":"2024-01-15T18:18:41.953440+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:18:41.953440+0000","last_undegraded":"2024-01-15T18:18:41.953440+0000","last_fullsized":"2024-01-15T18:18:41.953440+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:52.321006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.930839+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:18:40.930839+0000","last_peered":"2024-01-15T18:18:40.930839+0000","last_clean":"2024-01-15T18:18:40.930839+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:18:40.930839+0000","last_undegraded":"2024-01-15T18:18:40.930839+0000","last_fullsized":"2024-01-15T18:18:40.930839+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:47.752921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":239,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-01-15T18:18:39.904797+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:18:39.904797+0000","last_peered":"2024-01-15T18:18:39.904797+0000","last_clean":"2024-01-15T18:18:39.904797+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:18:39.904797+0000","last_undegraded":"2024-01-15T18:18:39.904797+0000","last_fullsized":"2024-01-15T18:18:39.904797+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":1758,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575019,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64960,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306880,"statfs":{"total":96636764160,"available":96570245120,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.86399999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.622},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.77400000000000002,"15min":0.85999999999999999},"min":{"1min":0.48299999999999998,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.92500000000000004,"5min":4.6100000000000003,"15min":13.744},"last":0.60799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.96799999999999997,"15min":1.0640000000000001},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":3.016,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.81100000000000005,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":4.7679999999999998,"15min":31.774999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.74099999999999999,"15min":0.98699999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":2.1680000000000001,"15min":30.846},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":1.073},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":17.039999999999999,"15min":39.695},"last":0.65300000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.82599999999999996,"15min":1.1579999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.88400000000000001,"5min":8.2650000000000006,"15min":26.463999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.94699999999999995,"15min":1.202},"min":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":16.879999999999999,"15min":42.692},"last":0.68200000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78800000000000003,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":1.147,"5min":14.087,"15min":14.087},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.629,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.63500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.81299999999999994,"15min":0.71199999999999997},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":1.0209999999999999,"5min":11.334,"15min":11.334},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":4.8179999999999996,"15min":8.423},"last":0.69299999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5369999999999999,"5min":4.7140000000000004,"15min":8.5030000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1970000000000001,"15min":3.3450000000000002},"last":0.45800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.81699999999999995,"15min":0.95299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.754},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74199999999999999,"15min":0.873},"min":{"1min":0.375,"5min":0.375,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":3.4940000000000002,"15min":12.449},"last":0.82599999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.85599999999999998,"15min":0.89700000000000002},"min":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.33900000000000002},"max":{"1min":1.0489999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.84199999999999997,"15min":1.004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.81899999999999995,"5min":8.9320000000000004,"15min":19.475000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.069,"15min":1.179},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":14.743,"15min":42.825000000000003},"last":0.997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.89000000000000001,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":9.4000000000000004,"15min":42.851999999999997},"last":1.0720000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78300000000000003,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":0.82099999999999995,"5min":3.016,"15min":24.199000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0629999999999999,"15min":1.087},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":2.956,"5min":18.791,"15min":18.791},"last":0.872}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.89300000000000002,"15min":0.745},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":19.114999999999998,"15min":19.114999999999998},"last":1.044},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33700000000000002},"max":{"1min":0.875,"5min":1.7450000000000001,"15min":12.491},"last":0.57999999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81899999999999995,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.89800000000000002},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.5259999999999998,"15min":12.945},"last":1.1120000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":1.006,"15min":0.83899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":11.907,"15min":11.907},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82499999999999996,"15min":0.88400000000000001},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":9.9440000000000008,"15min":27.651},"last":1.1910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575019,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76860,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294980,"statfs":{"total":96636764160,"available":96558059520,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":4.4770000000000003,"15min":11.519},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.874,"15min":0.92400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.91700000000000004,"5min":7.6260000000000003,"15min":12.952},"last":0.68899999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.92700000000000005,"15min":0.90300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.95499999999999996,"5min":4.3140000000000001,"15min":13.358000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.94699999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0169999999999999,"5min":4.1580000000000004,"15min":21.856999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":1,"5min":6.7160000000000002,"15min":18.314},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.145,"15min":1.0109999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.0580000000000001,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.752}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.85499999999999998,"15min":0.94499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":3.2400000000000002,"15min":13.317},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":1.01},"min":{"1min":0.437,"5min":0.437,"15min":0.42899999999999999},"max":{"1min":0.998,"5min":3.931,"15min":19.638000000000002},"last":0.72799999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":0.96099999999999997,"5min":5.899,"15min":11.093},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":0.88700000000000001,"5min":2.516,"15min":3.3889999999999998},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84399999999999997,"15min":0.79500000000000004},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97899999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.308},"max":{"1min":1.0840000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.85899999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":1.071,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":1.034,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":1}]}]},{"osd":4,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81020,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290820,"statfs":{"total":96636764160,"available":96553799680,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.79800000000000004,"15min":0.93300000000000005},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":9.4179999999999993,"15min":34.802},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":14.488,"15min":22.335999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.95699999999999996},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":2.2130000000000001,"15min":38.198},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.80500000000000005,"15min":0.95599999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.86599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95099999999999996,"5min":1.6259999999999999,"15min":7.6989999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.78500000000000003,"15min":0.84799999999999998},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":4.4089999999999998,"15min":7.4829999999999997},"last":0.65600000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.91300000000000003,"15min":0.95399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":14.529,"15min":14.529},"last":0.628},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72299999999999998,"15min":0.92900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":1.4279999999999999,"15min":25.774000000000001},"last":0.60699999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":0.68200000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.97799999999999998,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.47499999999999998,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":25.887,"15min":25.887},"last":0.36299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":10.284000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.46200000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.82799999999999996,"15min":0.78300000000000003},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.77100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"max":{"1min":1.03,"5min":1.232,"15min":11.67},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.625,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.255},"max":{"1min":0.81899999999999995,"5min":7.2839999999999998,"15min":26.364000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.66300000000000003,"15min":0.95599999999999996},"min":{"1min":0.312,"5min":0.25,"15min":0.246},"max":{"1min":0.71499999999999997,"5min":7.8520000000000003,"15min":28.332000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.60799999999999998,"15min":0.98799999999999999},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.84799999999999998,"5min":3.8100000000000001,"15min":40.457999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.76300000000000001,"15min":1.1419999999999999},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":9.282,"15min":27.427},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.71199999999999997,"15min":0.98899999999999999},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.90400000000000003,"5min":9.1180000000000003,"15min":23.036999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.81499999999999995,"15min":1.089},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":1.048,"5min":9.3510000000000009,"15min":26.404},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":1.024},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.41199999999999998},"max":{"1min":1.0780000000000001,"5min":4.1040000000000001,"15min":22.998000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84599999999999997,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.0229999999999999,"5min":3.8540000000000001,"15min":20.399000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":1.097},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":6.5140000000000002,"15min":26.314},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85599999999999998,"15min":1.1220000000000001},"min":{"1min":0.46100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":4.165,"15min":22.908999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":1.048,"15min":1.153},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.42799999999999999},"max":{"1min":1.0609999999999999,"5min":6.843,"15min":20.445},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.1850000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.123,"5min":6.7640000000000002,"15min":26.091999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.98999999999999999,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":7.0250000000000004,"15min":23.111999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.92000000000000004,"15min":1.321},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46000000000000002},"max":{"1min":1.1419999999999999,"5min":4.4470000000000001,"15min":44.817999999999998},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64380,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307460,"statfs":{"total":96636764160,"available":96570839040,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54400000000000004,"15min":1.0529999999999999},"min":{"1min":0.31,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.373},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.86199999999999999,"15min":1.321},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.66400000000000003,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.39200000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.64700000000000002,"15min":1.5409999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.56299999999999994,"5min":7.476,"15min":113.149},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.96899999999999997,"15min":1.1080000000000001},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":20.619,"15min":41.302},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.341,"15min":1.379},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28899999999999998},"max":{"1min":0.67400000000000004,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66000000000000003,"15min":1.27},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":0.77300000000000002,"5min":7.3609999999999998,"15min":115.693},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":1.206},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":7.4169999999999998,"15min":24.366},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0740000000000001,"15min":1.23},"min":{"1min":0.51600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.5800000000000001,"15min":1.5089999999999999},"min":{"1min":0.54700000000000004,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84999999999999998,"15min":1.304},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":7.5099999999999998,"15min":26.084},"last":0.54900000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":1.0980000000000001,"15min":1.3149999999999999},"min":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":20.120999999999999,"15min":22.199000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.748,"5min":1.6240000000000001,"15min":1.5700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":1.165,"15min":1.262},"min":{"1min":0.55200000000000005,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":0.92600000000000005,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":1.613,"15min":1.6299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64404,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307436,"statfs":{"total":96636764160,"available":96570814464,"internally_reserved":0,"allocated":1376256,"data_stored":593098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.64200000000000002,"15min":0.91500000000000004},"min":{"1min":0.245,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.68200000000000005,"5min":9.452,"15min":39.948},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.68799999999999994,"15min":1.103},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":8.141,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.71799999999999997,"15min":0.98199999999999998},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":0.63800000000000001,"5min":15.137,"15min":49.229999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61799999999999999,"5min":9.8469999999999995,"15min":10.349},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61099999999999999,"15min":1.173},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":4.4669999999999996,"15min":27.643999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.90000000000000002,"15min":0.997},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.66800000000000004,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84299999999999997,"15min":1.222},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":4.633,"15min":27.489999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67600000000000005,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98599999999999999,"5min":9.6989999999999998,"15min":12.997},"last":0.83599999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.86599999999999999,"15min":1.212},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":4.673,"15min":27.547000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.0289999999999999,"15min":1.1200000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.496},"max":{"1min":1.028,"5min":9.7650000000000006,"15min":13.054},"last":0.78100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.86499999999999999,"15min":1.204},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":4.7060000000000004,"15min":27.579999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.87,"15min":1.0369999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.1319999999999999},"min":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":6.6950000000000003,"5min":9.5660000000000007,"15min":11.833},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1.0229999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.502},"max":{"1min":0.93200000000000005,"5min":9.8079999999999998,"15min":14.606},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.78600000000000003,"15min":1.1379999999999999},"min":{"1min":0.27100000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.94299999999999995,"5min":11.356,"15min":31.977},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.81299999999999994,"15min":1.0229999999999999},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0289999999999999,"5min":11.629,"15min":36.145000000000003},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.76600000000000001,"15min":1.1319999999999999},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":0.91800000000000004,"5min":5.8929999999999998,"15min":41.387999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.877,"15min":1.038},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.25},"max":{"1min":0.97499999999999998,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.97599999999999998,"15min":1.0860000000000001},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.093,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.79400000000000004,"15min":1.097},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":6.3739999999999997,"15min":45.128},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.0289999999999999,"15min":1.0740000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.372},"max":{"1min":2.1469999999999998,"5min":11.772,"15min":32.932000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.0840000000000001,"15min":1.1919999999999999},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":1.4370000000000001,"5min":11.220000000000001,"15min":31.782},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1.083,"15min":1.1850000000000001},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3979999999999999,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.841,"5min":11.958,"15min":33.151000000000003},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":1.0640000000000001},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.379,"5min":4.0970000000000004,"15min":13.055},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.749,"5min":1.0449999999999999,"15min":1.0920000000000001},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.522,"5min":11.833,"15min":33.098999999999997},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.92300000000000004,"15min":1.0740000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3160000000000001,"5min":4.1349999999999998,"15min":13.090999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.1120000000000001,"15min":1.242},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.464,"5min":11.321999999999999,"15min":31.905999999999999},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:46.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2024-01-15T18:18:46.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.908026+0000', 'last_change': '2024-01-15T18:18:40.908026+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.908026+0000', 'last_undegraded': '2024-01-15T18:18:40.908026+0000', 'last_fullsized': '2024-01-15T18:18:40.908026+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2024-01-15T18:18:46.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.913598+0000', 'last_change': '2024-01-15T18:18:40.913598+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.913598+0000', 'last_undegraded': '2024-01-15T18:18:40.913598+0000', 'last_fullsized': '2024-01-15T18:18:40.913598+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.925271+0000', 'last_change': '2024-01-15T18:18:40.925271+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.925271+0000', 'last_undegraded': '2024-01-15T18:18:40.925271+0000', 'last_fullsized': '2024-01-15T18:18:40.925271+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.919588+0000', 'last_change': '2024-01-15T18:18:40.919588+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.919588+0000', 'last_undegraded': '2024-01-15T18:18:40.919588+0000', 'last_fullsized': '2024-01-15T18:18:40.919588+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.913325+0000', 'last_change': '2024-01-15T18:18:40.913325+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.913325+0000', 'last_undegraded': '2024-01-15T18:18:40.913325+0000', 'last_fullsized': '2024-01-15T18:18:40.913325+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.907781+0000', 'last_change': '2024-01-15T18:18:40.907781+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.907781+0000', 'last_undegraded': '2024-01-15T18:18:40.907781+0000', 'last_fullsized': '2024-01-15T18:18:40.907781+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.913469+0000', 'last_change': '2024-01-15T18:18:40.913469+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.913469+0000', 'last_undegraded': '2024-01-15T18:18:40.913469+0000', 'last_fullsized': '2024-01-15T18:18:40.913469+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.914537+0000', 'last_change': '2024-01-15T18:18:40.914537+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.914537+0000', 'last_undegraded': '2024-01-15T18:18:40.914537+0000', 'last_fullsized': '2024-01-15T18:18:40.914537+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.910456+0000', 'last_change': '2024-01-15T18:18:40.910456+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.910456+0000', 'last_undegraded': '2024-01-15T18:18:40.910456+0000', 'last_fullsized': '2024-01-15T18:18:40.910456+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.909936+0000', 'last_change': '2024-01-15T18:18:40.909936+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.909936+0000', 'last_undegraded': '2024-01-15T18:18:40.909936+0000', 'last_fullsized': '2024-01-15T18:18:40.909936+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2024-01-15T18:18:46.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.913060+0000', 'last_change': '2024-01-15T18:18:40.913060+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.913060+0000', 'last_undegraded': '2024-01-15T18:18:40.913060+0000', 'last_fullsized': '2024-01-15T18:18:40.913060+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.907829+0000', 'last_change': '2024-01-15T18:18:40.907829+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.907829+0000', 'last_undegraded': '2024-01-15T18:18:40.907829+0000', 'last_fullsized': '2024-01-15T18:18:40.907829+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.907707+0000', 'last_change': '2024-01-15T18:18:40.907707+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.907707+0000', 'last_undegraded': '2024-01-15T18:18:40.907707+0000', 'last_fullsized': '2024-01-15T18:18:40.907707+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.906954+0000', 'last_change': '2024-01-15T18:18:40.906954+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.906954+0000', 'last_undegraded': '2024-01-15T18:18:40.906954+0000', 'last_fullsized': '2024-01-15T18:18:40.906954+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.908132+0000', 'last_change': '2024-01-15T18:18:40.908132+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.908132+0000', 'last_undegraded': '2024-01-15T18:18:40.908132+0000', 'last_fullsized': '2024-01-15T18:18:40.908132+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.910848+0000', 'last_change': '2024-01-15T18:18:40.910848+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.910848+0000', 'last_undegraded': '2024-01-15T18:18:40.910848+0000', 'last_fullsized': '2024-01-15T18:18:40.910848+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.917233+0000', 'last_change': '2024-01-15T18:18:40.917233+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.917233+0000', 'last_undegraded': '2024-01-15T18:18:40.917233+0000', 'last_fullsized': '2024-01-15T18:18:40.917233+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.908694+0000', 'last_change': '2024-01-15T18:18:40.908694+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.908694+0000', 'last_undegraded': '2024-01-15T18:18:40.908694+0000', 'last_fullsized': '2024-01-15T18:18:40.908694+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.912545+0000', 'last_change': '2024-01-15T18:18:40.912545+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.912545+0000', 'last_undegraded': '2024-01-15T18:18:40.912545+0000', 'last_fullsized': '2024-01-15T18:18:40.912545+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.909660+0000', 'last_change': '2024-01-15T18:18:40.909660+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.909660+0000', 'last_undegraded': '2024-01-15T18:18:40.909660+0000', 'last_fullsized': '2024-01-15T18:18:40.909660+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.918761+0000', 'last_change': '2024-01-15T18:18:40.918761+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.918761+0000', 'last_undegraded': '2024-01-15T18:18:40.918761+0000', 'last_fullsized': '2024-01-15T18:18:40.918761+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.916907+0000', 'last_change': '2024-01-15T18:18:40.916907+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.916907+0000', 'last_undegraded': '2024-01-15T18:18:40.916907+0000', 'last_fullsized': '2024-01-15T18:18:40.916907+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.907383+0000', 'last_change': '2024-01-15T18:18:40.907383+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.907383+0000', 'last_undegraded': '2024-01-15T18:18:40.907383+0000', 'last_fullsized': '2024-01-15T18:18:40.907383+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.916144+0000', 'last_change': '2024-01-15T18:18:40.916144+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.916144+0000', 'last_undegraded': '2024-01-15T18:18:40.916144+0000', 'last_fullsized': '2024-01-15T18:18:40.916144+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2024-01-15T18:18:46.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.921779+0000', 'last_change': '2024-01-15T18:18:40.921779+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.921779+0000', 'last_undegraded': '2024-01-15T18:18:40.921779+0000', 'last_fullsized': '2024-01-15T18:18:40.921779+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.911900+0000', 'last_change': '2024-01-15T18:18:40.911900+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.911900+0000', 'last_undegraded': '2024-01-15T18:18:40.911900+0000', 'last_fullsized': '2024-01-15T18:18:40.911900+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.911924+0000', 'last_change': '2024-01-15T18:18:40.911924+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.911924+0000', 'last_undegraded': '2024-01-15T18:18:40.911924+0000', 'last_fullsized': '2024-01-15T18:18:40.911924+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.914035+0000', 'last_change': '2024-01-15T18:18:40.914035+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.914035+0000', 'last_undegraded': '2024-01-15T18:18:40.914035+0000', 'last_fullsized': '2024-01-15T18:18:40.914035+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.912833+0000', 'last_change': '2024-01-15T18:18:40.912833+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.912833+0000', 'last_undegraded': '2024-01-15T18:18:40.912833+0000', 'last_fullsized': '2024-01-15T18:18:40.912833+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2024-01-15T18:18:46.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.908189+0000', 'last_change': '2024-01-15T18:18:40.908189+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.908189+0000', 'last_undegraded': '2024-01-15T18:18:40.908189+0000', 'last_fullsized': '2024-01-15T18:18:40.908189+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.907378+0000', 'last_change': '2024-01-15T18:18:40.907378+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.907378+0000', 'last_undegraded': '2024-01-15T18:18:40.907378+0000', 'last_fullsized': '2024-01-15T18:18:40.907378+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.908276+0000', 'last_change': '2024-01-15T18:18:40.908276+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.908276+0000', 'last_undegraded': '2024-01-15T18:18:40.908276+0000', 'last_fullsized': '2024-01-15T18:18:40.908276+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.909185+0000', 'last_change': '2024-01-15T18:18:40.909185+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.909185+0000', 'last_undegraded': '2024-01-15T18:18:40.909185+0000', 'last_fullsized': '2024-01-15T18:18:40.909185+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2024-01-15T18:18:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 109, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:18:40.912287+0000', 'last_change': '2024-01-15T18:18:40.912287+0000', 'last_active': '2024-01-15T18:18:40.902736+0000', 'last_peered': '2024-01-15T18:18:40.902736+0000', 'last_clean': '2024-01-15T18:18:40.902736+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:18:40.912287+0000', 'last_undegraded': '2024-01-15T18:18:40.912287+0000', 'last_fullsized': '2024-01-15T18:18:40.912287+0000', 'mapping_epoch': 109, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 109, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'last_clean_scrub_stamp': '2024-01-15T18:18:40.902736+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:18:46.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T18:18:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.939+0000 7f85f7400640 1 Processor -- start 2024-01-15T18:18:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.939+0000 7f85f7400640 1 -- start start 2024-01-15T18:18:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f7400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85f015d8f0 0x7f85f015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f006a490 0x7f85f006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f7400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f01613d0 con 0x7f85f0146b90 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f7400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f006f6d0 con 0x7f85f015d8f0 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f7400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f006ae60 con 0x7f85f006a490 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85efe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85f015d8f0 0x7f85f015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f4a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f85f0146b90 0x7f85f0161860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38034/0 (socket says 172.21.15.143:38034) 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f4a00640 1 -- 172.21.15.143:0/1445074824 learned_addr learned my addr 172.21.15.143:0/1445074824 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85f5400640 1 --2- 172.21.15.143:0/1445074824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f006a490 0x7f85f006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85efe00640 1 -- 172.21.15.143:0/1445074824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f006a490 msgr2=0x7f85f006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85efe00640 1 --2- 172.21.15.143:0/1445074824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f006a490 0x7f85f006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85efe00640 1 -- 172.21.15.143:0/1445074824 >> v1:172.21.15.143:6789/0 conn(0x7f85f0146b90 legacy=0x7f85f0161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85ef400640 1 -- 172.21.15.143:0/1445074824 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462425246 0 0) 0x7f85f01613d0 con 0x7f85f0146b90 2024-01-15T18:18:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.943+0000 7f85efe00640 1 -- 172.21.15.143:0/1445074824 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f0146f70 con 0x7f85f015d8f0 2024-01-15T18:18:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85efe00640 1 --2- 172.21.15.143:0/1445074824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f015d8f0 0x7f85f015dd50 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f85d80097f0 tx=0x7f85d801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7179e34a2cbf562 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85ef400640 1 -- 172.21.15.143:0/1445074824 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d80ab5f0 con 0x7f85f015d8f0 2024-01-15T18:18:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85ef400640 1 -- 172.21.15.143:0/1445074824 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d80ab750 con 0x7f85f015d8f0 2024-01-15T18:18:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85ef400640 1 -- 172.21.15.143:0/1445074824 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d80abae0 con 0x7f85f015d8f0 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 -- 172.21.15.143:0/1445074824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f015d8f0 msgr2=0x7f85f015dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 --2- 172.21.15.143:0/1445074824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f015d8f0 0x7f85f015dd50 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f85d80097f0 tx=0x7f85d801c3f0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 -- 172.21.15.143:0/1445074824 shutdown_connections 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 --2- 172.21.15.143:0/1445074824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f006a490 0x7f85f006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 --2- 172.21.15.143:0/1445074824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f015d8f0 0x7f85f015dd50 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 -- 172.21.15.143:0/1445074824 >> 172.21.15.143:0/1445074824 conn(0x7f85f00b1ce0 msgr2=0x7f85f00b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 -- 172.21.15.143:0/1445074824 shutdown_connections 2024-01-15T18:18:46.943 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 -- 172.21.15.143:0/1445074824 wait complete. 2024-01-15T18:18:46.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.947+0000 7f85f7400640 1 Processor -- start 2024-01-15T18:18:46.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- start start 2024-01-15T18:18:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 0x7f85f0143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f0146b90 0x7f85f0140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85f015d8f0 0x7f85f0140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0143920 con 0x7f85f006a490 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0144280 con 0x7f85f015d8f0 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0141880 con 0x7f85f0146b90 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 0x7f85f0143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 0x7f85f0143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48044/0 (socket says 172.21.15.143:48044) 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 -- 172.21.15.143:0/2202995940 learned_addr learned my addr 172.21.15.143:0/2202995940 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85efe00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f0146b90 0x7f85f0140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f5400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85f015d8f0 0x7f85f0140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 -- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f0146b90 msgr2=0x7f85f0140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f0146b90 0x7f85f0140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 -- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85f015d8f0 msgr2=0x7f85f0140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85f015d8f0 0x7f85f0140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 -- 172.21.15.143:0/2202995940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8009420 con 0x7f85f006a490 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f4a00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 0x7f85f0143d40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f85e00011f0 tx=0x7f85e000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00074e0 con 0x7f85f006a490 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0007640 con 0x7f85f006a490 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f0142f80 con 0x7f85f006a490 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0004a10 con 0x7f85f006a490 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f0143540 con 0x7f85f006a490 2024-01-15T18:18:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.951+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f0143a60 con 0x7f85f006a490 2024-01-15T18:18:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.955+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f85e0004b70 con 0x7f85f006a490 2024-01-15T18:18:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.955+0000 7f85ecc00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85c8067a60 0x7f85c8069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.955+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f85e0082840 con 0x7f85f006a490 2024-01-15T18:18:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.955+0000 7f85efe00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85c8067a60 0x7f85c8069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:46.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.955+0000 7f85efe00640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85c8067a60 0x7f85c8069f20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f85d80af000 tx=0x7f85d80a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:46.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:46.959+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85f0143a60 con 0x7f85f006a490 2024-01-15T18:18:47.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f85f0143a60 con 0x7f85f006a490 2024-01-15T18:18:47.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85ecc00640 1 -- 172.21.15.143:0/2202995940 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1261 (secure 0 0 0) 0x7f85e0018080 con 0x7f85f006a490 2024-01-15T18:18:47.158 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:47.158 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":853,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341886,"num_in_osds":8,"osd_in_since":1705341876,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":95},{"state_name":"creating+peering","count":2}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":592778,"bytes_used":571318272,"bytes_avail":772522795008,"bytes_total":773094113280,"inactive_pgs_ratio":0.020618556067347527,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":72,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":9021}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-01-15T18:18:28.892887+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-01-15T18:18:47.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85c8067a60 msgr2=0x7f85c8069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85c8067a60 0x7f85c8069f20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f85d80af000 tx=0x7f85d80a7000 comp rx=0 tx=0).stop 2024-01-15T18:18:47.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 msgr2=0x7f85f0143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 0x7f85f0143d40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f85e00011f0 tx=0x7f85e000c9b0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 shutdown_connections 2024-01-15T18:18:47.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85c8067a60 0x7f85c8069f20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85f015d8f0 0x7f85f0140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85f0146b90 0x7f85f0140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 --2- 172.21.15.143:0/2202995940 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85f006a490 0x7f85f0143d40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.163+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 >> 172.21.15.143:0/2202995940 conn(0x7f85f00b1ce0 msgr2=0x7f85f0145af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:47.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.167+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 shutdown_connections 2024-01-15T18:18:47.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.167+0000 7f85f7400640 1 -- 172.21.15.143:0/2202995940 wait complete. 2024-01-15T18:18:47.180 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T18:18:47.181 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:47.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 Processor -- start 2024-01-15T18:18:47.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 -- start start 2024-01-15T18:18:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd2180ce560 0x7fd21815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:47.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd218071d00 0x7fd218072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd218072650 con 0x7fd218141660 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21815fbb0 con 0x7fd218071d00 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd21d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21815f310 con 0x7fd2180ce560 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd216800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd218071d00 0x7fd218072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd215e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd218141660 0x7fd218141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46744/0 (socket says 172.21.15.143:46744) 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.323+0000 7fd215e00640 1 -- 172.21.15.143:0/3203663832 learned_addr learned my addr 172.21.15.143:0/3203663832 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd215400640 1 --2- 172.21.15.143:0/3203663832 >> v2:172.21.15.204:3300/0 conn(0x7fd2180ce560 0x7fd21815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd216800640 1 -- 172.21.15.143:0/3203663832 >> v2:172.21.15.204:3300/0 conn(0x7fd2180ce560 msgr2=0x7fd21815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd216800640 1 --2- 172.21.15.143:0/3203663832 >> v2:172.21.15.204:3300/0 conn(0x7fd2180ce560 0x7fd21815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd216800640 1 -- 172.21.15.143:0/3203663832 >> v1:172.21.15.143:6789/0 conn(0x7fd218141660 legacy=0x7fd218141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd216800640 1 -- 172.21.15.143:0/3203663832 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21815f5f0 con 0x7fd218071d00 2024-01-15T18:18:47.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd214a00640 1 -- 172.21.15.143:0/3203663832 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666393189 0 0) 0x7fd208001120 con 0x7fd218141660 2024-01-15T18:18:47.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd216800640 1 --2- 172.21.15.143:0/3203663832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218072110 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd21806f940 tx=0x7fd21001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90579d488ef9f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:47.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd214a00640 1 -- 172.21.15.143:0/3203663832 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100a58c0 con 0x7fd218071d00 2024-01-15T18:18:47.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd214a00640 1 -- 172.21.15.143:0/3203663832 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2100a5a20 con 0x7fd218071d00 2024-01-15T18:18:47.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.327+0000 7fd214a00640 1 -- 172.21.15.143:0/3203663832 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100a5db0 con 0x7fd218071d00 2024-01-15T18:18:47.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- 172.21.15.143:0/3203663832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 msgr2=0x7fd218072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 --2- 172.21.15.143:0/3203663832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218072110 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd21806f940 tx=0x7fd21001c910 comp rx=0 tx=0).stop 2024-01-15T18:18:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- 172.21.15.143:0/3203663832 shutdown_connections 2024-01-15T18:18:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 --2- 172.21.15.143:0/3203663832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218072110 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 --2- 172.21.15.143:0/3203663832 >> v2:172.21.15.204:3300/0 conn(0x7fd2180ce560 0x7fd21815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- 172.21.15.143:0/3203663832 >> 172.21.15.143:0/3203663832 conn(0x7fd21806c3f0 msgr2=0x7fd21806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- 172.21.15.143:0/3203663832 shutdown_connections 2024-01-15T18:18:47.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- 172.21.15.143:0/3203663832 wait complete. 2024-01-15T18:18:47.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 Processor -- start 2024-01-15T18:18:47.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- start start 2024-01-15T18:18:47.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2180ce560 0x7fd21806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd218141660 0x7fd2181394a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181483c0 con 0x7fd218071d00 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21815ada0 con 0x7fd2180ce560 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd21d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd218159a10 con 0x7fd218141660 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47194/0 (socket says 172.21.15.143:47194) 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 -- 172.21.15.143:0/1393405441 learned_addr learned my addr 172.21.15.143:0/1393405441 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215400640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2180ce560 0x7fd21806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd216800640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd218141660 0x7fd2181394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:47.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 -- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd218141660 msgr2=0x7fd2181394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd218141660 0x7fd2181394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 -- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2180ce560 msgr2=0x7fd21806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2180ce560 0x7fd21806d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.331+0000 7fd215e00640 1 -- 172.21.15.143:0/1393405441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210009a70 con 0x7fd218071d00 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd215e00640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218147e80 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c480 tx=0x7fd20800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080074e0 con 0x7fd218071d00 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208007660 con 0x7fd218071d00 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208004a30 con 0x7fd218071d00 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2180cd150 con 0x7fd218071d00 2024-01-15T18:18:47.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd218159fd0 con 0x7fd218071d00 2024-01-15T18:18:47.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd218067500 con 0x7fd218071d00 2024-01-15T18:18:47.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fd208004b90 con 0x7fd218071d00 2024-01-15T18:18:47.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd20e000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f0067a90 0x7fd1f0069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:47.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd208082210 con 0x7fd218071d00 2024-01-15T18:18:47.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.335+0000 7fd215400640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f0067a90 0x7fd1f0069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:47.332 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.339+0000 7fd215400640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f0067a90 0x7fd1f0069f50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd21806e270 tx=0x7fd200008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:47.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.339+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2080546f0 con 0x7fd218071d00 2024-01-15T18:18:47.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.487+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2180ccfd0 con 0x7fd1f0067a90 2024-01-15T18:18:47.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.495+0000 7fd20e000640 1 -- 172.21.15.143:0/1393405441 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288727 (secure 0 0 0) 0x7fd200010960 con 0x7fd1f0067a90 2024-01-15T18:18:47.494 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:47.496 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f0067a90 msgr2=0x7fd1f0069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f0067a90 0x7fd1f0069f50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd21806e270 tx=0x7fd200008040 comp rx=0 tx=0).stop 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 msgr2=0x7fd218147e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218147e80 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd20800c480 tx=0x7fd20800c940 comp rx=0 tx=0).stop 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 shutdown_connections 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd1f0067a90 0x7fd1f0069f50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd218141660 0x7fd2181394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2180ce560 0x7fd21806d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 --2- 172.21.15.143:0/1393405441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd218071d00 0x7fd218147e80 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 >> 172.21.15.143:0/1393405441 conn(0x7fd21806c3f0 msgr2=0x7fd2180afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 shutdown_connections 2024-01-15T18:18:47.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:47.499+0000 7fd21d000640 1 -- 172.21.15.143:0/1393405441 wait complete. 2024-01-15T18:18:47.515 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-01-15T18:18:46.896222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":557928,"kb_used_data":11888,"kb_used_omap":12,"kb_used_meta":545907,"kb_avail":754416792,"statfs":{"total":773094113280,"available":772522795008,"internally_reserved":0,"allocated":12173312,"data_stored":5930873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":559009328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001358"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955237+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:18:41.955237+0000","last_peered":"2024-01-15T18:18:41.955237+0000","last_clean":"2024-01-15T18:18:41.955237+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:18:41.955237+0000","last_undegraded":"2024-01-15T18:18:41.955237+0000","last_fullsized":"2024-01-15T18:18:41.955237+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.936016+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:18:42.936016+0000","last_peered":"2024-01-15T18:18:42.936016+0000","last_clean":"2024-01-15T18:18:42.936016+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:18:42.936016+0000","last_undegraded":"2024-01-15T18:18:42.936016+0000","last_fullsized":"2024-01-15T18:18:42.936016+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995853+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:18:42.995853+0000","last_peered":"2024-01-15T18:18:42.995853+0000","last_clean":"2024-01-15T18:18:42.995853+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:18:42.995853+0000","last_undegraded":"2024-01-15T18:18:42.995853+0000","last_fullsized":"2024-01-15T18:18:42.995853+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:07.807378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.945216+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:18:41.945216+0000","last_peered":"2024-01-15T18:18:41.945216+0000","last_clean":"2024-01-15T18:18:41.945216+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:18:41.945216+0000","last_undegraded":"2024-01-15T18:18:41.945216+0000","last_fullsized":"2024-01-15T18:18:41.945216+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.925271+0000","last_change":"2024-01-15T18:18:40.925271+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.925271+0000","last_undegraded":"2024-01-15T18:18:40.925271+0000","last_fullsized":"2024-01-15T18:18:40.925271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954702+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:18:41.954702+0000","last_peered":"2024-01-15T18:18:41.954702+0000","last_clean":"2024-01-15T18:18:41.954702+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:18:41.954702+0000","last_undegraded":"2024-01-15T18:18:41.954702+0000","last_fullsized":"2024-01-15T18:18:41.954702+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:04:30.848650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.933644+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:18:42.933644+0000","last_peered":"2024-01-15T18:18:42.933644+0000","last_clean":"2024-01-15T18:18:42.933644+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:18:42.933644+0000","last_undegraded":"2024-01-15T18:18:42.933644+0000","last_fullsized":"2024-01-15T18:18:42.933644+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995592+0000","last_change":"2024-01-15T18:18:41.932178+0000","last_active":"2024-01-15T18:18:42.995592+0000","last_peered":"2024-01-15T18:18:42.995592+0000","last_clean":"2024-01-15T18:18:42.995592+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:18:42.995592+0000","last_undegraded":"2024-01-15T18:18:42.995592+0000","last_fullsized":"2024-01-15T18:18:42.995592+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:32.753279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.956014+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:18:41.956014+0000","last_peered":"2024-01-15T18:18:41.956014+0000","last_clean":"2024-01-15T18:18:41.956014+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:18:41.956014+0000","last_undegraded":"2024-01-15T18:18:41.956014+0000","last_fullsized":"2024-01-15T18:18:41.956014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954228+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:18:41.954228+0000","last_peered":"2024-01-15T18:18:41.954228+0000","last_clean":"2024-01-15T18:18:41.954228+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:18:41.954228+0000","last_undegraded":"2024-01-15T18:18:41.954228+0000","last_fullsized":"2024-01-15T18:18:41.954228+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935645+0000","last_change":"2024-01-15T18:18:41.934119+0000","last_active":"2024-01-15T18:18:42.935645+0000","last_peered":"2024-01-15T18:18:42.935645+0000","last_clean":"2024-01-15T18:18:42.935645+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:18:42.935645+0000","last_undegraded":"2024-01-15T18:18:42.935645+0000","last_fullsized":"2024-01-15T18:18:42.935645+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951965+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:18:41.951965+0000","last_peered":"2024-01-15T18:18:41.951965+0000","last_clean":"2024-01-15T18:18:41.951965+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:18:41.951965+0000","last_undegraded":"2024-01-15T18:18:41.951965+0000","last_fullsized":"2024-01-15T18:18:41.951965+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:27.358403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996036+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:18:42.996036+0000","last_peered":"2024-01-15T18:18:42.996036+0000","last_clean":"2024-01-15T18:18:42.996036+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:18:42.996036+0000","last_undegraded":"2024-01-15T18:18:42.996036+0000","last_fullsized":"2024-01-15T18:18:42.996036+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995616+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:18:42.995616+0000","last_peered":"2024-01-15T18:18:42.995616+0000","last_clean":"2024-01-15T18:18:42.995616+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:18:42.995616+0000","last_undegraded":"2024-01-15T18:18:42.995616+0000","last_fullsized":"2024-01-15T18:18:42.995616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:06.614151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955421+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:18:41.955421+0000","last_peered":"2024-01-15T18:18:41.955421+0000","last_clean":"2024-01-15T18:18:41.955421+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:18:41.955421+0000","last_undegraded":"2024-01-15T18:18:41.955421+0000","last_fullsized":"2024-01-15T18:18:41.955421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:31:34.253432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947043+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:18:41.947043+0000","last_peered":"2024-01-15T18:18:41.947043+0000","last_clean":"2024-01-15T18:18:41.947043+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:18:41.947043+0000","last_undegraded":"2024-01-15T18:18:41.947043+0000","last_fullsized":"2024-01-15T18:18:41.947043+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935646+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:18:42.935646+0000","last_peered":"2024-01-15T18:18:42.935646+0000","last_clean":"2024-01-15T18:18:42.935646+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:18:42.935646+0000","last_undegraded":"2024-01-15T18:18:42.935646+0000","last_fullsized":"2024-01-15T18:18:42.935646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934604+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:18:42.934604+0000","last_peered":"2024-01-15T18:18:42.934604+0000","last_clean":"2024-01-15T18:18:42.934604+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:18:42.934604+0000","last_undegraded":"2024-01-15T18:18:42.934604+0000","last_fullsized":"2024-01-15T18:18:42.934604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:45.110826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996037+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:18:42.996037+0000","last_peered":"2024-01-15T18:18:42.996037+0000","last_clean":"2024-01-15T18:18:42.996037+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:18:42.996037+0000","last_undegraded":"2024-01-15T18:18:42.996037+0000","last_fullsized":"2024-01-15T18:18:42.996037+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935404+0000","last_change":"2024-01-15T18:18:41.927062+0000","last_active":"2024-01-15T18:18:42.935404+0000","last_peered":"2024-01-15T18:18:42.935404+0000","last_clean":"2024-01-15T18:18:42.935404+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:18:42.935404+0000","last_undegraded":"2024-01-15T18:18:42.935404+0000","last_fullsized":"2024-01-15T18:18:42.935404+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935805+0000","last_change":"2024-01-15T18:18:41.937906+0000","last_active":"2024-01-15T18:18:42.935805+0000","last_peered":"2024-01-15T18:18:42.935805+0000","last_clean":"2024-01-15T18:18:42.935805+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:18:42.935805+0000","last_undegraded":"2024-01-15T18:18:42.935805+0000","last_fullsized":"2024-01-15T18:18:42.935805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:20.422368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955395+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:18:41.955395+0000","last_peered":"2024-01-15T18:18:41.955395+0000","last_clean":"2024-01-15T18:18:41.955395+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:18:41.955395+0000","last_undegraded":"2024-01-15T18:18:41.955395+0000","last_fullsized":"2024-01-15T18:18:41.955395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934993+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:18:42.934993+0000","last_peered":"2024-01-15T18:18:42.934993+0000","last_clean":"2024-01-15T18:18:42.934993+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:18:42.934993+0000","last_undegraded":"2024-01-15T18:18:42.934993+0000","last_fullsized":"2024-01-15T18:18:42.934993+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955258+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:18:41.955258+0000","last_peered":"2024-01-15T18:18:41.955258+0000","last_clean":"2024-01-15T18:18:41.955258+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:18:41.955258+0000","last_undegraded":"2024-01-15T18:18:41.955258+0000","last_fullsized":"2024-01-15T18:18:41.955258+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935805+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:18:42.935805+0000","last_peered":"2024-01-15T18:18:42.935805+0000","last_clean":"2024-01-15T18:18:42.935805+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:18:42.935805+0000","last_undegraded":"2024-01-15T18:18:42.935805+0000","last_fullsized":"2024-01-15T18:18:42.935805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:12.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955709+0000","last_change":"2024-01-15T18:18:41.949864+0000","last_active":"2024-01-15T18:18:41.955709+0000","last_peered":"2024-01-15T18:18:41.955709+0000","last_clean":"2024-01-15T18:18:41.955709+0000","last_became_active":"2024-01-15T18:18:41.944900+0000","last_became_peered":"2024-01-15T18:18:41.944900+0000","last_unstale":"2024-01-15T18:18:41.955709+0000","last_undegraded":"2024-01-15T18:18:41.955709+0000","last_fullsized":"2024-01-15T18:18:41.955709+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933525+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:18:41.933525+0000","last_peered":"2024-01-15T18:18:41.933525+0000","last_clean":"2024-01-15T18:18:41.933525+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:18:41.933525+0000","last_undegraded":"2024-01-15T18:18:41.933525+0000","last_fullsized":"2024-01-15T18:18:41.933525+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:28.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957639+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:18:41.957639+0000","last_peered":"2024-01-15T18:18:41.957639+0000","last_clean":"2024-01-15T18:18:41.957639+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:18:41.957639+0000","last_undegraded":"2024-01-15T18:18:41.957639+0000","last_fullsized":"2024-01-15T18:18:41.957639+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935282+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:18:42.935282+0000","last_peered":"2024-01-15T18:18:42.935282+0000","last_clean":"2024-01-15T18:18:42.935282+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:18:42.935282+0000","last_undegraded":"2024-01-15T18:18:42.935282+0000","last_fullsized":"2024-01-15T18:18:42.935282+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.933944+0000","last_change":"2024-01-15T18:18:41.937559+0000","last_active":"2024-01-15T18:18:42.933944+0000","last_peered":"2024-01-15T18:18:42.933944+0000","last_clean":"2024-01-15T18:18:42.933944+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:18:42.933944+0000","last_undegraded":"2024-01-15T18:18:42.933944+0000","last_fullsized":"2024-01-15T18:18:42.933944+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955724+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:18:41.955724+0000","last_peered":"2024-01-15T18:18:41.955724+0000","last_clean":"2024-01-15T18:18:41.955724+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:18:41.955724+0000","last_undegraded":"2024-01-15T18:18:41.955724+0000","last_fullsized":"2024-01-15T18:18:41.955724+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955368+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:18:41.955368+0000","last_peered":"2024-01-15T18:18:41.955368+0000","last_clean":"2024-01-15T18:18:41.955368+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:18:41.955368+0000","last_undegraded":"2024-01-15T18:18:41.955368+0000","last_fullsized":"2024-01-15T18:18:41.955368+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954348+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:18:41.954348+0000","last_peered":"2024-01-15T18:18:41.954348+0000","last_clean":"2024-01-15T18:18:41.954348+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:18:41.954348+0000","last_undegraded":"2024-01-15T18:18:41.954348+0000","last_fullsized":"2024-01-15T18:18:41.954348+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:14.769436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.941137+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:18:41.941137+0000","last_peered":"2024-01-15T18:18:41.941137+0000","last_clean":"2024-01-15T18:18:41.941137+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:18:41.941137+0000","last_undegraded":"2024-01-15T18:18:41.941137+0000","last_fullsized":"2024-01-15T18:18:41.941137+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:01:16.930740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.936328+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:18:42.936328+0000","last_peered":"2024-01-15T18:18:42.936328+0000","last_clean":"2024-01-15T18:18:42.936328+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:18:42.936328+0000","last_undegraded":"2024-01-15T18:18:42.936328+0000","last_fullsized":"2024-01-15T18:18:42.936328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.932391+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:18:41.932391+0000","last_peered":"2024-01-15T18:18:41.932391+0000","last_clean":"2024-01-15T18:18:41.932391+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:18:41.932391+0000","last_undegraded":"2024-01-15T18:18:41.932391+0000","last_fullsized":"2024-01-15T18:18:41.932391+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:24.049276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954412+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:18:41.954412+0000","last_peered":"2024-01-15T18:18:41.954412+0000","last_clean":"2024-01-15T18:18:41.954412+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:18:41.954412+0000","last_undegraded":"2024-01-15T18:18:41.954412+0000","last_fullsized":"2024-01-15T18:18:41.954412+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:49.284112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946203+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:18:40.946203+0000","last_peered":"2024-01-15T18:18:40.946203+0000","last_clean":"2024-01-15T18:18:40.946203+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:18:40.946203+0000","last_undegraded":"2024-01-15T18:18:40.946203+0000","last_fullsized":"2024-01-15T18:18:40.946203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:08:09.308839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996080+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:18:42.996080+0000","last_peered":"2024-01-15T18:18:42.996080+0000","last_clean":"2024-01-15T18:18:42.996080+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:18:42.996080+0000","last_undegraded":"2024-01-15T18:18:42.996080+0000","last_fullsized":"2024-01-15T18:18:42.996080+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:02.239162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946121+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:18:40.946121+0000","last_peered":"2024-01-15T18:18:40.946121+0000","last_clean":"2024-01-15T18:18:40.946121+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:18:40.946121+0000","last_undegraded":"2024-01-15T18:18:40.946121+0000","last_fullsized":"2024-01-15T18:18:40.946121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:10.172303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955156+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:18:41.955156+0000","last_peered":"2024-01-15T18:18:41.955156+0000","last_clean":"2024-01-15T18:18:41.955156+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:18:41.955156+0000","last_undegraded":"2024-01-15T18:18:41.955156+0000","last_fullsized":"2024-01-15T18:18:41.955156+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.941822+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:18:41.941822+0000","last_peered":"2024-01-15T18:18:41.941822+0000","last_clean":"2024-01-15T18:18:41.941822+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:18:41.941822+0000","last_undegraded":"2024-01-15T18:18:41.941822+0000","last_fullsized":"2024-01-15T18:18:41.941822+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:23.630535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935414+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:18:42.935414+0000","last_peered":"2024-01-15T18:18:42.935414+0000","last_clean":"2024-01-15T18:18:42.935414+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:18:42.935414+0000","last_undegraded":"2024-01-15T18:18:42.935414+0000","last_fullsized":"2024-01-15T18:18:42.935414+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949383+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:18:41.949383+0000","last_peered":"2024-01-15T18:18:41.949383+0000","last_clean":"2024-01-15T18:18:41.949383+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:18:41.949383+0000","last_undegraded":"2024-01-15T18:18:41.949383+0000","last_fullsized":"2024-01-15T18:18:41.949383+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.954520+0000","last_change":"2024-01-15T18:18:41.940718+0000","last_active":"2024-01-15T18:18:41.954520+0000","last_peered":"2024-01-15T18:18:41.954520+0000","last_clean":"2024-01-15T18:18:41.954520+0000","last_became_active":"2024-01-15T18:18:41.940432+0000","last_became_peered":"2024-01-15T18:18:41.940432+0000","last_unstale":"2024-01-15T18:18:41.954520+0000","last_undegraded":"2024-01-15T18:18:41.954520+0000","last_fullsized":"2024-01-15T18:18:41.954520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.916211+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:18:41.916211+0000","last_peered":"2024-01-15T18:18:41.916211+0000","last_clean":"2024-01-15T18:18:41.916211+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:18:41.916211+0000","last_undegraded":"2024-01-15T18:18:41.916211+0000","last_fullsized":"2024-01-15T18:18:41.916211+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.933449+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:18:42.933449+0000","last_peered":"2024-01-15T18:18:42.933449+0000","last_clean":"2024-01-15T18:18:42.933449+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:18:42.933449+0000","last_undegraded":"2024-01-15T18:18:42.933449+0000","last_fullsized":"2024-01-15T18:18:42.933449+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918022+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:18:41.918022+0000","last_peered":"2024-01-15T18:18:41.918022+0000","last_clean":"2024-01-15T18:18:41.918022+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:18:41.918022+0000","last_undegraded":"2024-01-15T18:18:41.918022+0000","last_fullsized":"2024-01-15T18:18:41.918022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934590+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:18:42.934590+0000","last_peered":"2024-01-15T18:18:42.934590+0000","last_clean":"2024-01-15T18:18:42.934590+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:18:42.934590+0000","last_undegraded":"2024-01-15T18:18:42.934590+0000","last_fullsized":"2024-01-15T18:18:42.934590+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946140+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:18:40.946140+0000","last_peered":"2024-01-15T18:18:40.946140+0000","last_clean":"2024-01-15T18:18:40.946140+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:18:40.946140+0000","last_undegraded":"2024-01-15T18:18:40.946140+0000","last_fullsized":"2024-01-15T18:18:40.946140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.163209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.936464+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:18:42.936464+0000","last_peered":"2024-01-15T18:18:42.936464+0000","last_clean":"2024-01-15T18:18:42.936464+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:18:42.936464+0000","last_undegraded":"2024-01-15T18:18:42.936464+0000","last_fullsized":"2024-01-15T18:18:42.936464+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.926304+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:18:41.926304+0000","last_peered":"2024-01-15T18:18:41.926304+0000","last_clean":"2024-01-15T18:18:41.926304+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:18:41.926304+0000","last_undegraded":"2024-01-15T18:18:41.926304+0000","last_fullsized":"2024-01-15T18:18:41.926304+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955327+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:18:41.955327+0000","last_peered":"2024-01-15T18:18:41.955327+0000","last_clean":"2024-01-15T18:18:41.955327+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:18:41.955327+0000","last_undegraded":"2024-01-15T18:18:41.955327+0000","last_fullsized":"2024-01-15T18:18:41.955327+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957957+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:18:41.957957+0000","last_peered":"2024-01-15T18:18:41.957957+0000","last_clean":"2024-01-15T18:18:41.957957+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:18:41.957957+0000","last_undegraded":"2024-01-15T18:18:41.957957+0000","last_fullsized":"2024-01-15T18:18:41.957957+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934783+0000","last_change":"2024-01-15T18:18:41.931596+0000","last_active":"2024-01-15T18:18:42.934783+0000","last_peered":"2024-01-15T18:18:42.934783+0000","last_clean":"2024-01-15T18:18:42.934783+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:18:42.934783+0000","last_undegraded":"2024-01-15T18:18:42.934783+0000","last_fullsized":"2024-01-15T18:18:42.934783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.960177+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:18:41.960177+0000","last_peered":"2024-01-15T18:18:41.960177+0000","last_clean":"2024-01-15T18:18:41.960177+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:18:41.960177+0000","last_undegraded":"2024-01-15T18:18:41.960177+0000","last_fullsized":"2024-01-15T18:18:41.960177+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955377+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:18:41.955377+0000","last_peered":"2024-01-15T18:18:41.955377+0000","last_clean":"2024-01-15T18:18:41.955377+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:18:41.955377+0000","last_undegraded":"2024-01-15T18:18:41.955377+0000","last_fullsized":"2024-01-15T18:18:41.955377+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.949761+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:18:41.949761+0000","last_peered":"2024-01-15T18:18:41.949761+0000","last_clean":"2024-01-15T18:18:41.949761+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:18:41.949761+0000","last_undegraded":"2024-01-15T18:18:41.949761+0000","last_fullsized":"2024-01-15T18:18:41.949761+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955653+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:18:41.955653+0000","last_peered":"2024-01-15T18:18:41.955653+0000","last_clean":"2024-01-15T18:18:41.955653+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:18:41.955653+0000","last_undegraded":"2024-01-15T18:18:41.955653+0000","last_fullsized":"2024-01-15T18:18:41.955653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.702381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.912091+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:18:41.912091+0000","last_peered":"2024-01-15T18:18:41.912091+0000","last_clean":"2024-01-15T18:18:41.912091+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:18:41.912091+0000","last_undegraded":"2024-01-15T18:18:41.912091+0000","last_fullsized":"2024-01-15T18:18:41.912091+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955766+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:18:41.955766+0000","last_peered":"2024-01-15T18:18:41.955766+0000","last_clean":"2024-01-15T18:18:41.955766+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:18:41.955766+0000","last_undegraded":"2024-01-15T18:18:41.955766+0000","last_fullsized":"2024-01-15T18:18:41.955766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:17.438389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.939931+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:18:40.939931+0000","last_peered":"2024-01-15T18:18:40.939931+0000","last_clean":"2024-01-15T18:18:40.939931+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:18:40.939931+0000","last_undegraded":"2024-01-15T18:18:40.939931+0000","last_fullsized":"2024-01-15T18:18:40.939931+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:30.432388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-01-15T18:18:40.921779+0000","last_change":"2024-01-15T18:18:40.921779+0000","last_active":"2024-01-15T18:18:40.902736+0000","last_peered":"2024-01-15T18:18:40.902736+0000","last_clean":"2024-01-15T18:18:40.902736+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:18:40.921779+0000","last_undegraded":"2024-01-15T18:18:40.921779+0000","last_fullsized":"2024-01-15T18:18:40.921779+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.946443+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:18:41.946443+0000","last_peered":"2024-01-15T18:18:41.946443+0000","last_clean":"2024-01-15T18:18:41.946443+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:18:41.946443+0000","last_undegraded":"2024-01-15T18:18:41.946443+0000","last_fullsized":"2024-01-15T18:18:41.946443+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.933399+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:18:41.933399+0000","last_peered":"2024-01-15T18:18:41.933399+0000","last_clean":"2024-01-15T18:18:41.933399+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:18:41.933399+0000","last_undegraded":"2024-01-15T18:18:41.933399+0000","last_fullsized":"2024-01-15T18:18:41.933399+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.408977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.931086+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:18:40.931086+0000","last_peered":"2024-01-15T18:18:40.931086+0000","last_clean":"2024-01-15T18:18:40.931086+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:18:40.931086+0000","last_undegraded":"2024-01-15T18:18:40.931086+0000","last_fullsized":"2024-01-15T18:18:40.931086+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:51:44.688957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995970+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:18:42.995970+0000","last_peered":"2024-01-15T18:18:42.995970+0000","last_clean":"2024-01-15T18:18:42.995970+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:18:42.995970+0000","last_undegraded":"2024-01-15T18:18:42.995970+0000","last_fullsized":"2024-01-15T18:18:42.995970+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.948648+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:18:41.948648+0000","last_peered":"2024-01-15T18:18:41.948648+0000","last_clean":"2024-01-15T18:18:41.948648+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:18:41.948648+0000","last_undegraded":"2024-01-15T18:18:41.948648+0000","last_fullsized":"2024-01-15T18:18:41.948648+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:00:59.694593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996138+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:18:42.996138+0000","last_peered":"2024-01-15T18:18:42.996138+0000","last_clean":"2024-01-15T18:18:42.996138+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:18:42.996138+0000","last_undegraded":"2024-01-15T18:18:42.996138+0000","last_fullsized":"2024-01-15T18:18:42.996138+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:26.983917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.922691+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:18:41.922691+0000","last_peered":"2024-01-15T18:18:41.922691+0000","last_clean":"2024-01-15T18:18:41.922691+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:18:41.922691+0000","last_undegraded":"2024-01-15T18:18:41.922691+0000","last_fullsized":"2024-01-15T18:18:41.922691+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:42.895411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995605+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:18:42.995605+0000","last_peered":"2024-01-15T18:18:42.995605+0000","last_clean":"2024-01-15T18:18:42.995605+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:18:42.995605+0000","last_undegraded":"2024-01-15T18:18:42.995605+0000","last_fullsized":"2024-01-15T18:18:42.995605+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958826+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:18:41.958826+0000","last_peered":"2024-01-15T18:18:41.958826+0000","last_clean":"2024-01-15T18:18:41.958826+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:18:41.958826+0000","last_undegraded":"2024-01-15T18:18:41.958826+0000","last_fullsized":"2024-01-15T18:18:41.958826+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.940187+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:18:41.940187+0000","last_peered":"2024-01-15T18:18:41.940187+0000","last_clean":"2024-01-15T18:18:41.940187+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:18:41.940187+0000","last_undegraded":"2024-01-15T18:18:41.940187+0000","last_fullsized":"2024-01-15T18:18:41.940187+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.947226+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:18:40.947226+0000","last_peered":"2024-01-15T18:18:40.947226+0000","last_clean":"2024-01-15T18:18:40.947226+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:18:40.947226+0000","last_undegraded":"2024-01-15T18:18:40.947226+0000","last_fullsized":"2024-01-15T18:18:40.947226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:36:53.075302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995617+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:18:42.995617+0000","last_peered":"2024-01-15T18:18:42.995617+0000","last_clean":"2024-01-15T18:18:42.995617+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:18:42.995617+0000","last_undegraded":"2024-01-15T18:18:42.995617+0000","last_fullsized":"2024-01-15T18:18:42.995617+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.924617+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:18:41.924617+0000","last_peered":"2024-01-15T18:18:41.924617+0000","last_clean":"2024-01-15T18:18:41.924617+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:18:41.924617+0000","last_undegraded":"2024-01-15T18:18:41.924617+0000","last_fullsized":"2024-01-15T18:18:41.924617+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.947927+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:18:41.947927+0000","last_peered":"2024-01-15T18:18:41.947927+0000","last_clean":"2024-01-15T18:18:41.947927+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:18:41.947927+0000","last_undegraded":"2024-01-15T18:18:41.947927+0000","last_fullsized":"2024-01-15T18:18:41.947927+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:43.903199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918200+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:18:41.918200+0000","last_peered":"2024-01-15T18:18:41.918200+0000","last_clean":"2024-01-15T18:18:41.918200+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:18:41.918200+0000","last_undegraded":"2024-01-15T18:18:41.918200+0000","last_fullsized":"2024-01-15T18:18:41.918200+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935854+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:18:42.935854+0000","last_peered":"2024-01-15T18:18:42.935854+0000","last_clean":"2024-01-15T18:18:42.935854+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:18:42.935854+0000","last_undegraded":"2024-01-15T18:18:42.935854+0000","last_fullsized":"2024-01-15T18:18:42.935854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957662+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:18:41.957662+0000","last_peered":"2024-01-15T18:18:41.957662+0000","last_clean":"2024-01-15T18:18:41.957662+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:18:41.957662+0000","last_undegraded":"2024-01-15T18:18:41.957662+0000","last_fullsized":"2024-01-15T18:18:41.957662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.944610+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:18:41.944610+0000","last_peered":"2024-01-15T18:18:41.944610+0000","last_clean":"2024-01-15T18:18:41.944610+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:18:41.944610+0000","last_undegraded":"2024-01-15T18:18:41.944610+0000","last_fullsized":"2024-01-15T18:18:41.944610+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.912207+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:18:41.912207+0000","last_peered":"2024-01-15T18:18:41.912207+0000","last_clean":"2024-01-15T18:18:41.912207+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:18:41.912207+0000","last_undegraded":"2024-01-15T18:18:41.912207+0000","last_fullsized":"2024-01-15T18:18:41.912207+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:51.313218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.951944+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:18:41.951944+0000","last_peered":"2024-01-15T18:18:41.951944+0000","last_clean":"2024-01-15T18:18:41.951944+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:18:41.951944+0000","last_undegraded":"2024-01-15T18:18:41.951944+0000","last_fullsized":"2024-01-15T18:18:41.951944+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918797+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:18:41.918797+0000","last_peered":"2024-01-15T18:18:41.918797+0000","last_clean":"2024-01-15T18:18:41.918797+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:18:41.918797+0000","last_undegraded":"2024-01-15T18:18:41.918797+0000","last_fullsized":"2024-01-15T18:18:41.918797+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934711+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:18:42.934711+0000","last_peered":"2024-01-15T18:18:42.934711+0000","last_clean":"2024-01-15T18:18:42.934711+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:18:42.934711+0000","last_undegraded":"2024-01-15T18:18:42.934711+0000","last_fullsized":"2024-01-15T18:18:42.934711+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:27.713450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.927483+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:18:41.927483+0000","last_peered":"2024-01-15T18:18:41.927483+0000","last_clean":"2024-01-15T18:18:41.927483+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:18:41.927483+0000","last_undegraded":"2024-01-15T18:18:41.927483+0000","last_fullsized":"2024-01-15T18:18:41.927483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935125+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:18:42.935125+0000","last_peered":"2024-01-15T18:18:42.935125+0000","last_clean":"2024-01-15T18:18:42.935125+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:18:42.935125+0000","last_undegraded":"2024-01-15T18:18:42.935125+0000","last_fullsized":"2024-01-15T18:18:42.935125+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.922187+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:18:41.922187+0000","last_peered":"2024-01-15T18:18:41.922187+0000","last_clean":"2024-01-15T18:18:41.922187+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:18:41.922187+0000","last_undegraded":"2024-01-15T18:18:41.922187+0000","last_fullsized":"2024-01-15T18:18:41.922187+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:54.586280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935563+0000","last_change":"2024-01-15T18:18:41.935993+0000","last_active":"2024-01-15T18:18:42.935563+0000","last_peered":"2024-01-15T18:18:42.935563+0000","last_clean":"2024-01-15T18:18:42.935563+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:18:42.935563+0000","last_undegraded":"2024-01-15T18:18:42.935563+0000","last_fullsized":"2024-01-15T18:18:42.935563+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.916297+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:18:41.916297+0000","last_peered":"2024-01-15T18:18:41.916297+0000","last_clean":"2024-01-15T18:18:41.916297+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:18:41.916297+0000","last_undegraded":"2024-01-15T18:18:41.916297+0000","last_fullsized":"2024-01-15T18:18:41.916297+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:26.705662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995855+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:18:42.995855+0000","last_peered":"2024-01-15T18:18:42.995855+0000","last_clean":"2024-01-15T18:18:42.995855+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:18:42.995855+0000","last_undegraded":"2024-01-15T18:18:42.995855+0000","last_fullsized":"2024-01-15T18:18:42.995855+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-01-15T18:18:40.946157+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:18:40.946157+0000","last_peered":"2024-01-15T18:18:40.946157+0000","last_clean":"2024-01-15T18:18:40.946157+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:18:40.946157+0000","last_undegraded":"2024-01-15T18:18:40.946157+0000","last_fullsized":"2024-01-15T18:18:40.946157+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:50.249980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.955068+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:18:41.955068+0000","last_peered":"2024-01-15T18:18:41.955068+0000","last_clean":"2024-01-15T18:18:41.955068+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:18:41.955068+0000","last_undegraded":"2024-01-15T18:18:41.955068+0000","last_fullsized":"2024-01-15T18:18:41.955068+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.918226+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:18:41.918226+0000","last_peered":"2024-01-15T18:18:41.918226+0000","last_clean":"2024-01-15T18:18:41.918226+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:18:41.918226+0000","last_undegraded":"2024-01-15T18:18:41.918226+0000","last_fullsized":"2024-01-15T18:18:41.918226+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.953440+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:18:41.953440+0000","last_peered":"2024-01-15T18:18:41.953440+0000","last_clean":"2024-01-15T18:18:41.953440+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:18:41.953440+0000","last_undegraded":"2024-01-15T18:18:41.953440+0000","last_fullsized":"2024-01-15T18:18:41.953440+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:52.321006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.922045+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:18:41.922045+0000","last_peered":"2024-01-15T18:18:41.922045+0000","last_clean":"2024-01-15T18:18:41.922045+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:18:41.922045+0000","last_undegraded":"2024-01-15T18:18:41.922045+0000","last_fullsized":"2024-01-15T18:18:41.922045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:35.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":241,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.921952+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:18:41.921952+0000","last_peered":"2024-01-15T18:18:41.921952+0000","last_clean":"2024-01-15T18:18:41.921952+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:18:41.921952+0000","last_undegraded":"2024-01-15T18:18:41.921952+0000","last_fullsized":"2024-01-15T18:18:41.921952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24,"ondisk_log_size":24,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64988,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306852,"statfs":{"total":96636764160,"available":96570216448,"internally_reserved":0,"allocated":1982464,"data_stored":1190314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.86399999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.77400000000000002,"15min":0.85999999999999999},"min":{"1min":0.48299999999999998,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.92500000000000004,"5min":4.6100000000000003,"15min":13.744},"last":0.84799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.96799999999999997,"15min":1.0640000000000001},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":3.016,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.81100000000000005,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":4.7679999999999998,"15min":31.774999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.74099999999999999,"15min":0.98699999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":2.1680000000000001,"15min":30.846},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":1.073},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":17.039999999999999,"15min":39.695},"last":0.72999999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.82599999999999996,"15min":1.1579999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.88400000000000001,"5min":8.2650000000000006,"15min":26.463999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.94699999999999995,"15min":1.202},"min":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":16.879999999999999,"15min":42.692},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78800000000000003,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":1.147,"5min":14.087,"15min":14.087},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.629,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.52800000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.81299999999999994,"15min":0.71199999999999997},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":1.0209999999999999,"5min":11.334,"15min":11.334},"last":0.628},{"interface":"front","average":{"1min":0.627,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":4.8179999999999996,"15min":8.423},"last":0.746}]},{"osd":6,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5369999999999999,"5min":4.7140000000000004,"15min":8.5030000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1970000000000001,"15min":3.3450000000000002},"last":0.77100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64412,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307428,"statfs":{"total":96636764160,"available":96570806272,"internally_reserved":0,"allocated":1384448,"data_stored":599400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.81699999999999995,"15min":0.95299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74199999999999999,"15min":0.873},"min":{"1min":0.375,"5min":0.375,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":3.4940000000000002,"15min":12.449},"last":0.752}]},{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.85599999999999998,"15min":0.89700000000000002},"min":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.33900000000000002},"max":{"1min":1.0489999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.84199999999999997,"15min":1.004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.81899999999999995,"5min":8.9320000000000004,"15min":19.475000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.069,"15min":1.179},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":14.743,"15min":42.825000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.89000000000000001,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.86699999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78300000000000003,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":0.82099999999999995,"5min":3.016,"15min":24.199000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0629999999999999,"15min":1.087},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":2.956,"5min":18.791,"15min":18.791},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.89300000000000002,"15min":0.745},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33700000000000002},"max":{"1min":0.875,"5min":1.7450000000000001,"15min":12.491},"last":0.96599999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81899999999999995,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.89800000000000002},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.5259999999999998,"15min":12.945},"last":0.93200000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":1.006,"15min":0.83899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":11.907,"15min":11.907},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82499999999999996,"15min":0.88400000000000001},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":9.9440000000000008,"15min":27.651},"last":0.89500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76888,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294952,"statfs":{"total":96636764160,"available":96558030848,"internally_reserved":0,"allocated":1388544,"data_stored":599934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":4.4770000000000003,"15min":11.519},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.874,"15min":0.92400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.91700000000000004,"5min":7.6260000000000003,"15min":12.952},"last":0.68100000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.92700000000000005,"15min":0.90300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.95499999999999996,"5min":4.3140000000000001,"15min":13.358000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.94699999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0169999999999999,"5min":4.1580000000000004,"15min":21.856999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":1,"5min":6.7160000000000002,"15min":18.314},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.145,"15min":1.0109999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.0580000000000001,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.85499999999999998,"15min":0.94499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":3.2400000000000002,"15min":13.317},"last":0.747},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":1.01},"min":{"1min":0.437,"5min":0.437,"15min":0.42899999999999999},"max":{"1min":0.998,"5min":3.931,"15min":19.638000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":0.96099999999999997,"5min":5.899,"15min":11.093},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":0.88700000000000001,"5min":2.516,"15min":3.3889999999999998},"last":0.56899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84399999999999997,"15min":0.79500000000000004},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97899999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.308},"max":{"1min":1.0840000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.76300000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":1.071,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":1.034,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.89800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81040,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290800,"statfs":{"total":96636764160,"available":96553779200,"internally_reserved":0,"allocated":1380352,"data_stored":599330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.79800000000000004,"15min":0.93300000000000005},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":9.4179999999999993,"15min":34.802},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":14.488,"15min":22.335999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.95699999999999996},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":2.2130000000000001,"15min":38.198},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.80500000000000005,"15min":0.95599999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.86599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95099999999999996,"5min":1.6259999999999999,"15min":7.6989999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.78500000000000003,"15min":0.84799999999999998},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":4.4089999999999998,"15min":7.4829999999999997},"last":0.73199999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.91300000000000003,"15min":0.95399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":14.529,"15min":14.529},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72299999999999998,"15min":0.92900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":1.4279999999999999,"15min":25.774000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":0.68200000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.502},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.97799999999999998,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.47499999999999998,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":25.887,"15min":25.887},"last":0.57899999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":10.284000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.82799999999999996,"15min":0.78300000000000003},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.77100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"max":{"1min":1.03,"5min":1.232,"15min":11.67},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575018,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64376,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307464,"statfs":{"total":96636764160,"available":96570843136,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.625,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.255},"max":{"1min":0.81899999999999995,"5min":7.2839999999999998,"15min":26.364000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.66300000000000003,"15min":0.95599999999999996},"min":{"1min":0.312,"5min":0.25,"15min":0.246},"max":{"1min":0.71499999999999997,"5min":7.8520000000000003,"15min":28.332000000000001},"last":0.47299999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.60799999999999998,"15min":0.98799999999999999},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.84799999999999998,"5min":3.8100000000000001,"15min":40.457999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.76300000000000001,"15min":1.1419999999999999},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":9.282,"15min":27.427},"last":0.45500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.71199999999999997,"15min":0.98899999999999999},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.90400000000000003,"5min":9.1180000000000003,"15min":23.036999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.81499999999999995,"15min":1.089},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":1.048,"5min":9.3510000000000009,"15min":26.404},"last":0.495}]},{"osd":4,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":1.024},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.41199999999999998},"max":{"1min":1.0780000000000001,"5min":4.1040000000000001,"15min":22.998000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84599999999999997,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.0229999999999999,"5min":3.8540000000000001,"15min":20.399000000000001},"last":0.82699999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":1.097},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":6.5140000000000002,"15min":26.314},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85599999999999998,"15min":1.1220000000000001},"min":{"1min":0.46100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":4.165,"15min":22.908999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":1.048,"15min":1.153},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.42799999999999999},"max":{"1min":1.0609999999999999,"5min":6.843,"15min":20.445},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.1850000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.123,"5min":6.7640000000000002,"15min":26.091999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.98999999999999999,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":7.0250000000000004,"15min":23.111999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.92000000000000004,"15min":1.321},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46000000000000002},"max":{"1min":1.1419999999999999,"5min":4.4470000000000001,"15min":44.817999999999998},"last":0.71299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64380,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307460,"statfs":{"total":96636764160,"available":96570839040,"internally_reserved":0,"allocated":1355776,"data_stored":586143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54400000000000004,"15min":1.0529999999999999},"min":{"1min":0.31,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.373},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.86199999999999999,"15min":1.321},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.66400000000000003,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.39200000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.64700000000000002,"15min":1.5409999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.56299999999999994,"5min":7.476,"15min":113.149},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.96899999999999997,"15min":1.1080000000000001},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":20.619,"15min":41.302},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.341,"15min":1.379},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28899999999999998},"max":{"1min":0.67400000000000004,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66000000000000003,"15min":1.27},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":0.77300000000000002,"5min":7.3609999999999998,"15min":115.693},"last":0.436}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":1.206},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":7.4169999999999998,"15min":24.366},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0740000000000001,"15min":1.23},"min":{"1min":0.51600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.45600000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.5800000000000001,"15min":1.5089999999999999},"min":{"1min":0.54700000000000004,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84999999999999998,"15min":1.304},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":7.5099999999999998,"15min":26.084},"last":0.54900000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":1.0980000000000001,"15min":1.3149999999999999},"min":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":20.120999999999999,"15min":22.199000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.748,"5min":1.6240000000000001,"15min":1.5700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.57199999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":1.165,"15min":1.262},"min":{"1min":0.55200000000000005,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":0.92600000000000005,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":1.613,"15min":1.6299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64404,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307436,"statfs":{"total":96636764160,"available":96570814464,"internally_reserved":0,"allocated":1376256,"data_stored":593098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.64200000000000002,"15min":0.91500000000000004},"min":{"1min":0.245,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.68200000000000005,"5min":9.452,"15min":39.948},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.68799999999999994,"15min":1.103},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":8.141,"15min":35.790999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.71799999999999997,"15min":0.98199999999999998},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":0.63800000000000001,"5min":15.137,"15min":49.229999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61799999999999999,"5min":9.8469999999999995,"15min":10.349},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61099999999999999,"15min":1.173},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":4.4669999999999996,"15min":27.643999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.90000000000000002,"15min":0.997},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.66800000000000004,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84299999999999997,"15min":1.222},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":4.633,"15min":27.489999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67600000000000005,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98599999999999999,"5min":9.6989999999999998,"15min":12.997},"last":0.83599999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.86599999999999999,"15min":1.212},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":4.673,"15min":27.547000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.0289999999999999,"15min":1.1200000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.496},"max":{"1min":1.028,"5min":9.7650000000000006,"15min":13.054},"last":0.78100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.86499999999999999,"15min":1.204},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":4.7060000000000004,"15min":27.579999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.87,"15min":1.0369999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.1319999999999999},"min":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":6.6950000000000003,"5min":9.5660000000000007,"15min":11.833},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1.0229999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.502},"max":{"1min":0.93200000000000005,"5min":9.8079999999999998,"15min":14.606},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575018,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1949696,"data_stored":1176511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.78600000000000003,"15min":1.1379999999999999},"min":{"1min":0.27100000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.94299999999999995,"5min":11.356,"15min":31.977},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.81299999999999994,"15min":1.0229999999999999},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0289999999999999,"5min":11.629,"15min":36.145000000000003},"last":0.58599999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.76600000000000001,"15min":1.1319999999999999},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":0.91800000000000004,"5min":5.8929999999999998,"15min":41.387999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.877,"15min":1.038},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.25},"max":{"1min":0.97499999999999998,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.97599999999999998,"15min":1.0860000000000001},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.093,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.79400000000000004,"15min":1.097},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":6.3739999999999997,"15min":45.128},"last":0.51500000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.0289999999999999,"15min":1.0740000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.372},"max":{"1min":2.1469999999999998,"5min":11.772,"15min":32.932000000000002},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.0840000000000001,"15min":1.1919999999999999},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":1.4370000000000001,"5min":11.220000000000001,"15min":31.782},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1.083,"15min":1.1850000000000001},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3979999999999999,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.841,"5min":11.958,"15min":33.151000000000003},"last":0.76500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":1.0640000000000001},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.379,"5min":4.0970000000000004,"15min":13.055},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.749,"5min":1.0449999999999999,"15min":1.0920000000000001},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.522,"5min":11.833,"15min":33.098999999999997},"last":0.79500000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.92300000000000004,"15min":1.0740000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3160000000000001,"5min":4.1349999999999998,"15min":13.090999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.1120000000000001,"15min":1.242},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.464,"5min":11.321999999999999,"15min":31.905999999999999},"last":0.69599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:50.521 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:50.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.663+0000 7f5091000640 1 Processor -- start 2024-01-15T18:18:50.659 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.663+0000 7f5091000640 1 -- start start 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5091000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f508c070d60 0x7f508c0711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5091000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c0acbc0 0x7f508c0acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5091000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0b3370 con 0x7f508c15d1f0 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5091000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c071730 con 0x7f508c070d60 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5091000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15d5d0 con 0x7f508c0acbc0 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f508c070d60 0x7f508c0711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f508c070d60 0x7f508c0711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47204/0 (socket says 172.21.15.143:47204) 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 -- 172.21.15.143:0/3503754596 learned_addr learned my addr 172.21.15.143:0/3503754596 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f508a800640 1 --2- 172.21.15.143:0/3503754596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c0acbc0 0x7f508c0acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 -- 172.21.15.143:0/3503754596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c0acbc0 msgr2=0x7f508c0acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 --2- 172.21.15.143:0/3503754596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c0acbc0 0x7f508c0acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 -- 172.21.15.143:0/3503754596 >> v1:172.21.15.204:6789/0 conn(0x7f508c15d1f0 legacy=0x7f508c06b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:50.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.667+0000 7f5089400640 1 -- 172.21.15.143:0/3503754596 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c06a800 con 0x7f508c070d60 2024-01-15T18:18:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5089400640 1 --2- 172.21.15.143:0/3503754596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c0711f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5074009870 tx=0x7f507401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e02b9f9d1f5732c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5088a00640 1 -- 172.21.15.143:0/3503754596 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50740aa670 con 0x7f508c070d60 2024-01-15T18:18:50.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5088a00640 1 -- 172.21.15.143:0/3503754596 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50740aa7d0 con 0x7f508c070d60 2024-01-15T18:18:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5088a00640 1 -- 172.21.15.143:0/3503754596 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50740aab60 con 0x7f508c070d60 2024-01-15T18:18:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 -- 172.21.15.143:0/3503754596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 msgr2=0x7f508c0711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 --2- 172.21.15.143:0/3503754596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c0711f0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5074009870 tx=0x7f507401c510 comp rx=0 tx=0).stop 2024-01-15T18:18:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 -- 172.21.15.143:0/3503754596 shutdown_connections 2024-01-15T18:18:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 --2- 172.21.15.143:0/3503754596 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c0acbc0 0x7f508c0acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 --2- 172.21.15.143:0/3503754596 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c0711f0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 -- 172.21.15.143:0/3503754596 >> 172.21.15.143:0/3503754596 conn(0x7f508c0b1610 msgr2=0x7f508c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 -- 172.21.15.143:0/3503754596 shutdown_connections 2024-01-15T18:18:50.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 -- 172.21.15.143:0/3503754596 wait complete. 2024-01-15T18:18:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 Processor -- start 2024-01-15T18:18:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.671+0000 7f5091000640 1 -- start start 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508c0acbc0 0x7f508c1602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c15d1f0 0x7f508c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15ddb0 con 0x7f508c070d60 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15e090 con 0x7f508c0acbc0 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c15e370 con 0x7f508c15d1f0 2024-01-15T18:18:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47220/0 (socket says 172.21.15.143:47220) 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 -- 172.21.15.143:0/2964573667 learned_addr learned my addr 172.21.15.143:0/2964573667 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089400640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508c0acbc0 0x7f508c1602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f508a800640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c15d1f0 0x7f508c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 -- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c15d1f0 msgr2=0x7f508c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c15d1f0 0x7f508c06cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 -- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508c0acbc0 msgr2=0x7f508c1602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508c0acbc0 0x7f508c1602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 -- 172.21.15.143:0/2964573667 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740094d0 con 0x7f508c070d60 2024-01-15T18:18:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5089e00640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c15d7d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c3c0 tx=0x7f507c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c0052d0 con 0x7f508c070d60 2024-01-15T18:18:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c005430 con 0x7f508c070d60 2024-01-15T18:18:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c1599c0 con 0x7f508c070d60 2024-01-15T18:18:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c003840 con 0x7f508c070d60 2024-01-15T18:18:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c159040 con 0x7f508c070d60 2024-01-15T18:18:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c074b00 con 0x7f508c070d60 2024-01-15T18:18:50.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.675+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f507c0039a0 con 0x7f508c070d60 2024-01-15T18:18:50.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.679+0000 7f5082000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5064067a90 0x7f5064069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:50.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.679+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f507c0827c0 con 0x7f508c070d60 2024-01-15T18:18:50.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.679+0000 7f5089400640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5064067a90 0x7f5064069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:50.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.679+0000 7f5089400640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5064067a90 0x7f5064069f50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f50740ae000 tx=0x7f50740b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:50.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.683+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f507c0511f0 con 0x7f508c070d60 2024-01-15T18:18:50.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.827+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f508c15e200 con 0x7f5064067a90 2024-01-15T18:18:50.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.835+0000 7f5082000640 1 -- 172.21.15.143:0/2964573667 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288929 (secure 0 0 0) 0x7f508c15e200 con 0x7f5064067a90 2024-01-15T18:18:50.834 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5064067a90 msgr2=0x7f5064069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5064067a90 0x7f5064069f50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f50740ae000 tx=0x7f50740b5040 comp rx=0 tx=0).stop 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 msgr2=0x7f508c15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c15d7d0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c3c0 tx=0x7f507c00c880 comp rx=0 tx=0).stop 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 shutdown_connections 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5064067a90 0x7f5064069f50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f508c15d1f0 0x7f508c06cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f508c0acbc0 0x7f508c1602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 --2- 172.21.15.143:0/2964573667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f508c070d60 0x7f508c15d7d0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 >> 172.21.15.143:0/2964573667 conn(0x7f508c0b1610 msgr2=0x7f508c144760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 shutdown_connections 2024-01-15T18:18:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:50.839+0000 7f5091000640 1 -- 172.21.15.143:0/2964573667 wait complete. 2024-01-15T18:18:50.855 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-01-15T18:18:48.896944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":558044,"kb_used_data":12008,"kb_used_omap":12,"kb_used_meta":545907,"kb_avail":754416676,"statfs":{"total":773094113280,"available":772522676224,"internally_reserved":0,"allocated":12296192,"data_stored":5980196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":559009328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.985217"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005355+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:18:43.005355+0000","last_peered":"2024-01-15T18:18:43.005355+0000","last_clean":"2024-01-15T18:18:43.005355+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:18:43.005355+0000","last_undegraded":"2024-01-15T18:18:43.005355+0000","last_fullsized":"2024-01-15T18:18:43.005355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:00.418536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.936016+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:18:42.936016+0000","last_peered":"2024-01-15T18:18:42.936016+0000","last_clean":"2024-01-15T18:18:42.936016+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:18:42.936016+0000","last_undegraded":"2024-01-15T18:18:42.936016+0000","last_fullsized":"2024-01-15T18:18:42.936016+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995853+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:18:42.995853+0000","last_peered":"2024-01-15T18:18:42.995853+0000","last_clean":"2024-01-15T18:18:42.995853+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:18:42.995853+0000","last_undegraded":"2024-01-15T18:18:42.995853+0000","last_fullsized":"2024-01-15T18:18:42.995853+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:07.807378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004959+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:18:43.004959+0000","last_peered":"2024-01-15T18:18:43.004959+0000","last_clean":"2024-01-15T18:18:43.004959+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:18:43.004959+0000","last_undegraded":"2024-01-15T18:18:43.004959+0000","last_fullsized":"2024-01-15T18:18:43.004959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:44.886136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003133+0000","last_change":"2024-01-15T18:18:41.951758+0000","last_active":"2024-01-15T18:18:43.003133+0000","last_peered":"2024-01-15T18:18:43.003133+0000","last_clean":"2024-01-15T18:18:43.003133+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:18:43.003133+0000","last_undegraded":"2024-01-15T18:18:43.003133+0000","last_fullsized":"2024-01-15T18:18:43.003133+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:51.159719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003768+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:18:43.003768+0000","last_peered":"2024-01-15T18:18:43.003768+0000","last_clean":"2024-01-15T18:18:43.003768+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:18:43.003768+0000","last_undegraded":"2024-01-15T18:18:43.003768+0000","last_fullsized":"2024-01-15T18:18:43.003768+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:48.760709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.933644+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:18:42.933644+0000","last_peered":"2024-01-15T18:18:42.933644+0000","last_clean":"2024-01-15T18:18:42.933644+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:18:42.933644+0000","last_undegraded":"2024-01-15T18:18:42.933644+0000","last_fullsized":"2024-01-15T18:18:42.933644+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995592+0000","last_change":"2024-01-15T18:18:41.932178+0000","last_active":"2024-01-15T18:18:42.995592+0000","last_peered":"2024-01-15T18:18:42.995592+0000","last_clean":"2024-01-15T18:18:42.995592+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:18:42.995592+0000","last_undegraded":"2024-01-15T18:18:42.995592+0000","last_fullsized":"2024-01-15T18:18:42.995592+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:32.753279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005397+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:18:43.005397+0000","last_peered":"2024-01-15T18:18:43.005397+0000","last_clean":"2024-01-15T18:18:43.005397+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:18:43.005397+0000","last_undegraded":"2024-01-15T18:18:43.005397+0000","last_fullsized":"2024-01-15T18:18:43.005397+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:16.576477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005034+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:18:43.005034+0000","last_peered":"2024-01-15T18:18:43.005034+0000","last_clean":"2024-01-15T18:18:43.005034+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:18:43.005034+0000","last_undegraded":"2024-01-15T18:18:43.005034+0000","last_fullsized":"2024-01-15T18:18:43.005034+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:16.172780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935645+0000","last_change":"2024-01-15T18:18:41.934119+0000","last_active":"2024-01-15T18:18:42.935645+0000","last_peered":"2024-01-15T18:18:42.935645+0000","last_clean":"2024-01-15T18:18:42.935645+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:18:42.935645+0000","last_undegraded":"2024-01-15T18:18:42.935645+0000","last_fullsized":"2024-01-15T18:18:42.935645+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005073+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:18:43.005073+0000","last_peered":"2024-01-15T18:18:43.005073+0000","last_clean":"2024-01-15T18:18:43.005073+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:18:43.005073+0000","last_undegraded":"2024-01-15T18:18:43.005073+0000","last_fullsized":"2024-01-15T18:18:43.005073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:16.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996036+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:18:42.996036+0000","last_peered":"2024-01-15T18:18:42.996036+0000","last_clean":"2024-01-15T18:18:42.996036+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:18:42.996036+0000","last_undegraded":"2024-01-15T18:18:42.996036+0000","last_fullsized":"2024-01-15T18:18:42.996036+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995616+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:18:42.995616+0000","last_peered":"2024-01-15T18:18:42.995616+0000","last_clean":"2024-01-15T18:18:42.995616+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:18:42.995616+0000","last_undegraded":"2024-01-15T18:18:42.995616+0000","last_fullsized":"2024-01-15T18:18:42.995616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:06.614151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003775+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:18:43.003775+0000","last_peered":"2024-01-15T18:18:43.003775+0000","last_clean":"2024-01-15T18:18:43.003775+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:18:43.003775+0000","last_undegraded":"2024-01-15T18:18:43.003775+0000","last_fullsized":"2024-01-15T18:18:43.003775+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:34.739612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.002796+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:18:43.002796+0000","last_peered":"2024-01-15T18:18:43.002796+0000","last_clean":"2024-01-15T18:18:43.002796+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:18:43.002796+0000","last_undegraded":"2024-01-15T18:18:43.002796+0000","last_fullsized":"2024-01-15T18:18:43.002796+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:47:03.406736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935646+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:18:42.935646+0000","last_peered":"2024-01-15T18:18:42.935646+0000","last_clean":"2024-01-15T18:18:42.935646+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:18:42.935646+0000","last_undegraded":"2024-01-15T18:18:42.935646+0000","last_fullsized":"2024-01-15T18:18:42.935646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934604+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:18:42.934604+0000","last_peered":"2024-01-15T18:18:42.934604+0000","last_clean":"2024-01-15T18:18:42.934604+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:18:42.934604+0000","last_undegraded":"2024-01-15T18:18:42.934604+0000","last_fullsized":"2024-01-15T18:18:42.934604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:45.110826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996037+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:18:42.996037+0000","last_peered":"2024-01-15T18:18:42.996037+0000","last_clean":"2024-01-15T18:18:42.996037+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:18:42.996037+0000","last_undegraded":"2024-01-15T18:18:42.996037+0000","last_fullsized":"2024-01-15T18:18:42.996037+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935404+0000","last_change":"2024-01-15T18:18:41.927062+0000","last_active":"2024-01-15T18:18:42.935404+0000","last_peered":"2024-01-15T18:18:42.935404+0000","last_clean":"2024-01-15T18:18:42.935404+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:18:42.935404+0000","last_undegraded":"2024-01-15T18:18:42.935404+0000","last_fullsized":"2024-01-15T18:18:42.935404+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935805+0000","last_change":"2024-01-15T18:18:41.937906+0000","last_active":"2024-01-15T18:18:42.935805+0000","last_peered":"2024-01-15T18:18:42.935805+0000","last_clean":"2024-01-15T18:18:42.935805+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:18:42.935805+0000","last_undegraded":"2024-01-15T18:18:42.935805+0000","last_fullsized":"2024-01-15T18:18:42.935805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:20.422368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003692+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:18:43.003692+0000","last_peered":"2024-01-15T18:18:43.003692+0000","last_clean":"2024-01-15T18:18:43.003692+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:18:43.003692+0000","last_undegraded":"2024-01-15T18:18:43.003692+0000","last_fullsized":"2024-01-15T18:18:43.003692+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934993+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:18:42.934993+0000","last_peered":"2024-01-15T18:18:42.934993+0000","last_clean":"2024-01-15T18:18:42.934993+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:18:42.934993+0000","last_undegraded":"2024-01-15T18:18:42.934993+0000","last_fullsized":"2024-01-15T18:18:42.934993+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003271+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:18:43.003271+0000","last_peered":"2024-01-15T18:18:43.003271+0000","last_clean":"2024-01-15T18:18:43.003271+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:18:43.003271+0000","last_undegraded":"2024-01-15T18:18:43.003271+0000","last_fullsized":"2024-01-15T18:18:43.003271+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935805+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:18:42.935805+0000","last_peered":"2024-01-15T18:18:42.935805+0000","last_clean":"2024-01-15T18:18:42.935805+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:18:42.935805+0000","last_undegraded":"2024-01-15T18:18:42.935805+0000","last_fullsized":"2024-01-15T18:18:42.935805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:12.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005024+0000","last_change":"2024-01-15T18:18:41.949864+0000","last_active":"2024-01-15T18:18:43.005024+0000","last_peered":"2024-01-15T18:18:43.005024+0000","last_clean":"2024-01-15T18:18:43.005024+0000","last_became_active":"2024-01-15T18:18:41.944900+0000","last_became_peered":"2024-01-15T18:18:41.944900+0000","last_unstale":"2024-01-15T18:18:43.005024+0000","last_undegraded":"2024-01-15T18:18:43.005024+0000","last_fullsized":"2024-01-15T18:18:43.005024+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:40.334827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004112+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:18:43.004112+0000","last_peered":"2024-01-15T18:18:43.004112+0000","last_clean":"2024-01-15T18:18:43.004112+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:18:43.004112+0000","last_undegraded":"2024-01-15T18:18:43.004112+0000","last_fullsized":"2024-01-15T18:18:43.004112+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:18.174579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004372+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:18:43.004372+0000","last_peered":"2024-01-15T18:18:43.004372+0000","last_clean":"2024-01-15T18:18:43.004372+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:18:43.004372+0000","last_undegraded":"2024-01-15T18:18:43.004372+0000","last_fullsized":"2024-01-15T18:18:43.004372+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935282+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:18:42.935282+0000","last_peered":"2024-01-15T18:18:42.935282+0000","last_clean":"2024-01-15T18:18:42.935282+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:18:42.935282+0000","last_undegraded":"2024-01-15T18:18:42.935282+0000","last_fullsized":"2024-01-15T18:18:42.935282+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.933944+0000","last_change":"2024-01-15T18:18:41.937559+0000","last_active":"2024-01-15T18:18:42.933944+0000","last_peered":"2024-01-15T18:18:42.933944+0000","last_clean":"2024-01-15T18:18:42.933944+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:18:42.933944+0000","last_undegraded":"2024-01-15T18:18:42.933944+0000","last_fullsized":"2024-01-15T18:18:42.933944+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004735+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:18:43.004735+0000","last_peered":"2024-01-15T18:18:43.004735+0000","last_clean":"2024-01-15T18:18:43.004735+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:18:43.004735+0000","last_undegraded":"2024-01-15T18:18:43.004735+0000","last_fullsized":"2024-01-15T18:18:43.004735+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:18:31.192027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003606+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:18:43.003606+0000","last_peered":"2024-01-15T18:18:43.003606+0000","last_clean":"2024-01-15T18:18:43.003606+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:18:43.003606+0000","last_undegraded":"2024-01-15T18:18:43.003606+0000","last_fullsized":"2024-01-15T18:18:43.003606+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:56:19.546961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004226+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:18:43.004226+0000","last_peered":"2024-01-15T18:18:43.004226+0000","last_clean":"2024-01-15T18:18:43.004226+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:18:43.004226+0000","last_undegraded":"2024-01-15T18:18:43.004226+0000","last_fullsized":"2024-01-15T18:18:43.004226+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:47.919946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.941137+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:18:41.941137+0000","last_peered":"2024-01-15T18:18:41.941137+0000","last_clean":"2024-01-15T18:18:41.941137+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:18:41.941137+0000","last_undegraded":"2024-01-15T18:18:41.941137+0000","last_fullsized":"2024-01-15T18:18:41.941137+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:01:16.930740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.936328+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:18:42.936328+0000","last_peered":"2024-01-15T18:18:42.936328+0000","last_clean":"2024-01-15T18:18:42.936328+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:18:42.936328+0000","last_undegraded":"2024-01-15T18:18:42.936328+0000","last_fullsized":"2024-01-15T18:18:42.936328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.932391+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:18:41.932391+0000","last_peered":"2024-01-15T18:18:41.932391+0000","last_clean":"2024-01-15T18:18:41.932391+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:18:41.932391+0000","last_undegraded":"2024-01-15T18:18:41.932391+0000","last_fullsized":"2024-01-15T18:18:41.932391+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:24.049276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003918+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:18:43.003918+0000","last_peered":"2024-01-15T18:18:43.003918+0000","last_clean":"2024-01-15T18:18:43.003918+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:18:43.003918+0000","last_undegraded":"2024-01-15T18:18:43.003918+0000","last_fullsized":"2024-01-15T18:18:43.003918+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:24.340948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.957666+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:18:41.957666+0000","last_peered":"2024-01-15T18:18:41.957666+0000","last_clean":"2024-01-15T18:18:41.957666+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:18:41.957666+0000","last_undegraded":"2024-01-15T18:18:41.957666+0000","last_fullsized":"2024-01-15T18:18:41.957666+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:51.311221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996080+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:18:42.996080+0000","last_peered":"2024-01-15T18:18:42.996080+0000","last_clean":"2024-01-15T18:18:42.996080+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:18:42.996080+0000","last_undegraded":"2024-01-15T18:18:42.996080+0000","last_fullsized":"2024-01-15T18:18:42.996080+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:02.239162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958742+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:18:41.958742+0000","last_peered":"2024-01-15T18:18:41.958742+0000","last_clean":"2024-01-15T18:18:41.958742+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:18:41.958742+0000","last_undegraded":"2024-01-15T18:18:41.958742+0000","last_fullsized":"2024-01-15T18:18:41.958742+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:59.159845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004746+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:18:43.004746+0000","last_peered":"2024-01-15T18:18:43.004746+0000","last_clean":"2024-01-15T18:18:43.004746+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:18:43.004746+0000","last_undegraded":"2024-01-15T18:18:43.004746+0000","last_fullsized":"2024-01-15T18:18:43.004746+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:01:50.217450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.941822+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:18:41.941822+0000","last_peered":"2024-01-15T18:18:41.941822+0000","last_clean":"2024-01-15T18:18:41.941822+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:18:41.941822+0000","last_undegraded":"2024-01-15T18:18:41.941822+0000","last_fullsized":"2024-01-15T18:18:41.941822+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:23.630535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935414+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:18:42.935414+0000","last_peered":"2024-01-15T18:18:42.935414+0000","last_clean":"2024-01-15T18:18:42.935414+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:18:42.935414+0000","last_undegraded":"2024-01-15T18:18:42.935414+0000","last_fullsized":"2024-01-15T18:18:42.935414+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.269883+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:18:42.269883+0000","last_peered":"2024-01-15T18:18:42.269883+0000","last_clean":"2024-01-15T18:18:42.269883+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:18:42.269883+0000","last_undegraded":"2024-01-15T18:18:42.269883+0000","last_fullsized":"2024-01-15T18:18:42.269883+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003348+0000","last_change":"2024-01-15T18:18:41.940718+0000","last_active":"2024-01-15T18:18:43.003348+0000","last_peered":"2024-01-15T18:18:43.003348+0000","last_clean":"2024-01-15T18:18:43.003348+0000","last_became_active":"2024-01-15T18:18:41.940432+0000","last_became_peered":"2024-01-15T18:18:41.940432+0000","last_unstale":"2024-01-15T18:18:43.003348+0000","last_undegraded":"2024-01-15T18:18:43.003348+0000","last_fullsized":"2024-01-15T18:18:43.003348+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:22.038786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.916211+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:18:41.916211+0000","last_peered":"2024-01-15T18:18:41.916211+0000","last_clean":"2024-01-15T18:18:41.916211+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:18:41.916211+0000","last_undegraded":"2024-01-15T18:18:41.916211+0000","last_fullsized":"2024-01-15T18:18:41.916211+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.933449+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:18:42.933449+0000","last_peered":"2024-01-15T18:18:42.933449+0000","last_clean":"2024-01-15T18:18:42.933449+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:18:42.933449+0000","last_undegraded":"2024-01-15T18:18:42.933449+0000","last_fullsized":"2024-01-15T18:18:42.933449+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060486+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:18:42.060486+0000","last_peered":"2024-01-15T18:18:42.060486+0000","last_clean":"2024-01-15T18:18:42.060486+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:18:42.060486+0000","last_undegraded":"2024-01-15T18:18:42.060486+0000","last_fullsized":"2024-01-15T18:18:42.060486+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934590+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:18:42.934590+0000","last_peered":"2024-01-15T18:18:42.934590+0000","last_clean":"2024-01-15T18:18:42.934590+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:18:42.934590+0000","last_undegraded":"2024-01-15T18:18:42.934590+0000","last_fullsized":"2024-01-15T18:18:42.934590+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.928543+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:18:41.928543+0000","last_peered":"2024-01-15T18:18:41.928543+0000","last_clean":"2024-01-15T18:18:41.928543+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:18:41.928543+0000","last_undegraded":"2024-01-15T18:18:41.928543+0000","last_fullsized":"2024-01-15T18:18:41.928543+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:01.739271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.936464+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:18:42.936464+0000","last_peered":"2024-01-15T18:18:42.936464+0000","last_clean":"2024-01-15T18:18:42.936464+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:18:42.936464+0000","last_undegraded":"2024-01-15T18:18:42.936464+0000","last_fullsized":"2024-01-15T18:18:42.936464+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.269964+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:18:42.269964+0000","last_peered":"2024-01-15T18:18:42.269964+0000","last_clean":"2024-01-15T18:18:42.269964+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:18:42.269964+0000","last_undegraded":"2024-01-15T18:18:42.269964+0000","last_fullsized":"2024-01-15T18:18:42.269964+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004500+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:18:43.004500+0000","last_peered":"2024-01-15T18:18:43.004500+0000","last_clean":"2024-01-15T18:18:43.004500+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:18:43.004500+0000","last_undegraded":"2024-01-15T18:18:43.004500+0000","last_fullsized":"2024-01-15T18:18:43.004500+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"110'3","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060685+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:18:42.060685+0000","last_peered":"2024-01-15T18:18:42.060685+0000","last_clean":"2024-01-15T18:18:42.060685+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:18:42.060685+0000","last_undegraded":"2024-01-15T18:18:42.060685+0000","last_fullsized":"2024-01-15T18:18:42.060685+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934783+0000","last_change":"2024-01-15T18:18:41.931596+0000","last_active":"2024-01-15T18:18:42.934783+0000","last_peered":"2024-01-15T18:18:42.934783+0000","last_clean":"2024-01-15T18:18:42.934783+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:18:42.934783+0000","last_undegraded":"2024-01-15T18:18:42.934783+0000","last_fullsized":"2024-01-15T18:18:42.934783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"110'6","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060666+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:18:42.060666+0000","last_peered":"2024-01-15T18:18:42.060666+0000","last_clean":"2024-01-15T18:18:42.060666+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:18:42.060666+0000","last_undegraded":"2024-01-15T18:18:42.060666+0000","last_fullsized":"2024-01-15T18:18:42.060666+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003888+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:18:43.003888+0000","last_peered":"2024-01-15T18:18:43.003888+0000","last_clean":"2024-01-15T18:18:43.003888+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:18:43.003888+0000","last_undegraded":"2024-01-15T18:18:43.003888+0000","last_fullsized":"2024-01-15T18:18:43.003888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.991431+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:18:41.991431+0000","last_peered":"2024-01-15T18:18:41.991431+0000","last_clean":"2024-01-15T18:18:41.991431+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:18:41.991431+0000","last_undegraded":"2024-01-15T18:18:41.991431+0000","last_fullsized":"2024-01-15T18:18:41.991431+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004204+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:18:43.004204+0000","last_peered":"2024-01-15T18:18:43.004204+0000","last_clean":"2024-01-15T18:18:43.004204+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:18:43.004204+0000","last_undegraded":"2024-01-15T18:18:43.004204+0000","last_fullsized":"2024-01-15T18:18:43.004204+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.912091+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:18:41.912091+0000","last_peered":"2024-01-15T18:18:41.912091+0000","last_clean":"2024-01-15T18:18:41.912091+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:18:41.912091+0000","last_undegraded":"2024-01-15T18:18:41.912091+0000","last_fullsized":"2024-01-15T18:18:41.912091+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003500+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:18:43.003500+0000","last_peered":"2024-01-15T18:18:43.003500+0000","last_clean":"2024-01-15T18:18:43.003500+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:18:43.003500+0000","last_undegraded":"2024-01-15T18:18:43.003500+0000","last_fullsized":"2024-01-15T18:18:43.003500+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:51:04.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958735+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:18:41.958735+0000","last_peered":"2024-01-15T18:18:41.958735+0000","last_clean":"2024-01-15T18:18:41.958735+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:18:41.958735+0000","last_undegraded":"2024-01-15T18:18:41.958735+0000","last_fullsized":"2024-01-15T18:18:41.958735+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:27:26.085701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003175+0000","last_change":"2024-01-15T18:18:41.936562+0000","last_active":"2024-01-15T18:18:43.003175+0000","last_peered":"2024-01-15T18:18:43.003175+0000","last_clean":"2024-01-15T18:18:43.003175+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:18:43.003175+0000","last_undegraded":"2024-01-15T18:18:43.003175+0000","last_fullsized":"2024-01-15T18:18:43.003175+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060816+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:18:42.060816+0000","last_peered":"2024-01-15T18:18:42.060816+0000","last_clean":"2024-01-15T18:18:42.060816+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:18:42.060816+0000","last_undegraded":"2024-01-15T18:18:42.060816+0000","last_fullsized":"2024-01-15T18:18:42.060816+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004482+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:18:43.004482+0000","last_peered":"2024-01-15T18:18:43.004482+0000","last_clean":"2024-01-15T18:18:43.004482+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:18:43.004482+0000","last_undegraded":"2024-01-15T18:18:43.004482+0000","last_fullsized":"2024-01-15T18:18:43.004482+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:48:34.695980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.958719+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:18:41.958719+0000","last_peered":"2024-01-15T18:18:41.958719+0000","last_clean":"2024-01-15T18:18:41.958719+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:18:41.958719+0000","last_undegraded":"2024-01-15T18:18:41.958719+0000","last_fullsized":"2024-01-15T18:18:41.958719+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:38.154337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995970+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:18:42.995970+0000","last_peered":"2024-01-15T18:18:42.995970+0000","last_clean":"2024-01-15T18:18:42.995970+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:18:42.995970+0000","last_undegraded":"2024-01-15T18:18:42.995970+0000","last_fullsized":"2024-01-15T18:18:42.995970+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":13,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.948648+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:18:41.948648+0000","last_peered":"2024-01-15T18:18:41.948648+0000","last_clean":"2024-01-15T18:18:41.948648+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:18:41.948648+0000","last_undegraded":"2024-01-15T18:18:41.948648+0000","last_fullsized":"2024-01-15T18:18:41.948648+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:00:59.694593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.996138+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:18:42.996138+0000","last_peered":"2024-01-15T18:18:42.996138+0000","last_clean":"2024-01-15T18:18:42.996138+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:18:42.996138+0000","last_undegraded":"2024-01-15T18:18:42.996138+0000","last_fullsized":"2024-01-15T18:18:42.996138+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:26.983917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.922691+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:18:41.922691+0000","last_peered":"2024-01-15T18:18:41.922691+0000","last_clean":"2024-01-15T18:18:41.922691+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:18:41.922691+0000","last_undegraded":"2024-01-15T18:18:41.922691+0000","last_fullsized":"2024-01-15T18:18:41.922691+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:42.895411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995605+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:18:42.995605+0000","last_peered":"2024-01-15T18:18:42.995605+0000","last_clean":"2024-01-15T18:18:42.995605+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:18:42.995605+0000","last_undegraded":"2024-01-15T18:18:42.995605+0000","last_fullsized":"2024-01-15T18:18:42.995605+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.991441+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:18:41.991441+0000","last_peered":"2024-01-15T18:18:41.991441+0000","last_clean":"2024-01-15T18:18:41.991441+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:18:41.991441+0000","last_undegraded":"2024-01-15T18:18:41.991441+0000","last_fullsized":"2024-01-15T18:18:41.991441+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004791+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:18:43.004791+0000","last_peered":"2024-01-15T18:18:43.004791+0000","last_clean":"2024-01-15T18:18:43.004791+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:18:43.004791+0000","last_undegraded":"2024-01-15T18:18:43.004791+0000","last_fullsized":"2024-01-15T18:18:43.004791+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:14.044300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.927818+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:18:41.927818+0000","last_peered":"2024-01-15T18:18:41.927818+0000","last_clean":"2024-01-15T18:18:41.927818+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:18:41.927818+0000","last_undegraded":"2024-01-15T18:18:41.927818+0000","last_fullsized":"2024-01-15T18:18:41.927818+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:51:31.110057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995617+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:18:42.995617+0000","last_peered":"2024-01-15T18:18:42.995617+0000","last_clean":"2024-01-15T18:18:42.995617+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:18:42.995617+0000","last_undegraded":"2024-01-15T18:18:42.995617+0000","last_fullsized":"2024-01-15T18:18:42.995617+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.991412+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:18:41.991412+0000","last_peered":"2024-01-15T18:18:41.991412+0000","last_clean":"2024-01-15T18:18:41.991412+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:18:41.991412+0000","last_undegraded":"2024-01-15T18:18:41.991412+0000","last_fullsized":"2024-01-15T18:18:41.991412+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.003498+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:18:43.003498+0000","last_peered":"2024-01-15T18:18:43.003498+0000","last_clean":"2024-01-15T18:18:43.003498+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:18:43.003498+0000","last_undegraded":"2024-01-15T18:18:43.003498+0000","last_fullsized":"2024-01-15T18:18:43.003498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060503+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:18:42.060503+0000","last_peered":"2024-01-15T18:18:42.060503+0000","last_clean":"2024-01-15T18:18:42.060503+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:18:42.060503+0000","last_undegraded":"2024-01-15T18:18:42.060503+0000","last_fullsized":"2024-01-15T18:18:42.060503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935854+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:18:42.935854+0000","last_peered":"2024-01-15T18:18:42.935854+0000","last_clean":"2024-01-15T18:18:42.935854+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:18:42.935854+0000","last_undegraded":"2024-01-15T18:18:42.935854+0000","last_fullsized":"2024-01-15T18:18:42.935854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.269980+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:18:42.269980+0000","last_peered":"2024-01-15T18:18:42.269980+0000","last_clean":"2024-01-15T18:18:42.269980+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:18:42.269980+0000","last_undegraded":"2024-01-15T18:18:42.269980+0000","last_fullsized":"2024-01-15T18:18:42.269980+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005507+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:18:43.005507+0000","last_peered":"2024-01-15T18:18:43.005507+0000","last_clean":"2024-01-15T18:18:43.005507+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:18:43.005507+0000","last_undegraded":"2024-01-15T18:18:43.005507+0000","last_fullsized":"2024-01-15T18:18:43.005507+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:38:55.463538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.912207+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:18:41.912207+0000","last_peered":"2024-01-15T18:18:41.912207+0000","last_clean":"2024-01-15T18:18:41.912207+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:18:41.912207+0000","last_undegraded":"2024-01-15T18:18:41.912207+0000","last_fullsized":"2024-01-15T18:18:41.912207+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:51.313218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.005198+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:18:43.005198+0000","last_peered":"2024-01-15T18:18:43.005198+0000","last_clean":"2024-01-15T18:18:43.005198+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:18:43.005198+0000","last_undegraded":"2024-01-15T18:18:43.005198+0000","last_fullsized":"2024-01-15T18:18:43.005198+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:47.175404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060706+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:18:42.060706+0000","last_peered":"2024-01-15T18:18:42.060706+0000","last_clean":"2024-01-15T18:18:42.060706+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:18:42.060706+0000","last_undegraded":"2024-01-15T18:18:42.060706+0000","last_fullsized":"2024-01-15T18:18:42.060706+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.934711+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:18:42.934711+0000","last_peered":"2024-01-15T18:18:42.934711+0000","last_clean":"2024-01-15T18:18:42.934711+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:18:42.934711+0000","last_undegraded":"2024-01-15T18:18:42.934711+0000","last_fullsized":"2024-01-15T18:18:42.934711+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:27.713450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.269948+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:18:42.269948+0000","last_peered":"2024-01-15T18:18:42.269948+0000","last_clean":"2024-01-15T18:18:42.269948+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:18:42.269948+0000","last_undegraded":"2024-01-15T18:18:42.269948+0000","last_fullsized":"2024-01-15T18:18:42.269948+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935125+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:18:42.935125+0000","last_peered":"2024-01-15T18:18:42.935125+0000","last_clean":"2024-01-15T18:18:42.935125+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:18:42.935125+0000","last_undegraded":"2024-01-15T18:18:42.935125+0000","last_fullsized":"2024-01-15T18:18:42.935125+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.922187+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:18:41.922187+0000","last_peered":"2024-01-15T18:18:41.922187+0000","last_clean":"2024-01-15T18:18:41.922187+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:18:41.922187+0000","last_undegraded":"2024-01-15T18:18:41.922187+0000","last_fullsized":"2024-01-15T18:18:41.922187+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:54.586280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.935563+0000","last_change":"2024-01-15T18:18:41.935993+0000","last_active":"2024-01-15T18:18:42.935563+0000","last_peered":"2024-01-15T18:18:42.935563+0000","last_clean":"2024-01-15T18:18:42.935563+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:18:42.935563+0000","last_undegraded":"2024-01-15T18:18:42.935563+0000","last_fullsized":"2024-01-15T18:18:42.935563+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.916297+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:18:41.916297+0000","last_peered":"2024-01-15T18:18:41.916297+0000","last_clean":"2024-01-15T18:18:41.916297+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:18:41.916297+0000","last_undegraded":"2024-01-15T18:18:41.916297+0000","last_fullsized":"2024-01-15T18:18:41.916297+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:26.705662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.995855+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:18:42.995855+0000","last_peered":"2024-01-15T18:18:42.995855+0000","last_clean":"2024-01-15T18:18:42.995855+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:18:42.995855+0000","last_undegraded":"2024-01-15T18:18:42.995855+0000","last_fullsized":"2024-01-15T18:18:42.995855+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.927702+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:18:41.927702+0000","last_peered":"2024-01-15T18:18:41.927702+0000","last_clean":"2024-01-15T18:18:41.927702+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:18:41.927702+0000","last_undegraded":"2024-01-15T18:18:41.927702+0000","last_fullsized":"2024-01-15T18:18:41.927702+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:03:44.404981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.004755+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:18:43.004755+0000","last_peered":"2024-01-15T18:18:43.004755+0000","last_clean":"2024-01-15T18:18:43.004755+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:18:43.004755+0000","last_undegraded":"2024-01-15T18:18:43.004755+0000","last_fullsized":"2024-01-15T18:18:43.004755+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:52.187345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.060520+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:18:42.060520+0000","last_peered":"2024-01-15T18:18:42.060520+0000","last_clean":"2024-01-15T18:18:42.060520+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:18:42.060520+0000","last_undegraded":"2024-01-15T18:18:42.060520+0000","last_fullsized":"2024-01-15T18:18:42.060520+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-01-15T18:18:43.002904+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:18:43.002904+0000","last_peered":"2024-01-15T18:18:43.002904+0000","last_clean":"2024-01-15T18:18:43.002904+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:18:43.002904+0000","last_undegraded":"2024-01-15T18:18:43.002904+0000","last_fullsized":"2024-01-15T18:18:43.002904+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.922045+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:18:41.922045+0000","last_peered":"2024-01-15T18:18:41.922045+0000","last_clean":"2024-01-15T18:18:41.922045+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:18:41.922045+0000","last_undegraded":"2024-01-15T18:18:41.922045+0000","last_fullsized":"2024-01-15T18:18:41.922045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:35.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":241,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.921952+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:18:41.921952+0000","last_peered":"2024-01-15T18:18:41.921952+0000","last_clean":"2024-01-15T18:18:41.921952+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:18:41.921952+0000","last_undegraded":"2024-01-15T18:18:41.921952+0000","last_fullsized":"2024-01-15T18:18:41.921952+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64988,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306852,"statfs":{"total":96636764160,"available":96570216448,"internally_reserved":0,"allocated":1982464,"data_stored":1190314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.93100000000000005,"15min":0.92200000000000004},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29599999999999999},"max":{"1min":0.86399999999999999,"5min":16.815999999999999,"15min":16.815999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.77400000000000002,"15min":0.85999999999999999},"min":{"1min":0.48299999999999998,"5min":0.441,"15min":0.28399999999999997},"max":{"1min":0.92500000000000004,"5min":4.6100000000000003,"15min":13.744},"last":0.84799999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.96799999999999997,"15min":1.0640000000000001},"min":{"1min":0.48799999999999999,"5min":0.39100000000000001,"15min":0.36699999999999999},"max":{"1min":3.016,"5min":16.920000000000002,"15min":16.920000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.81100000000000005,"15min":1.054},"min":{"1min":0.43099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":4.7679999999999998,"15min":31.774999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.74099999999999999,"15min":0.98699999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78100000000000003,"5min":2.1680000000000001,"15min":30.846},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":1.073},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39800000000000002},"max":{"1min":1.117,"5min":17.039999999999999,"15min":39.695},"last":0.72999999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.82599999999999996,"15min":1.1579999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38800000000000001},"max":{"1min":0.88400000000000001,"5min":8.2650000000000006,"15min":26.463999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.94699999999999995,"15min":1.202},"min":{"1min":0.42899999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91000000000000003,"5min":16.879999999999999,"15min":42.692},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.78800000000000003,"15min":0.70899999999999996},"min":{"1min":0.42199999999999999,"5min":0.377,"15min":0.33300000000000002},"max":{"1min":1.147,"5min":14.087,"15min":14.087},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.629,"15min":0.67600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36299999999999999},"max":{"1min":0.82599999999999996,"5min":2.2450000000000001,"15min":4.3710000000000004},"last":0.52800000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.81299999999999994,"15min":0.71199999999999997},"min":{"1min":0.38300000000000001,"5min":0.36899999999999999,"15min":0.32400000000000001},"max":{"1min":1.0209999999999999,"5min":11.334,"15min":11.334},"last":0.628},{"interface":"front","average":{"1min":0.627,"5min":0.74099999999999999,"15min":0.76100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":4.8179999999999996,"15min":8.423},"last":0.746}]},{"osd":6,"last update":"Mon Jan 15 18:18:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79500000000000004,"15min":0.80400000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.5369999999999999,"5min":4.7140000000000004,"15min":8.5030000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.68600000000000005,"15min":0.71999999999999997},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.96099999999999997,"5min":1.1970000000000001,"15min":3.3450000000000002},"last":0.77100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575019,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64412,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307428,"statfs":{"total":96636764160,"available":96570806272,"internally_reserved":0,"allocated":1384448,"data_stored":599400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.81699999999999995,"15min":0.95299999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.874,"5min":9.8919999999999995,"15min":22.754000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.74199999999999999,"15min":0.873},"min":{"1min":0.375,"5min":0.375,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":3.4940000000000002,"15min":12.449},"last":0.752}]},{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.85599999999999998,"15min":0.89700000000000002},"min":{"1min":0.42299999999999999,"5min":0.41199999999999998,"15min":0.33900000000000002},"max":{"1min":1.0489999999999999,"5min":6.8239999999999998,"15min":14.246},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.84199999999999997,"15min":1.004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.379},"max":{"1min":0.81899999999999995,"5min":8.9320000000000004,"15min":19.475000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":1.069,"15min":1.179},"min":{"1min":0.50700000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93200000000000005,"5min":14.743,"15min":42.825000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.89000000000000001,"15min":1.153},"min":{"1min":0.41199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89800000000000002,"5min":9.4000000000000004,"15min":42.851999999999997},"last":0.86699999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.78300000000000003,"15min":1.0269999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.41299999999999998},"max":{"1min":0.82099999999999995,"5min":3.016,"15min":24.199000000000002},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":1.0629999999999999,"15min":1.087},"min":{"1min":0.46400000000000002,"5min":0.45800000000000002,"15min":0.38300000000000001},"max":{"1min":2.956,"5min":18.791,"15min":18.791},"last":0.66600000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.89300000000000002,"15min":0.745},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79600000000000004,"5min":19.114999999999998,"15min":19.114999999999998},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65500000000000003,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33700000000000002},"max":{"1min":0.875,"5min":1.7450000000000001,"15min":12.491},"last":0.96599999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81899999999999995,"5min":7.5880000000000001,"15min":9.4819999999999993},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.89800000000000002},"min":{"1min":0.51600000000000001,"5min":0.47599999999999998,"15min":0.42099999999999999},"max":{"1min":0.96199999999999997,"5min":2.5259999999999998,"15min":12.945},"last":0.93200000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":1.006,"15min":0.83899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":11.907,"15min":11.907},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.82499999999999996,"15min":0.88400000000000001},"min":{"1min":0.432,"5min":0.42799999999999999,"15min":0.34200000000000003},"max":{"1min":0.85399999999999998,"5min":9.9440000000000008,"15min":27.651},"last":0.89500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76888,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294952,"statfs":{"total":96636764160,"available":96558030848,"internally_reserved":0,"allocated":1388544,"data_stored":599934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.81200000000000006,"15min":0.83199999999999996},"min":{"1min":0.34300000000000003,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":4.4770000000000003,"15min":11.519},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.874,"15min":0.92400000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34999999999999998},"max":{"1min":0.91700000000000004,"5min":7.6260000000000003,"15min":12.952},"last":0.68100000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.92700000000000005,"15min":0.90300000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.37},"max":{"1min":0.95499999999999996,"5min":4.3140000000000001,"15min":13.358000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.77500000000000002,"15min":0.94699999999999995},"min":{"1min":0.374,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.0169999999999999,"5min":4.1580000000000004,"15min":21.856999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.90600000000000003,"15min":0.93500000000000005},"min":{"1min":0.44,"5min":0.373,"15min":0.373},"max":{"1min":1,"5min":6.7160000000000002,"15min":18.314},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":1.145,"15min":1.0109999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.0580000000000001,"5min":21.120000000000001,"15min":21.120000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.85499999999999998,"15min":0.94499999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":3.2400000000000002,"15min":13.317},"last":0.747},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.86099999999999999,"15min":1.01},"min":{"1min":0.437,"5min":0.437,"15min":0.42899999999999999},"max":{"1min":0.998,"5min":3.931,"15min":19.638000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.33600000000000002,"5min":0.33500000000000002,"15min":0.28799999999999998},"max":{"1min":0.96099999999999997,"5min":5.899,"15min":11.093},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.35899999999999999,"5min":0.35699999999999998,"15min":0.32100000000000001},"max":{"1min":0.88700000000000001,"5min":2.516,"15min":3.3889999999999998},"last":0.56899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.84399999999999997,"15min":0.79500000000000004},"min":{"1min":0.443,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97899999999999998,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.75,"15min":0.72299999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.308},"max":{"1min":1.0840000000000001,"5min":5.7130000000000001,"15min":7.2240000000000002},"last":0.76300000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.79800000000000004,"15min":0.75900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.33200000000000002},"max":{"1min":1.071,"5min":3.2759999999999998,"15min":4.1799999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.86399999999999999,"15min":0.81799999999999995},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.40400000000000003},"max":{"1min":1.034,"5min":6.2060000000000004,"15min":6.2060000000000004},"last":0.89800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575019,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81040,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290800,"statfs":{"total":96636764160,"available":96553779200,"internally_reserved":0,"allocated":1380352,"data_stored":599330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":81590725},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.79800000000000004,"15min":0.93300000000000005},"min":{"1min":0.35499999999999998,"5min":0.33800000000000002,"15min":0.29499999999999998},"max":{"1min":0.85799999999999998,"5min":9.4179999999999993,"15min":34.802},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.91700000000000004,"15min":0.90700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79500000000000004,"5min":14.488,"15min":22.335999999999999},"last":0.67100000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.72899999999999998,"15min":0.95699999999999996},"min":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83199999999999996,"5min":2.2130000000000001,"15min":38.198},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.80500000000000005,"15min":0.95599999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.78300000000000003,"5min":4.4550000000000001,"15min":38.220999999999997},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.755,"15min":0.86599999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.95099999999999996,"5min":1.6259999999999999,"15min":7.6989999999999998},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.78500000000000003,"15min":0.84799999999999998},"min":{"1min":0.504,"5min":0.45600000000000002,"15min":0.38700000000000001},"max":{"1min":0.90400000000000003,"5min":4.4089999999999998,"15min":7.4829999999999997},"last":0.73199999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.91300000000000003,"15min":0.95399999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.81699999999999995,"5min":14.529,"15min":14.529},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.72299999999999998,"15min":0.92900000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":1.4279999999999999,"15min":25.774000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76100000000000001,"15min":0.68200000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.33200000000000002},"max":{"1min":0.84299999999999997,"5min":9.9619999999999997,"15min":9.9619999999999997},"last":0.502},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.97799999999999998,"15min":0.82399999999999995},"min":{"1min":0.53400000000000003,"5min":0.47499999999999998,"15min":0.34100000000000003},"max":{"1min":0.91500000000000004,"5min":25.887,"15min":25.887},"last":0.57899999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.71399999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.38100000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":10.284000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.70799999999999996,"15min":0.69499999999999995},"min":{"1min":0.47099999999999997,"5min":0.34499999999999997,"15min":0.33900000000000002},"max":{"1min":0.91400000000000003,"5min":3.6949999999999998,"15min":3.6949999999999998},"last":0.55400000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:18:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.82799999999999996,"15min":0.78300000000000003},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.30399999999999999},"max":{"1min":0.94999999999999996,"5min":13.760999999999999,"15min":13.760999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70499999999999996,"15min":0.77100000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.36199999999999999},"max":{"1min":1.03,"5min":1.232,"15min":11.67},"last":0.52300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575019,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64424,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307416,"statfs":{"total":96636764160,"available":96570793984,"internally_reserved":0,"allocated":1396736,"data_stored":601012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":64551367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.625,"15min":0.95299999999999996},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.255},"max":{"1min":0.81899999999999995,"5min":7.2839999999999998,"15min":26.364000000000001},"last":0.443},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.66300000000000003,"15min":0.95599999999999996},"min":{"1min":0.312,"5min":0.25,"15min":0.246},"max":{"1min":0.71499999999999997,"5min":7.8520000000000003,"15min":28.332000000000001},"last":0.39800000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.60799999999999998,"15min":0.98799999999999999},"min":{"1min":0.29599999999999999,"5min":0.29199999999999998,"15min":0.28599999999999998},"max":{"1min":0.84799999999999998,"5min":3.8100000000000001,"15min":40.457999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.76300000000000001,"15min":1.1419999999999999},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.876,"5min":9.282,"15min":27.427},"last":0.53200000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.71199999999999997,"15min":0.98899999999999999},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.28399999999999997},"max":{"1min":0.90400000000000003,"5min":9.1180000000000003,"15min":23.036999999999999},"last":0.502},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.81499999999999995,"15min":1.089},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.34300000000000003},"max":{"1min":1.048,"5min":9.3510000000000009,"15min":26.404},"last":0.57099999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.84599999999999997,"15min":1.024},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.41199999999999998},"max":{"1min":1.0780000000000001,"5min":4.1040000000000001,"15min":22.998000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84599999999999997,"15min":1.0740000000000001},"min":{"1min":0.496,"5min":0.372,"15min":0.372},"max":{"1min":1.0229999999999999,"5min":3.8540000000000001,"15min":20.399000000000001},"last":0.81299999999999994}]},{"osd":5,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92300000000000004,"15min":1.097},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.125,"5min":6.5140000000000002,"15min":26.314},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.85599999999999998,"15min":1.1220000000000001},"min":{"1min":0.46100000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":4.165,"15min":22.908999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":1.048,"15min":1.153},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.42799999999999999},"max":{"1min":1.0609999999999999,"5min":6.843,"15min":20.445},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":1.095,"15min":1.1850000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.495},"max":{"1min":1.123,"5min":6.7640000000000002,"15min":26.091999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:10 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.98999999999999999,"15min":1.1579999999999999},"min":{"1min":0.505,"5min":0.498,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":7.0250000000000004,"15min":23.111999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.92000000000000004,"15min":1.321},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46000000000000002},"max":{"1min":1.1419999999999999,"5min":4.4470000000000001,"15min":44.817999999999998},"last":0.91200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575020,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64404,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307436,"statfs":{"total":96636764160,"available":96570814464,"internally_reserved":0,"allocated":1380352,"data_stored":599330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.54400000000000004,"15min":1.0529999999999999},"min":{"1min":0.31,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":5.9950000000000001,"15min":29.527999999999999},"last":0.495},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.86199999999999999,"15min":1.321},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.66400000000000003,"5min":23.792999999999999,"15min":34.122999999999998},"last":0.47699999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.64700000000000002,"15min":1.5409999999999999},"min":{"1min":0.35599999999999998,"5min":0.313,"15min":0.30199999999999999},"max":{"1min":0.56299999999999994,"5min":7.476,"15min":113.149},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.96899999999999997,"15min":1.1080000000000001},"min":{"1min":0.35499999999999998,"5min":0.30399999999999999,"15min":0.30199999999999999},"max":{"1min":0.59499999999999997,"5min":20.619,"15min":41.302},"last":0.36399999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":1.341,"15min":1.379},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28899999999999998},"max":{"1min":0.67400000000000004,"5min":39.945999999999998,"15min":46.960000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.66000000000000003,"15min":1.27},"min":{"1min":0.42699999999999999,"5min":0.40500000000000003,"15min":0.34200000000000003},"max":{"1min":0.77300000000000002,"5min":7.3609999999999998,"15min":115.693},"last":0.54900000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":1.206},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.47199999999999998},"max":{"1min":0.90800000000000003,"5min":7.4169999999999998,"15min":24.366},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":1.0740000000000001,"15min":1.23},"min":{"1min":0.51600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.81699999999999995,"5min":20.053000000000001,"15min":20.489000000000001},"last":0.76900000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":1.5800000000000001,"15min":1.5089999999999999},"min":{"1min":0.54700000000000004,"5min":0.46899999999999997,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":41.683999999999997,"15min":41.683999999999997},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.84999999999999998,"15min":1.304},"min":{"1min":0.59599999999999997,"5min":0.497,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":7.5099999999999998,"15min":26.084},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":1.0980000000000001,"15min":1.3149999999999999},"min":{"1min":0.46999999999999997,"5min":0.46899999999999997,"15min":0.45900000000000002},"max":{"1min":0.88100000000000001,"5min":20.120999999999999,"15min":22.199000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.748,"5min":1.6240000000000001,"15min":1.5700000000000001},"min":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.47699999999999998},"max":{"1min":0.94499999999999995,"5min":41.738999999999997,"15min":41.738999999999997},"last":0.92200000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:18:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":1.165,"15min":1.262},"min":{"1min":0.55200000000000005,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":0.92600000000000005,"5min":20.158000000000001,"15min":20.446999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":1.613,"15min":1.6299999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97299999999999998,"5min":41.402000000000001,"15min":41.402000000000001},"last":0.88600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575020,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64408,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307432,"statfs":{"total":96636764160,"available":96570810368,"internally_reserved":0,"allocated":1388544,"data_stored":599454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.64200000000000002,"15min":0.91500000000000004},"min":{"1min":0.245,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.68200000000000005,"5min":9.452,"15min":39.948},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.68799999999999994,"15min":1.103},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.27600000000000002},"max":{"1min":0.78500000000000003,"5min":8.141,"15min":35.790999999999997},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.71799999999999997,"15min":0.98199999999999998},"min":{"1min":0.32900000000000001,"5min":0.314,"15min":0.27600000000000002},"max":{"1min":0.63800000000000001,"5min":15.137,"15min":49.229999999999997},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.751,"15min":0.85699999999999998},"min":{"1min":0.33000000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61799999999999999,"5min":9.8469999999999995,"15min":10.349},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.61099999999999999,"15min":1.173},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.88,"5min":4.4669999999999996,"15min":27.643999999999998},"last":0.44},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.90000000000000002,"15min":0.997},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.35599999999999998},"max":{"1min":0.66800000000000004,"5min":15.608000000000001,"15min":15.608000000000001},"last":0.38500000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.84299999999999997,"15min":1.222},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.40200000000000002},"max":{"1min":0.86599999999999999,"5min":4.633,"15min":27.489999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":1.028,"15min":1.0700000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.98599999999999999,"5min":9.6989999999999998,"15min":12.997},"last":0.77300000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.86599999999999999,"15min":1.212},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.50800000000000001},"max":{"1min":1.0029999999999999,"5min":4.673,"15min":27.547000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":1.0289999999999999,"15min":1.1200000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.496},"max":{"1min":1.028,"5min":9.7650000000000006,"15min":13.054},"last":0.79600000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.86499999999999999,"15min":1.204},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.46500000000000002},"max":{"1min":0.95499999999999996,"5min":4.7060000000000004,"15min":27.579999999999998},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.87,"15min":1.0369999999999999},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.42399999999999999},"max":{"1min":0.90900000000000003,"5min":5.9020000000000001,"15min":11.712999999999999},"last":0.92400000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:18:11 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.1319999999999999},"min":{"1min":0.53400000000000003,"5min":0.52200000000000002,"15min":0.46500000000000002},"max":{"1min":6.6950000000000003,"5min":9.5660000000000007,"15min":11.833},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":1.0229999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.505,"15min":0.502},"max":{"1min":0.93200000000000005,"5min":9.8079999999999998,"15min":14.606},"last":0.81799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575019,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77480,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294360,"statfs":{"total":96636764160,"available":96557424640,"internally_reserved":0,"allocated":1994752,"data_stored":1191422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.78600000000000003,"15min":1.1379999999999999},"min":{"1min":0.27100000000000002,"5min":0.23999999999999999,"15min":0.23599999999999999},"max":{"1min":0.94299999999999995,"5min":11.356,"15min":31.977},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.81299999999999994,"15min":1.0229999999999999},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0289999999999999,"5min":11.629,"15min":36.145000000000003},"last":0.40600000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.76600000000000001,"15min":1.1319999999999999},"min":{"1min":0.36299999999999999,"5min":0.307,"15min":0.30199999999999999},"max":{"1min":0.91800000000000004,"5min":5.8929999999999998,"15min":41.387999999999998},"last":0.505},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.877,"15min":1.038},"min":{"1min":0.32000000000000001,"5min":0.315,"15min":0.25},"max":{"1min":0.97499999999999998,"5min":11.539999999999999,"15min":33.219999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.97599999999999998,"15min":1.0860000000000001},"min":{"1min":0.39300000000000002,"5min":0.34699999999999998,"15min":0.29899999999999999},"max":{"1min":1.093,"5min":11.391999999999999,"15min":31.943999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.79400000000000004,"15min":1.097},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33300000000000002},"max":{"1min":1.048,"5min":6.3739999999999997,"15min":45.128},"last":0.45300000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.0289999999999999,"15min":1.0740000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.372},"max":{"1min":2.1469999999999998,"5min":11.772,"15min":32.932000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":1.0840000000000001,"15min":1.1919999999999999},"min":{"1min":0.52700000000000002,"5min":0.45500000000000002,"15min":0.44},"max":{"1min":1.4370000000000001,"5min":11.220000000000001,"15min":31.782},"last":0.63800000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":1.083,"15min":1.1850000000000001},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3979999999999999,"5min":11.287000000000001,"15min":31.858000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":1.0389999999999999,"15min":1.0629999999999999},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.841,"5min":11.958,"15min":33.151000000000003},"last":0.70299999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.93500000000000005,"15min":1.0640000000000001},"min":{"1min":0.54300000000000004,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.379,"5min":4.0970000000000004,"15min":13.055},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.749,"5min":1.0449999999999999,"15min":1.0920000000000001},"min":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.522,"5min":11.833,"15min":33.098999999999997},"last":0.65300000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:18:06 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.92300000000000004,"15min":1.0740000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3160000000000001,"5min":4.1349999999999998,"15min":13.090999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":1.1120000000000001,"15min":1.242},"min":{"1min":0.53900000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":1.464,"5min":11.321999999999999,"15min":31.905999999999999},"last":0.69099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:50.859 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:18:50.859 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:18:51.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 Processor -- start 2024-01-15T18:18:51.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 -- start start 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf200cded0 0x7faf200c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cf460 0x7faf200c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf200c4ee0 con 0x7faf2015b210 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2015b5f0 con 0x7faf200cded0 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf27000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf200cf9d0 con 0x7faf200cf460 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf200cded0 0x7faf200c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf200cded0 0x7faf200c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47228/0 (socket says 172.21.15.143:47228) 2024-01-15T18:18:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 -- 172.21.15.143:0/3436884384 learned_addr learned my addr 172.21.15.143:0/3436884384 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf25000640 1 --2- 172.21.15.143:0/3436884384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cf460 0x7faf200c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 -- 172.21.15.143:0/3436884384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cf460 msgr2=0x7faf200c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 --2- 172.21.15.143:0/3436884384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cf460 0x7faf200c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 -- 172.21.15.143:0/3436884384 >> v1:172.21.15.204:6789/0 conn(0x7faf2015b210 legacy=0x7faf200cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.047+0000 7faf1f400640 1 -- 172.21.15.143:0/3436884384 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200cfcb0 con 0x7faf200cded0 2024-01-15T18:18:51.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf1f400640 1 --2- 172.21.15.143:0/3436884384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf200cded0 0x7faf200c5370 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7faf08009870 tx=0x7faf0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970c83a090fb9460 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf1ea00640 1 -- 172.21.15.143:0/3436884384 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080a6040 con 0x7faf200cded0 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf1ea00640 1 -- 172.21.15.143:0/3436884384 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf080aa600 con 0x7faf200cded0 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf1ea00640 1 -- 172.21.15.143:0/3436884384 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf200ccf00 con 0x7faf200cded0 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf27000640 1 -- 172.21.15.143:0/3436884384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf200cded0 msgr2=0x7faf200c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf27000640 1 --2- 172.21.15.143:0/3436884384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf200cded0 0x7faf200c5370 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7faf08009870 tx=0x7faf0801c480 comp rx=0 tx=0).stop 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf27000640 1 -- 172.21.15.143:0/3436884384 shutdown_connections 2024-01-15T18:18:51.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf27000640 1 --2- 172.21.15.143:0/3436884384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cf460 0x7faf200c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf27000640 1 --2- 172.21.15.143:0/3436884384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf200cded0 0x7faf200c5370 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.051+0000 7faf27000640 1 -- 172.21.15.143:0/3436884384 >> 172.21.15.143:0/3436884384 conn(0x7faf200d0e80 msgr2=0x7faf200d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 -- 172.21.15.143:0/3436884384 shutdown_connections 2024-01-15T18:18:51.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 -- 172.21.15.143:0/3436884384 wait complete. 2024-01-15T18:18:51.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 Processor -- start 2024-01-15T18:18:51.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 -- start start 2024-01-15T18:18:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cded0 0x7faf20156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf200cf460 0x7faf20157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 0x7faf20159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf201593e0 con 0x7faf2015b210 2024-01-15T18:18:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf201584f0 con 0x7faf200cf460 2024-01-15T18:18:51.051 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf27000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf20158a40 con 0x7faf200cded0 2024-01-15T18:18:51.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 0x7faf20159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 0x7faf20159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47230/0 (socket says 172.21.15.143:47230) 2024-01-15T18:18:51.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 -- 172.21.15.143:0/1520988273 learned_addr learned my addr 172.21.15.143:0/1520988273 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:51.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf1f400640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf200cf460 0x7faf20157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.052 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf1fe00640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cded0 0x7faf20156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 -- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cded0 msgr2=0x7faf20156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cded0 0x7faf20156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 -- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf200cf460 msgr2=0x7faf20157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf200cf460 0x7faf20157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.055+0000 7faf25000640 1 -- 172.21.15.143:0/1520988273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf080094d0 con 0x7faf2015b210 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf25000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 0x7faf20159800 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7faf18007e60 tx=0x7faf1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18014420 con 0x7faf2015b210 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf180145a0 con 0x7faf2015b210 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18014950 con 0x7faf2015b210 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf20158d20 con 0x7faf2015b210 2024-01-15T18:18:51.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf20158070 con 0x7faf2015b210 2024-01-15T18:18:51.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf20158290 con 0x7faf2015b210 2024-01-15T18:18:51.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.059+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7faf18014ab0 con 0x7faf2015b210 2024-01-15T18:18:51.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.063+0000 7faf17e00640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faef8067a60 0x7faef8069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.063+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7faf18083b30 con 0x7faf2015b210 2024-01-15T18:18:51.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.063+0000 7faf1fe00640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faef8067a60 0x7faef8069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.063+0000 7faf1fe00640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faef8067a60 0x7faef8069f20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7faf10007fd0 tx=0x7faf10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:51.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.067+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faf18052520 con 0x7faf2015b210 2024-01-15T18:18:51.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.275+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faf20067500 con 0x7faf2015b210 2024-01-15T18:18:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.279+0000 7faf17e00640 1 -- 172.21.15.143:0/1520988273 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7faf18055b40 con 0x7faf2015b210 2024-01-15T18:18:51.275 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2024-01-15T18:18:51.275 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQCPc6Vl/wk6HxAAjMuVrCQleV8gtRwtpeL70A== 2024-01-15T18:18:51.275 INFO:teuthology.orchestra.run.smithi143.stdout: caps mds = "allow" 2024-01-15T18:18:51.275 INFO:teuthology.orchestra.run.smithi143.stdout: caps mon = "allow r" 2024-01-15T18:18:51.275 INFO:teuthology.orchestra.run.smithi143.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:18:51.275 INFO:teuthology.orchestra.run.smithi143.stderr:updated caps for client.0 2024-01-15T18:18:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faef8067a60 msgr2=0x7faef8069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faef8067a60 0x7faef8069f20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7faf10007fd0 tx=0x7faf10009040 comp rx=0 tx=0).stop 2024-01-15T18:18:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 msgr2=0x7faf20159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 0x7faf20159800 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7faf18007e60 tx=0x7faf1800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 shutdown_connections 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faef8067a60 0x7faef8069f20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf2015b210 0x7faf20159800 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faf200cf460 0x7faf20157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 --2- 172.21.15.143:0/1520988273 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faf200cded0 0x7faf20156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 >> 172.21.15.143:0/1520988273 conn(0x7faf200d0e80 msgr2=0x7faf200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 shutdown_connections 2024-01-15T18:18:51.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.283+0000 7faf27000640 1 -- 172.21.15.143:0/1520988273 wait complete. 2024-01-15T18:18:51.299 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:18:51.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 Processor -- start 2024-01-15T18:18:51.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 -- start start 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f095015b260 0x7f09500cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500ccaf0 0x7f09500cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09500c4d90 con 0x7f09500d19f0 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095015b640 con 0x7f095015b260 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0957400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500cf800 con 0x7f09500ccaf0 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f095015b260 0x7f09500cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f095015b260 0x7f09500cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47242/0 (socket says 172.21.15.143:47242) 2024-01-15T18:18:51.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 -- 172.21.15.143:0/1040099402 learned_addr learned my addr 172.21.15.143:0/1040099402 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0955400640 1 --2- 172.21.15.143:0/1040099402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500ccaf0 0x7f09500cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 -- 172.21.15.143:0/1040099402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500ccaf0 msgr2=0x7f09500cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 --2- 172.21.15.143:0/1040099402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500ccaf0 0x7f09500cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 -- 172.21.15.143:0/1040099402 >> v1:172.21.15.204:6789/0 conn(0x7f09500d19f0 legacy=0x7f09500c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:51.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.443+0000 7f0954a00640 1 -- 172.21.15.143:0/1040099402 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500cfae0 con 0x7f095015b260 2024-01-15T18:18:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0954a00640 1 --2- 172.21.15.143:0/1040099402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f095015b260 0x7f09500cd810 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0940009870 tx=0x7f094001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f928eceff76243 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f094f400640 1 -- 172.21.15.143:0/1040099402 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400ac410 con 0x7f095015b260 2024-01-15T18:18:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f094f400640 1 -- 172.21.15.143:0/1040099402 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09400ac570 con 0x7f095015b260 2024-01-15T18:18:51.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f094f400640 1 -- 172.21.15.143:0/1040099402 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400ac900 con 0x7f095015b260 2024-01-15T18:18:51.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 -- 172.21.15.143:0/1040099402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f095015b260 msgr2=0x7f09500cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 --2- 172.21.15.143:0/1040099402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f095015b260 0x7f09500cd810 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0940009870 tx=0x7f094001c500 comp rx=0 tx=0).stop 2024-01-15T18:18:51.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 -- 172.21.15.143:0/1040099402 shutdown_connections 2024-01-15T18:18:51.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 --2- 172.21.15.143:0/1040099402 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500ccaf0 0x7f09500cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 --2- 172.21.15.143:0/1040099402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f095015b260 0x7f09500cd810 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 -- 172.21.15.143:0/1040099402 >> 172.21.15.143:0/1040099402 conn(0x7f09500d0ed0 msgr2=0x7f09500d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:51.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 -- 172.21.15.143:0/1040099402 shutdown_connections 2024-01-15T18:18:51.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.447+0000 7f0957400640 1 -- 172.21.15.143:0/1040099402 wait complete. 2024-01-15T18:18:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 Processor -- start 2024-01-15T18:18:51.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- start start 2024-01-15T18:18:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 0x7f095015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500d19f0 0x7f09501591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095015b260 0x7f0950152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950158490 con 0x7f09500ccaf0 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950159700 con 0x7f095015b260 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095015a8e0 con 0x7f09500d19f0 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 0x7f095015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 0x7f095015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47246/0 (socket says 172.21.15.143:47246) 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 -- 172.21.15.143:0/3165034703 learned_addr learned my addr 172.21.15.143:0/3165034703 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 -- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500d19f0 msgr2=0x7f09501591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0955400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095015b260 0x7f0950152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f094fe00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500d19f0 0x7f09501591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500d19f0 0x7f09501591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 -- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095015b260 msgr2=0x7f0950152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095015b260 0x7f0950152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 -- 172.21.15.143:0/3165034703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400094d0 con 0x7f09500ccaf0 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0955400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095015b260 0x7f0950152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f094fe00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500d19f0 0x7f09501591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0954a00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 0x7f095015a300 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f09400af040 tx=0x7f09400192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400ac410 con 0x7f09500ccaf0 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09400b5630 con 0x7f09500ccaf0 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09501586e0 con 0x7f09500ccaf0 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400b59e0 con 0x7f09500ccaf0 2024-01-15T18:18:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0950158ca0 con 0x7f09500ccaf0 2024-01-15T18:18:51.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.451+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0950067500 con 0x7f09500ccaf0 2024-01-15T18:18:51.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.455+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f09400b3070 con 0x7f09500ccaf0 2024-01-15T18:18:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.455+0000 7f094cc00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0928067a60 0x7f0928069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.455+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f094011d5d0 con 0x7f09500ccaf0 2024-01-15T18:18:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.455+0000 7f094fe00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0928067a60 0x7f0928069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.455+0000 7f094fe00640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0928067a60 0x7f0928069f20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f09500ce050 tx=0x7f0938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:51.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.459+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f09400ec260 con 0x7f09500ccaf0 2024-01-15T18:18:51.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.639+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09500b6d50 con 0x7f09500ccaf0 2024-01-15T18:18:51.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.639+0000 7f094cc00640 1 -- 172.21.15.143:0/3165034703 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f09400b1020 con 0x7f09500ccaf0 2024-01-15T18:18:51.635 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:18:51.635 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8776,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2963172683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2963172683},{"type":"v1","addr":"172.21.15.204:6834","nonce":2963172683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":9012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/813203633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":813203633},{"type":"v1","addr":"172.21.15.143:6836","nonce":813203633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":9015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/3497546933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3497546933},{"type":"v1","addr":"172.21.15.143:6837","nonce":3497546933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:18:41.297206+0000","modified":"2024-01-15T18:18:44.320349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3419861535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3419861535},{"type":"v1","addr":"172.21.15.204:6835","nonce":3419861535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-01-15T18:18:51.636 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 72 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0928067a60 msgr2=0x7f0928069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0928067a60 0x7f0928069f20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f09500ce050 tx=0x7f0938008040 comp rx=0 tx=0).stop 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 msgr2=0x7f095015a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 0x7f095015a300 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f09400af040 tx=0x7f09400192a0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 shutdown_connections 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0928067a60 0x7f0928069f20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f095015b260 0x7f0950152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09500d19f0 0x7f09501591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 --2- 172.21.15.143:0/3165034703 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09500ccaf0 0x7f095015a300 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 >> 172.21.15.143:0/3165034703 conn(0x7f09500d0ed0 msgr2=0x7f09500c5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 shutdown_connections 2024-01-15T18:18:51.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:51.643+0000 7f0957400640 1 -- 172.21.15.143:0/3165034703 wait complete. 2024-01-15T18:18:52.060 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:18:41.297206+0000', 'modified': '2024-01-15T18:18:44.320349+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9021}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9021': {'gid': 9021, 'name': 'd', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.204:6835/3419861535', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6833', 'nonce': 3419861535}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 3419861535}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:18:52.060 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:18:52.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-15T18:18:52.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.203+0000 7ff209400640 1 Processor -- start 2024-01-15T18:18:52.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff209400640 1 -- start start 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff209400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040ce690 0x7ff2040ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff209400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2040b3bf0 0x7ff2040b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff209400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff204070bf0 con 0x7ff2040b1da0 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff209400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20415dba0 con 0x7ff2040b3bf0 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff209400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2040b4920 con 0x7ff2040ce690 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2040b3bf0 0x7ff2040b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff2040b3bf0 0x7ff2040b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47256/0 (socket says 172.21.15.143:47256) 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 -- 172.21.15.143:0/2400517399 learned_addr learned my addr 172.21.15.143:0/2400517399 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff202800640 1 --2- 172.21.15.143:0/2400517399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040ce690 0x7ff2040ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 -- 172.21.15.143:0/2400517399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040ce690 msgr2=0x7ff2040ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 --2- 172.21.15.143:0/2400517399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040ce690 0x7ff2040ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 -- 172.21.15.143:0/2400517399 >> v1:172.21.15.204:6789/0 conn(0x7ff2040b1da0 legacy=0x7ff2040b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.207+0000 7ff203200640 1 -- 172.21.15.143:0/2400517399 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2040b4c00 con 0x7ff2040b3bf0 2024-01-15T18:18:52.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff203200640 1 --2- 172.21.15.143:0/2400517399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040b3bf0 0x7ff2040b4080 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e50 tx=0x7ff1fc01d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1302aeffde57c3f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff201400640 1 -- 172.21.15.143:0/2400517399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0a58d0 con 0x7ff2040b3bf0 2024-01-15T18:18:52.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff201400640 1 -- 172.21.15.143:0/2400517399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc0a5a30 con 0x7ff2040b3bf0 2024-01-15T18:18:52.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff201400640 1 -- 172.21.15.143:0/2400517399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0a5dc0 con 0x7ff2040b3bf0 2024-01-15T18:18:52.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 -- 172.21.15.143:0/2400517399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040b3bf0 msgr2=0x7ff2040b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 --2- 172.21.15.143:0/2400517399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040b3bf0 0x7ff2040b4080 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e50 tx=0x7ff1fc01d440 comp rx=0 tx=0).stop 2024-01-15T18:18:52.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 -- 172.21.15.143:0/2400517399 shutdown_connections 2024-01-15T18:18:52.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 --2- 172.21.15.143:0/2400517399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040b3bf0 0x7ff2040b4080 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 --2- 172.21.15.143:0/2400517399 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040ce690 0x7ff2040ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 -- 172.21.15.143:0/2400517399 >> 172.21.15.143:0/2400517399 conn(0x7ff20406c3f0 msgr2=0x7ff20406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 -- 172.21.15.143:0/2400517399 shutdown_connections 2024-01-15T18:18:52.207 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.211+0000 7ff209400640 1 -- 172.21.15.143:0/2400517399 wait complete. 2024-01-15T18:18:52.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 Processor -- start 2024-01-15T18:18:52.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 -- start start 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2040b1da0 0x7ff2041598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040b3bf0 0x7ff204145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 0x7ff204140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20415ac40 con 0x7ff2040ce690 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff204146390 con 0x7ff2040b1da0 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff204159fc0 con 0x7ff2040b3bf0 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 0x7ff204140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 0x7ff204140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47264/0 (socket says 172.21.15.143:47264) 2024-01-15T18:18:52.211 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 -- 172.21.15.143:0/943523073 learned_addr learned my addr 172.21.15.143:0/943523073 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff202800640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2040b1da0 0x7ff2041598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff201e00640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040b3bf0 0x7ff204145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 -- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040b3bf0 msgr2=0x7ff204145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040b3bf0 0x7ff204145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 -- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2040b1da0 msgr2=0x7ff2041598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2040b1da0 0x7ff2041598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 -- 172.21.15.143:0/943523073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc009b30 con 0x7ff2040ce690 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff203200640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 0x7ff204140450 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc01a610 tx=0x7ff1fc019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff1faa00640 1 -- 172.21.15.143:0/943523073 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0b8670 con 0x7ff2040ce690 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff1faa00640 1 -- 172.21.15.143:0/943523073 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc0b87d0 con 0x7ff2040ce690 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff1faa00640 1 -- 172.21.15.143:0/943523073 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0b8b60 con 0x7ff2040ce690 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20415a2a0 con 0x7ff2040ce690 2024-01-15T18:18:52.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.215+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20415a860 con 0x7ff2040ce690 2024-01-15T18:18:52.213 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff1faa00640 1 -- 172.21.15.143:0/943523073 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7ff1fc0b8cc0 con 0x7ff2040ce690 2024-01-15T18:18:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff1faa00640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1dc067a60 0x7ff1dc069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff1faa00640 1 -- 172.21.15.143:0/943523073 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff1fc11e8a0 con 0x7ff2040ce690 2024-01-15T18:18:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff202800640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1dc067a60 0x7ff1dc069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 Processor -- start 2024-01-15T18:18:52.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 -- start start 2024-01-15T18:18:52.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff202800640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1dc067a60 0x7ff1dc069f20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40079c0 tx=0x7ff1f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb58884070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb588845b0 0x55fb58839de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb5883a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb586a4130 con 0x55fb58883c90 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb586919a0 con 0x55fb588845b0 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5868eed0 con 0x55fb5883a320 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff202800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb58884070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff202800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb58884070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47268/0 (socket says 172.21.15.143:47268) 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff202800640 1 -- 172.21.15.143:0/1218365519 learned_addr learned my addr 172.21.15.143:0/1218365519 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.219+0000 7ff201e00640 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb588845b0 0x55fb58839de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff203200640 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb5883a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff202800640 1 -- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 msgr2=0x55fb5883a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff202800640 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb5883a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff202800640 1 -- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb588845b0 msgr2=0x55fb58839de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff202800640 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb588845b0 0x55fb58839de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff202800640 1 -- 172.21.15.143:0/1218365519 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb58382890 con 0x55fb58883c90 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff202800640 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb58884070 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4010ac0 tx=0x7ff1f4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19535b1d79b8caa8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff1f3400640 1 -- 172.21.15.143:0/1218365519 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4016c10 con 0x55fb58883c90 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff1f3400640 1 -- 172.21.15.143:0/1218365519 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f401a070 con 0x55fb58883c90 2024-01-15T18:18:52.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff1f3400640 1 -- 172.21.15.143:0/1218365519 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4021440 con 0x55fb58883c90 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- 172.21.15.143:0/1218365519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 msgr2=0x55fb58884070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb58884070 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4010ac0 tx=0x7ff1f4013000 comp rx=0 tx=0).stop 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- 172.21.15.143:0/1218365519 shutdown_connections 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb5883a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb588845b0 0x55fb58839de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1218365519 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb58884070 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- 172.21.15.143:0/1218365519 >> 172.21.15.143:0/1218365519 conn(0x55fb588362d0 msgr2=0x55fb588386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- 172.21.15.143:0/1218365519 shutdown_connections 2024-01-15T18:18:52.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- 172.21.15.143:0/1218365519 wait complete. 2024-01-15T18:18:52.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- start start 2024-01-15T18:18:52.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 Processor -- start 2024-01-15T18:18:52.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 0x55fb5883ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb5883f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588845b0 0x55fb58852910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5868a7c0 con 0x55fb5883a320 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb58642bf0 con 0x55fb58883c90 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb58648a90 con 0x55fb588845b0 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.223+0000 7ff20c92a000 10 client.0 _ll_register_callbacks cb 0x55fb588253c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 0x55fb5883ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 0x55fb5883ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47282/0 (socket says 172.21.15.143:47282) 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 -- 172.21.15.143:0/51761304 learned_addr learned my addr 172.21.15.143:0/51761304 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff201e00640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb5883f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff203200640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588845b0 0x55fb58852910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 -- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588845b0 msgr2=0x55fb58852910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588845b0 0x55fb58852910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 -- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 msgr2=0x55fb5883f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb5883f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 -- 172.21.15.143:0/51761304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f400e040 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff202800640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 0x55fb5883ed40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4016850 tx=0x7ff1f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f65288db73c402 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 1 -- 172.21.15.143:0/51761304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4021b70 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 1 -- 172.21.15.143:0/51761304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff1f4021cd0 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb5864c470 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 1 -- 172.21.15.143:0/51761304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f4034460 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 10 client.9174 fetch_fsmap learned FSMap version 72 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55fb5864b190 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 1 -- 172.21.15.143:0/51761304 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb5864c470 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 1 -- 172.21.15.143:0/51761304 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1f40345c0 con 0x55fb5883a320 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 10 client.9174 fetch_fsmap finished waiting for FSMap version 72 2024-01-15T18:18:52.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 10 client.9174 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 20 client.9174 populate_metadata read hostname 'smithi143' 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb58853320 0x55fb58895f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 4 client.9174 mds_command: new command op to 9021 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 --> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb5864fb50 con 0x55fb58853320 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff201e00640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb58853320 0x55fb58895f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff201e00640 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb58853320 0x55fb58895f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84e63fcc3d3628d5 server_cookie=64d6964174eca31d in_seq=0 out_seq=0 2024-01-15T18:18:52.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 10 client.9174 ms_handle_connect on v2:172.21.15.204:6833/3419861535 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 1 -- 172.21.15.143:0/51761304 <== mds.0 v2:172.21.15.204:6833/3419861535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb5864fb50 con 0x55fb58853320 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff1ebe00640 10 client.9174 handle_command_reply: tid=0 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.227+0000 7ff20c92a000 1 client.9174 shutdown 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 msgr2=0x55fb5883ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 0x55fb5883ed40 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4016850 tx=0x7ff1f4004740 comp rx=0 tx=0).stop 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 shutdown_connections 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb58853320 0x55fb58895f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588845b0 0x55fb58852910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb5883f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 --2- 172.21.15.143:0/51761304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb5883a320 0x55fb5883ed40 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff1ebe00640 0 client.9174 ms_handle_reset on v2:172.21.15.204:6833/3419861535 2024-01-15T18:18:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 >> 172.21.15.143:0/51761304 conn(0x55fb588362d0 msgr2=0x55fb588386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 shutdown_connections 2024-01-15T18:18:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 1 -- 172.21.15.143:0/51761304 wait complete. 2024-01-15T18:18:52.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.231+0000 7ff20c92a000 20 client.9174 trim_cache size 0 max 16384 2024-01-15T18:18:52.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.243+0000 7ff20c92a000 1 Processor -- start 2024-01-15T18:18:52.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- start start 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588362d0 0x55fb58881b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb5883a320 0x55fb58882070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb588667a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb58343940 con 0x55fb58883c90 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb58687bc0 con 0x55fb5883a320 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb5883f820 con 0x55fb588362d0 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb588667a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb588667a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47284/0 (socket says 172.21.15.143:47284) 2024-01-15T18:18:52.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 -- 172.21.15.143:0/3737326017 learned_addr learned my addr 172.21.15.143:0/3737326017 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff201e00640 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb5883a320 0x55fb58882070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff202800640 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588362d0 0x55fb58881b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 -- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588362d0 msgr2=0x55fb58881b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588362d0 0x55fb58881b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 -- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb5883a320 msgr2=0x55fb58882070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb5883a320 0x55fb58882070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 -- 172.21.15.143:0/3737326017 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb58631490 con 0x55fb58883c90 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff203200640 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb588667a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e50 tx=0x7ff1fc0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24d324f2b124fa1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff1f3e00640 1 -- 172.21.15.143:0/3737326017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0f92c0 con 0x55fb58883c90 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff1f3e00640 1 -- 172.21.15.143:0/3737326017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc108070 con 0x55fb58883c90 2024-01-15T18:18:52.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff1f3e00640 1 -- 172.21.15.143:0/3737326017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0ed6f0 con 0x55fb58883c90 2024-01-15T18:18:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- 172.21.15.143:0/3737326017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 msgr2=0x55fb588667a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb588667a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009e50 tx=0x7ff1fc0ba040 comp rx=0 tx=0).stop 2024-01-15T18:18:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- 172.21.15.143:0/3737326017 shutdown_connections 2024-01-15T18:18:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb58883c90 0x55fb588667a0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb5883a320 0x55fb58882070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 --2- 172.21.15.143:0/3737326017 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb588362d0 0x55fb58881b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- 172.21.15.143:0/3737326017 >> 172.21.15.143:0/3737326017 conn(0x55fb5883ef10 msgr2=0x55fb588386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- 172.21.15.143:0/3737326017 shutdown_connections 2024-01-15T18:18:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- 172.21.15.143:0/3737326017 wait complete. 2024-01-15T18:18:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 -- start start 2024-01-15T18:18:52.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.247+0000 7ff20c92a000 1 Processor -- start 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 0x55fb58866bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb58895f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb58896490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb58643d60 con 0x55fb588362d0 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb5868e7c0 con 0x55fb58883c90 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb586258a0 con 0x55fb5883a320 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 10 client.0 _ll_register_callbacks cb 0x55fb586d8c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 0x55fb58866bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 0x55fb58866bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47294/0 (socket says 172.21.15.143:47294) 2024-01-15T18:18:52.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 -- 172.21.15.143:0/1128021989 learned_addr learned my addr 172.21.15.143:0/1128021989 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff201e00640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb58895f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff203200640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb58896490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 -- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 msgr2=0x55fb58895f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb58895f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 -- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 msgr2=0x55fb58896490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb58896490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 -- 172.21.15.143:0/1128021989 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20415a860 con 0x55fb588362d0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff202800640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 0x55fb58866bd0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4035fc0 tx=0x7ff1f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54113f28a04f5809 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 1 -- 172.21.15.143:0/1128021989 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4010770 con 0x55fb588362d0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 1 -- 172.21.15.143:0/1128021989 <== mon.0 v2:172.21.15.143:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff1f4021650 con 0x55fb588362d0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb58689110 con 0x55fb588362d0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 1 -- 172.21.15.143:0/1128021989 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1f4021b70 con 0x55fb588362d0 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 10 client.9180 fetch_fsmap learned FSMap version 72 2024-01-15T18:18:52.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x55fb58382cc0 con 0x55fb588362d0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 1 -- 172.21.15.143:0/1128021989 <== mon.0 v2:172.21.15.143:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb58689110 con 0x55fb588362d0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 1 -- 172.21.15.143:0/1128021989 <== mon.0 v2:172.21.15.143:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff1f4021cd0 con 0x55fb588362d0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 10 client.9180 fetch_fsmap finished waiting for FSMap version 72 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 10 client.9180 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 20 client.9180 populate_metadata read hostname 'smithi143' 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb588845b0 0x55fb588f5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 4 client.9180 mds_command: new command op to 9021 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 --> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb586b89a0 con 0x55fb588845b0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff201e00640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb588845b0 0x55fb588f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff201e00640 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb588845b0 0x55fb588f5650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42f08fd2735dcbd server_cookie=e320150c120ef494 in_seq=0 out_seq=0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.251+0000 7ff1ebe00640 10 client.9180 ms_handle_connect on v2:172.21.15.204:6833/3419861535 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff1ebe00640 1 -- 172.21.15.143:0/1128021989 <== mds.0 v2:172.21.15.204:6833/3419861535 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55fb586b89a0 con 0x55fb588845b0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff1ebe00640 10 client.9180 handle_command_reply: tid=0 2024-01-15T18:18:52.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 client.9180 shutdown 2024-01-15T18:18:52.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 msgr2=0x55fb58866bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 0x55fb58866bd0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4035fc0 tx=0x7ff1f4016000 comp rx=0 tx=0).stop 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 shutdown_connections 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:6833/3419861535,v1:172.21.15.204:6835/3419861535] conn(0x55fb588845b0 0x55fb588f5650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb58883c90 0x55fb58896490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x55fb5883a320 0x55fb58895f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff1ebe00640 0 client.9180 ms_handle_reset on v2:172.21.15.204:6833/3419861535 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 --2- 172.21.15.143:0/1128021989 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fb588362d0 0x55fb58866bd0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 >> 172.21.15.143:0/1128021989 conn(0x55fb5883ef10 msgr2=0x55fb58838f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 shutdown_connections 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 1 -- 172.21.15.143:0/1128021989 wait complete. 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff20c92a000 20 client.9180 trim_cache size 0 max 16384 2024-01-15T18:18:52.252 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "cluster_fsid": "10305fab-1773-4952-a8b6-aa6388f5534c", 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "whoami": 0, 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 9021, 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "want_state": "up:active", 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "up:active", 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs", 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "rank_uptime": 10.341124026999999, 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "mdsmap_epoch": 72, 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch": 111, 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_epoch_barrier": 111, 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout: "uptime": 15.22364044 2024-01-15T18:18:52.253 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1dc067a60 msgr2=0x7ff1dc069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1dc067a60 0x7ff1dc069f20 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40079c0 tx=0x7ff1f4008040 comp rx=0 tx=0).stop 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 msgr2=0x7ff204140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 0x7ff204140450 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc01a610 tx=0x7ff1fc019780 comp rx=0 tx=0).stop 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 shutdown_connections 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff1dc067a60 0x7ff1dc069f20 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2040ce690 0x7ff204140450 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff2040b3bf0 0x7ff204145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 --2- 172.21.15.143:0/943523073 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2040b1da0 0x7ff2041598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 >> 172.21.15.143:0/943523073 conn(0x7ff20406c3f0 msgr2=0x7ff204146850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 shutdown_connections 2024-01-15T18:18:52.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.255+0000 7ff209400640 1 -- 172.21.15.143:0/943523073 wait complete. 2024-01-15T18:18:52.270 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:18:52.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 Processor -- start 2024-01-15T18:18:52.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 -- start start 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67940cde50 0x7f67940d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67940ce7c0 con 0x7f67940cf410 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67940cef50 con 0x7f67940cde50 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6799e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67940ce290 con 0x7f67940c5600 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6792000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67940cde50 0x7f67940d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6792000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f67940cde50 0x7f67940d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47296/0 (socket says 172.21.15.143:47296) 2024-01-15T18:18:52.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.411+0000 7f6792000640 1 -- 172.21.15.143:0/50897360 learned_addr learned my addr 172.21.15.143:0/50897360 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6793400640 1 --2- 172.21.15.143:0/50897360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6792000640 1 -- 172.21.15.143:0/50897360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 msgr2=0x7f67940c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6792000640 1 --2- 172.21.15.143:0/50897360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6792000640 1 -- 172.21.15.143:0/50897360 >> v1:172.21.15.204:6789/0 conn(0x7f67940cf410 legacy=0x7f67940cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6792000640 1 -- 172.21.15.143:0/50897360 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67940ce4e0 con 0x7f67940cde50 2024-01-15T18:18:52.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6792000640 1 --2- 172.21.15.143:0/50897360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cde50 0x7f67940d1950 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f677c009870 tx=0x7f677c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1505bcfe0630d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6791600640 1 -- 172.21.15.143:0/50897360 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0a7070 con 0x7f67940cde50 2024-01-15T18:18:52.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6791600640 1 -- 172.21.15.143:0/50897360 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c0ab430 con 0x7f67940cde50 2024-01-15T18:18:52.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.415+0000 7f6791600640 1 -- 172.21.15.143:0/50897360 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0ab7c0 con 0x7f67940cde50 2024-01-15T18:18:52.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- 172.21.15.143:0/50897360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cde50 msgr2=0x7f67940d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 --2- 172.21.15.143:0/50897360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cde50 0x7f67940d1950 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f677c009870 tx=0x7f677c01c540 comp rx=0 tx=0).stop 2024-01-15T18:18:52.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- 172.21.15.143:0/50897360 shutdown_connections 2024-01-15T18:18:52.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 --2- 172.21.15.143:0/50897360 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 --2- 172.21.15.143:0/50897360 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cde50 0x7f67940d1950 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- 172.21.15.143:0/50897360 >> 172.21.15.143:0/50897360 conn(0x7f67940d0e30 msgr2=0x7f67940d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- 172.21.15.143:0/50897360 shutdown_connections 2024-01-15T18:18:52.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- 172.21.15.143:0/50897360 wait complete. 2024-01-15T18:18:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 Processor -- start 2024-01-15T18:18:52.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- start start 2024-01-15T18:18:52.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67940cde50 0x7f67940ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 0x7f6794157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67940cadd0 con 0x7f67940cf410 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67941581d0 con 0x7f67940cde50 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6799e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6794158630 con 0x7f67940c5600 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6793400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 0x7f6794157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6793400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 0x7f6794157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47310/0 (socket says 172.21.15.143:47310) 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6793400640 1 -- 172.21.15.143:0/598734359 learned_addr learned my addr 172.21.15.143:0/598734359 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6792a00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.419+0000 7f6792000640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67940cde50 0x7f67940ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6793400640 1 -- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 msgr2=0x7f67940c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6793400640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6793400640 1 -- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67940cde50 msgr2=0x7f67940ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6793400640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67940cde50 0x7f67940ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6793400640 1 -- 172.21.15.143:0/598734359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0094d0 con 0x7f67940cf410 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6792000640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67940cde50 0x7f67940ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6793400640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 0x7f6794157c90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f678c00efc0 tx=0x7f678c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c014480 con 0x7f67940cf410 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c00f040 con 0x7f67940cf410 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679415aa70 con 0x7f67940cf410 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c014820 con 0x7f67940cf410 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6794158f70 con 0x7f67940cf410 2024-01-15T18:18:52.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794067500 con 0x7f67940cf410 2024-01-15T18:18:52.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f678c014980 con 0x7f67940cf410 2024-01-15T18:18:52.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.423+0000 7f678b400640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f676c067a90 0x7f676c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:52.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.427+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f678c083e60 con 0x7f67940cf410 2024-01-15T18:18:52.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.427+0000 7f6792a00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f676c067a90 0x7f676c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:52.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.427+0000 7f6792a00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f676c067a90 0x7f676c069f50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f67940c5e10 tx=0x7f6784009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:52.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.427+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f678c0527a0 con 0x7f67940cf410 2024-01-15T18:18:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f67940b6cb0 con 0x7f67940cf410 2024-01-15T18:18:52.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f678b400640 1 -- 172.21.15.143:0/598734359 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f678c018070 con 0x7f67940cf410 2024-01-15T18:18:52.605 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:18:52.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f676c067a90 msgr2=0x7f676c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f676c067a90 0x7f676c069f50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f67940c5e10 tx=0x7f6784009040 comp rx=0 tx=0).stop 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 msgr2=0x7f6794157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 0x7f6794157c90 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f678c00efc0 tx=0x7f678c00c490 comp rx=0 tx=0).stop 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 shutdown_connections 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f676c067a90 0x7f676c069f50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f67940cf410 0x7f6794157c90 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67940cde50 0x7f67940ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 --2- 172.21.15.143:0/598734359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f67940c5600 0x7f67940c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 >> 172.21.15.143:0/598734359 conn(0x7f67940d0e30 msgr2=0x7f679406cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 shutdown_connections 2024-01-15T18:18:52.608 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:52.611+0000 7f6799e00640 1 -- 172.21.15.143:0/598734359 wait complete. 2024-01-15T18:18:52.627 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:18:52.627 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi143.front.sepia.ceph.com 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:18:52.628 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:18:52.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:52.628 DEBUG:teuthology.orchestra.run.smithi143:> ip addr 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 ::1/128 scope host 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:94 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f0 2024-01-15T18:18:52.638 INFO:teuthology.orchestra.run.smithi143.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:d9:3b:95 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: altname enp7s0f1 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:82 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0c:c4:7a:88:77:83 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.21.15.143/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft 41611sec preferred_lft 41611sec 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::ec4:7aff:fe88:7783/64 scope link 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 02:42:68:48:e7:0a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:18:52.639 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:18:52.640 INFO:teuthology.orchestra.run.smithi143.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:52.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:52.641 DEBUG:teuthology.orchestra.run.smithi143:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:18:52.701 INFO:teuthology.orchestra.run.smithi143.stdout:1 2024-01-15T18:18:52.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:52.703 DEBUG:teuthology.orchestra.run.smithi143:> route 2024-01-15T18:18:52.758 INFO:teuthology.orchestra.run.smithi143.stdout:Kernel IP routing table 2024-01-15T18:18:52.758 INFO:teuthology.orchestra.run.smithi143.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run.smithi143.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:18:52.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:52.759 DEBUG:teuthology.orchestra.run.smithi143:> (cd / && exec bash -c ' 2024-01-15T18:18:52.759 DEBUG:teuthology.orchestra.run.smithi143:> set -e 2024-01-15T18:18:52.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:18:52.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:18:52.760 DEBUG:teuthology.orchestra.run.smithi143:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:18:52.760 DEBUG:teuthology.orchestra.run.smithi143:> ') 2024-01-15T18:18:52.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:52.852 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:18:52.861 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:18:52.861 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:18:52.861 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:18:52.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:52.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link set brx.0 up 2024-01-15T18:18:52.923 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:18:52.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:18:52.923 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:52.971 INFO:teuthology.orchestra.run.smithi143.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:18:52.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:18:52.971 DEBUG:teuthology.orchestra.run.smithi143:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:53.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:18:53.088 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:18:53.141 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:18:53.142 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:18:53.143 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:18:53.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:53.144 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:18:53.195 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:18:53.195 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:18:53.238 DEBUG:teuthology.orchestra.run.smithi143:> sudo modprobe fuse 2024-01-15T18:18:53.258 DEBUG:teuthology.orchestra.run.smithi143:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/sys 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/proc 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/dev 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/pts 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/run 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/ 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/security 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/shm 2024-01-15T18:18:53.313 INFO:teuthology.orchestra.run.smithi143.stdout:/run/lock 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/cgroup 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/pstore 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/bpf 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/hugepages 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/dev/mqueue 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/debug 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/tracing 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/proc/fs/nfsd 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/fs/fuse/connections 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/sys/kernel/config 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/run/rpc_pipefs 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/run/user/1000 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph 2024-01-15T18:18:53.314 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:18:53.315 INFO:teuthology.orchestra.run.smithi143.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:18:53.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:53.316 DEBUG:teuthology.orchestra.run.smithi143:> ls /sys/fs/fuse/connections 2024-01-15T18:18:53.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:2024-01-15T18:18:53.335+0000 7f8c8b072580 -1 init, newargv = 0x55b9ac6123b0 newargc=15 2024-01-15T18:18:53.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[53117]: starting ceph client 2024-01-15T18:18:53.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[53117]: starting fuse 2024-01-15T18:18:53.340 INFO:teuthology.orchestra.run.smithi143.stdout:43 2024-01-15T18:18:53.340 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:18:53.340 DEBUG:teuthology.orchestra.run.smithi143:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:18:53.340 DEBUG:teuthology.orchestra.run.smithi143:> import glob 2024-01-15T18:18:53.340 DEBUG:teuthology.orchestra.run.smithi143:> import re 2024-01-15T18:18:53.340 DEBUG:teuthology.orchestra.run.smithi143:> import os 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> import subprocess 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> def _find_admin_socket(client_name): 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> files = glob.glob(asok_path) 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> # Given a non-glob path, it better be there 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> if "*" not in asok_path: 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> assert(len(files) == 1) 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> return files[0] 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> for f in files: 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:18:53.341 DEBUG:teuthology.orchestra.run.smithi143:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> contents = proc_f.read() 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> if mountpoint in contents: 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> return f 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> print(_find_admin_socket("client.0")) 2024-01-15T18:18:53.342 DEBUG:teuthology.orchestra.run.smithi143:> ' 2024-01-15T18:18:53.468 INFO:teuthology.orchestra.run.smithi143.stdout:/var/run/ceph/ceph-client.0.53117.asok 2024-01-15T18:18:53.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:18:53.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53117.asok 2024-01-15T18:18:53.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:53.479 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53117.asok status 2024-01-15T18:18:53.646 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:18:53.686 INFO:teuthology.orchestra.run.smithi143.stdout: "metadata": { 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "entity_id": "0", 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "hostname": "smithi143", 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "pid": "53117", 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "root": "/" 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_count": 0, 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "dentry_pinned_count": 0, 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "id": 9195, 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "inst": { 2024-01-15T18:18:53.687 INFO:teuthology.orchestra.run.smithi143.stdout: "name": { 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "client", 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "num": 9195 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3370562598 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": { 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "type": "v1", 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:18:53.688 INFO:teuthology.orchestra.run.smithi143.stdout: "nonce": 3370562598 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "inst_str": "client.9195 192.168.0.1:0/3370562598", 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "addr_str": "192.168.0.1:0/3370562598", 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "inode_count": 1, 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_epoch": 72, 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch": 111, 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "blocklisted": false, 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout: "fs_name": "cephfs" 2024-01-15T18:18:53.689 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:18:53.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:53.690 DEBUG:teuthology.orchestra.run.smithi143:> stat --file-system '--printf=%T 2024-01-15T18:18:53.690 DEBUG:teuthology.orchestra.run.smithi143:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:53.700 INFO:teuthology.orchestra.run.smithi143.stdout:fuseblk 2024-01-15T18:18:53.700 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:53.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:53.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:18:53.763 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:18:53.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 Processor -- start 2024-01-15T18:18:53.945 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 -- start start 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c06bfb0 0x7f946c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f946c0b0310 0x7f946c0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0cbff0 con 0x7f946c0cc9c0 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0d0c50 con 0x7f946c0b0310 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9473600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c06c8d0 con 0x7f946c06bfb0 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9471600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f946c0cc9c0 0x7f946c0cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46760/0 (socket says 172.21.15.143:46760) 2024-01-15T18:18:53.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9471600640 1 -- 172.21.15.143:0/2920833610 learned_addr learned my addr 172.21.15.143:0/2920833610 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9470c00640 1 --2- 172.21.15.143:0/2920833610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c06bfb0 0x7f946c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f946be00640 1 --2- 172.21.15.143:0/2920833610 >> v2:172.21.15.204:3300/0 conn(0x7f946c0b0310 0x7f946c0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f946b400640 1 -- 172.21.15.143:0/2920833610 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933307105 0 0) 0x7f946c0cbff0 con 0x7f946c0cc9c0 2024-01-15T18:18:53.947 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f946b400640 1 -- 172.21.15.143:0/2920833610 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f944c003530 con 0x7f946c0cc9c0 2024-01-15T18:18:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9470c00640 1 -- 172.21.15.143:0/2920833610 >> v2:172.21.15.204:3300/0 conn(0x7f946c0b0310 msgr2=0x7f946c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9470c00640 1 --2- 172.21.15.143:0/2920833610 >> v2:172.21.15.204:3300/0 conn(0x7f946c0b0310 0x7f946c0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9470c00640 1 -- 172.21.15.143:0/2920833610 >> v1:172.21.15.143:6789/0 conn(0x7f946c0cc9c0 legacy=0x7f946c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:53.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.951+0000 7f9470c00640 1 -- 172.21.15.143:0/2920833610 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0cb6c0 con 0x7f946c06bfb0 2024-01-15T18:18:53.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9470c00640 1 --2- 172.21.15.143:0/2920833610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c06bfb0 0x7f946c06c390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f945c0097c0 tx=0x7f945c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24aeffeca78ca8ca server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:53.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f946b400640 1 -- 172.21.15.143:0/2920833610 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0ac620 con 0x7f946c06bfb0 2024-01-15T18:18:53.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f946b400640 1 -- 172.21.15.143:0/2920833610 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c0ac780 con 0x7f946c06bfb0 2024-01-15T18:18:53.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f946b400640 1 -- 172.21.15.143:0/2920833610 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c0acb10 con 0x7f946c06bfb0 2024-01-15T18:18:53.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 -- 172.21.15.143:0/2920833610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c06bfb0 msgr2=0x7f946c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:53.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 --2- 172.21.15.143:0/2920833610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c06bfb0 0x7f946c06c390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f945c0097c0 tx=0x7f945c01c3c0 comp rx=0 tx=0).stop 2024-01-15T18:18:53.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 -- 172.21.15.143:0/2920833610 shutdown_connections 2024-01-15T18:18:53.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 --2- 172.21.15.143:0/2920833610 >> v2:172.21.15.204:3300/0 conn(0x7f946c0b0310 0x7f946c0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:53.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 --2- 172.21.15.143:0/2920833610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c06bfb0 0x7f946c06c390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:53.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 -- 172.21.15.143:0/2920833610 >> 172.21.15.143:0/2920833610 conn(0x7f946c0d0ed0 msgr2=0x7f946c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:53.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 -- 172.21.15.143:0/2920833610 shutdown_connections 2024-01-15T18:18:53.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.955+0000 7f9473600640 1 -- 172.21.15.143:0/2920833610 wait complete. 2024-01-15T18:18:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 Processor -- start 2024-01-15T18:18:53.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 -- start start 2024-01-15T18:18:53.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c06bfb0 0x7f946c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c0b0310 0x7f946c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 0x7f946c0c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c9af0 con 0x7f946c0cc9c0 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c8510 con 0x7f946c06bfb0 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0c87f0 con 0x7f946c0b0310 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 0x7f946c0c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 0x7f946c0c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47312/0 (socket says 172.21.15.143:47312) 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 -- 172.21.15.143:0/1396881172 learned_addr learned my addr 172.21.15.143:0/1396881172 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9470c00640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c06bfb0 0x7f946c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f946be00640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c0b0310 0x7f946c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 -- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c0b0310 msgr2=0x7f946c156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c0b0310 0x7f946c156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:53.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 -- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c06bfb0 msgr2=0x7f946c1558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c06bfb0 0x7f946c1558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 -- 172.21.15.143:0/1396881172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009420 con 0x7f946c0cc9c0 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9471600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 0x7f946c0c95b0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f946400cb40 tx=0x7f946400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400aca0 con 0x7f946c0cc9c0 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946400ae00 con 0x7f946c0cc9c0 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0c8ad0 con 0x7f946c0cc9c0 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464012620 con 0x7f946c0cc9c0 2024-01-15T18:18:53.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.959+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0c9090 con 0x7f946c0cc9c0 2024-01-15T18:18:53.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.963+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0c9280 con 0x7f946c0cc9c0 2024-01-15T18:18:53.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.963+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f9464007070 con 0x7f946c0cc9c0 2024-01-15T18:18:53.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.963+0000 7f9468c00640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9444067a60 0x7f9444069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:53.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.963+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9464083330 con 0x7f946c0cc9c0 2024-01-15T18:18:53.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.963+0000 7f9470c00640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9444067a60 0x7f9444069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:53.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.963+0000 7f9470c00640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9444067a60 0x7f9444069f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f945c0ae000 tx=0x7f945c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:53.963 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:53.967+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9464051d20 con 0x7f946c0cc9c0 2024-01-15T18:18:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.155+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f946c067500 con 0x7f946c0cc9c0 2024-01-15T18:18:54.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.155+0000 7f9468c00640 1 -- 172.21.15.143:0/1396881172 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9464017070 con 0x7f946c0cc9c0 2024-01-15T18:18:54.151 INFO:teuthology.orchestra.run.smithi143.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:18:54.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9444067a60 msgr2=0x7f9444069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9444067a60 0x7f9444069f20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f945c0ae000 tx=0x7f945c008040 comp rx=0 tx=0).stop 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 msgr2=0x7f946c0c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 0x7f946c0c95b0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f946400cb40 tx=0x7f946400a060 comp rx=0 tx=0).stop 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 shutdown_connections 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9444067a60 0x7f9444069f20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f946c0cc9c0 0x7f946c0c95b0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f946c0b0310 0x7f946c156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 --2- 172.21.15.143:0/1396881172 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f946c06bfb0 0x7f946c1558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 >> 172.21.15.143:0/1396881172 conn(0x7f946c0d0ed0 msgr2=0x7f946c06fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 shutdown_connections 2024-01-15T18:18:54.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:54.159+0000 7f9473600640 1 -- 172.21.15.143:0/1396881172 wait complete. 2024-01-15T18:18:54.174 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:18:54.174 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:18:54.174 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi204.front.sepia.ceph.com 2024-01-15T18:18:54.174 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:18:54.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:54.175 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:18:54.175 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:18:54.175 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:18:54.175 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:18:54.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.175 DEBUG:teuthology.orchestra.run.smithi204:> ip addr 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 ::1/128 scope host 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.185 INFO:teuthology.orchestra.run.smithi204.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f0 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:d9:36:2b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: altname enp7s0f1 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:aa brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0c:c4:7a:88:77:ab brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.21.15.204/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft 41603sec preferred_lft 41603sec 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::ec4:7aff:fe88:77ab/64 scope link 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 02:42:84:f8:53:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:54.186 INFO:teuthology.orchestra.run.smithi204.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-01-15T18:18:54.187 INFO:teuthology.orchestra.run.smithi204.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:18:54.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.188 DEBUG:teuthology.orchestra.run.smithi204:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:18:54.249 INFO:teuthology.orchestra.run.smithi204.stdout:1 2024-01-15T18:18:54.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.252 DEBUG:teuthology.orchestra.run.smithi204:> route 2024-01-15T18:18:54.305 INFO:teuthology.orchestra.run.smithi204.stdout:Kernel IP routing table 2024-01-15T18:18:54.305 INFO:teuthology.orchestra.run.smithi204.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:18:54.305 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run.smithi204.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:18:54.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.306 DEBUG:teuthology.orchestra.run.smithi204:> (cd / && exec bash -c ' 2024-01-15T18:18:54.306 DEBUG:teuthology.orchestra.run.smithi204:> set -e 2024-01-15T18:18:54.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:18:54.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:18:54.306 DEBUG:teuthology.orchestra.run.smithi204:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:18:54.306 DEBUG:teuthology.orchestra.run.smithi204:> ') 2024-01-15T18:18:54.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.400 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:18:54.409 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:18:54.409 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:18:54.409 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:18:54.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.409 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link set brx.0 up 2024-01-15T18:18:54.470 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:18:54.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:18:54.470 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:54.522 INFO:teuthology.orchestra.run.smithi204.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:18:54.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:18:54.522 DEBUG:teuthology.orchestra.run.smithi204:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:54.573 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:18:54.635 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:18:54.688 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:18:54.688 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:18:54.688 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:18:54.691 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:18:54.692 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:18:54.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:18:54.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:18:54.693 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:18:54.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.694 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:18:54.745 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:18:54.745 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:18:54.788 DEBUG:teuthology.orchestra.run.smithi204:> sudo modprobe fuse 2024-01-15T18:18:54.810 DEBUG:teuthology.orchestra.run.smithi204:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:18:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:/sys 2024-01-15T18:18:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:/proc 2024-01-15T18:18:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:/dev 2024-01-15T18:18:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/pts 2024-01-15T18:18:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:/run 2024-01-15T18:18:54.863 INFO:teuthology.orchestra.run.smithi204.stdout:/ 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/security 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/shm 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/run/lock 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/cgroup 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/pstore 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/bpf 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/hugepages 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/dev/mqueue 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/debug 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/tracing 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/proc/fs/nfsd 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/fs/fuse/connections 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/sys/kernel/config 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/run/credentials/systemd-sysusers.service 2024-01-15T18:18:54.864 INFO:teuthology.orchestra.run.smithi204.stdout:/run/rpc_pipefs 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/user/1000 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:18:54.865 INFO:teuthology.orchestra.run.smithi204.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:18:54.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:54.866 DEBUG:teuthology.orchestra.run.smithi204:> ls /sys/fs/fuse/connections 2024-01-15T18:18:54.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:2024-01-15T18:18:54.883+0000 7f7808074580 -1 init, newargv = 0x5602e3166190 newargc=15 2024-01-15T18:18:54.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34481]: starting ceph client 2024-01-15T18:18:54.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34481]: starting fuse 2024-01-15T18:18:54.897 INFO:teuthology.orchestra.run.smithi204.stdout:43 2024-01-15T18:18:54.897 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> import glob 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> import re 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> import os 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> import subprocess 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> def _find_admin_socket(client_name): 2024-01-15T18:18:54.897 DEBUG:teuthology.orchestra.run.smithi204:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> files = glob.glob(asok_path) 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> # Given a non-glob path, it better be there 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> if "*" not in asok_path: 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> assert(len(files) == 1) 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> return files[0] 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> for f in files: 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> contents = proc_f.read() 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> if mountpoint in contents: 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> return f 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> print(_find_admin_socket("client.1")) 2024-01-15T18:18:54.898 DEBUG:teuthology.orchestra.run.smithi204:> ' 2024-01-15T18:18:55.021 INFO:teuthology.orchestra.run.smithi204.stdout:/var/run/ceph/ceph-client.1.34481.asok 2024-01-15T18:18:55.024 INFO:teuthology.orchestra.run.smithi204.stderr:2024-01-15T18:18:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:18:55.032 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34481.asok 2024-01-15T18:18:55.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:55.032 DEBUG:teuthology.orchestra.run.smithi204:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34481.asok status 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout: "metadata": { 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout: "entity_id": "1", 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout: "hostname": "smithi204", 2024-01-15T18:18:55.200 INFO:teuthology.orchestra.run.smithi204.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "pid": "34481", 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "root": "/" 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_count": 0, 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "dentry_pinned_count": 0, 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "id": 8938, 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "inst": { 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "name": { 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "client", 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "num": 8938 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:18:55.201 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 2822520751 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: } 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": { 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "type": "v1", 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "nonce": 2822520751 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: }, 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "inst_str": "client.8938 192.168.0.1:0/2822520751", 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "addr_str": "192.168.0.1:0/2822520751", 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "inode_count": 1, 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "mds_epoch": 72, 2024-01-15T18:18:55.202 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch": 111, 2024-01-15T18:18:55.203 INFO:teuthology.orchestra.run.smithi204.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:18:55.203 INFO:teuthology.orchestra.run.smithi204.stdout: "blocklisted": false, 2024-01-15T18:18:55.203 INFO:teuthology.orchestra.run.smithi204.stdout: "fs_name": "cephfs" 2024-01-15T18:18:55.203 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-01-15T18:18:55.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:55.215 DEBUG:teuthology.orchestra.run.smithi204:> stat --file-system '--printf=%T 2024-01-15T18:18:55.215 DEBUG:teuthology.orchestra.run.smithi204:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:55.224 INFO:teuthology.orchestra.run.smithi204.stdout:fuseblk 2024-01-15T18:18:55.224 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:55.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:55.225 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:18:55.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:18:55.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 Processor -- start 2024-01-15T18:18:55.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 -- start start 2024-01-15T18:18:55.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f843c1607c0 0x7f843c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f843c0cbf50 0x7f843c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0cffd0 con 0x7f843c0cfb30 2024-01-15T18:18:55.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0d1ce0 con 0x7f843c0cbf50 2024-01-15T18:18:55.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8441a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0d02b0 con 0x7f843c1607c0 2024-01-15T18:18:55.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f843c0cbf50 0x7f843c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f843c0cfb30 0x7f843c06bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46762/0 (socket says 172.21.15.143:46762) 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843a000640 1 -- 172.21.15.143:0/1739382496 learned_addr learned my addr 172.21.15.143:0/1739382496 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843aa00640 1 --2- 172.21.15.143:0/1739382496 >> v2:172.21.15.204:3300/0 conn(0x7f843c1607c0 0x7f843c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843b400640 1 -- 172.21.15.143:0/1739382496 >> v2:172.21.15.204:3300/0 conn(0x7f843c1607c0 msgr2=0x7f843c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843b400640 1 --2- 172.21.15.143:0/1739382496 >> v2:172.21.15.204:3300/0 conn(0x7f843c1607c0 0x7f843c06b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843b400640 1 -- 172.21.15.143:0/1739382496 >> v1:172.21.15.143:6789/0 conn(0x7f843c0cfb30 legacy=0x7f843c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f843b400640 1 -- 172.21.15.143:0/1739382496 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0cb470 con 0x7f843c0cbf50 2024-01-15T18:18:55.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.443+0000 7f8439600640 1 -- 172.21.15.143:0/1739382496 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655612954 0 0) 0x7f843c0cffd0 con 0x7f843c0cfb30 2024-01-15T18:18:55.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f843b400640 1 --2- 172.21.15.143:0/1739382496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c0cbf50 0x7f843c0cc3c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f843c071a00 tx=0x7f843401f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad09a5edf4b57f7b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:55.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8439600640 1 -- 172.21.15.143:0/1739382496 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340b0440 con 0x7f843c0cbf50 2024-01-15T18:18:55.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8439600640 1 -- 172.21.15.143:0/1739382496 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84340b05a0 con 0x7f843c0cbf50 2024-01-15T18:18:55.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8439600640 1 -- 172.21.15.143:0/1739382496 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340b0930 con 0x7f843c0cbf50 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 -- 172.21.15.143:0/1739382496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c0cbf50 msgr2=0x7f843c0cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 --2- 172.21.15.143:0/1739382496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c0cbf50 0x7f843c0cc3c0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f843c071a00 tx=0x7f843401f0f0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 -- 172.21.15.143:0/1739382496 shutdown_connections 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 --2- 172.21.15.143:0/1739382496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c0cbf50 0x7f843c0cc3c0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 --2- 172.21.15.143:0/1739382496 >> v2:172.21.15.204:3300/0 conn(0x7f843c1607c0 0x7f843c06b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 -- 172.21.15.143:0/1739382496 >> 172.21.15.143:0/1739382496 conn(0x7f843c0d0e20 msgr2=0x7f843c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 -- 172.21.15.143:0/1739382496 shutdown_connections 2024-01-15T18:18:55.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.447+0000 7f8441a00640 1 -- 172.21.15.143:0/1739382496 wait complete. 2024-01-15T18:18:55.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 Processor -- start 2024-01-15T18:18:55.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- start start 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 0x7f843c1f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f843c0cfb30 0x7f843c146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c1607c0 0x7f843c0c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c159d90 con 0x7f843c1607c0 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c146770 con 0x7f843c0cbf50 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1469f0 con 0x7f843c0cfb30 2024-01-15T18:18:55.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 0x7f843c1f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 0x7f843c1f32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:58486/0 (socket says 172.21.15.143:58486) 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c1607c0 0x7f843c0c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 -- 172.21.15.143:0/3732889561 learned_addr learned my addr 172.21.15.143:0/3732889561 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843a000640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f843c0cfb30 0x7f843c146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 -- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f843c0cfb30 msgr2=0x7f843c146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f843c0cfb30 0x7f843c146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 -- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c1607c0 msgr2=0x7f843c0c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c1607c0 0x7f843c0c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 -- 172.21.15.143:0/3732889561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434009a70 con 0x7f843c0cbf50 2024-01-15T18:18:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f843aa00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 0x7f843c1f32e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f842c00e870 tx=0x7f842c00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c009510 con 0x7f843c0cbf50 2024-01-15T18:18:55.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c00eec0 con 0x7f843c0cbf50 2024-01-15T18:18:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c146cd0 con 0x7f843c0cbf50 2024-01-15T18:18:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00cb40 con 0x7f843c0cbf50 2024-01-15T18:18:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c0c9480 con 0x7f843c0cbf50 2024-01-15T18:18:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.451+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c06d4b0 con 0x7f843c0cbf50 2024-01-15T18:18:55.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.455+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f842c00cca0 con 0x7f843c0cbf50 2024-01-15T18:18:55.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.455+0000 7f8432a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8414067a90 0x7f8414069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.455+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f842c083bc0 con 0x7f843c0cbf50 2024-01-15T18:18:55.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.455+0000 7f843a000640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8414067a90 0x7f8414069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.455+0000 7f843a000640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8414067a90 0x7f8414069f50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cf750 tx=0x7f8424007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:55.453 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.459+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f842c0525b0 con 0x7f843c0cbf50 2024-01-15T18:18:55.636 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.643+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f843c067500 con 0x7f843c0cbf50 2024-01-15T18:18:55.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.839+0000 7f8432a00640 1 -- 172.21.15.143:0/3732889561 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 136+0+0 (secure 0 0 0) 0x7f842c0170c0 con 0x7f843c0cbf50 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8414067a90 msgr2=0x7f8414069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8414067a90 0x7f8414069f50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f843c0cf750 tx=0x7f8424007d60 comp rx=0 tx=0).stop 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 msgr2=0x7f843c1f32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 0x7f843c1f32e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f842c00e870 tx=0x7f842c00ec70 comp rx=0 tx=0).stop 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 shutdown_connections 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8414067a90 0x7f8414069f50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f843c1607c0 0x7f843c0c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f843c0cfb30 0x7f843c146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 --2- 172.21.15.143:0/3732889561 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f843c0cbf50 0x7f843c1f32e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:55.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 >> 172.21.15.143:0/3732889561 conn(0x7f843c0d0e20 msgr2=0x7f843c06b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:55.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 shutdown_connections 2024-01-15T18:18:55.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.843+0000 7f8441a00640 1 -- 172.21.15.143:0/3732889561 wait complete. 2024-01-15T18:18:55.858 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:18:55.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.999+0000 7fc15aa00640 1 Processor -- start 2024-01-15T18:18:55.995 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:55.999+0000 7fc15aa00640 1 -- start start 2024-01-15T18:18:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc15aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc154071bb0 0x7fc154072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc15aa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1540ceb50 con 0x7fc154143780 2024-01-15T18:18:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc15aa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15415c110 con 0x7fc15415c5a0 2024-01-15T18:18:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc15aa00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc154072570 con 0x7fc154071bb0 2024-01-15T18:18:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc154071bb0 0x7fc154072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:55.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc154143780 0x7fc1540cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46778/0 (socket says 172.21.15.143:46778) 2024-01-15T18:18:55.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153e00640 1 -- 172.21.15.143:0/511629489 learned_addr learned my addr 172.21.15.143:0/511629489 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:55.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153400640 1 -- 172.21.15.143:0/511629489 >> v1:172.21.15.204:6789/0 conn(0x7fc15415c5a0 legacy=0x7fc15415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153400640 1 -- 172.21.15.143:0/511629489 >> v1:172.21.15.143:6789/0 conn(0x7fc154143780 legacy=0x7fc1540cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:55.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153400640 1 -- 172.21.15.143:0/511629489 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15415b7e0 con 0x7fc154071bb0 2024-01-15T18:18:55.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc152a00640 1 -- 172.21.15.143:0/511629489 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726737888 0 0) 0x7fc144001120 con 0x7fc154143780 2024-01-15T18:18:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.003+0000 7fc153400640 1 --2- 172.21.15.143:0/511629489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154072030 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0097c0 tx=0x7fc13c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4a83d84f0244c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc152a00640 1 -- 172.21.15.143:0/511629489 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0ad680 con 0x7fc154071bb0 2024-01-15T18:18:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc152a00640 1 -- 172.21.15.143:0/511629489 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c0ad7e0 con 0x7fc154071bb0 2024-01-15T18:18:56.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc152a00640 1 -- 172.21.15.143:0/511629489 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0adb70 con 0x7fc154071bb0 2024-01-15T18:18:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 -- 172.21.15.143:0/511629489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 msgr2=0x7fc154072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 --2- 172.21.15.143:0/511629489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154072030 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0097c0 tx=0x7fc13c01c480 comp rx=0 tx=0).stop 2024-01-15T18:18:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 -- 172.21.15.143:0/511629489 shutdown_connections 2024-01-15T18:18:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 --2- 172.21.15.143:0/511629489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154072030 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 -- 172.21.15.143:0/511629489 >> 172.21.15.143:0/511629489 conn(0x7fc15406c3f0 msgr2=0x7fc15406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 -- 172.21.15.143:0/511629489 shutdown_connections 2024-01-15T18:18:56.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 -- 172.21.15.143:0/511629489 wait complete. 2024-01-15T18:18:56.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 Processor -- start 2024-01-15T18:18:56.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 -- start start 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.007+0000 7fc15aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc154143780 0x7fc154131190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15415c5a0 0x7fc1541587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154138e10 con 0x7fc154071bb0 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154071570 con 0x7fc15415c5a0 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc154138570 con 0x7fc154143780 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47356/0 (socket says 172.21.15.143:47356) 2024-01-15T18:18:56.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 -- 172.21.15.143:0/1221202368 learned_addr learned my addr 172.21.15.143:0/1221202368 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc158a00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15415c5a0 0x7fc1541587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153400640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc154143780 0x7fc154131190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 -- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc154143780 msgr2=0x7fc154131190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc154143780 0x7fc154131190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 -- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15415c5a0 msgr2=0x7fc1541587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15415c5a0 0x7fc1541587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 -- 172.21.15.143:0/1221202368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c009420 con 0x7fc154071bb0 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc153e00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154159370 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc14400c780 tx=0x7fc14400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1440072d0 con 0x7fc154071bb0 2024-01-15T18:18:56.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc144007430 con 0x7fc154071bb0 2024-01-15T18:18:56.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc144001d20 con 0x7fc154071bb0 2024-01-15T18:18:56.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc154138850 con 0x7fc154071bb0 2024-01-15T18:18:56.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc154071190 con 0x7fc154071bb0 2024-01-15T18:18:56.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc154067500 con 0x7fc154071bb0 2024-01-15T18:18:56.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.011+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fc1440047f0 con 0x7fc154071bb0 2024-01-15T18:18:56.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.015+0000 7fc14be00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc12c067a60 0x7fc12c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.015+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc144082450 con 0x7fc154071bb0 2024-01-15T18:18:56.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.015+0000 7fc153400640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc12c067a60 0x7fc12c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.015+0000 7fc153400640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc12c067a60 0x7fc12c069f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0af000 tx=0x7fc13c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.019+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc144050e40 con 0x7fc154071bb0 2024-01-15T18:18:56.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.187+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1541386e0 con 0x7fc12c067a60 2024-01-15T18:18:56.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.187+0000 7fc14be00640 1 -- 172.21.15.143:0/1221202368 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fc1541386e0 con 0x7fc12c067a60 2024-01-15T18:18:56.183 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:18:56.183 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:18:56.183 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:18:56.183 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:18:56.183 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:18:56.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc12c067a60 msgr2=0x7fc12c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc12c067a60 0x7fc12c069f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0af000 tx=0x7fc13c008040 comp rx=0 tx=0).stop 2024-01-15T18:18:56.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 msgr2=0x7fc154159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154159370 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc14400c780 tx=0x7fc14400cc40 comp rx=0 tx=0).stop 2024-01-15T18:18:56.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 shutdown_connections 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc12c067a60 0x7fc12c069f20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc15415c5a0 0x7fc1541587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc154143780 0x7fc154131190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 --2- 172.21.15.143:0/1221202368 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc154071bb0 0x7fc154159370 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 >> 172.21.15.143:0/1221202368 conn(0x7fc15406c3f0 msgr2=0x7fc15413a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 shutdown_connections 2024-01-15T18:18:56.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.191+0000 7fc15aa00640 1 -- 172.21.15.143:0/1221202368 wait complete. 2024-01-15T18:18:56.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:18:56.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 Processor -- start 2024-01-15T18:18:56.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 -- start start 2024-01-15T18:18:56.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fc4071bb0 0x7f0fc4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.346 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc40ceb50 con 0x7f0fc4143780 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc4072570 con 0x7f0fc4071bb0 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fca000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc415c110 con 0x7f0fc415c5a0 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fc4071bb0 0x7f0fc4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0fc4071bb0 0x7f0fc4072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47368/0 (socket says 172.21.15.143:47368) 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 -- 172.21.15.143:0/1307219493 learned_addr learned my addr 172.21.15.143:0/1307219493 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc3e00640 1 --2- 172.21.15.143:0/1307219493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 -- 172.21.15.143:0/1307219493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 msgr2=0x7f0fc415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 --2- 172.21.15.143:0/1307219493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 -- 172.21.15.143:0/1307219493 >> v1:172.21.15.204:6789/0 conn(0x7f0fc4143780 legacy=0x7f0fc40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.351+0000 7f0fc2a00640 1 -- 172.21.15.143:0/1307219493 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc415b7e0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fc2a00640 1 --2- 172.21.15.143:0/1307219493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4072030 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009870 tx=0x7f0fac01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fc96c8ba3042a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fc2000640 1 -- 172.21.15.143:0/1307219493 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0a9400 con 0x7f0fc4071bb0 2024-01-15T18:18:56.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fc2000640 1 -- 172.21.15.143:0/1307219493 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac0a9560 con 0x7f0fc4071bb0 2024-01-15T18:18:56.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fc2000640 1 -- 172.21.15.143:0/1307219493 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0a98f0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 -- 172.21.15.143:0/1307219493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 msgr2=0x7f0fc4072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 --2- 172.21.15.143:0/1307219493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4072030 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009870 tx=0x7f0fac01c540 comp rx=0 tx=0).stop 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 -- 172.21.15.143:0/1307219493 shutdown_connections 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 --2- 172.21.15.143:0/1307219493 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 --2- 172.21.15.143:0/1307219493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4072030 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 -- 172.21.15.143:0/1307219493 >> 172.21.15.143:0/1307219493 conn(0x7f0fc406c3f0 msgr2=0x7f0fc406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 -- 172.21.15.143:0/1307219493 shutdown_connections 2024-01-15T18:18:56.351 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.355+0000 7f0fca000640 1 -- 172.21.15.143:0/1307219493 wait complete. 2024-01-15T18:18:56.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 Processor -- start 2024-01-15T18:18:56.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- start start 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4134b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fc4143780 0x7f0fc4158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc4135ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4136580 con 0x7f0fc4071bb0 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4134030 con 0x7f0fc4143780 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4134310 con 0x7f0fc415c5a0 2024-01-15T18:18:56.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4134b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4134b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47376/0 (socket says 172.21.15.143:47376) 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 -- 172.21.15.143:0/4029182790 learned_addr learned my addr 172.21.15.143:0/4029182790 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc2a00640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fc4143780 0x7f0fc4158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3e00640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc4135ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 -- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 msgr2=0x7f0fc4135ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc4135ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 -- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fc4143780 msgr2=0x7f0fc4158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fc4143780 0x7f0fc4158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 -- 172.21.15.143:0/4029182790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0094d0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fc3400640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4134b10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c450 tx=0x7f0fb400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40052d0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb4005450 con 0x7f0fc4071bb0 2024-01-15T18:18:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc41345f0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4012870 con 0x7f0fc4071bb0 2024-01-15T18:18:56.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc4159000 con 0x7f0fc4071bb0 2024-01-15T18:18:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.359+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc4133e90 con 0x7f0fc4071bb0 2024-01-15T18:18:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.363+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f0fb40129d0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.363+0000 7f0fbb400640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f9c067a60 0x7f0f9c069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.363+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0fb4082bf0 con 0x7f0fc4071bb0 2024-01-15T18:18:56.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.363+0000 7f0fc2a00640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f9c067a60 0x7f0f9c069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.363+0000 7f0fc2a00640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f9c067a60 0x7f0f9c069f20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0ae000 tx=0x7f0fac0b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.367+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0fc4133e90 con 0x7f0fc4071bb0 2024-01-15T18:18:56.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.515+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0fc40ceb50 con 0x7f0fc4071bb0 2024-01-15T18:18:56.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.515+0000 7f0fbb400640 1 -- 172.21.15.143:0/4029182790 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0fc40ceb50 con 0x7f0fc4071bb0 2024-01-15T18:18:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f9c067a60 msgr2=0x7f0f9c069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f9c067a60 0x7f0f9c069f20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0ae000 tx=0x7f0fac0b3040 comp rx=0 tx=0).stop 2024-01-15T18:18:56.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 msgr2=0x7f0fc4134b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4134b10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c450 tx=0x7f0fb400c910 comp rx=0 tx=0).stop 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 shutdown_connections 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0f9c067a60 0x7f0f9c069f20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0fc415c5a0 0x7f0fc4135ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0fc4143780 0x7f0fc4158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 --2- 172.21.15.143:0/4029182790 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fc4071bb0 0x7f0fc4134b10 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 >> 172.21.15.143:0/4029182790 conn(0x7f0fc406c3f0 msgr2=0x7f0fc4071330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 shutdown_connections 2024-01-15T18:18:56.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.519+0000 7f0fca000640 1 -- 172.21.15.143:0/4029182790 wait complete. 2024-01-15T18:18:56.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:18:56.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f807d200640 1 Processor -- start 2024-01-15T18:18:56.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f807d200640 1 -- start start 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f807d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807806aba0 0x7f80780aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f807d200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8078160eb0 con 0x7f807815d130 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f807d200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8078161190 con 0x7f807815d510 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f807d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8078161470 con 0x7f807806aba0 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8075e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f807815d130 0x7f807806a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46794/0 (socket says 172.21.15.143:46794) 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8075e00640 1 -- 172.21.15.143:0/3165706251 learned_addr learned my addr 172.21.15.143:0/3165706251 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8075400640 1 --2- 172.21.15.143:0/3165706251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807806aba0 0x7f80780aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917509458 0 0) 0x7f8078160eb0 con 0x7f807815d130 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8058003530 con 0x7f807815d130 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901302187 0 0) 0x7f8078161190 con 0x7f807815d510 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8078160eb0 con 0x7f807815d510 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2741636654 0 0) 0x7f8058003530 con 0x7f807815d130 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8075400640 1 -- 172.21.15.143:0/3165706251 >> v1:172.21.15.204:6789/0 conn(0x7f807815d510 legacy=0x7f80780b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8075400640 1 -- 172.21.15.143:0/3165706251 >> v1:172.21.15.143:6789/0 conn(0x7f807815d130 legacy=0x7f807806a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.679+0000 7f8075400640 1 -- 172.21.15.143:0/3165706251 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8078161750 con 0x7f807806aba0 2024-01-15T18:18:56.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f8075400640 1 --2- 172.21.15.143:0/3165706251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807806aba0 0x7f80780aeed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80600097f0 tx=0x7f806001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40922d3c7768102c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068001680 con 0x7f807806aba0 2024-01-15T18:18:56.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80600a7050 con 0x7f807806aba0 2024-01-15T18:18:56.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f8074a00640 1 -- 172.21.15.143:0/3165706251 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80600ab600 con 0x7f807806aba0 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f807d200640 1 -- 172.21.15.143:0/3165706251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807806aba0 msgr2=0x7f80780aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f807d200640 1 --2- 172.21.15.143:0/3165706251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807806aba0 0x7f80780aeed0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80600097f0 tx=0x7f806001c3e0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f807d200640 1 -- 172.21.15.143:0/3165706251 shutdown_connections 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f807d200640 1 --2- 172.21.15.143:0/3165706251 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807806aba0 0x7f80780aeed0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f807d200640 1 -- 172.21.15.143:0/3165706251 >> 172.21.15.143:0/3165706251 conn(0x7f80780b1b90 msgr2=0x7f80780b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.683+0000 7f807d200640 1 -- 172.21.15.143:0/3165706251 shutdown_connections 2024-01-15T18:18:56.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 -- 172.21.15.143:0/3165706251 wait complete. 2024-01-15T18:18:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 Processor -- start 2024-01-15T18:18:56.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 -- start start 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f807806aba0 0x7f8078153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807815d130 0x7f80781454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 0x7f8078145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807815c0a0 con 0x7f807815d510 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807815bae0 con 0x7f807806aba0 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f807d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807815bdc0 con 0x7f807815d130 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 0x7f8078145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 0x7f8078145a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47378/0 (socket says 172.21.15.143:47378) 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 -- 172.21.15.143:0/3113909680 learned_addr learned my addr 172.21.15.143:0/3113909680 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8075e00640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f807806aba0 0x7f8078153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8075400640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807815d130 0x7f80781454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 -- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807815d130 msgr2=0x7f80781454c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807815d130 0x7f80781454c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 -- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f807806aba0 msgr2=0x7f8078153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f807806aba0 0x7f8078153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 -- 172.21.15.143:0/3113909680 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009420 con 0x7f807815d510 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8075400640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807815d130 0x7f80781454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f8076800640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 0x7f8078145a00 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f807000c730 tx=0x7f807000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070003b80 con 0x7f807815d510 2024-01-15T18:18:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070003ce0 con 0x7f807815d510 2024-01-15T18:18:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.687+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000aa90 con 0x7f807815d510 2024-01-15T18:18:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807815e260 con 0x7f807815d510 2024-01-15T18:18:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8078153db0 con 0x7f807815d510 2024-01-15T18:18:56.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8078153fa0 con 0x7f807815d510 2024-01-15T18:18:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f807000abf0 con 0x7f807815d510 2024-01-15T18:18:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f806e000640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8050067a60 0x7f8050069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8070083080 con 0x7f807815d510 2024-01-15T18:18:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f8075e00640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8050067a60 0x7f8050069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:56.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.691+0000 7f8075e00640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8050067a60 0x7f8050069f20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f80680011f0 tx=0x7f80680016a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:56.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.695+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8078153fa0 con 0x7f807815d510 2024-01-15T18:18:56.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.851+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7f8078153fa0 con 0x7f807815d510 2024-01-15T18:18:56.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.855+0000 7f806e000640 1 -- 172.21.15.143:0/3113909680 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v41)=0 v41) v1 ==== 131+0+0 (secure 0 0 0) 0x7f8078153fa0 con 0x7f807815d510 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8050067a60 msgr2=0x7f8050069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8050067a60 0x7f8050069f20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f80680011f0 tx=0x7f80680016a0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 msgr2=0x7f8078145a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 0x7f8078145a00 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f807000c730 tx=0x7f807000cbf0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 shutdown_connections 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8050067a60 0x7f8050069f20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f807815d510 0x7f8078145a00 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f807815d130 0x7f80781454c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 --2- 172.21.15.143:0/3113909680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f807806aba0 0x7f8078153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 >> 172.21.15.143:0/3113909680 conn(0x7f80780b1b90 msgr2=0x7f8078156960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 shutdown_connections 2024-01-15T18:18:56.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:56.859+0000 7f807d200640 1 -- 172.21.15.143:0/3113909680 wait complete. 2024-01-15T18:18:56.874 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 120 2024-01-15T18:18:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdf6600640 1 Processor -- start 2024-01-15T18:18:57.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdf6600640 1 -- start start 2024-01-15T18:18:57.018 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdf6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf0071bb0 0x7fbdf0072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdf6600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf00ceb50 con 0x7fbdf0143780 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdf6600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf015c110 con 0x7fbdf015c5a0 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdf6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0072570 con 0x7fbdf0071bb0 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdef400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbdf0143780 0x7fbdf00cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46804/0 (socket says 172.21.15.143:46804) 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdef400640 1 -- 172.21.15.143:0/3691222858 learned_addr learned my addr 172.21.15.143:0/3691222858 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdeea00640 1 --2- 172.21.15.143:0/3691222858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf0071bb0 0x7fbdf0072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183190597 0 0) 0x7fbde0001120 con 0x7fbdf0143780 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd0003530 con 0x7fbdf0143780 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419944643 0 0) 0x7fbdf015c110 con 0x7fbdf015c5a0 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde0001120 con 0x7fbdf015c5a0 2024-01-15T18:18:57.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdeea00640 1 -- 172.21.15.143:0/3691222858 >> v1:172.21.15.204:6789/0 conn(0x7fbdf015c5a0 legacy=0x7fbdf015ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdeea00640 1 -- 172.21.15.143:0/3691222858 >> v1:172.21.15.143:6789/0 conn(0x7fbdf0143780 legacy=0x7fbdf00cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.023+0000 7fbdeea00640 1 -- 172.21.15.143:0/3691222858 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf015b7e0 con 0x7fbdf0071bb0 2024-01-15T18:18:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdeea00640 1 --2- 172.21.15.143:0/3691222858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf0071bb0 0x7fbdf0072030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80097f0 tx=0x7fbdd801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0fa74ab32530f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80aa620 con 0x7fbdf0071bb0 2024-01-15T18:18:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80aa780 con 0x7fbdf0071bb0 2024-01-15T18:18:57.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdee000640 1 -- 172.21.15.143:0/3691222858 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80aaaf0 con 0x7fbdf0071bb0 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 -- 172.21.15.143:0/3691222858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf0071bb0 msgr2=0x7fbdf0072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 --2- 172.21.15.143:0/3691222858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf0071bb0 0x7fbdf0072030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80097f0 tx=0x7fbdd801c4c0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 -- 172.21.15.143:0/3691222858 shutdown_connections 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 --2- 172.21.15.143:0/3691222858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf0071bb0 0x7fbdf0072030 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 -- 172.21.15.143:0/3691222858 >> 172.21.15.143:0/3691222858 conn(0x7fbdf006c3f0 msgr2=0x7fbdf006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 -- 172.21.15.143:0/3691222858 shutdown_connections 2024-01-15T18:18:57.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.027+0000 7fbdf6600640 1 -- 172.21.15.143:0/3691222858 wait complete. 2024-01-15T18:18:57.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 Processor -- start 2024-01-15T18:18:57.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 -- start start 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0071bb0 0x7fbdf0137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 0x7fbdf0140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf015c5a0 0x7fbdf0138db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf01414f0 con 0x7fbdf0143780 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0137d10 con 0x7fbdf0071bb0 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0137ff0 con 0x7fbdf015c5a0 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 0x7fbdf0140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 0x7fbdf0140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47388/0 (socket says 172.21.15.143:47388) 2024-01-15T18:18:57.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 -- 172.21.15.143:0/1911990794 learned_addr learned my addr 172.21.15.143:0/1911990794 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdef400640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0071bb0 0x7fbdf0137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdefe00640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf015c5a0 0x7fbdf0138db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 -- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf015c5a0 msgr2=0x7fbdf0138db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf015c5a0 0x7fbdf0138db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 -- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0071bb0 msgr2=0x7fbdf0137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0071bb0 0x7fbdf0137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 -- 172.21.15.143:0/1911990794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8009420 con 0x7fbdf0143780 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdeea00640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 0x7fbdf0140fb0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80097f0 tx=0x7fbdd801ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80aad00 con 0x7fbdf0143780 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd80aae60 con 0x7fbdf0143780 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80be610 con 0x7fbdf0143780 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf01382d0 con 0x7fbdf0143780 2024-01-15T18:18:57.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.031+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf0138890 con 0x7fbdf0143780 2024-01-15T18:18:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.035+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf0138a80 con 0x7fbdf0143780 2024-01-15T18:18:57.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.035+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7fbdd80be770 con 0x7fbdf0143780 2024-01-15T18:18:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.035+0000 7fbde7400640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc8067a60 0x7fbdc8069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.035+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbdd811d2a0 con 0x7fbdf0143780 2024-01-15T18:18:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.035+0000 7fbdef400640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc8067a60 0x7fbdc8069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.035+0000 7fbdef400640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc8067a60 0x7fbdc8069f20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbde0007fd0 tx=0x7fbde0007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.039+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbdd80ebfe0 con 0x7fbdf0143780 2024-01-15T18:18:57.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.191+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7fbdf0067500 con 0x7fbdf0143780 2024-01-15T18:18:57.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.195+0000 7fbde7400640 1 -- 172.21.15.143:0/1911990794 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v42)=0 v42) v1 ==== 137+0+0 (secure 0 0 0) 0x7fbdd80ef600 con 0x7fbdf0143780 2024-01-15T18:18:57.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc8067a60 msgr2=0x7fbdc8069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc8067a60 0x7fbdc8069f20 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbde0007fd0 tx=0x7fbde0007d60 comp rx=0 tx=0).stop 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 msgr2=0x7fbdf0140fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 0x7fbdf0140fb0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80097f0 tx=0x7fbdd801ad00 comp rx=0 tx=0).stop 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 shutdown_connections 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbdc8067a60 0x7fbdc8069f20 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbdf015c5a0 0x7fbdf0138db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbdf0143780 0x7fbdf0140fb0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 --2- 172.21.15.143:0/1911990794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf0071bb0 0x7fbdf0137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 >> 172.21.15.143:0/1911990794 conn(0x7fbdf006c3f0 msgr2=0x7fbdf00afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 shutdown_connections 2024-01-15T18:18:57.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.199+0000 7fbdf6600640 1 -- 172.21.15.143:0/1911990794 wait complete. 2024-01-15T18:18:57.216 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:18:57.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.359+0000 7f2f64a00640 1 Processor -- start 2024-01-15T18:18:57.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.359+0000 7f2f64a00640 1 -- start start 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f64a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c1607c0 0x7f2f5c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f64a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f5c0cfb30 0x7f2f5c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f64a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2f5c06c230 0x7f2f5c0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f64a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5c06c670 con 0x7f2f5c0cfb30 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f64a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5c0d1ce0 con 0x7f2f5c06c230 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f64a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0cffd0 con 0x7f2f5c1607c0 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f5c0cfb30 0x7f2f5c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f5c0cfb30 0x7f2f5c06bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56902/0 (socket says 172.21.15.143:56902) 2024-01-15T18:18:57.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 -- 172.21.15.143:0/2775809423 learned_addr learned my addr 172.21.15.143:0/2775809423 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f62a00640 1 --2- 172.21.15.143:0/2775809423 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c06c230 0x7f2f5c0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f62000640 1 --2- 172.21.15.143:0/2775809423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c1607c0 0x7f2f5c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 -- 172.21.15.143:0/2775809423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c1607c0 msgr2=0x7f2f5c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 --2- 172.21.15.143:0/2775809423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c1607c0 0x7f2f5c06b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 -- 172.21.15.143:0/2775809423 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c06c230 msgr2=0x7f2f5c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 --2- 172.21.15.143:0/2775809423 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c06c230 0x7f2f5c0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.363+0000 7f2f61600640 1 -- 172.21.15.143:0/2775809423 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0d02b0 con 0x7f2f5c0cfb30 2024-01-15T18:18:57.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f61600640 1 --2- 172.21.15.143:0/2775809423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c06bcf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0d0590 tx=0x7f2f4401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f43c58517f82d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f60c00640 1 -- 172.21.15.143:0/2775809423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440a89b0 con 0x7f2f5c0cfb30 2024-01-15T18:18:57.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f60c00640 1 -- 172.21.15.143:0/2775809423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440a8b10 con 0x7f2f5c0cfb30 2024-01-15T18:18:57.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f60c00640 1 -- 172.21.15.143:0/2775809423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440a8ea0 con 0x7f2f5c0cfb30 2024-01-15T18:18:57.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 -- 172.21.15.143:0/2775809423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c0cfb30 msgr2=0x7f2f5c06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 --2- 172.21.15.143:0/2775809423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c06bcf0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0d0590 tx=0x7f2f4401ecf0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 -- 172.21.15.143:0/2775809423 shutdown_connections 2024-01-15T18:18:57.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 --2- 172.21.15.143:0/2775809423 >> v2:172.21.15.204:3300/0 conn(0x7f2f5c06c230 0x7f2f5c0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 --2- 172.21.15.143:0/2775809423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c06bcf0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 --2- 172.21.15.143:0/2775809423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c1607c0 0x7f2f5c06b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 -- 172.21.15.143:0/2775809423 >> 172.21.15.143:0/2775809423 conn(0x7f2f5c0d0e20 msgr2=0x7f2f5c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:57.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 -- 172.21.15.143:0/2775809423 shutdown_connections 2024-01-15T18:18:57.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 -- 172.21.15.143:0/2775809423 wait complete. 2024-01-15T18:18:57.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 Processor -- start 2024-01-15T18:18:57.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.367+0000 7f2f64a00640 1 -- start start 2024-01-15T18:18:57.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c06c230 0x7f2f5c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 0x7f2f5c0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c157680 con 0x7f2f5c1607c0 2024-01-15T18:18:57.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0c8580 con 0x7f2f5c0cfb30 2024-01-15T18:18:57.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c0c8860 con 0x7f2f5c06c230 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 0x7f2f5c0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 0x7f2f5c0c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56918/0 (socket says 172.21.15.143:56918) 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 -- 172.21.15.143:0/704888159 learned_addr learned my addr 172.21.15.143:0/704888159 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62000640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c06c230 0x7f2f5c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f61600640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 -- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c06c230 msgr2=0x7f2f5c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c06c230 0x7f2f5c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 -- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f5c0cfb30 msgr2=0x7f2f5c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c0c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 -- 172.21.15.143:0/704888159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440094d0 con 0x7f2f5c1607c0 2024-01-15T18:18:57.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f62a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 0x7f2f5c0c7ef0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400d800 tx=0x7f2f5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f540096c0 con 0x7f2f5c1607c0 2024-01-15T18:18:57.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c0c8b40 con 0x7f2f5c1607c0 2024-01-15T18:18:57.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f54009820 con 0x7f2f5c1607c0 2024-01-15T18:18:57.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54009bd0 con 0x7f2f5c1607c0 2024-01-15T18:18:57.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c0c9100 con 0x7f2f5c1607c0 2024-01-15T18:18:57.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.371+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5c06d4b0 con 0x7f2f5c1607c0 2024-01-15T18:18:57.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.375+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f2f54009d30 con 0x7f2f5c1607c0 2024-01-15T18:18:57.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.375+0000 7f2f5a000640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f34067a60 0x7f2f34069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.375+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2f540839b0 con 0x7f2f5c1607c0 2024-01-15T18:18:57.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.375+0000 7f2f62000640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f34067a60 0x7f2f34069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.375+0000 7f2f62000640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f34067a60 0x7f2f34069f20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c003ab0 tx=0x7f2f4c009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.379+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2f54052320 con 0x7f2f5c1607c0 2024-01-15T18:18:57.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.547+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f5c0cba30 con 0x7f2f34067a60 2024-01-15T18:18:57.551 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:57.555+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:57.551 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:57.555+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:57.551 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:57.555+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:57.551 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:57.555+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:57.551 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:18:57.555+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:18:57.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.587+0000 7f2f5a000640 1 -- 172.21.15.143:0/704888159 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2f4c009d20 con 0x7f2f34067a60 2024-01-15T18:18:57.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f34067a60 msgr2=0x7f2f34069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f34067a60 0x7f2f34069f20 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c003ab0 tx=0x7f2f4c009830 comp rx=0 tx=0).stop 2024-01-15T18:18:57.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 msgr2=0x7f2f5c0c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 0x7f2f5c0c7ef0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400d800 tx=0x7f2f5400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 shutdown_connections 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2f34067a60 0x7f2f34069f20 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f5c1607c0 0x7f2f5c0c7ef0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f5c0cfb30 0x7f2f5c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 --2- 172.21.15.143:0/704888159 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2f5c06c230 0x7f2f5c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 >> 172.21.15.143:0/704888159 conn(0x7f2f5c0d0e20 msgr2=0x7f2f5c0c4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 shutdown_connections 2024-01-15T18:18:57.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.591+0000 7f2f64a00640 1 -- 172.21.15.143:0/704888159 wait complete. 2024-01-15T18:18:57.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:18:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58b5000640 1 Processor -- start 2024-01-15T18:18:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58b5000640 1 -- start start 2024-01-15T18:18:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58b5000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58b00ce560 0x7f58b015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58b5000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b015f310 con 0x7f58b0141660 2024-01-15T18:18:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58b5000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b015f5f0 con 0x7f58b0071d00 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58b5000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b015fbb0 con 0x7f58b00ce560 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58ad400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58b00ce560 0x7f58b015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58ade00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f58b0141660 0x7f58b0141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35782/0 (socket says 172.21.15.143:35782) 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58ade00640 1 -- 172.21.15.143:0/296790201 learned_addr learned my addr 172.21.15.143:0/296790201 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58ad400640 1 -- 172.21.15.143:0/296790201 >> v1:172.21.15.204:6789/0 conn(0x7f58b0071d00 legacy=0x7f58b00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58ad400640 1 -- 172.21.15.143:0/296790201 >> v1:172.21.15.143:6789/0 conn(0x7f58b0141660 legacy=0x7f58b0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58ad400640 1 -- 172.21.15.143:0/296790201 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b015f8d0 con 0x7f58b00ce560 2024-01-15T18:18:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.751+0000 7f58aca00640 1 -- 172.21.15.143:0/296790201 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345668464 0 0) 0x7f58b015f310 con 0x7f58b0141660 2024-01-15T18:18:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58ad400640 1 --2- 172.21.15.143:0/296790201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b00ce560 0x7f58b015ec80 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f58980097c0 tx=0x7f589801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1ba7267c8a75de server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58aca00640 1 -- 172.21.15.143:0/296790201 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898004230 con 0x7f58b00ce560 2024-01-15T18:18:57.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58aca00640 1 -- 172.21.15.143:0/296790201 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58980a7050 con 0x7f58b00ce560 2024-01-15T18:18:57.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58aca00640 1 -- 172.21.15.143:0/296790201 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898003620 con 0x7f58b00ce560 2024-01-15T18:18:57.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 -- 172.21.15.143:0/296790201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b00ce560 msgr2=0x7f58b015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 --2- 172.21.15.143:0/296790201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b00ce560 0x7f58b015ec80 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f58980097c0 tx=0x7f589801c440 comp rx=0 tx=0).stop 2024-01-15T18:18:57.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 -- 172.21.15.143:0/296790201 shutdown_connections 2024-01-15T18:18:57.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 --2- 172.21.15.143:0/296790201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b00ce560 0x7f58b015ec80 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 -- 172.21.15.143:0/296790201 >> 172.21.15.143:0/296790201 conn(0x7f58b006c3f0 msgr2=0x7f58b006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:57.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 -- 172.21.15.143:0/296790201 shutdown_connections 2024-01-15T18:18:57.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.755+0000 7f58b5000640 1 -- 172.21.15.143:0/296790201 wait complete. 2024-01-15T18:18:57.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 Processor -- start 2024-01-15T18:18:57.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 -- start start 2024-01-15T18:18:57.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b0071d00 0x7f58b015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58b00ce560 0x7f58b0156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 0x7f58b00ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b015a150 con 0x7f58b0141660 2024-01-15T18:18:57.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b015b380 con 0x7f58b0071d00 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b015a9b0 con 0x7f58b00ce560 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 0x7f58b00ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 0x7f58b00ac910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56950/0 (socket says 172.21.15.143:56950) 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ade00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b0071d00 0x7f58b015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ad400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58b00ce560 0x7f58b0156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 -- 172.21.15.143:0/1969890496 learned_addr learned my addr 172.21.15.143:0/1969890496 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 -- 172.21.15.143:0/1969890496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58b00ce560 msgr2=0x7f58b0156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58b00ce560 0x7f58b0156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 -- 172.21.15.143:0/1969890496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b0071d00 msgr2=0x7f58b015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b0071d00 0x7f58b015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 -- 172.21.15.143:0/1969890496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898009420 con 0x7f58b0141660 2024-01-15T18:18:57.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58ae800640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 0x7f58b00ac910 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c780 tx=0x7f58a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8004b20 con 0x7f58b0141660 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b00cd150 con 0x7f58b0141660 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8004c80 con 0x7f58b0141660 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a800aa30 con 0x7f58b0141660 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.759+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b015a360 con 0x7f58b0141660 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.763+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b015ab40 con 0x7f58b0141660 2024-01-15T18:18:57.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.763+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f58a800ab90 con 0x7f58b0141660 2024-01-15T18:18:57.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.763+0000 7f58a6000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5888067a60 0x7f5888069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:57.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.763+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f58a8083320 con 0x7f58b0141660 2024-01-15T18:18:57.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.763+0000 7f58ade00640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5888067a60 0x7f5888069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:57.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.763+0000 7f58ade00640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5888067a60 0x7f5888069f20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000f70 tx=0x7f58a0007df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:57.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.767+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f58a808d080 con 0x7f58b0141660 2024-01-15T18:18:57.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.947+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f58b013cbb0 con 0x7f5888067a60 2024-01-15T18:18:57.946 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.951+0000 7f58a6000640 1 -- 172.21.15.143:0/1969890496 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f58b013cbb0 con 0x7f5888067a60 2024-01-15T18:18:57.946 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080 2024-01-15T18:18:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.951+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5888067a60 msgr2=0x7f5888069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.951+0000 7f58b5000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5888067a60 0x7f5888069f20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f58a0000f70 tx=0x7f58a0007df0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.951+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 msgr2=0x7f58b00ac910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.951+0000 7f58b5000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 0x7f58b00ac910 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f58a800c780 tx=0x7f58a800cc40 comp rx=0 tx=0).stop 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 shutdown_connections 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5888067a60 0x7f5888069f20 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58b0141660 0x7f58b00ac910 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f58b00ce560 0x7f58b0156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 --2- 172.21.15.143:0/1969890496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58b0071d00 0x7f58b015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 >> 172.21.15.143:0/1969890496 conn(0x7f58b006c3f0 msgr2=0x7f58b013c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 shutdown_connections 2024-01-15T18:18:57.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:57.955+0000 7f58b5000640 1 -- 172.21.15.143:0/1969890496 wait complete. 2024-01-15T18:18:57.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:18:58.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f389d000640 1 Processor -- start 2024-01-15T18:18:58.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f389d000640 1 -- start start 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f389d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f38980b24b0 0x7f38980ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f389d000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3898070bf0 con 0x7f38980b3bf0 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f389d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38980b4920 con 0x7f38980ce690 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f389d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389815dba0 con 0x7f38980b24b0 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3896800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f38980b3bf0 0x7f38980b40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35794/0 (socket says 172.21.15.143:35794) 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3896800640 1 -- 172.21.15.143:0/1513681227 learned_addr learned my addr 172.21.15.143:0/1513681227 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3895400640 1 --2- 172.21.15.143:0/1513681227 >> v2:172.21.15.204:3300/0 conn(0x7f38980b24b0 0x7f38980ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345718110 0 0) 0x7f3898070bf0 con 0x7f38980b3bf0 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3878003530 con 0x7f38980b3bf0 2024-01-15T18:18:58.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312576101 0 0) 0x7f38980b4920 con 0x7f38980ce690 2024-01-15T18:18:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3898070bf0 con 0x7f38980ce690 2024-01-15T18:18:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3895400640 1 -- 172.21.15.143:0/1513681227 >> v1:172.21.15.204:6789/0 conn(0x7f38980ce690 legacy=0x7f38980b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3895400640 1 -- 172.21.15.143:0/1513681227 >> v1:172.21.15.143:6789/0 conn(0x7f38980b3bf0 legacy=0x7f38980b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.115+0000 7f3895400640 1 -- 172.21.15.143:0/1513681227 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38980b4c00 con 0x7f38980b24b0 2024-01-15T18:18:58.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f3895400640 1 --2- 172.21.15.143:0/1513681227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980b24b0 0x7f38980ac8e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f38800097c0 tx=0x7f388001ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3bdc57290e77028 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:58.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800a7070 con 0x7f38980b24b0 2024-01-15T18:18:58.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38800ab430 con 0x7f38980b24b0 2024-01-15T18:18:58.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f3894a00640 1 -- 172.21.15.143:0/1513681227 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800ab7c0 con 0x7f38980b24b0 2024-01-15T18:18:58.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 -- 172.21.15.143:0/1513681227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980b24b0 msgr2=0x7f38980ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 --2- 172.21.15.143:0/1513681227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980b24b0 0x7f38980ac8e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f38800097c0 tx=0x7f388001ce70 comp rx=0 tx=0).stop 2024-01-15T18:18:58.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 -- 172.21.15.143:0/1513681227 shutdown_connections 2024-01-15T18:18:58.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 --2- 172.21.15.143:0/1513681227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980b24b0 0x7f38980ac8e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 -- 172.21.15.143:0/1513681227 >> 172.21.15.143:0/1513681227 conn(0x7f389806c3f0 msgr2=0x7f389806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:58.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 -- 172.21.15.143:0/1513681227 shutdown_connections 2024-01-15T18:18:58.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.119+0000 7f389d000640 1 -- 172.21.15.143:0/1513681227 wait complete. 2024-01-15T18:18:58.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 Processor -- start 2024-01-15T18:18:58.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 -- start start 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38980b24b0 0x7f389815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 0x7f38980b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980ce690 0x7f38980ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389813e6d0 con 0x7f38980b3bf0 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389813eaf0 con 0x7f38980ce690 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389815a6f0 con 0x7f38980b24b0 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 0x7f38980b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 0x7f38980b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56966/0 (socket says 172.21.15.143:56966) 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 -- 172.21.15.143:0/1394648302 learned_addr learned my addr 172.21.15.143:0/1394648302 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:18:58.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3896800640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980ce690 0x7f38980ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895e00640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38980b24b0 0x7f389815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 -- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38980b24b0 msgr2=0x7f389815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38980b24b0 0x7f389815a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 -- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980ce690 msgr2=0x7f38980ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980ce690 0x7f38980ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 -- 172.21.15.143:0/1394648302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880009420 con 0x7f38980b3bf0 2024-01-15T18:18:58.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f3895400640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 0x7f38980b0020 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f388001a040 tx=0x7f388001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:58.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800be5f0 con 0x7f38980b3bf0 2024-01-15T18:18:58.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38800be770 con 0x7f38980b3bf0 2024-01-15T18:18:58.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389813e330 con 0x7f38980b3bf0 2024-01-15T18:18:58.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800beb20 con 0x7f38980b3bf0 2024-01-15T18:18:58.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.123+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3898143770 con 0x7f38980b3bf0 2024-01-15T18:18:58.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.127+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3898067500 con 0x7f38980b3bf0 2024-01-15T18:18:58.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.127+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 38) v1 ==== 87741+0+0 (secure 0 0 0) 0x7f38800b43f0 con 0x7f38980b3bf0 2024-01-15T18:18:58.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.127+0000 7f388e000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3870067a60 0x7f3870069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:58.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.127+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f388011d420 con 0x7f38980b3bf0 2024-01-15T18:18:58.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.127+0000 7f3895e00640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3870067a60 0x7f3870069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:58.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.127+0000 7f3895e00640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3870067a60 0x7f3870069f20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3888000f70 tx=0x7f3888009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:58.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.131+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f38800ebe70 con 0x7f38980b3bf0 2024-01-15T18:18:58.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.303+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f389806e260 con 0x7f3870067a60 2024-01-15T18:18:58.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f388e000640 1 -- 172.21.15.143:0/1394648302 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f389806e260 con 0x7f3870067a60 2024-01-15T18:18:58.304 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080 2024-01-15T18:18:58.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3870067a60 msgr2=0x7f3870069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3870067a60 0x7f3870069f20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3888000f70 tx=0x7f3888009040 comp rx=0 tx=0).stop 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 msgr2=0x7f38980b0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 0x7f38980b0020 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f388001a040 tx=0x7f388001ac50 comp rx=0 tx=0).stop 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 shutdown_connections 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3870067a60 0x7f3870069f20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f38980ce690 0x7f38980ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980b3bf0 0x7f38980b0020 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 --2- 172.21.15.143:0/1394648302 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f38980b24b0 0x7f389815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:58.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 >> 172.21.15.143:0/1394648302 conn(0x7f389806c3f0 msgr2=0x7f389806db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 shutdown_connections 2024-01-15T18:18:58.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:18:58.311+0000 7f389d000640 1 -- 172.21.15.143:0/1394648302 wait complete. 2024-01-15T18:18:58.326 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080 2024-01-15T18:18:58.326 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:18:58.326 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:18:58.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:18:58.336 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:18:59.928 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:59.933 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:05.517 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:19:05.517 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:19:05.517 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 7.11423 s, 151 MB/s 2024-01-15T18:19:05.518 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:19:05.519 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:19:05.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:19:05.528 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:19:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:19:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:19:12.455 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.87109 s, 156 MB/s 2024-01-15T18:19:12.456 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:19:12.456 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:19:12.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:19:12.465 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:19:19.350 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records in 2024-01-15T18:19:19.350 INFO:teuthology.orchestra.run.smithi143.stderr:1024+0 records out 2024-01-15T18:19:19.350 INFO:teuthology.orchestra.run.smithi143.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 6.83009 s, 157 MB/s 2024-01-15T18:19:19.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:19:19.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8e4c00640 1 Processor -- start 2024-01-15T18:19:19.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8e4c00640 1 -- start start 2024-01-15T18:19:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8e4c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff8e0141660 0x7ff8e00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8e4c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e015f310 con 0x7ff8e0071d00 2024-01-15T18:19:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8e4c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e015f5f0 con 0x7ff8e015ec80 2024-01-15T18:19:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8e4c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e015fbb0 con 0x7ff8e0141660 2024-01-15T18:19:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8de800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff8e0071d00 0x7ff8e00b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33532/0 (socket says 172.21.15.143:33532) 2024-01-15T18:19:19.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8de800640 1 -- 172.21.15.143:0/2014981390 learned_addr learned my addr 172.21.15.143:0/2014981390 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dde00640 1 --2- 172.21.15.143:0/2014981390 >> v2:172.21.15.204:3300/0 conn(0x7ff8e0141660 0x7ff8e00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607307170 0 0) 0x7ff8e015f310 con 0x7ff8e0071d00 2024-01-15T18:19:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c0003530 con 0x7ff8e0071d00 2024-01-15T18:19:19.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3158097573 0 0) 0x7ff8c0003530 con 0x7ff8e0071d00 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 >> v2:172.21.15.204:3300/0 conn(0x7ff8e0141660 msgr2=0x7ff8e00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 --2- 172.21.15.143:0/2014981390 >> v2:172.21.15.204:3300/0 conn(0x7ff8e0141660 0x7ff8e00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 >> v1:172.21.15.204:6789/0 conn(0x7ff8e015ec80 legacy=0x7ff8e0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e015f8d0 con 0x7ff8e0071d00 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7ff8d8001a00 con 0x7ff8e0071d00 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 >> v1:172.21.15.143:6789/0 conn(0x7ff8e0071d00 legacy=0x7ff8e00b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c00047d0 0x7ff8c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8c0005190 0x7ff8c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e015fbb0 con 0x7ff8c0003980 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c000c030 con 0x7ff8c00047d0 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c000c1f0 con 0x7ff8c0005190 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dde00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8dd400640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c00047d0 0x7ff8c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.498+0000 7ff8de800640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8c0005190 0x7ff8c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dde00640 1 -- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8c0005190 msgr2=0x7ff8c000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dde00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8c0005190 0x7ff8c000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dde00640 1 -- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c00047d0 msgr2=0x7ff8c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dde00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8c00047d0 0x7ff8c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dde00640 1 -- 172.21.15.143:0/2014981390 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0003530 con 0x7ff8c0003980 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8de800640 1 -- 172.21.15.143:0/2014981390 reap_dead start 2024-01-15T18:19:19.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dde00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8c0003da0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009ae0 tx=0x7ff8d001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15ff4fd4020d3a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:19.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00a6510 con 0x7ff8c0003980 2024-01-15T18:19:19.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d00a6670 con 0x7ff8c0003980 2024-01-15T18:19:19.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8dca00640 1 -- 172.21.15.143:0/2014981390 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00a6a20 con 0x7ff8c0003980 2024-01-15T18:19:19.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8e4c00640 1 -- 172.21.15.143:0/2014981390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 msgr2=0x7ff8c0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.502+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8c0003da0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009ae0 tx=0x7ff8d001ca20 comp rx=0 tx=0).stop 2024-01-15T18:19:19.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- 172.21.15.143:0/2014981390 shutdown_connections 2024-01-15T18:19:19.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/2014981390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8c0003da0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- 172.21.15.143:0/2014981390 >> 172.21.15.143:0/2014981390 conn(0x7ff8e006c3f0 msgr2=0x7ff8e006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:19.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- 172.21.15.143:0/2014981390 shutdown_connections 2024-01-15T18:19:19.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- 172.21.15.143:0/2014981390 wait complete. 2024-01-15T18:19:19.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 Processor -- start 2024-01-15T18:19:19.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- start start 2024-01-15T18:19:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8e013c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8e01e5610 0x7ff8e01f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e015ccd0 0x7ff8e015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e015f130 con 0x7ff8c0003980 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01f1fb0 con 0x7ff8e015ccd0 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8e4c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e015d690 con 0x7ff8e01e5610 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8e013c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8e013c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57600/0 (socket says 172.21.15.143:57600) 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dde00640 1 -- 172.21.15.143:0/3976909264 learned_addr learned my addr 172.21.15.143:0/3976909264 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8de800640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e015ccd0 0x7ff8e015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dd400640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8e01e5610 0x7ff8e01f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dd400640 1 -- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e015ccd0 msgr2=0x7ff8e015d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dd400640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e015ccd0 0x7ff8e015d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dd400640 1 -- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 msgr2=0x7ff8e013c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dd400640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8e013c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8dd400640 1 -- 172.21.15.143:0/3976909264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00cd150 con 0x7ff8e01e5610 2024-01-15T18:19:19.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.506+0000 7ff8de800640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e015ccd0 0x7ff8e015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:19:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8dd400640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8e01e5610 0x7ff8e01f1a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800f5c0 tx=0x7ff8c800fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80052e0 con 0x7ff8e01e5610 2024-01-15T18:19:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c8005440 con 0x7ff8e01e5610 2024-01-15T18:19:19.505 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8003820 con 0x7ff8e01e5610 2024-01-15T18:19:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e0139ec0 con 0x7ff8e01e5610 2024-01-15T18:19:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e013cc00 con 0x7ff8e01e5610 2024-01-15T18:19:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0067500 con 0x7ff8e01e5610 2024-01-15T18:19:19.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 39) v1 ==== 87792+0+0 (secure 0 0 0) 0x7ff8c8003980 con 0x7ff8e01e5610 2024-01-15T18:19:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8d6000640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b8067af0 0x7ff8b8069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff8c80830c0 con 0x7ff8e01e5610 2024-01-15T18:19:19.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.510+0000 7ff8dde00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b8067af0 0x7ff8b8069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.514+0000 7ff8dde00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b8067af0 0x7ff8b8069fb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0000c10 tx=0x7ff8d00046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.514+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff8c8051a90 con 0x7ff8e01e5610 2024-01-15T18:19:19.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.690+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8e00ccfd0 con 0x7ff8b8067af0 2024-01-15T18:19:19.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8d6000640 1 -- 172.21.15.143:0/3976909264 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff8d00059a0 con 0x7ff8b8067af0 2024-01-15T18:19:19.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b8067af0 msgr2=0x7ff8b8069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b8067af0 0x7ff8b8069fb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0000c10 tx=0x7ff8d00046d0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8e01e5610 msgr2=0x7ff8e01f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8e01e5610 0x7ff8e01f1a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800f5c0 tx=0x7ff8c800fa80 comp rx=0 tx=0).stop 2024-01-15T18:19:19.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 shutdown_connections 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff8b8067af0 0x7ff8b8069fb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8e015ccd0 0x7ff8e015d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8e01e5610 0x7ff8e01f1a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 --2- 172.21.15.143:0/3976909264 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8c0003980 0x7ff8e013c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 >> 172.21.15.143:0/3976909264 conn(0x7ff8e006c3f0 msgr2=0x7ff8e00cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 shutdown_connections 2024-01-15T18:19:19.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.706+0000 7ff8e4c00640 1 -- 172.21.15.143:0/3976909264 wait complete. 2024-01-15T18:19:19.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:19:19.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a58e00640 1 Processor -- start 2024-01-15T18:19:19.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a58e00640 1 -- start start 2024-01-15T18:19:19.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a58e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4a54143780 0x7f4a540cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a58e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a540ceb50 con 0x7f4a540ce520 2024-01-15T18:19:19.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a58e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5415c110 con 0x7f4a540722c0 2024-01-15T18:19:19.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a58e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5415b7e0 con 0x7f4a54143780 2024-01-15T18:19:19.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a51400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4a540ce520 0x7f4a54071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33544/0 (socket says 172.21.15.143:33544) 2024-01-15T18:19:19.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a51400640 1 -- 172.21.15.143:0/1012146995 learned_addr learned my addr 172.21.15.143:0/1012146995 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:19.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a51e00640 1 --2- 172.21.15.143:0/1012146995 >> v2:172.21.15.204:3300/0 conn(0x7f4a54143780 0x7f4a540cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a50a00640 1 -- 172.21.15.143:0/1012146995 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465376380 0 0) 0x7f4a3c001120 con 0x7f4a540ce520 2024-01-15T18:19:19.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a50a00640 1 -- 172.21.15.143:0/1012146995 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a34003530 con 0x7f4a540ce520 2024-01-15T18:19:19.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a51e00640 1 -- 172.21.15.143:0/1012146995 >> v1:172.21.15.204:6789/0 conn(0x7f4a540722c0 legacy=0x7f4a5415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a51e00640 1 -- 172.21.15.143:0/1012146995 >> v1:172.21.15.143:6789/0 conn(0x7f4a540ce520 legacy=0x7f4a54071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.866+0000 7f4a51e00640 1 -- 172.21.15.143:0/1012146995 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5415bac0 con 0x7f4a54143780 2024-01-15T18:19:19.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a51e00640 1 --2- 172.21.15.143:0/1012146995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cdfe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4a440097c0 tx=0x7f4a4401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fe007d17ac4320f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:19.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a50a00640 1 -- 172.21.15.143:0/1012146995 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440a8050 con 0x7f4a54143780 2024-01-15T18:19:19.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a50a00640 1 -- 172.21.15.143:0/1012146995 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440ac5f0 con 0x7f4a54143780 2024-01-15T18:19:19.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a50a00640 1 -- 172.21.15.143:0/1012146995 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440ac960 con 0x7f4a54143780 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 -- 172.21.15.143:0/1012146995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 msgr2=0x7f4a540cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1012146995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cdfe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4a440097c0 tx=0x7f4a4401c480 comp rx=0 tx=0).stop 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 -- 172.21.15.143:0/1012146995 shutdown_connections 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1012146995 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cdfe0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 -- 172.21.15.143:0/1012146995 >> 172.21.15.143:0/1012146995 conn(0x7f4a5406c3f0 msgr2=0x7f4a5406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 -- 172.21.15.143:0/1012146995 shutdown_connections 2024-01-15T18:19:19.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.870+0000 7f4a58e00640 1 -- 172.21.15.143:0/1012146995 wait complete. 2024-01-15T18:19:19.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 Processor -- start 2024-01-15T18:19:19.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 -- start start 2024-01-15T18:19:19.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 0x7f4a54130440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a540ce520 0x7f4a5413c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5413c1a0 con 0x7f4a540722c0 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a540ae3f0 con 0x7f4a54143780 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a540ad9f0 con 0x7f4a540ce520 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 0x7f4a54130440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 0x7f4a54130440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57610/0 (socket says 172.21.15.143:57610) 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 -- 172.21.15.143:0/1572125014 learned_addr learned my addr 172.21.15.143:0/1572125014 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a52800640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51400640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a540ce520 0x7f4a5413c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 -- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a540ce520 msgr2=0x7f4a5413c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a540ce520 0x7f4a5413c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.871 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 -- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 msgr2=0x7f4a540cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:19.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:19.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 -- 172.21.15.143:0/1572125014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44009420 con 0x7f4a540722c0 2024-01-15T18:19:19.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a52800640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a51e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 0x7f4a54130440 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f4a4401a0b0 tx=0x7f4a4401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44006e20 con 0x7f4a540722c0 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a44004020 con 0x7f4a540722c0 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a540ade30 con 0x7f4a540722c0 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.874+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44004390 con 0x7f4a540722c0 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.878+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5413c3b0 con 0x7f4a540722c0 2024-01-15T18:19:19.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.878+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5413c5a0 con 0x7f4a540722c0 2024-01-15T18:19:19.874 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.878+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 39) v1 ==== 87792+0+0 (secure 0 0 0) 0x7f4a440b7630 con 0x7f4a540722c0 2024-01-15T18:19:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.878+0000 7f4a4a000640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a2c067b20 0x7f4a2c069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.878+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4a4411d550 con 0x7f4a540722c0 2024-01-15T18:19:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.878+0000 7f4a51400640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a2c067b20 0x7f4a2c069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:19.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.882+0000 7f4a51400640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a2c067b20 0x7f4a2c069fe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4a541f9250 tx=0x7f4a3c007bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:19.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:19.882+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4a440ebf20 con 0x7f4a540722c0 2024-01-15T18:19:20.053 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.058+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a5415d750 con 0x7f4a2c067b20 2024-01-15T18:19:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.062+0000 7f4a4a000640 1 -- 172.21.15.143:0/1572125014 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f4a3c005860 con 0x7f4a2c067b20 2024-01-15T18:19:20.058 INFO:teuthology.orchestra.run.smithi143.stdout:/volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080 2024-01-15T18:19:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a2c067b20 msgr2=0x7f4a2c069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a2c067b20 0x7f4a2c069fe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f4a541f9250 tx=0x7f4a3c007bc0 comp rx=0 tx=0).stop 2024-01-15T18:19:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 msgr2=0x7f4a54130440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 0x7f4a54130440 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f4a4401a0b0 tx=0x7f4a4401acc0 comp rx=0 tx=0).stop 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 shutdown_connections 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4a2c067b20 0x7f4a2c069fe0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a54143780 0x7f4a540cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4a540ce520 0x7f4a5413c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 --2- 172.21.15.143:0/1572125014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a540722c0 0x7f4a54130440 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 >> 172.21.15.143:0/1572125014 conn(0x7f4a5406c3f0 msgr2=0x7f4a540af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 shutdown_connections 2024-01-15T18:19:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:20.066+0000 7f4a58e00640 1 -- 172.21.15.143:0/1572125014 wait complete. 2024-01-15T18:19:20.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:19:20.080 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080) 2024-01-15T18:19:20.094 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080 2024-01-15T18:19:20.094 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:19:20.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:19:20.095 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:19:22.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:19:22.096 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080) 2024-01-15T18:19:22.112 INFO:teuthology.orchestra.run.smithi143.stdout:# file: volumes/_nogroup/sv1/b7e79b87-0906-4d38-8637-9065bcea5080 2024-01-15T18:19:22.112 INFO:teuthology.orchestra.run.smithi143.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:19:22.112 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:19:22.113 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:19:22.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone39061 2024-01-15T18:19:22.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.310+0000 7f6661600640 1 Processor -- start 2024-01-15T18:19:22.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.310+0000 7f6661600640 1 -- start start 2024-01-15T18:19:22.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6661600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f665c0cfb30 0x7f665c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6661600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 0x7f665c0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6661600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c0cffd0 con 0x7f665c1607c0 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6661600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c0d1ce0 con 0x7f665c0cfb30 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6661600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0d02b0 con 0x7f665c0cbf50 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f665c1607c0 0x7f665c06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33554/0 (socket says 172.21.15.143:33554) 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 -- 172.21.15.143:0/1766609304 learned_addr learned my addr 172.21.15.143:0/1766609304 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665a000640 1 --2- 172.21.15.143:0/1766609304 >> v2:172.21.15.204:3300/0 conn(0x7f665c0cfb30 0x7f665c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665b400640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 0x7f665c0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559244164 0 0) 0x7f665c0cffd0 con 0x7f665c1607c0 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f663c003530 con 0x7f665c1607c0 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4037962510 0 0) 0x7f663c003530 con 0x7f665c1607c0 2024-01-15T18:19:22.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 msgr2=0x7f665c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 0x7f665c0cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 >> v2:172.21.15.204:3300/0 conn(0x7f665c0cfb30 msgr2=0x7f665c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 --2- 172.21.15.143:0/1766609304 >> v2:172.21.15.204:3300/0 conn(0x7f665c0cfb30 0x7f665c06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c0cb470 con 0x7f665c1607c0 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f664c001680 con 0x7f665c1607c0 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 >> v1:172.21.15.143:6789/0 conn(0x7f665c1607c0 legacy=0x7f665c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663c003980 0x7f663c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 0x7f663c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c004d10 0x7f663c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0d1ce0 con 0x7f663c003980 2024-01-15T18:19:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0d02b0 con 0x7f663c004d10 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c00b990 con 0x7f665c0cbf50 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663c003980 0x7f663c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665b400640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c004d10 0x7f663c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665a000640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 0x7f663c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 -- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 msgr2=0x7f663c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c0cbf50 0x7f663c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 -- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c004d10 msgr2=0x7f663c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c004d10 0x7f663c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.314+0000 7f665aa00640 1 -- 172.21.15.143:0/1766609304 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c003530 con 0x7f663c003980 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f665aa00640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663c003980 0x7f663c003da0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c240 tx=0x7f664c01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd24be68fa4fe52 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c005ae0 con 0x7f663c003980 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c0ab070 con 0x7f663c003980 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6659600640 1 -- 172.21.15.143:0/1766609304 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c005e80 con 0x7f663c003980 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6661600640 1 -- 172.21.15.143:0/1766609304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663c003980 msgr2=0x7f663c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6661600640 1 --2- 172.21.15.143:0/1766609304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f663c003980 0x7f663c003da0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f664c00c240 tx=0x7f664c01f3d0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f665b400640 1 -- 172.21.15.143:0/1766609304 reap_dead start 2024-01-15T18:19:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6661600640 1 -- 172.21.15.143:0/1766609304 shutdown_connections 2024-01-15T18:19:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6661600640 1 -- 172.21.15.143:0/1766609304 >> 172.21.15.143:0/1766609304 conn(0x7f665c0d0e20 msgr2=0x7f665c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6661600640 1 -- 172.21.15.143:0/1766609304 shutdown_connections 2024-01-15T18:19:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.318+0000 7f6661600640 1 -- 172.21.15.143:0/1766609304 wait complete. 2024-01-15T18:19:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 Processor -- start 2024-01-15T18:19:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 -- start start 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c003980 0x7f665c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 0x7f665c0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c156190 0x7f665c0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c158850 con 0x7f665c1531a0 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c156b90 con 0x7f663c003980 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c156e70 con 0x7f665c156190 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 0x7f665c0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 0x7f665c0c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57634/0 (socket says 172.21.15.143:57634) 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 -- 172.21.15.143:0/3930867371 learned_addr learned my addr 172.21.15.143:0/3930867371 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:22.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665aa00640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c003980 0x7f665c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665b400640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c156190 0x7f665c0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 -- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c156190 msgr2=0x7f665c0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c156190 0x7f665c0c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 -- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c003980 msgr2=0x7f665c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c003980 0x7f665c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 -- 172.21.15.143:0/3930867371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c158470 con 0x7f665c1531a0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f665a000640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 0x7f665c0c7ef0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f66440119d0 tx=0x7f6644011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664400fb50 con 0x7f665c1531a0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400fcb0 con 0x7f665c1531a0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644014440 con 0x7f665c1531a0 2024-01-15T18:19:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c153600 con 0x7f665c1531a0 2024-01-15T18:19:22.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.322+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c0c8600 con 0x7f665c1531a0 2024-01-15T18:19:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.326+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c067500 con 0x7f665c1531a0 2024-01-15T18:19:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.326+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 39) v1 ==== 87792+0+0 (secure 0 0 0) 0x7f66440145a0 con 0x7f665c1531a0 2024-01-15T18:19:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.326+0000 7f6652a00640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6634067af0 0x7f6634069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.326+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6644083a80 con 0x7f665c1531a0 2024-01-15T18:19:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.326+0000 7f665aa00640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6634067af0 0x7f6634069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.326+0000 7f665aa00640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6634067af0 0x7f6634069fb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f664c0ae000 tx=0x7f664c0b4040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:22.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.330+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f6644052510 con 0x7f665c1531a0 2024-01-15T18:19:22.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.506+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone39061", "target": ["mon-mgr", ""]}) v1 -- 0x7f665c0ce6b0 con 0x7f6634067af0 2024-01-15T18:19:22.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.566+0000 7f6652a00640 1 -- 172.21.15.143:0/3930867371 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f664c0b8040 con 0x7f6634067af0 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6634067af0 msgr2=0x7f6634069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6634067af0 0x7f6634069fb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f664c0ae000 tx=0x7f664c0b4040 comp rx=0 tx=0).stop 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 msgr2=0x7f665c0c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 0x7f665c0c7ef0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f66440119d0 tx=0x7f6644011e90 comp rx=0 tx=0).stop 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 shutdown_connections 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6634067af0 0x7f6634069fb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f665c156190 0x7f665c0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f665c1531a0 0x7f665c0c7ef0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 --2- 172.21.15.143:0/3930867371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f663c003980 0x7f665c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 >> 172.21.15.143:0/3930867371 conn(0x7f665c0d0e20 msgr2=0x7f665c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 shutdown_connections 2024-01-15T18:19:22.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.570+0000 7f6661600640 1 -- 172.21.15.143:0/3930867371 wait complete. 2024-01-15T18:19:22.570 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.574+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.570 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.574+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.570 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.574+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.570 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.574+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.570 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.574+0000 7f3cb1000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.587 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.590+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.587 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.590+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.587 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.590+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.587 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.590+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.587 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:22.590+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:22.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone39062 2024-01-15T18:19:22.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc053a00640 1 Processor -- start 2024-01-15T18:19:22.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc053a00640 1 -- start start 2024-01-15T18:19:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc053a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc04c141660 0x7fc04c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc053a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c15f310 con 0x7fc04c15ec80 2024-01-15T18:19:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc053a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c15f5f0 con 0x7fc04c071d00 2024-01-15T18:19:22.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc053a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c15fbb0 con 0x7fc04c141660 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc04be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc04c15ec80 0x7fc04c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33570/0 (socket says 172.21.15.143:33570) 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc04be00640 1 -- 172.21.15.143:0/1507081291 learned_addr learned my addr 172.21.15.143:0/1507081291 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc051000640 1 --2- 172.21.15.143:0/1507081291 >> v2:172.21.15.204:3300/0 conn(0x7fc04c141660 0x7fc04c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806430243 0 0) 0x7fc04c15f310 con 0x7fc04c15ec80 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc02c003530 con 0x7fc04c15ec80 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649902899 0 0) 0x7fc04c15f5f0 con 0x7fc04c071d00 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc04c15f310 con 0x7fc04c071d00 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc051000640 1 -- 172.21.15.143:0/1507081291 >> v1:172.21.15.204:6789/0 conn(0x7fc04c071d00 legacy=0x7fc04c0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc051000640 1 -- 172.21.15.143:0/1507081291 >> v1:172.21.15.143:6789/0 conn(0x7fc04c15ec80 legacy=0x7fc04c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.738+0000 7fc051000640 1 -- 172.21.15.143:0/1507081291 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c15f8d0 con 0x7fc04c141660 2024-01-15T18:19:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc051000640 1 --2- 172.21.15.143:0/1507081291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c141660 0x7fc04c0ce560 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0097c0 tx=0x7fc03c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869f6c0dfe62bbae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0a7070 con 0x7fc04c141660 2024-01-15T18:19:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c0ab430 con 0x7fc04c141660 2024-01-15T18:19:22.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc04b400640 1 -- 172.21.15.143:0/1507081291 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0ab7a0 con 0x7fc04c141660 2024-01-15T18:19:22.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc053a00640 1 -- 172.21.15.143:0/1507081291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c141660 msgr2=0x7fc04c0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc053a00640 1 --2- 172.21.15.143:0/1507081291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c141660 0x7fc04c0ce560 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0097c0 tx=0x7fc03c01c410 comp rx=0 tx=0).stop 2024-01-15T18:19:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc053a00640 1 -- 172.21.15.143:0/1507081291 shutdown_connections 2024-01-15T18:19:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc053a00640 1 --2- 172.21.15.143:0/1507081291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c141660 0x7fc04c0ce560 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc053a00640 1 -- 172.21.15.143:0/1507081291 >> 172.21.15.143:0/1507081291 conn(0x7fc04c06c3f0 msgr2=0x7fc04c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.742+0000 7fc053a00640 1 -- 172.21.15.143:0/1507081291 shutdown_connections 2024-01-15T18:19:22.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 -- 172.21.15.143:0/1507081291 wait complete. 2024-01-15T18:19:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 Processor -- start 2024-01-15T18:19:22.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 -- start start 2024-01-15T18:19:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc04c071d00 0x7fc04c137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 0x7fc04c0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c15ec80 0x7fc04c06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c15f130 con 0x7fc04c141660 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c140ea0 con 0x7fc04c15ec80 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc053a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c1368c0 con 0x7fc04c071d00 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 0x7fc04c0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 0x7fc04c0b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57666/0 (socket says 172.21.15.143:57666) 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 -- 172.21.15.143:0/2037856725 learned_addr learned my addr 172.21.15.143:0/2037856725 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc051000640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc04c071d00 0x7fc04c137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc051a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c15ec80 0x7fc04c06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 -- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc04c071d00 msgr2=0x7fc04c137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc04c071d00 0x7fc04c137af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 -- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c15ec80 msgr2=0x7fc04c06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c15ec80 0x7fc04c06ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 -- 172.21.15.143:0/2037856725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c009420 con 0x7fc04c141660 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc051a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c15ec80 0x7fc04c06ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc04be00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 0x7fc04c0b1a30 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c450 tx=0x7fc03400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034005cf0 con 0x7fc04c141660 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc034005e50 con 0x7fc04c141660 2024-01-15T18:19:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.746+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034007270 con 0x7fc04c141660 2024-01-15T18:19:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c0cd150 con 0x7fc04c141660 2024-01-15T18:19:22.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c136270 con 0x7fc04c141660 2024-01-15T18:19:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c067500 con 0x7fc04c141660 2024-01-15T18:19:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 39) v1 ==== 87792+0+0 (secure 0 0 0) 0x7fc0340073d0 con 0x7fc04c141660 2024-01-15T18:19:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc048c00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc024067af0 0x7fc024069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc034082750 con 0x7fc04c141660 2024-01-15T18:19:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.750+0000 7fc051000640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc024067af0 0x7fc024069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:22.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.754+0000 7fc051000640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc024067af0 0x7fc024069fb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0af000 tx=0x7fc03c0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:22.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.754+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc034051120 con 0x7fc04c141660 2024-01-15T18:19:22.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.938+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone39062", "target": ["mon-mgr", ""]}) v1 -- 0x7fc04c0ccfd0 con 0x7fc024067af0 2024-01-15T18:19:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.994+0000 7fc048c00640 1 -- 172.21.15.143:0/2037856725 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc03c0ab7c0 con 0x7fc024067af0 2024-01-15T18:19:22.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc024067af0 msgr2=0x7fc024069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc024067af0 0x7fc024069fb0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0af000 tx=0x7fc03c0a7000 comp rx=0 tx=0).stop 2024-01-15T18:19:22.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 msgr2=0x7fc04c0b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 0x7fc04c0b1a30 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c450 tx=0x7fc03400c910 comp rx=0 tx=0).stop 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 shutdown_connections 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc024067af0 0x7fc024069fb0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc04c15ec80 0x7fc04c06ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc04c141660 0x7fc04c0b1a30 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 --2- 172.21.15.143:0/2037856725 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc04c071d00 0x7fc04c137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 >> 172.21.15.143:0/2037856725 conn(0x7fc04c06c3f0 msgr2=0x7fc04c144020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 shutdown_connections 2024-01-15T18:19:22.994 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:22.998+0000 7fc053a00640 1 -- 172.21.15.143:0/2037856725 wait complete. 2024-01-15T18:19:23.015 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone39063 2024-01-15T18:19:23.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff979c00640 1 Processor -- start 2024-01-15T18:19:23.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff979c00640 1 -- start start 2024-01-15T18:19:23.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff979c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff9740cfb30 0x7ff97406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:23.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff979c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740cffd0 con 0x7ff9741607c0 2024-01-15T18:19:23.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff979c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740d02b0 con 0x7ff9740cbf50 2024-01-15T18:19:23.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff979c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9740d1ce0 con 0x7ff9740cfb30 2024-01-15T18:19:23.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff972a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff9741607c0 0x7ff97406b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33578/0 (socket says 172.21.15.143:33578) 2024-01-15T18:19:23.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.162+0000 7ff972a00640 1 -- 172.21.15.143:0/389764161 learned_addr learned my addr 172.21.15.143:0/389764161 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:23.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff972000640 1 --2- 172.21.15.143:0/389764161 >> v2:172.21.15.204:3300/0 conn(0x7ff9740cfb30 0x7ff97406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412914787 0 0) 0x7ff9740cffd0 con 0x7ff9741607c0 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff954003530 con 0x7ff9741607c0 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220163963 0 0) 0x7ff9740d02b0 con 0x7ff9740cbf50 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9740cffd0 con 0x7ff9740cbf50 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff972000640 1 -- 172.21.15.143:0/389764161 >> v1:172.21.15.204:6789/0 conn(0x7ff9740cbf50 legacy=0x7ff9740cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff972000640 1 -- 172.21.15.143:0/389764161 >> v1:172.21.15.143:6789/0 conn(0x7ff9741607c0 legacy=0x7ff97406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:23.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff972000640 1 -- 172.21.15.143:0/389764161 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740cb470 con 0x7ff9740cfb30 2024-01-15T18:19:23.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff972000640 1 --2- 172.21.15.143:0/389764161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9740cfb30 0x7ff97406bec0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0097c0 tx=0x7ff95c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18e0f0fe4c9027a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:23.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0a8970 con 0x7ff9740cfb30 2024-01-15T18:19:23.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c0a8ad0 con 0x7ff9740cfb30 2024-01-15T18:19:23.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.166+0000 7ff971600640 1 -- 172.21.15.143:0/389764161 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c0a8e60 con 0x7ff9740cfb30 2024-01-15T18:19:23.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- 172.21.15.143:0/389764161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9740cfb30 msgr2=0x7ff97406bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:23.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 --2- 172.21.15.143:0/389764161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9740cfb30 0x7ff97406bec0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0097c0 tx=0x7ff95c01ec70 comp rx=0 tx=0).stop 2024-01-15T18:19:23.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- 172.21.15.143:0/389764161 shutdown_connections 2024-01-15T18:19:23.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 --2- 172.21.15.143:0/389764161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9740cfb30 0x7ff97406bec0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:23.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- 172.21.15.143:0/389764161 >> 172.21.15.143:0/389764161 conn(0x7ff9740d0e20 msgr2=0x7ff9740d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:23.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- 172.21.15.143:0/389764161 shutdown_connections 2024-01-15T18:19:23.165 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- 172.21.15.143:0/389764161 wait complete. 2024-01-15T18:19:23.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 Processor -- start 2024-01-15T18:19:23.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- start start 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 0x7ff974148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9740cfb30 0x7ff9741f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9741607c0 0x7ff9741f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974149b70 con 0x7ff9740cbf50 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740c9b60 con 0x7ff9741607c0 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff979c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740c9880 con 0x7ff9740cfb30 2024-01-15T18:19:23.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff972a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 0x7ff974148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff972a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 0x7ff974148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57682/0 (socket says 172.21.15.143:57682) 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.170+0000 7ff972a00640 1 -- 172.21.15.143:0/3228391229 learned_addr learned my addr 172.21.15.143:0/3228391229 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972000640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9740cfb30 0x7ff9741f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff973400640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9741607c0 0x7ff9741f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972a00640 1 -- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9740cfb30 msgr2=0x7ff9741f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972a00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9740cfb30 0x7ff9741f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972a00640 1 -- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9741607c0 msgr2=0x7ff9741f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972a00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9741607c0 0x7ff9741f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972a00640 1 -- 172.21.15.143:0/3228391229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c009420 con 0x7ff9740cbf50 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff972a00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 0x7ff974148950 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c420 tx=0x7ff96400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:23.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964005cf0 con 0x7ff9740cbf50 2024-01-15T18:19:23.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964005e50 con 0x7ff9740cbf50 2024-01-15T18:19:23.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9741494b0 con 0x7ff9740cbf50 2024-01-15T18:19:23.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964007310 con 0x7ff9740cbf50 2024-01-15T18:19:23.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff974148e90 con 0x7ff9740cbf50 2024-01-15T18:19:23.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff97406d4b0 con 0x7ff9740cbf50 2024-01-15T18:19:23.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 39) v1 ==== 87792+0+0 (secure 0 0 0) 0x7ff964007470 con 0x7ff9740cbf50 2024-01-15T18:19:23.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.174+0000 7ff96aa00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff94c067af0 0x7ff94c069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:23.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.178+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff9640828f0 con 0x7ff9740cbf50 2024-01-15T18:19:23.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.178+0000 7ff972000640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff94c067af0 0x7ff94c069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:23.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.178+0000 7ff972000640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff94c067af0 0x7ff94c069fb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0b2000 tx=0x7ff95c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:23.175 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.178+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff9640512c0 con 0x7ff9740cbf50 2024-01-15T18:19:23.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.358+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone39063", "target": ["mon-mgr", ""]}) v1 -- 0x7ff97406c460 con 0x7ff94c067af0 2024-01-15T18:19:23.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:23.962+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 40) v1 ==== 87894+0+0 (secure 0 0 0) 0x7ff964051690 con 0x7ff9740cbf50 2024-01-15T18:19:27.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.110+0000 7ff96aa00640 1 -- 172.21.15.143:0/3228391229 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff97406c460 con 0x7ff94c067af0 2024-01-15T18:19:27.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.110+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff94c067af0 msgr2=0x7ff94c069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:27.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.110+0000 7ff979c00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff94c067af0 0x7ff94c069fb0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0b2000 tx=0x7ff95c008040 comp rx=0 tx=0).stop 2024-01-15T18:19:27.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.110+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 msgr2=0x7ff974148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:27.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.110+0000 7ff979c00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 0x7ff974148950 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c420 tx=0x7ff96400c8e0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 shutdown_connections 2024-01-15T18:19:27.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff94c067af0 0x7ff94c069fb0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff9741607c0 0x7ff9741f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff9740cfb30 0x7ff9741f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 --2- 172.21.15.143:0/3228391229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff9740cbf50 0x7ff974148950 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 >> 172.21.15.143:0/3228391229 conn(0x7ff9740d0e20 msgr2=0x7ff97406b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:27.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 shutdown_connections 2024-01-15T18:19:27.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.114+0000 7ff979c00640 1 -- 172.21.15.143:0/3228391229 wait complete. 2024-01-15T18:19:27.131 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:19:27.131 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone39063 2024-01-15T18:19:27.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 Processor -- start 2024-01-15T18:19:27.273 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 -- start start 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feaf8143780 0x7feaf80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf8071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf80ceb50 con 0x7feaf80720f0 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf8072580 con 0x7feaf8143780 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf815c110 con 0x7feaf80ce520 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf7e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7feaf8143780 0x7feaf80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf7400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf8071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafce00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7feaf80720f0 0x7feaf815a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:51472/0 (socket says 172.21.15.143:51472) 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feafce00640 1 -- 172.21.15.143:0/963662323 learned_addr learned my addr 172.21.15.143:0/963662323 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf7e00640 1 -- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 msgr2=0x7feaf8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf7e00640 1 --2- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf8071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf7e00640 1 -- 172.21.15.143:0/963662323 >> v1:172.21.15.204:6789/0 conn(0x7feaf80720f0 legacy=0x7feaf815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf7e00640 1 -- 172.21.15.143:0/963662323 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf815b7e0 con 0x7feaf8143780 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.278+0000 7feaf6a00640 1 -- 172.21.15.143:0/963662323 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345684995 0 0) 0x7feaf00014a0 con 0x7feaf80720f0 2024-01-15T18:19:27.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feaf7e00640 1 --2- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf8143780 0x7feaf80cdfe0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7feae80097c0 tx=0x7feae801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9084721f8f5e2d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:27.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feaf6a00640 1 -- 172.21.15.143:0/963662323 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80ad640 con 0x7feaf8143780 2024-01-15T18:19:27.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feaf6a00640 1 -- 172.21.15.143:0/963662323 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80ad7a0 con 0x7feaf8143780 2024-01-15T18:19:27.278 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feaf6a00640 1 -- 172.21.15.143:0/963662323 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80adb30 con 0x7feaf8143780 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 -- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf8143780 msgr2=0x7feaf80cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 --2- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf8143780 0x7feaf80cdfe0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7feae80097c0 tx=0x7feae801c480 comp rx=0 tx=0).stop 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 -- 172.21.15.143:0/963662323 shutdown_connections 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 --2- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf8071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 --2- 172.21.15.143:0/963662323 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf8143780 0x7feaf80cdfe0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 -- 172.21.15.143:0/963662323 >> 172.21.15.143:0/963662323 conn(0x7feaf806c3f0 msgr2=0x7feaf806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 -- 172.21.15.143:0/963662323 shutdown_connections 2024-01-15T18:19:27.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.282+0000 7feafee00640 1 -- 172.21.15.143:0/963662323 wait complete. 2024-01-15T18:19:27.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 Processor -- start 2024-01-15T18:19:27.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 -- start start 2024-01-15T18:19:27.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf80720f0 0x7feaf813b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 0x7feaf80cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf813b960 con 0x7feaf8143780 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8158740 con 0x7feaf80720f0 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf8158a20 con 0x7feaf80ce520 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 0x7feaf80cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 0x7feaf80cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51552/0 (socket says 172.21.15.143:51552) 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 -- 172.21.15.143:0/2699776248 learned_addr learned my addr 172.21.15.143:0/2699776248 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feaf7400640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feaf7e00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf80720f0 0x7feaf813b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 -- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 msgr2=0x7feaf814a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf814a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 -- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf80720f0 msgr2=0x7feaf813b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf80720f0 0x7feaf813b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:27.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 -- 172.21.15.143:0/2699776248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae8009420 con 0x7feaf8143780 2024-01-15T18:19:27.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafce00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 0x7feaf80cd490 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c780 tx=0x7feaf000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:27.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000ce30 con 0x7feaf8143780 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf000a640 con 0x7feaf8143780 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf000aa30 con 0x7feaf8143780 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf8158d00 con 0x7feaf8143780 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.286+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf81592c0 con 0x7feaf8143780 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.290+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf81594d0 con 0x7feaf8143780 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.290+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 40) v1 ==== 87894+0+0 (secure 0 0 0) 0x7feaf000ab90 con 0x7feaf8143780 2024-01-15T18:19:27.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.290+0000 7feaefe00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fead0067c10 0x7fead006a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:27.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.290+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7feaf00826c0 con 0x7feaf8143780 2024-01-15T18:19:27.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.290+0000 7feaf7e00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fead0067c10 0x7fead006a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:27.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.290+0000 7feaf7e00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fead0067c10 0x7fead006a0d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7feae80af000 tx=0x7feae80a7070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:27.290 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.294+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7feaf0051060 con 0x7feaf8143780 2024-01-15T18:19:27.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:27.478+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone39063", "target": ["mon-mgr", ""]}) v1 -- 0x7feaf80ceb50 con 0x7fead0067c10 2024-01-15T18:19:29.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:29.990 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.050+0000 7feaefe00640 1 -- 172.21.15.143:0/2699776248 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7feaf80ceb50 con 0x7fead0067c10 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: "status": { 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: "state": "pending", 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: "source": { 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: "volume": "cephfs", 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: "subvolume": "sv1", 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: "snapshot": "ss1" 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:19:32.047 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:19:32.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fead0067c10 msgr2=0x7fead006a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:32.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fead0067c10 0x7fead006a0d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7feae80af000 tx=0x7feae80a7070 comp rx=0 tx=0).stop 2024-01-15T18:19:32.049 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 msgr2=0x7feaf80cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 0x7feaf80cd490 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7feaf000c780 tx=0x7feaf000cc40 comp rx=0 tx=0).stop 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 shutdown_connections 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fead0067c10 0x7fead006a0d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feaf8143780 0x7feaf80cd490 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7feaf80ce520 0x7feaf814a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 --2- 172.21.15.143:0/2699776248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feaf80720f0 0x7feaf813b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 >> 172.21.15.143:0/2699776248 conn(0x7feaf806c3f0 msgr2=0x7feaf80ada70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 shutdown_connections 2024-01-15T18:19:32.050 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.054+0000 7feafee00640 1 -- 172.21.15.143:0/2699776248 wait complete. 2024-01-15T18:19:32.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone39061 2024-01-15T18:19:32.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 Processor -- start 2024-01-15T18:19:32.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 -- start start 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94d00ce560 0x7f94d015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d0071d00 0x7f94d0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d015f310 con 0x7f94d0141660 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d015fbb0 con 0x7f94d00ce560 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94d6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d0072690 con 0x7f94d0071d00 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cf400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f94d0141660 0x7f94d0141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34070/0 (socket says 172.21.15.143:34070) 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f94d00ce560 0x7f94d015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:32.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cf400640 1 -- 172.21.15.143:0/2782966431 learned_addr learned my addr 172.21.15.143:0/2782966431 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cfe00640 1 --2- 172.21.15.143:0/2782966431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d0071d00 0x7f94d0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cea00640 1 -- 172.21.15.143:0/2782966431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d0071d00 msgr2=0x7f94d0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cea00640 1 --2- 172.21.15.143:0/2782966431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d0071d00 0x7f94d0072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cea00640 1 -- 172.21.15.143:0/2782966431 >> v1:172.21.15.143:6789/0 conn(0x7f94d0141660 legacy=0x7f94d0141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94cea00640 1 -- 172.21.15.143:0/2782966431 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d015f5f0 con 0x7f94d00ce560 2024-01-15T18:19:32.219 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.222+0000 7f94ce000640 1 -- 172.21.15.143:0/2782966431 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538761221 0 0) 0x7f94d015f310 con 0x7f94d0141660 2024-01-15T18:19:32.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94cea00640 1 --2- 172.21.15.143:0/2782966431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d00ce560 0x7f94d015ec80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f94b80097c0 tx=0x7f94b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda79296b276c671 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:32.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94ce000640 1 -- 172.21.15.143:0/2782966431 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b80ab600 con 0x7f94d00ce560 2024-01-15T18:19:32.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94ce000640 1 -- 172.21.15.143:0/2782966431 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b80a7050 con 0x7f94d00ce560 2024-01-15T18:19:32.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94ce000640 1 -- 172.21.15.143:0/2782966431 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b80ab980 con 0x7f94d00ce560 2024-01-15T18:19:32.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 -- 172.21.15.143:0/2782966431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d00ce560 msgr2=0x7f94d015ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:32.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 --2- 172.21.15.143:0/2782966431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d00ce560 0x7f94d015ec80 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f94b80097c0 tx=0x7f94b801c480 comp rx=0 tx=0).stop 2024-01-15T18:19:32.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 -- 172.21.15.143:0/2782966431 shutdown_connections 2024-01-15T18:19:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 --2- 172.21.15.143:0/2782966431 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d0071d00 0x7f94d0072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 --2- 172.21.15.143:0/2782966431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d00ce560 0x7f94d015ec80 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 -- 172.21.15.143:0/2782966431 >> 172.21.15.143:0/2782966431 conn(0x7f94d006c3f0 msgr2=0x7f94d006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 -- 172.21.15.143:0/2782966431 shutdown_connections 2024-01-15T18:19:32.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.226+0000 7f94d6400640 1 -- 172.21.15.143:0/2782966431 wait complete. 2024-01-15T18:19:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 Processor -- start 2024-01-15T18:19:32.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 -- start start 2024-01-15T18:19:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 0x7f94d015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d00ce560 0x7f94d015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94d0141660 0x7f94d00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d015ae40 con 0x7f94d0071d00 2024-01-15T18:19:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00cc480 con 0x7f94d0141660 2024-01-15T18:19:32.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d00cc760 con 0x7f94d00ce560 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 0x7f94d015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 0x7f94d015a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51572/0 (socket says 172.21.15.143:51572) 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 -- 172.21.15.143:0/2620112109 learned_addr learned my addr 172.21.15.143:0/2620112109 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cea00640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d00ce560 0x7f94d015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cfe00640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94d0141660 0x7f94d00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 -- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d00ce560 msgr2=0x7f94d015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d00ce560 0x7f94d015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 -- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94d0141660 msgr2=0x7f94d00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94d0141660 0x7f94d00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 -- 172.21.15.143:0/2620112109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009420 con 0x7f94d0071d00 2024-01-15T18:19:32.227 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94cf400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 0x7f94d015a900 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c480 tx=0x7f94c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:32.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c00074e0 con 0x7f94d0071d00 2024-01-15T18:19:32.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c0007660 con 0x7f94d0071d00 2024-01-15T18:19:32.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d00cd150 con 0x7f94d0071d00 2024-01-15T18:19:32.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004a30 con 0x7f94d0071d00 2024-01-15T18:19:32.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.230+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d00b1ea0 con 0x7f94d0071d00 2024-01-15T18:19:32.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.234+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d0067500 con 0x7f94d0071d00 2024-01-15T18:19:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.234+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 40) v1 ==== 87894+0+0 (secure 0 0 0) 0x7f94c0004b90 con 0x7f94d0071d00 2024-01-15T18:19:32.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.234+0000 7f94c7400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94a8067c10 0x7f94a806a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.234+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f94c0082980 con 0x7f94d0071d00 2024-01-15T18:19:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.234+0000 7f94cea00640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94a8067c10 0x7f94a806a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:32.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.234+0000 7f94cea00640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94a8067c10 0x7f94a806a0d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f94b80ad000 tx=0x7f94b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:32.234 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.238+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f94c0051270 con 0x7f94d0071d00 2024-01-15T18:19:32.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:32.418+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone39061", "target": ["mon-mgr", ""]}) v1 -- 0x7f94d013e560 con 0x7f94a8067c10 2024-01-15T18:19:37.053 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:37.058+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:37.053 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:37.058+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:37.053 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:37.058+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:37.053 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:37.058+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:37.053 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:19:37.058+0000 7f3cb1a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:37.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.138+0000 7f94c7400640 1 -- 172.21.15.143:0/2620112109 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f94d013e560 con 0x7f94a8067c10 2024-01-15T18:19:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94a8067c10 msgr2=0x7f94a806a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94a8067c10 0x7f94a806a0d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f94b80ad000 tx=0x7f94b8008040 comp rx=0 tx=0).stop 2024-01-15T18:19:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 msgr2=0x7f94d015a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:37.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 0x7f94d015a900 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f94c000c480 tx=0x7f94c000c940 comp rx=0 tx=0).stop 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 shutdown_connections 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f94a8067c10 0x7f94a806a0d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94d0141660 0x7f94d00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f94d00ce560 0x7f94d015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 --2- 172.21.15.143:0/2620112109 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f94d0071d00 0x7f94d015a900 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 >> 172.21.15.143:0/2620112109 conn(0x7f94d006c3f0 msgr2=0x7f94d013de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 shutdown_connections 2024-01-15T18:19:37.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.142+0000 7f94d6400640 1 -- 172.21.15.143:0/2620112109 wait complete. 2024-01-15T18:19:37.159 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone39062 2024-01-15T18:19:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.302+0000 7fa745a00640 1 Processor -- start 2024-01-15T18:19:37.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.302+0000 7fa745a00640 1 -- start start 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa745a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 0x7fa74015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa745a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa74015de90 0x7fa7400ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa745a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7400ae830 con 0x7fa7400ae450 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa745a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7400ad900 con 0x7fa74015de90 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa745a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74015e6c0 con 0x7fa74006aba0 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73ea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa7400ae450 0x7fa74006a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33426/0 (socket says 172.21.15.143:33426) 2024-01-15T18:19:37.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73ea00640 1 -- 172.21.15.143:0/974999278 learned_addr learned my addr 172.21.15.143:0/974999278 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:19:37.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73e000640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 0x7fa74015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 --2- 172.21.15.143:0/974999278 >> v2:172.21.15.204:3300/0 conn(0x7fa74015de90 0x7fa7400ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941114489 0 0) 0x7fa7400ae830 con 0x7fa7400ae450 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa720003530 con 0x7fa7400ae450 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718747755 0 0) 0x7fa720003530 con 0x7fa7400ae450 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 msgr2=0x7fa74015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 0x7fa74015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 >> v2:172.21.15.204:3300/0 conn(0x7fa74015de90 msgr2=0x7fa7400ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 --2- 172.21.15.143:0/974999278 >> v2:172.21.15.204:3300/0 conn(0x7fa74015de90 0x7fa7400ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa740160e20 con 0x7fa7400ae450 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fa730001680 con 0x7fa7400ae450 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 >> v1:172.21.15.143:6789/0 conn(0x7fa7400ae450 legacy=0x7fa74006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 0x7fa720003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa720003f00 0x7fa7200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa720004d10 0x7fa720005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7400ad900 con 0x7fa720004d10 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74015e6c0 con 0x7fa720003f00 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72000b990 con 0x7fa74006aba0 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa720004d10 0x7fa720005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73ea00640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 0x7fa720003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73e000640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa720003f00 0x7fa7200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 -- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 msgr2=0x7fa720003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa74006aba0 0x7fa720003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 -- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa720003f00 msgr2=0x7fa7200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa720003f00 0x7fa7200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.306+0000 7fa73f400640 1 -- 172.21.15.143:0/974999278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720003530 con 0x7fa720004d10 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa73f400640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa720004d10 0x7fa720005190 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa738009fc0 tx=0x7fa73801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7339d662e76e600e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380a5cb0 con 0x7fa720004d10 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73800f040 con 0x7fa720004d10 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa73d600640 1 -- 172.21.15.143:0/974999278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380ad680 con 0x7fa720004d10 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa745a00640 1 -- 172.21.15.143:0/974999278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa720004d10 msgr2=0x7fa720005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa745a00640 1 --2- 172.21.15.143:0/974999278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa720004d10 0x7fa720005190 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fa738009fc0 tx=0x7fa73801cc40 comp rx=0 tx=0).stop 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa73f400640 1 -- 172.21.15.143:0/974999278 reap_dead start 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa745a00640 1 -- 172.21.15.143:0/974999278 shutdown_connections 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa745a00640 1 -- 172.21.15.143:0/974999278 >> 172.21.15.143:0/974999278 conn(0x7fa7400cfd20 msgr2=0x7fa7400d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa745a00640 1 -- 172.21.15.143:0/974999278 shutdown_connections 2024-01-15T18:19:37.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.310+0000 7fa745a00640 1 -- 172.21.15.143:0/974999278 wait complete. 2024-01-15T18:19:37.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 Processor -- start 2024-01-15T18:19:37.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- start start 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa720004d10 0x7fa740154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7400c5fd0 0x7fa7401555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7400c6410 0x7fa74015e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401551c0 con 0x7fa7400c6410 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7400c3850 con 0x7fa7400c5fd0 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7400c4060 con 0x7fa720004d10 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7400c6410 0x7fa74015e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7400c5fd0 0x7fa7401555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73ea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa720004d10 0x7fa740154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7400c5fd0 0x7fa7401555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:60186/0 (socket says 172.21.15.143:60186) 2024-01-15T18:19:37.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73e000640 1 -- 172.21.15.143:0/127738433 learned_addr learned my addr 172.21.15.143:0/127738433 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 -- 172.21.15.143:0/127738433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa720004d10 msgr2=0x7fa740154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 --2- 172.21.15.143:0/127738433 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa720004d10 0x7fa740154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 -- 172.21.15.143:0/127738433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7400c5fd0 msgr2=0x7fa7401555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 --2- 172.21.15.143:0/127738433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7400c5fd0 0x7fa7401555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 -- 172.21.15.143:0/127738433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa740160840 con 0x7fa7400c6410 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa73f400640 1 --2- 172.21.15.143:0/127738433 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7400c6410 0x7fa74015e800 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa720004670 tx=0x7fa738009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7380bc070 con 0x7fa7400c6410 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7380039a0 con 0x7fa7400c6410 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- 172.21.15.143:0/127738433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7400c3aa0 con 0x7fa7400c6410 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa738003b50 con 0x7fa7400c6410 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- 172.21.15.143:0/127738433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa740154de0 con 0x7fa7400c6410 2024-01-15T18:19:37.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.314+0000 7fa745a00640 1 -- 172.21.15.143:0/127738433 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa740154c40 con 0x7fa7400c6410 2024-01-15T18:19:37.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.318+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 40) v1 ==== 87894+0+0 (secure 0 0 0) 0x7fa7380b4040 con 0x7fa7400c6410 2024-01-15T18:19:37.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.318+0000 7fa736a00640 1 --2- 172.21.15.143:0/127738433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa718067c40 0x7fa71806a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:37.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.318+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa73811d0c0 con 0x7fa7400c6410 2024-01-15T18:19:37.313 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.318+0000 7fa73ea00640 1 --2- 172.21.15.143:0/127738433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa718067c40 0x7fa71806a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:37.314 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.318+0000 7fa73ea00640 1 --2- 172.21.15.143:0/127738433 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa718067c40 0x7fa71806a100 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa730000c10 tx=0x7fa730007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:37.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.322+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa740154c40 con 0x7fa7400c6410 2024-01-15T18:19:37.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:37.502+0000 7fa745a00640 1 -- 172.21.15.143:0/127738433 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone39062", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7401616c0 con 0x7fa718067c40 2024-01-15T18:19:39.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:19:39.858+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 41) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fa7380ec240 con 0x7fa7400c6410 2024-01-15T18:20:00.090 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:20:00.093 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:20:30.147 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:20:30.151 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:20:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:20:41.985+0000 7fa736a00640 1 -- 172.21.15.143:0/127738433 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 42) v1 ==== 87894+0+0 (secure 0 0 0) 0x7fa73810d680 con 0x7fa7400c6410 2024-01-15T18:21:00.228 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:21:00.233 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:21:30.342 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:21:30.346 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:21:37.187 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-01-15T18:21:37.188 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:21:37.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fdedaa00640 1 Processor -- start 2024-01-15T18:21:37.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fdedaa00640 1 -- start start 2024-01-15T18:21:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fdedaa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fded4072030 0x7fded415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fdedaa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40ceb50 con 0x7fded4143780 2024-01-15T18:21:37.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fdedaa00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded415c110 con 0x7fded4071bb0 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fdedaa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded415b7e0 con 0x7fded4072030 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fded4143780 0x7fded40cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50720/0 (socket says 172.21.15.143:50720) 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded3e00640 1 -- 172.21.15.143:0/211453170 learned_addr learned my addr 172.21.15.143:0/211453170 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded8a00640 1 --2- 172.21.15.143:0/211453170 >> v2:172.21.15.204:3300/0 conn(0x7fded4072030 0x7fded415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579580713 0 0) 0x7fdec4001120 con 0x7fded4143780 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb4005740 con 0x7fded4143780 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125117024 0 0) 0x7fded415c110 con 0x7fded4071bb0 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec4001120 con 0x7fded4071bb0 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded8a00640 1 -- 172.21.15.143:0/211453170 >> v1:172.21.15.204:6789/0 conn(0x7fded4071bb0 legacy=0x7fded415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded8a00640 1 -- 172.21.15.143:0/211453170 >> v1:172.21.15.143:6789/0 conn(0x7fded4143780 legacy=0x7fded40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.352+0000 7fded8a00640 1 -- 172.21.15.143:0/211453170 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded415bac0 con 0x7fded4072030 2024-01-15T18:21:37.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fded8a00640 1 --2- 172.21.15.143:0/211453170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded415ccb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009d90 tx=0x7fdecc01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ba97a18b8b3b11 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:37.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0a59d0 con 0x7fded4072030 2024-01-15T18:21:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc0a5b30 con 0x7fded4072030 2024-01-15T18:21:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fded2a00640 1 -- 172.21.15.143:0/211453170 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0a5ea0 con 0x7fded4072030 2024-01-15T18:21:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fdedaa00640 1 -- 172.21.15.143:0/211453170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 msgr2=0x7fded415ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fdedaa00640 1 --2- 172.21.15.143:0/211453170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded415ccb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009d90 tx=0x7fdecc01c950 comp rx=0 tx=0).stop 2024-01-15T18:21:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fdedaa00640 1 -- 172.21.15.143:0/211453170 shutdown_connections 2024-01-15T18:21:37.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fdedaa00640 1 --2- 172.21.15.143:0/211453170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded415ccb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.356+0000 7fdedaa00640 1 -- 172.21.15.143:0/211453170 >> 172.21.15.143:0/211453170 conn(0x7fded406c3f0 msgr2=0x7fded406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 -- 172.21.15.143:0/211453170 shutdown_connections 2024-01-15T18:21:37.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 -- 172.21.15.143:0/211453170 wait complete. 2024-01-15T18:21:37.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 Processor -- start 2024-01-15T18:21:37.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 -- start start 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 0x7fded41f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded4070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fded4143780 0x7fded4071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4132ea0 con 0x7fded4071bb0 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4133180 con 0x7fded4072030 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdedaa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded4133460 con 0x7fded4143780 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 0x7fded41f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 0x7fded41f1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38776/0 (socket says 172.21.15.143:38776) 2024-01-15T18:21:37.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 -- 172.21.15.143:0/4061763054 learned_addr learned my addr 172.21.15.143:0/4061763054 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3400640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded4070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded8a00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fded4143780 0x7fded4071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 -- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fded4143780 msgr2=0x7fded4071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fded4143780 0x7fded4071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 -- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 msgr2=0x7fded4070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded4070e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 -- 172.21.15.143:0/4061763054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc009a70 con 0x7fded4071bb0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fded3e00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 0x7fded41f1b40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c700 tx=0x7fdec400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.360+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4005ac0 con 0x7fded4071bb0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec4005c40 con 0x7fded4071bb0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4007070 con 0x7fded4071bb0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded4133740 con 0x7fded4071bb0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded41f2080 con 0x7fded4071bb0 2024-01-15T18:21:37.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded41f2300 con 0x7fded4071bb0 2024-01-15T18:21:37.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 42) v1 ==== 87894+0+0 (secure 0 0 0) 0x7fdec40071f0 con 0x7fded4071bb0 2024-01-15T18:21:37.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdecbe00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeac067c10 0x7fdeac06a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fdec4082940 con 0x7fded4071bb0 2024-01-15T18:21:37.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.364+0000 7fded3400640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeac067c10 0x7fdeac06a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.368+0000 7fded3400640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeac067c10 0x7fdeac06a0d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007fd0 tx=0x7fdebc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:37.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.368+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fdec4051290 con 0x7fded4071bb0 2024-01-15T18:21:37.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.536+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fded4070aa0 con 0x7fdeac067c10 2024-01-15T18:21:37.535 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdecbe00640 1 -- 172.21.15.143:0/4061763054 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdebc001360 con 0x7fdeac067c10 2024-01-15T18:21:37.535 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:21:37.535 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:21:37.535 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "cephfs" 2024-01-15T18:21:37.535 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:21:37.535 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:21:37.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeac067c10 msgr2=0x7fdeac06a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeac067c10 0x7fdeac06a0d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdebc007fd0 tx=0x7fdebc009040 comp rx=0 tx=0).stop 2024-01-15T18:21:37.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 msgr2=0x7fded41f1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 0x7fded41f1b40 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c700 tx=0x7fdec400cbc0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 shutdown_connections 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fdeac067c10 0x7fdeac06a0d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fded4143780 0x7fded4071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fded4072030 0x7fded4070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 --2- 172.21.15.143:0/4061763054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fded4071bb0 0x7fded41f1b40 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 >> 172.21.15.143:0/4061763054 conn(0x7fded406c3f0 msgr2=0x7fded40af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 shutdown_connections 2024-01-15T18:21:37.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.540+0000 7fdedaa00640 1 -- 172.21.15.143:0/4061763054 wait complete. 2024-01-15T18:21:37.556 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:21:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 Processor -- start 2024-01-15T18:21:37.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 -- start start 2024-01-15T18:21:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f225815f1b0 0x7f225806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22580cc810 con 0x7f225815bf40 2024-01-15T18:21:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225806e9f0 con 0x7f225815f1b0 2024-01-15T18:21:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f225d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225815ea80 con 0x7f2258070cc0 2024-01-15T18:21:37.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2255400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f225815bf40 0x7f22580710a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50732/0 (socket says 172.21.15.143:50732) 2024-01-15T18:21:37.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2255400640 1 -- 172.21.15.143:0/3854030237 learned_addr learned my addr 172.21.15.143:0/3854030237 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:37.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2256800640 1 --2- 172.21.15.143:0/3854030237 >> v2:172.21.15.204:3300/0 conn(0x7f225815f1b0 0x7f225806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2255e00640 1 --2- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2254a00640 1 -- 172.21.15.143:0/3854030237 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440259962 0 0) 0x7f2240001120 con 0x7f225815bf40 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2254a00640 1 -- 172.21.15.143:0/3854030237 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2238003530 con 0x7f225815bf40 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2254a00640 1 -- 172.21.15.143:0/3854030237 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289112030 0 0) 0x7f2238003530 con 0x7f225815bf40 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2256800640 1 -- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 msgr2=0x7f225806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2256800640 1 --2- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2256800640 1 -- 172.21.15.143:0/3854030237 >> v1:172.21.15.143:6789/0 conn(0x7f225815bf40 legacy=0x7f22580710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.700+0000 7f2256800640 1 -- 172.21.15.143:0/3854030237 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225815c380 con 0x7f225815f1b0 2024-01-15T18:21:37.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f2256800640 1 --2- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815f1b0 0x7f225806a490 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2250009dc0 tx=0x7f225001c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deaab170362750e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:37.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f2254a00640 1 -- 172.21.15.143:0/3854030237 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240001810 con 0x7f225815f1b0 2024-01-15T18:21:37.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f2254a00640 1 -- 172.21.15.143:0/3854030237 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22500a5660 con 0x7f225815f1b0 2024-01-15T18:21:37.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f2254a00640 1 -- 172.21.15.143:0/3854030237 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500a59f0 con 0x7f225815f1b0 2024-01-15T18:21:37.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 -- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815f1b0 msgr2=0x7f225806a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.700 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 --2- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815f1b0 0x7f225806a490 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2250009dc0 tx=0x7f225001c820 comp rx=0 tx=0).stop 2024-01-15T18:21:37.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 -- 172.21.15.143:0/3854030237 shutdown_connections 2024-01-15T18:21:37.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 --2- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815f1b0 0x7f225806a490 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 --2- 172.21.15.143:0/3854030237 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225806dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 -- 172.21.15.143:0/3854030237 >> 172.21.15.143:0/3854030237 conn(0x7f22580b1610 msgr2=0x7f22580b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:37.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 -- 172.21.15.143:0/3854030237 shutdown_connections 2024-01-15T18:21:37.701 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.704+0000 7f225d200640 1 -- 172.21.15.143:0/3854030237 wait complete. 2024-01-15T18:21:37.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 Processor -- start 2024-01-15T18:21:37.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 -- start start 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815bf40 0x7f22580b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 0x7f225814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22581504f0 con 0x7f225815f1b0 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225814fea0 con 0x7f225815bf40 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225814dcd0 con 0x7f2258070cc0 2024-01-15T18:21:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 0x7f225814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 0x7f225814e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38792/0 (socket says 172.21.15.143:38792) 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 -- 172.21.15.143:0/3626129248 learned_addr learned my addr 172.21.15.143:0/3626129248 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2255400640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815bf40 0x7f22580b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2255e00640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 -- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 msgr2=0x7f225815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 -- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815bf40 msgr2=0x7f22580b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815bf40 0x7f22580b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 -- 172.21.15.143:0/3626129248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2250009a70 con 0x7f225815f1b0 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2255400640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815bf40 0x7f22580b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2255e00640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f2256800640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 0x7f225814e890 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f22500b0040 tx=0x7f22500195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500b9860 con 0x7f225815f1b0 2024-01-15T18:21:37.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22500b99e0 con 0x7f225815f1b0 2024-01-15T18:21:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22500b9d90 con 0x7f225815f1b0 2024-01-15T18:21:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22580b2650 con 0x7f225815f1b0 2024-01-15T18:21:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.708+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22580b2300 con 0x7f225815f1b0 2024-01-15T18:21:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.712+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2258074b00 con 0x7f225815f1b0 2024-01-15T18:21:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.712+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 42) v1 ==== 87894+0+0 (secure 0 0 0) 0x7f22500c4460 con 0x7f225815f1b0 2024-01-15T18:21:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.712+0000 7f224e000640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2230067c10 0x7f223006a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.712+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f225011e620 con 0x7f225815f1b0 2024-01-15T18:21:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.712+0000 7f2255e00640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2230067c10 0x7f223006a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.712+0000 7f2255e00640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2230067c10 0x7f223006a0d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2258153690 tx=0x7f2248008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.716+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f22500ecfd0 con 0x7f225815f1b0 2024-01-15T18:21:37.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.888+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f225806ade0 con 0x7f2230067c10 2024-01-15T18:21:37.889 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:37.892+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:21:37.889 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:37.892+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:21:37.889 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:37.892+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:21:37.889 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:37.892+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:21:37.889 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:37.892+0000 7f3caa200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:21:37.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.904+0000 7f224e000640 1 -- 172.21.15.143:0/3626129248 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+185 (secure 0 0 0) 0x7f2248010930 con 0x7f2230067c10 2024-01-15T18:21:37.901 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:37.901 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:21:37.901 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:21:37.901 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "sv1" 2024-01-15T18:21:37.901 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone39063" 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone39062" 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "subvol_clone39061" 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:21:37.902 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2230067c10 msgr2=0x7f223006a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2230067c10 0x7f223006a0d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2258153690 tx=0x7f2248008040 comp rx=0 tx=0).stop 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 msgr2=0x7f225814e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 0x7f225814e890 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f22500b0040 tx=0x7f22500195c0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 shutdown_connections 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2230067c10 0x7f223006a0d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f225815f1b0 0x7f225814e890 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f225815bf40 0x7f22580b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 --2- 172.21.15.143:0/3626129248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2258070cc0 0x7f225815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 >> 172.21.15.143:0/3626129248 conn(0x7f22580b1610 msgr2=0x7f225815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:37.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 shutdown_connections 2024-01-15T18:21:37.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:37.908+0000 7f225d200640 1 -- 172.21.15.143:0/3626129248 wait complete. 2024-01-15T18:21:37.923 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:21:38.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 Processor -- start 2024-01-15T18:21:38.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 -- start start 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f5d9c15b610 0x7f5d9c140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c071920 0x7f5d9c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5d9c06a490 0x7f5d9c06a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c13e420 con 0x7f5d9c06a490 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c1416a0 con 0x7f5d9c15b610 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5da0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06ae60 con 0x7f5d9c071920 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5d9c06a490 0x7f5d9c06a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.069 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5d9c06a490 0x7f5d9c06a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38820/0 (socket says 172.21.15.143:38820) 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 -- 172.21.15.143:0/1310004463 learned_addr learned my addr 172.21.15.143:0/1310004463 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d99400640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c071920 0x7f5d9c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d99e00640 1 --2- 172.21.15.143:0/1310004463 >> v2:172.21.15.204:3300/0 conn(0x7f5d9c15b610 0x7f5d9c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 -- 172.21.15.143:0/1310004463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c071920 msgr2=0x7f5d9c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c071920 0x7f5d9c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 -- 172.21.15.143:0/1310004463 >> v2:172.21.15.204:3300/0 conn(0x7f5d9c15b610 msgr2=0x7f5d9c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 --2- 172.21.15.143:0/1310004463 >> v2:172.21.15.204:3300/0 conn(0x7f5d9c15b610 0x7f5d9c140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d9a800640 1 -- 172.21.15.143:0/1310004463 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c140e00 con 0x7f5d9c06a490 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d99400640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c071920 0x7f5d9c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:38.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.072+0000 7f5d99e00640 1 --2- 172.21.15.143:0/1310004463 >> v2:172.21.15.204:3300/0 conn(0x7f5d9c15b610 0x7f5d9c140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:38.071 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5d9a800640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c06a490 0x7f5d9c06a920 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c06b470 tx=0x7f5d9401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366eb979e4ebc666 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5d98a00640 1 -- 172.21.15.143:0/1310004463 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d940aa620 con 0x7f5d9c06a490 2024-01-15T18:21:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5d98a00640 1 -- 172.21.15.143:0/1310004463 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d940aa780 con 0x7f5d9c06a490 2024-01-15T18:21:38.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5d98a00640 1 -- 172.21.15.143:0/1310004463 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d940aab10 con 0x7f5d9c06a490 2024-01-15T18:21:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 -- 172.21.15.143:0/1310004463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c06a490 msgr2=0x7f5d9c06a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c06a490 0x7f5d9c06a920 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c06b470 tx=0x7f5d9401c500 comp rx=0 tx=0).stop 2024-01-15T18:21:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 -- 172.21.15.143:0/1310004463 shutdown_connections 2024-01-15T18:21:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c06a490 0x7f5d9c06a920 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1310004463 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c071920 0x7f5d9c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.073 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1310004463 >> v2:172.21.15.204:3300/0 conn(0x7f5d9c15b610 0x7f5d9c140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 -- 172.21.15.143:0/1310004463 >> 172.21.15.143:0/1310004463 conn(0x7f5d9c0b1b90 msgr2=0x7f5d9c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:38.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 -- 172.21.15.143:0/1310004463 shutdown_connections 2024-01-15T18:21:38.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.076+0000 7f5da0e00640 1 -- 172.21.15.143:0/1310004463 wait complete. 2024-01-15T18:21:38.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 Processor -- start 2024-01-15T18:21:38.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 -- start start 2024-01-15T18:21:38.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d9c06a490 0x7f5d9c138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 0x7f5d9c0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.076 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c15b610 0x7f5d9c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b4460 con 0x7f5d9c071920 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c138960 con 0x7f5d9c06a490 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c13dc40 con 0x7f5d9c15b610 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 0x7f5d9c0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 0x7f5d9c0b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38832/0 (socket says 172.21.15.143:38832) 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 -- 172.21.15.143:0/1736057038 learned_addr learned my addr 172.21.15.143:0/1736057038 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d9c06a490 0x7f5d9c138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d9a800640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c15b610 0x7f5d9c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 -- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c15b610 msgr2=0x7f5d9c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c15b610 0x7f5d9c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 -- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d9c06a490 msgr2=0x7f5d9c138050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d9c06a490 0x7f5d9c138050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 -- 172.21.15.143:0/1736057038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940094d0 con 0x7f5d9c071920 2024-01-15T18:21:38.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d99400640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 0x7f5d9c0b39e0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c7c0 tx=0x7f5d8400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84007a30 con 0x7f5d9c071920 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d84007b90 con 0x7f5d9c071920 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c13f620 con 0x7f5d9c071920 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8400ce70 con 0x7f5d9c071920 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.080+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c138c40 con 0x7f5d9c071920 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.084+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c067500 con 0x7f5d9c071920 2024-01-15T18:21:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.084+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 42) v1 ==== 87894+0+0 (secure 0 0 0) 0x7f5d840113f0 con 0x7f5d9c071920 2024-01-15T18:21:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.084+0000 7f5d92000640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d74067c10 0x7f5d7406a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.084+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f5d840834f0 con 0x7f5d9c071920 2024-01-15T18:21:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.084+0000 7f5d99e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d74067c10 0x7f5d7406a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.084+0000 7f5d99e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d74067c10 0x7f5d7406a0d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d0840 tx=0x7f5d8c007770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:38.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.088+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5d84051e90 con 0x7f5d9c071920 2024-01-15T18:21:38.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.264+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d9c15b2f0 con 0x7f5d74067c10 2024-01-15T18:21:38.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.276+0000 7f5d92000640 1 -- 172.21.15.143:0/1736057038 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f5d9c15b2f0 con 0x7f5d74067c10 2024-01-15T18:21:38.273 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:38.273 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2024-01-15T18:21:38.273 INFO:teuthology.orchestra.run.smithi143.stdout: { 2024-01-15T18:21:38.273 INFO:teuthology.orchestra.run.smithi143.stdout: "name": "ss1" 2024-01-15T18:21:38.273 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-01-15T18:21:38.273 INFO:teuthology.orchestra.run.smithi143.stdout:] 2024-01-15T18:21:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d74067c10 msgr2=0x7f5d7406a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d74067c10 0x7f5d7406a0d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d0840 tx=0x7f5d8c007770 comp rx=0 tx=0).stop 2024-01-15T18:21:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 msgr2=0x7f5d9c0b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 0x7f5d9c0b39e0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400c7c0 tx=0x7f5d8400cc80 comp rx=0 tx=0).stop 2024-01-15T18:21:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 shutdown_connections 2024-01-15T18:21:38.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5d74067c10 0x7f5d7406a0d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5d9c15b610 0x7f5d9c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5d9c071920 0x7f5d9c0b39e0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 --2- 172.21.15.143:0/1736057038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d9c06a490 0x7f5d9c138050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 >> 172.21.15.143:0/1736057038 conn(0x7f5d9c0b1b90 msgr2=0x7f5d9c15d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 shutdown_connections 2024-01-15T18:21:38.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.280+0000 7f5da0e00640 1 -- 172.21.15.143:0/1736057038 wait complete. 2024-01-15T18:21:38.294 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.440+0000 7f5a4de00640 1 Processor -- start 2024-01-15T18:21:38.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.440+0000 7f5a4de00640 1 -- start start 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a4de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a480ce520 0x7f5a48071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a4de00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a480ceb50 con 0x7f5a480720f0 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a4de00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a48072580 con 0x7f5a480ce520 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a4de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815c110 con 0x7f5a48143780 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a480ce520 0x7f5a48071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5a480ce520 0x7f5a48071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38834/0 (socket says 172.21.15.143:38834) 2024-01-15T18:21:38.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 -- 172.21.15.143:0/1298404436 learned_addr learned my addr 172.21.15.143:0/1298404436 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46a00640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 -- 172.21.15.143:0/1298404436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 msgr2=0x7f5a480cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 -- 172.21.15.143:0/1298404436 >> v1:172.21.15.204:6789/0 conn(0x7f5a480720f0 legacy=0x7f5a4815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 -- 172.21.15.143:0/1298404436 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4815b7e0 con 0x7f5a480ce520 2024-01-15T18:21:38.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46a00640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:38.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.444+0000 7f5a46000640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480ce520 0x7f5a48071bb0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009870 tx=0x7f5a3001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da6441506e8c9e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:38.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a45600640 1 -- 172.21.15.143:0/1298404436 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a300ac630 con 0x7f5a480ce520 2024-01-15T18:21:38.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a45600640 1 -- 172.21.15.143:0/1298404436 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a300ac790 con 0x7f5a480ce520 2024-01-15T18:21:38.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a45600640 1 -- 172.21.15.143:0/1298404436 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a300acb20 con 0x7f5a480ce520 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- 172.21.15.143:0/1298404436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480ce520 msgr2=0x7f5a48071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480ce520 0x7f5a48071bb0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009870 tx=0x7f5a3001c500 comp rx=0 tx=0).stop 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- 172.21.15.143:0/1298404436 shutdown_connections 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480ce520 0x7f5a48071bb0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 --2- 172.21.15.143:0/1298404436 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- 172.21.15.143:0/1298404436 >> 172.21.15.143:0/1298404436 conn(0x7f5a4806c3f0 msgr2=0x7f5a4806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- 172.21.15.143:0/1298404436 shutdown_connections 2024-01-15T18:21:38.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- 172.21.15.143:0/1298404436 wait complete. 2024-01-15T18:21:38.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 Processor -- start 2024-01-15T18:21:38.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- start start 2024-01-15T18:21:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480720f0 0x7f5a481676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a480ce520 0x7f5a48165240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48139720 con 0x7f5a480720f0 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48139a00 con 0x7f5a480ce520 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.448+0000 7f5a4de00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48139ce0 con 0x7f5a48143780 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480720f0 0x7f5a481676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480720f0 0x7f5a481676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38848/0 (socket says 172.21.15.143:38848) 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 -- 172.21.15.143:0/3489158188 learned_addr learned my addr 172.21.15.143:0/3489158188 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46000640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a480ce520 0x7f5a48165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a47400640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 -- 172.21.15.143:0/3489158188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 msgr2=0x7f5a480af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f5a48143780 0x7f5a480af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 -- 172.21.15.143:0/3489158188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a480ce520 msgr2=0x7f5a48165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:38.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a480ce520 0x7f5a48165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 -- 172.21.15.143:0/3489158188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a300094d0 con 0x7f5a480720f0 2024-01-15T18:21:38.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a46a00640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a480720f0 0x7f5a481676b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c7f0 tx=0x7f5a3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38007820 con 0x7f5a480720f0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38007980 con 0x7f5a480720f0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a4de00640 1 -- 172.21.15.143:0/3489158188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48166eb0 con 0x7f5a480720f0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38007d30 con 0x7f5a480720f0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a4de00640 1 -- 172.21.15.143:0/3489158188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48165780 con 0x7f5a480720f0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a4de00640 1 -- 172.21.15.143:0/3489158188 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48166d10 con 0x7f5a480720f0 2024-01-15T18:21:38.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.452+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 42) v1 ==== 87894+0+0 (secure 0 0 0) 0x7f5a380113f0 con 0x7f5a480720f0 2024-01-15T18:21:38.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.456+0000 7f5a3f400640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5a20067c10 0x7f5a2006a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:38.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.456+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f5a380836c0 con 0x7f5a480720f0 2024-01-15T18:21:38.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.456+0000 7f5a46000640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5a20067c10 0x7f5a2006a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:38.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.456+0000 7f5a46000640 1 --2- 172.21.15.143:0/3489158188 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5a20067c10 0x7f5a2006a0d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5a300ae000 tx=0x7f5a30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:38.454 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.460+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f5a48166d10 con 0x7f5a480720f0 2024-01-15T18:21:38.631 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.636+0000 7f5a4de00640 1 -- 172.21.15.143:0/3489158188 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a48139b70 con 0x7f5a20067c10 2024-01-15T18:21:38.636 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:38.640+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:38.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:38.640+0000 7f5a3f400640 1 -- 172.21.15.143:0/3489158188 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5a48139b70 con 0x7f5a20067c10 2024-01-15T18:21:38.637 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:38.637 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:38.646 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:40.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:40.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd61e800640 1 Processor -- start 2024-01-15T18:21:40.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd61e800640 1 -- start start 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd61e800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd6180c6230 0x7fd6180cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd61e800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6180c4d90 con 0x7fd6180cdf20 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd61e800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61815b640 con 0x7fd61815b260 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd61e800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6180cf800 con 0x7fd6180c6230 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd616a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd6180cdf20 0x7fd6180cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50744/0 (socket says 172.21.15.143:50744) 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd616a00640 1 -- 172.21.15.143:0/10217598 learned_addr learned my addr 172.21.15.143:0/10217598 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd617e00640 1 --2- 172.21.15.143:0/10217598 >> v2:172.21.15.204:3300/0 conn(0x7fd6180c6230 0x7fd6180cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd616000640 1 -- 172.21.15.143:0/10217598 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838655284 0 0) 0x7fd6180c4d90 con 0x7fd6180cdf20 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.804+0000 7fd616000640 1 -- 172.21.15.143:0/10217598 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5f8003530 con 0x7fd6180cdf20 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd617e00640 1 -- 172.21.15.143:0/10217598 >> v1:172.21.15.204:6789/0 conn(0x7fd61815b260 legacy=0x7fd6180cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd617e00640 1 -- 172.21.15.143:0/10217598 >> v1:172.21.15.143:6789/0 conn(0x7fd6180cdf20 legacy=0x7fd6180cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:40.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd616000640 1 -- 172.21.15.143:0/10217598 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827655184 0 0) 0x7fd5f8003530 con 0x7fd6180cdf20 2024-01-15T18:21:40.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd617e00640 1 -- 172.21.15.143:0/10217598 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6180cfae0 con 0x7fd6180c6230 2024-01-15T18:21:40.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd617e00640 1 --2- 172.21.15.143:0/10217598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 0x7fd6180cc1f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd610009d90 tx=0x7fd61001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38bb8cdddf4d0faa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd616000640 1 -- 172.21.15.143:0/10217598 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6100a59e0 con 0x7fd6180c6230 2024-01-15T18:21:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd616000640 1 -- 172.21.15.143:0/10217598 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6180ccf40 con 0x7fd6180c6230 2024-01-15T18:21:40.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.808+0000 7fd616000640 1 -- 172.21.15.143:0/10217598 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6100a5d60 con 0x7fd6180c6230 2024-01-15T18:21:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- 172.21.15.143:0/10217598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 msgr2=0x7fd6180cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 --2- 172.21.15.143:0/10217598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 0x7fd6180cc1f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd610009d90 tx=0x7fd61001c920 comp rx=0 tx=0).stop 2024-01-15T18:21:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- 172.21.15.143:0/10217598 shutdown_connections 2024-01-15T18:21:40.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 --2- 172.21.15.143:0/10217598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 0x7fd6180cc1f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:40.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- 172.21.15.143:0/10217598 >> 172.21.15.143:0/10217598 conn(0x7fd6180d0ed0 msgr2=0x7fd6180d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:40.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- 172.21.15.143:0/10217598 shutdown_connections 2024-01-15T18:21:40.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- 172.21.15.143:0/10217598 wait complete. 2024-01-15T18:21:40.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 Processor -- start 2024-01-15T18:21:40.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- start start 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 0x7fd61806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6180cdf20 0x7fd6180b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd61815b260 0x7fd6180b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c8ae0 con 0x7fd6180cdf20 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618152590 con 0x7fd6180c6230 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd61e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6180c8140 con 0x7fd61815b260 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6180cdf20 0x7fd6180b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6180cdf20 0x7fd6180b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38860/0 (socket says 172.21.15.143:38860) 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 -- 172.21.15.143:0/3973649663 learned_addr learned my addr 172.21.15.143:0/3973649663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd617400640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 0x7fd61806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:40.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd617e00640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd61815b260 0x7fd6180b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 -- 172.21.15.143:0/3973649663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd61815b260 msgr2=0x7fd6180b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd61815b260 0x7fd6180b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 -- 172.21.15.143:0/3973649663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 msgr2=0x7fd61806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd6180c6230 0x7fd61806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.812+0000 7fd616a00640 1 -- 172.21.15.143:0/3973649663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610009a70 con 0x7fd6180cdf20 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd616a00640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd6180cdf20 0x7fd6180b27c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fd60000c3e0 tx=0x7fd60000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600005cf0 con 0x7fd6180cdf20 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600005e70 con 0x7fd6180cdf20 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd61e800640 1 -- 172.21.15.143:0/3973649663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6180c8420 con 0x7fd6180cdf20 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000072f0 con 0x7fd6180cdf20 2024-01-15T18:21:40.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd61e800640 1 -- 172.21.15.143:0/3973649663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6180b2d00 con 0x7fd6180cdf20 2024-01-15T18:21:40.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd61e800640 1 -- 172.21.15.143:0/3973649663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6180b2f50 con 0x7fd6180cdf20 2024-01-15T18:21:40.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fd600007510 con 0x7fd6180cdf20 2024-01-15T18:21:40.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd60f400640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5f0067ca0 0x7fd5f006a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:40.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd600082ab0 con 0x7fd6180cdf20 2024-01-15T18:21:40.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.816+0000 7fd617400640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5f0067ca0 0x7fd5f006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:40.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.820+0000 7fd617400640 1 --2- 172.21.15.143:0/3973649663 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd5f0067ca0 0x7fd5f006a160 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd608007fd0 tx=0x7fd608009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:40.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:40.820+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd6180b2f50 con 0x7fd6180cdf20 2024-01-15T18:21:40.997 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:41.000+0000 7fd61e800640 1 -- 172.21.15.143:0/3973649663 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6180c88d0 con 0x7fd5f0067ca0 2024-01-15T18:21:41.003 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:41.008+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:41.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:41.008+0000 7fd60f400640 1 -- 172.21.15.143:0/3973649663 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd6080014f0 con 0x7fd5f0067ca0 2024-01-15T18:21:41.003 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:41.004 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:41.012 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:43.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.324+0000 7f0e0f000640 1 Processor -- start 2024-01-15T18:21:43.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.324+0000 7f0e0f000640 1 -- start start 2024-01-15T18:21:43.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 0x7f0e080d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:43.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0f000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0e080cbef0 0x7f0e080cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:43.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0f000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e080c6c70 con 0x7f0e08099a70 2024-01-15T18:21:43.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0f000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e080b29a0 con 0x7f0e080cbef0 2024-01-15T18:21:43.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080d0330 con 0x7f0e080c5f60 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0dc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0e08099a70 0x7f0e08099e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50746/0 (socket says 172.21.15.143:50746) 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0dc00640 1 -- 172.21.15.143:0/1803822855 learned_addr learned my addr 172.21.15.143:0/1803822855 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0e600640 1 --2- 172.21.15.143:0/1803822855 >> v2:172.21.15.143:3300/0 conn(0x7f0e080cbef0 0x7f0e080cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0d200640 1 --2- 172.21.15.143:0/1803822855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 0x7f0e080d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0e600640 1 -- 172.21.15.143:0/1803822855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 msgr2=0x7f0e080d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0e600640 1 --2- 172.21.15.143:0/1803822855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 0x7f0e080d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0e600640 1 -- 172.21.15.143:0/1803822855 >> v1:172.21.15.143:6789/0 conn(0x7f0e08099a70 legacy=0x7f0e08099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:43.326 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.328+0000 7f0e0e600640 1 -- 172.21.15.143:0/1803822855 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0815b4d0 con 0x7f0e080cbef0 2024-01-15T18:21:43.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0e600640 1 --2- 172.21.15.143:0/1803822855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e080cc380 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009e10 tx=0x7f0e00025160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55243c822936c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:43.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e07e00640 1 -- 172.21.15.143:0/1803822855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000b4400 con 0x7f0e080cbef0 2024-01-15T18:21:43.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e07e00640 1 -- 172.21.15.143:0/1803822855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000b0050 con 0x7f0e080cbef0 2024-01-15T18:21:43.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e07e00640 1 -- 172.21.15.143:0/1803822855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000b4780 con 0x7f0e080cbef0 2024-01-15T18:21:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 -- 172.21.15.143:0/1803822855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 msgr2=0x7f0e080cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 --2- 172.21.15.143:0/1803822855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e080cc380 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009e10 tx=0x7f0e00025160 comp rx=0 tx=0).stop 2024-01-15T18:21:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 -- 172.21.15.143:0/1803822855 shutdown_connections 2024-01-15T18:21:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 --2- 172.21.15.143:0/1803822855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e080cc380 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 --2- 172.21.15.143:0/1803822855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 0x7f0e080d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:43.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 -- 172.21.15.143:0/1803822855 >> 172.21.15.143:0/1803822855 conn(0x7f0e080d0e90 msgr2=0x7f0e080d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:43.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 -- 172.21.15.143:0/1803822855 shutdown_connections 2024-01-15T18:21:43.334 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.336+0000 7f0e0f000640 1 -- 172.21.15.143:0/1803822855 wait complete. 2024-01-15T18:21:43.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.340+0000 7f0e0f000640 1 Processor -- start 2024-01-15T18:21:43.336 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.340+0000 7f0e0f000640 1 -- start start 2024-01-15T18:21:43.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e08099a70 0x7f0e080b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:43.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 0x7f0e08158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:43.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e081569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:43.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08156f20 con 0x7f0e080cbef0 2024-01-15T18:21:43.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081571a0 con 0x7f0e08099a70 2024-01-15T18:21:43.339 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08157480 con 0x7f0e080c5f60 2024-01-15T18:21:43.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e081569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:43.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e081569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38878/0 (socket says 172.21.15.143:38878) 2024-01-15T18:21:43.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 -- 172.21.15.143:0/3023163 learned_addr learned my addr 172.21.15.143:0/3023163 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:43.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 -- 172.21.15.143:0/3023163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 msgr2=0x7f0e08158240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T18:21:43.340 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 --2- 172.21.15.143:0/3023163 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0e080c5f60 0x7f0e08158240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:43.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 -- 172.21.15.143:0/3023163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e08099a70 msgr2=0x7f0e080b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:43.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 --2- 172.21.15.143:0/3023163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e08099a70 0x7f0e080b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:43.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 -- 172.21.15.143:0/3023163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00009a70 con 0x7f0e080cbef0 2024-01-15T18:21:43.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0e600640 1 --2- 172.21.15.143:0/3023163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e080cbef0 0x7f0e081569e0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0e000b7040 tx=0x7f0e00021f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:43.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000bf680 con 0x7f0e080cbef0 2024-01-15T18:21:43.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000bf7e0 con 0x7f0e080cbef0 2024-01-15T18:21:43.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 -- 172.21.15.143:0/3023163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e08157760 con 0x7f0e080cbef0 2024-01-15T18:21:43.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000bfb70 con 0x7f0e080cbef0 2024-01-15T18:21:43.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.344+0000 7f0e0f000640 1 -- 172.21.15.143:0/3023163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08157d20 con 0x7f0e080cbef0 2024-01-15T18:21:43.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.348+0000 7f0e0f000640 1 -- 172.21.15.143:0/3023163 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e08157f10 con 0x7f0e080cbef0 2024-01-15T18:21:43.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.348+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f0e000bb040 con 0x7f0e080cbef0 2024-01-15T18:21:43.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.352+0000 7f0e05600640 1 --2- 172.21.15.143:0/3023163 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0de0067ca0 0x7f0de006a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:43.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.352+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f0e00126e00 con 0x7f0e080cbef0 2024-01-15T18:21:43.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.352+0000 7f0e0dc00640 1 --2- 172.21.15.143:0/3023163 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0de0067ca0 0x7f0de006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:43.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.356+0000 7f0e0dc00640 1 --2- 172.21.15.143:0/3023163 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0de0067ca0 0x7f0de006a160 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0e08156930 tx=0x7f0df8007a50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.356+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0e000f56e0 con 0x7f0e080cbef0 2024-01-15T18:21:43.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.700+0000 7f0e0f000640 1 -- 172.21.15.143:0/3023163 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e08067c10 con 0x7f0de0067ca0 2024-01-15T18:21:43.725 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:43.728+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:43.725 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:43.728+0000 7f0e05600640 1 -- 172.21.15.143:0/3023163 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0e08067c10 con 0x7f0de0067ca0 2024-01-15T18:21:43.726 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:43.726 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:43.746 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:45.748 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:45.900 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.904+0000 7f49c8600640 1 Processor -- start 2024-01-15T18:21:45.901 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.904+0000 7f49c8600640 1 -- start start 2024-01-15T18:21:45.902 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c8600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49c00b4ac0 0x7f49c0164540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c8600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b2a20 con 0x7f49c0159b70 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c8600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b2d00 con 0x7f49c0157300 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c8600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00b35a0 con 0x7f49c00b4ac0 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49c00b4ac0 0x7f49c0164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49c00b4ac0 0x7f49c0164540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38892/0 (socket says 172.21.15.143:38892) 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 -- 172.21.15.143:0/2497925508 learned_addr learned my addr 172.21.15.143:0/2497925508 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c5200640 1 --1- 172.21.15.143:0/2497925508 >> v1:172.21.15.143:6789/0 conn(0x7f49c0159b70 0x7f49c00b43b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50754/0 (socket says 172.21.15.143:50754) 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 -- 172.21.15.143:0/2497925508 >> v1:172.21.15.204:6789/0 conn(0x7f49c0157300 legacy=0x7f49c00b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 -- 172.21.15.143:0/2497925508 >> v1:172.21.15.143:6789/0 conn(0x7f49c0159b70 legacy=0x7f49c00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:45.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 -- 172.21.15.143:0/2497925508 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00b2fe0 con 0x7f49c00b4ac0 2024-01-15T18:21:45.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49c6600640 1 --2- 172.21.15.143:0/2497925508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c00b4ac0 0x7f49c0164540 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d150 tx=0x7f49b801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2687cb10a05625b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:45.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.908+0000 7f49bfe00640 1 -- 172.21.15.143:0/2497925508 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800b2d0 con 0x7f49c00b4ac0 2024-01-15T18:21:45.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49bfe00640 1 -- 172.21.15.143:0/2497925508 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b800b430 con 0x7f49c00b4ac0 2024-01-15T18:21:45.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49bfe00640 1 -- 172.21.15.143:0/2497925508 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80a7cb0 con 0x7f49c00b4ac0 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- 172.21.15.143:0/2497925508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c00b4ac0 msgr2=0x7f49c0164540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 --2- 172.21.15.143:0/2497925508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c00b4ac0 0x7f49c0164540 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d150 tx=0x7f49b801eba0 comp rx=0 tx=0).stop 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- 172.21.15.143:0/2497925508 shutdown_connections 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 --2- 172.21.15.143:0/2497925508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c00b4ac0 0x7f49c0164540 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- 172.21.15.143:0/2497925508 >> 172.21.15.143:0/2497925508 conn(0x7f49c006c3f0 msgr2=0x7f49c006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- 172.21.15.143:0/2497925508 shutdown_connections 2024-01-15T18:21:45.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- 172.21.15.143:0/2497925508 wait complete. 2024-01-15T18:21:45.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 Processor -- start 2024-01-15T18:21:45.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- start start 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b4ac0 0x7f49c00ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c0157300 0x7f49c0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c0159b70 0x7f49c0151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0159fd0 con 0x7f49c0159b70 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0151040 con 0x7f49c0157300 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.912+0000 7f49c8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c013b100 con 0x7f49c00b4ac0 2024-01-15T18:21:45.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c0159b70 0x7f49c0151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c0159b70 0x7f49c0151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38896/0 (socket says 172.21.15.143:38896) 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 -- 172.21.15.143:0/2104262330 learned_addr learned my addr 172.21.15.143:0/2104262330 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c5200640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c0157300 0x7f49c0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c5c00640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b4ac0 0x7f49c00ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 -- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b4ac0 msgr2=0x7f49c00ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b4ac0 0x7f49c00ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 -- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c0157300 msgr2=0x7f49c0150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c0157300 0x7f49c0150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 -- 172.21.15.143:0/2104262330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b800bd40 con 0x7f49c0159b70 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c5c00640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49c00b4ac0 0x7f49c00ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:45.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c5200640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49c0157300 0x7f49c0150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:21:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c6600640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49c0159b70 0x7f49c0151890 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f49b801b7b0 tx=0x7f49b801b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b80bd070 con 0x7f49c0159b70 2024-01-15T18:21:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8009890 con 0x7f49c0159b70 2024-01-15T18:21:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8009c40 con 0x7f49c0159b70 2024-01-15T18:21:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c8600640 1 -- 172.21.15.143:0/2104262330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c013bcf0 con 0x7f49c0159b70 2024-01-15T18:21:45.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c8600640 1 -- 172.21.15.143:0/2104262330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c013b6a0 con 0x7f49c0159b70 2024-01-15T18:21:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49c8600640 1 -- 172.21.15.143:0/2104262330 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c013be80 con 0x7f49c0159b70 2024-01-15T18:21:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.916+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f49b80b9430 con 0x7f49c0159b70 2024-01-15T18:21:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.920+0000 7f49bd600640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4998067ca0 0x7f499806a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.920+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f49b811e970 con 0x7f49c0159b70 2024-01-15T18:21:45.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.920+0000 7f49c5c00640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4998067ca0 0x7f499806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:45.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.920+0000 7f49c5c00640 1 --2- 172.21.15.143:0/2104262330 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4998067ca0 0x7f499806a160 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f49b0003e40 tx=0x7f49b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:45.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:45.924+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f49b80ed2f0 con 0x7f49c0159b70 2024-01-15T18:21:46.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:46.104+0000 7f49c8600640 1 -- 172.21.15.143:0/2104262330 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49c013be80 con 0x7f4998067ca0 2024-01-15T18:21:46.106 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:46.112+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:46.107 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:46.112+0000 7f49bd600640 1 -- 172.21.15.143:0/2104262330 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49c013be80 con 0x7f4998067ca0 2024-01-15T18:21:46.107 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:46.107 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:46.115 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:48.117 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:48.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.272+0000 7f40f3e00640 1 Processor -- start 2024-01-15T18:21:48.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.272+0000 7f40f3e00640 1 -- start start 2024-01-15T18:21:48.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40ec0b24b0 0x7f40ec0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:48.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b3bf0 0x7f40ec0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:48.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f3e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec070bf0 con 0x7f40ec0ce690 2024-01-15T18:21:48.270 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f3e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec15dba0 con 0x7f40ec0b24b0 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0b4920 con 0x7f40ec0b3bf0 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f0a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f40ec0b24b0 0x7f40ec0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f1400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f40ec0ce690 0x7f40ec0b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43612/0 (socket says 172.21.15.143:43612) 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f1400640 1 -- 172.21.15.143:0/1894131969 learned_addr learned my addr 172.21.15.143:0/1894131969 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f1e00640 1 --2- 172.21.15.143:0/1894131969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b3bf0 0x7f40ec0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f0a00640 1 -- 172.21.15.143:0/1894131969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b3bf0 msgr2=0x7f40ec0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f0a00640 1 --2- 172.21.15.143:0/1894131969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b3bf0 0x7f40ec0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:48.271 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f0a00640 1 -- 172.21.15.143:0/1894131969 >> v1:172.21.15.143:6789/0 conn(0x7f40ec0ce690 legacy=0x7f40ec0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:48.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f0a00640 1 -- 172.21.15.143:0/1894131969 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0b4c00 con 0x7f40ec0b24b0 2024-01-15T18:21:48.272 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40ebe00640 1 -- 172.21.15.143:0/1894131969 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061931219 0 0) 0x7f40ec070bf0 con 0x7f40ec0ce690 2024-01-15T18:21:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.276+0000 7f40f0a00640 1 --2- 172.21.15.143:0/1894131969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0b24b0 0x7f40ec0ac8e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097c0 tx=0x7f40d401ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fab27a8dccf231 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40ebe00640 1 -- 172.21.15.143:0/1894131969 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40ab670 con 0x7f40ec0b24b0 2024-01-15T18:21:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40ebe00640 1 -- 172.21.15.143:0/1894131969 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d40ab7d0 con 0x7f40ec0b24b0 2024-01-15T18:21:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40ebe00640 1 -- 172.21.15.143:0/1894131969 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d40abb60 con 0x7f40ec0b24b0 2024-01-15T18:21:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 -- 172.21.15.143:0/1894131969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0b24b0 msgr2=0x7f40ec0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:48.274 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 --2- 172.21.15.143:0/1894131969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0b24b0 0x7f40ec0ac8e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f40d40097c0 tx=0x7f40d401ceb0 comp rx=0 tx=0).stop 2024-01-15T18:21:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 -- 172.21.15.143:0/1894131969 shutdown_connections 2024-01-15T18:21:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 --2- 172.21.15.143:0/1894131969 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b3bf0 0x7f40ec0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 --2- 172.21.15.143:0/1894131969 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0b24b0 0x7f40ec0ac8e0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 -- 172.21.15.143:0/1894131969 >> 172.21.15.143:0/1894131969 conn(0x7f40ec06c3f0 msgr2=0x7f40ec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 -- 172.21.15.143:0/1894131969 shutdown_connections 2024-01-15T18:21:48.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 -- 172.21.15.143:0/1894131969 wait complete. 2024-01-15T18:21:48.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 Processor -- start 2024-01-15T18:21:48.277 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 -- start start 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b24b0 0x7f40ec1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40ec0b3bf0 0x7f40ec158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.280+0000 7f40f3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0ce690 0x7f40ec13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f3e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec1445a0 con 0x7f40ec0ce690 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec145830 con 0x7f40ec0b3bf0 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec145b10 con 0x7f40ec0b24b0 2024-01-15T18:21:48.279 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0ce690 0x7f40ec13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0ce690 0x7f40ec13f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39082/0 (socket says 172.21.15.143:39082) 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 -- 172.21.15.143:0/2904832583 learned_addr learned my addr 172.21.15.143:0/2904832583 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f0a00640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40ec0b3bf0 0x7f40ec158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1400640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b24b0 0x7f40ec1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 -- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b24b0 msgr2=0x7f40ec1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b24b0 0x7f40ec1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 -- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40ec0b3bf0 msgr2=0x7f40ec158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40ec0b3bf0 0x7f40ec158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 -- 172.21.15.143:0/2904832583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d4009420 con 0x7f40ec0ce690 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f0a00640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f40ec0b3bf0 0x7f40ec158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1400640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f40ec0b24b0 0x7f40ec1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f1e00640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f40ec0ce690 0x7f40ec13f3f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f40e400efc0 tx=0x7f40e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:48.280 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4009b10 con 0x7f40ec0ce690 2024-01-15T18:21:48.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e400f040 con 0x7f40ec0ce690 2024-01-15T18:21:48.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f3e00640 1 -- 172.21.15.143:0/2904832583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec145df0 con 0x7f40ec0ce690 2024-01-15T18:21:48.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e40146c0 con 0x7f40ec0ce690 2024-01-15T18:21:48.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f3e00640 1 -- 172.21.15.143:0/2904832583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec1463b0 con 0x7f40ec0ce690 2024-01-15T18:21:48.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40f3e00640 1 -- 172.21.15.143:0/2904832583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec1446e0 con 0x7f40ec0ce690 2024-01-15T18:21:48.281 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.284+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f40e4014820 con 0x7f40ec0ce690 2024-01-15T18:21:48.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.288+0000 7f40e9600640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40c4067cd0 0x7f40c406a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:48.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.288+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f40e4083f10 con 0x7f40ec0ce690 2024-01-15T18:21:48.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.288+0000 7f40f1400640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40c4067cd0 0x7f40c406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:48.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.288+0000 7f40f1400640 1 --2- 172.21.15.143:0/2904832583 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f40c4067cd0 0x7f40c406a190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f40dc007fd0 tx=0x7f40dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:48.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.292+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f40ec1446e0 con 0x7f40ec0ce690 2024-01-15T18:21:48.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.468+0000 7f40f3e00640 1 -- 172.21.15.143:0/2904832583 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40ec1446e0 con 0x7f40c4067cd0 2024-01-15T18:21:48.470 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:48.476+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:48.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:48.476+0000 7f40e9600640 1 -- 172.21.15.143:0/2904832583 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f40ec1446e0 con 0x7f40c4067cd0 2024-01-15T18:21:48.471 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:48.471 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:48.479 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:50.481 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:50.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.636+0000 7fcb47800640 1 Processor -- start 2024-01-15T18:21:50.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.636+0000 7fcb47800640 1 -- start start 2024-01-15T18:21:50.633 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.636+0000 7fcb47800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fcb40143780 0x7fcb400cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:50.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.636+0000 7fcb47800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb400ceb50 con 0x7fcb400722c0 2024-01-15T18:21:50.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.636+0000 7fcb47800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4015c110 con 0x7fcb400ce520 2024-01-15T18:21:50.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.636+0000 7fcb47800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4015b7e0 con 0x7fcb40143780 2024-01-15T18:21:50.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb45800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcb400722c0 0x7fcb4015c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43622/0 (socket says 172.21.15.143:43622) 2024-01-15T18:21:50.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb45800640 1 -- 172.21.15.143:0/2433460969 learned_addr learned my addr 172.21.15.143:0/2433460969 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:50.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb44e00640 1 --2- 172.21.15.143:0/2433460969 >> v2:172.21.15.204:3300/0 conn(0x7fcb40143780 0x7fcb400cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb3f400640 1 -- 172.21.15.143:0/2433460969 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976582605 0 0) 0x7fcb380014a0 con 0x7fcb400722c0 2024-01-15T18:21:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb3f400640 1 -- 172.21.15.143:0/2433460969 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb20003530 con 0x7fcb400722c0 2024-01-15T18:21:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb44e00640 1 -- 172.21.15.143:0/2433460969 >> v1:172.21.15.204:6789/0 conn(0x7fcb400ce520 legacy=0x7fcb40071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb44e00640 1 -- 172.21.15.143:0/2433460969 >> v1:172.21.15.143:6789/0 conn(0x7fcb400722c0 legacy=0x7fcb4015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb3f400640 1 -- 172.21.15.143:0/2433460969 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 975859594 0 0) 0x7fcb20003530 con 0x7fcb400722c0 2024-01-15T18:21:50.635 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb44e00640 1 -- 172.21.15.143:0/2433460969 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4015bac0 con 0x7fcb40143780 2024-01-15T18:21:50.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb44e00640 1 --2- 172.21.15.143:0/2433460969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb40143780 0x7fcb400cdfe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcb300097c0 tx=0x7fcb3001c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09ed552351699f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:50.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb3f400640 1 -- 172.21.15.143:0/2433460969 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38001b90 con 0x7fcb40143780 2024-01-15T18:21:50.637 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb3f400640 1 -- 172.21.15.143:0/2433460969 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb300a7050 con 0x7fcb40143780 2024-01-15T18:21:50.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.640+0000 7fcb3f400640 1 -- 172.21.15.143:0/2433460969 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300ab600 con 0x7fcb40143780 2024-01-15T18:21:50.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- 172.21.15.143:0/2433460969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb40143780 msgr2=0x7fcb400cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:50.638 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 --2- 172.21.15.143:0/2433460969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb40143780 0x7fcb400cdfe0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fcb300097c0 tx=0x7fcb3001c3a0 comp rx=0 tx=0).stop 2024-01-15T18:21:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- 172.21.15.143:0/2433460969 shutdown_connections 2024-01-15T18:21:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 --2- 172.21.15.143:0/2433460969 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb40143780 0x7fcb400cdfe0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- 172.21.15.143:0/2433460969 >> 172.21.15.143:0/2433460969 conn(0x7fcb4006c3f0 msgr2=0x7fcb4006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- 172.21.15.143:0/2433460969 shutdown_connections 2024-01-15T18:21:50.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- 172.21.15.143:0/2433460969 wait complete. 2024-01-15T18:21:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 Processor -- start 2024-01-15T18:21:50.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- start start 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb400722c0 0x7fcb40140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb400ce520 0x7fcb40141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb40143780 0x7fcb4014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.644+0000 7fcb47800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4013e5d0 con 0x7fcb400722c0 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb47800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4013d530 con 0x7fcb400ce520 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb47800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4013d810 con 0x7fcb40143780 2024-01-15T18:21:50.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb400722c0 0x7fcb40140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb400722c0 0x7fcb40140bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39088/0 (socket says 172.21.15.143:39088) 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 -- 172.21.15.143:0/421342593 learned_addr learned my addr 172.21.15.143:0/421342593 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb3fe00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb400ce520 0x7fcb40141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb45800640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb40143780 0x7fcb4014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 -- 172.21.15.143:0/421342593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb40143780 msgr2=0x7fcb4014a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fcb40143780 0x7fcb4014a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 -- 172.21.15.143:0/421342593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb400ce520 msgr2=0x7fcb40141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcb400ce520 0x7fcb40141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 -- 172.21.15.143:0/421342593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb30009420 con 0x7fcb400722c0 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb44e00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcb400722c0 0x7fcb40140bf0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fcb30000c10 tx=0x7fcb30019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300b4400 con 0x7fcb400722c0 2024-01-15T18:21:50.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb300b4560 con 0x7fcb400722c0 2024-01-15T18:21:50.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb47800640 1 -- 172.21.15.143:0/421342593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4013daf0 con 0x7fcb400722c0 2024-01-15T18:21:50.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300b48d0 con 0x7fcb400722c0 2024-01-15T18:21:50.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb47800640 1 -- 172.21.15.143:0/421342593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4013e0b0 con 0x7fcb400722c0 2024-01-15T18:21:50.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb47800640 1 -- 172.21.15.143:0/421342593 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4013e2a0 con 0x7fcb400722c0 2024-01-15T18:21:50.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.648+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fcb300b2040 con 0x7fcb400722c0 2024-01-15T18:21:50.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.652+0000 7fcb3cc00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb18067ca0 0x7fcb1806a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:50.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.652+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fcb3011d160 con 0x7fcb400722c0 2024-01-15T18:21:50.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.652+0000 7fcb3fe00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb18067ca0 0x7fcb1806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:50.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.652+0000 7fcb3fe00640 1 --2- 172.21.15.143:0/421342593 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fcb18067ca0 0x7fcb1806a160 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcb400b0960 tx=0x7fcb28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:50.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.656+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fcb300eba40 con 0x7fcb400722c0 2024-01-15T18:21:50.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.832+0000 7fcb47800640 1 -- 172.21.15.143:0/421342593 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb4015d790 con 0x7fcb18067ca0 2024-01-15T18:21:50.834 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:50.840+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:50.840+0000 7fcb3cc00640 1 -- 172.21.15.143:0/421342593 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcb280014f0 con 0x7fcb18067ca0 2024-01-15T18:21:50.834 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:50.843 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:52.844 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 Processor -- start 2024-01-15T18:21:53.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 -- start start 2024-01-15T18:21:53.003 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0301607c0 0x7fd03006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd0300cbf50 0x7fd0300cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0300cffd0 con 0x7fd0300cfb30 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0300d1ce0 con 0x7fd0301607c0 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd036e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0300d02b0 con 0x7fd0300cbf50 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0301607c0 0x7fd03006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd0301607c0 0x7fd03006b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39090/0 (socket says 172.21.15.143:39090) 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 -- 172.21.15.143:0/2331504995 learned_addr learned my addr 172.21.15.143:0/2331504995 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02f400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd0300cfb30 0x7fd03006bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43632/0 (socket says 172.21.15.143:43632) 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd034e00640 1 --2- 172.21.15.143:0/2331504995 >> v2:172.21.15.204:3300/0 conn(0x7fd0300cbf50 0x7fd0300cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 -- 172.21.15.143:0/2331504995 >> v2:172.21.15.204:3300/0 conn(0x7fd0300cbf50 msgr2=0x7fd0300cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:53.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 --2- 172.21.15.143:0/2331504995 >> v2:172.21.15.204:3300/0 conn(0x7fd0300cbf50 0x7fd0300cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 -- 172.21.15.143:0/2331504995 >> v1:172.21.15.143:6789/0 conn(0x7fd0300cfb30 legacy=0x7fd03006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd02fe00640 1 -- 172.21.15.143:0/2331504995 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0300cb470 con 0x7fd0301607c0 2024-01-15T18:21:53.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.008+0000 7fd034e00640 1 --2- 172.21.15.143:0/2331504995 >> v2:172.21.15.204:3300/0 conn(0x7fd0300cbf50 0x7fd0300cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:53.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd02fe00640 1 --2- 172.21.15.143:0/2331504995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0301607c0 0x7fd03006b7b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd0300cef70 tx=0x7fd02001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231f6b2b3aeeb00 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd02ea00640 1 -- 172.21.15.143:0/2331504995 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200a89f0 con 0x7fd0301607c0 2024-01-15T18:21:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd02ea00640 1 -- 172.21.15.143:0/2331504995 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0200a8b50 con 0x7fd0301607c0 2024-01-15T18:21:53.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd02ea00640 1 -- 172.21.15.143:0/2331504995 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200b0440 con 0x7fd0301607c0 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 -- 172.21.15.143:0/2331504995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0301607c0 msgr2=0x7fd03006b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 --2- 172.21.15.143:0/2331504995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0301607c0 0x7fd03006b7b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd0300cef70 tx=0x7fd02001ecf0 comp rx=0 tx=0).stop 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 -- 172.21.15.143:0/2331504995 shutdown_connections 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 --2- 172.21.15.143:0/2331504995 >> v2:172.21.15.204:3300/0 conn(0x7fd0300cbf50 0x7fd0300cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 --2- 172.21.15.143:0/2331504995 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0301607c0 0x7fd03006b7b0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 -- 172.21.15.143:0/2331504995 >> 172.21.15.143:0/2331504995 conn(0x7fd0300d0e20 msgr2=0x7fd0300d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 -- 172.21.15.143:0/2331504995 shutdown_connections 2024-01-15T18:21:53.008 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.012+0000 7fd036e00640 1 -- 172.21.15.143:0/2331504995 wait complete. 2024-01-15T18:21:53.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 Processor -- start 2024-01-15T18:21:53.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- start start 2024-01-15T18:21:53.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0300cbf50 0x7fd0300af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0300cfb30 0x7fd0301f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0301607c0 0x7fd0300c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd030159d10 con 0x7fd0300cbf50 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300c5a00 con 0x7fd0301607c0 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300c5ce0 con 0x7fd0300cfb30 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0300cbf50 0x7fd0300af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0300cbf50 0x7fd0300af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39092/0 (socket says 172.21.15.143:39092) 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 -- 172.21.15.143:0/4131507384 learned_addr learned my addr 172.21.15.143:0/4131507384 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd034e00640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0301607c0 0x7fd0300c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02f400640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0300cfb30 0x7fd0301f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 -- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0300cfb30 msgr2=0x7fd0301f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0300cfb30 0x7fd0301f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:53.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 -- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0301607c0 msgr2=0x7fd0300c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0301607c0 0x7fd0300c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 -- 172.21.15.143:0/4131507384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200094d0 con 0x7fd0300cbf50 2024-01-15T18:21:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02f400640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd0300cfb30 0x7fd0301f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:21:53.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd02fe00640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd0300cbf50 0x7fd0300af520 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fd02001b900 tx=0x7fd02001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200ba600 con 0x7fd0300cbf50 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0200ba780 con 0x7fd0300cbf50 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0200bab30 con 0x7fd0300cbf50 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- 172.21.15.143:0/4131507384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0300c5fc0 con 0x7fd0300cbf50 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- 172.21.15.143:0/4131507384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0300c6580 con 0x7fd0300cbf50 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.016+0000 7fd036e00640 1 -- 172.21.15.143:0/4131507384 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03006d4b0 con 0x7fd0300cbf50 2024-01-15T18:21:53.014 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.020+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fd0200bac90 con 0x7fd0300cbf50 2024-01-15T18:21:53.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.020+0000 7fd027e00640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd008067ca0 0x7fd00806a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:53.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.020+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd020121ad0 con 0x7fd0300cbf50 2024-01-15T18:21:53.015 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.020+0000 7fd02f400640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd008067ca0 0x7fd00806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:53.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.020+0000 7fd02f400640 1 --2- 172.21.15.143:0/4131507384 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd008067ca0 0x7fd00806a160 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd0301f6040 tx=0x7fd018009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:53.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.024+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd0200f03b0 con 0x7fd0300cbf50 2024-01-15T18:21:53.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.204+0000 7fd036e00640 1 -- 172.21.15.143:0/4131507384 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0300ce6b0 con 0x7fd008067ca0 2024-01-15T18:21:53.204 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:53.208+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:53.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:53.208+0000 7fd027e00640 1 -- 172.21.15.143:0/4131507384 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd018001cd0 con 0x7fd008067ca0 2024-01-15T18:21:53.205 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:53.205 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:53.214 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:55.215 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f0122400640 1 Processor -- start 2024-01-15T18:21:55.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f0122400640 1 -- start start 2024-01-15T18:21:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f0122400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f011c0cc4e0 0x7f011c13ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f0122400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c140500 con 0x7f011c071970 2024-01-15T18:21:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f0122400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c13d290 con 0x7f011c15dae0 2024-01-15T18:21:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f0122400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c13f5d0 con 0x7f011c0cc4e0 2024-01-15T18:21:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f011be00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f011c071970 0x7f011c0b4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43638/0 (socket says 172.21.15.143:43638) 2024-01-15T18:21:55.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f011be00640 1 -- 172.21.15.143:0/3314999511 learned_addr learned my addr 172.21.15.143:0/3314999511 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:55.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365203883 0 0) 0x7f01140014a0 con 0x7f011c071970 2024-01-15T18:21:55.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f011b400640 1 --2- 172.21.15.143:0/3314999511 >> v2:172.21.15.204:3300/0 conn(0x7f011c0cc4e0 0x7f011c13ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.372+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00fc005740 con 0x7f011c071970 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792935805 0 0) 0x7f00fc005740 con 0x7f011c071970 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 >> v2:172.21.15.204:3300/0 conn(0x7f011c0cc4e0 msgr2=0x7f011c13ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 --2- 172.21.15.143:0/3314999511 >> v2:172.21.15.204:3300/0 conn(0x7f011c0cc4e0 0x7f011c13ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 >> v1:172.21.15.204:6789/0 conn(0x7f011c15dae0 legacy=0x7f011c071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c15e530 con 0x7f011c071970 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f0114001b90 con 0x7f011c071970 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 >> v1:172.21.15.143:6789/0 conn(0x7f011c071970 legacy=0x7f011c0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00fc005b90 0x7f00fc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00fc0069e0 0x7f00fc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f00fc00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c13f5d0 con 0x7f00fc0073a0 2024-01-15T18:21:55.372 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc00e240 con 0x7f00fc0069e0 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc00e400 con 0x7f00fc005b90 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f00fc00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011aa00640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00fc0069e0 0x7f00fc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011b400640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00fc005b90 0x7f00fc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 -- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00fc005b90 msgr2=0x7f00fc005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00fc005b90 0x7f00fc005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 -- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00fc0069e0 msgr2=0x7f00fc006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00fc0069e0 0x7f00fc006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:55.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 -- 172.21.15.143:0/3314999511 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc005740 con 0x7f00fc0073a0 2024-01-15T18:21:55.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 -- 172.21.15.143:0/3314999511 reap_dead start 2024-01-15T18:21:55.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.376+0000 7f011be00640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f00fc00dba0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f011400a800 tx=0x7f011401d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6dd858e179fc5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:55.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01140ad630 con 0x7f00fc0073a0 2024-01-15T18:21:55.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01140a8070 con 0x7f00fc0073a0 2024-01-15T18:21:55.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f011a000640 1 -- 172.21.15.143:0/3314999511 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01140ad9f0 con 0x7f00fc0073a0 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 -- 172.21.15.143:0/3314999511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 msgr2=0x7f00fc00dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f00fc00dba0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f011400a800 tx=0x7f011401d300 comp rx=0 tx=0).stop 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 -- 172.21.15.143:0/3314999511 shutdown_connections 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 --2- 172.21.15.143:0/3314999511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f00fc00dba0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 -- 172.21.15.143:0/3314999511 >> 172.21.15.143:0/3314999511 conn(0x7f011c0b1b90 msgr2=0x7f011c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 -- 172.21.15.143:0/3314999511 shutdown_connections 2024-01-15T18:21:55.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 -- 172.21.15.143:0/3314999511 wait complete. 2024-01-15T18:21:55.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 Processor -- start 2024-01-15T18:21:55.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.380+0000 7f0122400640 1 -- start start 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f011c0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f011c130f30 0x7f011c150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f011c0d05e0 0x7f011c158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1506b0 con 0x7f00fc0073a0 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0b5170 con 0x7f011c0d05e0 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c0cd4d0 con 0x7f011c130f30 2024-01-15T18:21:55.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f011c0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f011c0b4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:39106/0 (socket says 172.21.15.143:39106) 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 -- 172.21.15.143:0/63306472 learned_addr learned my addr 172.21.15.143:0/63306472 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011aa00640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f011c130f30 0x7f011c150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011be00640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f011c0d05e0 0x7f011c158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 -- 172.21.15.143:0/63306472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f011c130f30 msgr2=0x7f011c150170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f011c130f30 0x7f011c150170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 -- 172.21.15.143:0/63306472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f011c0d05e0 msgr2=0x7f011c158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f011c0d05e0 0x7f011c158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 -- 172.21.15.143:0/63306472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c0cd7b0 con 0x7f00fc0073a0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f011b400640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00fc0073a0 0x7f011c0b4b90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f010c011960 tx=0x7f010c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c00faf0 con 0x7f00fc0073a0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c00fc50 con 0x7f00fc0073a0 2024-01-15T18:21:55.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 -- 172.21.15.143:0/63306472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c0cda90 con 0x7f00fc0073a0 2024-01-15T18:21:55.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c014440 con 0x7f00fc0073a0 2024-01-15T18:21:55.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 -- 172.21.15.143:0/63306472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c0d0a60 con 0x7f00fc0073a0 2024-01-15T18:21:55.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0122400640 1 -- 172.21.15.143:0/63306472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c067500 con 0x7f00fc0073a0 2024-01-15T18:21:55.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.384+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f010c0145a0 con 0x7f00fc0073a0 2024-01-15T18:21:55.382 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.388+0000 7f0113400640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00f4067ca0 0x7f00f406a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:55.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.388+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f010c083b80 con 0x7f00fc0073a0 2024-01-15T18:21:55.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.388+0000 7f011aa00640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00f4067ca0 0x7f00f406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:55.383 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.388+0000 7f011aa00640 1 --2- 172.21.15.143:0/63306472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00f4067ca0 0x7f00f406a160 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f011c13af10 tx=0x7f0104009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:55.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.392+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f010c052570 con 0x7f00fc0073a0 2024-01-15T18:21:55.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.568+0000 7f0122400640 1 -- 172.21.15.143:0/63306472 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f011c13fac0 con 0x7f00f4067ca0 2024-01-15T18:21:55.570 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:55.576+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:55.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:55.576+0000 7f0113400640 1 -- 172.21.15.143:0/63306472 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0104001530 con 0x7f00f4067ca0 2024-01-15T18:21:55.571 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:55.571 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:55.579 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:57.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:21:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.736+0000 7f251fc00640 1 Processor -- start 2024-01-15T18:21:57.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.736+0000 7f251fc00640 1 -- start start 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251fc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2518072410 0x7f251815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251fc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251815f310 con 0x7f2518141660 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251fc00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251815f5f0 con 0x7f25180ce560 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251fc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f251815fbb0 con 0x7f2518072410 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251dc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2518072410 0x7f251815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251d200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2518141660 0x7f2518141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41058/0 (socket says 172.21.15.143:41058) 2024-01-15T18:21:57.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251d200640 1 -- 172.21.15.143:0/3182261243 learned_addr learned my addr 172.21.15.143:0/3182261243 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:21:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251dc00640 1 -- 172.21.15.143:0/3182261243 >> v1:172.21.15.204:6789/0 conn(0x7f25180ce560 legacy=0x7f2518071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:21:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251dc00640 1 -- 172.21.15.143:0/3182261243 >> v1:172.21.15.143:6789/0 conn(0x7f2518141660 legacy=0x7f2518141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f251dc00640 1 -- 172.21.15.143:0/3182261243 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251815f8d0 con 0x7f2518072410 2024-01-15T18:21:57.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.740+0000 7f2517400640 1 -- 172.21.15.143:0/3182261243 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248148950 0 0) 0x7f251815f310 con 0x7f2518141660 2024-01-15T18:21:57.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251dc00640 1 --2- 172.21.15.143:0/3182261243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2518072410 0x7f251815ec80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f2510009d90 tx=0x7f251001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7dfa9fdfad1ebc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f2517400640 1 -- 172.21.15.143:0/3182261243 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100a58e0 con 0x7f2518072410 2024-01-15T18:21:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f2517400640 1 -- 172.21.15.143:0/3182261243 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25100a5a40 con 0x7f2518072410 2024-01-15T18:21:57.739 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f2517400640 1 -- 172.21.15.143:0/3182261243 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100a5dd0 con 0x7f2518072410 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 -- 172.21.15.143:0/3182261243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2518072410 msgr2=0x7f251815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 --2- 172.21.15.143:0/3182261243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2518072410 0x7f251815ec80 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f2510009d90 tx=0x7f251001c910 comp rx=0 tx=0).stop 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 -- 172.21.15.143:0/3182261243 shutdown_connections 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 --2- 172.21.15.143:0/3182261243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2518072410 0x7f251815ec80 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 -- 172.21.15.143:0/3182261243 >> 172.21.15.143:0/3182261243 conn(0x7f251806c3f0 msgr2=0x7f251806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 -- 172.21.15.143:0/3182261243 shutdown_connections 2024-01-15T18:21:57.740 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 -- 172.21.15.143:0/3182261243 wait complete. 2024-01-15T18:21:57.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 Processor -- start 2024-01-15T18:21:57.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.744+0000 7f251fc00640 1 -- start start 2024-01-15T18:21:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2518072410 0x7f2518166660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:57.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25180ce560 0x7f25181f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2518141660 0x7f25181fa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251813bd40 con 0x7f25180ce560 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251813bfb0 con 0x7f2518141660 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251813c290 con 0x7f2518072410 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25180ce560 0x7f25181f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25180ce560 0x7f25181f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56138/0 (socket says 172.21.15.143:56138) 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 -- 172.21.15.143:0/1055922211 learned_addr learned my addr 172.21.15.143:0/1055922211 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251d200640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2518072410 0x7f2518166660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251dc00640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2518141660 0x7f25181fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 -- 172.21.15.143:0/1055922211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2518072410 msgr2=0x7f2518166660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2518072410 0x7f2518166660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 -- 172.21.15.143:0/1055922211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2518141660 msgr2=0x7f25181fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2518141660 0x7f25181fa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 -- 172.21.15.143:0/1055922211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510009a70 con 0x7f25180ce560 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2517e00640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25180ce560 0x7f25181f9c30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f250000c450 tx=0x7f250000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500012830 con 0x7f25180ce560 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000129b0 con 0x7f25180ce560 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 -- 172.21.15.143:0/1055922211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25180cd150 con 0x7f25180ce560 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500012d60 con 0x7f25180ce560 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 -- 172.21.15.143:0/1055922211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251813c850 con 0x7f25180ce560 2024-01-15T18:21:57.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.748+0000 7f251fc00640 1 -- 172.21.15.143:0/1055922211 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25180b3ee0 con 0x7f25180ce560 2024-01-15T18:21:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.752+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f250001c460 con 0x7f25180ce560 2024-01-15T18:21:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.752+0000 7f2514c00640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24f0067ca0 0x7f24f006a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:21:57.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.752+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2500082e40 con 0x7f25180ce560 2024-01-15T18:21:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.752+0000 7f251d200640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24f0067ca0 0x7f24f006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:21:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.752+0000 7f251d200640 1 --2- 172.21.15.143:0/1055922211 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f24f0067ca0 0x7f24f006a160 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2508000f70 tx=0x7f2508009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:21:57.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.756+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2500051b80 con 0x7f25180ce560 2024-01-15T18:21:57.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.936+0000 7f251fc00640 1 -- 172.21.15.143:0/1055922211 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f251806fa80 con 0x7f24f0067ca0 2024-01-15T18:21:57.939 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:21:57.944+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:21:57.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:21:57.944+0000 7f2514c00640 1 -- 172.21.15.143:0/1055922211 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f251806fa80 con 0x7f24f0067ca0 2024-01-15T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:21:57.940 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:21:57.948 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:21:59.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:00.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 Processor -- start 2024-01-15T18:22:00.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 -- start start 2024-01-15T18:22:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c15b180 0x7f151c0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f151c0ce4a0 0x7f151c0cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f151c0cddb0 0x7f151c0cb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0ce200 con 0x7f151c0ce4a0 2024-01-15T18:22:00.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0c4ea0 con 0x7f151c0cddb0 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1523a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c4b70 con 0x7f151c15b180 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f151c0ce4a0 0x7f151c0cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f151c0ce4a0 0x7f151c0cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56146/0 (socket says 172.21.15.143:56146) 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 -- 172.21.15.143:0/276240762 learned_addr learned my addr 172.21.15.143:0/276240762 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 -- 172.21.15.143:0/276240762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c15b180 msgr2=0x7f151c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1521000640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c15b180 0x7f151c0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1521a00640 1 --2- 172.21.15.143:0/276240762 >> v2:172.21.15.204:3300/0 conn(0x7f151c0cddb0 0x7f151c0cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c15b180 0x7f151c0d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 -- 172.21.15.143:0/276240762 >> v2:172.21.15.204:3300/0 conn(0x7f151c0cddb0 msgr2=0x7f151c0cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 --2- 172.21.15.143:0/276240762 >> v2:172.21.15.204:3300/0 conn(0x7f151c0cddb0 0x7f151c0cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f151be00640 1 -- 172.21.15.143:0/276240762 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c0cfdc0 con 0x7f151c0ce4a0 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1521000640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c15b180 0x7f151c0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:22:00.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.108+0000 7f1521a00640 1 --2- 172.21.15.143:0/276240762 >> v2:172.21.15.204:3300/0 conn(0x7f151c0cddb0 0x7f151c0cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:22:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f151be00640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0ce4a0 0x7f151c0cd870 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f151c072310 tx=0x7f150401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8342fddf3948d1a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f151b400640 1 -- 172.21.15.143:0/276240762 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15040aa5f0 con 0x7f151c0ce4a0 2024-01-15T18:22:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f151b400640 1 -- 172.21.15.143:0/276240762 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15040aa750 con 0x7f151c0ce4a0 2024-01-15T18:22:00.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f151b400640 1 -- 172.21.15.143:0/276240762 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15040aaae0 con 0x7f151c0ce4a0 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 -- 172.21.15.143:0/276240762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0ce4a0 msgr2=0x7f151c0cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0ce4a0 0x7f151c0cd870 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f151c072310 tx=0x7f150401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 -- 172.21.15.143:0/276240762 shutdown_connections 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 --2- 172.21.15.143:0/276240762 >> v2:172.21.15.204:3300/0 conn(0x7f151c0cddb0 0x7f151c0cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0ce4a0 0x7f151c0cd870 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 --2- 172.21.15.143:0/276240762 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c15b180 0x7f151c0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 -- 172.21.15.143:0/276240762 >> 172.21.15.143:0/276240762 conn(0x7f151c0d0e80 msgr2=0x7f151c0d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:00.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 -- 172.21.15.143:0/276240762 shutdown_connections 2024-01-15T18:22:00.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.112+0000 7f1523a00640 1 -- 172.21.15.143:0/276240762 wait complete. 2024-01-15T18:22:00.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 Processor -- start 2024-01-15T18:22:00.111 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 -- start start 2024-01-15T18:22:00.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0cddb0 0x7f151c0ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0ce4a0 0x7f151c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c15b180 0x7f151c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c156800 con 0x7f151c0cddb0 2024-01-15T18:22:00.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c09bc60 con 0x7f151c15b180 2024-01-15T18:22:00.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c156ed0 con 0x7f151c0ce4a0 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0cddb0 0x7f151c0ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0cddb0 0x7f151c0ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56148/0 (socket says 172.21.15.143:56148) 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 -- 172.21.15.143:0/850268205 learned_addr learned my addr 172.21.15.143:0/850268205 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f151be00640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0ce4a0 0x7f151c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521a00640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c15b180 0x7f151c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 -- 172.21.15.143:0/850268205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0ce4a0 msgr2=0x7f151c156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f151c0ce4a0 0x7f151c156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 -- 172.21.15.143:0/850268205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c15b180 msgr2=0x7f151c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f151c15b180 0x7f151c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 -- 172.21.15.143:0/850268205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040094d0 con 0x7f151c0cddb0 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1521000640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f151c0cddb0 0x7f151c0ca600 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c7f0 tx=0x7f150c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:00.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c007820 con 0x7f151c0cddb0 2024-01-15T18:22:00.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c007980 con 0x7f151c0cddb0 2024-01-15T18:22:00.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 -- 172.21.15.143:0/850268205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c156240 con 0x7f151c0cddb0 2024-01-15T18:22:00.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c007d30 con 0x7f151c0cddb0 2024-01-15T18:22:00.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.116+0000 7f1523a00640 1 -- 172.21.15.143:0/850268205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c09bee0 con 0x7f151c0cddb0 2024-01-15T18:22:00.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.120+0000 7f1523a00640 1 -- 172.21.15.143:0/850268205 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c067500 con 0x7f151c0cddb0 2024-01-15T18:22:00.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.120+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f150c0113f0 con 0x7f151c0cddb0 2024-01-15T18:22:00.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.120+0000 7f1518c00640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14f4067ca0 0x7f14f406a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:00.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.120+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f150c0836c0 con 0x7f151c0cddb0 2024-01-15T18:22:00.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.120+0000 7f151be00640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14f4067ca0 0x7f14f406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:00.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.120+0000 7f151be00640 1 --2- 172.21.15.143:0/850268205 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f14f4067ca0 0x7f14f406a160 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f15040ae000 tx=0x7f15040b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:00.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.124+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f150c052050 con 0x7f151c0cddb0 2024-01-15T18:22:00.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.300+0000 7f1523a00640 1 -- 172.21.15.143:0/850268205 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f151c0c5310 con 0x7f14f4067ca0 2024-01-15T18:22:00.303 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:00.308+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:00.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:00.308+0000 7f1518c00640 1 -- 172.21.15.143:0/850268205 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f15040aa940 con 0x7f14f4067ca0 2024-01-15T18:22:00.303 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:00.304 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:00.312 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:00.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:22:00.401 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:22:02.314 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:02.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 Processor -- start 2024-01-15T18:22:02.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 -- start start 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 0x7fe4940cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe4940ce520 0x7fe494071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4940ceb50 con 0x7fe4940720f0 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe494072580 con 0x7fe4940ce520 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe499000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49415c110 con 0x7fe494143780 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe492800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe4940720f0 0x7fe49415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:41068/0 (socket says 172.21.15.143:41068) 2024-01-15T18:22:02.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe492800640 1 -- 172.21.15.143:0/3184417041 learned_addr learned my addr 172.21.15.143:0/3184417041 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:02.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe491400640 1 --2- 172.21.15.143:0/3184417041 >> v2:172.21.15.204:3300/0 conn(0x7fe4940ce520 0x7fe494071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe491e00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 0x7fe4940cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764762787 0 0) 0x7fe48c0014a0 con 0x7fe4940720f0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe474003530 con 0x7fe4940720f0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413331718 0 0) 0x7fe474003530 con 0x7fe4940720f0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 msgr2=0x7fe4940cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 0x7fe4940cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 >> v2:172.21.15.204:3300/0 conn(0x7fe4940ce520 msgr2=0x7fe494071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 --2- 172.21.15.143:0/3184417041 >> v2:172.21.15.204:3300/0 conn(0x7fe4940ce520 0x7fe494071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49415b7e0 con 0x7fe4940720f0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fe48c001b90 con 0x7fe4940720f0 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 >> v1:172.21.15.143:6789/0 conn(0x7fe4940720f0 legacy=0x7fe49415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:02.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe474003980 0x7fe474003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 0x7fe4740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474004d10 0x7fe474005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494072580 con 0x7fe474003980 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49415c110 con 0x7fe474004d10 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe474004630 con 0x7fe494143780 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.472+0000 7fe491e00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe474003980 0x7fe474003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491400640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 0x7fe4740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe492800640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474004d10 0x7fe474005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491e00640 1 -- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 msgr2=0x7fe4740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491e00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494143780 0x7fe4740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491e00640 1 -- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474004d10 msgr2=0x7fe474005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491e00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474004d10 0x7fe474005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:02.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491e00640 1 -- 172.21.15.143:0/3184417041 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474003530 con 0x7fe474003980 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe491e00640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe474003980 0x7fe474003d60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe484009ae0 tx=0x7fe48401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6828d411008e6c8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4840a5530 con 0x7fe474003980 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4840a5690 con 0x7fe474003980 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.476+0000 7fe490a00640 1 -- 172.21.15.143:0/3184417041 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4840a5a40 con 0x7fe474003980 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- 172.21.15.143:0/3184417041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe474003980 msgr2=0x7fe474003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 --2- 172.21.15.143:0/3184417041 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe474003980 0x7fe474003d60 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fe484009ae0 tx=0x7fe48401ca50 comp rx=0 tx=0).stop 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe492800640 1 -- 172.21.15.143:0/3184417041 reap_dead start 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- 172.21.15.143:0/3184417041 shutdown_connections 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- 172.21.15.143:0/3184417041 >> 172.21.15.143:0/3184417041 conn(0x7fe49406c3f0 msgr2=0x7fe49406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- 172.21.15.143:0/3184417041 shutdown_connections 2024-01-15T18:22:02.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- 172.21.15.143:0/3184417041 wait complete. 2024-01-15T18:22:02.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 Processor -- start 2024-01-15T18:22:02.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- start start 2024-01-15T18:22:02.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474003980 0x7fe49413e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494159330 0x7fe494159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4941411e0 0x7fe494141660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494142fd0 con 0x7fe4941411e0 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe494159cd0 con 0x7fe474003980 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe499000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe494141f30 con 0x7fe494159330 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe492800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4941411e0 0x7fe494141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe492800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4941411e0 0x7fe494141660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56162/0 (socket says 172.21.15.143:56162) 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe492800640 1 -- 172.21.15.143:0/2675202089 learned_addr learned my addr 172.21.15.143:0/2675202089 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.480+0000 7fe491e00640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474003980 0x7fe49413e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe491400640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494159330 0x7fe494159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe492800640 1 -- 172.21.15.143:0/2675202089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494159330 msgr2=0x7fe494159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe492800640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe494159330 0x7fe494159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe492800640 1 -- 172.21.15.143:0/2675202089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474003980 msgr2=0x7fe49413e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe492800640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe474003980 0x7fe49413e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:02.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe492800640 1 -- 172.21.15.143:0/2675202089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe494142210 con 0x7fe4941411e0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe492800640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe4941411e0 0x7fe494141660 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0051e0 tx=0x7fe48c006670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c015480 con 0x7fe4941411e0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c015600 con 0x7fe4941411e0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0159b0 con 0x7fe4941411e0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe499000640 1 -- 172.21.15.143:0/2675202089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4941424f0 con 0x7fe4941411e0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe499000640 1 -- 172.21.15.143:0/2675202089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe494142ab0 con 0x7fe4941411e0 2024-01-15T18:22:02.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe499000640 1 -- 172.21.15.143:0/2675202089 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe494142ca0 con 0x7fe4941411e0 2024-01-15T18:22:02.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.484+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fe48c013380 con 0x7fe4941411e0 2024-01-15T18:22:02.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.488+0000 7fe48a000640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe46c067c90 0x7fe46c06a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:02.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.488+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fe48c082ec0 con 0x7fe4941411e0 2024-01-15T18:22:02.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.488+0000 7fe491e00640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe46c067c90 0x7fe46c06a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:02.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.488+0000 7fe491e00640 1 --2- 172.21.15.143:0/2675202089 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe46c067c90 0x7fe46c06a150 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe4840af000 tx=0x7fe4840046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:02.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.492+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe48c051c60 con 0x7fe4941411e0 2024-01-15T18:22:02.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.668+0000 7fe499000640 1 -- 172.21.15.143:0/2675202089 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4940b4fc0 con 0x7fe46c067c90 2024-01-15T18:22:02.671 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:02.676+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:02.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:02.676+0000 7fe48a000640 1 -- 172.21.15.143:0/2675202089 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe4940b4fc0 con 0x7fe46c067c90 2024-01-15T18:22:02.671 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:02.672 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:02.680 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:04.682 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 Processor -- start 2024-01-15T18:22:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 -- start start 2024-01-15T18:22:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a90157300 0x7f8a90159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:04.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a900b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:04.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a900b2a20 con 0x7f8a900b48f0 2024-01-15T18:22:04.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a900b35a0 con 0x7f8a90157300 2024-01-15T18:22:04.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a97600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900b4d80 con 0x7f8a900b21d0 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a94c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8a90157300 0x7f8a90159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a8fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a900b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a95600640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8a900b48f0 0x7f8a9015d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:49854/0 (socket says 172.21.15.143:49854) 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a95600640 1 -- 172.21.15.143:0/3182105514 learned_addr learned my addr 172.21.15.143:0/3182105514 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a94c00640 1 -- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 msgr2=0x7f8a900b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a94c00640 1 --2- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a900b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a94c00640 1 -- 172.21.15.143:0/3182105514 >> v1:172.21.15.204:6789/0 conn(0x7f8a900b48f0 legacy=0x7f8a9015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a8f400640 1 -- 172.21.15.143:0/3182105514 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502524638 0 0) 0x7f8a900b2a20 con 0x7f8a900b48f0 2024-01-15T18:22:04.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.840+0000 7f8a94c00640 1 -- 172.21.15.143:0/3182105514 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900b2d00 con 0x7f8a90157300 2024-01-15T18:22:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a94c00640 1 --2- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a90157300 0x7f8a90159b70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8a800097c0 tx=0x7f8a8001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2139a13e1853793 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a8f400640 1 -- 172.21.15.143:0/3182105514 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800ab650 con 0x7f8a90157300 2024-01-15T18:22:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a8f400640 1 -- 172.21.15.143:0/3182105514 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a800ab7b0 con 0x7f8a90157300 2024-01-15T18:22:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a8f400640 1 -- 172.21.15.143:0/3182105514 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a800abb40 con 0x7f8a90157300 2024-01-15T18:22:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 -- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a90157300 msgr2=0x7f8a90159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 --2- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a90157300 0x7f8a90159b70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8a800097c0 tx=0x7f8a8001c450 comp rx=0 tx=0).stop 2024-01-15T18:22:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 -- 172.21.15.143:0/3182105514 shutdown_connections 2024-01-15T18:22:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 --2- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a900b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 --2- 172.21.15.143:0/3182105514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a90157300 0x7f8a90159b70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 -- 172.21.15.143:0/3182105514 >> 172.21.15.143:0/3182105514 conn(0x7f8a9006c3f0 msgr2=0x7f8a9006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 -- 172.21.15.143:0/3182105514 shutdown_connections 2024-01-15T18:22:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.844+0000 7f8a97600640 1 -- 172.21.15.143:0/3182105514 wait complete. 2024-01-15T18:22:04.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 Processor -- start 2024-01-15T18:22:04.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 -- start start 2024-01-15T18:22:04.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a9006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:04.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a900b48f0 0x7f8a90151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:04.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a90157300 0x7f8a9015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a90151fc0 con 0x7f8a90157300 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9015ec10 con 0x7f8a900b48f0 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9006c8e0 con 0x7f8a900b21d0 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a90157300 0x7f8a9015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a94c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a9006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a90157300 0x7f8a9015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56166/0 (socket says 172.21.15.143:56166) 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 -- 172.21.15.143:0/1116988708 learned_addr learned my addr 172.21.15.143:0/1116988708 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a8fe00640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a900b48f0 0x7f8a90151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 -- 172.21.15.143:0/1116988708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 msgr2=0x7f8a9006d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8a900b21d0 0x7f8a9006d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 -- 172.21.15.143:0/1116988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a900b48f0 msgr2=0x7f8a90151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a900b48f0 0x7f8a90151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 -- 172.21.15.143:0/1116988708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80009420 con 0x7f8a90157300 2024-01-15T18:22:04.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a95600640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a90157300 0x7f8a9015e1c0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8a88001570 tx=0x7f8a8800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800cea0 con 0x7f8a90157300 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8800a5e0 con 0x7f8a90157300 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800a990 con 0x7f8a90157300 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 -- 172.21.15.143:0/1116988708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9006cbc0 con 0x7f8a90157300 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.848+0000 7f8a97600640 1 -- 172.21.15.143:0/1116988708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9015eef0 con 0x7f8a90157300 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.852+0000 7f8a97600640 1 -- 172.21.15.143:0/1116988708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9006d0e0 con 0x7f8a90157300 2024-01-15T18:22:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.852+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f8a8800aaf0 con 0x7f8a90157300 2024-01-15T18:22:04.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.852+0000 7f8a8cc00640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a68067ca0 0x7f8a6806a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:04.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.852+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8a880828a0 con 0x7f8a90157300 2024-01-15T18:22:04.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.852+0000 7f8a94c00640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a68067ca0 0x7f8a6806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.852+0000 7f8a94c00640 1 --2- 172.21.15.143:0/1116988708 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a68067ca0 0x7f8a6806a160 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8a800af000 tx=0x7f8a800a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:04.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:04.856+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a88051290 con 0x7f8a90157300 2024-01-15T18:22:05.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:05.040+0000 7f8a97600640 1 -- 172.21.15.143:0/1116988708 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a9006ca50 con 0x7f8a68067ca0 2024-01-15T18:22:05.040 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:05.044+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:05.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:05.044+0000 7f8a8cc00640 1 -- 172.21.15.143:0/1116988708 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8a9006ca50 con 0x7f8a68067ca0 2024-01-15T18:22:05.040 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:05.041 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:05.049 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:07.051 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:07.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 Processor -- start 2024-01-15T18:22:07.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 -- start start 2024-01-15T18:22:07.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7cc06bfb0 0x7fd7cc06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:07.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0b0310 0x7fd7cc0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:07.204 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0cbff0 con 0x7fd7cc0cc9c0 2024-01-15T18:22:07.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0d0c50 con 0x7fd7cc06bfb0 2024-01-15T18:22:07.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc06c8d0 con 0x7fd7cc0b0310 2024-01-15T18:22:07.205 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7cc06bfb0 0x7fd7cc06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0b0310 0x7fd7cc0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d1400640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7fd7cc0cc9c0 0x7fd7cc0cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:53716/0 (socket says 172.21.15.143:53716) 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d0a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7cc06bfb0 0x7fd7cc06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:49054/0 (socket says 172.21.15.143:49054) 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.208+0000 7fd7d1400640 1 -- 172.21.15.143:0/1130923150 learned_addr learned my addr 172.21.15.143:0/1130923150 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7d0a00640 1 -- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0b0310 msgr2=0x7fd7cc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7d0a00640 1 --2- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0b0310 0x7fd7cc0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7d0a00640 1 -- 172.21.15.143:0/1130923150 >> v1:172.21.15.204:6789/0 conn(0x7fd7cc0cc9c0 legacy=0x7fd7cc0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7d0a00640 1 -- 172.21.15.143:0/1130923150 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0cb6c0 con 0x7fd7cc06bfb0 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7cb400640 1 -- 172.21.15.143:0/1130923150 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741735612 0 0) 0x7fd7cc0cbff0 con 0x7fd7cc0cc9c0 2024-01-15T18:22:07.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7cbe00640 1 --2- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0b0310 0x7fd7cc0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:22:07.208 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7d0a00640 1 --2- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc06c390 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0097c0 tx=0x7fd7bc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d355c365f9cb16c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7cb400640 1 -- 172.21.15.143:0/1130923150 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0ac620 con 0x7fd7cc06bfb0 2024-01-15T18:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7cb400640 1 -- 172.21.15.143:0/1130923150 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc0ac780 con 0x7fd7cc06bfb0 2024-01-15T18:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.212+0000 7fd7cb400640 1 -- 172.21.15.143:0/1130923150 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc0acb10 con 0x7fd7cc06bfb0 2024-01-15T18:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 msgr2=0x7fd7cc06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:07.209 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 --2- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc06c390 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0097c0 tx=0x7fd7bc01c3c0 comp rx=0 tx=0).stop 2024-01-15T18:22:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- 172.21.15.143:0/1130923150 shutdown_connections 2024-01-15T18:22:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 --2- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0b0310 0x7fd7cc0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 --2- 172.21.15.143:0/1130923150 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc06c390 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- 172.21.15.143:0/1130923150 >> 172.21.15.143:0/1130923150 conn(0x7fd7cc0d0ed0 msgr2=0x7fd7cc0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- 172.21.15.143:0/1130923150 shutdown_connections 2024-01-15T18:22:07.210 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- 172.21.15.143:0/1130923150 wait complete. 2024-01-15T18:22:07.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 Processor -- start 2024-01-15T18:22:07.212 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- start start 2024-01-15T18:22:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc0ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7cc0b0310 0x7fd7cc0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0cc9c0 0x7fd7cc141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc13ffc0 con 0x7fd7cc0b0310 2024-01-15T18:22:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc1402a0 con 0x7fd7cc06bfb0 2024-01-15T18:22:07.214 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d3400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc140580 con 0x7fd7cc0cc9c0 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7cc0b0310 0x7fd7cc0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7cbe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7cc0b0310 0x7fd7cc0c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48792/0 (socket says 172.21.15.143:48792) 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7cbe00640 1 -- 172.21.15.143:0/1916042765 learned_addr learned my addr 172.21.15.143:0/1916042765 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d1400640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0cc9c0 0x7fd7cc141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.216+0000 7fd7d0a00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc0ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7cbe00640 1 -- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0cc9c0 msgr2=0x7fd7cc141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7cbe00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0cc9c0 0x7fd7cc141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7cbe00640 1 -- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 msgr2=0x7fd7cc0ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7cbe00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc0ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7cbe00640 1 -- 172.21.15.143:0/1916042765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4009420 con 0x7fd7cc0b0310 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7d1400640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7cc0cc9c0 0x7fd7cc141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7d0a00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7cc06bfb0 0x7fd7cc0ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7cbe00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7cc0b0310 0x7fd7cc0c71a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e9b0 tx=0x7fd7b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:07.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400c9a0 con 0x7fd7cc0b0310 2024-01-15T18:22:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b400cb00 con 0x7fd7cc0b0310 2024-01-15T18:22:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4013470 con 0x7fd7cc0b0310 2024-01-15T18:22:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7d3400640 1 -- 172.21.15.143:0/1916042765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc009420 con 0x7fd7cc0b0310 2024-01-15T18:22:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7d3400640 1 -- 172.21.15.143:0/1916042765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc140b40 con 0x7fd7cc0b0310 2024-01-15T18:22:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7d3400640 1 -- 172.21.15.143:0/1916042765 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc067500 con 0x7fd7cc0b0310 2024-01-15T18:22:07.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fd7b40135d0 con 0x7fd7cc0b0310 2024-01-15T18:22:07.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7c8c00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd7a4067ca0 0x7fd7a406a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:07.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.220+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd7b4083310 con 0x7fd7cc0b0310 2024-01-15T18:22:07.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.224+0000 7fd7d0a00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd7a4067ca0 0x7fd7a406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:07.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.224+0000 7fd7d0a00640 1 --2- 172.21.15.143:0/1916042765 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd7a4067ca0 0x7fd7a406a160 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0ae000 tx=0x7fd7bc0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:07.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.228+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd7b4051bf0 con 0x7fd7cc0b0310 2024-01-15T18:22:07.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.404+0000 7fd7d3400640 1 -- 172.21.15.143:0/1916042765 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7cc06b610 con 0x7fd7a4067ca0 2024-01-15T18:22:07.404 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:07.408+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:07.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:07.408+0000 7fd7c8c00640 1 -- 172.21.15.143:0/1916042765 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd7cc06b610 con 0x7fd7a4067ca0 2024-01-15T18:22:07.404 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:07.405 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:07.413 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:09.414 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 Processor -- start 2024-01-15T18:22:09.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 -- start start 2024-01-15T18:22:09.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f215c157300 0x7f215c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:09.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f215c0b4ac0 0x7f215c070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215c0b2a20 con 0x7f215c0b21d0 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c0b35a0 con 0x7f215c0b4ac0 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2163200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c0b2d00 con 0x7f215c157300 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2161200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f215c0b4ac0 0x7f215c070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2161200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f215c0b4ac0 0x7f215c070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48806/0 (socket says 172.21.15.143:48806) 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2161200640 1 -- 172.21.15.143:0/738955119 learned_addr learned my addr 172.21.15.143:0/738955119 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f215be00640 1 --2- 172.21.15.143:0/738955119 >> v2:172.21.15.204:3300/0 conn(0x7f215c157300 0x7f215c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2161200640 1 -- 172.21.15.143:0/738955119 >> v2:172.21.15.204:3300/0 conn(0x7f215c157300 msgr2=0x7f215c159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2161200640 1 --2- 172.21.15.143:0/738955119 >> v2:172.21.15.204:3300/0 conn(0x7f215c157300 0x7f215c159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.572+0000 7f2161200640 1 -- 172.21.15.143:0/738955119 >> v1:172.21.15.204:6789/0 conn(0x7f215c0b21d0 legacy=0x7f215c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f2161200640 1 -- 172.21.15.143:0/738955119 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0b2fe0 con 0x7f215c0b4ac0 2024-01-15T18:22:09.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f215be00640 1 --2- 172.21.15.143:0/738955119 >> v2:172.21.15.204:3300/0 conn(0x7f215c157300 0x7f215c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:09.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f2161200640 1 --2- 172.21.15.143:0/738955119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b4ac0 0x7f215c070e80 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f215c0b1aa0 tx=0x7f215401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4263313372da9c6f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:09.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f215aa00640 1 -- 172.21.15.143:0/738955119 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21540a5880 con 0x7f215c0b4ac0 2024-01-15T18:22:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f215aa00640 1 -- 172.21.15.143:0/738955119 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21540a59e0 con 0x7f215c0b4ac0 2024-01-15T18:22:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f215aa00640 1 -- 172.21.15.143:0/738955119 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21540a5d70 con 0x7f215c0b4ac0 2024-01-15T18:22:09.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f2163200640 1 -- 172.21.15.143:0/738955119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b4ac0 msgr2=0x7f215c070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.576+0000 7f2163200640 1 --2- 172.21.15.143:0/738955119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b4ac0 0x7f215c070e80 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f215c0b1aa0 tx=0x7f215401c9a0 comp rx=0 tx=0).stop 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- 172.21.15.143:0/738955119 shutdown_connections 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 --2- 172.21.15.143:0/738955119 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b4ac0 0x7f215c070e80 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 --2- 172.21.15.143:0/738955119 >> v2:172.21.15.204:3300/0 conn(0x7f215c157300 0x7f215c159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- 172.21.15.143:0/738955119 >> 172.21.15.143:0/738955119 conn(0x7f215c06c3f0 msgr2=0x7f215c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- 172.21.15.143:0/738955119 shutdown_connections 2024-01-15T18:22:09.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- 172.21.15.143:0/738955119 wait complete. 2024-01-15T18:22:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 Processor -- start 2024-01-15T18:22:09.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- start start 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b21d0 0x7f215c14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f215c0b4ac0 0x7f215c14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f215c157300 0x7f215c14ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c14ba10 con 0x7f215c0b21d0 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c14f450 con 0x7f215c0b4ac0 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2163200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215c1f9750 con 0x7f215c157300 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b21d0 0x7f215c14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b21d0 0x7f215c14f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48814/0 (socket says 172.21.15.143:48814) 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 -- 172.21.15.143:0/2996247951 learned_addr learned my addr 172.21.15.143:0/2996247951 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f2161200640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f215c157300 0x7f215c14ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:09.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215b400640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f215c0b4ac0 0x7f215c14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 -- 172.21.15.143:0/2996247951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f215c157300 msgr2=0x7f215c14ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f215c157300 0x7f215c14ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 -- 172.21.15.143:0/2996247951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f215c0b4ac0 msgr2=0x7f215c14b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f215c0b4ac0 0x7f215c14b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.580+0000 7f215be00640 1 -- 172.21.15.143:0/2996247951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154009b30 con 0x7f215c0b21d0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f215be00640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f215c0b21d0 0x7f215c14f660 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f214c00a780 tx=0x7f214c00ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c004220 con 0x7f215c0b21d0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c004380 con 0x7f215c0b21d0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c00d970 con 0x7f215c0b21d0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2163200640 1 -- 172.21.15.143:0/2996247951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c1f9a30 con 0x7f215c0b21d0 2024-01-15T18:22:09.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2163200640 1 -- 172.21.15.143:0/2996247951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c1f9ff0 con 0x7f215c0b21d0 2024-01-15T18:22:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2163200640 1 -- 172.21.15.143:0/2996247951 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c067500 con 0x7f215c0b21d0 2024-01-15T18:22:09.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f214c00dad0 con 0x7f215c0b21d0 2024-01-15T18:22:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2153e00640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2134067ca0 0x7f213406a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f214c083160 con 0x7f215c0b21d0 2024-01-15T18:22:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.584+0000 7f215b400640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2134067ca0 0x7f213406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:09.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.588+0000 7f215b400640 1 --2- 172.21.15.143:0/2996247951 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2134067ca0 0x7f213406a160 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f215c06de90 tx=0x7f2144009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:09.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.588+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f214c051a40 con 0x7f215c0b21d0 2024-01-15T18:22:09.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.768+0000 7f2163200640 1 -- 172.21.15.143:0/2996247951 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f215c0b4070 con 0x7f2134067ca0 2024-01-15T18:22:09.769 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:09.776+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:09.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:09.776+0000 7f2153e00640 1 -- 172.21.15.143:0/2996247951 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f215c0b4070 con 0x7f2134067ca0 2024-01-15T18:22:09.770 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:09.770 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:09.778 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:11.780 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:11.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 Processor -- start 2024-01-15T18:22:11.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 -- start start 2024-01-15T18:22:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de015aef0 0x7f8de006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8de006c560 0x7f8de00b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de015f7c0 con 0x7f8de00b27d0 2024-01-15T18:22:11.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de006c310 con 0x7f8de006c560 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de015ebb0 con 0x7f8de015aef0 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8de006c560 0x7f8de00b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8de006c560 0x7f8de00b2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48824/0 (socket says 172.21.15.143:48824) 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 -- 172.21.15.143:0/2633212543 learned_addr learned my addr 172.21.15.143:0/2633212543 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5c00640 1 --2- 172.21.15.143:0/2633212543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de015aef0 0x7f8de006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 -- 172.21.15.143:0/2633212543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de015aef0 msgr2=0x7f8de006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 --2- 172.21.15.143:0/2633212543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de015aef0 0x7f8de006b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 -- 172.21.15.143:0/2633212543 >> v1:172.21.15.204:6789/0 conn(0x7f8de00b27d0 legacy=0x7f8de0071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8de5200640 1 -- 172.21.15.143:0/2633212543 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00b2c30 con 0x7f8de006c560 2024-01-15T18:22:11.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.932+0000 7f8ddfe00640 1 -- 172.21.15.143:0/2633212543 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089922622 0 0) 0x7f8de015f7c0 con 0x7f8de00b27d0 2024-01-15T18:22:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de5200640 1 --2- 172.21.15.143:0/2633212543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de006c560 0x7f8de00b2290 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009870 tx=0x7f8dc801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446715a305dff6e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8ddfe00640 1 -- 172.21.15.143:0/2633212543 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80ab620 con 0x7f8de006c560 2024-01-15T18:22:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8ddfe00640 1 -- 172.21.15.143:0/2633212543 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc80ab780 con 0x7f8de006c560 2024-01-15T18:22:11.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8ddfe00640 1 -- 172.21.15.143:0/2633212543 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80abb10 con 0x7f8de006c560 2024-01-15T18:22:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de8600640 1 -- 172.21.15.143:0/2633212543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de006c560 msgr2=0x7f8de00b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de8600640 1 --2- 172.21.15.143:0/2633212543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de006c560 0x7f8de00b2290 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009870 tx=0x7f8dc801c510 comp rx=0 tx=0).stop 2024-01-15T18:22:11.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de8600640 1 -- 172.21.15.143:0/2633212543 shutdown_connections 2024-01-15T18:22:11.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de8600640 1 --2- 172.21.15.143:0/2633212543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de006c560 0x7f8de00b2290 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:11.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de8600640 1 --2- 172.21.15.143:0/2633212543 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de015aef0 0x7f8de006b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:11.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.936+0000 7f8de8600640 1 -- 172.21.15.143:0/2633212543 >> 172.21.15.143:0/2633212543 conn(0x7f8de0066720 msgr2=0x7f8de006dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:11.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 -- 172.21.15.143:0/2633212543 shutdown_connections 2024-01-15T18:22:11.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 -- 172.21.15.143:0/2633212543 wait complete. 2024-01-15T18:22:11.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 Processor -- start 2024-01-15T18:22:11.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 -- start start 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de006c560 0x7f8de014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de00b27d0 0x7f8de015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de015aef0 0x7f8de01495a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de015df10 con 0x7f8de00b27d0 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de015d300 con 0x7f8de015aef0 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de8600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de015d5e0 con 0x7f8de006c560 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de00b27d0 0x7f8de015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de00b27d0 0x7f8de015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48830/0 (socket says 172.21.15.143:48830) 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 -- 172.21.15.143:0/1558021898 learned_addr learned my addr 172.21.15.143:0/1558021898 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de7000640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de015aef0 0x7f8de01495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5c00640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de006c560 0x7f8de014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:11.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 -- 172.21.15.143:0/1558021898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de006c560 msgr2=0x7f8de014c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8de006c560 0x7f8de014c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 -- 172.21.15.143:0/1558021898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de015aef0 msgr2=0x7f8de01495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8de015aef0 0x7f8de01495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 -- 172.21.15.143:0/1558021898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc80094d0 con 0x7f8de00b27d0 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.940+0000 7f8de5200640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8de00b27d0 0x7f8de015c840 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80ad040 tx=0x7f8dc801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80abe40 con 0x7f8de00b27d0 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc80be3f0 con 0x7f8de00b27d0 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80be7a0 con 0x7f8de00b27d0 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8de8600640 1 -- 172.21.15.143:0/1558021898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de015d8c0 con 0x7f8de00b27d0 2024-01-15T18:22:11.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8de8600640 1 -- 172.21.15.143:0/1558021898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de014a2f0 con 0x7f8de00b27d0 2024-01-15T18:22:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8de8600640 1 -- 172.21.15.143:0/1558021898 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de015e080 con 0x7f8de00b27d0 2024-01-15T18:22:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f8dc80b43f0 con 0x7f8de00b27d0 2024-01-15T18:22:11.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8ddd600640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8db8067ca0 0x7f8db806a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8dc811d330 con 0x7f8de00b27d0 2024-01-15T18:22:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8de5c00640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8db8067ca0 0x7f8db806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:11.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.944+0000 7f8de5c00640 1 --2- 172.21.15.143:0/1558021898 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8db8067ca0 0x7f8db806a160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8de0154c50 tx=0x7f8dd0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:11.944 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:11.948+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8dc80ec070 con 0x7f8de00b27d0 2024-01-15T18:22:12.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:12.128+0000 7f8de8600640 1 -- 172.21.15.143:0/1558021898 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8de00ce610 con 0x7f8db8067ca0 2024-01-15T18:22:12.130 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:12.136+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:12.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:12.136+0000 7f8ddd600640 1 -- 172.21.15.143:0/1558021898 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8de00ce610 con 0x7f8db8067ca0 2024-01-15T18:22:12.130 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:12.131 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:12.138 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:14.140 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:14.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 Processor -- start 2024-01-15T18:22:14.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 -- start start 2024-01-15T18:22:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3100cfb30 0x7fd31006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3100cbf50 0x7fd3100cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3100cffd0 con 0x7fd3101607c0 2024-01-15T18:22:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3100d1ce0 con 0x7fd3100cfb30 2024-01-15T18:22:14.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd318200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3100d02b0 con 0x7fd3100cbf50 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd314e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd3100cfb30 0x7fd31006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd315800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd3101607c0 0x7fd31006b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38646/0 (socket says 172.21.15.143:38646) 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd315800640 1 -- 172.21.15.143:0/1053373189 learned_addr learned my addr 172.21.15.143:0/1053373189 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd316200640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3100cbf50 0x7fd3100cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd314e00640 1 -- 172.21.15.143:0/1053373189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3100cbf50 msgr2=0x7fd3100cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd314e00640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3100cbf50 0x7fd3100cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd314e00640 1 -- 172.21.15.143:0/1053373189 >> v1:172.21.15.143:6789/0 conn(0x7fd3101607c0 legacy=0x7fd31006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd314e00640 1 -- 172.21.15.143:0/1053373189 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100cb470 con 0x7fd3100cfb30 2024-01-15T18:22:14.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd30fe00640 1 -- 172.21.15.143:0/1053373189 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214137464 0 0) 0x7fd3100cffd0 con 0x7fd3101607c0 2024-01-15T18:22:14.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.300+0000 7fd316200640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3100cbf50 0x7fd3100cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:22:14.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd314e00640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cfb30 0x7fd31006bec0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80097c0 tx=0x7fd2f801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f2d0aeb6fba3e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd30fe00640 1 -- 172.21.15.143:0/1053373189 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80a8970 con 0x7fd3100cfb30 2024-01-15T18:22:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd30fe00640 1 -- 172.21.15.143:0/1053373189 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f80a8ad0 con 0x7fd3100cfb30 2024-01-15T18:22:14.298 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd30fe00640 1 -- 172.21.15.143:0/1053373189 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80a8e60 con 0x7fd3100cfb30 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 -- 172.21.15.143:0/1053373189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cfb30 msgr2=0x7fd31006bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cfb30 0x7fd31006bec0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80097c0 tx=0x7fd2f801ec70 comp rx=0 tx=0).stop 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 -- 172.21.15.143:0/1053373189 shutdown_connections 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3100cbf50 0x7fd3100cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 --2- 172.21.15.143:0/1053373189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cfb30 0x7fd31006bec0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 -- 172.21.15.143:0/1053373189 >> 172.21.15.143:0/1053373189 conn(0x7fd3100d0e20 msgr2=0x7fd3100d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 -- 172.21.15.143:0/1053373189 shutdown_connections 2024-01-15T18:22:14.299 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 -- 172.21.15.143:0/1053373189 wait complete. 2024-01-15T18:22:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 Processor -- start 2024-01-15T18:22:14.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.304+0000 7fd318200640 1 -- start start 2024-01-15T18:22:14.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cbf50 0x7fd31009ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3100cfb30 0x7fd31014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3101607c0 0x7fd31014f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31014f980 con 0x7fd3100cbf50 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31009b0e0 con 0x7fd3100cfb30 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31009b3c0 con 0x7fd3101607c0 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cbf50 0x7fd31009ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cbf50 0x7fd31009ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48846/0 (socket says 172.21.15.143:48846) 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 -- 172.21.15.143:0/617358858 learned_addr learned my addr 172.21.15.143:0/617358858 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd316200640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3101607c0 0x7fd31014f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd314e00640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3100cfb30 0x7fd31014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 -- 172.21.15.143:0/617358858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3101607c0 msgr2=0x7fd31014f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd3101607c0 0x7fd31014f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 -- 172.21.15.143:0/617358858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3100cfb30 msgr2=0x7fd31014ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:14.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3100cfb30 0x7fd31014ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 -- 172.21.15.143:0/617358858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8009420 con 0x7fd3100cbf50 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd315800640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3100cbf50 0x7fd31009ca10 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c480 tx=0x7fd30000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000074e0 con 0x7fd3100cbf50 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300007660 con 0x7fd3100cbf50 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 -- 172.21.15.143:0/617358858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31009b6a0 con 0x7fd3100cbf50 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300004a30 con 0x7fd3100cbf50 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 -- 172.21.15.143:0/617358858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31009bc60 con 0x7fd3100cbf50 2024-01-15T18:22:14.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd318200640 1 -- 172.21.15.143:0/617358858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd31006d4b0 con 0x7fd3100cbf50 2024-01-15T18:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.308+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fd300004b90 con 0x7fd3100cbf50 2024-01-15T18:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.312+0000 7fd30d600640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd2e8067ca0 0x7fd2e806a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.312+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd300082680 con 0x7fd3100cbf50 2024-01-15T18:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.312+0000 7fd314e00640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd2e8067ca0 0x7fd2e806a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:14.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.312+0000 7fd314e00640 1 --2- 172.21.15.143:0/617358858 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd2e8067ca0 0x7fd2e806a160 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80b2000 tx=0x7fd2f80aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:14.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.316+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd300050f90 con 0x7fd3100cbf50 2024-01-15T18:22:14.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.492+0000 7fd318200640 1 -- 172.21.15.143:0/617358858 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3100ce6b0 con 0x7fd2e8067ca0 2024-01-15T18:22:14.495 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:14.500+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:14.500+0000 7fd30d600640 1 -- 172.21.15.143:0/617358858 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd2f80a8770 con 0x7fd2e8067ca0 2024-01-15T18:22:14.496 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:14.496 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:14.504 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:16.505 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.664+0000 7fe81d000640 1 Processor -- start 2024-01-15T18:22:16.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.664+0000 7fe81d000640 1 -- start start 2024-01-15T18:22:16.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe81d000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe81815d1f0 0x7fe81806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe81d000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8180b3370 con 0x7fe81806bb70 2024-01-15T18:22:16.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe81d000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81815d5d0 con 0x7fe818071470 2024-01-15T18:22:16.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe81d000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81806a800 con 0x7fe81815d1f0 2024-01-15T18:22:16.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe815400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe81806bb70 0x7fe818070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38652/0 (socket says 172.21.15.143:38652) 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe815400640 1 -- 172.21.15.143:0/1563220620 learned_addr learned my addr 172.21.15.143:0/1563220620 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe815e00640 1 --2- 172.21.15.143:0/1563220620 >> v2:172.21.15.204:3300/0 conn(0x7fe81815d1f0 0x7fe81806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381794712 0 0) 0x7fe8180b3370 con 0x7fe81806bb70 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f8003530 con 0x7fe81806bb70 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2526809391 0 0) 0x7fe7f8003530 con 0x7fe81806bb70 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 >> v2:172.21.15.204:3300/0 conn(0x7fe81815d1f0 msgr2=0x7fe81806b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 --2- 172.21.15.143:0/1563220620 >> v2:172.21.15.204:3300/0 conn(0x7fe81815d1f0 0x7fe81806b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 >> v1:172.21.15.204:6789/0 conn(0x7fe818071470 legacy=0x7fe8180acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81806aae0 con 0x7fe81806bb70 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fe800001680 con 0x7fe81806bb70 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 >> v1:172.21.15.143:6789/0 conn(0x7fe81806bb70 legacy=0x7fe818070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8003980 0x7fe7f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7f80047d0 0x7fe7f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7f8005190 0x7fe7f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81806a800 con 0x7fe7f8005190 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f800c030 con 0x7fe7f8003980 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f800c1f0 con 0x7fe7f80047d0 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe815e00640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8003980 0x7fe7f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe816800640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7f8005190 0x7fe7f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe815400640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7f80047d0 0x7fe7f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe816800640 1 -- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7f80047d0 msgr2=0x7fe7f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe816800640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe7f80047d0 0x7fe7f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe816800640 1 -- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8003980 msgr2=0x7fe7f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe816800640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8003980 0x7fe7f8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:16.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.668+0000 7fe816800640 1 -- 172.21.15.143:0/1563220620 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f8003530 con 0x7fe7f8005190 2024-01-15T18:22:16.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe816800640 1 -- 172.21.15.143:0/1563220620 reap_dead start 2024-01-15T18:22:16.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe816800640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7f8005190 0x7fe7f800b990 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe810001e70 tx=0x7fe81001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce50372f2f7064b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810003b80 con 0x7fe7f8005190 2024-01-15T18:22:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810003d00 con 0x7fe7f8005190 2024-01-15T18:22:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe814a00640 1 -- 172.21.15.143:0/1563220620 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810005260 con 0x7fe7f8005190 2024-01-15T18:22:16.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 -- 172.21.15.143:0/1563220620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7f8005190 msgr2=0x7fe7f800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7f8005190 0x7fe7f800b990 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe810001e70 tx=0x7fe81001c760 comp rx=0 tx=0).stop 2024-01-15T18:22:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 -- 172.21.15.143:0/1563220620 shutdown_connections 2024-01-15T18:22:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 --2- 172.21.15.143:0/1563220620 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe7f8005190 0x7fe7f800b990 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 -- 172.21.15.143:0/1563220620 >> 172.21.15.143:0/1563220620 conn(0x7fe8180b1610 msgr2=0x7fe8180b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 -- 172.21.15.143:0/1563220620 shutdown_connections 2024-01-15T18:22:16.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.672+0000 7fe81d000640 1 -- 172.21.15.143:0/1563220620 wait complete. 2024-01-15T18:22:16.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 Processor -- start 2024-01-15T18:22:16.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 -- start start 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8005190 0x7fe8181f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8180b02a0 0x7fe8180c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8180c53e0 0x7fe8180adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180c4fc0 con 0x7fe8180b02a0 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180b0ac0 con 0x7fe7f8005190 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180c4620 con 0x7fe8180c53e0 2024-01-15T18:22:16.668 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8180b02a0 0x7fe8180c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8180b02a0 0x7fe8180c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48866/0 (socket says 172.21.15.143:48866) 2024-01-15T18:22:16.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 -- 172.21.15.143:0/4134681391 learned_addr learned my addr 172.21.15.143:0/4134681391 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:16.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe816800640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8180c53e0 0x7fe8180adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815e00640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8005190 0x7fe8181f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 -- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8180c53e0 msgr2=0x7fe8180adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe8180c53e0 0x7fe8180adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 -- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8005190 msgr2=0x7fe8181f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8005190 0x7fe8181f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 -- 172.21.15.143:0/4134681391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8180c4900 con 0x7fe8180b02a0 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815e00640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe7f8005190 0x7fe8181f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe815400640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8180b02a0 0x7fe8180c60b0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe8000072c0 tx=0x7fe800007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000f940 con 0x7fe8180b02a0 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80000fac0 con 0x7fe8180b02a0 2024-01-15T18:22:16.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000fe70 con 0x7fe8180b02a0 2024-01-15T18:22:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 -- 172.21.15.143:0/4134681391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8180c4be0 con 0x7fe8180b02a0 2024-01-15T18:22:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.676+0000 7fe81d000640 1 -- 172.21.15.143:0/4134681391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8180c65f0 con 0x7fe8180b02a0 2024-01-15T18:22:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.680+0000 7fe81d000640 1 -- 172.21.15.143:0/4134681391 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe818074b00 con 0x7fe8180b02a0 2024-01-15T18:22:16.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.680+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fe80001d460 con 0x7fe8180b02a0 2024-01-15T18:22:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.680+0000 7fe80e000640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f0067cd0 0x7fe7f006a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.680+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fe8000834c0 con 0x7fe8180b02a0 2024-01-15T18:22:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.680+0000 7fe815e00640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f0067cd0 0x7fe7f006a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:16.672 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.680+0000 7fe815e00640 1 --2- 172.21.15.143:0/4134681391 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe7f0067cd0 0x7fe7f006a190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe808009b60 tx=0x7fe808008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:16.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.684+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe800051da0 con 0x7fe8180b02a0 2024-01-15T18:22:16.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.860+0000 7fe81d000640 1 -- 172.21.15.143:0/4134681391 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe81806b380 con 0x7fe7f0067cd0 2024-01-15T18:22:16.858 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:16.868+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:16.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:16.868+0000 7fe80e000640 1 -- 172.21.15.143:0/4134681391 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe8080097b0 con 0x7fe7f0067cd0 2024-01-15T18:22:16.858 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:16.859 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:16.866 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:18.867 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:19.021 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 Processor -- start 2024-01-15T18:22:19.022 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 -- start start 2024-01-15T18:22:19.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b001607c0 0x7f4b0006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:19.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4b000cfb30 0x7f4b0006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:19.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b000cffd0 con 0x7f4b0006c230 2024-01-15T18:22:19.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0006c670 con 0x7f4b000cfb30 2024-01-15T18:22:19.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b07c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000d1ce0 con 0x7f4b001607c0 2024-01-15T18:22:19.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b05c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4b0006c230 0x7f4b000cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55658/0 (socket says 172.21.15.143:55658) 2024-01-15T18:22:19.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b05c00640 1 -- 172.21.15.143:0/2758205038 learned_addr learned my addr 172.21.15.143:0/2758205038 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:19.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4affe00640 1 --2- 172.21.15.143:0/2758205038 >> v2:172.21.15.204:3300/0 conn(0x7f4b000cfb30 0x7f4b0006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:19.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4b05200640 1 --2- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b001607c0 0x7f4b0006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4aff400640 1 -- 172.21.15.143:0/2758205038 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404159459 0 0) 0x7f4b000cffd0 con 0x7f4b0006c230 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4aff400640 1 -- 172.21.15.143:0/2758205038 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae0003530 con 0x7f4b0006c230 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4affe00640 1 -- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b001607c0 msgr2=0x7f4b0006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4affe00640 1 --2- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b001607c0 0x7f4b0006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4aff400640 1 -- 172.21.15.143:0/2758205038 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3233652532 0 0) 0x7f4ae0003530 con 0x7f4b0006c230 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4affe00640 1 -- 172.21.15.143:0/2758205038 >> v1:172.21.15.143:6789/0 conn(0x7f4b0006c230 legacy=0x7f4b000cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:19.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.032+0000 7f4affe00640 1 -- 172.21.15.143:0/2758205038 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b000d02b0 con 0x7f4b000cfb30 2024-01-15T18:22:19.026 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4affe00640 1 --2- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b000cfb30 0x7f4b0006bcf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80097c0 tx=0x7f4ae801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15572f6dc8795757 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:19.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4aff400640 1 -- 172.21.15.143:0/2758205038 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80a8970 con 0x7f4b000cfb30 2024-01-15T18:22:19.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4aff400640 1 -- 172.21.15.143:0/2758205038 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae80a8ad0 con 0x7f4b000cfb30 2024-01-15T18:22:19.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4aff400640 1 -- 172.21.15.143:0/2758205038 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80a8e60 con 0x7f4b000cfb30 2024-01-15T18:22:19.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 -- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b000cfb30 msgr2=0x7f4b0006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:19.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 --2- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b000cfb30 0x7f4b0006bcf0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80097c0 tx=0x7f4ae801ec70 comp rx=0 tx=0).stop 2024-01-15T18:22:19.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 -- 172.21.15.143:0/2758205038 shutdown_connections 2024-01-15T18:22:19.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 --2- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b000cfb30 0x7f4b0006bcf0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:19.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 --2- 172.21.15.143:0/2758205038 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b001607c0 0x7f4b0006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:19.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 -- 172.21.15.143:0/2758205038 >> 172.21.15.143:0/2758205038 conn(0x7f4b000d0e20 msgr2=0x7f4b000d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:19.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 -- 172.21.15.143:0/2758205038 shutdown_connections 2024-01-15T18:22:19.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.036+0000 7f4b07c00640 1 -- 172.21.15.143:0/2758205038 wait complete. 2024-01-15T18:22:19.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 Processor -- start 2024-01-15T18:22:19.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 -- start start 2024-01-15T18:22:19.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0006c230 0x7f4b00151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b000cfb30 0x7f4b000aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b001607c0 0x7f4b000ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000b15f0 con 0x7f4b0006c230 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b000b1a10 con 0x7f4b001607c0 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b00151840 con 0x7f4b000cfb30 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0006c230 0x7f4b00151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0006c230 0x7f4b00151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34472/0 (socket says 172.21.15.143:34472) 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 -- 172.21.15.143:0/276416822 learned_addr learned my addr 172.21.15.143:0/276416822 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4affe00640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b000cfb30 0x7f4b000aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05c00640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b001607c0 0x7f4b000ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 -- 172.21.15.143:0/276416822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b000cfb30 msgr2=0x7f4b000aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b000cfb30 0x7f4b000aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 -- 172.21.15.143:0/276416822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b001607c0 msgr2=0x7f4b000ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:19.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b001607c0 0x7f4b000ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 -- 172.21.15.143:0/276416822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009420 con 0x7f4b0006c230 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b05200640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0006c230 0x7f4b00151300 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f4af000e870 tx=0x7f4af000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0009510 con 0x7f4b0006c230 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af000eec0 con 0x7f4b0006c230 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000cb40 con 0x7f4b0006c230 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 -- 172.21.15.143:0/276416822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b000b0970 con 0x7f4b0006c230 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.040+0000 7f4b07c00640 1 -- 172.21.15.143:0/276416822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b000b0f30 con 0x7f4b0006c230 2024-01-15T18:22:19.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.044+0000 7f4b07c00640 1 -- 172.21.15.143:0/276416822 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0006d4b0 con 0x7f4b0006c230 2024-01-15T18:22:19.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.044+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f4af000cca0 con 0x7f4b0006c230 2024-01-15T18:22:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.044+0000 7f4afcc00640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ad8067cd0 0x7f4ad806a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.044+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f4af0083530 con 0x7f4b0006c230 2024-01-15T18:22:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.044+0000 7f4affe00640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ad8067cd0 0x7f4ad806a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:19.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.044+0000 7f4affe00640 1 --2- 172.21.15.143:0/276416822 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ad8067cd0 0x7f4ad806a190 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80b2000 tx=0x7f4ae8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:19.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.048+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4af0051ec0 con 0x7f4b0006c230 2024-01-15T18:22:19.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.224+0000 7f4b07c00640 1 -- 172.21.15.143:0/276416822 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b000cba30 con 0x7f4ad8067cd0 2024-01-15T18:22:19.222 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:19.232+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:19.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:19.232+0000 7f4afcc00640 1 -- 172.21.15.143:0/276416822 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ae80b0440 con 0x7f4ad8067cd0 2024-01-15T18:22:19.223 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:19.223 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:19.232 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:21.233 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:21.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 Processor -- start 2024-01-15T18:22:21.385 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 -- start start 2024-01-15T18:22:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2d80b3e10 0x7fd2d81639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d806be00 con 0x7fd2d815a800 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d806b840 con 0x7fd2d80b3e10 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2dd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d806bb20 con 0x7fd2d815a420 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2d80b3e10 0x7fd2d81639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2d80b3e10 0x7fd2d81639d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34478/0 (socket says 172.21.15.143:34478) 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d5400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd2d815a800 0x7fd2d815ac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55662/0 (socket says 172.21.15.143:55662) 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 -- 172.21.15.143:0/3418755594 learned_addr learned my addr 172.21.15.143:0/3418755594 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d5e00640 1 --2- 172.21.15.143:0/3418755594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 -- 172.21.15.143:0/3418755594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 msgr2=0x7fd2d806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 --2- 172.21.15.143:0/3418755594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d806aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:21.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 -- 172.21.15.143:0/3418755594 >> v1:172.21.15.143:6789/0 conn(0x7fd2d815a800 legacy=0x7fd2d815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:21.389 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.396+0000 7fd2d6800640 1 -- 172.21.15.143:0/3418755594 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80b2620 con 0x7fd2d80b3e10 2024-01-15T18:22:21.390 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2d6800640 1 --2- 172.21.15.143:0/3418755594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d81639d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000d150 tx=0x7fd2d001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccf6e3b195ebf51 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2d4a00640 1 -- 172.21.15.143:0/3418755594 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d000b2d0 con 0x7fd2d80b3e10 2024-01-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2d4a00640 1 -- 172.21.15.143:0/3418755594 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d000b430 con 0x7fd2d80b3e10 2024-01-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2d4a00640 1 -- 172.21.15.143:0/3418755594 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00a7a80 con 0x7fd2d80b3e10 2024-01-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 -- 172.21.15.143:0/3418755594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 msgr2=0x7fd2d81639d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 --2- 172.21.15.143:0/3418755594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d81639d0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000d150 tx=0x7fd2d001eb80 comp rx=0 tx=0).stop 2024-01-15T18:22:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 -- 172.21.15.143:0/3418755594 shutdown_connections 2024-01-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 --2- 172.21.15.143:0/3418755594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d81639d0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 --2- 172.21.15.143:0/3418755594 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d806aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 -- 172.21.15.143:0/3418755594 >> 172.21.15.143:0/3418755594 conn(0x7fd2d80b1610 msgr2=0x7fd2d80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 -- 172.21.15.143:0/3418755594 shutdown_connections 2024-01-15T18:22:21.392 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.400+0000 7fd2dd200640 1 -- 172.21.15.143:0/3418755594 wait complete. 2024-01-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 Processor -- start 2024-01-15T18:22:21.393 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- start start 2024-01-15T18:22:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d80ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d815a800 0x7fd2d8158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:21.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8140490 con 0x7fd2d80b3e10 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d813fe20 con 0x7fd2d815a800 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80adb20 con 0x7fd2d815a420 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d813f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34490/0 (socket says 172.21.15.143:34490) 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 -- 172.21.15.143:0/3959157212 learned_addr learned my addr 172.21.15.143:0/3959157212 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5400640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d80ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d6800640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d815a800 0x7fd2d8158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 -- 172.21.15.143:0/3959157212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 msgr2=0x7fd2d80ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd2d815a420 0x7fd2d80ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 -- 172.21.15.143:0/3959157212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d815a800 msgr2=0x7fd2d8158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2d815a800 0x7fd2d8158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:21.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 -- 172.21.15.143:0/3959157212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d000bd40 con 0x7fd2d80b3e10 2024-01-15T18:22:21.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2d5e00640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b3e10 0x7fd2d813f8e0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d5f0 tx=0x7fd2c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:21.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80125f0 con 0x7fd2d80b3e10 2024-01-15T18:22:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c8012750 con 0x7fd2d80b3e10 2024-01-15T18:22:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8012ae0 con 0x7fd2d80b3e10 2024-01-15T18:22:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- 172.21.15.143:0/3959157212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d80ade00 con 0x7fd2d80b3e10 2024-01-15T18:22:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- 172.21.15.143:0/3959157212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d81400b0 con 0x7fd2d80b3e10 2024-01-15T18:22:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.404+0000 7fd2dd200640 1 -- 172.21.15.143:0/3959157212 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d8074b00 con 0x7fd2d80b3e10 2024-01-15T18:22:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.408+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fd2c8012c40 con 0x7fd2d80b3e10 2024-01-15T18:22:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.408+0000 7fd2ce000640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd2b0067ca0 0x7fd2b006a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.408+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd2c8083780 con 0x7fd2d80b3e10 2024-01-15T18:22:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.408+0000 7fd2d5400640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd2b0067ca0 0x7fd2b006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.408+0000 7fd2d5400640 1 --2- 172.21.15.143:0/3959157212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd2b0067ca0 0x7fd2b006a160 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd2d81f94c0 tx=0x7fd2c0007e50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:21.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.412+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd2c8052170 con 0x7fd2d80b3e10 2024-01-15T18:22:21.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.592+0000 7fd2dd200640 1 -- 172.21.15.143:0/3959157212 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2d806b5f0 con 0x7fd2b0067ca0 2024-01-15T18:22:21.588 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:21.596+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:21.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:21.596+0000 7fd2ce000640 1 -- 172.21.15.143:0/3959157212 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd2c0004a70 con 0x7fd2b0067ca0 2024-01-15T18:22:21.588 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:21.597 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:23.599 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 Processor -- start 2024-01-15T18:22:23.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 -- start start 2024-01-15T18:22:23.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f63d40b2320 0x7f63d40cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f63d40d1b30 0x7f63d406beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d40cb670 con 0x7f63d40d0110 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40cb950 con 0x7f63d40d1b30 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63dac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40cbc30 con 0x7f63d40b2320 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f63d40d1b30 0x7f63d406beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f63d40d1b30 0x7f63d406beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34496/0 (socket says 172.21.15.143:34496) 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 -- 172.21.15.143:0/2624141907 learned_addr learned my addr 172.21.15.143:0/2624141907 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:23.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3e00640 1 --2- 172.21.15.143:0/2624141907 >> v2:172.21.15.204:3300/0 conn(0x7f63d40b2320 0x7f63d40cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 -- 172.21.15.143:0/2624141907 >> v2:172.21.15.204:3300/0 conn(0x7f63d40b2320 msgr2=0x7f63d40cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 --2- 172.21.15.143:0/2624141907 >> v2:172.21.15.204:3300/0 conn(0x7f63d40b2320 0x7f63d40cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 -- 172.21.15.143:0/2624141907 >> v1:172.21.15.204:6789/0 conn(0x7f63d40d0110 legacy=0x7f63d40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:23.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.760+0000 7f63d3400640 1 -- 172.21.15.143:0/2624141907 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40cbf10 con 0x7f63d40d1b30 2024-01-15T18:22:23.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.764+0000 7f63d3400640 1 --2- 172.21.15.143:0/2624141907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d406beb0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f63d4071160 tx=0x7f63bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8dfc4e3d28e440 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.764+0000 7f63d2a00640 1 -- 172.21.15.143:0/2624141907 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0aa620 con 0x7f63d40d1b30 2024-01-15T18:22:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.764+0000 7f63d2a00640 1 -- 172.21.15.143:0/2624141907 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc0aa780 con 0x7f63d40d1b30 2024-01-15T18:22:23.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.764+0000 7f63d2a00640 1 -- 172.21.15.143:0/2624141907 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0aab10 con 0x7f63d40d1b30 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- 172.21.15.143:0/2624141907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 msgr2=0x7f63d406beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 --2- 172.21.15.143:0/2624141907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d406beb0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f63d4071160 tx=0x7f63bc01c4a0 comp rx=0 tx=0).stop 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- 172.21.15.143:0/2624141907 shutdown_connections 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 --2- 172.21.15.143:0/2624141907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d406beb0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 --2- 172.21.15.143:0/2624141907 >> v2:172.21.15.204:3300/0 conn(0x7f63d40b2320 0x7f63d40cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- 172.21.15.143:0/2624141907 >> 172.21.15.143:0/2624141907 conn(0x7f63d40d0e80 msgr2=0x7f63d40d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- 172.21.15.143:0/2624141907 shutdown_connections 2024-01-15T18:22:23.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- 172.21.15.143:0/2624141907 wait complete. 2024-01-15T18:22:23.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 Processor -- start 2024-01-15T18:22:23.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- start start 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63d40b2320 0x7f63d4150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63d40d0110 0x7f63d4153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d4153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d415a7b0 con 0x7f63d40d1b30 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4153c70 con 0x7f63d40b2320 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63dac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d4153f50 con 0x7f63d40d0110 2024-01-15T18:22:23.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.768+0000 7f63d8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d4153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:23.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d4153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34506/0 (socket says 172.21.15.143:34506) 2024-01-15T18:22:23.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 -- 172.21.15.143:0/3892454515 learned_addr learned my addr 172.21.15.143:0/3892454515 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:23.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d3400640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63d40d0110 0x7f63d4153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:23.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d3e00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63d40b2320 0x7f63d4150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 -- 172.21.15.143:0/3892454515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63d40d0110 msgr2=0x7f63d4153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63d40d0110 0x7f63d4153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 -- 172.21.15.143:0/3892454515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63d40b2320 msgr2=0x7f63d4150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63d40b2320 0x7f63d4150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 -- 172.21.15.143:0/3892454515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0094d0 con 0x7f63d40d1b30 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63d8c00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63d40d1b30 0x7f63d4153660 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f63cc00c770 tx=0x7f63cc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc003b80 con 0x7f63d40d1b30 2024-01-15T18:22:23.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc003ce0 con 0x7f63d40d1b30 2024-01-15T18:22:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00aa90 con 0x7f63d40d1b30 2024-01-15T18:22:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63dac00640 1 -- 172.21.15.143:0/3892454515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d4155640 con 0x7f63d40d1b30 2024-01-15T18:22:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63dac00640 1 -- 172.21.15.143:0/3892454515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d415da50 con 0x7f63d40d1b30 2024-01-15T18:22:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63dac00640 1 -- 172.21.15.143:0/3892454515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d4155480 con 0x7f63d40d1b30 2024-01-15T18:22:23.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f63cc00abf0 con 0x7f63d40d1b30 2024-01-15T18:22:23.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.772+0000 7f63cbe00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63ac067ca0 0x7f63ac06a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:23.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.776+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f63cc082dd0 con 0x7f63d40d1b30 2024-01-15T18:22:23.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.776+0000 7f63d3e00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63ac067ca0 0x7f63ac06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:23.766 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.776+0000 7f63d3e00640 1 --2- 172.21.15.143:0/3892454515 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f63ac067ca0 0x7f63ac06a160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f63d40d08a0 tx=0x7f63c4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:23.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.780+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63cc051b10 con 0x7f63d40d1b30 2024-01-15T18:22:23.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.960+0000 7f63dac00640 1 -- 172.21.15.143:0/3892454515 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63d415dc40 con 0x7f63ac067ca0 2024-01-15T18:22:23.957 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:23.964+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:23.964+0000 7f63cbe00640 1 -- 172.21.15.143:0/3892454515 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f63d415dc40 con 0x7f63ac067ca0 2024-01-15T18:22:23.957 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:23.957 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:23.966 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:25.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:26.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 Processor -- start 2024-01-15T18:22:26.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 -- start start 2024-01-15T18:22:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9324141660 0x7f93240ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f932415ec80 0x7f9324141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932415f310 con 0x7f9324142470 2024-01-15T18:22:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932415fbb0 con 0x7f932415ec80 2024-01-15T18:22:26.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9329400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932415f100 con 0x7f9324141660 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9321e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f932415ec80 0x7f9324141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9321e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f932415ec80 0x7f9324141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34522/0 (socket says 172.21.15.143:34522) 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9321e00640 1 -- 172.21.15.143:0/1355998777 learned_addr learned my addr 172.21.15.143:0/1355998777 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9323200640 1 --1- 172.21.15.143:0/1355998777 >> v1:172.21.15.143:6789/0 conn(0x7f9324142470 0x7f9324071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55678/0 (socket says 172.21.15.143:55678) 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.128+0000 7f9322800640 1 --2- 172.21.15.143:0/1355998777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9324141660 0x7f93240ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321e00640 1 -- 172.21.15.143:0/1355998777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9324141660 msgr2=0x7f93240ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321e00640 1 --2- 172.21.15.143:0/1355998777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9324141660 0x7f93240ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321e00640 1 -- 172.21.15.143:0/1355998777 >> v1:172.21.15.143:6789/0 conn(0x7f9324142470 legacy=0x7f9324071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321e00640 1 -- 172.21.15.143:0/1355998777 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932415f5f0 con 0x7f932415ec80 2024-01-15T18:22:26.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321400640 1 -- 172.21.15.143:0/1355998777 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236758621 0 0) 0x7f932415f310 con 0x7f9324142470 2024-01-15T18:22:26.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321e00640 1 --2- 172.21.15.143:0/1355998777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f932415ec80 0x7f9324141f30 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f930c009870 tx=0x7f930c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c07f0dacac6d8a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:26.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321400640 1 -- 172.21.15.143:0/1355998777 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0aa670 con 0x7f932415ec80 2024-01-15T18:22:26.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321400640 1 -- 172.21.15.143:0/1355998777 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c0aa7d0 con 0x7f932415ec80 2024-01-15T18:22:26.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.132+0000 7f9321400640 1 -- 172.21.15.143:0/1355998777 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0aab60 con 0x7f932415ec80 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- 172.21.15.143:0/1355998777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f932415ec80 msgr2=0x7f9324141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 --2- 172.21.15.143:0/1355998777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f932415ec80 0x7f9324141f30 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f930c009870 tx=0x7f930c01c510 comp rx=0 tx=0).stop 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- 172.21.15.143:0/1355998777 shutdown_connections 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 --2- 172.21.15.143:0/1355998777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f932415ec80 0x7f9324141f30 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 --2- 172.21.15.143:0/1355998777 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9324141660 0x7f93240ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- 172.21.15.143:0/1355998777 >> 172.21.15.143:0/1355998777 conn(0x7f932406c3f0 msgr2=0x7f932406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- 172.21.15.143:0/1355998777 shutdown_connections 2024-01-15T18:22:26.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- 172.21.15.143:0/1355998777 wait complete. 2024-01-15T18:22:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 Processor -- start 2024-01-15T18:22:26.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- start start 2024-01-15T18:22:26.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324141660 0x7f932413ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9324142470 0x7f932415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932415ec80 0x7f932415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932413db40 con 0x7f9324142470 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932413de20 con 0x7f9324141660 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9329400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932413e100 con 0x7f932415ec80 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9324142470 0x7f932415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9324142470 0x7f932415c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34530/0 (socket says 172.21.15.143:34530) 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 -- 172.21.15.143:0/3129047278 learned_addr learned my addr 172.21.15.143:0/3129047278 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9322800640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324141660 0x7f932413ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9323200640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932415ec80 0x7f932415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 -- 172.21.15.143:0/3129047278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932415ec80 msgr2=0x7f932415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f932415ec80 0x7f932415c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:26.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 -- 172.21.15.143:0/3129047278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324141660 msgr2=0x7f932413ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.136+0000 7f9321e00640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324141660 0x7f932413ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f9321e00640 1 -- 172.21.15.143:0/3129047278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0094d0 con 0x7f9324142470 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f9321e00640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9324142470 0x7f932415c0d0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ac040 tx=0x7f930c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0b3800 con 0x7f9324142470 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c0b3980 con 0x7f9324142470 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f9329400640 1 -- 172.21.15.143:0/3129047278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93240cd150 con 0x7f9324142470 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0b3d30 con 0x7f9324142470 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f9329400640 1 -- 172.21.15.143:0/3129047278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932413e6c0 con 0x7f9324142470 2024-01-15T18:22:26.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f9329400640 1 -- 172.21.15.143:0/3129047278 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9324067500 con 0x7f9324142470 2024-01-15T18:22:26.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f930c0c6390 con 0x7f9324142470 2024-01-15T18:22:26.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f931aa00640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92fc067ca0 0x7f92fc06a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f930c11dc50 con 0x7f9324142470 2024-01-15T18:22:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.140+0000 7f9322800640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92fc067ca0 0x7f92fc06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:26.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.144+0000 7f9322800640 1 --2- 172.21.15.143:0/3129047278 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f92fc067ca0 0x7f92fc06a160 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f931400b9d0 tx=0x7f9314008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:26.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.144+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f930c0ec8e0 con 0x7f9324142470 2024-01-15T18:22:26.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.324+0000 7f9329400640 1 -- 172.21.15.143:0/3129047278 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93240ccfd0 con 0x7f92fc067ca0 2024-01-15T18:22:26.321 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:26.328+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:26.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:26.328+0000 7f931aa00640 1 -- 172.21.15.143:0/3129047278 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f931400b5c0 con 0x7f92fc067ca0 2024-01-15T18:22:26.321 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:26.321 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:26.330 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:28.331 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.492+0000 7f21b4200640 1 Processor -- start 2024-01-15T18:22:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.492+0000 7f21b4200640 1 -- start start 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b4200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21ac0cf400 0x7f21ac0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b4200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce390 0x7f21ac0b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b4200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac15b5b0 con 0x7f21ac0cda70 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b4200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0cba70 con 0x7f21ac0cf400 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b4200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0cbd10 con 0x7f21ac0ce390 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21ac0cf400 0x7f21ac0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21ac0cf400 0x7f21ac0cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59336/0 (socket says 172.21.15.143:59336) 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 -- 172.21.15.143:0/3130459953 learned_addr learned my addr 172.21.15.143:0/3130459953 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b2200640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce390 0x7f21ac0b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 -- 172.21.15.143:0/3130459953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce390 msgr2=0x7f21ac0b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce390 0x7f21ac0b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 -- 172.21.15.143:0/3130459953 >> v1:172.21.15.204:6789/0 conn(0x7f21ac0cda70 legacy=0x7f21ac0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b0e00640 1 -- 172.21.15.143:0/3130459953 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0cbff0 con 0x7f21ac0cf400 2024-01-15T18:22:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.496+0000 7f21b2200640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce390 0x7f21ac0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:22:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b0e00640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0cf400 0x7f21ac0cde50 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e94ac700b5d8dd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21abe00640 1 -- 172.21.15.143:0/3130459953 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940aa610 con 0x7f21ac0cf400 2024-01-15T18:22:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21abe00640 1 -- 172.21.15.143:0/3130459953 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21940a6040 con 0x7f21ac0cf400 2024-01-15T18:22:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21abe00640 1 -- 172.21.15.143:0/3130459953 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940aa990 con 0x7f21ac0cf400 2024-01-15T18:22:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 -- 172.21.15.143:0/3130459953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0cf400 msgr2=0x7f21ac0cde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:28.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0cf400 0x7f21ac0cde50 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f2194009870 tx=0x7f219401c4d0 comp rx=0 tx=0).stop 2024-01-15T18:22:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 -- 172.21.15.143:0/3130459953 shutdown_connections 2024-01-15T18:22:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0ce390 0x7f21ac0b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 --2- 172.21.15.143:0/3130459953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0cf400 0x7f21ac0cde50 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 -- 172.21.15.143:0/3130459953 >> 172.21.15.143:0/3130459953 conn(0x7f21ac0d0e20 msgr2=0x7f21ac0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 -- 172.21.15.143:0/3130459953 shutdown_connections 2024-01-15T18:22:28.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 -- 172.21.15.143:0/3130459953 wait complete. 2024-01-15T18:22:28.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 Processor -- start 2024-01-15T18:22:28.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.500+0000 7f21b4200640 1 -- start start 2024-01-15T18:22:28.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0cda70 0x7f21ac14f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:28.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0ce390 0x7f21ac14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:28.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac0cf400 0x7f21ac150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0af9f0 con 0x7f21ac0ce390 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac152fe0 con 0x7f21ac0cf400 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0af430 con 0x7f21ac0cda70 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0ce390 0x7f21ac14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0ce390 0x7f21ac14fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59352/0 (socket says 172.21.15.143:59352) 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 -- 172.21.15.143:0/1983640583 learned_addr learned my addr 172.21.15.143:0/1983640583 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b2200640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac0cf400 0x7f21ac150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b1800640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0cda70 0x7f21ac14f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 -- 172.21.15.143:0/1983640583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0cda70 msgr2=0x7f21ac14f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f21ac0cda70 0x7f21ac14f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 -- 172.21.15.143:0/1983640583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac0cf400 msgr2=0x7f21ac150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f21ac0cf400 0x7f21ac150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:28.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 -- 172.21.15.143:0/1983640583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940094d0 con 0x7f21ac0ce390 2024-01-15T18:22:28.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b0e00640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21ac0ce390 0x7f21ac14fed0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f21940ac040 tx=0x7f219401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:28.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940bd5f0 con 0x7f21ac0ce390 2024-01-15T18:22:28.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21940bd770 con 0x7f21ac0ce390 2024-01-15T18:22:28.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 -- 172.21.15.143:0/1983640583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac0af710 con 0x7f21ac0ce390 2024-01-15T18:22:28.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940bdb20 con 0x7f21ac0ce390 2024-01-15T18:22:28.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 -- 172.21.15.143:0/1983640583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac06b990 con 0x7f21ac0ce390 2024-01-15T18:22:28.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.504+0000 7f21b4200640 1 -- 172.21.15.143:0/1983640583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac067500 con 0x7f21ac0ce390 2024-01-15T18:22:28.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.508+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f21940b33f0 con 0x7f21ac0ce390 2024-01-15T18:22:28.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.508+0000 7f21a9600640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067cd0 0x7f218406a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:28.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.508+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f219411ddb0 con 0x7f21ac0ce390 2024-01-15T18:22:28.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.508+0000 7f21b1800640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067cd0 0x7f218406a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:28.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.508+0000 7f21b1800640 1 --2- 172.21.15.143:0/1983640583 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2184067cd0 0x7f218406a190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f21ac165790 tx=0x7f219c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:28.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.512+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f21940ecaf0 con 0x7f21ac0ce390 2024-01-15T18:22:28.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.692+0000 7f21b4200640 1 -- 172.21.15.143:0/1983640583 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21ac0d1ca0 con 0x7f2184067cd0 2024-01-15T18:22:28.688 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:28.696+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:28.696+0000 7f21a9600640 1 -- 172.21.15.143:0/1983640583 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f219c009550 con 0x7f2184067cd0 2024-01-15T18:22:28.689 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:28.697 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:30.478 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:22:30.482 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:22:30.698 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:30.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 Processor -- start 2024-01-15T18:22:30.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 -- start start 2024-01-15T18:22:30.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:30.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f04cc0cbf50 0x7f04cc0cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:30.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04cc0cffd0 con 0x7f04cc1607c0 2024-01-15T18:22:30.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04cc0d1ce0 con 0x7f04cc0cbf50 2024-01-15T18:22:30.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04d0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0d02b0 con 0x7f04cc0cfb30 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f04cc0cbf50 0x7f04cc0cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04c9400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f04cc0cbf50 0x7f04cc0cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:49708/0 (socket says 172.21.15.143:49708) 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 -- 172.21.15.143:0/121490190 learned_addr learned my addr 172.21.15.143:0/121490190 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04c9e00640 1 --1- 172.21.15.143:0/121490190 >> v1:172.21.15.204:6789/0 conn(0x7f04cc1607c0 0x7f04cc06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58902/0 (socket says 172.21.15.143:58902) 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 -- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 msgr2=0x7f04cc06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 --2- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:30.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 -- 172.21.15.143:0/121490190 >> v1:172.21.15.204:6789/0 conn(0x7f04cc1607c0 legacy=0x7f04cc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:30.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.860+0000 7f04ca800640 1 -- 172.21.15.143:0/121490190 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc0cb470 con 0x7f04cc0cbf50 2024-01-15T18:22:30.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04ca800640 1 --2- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 0x7f04cc0cc390 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009e50 tx=0x7f04c401f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38fa5fd0979cb38 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:30.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04c8a00640 1 -- 172.21.15.143:0/121490190 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40af400 con 0x7f04cc0cbf50 2024-01-15T18:22:30.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04c8a00640 1 -- 172.21.15.143:0/121490190 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c40ab050 con 0x7f04cc0cbf50 2024-01-15T18:22:30.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04c8a00640 1 -- 172.21.15.143:0/121490190 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40af780 con 0x7f04cc0cbf50 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04d0e00640 1 -- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 msgr2=0x7f04cc0cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04d0e00640 1 --2- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 0x7f04cc0cc390 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009e50 tx=0x7f04c401f1f0 comp rx=0 tx=0).stop 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04d0e00640 1 -- 172.21.15.143:0/121490190 shutdown_connections 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04d0e00640 1 --2- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 0x7f04cc0cc390 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04d0e00640 1 --2- 172.21.15.143:0/121490190 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc06bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.864+0000 7f04d0e00640 1 -- 172.21.15.143:0/121490190 >> 172.21.15.143:0/121490190 conn(0x7f04cc0d0e20 msgr2=0x7f04cc0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:30.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 -- 172.21.15.143:0/121490190 shutdown_connections 2024-01-15T18:22:30.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 -- 172.21.15.143:0/121490190 wait complete. 2024-01-15T18:22:30.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 Processor -- start 2024-01-15T18:22:30.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 -- start start 2024-01-15T18:22:30.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 0x7f04cc0c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:30.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc0c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04cc1607c0 0x7f04cc0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0c5260 con 0x7f04cc1607c0 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0c5540 con 0x7f04cc0cbf50 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04d0e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc0c5820 con 0x7f04cc0cfb30 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04cc1607c0 0x7f04cc0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04cc1607c0 0x7f04cc0c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59362/0 (socket says 172.21.15.143:59362) 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 -- 172.21.15.143:0/1245564781 learned_addr learned my addr 172.21.15.143:0/1245564781 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04c9e00640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 0x7f04cc0c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04c9400640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc0c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 -- 172.21.15.143:0/1245564781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 msgr2=0x7f04cc0c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:30.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f04cc0cfb30 0x7f04cc0c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 -- 172.21.15.143:0/1245564781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 msgr2=0x7f04cc0c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04cc0cbf50 0x7f04cc0c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 -- 172.21.15.143:0/1245564781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4009b30 con 0x7f04cc1607c0 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.868+0000 7f04ca800640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f04cc1607c0 0x7f04cc0c9cc0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f04c401ce20 tx=0x7f04c401bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40afd30 con 0x7f04cc1607c0 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c40afe90 con 0x7f04cc1607c0 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c40ba660 con 0x7f04cc1607c0 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04d0e00640 1 -- 172.21.15.143:0/1245564781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc0c5b00 con 0x7f04cc1607c0 2024-01-15T18:22:30.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04d0e00640 1 -- 172.21.15.143:0/1245564781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc0c6300 con 0x7f04cc1607c0 2024-01-15T18:22:30.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04d0e00640 1 -- 172.21.15.143:0/1245564781 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc06d4b0 con 0x7f04cc1607c0 2024-01-15T18:22:30.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f04c40b6040 con 0x7f04cc1607c0 2024-01-15T18:22:30.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c2000640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04a4067ca0 0x7f04a406a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:30.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f04c4121650 con 0x7f04cc1607c0 2024-01-15T18:22:30.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.872+0000 7f04c9e00640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04a4067ca0 0x7f04a406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:30.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.876+0000 7f04c9e00640 1 --2- 172.21.15.143:0/1245564781 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f04a4067ca0 0x7f04a406a160 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000f70 tx=0x7f04bc007c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:30.869 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:30.876+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f04c40efff0 con 0x7f04cc1607c0 2024-01-15T18:22:31.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:31.056+0000 7f04d0e00640 1 -- 172.21.15.143:0/1245564781 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04cc0c56b0 con 0x7f04a4067ca0 2024-01-15T18:22:31.055 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:31.064+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:31.055 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:31.064+0000 7f04c2000640 1 -- 172.21.15.143:0/1245564781 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f04cc0c56b0 con 0x7f04a4067ca0 2024-01-15T18:22:31.055 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:31.056 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:31.063 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:33.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:33.215 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.224+0000 7f4faac00640 1 Processor -- start 2024-01-15T18:22:33.216 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.224+0000 7f4faac00640 1 -- start start 2024-01-15T18:22:33.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.224+0000 7f4faac00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f4fa4071bb0 0x7f4fa4072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.217 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.224+0000 7f4faac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4fa415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4faac00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa40ceb50 con 0x7f4fa4143780 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4faac00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa4072570 con 0x7f4fa4071bb0 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4faac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415c110 con 0x7f4fa415c5a0 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4fa4143780 0x7f4fa40cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:35326/0 (socket says 172.21.15.143:35326) 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 -- 172.21.15.143:0/1431466243 learned_addr learned my addr 172.21.15.143:0/1431466243 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3400640 1 --2- 172.21.15.143:0/1431466243 >> v2:172.21.15.204:3300/0 conn(0x7f4fa4071bb0 0x7f4fa4072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.218 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa8c00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4fa415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869619536 0 0) 0x7f4f94001120 con 0x7f4fa4143780 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f84003530 con 0x7f4fa4143780 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2182618155 0 0) 0x7f4f84003530 con 0x7f4fa4143780 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 msgr2=0x7f4fa415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4fa415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 >> v2:172.21.15.204:3300/0 conn(0x7f4fa4071bb0 msgr2=0x7f4fa4072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 --2- 172.21.15.143:0/1431466243 >> v2:172.21.15.204:3300/0 conn(0x7f4fa4071bb0 0x7f4fa4072030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa415b7e0 con 0x7f4fa4143780 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f4f94001830 con 0x7f4fa4143780 2024-01-15T18:22:33.220 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa8c00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4fa415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 >> v1:172.21.15.143:6789/0 conn(0x7f4fa4143780 legacy=0x7f4fa40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f84003980 0x7f4f84003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4f840047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f84004d10 0x7f4f84005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa4072570 con 0x7f4f84003980 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa415c110 con 0x7f4f84004d10 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8400b990 con 0x7f4fa415c5a0 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3400640 1 --2- 172.21.15.143:0/1431466243 >> v2:172.21.15.204:3300/0 conn(0x7f4fa4071bb0 0x7f4fa4072030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f84003980 0x7f4f84003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa8c00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f84004d10 0x7f4f84005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3400640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4f840047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 -- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 msgr2=0x7f4f840047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4f840047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:33.221 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 -- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f84004d10 msgr2=0x7f4f84005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4f84004d10 0x7f4f84005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3e00640 1 -- 172.21.15.143:0/1431466243 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c009a70 con 0x7f4f84003980 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.228+0000 7f4fa3400640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4fa415c5a0 0x7f4f840047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f84003980 0x7f4f84003da0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400c0e0 tx=0x7f4f9401cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbc15c6b7853bce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940058e0 con 0x7f4f84003980 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94005a60 con 0x7f4f84003980 2024-01-15T18:22:33.222 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4fa2a00640 1 -- 172.21.15.143:0/1431466243 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94005e10 con 0x7f4f84003980 2024-01-15T18:22:33.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4faac00640 1 -- 172.21.15.143:0/1431466243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f84003980 msgr2=0x7f4f84003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:33.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4faac00640 1 --2- 172.21.15.143:0/1431466243 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4f84003980 0x7f4f84003da0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400c0e0 tx=0x7f4f9401cb00 comp rx=0 tx=0).stop 2024-01-15T18:22:33.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4fa8c00640 1 -- 172.21.15.143:0/1431466243 reap_dead start 2024-01-15T18:22:33.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4faac00640 1 -- 172.21.15.143:0/1431466243 shutdown_connections 2024-01-15T18:22:33.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4faac00640 1 -- 172.21.15.143:0/1431466243 >> 172.21.15.143:0/1431466243 conn(0x7f4fa406c3f0 msgr2=0x7f4fa406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:33.223 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4faac00640 1 -- 172.21.15.143:0/1431466243 shutdown_connections 2024-01-15T18:22:33.224 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.232+0000 7f4faac00640 1 -- 172.21.15.143:0/1431466243 wait complete. 2024-01-15T18:22:33.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 Processor -- start 2024-01-15T18:22:33.225 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- start start 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f84003980 0x7f4fa41f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa413f260 0x7f4fa4159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fa41f5a30 0x7f4fa4070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa41f9510 con 0x7f4fa41f5a30 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa41f97e0 con 0x7f4fa413f260 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa41f9ac0 con 0x7f4f84003980 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fa41f5a30 0x7f4fa4070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fa41f5a30 0x7f4fa4070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59376/0 (socket says 172.21.15.143:59376) 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 -- 172.21.15.143:0/2094445359 learned_addr learned my addr 172.21.15.143:0/2094445359 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f84003980 0x7f4fa41f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.228 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa3400640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa413f260 0x7f4fa4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 -- 172.21.15.143:0/2094445359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f84003980 msgr2=0x7f4fa41f8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4f84003980 0x7f4fa41f8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 -- 172.21.15.143:0/2094445359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa413f260 msgr2=0x7f4fa4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fa413f260 0x7f4fa4159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 -- 172.21.15.143:0/2094445359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa413a360 con 0x7f4fa41f5a30 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4fa8c00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4fa41f5a30 0x7f4fa4070aa0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40b46a0 tx=0x7f4f9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c00e040 con 0x7f4fa41f5a30 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f9c00f040 con 0x7f4fa41f5a30 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- 172.21.15.143:0/2094445359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa413a640 con 0x7f4fa41f5a30 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c003be0 con 0x7f4fa41f5a30 2024-01-15T18:22:33.229 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- 172.21.15.143:0/2094445359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa41f9da0 con 0x7f4fa41f5a30 2024-01-15T18:22:33.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.236+0000 7f4faac00640 1 -- 172.21.15.143:0/2094445359 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa413a830 con 0x7f4fa41f5a30 2024-01-15T18:22:33.230 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.240+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f4f9c015430 con 0x7f4fa41f5a30 2024-01-15T18:22:33.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.240+0000 7f4f9be00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f7c067cd0 0x7f4f7c06a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:33.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.240+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f4f9c083c90 con 0x7f4fa41f5a30 2024-01-15T18:22:33.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.240+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f7c067cd0 0x7f4f7c06a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:33.231 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.240+0000 7f4fa3e00640 1 --2- 172.21.15.143:0/2094445359 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4f7c067cd0 0x7f4f7c06a190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4f940aa000 tx=0x7f4f94001af0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:33.235 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.244+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4f9c052600 con 0x7f4fa41f5a30 2024-01-15T18:22:33.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.420+0000 7f4faac00640 1 -- 172.21.15.143:0/2094445359 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4fa415d1c0 con 0x7f4f7c067cd0 2024-01-15T18:22:33.418 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:33.428+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:33.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:33.428+0000 7f4f9be00640 1 -- 172.21.15.143:0/2094445359 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4f94005bd0 con 0x7f4f7c067cd0 2024-01-15T18:22:33.419 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:33.419 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:33.427 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:35.429 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:35.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 Processor -- start 2024-01-15T18:22:35.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 -- start start 2024-01-15T18:22:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb4143780 0x7f8bb40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8bb40ce520 0x7f8bb4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:35.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40ceb50 con 0x7f8bb40720f0 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb4072580 con 0x7f8bb40ce520 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bbba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415c110 con 0x7f8bb4143780 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8bb40ce520 0x7f8bb4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb9000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb4143780 0x7f8bb40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8bb40ce520 0x7f8bb4071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:49756/0 (socket says 172.21.15.143:49756) 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb9a00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f8bb40720f0 0x7f8bb415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58908/0 (socket says 172.21.15.143:58908) 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 -- 172.21.15.143:0/1330369025 learned_addr learned my addr 172.21.15.143:0/1330369025 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:35.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 -- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb4143780 msgr2=0x7f8bb40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 --2- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb4143780 0x7f8bb40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 -- 172.21.15.143:0/1330369025 >> v1:172.21.15.204:6789/0 conn(0x7f8bb40720f0 legacy=0x7f8bb415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:35.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.592+0000 7f8bb3e00640 1 -- 172.21.15.143:0/1330369025 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb415b7e0 con 0x7f8bb40ce520 2024-01-15T18:22:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.596+0000 7f8bb3e00640 1 --2- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 0x7f8bb4071bb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009870 tx=0x7f8b9c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=820e055ac4ce334 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.596+0000 7f8bb3400640 1 -- 172.21.15.143:0/1330369025 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0ab650 con 0x7f8bb40ce520 2024-01-15T18:22:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.596+0000 7f8bb3400640 1 -- 172.21.15.143:0/1330369025 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c0ab7b0 con 0x7f8bb40ce520 2024-01-15T18:22:35.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.596+0000 7f8bb3400640 1 -- 172.21.15.143:0/1330369025 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0abb40 con 0x7f8bb40ce520 2024-01-15T18:22:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.596+0000 7f8bbba00640 1 -- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 msgr2=0x7f8bb4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:35.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.596+0000 7f8bbba00640 1 --2- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 0x7f8bb4071bb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009870 tx=0x7f8b9c01c540 comp rx=0 tx=0).stop 2024-01-15T18:22:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- 172.21.15.143:0/1330369025 shutdown_connections 2024-01-15T18:22:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 --2- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 0x7f8bb4071bb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 --2- 172.21.15.143:0/1330369025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb4143780 0x7f8bb40cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- 172.21.15.143:0/1330369025 >> 172.21.15.143:0/1330369025 conn(0x7f8bb406c3f0 msgr2=0x7f8bb406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- 172.21.15.143:0/1330369025 shutdown_connections 2024-01-15T18:22:35.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- 172.21.15.143:0/1330369025 wait complete. 2024-01-15T18:22:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 Processor -- start 2024-01-15T18:22:35.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- start start 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb40720f0 0x7f8bb4155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 0x7f8bb4141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb4143780 0x7f8bb41421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41370f0 con 0x7f8bb4143780 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4155770 con 0x7f8bb40ce520 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bbba00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4156870 con 0x7f8bb40720f0 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb4143780 0x7f8bb41421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb4143780 0x7f8bb41421f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59390/0 (socket says 172.21.15.143:59390) 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 -- 172.21.15.143:0/2566119927 learned_addr learned my addr 172.21.15.143:0/2566119927 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb3e00640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 0x7f8bb4141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9000640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb40720f0 0x7f8bb4155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:35.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 -- 172.21.15.143:0/2566119927 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb40720f0 msgr2=0x7f8bb4155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8bb40720f0 0x7f8bb4155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 -- 172.21.15.143:0/2566119927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 msgr2=0x7f8bb4141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8bb40ce520 0x7f8bb4141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.600+0000 7f8bb9a00640 1 -- 172.21.15.143:0/2566119927 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c0094d0 con 0x7f8bb4143780 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb9a00640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb4143780 0x7f8bb41421f0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c760 tx=0x7f8bac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00ce70 con 0x7f8bb4143780 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00a640 con 0x7f8bb4143780 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00a9b0 con 0x7f8bb4143780 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bbba00640 1 -- 172.21.15.143:0/2566119927 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb4156b50 con 0x7f8bb4143780 2024-01-15T18:22:35.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bbba00640 1 -- 172.21.15.143:0/2566119927 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb4137620 con 0x7f8bb4143780 2024-01-15T18:22:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bbba00640 1 -- 172.21.15.143:0/2566119927 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb4137810 con 0x7f8bb4143780 2024-01-15T18:22:35.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7f8bac00ab10 con 0x7f8bb4143780 2024-01-15T18:22:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb0c00640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b8c067ca0 0x7f8b8c06a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8bac082a50 con 0x7f8bb4143780 2024-01-15T18:22:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.604+0000 7f8bb9000640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b8c067ca0 0x7f8b8c06a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:35.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.608+0000 7f8bb9000640 1 --2- 172.21.15.143:0/2566119927 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b8c067ca0 0x7f8b8c06a160 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009ac0 tx=0x7f8ba4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:35.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.608+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8bac051360 con 0x7f8bb4143780 2024-01-15T18:22:35.782 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.792+0000 7f8bbba00640 1 -- 172.21.15.143:0/2566119927 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bb40ceb50 con 0x7f8b8c067ca0 2024-01-15T18:22:35.788 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:35.796+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:35.788 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:35.796+0000 7f8bb0c00640 1 -- 172.21.15.143:0/2566119927 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8bb40ceb50 con 0x7f8b8c067ca0 2024-01-15T18:22:35.789 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:35.789 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:35.797 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:37.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:37.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 Processor -- start 2024-01-15T18:22:37.954 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 -- start start 2024-01-15T18:22:37.955 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9580cfb30 0x7fb95806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9580cbf50 0x7fb9580cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9580cffd0 con 0x7fb9581607c0 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580d1ce0 con 0x7fb9580cfb30 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb95ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580d02b0 con 0x7fb9580cbf50 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb956800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9580cbf50 0x7fb9580cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9580cfb30 0x7fb95806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb9580cfb30 0x7fb95806bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:37820/0 (socket says 172.21.15.143:37820) 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 -- 172.21.15.143:0/4248247021 learned_addr learned my addr 172.21.15.143:0/4248247021 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb956800640 1 --2- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9580cbf50 0x7fb9580cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:33826/0 (socket says 172.21.15.143:33826) 2024-01-15T18:22:37.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 -- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9580cbf50 msgr2=0x7fb9580cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:37.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 --2- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9580cbf50 0x7fb9580cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:37.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 -- 172.21.15.143:0/4248247021 >> v1:172.21.15.204:6789/0 conn(0x7fb9581607c0 legacy=0x7fb95806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:37.957 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.963+0000 7fb955400640 1 -- 172.21.15.143:0/4248247021 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580cb470 con 0x7fb9580cfb30 2024-01-15T18:22:37.958 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb955400640 1 --2- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cfb30 0x7fb95806bec0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb940009870 tx=0x7fb94001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=806b3fb6009cdbe1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:37.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb954a00640 1 -- 172.21.15.143:0/4248247021 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400a8a30 con 0x7fb9580cfb30 2024-01-15T18:22:37.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb954a00640 1 -- 172.21.15.143:0/4248247021 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9400a8b90 con 0x7fb9580cfb30 2024-01-15T18:22:37.959 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb954a00640 1 -- 172.21.15.143:0/4248247021 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400b0440 con 0x7fb9580cfb30 2024-01-15T18:22:37.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 -- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cfb30 msgr2=0x7fb95806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:37.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 --2- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cfb30 0x7fb95806bec0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb940009870 tx=0x7fb94001ed30 comp rx=0 tx=0).stop 2024-01-15T18:22:37.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 -- 172.21.15.143:0/4248247021 shutdown_connections 2024-01-15T18:22:37.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 --2- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9580cbf50 0x7fb9580cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:37.960 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 --2- 172.21.15.143:0/4248247021 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cfb30 0x7fb95806bec0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:37.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 -- 172.21.15.143:0/4248247021 >> 172.21.15.143:0/4248247021 conn(0x7fb9580d0e20 msgr2=0x7fb9580d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:37.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 -- 172.21.15.143:0/4248247021 shutdown_connections 2024-01-15T18:22:37.961 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.967+0000 7fb95ce00640 1 -- 172.21.15.143:0/4248247021 wait complete. 2024-01-15T18:22:37.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 Processor -- start 2024-01-15T18:22:37.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 -- start start 2024-01-15T18:22:37.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cbf50 0x7fb958157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9580cfb30 0x7fb9580b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9581607c0 0x7fb9580cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580adb40 con 0x7fb9580cfb30 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581f3280 con 0x7fb9580cbf50 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580ca130 con 0x7fb9581607c0 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9580cfb30 0x7fb9580b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9580cfb30 0x7fb9580b6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38278/0 (socket says 172.21.15.143:38278) 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 -- 172.21.15.143:0/3320321404 learned_addr learned my addr 172.21.15.143:0/3320321404 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955e00640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cbf50 0x7fb958157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb956800640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9581607c0 0x7fb9580cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:37.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 -- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9581607c0 msgr2=0x7fb9580cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb9581607c0 0x7fb9580cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 -- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cbf50 msgr2=0x7fb958157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cbf50 0x7fb958157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 -- 172.21.15.143:0/3320321404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400094d0 con 0x7fb9580cfb30 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955e00640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9580cbf50 0x7fb958157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb955400640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9580cfb30 0x7fb9580b6520 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fb9400b3040 tx=0x7fb94001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400b0dd0 con 0x7fb9580cfb30 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9400bb460 con 0x7fb9580cfb30 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 -- 172.21.15.143:0/3320321404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9580ca410 con 0x7fb9580cfb30 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400bb810 con 0x7fb9580cfb30 2024-01-15T18:22:37.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.971+0000 7fb95ce00640 1 -- 172.21.15.143:0/3320321404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9580ca9d0 con 0x7fb9580cfb30 2024-01-15T18:22:37.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.975+0000 7fb95ce00640 1 -- 172.21.15.143:0/3320321404 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95806d4b0 con 0x7fb9580cfb30 2024-01-15T18:22:37.967 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.975+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 43) v1 ==== 87945+0+0 (secure 0 0 0) 0x7fb9400b83f0 con 0x7fb9580cfb30 2024-01-15T18:22:37.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.975+0000 7fb94e000640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb930067ca0 0x7fb93006a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:37.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.975+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fb940120de0 con 0x7fb9580cfb30 2024-01-15T18:22:37.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.975+0000 7fb955e00640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb930067ca0 0x7fb93006a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:37.968 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.975+0000 7fb955e00640 1 --2- 172.21.15.143:0/3320321404 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb930067ca0 0x7fb93006a160 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb9580d1970 tx=0x7fb948009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:37.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:37.979+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb9400ef770 con 0x7fb9580cfb30 2024-01-15T18:22:38.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:38.159+0000 7fb95ce00640 1 -- 172.21.15.143:0/3320321404 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9580ce6b0 con 0x7fb930067ca0 2024-01-15T18:22:38.155 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:38.163+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:38.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:38.163+0000 7fb94e000640 1 -- 172.21.15.143:0/3320321404 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb948001860 con 0x7fb930067ca0 2024-01-15T18:22:38.156 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:38.156 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:38.165 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:40.167 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:40.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.323+0000 7f9e61a00640 1 Processor -- start 2024-01-15T18:22:40.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.323+0000 7f9e61a00640 1 -- start start 2024-01-15T18:22:40.317 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e61a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cfb30 0x7f9e5c06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:40.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e61a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c0cffd0 con 0x7f9e5c1607c0 2024-01-15T18:22:40.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e61a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c0d02b0 con 0x7f9e5c0cbf50 2024-01-15T18:22:40.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e61a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0d1ce0 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e5aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9e5c1607c0 0x7f9e5c06b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38280/0 (socket says 172.21.15.143:38280) 2024-01-15T18:22:40.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e5aa00640 1 -- 172.21.15.143:0/385319747 learned_addr learned my addr 172.21.15.143:0/385319747 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:40.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e5a000640 1 --2- 172.21.15.143:0/385319747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cfb30 0x7f9e5c06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:40.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053330559 0 0) 0x7f9e5c0cffd0 con 0x7f9e5c1607c0 2024-01-15T18:22:40.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e3c003530 con 0x7f9e5c1607c0 2024-01-15T18:22:40.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720382589 0 0) 0x7f9e5c0d02b0 con 0x7f9e5c0cbf50 2024-01-15T18:22:40.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e5c0cffd0 con 0x7f9e5c0cbf50 2024-01-15T18:22:40.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2816240762 0 0) 0x7f9e3c003530 con 0x7f9e5c1607c0 2024-01-15T18:22:40.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e5a000640 1 -- 172.21.15.143:0/385319747 >> v1:172.21.15.204:6789/0 conn(0x7f9e5c0cbf50 legacy=0x7f9e5c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:40.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e5a000640 1 -- 172.21.15.143:0/385319747 >> v1:172.21.15.143:6789/0 conn(0x7f9e5c1607c0 legacy=0x7f9e5c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:40.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.327+0000 7f9e5a000640 1 -- 172.21.15.143:0/385319747 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c0cb470 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e5a000640 1 --2- 172.21.15.143:0/385319747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cfb30 0x7f9e5c06bec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097f0 tx=0x7f9e4401ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa0d19af7bf20ddb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:40.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c001680 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e440a8650 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.322 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e59600640 1 -- 172.21.15.143:0/385319747 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440a89e0 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 -- 172.21.15.143:0/385319747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cfb30 msgr2=0x7f9e5c06bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 --2- 172.21.15.143:0/385319747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cfb30 0x7f9e5c06bec0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097f0 tx=0x7f9e4401ebd0 comp rx=0 tx=0).stop 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 -- 172.21.15.143:0/385319747 shutdown_connections 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 --2- 172.21.15.143:0/385319747 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cfb30 0x7f9e5c06bec0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 -- 172.21.15.143:0/385319747 >> 172.21.15.143:0/385319747 conn(0x7f9e5c0d0e20 msgr2=0x7f9e5c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 -- 172.21.15.143:0/385319747 shutdown_connections 2024-01-15T18:22:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 -- 172.21.15.143:0/385319747 wait complete. 2024-01-15T18:22:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 Processor -- start 2024-01-15T18:22:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.331+0000 7f9e61a00640 1 -- start start 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cbf50 0x7f9e5c06f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e5c0cfb30 0x7f9e5c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5c1607c0 0x7f9e5c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1f6e40 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0b4240 con 0x7f9e5c1607c0 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0c95a0 con 0x7f9e5c0cbf50 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e5c0cfb30 0x7f9e5c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e5c0cfb30 0x7f9e5c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38290/0 (socket says 172.21.15.143:38290) 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 -- 172.21.15.143:0/3869802208 learned_addr learned my addr 172.21.15.143:0/3869802208 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5aa00640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cbf50 0x7f9e5c06f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5b400640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5c1607c0 0x7f9e5c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 -- 172.21.15.143:0/3869802208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cbf50 msgr2=0x7f9e5c06f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9e5c0cbf50 0x7f9e5c06f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 -- 172.21.15.143:0/3869802208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5c1607c0 msgr2=0x7f9e5c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e5c1607c0 0x7f9e5c15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 -- 172.21.15.143:0/3869802208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44009420 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e5a000640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9e5c0cfb30 0x7f9e5c15b540 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097f0 tx=0x7f9e4401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440b0850 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e440b09d0 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 -- 172.21.15.143:0/3869802208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c1f6080 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e440b0d80 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 -- 172.21.15.143:0/3869802208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c1f6640 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.335+0000 7f9e61a00640 1 -- 172.21.15.143:0/3869802208 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c06d4b0 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.339+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 44) v1 ==== 87894+0+0 (secure 0 0 0) 0x7f9e440b7040 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.339+0000 7f9e52a00640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9e34067c10 0x7f9e3406a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:40.330 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.339+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f9e441208d0 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.339+0000 7f9e5aa00640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9e34067c10 0x7f9e3406a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.339+0000 7f9e5aa00640 1 --2- 172.21.15.143:0/3869802208 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9e34067c10 0x7f9e3406a0d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c001ec0 tx=0x7f9e4c00f5a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.343+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9e440ef2d0 con 0x7f9e5c0cfb30 2024-01-15T18:22:40.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.519+0000 7f9e61a00640 1 -- 172.21.15.143:0/3869802208 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e5c0d02d0 con 0x7f9e34067c10 2024-01-15T18:22:40.519 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:40.527+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:40.519 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:40.527+0000 7f9e52a00640 1 -- 172.21.15.143:0/3869802208 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9e5c0d02d0 con 0x7f9e34067c10 2024-01-15T18:22:40.519 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:40.519 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:40.528 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:42.530 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d4000640 1 Processor -- start 2024-01-15T18:22:42.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d4000640 1 -- start start 2024-01-15T18:22:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d4000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f25cc157300 0x7f25cc159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:42.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d4000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25cc0b2a20 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d4000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25cc0b2d00 con 0x7f25cc0b21d0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d4000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25cc0b35a0 con 0x7f25cc157300 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d2000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f25cc0b4ac0 0x7f25cc070e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38282/0 (socket says 172.21.15.143:38282) 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d2000640 1 -- 172.21.15.143:0/1976604249 learned_addr learned my addr 172.21.15.143:0/1976604249 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25d1600640 1 --2- 172.21.15.143:0/1976604249 >> v2:172.21.15.204:3300/0 conn(0x7f25cc157300 0x7f25cc159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.691+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483868923 0 0) 0x7f25cc0b2a20 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ac003530 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567317696 0 0) 0x7f25cc0b2d00 con 0x7f25cc0b21d0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25cc0b2a20 con 0x7f25cc0b21d0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061506706 0 0) 0x7f25ac003530 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25d1600640 1 -- 172.21.15.143:0/1976604249 >> v1:172.21.15.204:6789/0 conn(0x7f25cc0b21d0 legacy=0x7f25cc0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25d1600640 1 -- 172.21.15.143:0/1976604249 >> v1:172.21.15.143:6789/0 conn(0x7f25cc0b4ac0 legacy=0x7f25cc070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:42.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25d1600640 1 -- 172.21.15.143:0/1976604249 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0b2fe0 con 0x7f25cc157300 2024-01-15T18:22:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25d1600640 1 --2- 172.21.15.143:0/1976604249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc157300 0x7f25cc159b70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0097c0 tx=0x7f25bc01c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cf06945438384 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4001a00 con 0x7f25cc157300 2024-01-15T18:22:42.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc0ac440 con 0x7f25cc157300 2024-01-15T18:22:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.695+0000 7f25cbe00640 1 -- 172.21.15.143:0/1976604249 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc0ac7d0 con 0x7f25cc157300 2024-01-15T18:22:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- 172.21.15.143:0/1976604249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc157300 msgr2=0x7f25cc159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 --2- 172.21.15.143:0/1976604249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc157300 0x7f25cc159b70 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0097c0 tx=0x7f25bc01c370 comp rx=0 tx=0).stop 2024-01-15T18:22:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- 172.21.15.143:0/1976604249 shutdown_connections 2024-01-15T18:22:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 --2- 172.21.15.143:0/1976604249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc157300 0x7f25cc159b70 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:42.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- 172.21.15.143:0/1976604249 >> 172.21.15.143:0/1976604249 conn(0x7f25cc06c3f0 msgr2=0x7f25cc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- 172.21.15.143:0/1976604249 shutdown_connections 2024-01-15T18:22:42.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- 172.21.15.143:0/1976604249 wait complete. 2024-01-15T18:22:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 Processor -- start 2024-01-15T18:22:42.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- start start 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc0b21d0 0x7f25cc146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0b4ac0 0x7f25cc146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25cc157300 0x7f25cc14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc1468a0 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc154580 con 0x7f25cc0b21d0 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d4000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc154860 con 0x7f25cc157300 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0b4ac0 0x7f25cc146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d0c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0b4ac0 0x7f25cc146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38298/0 (socket says 172.21.15.143:38298) 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d0c00640 1 -- 172.21.15.143:0/324056824 learned_addr learned my addr 172.21.15.143:0/324056824 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d1600640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc0b21d0 0x7f25cc146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.699+0000 7f25d2000640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25cc157300 0x7f25cc14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d0c00640 1 -- 172.21.15.143:0/324056824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25cc157300 msgr2=0x7f25cc14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d0c00640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f25cc157300 0x7f25cc14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d0c00640 1 -- 172.21.15.143:0/324056824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc0b21d0 msgr2=0x7f25cc146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d0c00640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25cc0b21d0 0x7f25cc146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d0c00640 1 -- 172.21.15.143:0/324056824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009420 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d0c00640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25cc0b4ac0 0x7f25cc146360 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f25b400c500 tx=0x7f25b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40052d0 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4005450 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4012870 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d4000640 1 -- 172.21.15.143:0/324056824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc154b40 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d4000640 1 -- 172.21.15.143:0/324056824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc155100 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25d4000640 1 -- 172.21.15.143:0/324056824 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc0b48f0 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.703+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f25b40129d0 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.707+0000 7f25c9600640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f25a4067b80 0x7f25a406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.707+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f25b4082f90 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.707+0000 7f25d1600640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f25a4067b80 0x7f25a406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.707+0000 7f25d1600640 1 --2- 172.21.15.143:0/324056824 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f25a4067b80 0x7f25a406a040 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0ae000 tx=0x7f25bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.711+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f25b40518d0 con 0x7f25cc0b4ac0 2024-01-15T18:22:42.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.887+0000 7f25d4000640 1 -- 172.21.15.143:0/324056824 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25cc067500 con 0x7f25a4067b80 2024-01-15T18:22:42.886 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:42.895+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:42.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:42.895+0000 7f25c9600640 1 -- 172.21.15.143:0/324056824 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f25cc067500 con 0x7f25a4067b80 2024-01-15T18:22:42.887 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:42.887 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:42.896 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:44.897 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:45.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 Processor -- start 2024-01-15T18:22:45.054 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 -- start start 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f46b40ac060 0x7f46b40af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46b40adf10 0x7f46b40ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b40ad990 con 0x7f46b415d570 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40725b0 con 0x7f46b40adf10 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46bbc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b4072890 con 0x7f46b40ac060 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b9c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f46b40adf10 0x7f46b40ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:45.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b3e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f46b415d570 0x7f46b40af6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38288/0 (socket says 172.21.15.143:38288) 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b3e00640 1 -- 172.21.15.143:0/77593222 learned_addr learned my addr 172.21.15.143:0/77593222 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b9200640 1 --2- 172.21.15.143:0/77593222 >> v2:172.21.15.204:3300/0 conn(0x7f46b40ac060 0x7f46b40af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b9c00640 1 -- 172.21.15.143:0/77593222 >> v2:172.21.15.204:3300/0 conn(0x7f46b40ac060 msgr2=0x7f46b40af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b9c00640 1 --2- 172.21.15.143:0/77593222 >> v2:172.21.15.204:3300/0 conn(0x7f46b40ac060 0x7f46b40af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b9c00640 1 -- 172.21.15.143:0/77593222 >> v1:172.21.15.143:6789/0 conn(0x7f46b415d570 legacy=0x7f46b40af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b9c00640 1 -- 172.21.15.143:0/77593222 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b40b3f30 con 0x7f46b40adf10 2024-01-15T18:22:45.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.063+0000 7f46b3400640 1 -- 172.21.15.143:0/77593222 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917682287 0 0) 0x7f46b40ad990 con 0x7f46b415d570 2024-01-15T18:22:45.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46b9c00640 1 --2- 172.21.15.143:0/77593222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b40adf10 0x7f46b40ad450 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f46b40714a0 tx=0x7f46b40b0610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3d646faa2777b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:45.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46b3400640 1 -- 172.21.15.143:0/77593222 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0a59f0 con 0x7f46b40adf10 2024-01-15T18:22:45.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46b3400640 1 -- 172.21.15.143:0/77593222 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac0a5b50 con 0x7f46b40adf10 2024-01-15T18:22:45.059 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46b3400640 1 -- 172.21.15.143:0/77593222 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0ad440 con 0x7f46b40adf10 2024-01-15T18:22:45.060 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 -- 172.21.15.143:0/77593222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b40adf10 msgr2=0x7f46b40ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 --2- 172.21.15.143:0/77593222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b40adf10 0x7f46b40ad450 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f46b40714a0 tx=0x7f46b40b0610 comp rx=0 tx=0).stop 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 -- 172.21.15.143:0/77593222 shutdown_connections 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 --2- 172.21.15.143:0/77593222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b40adf10 0x7f46b40ad450 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 --2- 172.21.15.143:0/77593222 >> v2:172.21.15.204:3300/0 conn(0x7f46b40ac060 0x7f46b40af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 -- 172.21.15.143:0/77593222 >> 172.21.15.143:0/77593222 conn(0x7f46b4071a90 msgr2=0x7f46b4071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 -- 172.21.15.143:0/77593222 shutdown_connections 2024-01-15T18:22:45.061 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.067+0000 7f46bbc00640 1 -- 172.21.15.143:0/77593222 wait complete. 2024-01-15T18:22:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 Processor -- start 2024-01-15T18:22:45.062 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 -- start start 2024-01-15T18:22:45.064 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b40ac060 0x7f46b4154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46b40adf10 0x7f46b41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b413d4a0 0x7f46b4151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4154760 con 0x7f46b413d4a0 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b415c3f0 con 0x7f46b40ac060 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41f9d30 con 0x7f46b40adf10 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b413d4a0 0x7f46b4151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b413d4a0 0x7f46b4151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38314/0 (socket says 172.21.15.143:38314) 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 -- 172.21.15.143:0/4291209412 learned_addr learned my addr 172.21.15.143:0/4291209412 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b3e00640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46b40adf10 0x7f46b41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9200640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b40ac060 0x7f46b4154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:45.065 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 -- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46b40adf10 msgr2=0x7f46b41511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46b40adf10 0x7f46b41511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 -- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b40ac060 msgr2=0x7f46b4154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f46b40ac060 0x7f46b4154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 -- 172.21.15.143:0/4291209412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009a70 con 0x7f46b413d4a0 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b3e00640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f46b40adf10 0x7f46b41511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b9c00640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46b413d4a0 0x7f46b4151720 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f46ac01cdf0 tx=0x7f46ac0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0ade80 con 0x7f46b413d4a0 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac009450 con 0x7f46b413d4a0 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 -- 172.21.15.143:0/4291209412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b415c690 con 0x7f46b413d4a0 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003ba0 con 0x7f46b413d4a0 2024-01-15T18:22:45.066 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.071+0000 7f46bbc00640 1 -- 172.21.15.143:0/4291209412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b413db10 con 0x7f46b413d4a0 2024-01-15T18:22:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.075+0000 7f46bbc00640 1 -- 172.21.15.143:0/4291209412 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4067500 con 0x7f46b413d4a0 2024-01-15T18:22:45.067 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.075+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f46ac003d00 con 0x7f46b413d4a0 2024-01-15T18:22:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.075+0000 7f46b0c00640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f468c067b80 0x7f468c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.075+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f46ac11e3f0 con 0x7f46b413d4a0 2024-01-15T18:22:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.075+0000 7f46b9200640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f468c067b80 0x7f468c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:45.068 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.075+0000 7f46b9200640 1 --2- 172.21.15.143:0/4291209412 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f468c067b80 0x7f468c06a040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f46a4007a60 tx=0x7f46a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:45.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.079+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f46ac0ed1a0 con 0x7f46b413d4a0 2024-01-15T18:22:45.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.267+0000 7f46bbc00640 1 -- 172.21.15.143:0/4291209412 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46b415cca0 con 0x7f468c067b80 2024-01-15T18:22:45.264 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:45.271+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:45.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:45.271+0000 7f46b0c00640 1 -- 172.21.15.143:0/4291209412 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f46b415cca0 con 0x7f468c067b80 2024-01-15T18:22:45.265 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:45.265 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:45.273 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:47.275 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:47.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.435+0000 7f5890c00640 1 Processor -- start 2024-01-15T18:22:47.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.435+0000 7f5890c00640 1 -- start start 2024-01-15T18:22:47.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5890c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f588c0adf10 0x7f588c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:47.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5890c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c0725b0 con 0x7f588c15d570 2024-01-15T18:22:47.429 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5890c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c072890 con 0x7f588c0ac060 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5890c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c0ae2f0 con 0x7f588c0adf10 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f588a800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f588c0adf10 0x7f588c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5889400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f588c15d570 0x7f588c0ad450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36042/0 (socket says 172.21.15.143:36042) 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5889400640 1 -- 172.21.15.143:0/1085135295 learned_addr learned my addr 172.21.15.143:0/1085135295 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f588a800640 1 -- 172.21.15.143:0/1085135295 >> v1:172.21.15.204:6789/0 conn(0x7f588c0ac060 legacy=0x7f588c0af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f588a800640 1 -- 172.21.15.143:0/1085135295 >> v1:172.21.15.143:6789/0 conn(0x7f588c15d570 legacy=0x7f588c0ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f588a800640 1 -- 172.21.15.143:0/1085135295 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c0b3f30 con 0x7f588c0adf10 2024-01-15T18:22:47.430 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f5888a00640 1 -- 172.21.15.143:0/1085135295 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726091731 0 0) 0x7f588c0725b0 con 0x7f588c15d570 2024-01-15T18:22:47.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.439+0000 7f588a800640 1 --2- 172.21.15.143:0/1085135295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c0adf10 0x7f588c0af8c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5884009d90 tx=0x7f588c0b0610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3489273b5eb982e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:47.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5888a00640 1 -- 172.21.15.143:0/1085135295 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840a59c0 con 0x7f588c0adf10 2024-01-15T18:22:47.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5888a00640 1 -- 172.21.15.143:0/1085135295 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58840a5b20 con 0x7f588c0adf10 2024-01-15T18:22:47.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5888a00640 1 -- 172.21.15.143:0/1085135295 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840ad440 con 0x7f588c0adf10 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 -- 172.21.15.143:0/1085135295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c0adf10 msgr2=0x7f588c0af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 --2- 172.21.15.143:0/1085135295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c0adf10 0x7f588c0af8c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f5884009d90 tx=0x7f588c0b0610 comp rx=0 tx=0).stop 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 -- 172.21.15.143:0/1085135295 shutdown_connections 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 --2- 172.21.15.143:0/1085135295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c0adf10 0x7f588c0af8c0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 -- 172.21.15.143:0/1085135295 >> 172.21.15.143:0/1085135295 conn(0x7f588c071a90 msgr2=0x7f588c071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 -- 172.21.15.143:0/1085135295 shutdown_connections 2024-01-15T18:22:47.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 -- 172.21.15.143:0/1085135295 wait complete. 2024-01-15T18:22:47.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 Processor -- start 2024-01-15T18:22:47.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 -- start start 2024-01-15T18:22:47.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c0ac060 0x7f588c157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:47.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.443+0000 7f5890c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f588c0adf10 0x7f588c149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:47.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c14c100 0x7f588c14e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c150f20 con 0x7f588c14c100 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c158140 con 0x7f588c0ac060 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c15e5a0 con 0x7f588c0adf10 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c14c100 0x7f588c14e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c14c100 0x7f588c14e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54360/0 (socket says 172.21.15.143:54360) 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 -- 172.21.15.143:0/3538246051 learned_addr learned my addr 172.21.15.143:0/3538246051 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5889400640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f588c0adf10 0x7f588c149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5889e00640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c0ac060 0x7f588c157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 -- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f588c0adf10 msgr2=0x7f588c149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f588c0adf10 0x7f588c149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 -- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c0ac060 msgr2=0x7f588c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c0ac060 0x7f588c157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 -- 172.21.15.143:0/3538246051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884009a70 con 0x7f588c14c100 2024-01-15T18:22:47.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5889400640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f588c0adf10 0x7f588c149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5889e00640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f588c0ac060 0x7f588c157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f588a800640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f588c14c100 0x7f588c14e340 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f58840194f0 tx=0x7f5884019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840ade00 con 0x7f588c14c100 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58840b6400 con 0x7f588c14c100 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58840b67b0 con 0x7f588c14c100 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 -- 172.21.15.143:0/3538246051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c150960 con 0x7f588c14c100 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 -- 172.21.15.143:0/3538246051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c157360 con 0x7f588c14c100 2024-01-15T18:22:47.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5890c00640 1 -- 172.21.15.143:0/3538246051 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c15e370 con 0x7f588c14c100 2024-01-15T18:22:47.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.447+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f58840b4040 con 0x7f588c14c100 2024-01-15T18:22:47.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.451+0000 7f5882000640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5864067b80 0x7f586406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:47.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.451+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f588411e410 con 0x7f588c14c100 2024-01-15T18:22:47.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.451+0000 7f5889e00640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5864067b80 0x7f586406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:47.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.451+0000 7f5889e00640 1 --2- 172.21.15.143:0/3538246051 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f5864067b80 0x7f586406a040 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f587c0017e0 tx=0x7f587c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:47.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.455+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f588c15e370 con 0x7f588c14c100 2024-01-15T18:22:47.626 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.635+0000 7f5890c00640 1 -- 172.21.15.143:0/3538246051 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f588c15e370 con 0x7f5864067b80 2024-01-15T18:22:47.631 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:47.639+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:47.632 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:47.639+0000 7f5882000640 1 -- 172.21.15.143:0/3538246051 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f588c15e370 con 0x7f5864067b80 2024-01-15T18:22:47.632 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:47.632 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:47.640 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:49.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:49.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 Processor -- start 2024-01-15T18:22:49.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 -- start start 2024-01-15T18:22:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f05ec0ce560 0x7f05ec15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05ec071d00 0x7f05ec072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec072650 con 0x7f05ec141660 2024-01-15T18:22:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec15fbb0 con 0x7f05ec071d00 2024-01-15T18:22:49.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05f1c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec15f310 con 0x7f05ec0ce560 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eb400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f05ec071d00 0x7f05ec072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eaa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f05ec141660 0x7f05ec141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36054/0 (socket says 172.21.15.143:36054) 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eaa00640 1 -- 172.21.15.143:0/2158991964 learned_addr learned my addr 172.21.15.143:0/2158991964 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05ea000640 1 --2- 172.21.15.143:0/2158991964 >> v2:172.21.15.204:3300/0 conn(0x7f05ec0ce560 0x7f05ec15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eb400640 1 -- 172.21.15.143:0/2158991964 >> v2:172.21.15.204:3300/0 conn(0x7f05ec0ce560 msgr2=0x7f05ec15ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eb400640 1 --2- 172.21.15.143:0/2158991964 >> v2:172.21.15.204:3300/0 conn(0x7f05ec0ce560 0x7f05ec15ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eb400640 1 -- 172.21.15.143:0/2158991964 >> v1:172.21.15.143:6789/0 conn(0x7f05ec141660 legacy=0x7f05ec141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05eb400640 1 -- 172.21.15.143:0/2158991964 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec15f5f0 con 0x7f05ec071d00 2024-01-15T18:22:49.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.803+0000 7f05e9600640 1 -- 172.21.15.143:0/2158991964 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646473707 0 0) 0x7f05dc001120 con 0x7f05ec141660 2024-01-15T18:22:49.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05eb400640 1 --2- 172.21.15.143:0/2158991964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec072110 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f05ec06f940 tx=0x7f05e401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48508e662028ac5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:49.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05e9600640 1 -- 172.21.15.143:0/2158991964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a58c0 con 0x7f05ec071d00 2024-01-15T18:22:49.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05e9600640 1 -- 172.21.15.143:0/2158991964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40a5a20 con 0x7f05ec071d00 2024-01-15T18:22:49.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05e9600640 1 -- 172.21.15.143:0/2158991964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40a5db0 con 0x7f05ec071d00 2024-01-15T18:22:49.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05f1c00640 1 -- 172.21.15.143:0/2158991964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 msgr2=0x7f05ec072110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:49.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05f1c00640 1 --2- 172.21.15.143:0/2158991964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec072110 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f05ec06f940 tx=0x7f05e401c910 comp rx=0 tx=0).stop 2024-01-15T18:22:49.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05f1c00640 1 -- 172.21.15.143:0/2158991964 shutdown_connections 2024-01-15T18:22:49.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05f1c00640 1 --2- 172.21.15.143:0/2158991964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec072110 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:49.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05f1c00640 1 --2- 172.21.15.143:0/2158991964 >> v2:172.21.15.204:3300/0 conn(0x7f05ec0ce560 0x7f05ec15ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:49.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.807+0000 7f05f1c00640 1 -- 172.21.15.143:0/2158991964 >> 172.21.15.143:0/2158991964 conn(0x7f05ec06c3f0 msgr2=0x7f05ec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:49.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 -- 172.21.15.143:0/2158991964 shutdown_connections 2024-01-15T18:22:49.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 -- 172.21.15.143:0/2158991964 wait complete. 2024-01-15T18:22:49.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 Processor -- start 2024-01-15T18:22:49.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 -- start start 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05ec0ce560 0x7f05ec0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec141660 0x7f05ec159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec138b20 con 0x7f05ec071d00 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0b2f00 con 0x7f05ec141660 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05f1c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec137f10 con 0x7f05ec0ce560 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:49.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54378/0 (socket says 172.21.15.143:54378) 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 -- 172.21.15.143:0/661192210 learned_addr learned my addr 172.21.15.143:0/661192210 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eb400640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec141660 0x7f05ec159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05ea000640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05ec0ce560 0x7f05ec0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 -- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05ec0ce560 msgr2=0x7f05ec0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f05ec0ce560 0x7f05ec0acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 -- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec141660 msgr2=0x7f05ec159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec141660 0x7f05ec159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 -- 172.21.15.143:0/661192210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009a70 con 0x7f05ec071d00 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eb400640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f05ec141660 0x7f05ec159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.811+0000 7f05eaa00640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05ec071d00 0x7f05ec15b760 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c480 tx=0x7f05dc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0074e0 con 0x7f05ec071d00 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc007660 con 0x7f05ec071d00 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc004a30 con 0x7f05ec071d00 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05f1c00640 1 -- 172.21.15.143:0/661192210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec0cd150 con 0x7f05ec071d00 2024-01-15T18:22:49.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05f1c00640 1 -- 172.21.15.143:0/661192210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec1384d0 con 0x7f05ec071d00 2024-01-15T18:22:49.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05f1c00640 1 -- 172.21.15.143:0/661192210 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec067500 con 0x7f05ec071d00 2024-01-15T18:22:49.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f05dc004b90 con 0x7f05ec071d00 2024-01-15T18:22:49.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05e2a00640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05c4067bb0 0x7f05c406a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:49.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f05dc082410 con 0x7f05ec071d00 2024-01-15T18:22:49.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05ea000640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05c4067bb0 0x7f05c406a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:49.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.815+0000 7f05ea000640 1 --2- 172.21.15.143:0/661192210 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f05c4067bb0 0x7f05c406a070 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0b33f0 tx=0x7f05d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:49.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:49.819+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f05dc050d20 con 0x7f05ec071d00 2024-01-15T18:22:50.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:50.019+0000 7f05f1c00640 1 -- 172.21.15.143:0/661192210 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05ec0ccfd0 con 0x7f05c4067bb0 2024-01-15T18:22:50.016 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:50.023+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:50.023+0000 7f05e2a00640 1 -- 172.21.15.143:0/661192210 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f05d400b5c0 con 0x7f05c4067bb0 2024-01-15T18:22:50.017 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:50.017 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:50.027 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:52.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:52.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63ca800640 1 Processor -- start 2024-01-15T18:22:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63ca800640 1 -- start start 2024-01-15T18:22:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63ca800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f63c40ce560 0x7f63c415ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63ca800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c415f310 con 0x7f63c4141660 2024-01-15T18:22:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63ca800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c415f5f0 con 0x7f63c4071d00 2024-01-15T18:22:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63ca800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c415fbb0 con 0x7f63c40ce560 2024-01-15T18:22:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f63c4141660 0x7f63c4141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36058/0 (socket says 172.21.15.143:36058) 2024-01-15T18:22:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c3400640 1 -- 172.21.15.143:0/2776110953 learned_addr learned my addr 172.21.15.143:0/2776110953 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 --2- 172.21.15.143:0/2776110953 >> v2:172.21.15.204:3300/0 conn(0x7f63c40ce560 0x7f63c415ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056091472 0 0) 0x7f63c415f310 con 0x7f63c4141660 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a4003530 con 0x7f63c4141660 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2401731267 0 0) 0x7f63a4003530 con 0x7f63c4141660 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 >> v2:172.21.15.204:3300/0 conn(0x7f63c40ce560 msgr2=0x7f63c415ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 --2- 172.21.15.143:0/2776110953 >> v2:172.21.15.204:3300/0 conn(0x7f63c40ce560 0x7f63c415ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 >> v1:172.21.15.204:6789/0 conn(0x7f63c4071d00 legacy=0x7f63c40b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c415f8d0 con 0x7f63c4141660 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f63b4001680 con 0x7f63c4141660 2024-01-15T18:22:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 >> v1:172.21.15.143:6789/0 conn(0x7f63c4141660 legacy=0x7f63c4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4003980 0x7f63a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63a4005190 0x7f63a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c415fbb0 con 0x7f63a4005190 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a400c030 con 0x7f63a40047d0 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a400c1f0 con 0x7f63a4003980 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c3e00640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63a4005190 0x7f63a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c3400640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4003980 0x7f63a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 -- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4003980 msgr2=0x7f63a4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63a4003980 0x7f63a4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 -- 172.21.15.143:0/2776110953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63a4005190 msgr2=0x7f63a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63a4005190 0x7f63a400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:52.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c2a00640 1 -- 172.21.15.143:0/2776110953 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4003530 con 0x7f63a40047d0 2024-01-15T18:22:52.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c3e00640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63a4005190 0x7f63a400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:22:52.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.191+0000 7f63c3e00640 1 -- 172.21.15.143:0/2776110953 reap_dead start 2024-01-15T18:22:52.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63c2a00640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63a4004c50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009ae0 tx=0x7f63ac01ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3218c4752b8913 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0a5640 con 0x7f63a40047d0 2024-01-15T18:22:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac0a57a0 con 0x7f63a40047d0 2024-01-15T18:22:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63c2000640 1 -- 172.21.15.143:0/2776110953 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac0a5b50 con 0x7f63a40047d0 2024-01-15T18:22:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63ca800640 1 -- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 msgr2=0x7f63a4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63ca800640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63a4004c50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009ae0 tx=0x7f63ac01ca50 comp rx=0 tx=0).stop 2024-01-15T18:22:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63ca800640 1 -- 172.21.15.143:0/2776110953 shutdown_connections 2024-01-15T18:22:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63ca800640 1 --2- 172.21.15.143:0/2776110953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63a4004c50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.195+0000 7f63ca800640 1 -- 172.21.15.143:0/2776110953 >> 172.21.15.143:0/2776110953 conn(0x7f63c406c3f0 msgr2=0x7f63c406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 -- 172.21.15.143:0/2776110953 shutdown_connections 2024-01-15T18:22:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 -- 172.21.15.143:0/2776110953 wait complete. 2024-01-15T18:22:52.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 Processor -- start 2024-01-15T18:22:52.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 -- start start 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63c41f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63c4138480 0x7f63c4138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c4143ca0 0x7f63c4144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c4138e40 con 0x7f63c4138480 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c41f63d0 con 0x7f63a40047d0 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63ca800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c413ba00 con 0x7f63c4143ca0 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63c4138480 0x7f63c4138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63c4138480 0x7f63c4138900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54388/0 (socket says 172.21.15.143:54388) 2024-01-15T18:22:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 -- 172.21.15.143:0/1244499207 learned_addr learned my addr 172.21.15.143:0/1244499207 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c3400640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63c41f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c3e00640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c4143ca0 0x7f63c4144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 -- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c4143ca0 msgr2=0x7f63c4144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c4143ca0 0x7f63c4144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 -- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 msgr2=0x7f63c41f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63c41f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 -- 172.21.15.143:0/1244499207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c40cd150 con 0x7f63c4138480 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c3e00640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63c4143ca0 0x7f63c4144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c3400640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63a40047d0 0x7f63c41f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.199+0000 7f63c2a00640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f63c4138480 0x7f63c4138900 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0b0040 tx=0x7f63ac0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac004220 con 0x7f63c4138480 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac0043a0 con 0x7f63c4138480 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63ca800640 1 -- 172.21.15.143:0/1244499207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c4144660 con 0x7f63c4138480 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac004510 con 0x7f63c4138480 2024-01-15T18:22:52.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63ca800640 1 -- 172.21.15.143:0/1244499207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c415c7a0 con 0x7f63c4138480 2024-01-15T18:22:52.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63ca800640 1 -- 172.21.15.143:0/1244499207 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c415e9f0 con 0x7f63c4138480 2024-01-15T18:22:52.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f63ac0b4070 con 0x7f63c4138480 2024-01-15T18:22:52.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63bb400640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f639c067bb0 0x7f639c06a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:52.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f63ac11cf30 con 0x7f63c4138480 2024-01-15T18:22:52.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.203+0000 7f63c3400640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f639c067bb0 0x7f639c06a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:52.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.207+0000 7f63c3400640 1 --2- 172.21.15.143:0/1244499207 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f639c067bb0 0x7f639c06a070 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f63c40cec00 tx=0x7f63b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:52.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.207+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f63ac0ebbc0 con 0x7f63c4138480 2024-01-15T18:22:52.381 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.391+0000 7f63ca800640 1 -- 172.21.15.143:0/1244499207 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63c40ccfd0 con 0x7f639c067bb0 2024-01-15T18:22:52.388 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:52.395+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:52.388 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:52.395+0000 7f63bb400640 1 -- 172.21.15.143:0/1244499207 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f63b4001640 con 0x7f639c067bb0 2024-01-15T18:22:52.388 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:52.389 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:52.397 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:54.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:54.551 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.559+0000 7f8b6c600640 1 Processor -- start 2024-01-15T18:22:54.552 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.559+0000 7f8b6c600640 1 -- start start 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.559+0000 7f8b6c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b641607c0 0x7f8b6406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.559+0000 7f8b6c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8b640cfb30 0x7f8b6406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.559+0000 7f8b6c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b6406c230 0x7f8b640cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b6c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6406c670 con 0x7f8b641607c0 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b6c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b640d1ce0 con 0x7f8b640cfb30 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b6c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640cffd0 con 0x7f8b6406c230 2024-01-15T18:22:54.553 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b641607c0 0x7f8b6406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b641607c0 0x7f8b6406b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54400/0 (socket says 172.21.15.143:54400) 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 -- 172.21.15.143:0/4224224365 learned_addr learned my addr 172.21.15.143:0/4224224365 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69200640 1 --2- 172.21.15.143:0/4224224365 >> v2:172.21.15.204:3300/0 conn(0x7f8b640cfb30 0x7f8b6406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b6a600640 1 --2- 172.21.15.143:0/4224224365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b6406c230 0x7f8b640cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 -- 172.21.15.143:0/4224224365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b6406c230 msgr2=0x7f8b640cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 --2- 172.21.15.143:0/4224224365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b6406c230 0x7f8b640cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 -- 172.21.15.143:0/4224224365 >> v2:172.21.15.204:3300/0 conn(0x7f8b640cfb30 msgr2=0x7f8b6406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:54.554 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 --2- 172.21.15.143:0/4224224365 >> v2:172.21.15.204:3300/0 conn(0x7f8b640cfb30 0x7f8b6406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.555 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.563+0000 7f8b69c00640 1 -- 172.21.15.143:0/4224224365 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640d02b0 con 0x7f8b641607c0 2024-01-15T18:22:54.556 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b69c00640 1 --2- 172.21.15.143:0/4224224365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b6406b7b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f8b6409a980 tx=0x7f8b5401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99692d2e2a8843d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b63e00640 1 -- 172.21.15.143:0/4224224365 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b540a89b0 con 0x7f8b641607c0 2024-01-15T18:22:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b63e00640 1 -- 172.21.15.143:0/4224224365 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b540a8b10 con 0x7f8b641607c0 2024-01-15T18:22:54.557 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b63e00640 1 -- 172.21.15.143:0/4224224365 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b540a8ea0 con 0x7f8b641607c0 2024-01-15T18:22:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 -- 172.21.15.143:0/4224224365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 msgr2=0x7f8b6406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 --2- 172.21.15.143:0/4224224365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b6406b7b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f8b6409a980 tx=0x7f8b5401ecf0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.558 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 -- 172.21.15.143:0/4224224365 shutdown_connections 2024-01-15T18:22:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 --2- 172.21.15.143:0/4224224365 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b6406c230 0x7f8b640cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 --2- 172.21.15.143:0/4224224365 >> v2:172.21.15.204:3300/0 conn(0x7f8b640cfb30 0x7f8b6406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 --2- 172.21.15.143:0/4224224365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b6406b7b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 -- 172.21.15.143:0/4224224365 >> 172.21.15.143:0/4224224365 conn(0x7f8b640d0e20 msgr2=0x7f8b640d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 -- 172.21.15.143:0/4224224365 shutdown_connections 2024-01-15T18:22:54.559 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 -- 172.21.15.143:0/4224224365 wait complete. 2024-01-15T18:22:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 Processor -- start 2024-01-15T18:22:54.560 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.567+0000 7f8b6c600640 1 -- start start 2024-01-15T18:22:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b6406c230 0x7f8b6415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b640cfb30 0x7f8b641f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.562 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b640ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640b4a00 con 0x7f8b641607c0 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640b2180 con 0x7f8b6406c230 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6415c980 con 0x7f8b640cfb30 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b640ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b640ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54416/0 (socket says 172.21.15.143:54416) 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 -- 172.21.15.143:0/537406679 learned_addr learned my addr 172.21.15.143:0/537406679 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b69200640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b640cfb30 0x7f8b641f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b69c00640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b6406c230 0x7f8b6415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 -- 172.21.15.143:0/537406679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b640cfb30 msgr2=0x7f8b641f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8b640cfb30 0x7f8b641f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 -- 172.21.15.143:0/537406679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b6406c230 msgr2=0x7f8b6415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8b6406c230 0x7f8b6415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:54.563 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 -- 172.21.15.143:0/537406679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540094d0 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6a600640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b641607c0 0x7f8b640ca890 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00eed0 tx=0x7f8b5c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c010040 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c014400 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c009ea0 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 -- 172.21.15.143:0/537406679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b640c7ef0 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 -- 172.21.15.143:0/537406679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b641f6960 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b6c600640 1 -- 172.21.15.143:0/537406679 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6406d4b0 con 0x7f8b641607c0 2024-01-15T18:22:54.564 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.571+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f8b5c0147b0 con 0x7f8b641607c0 2024-01-15T18:22:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.575+0000 7f8b61600640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b3c067b80 0x7f8b3c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.575+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8b5c0839c0 con 0x7f8b641607c0 2024-01-15T18:22:54.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.575+0000 7f8b69c00640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b3c067b80 0x7f8b3c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:54.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.575+0000 7f8b69c00640 1 --2- 172.21.15.143:0/537406679 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8b3c067b80 0x7f8b3c06a040 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f8b540b2000 tx=0x7f8b540aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:54.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.579+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8b5c052380 con 0x7f8b641607c0 2024-01-15T18:22:54.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.759+0000 7f8b6c600640 1 -- 172.21.15.143:0/537406679 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b640cba30 con 0x7f8b3c067b80 2024-01-15T18:22:54.757 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:54.767+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:54.767+0000 7f8b61600640 1 -- 172.21.15.143:0/537406679 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8b540c00f0 con 0x7f8b3c067b80 2024-01-15T18:22:54.758 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:54.758 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:54.766 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:56.767 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 Processor -- start 2024-01-15T18:22:56.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 -- start start 2024-01-15T18:22:56.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f178806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:56.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f178815f1b0 0x7f178806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17880cc810 con 0x7f178815bf40 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178806e9f0 con 0x7f178815f1b0 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f178ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178815ea80 con 0x7f1788070cc0 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f178815f1b0 0x7f178806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f178815f1b0 0x7f178806a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54428/0 (socket says 172.21.15.143:54428) 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 -- 172.21.15.143:0/2929576117 learned_addr learned my addr 172.21.15.143:0/2929576117 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1785e00640 1 --2- 172.21.15.143:0/2929576117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f178806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 -- 172.21.15.143:0/2929576117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 msgr2=0x7f178806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 --2- 172.21.15.143:0/2929576117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f178806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 -- 172.21.15.143:0/2929576117 >> v1:172.21.15.204:6789/0 conn(0x7f178815bf40 legacy=0x7f17880710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:56.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.931+0000 7f1786800640 1 -- 172.21.15.143:0/2929576117 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178815c380 con 0x7f178815f1b0 2024-01-15T18:22:56.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f1786800640 1 --2- 172.21.15.143:0/2929576117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f178806a490 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f1780009e80 tx=0x7f178001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b4172b3beb6f47 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:56.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f1784a00640 1 -- 172.21.15.143:0/2929576117 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800a5800 con 0x7f178815f1b0 2024-01-15T18:22:56.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f1784a00640 1 -- 172.21.15.143:0/2929576117 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17800a5960 con 0x7f178815f1b0 2024-01-15T18:22:56.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f1784a00640 1 -- 172.21.15.143:0/2929576117 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800a5cf0 con 0x7f178815f1b0 2024-01-15T18:22:56.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 -- 172.21.15.143:0/2929576117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 msgr2=0x7f178806a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:56.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 --2- 172.21.15.143:0/2929576117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f178806a490 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f1780009e80 tx=0x7f178001c980 comp rx=0 tx=0).stop 2024-01-15T18:22:56.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 -- 172.21.15.143:0/2929576117 shutdown_connections 2024-01-15T18:22:56.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 --2- 172.21.15.143:0/2929576117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f178806a490 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:56.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 --2- 172.21.15.143:0/2929576117 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f178806dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:56.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 -- 172.21.15.143:0/2929576117 >> 172.21.15.143:0/2929576117 conn(0x7f17880b1610 msgr2=0x7f17880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:56.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 -- 172.21.15.143:0/2929576117 shutdown_connections 2024-01-15T18:22:56.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.935+0000 7f178ce00640 1 -- 172.21.15.143:0/2929576117 wait complete. 2024-01-15T18:22:56.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 Processor -- start 2024-01-15T18:22:56.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 -- start start 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f17880af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178815bf40 0x7f178814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f1788164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1788164db0 con 0x7f178815f1b0 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17880af720 con 0x7f178815bf40 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f178ce00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17880afa00 con 0x7f1788070cc0 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f1788164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1785400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178815bf40 0x7f178814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f1788164870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54438/0 (socket says 172.21.15.143:54438) 2024-01-15T18:22:56.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 -- 172.21.15.143:0/1239582025 learned_addr learned my addr 172.21.15.143:0/1239582025 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1785e00640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f17880af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 -- 172.21.15.143:0/1239582025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 msgr2=0x7f17880af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1788070cc0 0x7f17880af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 -- 172.21.15.143:0/1239582025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178815bf40 msgr2=0x7f178814f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f178815bf40 0x7f178814f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 -- 172.21.15.143:0/1239582025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009b30 con 0x7f178815f1b0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f1786800640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f178815f1b0 0x7f1788164870 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f17800af040 tx=0x7f1780019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.939+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800b8640 con 0x7f178815f1b0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17800b87a0 con 0x7f178815f1b0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17800b8b10 con 0x7f178815f1b0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f178ce00640 1 -- 172.21.15.143:0/1239582025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17880afce0 con 0x7f178815f1b0 2024-01-15T18:22:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f178ce00640 1 -- 172.21.15.143:0/1239582025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178815c5f0 con 0x7f178815f1b0 2024-01-15T18:22:56.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f178ce00640 1 -- 172.21.15.143:0/1239582025 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1788074b00 con 0x7f178815f1b0 2024-01-15T18:22:56.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f17800b8c70 con 0x7f178815f1b0 2024-01-15T18:22:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f177e000640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1760067b80 0x7f176006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f178011e260 con 0x7f178815f1b0 2024-01-15T18:22:56.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.943+0000 7f1785e00640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1760067b80 0x7f176006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:56.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.947+0000 7f1785e00640 1 --2- 172.21.15.143:0/1239582025 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1760067b80 0x7f176006a040 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f17780079c0 tx=0x7f1778008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:56.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:56.947+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f17800ecfa0 con 0x7f178815f1b0 2024-01-15T18:22:57.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:57.135+0000 7f178ce00640 1 -- 172.21.15.143:0/1239582025 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f178806ade0 con 0x7f1760067b80 2024-01-15T18:22:57.131 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:57.139+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:57.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:57.139+0000 7f177e000640 1 -- 172.21.15.143:0/1239582025 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f17780108d0 con 0x7f1760067b80 2024-01-15T18:22:57.132 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:57.132 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:57.140 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:22:59.142 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:22:59.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 Processor -- start 2024-01-15T18:22:59.294 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 -- start start 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0468141660 0x7f04680ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f046815ec80 0x7f0468141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046815f310 con 0x7f0468142470 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046815fbb0 con 0x7f046815ec80 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046fe00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046815f130 con 0x7f0468141660 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046ca00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f046815ec80 0x7f0468141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046de00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0468142470 0x7f0468071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47050/0 (socket says 172.21.15.143:47050) 2024-01-15T18:22:59.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046de00640 1 -- 172.21.15.143:0/1161126071 learned_addr learned my addr 172.21.15.143:0/1161126071 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046d400640 1 --2- 172.21.15.143:0/1161126071 >> v2:172.21.15.204:3300/0 conn(0x7f0468141660 0x7f04680ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046ca00640 1 -- 172.21.15.143:0/1161126071 >> v2:172.21.15.204:3300/0 conn(0x7f0468141660 msgr2=0x7f04680ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046ca00640 1 --2- 172.21.15.143:0/1161126071 >> v2:172.21.15.204:3300/0 conn(0x7f0468141660 0x7f04680ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046ca00640 1 -- 172.21.15.143:0/1161126071 >> v1:172.21.15.143:6789/0 conn(0x7f0468142470 legacy=0x7f0468071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f046ca00640 1 -- 172.21.15.143:0/1161126071 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046815f5f0 con 0x7f046815ec80 2024-01-15T18:22:59.297 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.303+0000 7f0467e00640 1 -- 172.21.15.143:0/1161126071 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767825618 0 0) 0x7f046815f310 con 0x7f0468142470 2024-01-15T18:22:59.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.307+0000 7f046ca00640 1 --2- 172.21.15.143:0/1161126071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f046815ec80 0x7f0468141f30 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f04680b1aa0 tx=0x7f045001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5962aa96d3d1f6ef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:59.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.307+0000 7f0467e00640 1 -- 172.21.15.143:0/1161126071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04500a7070 con 0x7f046815ec80 2024-01-15T18:22:59.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.307+0000 7f0467e00640 1 -- 172.21.15.143:0/1161126071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04500ab430 con 0x7f046815ec80 2024-01-15T18:22:59.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.307+0000 7f0467e00640 1 -- 172.21.15.143:0/1161126071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04500ab7c0 con 0x7f046815ec80 2024-01-15T18:22:59.300 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- 172.21.15.143:0/1161126071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f046815ec80 msgr2=0x7f0468141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:59.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 --2- 172.21.15.143:0/1161126071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f046815ec80 0x7f0468141f30 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f04680b1aa0 tx=0x7f045001c430 comp rx=0 tx=0).stop 2024-01-15T18:22:59.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- 172.21.15.143:0/1161126071 shutdown_connections 2024-01-15T18:22:59.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 --2- 172.21.15.143:0/1161126071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f046815ec80 0x7f0468141f30 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:59.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 --2- 172.21.15.143:0/1161126071 >> v2:172.21.15.204:3300/0 conn(0x7f0468141660 0x7f04680ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:59.301 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- 172.21.15.143:0/1161126071 >> 172.21.15.143:0/1161126071 conn(0x7f046806c3f0 msgr2=0x7f046806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:22:59.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- 172.21.15.143:0/1161126071 shutdown_connections 2024-01-15T18:22:59.302 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- 172.21.15.143:0/1161126071 wait complete. 2024-01-15T18:22:59.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 Processor -- start 2024-01-15T18:22:59.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- start start 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0468141660 0x7f046813e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0468142470 0x7f0468149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f046815ec80 0x7f04681f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046815d130 con 0x7f0468142470 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04681f9e90 con 0x7f046815ec80 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046fe00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046815d3b0 con 0x7f0468141660 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0468142470 0x7f0468149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046ca00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0468142470 0x7f0468149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48482/0 (socket says 172.21.15.143:48482) 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046ca00640 1 -- 172.21.15.143:0/1450301980 learned_addr learned my addr 172.21.15.143:0/1450301980 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:22:59.305 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046de00640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f046815ec80 0x7f04681f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.311+0000 7f046d400640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0468141660 0x7f046813e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046ca00640 1 -- 172.21.15.143:0/1450301980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0468141660 msgr2=0x7f046813e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046ca00640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0468141660 0x7f046813e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046ca00640 1 -- 172.21.15.143:0/1450301980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f046815ec80 msgr2=0x7f04681f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046ca00640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f046815ec80 0x7f04681f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046ca00640 1 -- 172.21.15.143:0/1450301980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0450009420 con 0x7f0468142470 2024-01-15T18:22:59.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046ca00640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0468142470 0x7f0468149160 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f0450019040 tx=0x7f045001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04500b5810 con 0x7f0468142470 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04500b5990 con 0x7f0468142470 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046fe00640 1 -- 172.21.15.143:0/1450301980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04680cd150 con 0x7f0468142470 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04500b5d40 con 0x7f0468142470 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046fe00640 1 -- 172.21.15.143:0/1450301980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0468149820 con 0x7f0468142470 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046fe00640 1 -- 172.21.15.143:0/1450301980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046815d540 con 0x7f0468142470 2024-01-15T18:22:59.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f04500c7390 con 0x7f0468142470 2024-01-15T18:22:59.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f0465600640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0440067b80 0x7f044006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:22:59.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f045011de10 con 0x7f0468142470 2024-01-15T18:22:59.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.315+0000 7f046d400640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0440067b80 0x7f044006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:22:59.309 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.319+0000 7f046d400640 1 --2- 172.21.15.143:0/1450301980 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0440067b80 0x7f044006a040 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0458006aa0 tx=0x7f0458008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:22:59.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.323+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f04500ec8f0 con 0x7f0468142470 2024-01-15T18:22:59.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.499+0000 7f046fe00640 1 -- 172.21.15.143:0/1450301980 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04680ccfd0 con 0x7f0440067b80 2024-01-15T18:22:59.498 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:22:59.507+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:22:59.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:22:59.507+0000 7f0465600640 1 -- 172.21.15.143:0/1450301980 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f04580076d0 con 0x7f0440067b80 2024-01-15T18:22:59.499 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:22:59.499 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:22:59.507 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:00.679 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:23:00.683 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:23:01.509 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 Processor -- start 2024-01-15T18:23:01.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 -- start start 2024-01-15T18:23:01.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8380ce560 0x7ff83815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:01.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff838071d00 0x7ff838072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:01.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83815f310 con 0x7ff838141660 2024-01-15T18:23:01.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83815fbb0 con 0x7ff8380ce560 2024-01-15T18:23:01.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff840800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff838072690 con 0x7ff838071d00 2024-01-15T18:23:01.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8380ce560 0x7ff83815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8380ce560 0x7ff83815ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48496/0 (socket says 172.21.15.143:48496) 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 -- 172.21.15.143:0/660534645 learned_addr learned my addr 172.21.15.143:0/660534645 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83e800640 1 --2- 172.21.15.143:0/660534645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff838071d00 0x7ff838072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 -- 172.21.15.143:0/660534645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff838071d00 msgr2=0x7ff838072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 --2- 172.21.15.143:0/660534645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff838071d00 0x7ff838072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 -- 172.21.15.143:0/660534645 >> v1:172.21.15.204:6789/0 conn(0x7ff838141660 legacy=0x7ff838141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:01.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.671+0000 7ff83d400640 1 -- 172.21.15.143:0/660534645 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83815f5f0 con 0x7ff8380ce560 2024-01-15T18:23:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.675+0000 7ff83d400640 1 --2- 172.21.15.143:0/660534645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8380ce560 0x7ff83815ec80 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ff820009870 tx=0x7ff82001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d8735c31bd0a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.675+0000 7ff83ca00640 1 -- 172.21.15.143:0/660534645 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8200a7070 con 0x7ff8380ce560 2024-01-15T18:23:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.675+0000 7ff83ca00640 1 -- 172.21.15.143:0/660534645 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8200ab430 con 0x7ff8380ce560 2024-01-15T18:23:01.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.675+0000 7ff83ca00640 1 -- 172.21.15.143:0/660534645 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8200ab7c0 con 0x7ff8380ce560 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.675+0000 7ff840800640 1 -- 172.21.15.143:0/660534645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8380ce560 msgr2=0x7ff83815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.675+0000 7ff840800640 1 --2- 172.21.15.143:0/660534645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8380ce560 0x7ff83815ec80 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ff820009870 tx=0x7ff82001c540 comp rx=0 tx=0).stop 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- 172.21.15.143:0/660534645 shutdown_connections 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 --2- 172.21.15.143:0/660534645 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff838071d00 0x7ff838072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 --2- 172.21.15.143:0/660534645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8380ce560 0x7ff83815ec80 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- 172.21.15.143:0/660534645 >> 172.21.15.143:0/660534645 conn(0x7ff83806c3f0 msgr2=0x7ff83806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- 172.21.15.143:0/660534645 shutdown_connections 2024-01-15T18:23:01.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- 172.21.15.143:0/660534645 wait complete. 2024-01-15T18:23:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 Processor -- start 2024-01-15T18:23:01.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- start start 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff838071d00 0x7ff83815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8380ce560 0x7ff8380acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838141660 0x7ff83813b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83813cfc0 con 0x7ff838071d00 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83813d260 con 0x7ff838141660 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff840800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83813d540 con 0x7ff8380ce560 2024-01-15T18:23:01.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff83de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff838071d00 0x7ff83815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff83de00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff838071d00 0x7ff83815e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48508/0 (socket says 172.21.15.143:48508) 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff83de00640 1 -- 172.21.15.143:0/1075033694 learned_addr learned my addr 172.21.15.143:0/1075033694 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff83e800640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838141660 0x7ff83813b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.679+0000 7ff83d400640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8380ce560 0x7ff8380acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83de00640 1 -- 172.21.15.143:0/1075033694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8380ce560 msgr2=0x7ff8380acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83de00640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8380ce560 0x7ff8380acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83de00640 1 -- 172.21.15.143:0/1075033694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838141660 msgr2=0x7ff83813b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83de00640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff838141660 0x7ff83813b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83de00640 1 -- 172.21.15.143:0/1075033694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200094d0 con 0x7ff838071d00 2024-01-15T18:23:01.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83de00640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff838071d00 0x7ff83815e000 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7ff82800c3c0 tx=0x7ff82800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8280052d0 con 0x7ff838071d00 2024-01-15T18:23:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff828005430 con 0x7ff838071d00 2024-01-15T18:23:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff840800640 1 -- 172.21.15.143:0/1075033694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8380cd150 con 0x7ff838071d00 2024-01-15T18:23:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff828003840 con 0x7ff838071d00 2024-01-15T18:23:01.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff840800640 1 -- 172.21.15.143:0/1075033694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83813db00 con 0x7ff838071d00 2024-01-15T18:23:01.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff840800640 1 -- 172.21.15.143:0/1075033694 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff838067500 con 0x7ff838071d00 2024-01-15T18:23:01.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7ff8280039a0 con 0x7ff838071d00 2024-01-15T18:23:01.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff836000640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff810067b80 0x7ff81006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:01.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7ff8280828a0 con 0x7ff838071d00 2024-01-15T18:23:01.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.683+0000 7ff83d400640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff810067b80 0x7ff81006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:01.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.687+0000 7ff83d400640 1 --2- 172.21.15.143:0/1075033694 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff810067b80 0x7ff81006a040 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff8200af000 tx=0x7ff820008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:01.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.687+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff828051260 con 0x7ff838071d00 2024-01-15T18:23:01.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.867+0000 7ff840800640 1 -- 172.21.15.143:0/1075033694 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8380cce10 con 0x7ff810067b80 2024-01-15T18:23:01.865 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:01.875+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:01.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:01.875+0000 7ff836000640 1 -- 172.21.15.143:0/1075033694 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff8380cce10 con 0x7ff810067b80 2024-01-15T18:23:01.866 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:01.866 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:01.874 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:03.876 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:04.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 Processor -- start 2024-01-15T18:23:04.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 -- start start 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc694143780 0x7fc6940cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc6940ce520 0x7fc694071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940ceb50 con 0x7fc6940720f0 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69415c110 con 0x7fc694143780 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc699200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc694072530 con 0x7fc6940ce520 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc694143780 0x7fc6940cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc692800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc6940720f0 0x7fc69415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:47058/0 (socket says 172.21.15.143:47058) 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc692800640 1 -- 172.21.15.143:0/3917619879 learned_addr learned my addr 172.21.15.143:0/3917619879 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691400640 1 --2- 172.21.15.143:0/3917619879 >> v2:172.21.15.204:3300/0 conn(0x7fc6940ce520 0x7fc694071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:04.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691e00640 1 -- 172.21.15.143:0/3917619879 >> v2:172.21.15.204:3300/0 conn(0x7fc6940ce520 msgr2=0x7fc694071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691e00640 1 --2- 172.21.15.143:0/3917619879 >> v2:172.21.15.204:3300/0 conn(0x7fc6940ce520 0x7fc694071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691e00640 1 -- 172.21.15.143:0/3917619879 >> v1:172.21.15.143:6789/0 conn(0x7fc6940720f0 legacy=0x7fc69415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691e00640 1 -- 172.21.15.143:0/3917619879 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69415b7e0 con 0x7fc694143780 2024-01-15T18:23:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc690a00640 1 -- 172.21.15.143:0/3917619879 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154920899 0 0) 0x7fc68c0014a0 con 0x7fc6940720f0 2024-01-15T18:23:04.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.039+0000 7fc691400640 1 --2- 172.21.15.143:0/3917619879 >> v2:172.21.15.204:3300/0 conn(0x7fc6940ce520 0x7fc694071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc691e00640 1 --2- 172.21.15.143:0/3917619879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc694143780 0x7fc6940cdfe0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc69406beb0 tx=0x7fc68401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cfe62c18e520bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc690a00640 1 -- 172.21.15.143:0/3917619879 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840ab600 con 0x7fc694143780 2024-01-15T18:23:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc690a00640 1 -- 172.21.15.143:0/3917619879 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6840a7050 con 0x7fc694143780 2024-01-15T18:23:04.035 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc690a00640 1 -- 172.21.15.143:0/3917619879 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840ab980 con 0x7fc694143780 2024-01-15T18:23:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 -- 172.21.15.143:0/3917619879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc694143780 msgr2=0x7fc6940cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 --2- 172.21.15.143:0/3917619879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc694143780 0x7fc6940cdfe0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fc69406beb0 tx=0x7fc68401c460 comp rx=0 tx=0).stop 2024-01-15T18:23:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 -- 172.21.15.143:0/3917619879 shutdown_connections 2024-01-15T18:23:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 --2- 172.21.15.143:0/3917619879 >> v2:172.21.15.204:3300/0 conn(0x7fc6940ce520 0x7fc694071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:04.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 --2- 172.21.15.143:0/3917619879 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc694143780 0x7fc6940cdfe0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 -- 172.21.15.143:0/3917619879 >> 172.21.15.143:0/3917619879 conn(0x7fc69406c3f0 msgr2=0x7fc69406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 -- 172.21.15.143:0/3917619879 shutdown_connections 2024-01-15T18:23:04.037 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.043+0000 7fc699200640 1 -- 172.21.15.143:0/3917619879 wait complete. 2024-01-15T18:23:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 Processor -- start 2024-01-15T18:23:04.038 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 -- start start 2024-01-15T18:23:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6940720f0 0x7fc694157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:04.040 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6940ce520 0x7fc6940ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc694143780 0x7fc69413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940ad250 con 0x7fc6940720f0 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940ad530 con 0x7fc694143780 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940ad810 con 0x7fc6940ce520 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6940720f0 0x7fc694157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6940720f0 0x7fc694157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48530/0 (socket says 172.21.15.143:48530) 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 -- 172.21.15.143:0/1380072039 learned_addr learned my addr 172.21.15.143:0/1380072039 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691400640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6940ce520 0x7fc6940ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc692800640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc694143780 0x7fc69413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 -- 172.21.15.143:0/1380072039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6940ce520 msgr2=0x7fc6940ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc6940ce520 0x7fc6940ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 -- 172.21.15.143:0/1380072039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc694143780 msgr2=0x7fc69413b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc694143780 0x7fc69413b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:04.041 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 -- 172.21.15.143:0/1380072039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684009420 con 0x7fc6940720f0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc691e00640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc6940720f0 0x7fc694157f00 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fc684019070 tx=0x7fc68401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684004230 con 0x7fc6940720f0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684004390 con 0x7fc6940720f0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684003820 con 0x7fc6940720f0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 -- 172.21.15.143:0/1380072039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6940adaf0 con 0x7fc6940720f0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.047+0000 7fc699200640 1 -- 172.21.15.143:0/1380072039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc694158440 con 0x7fc6940720f0 2024-01-15T18:23:04.042 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.051+0000 7fc699200640 1 -- 172.21.15.143:0/1380072039 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6940ae2f0 con 0x7fc6940720f0 2024-01-15T18:23:04.043 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.051+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fc684003980 con 0x7fc6940720f0 2024-01-15T18:23:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.051+0000 7fc68a000640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc66c067b80 0x7fc66c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.051+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fc68411d790 con 0x7fc6940720f0 2024-01-15T18:23:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.051+0000 7fc691400640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc66c067b80 0x7fc66c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:04.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.051+0000 7fc691400640 1 --2- 172.21.15.143:0/1380072039 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc66c067b80 0x7fc66c06a040 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc694164a80 tx=0x7fc67c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:04.048 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.055+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc6840ec420 con 0x7fc6940720f0 2024-01-15T18:23:04.226 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.235+0000 7fc699200640 1 -- 172.21.15.143:0/1380072039 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc694067500 con 0x7fc66c067b80 2024-01-15T18:23:04.232 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:04.239+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:04.232 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:04.239+0000 7fc68a000640 1 -- 172.21.15.143:0/1380072039 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc694067500 con 0x7fc66c067b80 2024-01-15T18:23:04.233 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:04.233 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:04.240 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:06.241 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:06.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 Processor -- start 2024-01-15T18:23:06.394 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 -- start start 2024-01-15T18:23:06.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f16c006bfb0 0x7f16c006c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:06.395 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c00b0310 0x7f16c00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:06.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c00cbff0 con 0x7f16c00cc9c0 2024-01-15T18:23:06.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c00d0c50 con 0x7f16c006bfb0 2024-01-15T18:23:06.396 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16c6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c006c8d0 con 0x7f16c00b0310 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16bf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f16c006bfb0 0x7f16c006c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16bf400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f16c006bfb0 0x7f16c006c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:53378/0 (socket says 172.21.15.143:53378) 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16bea00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c00b0310 0x7f16c00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16bfe00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f16c00cc9c0 0x7f16c00cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:57190/0 (socket says 172.21.15.143:57190) 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.403+0000 7f16bf400640 1 -- 172.21.15.143:0/3001928728 learned_addr learned my addr 172.21.15.143:0/3001928728 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16bf400640 1 -- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c00b0310 msgr2=0x7f16c00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16bf400640 1 --2- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c00b0310 0x7f16c00cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16bf400640 1 -- 172.21.15.143:0/3001928728 >> v1:172.21.15.204:6789/0 conn(0x7f16c00cc9c0 legacy=0x7f16c00cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16bf400640 1 -- 172.21.15.143:0/3001928728 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c00cb6c0 con 0x7f16c006bfb0 2024-01-15T18:23:06.397 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16bea00640 1 --2- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c00b0310 0x7f16c00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:06.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16bf400640 1 --2- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c006bfb0 0x7f16c006c390 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009870 tx=0x7f16b001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34da61d1872f3492 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:06.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16be000640 1 -- 172.21.15.143:0/3001928728 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b00ab680 con 0x7f16c006bfb0 2024-01-15T18:23:06.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16be000640 1 -- 172.21.15.143:0/3001928728 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b00ab7e0 con 0x7f16c006bfb0 2024-01-15T18:23:06.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.407+0000 7f16be000640 1 -- 172.21.15.143:0/3001928728 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b00abb70 con 0x7f16c006bfb0 2024-01-15T18:23:06.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c006bfb0 msgr2=0x7f16c006c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:06.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 --2- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c006bfb0 0x7f16c006c390 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009870 tx=0x7f16b001c480 comp rx=0 tx=0).stop 2024-01-15T18:23:06.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- 172.21.15.143:0/3001928728 shutdown_connections 2024-01-15T18:23:06.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 --2- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c00b0310 0x7f16c00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:06.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 --2- 172.21.15.143:0/3001928728 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c006bfb0 0x7f16c006c390 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:06.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- 172.21.15.143:0/3001928728 >> 172.21.15.143:0/3001928728 conn(0x7f16c00d0ed0 msgr2=0x7f16c00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:06.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- 172.21.15.143:0/3001928728 shutdown_connections 2024-01-15T18:23:06.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- 172.21.15.143:0/3001928728 wait complete. 2024-01-15T18:23:06.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 Processor -- start 2024-01-15T18:23:06.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- start start 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c006bfb0 0x7f16c01604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c00b0310 0x7f16c00c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c00cc9c0 0x7f16c0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c4f40 con 0x7f16c00cc9c0 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c9740 con 0x7f16c00b0310 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16c6400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00c9a20 con 0x7f16c006bfb0 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c00cc9c0 0x7f16c0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.411+0000 7f16bfe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c00cc9c0 0x7f16c0157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48534/0 (socket says 172.21.15.143:48534) 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 -- 172.21.15.143:0/463331665 learned_addr learned my addr 172.21.15.143:0/463331665 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bea00640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c00b0310 0x7f16c00c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bf400640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c006bfb0 0x7f16c01604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:06.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 -- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c006bfb0 msgr2=0x7f16c01604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c006bfb0 0x7f16c01604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 -- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c00b0310 msgr2=0x7f16c00c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c00b0310 0x7f16c00c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 -- 172.21.15.143:0/463331665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009420 con 0x7f16c00cc9c0 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bea00640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f16c00b0310 0x7f16c00c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bf400640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f16c006bfb0 0x7f16c01604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16bfe00640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f16c00cc9c0 0x7f16c0157770 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f16b800c7a0 tx=0x7f16b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800a640 con 0x7f16c00cc9c0 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800a7a0 con 0x7f16c00cc9c0 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800ab10 con 0x7f16c00cc9c0 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16c6400640 1 -- 172.21.15.143:0/463331665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b00094d0 con 0x7f16c00cc9c0 2024-01-15T18:23:06.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16c6400640 1 -- 172.21.15.143:0/463331665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c00c9fe0 con 0x7f16c00cc9c0 2024-01-15T18:23:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16c6400640 1 -- 172.21.15.143:0/463331665 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0067500 con 0x7f16c00cc9c0 2024-01-15T18:23:06.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.415+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f16b800ac70 con 0x7f16c00cc9c0 2024-01-15T18:23:06.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.419+0000 7f16b7400640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1698067b80 0x7f169806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:06.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.419+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f16b8082b20 con 0x7f16c00cc9c0 2024-01-15T18:23:06.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.419+0000 7f16bf400640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1698067b80 0x7f169806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:06.409 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.419+0000 7f16bf400640 1 --2- 172.21.15.143:0/463331665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1698067b80 0x7f169806a040 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f16b00af000 tx=0x7f16b00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:06.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.423+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f16b80514e0 con 0x7f16c00cc9c0 2024-01-15T18:23:06.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.599+0000 7f16c6400640 1 -- 172.21.15.143:0/463331665 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16c00cd200 con 0x7f1698067b80 2024-01-15T18:23:06.595 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:06.603+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:06.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:06.603+0000 7f16b7400640 1 -- 172.21.15.143:0/463331665 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f16b00aba60 con 0x7f1698067b80 2024-01-15T18:23:06.596 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:06.596 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:06.605 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:08.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:08.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 Processor -- start 2024-01-15T18:23:08.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 -- start start 2024-01-15T18:23:08.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d80cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:08.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d80722c0 0x7f85d815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:08.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d80ceb50 con 0x7f85d80ce520 2024-01-15T18:23:08.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d815c110 con 0x7f85d80722c0 2024-01-15T18:23:08.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85e0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d815b7e0 con 0x7f85d8143780 2024-01-15T18:23:08.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85dea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f85d80722c0 0x7f85d815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:08.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85dd600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f85d80ce520 0x7f85d8071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38350/0 (socket says 172.21.15.143:38350) 2024-01-15T18:23:08.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.767+0000 7f85dd600640 1 -- 172.21.15.143:0/630712636 learned_addr learned my addr 172.21.15.143:0/630712636 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:08.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85de000640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d80cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:08.761 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dea00640 1 -- 172.21.15.143:0/630712636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 msgr2=0x7f85d80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:08.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dea00640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d80cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:08.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dea00640 1 -- 172.21.15.143:0/630712636 >> v1:172.21.15.143:6789/0 conn(0x7f85d80ce520 legacy=0x7f85d8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:08.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dcc00640 1 -- 172.21.15.143:0/630712636 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515279631 0 0) 0x7f85c0001120 con 0x7f85d80ce520 2024-01-15T18:23:08.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dea00640 1 -- 172.21.15.143:0/630712636 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d815bac0 con 0x7f85d80722c0 2024-01-15T18:23:08.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85de000640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:08.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dea00640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d815c5a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f85d00097c0 tx=0x7f85d001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70210e6c53f16c20 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:08.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dcc00640 1 -- 172.21.15.143:0/630712636 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00ad640 con 0x7f85d80722c0 2024-01-15T18:23:08.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dcc00640 1 -- 172.21.15.143:0/630712636 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d00ad7a0 con 0x7f85d80722c0 2024-01-15T18:23:08.764 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.771+0000 7f85dcc00640 1 -- 172.21.15.143:0/630712636 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00adb30 con 0x7f85d80722c0 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- 172.21.15.143:0/630712636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 msgr2=0x7f85d815c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d815c5a0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f85d00097c0 tx=0x7f85d001c480 comp rx=0 tx=0).stop 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- 172.21.15.143:0/630712636 shutdown_connections 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d815c5a0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 --2- 172.21.15.143:0/630712636 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d80cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- 172.21.15.143:0/630712636 >> 172.21.15.143:0/630712636 conn(0x7f85d806c3f0 msgr2=0x7f85d806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- 172.21.15.143:0/630712636 shutdown_connections 2024-01-15T18:23:08.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- 172.21.15.143:0/630712636 wait complete. 2024-01-15T18:23:08.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 Processor -- start 2024-01-15T18:23:08.767 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- start start 2024-01-15T18:23:08.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d8138620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:08.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80ce520 0x7f85d80b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:08.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d81599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8138b60 con 0x7f85d80722c0 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8139f50 con 0x7f85d80ce520 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85e0a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80b3930 con 0x7f85d8143780 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85de000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d8138620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85de000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d8138620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36586/0 (socket says 172.21.15.143:36586) 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85de000640 1 -- 172.21.15.143:0/2628360523 learned_addr learned my addr 172.21.15.143:0/2628360523 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85dd600640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80ce520 0x7f85d80b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.775+0000 7f85dea00640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d81599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85de000640 1 -- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 msgr2=0x7f85d81599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85de000640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f85d8143780 0x7f85d81599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85de000640 1 -- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80ce520 msgr2=0x7f85d80b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85de000640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80ce520 0x7f85d80b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85de000640 1 -- 172.21.15.143:0/2628360523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d0009420 con 0x7f85d80722c0 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85dd600640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85d80ce520 0x7f85d80b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85de000640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f85d80722c0 0x7f85d8138620 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f85c800ce00 tx=0x7f85c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:08.770 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800f890 con 0x7f85d80722c0 2024-01-15T18:23:08.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c800f9f0 con 0x7f85d80722c0 2024-01-15T18:23:08.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85e0a00640 1 -- 172.21.15.143:0/2628360523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d813cb50 con 0x7f85d80722c0 2024-01-15T18:23:08.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800fdc0 con 0x7f85d80722c0 2024-01-15T18:23:08.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85e0a00640 1 -- 172.21.15.143:0/2628360523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d8159f00 con 0x7f85d80722c0 2024-01-15T18:23:08.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85e0a00640 1 -- 172.21.15.143:0/2628360523 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d8067500 con 0x7f85d80722c0 2024-01-15T18:23:08.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f85c800b040 con 0x7f85d80722c0 2024-01-15T18:23:08.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85d6000640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067b80 0x7f85b006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:08.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.779+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f85c80839c0 con 0x7f85d80722c0 2024-01-15T18:23:08.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.783+0000 7f85dd600640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067b80 0x7f85b006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:08.773 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.783+0000 7f85dd600640 1 --2- 172.21.15.143:0/2628360523 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f85b0067b80 0x7f85b006a040 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f85d813ddd0 tx=0x7f85c0007a60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:08.776 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.783+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f85c80526b0 con 0x7f85d80722c0 2024-01-15T18:23:08.956 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.963+0000 7f85e0a00640 1 -- 172.21.15.143:0/2628360523 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85d813a280 con 0x7f85b0067b80 2024-01-15T18:23:08.962 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:08.971+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:08.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:08.971+0000 7f85d6000640 1 -- 172.21.15.143:0/2628360523 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f85d813a280 con 0x7f85b0067b80 2024-01-15T18:23:08.962 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:08.963 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:08.971 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:10.972 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:11.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 Processor -- start 2024-01-15T18:23:11.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 -- start start 2024-01-15T18:23:11.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd4480ce560 0x7fd44815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:11.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd448072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:11.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44815f310 con 0x7fd448141660 2024-01-15T18:23:11.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44815fbb0 con 0x7fd4480ce560 2024-01-15T18:23:11.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd448072690 con 0x7fd448071d00 2024-01-15T18:23:11.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44d200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd448141660 0x7fd448141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38362/0 (socket says 172.21.15.143:38362) 2024-01-15T18:23:11.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44d200640 1 -- 172.21.15.143:0/3184484889 learned_addr learned my addr 172.21.15.143:0/3184484889 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:11.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd447e00640 1 --2- 172.21.15.143:0/3184484889 >> v2:172.21.15.204:3300/0 conn(0x7fd4480ce560 0x7fd44815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:11.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.135+0000 7fd44dc00640 1 --2- 172.21.15.143:0/3184484889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd448072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd447400640 1 -- 172.21.15.143:0/3184484889 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131470646 0 0) 0x7fd44815f310 con 0x7fd448141660 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd447400640 1 -- 172.21.15.143:0/3184484889 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd428003530 con 0x7fd448141660 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd44dc00640 1 -- 172.21.15.143:0/3184484889 >> v2:172.21.15.204:3300/0 conn(0x7fd4480ce560 msgr2=0x7fd44815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd44dc00640 1 --2- 172.21.15.143:0/3184484889 >> v2:172.21.15.204:3300/0 conn(0x7fd4480ce560 0x7fd44815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd44dc00640 1 -- 172.21.15.143:0/3184484889 >> v1:172.21.15.143:6789/0 conn(0x7fd448141660 legacy=0x7fd448141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd44dc00640 1 -- 172.21.15.143:0/3184484889 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44815f5f0 con 0x7fd448071d00 2024-01-15T18:23:11.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd447e00640 1 --2- 172.21.15.143:0/3184484889 >> v2:172.21.15.204:3300/0 conn(0x7fd4480ce560 0x7fd44815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:11.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd44dc00640 1 --2- 172.21.15.143:0/3184484889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd448072150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd440009d90 tx=0x7fd44001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c365b9af8c6181 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:11.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd447400640 1 -- 172.21.15.143:0/3184484889 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400a5900 con 0x7fd448071d00 2024-01-15T18:23:11.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd447400640 1 -- 172.21.15.143:0/3184484889 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4400a5a60 con 0x7fd448071d00 2024-01-15T18:23:11.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.139+0000 7fd447400640 1 -- 172.21.15.143:0/3184484889 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400a5dd0 con 0x7fd448071d00 2024-01-15T18:23:11.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- 172.21.15.143:0/3184484889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 msgr2=0x7fd448072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 --2- 172.21.15.143:0/3184484889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd448072150 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd440009d90 tx=0x7fd44001c950 comp rx=0 tx=0).stop 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- 172.21.15.143:0/3184484889 shutdown_connections 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 --2- 172.21.15.143:0/3184484889 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd448072150 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 --2- 172.21.15.143:0/3184484889 >> v2:172.21.15.204:3300/0 conn(0x7fd4480ce560 0x7fd44815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- 172.21.15.143:0/3184484889 >> 172.21.15.143:0/3184484889 conn(0x7fd44806c3f0 msgr2=0x7fd44806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- 172.21.15.143:0/3184484889 shutdown_connections 2024-01-15T18:23:11.134 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- 172.21.15.143:0/3184484889 wait complete. 2024-01-15T18:23:11.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 Processor -- start 2024-01-15T18:23:11.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- start start 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd44813e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4480ce560 0x7fd44813b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd448141660 0x7fd44815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44815a090 con 0x7fd4480ce560 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44815a370 con 0x7fd448141660 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd44fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44815a650 con 0x7fd448071d00 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd447e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4480ce560 0x7fd44813b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:11.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd447e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4480ce560 0x7fd44813b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36592/0 (socket says 172.21.15.143:36592) 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.143+0000 7fd447e00640 1 -- 172.21.15.143:0/3502954820 learned_addr learned my addr 172.21.15.143:0/3502954820 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44d200640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd44813e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44dc00640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd448141660 0x7fd44815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd447e00640 1 -- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 msgr2=0x7fd44813e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd447e00640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd44813e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd447e00640 1 -- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd448141660 msgr2=0x7fd44815d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd447e00640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd448141660 0x7fd44815d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd447e00640 1 -- 172.21.15.143:0/3502954820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd430009420 con 0x7fd4480ce560 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44d200640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd448071d00 0x7fd44813e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44dc00640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd448141660 0x7fd44815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd447e00640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4480ce560 0x7fd44813b340 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fd43000ebe0 tx=0x7fd43000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:11.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43000eee0 con 0x7fd4480ce560 2024-01-15T18:23:11.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43000cc30 con 0x7fd4480ce560 2024-01-15T18:23:11.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430013470 con 0x7fd4480ce560 2024-01-15T18:23:11.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44fc00640 1 -- 172.21.15.143:0/3502954820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd440009a70 con 0x7fd4480ce560 2024-01-15T18:23:11.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44fc00640 1 -- 172.21.15.143:0/3502954820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4480cd150 con 0x7fd4480ce560 2024-01-15T18:23:11.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd44fc00640 1 -- 172.21.15.143:0/3502954820 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4480ad6b0 con 0x7fd4480ce560 2024-01-15T18:23:11.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.147+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fd4300135d0 con 0x7fd4480ce560 2024-01-15T18:23:11.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.151+0000 7fd444c00640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd420067b80 0x7fd42006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:11.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.151+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd430083380 con 0x7fd4480ce560 2024-01-15T18:23:11.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.151+0000 7fd44d200640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd420067b80 0x7fd42006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:11.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.151+0000 7fd44d200640 1 --2- 172.21.15.143:0/3502954820 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd420067b80 0x7fd42006a040 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd438005890 tx=0x7fd438009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:11.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.155+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd430052010 con 0x7fd4480ce560 2024-01-15T18:23:11.325 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.335+0000 7fd44fc00640 1 -- 172.21.15.143:0/3502954820 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd44815a4e0 con 0x7fd420067b80 2024-01-15T18:23:11.333 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:11.343+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:11.333 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:11.343+0000 7fd444c00640 1 -- 172.21.15.143:0/3502954820 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd44815a4e0 con 0x7fd420067b80 2024-01-15T18:23:11.333 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:11.334 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:11.341 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:13.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.503+0000 7ff691600640 1 Processor -- start 2024-01-15T18:23:13.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.503+0000 7ff691600640 1 -- start start 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.503+0000 7ff691600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c0ae620 0x7ff68c15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.503+0000 7ff691600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c15e650 con 0x7ff68c0ae240 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff691600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c0ad900 con 0x7ff68c06aba0 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff691600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c160d80 con 0x7ff68c0ae620 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff68aa00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff68c0ae240 0x7ff68c06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38370/0 (socket says 172.21.15.143:38370) 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff68aa00640 1 -- 172.21.15.143:0/2770155170 learned_addr learned my addr 172.21.15.143:0/2770155170 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff68a000640 1 --2- 172.21.15.143:0/2770155170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c0ae620 0x7ff68c15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804101072 0 0) 0x7ff68c15e650 con 0x7ff68c0ae240 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c003530 con 0x7ff68c0ae240 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200564287 0 0) 0x7ff68c0ad900 con 0x7ff68c06aba0 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff68c15e650 con 0x7ff68c06aba0 2024-01-15T18:23:13.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357545846 0 0) 0x7ff66c003530 con 0x7ff68c0ae240 2024-01-15T18:23:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff68a000640 1 -- 172.21.15.143:0/2770155170 >> v1:172.21.15.204:6789/0 conn(0x7ff68c06aba0 legacy=0x7ff68c15de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff68a000640 1 -- 172.21.15.143:0/2770155170 >> v1:172.21.15.143:6789/0 conn(0x7ff68c0ae240 legacy=0x7ff68c06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:13.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.507+0000 7ff68a000640 1 -- 172.21.15.143:0/2770155170 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c161060 con 0x7ff68c0ae620 2024-01-15T18:23:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff68a000640 1 --2- 172.21.15.143:0/2770155170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c0ae620 0x7ff68c15d8e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff6740097f0 tx=0x7ff67401c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4ea551d7588542 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c001680 con 0x7ff68c0ae620 2024-01-15T18:23:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740a7050 con 0x7ff68c0ae620 2024-01-15T18:23:13.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff689600640 1 -- 172.21.15.143:0/2770155170 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740ab600 con 0x7ff68c0ae620 2024-01-15T18:23:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 -- 172.21.15.143:0/2770155170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c0ae620 msgr2=0x7ff68c15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:13.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 --2- 172.21.15.143:0/2770155170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c0ae620 0x7ff68c15d8e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff6740097f0 tx=0x7ff67401c3b0 comp rx=0 tx=0).stop 2024-01-15T18:23:13.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 -- 172.21.15.143:0/2770155170 shutdown_connections 2024-01-15T18:23:13.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 --2- 172.21.15.143:0/2770155170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c0ae620 0x7ff68c15d8e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:13.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 -- 172.21.15.143:0/2770155170 >> 172.21.15.143:0/2770155170 conn(0x7ff68c0cfd20 msgr2=0x7ff68c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:13.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 -- 172.21.15.143:0/2770155170 shutdown_connections 2024-01-15T18:23:13.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 -- 172.21.15.143:0/2770155170 wait complete. 2024-01-15T18:23:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 Processor -- start 2024-01-15T18:23:13.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.511+0000 7ff691600640 1 -- start start 2024-01-15T18:23:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c06aba0 0x7ff68c14b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68c0ae240 0x7ff68c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68c0c6630 0x7ff68c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c14beb0 con 0x7ff68c0ae240 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c14b210 con 0x7ff68c0c6630 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c160130 con 0x7ff68c06aba0 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68c0ae240 0x7ff68c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68c0ae240 0x7ff68c157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36602/0 (socket says 172.21.15.143:36602) 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 -- 172.21.15.143:0/3154330900 learned_addr learned my addr 172.21.15.143:0/3154330900 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68aa00640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c06aba0 0x7ff68c14b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68b400640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68c0c6630 0x7ff68c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 -- 172.21.15.143:0/3154330900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c06aba0 msgr2=0x7ff68c14b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff68c06aba0 0x7ff68c14b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 -- 172.21.15.143:0/3154330900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68c0c6630 msgr2=0x7ff68c14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff68c0c6630 0x7ff68c14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 -- 172.21.15.143:0/3154330900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674009420 con 0x7ff68c0ae240 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff68a000640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff68c0ae240 0x7ff68c157210 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7ff674019fe0 tx=0x7ff674019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674003a30 con 0x7ff68c0ae240 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674003b90 con 0x7ff68c0ae240 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 -- 172.21.15.143:0/3154330900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c15f890 con 0x7ff68c0ae240 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740b5410 con 0x7ff68c0ae240 2024-01-15T18:23:13.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 -- 172.21.15.143:0/3154330900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c15fe50 con 0x7ff68c0ae240 2024-01-15T18:23:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.515+0000 7ff691600640 1 -- 172.21.15.143:0/3154330900 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c067500 con 0x7ff68c0ae240 2024-01-15T18:23:13.509 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.519+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7ff6740b2050 con 0x7ff68c0ae240 2024-01-15T18:23:13.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.519+0000 7ff682a00640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff664067b80 0x7ff66406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:13.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.519+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7ff67411d290 con 0x7ff68c0ae240 2024-01-15T18:23:13.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.519+0000 7ff68aa00640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff664067b80 0x7ff66406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:13.510 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.519+0000 7ff68aa00640 1 --2- 172.21.15.143:0/3154330900 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff664067b80 0x7ff66406a040 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff67c000fe0 tx=0x7ff67c0016a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:13.513 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.523+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff6740ebfb0 con 0x7ff68c0ae240 2024-01-15T18:23:13.703 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.711+0000 7ff691600640 1 -- 172.21.15.143:0/3154330900 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff68c152b00 con 0x7ff664067b80 2024-01-15T18:23:13.710 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:13.719+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:13.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:13.719+0000 7ff682a00640 1 -- 172.21.15.143:0/3154330900 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff68c152b00 con 0x7ff664067b80 2024-01-15T18:23:13.710 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:13.711 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:13.718 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:15.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:15.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 Processor -- start 2024-01-15T18:23:15.873 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 -- start start 2024-01-15T18:23:15.875 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f754015d130 0x7f75400aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f754015d510 0x7f754006e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754006a490 0x7f754006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754006ae10 con 0x7f754015d510 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7540160eb0 con 0x7f754015d130 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f7546200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7540161190 con 0x7f754006a490 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f754015d510 0x7f754006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f754015d510 0x7f754006e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36616/0 (socket says 172.21.15.143:36616) 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 -- 172.21.15.143:0/14180207 learned_addr learned my addr 172.21.15.143:0/14180207 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:15.876 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753f400640 1 --2- 172.21.15.143:0/14180207 >> v2:172.21.15.204:3300/0 conn(0x7f754015d130 0x7f75400aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753fe00640 1 --2- 172.21.15.143:0/14180207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754006a490 0x7f754006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 -- 172.21.15.143:0/14180207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754006a490 msgr2=0x7f754006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 --2- 172.21.15.143:0/14180207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754006a490 0x7f754006a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 -- 172.21.15.143:0/14180207 >> v2:172.21.15.204:3300/0 conn(0x7f754015d130 msgr2=0x7f75400aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 --2- 172.21.15.143:0/14180207 >> v2:172.21.15.204:3300/0 conn(0x7f754015d130 0x7f75400aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.877 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.883+0000 7f753ea00640 1 -- 172.21.15.143:0/14180207 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7540161470 con 0x7f754015d510 2024-01-15T18:23:15.878 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f753ea00640 1 --2- 172.21.15.143:0/14180207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f754006e060 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f754006b550 tx=0x7f752801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1155cdc813431 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f753e000640 1 -- 172.21.15.143:0/14180207 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280aa640 con 0x7f754015d510 2024-01-15T18:23:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f753e000640 1 -- 172.21.15.143:0/14180207 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75280aa7a0 con 0x7f754015d510 2024-01-15T18:23:15.879 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f753e000640 1 -- 172.21.15.143:0/14180207 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75280aab30 con 0x7f754015d510 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 -- 172.21.15.143:0/14180207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 msgr2=0x7f754006e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 --2- 172.21.15.143:0/14180207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f754006e060 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f754006b550 tx=0x7f752801c500 comp rx=0 tx=0).stop 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 -- 172.21.15.143:0/14180207 shutdown_connections 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 --2- 172.21.15.143:0/14180207 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754006a490 0x7f754006a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 --2- 172.21.15.143:0/14180207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f754006e060 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 --2- 172.21.15.143:0/14180207 >> v2:172.21.15.204:3300/0 conn(0x7f754015d130 0x7f75400aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.880 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 -- 172.21.15.143:0/14180207 >> 172.21.15.143:0/14180207 conn(0x7f75400b1b90 msgr2=0x7f75400b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 -- 172.21.15.143:0/14180207 shutdown_connections 2024-01-15T18:23:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.887+0000 7f7546200640 1 -- 172.21.15.143:0/14180207 wait complete. 2024-01-15T18:23:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 Processor -- start 2024-01-15T18:23:15.881 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 -- start start 2024-01-15T18:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754006a490 0x7f754014eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754015d130 0x7f7540154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f7540155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754015c0a0 con 0x7f754015d510 2024-01-15T18:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754014f2e0 con 0x7f754006a490 2024-01-15T18:23:15.884 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7540154da0 con 0x7f754015d130 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f7540155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f7540155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36624/0 (socket says 172.21.15.143:36624) 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 -- 172.21.15.143:0/265046610 learned_addr learned my addr 172.21.15.143:0/265046610 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753f400640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754006a490 0x7f754014eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753ea00640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754015d130 0x7f7540154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 -- 172.21.15.143:0/265046610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754015d130 msgr2=0x7f7540154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f754015d130 0x7f7540154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.885 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 -- 172.21.15.143:0/265046610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754006a490 msgr2=0x7f754014eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f754006a490 0x7f754014eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 -- 172.21.15.143:0/265046610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280094d0 con 0x7f754015d510 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f753fe00640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f754015d510 0x7f7540155270 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7538007e60 tx=0x7f753800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7538014620 con 0x7f754015d510 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753800ee90 con 0x7f754015d510 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75380149e0 con 0x7f754015d510 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 -- 172.21.15.143:0/265046610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754015b2e0 con 0x7f754015d510 2024-01-15T18:23:15.886 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.891+0000 7f7546200640 1 -- 172.21.15.143:0/265046610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754015b8a0 con 0x7f754015d510 2024-01-15T18:23:15.887 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.895+0000 7f7546200640 1 -- 172.21.15.143:0/265046610 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7540067500 con 0x7f754015d510 2024-01-15T18:23:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.895+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f7538014b40 con 0x7f754015d510 2024-01-15T18:23:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.895+0000 7f7537400640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7518067b80 0x7f751806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.895+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f7538083bc0 con 0x7f754015d510 2024-01-15T18:23:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.895+0000 7f753f400640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7518067b80 0x7f751806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:15.888 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.895+0000 7f753f400640 1 --2- 172.21.15.143:0/265046610 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f7518067b80 0x7f751806a040 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7530007a60 tx=0x7f7530008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:15.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:15.899+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f75380524f0 con 0x7f754015d510 2024-01-15T18:23:16.070 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:16.079+0000 7f7546200640 1 -- 172.21.15.143:0/265046610 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75400af7e0 con 0x7f7518067b80 2024-01-15T18:23:16.078 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:16.087+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:16.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:16.087+0000 7f7537400640 1 -- 172.21.15.143:0/265046610 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f75400af7e0 con 0x7f7518067b80 2024-01-15T18:23:16.078 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:16.079 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:16.086 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:18.088 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:18.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 Processor -- start 2024-01-15T18:23:18.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 -- start start 2024-01-15T18:23:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00f8071bb0 0x7f00f8072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f80ceb50 con 0x7f00f8143780 2024-01-15T18:23:18.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f8072570 con 0x7f00f8071bb0 2024-01-15T18:23:18.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00fd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f815c110 con 0x7f00f815c5a0 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00f8071bb0 0x7f00f8072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f00f8071bb0 0x7f00f8072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35142/0 (socket says 172.21.15.143:35142) 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 -- 172.21.15.143:0/1661683772 learned_addr learned my addr 172.21.15.143:0/1661683772 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f6800640 1 --2- 172.21.15.143:0/1661683772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 -- 172.21.15.143:0/1661683772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 msgr2=0x7f00f815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 --2- 172.21.15.143:0/1661683772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f815ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 -- 172.21.15.143:0/1661683772 >> v1:172.21.15.204:6789/0 conn(0x7f00f8143780 legacy=0x7f00f80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.251+0000 7f00f5400640 1 -- 172.21.15.143:0/1661683772 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f815b7e0 con 0x7f00f8071bb0 2024-01-15T18:23:18.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00f5400640 1 --2- 172.21.15.143:0/1661683772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8071bb0 0x7f00f8072030 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009870 tx=0x7f00e001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9766d3b9624c74f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:18.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00f4a00640 1 -- 172.21.15.143:0/1661683772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00a9400 con 0x7f00f8071bb0 2024-01-15T18:23:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00f4a00640 1 -- 172.21.15.143:0/1661683772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e00a9560 con 0x7f00f8071bb0 2024-01-15T18:23:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00f4a00640 1 -- 172.21.15.143:0/1661683772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00a98f0 con 0x7f00f8071bb0 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 -- 172.21.15.143:0/1661683772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8071bb0 msgr2=0x7f00f8072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 --2- 172.21.15.143:0/1661683772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8071bb0 0x7f00f8072030 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009870 tx=0x7f00e001c540 comp rx=0 tx=0).stop 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 -- 172.21.15.143:0/1661683772 shutdown_connections 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 --2- 172.21.15.143:0/1661683772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 --2- 172.21.15.143:0/1661683772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8071bb0 0x7f00f8072030 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 -- 172.21.15.143:0/1661683772 >> 172.21.15.143:0/1661683772 conn(0x7f00f806c3f0 msgr2=0x7f00f806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.255+0000 7f00fd200640 1 -- 172.21.15.143:0/1661683772 shutdown_connections 2024-01-15T18:23:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 -- 172.21.15.143:0/1661683772 wait complete. 2024-01-15T18:23:18.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 Processor -- start 2024-01-15T18:23:18.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 -- start start 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00f8071bb0 0x7f00f8142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8143780 0x7f00f806db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f813edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f813fbf0 con 0x7f00f8143780 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f813f910 con 0x7f00f8071bb0 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f81342a0 con 0x7f00f815c5a0 2024-01-15T18:23:18.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8143780 0x7f00f806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8143780 0x7f00f806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35158/0 (socket says 172.21.15.143:35158) 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 -- 172.21.15.143:0/3043215160 learned_addr learned my addr 172.21.15.143:0/3043215160 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f6800640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f813edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5e00640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00f8071bb0 0x7f00f8142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 -- 172.21.15.143:0/3043215160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 msgr2=0x7f00f813edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f00f815c5a0 0x7f00f813edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 -- 172.21.15.143:0/3043215160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00f8071bb0 msgr2=0x7f00f8142c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f00f8071bb0 0x7f00f8142c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 -- 172.21.15.143:0/3043215160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e00094d0 con 0x7f00f8143780 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00f5400640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f00f8143780 0x7f00f806db60 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f00e001abf0 tx=0x7f00e001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:18.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00a9400 con 0x7f00f8143780 2024-01-15T18:23:18.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e00b1800 con 0x7f00f8143780 2024-01-15T18:23:18.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00b1bb0 con 0x7f00f8143780 2024-01-15T18:23:18.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.259+0000 7f00fd200640 1 -- 172.21.15.143:0/3043215160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f8134580 con 0x7f00f8143780 2024-01-15T18:23:18.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00fd200640 1 -- 172.21.15.143:0/3043215160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f813f330 con 0x7f00f8143780 2024-01-15T18:23:18.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00fd200640 1 -- 172.21.15.143:0/3043215160 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f8134100 con 0x7f00f8143780 2024-01-15T18:23:18.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f00e00b1d10 con 0x7f00f8143780 2024-01-15T18:23:18.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00ee000640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00d0067b80 0x7f00d006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:18.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f00e011d900 con 0x7f00f8143780 2024-01-15T18:23:18.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00f5e00640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00d0067b80 0x7f00d006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:18.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.263+0000 7f00f5e00640 1 --2- 172.21.15.143:0/3043215160 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f00d0067b80 0x7f00d006a040 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f00e8007fd0 tx=0x7f00e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:18.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.267+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f00f8134100 con 0x7f00f8143780 2024-01-15T18:23:18.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.447+0000 7f00fd200640 1 -- 172.21.15.143:0/3043215160 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00f8166ac0 con 0x7f00d0067b80 2024-01-15T18:23:18.443 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:18.451+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:18.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:18.451+0000 7f00ee000640 1 -- 172.21.15.143:0/3043215160 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f00e80014f0 con 0x7f00d0067b80 2024-01-15T18:23:18.443 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:18.444 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:18.452 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:20.453 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.615+0000 7fc73d400640 1 Processor -- start 2024-01-15T18:23:20.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.615+0000 7fc73d400640 1 -- start start 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc73d400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc738143780 0x7fc7380cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc73d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc73815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc73d400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7380ceb50 con 0x7fc7380ce520 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc73d400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73815c110 con 0x7fc738143780 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc73d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73815b7e0 con 0x7fc7380722c0 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc7380ce520 0x7fc738071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37564/0 (socket says 172.21.15.143:37564) 2024-01-15T18:23:20.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735e00640 1 -- 172.21.15.143:0/2369285121 learned_addr learned my addr 172.21.15.143:0/2369285121 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:20.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc736800640 1 --2- 172.21.15.143:0/2369285121 >> v2:172.21.15.204:3300/0 conn(0x7fc738143780 0x7fc7380cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc73815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568792212 0 0) 0x7fc720001120 con 0x7fc7380ce520 2024-01-15T18:23:20.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc718003530 con 0x7fc7380ce520 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840723132 0 0) 0x7fc718003530 con 0x7fc7380ce520 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 msgr2=0x7fc73815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc73815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 >> v2:172.21.15.204:3300/0 conn(0x7fc738143780 msgr2=0x7fc7380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 --2- 172.21.15.143:0/2369285121 >> v2:172.21.15.204:3300/0 conn(0x7fc738143780 0x7fc7380cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73815bac0 con 0x7fc7380ce520 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fc720001810 con 0x7fc7380ce520 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 >> v1:172.21.15.143:6789/0 conn(0x7fc7380ce520 legacy=0x7fc738071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc718003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718003f00 0x7fc7180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc718004d10 0x7fc718005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73815c110 con 0x7fc718004d10 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73815b7e0 con 0x7fc718003f00 2024-01-15T18:23:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71800b990 con 0x7fc7380722c0 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc718004d10 0x7fc718005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc736800640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc718003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735e00640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718003f00 0x7fc7180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 -- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 msgr2=0x7fc718003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc718003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 -- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718003f00 msgr2=0x7fc7180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718003f00 0x7fc7180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc737200640 1 -- 172.21.15.143:0/2369285121 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718003530 con 0x7fc718004d10 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc736800640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380722c0 0x7fc718003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:20.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.619+0000 7fc735e00640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718003f00 0x7fc7180047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc737200640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc718004d10 0x7fc718005190 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc730009ab0 tx=0x7fc73001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10c1eebe1567704 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7300a5620 con 0x7fc718004d10 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7300a5780 con 0x7fc718004d10 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc735400640 1 -- 172.21.15.143:0/2369285121 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7300a5b30 con 0x7fc718004d10 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 -- 172.21.15.143:0/2369285121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc718004d10 msgr2=0x7fc718005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 --2- 172.21.15.143:0/2369285121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc718004d10 0x7fc718005190 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc730009ab0 tx=0x7fc73001c9d0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc737200640 1 -- 172.21.15.143:0/2369285121 reap_dead start 2024-01-15T18:23:20.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 -- 172.21.15.143:0/2369285121 shutdown_connections 2024-01-15T18:23:20.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 -- 172.21.15.143:0/2369285121 >> 172.21.15.143:0/2369285121 conn(0x7fc73806c3f0 msgr2=0x7fc73806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:20.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 -- 172.21.15.143:0/2369285121 shutdown_connections 2024-01-15T18:23:20.615 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 -- 172.21.15.143:0/2369285121 wait complete. 2024-01-15T18:23:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.623+0000 7fc73d400640 1 Processor -- start 2024-01-15T18:23:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- start start 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718004d10 0x7fc73813f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380c7d30 0x7fc7380c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc738141980 0x7fc738141e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738142340 con 0x7fc738141980 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381f64f0 con 0x7fc718004d10 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7381f7350 con 0x7fc7380c7d30 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc738141980 0x7fc738141e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc738141980 0x7fc738141e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35168/0 (socket says 172.21.15.143:35168) 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 -- 172.21.15.143:0/4157240654 learned_addr learned my addr 172.21.15.143:0/4157240654 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc736800640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718004d10 0x7fc73813f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc735e00640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380c7d30 0x7fc7380c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 -- 172.21.15.143:0/4157240654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380c7d30 msgr2=0x7fc7380c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc7380c7d30 0x7fc7380c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.619 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 -- 172.21.15.143:0/4157240654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718004d10 msgr2=0x7fc73813f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc718004d10 0x7fc73813f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 -- 172.21.15.143:0/4157240654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73813f730 con 0x7fc738141980 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc737200640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc738141980 0x7fc738141e00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc7300b0040 tx=0x7fc718005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730004230 con 0x7fc738141980 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7300043b0 con 0x7fc738141980 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- 172.21.15.143:0/4157240654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7381f6740 con 0x7fc738141980 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730003860 con 0x7fc738141980 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- 172.21.15.143:0/4157240654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7381f6d00 con 0x7fc738141980 2024-01-15T18:23:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.627+0000 7fc73d400640 1 -- 172.21.15.143:0/4157240654 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738067500 con 0x7fc738141980 2024-01-15T18:23:20.621 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.631+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fc7300b4070 con 0x7fc738141980 2024-01-15T18:23:20.622 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.631+0000 7fc72ea00640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc710067bb0 0x7fc71006a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:20.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.631+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fc73011cbf0 con 0x7fc738141980 2024-01-15T18:23:20.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.631+0000 7fc736800640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc710067bb0 0x7fc71006a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:20.623 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.631+0000 7fc736800640 1 --2- 172.21.15.143:0/4157240654 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc710067bb0 0x7fc71006a070 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fc728007540 tx=0x7fc72800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:20.627 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.635+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fc7300eb5b0 con 0x7fc738141980 2024-01-15T18:23:20.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.815+0000 7fc73d400640 1 -- 172.21.15.143:0/4157240654 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7380b4ce0 con 0x7fc710067bb0 2024-01-15T18:23:20.811 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:20.819+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:20.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:20.819+0000 7fc72ea00640 1 -- 172.21.15.143:0/4157240654 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc72800f040 con 0x7fc710067bb0 2024-01-15T18:23:20.811 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:20.812 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:20.819 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:22.820 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 Processor -- start 2024-01-15T18:23:22.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 -- start start 2024-01-15T18:23:22.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd7240ce690 0x7fd7240ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 0x7fd7240b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd724070bf0 con 0x7fd7240b3bf0 2024-01-15T18:23:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72415dba0 con 0x7fd7240ce690 2024-01-15T18:23:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b2740 con 0x7fd7240b1da0 2024-01-15T18:23:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72a600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd7240b3bf0 0x7fd7240b4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37574/0 (socket says 172.21.15.143:37574) 2024-01-15T18:23:22.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd72a600640 1 -- 172.21.15.143:0/2460379320 learned_addr learned my addr 172.21.15.143:0/2460379320 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:22.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd729c00640 1 --2- 172.21.15.143:0/2460379320 >> v2:172.21.15.204:3300/0 conn(0x7fd7240ce690 0x7fd7240ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489934154 0 0) 0x7fd724070bf0 con 0x7fd7240b3bf0 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd704003530 con 0x7fd7240b3bf0 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd729200640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 0x7fd7240b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367352240 0 0) 0x7fd704003530 con 0x7fd7240b3bf0 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 msgr2=0x7fd7240b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 0x7fd7240b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 >> v2:172.21.15.204:3300/0 conn(0x7fd7240ce690 msgr2=0x7fd7240ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 --2- 172.21.15.143:0/2460379320 >> v2:172.21.15.204:3300/0 conn(0x7fd7240ce690 0x7fd7240ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240b4920 con 0x7fd7240b3bf0 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fd71c001a00 con 0x7fd7240b3bf0 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 >> v1:172.21.15.143:6789/0 conn(0x7fd7240b3bf0 legacy=0x7fd7240b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:22.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd704003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.983+0000 7fd723e00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 0x7fd7040047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd723e00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd704004d10 0x7fd704005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72415dba0 con 0x7fd704004d10 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240b2740 con 0x7fd704003980 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70400b990 con 0x7fd7240b1da0 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd704003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd72a600640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd704004d10 0x7fd704005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729200640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 0x7fd7040047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.980 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 -- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 msgr2=0x7fd7040047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:22.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7240b1da0 0x7fd7040047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:22.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 -- 172.21.15.143:0/2460379320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd704004d10 msgr2=0x7fd704005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:22.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd704004d10 0x7fd704005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:22.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 -- 172.21.15.143:0/2460379320 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704003530 con 0x7fd704003980 2024-01-15T18:23:22.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd72a600640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd704004d10 0x7fd704005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:22.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.987+0000 7fd729c00640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd704003da0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd714009ae0 tx=0x7fd71401ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6abe11301dbe3d9c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:22.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140a5680 con 0x7fd704003980 2024-01-15T18:23:22.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7140a57e0 con 0x7fd704003980 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd723e00640 1 -- 172.21.15.143:0/2460379320 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7140a5b90 con 0x7fd704003980 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 -- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 msgr2=0x7fd704003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 --2- 172.21.15.143:0/2460379320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd704003da0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd714009ae0 tx=0x7fd71401ca90 comp rx=0 tx=0).stop 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72a600640 1 -- 172.21.15.143:0/2460379320 reap_dead start 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 -- 172.21.15.143:0/2460379320 shutdown_connections 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 -- 172.21.15.143:0/2460379320 >> 172.21.15.143:0/2460379320 conn(0x7fd72406c3f0 msgr2=0x7fd72406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 -- 172.21.15.143:0/2460379320 shutdown_connections 2024-01-15T18:23:22.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 -- 172.21.15.143:0/2460379320 wait complete. 2024-01-15T18:23:22.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 Processor -- start 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.991+0000 7fd72c600640 1 -- start start 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd72413afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd72413b650 0x7fd724140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72413e0b0 0x7fd7240afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72413ec90 con 0x7fd72413b650 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd724140990 con 0x7fd704003980 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72413e9b0 con 0x7fd72413e0b0 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd72413b650 0x7fd724140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd72413b650 0x7fd724140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35182/0 (socket says 172.21.15.143:35182) 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 -- 172.21.15.143:0/498478742 learned_addr learned my addr 172.21.15.143:0/498478742 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729c00640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd72413afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.987 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72a600640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72413e0b0 0x7fd7240afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 -- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72413e0b0 msgr2=0x7fd7240afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72413e0b0 0x7fd7240afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 -- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 msgr2=0x7fd72413afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd72413afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 -- 172.21.15.143:0/498478742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240add00 con 0x7fd72413b650 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729c00640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd704003980 0x7fd72413afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72a600640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd72413e0b0 0x7fd7240afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd729200640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd72413b650 0x7fd724140450 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd70c011a00 tx=0x7fd70c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00fb50 con 0x7fd72413b650 2024-01-15T18:23:22.988 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00fcb0 con 0x7fd72413b650 2024-01-15T18:23:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c014440 con 0x7fd72413b650 2024-01-15T18:23:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 -- 172.21.15.143:0/498478742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7240adfe0 con 0x7fd72413b650 2024-01-15T18:23:22.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 -- 172.21.15.143:0/498478742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72413e560 con 0x7fd72413b650 2024-01-15T18:23:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd72c600640 1 -- 172.21.15.143:0/498478742 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7240adb60 con 0x7fd72413b650 2024-01-15T18:23:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.995+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fd70c0145a0 con 0x7fd72413b650 2024-01-15T18:23:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.999+0000 7fd721600640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6fc067bb0 0x7fd6fc06a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:22.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.999+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd70c083cd0 con 0x7fd72413b650 2024-01-15T18:23:22.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.999+0000 7fd729c00640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6fc067bb0 0x7fd6fc06a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:22.991 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:22.999+0000 7fd729c00640 1 --2- 172.21.15.143:0/498478742 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd6fc067bb0 0x7fd6fc06a070 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fd7140af000 tx=0x7fd7140046d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:22.993 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:23.003+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd7240adb60 con 0x7fd72413b650 2024-01-15T18:23:23.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:23.183+0000 7fd72c600640 1 -- 172.21.15.143:0/498478742 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7240cea70 con 0x7fd6fc067bb0 2024-01-15T18:23:23.178 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:23.187+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:23.179 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:23.187+0000 7fd721600640 1 -- 172.21.15.143:0/498478742 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd714004020 con 0x7fd6fc067bb0 2024-01-15T18:23:23.179 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:23.179 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:23.187 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:25.189 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:25.341 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 Processor -- start 2024-01-15T18:23:25.342 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 -- start start 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6458157300 0x7f6458159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64580b21d0 0x7f64580b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f64580b48f0 0x7f645815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64580b35a0 con 0x7f6458157300 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64580b4d80 con 0x7f64580b48f0 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580b2a20 con 0x7f64580b21d0 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6458157300 0x7f6458159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6458157300 0x7f6458159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35184/0 (socket says 172.21.15.143:35184) 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 -- 172.21.15.143:0/2507964045 learned_addr learned my addr 172.21.15.143:0/2507964045 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 -- 172.21.15.143:0/2507964045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64580b21d0 msgr2=0x7f64580b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 --2- 172.21.15.143:0/2507964045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64580b21d0 0x7f64580b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 -- 172.21.15.143:0/2507964045 >> v2:172.21.15.204:3300/0 conn(0x7f64580b48f0 msgr2=0x7f645815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645d200640 1 --2- 172.21.15.143:0/2507964045 >> v2:172.21.15.204:3300/0 conn(0x7f64580b48f0 0x7f645815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 --2- 172.21.15.143:0/2507964045 >> v2:172.21.15.204:3300/0 conn(0x7f64580b48f0 0x7f645815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f6457e00640 1 -- 172.21.15.143:0/2507964045 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580b2d00 con 0x7f6458157300 2024-01-15T18:23:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.351+0000 7f645d200640 1 --2- 172.21.15.143:0/2507964045 >> v2:172.21.15.204:3300/0 conn(0x7f64580b48f0 0x7f645815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:25.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f6457e00640 1 --2- 172.21.15.143:0/2507964045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458157300 0x7f6458159b70 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f645815a0b0 tx=0x7f644801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e06406900def26 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:25.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f6456a00640 1 -- 172.21.15.143:0/2507964045 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480aa5f0 con 0x7f6458157300 2024-01-15T18:23:25.347 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f6456a00640 1 -- 172.21.15.143:0/2507964045 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64480aa750 con 0x7f6458157300 2024-01-15T18:23:25.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f6456a00640 1 -- 172.21.15.143:0/2507964045 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480aaae0 con 0x7f6458157300 2024-01-15T18:23:25.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 -- 172.21.15.143:0/2507964045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458157300 msgr2=0x7f6458159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:25.348 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 --2- 172.21.15.143:0/2507964045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458157300 0x7f6458159b70 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f645815a0b0 tx=0x7f644801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 -- 172.21.15.143:0/2507964045 shutdown_connections 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 --2- 172.21.15.143:0/2507964045 >> v2:172.21.15.204:3300/0 conn(0x7f64580b48f0 0x7f645815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 --2- 172.21.15.143:0/2507964045 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f64580b21d0 0x7f64580b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 --2- 172.21.15.143:0/2507964045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6458157300 0x7f6458159b70 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 -- 172.21.15.143:0/2507964045 >> 172.21.15.143:0/2507964045 conn(0x7f645806c3f0 msgr2=0x7f645806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.355+0000 7f645f200640 1 -- 172.21.15.143:0/2507964045 shutdown_connections 2024-01-15T18:23:25.349 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 -- 172.21.15.143:0/2507964045 wait complete. 2024-01-15T18:23:25.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 Processor -- start 2024-01-15T18:23:25.350 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 -- start start 2024-01-15T18:23:25.352 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64580b21d0 0x7f64580ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64580b48f0 0x7f6458150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6458157300 0x7f6458150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458146b20 con 0x7f64580b21d0 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580ace20 con 0x7f64580b48f0 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645f200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64580ad0a0 con 0x7f6458157300 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64580b21d0 0x7f64580ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64580b21d0 0x7f64580ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35186/0 (socket says 172.21.15.143:35186) 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 -- 172.21.15.143:0/2118592598 learned_addr learned my addr 172.21.15.143:0/2118592598 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645d200640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6458157300 0x7f6458150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457400640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64580b48f0 0x7f6458150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 -- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6458157300 msgr2=0x7f6458150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6458157300 0x7f6458150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 -- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64580b48f0 msgr2=0x7f6458150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64580b48f0 0x7f6458150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:25.353 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 -- 172.21.15.143:0/2118592598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64480094d0 con 0x7f64580b21d0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f645d200640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f6458157300 0x7f6458150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457400640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64580b48f0 0x7f6458150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.359+0000 7f6457e00640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64580b21d0 0x7f64580ac8e0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f644801ab80 tx=0x7f644801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480b3800 con 0x7f64580b21d0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64480b3980 con 0x7f64580b21d0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f645f200640 1 -- 172.21.15.143:0/2118592598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64580ad380 con 0x7f64580b21d0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480b3d30 con 0x7f64580b21d0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f645f200640 1 -- 172.21.15.143:0/2118592598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64580ad940 con 0x7f64580b21d0 2024-01-15T18:23:25.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f645f200640 1 -- 172.21.15.143:0/2118592598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6458146c90 con 0x7f64580b21d0 2024-01-15T18:23:25.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f64480c6390 con 0x7f64580b21d0 2024-01-15T18:23:25.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f644fe00640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6430067b80 0x7f643006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f64480a7070 con 0x7f64580b21d0 2024-01-15T18:23:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.363+0000 7f6457400640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6430067b80 0x7f643006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:25.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.367+0000 7f6457400640 1 --2- 172.21.15.143:0/2118592598 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6430067b80 0x7f643006a040 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6440006fd0 tx=0x7f6440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:25.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.367+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f64480ec6e0 con 0x7f64580b21d0 2024-01-15T18:23:25.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.547+0000 7f645f200640 1 -- 172.21.15.143:0/2118592598 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64580adb30 con 0x7f6430067b80 2024-01-15T18:23:25.543 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:25.551+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:25.543 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:25.551+0000 7f644fe00640 1 -- 172.21.15.143:0/2118592598 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f64580adb30 con 0x7f6430067b80 2024-01-15T18:23:25.544 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:25.544 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:25.552 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:27.553 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:27.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.711+0000 7ff459200640 1 Processor -- start 2024-01-15T18:23:27.704 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.711+0000 7ff459200640 1 -- start start 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff459200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 0x7ff4540ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff459200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff45415ec80 0x7ff454141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff459200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff45415f310 con 0x7ff454142470 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff459200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45415fbb0 con 0x7ff45415ec80 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff459200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415f100 con 0x7ff454141660 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 0x7ff4540ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:27.706 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff45415ec80 0x7ff454141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 0x7ff4540ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:41912/0 (socket says 172.21.15.143:41912) 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7ff45415ec80 0x7ff454141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48194/0 (socket says 172.21.15.143:48194) 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 -- 172.21.15.143:0/120295609 learned_addr learned my addr 172.21.15.143:0/120295609 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff452800640 1 --1- 172.21.15.143:0/120295609 >> v1:172.21.15.204:6789/0 conn(0x7ff454142470 0x7ff454071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43774/0 (socket says 172.21.15.143:43774) 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 -- 172.21.15.143:0/120295609 >> v2:172.21.15.204:3300/0 conn(0x7ff45415ec80 msgr2=0x7ff454141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 --2- 172.21.15.143:0/120295609 >> v2:172.21.15.204:3300/0 conn(0x7ff45415ec80 0x7ff454141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 -- 172.21.15.143:0/120295609 >> v1:172.21.15.204:6789/0 conn(0x7ff454142470 legacy=0x7ff454071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:27.707 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.715+0000 7ff451e00640 1 -- 172.21.15.143:0/120295609 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45415f5f0 con 0x7ff454141660 2024-01-15T18:23:27.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff451e00640 1 --2- 172.21.15.143:0/120295609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 0x7ff4540ce560 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff444009870 tx=0x7ff44401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d3450da9692fb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:27.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff450a00640 1 -- 172.21.15.143:0/120295609 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440aa670 con 0x7ff454141660 2024-01-15T18:23:27.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff450a00640 1 -- 172.21.15.143:0/120295609 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4440aa7d0 con 0x7ff454141660 2024-01-15T18:23:27.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff450a00640 1 -- 172.21.15.143:0/120295609 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440aab60 con 0x7ff454141660 2024-01-15T18:23:27.710 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 -- 172.21.15.143:0/120295609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 msgr2=0x7ff4540ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 --2- 172.21.15.143:0/120295609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 0x7ff4540ce560 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff444009870 tx=0x7ff44401c510 comp rx=0 tx=0).stop 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 -- 172.21.15.143:0/120295609 shutdown_connections 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 --2- 172.21.15.143:0/120295609 >> v2:172.21.15.204:3300/0 conn(0x7ff45415ec80 0x7ff454141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 --2- 172.21.15.143:0/120295609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff454141660 0x7ff4540ce560 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 -- 172.21.15.143:0/120295609 >> 172.21.15.143:0/120295609 conn(0x7ff45406c3f0 msgr2=0x7ff45406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 -- 172.21.15.143:0/120295609 shutdown_connections 2024-01-15T18:23:27.711 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.719+0000 7ff459200640 1 -- 172.21.15.143:0/120295609 wait complete. 2024-01-15T18:23:27.712 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 Processor -- start 2024-01-15T18:23:27.713 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- start start 2024-01-15T18:23:27.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff454141660 0x7ff45415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff454142470 0x7ff45415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff45415ec80 0x7ff45415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415e260 con 0x7ff454141660 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415d3a0 con 0x7ff454142470 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45415d650 con 0x7ff45415ec80 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff454141660 0x7ff45415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff454141660 0x7ff45415a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52198/0 (socket says 172.21.15.143:52198) 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 -- 172.21.15.143:0/1902417321 learned_addr learned my addr 172.21.15.143:0/1902417321 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451400640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff454142470 0x7ff45415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff452800640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff45415ec80 0x7ff45415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 -- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff45415ec80 msgr2=0x7ff45415ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff45415ec80 0x7ff45415ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 -- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff454142470 msgr2=0x7ff45415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:27.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff454142470 0x7ff45415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:27.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 -- 172.21.15.143:0/1902417321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440094d0 con 0x7ff454141660 2024-01-15T18:23:27.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff452800640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff45415ec80 0x7ff45415ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:27.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff451e00640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff454141660 0x7ff45415a1b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff4440ac040 tx=0x7ff44401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:27.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440b3800 con 0x7ff454141660 2024-01-15T18:23:27.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4440b3960 con 0x7ff454141660 2024-01-15T18:23:27.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440b3d10 con 0x7ff454141660 2024-01-15T18:23:27.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- 172.21.15.143:0/1902417321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4540cd150 con 0x7ff454141660 2024-01-15T18:23:27.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- 172.21.15.143:0/1902417321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45415dc10 con 0x7ff454141660 2024-01-15T18:23:27.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.723+0000 7ff459200640 1 -- 172.21.15.143:0/1902417321 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45415e6d0 con 0x7ff454141660 2024-01-15T18:23:27.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.727+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7ff4440b2380 con 0x7ff454141660 2024-01-15T18:23:27.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.727+0000 7ff44a000640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff42c067bb0 0x7ff42c06a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:27.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.727+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7ff44411d7e0 con 0x7ff454141660 2024-01-15T18:23:27.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.727+0000 7ff451400640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff42c067bb0 0x7ff42c06a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:27.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.727+0000 7ff451400640 1 --2- 172.21.15.143:0/1902417321 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff42c067bb0 0x7ff42c06a070 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7ff4541f92a0 tx=0x7ff43c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:27.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.731+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff4440ec520 con 0x7ff454141660 2024-01-15T18:23:27.899 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.907+0000 7ff459200640 1 -- 172.21.15.143:0/1902417321 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4540cd970 con 0x7ff42c067bb0 2024-01-15T18:23:27.905 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:27.915+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:27.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:27.915+0000 7ff44a000640 1 -- 172.21.15.143:0/1902417321 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff4540cd970 con 0x7ff42c067bb0 2024-01-15T18:23:27.906 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:27.906 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:27.914 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:29.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:30.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.079+0000 7eff7fc00640 1 Processor -- start 2024-01-15T18:23:30.072 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.079+0000 7eff7fc00640 1 -- start start 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff78143780 0x7eff780cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7fc00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff780ce520 0x7eff78071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7fc00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7eff780720f0 0x7eff7815a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7fc00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff78072580 con 0x7eff780ce520 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7fc00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780ceb50 con 0x7eff780720f0 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815c110 con 0x7eff78143780 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff780ce520 0x7eff78071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.074 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7eff780ce520 0x7eff78071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52202/0 (socket says 172.21.15.143:52202) 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 -- 172.21.15.143:0/2149048606 learned_addr learned my addr 172.21.15.143:0/2149048606 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7d200640 1 --2- 172.21.15.143:0/2149048606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff78143780 0x7eff780cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff7dc00640 1 --2- 172.21.15.143:0/2149048606 >> v2:172.21.15.204:3300/0 conn(0x7eff780720f0 0x7eff7815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 -- 172.21.15.143:0/2149048606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff78143780 msgr2=0x7eff780cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 --2- 172.21.15.143:0/2149048606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff78143780 0x7eff780cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 -- 172.21.15.143:0/2149048606 >> v2:172.21.15.204:3300/0 conn(0x7eff780720f0 msgr2=0x7eff7815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 --2- 172.21.15.143:0/2149048606 >> v2:172.21.15.204:3300/0 conn(0x7eff780720f0 0x7eff7815a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.083+0000 7eff77e00640 1 -- 172.21.15.143:0/2149048606 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7815b7e0 con 0x7eff780ce520 2024-01-15T18:23:30.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff77e00640 1 --2- 172.21.15.143:0/2149048606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff780ce520 0x7eff78071bb0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7eff780b1da0 tx=0x7eff6001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d325db3368202076 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:30.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff77400640 1 -- 172.21.15.143:0/2149048606 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600ac630 con 0x7eff780ce520 2024-01-15T18:23:30.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff77400640 1 -- 172.21.15.143:0/2149048606 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff600ac790 con 0x7eff780ce520 2024-01-15T18:23:30.077 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff77400640 1 -- 172.21.15.143:0/2149048606 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff600acb20 con 0x7eff780ce520 2024-01-15T18:23:30.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 -- 172.21.15.143:0/2149048606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff780ce520 msgr2=0x7eff78071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:30.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 --2- 172.21.15.143:0/2149048606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff780ce520 0x7eff78071bb0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7eff780b1da0 tx=0x7eff6001c500 comp rx=0 tx=0).stop 2024-01-15T18:23:30.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 -- 172.21.15.143:0/2149048606 shutdown_connections 2024-01-15T18:23:30.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 --2- 172.21.15.143:0/2149048606 >> v2:172.21.15.204:3300/0 conn(0x7eff780720f0 0x7eff7815a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 --2- 172.21.15.143:0/2149048606 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff780ce520 0x7eff78071bb0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 --2- 172.21.15.143:0/2149048606 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff78143780 0x7eff780cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 -- 172.21.15.143:0/2149048606 >> 172.21.15.143:0/2149048606 conn(0x7eff7806c3f0 msgr2=0x7eff7806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:30.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 -- 172.21.15.143:0/2149048606 shutdown_connections 2024-01-15T18:23:30.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 -- 172.21.15.143:0/2149048606 wait complete. 2024-01-15T18:23:30.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 Processor -- start 2024-01-15T18:23:30.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.087+0000 7eff7fc00640 1 -- start start 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff780720f0 0x7eff781408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff780ce520 0x7eff78141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff78143780 0x7eff78141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7813f590 con 0x7eff78143780 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7813f870 con 0x7eff780720f0 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7813fb50 con 0x7eff780ce520 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff78143780 0x7eff78141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff78143780 0x7eff78141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52204/0 (socket says 172.21.15.143:52204) 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 -- 172.21.15.143:0/1751885137 learned_addr learned my addr 172.21.15.143:0/1751885137 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7d200640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff780720f0 0x7eff781408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff77e00640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff780ce520 0x7eff78141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 -- 172.21.15.143:0/1751885137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff780ce520 msgr2=0x7eff78141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7eff780ce520 0x7eff78141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 -- 172.21.15.143:0/1751885137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff780720f0 msgr2=0x7eff781408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff780720f0 0x7eff781408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 -- 172.21.15.143:0/1751885137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600094d0 con 0x7eff78143780 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7dc00640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff78143780 0x7eff78141c60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7eff70007d10 tx=0x7eff7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70014420 con 0x7eff78143780 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff700145a0 con 0x7eff78143780 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff70014950 con 0x7eff78143780 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 -- 172.21.15.143:0/1751885137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7813fe30 con 0x7eff78143780 2024-01-15T18:23:30.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 -- 172.21.15.143:0/1751885137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff78140e00 con 0x7eff78143780 2024-01-15T18:23:30.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.091+0000 7eff7fc00640 1 -- 172.21.15.143:0/1751885137 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff781410b0 con 0x7eff78143780 2024-01-15T18:23:30.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.095+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7eff70014ab0 con 0x7eff78143780 2024-01-15T18:23:30.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.095+0000 7eff74c00640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff50067b80 0x7eff5006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:30.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.095+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7eff70083bb0 con 0x7eff78143780 2024-01-15T18:23:30.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.095+0000 7eff7d200640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff50067b80 0x7eff5006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:30.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.095+0000 7eff7d200640 1 --2- 172.21.15.143:0/1751885137 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7eff50067b80 0x7eff5006a040 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7eff680079c0 tx=0x7eff68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:30.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.099+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7eff781410b0 con 0x7eff78143780 2024-01-15T18:23:30.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.279+0000 7eff7fc00640 1 -- 172.21.15.143:0/1751885137 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff78140350 con 0x7eff50067b80 2024-01-15T18:23:30.274 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:30.283+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:30.275 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:30.283+0000 7eff74c00640 1 -- 172.21.15.143:0/1751885137 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7eff78140350 con 0x7eff50067b80 2024-01-15T18:23:30.275 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:30.275 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:30.283 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:30.950 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:23:30.953 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:23:32.285 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:32.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90cc400640 1 Processor -- start 2024-01-15T18:23:32.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90cc400640 1 -- start start 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90cc400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90c415b2d0 0x7f90c40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90cc400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40b0b30 con 0x7f90c40cdd80 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90cc400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c40cf960 con 0x7f90c40cf410 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90cc400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c40ce210 con 0x7f90c415b2d0 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90c9a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90c415b2d0 0x7f90c40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90ca400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f90c40cdd80 0x7f90c40ccc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39476/0 (socket says 172.21.15.143:39476) 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90ca400640 1 -- 172.21.15.143:0/1496006163 learned_addr learned my addr 172.21.15.143:0/1496006163 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:32.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90c9a00640 1 -- 172.21.15.143:0/1496006163 >> v1:172.21.15.204:6789/0 conn(0x7f90c40cf410 legacy=0x7f90c40b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:32.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90c9a00640 1 -- 172.21.15.143:0/1496006163 >> v1:172.21.15.143:6789/0 conn(0x7f90c40cdd80 legacy=0x7f90c40ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:32.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90c9a00640 1 -- 172.21.15.143:0/1496006163 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c40cfc40 con 0x7f90c415b2d0 2024-01-15T18:23:32.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.447+0000 7f90c3e00640 1 -- 172.21.15.143:0/1496006163 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918543834 0 0) 0x7f90c40b0b30 con 0x7f90c40cdd80 2024-01-15T18:23:32.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90c9a00640 1 --2- 172.21.15.143:0/1496006163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c415b2d0 0x7f90c40cd840 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b2c8c7786c770d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:32.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90c3e00640 1 -- 172.21.15.143:0/1496006163 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40ab600 con 0x7f90c415b2d0 2024-01-15T18:23:32.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90c3e00640 1 -- 172.21.15.143:0/1496006163 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b40a7050 con 0x7f90c415b2d0 2024-01-15T18:23:32.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90c3e00640 1 -- 172.21.15.143:0/1496006163 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40ab980 con 0x7f90c415b2d0 2024-01-15T18:23:32.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90cc400640 1 -- 172.21.15.143:0/1496006163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c415b2d0 msgr2=0x7f90c40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:32.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90cc400640 1 --2- 172.21.15.143:0/1496006163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c415b2d0 0x7f90c40cd840 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f90b40097a0 tx=0x7f90b401c460 comp rx=0 tx=0).stop 2024-01-15T18:23:32.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90cc400640 1 -- 172.21.15.143:0/1496006163 shutdown_connections 2024-01-15T18:23:32.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90cc400640 1 --2- 172.21.15.143:0/1496006163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c415b2d0 0x7f90c40cd840 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:32.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.451+0000 7f90cc400640 1 -- 172.21.15.143:0/1496006163 >> 172.21.15.143:0/1496006163 conn(0x7f90c40d0e30 msgr2=0x7f90c40d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:32.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 -- 172.21.15.143:0/1496006163 shutdown_connections 2024-01-15T18:23:32.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 -- 172.21.15.143:0/1496006163 wait complete. 2024-01-15T18:23:32.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 Processor -- start 2024-01-15T18:23:32.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 -- start start 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c40cdd80 0x7f90c41556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90c40cf410 0x7f90c4165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90c415b2d0 0x7f90c406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40ca220 con 0x7f90c40cdd80 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40ca500 con 0x7f90c40cf410 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90cc400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c40ca7e0 con 0x7f90c415b2d0 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c40cdd80 0x7f90c41556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c40cdd80 0x7f90c41556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52216/0 (socket says 172.21.15.143:52216) 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 -- 172.21.15.143:0/3323348737 learned_addr learned my addr 172.21.15.143:0/3323348737 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:32.449 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9000640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90c40cf410 0x7f90c4165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90ca400640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90c415b2d0 0x7f90c406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 -- 172.21.15.143:0/3323348737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90c415b2d0 msgr2=0x7f90c406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f90c415b2d0 0x7f90c406e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 -- 172.21.15.143:0/3323348737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90c40cf410 msgr2=0x7f90c4165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f90c40cf410 0x7f90c4165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.455+0000 7f90c9a00640 1 -- 172.21.15.143:0/3323348737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40093b0 con 0x7f90c40cdd80 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c9a00640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90c40cdd80 0x7f90c41556d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f90b4019070 tx=0x7f90b4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40b4610 con 0x7f90c40cdd80 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b40b4770 con 0x7f90c40cdd80 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90cc400640 1 -- 172.21.15.143:0/3323348737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4165e90 con 0x7f90c40cdd80 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b40b4b20 con 0x7f90c40cdd80 2024-01-15T18:23:32.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90cc400640 1 -- 172.21.15.143:0/3323348737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c406ea70 con 0x7f90c40cdd80 2024-01-15T18:23:32.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90cc400640 1 -- 172.21.15.143:0/3323348737 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c4067500 con 0x7f90c40cdd80 2024-01-15T18:23:32.451 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f90b40b2040 con 0x7f90c40cdd80 2024-01-15T18:23:32.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c1600640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f909c067b80 0x7f909c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:32.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f90b411d1c0 con 0x7f90c40cdd80 2024-01-15T18:23:32.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.459+0000 7f90c9000640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f909c067b80 0x7f909c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:32.452 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.463+0000 7f90c9000640 1 --2- 172.21.15.143:0/3323348737 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f909c067b80 0x7f909c06a040 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f90c40c45a0 tx=0x7f90ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:32.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.463+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f90b40ebf00 con 0x7f90c40cdd80 2024-01-15T18:23:32.634 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.643+0000 7f90cc400640 1 -- 172.21.15.143:0/3323348737 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90c4168d00 con 0x7f909c067b80 2024-01-15T18:23:32.641 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:23:32.651+0000 7f3caa200640 -1 mgr.server reply reply (11) Resource temporarily unavailable snapshot 'ss1' has pending clones 2024-01-15T18:23:32.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:32.651+0000 7f90c1600640 1 -- 172.21.15.143:0/3323348737 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: -11 snapshot 'ss1' has pending clones) v1 ==== 41+0+0 (secure 0 0 0) 0x7f90ac0014f0 con 0x7f909c067b80 2024-01-15T18:23:32.641 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:32.642 INFO:teuthology.orchestra.run.smithi143.stderr:Error EAGAIN: snapshot 'ss1' has pending clones 2024-01-15T18:23:32.650 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:23:32.654 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the ... ERROR 2024-01-15T18:23:32.655 INFO:tasks.cephfs_test_runner:ERROR 2024-01-15T18:23:32.655 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.655 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:23:32.655 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:23:32.655 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:23:32.655 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 8163, in test_when_3_clones_are_cancelled 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: self._cancel_clones_and_assert(c) 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7984, in _cancel_clones_and_assert 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: self._cancel_clones(clones, check_status=True) 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7965, in _cancel_clones 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: self.run_ceph_cmd(f'fs clone cancel {v} {i}', 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/remote.py", line 523, in run 2024-01-15T18:23:32.656 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 455, in run 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: r.wait() 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone39062' 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:23:32.657 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7597, in tearDown 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner: self._loop_cmd('fs subvolume snapshot rm --force --format ' 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7566, in _loop_cmd 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 50 times: {cmdargs}') 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 50 times: fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner:Ran 7 tests in 1084.721s 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:23:32.658 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 8163, in test_when_3_clones_are_cancelled 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: self._cancel_clones_and_assert(c) 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7984, in _cancel_clones_and_assert 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: self._cancel_clones(clones, check_status=True) 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7965, in _cancel_clones 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: self.run_ceph_cmd(f'fs clone cancel {v} {i}', 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2024-01-15T18:23:32.659 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/remote.py", line 523, in run 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 455, in run 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: r.wait() 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi143 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone39062' 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:23:32.660 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7597, in tearDown 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner: self._loop_cmd('fs subvolume snapshot rm --force --format ' 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7566, in _loop_cmd 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 50 times: {cmdargs}') 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 50 times: fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:23:32.661 INFO:tasks.cephfs_test_runner: 2024-01-15T18:23:32.661 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress), test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:23:32.856 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=52feaa06080c4043a77be1063634d8cb Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress), test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:23:32.857 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-15T18:23:32.870 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-01-15T18:23:32.893 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-01-15T18:23:32.893 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:23:32.893 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:32.908 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:23:32.908 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:32.956 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:23:33.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 Processor -- start 2024-01-15T18:23:33.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 -- start start 2024-01-15T18:23:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f37d40ade20 0x7f37d40ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ae740 0x7f37d415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d415b540 con 0x7f37d40725b0 2024-01-15T18:23:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d40b03d0 con 0x7f37d40ade20 2024-01-15T18:23:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37dae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d40af950 con 0x7f37d40ae740 2024-01-15T18:23:33.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37d3400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f37d40725b0 0x7f37d415b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39484/0 (socket says 172.21.15.143:39484) 2024-01-15T18:23:33.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37d3400640 1 -- 172.21.15.143:0/1344653913 learned_addr learned my addr 172.21.15.143:0/1344653913 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.147+0000 7f37d3e00640 1 --2- 172.21.15.143:0/1344653913 >> v2:172.21.15.204:3300/0 conn(0x7f37d40ade20 0x7f37d40ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d8e00640 1 --2- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ae740 0x7f37d415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d2a00640 1 -- 172.21.15.143:0/1344653913 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412775498 0 0) 0x7f37d415b540 con 0x7f37d40725b0 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d2a00640 1 -- 172.21.15.143:0/1344653913 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b4003530 con 0x7f37d40725b0 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d3e00640 1 -- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ae740 msgr2=0x7f37d415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d3e00640 1 --2- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ae740 0x7f37d415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d3e00640 1 -- 172.21.15.143:0/1344653913 >> v1:172.21.15.143:6789/0 conn(0x7f37d40725b0 legacy=0x7f37d415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d3e00640 1 -- 172.21.15.143:0/1344653913 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d40afc30 con 0x7f37d40ade20 2024-01-15T18:23:33.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d3e00640 1 --2- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40ade20 0x7f37d40ae200 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37c40097c0 tx=0x7f37d415cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6876ab8b6c17ac39 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d2a00640 1 -- 172.21.15.143:0/1344653913 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40ac620 con 0x7f37d40ade20 2024-01-15T18:23:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d2a00640 1 -- 172.21.15.143:0/1344653913 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c40ac780 con 0x7f37d40ade20 2024-01-15T18:23:33.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.151+0000 7f37d2a00640 1 -- 172.21.15.143:0/1344653913 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40acaf0 con 0x7f37d40ade20 2024-01-15T18:23:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40ade20 msgr2=0x7f37d40ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 --2- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40ade20 0x7f37d40ae200 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37c40097c0 tx=0x7f37d415cc80 comp rx=0 tx=0).stop 2024-01-15T18:23:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- 172.21.15.143:0/1344653913 shutdown_connections 2024-01-15T18:23:33.145 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 --2- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ae740 0x7f37d415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 --2- 172.21.15.143:0/1344653913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40ade20 0x7f37d40ae200 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- 172.21.15.143:0/1344653913 >> 172.21.15.143:0/1344653913 conn(0x7f37d4071a90 msgr2=0x7f37d4071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- 172.21.15.143:0/1344653913 shutdown_connections 2024-01-15T18:23:33.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- 172.21.15.143:0/1344653913 wait complete. 2024-01-15T18:23:33.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 Processor -- start 2024-01-15T18:23:33.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- start start 2024-01-15T18:23:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 0x7f37d414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ade20 0x7f37d41546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37d40ae740 0x7f37d41551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d41542d0 con 0x7f37d40ae740 2024-01-15T18:23:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d41536e0 con 0x7f37d40725b0 2024-01-15T18:23:33.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.155+0000 7f37dae00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4153930 con 0x7f37d40ade20 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 0x7f37d414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d8e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37d40ae740 0x7f37d41551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ade20 0x7f37d41546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 0x7f37d414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48236/0 (socket says 172.21.15.143:48236) 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 -- 172.21.15.143:0/800430032 learned_addr learned my addr 172.21.15.143:0/800430032 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 -- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ade20 msgr2=0x7f37d41546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ade20 0x7f37d41546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 -- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37d40ae740 msgr2=0x7f37d41551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37d40ae740 0x7f37d41551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 -- 172.21.15.143:0/800430032 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c4009420 con 0x7f37d40725b0 2024-01-15T18:23:33.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37d3e00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 0x7f37d414f830 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f37c4000c10 tx=0x7f37c4019160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40ac8f0 con 0x7f37d40725b0 2024-01-15T18:23:33.152 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c40aca50 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c40acde0 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d4153c10 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d414fd70 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4154410 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.159+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f37c40b3040 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.163+0000 7f37cbe00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37ac067b80 0x7f37ac06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.163+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f37c411d240 con 0x7f37d40725b0 2024-01-15T18:23:33.153 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.163+0000 7f37d3400640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37ac067b80 0x7f37ac06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.163+0000 7f37d3400640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37ac067b80 0x7f37ac06a040 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007fd0 tx=0x7f37bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.167+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f37d4154410 con 0x7f37d40725b0 2024-01-15T18:23:33.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.315+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37d4154410 con 0x7f37d40725b0 2024-01-15T18:23:33.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.315+0000 7f37cbe00640 1 -- 172.21.15.143:0/800430032 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16745 (secure 0 0 0) 0x7f37c40ebc00 con 0x7f37d40725b0 2024-01-15T18:23:33.308 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:33.309 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":113,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:19:41.863194+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:23:33.310 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37ac067b80 msgr2=0x7f37ac06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37ac067b80 0x7f37ac06a040 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007fd0 tx=0x7f37bc009040 comp rx=0 tx=0).stop 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 msgr2=0x7f37d414f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 0x7f37d414f830 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f37c4000c10 tx=0x7f37c4019160 comp rx=0 tx=0).stop 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 shutdown_connections 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f37ac067b80 0x7f37ac06a040 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37d40ae740 0x7f37d41551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f37d40ade20 0x7f37d41546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.311 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 --2- 172.21.15.143:0/800430032 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f37d40725b0 0x7f37d414f830 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 >> 172.21.15.143:0/800430032 conn(0x7f37d4071a90 msgr2=0x7f37d406daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:33.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 shutdown_connections 2024-01-15T18:23:33.312 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.319+0000 7f37dae00640 1 -- 172.21.15.143:0/800430032 wait complete. 2024-01-15T18:23:33.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:18:39.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:18:40.286820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:23:33.330 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:23:33.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 Processor -- start 2024-01-15T18:23:33.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 -- start start 2024-01-15T18:23:33.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8aa00ce690 0x7f8aa00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00b1da0 0x7f8aa00b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa0070bf0 con 0x7f8aa00b3bf0 2024-01-15T18:23:33.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa015dba0 con 0x7f8aa00ce690 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00b2740 con 0x7f8aa00b1da0 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa6200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8aa00b3bf0 0x7f8aa00b4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39498/0 (socket says 172.21.15.143:39498) 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa6200640 1 -- 172.21.15.143:0/2328034106 learned_addr learned my addr 172.21.15.143:0/2328034106 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa5800640 1 --2- 172.21.15.143:0/2328034106 >> v2:172.21.15.143:3300/0 conn(0x7f8aa00ce690 0x7f8aa00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa4e00640 1 --2- 172.21.15.143:0/2328034106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00b1da0 0x7f8aa00b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa5800640 1 -- 172.21.15.143:0/2328034106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00b1da0 msgr2=0x7f8aa00b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:33.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa5800640 1 --2- 172.21.15.143:0/2328034106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00b1da0 0x7f8aa00b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa5800640 1 -- 172.21.15.143:0/2328034106 >> v1:172.21.15.143:6789/0 conn(0x7f8aa00b3bf0 legacy=0x7f8aa00b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.479+0000 7f8aa5800640 1 -- 172.21.15.143:0/2328034106 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00b4920 con 0x7f8aa00ce690 2024-01-15T18:23:33.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa5800640 1 --2- 172.21.15.143:0/2328034106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00ce690 0x7f8aa00ac8e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8a900097c0 tx=0x7f8a9001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdaba8ac4b07306 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8a9fe00640 1 -- 172.21.15.143:0/2328034106 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900ab650 con 0x7f8aa00ce690 2024-01-15T18:23:33.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8a9fe00640 1 -- 172.21.15.143:0/2328034106 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a900ab7b0 con 0x7f8aa00ce690 2024-01-15T18:23:33.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8a9fe00640 1 -- 172.21.15.143:0/2328034106 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900abb40 con 0x7f8aa00ce690 2024-01-15T18:23:33.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 -- 172.21.15.143:0/2328034106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00ce690 msgr2=0x7f8aa00ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2328034106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00ce690 0x7f8aa00ac8e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8a900097c0 tx=0x7f8a9001c450 comp rx=0 tx=0).stop 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 -- 172.21.15.143:0/2328034106 shutdown_connections 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2328034106 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00b1da0 0x7f8aa00b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2328034106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00ce690 0x7f8aa00ac8e0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 -- 172.21.15.143:0/2328034106 >> 172.21.15.143:0/2328034106 conn(0x7f8aa006c3f0 msgr2=0x7f8aa006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 -- 172.21.15.143:0/2328034106 shutdown_connections 2024-01-15T18:23:33.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.483+0000 7f8aa8200640 1 -- 172.21.15.143:0/2328034106 wait complete. 2024-01-15T18:23:33.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 Processor -- start 2024-01-15T18:23:33.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 -- start start 2024-01-15T18:23:33.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 0x7f8aa01343f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa00b3bf0 0x7f8aa013d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00ce690 0x7f8aa0145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0140f20 con 0x7f8aa00b1da0 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01411c0 con 0x7f8aa00b3bf0 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa01414a0 con 0x7f8aa00ce690 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 0x7f8aa01343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 0x7f8aa01343f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52228/0 (socket says 172.21.15.143:52228) 2024-01-15T18:23:33.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 -- 172.21.15.143:0/2601728430 learned_addr learned my addr 172.21.15.143:0/2601728430 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa4e00640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa00b3bf0 0x7f8aa013d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa6200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00ce690 0x7f8aa0145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 -- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00ce690 msgr2=0x7f8aa0145450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00ce690 0x7f8aa0145450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 -- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa00b3bf0 msgr2=0x7f8aa013d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa00b3bf0 0x7f8aa013d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 -- 172.21.15.143:0/2601728430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90009420 con 0x7f8aa00b1da0 2024-01-15T18:23:33.481 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa4e00640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa00b3bf0 0x7f8aa013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:33.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa5800640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 0x7f8aa01343f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8a9001a080 tx=0x7f8a9001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900bf610 con 0x7f8aa00b1da0 2024-01-15T18:23:33.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a900bf770 con 0x7f8aa00b1da0 2024-01-15T18:23:33.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa0141780 con 0x7f8aa00b1da0 2024-01-15T18:23:33.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900bfb20 con 0x7f8aa00b1da0 2024-01-15T18:23:33.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.487+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa0134930 con 0x7f8aa00b1da0 2024-01-15T18:23:33.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.491+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa0067500 con 0x7f8aa00b1da0 2024-01-15T18:23:33.483 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.491+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f8a900bfc80 con 0x7f8aa00b1da0 2024-01-15T18:23:33.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.491+0000 7f8a9d600640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a78067bb0 0x7f8a7806a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.491+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8a9011e050 con 0x7f8aa00b1da0 2024-01-15T18:23:33.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.491+0000 7f8aa4e00640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a78067bb0 0x7f8a7806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.491+0000 7f8aa4e00640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a78067bb0 0x7f8a7806a070 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00b1cd0 tx=0x7f8a88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.495+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8a900ecae0 con 0x7f8aa00b1da0 2024-01-15T18:23:33.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.647+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8aa006e260 con 0x7f8aa00b1da0 2024-01-15T18:23:33.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.647+0000 7f8a9d600640 1 -- 172.21.15.143:0/2601728430 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a900f0100 con 0x7f8aa00b1da0 2024-01-15T18:23:33.640 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:23:33.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a78067bb0 msgr2=0x7f8a7806a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a78067bb0 0x7f8a7806a070 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00b1cd0 tx=0x7f8a88008040 comp rx=0 tx=0).stop 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 msgr2=0x7f8aa01343f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 0x7f8aa01343f0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8a9001a080 tx=0x7f8a9001ac90 comp rx=0 tx=0).stop 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 shutdown_connections 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8a78067bb0 0x7f8a7806a070 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8aa00ce690 0x7f8aa0145450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8aa00b3bf0 0x7f8aa013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 --2- 172.21.15.143:0/2601728430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa00b1da0 0x7f8aa01343f0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 >> 172.21.15.143:0/2601728430 conn(0x7f8aa006c3f0 msgr2=0x7f8aa01466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:33.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 shutdown_connections 2024-01-15T18:23:33.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.651+0000 7f8aa8200640 1 -- 172.21.15.143:0/2601728430 wait complete. 2024-01-15T18:23:33.662 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:23:33.800 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.807+0000 7f3fbd800640 1 Processor -- start 2024-01-15T18:23:33.801 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 -- start start 2024-01-15T18:23:33.802 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fb815d3f0 0x7f3fb80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f3fb80d0050 0x7f3fb806bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb806c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb80cb480 con 0x7f3fb815d3f0 2024-01-15T18:23:33.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb80cb6d0 con 0x7f3fb80d0050 2024-01-15T18:23:33.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fbd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80cb9b0 con 0x7f3fb80b0490 2024-01-15T18:23:33.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fb815d3f0 0x7f3fb80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f3fb815d3f0 0x7f3fb80cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52232/0 (socket says 172.21.15.143:52232) 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 -- 172.21.15.143:0/2196031935 learned_addr learned my addr 172.21.15.143:0/2196031935 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6000640 1 --2- 172.21.15.143:0/2196031935 >> v2:172.21.15.204:3300/0 conn(0x7f3fb80d0050 0x7f3fb806bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb7400640 1 --2- 172.21.15.143:0/2196031935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb806c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 -- 172.21.15.143:0/2196031935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 msgr2=0x7f3fb806c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 --2- 172.21.15.143:0/2196031935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb806c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.804 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 -- 172.21.15.143:0/2196031935 >> v2:172.21.15.204:3300/0 conn(0x7f3fb80d0050 msgr2=0x7f3fb806bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:33.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 --2- 172.21.15.143:0/2196031935 >> v2:172.21.15.204:3300/0 conn(0x7f3fb80d0050 0x7f3fb806bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.805 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.811+0000 7f3fb6a00640 1 -- 172.21.15.143:0/2196031935 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80cbc90 con 0x7f3fb815d3f0 2024-01-15T18:23:33.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fb6a00640 1 --2- 172.21.15.143:0/2196031935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb80cfb10 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8070fe0 tx=0x7f3fa801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc32956ce953d25 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fb5600640 1 -- 172.21.15.143:0/2196031935 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80aa610 con 0x7f3fb815d3f0 2024-01-15T18:23:33.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fb5600640 1 -- 172.21.15.143:0/2196031935 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa80a6040 con 0x7f3fb815d3f0 2024-01-15T18:23:33.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fb5600640 1 -- 172.21.15.143:0/2196031935 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa80aa990 con 0x7f3fb815d3f0 2024-01-15T18:23:33.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 -- 172.21.15.143:0/2196031935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 msgr2=0x7f3fb80cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 --2- 172.21.15.143:0/2196031935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb80cfb10 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8070fe0 tx=0x7f3fa801c4d0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 -- 172.21.15.143:0/2196031935 shutdown_connections 2024-01-15T18:23:33.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 --2- 172.21.15.143:0/2196031935 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb806c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 --2- 172.21.15.143:0/2196031935 >> v2:172.21.15.204:3300/0 conn(0x7f3fb80d0050 0x7f3fb806bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.808 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 --2- 172.21.15.143:0/2196031935 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb80cfb10 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 -- 172.21.15.143:0/2196031935 >> 172.21.15.143:0/2196031935 conn(0x7f3fb80d0e30 msgr2=0x7f3fb80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:33.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 -- 172.21.15.143:0/2196031935 shutdown_connections 2024-01-15T18:23:33.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.815+0000 7f3fbd800640 1 -- 172.21.15.143:0/2196031935 wait complete. 2024-01-15T18:23:33.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 Processor -- start 2024-01-15T18:23:33.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- start start 2024-01-15T18:23:33.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fb80d0050 0x7f3fb80c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb8149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb8156af0 con 0x7f3fb815d3f0 2024-01-15T18:23:33.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb80c7810 con 0x7f3fb80d0050 2024-01-15T18:23:33.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb8156750 con 0x7f3fb80b0490 2024-01-15T18:23:33.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb8149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb8149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52236/0 (socket says 172.21.15.143:52236) 2024-01-15T18:23:33.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 -- 172.21.15.143:0/3188151488 learned_addr learned my addr 172.21.15.143:0/3188151488 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb6a00640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb6000640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fb80d0050 0x7f3fb80c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 -- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 msgr2=0x7f3fb815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 -- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fb80d0050 msgr2=0x7f3fb80c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fb80d0050 0x7f3fb80c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 -- 172.21.15.143:0/3188151488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa80094d0 con 0x7f3fb815d3f0 2024-01-15T18:23:33.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fb7400640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb8149040 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0007e30 tx=0x7f3fb000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0014620 con 0x7f3fb815d3f0 2024-01-15T18:23:33.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb000ee90 con 0x7f3fb815d3f0 2024-01-15T18:23:33.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb80c7250 con 0x7f3fb815d3f0 2024-01-15T18:23:33.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00149c0 con 0x7f3fb815d3f0 2024-01-15T18:23:33.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb80c6f00 con 0x7f3fb815d3f0 2024-01-15T18:23:33.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.819+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb81565b0 con 0x7f3fb815d3f0 2024-01-15T18:23:33.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.823+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f3fb0014b20 con 0x7f3fb815d3f0 2024-01-15T18:23:33.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.823+0000 7f3faea00640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3f90067b80 0x7f3f9006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:33.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.823+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f3fb0083c40 con 0x7f3fb815d3f0 2024-01-15T18:23:33.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.823+0000 7f3fb6a00640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3f90067b80 0x7f3f9006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:33.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.823+0000 7f3fb6a00640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3f90067b80 0x7f3f9006a040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80ae000 tx=0x7f3fa80b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:33.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.827+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f3fb81565b0 con 0x7f3fb815d3f0 2024-01-15T18:23:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.979+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fb8156c30 con 0x7f3fb815d3f0 2024-01-15T18:23:33.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3faea00640 1 -- 172.21.15.143:0/3188151488 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fb8156c30 con 0x7f3fb815d3f0 2024-01-15T18:23:33.973 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:23:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3f90067b80 msgr2=0x7f3f9006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3f90067b80 0x7f3f9006a040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80ae000 tx=0x7f3fa80b5040 comp rx=0 tx=0).stop 2024-01-15T18:23:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 msgr2=0x7f3fb8149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:33.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb8149040 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0007e30 tx=0x7f3fb000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 shutdown_connections 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f3f90067b80 0x7f3f9006a040 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3fb815d3f0 0x7f3fb8149040 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3fb80d0050 0x7f3fb80c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 --2- 172.21.15.143:0/3188151488 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f3fb80b0490 0x7f3fb815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 >> 172.21.15.143:0/3188151488 conn(0x7f3fb80d0e30 msgr2=0x7f3fb8158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 shutdown_connections 2024-01-15T18:23:33.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:33.983+0000 7f3fbd800640 1 -- 172.21.15.143:0/3188151488 wait complete. 2024-01-15T18:23:33.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:23:34.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.143+0000 7ff839e00640 1 Processor -- start 2024-01-15T18:23:34.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.143+0000 7ff839e00640 1 -- start start 2024-01-15T18:23:34.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff839e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff8340ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff839e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8340b3bf0 0x7ff8340b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff839e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff834070bf0 con 0x7ff8340b1da0 2024-01-15T18:23:34.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff839e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83415dba0 con 0x7ff8340b3bf0 2024-01-15T18:23:34.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff839e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340b4920 con 0x7ff8340ce690 2024-01-15T18:23:34.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff833400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff8340b3bf0 0x7ff8340b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff832000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff8340b1da0 0x7ff8340b2200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39510/0 (socket says 172.21.15.143:39510) 2024-01-15T18:23:34.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff832000640 1 -- 172.21.15.143:0/977283196 learned_addr learned my addr 172.21.15.143:0/977283196 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:34.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff832a00640 1 --2- 172.21.15.143:0/977283196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff8340ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff833400640 1 -- 172.21.15.143:0/977283196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 msgr2=0x7ff8340ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:34.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff833400640 1 --2- 172.21.15.143:0/977283196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff8340ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff833400640 1 -- 172.21.15.143:0/977283196 >> v1:172.21.15.143:6789/0 conn(0x7ff8340b1da0 legacy=0x7ff8340b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff831600640 1 -- 172.21.15.143:0/977283196 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296174139 0 0) 0x7ff834070bf0 con 0x7ff8340b1da0 2024-01-15T18:23:34.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.147+0000 7ff833400640 1 -- 172.21.15.143:0/977283196 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8340b4c00 con 0x7ff8340b3bf0 2024-01-15T18:23:34.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff833400640 1 --2- 172.21.15.143:0/977283196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff8340b4080 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009d90 tx=0x7ff82c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba0fab9fcd75e69 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff831600640 1 -- 172.21.15.143:0/977283196 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0a5a00 con 0x7ff8340b3bf0 2024-01-15T18:23:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff831600640 1 -- 172.21.15.143:0/977283196 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c0a5b60 con 0x7ff8340b3bf0 2024-01-15T18:23:34.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff831600640 1 -- 172.21.15.143:0/977283196 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0ad440 con 0x7ff8340b3bf0 2024-01-15T18:23:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 -- 172.21.15.143:0/977283196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 msgr2=0x7ff8340b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 --2- 172.21.15.143:0/977283196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff8340b4080 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009d90 tx=0x7ff82c01d380 comp rx=0 tx=0).stop 2024-01-15T18:23:34.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 -- 172.21.15.143:0/977283196 shutdown_connections 2024-01-15T18:23:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 --2- 172.21.15.143:0/977283196 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff8340b4080 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 --2- 172.21.15.143:0/977283196 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff8340ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 -- 172.21.15.143:0/977283196 >> 172.21.15.143:0/977283196 conn(0x7ff83406c3f0 msgr2=0x7ff83406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 -- 172.21.15.143:0/977283196 shutdown_connections 2024-01-15T18:23:34.143 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 -- 172.21.15.143:0/977283196 wait complete. 2024-01-15T18:23:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 Processor -- start 2024-01-15T18:23:34.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.151+0000 7ff839e00640 1 -- start start 2024-01-15T18:23:34.146 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8340b1da0 0x7ff834141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff834136490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff83406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8341478c0 con 0x7ff8340b3bf0 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff834141c50 con 0x7ff8340b1da0 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff834141970 con 0x7ff8340ce690 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff834136490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.147 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff834136490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52268/0 (socket says 172.21.15.143:52268) 2024-01-15T18:23:34.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 -- 172.21.15.143:0/4128657233 learned_addr learned my addr 172.21.15.143:0/4128657233 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:34.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832a00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8340b1da0 0x7ff834141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff833400640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff83406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 -- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 msgr2=0x7ff83406d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff83406d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.148 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 -- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8340b1da0 msgr2=0x7ff834141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8340b1da0 0x7ff834141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 -- 172.21.15.143:0/4128657233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009a70 con 0x7ff8340b3bf0 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832a00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8340b1da0 0x7ff834141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff833400640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff83406d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff832000640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff834136490 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0011f0 tx=0x7ff81c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0074e0 con 0x7ff8340b3bf0 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c007640 con 0x7ff8340b3bf0 2024-01-15T18:23:34.149 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004a10 con 0x7ff8340b3bf0 2024-01-15T18:23:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff834147200 con 0x7ff8340b3bf0 2024-01-15T18:23:34.150 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff834141680 con 0x7ff8340b3bf0 2024-01-15T18:23:34.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.155+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff834147a00 con 0x7ff8340b3bf0 2024-01-15T18:23:34.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.159+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7ff81c004b70 con 0x7ff8340b3bf0 2024-01-15T18:23:34.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.159+0000 7ff82b400640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff80c067b80 0x7ff80c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.159+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7ff81c0826a0 con 0x7ff8340b3bf0 2024-01-15T18:23:34.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.159+0000 7ff832a00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff80c067b80 0x7ff80c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.151 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.159+0000 7ff832a00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff80c067b80 0x7ff80c06a040 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff8240097a0 tx=0x7ff824008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.154 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.163+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7ff834147a00 con 0x7ff8340b3bf0 2024-01-15T18:23:34.303 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.311+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff834147a00 con 0x7ff8340b3bf0 2024-01-15T18:23:34.304 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.311+0000 7ff82b400640 1 -- 172.21.15.143:0/4128657233 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff834147a00 con 0x7ff8340b3bf0 2024-01-15T18:23:34.304 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:23:34.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff80c067b80 msgr2=0x7ff80c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.306 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff80c067b80 0x7ff80c06a040 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff8240097a0 tx=0x7ff824008040 comp rx=0 tx=0).stop 2024-01-15T18:23:34.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 msgr2=0x7ff834136490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff834136490 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0011f0 tx=0x7ff81c00c9b0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 shutdown_connections 2024-01-15T18:23:34.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7ff80c067b80 0x7ff80c06a040 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7ff8340ce690 0x7ff83406d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff8340b3bf0 0x7ff834136490 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 --2- 172.21.15.143:0/4128657233 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8340b1da0 0x7ff834141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 >> 172.21.15.143:0/4128657233 conn(0x7ff83406c3f0 msgr2=0x7ff834146800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:34.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 shutdown_connections 2024-01-15T18:23:34.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.315+0000 7ff839e00640 1 -- 172.21.15.143:0/4128657233 wait complete. 2024-01-15T18:23:34.326 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:23:34.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.471+0000 7fa62b400640 1 Processor -- start 2024-01-15T18:23:34.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.471+0000 7fa62b400640 1 -- start start 2024-01-15T18:23:34.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa62b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa624157300 0x7fa624159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa62b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b21d0 0x7fa6240b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa62b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6240b48f0 0x7fa62415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa62b400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6240b35a0 con 0x7fa6240b48f0 2024-01-15T18:23:34.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa62b400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6240b4d80 con 0x7fa624157300 2024-01-15T18:23:34.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa62b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6240b2a20 con 0x7fa6240b21d0 2024-01-15T18:23:34.472 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6240b48f0 0x7fa62415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa6240b48f0 0x7fa62415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52274/0 (socket says 172.21.15.143:52274) 2024-01-15T18:23:34.473 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 -- 172.21.15.143:0/466606964 learned_addr learned my addr 172.21.15.143:0/466606964 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:34.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa628a00640 1 --2- 172.21.15.143:0/466606964 >> v2:172.21.15.204:3300/0 conn(0x7fa624157300 0x7fa624159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa623e00640 1 --2- 172.21.15.143:0/466606964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b21d0 0x7fa6240b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.474 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 -- 172.21.15.143:0/466606964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b21d0 msgr2=0x7fa6240b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:34.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 --2- 172.21.15.143:0/466606964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b21d0 0x7fa6240b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 -- 172.21.15.143:0/466606964 >> v2:172.21.15.204:3300/0 conn(0x7fa624157300 msgr2=0x7fa624159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:34.475 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 --2- 172.21.15.143:0/466606964 >> v2:172.21.15.204:3300/0 conn(0x7fa624157300 0x7fa624159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 -- 172.21.15.143:0/466606964 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240b2d00 con 0x7fa6240b48f0 2024-01-15T18:23:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa629400640 1 --2- 172.21.15.143:0/466606964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b48f0 0x7fa62415d1b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b1aa0 tx=0x7fa61c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02fddbeec519820 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.476 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.475+0000 7fa623400640 1 -- 172.21.15.143:0/466606964 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0a5820 con 0x7fa6240b48f0 2024-01-15T18:23:34.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa623400640 1 -- 172.21.15.143:0/466606964 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c0a5980 con 0x7fa6240b48f0 2024-01-15T18:23:34.477 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa623400640 1 -- 172.21.15.143:0/466606964 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0a5d10 con 0x7fa6240b48f0 2024-01-15T18:23:34.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- 172.21.15.143:0/466606964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b48f0 msgr2=0x7fa62415d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- 172.21.15.143:0/466606964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b48f0 0x7fa62415d1b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b1aa0 tx=0x7fa61c01c9a0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.478 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- 172.21.15.143:0/466606964 shutdown_connections 2024-01-15T18:23:34.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- 172.21.15.143:0/466606964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b48f0 0x7fa62415d1b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- 172.21.15.143:0/466606964 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b21d0 0x7fa6240b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.479 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- 172.21.15.143:0/466606964 >> v2:172.21.15.204:3300/0 conn(0x7fa624157300 0x7fa624159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- 172.21.15.143:0/466606964 >> 172.21.15.143:0/466606964 conn(0x7fa62406c3f0 msgr2=0x7fa62406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:34.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- 172.21.15.143:0/466606964 shutdown_connections 2024-01-15T18:23:34.480 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- 172.21.15.143:0/466606964 wait complete. 2024-01-15T18:23:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 Processor -- start 2024-01-15T18:23:34.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- start start 2024-01-15T18:23:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 0x7fa62406d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b48f0 0x7fa62414ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa624157300 0x7fa62414b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62414b7b0 con 0x7fa6240b21d0 2024-01-15T18:23:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6241576e0 con 0x7fa624157300 2024-01-15T18:23:34.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.479+0000 7fa62b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa62406d850 con 0x7fa6240b48f0 2024-01-15T18:23:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 0x7fa62406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 0x7fa62406d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52280/0 (socket says 172.21.15.143:52280) 2024-01-15T18:23:34.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 -- 172.21.15.143:0/1178532772 learned_addr learned my addr 172.21.15.143:0/1178532772 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:34.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa629400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa624157300 0x7fa62414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa623e00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b48f0 0x7fa62414ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 -- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b48f0 msgr2=0x7fa62414ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b48f0 0x7fa62414ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 -- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa624157300 msgr2=0x7fa62414b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa624157300 0x7fa62414b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 -- 172.21.15.143:0/1178532772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c009b30 con 0x7fa6240b21d0 2024-01-15T18:23:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa628a00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 0x7fa62406d310 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa61400d5f0 tx=0x7fa61400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6140125f0 con 0x7fa6240b21d0 2024-01-15T18:23:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa614012750 con 0x7fa6240b21d0 2024-01-15T18:23:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614012ae0 con 0x7fa6240b21d0 2024-01-15T18:23:34.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa624157930 con 0x7fa6240b21d0 2024-01-15T18:23:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa624157ef0 con 0x7fa6240b21d0 2024-01-15T18:23:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa624067500 con 0x7fa6240b21d0 2024-01-15T18:23:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.483+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fa614012c40 con 0x7fa6240b21d0 2024-01-15T18:23:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.487+0000 7fa620c00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5fc067b80 0x7fa5fc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.487+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fa6140836c0 con 0x7fa6240b21d0 2024-01-15T18:23:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.487+0000 7fa623e00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5fc067b80 0x7fa5fc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.487+0000 7fa623e00640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5fc067b80 0x7fa5fc06a040 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa62415d120 tx=0x7fa60c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.491+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa6140520e0 con 0x7fa6240b21d0 2024-01-15T18:23:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.651+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa62406e4e0 con 0x7fa6240b21d0 2024-01-15T18:23:34.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.651+0000 7fa620c00640 1 -- 172.21.15.143:0/1178532772 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa614007380 con 0x7fa6240b21d0 2024-01-15T18:23:34.645 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:23:34.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5fc067b80 msgr2=0x7fa5fc06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5fc067b80 0x7fa5fc06a040 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa62415d120 tx=0x7fa60c008040 comp rx=0 tx=0).stop 2024-01-15T18:23:34.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 msgr2=0x7fa62406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 0x7fa62406d310 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fa61400d5f0 tx=0x7fa61400dab0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 shutdown_connections 2024-01-15T18:23:34.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fa5fc067b80 0x7fa5fc06a040 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa624157300 0x7fa62414b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa6240b48f0 0x7fa62414ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 --2- 172.21.15.143:0/1178532772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6240b21d0 0x7fa62406d310 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 >> 172.21.15.143:0/1178532772 conn(0x7fa62406c3f0 msgr2=0x7fa62415c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:34.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 shutdown_connections 2024-01-15T18:23:34.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.655+0000 7fa62b400640 1 -- 172.21.15.143:0/1178532772 wait complete. 2024-01-15T18:23:34.666 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:23:34.667 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:34.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:23:34.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd75e800640 1 Processor -- start 2024-01-15T18:23:34.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd75e800640 1 -- start start 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd75e800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd758140700 0x7fd75806a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd75e800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd758140b90 con 0x7fd75815b610 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd75e800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd758140e00 con 0x7fd75806a9d0 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd75e800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75813e420 con 0x7fd758140700 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd756a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd758140700 0x7fd75806a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd757400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd75815b610 0x7fd758071920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39518/0 (socket says 172.21.15.143:39518) 2024-01-15T18:23:34.856 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.863+0000 7fd757400640 1 -- 172.21.15.143:0/1349335842 learned_addr learned my addr 172.21.15.143:0/1349335842 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756a00640 1 -- 172.21.15.143:0/1349335842 >> v1:172.21.15.204:6789/0 conn(0x7fd75806a9d0 legacy=0x7fd75815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756a00640 1 -- 172.21.15.143:0/1349335842 >> v1:172.21.15.143:6789/0 conn(0x7fd75815b610 legacy=0x7fd758071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756a00640 1 -- 172.21.15.143:0/1349335842 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7581410e0 con 0x7fd758140700 2024-01-15T18:23:34.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756a00640 1 --2- 172.21.15.143:0/1349335842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd758140700 0x7fd75806a490 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097f0 tx=0x7fd74001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fffdb2d451dc47a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756000640 1 -- 172.21.15.143:0/1349335842 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7400ab600 con 0x7fd758140700 2024-01-15T18:23:34.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756000640 1 -- 172.21.15.143:0/1349335842 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7400a7050 con 0x7fd758140700 2024-01-15T18:23:34.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd756000640 1 -- 172.21.15.143:0/1349335842 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7400ab980 con 0x7fd758140700 2024-01-15T18:23:34.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd75e800640 1 -- 172.21.15.143:0/1349335842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd758140700 msgr2=0x7fd75806a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.867+0000 7fd75e800640 1 --2- 172.21.15.143:0/1349335842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd758140700 0x7fd75806a490 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fd7400097f0 tx=0x7fd74001c480 comp rx=0 tx=0).stop 2024-01-15T18:23:34.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- 172.21.15.143:0/1349335842 shutdown_connections 2024-01-15T18:23:34.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 --2- 172.21.15.143:0/1349335842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd758140700 0x7fd75806a490 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- 172.21.15.143:0/1349335842 >> 172.21.15.143:0/1349335842 conn(0x7fd7580b1b90 msgr2=0x7fd7580b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:34.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- 172.21.15.143:0/1349335842 shutdown_connections 2024-01-15T18:23:34.861 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- 172.21.15.143:0/1349335842 wait complete. 2024-01-15T18:23:34.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 Processor -- start 2024-01-15T18:23:34.862 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- start start 2024-01-15T18:23:34.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 0x7fd75813a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd758140700 0x7fd75813bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd75815b610 0x7fd758139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75813d2c0 con 0x7fd75806a9d0 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75813c420 con 0x7fd75815b610 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd75e800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75813c990 con 0x7fd758140700 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 0x7fd75813a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 0x7fd75813a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52288/0 (socket says 172.21.15.143:52288) 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 -- 172.21.15.143:0/2908721931 learned_addr learned my addr 172.21.15.143:0/2908721931 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:34.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd756a00640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd758140700 0x7fd75813bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757e00640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd75815b610 0x7fd758139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 -- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd758140700 msgr2=0x7fd75813bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd758140700 0x7fd75813bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 -- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd75815b610 msgr2=0x7fd758139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd75815b610 0x7fd758139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757400640 1 -- 172.21.15.143:0/2908721931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd740009420 con 0x7fd75806a9d0 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.871+0000 7fd757e00640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd75815b610 0x7fd758139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd757400640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 0x7fd75813a460 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c400 tx=0x7fd74800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd748005cf0 con 0x7fd75806a9d0 2024-01-15T18:23:34.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd748005e50 con 0x7fd75806a9d0 2024-01-15T18:23:34.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480072d0 con 0x7fd75806a9d0 2024-01-15T18:23:34.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75813cc70 con 0x7fd75806a9d0 2024-01-15T18:23:34.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75813a1b0 con 0x7fd75806a9d0 2024-01-15T18:23:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758067500 con 0x7fd75806a9d0 2024-01-15T18:23:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fd748007430 con 0x7fd75806a9d0 2024-01-15T18:23:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd74f400640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd730067b80 0x7fd73006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd7480828c0 con 0x7fd75806a9d0 2024-01-15T18:23:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.875+0000 7fd756a00640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd730067b80 0x7fd73006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:34.868 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.879+0000 7fd756a00640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd730067b80 0x7fd73006a040 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd7400ad000 tx=0x7fd7400a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:34.872 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:34.879+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd748051280 con 0x7fd75806a9d0 2024-01-15T18:23:35.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.031+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd758166d10 con 0x7fd75806a9d0 2024-01-15T18:23:35.025 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd74f400640 1 -- 172.21.15.143:0/2908721931 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16745 (secure 0 0 0) 0x7fd748017090 con 0x7fd75806a9d0 2024-01-15T18:23:35.025 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:35.025 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":113,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:19:41.863194+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:23:35.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd730067b80 msgr2=0x7fd73006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.027 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd730067b80 0x7fd73006a040 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd7400ad000 tx=0x7fd7400a6000 comp rx=0 tx=0).stop 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 msgr2=0x7fd75813a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 0x7fd75813a460 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd74800c400 tx=0x7fd74800c8c0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 shutdown_connections 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd730067b80 0x7fd73006a040 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd75815b610 0x7fd758139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd758140700 0x7fd75813bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.028 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 --2- 172.21.15.143:0/2908721931 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd75806a9d0 0x7fd75813a460 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 >> 172.21.15.143:0/2908721931 conn(0x7fd7580b1b90 msgr2=0x7fd75815da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:35.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 shutdown_connections 2024-01-15T18:23:35.029 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.035+0000 7fd75e800640 1 -- 172.21.15.143:0/2908721931 wait complete. 2024-01-15T18:23:35.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:18:39.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:18:40.286820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:23:35.048 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:23:35.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.195+0000 7fbeb4600640 1 Processor -- start 2024-01-15T18:23:35.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.195+0000 7fbeb4600640 1 -- start start 2024-01-15T18:23:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb4600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbeac15f570 0x7fbeac070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb4600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbeac071200 0x7fbeac0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb4600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeac15f0e0 con 0x7fbeac158a70 2024-01-15T18:23:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb4600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac15f9e0 con 0x7fbeac071200 2024-01-15T18:23:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb4600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeac071640 con 0x7fbeac15f570 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbeac071200 0x7fbeac0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fbeac071200 0x7fbeac0cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52300/0 (socket says 172.21.15.143:52300) 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 -- 172.21.15.143:0/2367162367 learned_addr learned my addr 172.21.15.143:0/2367162367 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb1200640 1 --2- 172.21.15.143:0/2367162367 >> v2:172.21.15.204:3300/0 conn(0x7fbeac15f570 0x7fbeac070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 -- 172.21.15.143:0/2367162367 >> v2:172.21.15.204:3300/0 conn(0x7fbeac15f570 msgr2=0x7fbeac070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 --2- 172.21.15.143:0/2367162367 >> v2:172.21.15.204:3300/0 conn(0x7fbeac15f570 0x7fbeac070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 -- 172.21.15.143:0/2367162367 >> v1:172.21.15.204:6789/0 conn(0x7fbeac158a70 legacy=0x7fbeac15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:35.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.199+0000 7fbeb2600640 1 -- 172.21.15.143:0/2367162367 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac158e50 con 0x7fbeac071200 2024-01-15T18:23:35.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb2600640 1 --2- 172.21.15.143:0/2367162367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac0cdcc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fbeac06bd20 tx=0x7fbea401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56924ce8e3a6d5c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeabe00640 1 -- 172.21.15.143:0/2367162367 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40a5850 con 0x7fbeac071200 2024-01-15T18:23:35.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeabe00640 1 -- 172.21.15.143:0/2367162367 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40a59b0 con 0x7fbeac071200 2024-01-15T18:23:35.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeabe00640 1 -- 172.21.15.143:0/2367162367 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40a5d40 con 0x7fbeac071200 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 -- 172.21.15.143:0/2367162367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 msgr2=0x7fbeac0cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 --2- 172.21.15.143:0/2367162367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac0cdcc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fbeac06bd20 tx=0x7fbea401c990 comp rx=0 tx=0).stop 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 -- 172.21.15.143:0/2367162367 shutdown_connections 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 --2- 172.21.15.143:0/2367162367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac0cdcc0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 --2- 172.21.15.143:0/2367162367 >> v2:172.21.15.204:3300/0 conn(0x7fbeac15f570 0x7fbeac070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 -- 172.21.15.143:0/2367162367 >> 172.21.15.143:0/2367162367 conn(0x7fbeac0b1610 msgr2=0x7fbeac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 -- 172.21.15.143:0/2367162367 shutdown_connections 2024-01-15T18:23:35.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.203+0000 7fbeb4600640 1 -- 172.21.15.143:0/2367162367 wait complete. 2024-01-15T18:23:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 Processor -- start 2024-01-15T18:23:35.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- start start 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbeac158a70 0x7fbeac1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbeac15f570 0x7fbeac0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac156020 con 0x7fbeac071200 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac15cfe0 con 0x7fbeac158a70 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac15d750 con 0x7fbeac15f570 2024-01-15T18:23:35.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52310/0 (socket says 172.21.15.143:52310) 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 -- 172.21.15.143:0/1518500670 learned_addr learned my addr 172.21.15.143:0/1518500670 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1200640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbeac158a70 0x7fbeac1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb2600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbeac15f570 0x7fbeac0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 -- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbeac15f570 msgr2=0x7fbeac0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbeac15f570 0x7fbeac0b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 -- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbeac158a70 msgr2=0x7fbeac1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbeac158a70 0x7fbeac1524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 -- 172.21.15.143:0/1518500670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4009b30 con 0x7fbeac071200 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb1c00640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac152c40 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c480 tx=0x7fbe9c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c012830 con 0x7fbeac071200 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c0129b0 con 0x7fbeac071200 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeac156270 con 0x7fbeac071200 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c012d60 con 0x7fbeac071200 2024-01-15T18:23:35.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeac15d230 con 0x7fbeac071200 2024-01-15T18:23:35.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.207+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeac074b00 con 0x7fbeac071200 2024-01-15T18:23:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.211+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fbe9c01c460 con 0x7fbeac071200 2024-01-15T18:23:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.211+0000 7fbea9600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe84067bb0 0x7fbe8406a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.211+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fbe9c083150 con 0x7fbeac071200 2024-01-15T18:23:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.211+0000 7fbeb1200640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe84067bb0 0x7fbe8406a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.211+0000 7fbeb1200640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe84067bb0 0x7fbe8406a070 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbeac153800 tx=0x7fbe94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.215+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbe9c051b10 con 0x7fbeac071200 2024-01-15T18:23:35.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.371+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbeac15d450 con 0x7fbeac071200 2024-01-15T18:23:35.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.371+0000 7fbea9600640 1 -- 172.21.15.143:0/1518500670 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbe9c055130 con 0x7fbeac071200 2024-01-15T18:23:35.363 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.371+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe84067bb0 msgr2=0x7fbe8406a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.371+0000 7fbeb4600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe84067bb0 0x7fbe8406a070 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbeac153800 tx=0x7fbe94008040 comp rx=0 tx=0).stop 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 msgr2=0x7fbeac152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac152c40 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c480 tx=0x7fbe9c00c940 comp rx=0 tx=0).stop 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 shutdown_connections 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbe84067bb0 0x7fbe8406a070 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbeac15f570 0x7fbeac0b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbeac158a70 0x7fbeac1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 --2- 172.21.15.143:0/1518500670 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbeac071200 0x7fbeac152c40 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 >> 172.21.15.143:0/1518500670 conn(0x7fbeac0b1610 msgr2=0x7fbeac1648d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 shutdown_connections 2024-01-15T18:23:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.375+0000 7fbeb4600640 1 -- 172.21.15.143:0/1518500670 wait complete. 2024-01-15T18:23:35.385 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:23:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.531+0000 7fb306c00640 1 Processor -- start 2024-01-15T18:23:35.524 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.531+0000 7fb306c00640 1 -- start start 2024-01-15T18:23:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb306c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb3000cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.525 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb306c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3000c4d90 con 0x7fb3000c6230 2024-01-15T18:23:35.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb306c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30015b640 con 0x7fb30015b260 2024-01-15T18:23:35.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb306c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000cf800 con 0x7fb3000cdf20 2024-01-15T18:23:35.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb304c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb3000c6230 0x7fb3000cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:39532/0 (socket says 172.21.15.143:39532) 2024-01-15T18:23:35.526 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb304c00640 1 -- 172.21.15.143:0/117852746 learned_addr learned my addr 172.21.15.143:0/117852746 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb3000cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138856135 0 0) 0x7fb3000c4d90 con 0x7fb3000c6230 2024-01-15T18:23:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2e0003530 con 0x7fb3000c6230 2024-01-15T18:23:35.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344537034 0 0) 0x7fb2e0003530 con 0x7fb3000c6230 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 msgr2=0x7fb3000cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb3000cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 >> v1:172.21.15.204:6789/0 conn(0x7fb30015b260 legacy=0x7fb3000cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000cfae0 con 0x7fb3000c6230 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7fb2f8001a00 con 0x7fb3000c6230 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 >> v1:172.21.15.143:6789/0 conn(0x7fb3000c6230 legacy=0x7fb3000cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.529 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e00039b0 0x7fb2e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb2e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e0004d10 0x7fb2e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000cf800 con 0x7fb2e0004d10 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e000ba70 con 0x7fb2e00039b0 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e000bc30 con 0x7fb3000cdf20 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb2e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ffe00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e00039b0 0x7fb2e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb304c00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e0004d10 0x7fb2e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 -- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e00039b0 msgr2=0x7fb2e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2e00039b0 0x7fb2e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 -- 172.21.15.143:0/117852746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e0004d10 msgr2=0x7fb2e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e0004d10 0x7fb2e0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb2ff400640 1 -- 172.21.15.143:0/117852746 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e0003530 con 0x7fb3000cdf20 2024-01-15T18:23:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.535+0000 7fb304c00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb2e0004d10 0x7fb2e0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb2ff400640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb2e00047d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009ab0 tx=0x7fb2e801ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d5d63fba72af5bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80a5660 con 0x7fb3000cdf20 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e80a57c0 con 0x7fb3000cdf20 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb2fea00640 1 -- 172.21.15.143:0/117852746 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80a5b70 con 0x7fb3000cdf20 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb306c00640 1 -- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 msgr2=0x7fb2e00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb306c00640 1 --2- 172.21.15.143:0/117852746 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3000cdf20 0x7fb2e00047d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009ab0 tx=0x7fb2e801ca70 comp rx=0 tx=0).stop 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb304c00640 1 -- 172.21.15.143:0/117852746 reap_dead start 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb306c00640 1 -- 172.21.15.143:0/117852746 shutdown_connections 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb306c00640 1 -- 172.21.15.143:0/117852746 >> 172.21.15.143:0/117852746 conn(0x7fb3000d0ed0 msgr2=0x7fb3000d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb306c00640 1 -- 172.21.15.143:0/117852746 shutdown_connections 2024-01-15T18:23:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.539+0000 7fb306c00640 1 -- 172.21.15.143:0/117852746 wait complete. 2024-01-15T18:23:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 Processor -- start 2024-01-15T18:23:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 -- start start 2024-01-15T18:23:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3000cdf20 0x7fb3001526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3001f3eb0 0x7fb30014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 0x7fb30006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30006bf90 con 0x7fb30006b5a0 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30014e7e0 con 0x7fb3000cdf20 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb300152d20 con 0x7fb3001f3eb0 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 0x7fb30006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 0x7fb30006ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52320/0 (socket says 172.21.15.143:52320) 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 -- 172.21.15.143:0/805501457 learned_addr learned my addr 172.21.15.143:0/805501457 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb2ff400640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3001f3eb0 0x7fb30014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb2ffe00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3000cdf20 0x7fb3001526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 -- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3001f3eb0 msgr2=0x7fb30014e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3001f3eb0 0x7fb30014e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 -- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3000cdf20 msgr2=0x7fb3001526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3000cdf20 0x7fb3001526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 -- 172.21.15.143:0/805501457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300153000 con 0x7fb30006b5a0 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb2ff400640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3001f3eb0 0x7fb30014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb304c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 0x7fb30006ba50 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fb30006b3d0 tx=0x7fb2f800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8007920 con 0x7fb30006b5a0 2024-01-15T18:23:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f8007a80 con 0x7fb30006b5a0 2024-01-15T18:23:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80135f0 con 0x7fb30006b5a0 2024-01-15T18:23:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3001532e0 con 0x7fb30006b5a0 2024-01-15T18:23:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.543+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3001538a0 con 0x7fb30006b5a0 2024-01-15T18:23:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.547+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb300067500 con 0x7fb30006b5a0 2024-01-15T18:23:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.547+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fb2f8013750 con 0x7fb30006b5a0 2024-01-15T18:23:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.547+0000 7fb2f7e00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2d8067bb0 0x7fb2d806a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.547+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fb2f80836c0 con 0x7fb30006b5a0 2024-01-15T18:23:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.547+0000 7fb2ffe00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2d8067bb0 0x7fb2d806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.541 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.547+0000 7fb2ffe00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2d8067bb0 0x7fb2d806a070 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb3000cf420 tx=0x7fb2f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.551+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fb2f80522d0 con 0x7fb30006b5a0 2024-01-15T18:23:35.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.699+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3000ce440 con 0x7fb30006b5a0 2024-01-15T18:23:35.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.699+0000 7fb2f7e00640 1 -- 172.21.15.143:0/805501457 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2f800a3b0 con 0x7fb30006b5a0 2024-01-15T18:23:35.692 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:23:35.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2d8067bb0 msgr2=0x7fb2d806a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2d8067bb0 0x7fb2d806a070 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb3000cf420 tx=0x7fb2f0009040 comp rx=0 tx=0).stop 2024-01-15T18:23:35.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 msgr2=0x7fb30006ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 0x7fb30006ba50 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fb30006b3d0 tx=0x7fb2f800af90 comp rx=0 tx=0).stop 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 shutdown_connections 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb2d8067bb0 0x7fb2d806a070 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb30006b5a0 0x7fb30006ba50 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb3001f3eb0 0x7fb30014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 --2- 172.21.15.143:0/805501457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3000cdf20 0x7fb3001526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 >> 172.21.15.143:0/805501457 conn(0x7fb3000d0ed0 msgr2=0x7fb3000cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 shutdown_connections 2024-01-15T18:23:35.695 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.703+0000 7fb306c00640 1 -- 172.21.15.143:0/805501457 wait complete. 2024-01-15T18:23:35.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:23:35.851 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.859+0000 7faede600640 1 Processor -- start 2024-01-15T18:23:35.852 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.859+0000 7faede600640 1 -- start start 2024-01-15T18:23:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faede600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faed8157300 0x7faed8159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faede600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 0x7faed80b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faede600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed80b2a20 con 0x7faed80b48f0 2024-01-15T18:23:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faede600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed80b35a0 con 0x7faed8157300 2024-01-15T18:23:35.853 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faede600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed80b4d80 con 0x7faed80b21d0 2024-01-15T18:23:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faed8157300 0x7faed8159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.854 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 0x7faed80b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed7400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7faed8157300 0x7faed8159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48340/0 (socket says 172.21.15.143:48340) 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed7400640 1 -- 172.21.15.143:0/3679687210 learned_addr learned my addr 172.21.15.143:0/3679687210 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed6a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 0x7faed80b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42090/0 (socket says 172.21.15.143:42090) 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed6a00640 1 -- 172.21.15.143:0/3679687210 >> v2:172.21.15.204:3300/0 conn(0x7faed8157300 msgr2=0x7faed8159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed6a00640 1 --2- 172.21.15.143:0/3679687210 >> v2:172.21.15.204:3300/0 conn(0x7faed8157300 0x7faed8159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed6a00640 1 -- 172.21.15.143:0/3679687210 >> v1:172.21.15.204:6789/0 conn(0x7faed80b48f0 legacy=0x7faed815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.855 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.863+0000 7faed6a00640 1 -- 172.21.15.143:0/3679687210 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed80b2d00 con 0x7faed80b21d0 2024-01-15T18:23:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faed6a00640 1 --2- 172.21.15.143:0/3679687210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 0x7faed80b43b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faec00097c0 tx=0x7faec001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e544230423a0239 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.857 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faed6000640 1 -- 172.21.15.143:0/3679687210 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00ab650 con 0x7faed80b21d0 2024-01-15T18:23:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faed6000640 1 -- 172.21.15.143:0/3679687210 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec00ab7b0 con 0x7faed80b21d0 2024-01-15T18:23:35.858 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faed6000640 1 -- 172.21.15.143:0/3679687210 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec00abb40 con 0x7faed80b21d0 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 -- 172.21.15.143:0/3679687210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 msgr2=0x7faed80b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 --2- 172.21.15.143:0/3679687210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 0x7faed80b43b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faec00097c0 tx=0x7faec001c450 comp rx=0 tx=0).stop 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 -- 172.21.15.143:0/3679687210 shutdown_connections 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 --2- 172.21.15.143:0/3679687210 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed80b21d0 0x7faed80b43b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 --2- 172.21.15.143:0/3679687210 >> v2:172.21.15.204:3300/0 conn(0x7faed8157300 0x7faed8159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 -- 172.21.15.143:0/3679687210 >> 172.21.15.143:0/3679687210 conn(0x7faed806c3f0 msgr2=0x7faed806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:35.859 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 -- 172.21.15.143:0/3679687210 shutdown_connections 2024-01-15T18:23:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 -- 172.21.15.143:0/3679687210 wait complete. 2024-01-15T18:23:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.867+0000 7faede600640 1 Processor -- start 2024-01-15T18:23:35.860 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- start start 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed80b21d0 0x7faed81f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed80b48f0 0x7faed81f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed8157300 0x7faed814d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed80aff50 con 0x7faed80b48f0 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed814bc30 con 0x7faed80b21d0 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed8147d30 con 0x7faed8157300 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed80b48f0 0x7faed81f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.863 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed6a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed80b48f0 0x7faed81f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52332/0 (socket says 172.21.15.143:52332) 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed6a00640 1 -- 172.21.15.143:0/1907645771 learned_addr learned my addr 172.21.15.143:0/1907645771 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed80b21d0 0x7faed81f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7e00640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed8157300 0x7faed814d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 -- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed8157300 msgr2=0x7faed814d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed8157300 0x7faed814d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 -- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed80b48f0 msgr2=0x7faed81f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:35.864 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed80b48f0 0x7faed81f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 -- 172.21.15.143:0/1907645771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec0009420 con 0x7faed80b21d0 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faed7400640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed80b21d0 0x7faed81f9fa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faec800e870 tx=0x7faec800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800ee60 con 0x7faed80b21d0 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800c940 con 0x7faed80b21d0 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed8148010 con 0x7faed80b21d0 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800ccd0 con 0x7faed80b21d0 2024-01-15T18:23:35.865 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed81f5dd0 con 0x7faed80b21d0 2024-01-15T18:23:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.871+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8067500 con 0x7faed80b21d0 2024-01-15T18:23:35.866 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.875+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7faec8013470 con 0x7faed80b21d0 2024-01-15T18:23:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.875+0000 7faecf400640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faeb0067bb0 0x7faeb006a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.875+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7faec8082f50 con 0x7faed80b21d0 2024-01-15T18:23:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.875+0000 7faed6a00640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faeb0067bb0 0x7faeb006a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:35.867 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.875+0000 7faed6a00640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faeb0067bb0 0x7faeb006a070 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faec001c490 tx=0x7faec00a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:35.870 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:35.879+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7faec8051c90 con 0x7faed80b21d0 2024-01-15T18:23:36.019 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.027+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faed80cc030 con 0x7faed80b21d0 2024-01-15T18:23:36.020 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faecf400640 1 -- 172.21.15.143:0/1907645771 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7faec80210a0 con 0x7faed80b21d0 2024-01-15T18:23:36.020 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faeb0067bb0 msgr2=0x7faeb006a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faeb0067bb0 0x7faeb006a070 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faec001c490 tx=0x7faec00a7000 comp rx=0 tx=0).stop 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed80b21d0 msgr2=0x7faed81f9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed80b21d0 0x7faed81f9fa0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7faec800e870 tx=0x7faec800ed30 comp rx=0 tx=0).stop 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 shutdown_connections 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7faeb0067bb0 0x7faeb006a070 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.023 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7faed8157300 0x7faed814d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faed80b48f0 0x7faed81f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 --2- 172.21.15.143:0/1907645771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faed80b21d0 0x7faed81f9fa0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 >> 172.21.15.143:0/1907645771 conn(0x7faed806c3f0 msgr2=0x7faed8156800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:36.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 shutdown_connections 2024-01-15T18:23:36.024 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.031+0000 7faede600640 1 -- 172.21.15.143:0/1907645771 wait complete. 2024-01-15T18:23:36.043 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:23:36.181 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 Processor -- start 2024-01-15T18:23:36.182 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 -- start start 2024-01-15T18:23:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 0x7f79fc0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79fc15d510 0x7f79fc06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79fc160eb0 con 0x7f79fc06a490 2024-01-15T18:23:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79fc161190 con 0x7f79fc15d510 2024-01-15T18:23:36.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f7a02000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc161470 con 0x7f79fc15d130 2024-01-15T18:23:36.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79faa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79fc15d510 0x7f79fc06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:36.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 0x7f79fc0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 0x7f79fc0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:42098/0 (socket says 172.21.15.143:42098) 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 -- 172.21.15.143:0/3106880283 learned_addr learned my addr 172.21.15.143:0/3106880283 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79faa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f79fc15d510 0x7f79fc06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48362/0 (socket says 172.21.15.143:48362) 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 -- 172.21.15.143:0/3106880283 >> v2:172.21.15.204:3300/0 conn(0x7f79fc15d510 msgr2=0x7f79fc06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 --2- 172.21.15.143:0/3106880283 >> v2:172.21.15.204:3300/0 conn(0x7f79fc15d510 0x7f79fc06e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 -- 172.21.15.143:0/3106880283 >> v1:172.21.15.204:6789/0 conn(0x7f79fc06a490 legacy=0x7f79fc071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.191+0000 7f79fb400640 1 -- 172.21.15.143:0/3106880283 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc161750 con 0x7f79fc15d130 2024-01-15T18:23:36.186 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f79fb400640 1 --2- 172.21.15.143:0/3106880283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 0x7f79fc0aeed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009870 tx=0x7f79ec01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20188ce0d7303e9d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f79fa000640 1 -- 172.21.15.143:0/3106880283 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0a9620 con 0x7f79fc15d130 2024-01-15T18:23:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f79fa000640 1 -- 172.21.15.143:0/3106880283 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0a9780 con 0x7f79fc15d130 2024-01-15T18:23:36.187 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f79fa000640 1 -- 172.21.15.143:0/3106880283 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0a9b10 con 0x7f79fc15d130 2024-01-15T18:23:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f7a02000640 1 -- 172.21.15.143:0/3106880283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 msgr2=0x7f79fc0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f7a02000640 1 --2- 172.21.15.143:0/3106880283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 0x7f79fc0aeed0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009870 tx=0x7f79ec01c540 comp rx=0 tx=0).stop 2024-01-15T18:23:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f7a02000640 1 -- 172.21.15.143:0/3106880283 shutdown_connections 2024-01-15T18:23:36.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f7a02000640 1 --2- 172.21.15.143:0/3106880283 >> v2:172.21.15.204:3300/0 conn(0x7f79fc15d510 0x7f79fc06e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f7a02000640 1 --2- 172.21.15.143:0/3106880283 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d130 0x7f79fc0aeed0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.195+0000 7f7a02000640 1 -- 172.21.15.143:0/3106880283 >> 172.21.15.143:0/3106880283 conn(0x7f79fc0b1b90 msgr2=0x7f79fc0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 -- 172.21.15.143:0/3106880283 shutdown_connections 2024-01-15T18:23:36.189 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 -- 172.21.15.143:0/3106880283 wait complete. 2024-01-15T18:23:36.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 Processor -- start 2024-01-15T18:23:36.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 -- start start 2024-01-15T18:23:36.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 0x7f79fc148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79fc15d130 0x7f79fc0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d510 0x7f79fc0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc14c0b0 con 0x7f79fc15d130 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc148a90 con 0x7f79fc06a490 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f7a02000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79fc148580 con 0x7f79fc15d510 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79faa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79fc15d130 0x7f79fc0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:36.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 0x7f79fc148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79faa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79fc15d130 0x7f79fc0ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:52340/0 (socket says 172.21.15.143:52340) 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79faa00640 1 -- 172.21.15.143:0/4275199817 learned_addr learned my addr 172.21.15.143:0/4275199817 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 0x7f79fc148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:48374/0 (socket says 172.21.15.143:48374) 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fbe00640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d510 0x7f79fc0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 -- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d510 msgr2=0x7f79fc0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d510 0x7f79fc0adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 -- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79fc15d130 msgr2=0x7f79fc0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.193 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79fc15d130 0x7f79fc0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 -- 172.21.15.143:0/4275199817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0094d0 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.199+0000 7f79fb400640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 0x7f79fc148040 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0aa000 tx=0x7f79ec01ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0bc650 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec0bc7f0 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec0bcba0 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc14b4a0 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79fc14ba60 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79fc067500 con 0x7f79fc06a490 2024-01-15T18:23:36.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f79ec0b13f0 con 0x7f79fc06a490 2024-01-15T18:23:36.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79f3400640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79d4067b80 0x7f79d406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:36.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f79ec11d5f0 con 0x7f79fc06a490 2024-01-15T18:23:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79faa00640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79d4067b80 0x7f79d406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:36.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.203+0000 7f79faa00640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79d4067b80 0x7f79d406a040 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f79fc133050 tx=0x7f79e4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:36.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.207+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79ec0ec450 con 0x7f79fc06a490 2024-01-15T18:23:36.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.367+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79fc06ada0 con 0x7f79fc06a490 2024-01-15T18:23:36.361 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f79f3400640 1 -- 172.21.15.143:0/4275199817 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79ec0efa70 con 0x7f79fc06a490 2024-01-15T18:23:36.361 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:23:36.363 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79d4067b80 msgr2=0x7f79d406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79d4067b80 0x7f79d406a040 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f79fc133050 tx=0x7f79e4009190 comp rx=0 tx=0).stop 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 msgr2=0x7f79fc148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 0x7f79fc148040 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0aa000 tx=0x7f79ec01ad80 comp rx=0 tx=0).stop 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 shutdown_connections 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79d4067b80 0x7f79d406a040 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79fc15d510 0x7f79fc0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79fc15d130 0x7f79fc0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 --2- 172.21.15.143:0/4275199817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79fc06a490 0x7f79fc148040 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:36.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 >> 172.21.15.143:0/4275199817 conn(0x7f79fc0b1b90 msgr2=0x7f79fc1523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:36.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 shutdown_connections 2024-01-15T18:23:36.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:36.371+0000 7f7a02000640 1 -- 172.21.15.143:0/4275199817 wait complete. 2024-01-15T18:23:36.383 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi143.front.sepia.ceph.com... 2024-01-15T18:23:36.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:36.384 DEBUG:teuthology.orchestra.run.smithi143:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:23:36.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi143.stderr:ceph-fuse[53117]: fuse finished with error 0 and tester_r 0 2024-01-15T18:23:36.978 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:23:36.978 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:23:36.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:36.978 DEBUG:teuthology.orchestra.run.smithi143:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:23:36.988 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-01-15T18:23:36.988 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:37.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354644k,nr_inodes=4088661,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2774 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275372k,nr_inodes=818843,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275376k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:23:37.040 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:23:37.040 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:37.050 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:23:37.050 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:37.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:23:37.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.247+0000 7f0f0be00640 1 Processor -- start 2024-01-15T18:23:37.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.247+0000 7f0f0be00640 1 -- start start 2024-01-15T18:23:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f0be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0f04071bb0 0x7f0f04072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f0be00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f0f0415c5a0 0x7f0f0415c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f0be00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f040ceb50 con 0x7f0f04143780 2024-01-15T18:23:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f0be00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f04072570 con 0x7f0f04071bb0 2024-01-15T18:23:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f0be00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0415c110 con 0x7f0f0415c5a0 2024-01-15T18:23:37.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f09400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0f04143780 0x7f0f040cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60592/0 (socket says 172.21.15.143:60592) 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f08a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f0f04071bb0 0x7f0f04072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f09400640 1 -- 172.21.15.143:0/1791140562 learned_addr learned my addr 172.21.15.143:0/1791140562 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f09e00640 1 --2- 172.21.15.143:0/1791140562 >> v2:172.21.15.204:3300/0 conn(0x7f0f0415c5a0 0x7f0f0415c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f08a00640 1 -- 172.21.15.143:0/1791140562 >> v2:172.21.15.204:3300/0 conn(0x7f0f0415c5a0 msgr2=0x7f0f0415c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f08a00640 1 --2- 172.21.15.143:0/1791140562 >> v2:172.21.15.204:3300/0 conn(0x7f0f0415c5a0 0x7f0f0415c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f08a00640 1 -- 172.21.15.143:0/1791140562 >> v1:172.21.15.143:6789/0 conn(0x7f0f04143780 legacy=0x7f0f040cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.251+0000 7f0f08a00640 1 -- 172.21.15.143:0/1791140562 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0415b7e0 con 0x7f0f04071bb0 2024-01-15T18:23:37.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f08a00640 1 --2- 172.21.15.143:0/1791140562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04071bb0 0x7f0f04072030 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0f04070a20 tx=0x7f0eec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6f7e0e71bc9aee server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f03e00640 1 -- 172.21.15.143:0/1791140562 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0ab660 con 0x7f0f04071bb0 2024-01-15T18:23:37.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f03e00640 1 -- 172.21.15.143:0/1791140562 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec0ab7c0 con 0x7f0f04071bb0 2024-01-15T18:23:37.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f03e00640 1 -- 172.21.15.143:0/1791140562 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0abb50 con 0x7f0f04071bb0 2024-01-15T18:23:37.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 -- 172.21.15.143:0/1791140562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04071bb0 msgr2=0x7f0f04072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 --2- 172.21.15.143:0/1791140562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04071bb0 0x7f0f04072030 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f0f04070a20 tx=0x7f0eec01c460 comp rx=0 tx=0).stop 2024-01-15T18:23:37.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 -- 172.21.15.143:0/1791140562 shutdown_connections 2024-01-15T18:23:37.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 --2- 172.21.15.143:0/1791140562 >> v2:172.21.15.204:3300/0 conn(0x7f0f0415c5a0 0x7f0f0415c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 --2- 172.21.15.143:0/1791140562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04071bb0 0x7f0f04072030 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 -- 172.21.15.143:0/1791140562 >> 172.21.15.143:0/1791140562 conn(0x7f0f0406c3f0 msgr2=0x7f0f0406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:37.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 -- 172.21.15.143:0/1791140562 shutdown_connections 2024-01-15T18:23:37.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 -- 172.21.15.143:0/1791140562 wait complete. 2024-01-15T18:23:37.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 Processor -- start 2024-01-15T18:23:37.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.255+0000 7f0f0be00640 1 -- start start 2024-01-15T18:23:37.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f04071bb0 0x7f0f04159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 0x7f0f040b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f0415c5a0 0x7f0f040b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041423a0 con 0x7f0f04143780 2024-01-15T18:23:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04159550 con 0x7f0f04071bb0 2024-01-15T18:23:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f041427c0 con 0x7f0f0415c5a0 2024-01-15T18:23:37.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 0x7f0f040b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 0x7f0f040b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37890/0 (socket says 172.21.15.143:37890) 2024-01-15T18:23:37.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 -- 172.21.15.143:0/2389125079 learned_addr learned my addr 172.21.15.143:0/2389125079 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:37.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f09400640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f04071bb0 0x7f0f04159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f09e00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f0415c5a0 0x7f0f040b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 -- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f0415c5a0 msgr2=0x7f0f040b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f0415c5a0 0x7f0f040b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 -- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f04071bb0 msgr2=0x7f0f04159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f04071bb0 0x7f0f04159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 -- 172.21.15.143:0/2389125079 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec009420 con 0x7f0f04143780 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f08a00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 0x7f0f040b3010 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0ad040 tx=0x7f0eec01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0ab660 con 0x7f0f04143780 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec0b5830 con 0x7f0f04143780 2024-01-15T18:23:37.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f04141720 con 0x7f0f04143780 2024-01-15T18:23:37.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0b5be0 con 0x7f0f04143780 2024-01-15T18:23:37.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f04141ce0 con 0x7f0f04143780 2024-01-15T18:23:37.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f041424e0 con 0x7f0f04143780 2024-01-15T18:23:37.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.259+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f0eec0a9070 con 0x7f0f04143780 2024-01-15T18:23:37.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.263+0000 7f0f01600640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0edc067b80 0x7f0edc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.263+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f0eec11dc00 con 0x7f0f04143780 2024-01-15T18:23:37.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.263+0000 7f0f09400640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0edc067b80 0x7f0edc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.263+0000 7f0f09400640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0edc067b80 0x7f0edc06a040 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005890 tx=0x7f0ef4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.267+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f0f041424e0 con 0x7f0f04143780 2024-01-15T18:23:37.410 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.419+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f041424e0 con 0x7f0f04143780 2024-01-15T18:23:37.411 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.419+0000 7f0f01600640 1 -- 172.21.15.143:0/2389125079 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16745 (secure 0 0 0) 0x7f0f041424e0 con 0x7f0f04143780 2024-01-15T18:23:37.412 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:37.412 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":113,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:19:41.863194+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:23:37.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0edc067b80 msgr2=0x7f0edc06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0edc067b80 0x7f0edc06a040 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005890 tx=0x7f0ef4009040 comp rx=0 tx=0).stop 2024-01-15T18:23:37.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 msgr2=0x7f0f040b3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 0x7f0f040b3010 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0ad040 tx=0x7f0eec01aca0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 shutdown_connections 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f0edc067b80 0x7f0edc06a040 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f0f0415c5a0 0x7f0f040b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f04143780 0x7f0f040b3010 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 --2- 172.21.15.143:0/2389125079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f04071bb0 0x7f0f04159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 >> 172.21.15.143:0/2389125079 conn(0x7f0f0406c3f0 msgr2=0x7f0f040ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:37.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 shutdown_connections 2024-01-15T18:23:37.416 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.423+0000 7f0f0be00640 1 -- 172.21.15.143:0/2389125079 wait complete. 2024-01-15T18:23:37.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:18:39.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:18:40.286820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:23:37.434 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:23:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11bd800640 1 Processor -- start 2024-01-15T18:23:37.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11bd800640 1 -- start start 2024-01-15T18:23:37.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11bd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 0x7f11b815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11bd800640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b80ceb50 con 0x7f11b8071bb0 2024-01-15T18:23:37.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11bd800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b815c110 con 0x7f11b8143780 2024-01-15T18:23:37.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11bd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80724c0 con 0x7f11b8072030 2024-01-15T18:23:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f11b8071bb0 0x7f11b815c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60604/0 (socket says 172.21.15.143:60604) 2024-01-15T18:23:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 -- 172.21.15.143:0/1923052085 learned_addr learned my addr 172.21.15.143:0/1923052085 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b7400640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 0x7f11b815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916254771 0 0) 0x7f11a0001120 con 0x7f11b8071bb0 2024-01-15T18:23:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1198003530 con 0x7f11b8071bb0 2024-01-15T18:23:37.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959053337 0 0) 0x7f11b815c110 con 0x7f11b8143780 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a0001120 con 0x7f11b8143780 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557464843 0 0) 0x7f1198003530 con 0x7f11b8071bb0 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 msgr2=0x7f11b815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 0x7f11b815ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 >> v1:172.21.15.204:6789/0 conn(0x7f11b8143780 legacy=0x7f11b80cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b815b7e0 con 0x7f11b8071bb0 2024-01-15T18:23:37.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f11a0001830 con 0x7f11b8071bb0 2024-01-15T18:23:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 >> v1:172.21.15.143:6789/0 conn(0x7f11b8071bb0 legacy=0x7f11b815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1198003b40 0x7f1198003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11980047d0 0x7f1198004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 0x7f1198005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b80724c0 con 0x7f11980047d0 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119800ba40 con 0x7f1198003b40 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119800bbd0 con 0x7f11b8072030 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11980047d0 0x7f1198004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6a00640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1198003b40 0x7f1198003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b7400640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 0x7f1198005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 -- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 msgr2=0x7f1198005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11b8072030 0x7f1198005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 -- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1198003b40 msgr2=0x7f1198003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:37.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1198003b40 0x7f1198003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.583+0000 7f11b6000640 1 -- 172.21.15.143:0/1923052085 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1198003530 con 0x7f11980047d0 2024-01-15T18:23:37.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11b6000640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11980047d0 0x7f1198004c50 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f11a000c160 tx=0x7f11a001cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db92a16c01005eef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a00058e0 con 0x7f11980047d0 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a0005a40 con 0x7f11980047d0 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11b5600640 1 -- 172.21.15.143:0/1923052085 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0005df0 con 0x7f11980047d0 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11bd800640 1 -- 172.21.15.143:0/1923052085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11980047d0 msgr2=0x7f1198004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11bd800640 1 --2- 172.21.15.143:0/1923052085 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11980047d0 0x7f1198004c50 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f11a000c160 tx=0x7f11a001cb20 comp rx=0 tx=0).stop 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11b7400640 1 -- 172.21.15.143:0/1923052085 reap_dead start 2024-01-15T18:23:37.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11bd800640 1 -- 172.21.15.143:0/1923052085 shutdown_connections 2024-01-15T18:23:37.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.587+0000 7f11bd800640 1 -- 172.21.15.143:0/1923052085 >> 172.21.15.143:0/1923052085 conn(0x7f11b806c3f0 msgr2=0x7f11b806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:37.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 -- 172.21.15.143:0/1923052085 shutdown_connections 2024-01-15T18:23:37.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 -- 172.21.15.143:0/1923052085 wait complete. 2024-01-15T18:23:37.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 Processor -- start 2024-01-15T18:23:37.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 -- start start 2024-01-15T18:23:37.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11980047d0 0x7f11b81e6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b8158f10 0x7f11b81f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 0x7f11b813df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b813d700 con 0x7f11b813da80 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b813e470 con 0x7f11b8158f10 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11bd800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81e6050 con 0x7f11980047d0 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 0x7f11b813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 0x7f11b813df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37912/0 (socket says 172.21.15.143:37912) 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 -- 172.21.15.143:0/1416539381 learned_addr learned my addr 172.21.15.143:0/1416539381 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b6000640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b8158f10 0x7f11b81f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b6a00640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11980047d0 0x7f11b81e6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 -- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11980047d0 msgr2=0x7f11b81e6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11980047d0 0x7f11b81e6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 -- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b8158f10 msgr2=0x7f11b81f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b8158f10 0x7f11b81f2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.591+0000 7f11b7400640 1 -- 172.21.15.143:0/1416539381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b8159aa0 con 0x7f11b813da80 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11b7400640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 0x7f11b813df30 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f11b8074700 tx=0x7f11b0011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b000e040 con 0x7f11b813da80 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b000f040 con 0x7f11b813da80 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0015430 con 0x7f11b813da80 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b8140de0 con 0x7f11b813da80 2024-01-15T18:23:37.591 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b81413a0 con 0x7f11b813da80 2024-01-15T18:23:37.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b8067500 con 0x7f11b813da80 2024-01-15T18:23:37.592 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f11b0015590 con 0x7f11b813da80 2024-01-15T18:23:37.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11aea00640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1190067b80 0x7f119006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f11b00838f0 con 0x7f11b813da80 2024-01-15T18:23:37.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11b6a00640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1190067b80 0x7f119006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.595+0000 7f11b6a00640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1190067b80 0x7f119006a040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f11a80014f0 tx=0x7f11a80013e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.599+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f11b0052200 con 0x7f11b813da80 2024-01-15T18:23:37.741 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11b813f9f0 con 0x7f11b813da80 2024-01-15T18:23:37.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11aea00640 1 -- 172.21.15.143:0/1416539381 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11b0055820 con 0x7f11b813da80 2024-01-15T18:23:37.742 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:23:37.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1190067b80 msgr2=0x7f119006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1190067b80 0x7f119006a040 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f11a80014f0 tx=0x7f11a80013e0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 msgr2=0x7f11b813df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 0x7f11b813df30 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f11b8074700 tx=0x7f11b0011f50 comp rx=0 tx=0).stop 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 shutdown_connections 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1190067b80 0x7f119006a040 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f11b813da80 0x7f11b813df30 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f11b8158f10 0x7f11b81f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 --2- 172.21.15.143:0/1416539381 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f11980047d0 0x7f11b81e6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 >> 172.21.15.143:0/1416539381 conn(0x7f11b806c3f0 msgr2=0x7f11b813ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:37.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 shutdown_connections 2024-01-15T18:23:37.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.751+0000 7f11bd800640 1 -- 172.21.15.143:0/1416539381 wait complete. 2024-01-15T18:23:37.764 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:23:37.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.911+0000 7fe36f600640 1 Processor -- start 2024-01-15T18:23:37.903 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.911+0000 7fe36f600640 1 -- start start 2024-01-15T18:23:37.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe36f600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe368159b70 0x7fe3680b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe36f600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3680b4d20 con 0x7fe368157300 2024-01-15T18:23:37.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe36f600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3680b2a20 con 0x7fe3680b48f0 2024-01-15T18:23:37.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe36f600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3680b35a0 con 0x7fe368159b70 2024-01-15T18:23:37.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe36cc00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe368157300 0x7fe3680b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60616/0 (socket says 172.21.15.143:60616) 2024-01-15T18:23:37.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe36cc00640 1 -- 172.21.15.143:0/576362346 learned_addr learned my addr 172.21.15.143:0/576362346 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:37.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe367e00640 1 --2- 172.21.15.143:0/576362346 >> v2:172.21.15.204:3300/0 conn(0x7fe368159b70 0x7fe3680b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe367400640 1 -- 172.21.15.143:0/576362346 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081809991 0 0) 0x7fe3680b4d20 con 0x7fe368157300 2024-01-15T18:23:37.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe367400640 1 -- 172.21.15.143:0/576362346 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348003450 con 0x7fe368157300 2024-01-15T18:23:37.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe367e00640 1 -- 172.21.15.143:0/576362346 >> v1:172.21.15.204:6789/0 conn(0x7fe3680b48f0 legacy=0x7fe368070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe367e00640 1 -- 172.21.15.143:0/576362346 >> v1:172.21.15.143:6789/0 conn(0x7fe368157300 legacy=0x7fe3680b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.915+0000 7fe367e00640 1 -- 172.21.15.143:0/576362346 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680b2d00 con 0x7fe368159b70 2024-01-15T18:23:37.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe367e00640 1 --2- 172.21.15.143:0/576362346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368159b70 0x7fe3680b43b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe3500097c0 tx=0x7fe35001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31fa702199f63a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe367400640 1 -- 172.21.15.143:0/576362346 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500ab650 con 0x7fe368159b70 2024-01-15T18:23:37.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe367400640 1 -- 172.21.15.143:0/576362346 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3500ab7b0 con 0x7fe368159b70 2024-01-15T18:23:37.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe367400640 1 -- 172.21.15.143:0/576362346 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500abb40 con 0x7fe368159b70 2024-01-15T18:23:37.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 -- 172.21.15.143:0/576362346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368159b70 msgr2=0x7fe3680b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 --2- 172.21.15.143:0/576362346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368159b70 0x7fe3680b43b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe3500097c0 tx=0x7fe35001c450 comp rx=0 tx=0).stop 2024-01-15T18:23:37.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 -- 172.21.15.143:0/576362346 shutdown_connections 2024-01-15T18:23:37.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 --2- 172.21.15.143:0/576362346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368159b70 0x7fe3680b43b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 -- 172.21.15.143:0/576362346 >> 172.21.15.143:0/576362346 conn(0x7fe36806c3f0 msgr2=0x7fe36806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:37.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 -- 172.21.15.143:0/576362346 shutdown_connections 2024-01-15T18:23:37.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 -- 172.21.15.143:0/576362346 wait complete. 2024-01-15T18:23:37.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 Processor -- start 2024-01-15T18:23:37.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.919+0000 7fe36f600640 1 -- start start 2024-01-15T18:23:37.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b48f0 0x7fe36815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368157300 0x7fe368154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 0x7fe368154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368159fd0 con 0x7fe368159b70 2024-01-15T18:23:37.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681557b0 con 0x7fe368157300 2024-01-15T18:23:37.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3681508d0 con 0x7fe3680b48f0 2024-01-15T18:23:37.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 0x7fe368154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 0x7fe368154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37926/0 (socket says 172.21.15.143:37926) 2024-01-15T18:23:37.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 -- 172.21.15.143:0/3375716979 learned_addr learned my addr 172.21.15.143:0/3375716979 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:37.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe367e00640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368157300 0x7fe368154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36cc00640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b48f0 0x7fe36815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 -- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b48f0 msgr2=0x7fe36815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b48f0 0x7fe36815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 -- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368157300 msgr2=0x7fe368154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368157300 0x7fe368154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 -- 172.21.15.143:0/3375716979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009420 con 0x7fe368159b70 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe367e00640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368157300 0x7fe368154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36d600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 0x7fe368154ca0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c6c0 tx=0x7fe36000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360003b80 con 0x7fe368159b70 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe360003ce0 con 0x7fe368159b70 2024-01-15T18:23:37.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3681554d0 con 0x7fe368159b70 2024-01-15T18:23:37.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3600052f0 con 0x7fe368159b70 2024-01-15T18:23:37.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe368150c20 con 0x7fe368159b70 2024-01-15T18:23:37.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.923+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368155240 con 0x7fe368159b70 2024-01-15T18:23:37.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.927+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fe360005450 con 0x7fe368159b70 2024-01-15T18:23:37.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.927+0000 7fe364c00640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe340067b80 0x7fe34006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:37.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.927+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fe360083040 con 0x7fe368159b70 2024-01-15T18:23:37.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.927+0000 7fe36cc00640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe340067b80 0x7fe34006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:37.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.927+0000 7fe36cc00640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe340067b80 0x7fe34006a040 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe3580011f0 tx=0x7fe358007d40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:37.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:37.931+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe360051a00 con 0x7fe368159b70 2024-01-15T18:23:38.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.083+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe368150a10 con 0x7fe368159b70 2024-01-15T18:23:38.075 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.083+0000 7fe364c00640 1 -- 172.21.15.143:0/3375716979 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe360055020 con 0x7fe368159b70 2024-01-15T18:23:38.076 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:23:38.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe340067b80 msgr2=0x7fe34006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.078 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe340067b80 0x7fe34006a040 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe3580011f0 tx=0x7fe358007d40 comp rx=0 tx=0).stop 2024-01-15T18:23:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 msgr2=0x7fe368154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 0x7fe368154ca0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c6c0 tx=0x7fe36000cb80 comp rx=0 tx=0).stop 2024-01-15T18:23:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 shutdown_connections 2024-01-15T18:23:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe340067b80 0x7fe34006a040 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe368159b70 0x7fe368154ca0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe368157300 0x7fe368154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 --2- 172.21.15.143:0/3375716979 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3680b48f0 0x7fe36815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 >> 172.21.15.143:0/3375716979 conn(0x7fe36806c3f0 msgr2=0x7fe36806d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 shutdown_connections 2024-01-15T18:23:38.080 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.087+0000 7fe36f600640 1 -- 172.21.15.143:0/3375716979 wait complete. 2024-01-15T18:23:38.098 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:23:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 Processor -- start 2024-01-15T18:23:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 -- start start 2024-01-15T18:23:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fe3e4143780 0x7fe3e40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.238 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3e40ce520 0x7fe3e4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e40ceb50 con 0x7fe3e40720f0 2024-01-15T18:23:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e415c110 con 0x7fe3e40ce520 2024-01-15T18:23:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3eb200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e4072530 con 0x7fe3e4143780 2024-01-15T18:23:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e9200640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe3e40720f0 0x7fe3e415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60628/0 (socket says 172.21.15.143:60628) 2024-01-15T18:23:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e3400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe3e40ce520 0x7fe3e4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.239 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e9200640 1 -- 172.21.15.143:0/1894868825 learned_addr learned my addr 172.21.15.143:0/1894868825 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/1894868825 >> v2:172.21.15.204:3300/0 conn(0x7fe3e4143780 0x7fe3e40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e3400640 1 -- 172.21.15.143:0/1894868825 >> v2:172.21.15.204:3300/0 conn(0x7fe3e4143780 msgr2=0x7fe3e40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e3400640 1 --2- 172.21.15.143:0/1894868825 >> v2:172.21.15.204:3300/0 conn(0x7fe3e4143780 0x7fe3e40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.240 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e3400640 1 -- 172.21.15.143:0/1894868825 >> v1:172.21.15.143:6789/0 conn(0x7fe3e40720f0 legacy=0x7fe3e415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e3400640 1 -- 172.21.15.143:0/1894868825 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e415b7e0 con 0x7fe3e40ce520 2024-01-15T18:23:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.247+0000 7fe3e2a00640 1 -- 172.21.15.143:0/1894868825 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103876699 0 0) 0x7fe3dc0014a0 con 0x7fe3e40720f0 2024-01-15T18:23:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3e3400640 1 --2- 172.21.15.143:0/1894868825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e4071bb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe3e406be30 tx=0x7fe3cc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed557139d3a72e59 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3e2a00640 1 -- 172.21.15.143:0/1894868825 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0ab600 con 0x7fe3e40ce520 2024-01-15T18:23:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3e2a00640 1 -- 172.21.15.143:0/1894868825 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0a7050 con 0x7fe3e40ce520 2024-01-15T18:23:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3e2a00640 1 -- 172.21.15.143:0/1894868825 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0ab980 con 0x7fe3e40ce520 2024-01-15T18:23:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 -- 172.21.15.143:0/1894868825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 msgr2=0x7fe3e4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 --2- 172.21.15.143:0/1894868825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e4071bb0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe3e406be30 tx=0x7fe3cc01c460 comp rx=0 tx=0).stop 2024-01-15T18:23:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 -- 172.21.15.143:0/1894868825 shutdown_connections 2024-01-15T18:23:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 --2- 172.21.15.143:0/1894868825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e4071bb0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 --2- 172.21.15.143:0/1894868825 >> v2:172.21.15.204:3300/0 conn(0x7fe3e4143780 0x7fe3e40cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 -- 172.21.15.143:0/1894868825 >> 172.21.15.143:0/1894868825 conn(0x7fe3e406c3f0 msgr2=0x7fe3e406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 -- 172.21.15.143:0/1894868825 shutdown_connections 2024-01-15T18:23:38.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.251+0000 7fe3eb200640 1 -- 172.21.15.143:0/1894868825 wait complete. 2024-01-15T18:23:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 Processor -- start 2024-01-15T18:23:38.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 -- start start 2024-01-15T18:23:38.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40720f0 0x7fe3e4159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e413bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e4143780 0x7fe3e413c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41fa340 con 0x7fe3e40ce520 2024-01-15T18:23:38.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41f9730 con 0x7fe3e4143780 2024-01-15T18:23:38.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e41f9a10 con 0x7fe3e40720f0 2024-01-15T18:23:38.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e413bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e413bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37940/0 (socket says 172.21.15.143:37940) 2024-01-15T18:23:38.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 -- 172.21.15.143:0/3264687084 learned_addr learned my addr 172.21.15.143:0/3264687084 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:38.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e9200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e4143780 0x7fe3e413c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40720f0 0x7fe3e4159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 -- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40720f0 msgr2=0x7fe3e4159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40720f0 0x7fe3e4159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 -- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e4143780 msgr2=0x7fe3e413c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e4143780 0x7fe3e413c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 -- 172.21.15.143:0/3264687084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009420 con 0x7fe3e40ce520 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40720f0 0x7fe3e4159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e9200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e4143780 0x7fe3e413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:38.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3e3400640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e413bda0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc019070 tx=0x7fe3cc019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc006e20 con 0x7fe3e40ce520 2024-01-15T18:23:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc004030 con 0x7fe3e40ce520 2024-01-15T18:23:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0043a0 con 0x7fe3e40ce520 2024-01-15T18:23:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e41f9cf0 con 0x7fe3e40ce520 2024-01-15T18:23:38.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.255+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4135260 con 0x7fe3e40ce520 2024-01-15T18:23:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.259+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e41fa4f0 con 0x7fe3e40ce520 2024-01-15T18:23:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.259+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fe3cc0b2040 con 0x7fe3e40ce520 2024-01-15T18:23:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.259+0000 7fe3dbe00640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc067b80 0x7fe3bc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.259+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fe3cc11c4d0 con 0x7fe3e40ce520 2024-01-15T18:23:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.259+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc067b80 0x7fe3bc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.259+0000 7fe3e3e00640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc067b80 0x7fe3bc06a040 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009a40 tx=0x7fe3d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.263+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fe3e41fa4f0 con 0x7fe3e40ce520 2024-01-15T18:23:38.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.411+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3e4067500 con 0x7fe3e40ce520 2024-01-15T18:23:38.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.411+0000 7fe3dbe00640 1 -- 172.21.15.143:0/3264687084 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3cc0bc070 con 0x7fe3e40ce520 2024-01-15T18:23:38.404 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:23:38.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc067b80 msgr2=0x7fe3bc06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc067b80 0x7fe3bc06a040 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009a40 tx=0x7fe3d4008040 comp rx=0 tx=0).stop 2024-01-15T18:23:38.406 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 msgr2=0x7fe3e413bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e413bda0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc019070 tx=0x7fe3cc019200 comp rx=0 tx=0).stop 2024-01-15T18:23:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 shutdown_connections 2024-01-15T18:23:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fe3bc067b80 0x7fe3bc06a040 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe3e4143780 0x7fe3e413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3e40ce520 0x7fe3e413bda0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.407 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 --2- 172.21.15.143:0/3264687084 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fe3e40720f0 0x7fe3e4159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 >> 172.21.15.143:0/3264687084 conn(0x7fe3e406c3f0 msgr2=0x7fe3e40ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:38.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 shutdown_connections 2024-01-15T18:23:38.408 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.415+0000 7fe3eb200640 1 -- 172.21.15.143:0/3264687084 wait complete. 2024-01-15T18:23:38.426 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:23:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 Processor -- start 2024-01-15T18:23:38.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 -- start start 2024-01-15T18:23:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec15d130 0x7f9bec0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9bec06aba0 0x7f9bec06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bec160eb0 con 0x7f9bec15d510 2024-01-15T18:23:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bec161190 con 0x7f9bec06aba0 2024-01-15T18:23:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec161470 con 0x7f9bec15d130 2024-01-15T18:23:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf1e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9bec06aba0 0x7f9bec06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf0a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9bec15d510 0x7f9bec06a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60638/0 (socket says 172.21.15.143:60638) 2024-01-15T18:23:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf0a00640 1 -- 172.21.15.143:0/1243836288 learned_addr learned my addr 172.21.15.143:0/1243836288 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:38.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf1400640 1 --2- 172.21.15.143:0/1243836288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec15d130 0x7f9bec0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf1e00640 1 -- 172.21.15.143:0/1243836288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec15d130 msgr2=0x7f9bec0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf1e00640 1 --2- 172.21.15.143:0/1243836288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec15d130 0x7f9bec0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf1e00640 1 -- 172.21.15.143:0/1243836288 >> v1:172.21.15.143:6789/0 conn(0x7f9bec15d510 legacy=0x7f9bec06a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bf1e00640 1 -- 172.21.15.143:0/1243836288 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec161750 con 0x7f9bec06aba0 2024-01-15T18:23:38.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.575+0000 7f9bebe00640 1 -- 172.21.15.143:0/1243836288 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635569166 0 0) 0x7f9bec160eb0 con 0x7f9bec15d510 2024-01-15T18:23:38.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf1e00640 1 --2- 172.21.15.143:0/1243836288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec06aba0 0x7f9bec06e060 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009d90 tx=0x7f9be401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5679c00bcc7906c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bebe00640 1 -- 172.21.15.143:0/1243836288 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40a5a30 con 0x7f9bec06aba0 2024-01-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bebe00640 1 -- 172.21.15.143:0/1243836288 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be40a5b90 con 0x7f9bec06aba0 2024-01-15T18:23:38.570 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bebe00640 1 -- 172.21.15.143:0/1243836288 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40ad440 con 0x7f9bec06aba0 2024-01-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 -- 172.21.15.143:0/1243836288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec06aba0 msgr2=0x7f9bec06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/1243836288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec06aba0 0x7f9bec06e060 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009d90 tx=0x7f9be401c950 comp rx=0 tx=0).stop 2024-01-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 -- 172.21.15.143:0/1243836288 shutdown_connections 2024-01-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/1243836288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec06aba0 0x7f9bec06e060 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/1243836288 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec15d130 0x7f9bec0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 -- 172.21.15.143:0/1243836288 >> 172.21.15.143:0/1243836288 conn(0x7f9bec0b1b90 msgr2=0x7f9bec0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:38.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 -- 172.21.15.143:0/1243836288 shutdown_connections 2024-01-15T18:23:38.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.579+0000 7f9bf3e00640 1 -- 172.21.15.143:0/1243836288 wait complete. 2024-01-15T18:23:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 Processor -- start 2024-01-15T18:23:38.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- start start 2024-01-15T18:23:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec06aba0 0x7f9bec158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15d130 0x7f9bec15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 0x7f9bec140400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec152f90 con 0x7f9bec15d510 2024-01-15T18:23:38.575 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec153270 con 0x7f9bec15d130 2024-01-15T18:23:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec153550 con 0x7f9bec06aba0 2024-01-15T18:23:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 0x7f9bec140400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 0x7f9bec140400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37962/0 (socket says 172.21.15.143:37962) 2024-01-15T18:23:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 -- 172.21.15.143:0/3427514731 learned_addr learned my addr 172.21.15.143:0/3427514731 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf0a00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15d130 0x7f9bec15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1400640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec06aba0 0x7f9bec158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 -- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec06aba0 msgr2=0x7f9bec158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec06aba0 0x7f9bec158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 -- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15d130 msgr2=0x7f9bec15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15d130 0x7f9bec15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 -- 172.21.15.143:0/3427514731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be4009a70 con 0x7f9bec15d510 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf1e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 0x7f9bec140400 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9be401b000 tx=0x7f9be40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40a5a30 con 0x7f9bec15d510 2024-01-15T18:23:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be4009450 con 0x7f9bec15d510 2024-01-15T18:23:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec153830 con 0x7f9bec15d510 2024-01-15T18:23:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be40b9670 con 0x7f9bec15d510 2024-01-15T18:23:38.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec1589a0 con 0x7f9bec15d510 2024-01-15T18:23:38.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.583+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec158b90 con 0x7f9bec15d510 2024-01-15T18:23:38.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.587+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f9be40b97d0 con 0x7f9bec15d510 2024-01-15T18:23:38.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.587+0000 7f9be9600640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bc4067b80 0x7f9bc406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.587+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f9be411ead0 con 0x7f9bec15d510 2024-01-15T18:23:38.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.587+0000 7f9bf1400640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bc4067b80 0x7f9bc406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.587+0000 7f9bf1400640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bc4067b80 0x7f9bc406a040 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009b60 tx=0x7f9bdc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.581 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.591+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9bec158b90 con 0x7f9bec15d510 2024-01-15T18:23:38.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9bec158b90 con 0x7f9bec15d510 2024-01-15T18:23:38.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9be9600640 1 -- 172.21.15.143:0/3427514731 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9bec158b90 con 0x7f9bec15d510 2024-01-15T18:23:38.746 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:23:38.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bc4067b80 msgr2=0x7f9bc406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bc4067b80 0x7f9bc406a040 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009b60 tx=0x7f9bdc009190 comp rx=0 tx=0).stop 2024-01-15T18:23:38.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 msgr2=0x7f9bec140400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 0x7f9bec140400 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f9be401b000 tx=0x7f9be40196f0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 shutdown_connections 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9bc4067b80 0x7f9bc406a040 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9bec15d510 0x7f9bec140400 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9bec15d130 0x7f9bec15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 --2- 172.21.15.143:0/3427514731 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9bec06aba0 0x7f9bec158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 >> 172.21.15.143:0/3427514731 conn(0x7f9bec0b1b90 msgr2=0x7f9bec1598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 shutdown_connections 2024-01-15T18:23:38.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.755+0000 7f9bf3e00640 1 -- 172.21.15.143:0/3427514731 wait complete. 2024-01-15T18:23:38.768 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:23:38.768 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:38.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:23:38.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 Processor -- start 2024-01-15T18:23:38.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 -- start start 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fbcc00cfb30 0x7fbcc006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc00cffd0 con 0x7fbcc006c230 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc006c670 con 0x7fbcc00cfb30 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcc4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00d1ce0 con 0x7fbcc01607c0 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcbe800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbcc006c230 0x7fbcc00cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60654/0 (socket says 172.21.15.143:60654) 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcbe800640 1 -- 172.21.15.143:0/3328172958 learned_addr learned my addr 172.21.15.143:0/3328172958 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:38.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcbd400640 1 --2- 172.21.15.143:0/3328172958 >> v2:172.21.15.204:3300/0 conn(0x7fbcc00cfb30 0x7fbcc006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.923+0000 7fbcbde00640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbca00640 1 -- 172.21.15.143:0/3328172958 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273648335 0 0) 0x7fbcc00cffd0 con 0x7fbcc006c230 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbca00640 1 -- 172.21.15.143:0/3328172958 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca0003530 con 0x7fbcc006c230 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbd400640 1 -- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 msgr2=0x7fbcc006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbd400640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbd400640 1 -- 172.21.15.143:0/3328172958 >> v1:172.21.15.143:6789/0 conn(0x7fbcc006c230 legacy=0x7fbcc00cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbd400640 1 -- 172.21.15.143:0/3328172958 --> v2:172.21.15.204:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00d02b0 con 0x7fbcc00cfb30 2024-01-15T18:23:38.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbde00640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbd400640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc00cfb30 0x7fbcc006bcf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097c0 tx=0x7fbca801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57e3991edf617be server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbca00640 1 -- 172.21.15.143:0/3328172958 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80a8970 con 0x7fbcc00cfb30 2024-01-15T18:23:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbca00640 1 -- 172.21.15.143:0/3328172958 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80a8ad0 con 0x7fbcc00cfb30 2024-01-15T18:23:38.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.927+0000 7fbcbca00640 1 -- 172.21.15.143:0/3328172958 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80a8e40 con 0x7fbcc00cfb30 2024-01-15T18:23:38.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc00cfb30 msgr2=0x7fbcc006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc00cfb30 0x7fbcc006bcf0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbca80097c0 tx=0x7fbca801ec70 comp rx=0 tx=0).stop 2024-01-15T18:23:38.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- 172.21.15.143:0/3328172958 shutdown_connections 2024-01-15T18:23:38.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc00cfb30 0x7fbcc006bcf0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/3328172958 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- 172.21.15.143:0/3328172958 >> 172.21.15.143:0/3328172958 conn(0x7fbcc00d0e20 msgr2=0x7fbcc00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:38.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- 172.21.15.143:0/3328172958 shutdown_connections 2024-01-15T18:23:38.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- 172.21.15.143:0/3328172958 wait complete. 2024-01-15T18:23:38.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 Processor -- start 2024-01-15T18:23:38.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- start start 2024-01-15T18:23:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc006c230 0x7fbcc01440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 0x7fbcc015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc01577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00ccb80 con 0x7fbcc00cfb30 2024-01-15T18:23:38.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015a980 con 0x7fbcc006c230 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.931+0000 7fbcc4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00ccd90 con 0x7fbcc01607c0 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 0x7fbcc015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 0x7fbcc015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37964/0 (socket says 172.21.15.143:37964) 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 -- 172.21.15.143:0/1858101194 learned_addr learned my addr 172.21.15.143:0/1858101194 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbe800640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc01577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbde00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc006c230 0x7fbcc01440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 -- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 msgr2=0x7fbcc01577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc01577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 -- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc006c230 msgr2=0x7fbcc01440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:38.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc006c230 0x7fbcc01440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:38.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 -- 172.21.15.143:0/1858101194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb0009420 con 0x7fbcc00cfb30 2024-01-15T18:23:38.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcbd400640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 0x7fbcc015a120 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fbca801d320 tx=0x7fbca801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80bf070 con 0x7fbcc00cfb30 2024-01-15T18:23:38.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80b1b50 con 0x7fbcc00cfb30 2024-01-15T18:23:38.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80b9410 con 0x7fbcc00cfb30 2024-01-15T18:23:38.928 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca8009420 con 0x7fbcc00cfb30 2024-01-15T18:23:38.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.935+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc00cd350 con 0x7fbcc00cfb30 2024-01-15T18:23:38.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.939+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc006d4b0 con 0x7fbcc00cfb30 2024-01-15T18:23:38.930 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.939+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fbca80b7040 con 0x7fbcc00cfb30 2024-01-15T18:23:38.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.939+0000 7fbcb6000640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc98067b80 0x7fbc9806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:38.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.939+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fbca81207a0 con 0x7fbcc00cfb30 2024-01-15T18:23:38.931 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.939+0000 7fbcbde00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc98067b80 0x7fbc9806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:38.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.939+0000 7fbcbde00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc98067b80 0x7fbc9806a040 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbcc006cfd0 tx=0x7fbcb0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:38.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:38.943+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fbca80ef160 con 0x7fbcc00cfb30 2024-01-15T18:23:39.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.095+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcc00cd540 con 0x7fbcc00cfb30 2024-01-15T18:23:39.088 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.095+0000 7fbcb6000640 1 -- 172.21.15.143:0/1858101194 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16745 (secure 0 0 0) 0x7fbca80b5020 con 0x7fbcc00cfb30 2024-01-15T18:23:39.088 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:39.089 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":113,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:19:41.863194+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc98067b80 msgr2=0x7fbc9806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc98067b80 0x7fbc9806a040 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbcc006cfd0 tx=0x7fbcb0009190 comp rx=0 tx=0).stop 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 msgr2=0x7fbcc015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 0x7fbcc015a120 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fbca801d320 tx=0x7fbca801ba10 comp rx=0 tx=0).stop 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 shutdown_connections 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fbc98067b80 0x7fbc9806a040 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fbcc01607c0 0x7fbcc01577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbcc00cfb30 0x7fbcc015a120 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 --2- 172.21.15.143:0/1858101194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbcc006c230 0x7fbcc01440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 >> 172.21.15.143:0/1858101194 conn(0x7fbcc00d0e20 msgr2=0x7fbcc00cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:39.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 shutdown_connections 2024-01-15T18:23:39.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.099+0000 7fbcc4e00640 1 -- 172.21.15.143:0/1858101194 wait complete. 2024-01-15T18:23:39.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T18:04:46.686433+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:18:39.661662+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:18:40.286820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:23:39.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:23:39.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.259+0000 7f639b400640 1 Processor -- start 2024-01-15T18:23:39.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.259+0000 7f639b400640 1 -- start start 2024-01-15T18:23:39.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f639b400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6394143780 0x7f63940cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f639b400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f63940722c0 0x7f639415c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f639b400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63940ceb50 con 0x7f63940ce520 2024-01-15T18:23:39.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f639b400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639415c110 con 0x7f6394143780 2024-01-15T18:23:39.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f639b400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639415b7e0 con 0x7f63940722c0 2024-01-15T18:23:39.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6398a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f6394143780 0x7f63940cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6393e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f63940ce520 0x7f6394071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60670/0 (socket says 172.21.15.143:60670) 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6393e00640 1 -- 172.21.15.143:0/964812711 learned_addr learned my addr 172.21.15.143:0/964812711 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6399400640 1 --2- 172.21.15.143:0/964812711 >> v2:172.21.15.204:3300/0 conn(0x7f63940722c0 0x7f639415c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6393400640 1 -- 172.21.15.143:0/964812711 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617851162 0 0) 0x7f637c001120 con 0x7f63940ce520 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6398a00640 1 -- 172.21.15.143:0/964812711 >> v2:172.21.15.204:3300/0 conn(0x7f63940722c0 msgr2=0x7f639415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6398a00640 1 --2- 172.21.15.143:0/964812711 >> v2:172.21.15.204:3300/0 conn(0x7f63940722c0 0x7f639415c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6398a00640 1 -- 172.21.15.143:0/964812711 >> v1:172.21.15.143:6789/0 conn(0x7f63940ce520 legacy=0x7f6394071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.263+0000 7f6398a00640 1 -- 172.21.15.143:0/964812711 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639415bac0 con 0x7f6394143780 2024-01-15T18:23:39.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f6398a00640 1 --2- 172.21.15.143:0/964812711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f63940cdfe0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f6394070a20 tx=0x7f638401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a46f7a6e953b0c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f6393400640 1 -- 172.21.15.143:0/964812711 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840ab600 con 0x7f6394143780 2024-01-15T18:23:39.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f6393400640 1 -- 172.21.15.143:0/964812711 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63840a7050 con 0x7f6394143780 2024-01-15T18:23:39.257 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f6393400640 1 -- 172.21.15.143:0/964812711 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63840ab980 con 0x7f6394143780 2024-01-15T18:23:39.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 -- 172.21.15.143:0/964812711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 msgr2=0x7f63940cdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 --2- 172.21.15.143:0/964812711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f63940cdfe0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f6394070a20 tx=0x7f638401c460 comp rx=0 tx=0).stop 2024-01-15T18:23:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 -- 172.21.15.143:0/964812711 shutdown_connections 2024-01-15T18:23:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 --2- 172.21.15.143:0/964812711 >> v2:172.21.15.204:3300/0 conn(0x7f63940722c0 0x7f639415c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 --2- 172.21.15.143:0/964812711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f63940cdfe0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 -- 172.21.15.143:0/964812711 >> 172.21.15.143:0/964812711 conn(0x7f639406c3f0 msgr2=0x7f639406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 -- 172.21.15.143:0/964812711 shutdown_connections 2024-01-15T18:23:39.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 -- 172.21.15.143:0/964812711 wait complete. 2024-01-15T18:23:39.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 Processor -- start 2024-01-15T18:23:39.260 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.267+0000 7f639b400640 1 -- start start 2024-01-15T18:23:39.262 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63940722c0 0x7f63940b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63940ce520 0x7f6394151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f6394152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6394126c10 con 0x7f6394143780 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6394151d90 con 0x7f63940722c0 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63940b2b60 con 0x7f63940ce520 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f6394152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f6394152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37986/0 (socket says 172.21.15.143:37986) 2024-01-15T18:23:39.263 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 -- 172.21.15.143:0/3285970589 learned_addr learned my addr 172.21.15.143:0/3285970589 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6398a00640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63940722c0 0x7f63940b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6393e00640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63940ce520 0x7f6394151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 -- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63940ce520 msgr2=0x7f6394151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63940ce520 0x7f6394151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 -- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63940722c0 msgr2=0x7f63940b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63940722c0 0x7f63940b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 -- 172.21.15.143:0/3285970589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384009420 con 0x7f6394143780 2024-01-15T18:23:39.264 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6399400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f6394152090 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f638c007d90 tx=0x7f638c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c014420 con 0x7f6394143780 2024-01-15T18:23:39.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c0145a0 con 0x7f6394143780 2024-01-15T18:23:39.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c014950 con 0x7f6394143780 2024-01-15T18:23:39.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63940b2e40 con 0x7f6394143780 2024-01-15T18:23:39.265 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639413c0e0 con 0x7f6394143780 2024-01-15T18:23:39.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6394126d80 con 0x7f6394143780 2024-01-15T18:23:39.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.271+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f638c014ab0 con 0x7f6394143780 2024-01-15T18:23:39.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.275+0000 7f6390c00640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f636c067b80 0x7f636c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.266 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.275+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f638c083d20 con 0x7f6394143780 2024-01-15T18:23:39.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.275+0000 7f6398a00640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f636c067b80 0x7f636c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.267 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.275+0000 7f6398a00640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f636c067b80 0x7f636c06a040 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f63840ad000 tx=0x7f63840a6000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.269 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.279+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f638c052630 con 0x7f6394143780 2024-01-15T18:23:39.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6394067500 con 0x7f6394143780 2024-01-15T18:23:39.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f6390c00640 1 -- 172.21.15.143:0/3285970589 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f638c055c50 con 0x7f6394143780 2024-01-15T18:23:39.421 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2024-01-15T18:23:39.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f636c067b80 msgr2=0x7f636c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f636c067b80 0x7f636c06a040 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f63840ad000 tx=0x7f63840a6000 comp rx=0 tx=0).stop 2024-01-15T18:23:39.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 msgr2=0x7f6394152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f6394152090 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f638c007d90 tx=0x7f638c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 shutdown_connections 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f636c067b80 0x7f636c06a040 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6394143780 0x7f6394152090 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f63940ce520 0x7f6394151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 --2- 172.21.15.143:0/3285970589 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f63940722c0 0x7f63940b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 >> 172.21.15.143:0/3285970589 conn(0x7f639406c3f0 msgr2=0x7f639406dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 shutdown_connections 2024-01-15T18:23:39.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.431+0000 7f639b400640 1 -- 172.21.15.143:0/3285970589 wait complete. 2024-01-15T18:23:39.444 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:23:39.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 Processor -- start 2024-01-15T18:23:39.582 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 -- start start 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fa0200cfb30 0x7fa02006bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0200cbf50 0x7fa0200cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0200cffd0 con 0x7fa0201607c0 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200d1ce0 con 0x7fa0200cbf50 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa025600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0200d02b0 con 0x7fa0200cfb30 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa01f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa0200cbf50 0x7fa0200cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa01ea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa0201607c0 0x7fa02006b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60676/0 (socket says 172.21.15.143:60676) 2024-01-15T18:23:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa01ea00640 1 -- 172.21.15.143:0/147682561 learned_addr learned my addr 172.21.15.143:0/147682561 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:39.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.591+0000 7fa01e000640 1 --2- 172.21.15.143:0/147682561 >> v2:172.21.15.204:3300/0 conn(0x7fa0200cfb30 0x7fa02006bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01f400640 1 -- 172.21.15.143:0/147682561 >> v2:172.21.15.204:3300/0 conn(0x7fa0200cfb30 msgr2=0x7fa02006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:39.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01f400640 1 --2- 172.21.15.143:0/147682561 >> v2:172.21.15.204:3300/0 conn(0x7fa0200cfb30 0x7fa02006bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.585 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01f400640 1 -- 172.21.15.143:0/147682561 >> v1:172.21.15.143:6789/0 conn(0x7fa0201607c0 legacy=0x7fa02006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01d600640 1 -- 172.21.15.143:0/147682561 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585584857 0 0) 0x7fa0200cffd0 con 0x7fa0201607c0 2024-01-15T18:23:39.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01f400640 1 -- 172.21.15.143:0/147682561 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200cb470 con 0x7fa0200cbf50 2024-01-15T18:23:39.586 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01e000640 1 --2- 172.21.15.143:0/147682561 >> v2:172.21.15.204:3300/0 conn(0x7fa0200cfb30 0x7fa02006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:39.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01f400640 1 --2- 172.21.15.143:0/147682561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0200cc3c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fa02009a980 tx=0x7fa01801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929bccb79acd7578 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01d600640 1 -- 172.21.15.143:0/147682561 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180b0440 con 0x7fa0200cbf50 2024-01-15T18:23:39.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01d600640 1 -- 172.21.15.143:0/147682561 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0180b05a0 con 0x7fa0200cbf50 2024-01-15T18:23:39.587 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa01d600640 1 -- 172.21.15.143:0/147682561 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180b0930 con 0x7fa0200cbf50 2024-01-15T18:23:39.588 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa025600640 1 -- 172.21.15.143:0/147682561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 msgr2=0x7fa0200cc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.595+0000 7fa025600640 1 --2- 172.21.15.143:0/147682561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0200cc3c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fa02009a980 tx=0x7fa01801f0f0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- 172.21.15.143:0/147682561 shutdown_connections 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 --2- 172.21.15.143:0/147682561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0200cc3c0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 --2- 172.21.15.143:0/147682561 >> v2:172.21.15.204:3300/0 conn(0x7fa0200cfb30 0x7fa02006bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- 172.21.15.143:0/147682561 >> 172.21.15.143:0/147682561 conn(0x7fa0200d0e20 msgr2=0x7fa0200d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- 172.21.15.143:0/147682561 shutdown_connections 2024-01-15T18:23:39.589 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- 172.21.15.143:0/147682561 wait complete. 2024-01-15T18:23:39.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 Processor -- start 2024-01-15T18:23:39.590 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- start start 2024-01-15T18:23:39.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0201f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0200cfb30 0x7fa0200cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0201607c0 0x7fa020144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020159c50 con 0x7fa0200cbf50 2024-01-15T18:23:39.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201f7070 con 0x7fa0200cfb30 2024-01-15T18:23:39.593 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa025600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201f67d0 con 0x7fa0201607c0 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0201f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0201f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38000/0 (socket says 172.21.15.143:38000) 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01ea00640 1 -- 172.21.15.143:0/2810983174 learned_addr learned my addr 172.21.15.143:0/2810983174 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01e000640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0200cfb30 0x7fa0200cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01f400640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0201607c0 0x7fa020144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01ea00640 1 -- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0201607c0 msgr2=0x7fa020144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.599+0000 7fa01ea00640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0201607c0 0x7fa020144680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa01ea00640 1 -- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0200cfb30 msgr2=0x7fa0200cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.594 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa01ea00640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0200cfb30 0x7fa0200cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa01ea00640 1 -- 172.21.15.143:0/2810983174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018009a70 con 0x7fa0200cbf50 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa01ea00640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0201f6110 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa01000c480 tx=0x7fa01000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100074e0 con 0x7fa0200cbf50 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010007660 con 0x7fa0200cbf50 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201f6ab0 con 0x7fa0200cbf50 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010004a30 con 0x7fa0200cbf50 2024-01-15T18:23:39.595 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa020159e60 con 0x7fa0200cbf50 2024-01-15T18:23:39.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02006d4b0 con 0x7fa0200cbf50 2024-01-15T18:23:39.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.603+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fa010004b90 con 0x7fa0200cbf50 2024-01-15T18:23:39.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.607+0000 7fa016a00640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ff8067b80 0x7f9ff806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.607+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fa010082550 con 0x7fa0200cbf50 2024-01-15T18:23:39.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.607+0000 7fa01e000640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ff8067b80 0x7f9ff806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.597 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.607+0000 7fa01e000640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ff8067b80 0x7f9ff806a040 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa02006c530 tx=0x7fa008008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.611+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fa010054950 con 0x7fa0200cbf50 2024-01-15T18:23:39.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.759+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa02015a080 con 0x7fa0200cbf50 2024-01-15T18:23:39.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.759+0000 7fa016a00640 1 -- 172.21.15.143:0/2810983174 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0100180a0 con 0x7fa0200cbf50 2024-01-15T18:23:39.751 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 32 2024-01-15T18:23:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ff8067b80 msgr2=0x7f9ff806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ff8067b80 0x7f9ff806a040 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa02006c530 tx=0x7fa008008040 comp rx=0 tx=0).stop 2024-01-15T18:23:39.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 msgr2=0x7fa0201f6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0201f6110 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fa01000c480 tx=0x7fa01000c940 comp rx=0 tx=0).stop 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 shutdown_connections 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9ff8067b80 0x7f9ff806a040 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fa0201607c0 0x7fa020144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa0200cfb30 0x7fa0200cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 --2- 172.21.15.143:0/2810983174 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa0200cbf50 0x7fa0201f6110 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 >> 172.21.15.143:0/2810983174 conn(0x7fa0200d0e20 msgr2=0x7fa02006f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 shutdown_connections 2024-01-15T18:23:39.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.763+0000 7fa025600640 1 -- 172.21.15.143:0/2810983174 wait complete. 2024-01-15T18:23:39.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:23:39.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.919+0000 7fd41ec00640 1 Processor -- start 2024-01-15T18:23:39.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.919+0000 7fd41ec00640 1 -- start start 2024-01-15T18:23:39.913 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41ec00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4180cfb30 0x7fd41806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41ec00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd4180cbf50 0x7fd4180cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41ec00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4180cffd0 con 0x7fd4181607c0 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41ec00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180d1ce0 con 0x7fd4180cfb30 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41ec00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180d02b0 con 0x7fd4180cbf50 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4180cfb30 0x7fd41806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417e00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd4181607c0 0x7fd41806b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60684/0 (socket says 172.21.15.143:60684) 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417e00640 1 -- 172.21.15.143:0/3227134489 learned_addr learned my addr 172.21.15.143:0/3227134489 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41cc00640 1 --2- 172.21.15.143:0/3227134489 >> v2:172.21.15.204:3300/0 conn(0x7fd4180cbf50 0x7fd4180cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.914 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417400640 1 -- 172.21.15.143:0/3227134489 >> v2:172.21.15.204:3300/0 conn(0x7fd4180cbf50 msgr2=0x7fd4180cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:39.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417400640 1 --2- 172.21.15.143:0/3227134489 >> v2:172.21.15.204:3300/0 conn(0x7fd4180cbf50 0x7fd4180cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417400640 1 -- 172.21.15.143:0/3227134489 >> v1:172.21.15.143:6789/0 conn(0x7fd4181607c0 legacy=0x7fd41806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd417400640 1 -- 172.21.15.143:0/3227134489 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180cb470 con 0x7fd4180cfb30 2024-01-15T18:23:39.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd41cc00640 1 --2- 172.21.15.143:0/3227134489 >> v2:172.21.15.204:3300/0 conn(0x7fd4180cbf50 0x7fd4180cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:39.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.923+0000 7fd416a00640 1 -- 172.21.15.143:0/3227134489 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297370508 0 0) 0x7fd4180cffd0 con 0x7fd4181607c0 2024-01-15T18:23:39.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd417400640 1 --2- 172.21.15.143:0/3227134489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4180cfb30 0x7fd41806bec0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd4180cef70 tx=0x7fd40001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcf542d760366f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd416a00640 1 -- 172.21.15.143:0/3227134489 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4000a8950 con 0x7fd4180cfb30 2024-01-15T18:23:39.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd416a00640 1 -- 172.21.15.143:0/3227134489 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4000a8ab0 con 0x7fd4180cfb30 2024-01-15T18:23:39.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd416a00640 1 -- 172.21.15.143:0/3227134489 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4000a8e40 con 0x7fd4180cfb30 2024-01-15T18:23:39.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 -- 172.21.15.143:0/3227134489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4180cfb30 msgr2=0x7fd41806bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 --2- 172.21.15.143:0/3227134489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4180cfb30 0x7fd41806bec0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fd4180cef70 tx=0x7fd40001ec50 comp rx=0 tx=0).stop 2024-01-15T18:23:39.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 -- 172.21.15.143:0/3227134489 shutdown_connections 2024-01-15T18:23:39.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 --2- 172.21.15.143:0/3227134489 >> v2:172.21.15.204:3300/0 conn(0x7fd4180cbf50 0x7fd4180cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 --2- 172.21.15.143:0/3227134489 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4180cfb30 0x7fd41806bec0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 -- 172.21.15.143:0/3227134489 >> 172.21.15.143:0/3227134489 conn(0x7fd4180d0e20 msgr2=0x7fd4180d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:39.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 -- 172.21.15.143:0/3227134489 shutdown_connections 2024-01-15T18:23:39.919 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 -- 172.21.15.143:0/3227134489 wait complete. 2024-01-15T18:23:39.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 Processor -- start 2024-01-15T18:23:39.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.927+0000 7fd41ec00640 1 -- start start 2024-01-15T18:23:39.922 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4180cbf50 0x7fd41806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4180cfb30 0x7fd4181f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 0x7fd4181f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181f4430 con 0x7fd4181607c0 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180b6250 con 0x7fd4180cbf50 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4181f3e00 con 0x7fd4180cfb30 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 0x7fd4181f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 0x7fd4181f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38026/0 (socket says 172.21.15.143:38026) 2024-01-15T18:23:39.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 -- 172.21.15.143:0/82273669 learned_addr learned my addr 172.21.15.143:0/82273669 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd417e00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4180cbf50 0x7fd41806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd417400640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4180cfb30 0x7fd4181f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 -- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4180cfb30 msgr2=0x7fd4181f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4180cfb30 0x7fd4181f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 -- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4180cbf50 msgr2=0x7fd41806f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4180cbf50 0x7fd41806f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 -- 172.21.15.143:0/82273669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400009420 con 0x7fd4181607c0 2024-01-15T18:23:39.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd417e00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4180cbf50 0x7fd41806f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd417400640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4180cfb30 0x7fd4181f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41cc00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 0x7fd4181f3180 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fd410007ef0 tx=0x7fd41000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410003ba0 con 0x7fd4181607c0 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000f040 con 0x7fd4181607c0 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000de80 con 0x7fd4181607c0 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4181f3800 con 0x7fd4181607c0 2024-01-15T18:23:39.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4181f40c0 con 0x7fd4181607c0 2024-01-15T18:23:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.931+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41806d4b0 con 0x7fd4181607c0 2024-01-15T18:23:39.926 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.935+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fd410004030 con 0x7fd4181607c0 2024-01-15T18:23:39.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.935+0000 7fd40fe00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3f0067b80 0x7fd3f006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:39.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.935+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd410083670 con 0x7fd4181607c0 2024-01-15T18:23:39.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.935+0000 7fd417e00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3f0067b80 0x7fd3f006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:39.927 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.935+0000 7fd417e00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3f0067b80 0x7fd3f006a040 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd408000f70 tx=0x7fd408007cb0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:39.929 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:39.939+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd410052030 con 0x7fd4181607c0 2024-01-15T18:23:40.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd41806cb80 con 0x7fd4181607c0 2024-01-15T18:23:40.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd40fe00640 1 -- 172.21.15.143:0/82273669 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd41001a020 con 0x7fd4181607c0 2024-01-15T18:23:40.082 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 64 2024-01-15T18:23:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3f0067b80 msgr2=0x7fd3f006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3f0067b80 0x7fd3f006a040 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd408000f70 tx=0x7fd408007cb0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 msgr2=0x7fd4181f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 0x7fd4181f3180 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fd410007ef0 tx=0x7fd41000d4a0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 shutdown_connections 2024-01-15T18:23:40.084 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd3f0067b80 0x7fd3f006a040 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4181607c0 0x7fd4181f3180 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd4180cfb30 0x7fd4181f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 --2- 172.21.15.143:0/82273669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4180cbf50 0x7fd41806f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 >> 172.21.15.143:0/82273669 conn(0x7fd4180d0e20 msgr2=0x7fd41806b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 shutdown_connections 2024-01-15T18:23:40.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.091+0000 7fd41ec00640 1 -- 172.21.15.143:0/82273669 wait complete. 2024-01-15T18:23:40.103 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:23:40.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 Processor -- start 2024-01-15T18:23:40.241 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 -- start start 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f2ff40d1950 0x7f2ff40cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ff40cb920 0x7f2ff40cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff40cef50 con 0x7f2ff40cb920 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff40ce7c0 con 0x7f2ff40d1950 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ffa400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40d1d90 con 0x7f2ff40cf410 2024-01-15T18:23:40.243 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ff40cb920 0x7f2ff40cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2ff40cb920 0x7f2ff40cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38032/0 (socket says 172.21.15.143:38032) 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 -- 172.21.15.143:0/516986032 learned_addr learned my addr 172.21.15.143:0/516986032 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff2a00640 1 --2- 172.21.15.143:0/516986032 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40d1950 0x7f2ff40cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3400640 1 --2- 172.21.15.143:0/516986032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 -- 172.21.15.143:0/516986032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 msgr2=0x7f2ff40cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 --2- 172.21.15.143:0/516986032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 -- 172.21.15.143:0/516986032 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40d1950 msgr2=0x7f2ff40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:40.244 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 --2- 172.21.15.143:0/516986032 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40d1950 0x7f2ff40cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff3e00640 1 -- 172.21.15.143:0/516986032 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff40ce4e0 con 0x7f2ff40cb920 2024-01-15T18:23:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.251+0000 7f2ff2a00640 1 --2- 172.21.15.143:0/516986032 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40d1950 0x7f2ff40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:40.245 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ff3e00640 1 --2- 172.21.15.143:0/516986032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff40cbdb0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4071a90 tx=0x7f2fec01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da7551372e95b45 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:40.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ff2000640 1 -- 172.21.15.143:0/516986032 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0a5740 con 0x7f2ff40cb920 2024-01-15T18:23:40.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ff2000640 1 -- 172.21.15.143:0/516986032 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec0a58a0 con 0x7f2ff40cb920 2024-01-15T18:23:40.246 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ff2000640 1 -- 172.21.15.143:0/516986032 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0a5c30 con 0x7f2ff40cb920 2024-01-15T18:23:40.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 -- 172.21.15.143:0/516986032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 msgr2=0x7f2ff40cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 --2- 172.21.15.143:0/516986032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff40cbdb0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4071a90 tx=0x7f2fec01c9d0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.247 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 -- 172.21.15.143:0/516986032 shutdown_connections 2024-01-15T18:23:40.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 --2- 172.21.15.143:0/516986032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff40cbdb0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 --2- 172.21.15.143:0/516986032 >> v2:172.21.15.204:3300/0 conn(0x7f2ff40d1950 0x7f2ff40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 --2- 172.21.15.143:0/516986032 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 -- 172.21.15.143:0/516986032 >> 172.21.15.143:0/516986032 conn(0x7f2ff40d0e30 msgr2=0x7f2ff40d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:40.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 -- 172.21.15.143:0/516986032 shutdown_connections 2024-01-15T18:23:40.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.255+0000 7f2ffa400640 1 -- 172.21.15.143:0/516986032 wait complete. 2024-01-15T18:23:40.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 Processor -- start 2024-01-15T18:23:40.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 -- start start 2024-01-15T18:23:40.250 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff40d1950 0x7f2ff409d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40c9b60 con 0x7f2ff40cb920 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff415dc30 con 0x7f2ff40d1950 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40c8f50 con 0x7f2ff40cf410 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff415d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38034/0 (socket says 172.21.15.143:38034) 2024-01-15T18:23:40.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 -- 172.21.15.143:0/2126984855 learned_addr learned my addr 172.21.15.143:0/2126984855 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3e00640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff40d1950 0x7f2ff409d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff2a00640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 -- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 msgr2=0x7f2ff40b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 -- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff40d1950 msgr2=0x7f2ff409d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff40d1950 0x7f2ff409d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 -- 172.21.15.143:0/2126984855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec009b30 con 0x7f2ff40cb920 2024-01-15T18:23:40.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ff3400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff415d6f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d800 tx=0x7f2fe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:40.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40096a0 con 0x7f2ff40cb920 2024-01-15T18:23:40.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe4009800 con 0x7f2ff40cb920 2024-01-15T18:23:40.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff40c9230 con 0x7f2ff40cb920 2024-01-15T18:23:40.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4009b70 con 0x7f2ff40cb920 2024-01-15T18:23:40.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.259+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff40c97f0 con 0x7f2ff40cb920 2024-01-15T18:23:40.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.263+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff4067500 con 0x7f2ff40cb920 2024-01-15T18:23:40.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.263+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f2fe4009cd0 con 0x7f2ff40cb920 2024-01-15T18:23:40.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.263+0000 7f2feb400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2fcc067b80 0x7f2fcc06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:40.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.263+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2fe40836f0 con 0x7f2ff40cb920 2024-01-15T18:23:40.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.263+0000 7f2ff2a00640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2fcc067b80 0x7f2fcc06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:40.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.263+0000 7f2ff2a00640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2fcc067b80 0x7f2fcc06a040 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2ff409ce00 tx=0x7f2fdc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:40.258 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.267+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2fe40523b0 con 0x7f2ff40cb920 2024-01-15T18:23:40.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.427+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ff4165600 con 0x7f2ff40cb920 2024-01-15T18:23:40.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.427+0000 7f2feb400640 1 -- 172.21.15.143:0/2126984855 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2fe40559d0 con 0x7f2ff40cb920 2024-01-15T18:23:40.419 INFO:teuthology.orchestra.run.smithi143.stderr:listed 0 entries 2024-01-15T18:23:40.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2fcc067b80 msgr2=0x7f2fcc06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2fcc067b80 0x7f2fcc06a040 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2ff409ce00 tx=0x7f2fdc008040 comp rx=0 tx=0).stop 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 msgr2=0x7f2ff415d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff415d6f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400d800 tx=0x7f2fe400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 shutdown_connections 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2fcc067b80 0x7f2fcc06a040 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2ff40d1950 0x7f2ff409d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f2ff40cf410 0x7f2ff40b5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 --2- 172.21.15.143:0/2126984855 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2ff40cb920 0x7f2ff415d6f0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:40.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 >> 172.21.15.143:0/2126984855 conn(0x7f2ff40d0e30 msgr2=0x7f2ff406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:40.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 shutdown_connections 2024-01-15T18:23:40.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:40.431+0000 7f2ffa400640 1 -- 172.21.15.143:0/2126984855 wait complete. 2024-01-15T18:23:40.440 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi204.front.sepia.ceph.com... 2024-01-15T18:23:40.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.441 DEBUG:teuthology.orchestra.run.smithi204:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:23:40.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi204.stderr:ceph-fuse[34481]: fuse finished with error 0 and tester_r 0 2024-01-15T18:23:40.464 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:23:40.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:23:40.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.498 DEBUG:teuthology.orchestra.run.smithi204:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:23:40.555 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-01-15T18:23:40.555 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:23:40.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354572k,nr_inodes=4088643,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=595 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:23:40.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.606 DEBUG:teuthology.orchestra.run.smithi204:> ip netns list 2024-01-15T18:23:40.659 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:23:40.659 INFO:teuthology.orchestra.run.smithi204.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:23:40.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.659 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:23:40.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.724 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:23:40.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.798 DEBUG:teuthology.orchestra.run.smithi204:> sudo ip link delete ceph-brx 2024-01-15T18:23:40.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.962 DEBUG:teuthology.orchestra.run.smithi143:> ip netns list 2024-01-15T18:23:40.971 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:23:40.971 INFO:teuthology.orchestra.run.smithi143.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:23:40.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:40.972 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:23:41.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:41.037 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:23:41.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:23:41.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo ip link delete ceph-brx 2024-01-15T18:23:41.287 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-15T18:23:41.295 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-15T18:23:41.295 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:23:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 Processor -- start 2024-01-15T18:23:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 -- start start 2024-01-15T18:23:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9dc4158bc0 0x7f9dc4071140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9dc406f3f0 0x7f9dc406a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f880 0x7f9dc406a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc4158fa0 con 0x7f9dc406f3f0 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc415b970 con 0x7f9dc4158bc0 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dca200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc415bc50 con 0x7f9dc406f880 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9dc406f3f0 0x7f9dc406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9dc406f3f0 0x7f9dc406a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38036/0 (socket says 172.21.15.143:38036) 2024-01-15T18:23:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 -- 172.21.15.143:0/3926147276 learned_addr learned my addr 172.21.15.143:0/3926147276 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc3e00640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f880 0x7f9dc406a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc3400640 1 --2- 172.21.15.143:0/3926147276 >> v2:172.21.15.204:3300/0 conn(0x7f9dc4158bc0 0x7f9dc4071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 -- 172.21.15.143:0/3926147276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f880 msgr2=0x7f9dc406a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f880 0x7f9dc406a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 -- 172.21.15.143:0/3926147276 >> v2:172.21.15.204:3300/0 conn(0x7f9dc4158bc0 msgr2=0x7f9dc4071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 --2- 172.21.15.143:0/3926147276 >> v2:172.21.15.204:3300/0 conn(0x7f9dc4158bc0 0x7f9dc4071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc2a00640 1 -- 172.21.15.143:0/3926147276 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc415bf30 con 0x7f9dc406f3f0 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc3e00640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f880 0x7f9dc406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.443+0000 7f9dc3400640 1 --2- 172.21.15.143:0/3926147276 >> v2:172.21.15.204:3300/0 conn(0x7f9dc4158bc0 0x7f9dc4071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:41.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dc2a00640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc406f3f0 0x7f9dc406a490 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4070270 tx=0x7f9dac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f257b5b6ef7459d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:41.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dc2000640 1 -- 172.21.15.143:0/3926147276 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0ab600 con 0x7f9dc406f3f0 2024-01-15T18:23:41.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dc2000640 1 -- 172.21.15.143:0/3926147276 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dac0a7050 con 0x7f9dc406f3f0 2024-01-15T18:23:41.439 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dc2000640 1 -- 172.21.15.143:0/3926147276 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0ab980 con 0x7f9dc406f3f0 2024-01-15T18:23:41.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 -- 172.21.15.143:0/3926147276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc406f3f0 msgr2=0x7f9dc406a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc406f3f0 0x7f9dc406a490 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4070270 tx=0x7f9dac01c470 comp rx=0 tx=0).stop 2024-01-15T18:23:41.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 -- 172.21.15.143:0/3926147276 shutdown_connections 2024-01-15T18:23:41.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f880 0x7f9dc406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 --2- 172.21.15.143:0/3926147276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc406f3f0 0x7f9dc406a490 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 --2- 172.21.15.143:0/3926147276 >> v2:172.21.15.204:3300/0 conn(0x7f9dc4158bc0 0x7f9dc4071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 -- 172.21.15.143:0/3926147276 >> 172.21.15.143:0/3926147276 conn(0x7f9dc40cfd20 msgr2=0x7f9dc40d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:41.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 -- 172.21.15.143:0/3926147276 shutdown_connections 2024-01-15T18:23:41.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.447+0000 7f9dca200640 1 -- 172.21.15.143:0/3926147276 wait complete. 2024-01-15T18:23:41.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 Processor -- start 2024-01-15T18:23:41.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 -- start start 2024-01-15T18:23:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f3f0 0x7f9dc41f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dc406f880 0x7f9dc40b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 0x7f9dc40b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40b2900 con 0x7f9dc4158bc0 2024-01-15T18:23:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40cda70 con 0x7f9dc406f880 2024-01-15T18:23:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41f5800 con 0x7f9dc406f3f0 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 0x7f9dc40b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 0x7f9dc40b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38046/0 (socket says 172.21.15.143:38046) 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f3f0 0x7f9dc41f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc2a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dc406f880 0x7f9dc40b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 -- 172.21.15.143:0/810561040 learned_addr learned my addr 172.21.15.143:0/810561040 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 -- 172.21.15.143:0/810561040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f3f0 msgr2=0x7f9dc41f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f3f0 0x7f9dc41f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.445 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 -- 172.21.15.143:0/810561040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dc406f880 msgr2=0x7f9dc40b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dc406f880 0x7f9dc40b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 -- 172.21.15.143:0/810561040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac0094d0 con 0x7f9dc4158bc0 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dc3e00640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 0x7f9dc40b0bb0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc007e10 tx=0x7f9dbc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc003ba0 con 0x7f9dc4158bc0 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc00f040 con 0x7f9dc4158bc0 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc40cdd30 con 0x7f9dc4158bc0 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc00de80 con 0x7f9dc4158bc0 2024-01-15T18:23:41.446 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.451+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc40ce2f0 con 0x7f9dc4158bc0 2024-01-15T18:23:41.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.455+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc4067500 con 0x7f9dc4158bc0 2024-01-15T18:23:41.447 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.455+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f9dbc004030 con 0x7f9dc4158bc0 2024-01-15T18:23:41.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.455+0000 7f9dbb400640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d9c067b80 0x7f9d9c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.455+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f9dbc083c10 con 0x7f9dc4158bc0 2024-01-15T18:23:41.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.455+0000 7f9dc3400640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d9c067b80 0x7f9d9c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.448 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.459+0000 7f9dc3400640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d9c067b80 0x7f9d9c06a040 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9db4006e90 tx=0x7f9db4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:41.450 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.459+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f9dbc0525d0 con 0x7f9dc4158bc0 2024-01-15T18:23:41.596 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.603+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dc40d0a60 con 0x7f9d9c067b80 2024-01-15T18:23:41.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.615+0000 7f9dbb400640 1 -- 172.21.15.143:0/810561040 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291090 (secure 0 0 0) 0x7f9dc40d0a60 con 0x7f9d9c067b80 2024-01-15T18:23:41.608 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:41.610 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:23:41.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d9c067b80 msgr2=0x7f9d9c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d9c067b80 0x7f9d9c06a040 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9db4006e90 tx=0x7f9db4008040 comp rx=0 tx=0).stop 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 msgr2=0x7f9dc40b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 0x7f9dc40b0bb0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc007e10 tx=0x7f9dbc00d4a0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 shutdown_connections 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9d9c067b80 0x7f9d9c06a040 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9dc4158bc0 0x7f9dc40b0bb0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9dc406f880 0x7f9dc40b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 --2- 172.21.15.143:0/810561040 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9dc406f3f0 0x7f9dc41f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 >> 172.21.15.143:0/810561040 conn(0x7f9dc40cfd20 msgr2=0x7f9dc4071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 shutdown_connections 2024-01-15T18:23:41.613 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.619+0000 7f9dca200640 1 -- 172.21.15.143:0/810561040 wait complete. 2024-01-15T18:23:41.629 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":673,"stamp":"2024-01-15T18:23:40.974303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4296130174,"num_objects":1055,"num_object_clones":1,"num_object_copies":2110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":1,"num_read":394,"num_read_kb":1048698,"num_write":1465,"num_write_kb":4197407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2909,"ondisk_log_size":2909,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8927144,"kb_used_data":8401960,"kb_used_omap":12,"kb_used_meta":525107,"kb_avail":746047576,"statfs":{"total":773094113280,"available":763952717824,"internally_reserved":0,"allocated":8603607040,"data_stored":8597151872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":537710128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":68,"apply_latency_ms":68,"commit_latency_ns":68000000,"apply_latency_ns":68000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002584"},"pg_stats":[{"pgid":"17.3f","version":"113'49","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480416+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:23:36.480416+0000","last_peered":"2024-01-15T18:23:36.480416+0000","last_clean":"2024-01-15T18:23:36.480416+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:23:36.480416+0000","last_undegraded":"2024-01-15T18:23:36.480416+0000","last_fullsized":"2024-01-15T18:23:36.480416+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:00.418536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'42","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480084+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:23:36.480084+0000","last_peered":"2024-01-15T18:23:36.480084+0000","last_clean":"2024-01-15T18:23:36.480084+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:23:36.480084+0000","last_undegraded":"2024-01-15T18:23:36.480084+0000","last_fullsized":"2024-01-15T18:23:36.480084+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'39","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481097+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:23:36.481097+0000","last_peered":"2024-01-15T18:23:36.481097+0000","last_clean":"2024-01-15T18:23:36.481097+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:23:36.481097+0000","last_undegraded":"2024-01-15T18:23:36.481097+0000","last_fullsized":"2024-01-15T18:23:36.481097+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:07.807378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":16,"num_write_kb":57345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.490145+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:23:36.490145+0000","last_peered":"2024-01-15T18:23:36.490145+0000","last_clean":"2024-01-15T18:23:36.490145+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:36.490145+0000","last_undegraded":"2024-01-15T18:23:36.490145+0000","last_fullsized":"2024-01-15T18:23:36.490145+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:44.886136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'25","reported_seq":42,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.450344+0000","last_change":"2024-01-15T18:18:41.951758+0000","last_active":"2024-01-15T18:23:36.450344+0000","last_peered":"2024-01-15T18:23:36.450344+0000","last_clean":"2024-01-15T18:23:36.450344+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:23:36.450344+0000","last_undegraded":"2024-01-15T18:23:36.450344+0000","last_fullsized":"2024-01-15T18:23:36.450344+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:51.159719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'45","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463710+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:23:36.463710+0000","last_peered":"2024-01-15T18:23:36.463710+0000","last_clean":"2024-01-15T18:23:36.463710+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:23:36.463710+0000","last_undegraded":"2024-01-15T18:23:36.463710+0000","last_fullsized":"2024-01-15T18:23:36.463710+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:48.760709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'41","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470502+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:23:36.470502+0000","last_peered":"2024-01-15T18:23:36.470502+0000","last_clean":"2024-01-15T18:23:36.470502+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:23:36.470502+0000","last_undegraded":"2024-01-15T18:23:36.470502+0000","last_fullsized":"2024-01-15T18:23:36.470502+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'42","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466790+0000","last_change":"2024-01-15T18:18:41.932178+0000","last_active":"2024-01-15T18:23:36.466790+0000","last_peered":"2024-01-15T18:23:36.466790+0000","last_clean":"2024-01-15T18:23:36.466790+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:23:36.466790+0000","last_undegraded":"2024-01-15T18:23:36.466790+0000","last_fullsized":"2024-01-15T18:23:36.466790+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:32.753279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'42","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475770+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:23:36.475770+0000","last_peered":"2024-01-15T18:23:36.475770+0000","last_clean":"2024-01-15T18:23:36.475770+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:23:36.475770+0000","last_undegraded":"2024-01-15T18:23:36.475770+0000","last_fullsized":"2024-01-15T18:23:36.475770+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:16.576477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'41","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.478544+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:23:36.478544+0000","last_peered":"2024-01-15T18:23:36.478544+0000","last_clean":"2024-01-15T18:23:36.478544+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:23:36.478544+0000","last_undegraded":"2024-01-15T18:23:36.478544+0000","last_fullsized":"2024-01-15T18:23:36.478544+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:16.172780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":11,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'41","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.474632+0000","last_change":"2024-01-15T18:18:41.934119+0000","last_active":"2024-01-15T18:23:36.474632+0000","last_peered":"2024-01-15T18:23:36.474632+0000","last_clean":"2024-01-15T18:23:36.474632+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:23:36.474632+0000","last_undegraded":"2024-01-15T18:23:36.474632+0000","last_fullsized":"2024-01-15T18:23:36.474632+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'43","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477840+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:23:36.477840+0000","last_peered":"2024-01-15T18:23:36.477840+0000","last_clean":"2024-01-15T18:23:36.477840+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:23:36.477840+0000","last_undegraded":"2024-01-15T18:23:36.477840+0000","last_fullsized":"2024-01-15T18:23:36.477840+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:16.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":20481,"num_write":25,"num_write_kb":94209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'34","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.453165+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:23:36.453165+0000","last_peered":"2024-01-15T18:23:36.453165+0000","last_clean":"2024-01-15T18:23:36.453165+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:23:36.453165+0000","last_undegraded":"2024-01-15T18:23:36.453165+0000","last_fullsized":"2024-01-15T18:23:36.453165+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'43","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.459261+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:23:36.459261+0000","last_peered":"2024-01-15T18:23:36.459261+0000","last_clean":"2024-01-15T18:23:36.459261+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:23:36.459261+0000","last_undegraded":"2024-01-15T18:23:36.459261+0000","last_fullsized":"2024-01-15T18:23:36.459261+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:06.614151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'29","reported_seq":47,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.469423+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:23:36.469423+0000","last_peered":"2024-01-15T18:23:36.469423+0000","last_clean":"2024-01-15T18:23:36.469423+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:23:36.469423+0000","last_undegraded":"2024-01-15T18:23:36.469423+0000","last_fullsized":"2024-01-15T18:23:36.469423+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:34.739612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'40","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481202+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:23:36.481202+0000","last_peered":"2024-01-15T18:23:36.481202+0000","last_clean":"2024-01-15T18:23:36.481202+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:23:36.481202+0000","last_undegraded":"2024-01-15T18:23:36.481202+0000","last_fullsized":"2024-01-15T18:23:36.481202+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:47:03.406736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.473510+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:23:36.473510+0000","last_peered":"2024-01-15T18:23:36.473510+0000","last_clean":"2024-01-15T18:23:36.473510+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:23:36.473510+0000","last_undegraded":"2024-01-15T18:23:36.473510+0000","last_fullsized":"2024-01-15T18:23:36.473510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470128+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:23:36.470128+0000","last_peered":"2024-01-15T18:23:36.470128+0000","last_clean":"2024-01-15T18:23:36.470128+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:23:36.470128+0000","last_undegraded":"2024-01-15T18:23:36.470128+0000","last_fullsized":"2024-01-15T18:23:36.470128+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:45.110826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455631+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:23:36.455631+0000","last_peered":"2024-01-15T18:23:36.455631+0000","last_clean":"2024-01-15T18:23:36.455631+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:23:36.455631+0000","last_undegraded":"2024-01-15T18:23:36.455631+0000","last_fullsized":"2024-01-15T18:23:36.455631+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'46","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481283+0000","last_change":"2024-01-15T18:18:41.927062+0000","last_active":"2024-01-15T18:23:36.481283+0000","last_peered":"2024-01-15T18:23:36.481283+0000","last_clean":"2024-01-15T18:23:36.481283+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:36.481283+0000","last_undegraded":"2024-01-15T18:23:36.481283+0000","last_fullsized":"2024-01-15T18:23:36.481283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'50","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477877+0000","last_change":"2024-01-15T18:18:41.937906+0000","last_active":"2024-01-15T18:23:36.477877+0000","last_peered":"2024-01-15T18:23:36.477877+0000","last_clean":"2024-01-15T18:23:36.477877+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:23:36.477877+0000","last_undegraded":"2024-01-15T18:23:36.477877+0000","last_fullsized":"2024-01-15T18:23:36.477877+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:20.422368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":11,"num_read_kb":36866,"num_write":22,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'41","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.464458+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:23:36.464458+0000","last_peered":"2024-01-15T18:23:36.464458+0000","last_clean":"2024-01-15T18:23:36.464458+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:23:36.464458+0000","last_undegraded":"2024-01-15T18:23:36.464458+0000","last_fullsized":"2024-01-15T18:23:36.464458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":17,"num_write_kb":61441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'35","reported_seq":52,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479224+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:23:36.479224+0000","last_peered":"2024-01-15T18:23:36.479224+0000","last_clean":"2024-01-15T18:23:36.479224+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:23:36.479224+0000","last_undegraded":"2024-01-15T18:23:36.479224+0000","last_fullsized":"2024-01-15T18:23:36.479224+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":11,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'39","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477262+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:23:36.477262+0000","last_peered":"2024-01-15T18:23:36.477262+0000","last_clean":"2024-01-15T18:23:36.477262+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:23:36.477262+0000","last_undegraded":"2024-01-15T18:23:36.477262+0000","last_fullsized":"2024-01-15T18:23:36.477262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'35","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477951+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:23:36.477951+0000","last_peered":"2024-01-15T18:23:36.477951+0000","last_clean":"2024-01-15T18:23:36.477951+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:23:36.477951+0000","last_undegraded":"2024-01-15T18:23:36.477951+0000","last_fullsized":"2024-01-15T18:23:36.477951+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:12.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":16386,"num_write":17,"num_write_kb":61441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'50","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.482075+0000","last_change":"2024-01-15T18:19:42.172768+0000","last_active":"2024-01-15T18:23:36.482075+0000","last_peered":"2024-01-15T18:23:36.482075+0000","last_clean":"2024-01-15T18:23:36.482075+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:23:36.482075+0000","last_undegraded":"2024-01-15T18:23:36.482075+0000","last_fullsized":"2024-01-15T18:23:36.482075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:36.268040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'37","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455733+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:23:36.455733+0000","last_peered":"2024-01-15T18:23:36.455733+0000","last_clean":"2024-01-15T18:23:36.455733+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:23:36.455733+0000","last_undegraded":"2024-01-15T18:23:36.455733+0000","last_fullsized":"2024-01-15T18:23:36.455733+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:18.174579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":12289,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'31","reported_seq":48,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.458498+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:23:36.458498+0000","last_peered":"2024-01-15T18:23:36.458498+0000","last_clean":"2024-01-15T18:23:36.458498+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:23:36.458498+0000","last_undegraded":"2024-01-15T18:23:36.458498+0000","last_fullsized":"2024-01-15T18:23:36.458498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'38","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480157+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:23:36.480157+0000","last_peered":"2024-01-15T18:23:36.480157+0000","last_clean":"2024-01-15T18:23:36.480157+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:23:36.480157+0000","last_undegraded":"2024-01-15T18:23:36.480157+0000","last_fullsized":"2024-01-15T18:23:36.480157+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'38","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.458139+0000","last_change":"2024-01-15T18:18:41.937559+0000","last_active":"2024-01-15T18:23:36.458139+0000","last_peered":"2024-01-15T18:23:36.458139+0000","last_clean":"2024-01-15T18:23:36.458139+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:23:36.458139+0000","last_undegraded":"2024-01-15T18:23:36.458139+0000","last_fullsized":"2024-01-15T18:23:36.458139+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5,"num_read_kb":16385,"num_write":15,"num_write_kb":53249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'59","reported_seq":81,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.482554+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:23:36.482554+0000","last_peered":"2024-01-15T18:23:36.482554+0000","last_clean":"2024-01-15T18:23:36.482554+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:23:36.482554+0000","last_undegraded":"2024-01-15T18:23:36.482554+0000","last_fullsized":"2024-01-15T18:23:36.482554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:18:31.192027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'50","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466535+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:23:36.466535+0000","last_peered":"2024-01-15T18:23:36.466535+0000","last_clean":"2024-01-15T18:23:36.466535+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:23:36.466535+0000","last_undegraded":"2024-01-15T18:23:36.466535+0000","last_fullsized":"2024-01-15T18:23:36.466535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:56:19.546961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'28","reported_seq":47,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466086+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:23:36.466086+0000","last_peered":"2024-01-15T18:23:36.466086+0000","last_clean":"2024-01-15T18:23:36.466086+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:23:36.466086+0000","last_undegraded":"2024-01-15T18:23:36.466086+0000","last_fullsized":"2024-01-15T18:23:36.466086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:47.919946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":9,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655110+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:20:46.655110+0000","last_peered":"2024-01-15T18:20:46.655110+0000","last_clean":"2024-01-15T18:20:46.655110+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:20:46.655110+0000","last_undegraded":"2024-01-15T18:20:46.655110+0000","last_fullsized":"2024-01-15T18:20:46.655110+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:01:16.930740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'44","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.464971+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:23:36.464971+0000","last_peered":"2024-01-15T18:23:36.464971+0000","last_clean":"2024-01-15T18:23:36.464971+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:23:36.464971+0000","last_undegraded":"2024-01-15T18:23:36.464971+0000","last_fullsized":"2024-01-15T18:23:36.464971+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652300+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:20:46.652300+0000","last_peered":"2024-01-15T18:20:46.652300+0000","last_clean":"2024-01-15T18:20:46.652300+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:20:46.652300+0000","last_undegraded":"2024-01-15T18:20:46.652300+0000","last_fullsized":"2024-01-15T18:20:46.652300+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:24.049276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.465808+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:23:36.465808+0000","last_peered":"2024-01-15T18:23:36.465808+0000","last_clean":"2024-01-15T18:23:36.465808+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:23:36.465808+0000","last_undegraded":"2024-01-15T18:23:36.465808+0000","last_fullsized":"2024-01-15T18:23:36.465808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:24.340948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'3","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.878194+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:19:41.878194+0000","last_peered":"2024-01-15T18:19:41.878194+0000","last_clean":"2024-01-15T18:19:41.878194+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:19:41.878194+0000","last_undegraded":"2024-01-15T18:19:41.878194+0000","last_fullsized":"2024-01-15T18:19:41.878194+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:51.311221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'51","reported_seq":75,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455773+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:23:36.455773+0000","last_peered":"2024-01-15T18:23:36.455773+0000","last_clean":"2024-01-15T18:23:36.455773+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:23:36.455773+0000","last_undegraded":"2024-01-15T18:23:36.455773+0000","last_fullsized":"2024-01-15T18:23:36.455773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:02.239162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.877947+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:19:41.877947+0000","last_peered":"2024-01-15T18:19:41.877947+0000","last_clean":"2024-01-15T18:19:41.877947+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:19:41.877947+0000","last_undegraded":"2024-01-15T18:19:41.877947+0000","last_fullsized":"2024-01-15T18:19:41.877947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:59.159845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'36","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475421+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:23:36.475421+0000","last_peered":"2024-01-15T18:23:36.475421+0000","last_clean":"2024-01-15T18:23:36.475421+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:36.475421+0000","last_undegraded":"2024-01-15T18:23:36.475421+0000","last_fullsized":"2024-01-15T18:23:36.475421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:01:50.217450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.868481+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:19:41.868481+0000","last_peered":"2024-01-15T18:19:41.868481+0000","last_clean":"2024-01-15T18:19:41.868481+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:19:41.868481+0000","last_undegraded":"2024-01-15T18:19:41.868481+0000","last_fullsized":"2024-01-15T18:19:41.868481+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:23.630535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'44","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.478124+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:23:36.478124+0000","last_peered":"2024-01-15T18:23:36.478124+0000","last_clean":"2024-01-15T18:23:36.478124+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:23:36.478124+0000","last_undegraded":"2024-01-15T18:23:36.478124+0000","last_fullsized":"2024-01-15T18:23:36.478124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:42.040607+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:19:42.040607+0000","last_peered":"2024-01-15T18:19:42.040607+0000","last_clean":"2024-01-15T18:19:42.040607+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:19:42.040607+0000","last_undegraded":"2024-01-15T18:19:42.040607+0000","last_fullsized":"2024-01-15T18:19:42.040607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'45","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.462420+0000","last_change":"2024-01-15T18:19:42.184955+0000","last_active":"2024-01-15T18:23:36.462420+0000","last_peered":"2024-01-15T18:23:36.462420+0000","last_clean":"2024-01-15T18:23:36.462420+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:23:36.462420+0000","last_undegraded":"2024-01-15T18:23:36.462420+0000","last_fullsized":"2024-01-15T18:23:36.462420+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303360,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:11.655302+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:21:11.655302+0000","last_peered":"2024-01-15T18:21:11.655302+0000","last_clean":"2024-01-15T18:21:11.655302+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:21:11.655302+0000","last_undegraded":"2024-01-15T18:21:11.655302+0000","last_fullsized":"2024-01-15T18:21:11.655302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.469773+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:23:36.469773+0000","last_peered":"2024-01-15T18:23:36.469773+0000","last_clean":"2024-01-15T18:23:36.469773+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:23:36.469773+0000","last_undegraded":"2024-01-15T18:23:36.469773+0000","last_fullsized":"2024-01-15T18:23:36.469773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":15,"num_write_kb":53249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:07.082656+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:21:07.082656+0000","last_peered":"2024-01-15T18:21:07.082656+0000","last_clean":"2024-01-15T18:21:07.082656+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:21:07.082656+0000","last_undegraded":"2024-01-15T18:21:07.082656+0000","last_fullsized":"2024-01-15T18:21:07.082656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'41","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463503+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:23:36.463503+0000","last_peered":"2024-01-15T18:23:36.463503+0000","last_clean":"2024-01-15T18:23:36.463503+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:23:36.463503+0000","last_undegraded":"2024-01-15T18:23:36.463503+0000","last_fullsized":"2024-01-15T18:23:36.463503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:06.664989+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:21:06.664989+0000","last_peered":"2024-01-15T18:21:06.664989+0000","last_clean":"2024-01-15T18:21:06.664989+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:21:06.664989+0000","last_undegraded":"2024-01-15T18:21:06.664989+0000","last_fullsized":"2024-01-15T18:21:06.664989+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:01.739271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'35","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.457523+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:23:36.457523+0000","last_peered":"2024-01-15T18:23:36.457523+0000","last_clean":"2024-01-15T18:23:36.457523+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:23:36.457523+0000","last_undegraded":"2024-01-15T18:23:36.457523+0000","last_fullsized":"2024-01-15T18:23:36.457523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:07.287483+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:21:07.287483+0000","last_peered":"2024-01-15T18:21:07.287483+0000","last_clean":"2024-01-15T18:21:07.287483+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:21:07.287483+0000","last_undegraded":"2024-01-15T18:21:07.287483+0000","last_fullsized":"2024-01-15T18:21:07.287483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'41","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466318+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:23:36.466318+0000","last_peered":"2024-01-15T18:23:36.466318+0000","last_clean":"2024-01-15T18:23:36.466318+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:23:36.466318+0000","last_undegraded":"2024-01-15T18:23:36.466318+0000","last_fullsized":"2024-01-15T18:23:36.466318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'11","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:40.071203+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:21:40.071203+0000","last_peered":"2024-01-15T18:21:40.071203+0000","last_clean":"2024-01-15T18:21:40.071203+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:21:40.071203+0000","last_undegraded":"2024-01-15T18:21:40.071203+0000","last_fullsized":"2024-01-15T18:21:40.071203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'40","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.467240+0000","last_change":"2024-01-15T18:18:41.931596+0000","last_active":"2024-01-15T18:23:36.467240+0000","last_peered":"2024-01-15T18:23:36.467240+0000","last_clean":"2024-01-15T18:23:36.467240+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:23:36.467240+0000","last_undegraded":"2024-01-15T18:23:36.467240+0000","last_fullsized":"2024-01-15T18:23:36.467240+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'170","reported_seq":186,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.430521+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:23:36.430521+0000","last_peered":"2024-01-15T18:23:36.430521+0000","last_clean":"2024-01-15T18:23:36.430521+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:23:36.430521+0000","last_undegraded":"2024-01-15T18:23:36.430521+0000","last_fullsized":"2024-01-15T18:23:36.430521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569375,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'36","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470792+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:23:36.470792+0000","last_peered":"2024-01-15T18:23:36.470792+0000","last_clean":"2024-01-15T18:23:36.470792+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:23:36.470792+0000","last_undegraded":"2024-01-15T18:23:36.470792+0000","last_fullsized":"2024-01-15T18:23:36.470792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.664156+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:21:01.664156+0000","last_peered":"2024-01-15T18:21:01.664156+0000","last_clean":"2024-01-15T18:21:01.664156+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:21:01.664156+0000","last_undegraded":"2024-01-15T18:21:01.664156+0000","last_fullsized":"2024-01-15T18:21:01.664156+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'46","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.476804+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:23:36.476804+0000","last_peered":"2024-01-15T18:23:36.476804+0000","last_clean":"2024-01-15T18:23:36.476804+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:23:36.476804+0000","last_undegraded":"2024-01-15T18:23:36.476804+0000","last_fullsized":"2024-01-15T18:23:36.476804+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.653868+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:21:01.653868+0000","last_peered":"2024-01-15T18:21:01.653868+0000","last_clean":"2024-01-15T18:21:01.653868+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:21:01.653868+0000","last_undegraded":"2024-01-15T18:21:01.653868+0000","last_fullsized":"2024-01-15T18:21:01.653868+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'43","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480708+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:23:36.480708+0000","last_peered":"2024-01-15T18:23:36.480708+0000","last_clean":"2024-01-15T18:23:36.480708+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:23:36.480708+0000","last_undegraded":"2024-01-15T18:23:36.480708+0000","last_fullsized":"2024-01-15T18:23:36.480708+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:51:04.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080645,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":22,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":24,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.664196+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:21:01.664196+0000","last_peered":"2024-01-15T18:21:01.664196+0000","last_clean":"2024-01-15T18:21:01.664196+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:21:01.664196+0000","last_undegraded":"2024-01-15T18:21:01.664196+0000","last_fullsized":"2024-01-15T18:21:01.664196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:27:26.085701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'21","reported_seq":38,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479169+0000","last_change":"2024-01-15T18:18:41.936562+0000","last_active":"2024-01-15T18:23:36.479169+0000","last_peered":"2024-01-15T18:23:36.479169+0000","last_clean":"2024-01-15T18:23:36.479169+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:23:36.479169+0000","last_undegraded":"2024-01-15T18:23:36.479169+0000","last_fullsized":"2024-01-15T18:23:36.479169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:57.081108+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:20:57.081108+0000","last_peered":"2024-01-15T18:20:57.081108+0000","last_clean":"2024-01-15T18:20:57.081108+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:20:57.081108+0000","last_undegraded":"2024-01-15T18:20:57.081108+0000","last_fullsized":"2024-01-15T18:20:57.081108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'35","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.472587+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:23:36.472587+0000","last_peered":"2024-01-15T18:23:36.472587+0000","last_clean":"2024-01-15T18:23:36.472587+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:23:36.472587+0000","last_undegraded":"2024-01-15T18:23:36.472587+0000","last_fullsized":"2024-01-15T18:23:36.472587+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:48:34.695980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663623+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:20:56.663623+0000","last_peered":"2024-01-15T18:20:56.663623+0000","last_clean":"2024-01-15T18:20:56.663623+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:20:56.663623+0000","last_undegraded":"2024-01-15T18:20:56.663623+0000","last_fullsized":"2024-01-15T18:20:56.663623+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:38.154337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'41","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.468898+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:23:36.468898+0000","last_peered":"2024-01-15T18:23:36.468898+0000","last_clean":"2024-01-15T18:23:36.468898+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:23:36.468898+0000","last_undegraded":"2024-01-15T18:23:36.468898+0000","last_fullsized":"2024-01-15T18:23:36.468898+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":7,"num_read_kb":12292,"num_write":19,"num_write_kb":61442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653254+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:20:56.653254+0000","last_peered":"2024-01-15T18:20:56.653254+0000","last_clean":"2024-01-15T18:20:56.653254+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:20:56.653254+0000","last_undegraded":"2024-01-15T18:20:56.653254+0000","last_fullsized":"2024-01-15T18:20:56.653254+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:00:59.694593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'40","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463772+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:23:36.463772+0000","last_peered":"2024-01-15T18:23:36.463772+0000","last_clean":"2024-01-15T18:23:36.463772+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:23:36.463772+0000","last_undegraded":"2024-01-15T18:23:36.463772+0000","last_fullsized":"2024-01-15T18:23:36.463772+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:26.983917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.656234+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:20:56.656234+0000","last_peered":"2024-01-15T18:20:56.656234+0000","last_clean":"2024-01-15T18:20:56.656234+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:20:56.656234+0000","last_undegraded":"2024-01-15T18:20:56.656234+0000","last_fullsized":"2024-01-15T18:20:56.656234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:42.895411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'34","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480616+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:23:36.480616+0000","last_peered":"2024-01-15T18:23:36.480616+0000","last_clean":"2024-01-15T18:23:36.480616+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:23:36.480616+0000","last_undegraded":"2024-01-15T18:23:36.480616+0000","last_fullsized":"2024-01-15T18:23:36.480616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'3","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663515+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:20:56.663515+0000","last_peered":"2024-01-15T18:20:56.663515+0000","last_clean":"2024-01-15T18:20:56.663515+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:20:56.663515+0000","last_undegraded":"2024-01-15T18:20:56.663515+0000","last_fullsized":"2024-01-15T18:20:56.663515+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481081+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:23:36.481081+0000","last_peered":"2024-01-15T18:23:36.481081+0000","last_clean":"2024-01-15T18:23:36.481081+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:23:36.481081+0000","last_undegraded":"2024-01-15T18:23:36.481081+0000","last_fullsized":"2024-01-15T18:23:36.481081+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:14.044300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663534+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:20:56.663534+0000","last_peered":"2024-01-15T18:20:56.663534+0000","last_clean":"2024-01-15T18:20:56.663534+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:20:56.663534+0000","last_undegraded":"2024-01-15T18:20:56.663534+0000","last_fullsized":"2024-01-15T18:20:56.663534+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:51:31.110057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'40","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.467085+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:23:36.467085+0000","last_peered":"2024-01-15T18:23:36.467085+0000","last_clean":"2024-01-15T18:23:36.467085+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:23:36.467085+0000","last_undegraded":"2024-01-15T18:23:36.467085+0000","last_fullsized":"2024-01-15T18:23:36.467085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:51.662713+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:20:51.662713+0000","last_peered":"2024-01-15T18:20:51.662713+0000","last_clean":"2024-01-15T18:20:51.662713+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:20:51.662713+0000","last_undegraded":"2024-01-15T18:20:51.662713+0000","last_fullsized":"2024-01-15T18:20:51.662713+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'40","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.462441+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:23:36.462441+0000","last_peered":"2024-01-15T18:23:36.462441+0000","last_clean":"2024-01-15T18:23:36.462441+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:23:36.462441+0000","last_undegraded":"2024-01-15T18:23:36.462441+0000","last_fullsized":"2024-01-15T18:23:36.462441+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720517,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":15,"num_write_kb":57345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:52.080245+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:20:52.080245+0000","last_peered":"2024-01-15T18:20:52.080245+0000","last_clean":"2024-01-15T18:20:52.080245+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:20:52.080245+0000","last_undegraded":"2024-01-15T18:20:52.080245+0000","last_fullsized":"2024-01-15T18:20:52.080245+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'33","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.461910+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:23:36.461910+0000","last_peered":"2024-01-15T18:23:36.461910+0000","last_clean":"2024-01-15T18:23:36.461910+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:23:36.461910+0000","last_undegraded":"2024-01-15T18:23:36.461910+0000","last_fullsized":"2024-01-15T18:23:36.461910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:52.284956+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:20:52.284956+0000","last_peered":"2024-01-15T18:20:52.284956+0000","last_clean":"2024-01-15T18:20:52.284956+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:20:52.284956+0000","last_undegraded":"2024-01-15T18:20:52.284956+0000","last_fullsized":"2024-01-15T18:20:52.284956+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'37","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.490017+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:23:36.490017+0000","last_peered":"2024-01-15T18:23:36.490017+0000","last_clean":"2024-01-15T18:23:36.490017+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:23:36.490017+0000","last_undegraded":"2024-01-15T18:23:36.490017+0000","last_fullsized":"2024-01-15T18:23:36.490017+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:38:55.463538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'15","reported_seq":31,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:42.035206+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:22:42.035206+0000","last_peered":"2024-01-15T18:22:42.035206+0000","last_clean":"2024-01-15T18:22:42.035206+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:22:42.035206+0000","last_undegraded":"2024-01-15T18:22:42.035206+0000","last_fullsized":"2024-01-15T18:22:42.035206+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:51.313218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'42","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479751+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:23:36.479751+0000","last_peered":"2024-01-15T18:23:36.479751+0000","last_clean":"2024-01-15T18:23:36.479751+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:23:36.479751+0000","last_undegraded":"2024-01-15T18:23:36.479751+0000","last_fullsized":"2024-01-15T18:23:36.479751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:47.175404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:47.079647+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:20:47.079647+0000","last_peered":"2024-01-15T18:20:47.079647+0000","last_clean":"2024-01-15T18:20:47.079647+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:20:47.079647+0000","last_undegraded":"2024-01-15T18:20:47.079647+0000","last_fullsized":"2024-01-15T18:20:47.079647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'34","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.460078+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:23:36.460078+0000","last_peered":"2024-01-15T18:23:36.460078+0000","last_clean":"2024-01-15T18:23:36.460078+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:23:36.460078+0000","last_undegraded":"2024-01-15T18:23:36.460078+0000","last_fullsized":"2024-01-15T18:23:36.460078+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:27.713450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331770,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":5,"num_read_kb":20480,"num_write":14,"num_write_kb":49153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:47.284362+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:20:47.284362+0000","last_peered":"2024-01-15T18:20:47.284362+0000","last_clean":"2024-01-15T18:20:47.284362+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:20:47.284362+0000","last_undegraded":"2024-01-15T18:20:47.284362+0000","last_fullsized":"2024-01-15T18:20:47.284362+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'44","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480400+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:23:36.480400+0000","last_peered":"2024-01-15T18:23:36.480400+0000","last_clean":"2024-01-15T18:23:36.480400+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:23:36.480400+0000","last_undegraded":"2024-01-15T18:23:36.480400+0000","last_fullsized":"2024-01-15T18:23:36.480400+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655069+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:20:46.655069+0000","last_peered":"2024-01-15T18:20:46.655069+0000","last_clean":"2024-01-15T18:20:46.655069+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:20:46.655069+0000","last_undegraded":"2024-01-15T18:20:46.655069+0000","last_fullsized":"2024-01-15T18:20:46.655069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:54.586280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'51","reported_seq":72,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480979+0000","last_change":"2024-01-15T18:18:41.935993+0000","last_active":"2024-01-15T18:23:36.480979+0000","last_peered":"2024-01-15T18:23:36.480979+0000","last_clean":"2024-01-15T18:23:36.480979+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:23:36.480979+0000","last_undegraded":"2024-01-15T18:23:36.480979+0000","last_fullsized":"2024-01-15T18:23:36.480979+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652145+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:20:46.652145+0000","last_peered":"2024-01-15T18:20:46.652145+0000","last_clean":"2024-01-15T18:20:46.652145+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:20:46.652145+0000","last_undegraded":"2024-01-15T18:20:46.652145+0000","last_fullsized":"2024-01-15T18:20:46.652145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:26.705662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'38","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466662+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:23:36.466662+0000","last_peered":"2024-01-15T18:23:36.466662+0000","last_clean":"2024-01-15T18:23:36.466662+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:23:36.466662+0000","last_undegraded":"2024-01-15T18:23:36.466662+0000","last_fullsized":"2024-01-15T18:23:36.466662+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'10","reported_seq":35,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:40.067251+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:21:40.067251+0000","last_peered":"2024-01-15T18:21:40.067251+0000","last_clean":"2024-01-15T18:21:40.067251+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:21:40.067251+0000","last_undegraded":"2024-01-15T18:21:40.067251+0000","last_fullsized":"2024-01-15T18:21:40.067251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:03:44.404981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'36","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475826+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:23:36.475826+0000","last_peered":"2024-01-15T18:23:36.475826+0000","last_clean":"2024-01-15T18:23:36.475826+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:36.475826+0000","last_undegraded":"2024-01-15T18:23:36.475826+0000","last_fullsized":"2024-01-15T18:23:36.475826+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:52.187345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:42.078947+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:20:42.078947+0000","last_peered":"2024-01-15T18:20:42.078947+0000","last_clean":"2024-01-15T18:20:42.078947+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:20:42.078947+0000","last_undegraded":"2024-01-15T18:20:42.078947+0000","last_fullsized":"2024-01-15T18:20:42.078947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'30","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475835+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:23:36.475835+0000","last_peered":"2024-01-15T18:23:36.475835+0000","last_clean":"2024-01-15T18:23:36.475835+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:23:36.475835+0000","last_undegraded":"2024-01-15T18:23:36.475835+0000","last_fullsized":"2024-01-15T18:23:36.475835+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.656696+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:21:01.656696+0000","last_peered":"2024-01-15T18:21:01.656696+0000","last_clean":"2024-01-15T18:21:01.656696+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:21:01.656696+0000","last_undegraded":"2024-01-15T18:21:01.656696+0000","last_fullsized":"2024-01-15T18:21:01.656696+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:35.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":246,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653639+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:20:56.653639+0000","last_peered":"2024-01-15T18:20:56.653639+0000","last_clean":"2024-01-15T18:20:56.653639+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:20:56.653639+0000","last_undegraded":"2024-01-15T18:20:56.653639+0000","last_fullsized":"2024-01-15T18:20:56.653639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":4294968312,"num_objects":1030,"num_object_clones":1,"num_object_copies":2060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1030,"num_whiteouts":1,"num_read":269,"num_read_kb":1048589,"num_write":1050,"num_write_kb":4194319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8589975552,"data_stored":8589936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2554,"ondisk_log_size":2554,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":571494,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1142988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461744,"kb_used_data":1398696,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92910096,"statfs":{"total":96636764160,"available":95139938304,"internally_reserved":0,"allocated":1432264704,"data_stored":1431460098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69099999999999995,"15min":0.94399999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.29999999999999999},"max":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.627},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.87},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":13.744},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":1.0840000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":16.920000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":1.05},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":1.04,"15min":31.774999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72599999999999998,"15min":1},"min":{"1min":0.57399999999999995,"5min":0.48699999999999999,"15min":0.377},"max":{"1min":1.056,"5min":1.085,"15min":30.846},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":1.0629999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":1.0229999999999999,"15min":39.695},"last":0.69499999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74099999999999999,"15min":1.1499999999999999},"min":{"1min":0.54500000000000004,"5min":0.437,"15min":0.38800000000000001},"max":{"1min":1.01,"5min":1.0109999999999999,"15min":26.463999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":1.198},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.41499999999999998},"max":{"1min":0.86699999999999999,"5min":1.0640000000000001,"15min":42.692},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86699999999999999,"5min":0.874,"15min":14.087},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.36299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":2.952},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":11.334},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.746},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91700000000000004,"5min":0.94499999999999995,"15min":8.423},"last":0.52300000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.78200000000000003},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.45500000000000002},"max":{"1min":0.95099999999999996,"5min":1.034,"15min":8.5030000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.35299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":3.3450000000000002},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":998316,"kb_used_data":935268,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93373524,"statfs":{"total":96636764160,"available":95614488576,"internally_reserved":0,"allocated":957714432,"data_stored":956912954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":22.754000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.82399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":1.018,"15min":9.1470000000000002},"last":0.65300000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.35299999999999998,"15min":0.34300000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":14.246},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":0.98199999999999998},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":0.97899999999999998,"15min":19.475000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":1.163},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.0549999999999999,"15min":42.825000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.747,"15min":1.107},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":1.103,"15min":42.851999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.98599999999999999},"min":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.8160000000000001,"15min":24.199000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":1.0700000000000001},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":1.03,"15min":18.791},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":1.044,"15min":19.114999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":8.3059999999999992},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.88800000000000001,"5min":1.0920000000000001,"15min":7.5880000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":0.93500000000000005,"5min":1.1120000000000001,"15min":12.945},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69099999999999995,"15min":0.80100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.38600000000000001},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":11.907},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.85399999999999998},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.86599999999999999,"5min":1.1910000000000001,"15min":27.651},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793712,"kb_used_data":718184,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93578128,"statfs":{"total":96636764160,"available":95824003072,"internally_reserved":0,"allocated":735420416,"data_stored":734615446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68400000000000005,"15min":0.85199999999999998},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.29899999999999999},"max":{"1min":0.98599999999999999,"5min":1.9350000000000001,"15min":11.519},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74099999999999999,"15min":0.94599999999999995},"min":{"1min":0.626,"5min":0.371,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":2.6819999999999999,"15min":12.952},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":2.1179999999999999,"15min":13.358000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.96699999999999997},"min":{"1min":0.56999999999999995,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":1.0289999999999999,"5min":2.8050000000000002,"15min":21.856999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":3.508,"15min":18.314},"last":0.62},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":1.0449999999999999},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":1.069,"5min":3.5310000000000001,"15min":21.120000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":6.7990000000000004,"15min":13.317},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":1.0369999999999999},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":2.3380000000000001,"15min":19.638000000000002},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.96399999999999997,"15min":11.093},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.95699999999999996,"15min":3.3889999999999998},"last":0.38900000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.78900000000000003},"min":{"1min":0.5,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":5.8680000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32600000000000001},"max":{"1min":0.91300000000000003,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.316}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":1.081,"15min":3.2759999999999998},"last":0.432},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.754,"15min":0.80300000000000005},"min":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.88300000000000001,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1387432,"kb_used_data":1324384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92984408,"statfs":{"total":96636764160,"available":95216033792,"internally_reserved":0,"allocated":1356169216,"data_stored":1355371903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.94899999999999995},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":1.02,"15min":34.802},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.91300000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":0.871,"5min":1.1319999999999999,"15min":22.335999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":0.90600000000000003,"5min":1.224,"15min":38.198},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.752,"15min":0.96799999999999997},"min":{"1min":0.52400000000000002,"5min":0.438,"15min":0.372},"max":{"1min":0.875,"5min":1.0269999999999999,"15min":38.220999999999997},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.89600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.34599999999999997},"max":{"1min":0.91600000000000004,"5min":1.0509999999999999,"15min":7.6989999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76000000000000001,"15min":0.85999999999999999},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":1.0740000000000001,"15min":7.4829999999999997},"last":0.77700000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.95199999999999996},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":1.214,"15min":14.529},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.94699999999999995},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":25.774000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.68899999999999995},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.33200000000000002},"max":{"1min":0.89800000000000002,"5min":0.90300000000000002,"15min":9.9619999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.79700000000000004},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":1.1719999999999999,"15min":25.887},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":10.284000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.97299999999999998,"15min":3.6949999999999998},"last":0.73099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":13.760999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.36199999999999999},"max":{"1min":0.96699999999999997,"5min":1.0269999999999999,"15min":11.67},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1154596,"kb_used_data":1087388,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93217244,"statfs":{"total":96636764160,"available":95454457856,"internally_reserved":0,"allocated":1113485312,"data_stored":1112671977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.66200000000000003,"5min":2.415,"15min":26.364000000000001},"last":0.44},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.95499999999999996},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.79500000000000004,"15min":28.332000000000001},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.98399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63300000000000001,"5min":0.82499999999999996,"15min":40.457999999999998},"last":0.315},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":1.1120000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":27.427},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.98799999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":23.036999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":1.081},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.873,"15min":26.404},"last":0.504}]},{"osd":4,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":1.0349999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.0529999999999999,"5min":1.1439999999999999,"15min":22.998000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76300000000000001,"15min":1.0740000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1,"5min":1.1439999999999999,"15min":20.399000000000001},"last":0.746}]},{"osd":5,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":1.0820000000000001},"min":{"1min":0.52800000000000002,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":0.97999999999999998,"5min":1.46,"15min":26.314},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":1.1040000000000001},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1,"5min":1.109,"15min":22.908999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79000000000000004,"15min":1.155},"min":{"1min":0.57399999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.105,"5min":1.105,"15min":20.445},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81399999999999995,"15min":1.1659999999999999},"min":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":1.4299999999999999,"15min":26.091999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":1.159},"min":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":23.111999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":1.3169999999999999},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.46400000000000002},"max":{"1min":1.0880000000000001,"5min":1.145,"15min":44.817999999999998},"last":0.85099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768936,"kb_used_data":705888,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93602904,"statfs":{"total":96636764160,"available":95849373696,"internally_reserved":0,"allocated":722829312,"data_stored":722033059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":1.054},"min":{"1min":0.28599999999999998,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.82899999999999996,"5min":4.2329999999999997,"15min":29.527999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":1.335},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":6.0670000000000002,"15min":34.122999999999998},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":1.534},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":4.5259999999999998,"15min":113.149},"last":0.498},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59099999999999997,"15min":1.127},"min":{"1min":0.40500000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84899999999999998,"5min":6.1020000000000003,"15min":41.302},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":1.3899999999999999},"min":{"1min":0.34699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76800000000000002,"5min":4.7370000000000001,"15min":46.960000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.625,"15min":1.2809999999999999},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":3.8809999999999998,"15min":115.693},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83199999999999996,"15min":1.194},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":1.1100000000000001,"5min":3.8399999999999999,"15min":24.366},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":1.228},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.432},"max":{"1min":1.0189999999999999,"5min":2.343,"15min":20.489000000000001},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":1.5289999999999999},"min":{"1min":0.61899999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":1.175,"5min":3.927,"15min":41.683999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80400000000000005,"15min":1.2889999999999999},"min":{"1min":0.60799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98599999999999999,"5min":2.3029999999999999,"15min":26.084},"last":0.70899999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":1.319},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.159,"5min":6.0220000000000002,"15min":22.199000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":1.569},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.0860000000000001,"5min":3.9590000000000001,"15min":41.738999999999997},"last":0.621}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85599999999999998,"15min":1.262},"min":{"1min":0.621,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.1339999999999999,"5min":2.556,"15min":20.446999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":1.6060000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1599999999999999,"5min":2.5939999999999999,"15min":41.402000000000001},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":941048,"kb_used_data":873840,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93430792,"statfs":{"total":96636764160,"available":95673131008,"internally_reserved":0,"allocated":894812160,"data_stored":894000019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.439,"15min":0.86099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.63700000000000001,"5min":1.4179999999999999,"15min":39.948},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.56599999999999995,"15min":1.0649999999999999},"min":{"1min":0.30399999999999999,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":4.9329999999999998,"15min":35.790999999999997},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.57699999999999996,"15min":0.97099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.27600000000000002},"max":{"1min":0.66100000000000003,"5min":9.2609999999999992,"15min":49.229999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":1.7649999999999999,"15min":10.349},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":1.0589999999999999},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68300000000000005,"5min":0.84999999999999998,"15min":27.643999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.90800000000000003},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":1.198,"15min":15.608000000000001},"last":0.498}]},{"osd":4,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76500000000000001,"15min":1.165},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":1.1080000000000001,"5min":1.383,"15min":27.489999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":1.0349999999999999},"min":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.218,"5min":6.6950000000000003,"15min":10.038},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76900000000000002,"15min":1.1779999999999999},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1739999999999999,"5min":1.2010000000000001,"15min":27.547000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.55400000000000005},"max":{"1min":1.1599999999999999,"5min":1.266,"15min":9.9610000000000003},"last":0.90000000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80600000000000005,"15min":1.171},"min":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.496},"max":{"1min":1.147,"5min":1.984,"15min":27.579999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":1},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.42399999999999999},"max":{"1min":1.1910000000000001,"5min":5.6470000000000002,"15min":11.712999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":1.0980000000000001},"min":{"1min":0.56899999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":5.819,"15min":11.833},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.89000000000000001,"15min":1.0920000000000001},"min":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.502},"max":{"1min":1.254,"5min":6.3570000000000002,"15min":14.606},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575077,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1421360,"kb_used_data":1358312,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92950480,"statfs":{"total":96636764160,"available":95181291520,"internally_reserved":0,"allocated":1390911488,"data_stored":1390086416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46300000000000002,"15min":1.054},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":31.977},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.51300000000000001,"15min":1.0349999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.61499999999999999,"5min":1.6579999999999999,"15min":36.145000000000003},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":1.1439999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":1.1759999999999999,"15min":41.387999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":1.0089999999999999},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70399999999999996,"5min":1.7689999999999999,"15min":33.219999999999999},"last":0.503}]},{"osd":3,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":1.006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80500000000000005,"15min":31.943999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":1.107},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.746,"5min":0.95599999999999996,"15min":45.128},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.38300000000000001},"max":{"1min":1.0469999999999999,"5min":1.4550000000000001,"15min":32.932000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.77400000000000002,"15min":1.1659999999999999},"min":{"1min":0.53600000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":1.115,"15min":31.782},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":1.149},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.38600000000000001},"max":{"1min":1.0249999999999999,"5min":1.179,"15min":31.858000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.79300000000000004,"15min":1.0409999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.7869999999999999,"15min":33.151000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":1.0660000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":13.055},"last":0.621},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.47699999999999998},"max":{"1min":1.262,"5min":1.7170000000000001,"15min":33.098999999999997},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":1.097},"min":{"1min":0.68200000000000005,"5min":0.55500000000000005,"15min":0.48899999999999999},"max":{"1min":1.0029999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":1.2050000000000001},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":0.997,"5min":1.075,"15min":31.905999999999999},"last":0.59799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":569487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":569445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1388326912,"data_stored":1388315187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":893394944,"data_stored":893387124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":721420288,"data_stored":721420288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1111498752,"data_stored":1111491082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1354764288,"data_stored":1354760453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":734007296,"data_stored":734003392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":956305408,"data_stored":956301434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1430257664,"data_stored":1430257664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:23:41.635 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:23:41.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.815+0000 7f82e5e00640 1 Processor -- start 2024-01-15T18:23:41.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.815+0000 7f82e5e00640 1 -- start start 2024-01-15T18:23:41.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82e5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 0x7f82e015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82e5e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00cc810 con 0x7f82e0070cc0 2024-01-15T18:23:41.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82e5e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e00cbf70 con 0x7f82e015f1b0 2024-01-15T18:23:41.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82e5e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e006e9f0 con 0x7f82e006dff0 2024-01-15T18:23:41.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82e0070cc0 0x7f82e00710a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60688/0 (socket says 172.21.15.143:60688) 2024-01-15T18:23:41.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 -- 172.21.15.143:0/3022801686 learned_addr learned my addr 172.21.15.143:0/3022801686 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:41.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666762777 0 0) 0x7f82d0001120 con 0x7f82e0070cc0 2024-01-15T18:23:41.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82de000640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 0x7f82e015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.811 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003530 con 0x7f82e0070cc0 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3770965951 0 0) 0x7f82c0003530 con 0x7f82e0070cc0 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 msgr2=0x7f82e015bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 0x7f82e015bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 >> v1:172.21.15.204:6789/0 conn(0x7f82e015f1b0 legacy=0x7f82e006a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e00cc250 con 0x7f82e0070cc0 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f82d0001810 con 0x7f82e0070cc0 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 >> v1:172.21.15.143:6789/0 conn(0x7f82e0070cc0 legacy=0x7f82e00710a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.812 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 0x7f82c00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c0004d10 0x7f82c0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e006e9f0 con 0x7f82c00039b0 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000baa0 con 0x7f82c0004d10 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000bc60 con 0x7f82e006dff0 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82de000640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 0x7f82c00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82df400640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c0004d10 0x7f82c0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 -- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 msgr2=0x7f82c00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:41.813 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e006dff0 0x7f82c00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 -- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c0004d10 msgr2=0x7f82c0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:41.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c0004d10 0x7f82c0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.814 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.819+0000 7f82dea00640 1 -- 172.21.15.143:0/3022801686 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0003530 con 0x7f82c00039b0 2024-01-15T18:23:41.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82dea00640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82c0003dd0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f82d0001ec0 tx=0x7f82d001cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318bed7cecfa5341 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:41.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0005ae0 con 0x7f82c00039b0 2024-01-15T18:23:41.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0005c60 con 0x7f82c00039b0 2024-01-15T18:23:41.815 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82dd600640 1 -- 172.21.15.143:0/3022801686 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0007070 con 0x7f82c00039b0 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82e5e00640 1 -- 172.21.15.143:0/3022801686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 msgr2=0x7f82c0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82e5e00640 1 --2- 172.21.15.143:0/3022801686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82c0003dd0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f82d0001ec0 tx=0x7f82d001cb20 comp rx=0 tx=0).stop 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82df400640 1 -- 172.21.15.143:0/3022801686 reap_dead start 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82e5e00640 1 -- 172.21.15.143:0/3022801686 shutdown_connections 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82e5e00640 1 -- 172.21.15.143:0/3022801686 >> 172.21.15.143:0/3022801686 conn(0x7f82e00b1610 msgr2=0x7f82e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82e5e00640 1 -- 172.21.15.143:0/3022801686 shutdown_connections 2024-01-15T18:23:41.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.823+0000 7f82e5e00640 1 -- 172.21.15.143:0/3022801686 wait complete. 2024-01-15T18:23:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 Processor -- start 2024-01-15T18:23:41.817 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 -- start start 2024-01-15T18:23:41.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82e0150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82e01510e0 0x7f82e0159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e0159570 0x7f82e0159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e0165000 con 0x7f82c00039b0 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e0150e80 con 0x7f82e01510e0 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e0164a60 con 0x7f82e0159570 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82e0150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82e0150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38052/0 (socket says 172.21.15.143:38052) 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 -- 172.21.15.143:0/795662062 learned_addr learned my addr 172.21.15.143:0/795662062 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82df400640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e0159570 0x7f82e0159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82de000640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82e01510e0 0x7f82e0159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 -- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e0159570 msgr2=0x7f82e0159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e0159570 0x7f82e0159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 -- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82e01510e0 msgr2=0x7f82e0159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82e01510e0 0x7f82e0159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 -- 172.21.15.143:0/795662062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e0164d20 con 0x7f82c00039b0 2024-01-15T18:23:41.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82dea00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82e0150940 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f82d0001ec0 tx=0x7f82c0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:41.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00b4070 con 0x7f82c00039b0 2024-01-15T18:23:41.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0005dd0 con 0x7f82c00039b0 2024-01-15T18:23:41.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e0155330 con 0x7f82c00039b0 2024-01-15T18:23:41.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0001790 con 0x7f82c00039b0 2024-01-15T18:23:41.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.827+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e01558f0 con 0x7f82c00039b0 2024-01-15T18:23:41.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.831+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e0074b00 con 0x7f82c00039b0 2024-01-15T18:23:41.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.831+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f82d0007040 con 0x7f82c00039b0 2024-01-15T18:23:41.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.831+0000 7f82d7400640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82b8067bb0 0x7f82b806a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:41.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.831+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f82d011cb00 con 0x7f82c00039b0 2024-01-15T18:23:41.823 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.831+0000 7f82de000640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82b8067bb0 0x7f82b806a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:41.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.831+0000 7f82de000640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82b8067bb0 0x7f82b806a070 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f82e0151c30 tx=0x7f82c8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.835+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f82d00eb410 con 0x7f82c00039b0 2024-01-15T18:23:41.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.983+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82e0158eb0 con 0x7f82b8067bb0 2024-01-15T18:23:41.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.991+0000 7f82d7400640 1 -- 172.21.15.143:0/795662062 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291090 (secure 0 0 0) 0x7f82c8019070 con 0x7f82b8067bb0 2024-01-15T18:23:41.986 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:41.987 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82b8067bb0 msgr2=0x7f82b806a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82b8067bb0 0x7f82b806a070 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f82e0151c30 tx=0x7f82c8009190 comp rx=0 tx=0).stop 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 msgr2=0x7f82e0150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82e0150940 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f82d0001ec0 tx=0x7f82c0005800 comp rx=0 tx=0).stop 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 shutdown_connections 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f82b8067bb0 0x7f82b806a070 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f82e0159570 0x7f82e0159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.989 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82e01510e0 0x7f82e0159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 --2- 172.21.15.143:0/795662062 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82c00039b0 0x7f82e0150940 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 >> 172.21.15.143:0/795662062 conn(0x7f82e00b1610 msgr2=0x7f82e015b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 shutdown_connections 2024-01-15T18:23:41.990 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:41.995+0000 7f82e5e00640 1 -- 172.21.15.143:0/795662062 wait complete. 2024-01-15T18:23:42.008 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":673,"stamp":"2024-01-15T18:23:40.974303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4296130174,"num_objects":1055,"num_object_clones":1,"num_object_copies":2110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":1,"num_read":394,"num_read_kb":1048698,"num_write":1465,"num_write_kb":4197407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2909,"ondisk_log_size":2909,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8927144,"kb_used_data":8401960,"kb_used_omap":12,"kb_used_meta":525107,"kb_avail":746047576,"statfs":{"total":773094113280,"available":763952717824,"internally_reserved":0,"allocated":8603607040,"data_stored":8597151872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":537710128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":68,"apply_latency_ms":68,"commit_latency_ns":68000000,"apply_latency_ns":68000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002584"},"pg_stats":[{"pgid":"17.3f","version":"113'49","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480416+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:23:36.480416+0000","last_peered":"2024-01-15T18:23:36.480416+0000","last_clean":"2024-01-15T18:23:36.480416+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:23:36.480416+0000","last_undegraded":"2024-01-15T18:23:36.480416+0000","last_fullsized":"2024-01-15T18:23:36.480416+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:00.418536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'42","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480084+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:23:36.480084+0000","last_peered":"2024-01-15T18:23:36.480084+0000","last_clean":"2024-01-15T18:23:36.480084+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:23:36.480084+0000","last_undegraded":"2024-01-15T18:23:36.480084+0000","last_fullsized":"2024-01-15T18:23:36.480084+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'39","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481097+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:23:36.481097+0000","last_peered":"2024-01-15T18:23:36.481097+0000","last_clean":"2024-01-15T18:23:36.481097+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:23:36.481097+0000","last_undegraded":"2024-01-15T18:23:36.481097+0000","last_fullsized":"2024-01-15T18:23:36.481097+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:07.807378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":16,"num_write_kb":57345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.490145+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:23:36.490145+0000","last_peered":"2024-01-15T18:23:36.490145+0000","last_clean":"2024-01-15T18:23:36.490145+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:36.490145+0000","last_undegraded":"2024-01-15T18:23:36.490145+0000","last_fullsized":"2024-01-15T18:23:36.490145+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:44.886136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'25","reported_seq":42,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.450344+0000","last_change":"2024-01-15T18:18:41.951758+0000","last_active":"2024-01-15T18:23:36.450344+0000","last_peered":"2024-01-15T18:23:36.450344+0000","last_clean":"2024-01-15T18:23:36.450344+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:23:36.450344+0000","last_undegraded":"2024-01-15T18:23:36.450344+0000","last_fullsized":"2024-01-15T18:23:36.450344+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:51.159719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'45","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463710+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:23:36.463710+0000","last_peered":"2024-01-15T18:23:36.463710+0000","last_clean":"2024-01-15T18:23:36.463710+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:23:36.463710+0000","last_undegraded":"2024-01-15T18:23:36.463710+0000","last_fullsized":"2024-01-15T18:23:36.463710+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:48.760709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'41","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470502+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:23:36.470502+0000","last_peered":"2024-01-15T18:23:36.470502+0000","last_clean":"2024-01-15T18:23:36.470502+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:23:36.470502+0000","last_undegraded":"2024-01-15T18:23:36.470502+0000","last_fullsized":"2024-01-15T18:23:36.470502+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'42","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466790+0000","last_change":"2024-01-15T18:18:41.932178+0000","last_active":"2024-01-15T18:23:36.466790+0000","last_peered":"2024-01-15T18:23:36.466790+0000","last_clean":"2024-01-15T18:23:36.466790+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:23:36.466790+0000","last_undegraded":"2024-01-15T18:23:36.466790+0000","last_fullsized":"2024-01-15T18:23:36.466790+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:32.753279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'42","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475770+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:23:36.475770+0000","last_peered":"2024-01-15T18:23:36.475770+0000","last_clean":"2024-01-15T18:23:36.475770+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:23:36.475770+0000","last_undegraded":"2024-01-15T18:23:36.475770+0000","last_fullsized":"2024-01-15T18:23:36.475770+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:16.576477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'41","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.478544+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:23:36.478544+0000","last_peered":"2024-01-15T18:23:36.478544+0000","last_clean":"2024-01-15T18:23:36.478544+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:23:36.478544+0000","last_undegraded":"2024-01-15T18:23:36.478544+0000","last_fullsized":"2024-01-15T18:23:36.478544+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:16.172780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":11,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'41","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.474632+0000","last_change":"2024-01-15T18:18:41.934119+0000","last_active":"2024-01-15T18:23:36.474632+0000","last_peered":"2024-01-15T18:23:36.474632+0000","last_clean":"2024-01-15T18:23:36.474632+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:23:36.474632+0000","last_undegraded":"2024-01-15T18:23:36.474632+0000","last_fullsized":"2024-01-15T18:23:36.474632+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'43","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477840+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:23:36.477840+0000","last_peered":"2024-01-15T18:23:36.477840+0000","last_clean":"2024-01-15T18:23:36.477840+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:23:36.477840+0000","last_undegraded":"2024-01-15T18:23:36.477840+0000","last_fullsized":"2024-01-15T18:23:36.477840+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:16.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":20481,"num_write":25,"num_write_kb":94209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'34","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.453165+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:23:36.453165+0000","last_peered":"2024-01-15T18:23:36.453165+0000","last_clean":"2024-01-15T18:23:36.453165+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:23:36.453165+0000","last_undegraded":"2024-01-15T18:23:36.453165+0000","last_fullsized":"2024-01-15T18:23:36.453165+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'43","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.459261+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:23:36.459261+0000","last_peered":"2024-01-15T18:23:36.459261+0000","last_clean":"2024-01-15T18:23:36.459261+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:23:36.459261+0000","last_undegraded":"2024-01-15T18:23:36.459261+0000","last_fullsized":"2024-01-15T18:23:36.459261+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:06.614151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'29","reported_seq":47,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.469423+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:23:36.469423+0000","last_peered":"2024-01-15T18:23:36.469423+0000","last_clean":"2024-01-15T18:23:36.469423+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:23:36.469423+0000","last_undegraded":"2024-01-15T18:23:36.469423+0000","last_fullsized":"2024-01-15T18:23:36.469423+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:34.739612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'40","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481202+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:23:36.481202+0000","last_peered":"2024-01-15T18:23:36.481202+0000","last_clean":"2024-01-15T18:23:36.481202+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:23:36.481202+0000","last_undegraded":"2024-01-15T18:23:36.481202+0000","last_fullsized":"2024-01-15T18:23:36.481202+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:47:03.406736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.473510+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:23:36.473510+0000","last_peered":"2024-01-15T18:23:36.473510+0000","last_clean":"2024-01-15T18:23:36.473510+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:23:36.473510+0000","last_undegraded":"2024-01-15T18:23:36.473510+0000","last_fullsized":"2024-01-15T18:23:36.473510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470128+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:23:36.470128+0000","last_peered":"2024-01-15T18:23:36.470128+0000","last_clean":"2024-01-15T18:23:36.470128+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:23:36.470128+0000","last_undegraded":"2024-01-15T18:23:36.470128+0000","last_fullsized":"2024-01-15T18:23:36.470128+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:45.110826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455631+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:23:36.455631+0000","last_peered":"2024-01-15T18:23:36.455631+0000","last_clean":"2024-01-15T18:23:36.455631+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:23:36.455631+0000","last_undegraded":"2024-01-15T18:23:36.455631+0000","last_fullsized":"2024-01-15T18:23:36.455631+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'46","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481283+0000","last_change":"2024-01-15T18:18:41.927062+0000","last_active":"2024-01-15T18:23:36.481283+0000","last_peered":"2024-01-15T18:23:36.481283+0000","last_clean":"2024-01-15T18:23:36.481283+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:36.481283+0000","last_undegraded":"2024-01-15T18:23:36.481283+0000","last_fullsized":"2024-01-15T18:23:36.481283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'50","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477877+0000","last_change":"2024-01-15T18:18:41.937906+0000","last_active":"2024-01-15T18:23:36.477877+0000","last_peered":"2024-01-15T18:23:36.477877+0000","last_clean":"2024-01-15T18:23:36.477877+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:23:36.477877+0000","last_undegraded":"2024-01-15T18:23:36.477877+0000","last_fullsized":"2024-01-15T18:23:36.477877+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:20.422368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":11,"num_read_kb":36866,"num_write":22,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'41","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.464458+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:23:36.464458+0000","last_peered":"2024-01-15T18:23:36.464458+0000","last_clean":"2024-01-15T18:23:36.464458+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:23:36.464458+0000","last_undegraded":"2024-01-15T18:23:36.464458+0000","last_fullsized":"2024-01-15T18:23:36.464458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":17,"num_write_kb":61441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'35","reported_seq":52,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479224+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:23:36.479224+0000","last_peered":"2024-01-15T18:23:36.479224+0000","last_clean":"2024-01-15T18:23:36.479224+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:23:36.479224+0000","last_undegraded":"2024-01-15T18:23:36.479224+0000","last_fullsized":"2024-01-15T18:23:36.479224+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":11,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'39","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477262+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:23:36.477262+0000","last_peered":"2024-01-15T18:23:36.477262+0000","last_clean":"2024-01-15T18:23:36.477262+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:23:36.477262+0000","last_undegraded":"2024-01-15T18:23:36.477262+0000","last_fullsized":"2024-01-15T18:23:36.477262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'35","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477951+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:23:36.477951+0000","last_peered":"2024-01-15T18:23:36.477951+0000","last_clean":"2024-01-15T18:23:36.477951+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:23:36.477951+0000","last_undegraded":"2024-01-15T18:23:36.477951+0000","last_fullsized":"2024-01-15T18:23:36.477951+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:12.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":16386,"num_write":17,"num_write_kb":61441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'50","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.482075+0000","last_change":"2024-01-15T18:19:42.172768+0000","last_active":"2024-01-15T18:23:36.482075+0000","last_peered":"2024-01-15T18:23:36.482075+0000","last_clean":"2024-01-15T18:23:36.482075+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:23:36.482075+0000","last_undegraded":"2024-01-15T18:23:36.482075+0000","last_fullsized":"2024-01-15T18:23:36.482075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:36.268040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'37","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455733+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:23:36.455733+0000","last_peered":"2024-01-15T18:23:36.455733+0000","last_clean":"2024-01-15T18:23:36.455733+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:23:36.455733+0000","last_undegraded":"2024-01-15T18:23:36.455733+0000","last_fullsized":"2024-01-15T18:23:36.455733+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:18.174579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":12289,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'31","reported_seq":48,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.458498+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:23:36.458498+0000","last_peered":"2024-01-15T18:23:36.458498+0000","last_clean":"2024-01-15T18:23:36.458498+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:23:36.458498+0000","last_undegraded":"2024-01-15T18:23:36.458498+0000","last_fullsized":"2024-01-15T18:23:36.458498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'38","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480157+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:23:36.480157+0000","last_peered":"2024-01-15T18:23:36.480157+0000","last_clean":"2024-01-15T18:23:36.480157+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:23:36.480157+0000","last_undegraded":"2024-01-15T18:23:36.480157+0000","last_fullsized":"2024-01-15T18:23:36.480157+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'38","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.458139+0000","last_change":"2024-01-15T18:18:41.937559+0000","last_active":"2024-01-15T18:23:36.458139+0000","last_peered":"2024-01-15T18:23:36.458139+0000","last_clean":"2024-01-15T18:23:36.458139+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:23:36.458139+0000","last_undegraded":"2024-01-15T18:23:36.458139+0000","last_fullsized":"2024-01-15T18:23:36.458139+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5,"num_read_kb":16385,"num_write":15,"num_write_kb":53249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'59","reported_seq":81,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.482554+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:23:36.482554+0000","last_peered":"2024-01-15T18:23:36.482554+0000","last_clean":"2024-01-15T18:23:36.482554+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:23:36.482554+0000","last_undegraded":"2024-01-15T18:23:36.482554+0000","last_fullsized":"2024-01-15T18:23:36.482554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:18:31.192027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'50","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466535+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:23:36.466535+0000","last_peered":"2024-01-15T18:23:36.466535+0000","last_clean":"2024-01-15T18:23:36.466535+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:23:36.466535+0000","last_undegraded":"2024-01-15T18:23:36.466535+0000","last_fullsized":"2024-01-15T18:23:36.466535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:56:19.546961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'28","reported_seq":47,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466086+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:23:36.466086+0000","last_peered":"2024-01-15T18:23:36.466086+0000","last_clean":"2024-01-15T18:23:36.466086+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:23:36.466086+0000","last_undegraded":"2024-01-15T18:23:36.466086+0000","last_fullsized":"2024-01-15T18:23:36.466086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:47.919946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":9,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655110+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:20:46.655110+0000","last_peered":"2024-01-15T18:20:46.655110+0000","last_clean":"2024-01-15T18:20:46.655110+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:20:46.655110+0000","last_undegraded":"2024-01-15T18:20:46.655110+0000","last_fullsized":"2024-01-15T18:20:46.655110+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:01:16.930740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'44","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.464971+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:23:36.464971+0000","last_peered":"2024-01-15T18:23:36.464971+0000","last_clean":"2024-01-15T18:23:36.464971+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:23:36.464971+0000","last_undegraded":"2024-01-15T18:23:36.464971+0000","last_fullsized":"2024-01-15T18:23:36.464971+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652300+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:20:46.652300+0000","last_peered":"2024-01-15T18:20:46.652300+0000","last_clean":"2024-01-15T18:20:46.652300+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:20:46.652300+0000","last_undegraded":"2024-01-15T18:20:46.652300+0000","last_fullsized":"2024-01-15T18:20:46.652300+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:24.049276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.465808+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:23:36.465808+0000","last_peered":"2024-01-15T18:23:36.465808+0000","last_clean":"2024-01-15T18:23:36.465808+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:23:36.465808+0000","last_undegraded":"2024-01-15T18:23:36.465808+0000","last_fullsized":"2024-01-15T18:23:36.465808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:24.340948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'3","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.878194+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:19:41.878194+0000","last_peered":"2024-01-15T18:19:41.878194+0000","last_clean":"2024-01-15T18:19:41.878194+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:19:41.878194+0000","last_undegraded":"2024-01-15T18:19:41.878194+0000","last_fullsized":"2024-01-15T18:19:41.878194+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:51.311221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'51","reported_seq":75,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455773+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:23:36.455773+0000","last_peered":"2024-01-15T18:23:36.455773+0000","last_clean":"2024-01-15T18:23:36.455773+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:23:36.455773+0000","last_undegraded":"2024-01-15T18:23:36.455773+0000","last_fullsized":"2024-01-15T18:23:36.455773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:02.239162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.877947+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:19:41.877947+0000","last_peered":"2024-01-15T18:19:41.877947+0000","last_clean":"2024-01-15T18:19:41.877947+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:19:41.877947+0000","last_undegraded":"2024-01-15T18:19:41.877947+0000","last_fullsized":"2024-01-15T18:19:41.877947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:59.159845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'36","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475421+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:23:36.475421+0000","last_peered":"2024-01-15T18:23:36.475421+0000","last_clean":"2024-01-15T18:23:36.475421+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:36.475421+0000","last_undegraded":"2024-01-15T18:23:36.475421+0000","last_fullsized":"2024-01-15T18:23:36.475421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:01:50.217450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.868481+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:19:41.868481+0000","last_peered":"2024-01-15T18:19:41.868481+0000","last_clean":"2024-01-15T18:19:41.868481+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:19:41.868481+0000","last_undegraded":"2024-01-15T18:19:41.868481+0000","last_fullsized":"2024-01-15T18:19:41.868481+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:23.630535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'44","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.478124+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:23:36.478124+0000","last_peered":"2024-01-15T18:23:36.478124+0000","last_clean":"2024-01-15T18:23:36.478124+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:23:36.478124+0000","last_undegraded":"2024-01-15T18:23:36.478124+0000","last_fullsized":"2024-01-15T18:23:36.478124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:42.040607+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:19:42.040607+0000","last_peered":"2024-01-15T18:19:42.040607+0000","last_clean":"2024-01-15T18:19:42.040607+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:19:42.040607+0000","last_undegraded":"2024-01-15T18:19:42.040607+0000","last_fullsized":"2024-01-15T18:19:42.040607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'45","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.462420+0000","last_change":"2024-01-15T18:19:42.184955+0000","last_active":"2024-01-15T18:23:36.462420+0000","last_peered":"2024-01-15T18:23:36.462420+0000","last_clean":"2024-01-15T18:23:36.462420+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:23:36.462420+0000","last_undegraded":"2024-01-15T18:23:36.462420+0000","last_fullsized":"2024-01-15T18:23:36.462420+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303360,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:11.655302+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:21:11.655302+0000","last_peered":"2024-01-15T18:21:11.655302+0000","last_clean":"2024-01-15T18:21:11.655302+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:21:11.655302+0000","last_undegraded":"2024-01-15T18:21:11.655302+0000","last_fullsized":"2024-01-15T18:21:11.655302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.469773+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:23:36.469773+0000","last_peered":"2024-01-15T18:23:36.469773+0000","last_clean":"2024-01-15T18:23:36.469773+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:23:36.469773+0000","last_undegraded":"2024-01-15T18:23:36.469773+0000","last_fullsized":"2024-01-15T18:23:36.469773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":15,"num_write_kb":53249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:07.082656+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:21:07.082656+0000","last_peered":"2024-01-15T18:21:07.082656+0000","last_clean":"2024-01-15T18:21:07.082656+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:21:07.082656+0000","last_undegraded":"2024-01-15T18:21:07.082656+0000","last_fullsized":"2024-01-15T18:21:07.082656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'41","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463503+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:23:36.463503+0000","last_peered":"2024-01-15T18:23:36.463503+0000","last_clean":"2024-01-15T18:23:36.463503+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:23:36.463503+0000","last_undegraded":"2024-01-15T18:23:36.463503+0000","last_fullsized":"2024-01-15T18:23:36.463503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:06.664989+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:21:06.664989+0000","last_peered":"2024-01-15T18:21:06.664989+0000","last_clean":"2024-01-15T18:21:06.664989+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:21:06.664989+0000","last_undegraded":"2024-01-15T18:21:06.664989+0000","last_fullsized":"2024-01-15T18:21:06.664989+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:01.739271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'35","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.457523+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:23:36.457523+0000","last_peered":"2024-01-15T18:23:36.457523+0000","last_clean":"2024-01-15T18:23:36.457523+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:23:36.457523+0000","last_undegraded":"2024-01-15T18:23:36.457523+0000","last_fullsized":"2024-01-15T18:23:36.457523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:07.287483+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:21:07.287483+0000","last_peered":"2024-01-15T18:21:07.287483+0000","last_clean":"2024-01-15T18:21:07.287483+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:21:07.287483+0000","last_undegraded":"2024-01-15T18:21:07.287483+0000","last_fullsized":"2024-01-15T18:21:07.287483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'41","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466318+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:23:36.466318+0000","last_peered":"2024-01-15T18:23:36.466318+0000","last_clean":"2024-01-15T18:23:36.466318+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:23:36.466318+0000","last_undegraded":"2024-01-15T18:23:36.466318+0000","last_fullsized":"2024-01-15T18:23:36.466318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'11","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:40.071203+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:21:40.071203+0000","last_peered":"2024-01-15T18:21:40.071203+0000","last_clean":"2024-01-15T18:21:40.071203+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:21:40.071203+0000","last_undegraded":"2024-01-15T18:21:40.071203+0000","last_fullsized":"2024-01-15T18:21:40.071203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'40","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.467240+0000","last_change":"2024-01-15T18:18:41.931596+0000","last_active":"2024-01-15T18:23:36.467240+0000","last_peered":"2024-01-15T18:23:36.467240+0000","last_clean":"2024-01-15T18:23:36.467240+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:23:36.467240+0000","last_undegraded":"2024-01-15T18:23:36.467240+0000","last_fullsized":"2024-01-15T18:23:36.467240+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'170","reported_seq":186,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.430521+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:23:36.430521+0000","last_peered":"2024-01-15T18:23:36.430521+0000","last_clean":"2024-01-15T18:23:36.430521+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:23:36.430521+0000","last_undegraded":"2024-01-15T18:23:36.430521+0000","last_fullsized":"2024-01-15T18:23:36.430521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569375,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'36","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470792+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:23:36.470792+0000","last_peered":"2024-01-15T18:23:36.470792+0000","last_clean":"2024-01-15T18:23:36.470792+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:23:36.470792+0000","last_undegraded":"2024-01-15T18:23:36.470792+0000","last_fullsized":"2024-01-15T18:23:36.470792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.664156+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:21:01.664156+0000","last_peered":"2024-01-15T18:21:01.664156+0000","last_clean":"2024-01-15T18:21:01.664156+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:21:01.664156+0000","last_undegraded":"2024-01-15T18:21:01.664156+0000","last_fullsized":"2024-01-15T18:21:01.664156+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'46","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.476804+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:23:36.476804+0000","last_peered":"2024-01-15T18:23:36.476804+0000","last_clean":"2024-01-15T18:23:36.476804+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:23:36.476804+0000","last_undegraded":"2024-01-15T18:23:36.476804+0000","last_fullsized":"2024-01-15T18:23:36.476804+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.653868+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:21:01.653868+0000","last_peered":"2024-01-15T18:21:01.653868+0000","last_clean":"2024-01-15T18:21:01.653868+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:21:01.653868+0000","last_undegraded":"2024-01-15T18:21:01.653868+0000","last_fullsized":"2024-01-15T18:21:01.653868+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'43","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480708+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:23:36.480708+0000","last_peered":"2024-01-15T18:23:36.480708+0000","last_clean":"2024-01-15T18:23:36.480708+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:23:36.480708+0000","last_undegraded":"2024-01-15T18:23:36.480708+0000","last_fullsized":"2024-01-15T18:23:36.480708+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:51:04.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080645,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":22,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":24,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.664196+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:21:01.664196+0000","last_peered":"2024-01-15T18:21:01.664196+0000","last_clean":"2024-01-15T18:21:01.664196+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:21:01.664196+0000","last_undegraded":"2024-01-15T18:21:01.664196+0000","last_fullsized":"2024-01-15T18:21:01.664196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:27:26.085701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'21","reported_seq":38,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479169+0000","last_change":"2024-01-15T18:18:41.936562+0000","last_active":"2024-01-15T18:23:36.479169+0000","last_peered":"2024-01-15T18:23:36.479169+0000","last_clean":"2024-01-15T18:23:36.479169+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:23:36.479169+0000","last_undegraded":"2024-01-15T18:23:36.479169+0000","last_fullsized":"2024-01-15T18:23:36.479169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:57.081108+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:20:57.081108+0000","last_peered":"2024-01-15T18:20:57.081108+0000","last_clean":"2024-01-15T18:20:57.081108+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:20:57.081108+0000","last_undegraded":"2024-01-15T18:20:57.081108+0000","last_fullsized":"2024-01-15T18:20:57.081108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'35","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.472587+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:23:36.472587+0000","last_peered":"2024-01-15T18:23:36.472587+0000","last_clean":"2024-01-15T18:23:36.472587+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:23:36.472587+0000","last_undegraded":"2024-01-15T18:23:36.472587+0000","last_fullsized":"2024-01-15T18:23:36.472587+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:48:34.695980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663623+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:20:56.663623+0000","last_peered":"2024-01-15T18:20:56.663623+0000","last_clean":"2024-01-15T18:20:56.663623+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:20:56.663623+0000","last_undegraded":"2024-01-15T18:20:56.663623+0000","last_fullsized":"2024-01-15T18:20:56.663623+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:38.154337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'41","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.468898+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:23:36.468898+0000","last_peered":"2024-01-15T18:23:36.468898+0000","last_clean":"2024-01-15T18:23:36.468898+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:23:36.468898+0000","last_undegraded":"2024-01-15T18:23:36.468898+0000","last_fullsized":"2024-01-15T18:23:36.468898+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":7,"num_read_kb":12292,"num_write":19,"num_write_kb":61442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653254+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:20:56.653254+0000","last_peered":"2024-01-15T18:20:56.653254+0000","last_clean":"2024-01-15T18:20:56.653254+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:20:56.653254+0000","last_undegraded":"2024-01-15T18:20:56.653254+0000","last_fullsized":"2024-01-15T18:20:56.653254+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:00:59.694593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'40","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463772+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:23:36.463772+0000","last_peered":"2024-01-15T18:23:36.463772+0000","last_clean":"2024-01-15T18:23:36.463772+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:23:36.463772+0000","last_undegraded":"2024-01-15T18:23:36.463772+0000","last_fullsized":"2024-01-15T18:23:36.463772+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:26.983917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.656234+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:20:56.656234+0000","last_peered":"2024-01-15T18:20:56.656234+0000","last_clean":"2024-01-15T18:20:56.656234+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:20:56.656234+0000","last_undegraded":"2024-01-15T18:20:56.656234+0000","last_fullsized":"2024-01-15T18:20:56.656234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:42.895411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'34","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480616+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:23:36.480616+0000","last_peered":"2024-01-15T18:23:36.480616+0000","last_clean":"2024-01-15T18:23:36.480616+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:23:36.480616+0000","last_undegraded":"2024-01-15T18:23:36.480616+0000","last_fullsized":"2024-01-15T18:23:36.480616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'3","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663515+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:20:56.663515+0000","last_peered":"2024-01-15T18:20:56.663515+0000","last_clean":"2024-01-15T18:20:56.663515+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:20:56.663515+0000","last_undegraded":"2024-01-15T18:20:56.663515+0000","last_fullsized":"2024-01-15T18:20:56.663515+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481081+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:23:36.481081+0000","last_peered":"2024-01-15T18:23:36.481081+0000","last_clean":"2024-01-15T18:23:36.481081+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:23:36.481081+0000","last_undegraded":"2024-01-15T18:23:36.481081+0000","last_fullsized":"2024-01-15T18:23:36.481081+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:14.044300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663534+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:20:56.663534+0000","last_peered":"2024-01-15T18:20:56.663534+0000","last_clean":"2024-01-15T18:20:56.663534+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:20:56.663534+0000","last_undegraded":"2024-01-15T18:20:56.663534+0000","last_fullsized":"2024-01-15T18:20:56.663534+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:51:31.110057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'40","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.467085+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:23:36.467085+0000","last_peered":"2024-01-15T18:23:36.467085+0000","last_clean":"2024-01-15T18:23:36.467085+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:23:36.467085+0000","last_undegraded":"2024-01-15T18:23:36.467085+0000","last_fullsized":"2024-01-15T18:23:36.467085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:51.662713+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:20:51.662713+0000","last_peered":"2024-01-15T18:20:51.662713+0000","last_clean":"2024-01-15T18:20:51.662713+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:20:51.662713+0000","last_undegraded":"2024-01-15T18:20:51.662713+0000","last_fullsized":"2024-01-15T18:20:51.662713+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'40","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.462441+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:23:36.462441+0000","last_peered":"2024-01-15T18:23:36.462441+0000","last_clean":"2024-01-15T18:23:36.462441+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:23:36.462441+0000","last_undegraded":"2024-01-15T18:23:36.462441+0000","last_fullsized":"2024-01-15T18:23:36.462441+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720517,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":15,"num_write_kb":57345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:52.080245+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:20:52.080245+0000","last_peered":"2024-01-15T18:20:52.080245+0000","last_clean":"2024-01-15T18:20:52.080245+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:20:52.080245+0000","last_undegraded":"2024-01-15T18:20:52.080245+0000","last_fullsized":"2024-01-15T18:20:52.080245+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'33","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.461910+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:23:36.461910+0000","last_peered":"2024-01-15T18:23:36.461910+0000","last_clean":"2024-01-15T18:23:36.461910+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:23:36.461910+0000","last_undegraded":"2024-01-15T18:23:36.461910+0000","last_fullsized":"2024-01-15T18:23:36.461910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:52.284956+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:20:52.284956+0000","last_peered":"2024-01-15T18:20:52.284956+0000","last_clean":"2024-01-15T18:20:52.284956+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:20:52.284956+0000","last_undegraded":"2024-01-15T18:20:52.284956+0000","last_fullsized":"2024-01-15T18:20:52.284956+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'37","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.490017+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:23:36.490017+0000","last_peered":"2024-01-15T18:23:36.490017+0000","last_clean":"2024-01-15T18:23:36.490017+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:23:36.490017+0000","last_undegraded":"2024-01-15T18:23:36.490017+0000","last_fullsized":"2024-01-15T18:23:36.490017+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:38:55.463538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'15","reported_seq":31,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:42.035206+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:22:42.035206+0000","last_peered":"2024-01-15T18:22:42.035206+0000","last_clean":"2024-01-15T18:22:42.035206+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:22:42.035206+0000","last_undegraded":"2024-01-15T18:22:42.035206+0000","last_fullsized":"2024-01-15T18:22:42.035206+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:51.313218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'42","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479751+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:23:36.479751+0000","last_peered":"2024-01-15T18:23:36.479751+0000","last_clean":"2024-01-15T18:23:36.479751+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:23:36.479751+0000","last_undegraded":"2024-01-15T18:23:36.479751+0000","last_fullsized":"2024-01-15T18:23:36.479751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:47.175404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:47.079647+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:20:47.079647+0000","last_peered":"2024-01-15T18:20:47.079647+0000","last_clean":"2024-01-15T18:20:47.079647+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:20:47.079647+0000","last_undegraded":"2024-01-15T18:20:47.079647+0000","last_fullsized":"2024-01-15T18:20:47.079647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'34","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.460078+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:23:36.460078+0000","last_peered":"2024-01-15T18:23:36.460078+0000","last_clean":"2024-01-15T18:23:36.460078+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:23:36.460078+0000","last_undegraded":"2024-01-15T18:23:36.460078+0000","last_fullsized":"2024-01-15T18:23:36.460078+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:27.713450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331770,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":5,"num_read_kb":20480,"num_write":14,"num_write_kb":49153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:47.284362+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:20:47.284362+0000","last_peered":"2024-01-15T18:20:47.284362+0000","last_clean":"2024-01-15T18:20:47.284362+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:20:47.284362+0000","last_undegraded":"2024-01-15T18:20:47.284362+0000","last_fullsized":"2024-01-15T18:20:47.284362+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'44","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480400+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:23:36.480400+0000","last_peered":"2024-01-15T18:23:36.480400+0000","last_clean":"2024-01-15T18:23:36.480400+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:23:36.480400+0000","last_undegraded":"2024-01-15T18:23:36.480400+0000","last_fullsized":"2024-01-15T18:23:36.480400+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655069+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:20:46.655069+0000","last_peered":"2024-01-15T18:20:46.655069+0000","last_clean":"2024-01-15T18:20:46.655069+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:20:46.655069+0000","last_undegraded":"2024-01-15T18:20:46.655069+0000","last_fullsized":"2024-01-15T18:20:46.655069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:54.586280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'51","reported_seq":72,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480979+0000","last_change":"2024-01-15T18:18:41.935993+0000","last_active":"2024-01-15T18:23:36.480979+0000","last_peered":"2024-01-15T18:23:36.480979+0000","last_clean":"2024-01-15T18:23:36.480979+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:23:36.480979+0000","last_undegraded":"2024-01-15T18:23:36.480979+0000","last_fullsized":"2024-01-15T18:23:36.480979+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652145+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:20:46.652145+0000","last_peered":"2024-01-15T18:20:46.652145+0000","last_clean":"2024-01-15T18:20:46.652145+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:20:46.652145+0000","last_undegraded":"2024-01-15T18:20:46.652145+0000","last_fullsized":"2024-01-15T18:20:46.652145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:26.705662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'38","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466662+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:23:36.466662+0000","last_peered":"2024-01-15T18:23:36.466662+0000","last_clean":"2024-01-15T18:23:36.466662+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:23:36.466662+0000","last_undegraded":"2024-01-15T18:23:36.466662+0000","last_fullsized":"2024-01-15T18:23:36.466662+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'10","reported_seq":35,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:40.067251+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:21:40.067251+0000","last_peered":"2024-01-15T18:21:40.067251+0000","last_clean":"2024-01-15T18:21:40.067251+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:21:40.067251+0000","last_undegraded":"2024-01-15T18:21:40.067251+0000","last_fullsized":"2024-01-15T18:21:40.067251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:03:44.404981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'36","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475826+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:23:36.475826+0000","last_peered":"2024-01-15T18:23:36.475826+0000","last_clean":"2024-01-15T18:23:36.475826+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:36.475826+0000","last_undegraded":"2024-01-15T18:23:36.475826+0000","last_fullsized":"2024-01-15T18:23:36.475826+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:52.187345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:42.078947+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:20:42.078947+0000","last_peered":"2024-01-15T18:20:42.078947+0000","last_clean":"2024-01-15T18:20:42.078947+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:20:42.078947+0000","last_undegraded":"2024-01-15T18:20:42.078947+0000","last_fullsized":"2024-01-15T18:20:42.078947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'30","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475835+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:23:36.475835+0000","last_peered":"2024-01-15T18:23:36.475835+0000","last_clean":"2024-01-15T18:23:36.475835+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:23:36.475835+0000","last_undegraded":"2024-01-15T18:23:36.475835+0000","last_fullsized":"2024-01-15T18:23:36.475835+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.656696+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:21:01.656696+0000","last_peered":"2024-01-15T18:21:01.656696+0000","last_clean":"2024-01-15T18:21:01.656696+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:21:01.656696+0000","last_undegraded":"2024-01-15T18:21:01.656696+0000","last_fullsized":"2024-01-15T18:21:01.656696+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:35.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":246,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653639+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:20:56.653639+0000","last_peered":"2024-01-15T18:20:56.653639+0000","last_clean":"2024-01-15T18:20:56.653639+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:20:56.653639+0000","last_undegraded":"2024-01-15T18:20:56.653639+0000","last_fullsized":"2024-01-15T18:20:56.653639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":4294968312,"num_objects":1030,"num_object_clones":1,"num_object_copies":2060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1030,"num_whiteouts":1,"num_read":269,"num_read_kb":1048589,"num_write":1050,"num_write_kb":4194319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8589975552,"data_stored":8589936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2554,"ondisk_log_size":2554,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":571494,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1142988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461744,"kb_used_data":1398696,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92910096,"statfs":{"total":96636764160,"available":95139938304,"internally_reserved":0,"allocated":1432264704,"data_stored":1431460098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69099999999999995,"15min":0.94399999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.29999999999999999},"max":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.627},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.87},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":13.744},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":1.0840000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":16.920000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":1.05},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":1.04,"15min":31.774999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72599999999999998,"15min":1},"min":{"1min":0.57399999999999995,"5min":0.48699999999999999,"15min":0.377},"max":{"1min":1.056,"5min":1.085,"15min":30.846},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":1.0629999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":1.0229999999999999,"15min":39.695},"last":0.69499999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74099999999999999,"15min":1.1499999999999999},"min":{"1min":0.54500000000000004,"5min":0.437,"15min":0.38800000000000001},"max":{"1min":1.01,"5min":1.0109999999999999,"15min":26.463999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":1.198},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.41499999999999998},"max":{"1min":0.86699999999999999,"5min":1.0640000000000001,"15min":42.692},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86699999999999999,"5min":0.874,"15min":14.087},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.36299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":2.952},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":11.334},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.746},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91700000000000004,"5min":0.94499999999999995,"15min":8.423},"last":0.52300000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.78200000000000003},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.45500000000000002},"max":{"1min":0.95099999999999996,"5min":1.034,"15min":8.5030000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.35299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":3.3450000000000002},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":998316,"kb_used_data":935268,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93373524,"statfs":{"total":96636764160,"available":95614488576,"internally_reserved":0,"allocated":957714432,"data_stored":956912954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":22.754000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.82399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":1.018,"15min":9.1470000000000002},"last":0.65300000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.35299999999999998,"15min":0.34300000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":14.246},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":0.98199999999999998},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":0.97899999999999998,"15min":19.475000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":1.163},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.0549999999999999,"15min":42.825000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.747,"15min":1.107},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":1.103,"15min":42.851999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.98599999999999999},"min":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.8160000000000001,"15min":24.199000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":1.0700000000000001},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":1.03,"15min":18.791},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":1.044,"15min":19.114999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":8.3059999999999992},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.88800000000000001,"5min":1.0920000000000001,"15min":7.5880000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":0.93500000000000005,"5min":1.1120000000000001,"15min":12.945},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69099999999999995,"15min":0.80100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.38600000000000001},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":11.907},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.85399999999999998},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.86599999999999999,"5min":1.1910000000000001,"15min":27.651},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793712,"kb_used_data":718184,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93578128,"statfs":{"total":96636764160,"available":95824003072,"internally_reserved":0,"allocated":735420416,"data_stored":734615446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68400000000000005,"15min":0.85199999999999998},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.29899999999999999},"max":{"1min":0.98599999999999999,"5min":1.9350000000000001,"15min":11.519},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74099999999999999,"15min":0.94599999999999995},"min":{"1min":0.626,"5min":0.371,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":2.6819999999999999,"15min":12.952},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":2.1179999999999999,"15min":13.358000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.96699999999999997},"min":{"1min":0.56999999999999995,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":1.0289999999999999,"5min":2.8050000000000002,"15min":21.856999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":3.508,"15min":18.314},"last":0.62},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":1.0449999999999999},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":1.069,"5min":3.5310000000000001,"15min":21.120000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":6.7990000000000004,"15min":13.317},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":1.0369999999999999},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":2.3380000000000001,"15min":19.638000000000002},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.96399999999999997,"15min":11.093},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.95699999999999996,"15min":3.3889999999999998},"last":0.38900000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.78900000000000003},"min":{"1min":0.5,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":5.8680000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32600000000000001},"max":{"1min":0.91300000000000003,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.316}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":1.081,"15min":3.2759999999999998},"last":0.432},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.754,"15min":0.80300000000000005},"min":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.88300000000000001,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1387432,"kb_used_data":1324384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92984408,"statfs":{"total":96636764160,"available":95216033792,"internally_reserved":0,"allocated":1356169216,"data_stored":1355371903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.94899999999999995},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":1.02,"15min":34.802},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.91300000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":0.871,"5min":1.1319999999999999,"15min":22.335999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":0.90600000000000003,"5min":1.224,"15min":38.198},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.752,"15min":0.96799999999999997},"min":{"1min":0.52400000000000002,"5min":0.438,"15min":0.372},"max":{"1min":0.875,"5min":1.0269999999999999,"15min":38.220999999999997},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.89600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.34599999999999997},"max":{"1min":0.91600000000000004,"5min":1.0509999999999999,"15min":7.6989999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76000000000000001,"15min":0.85999999999999999},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":1.0740000000000001,"15min":7.4829999999999997},"last":0.77700000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.95199999999999996},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":1.214,"15min":14.529},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.94699999999999995},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":25.774000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.68899999999999995},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.33200000000000002},"max":{"1min":0.89800000000000002,"5min":0.90300000000000002,"15min":9.9619999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.79700000000000004},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":1.1719999999999999,"15min":25.887},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":10.284000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.97299999999999998,"15min":3.6949999999999998},"last":0.73099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":13.760999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.36199999999999999},"max":{"1min":0.96699999999999997,"5min":1.0269999999999999,"15min":11.67},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1154596,"kb_used_data":1087388,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93217244,"statfs":{"total":96636764160,"available":95454457856,"internally_reserved":0,"allocated":1113485312,"data_stored":1112671977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.66200000000000003,"5min":2.415,"15min":26.364000000000001},"last":0.44},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.95499999999999996},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.79500000000000004,"15min":28.332000000000001},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.98399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63300000000000001,"5min":0.82499999999999996,"15min":40.457999999999998},"last":0.315},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":1.1120000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":27.427},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.98799999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":23.036999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":1.081},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.873,"15min":26.404},"last":0.504}]},{"osd":4,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":1.0349999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.0529999999999999,"5min":1.1439999999999999,"15min":22.998000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76300000000000001,"15min":1.0740000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1,"5min":1.1439999999999999,"15min":20.399000000000001},"last":0.746}]},{"osd":5,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":1.0820000000000001},"min":{"1min":0.52800000000000002,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":0.97999999999999998,"5min":1.46,"15min":26.314},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":1.1040000000000001},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1,"5min":1.109,"15min":22.908999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79000000000000004,"15min":1.155},"min":{"1min":0.57399999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.105,"5min":1.105,"15min":20.445},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81399999999999995,"15min":1.1659999999999999},"min":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":1.4299999999999999,"15min":26.091999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":1.159},"min":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":23.111999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":1.3169999999999999},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.46400000000000002},"max":{"1min":1.0880000000000001,"5min":1.145,"15min":44.817999999999998},"last":0.85099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768936,"kb_used_data":705888,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93602904,"statfs":{"total":96636764160,"available":95849373696,"internally_reserved":0,"allocated":722829312,"data_stored":722033059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":1.054},"min":{"1min":0.28599999999999998,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.82899999999999996,"5min":4.2329999999999997,"15min":29.527999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":1.335},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":6.0670000000000002,"15min":34.122999999999998},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":1.534},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":4.5259999999999998,"15min":113.149},"last":0.498},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59099999999999997,"15min":1.127},"min":{"1min":0.40500000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84899999999999998,"5min":6.1020000000000003,"15min":41.302},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":1.3899999999999999},"min":{"1min":0.34699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76800000000000002,"5min":4.7370000000000001,"15min":46.960000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.625,"15min":1.2809999999999999},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":3.8809999999999998,"15min":115.693},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83199999999999996,"15min":1.194},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":1.1100000000000001,"5min":3.8399999999999999,"15min":24.366},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":1.228},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.432},"max":{"1min":1.0189999999999999,"5min":2.343,"15min":20.489000000000001},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":1.5289999999999999},"min":{"1min":0.61899999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":1.175,"5min":3.927,"15min":41.683999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80400000000000005,"15min":1.2889999999999999},"min":{"1min":0.60799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98599999999999999,"5min":2.3029999999999999,"15min":26.084},"last":0.70899999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":1.319},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.159,"5min":6.0220000000000002,"15min":22.199000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":1.569},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.0860000000000001,"5min":3.9590000000000001,"15min":41.738999999999997},"last":0.621}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85599999999999998,"15min":1.262},"min":{"1min":0.621,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.1339999999999999,"5min":2.556,"15min":20.446999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":1.6060000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1599999999999999,"5min":2.5939999999999999,"15min":41.402000000000001},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":941048,"kb_used_data":873840,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93430792,"statfs":{"total":96636764160,"available":95673131008,"internally_reserved":0,"allocated":894812160,"data_stored":894000019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.439,"15min":0.86099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.63700000000000001,"5min":1.4179999999999999,"15min":39.948},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.56599999999999995,"15min":1.0649999999999999},"min":{"1min":0.30399999999999999,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":4.9329999999999998,"15min":35.790999999999997},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.57699999999999996,"15min":0.97099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.27600000000000002},"max":{"1min":0.66100000000000003,"5min":9.2609999999999992,"15min":49.229999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":1.7649999999999999,"15min":10.349},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":1.0589999999999999},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68300000000000005,"5min":0.84999999999999998,"15min":27.643999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.90800000000000003},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":1.198,"15min":15.608000000000001},"last":0.498}]},{"osd":4,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76500000000000001,"15min":1.165},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":1.1080000000000001,"5min":1.383,"15min":27.489999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":1.0349999999999999},"min":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.218,"5min":6.6950000000000003,"15min":10.038},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76900000000000002,"15min":1.1779999999999999},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1739999999999999,"5min":1.2010000000000001,"15min":27.547000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.55400000000000005},"max":{"1min":1.1599999999999999,"5min":1.266,"15min":9.9610000000000003},"last":0.90000000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80600000000000005,"15min":1.171},"min":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.496},"max":{"1min":1.147,"5min":1.984,"15min":27.579999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":1},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.42399999999999999},"max":{"1min":1.1910000000000001,"5min":5.6470000000000002,"15min":11.712999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":1.0980000000000001},"min":{"1min":0.56899999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":5.819,"15min":11.833},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.89000000000000001,"15min":1.0920000000000001},"min":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.502},"max":{"1min":1.254,"5min":6.3570000000000002,"15min":14.606},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575077,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1421360,"kb_used_data":1358312,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92950480,"statfs":{"total":96636764160,"available":95181291520,"internally_reserved":0,"allocated":1390911488,"data_stored":1390086416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46300000000000002,"15min":1.054},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":31.977},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.51300000000000001,"15min":1.0349999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.61499999999999999,"5min":1.6579999999999999,"15min":36.145000000000003},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":1.1439999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":1.1759999999999999,"15min":41.387999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":1.0089999999999999},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70399999999999996,"5min":1.7689999999999999,"15min":33.219999999999999},"last":0.503}]},{"osd":3,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":1.006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80500000000000005,"15min":31.943999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":1.107},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.746,"5min":0.95599999999999996,"15min":45.128},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.38300000000000001},"max":{"1min":1.0469999999999999,"5min":1.4550000000000001,"15min":32.932000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.77400000000000002,"15min":1.1659999999999999},"min":{"1min":0.53600000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":1.115,"15min":31.782},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":1.149},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.38600000000000001},"max":{"1min":1.0249999999999999,"5min":1.179,"15min":31.858000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.79300000000000004,"15min":1.0409999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.7869999999999999,"15min":33.151000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":1.0660000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":13.055},"last":0.621},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.47699999999999998},"max":{"1min":1.262,"5min":1.7170000000000001,"15min":33.098999999999997},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":1.097},"min":{"1min":0.68200000000000005,"5min":0.55500000000000005,"15min":0.48899999999999999},"max":{"1min":1.0029999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":1.2050000000000001},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":0.997,"5min":1.075,"15min":31.905999999999999},"last":0.59799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":569487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":569445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1388326912,"data_stored":1388315187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":893394944,"data_stored":893387124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":721420288,"data_stored":721420288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1111498752,"data_stored":1111491082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1354764288,"data_stored":1354760453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":734007296,"data_stored":734003392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":956305408,"data_stored":956301434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1430257664,"data_stored":1430257664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:23:42.013 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-15T18:23:42.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:23:42.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 Processor -- start 2024-01-15T18:23:42.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 -- start start 2024-01-15T18:23:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb4143780 0x7fabb40cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fabb40ce520 0x7fabb4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb40ceb50 con 0x7fabb40720f0 2024-01-15T18:23:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb4072580 con 0x7fabb40ce520 2024-01-15T18:23:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabbb600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415c110 con 0x7fabb4143780 2024-01-15T18:23:42.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb9600640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fabb40720f0 0x7fabb415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60704/0 (socket says 172.21.15.143:60704) 2024-01-15T18:23:42.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb9600640 1 -- 172.21.15.143:0/818999014 learned_addr learned my addr 172.21.15.143:0/818999014 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:42.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb3e00640 1 --2- 172.21.15.143:0/818999014 >> v2:172.21.15.143:3300/0 conn(0x7fabb40ce520 0x7fabb4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb8c00640 1 --2- 172.21.15.143:0/818999014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb4143780 0x7fabb40cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb3e00640 1 -- 172.21.15.143:0/818999014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb4143780 msgr2=0x7fabb40cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:42.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb3e00640 1 --2- 172.21.15.143:0/818999014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb4143780 0x7fabb40cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb3e00640 1 -- 172.21.15.143:0/818999014 >> v1:172.21.15.143:6789/0 conn(0x7fabb40720f0 legacy=0x7fabb415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb3e00640 1 -- 172.21.15.143:0/818999014 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb415b7e0 con 0x7fabb40ce520 2024-01-15T18:23:42.192 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.199+0000 7fabb3400640 1 -- 172.21.15.143:0/818999014 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3389505623 0 0) 0x7fabac0014a0 con 0x7fabb40720f0 2024-01-15T18:23:42.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabb3e00640 1 --2- 172.21.15.143:0/818999014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb40ce520 0x7fabb4071bb0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0097c0 tx=0x7fab9c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5931437b7dc71649 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:42.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabb3400640 1 -- 172.21.15.143:0/818999014 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0ad640 con 0x7fabb40ce520 2024-01-15T18:23:42.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabb3400640 1 -- 172.21.15.143:0/818999014 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c0ad7a0 con 0x7fabb40ce520 2024-01-15T18:23:42.194 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabb3400640 1 -- 172.21.15.143:0/818999014 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0adb30 con 0x7fabb40ce520 2024-01-15T18:23:42.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 -- 172.21.15.143:0/818999014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb40ce520 msgr2=0x7fabb4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 --2- 172.21.15.143:0/818999014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb40ce520 0x7fabb4071bb0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0097c0 tx=0x7fab9c01c480 comp rx=0 tx=0).stop 2024-01-15T18:23:42.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 -- 172.21.15.143:0/818999014 shutdown_connections 2024-01-15T18:23:42.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 --2- 172.21.15.143:0/818999014 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb40ce520 0x7fabb4071bb0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 --2- 172.21.15.143:0/818999014 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb4143780 0x7fabb40cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.195 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 -- 172.21.15.143:0/818999014 >> 172.21.15.143:0/818999014 conn(0x7fabb406c3f0 msgr2=0x7fabb406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:42.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 -- 172.21.15.143:0/818999014 shutdown_connections 2024-01-15T18:23:42.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.203+0000 7fabbb600640 1 -- 172.21.15.143:0/818999014 wait complete. 2024-01-15T18:23:42.196 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 Processor -- start 2024-01-15T18:23:42.197 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 -- start start 2024-01-15T18:23:42.198 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb40720f0 0x7fabb4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb40ce520 0x7fabb4162070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 0x7fabb413e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415f030 con 0x7fabb4143780 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415e3b0 con 0x7fabb40720f0 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb415e690 con 0x7fabb40ce520 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 0x7fabb413e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 0x7fabb413e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38072/0 (socket says 172.21.15.143:38072) 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 -- 172.21.15.143:0/2828778423 learned_addr learned my addr 172.21.15.143:0/2828778423 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:42.199 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb8c00640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb40720f0 0x7fabb4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb3e00640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb40ce520 0x7fabb4162070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 -- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb40ce520 msgr2=0x7fabb4162070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb40ce520 0x7fabb4162070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 -- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb40720f0 msgr2=0x7fabb4159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb40720f0 0x7fabb4159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 -- 172.21.15.143:0/2828778423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009420 con 0x7fabb4143780 2024-01-15T18:23:42.200 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb9600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 0x7fabb413e9f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c730 tx=0x7fabac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00ce40 con 0x7fabb4143780 2024-01-15T18:23:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac00a640 con 0x7fabb4143780 2024-01-15T18:23:42.201 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac00a9f0 con 0x7fabb4143780 2024-01-15T18:23:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb415e970 con 0x7fabb4143780 2024-01-15T18:23:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.207+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb4159940 con 0x7fabb4143780 2024-01-15T18:23:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.211+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb415f170 con 0x7fabb4143780 2024-01-15T18:23:42.202 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.211+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fabac00ab50 con 0x7fabb4143780 2024-01-15T18:23:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.211+0000 7fabb0c00640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab8c067b80 0x7fab8c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.211+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fabac0827e0 con 0x7fabb4143780 2024-01-15T18:23:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.211+0000 7fabb8c00640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab8c067b80 0x7fab8c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.203 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.211+0000 7fabb8c00640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab8c067b80 0x7fab8c06a040 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faba40079c0 tx=0x7faba4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:42.206 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.215+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fabb415f170 con 0x7fabb4143780 2024-01-15T18:23:42.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.367+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabb40ceb50 con 0x7fab8c067b80 2024-01-15T18:23:42.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.371+0000 7fabb0c00640 1 -- 172.21.15.143:0/2828778423 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291090 (secure 0 0 0) 0x7fabb40ceb50 con 0x7fab8c067b80 2024-01-15T18:23:42.367 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:42.368 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:23:42.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.375+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab8c067b80 msgr2=0x7fab8c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.375+0000 7fabbb600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab8c067b80 0x7fab8c06a040 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7faba40079c0 tx=0x7faba4008040 comp rx=0 tx=0).stop 2024-01-15T18:23:42.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.375+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 msgr2=0x7fabb413e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.375+0000 7fabbb600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 0x7fabb413e9f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fabac00c730 tx=0x7fabac00cbf0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 shutdown_connections 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fab8c067b80 0x7fab8c06a040 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabb4143780 0x7fabb413e9f0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fabb40ce520 0x7fabb4162070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 --2- 172.21.15.143:0/2828778423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fabb40720f0 0x7fabb4159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 >> 172.21.15.143:0/2828778423 conn(0x7fabb406c3f0 msgr2=0x7fabb40aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 shutdown_connections 2024-01-15T18:23:42.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.379+0000 7fabbb600640 1 -- 172.21.15.143:0/2828778423 wait complete. 2024-01-15T18:23:42.388 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":673,"stamp":"2024-01-15T18:23:40.974303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4296130174,"num_objects":1055,"num_object_clones":1,"num_object_copies":2110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1055,"num_whiteouts":1,"num_read":394,"num_read_kb":1048698,"num_write":1465,"num_write_kb":4197407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2909,"ondisk_log_size":2909,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8927144,"kb_used_data":8401960,"kb_used_omap":12,"kb_used_meta":525107,"kb_avail":746047576,"statfs":{"total":773094113280,"available":763952717824,"internally_reserved":0,"allocated":8603607040,"data_stored":8597151872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":537710128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":68,"apply_latency_ms":68,"commit_latency_ns":68000000,"apply_latency_ns":68000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002584"},"pg_stats":[{"pgid":"17.3f","version":"113'49","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480416+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:23:36.480416+0000","last_peered":"2024-01-15T18:23:36.480416+0000","last_clean":"2024-01-15T18:23:36.480416+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:23:36.480416+0000","last_undegraded":"2024-01-15T18:23:36.480416+0000","last_fullsized":"2024-01-15T18:23:36.480416+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:00.418536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'42","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480084+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:23:36.480084+0000","last_peered":"2024-01-15T18:23:36.480084+0000","last_clean":"2024-01-15T18:23:36.480084+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:23:36.480084+0000","last_undegraded":"2024-01-15T18:23:36.480084+0000","last_fullsized":"2024-01-15T18:23:36.480084+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'39","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481097+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:23:36.481097+0000","last_peered":"2024-01-15T18:23:36.481097+0000","last_clean":"2024-01-15T18:23:36.481097+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:23:36.481097+0000","last_undegraded":"2024-01-15T18:23:36.481097+0000","last_fullsized":"2024-01-15T18:23:36.481097+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:07.807378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":16,"num_write_kb":57345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.490145+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:23:36.490145+0000","last_peered":"2024-01-15T18:23:36.490145+0000","last_clean":"2024-01-15T18:23:36.490145+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:36.490145+0000","last_undegraded":"2024-01-15T18:23:36.490145+0000","last_fullsized":"2024-01-15T18:23:36.490145+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:44.886136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'25","reported_seq":42,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.450344+0000","last_change":"2024-01-15T18:18:41.951758+0000","last_active":"2024-01-15T18:23:36.450344+0000","last_peered":"2024-01-15T18:23:36.450344+0000","last_clean":"2024-01-15T18:23:36.450344+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:23:36.450344+0000","last_undegraded":"2024-01-15T18:23:36.450344+0000","last_fullsized":"2024-01-15T18:23:36.450344+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:51.159719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'45","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463710+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:23:36.463710+0000","last_peered":"2024-01-15T18:23:36.463710+0000","last_clean":"2024-01-15T18:23:36.463710+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:23:36.463710+0000","last_undegraded":"2024-01-15T18:23:36.463710+0000","last_fullsized":"2024-01-15T18:23:36.463710+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:25:48.760709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'41","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470502+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:23:36.470502+0000","last_peered":"2024-01-15T18:23:36.470502+0000","last_clean":"2024-01-15T18:23:36.470502+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:23:36.470502+0000","last_undegraded":"2024-01-15T18:23:36.470502+0000","last_fullsized":"2024-01-15T18:23:36.470502+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'42","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466790+0000","last_change":"2024-01-15T18:18:41.932178+0000","last_active":"2024-01-15T18:23:36.466790+0000","last_peered":"2024-01-15T18:23:36.466790+0000","last_clean":"2024-01-15T18:23:36.466790+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:23:36.466790+0000","last_undegraded":"2024-01-15T18:23:36.466790+0000","last_fullsized":"2024-01-15T18:23:36.466790+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:32.753279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'42","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475770+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:23:36.475770+0000","last_peered":"2024-01-15T18:23:36.475770+0000","last_clean":"2024-01-15T18:23:36.475770+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:23:36.475770+0000","last_undegraded":"2024-01-15T18:23:36.475770+0000","last_fullsized":"2024-01-15T18:23:36.475770+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:16.576477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'41","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.478544+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:23:36.478544+0000","last_peered":"2024-01-15T18:23:36.478544+0000","last_clean":"2024-01-15T18:23:36.478544+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:23:36.478544+0000","last_undegraded":"2024-01-15T18:23:36.478544+0000","last_fullsized":"2024-01-15T18:23:36.478544+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:16.172780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":11,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'41","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.474632+0000","last_change":"2024-01-15T18:18:41.934119+0000","last_active":"2024-01-15T18:23:36.474632+0000","last_peered":"2024-01-15T18:23:36.474632+0000","last_clean":"2024-01-15T18:23:36.474632+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:23:36.474632+0000","last_undegraded":"2024-01-15T18:23:36.474632+0000","last_fullsized":"2024-01-15T18:23:36.474632+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'43","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477840+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:23:36.477840+0000","last_peered":"2024-01-15T18:23:36.477840+0000","last_clean":"2024-01-15T18:23:36.477840+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:23:36.477840+0000","last_undegraded":"2024-01-15T18:23:36.477840+0000","last_fullsized":"2024-01-15T18:23:36.477840+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:16.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":20481,"num_write":25,"num_write_kb":94209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'34","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.453165+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:23:36.453165+0000","last_peered":"2024-01-15T18:23:36.453165+0000","last_clean":"2024-01-15T18:23:36.453165+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:23:36.453165+0000","last_undegraded":"2024-01-15T18:23:36.453165+0000","last_fullsized":"2024-01-15T18:23:36.453165+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'43","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.459261+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:23:36.459261+0000","last_peered":"2024-01-15T18:23:36.459261+0000","last_clean":"2024-01-15T18:23:36.459261+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:23:36.459261+0000","last_undegraded":"2024-01-15T18:23:36.459261+0000","last_fullsized":"2024-01-15T18:23:36.459261+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:06.614151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'29","reported_seq":47,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.469423+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:23:36.469423+0000","last_peered":"2024-01-15T18:23:36.469423+0000","last_clean":"2024-01-15T18:23:36.469423+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:23:36.469423+0000","last_undegraded":"2024-01-15T18:23:36.469423+0000","last_fullsized":"2024-01-15T18:23:36.469423+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:34.739612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'40","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481202+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:23:36.481202+0000","last_peered":"2024-01-15T18:23:36.481202+0000","last_clean":"2024-01-15T18:23:36.481202+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:23:36.481202+0000","last_undegraded":"2024-01-15T18:23:36.481202+0000","last_fullsized":"2024-01-15T18:23:36.481202+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:47:03.406736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.473510+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:23:36.473510+0000","last_peered":"2024-01-15T18:23:36.473510+0000","last_clean":"2024-01-15T18:23:36.473510+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:23:36.473510+0000","last_undegraded":"2024-01-15T18:23:36.473510+0000","last_fullsized":"2024-01-15T18:23:36.473510+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470128+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:23:36.470128+0000","last_peered":"2024-01-15T18:23:36.470128+0000","last_clean":"2024-01-15T18:23:36.470128+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:23:36.470128+0000","last_undegraded":"2024-01-15T18:23:36.470128+0000","last_fullsized":"2024-01-15T18:23:36.470128+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:45.110826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455631+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:23:36.455631+0000","last_peered":"2024-01-15T18:23:36.455631+0000","last_clean":"2024-01-15T18:23:36.455631+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:23:36.455631+0000","last_undegraded":"2024-01-15T18:23:36.455631+0000","last_fullsized":"2024-01-15T18:23:36.455631+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'46","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481283+0000","last_change":"2024-01-15T18:18:41.927062+0000","last_active":"2024-01-15T18:23:36.481283+0000","last_peered":"2024-01-15T18:23:36.481283+0000","last_clean":"2024-01-15T18:23:36.481283+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:36.481283+0000","last_undegraded":"2024-01-15T18:23:36.481283+0000","last_fullsized":"2024-01-15T18:23:36.481283+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'50","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477877+0000","last_change":"2024-01-15T18:18:41.937906+0000","last_active":"2024-01-15T18:23:36.477877+0000","last_peered":"2024-01-15T18:23:36.477877+0000","last_clean":"2024-01-15T18:23:36.477877+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:23:36.477877+0000","last_undegraded":"2024-01-15T18:23:36.477877+0000","last_fullsized":"2024-01-15T18:23:36.477877+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:20.422368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":11,"num_read_kb":36866,"num_write":22,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'41","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.464458+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:23:36.464458+0000","last_peered":"2024-01-15T18:23:36.464458+0000","last_clean":"2024-01-15T18:23:36.464458+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:23:36.464458+0000","last_undegraded":"2024-01-15T18:23:36.464458+0000","last_fullsized":"2024-01-15T18:23:36.464458+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":4098,"num_write":17,"num_write_kb":61441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'35","reported_seq":52,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479224+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:23:36.479224+0000","last_peered":"2024-01-15T18:23:36.479224+0000","last_clean":"2024-01-15T18:23:36.479224+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:23:36.479224+0000","last_undegraded":"2024-01-15T18:23:36.479224+0000","last_fullsized":"2024-01-15T18:23:36.479224+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46137344,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":11,"num_write_kb":45056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'39","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477262+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:23:36.477262+0000","last_peered":"2024-01-15T18:23:36.477262+0000","last_clean":"2024-01-15T18:23:36.477262+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:23:36.477262+0000","last_undegraded":"2024-01-15T18:23:36.477262+0000","last_fullsized":"2024-01-15T18:23:36.477262+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'35","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.477951+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:23:36.477951+0000","last_peered":"2024-01-15T18:23:36.477951+0000","last_clean":"2024-01-15T18:23:36.477951+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:23:36.477951+0000","last_undegraded":"2024-01-15T18:23:36.477951+0000","last_fullsized":"2024-01-15T18:23:36.477951+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:12.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":16386,"num_write":17,"num_write_kb":61441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'50","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.482075+0000","last_change":"2024-01-15T18:19:42.172768+0000","last_active":"2024-01-15T18:23:36.482075+0000","last_peered":"2024-01-15T18:23:36.482075+0000","last_clean":"2024-01-15T18:23:36.482075+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:23:36.482075+0000","last_undegraded":"2024-01-15T18:23:36.482075+0000","last_fullsized":"2024-01-15T18:23:36.482075+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:36.268040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'37","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455733+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:23:36.455733+0000","last_peered":"2024-01-15T18:23:36.455733+0000","last_clean":"2024-01-15T18:23:36.455733+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:23:36.455733+0000","last_undegraded":"2024-01-15T18:23:36.455733+0000","last_fullsized":"2024-01-15T18:23:36.455733+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:15:18.174579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4,"num_read_kb":12289,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'31","reported_seq":48,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.458498+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:23:36.458498+0000","last_peered":"2024-01-15T18:23:36.458498+0000","last_clean":"2024-01-15T18:23:36.458498+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:23:36.458498+0000","last_undegraded":"2024-01-15T18:23:36.458498+0000","last_fullsized":"2024-01-15T18:23:36.458498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'38","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480157+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:23:36.480157+0000","last_peered":"2024-01-15T18:23:36.480157+0000","last_clean":"2024-01-15T18:23:36.480157+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:23:36.480157+0000","last_undegraded":"2024-01-15T18:23:36.480157+0000","last_fullsized":"2024-01-15T18:23:36.480157+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'38","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.458139+0000","last_change":"2024-01-15T18:18:41.937559+0000","last_active":"2024-01-15T18:23:36.458139+0000","last_peered":"2024-01-15T18:23:36.458139+0000","last_clean":"2024-01-15T18:23:36.458139+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:23:36.458139+0000","last_undegraded":"2024-01-15T18:23:36.458139+0000","last_fullsized":"2024-01-15T18:23:36.458139+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5,"num_read_kb":16385,"num_write":15,"num_write_kb":53249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'59","reported_seq":81,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.482554+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:23:36.482554+0000","last_peered":"2024-01-15T18:23:36.482554+0000","last_clean":"2024-01-15T18:23:36.482554+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:23:36.482554+0000","last_undegraded":"2024-01-15T18:23:36.482554+0000","last_fullsized":"2024-01-15T18:23:36.482554+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:18:31.192027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'50","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466535+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:23:36.466535+0000","last_peered":"2024-01-15T18:23:36.466535+0000","last_clean":"2024-01-15T18:23:36.466535+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:23:36.466535+0000","last_undegraded":"2024-01-15T18:23:36.466535+0000","last_fullsized":"2024-01-15T18:23:36.466535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:56:19.546961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'28","reported_seq":47,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466086+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:23:36.466086+0000","last_peered":"2024-01-15T18:23:36.466086+0000","last_clean":"2024-01-15T18:23:36.466086+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:23:36.466086+0000","last_undegraded":"2024-01-15T18:23:36.466086+0000","last_fullsized":"2024-01-15T18:23:36.466086+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:47.919946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37748736,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":9,"num_write_kb":36864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655110+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:20:46.655110+0000","last_peered":"2024-01-15T18:20:46.655110+0000","last_clean":"2024-01-15T18:20:46.655110+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:20:46.655110+0000","last_undegraded":"2024-01-15T18:20:46.655110+0000","last_fullsized":"2024-01-15T18:20:46.655110+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:01:16.930740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'44","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.464971+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:23:36.464971+0000","last_peered":"2024-01-15T18:23:36.464971+0000","last_clean":"2024-01-15T18:23:36.464971+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:23:36.464971+0000","last_undegraded":"2024-01-15T18:23:36.464971+0000","last_fullsized":"2024-01-15T18:23:36.464971+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652300+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:20:46.652300+0000","last_peered":"2024-01-15T18:20:46.652300+0000","last_clean":"2024-01-15T18:20:46.652300+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:20:46.652300+0000","last_undegraded":"2024-01-15T18:20:46.652300+0000","last_fullsized":"2024-01-15T18:20:46.652300+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:24.049276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.465808+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:23:36.465808+0000","last_peered":"2024-01-15T18:23:36.465808+0000","last_clean":"2024-01-15T18:23:36.465808+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:23:36.465808+0000","last_undegraded":"2024-01-15T18:23:36.465808+0000","last_fullsized":"2024-01-15T18:23:36.465808+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:29:24.340948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'3","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.878194+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:19:41.878194+0000","last_peered":"2024-01-15T18:19:41.878194+0000","last_clean":"2024-01-15T18:19:41.878194+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:19:41.878194+0000","last_undegraded":"2024-01-15T18:19:41.878194+0000","last_fullsized":"2024-01-15T18:19:41.878194+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:51.311221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'51","reported_seq":75,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.455773+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:23:36.455773+0000","last_peered":"2024-01-15T18:23:36.455773+0000","last_clean":"2024-01-15T18:23:36.455773+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:23:36.455773+0000","last_undegraded":"2024-01-15T18:23:36.455773+0000","last_fullsized":"2024-01-15T18:23:36.455773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:02.239162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.877947+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:19:41.877947+0000","last_peered":"2024-01-15T18:19:41.877947+0000","last_clean":"2024-01-15T18:19:41.877947+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:19:41.877947+0000","last_undegraded":"2024-01-15T18:19:41.877947+0000","last_fullsized":"2024-01-15T18:19:41.877947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:59.159845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'36","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475421+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:23:36.475421+0000","last_peered":"2024-01-15T18:23:36.475421+0000","last_clean":"2024-01-15T18:23:36.475421+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:36.475421+0000","last_undegraded":"2024-01-15T18:23:36.475421+0000","last_fullsized":"2024-01-15T18:23:36.475421+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:01:50.217450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:41.868481+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:19:41.868481+0000","last_peered":"2024-01-15T18:19:41.868481+0000","last_clean":"2024-01-15T18:19:41.868481+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:19:41.868481+0000","last_undegraded":"2024-01-15T18:19:41.868481+0000","last_fullsized":"2024-01-15T18:19:41.868481+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:23.630535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'44","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.478124+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:23:36.478124+0000","last_peered":"2024-01-15T18:23:36.478124+0000","last_clean":"2024-01-15T18:23:36.478124+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:23:36.478124+0000","last_undegraded":"2024-01-15T18:23:36.478124+0000","last_fullsized":"2024-01-15T18:23:36.478124+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:19:42.040607+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:19:42.040607+0000","last_peered":"2024-01-15T18:19:42.040607+0000","last_clean":"2024-01-15T18:19:42.040607+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:19:42.040607+0000","last_undegraded":"2024-01-15T18:19:42.040607+0000","last_fullsized":"2024-01-15T18:19:42.040607+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'45","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.462420+0000","last_change":"2024-01-15T18:19:42.184955+0000","last_active":"2024-01-15T18:23:36.462420+0000","last_peered":"2024-01-15T18:23:36.462420+0000","last_clean":"2024-01-15T18:23:36.462420+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:23:36.462420+0000","last_undegraded":"2024-01-15T18:23:36.462420+0000","last_fullsized":"2024-01-15T18:23:36.462420+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303360,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":18,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:11.655302+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:21:11.655302+0000","last_peered":"2024-01-15T18:21:11.655302+0000","last_clean":"2024-01-15T18:21:11.655302+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:21:11.655302+0000","last_undegraded":"2024-01-15T18:21:11.655302+0000","last_fullsized":"2024-01-15T18:21:11.655302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'40","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.469773+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:23:36.469773+0000","last_peered":"2024-01-15T18:23:36.469773+0000","last_clean":"2024-01-15T18:23:36.469773+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:23:36.469773+0000","last_undegraded":"2024-01-15T18:23:36.469773+0000","last_fullsized":"2024-01-15T18:23:36.469773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":15,"num_write_kb":53249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:07.082656+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:21:07.082656+0000","last_peered":"2024-01-15T18:21:07.082656+0000","last_clean":"2024-01-15T18:21:07.082656+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:21:07.082656+0000","last_undegraded":"2024-01-15T18:21:07.082656+0000","last_fullsized":"2024-01-15T18:21:07.082656+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:18.870315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'41","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463503+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:23:36.463503+0000","last_peered":"2024-01-15T18:23:36.463503+0000","last_clean":"2024-01-15T18:23:36.463503+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:23:36.463503+0000","last_undegraded":"2024-01-15T18:23:36.463503+0000","last_fullsized":"2024-01-15T18:23:36.463503+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:06.664989+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:21:06.664989+0000","last_peered":"2024-01-15T18:21:06.664989+0000","last_clean":"2024-01-15T18:21:06.664989+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:21:06.664989+0000","last_undegraded":"2024-01-15T18:21:06.664989+0000","last_fullsized":"2024-01-15T18:21:06.664989+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:01.739271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'35","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.457523+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:23:36.457523+0000","last_peered":"2024-01-15T18:23:36.457523+0000","last_clean":"2024-01-15T18:23:36.457523+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:23:36.457523+0000","last_undegraded":"2024-01-15T18:23:36.457523+0000","last_fullsized":"2024-01-15T18:23:36.457523+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:07.287483+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:21:07.287483+0000","last_peered":"2024-01-15T18:21:07.287483+0000","last_clean":"2024-01-15T18:21:07.287483+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:21:07.287483+0000","last_undegraded":"2024-01-15T18:21:07.287483+0000","last_fullsized":"2024-01-15T18:21:07.287483+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:10.236338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'41","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466318+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:23:36.466318+0000","last_peered":"2024-01-15T18:23:36.466318+0000","last_clean":"2024-01-15T18:23:36.466318+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:23:36.466318+0000","last_undegraded":"2024-01-15T18:23:36.466318+0000","last_fullsized":"2024-01-15T18:23:36.466318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'11","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:40.071203+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:21:40.071203+0000","last_peered":"2024-01-15T18:21:40.071203+0000","last_clean":"2024-01-15T18:21:40.071203+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:21:40.071203+0000","last_undegraded":"2024-01-15T18:21:40.071203+0000","last_fullsized":"2024-01-15T18:21:40.071203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:05.606364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'40","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.467240+0000","last_change":"2024-01-15T18:18:41.931596+0000","last_active":"2024-01-15T18:23:36.467240+0000","last_peered":"2024-01-15T18:23:36.467240+0000","last_clean":"2024-01-15T18:23:36.467240+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:23:36.467240+0000","last_undegraded":"2024-01-15T18:23:36.467240+0000","last_fullsized":"2024-01-15T18:23:36.467240+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'170","reported_seq":186,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.430521+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:23:36.430521+0000","last_peered":"2024-01-15T18:23:36.430521+0000","last_clean":"2024-01-15T18:23:36.430521+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:23:36.430521+0000","last_undegraded":"2024-01-15T18:23:36.430521+0000","last_fullsized":"2024-01-15T18:23:36.430521+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:06.799590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569375,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'36","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.470792+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:23:36.470792+0000","last_peered":"2024-01-15T18:23:36.470792+0000","last_clean":"2024-01-15T18:23:36.470792+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:23:36.470792+0000","last_undegraded":"2024-01-15T18:23:36.470792+0000","last_fullsized":"2024-01-15T18:23:36.470792+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.664156+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:21:01.664156+0000","last_peered":"2024-01-15T18:21:01.664156+0000","last_clean":"2024-01-15T18:21:01.664156+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:21:01.664156+0000","last_undegraded":"2024-01-15T18:21:01.664156+0000","last_fullsized":"2024-01-15T18:21:01.664156+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:54:50.724126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'46","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.476804+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:23:36.476804+0000","last_peered":"2024-01-15T18:23:36.476804+0000","last_clean":"2024-01-15T18:23:36.476804+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:23:36.476804+0000","last_undegraded":"2024-01-15T18:23:36.476804+0000","last_fullsized":"2024-01-15T18:23:36.476804+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.653868+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:21:01.653868+0000","last_peered":"2024-01-15T18:21:01.653868+0000","last_clean":"2024-01-15T18:21:01.653868+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:21:01.653868+0000","last_undegraded":"2024-01-15T18:21:01.653868+0000","last_fullsized":"2024-01-15T18:21:01.653868+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'43","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480708+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:23:36.480708+0000","last_peered":"2024-01-15T18:23:36.480708+0000","last_clean":"2024-01-15T18:23:36.480708+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:23:36.480708+0000","last_undegraded":"2024-01-15T18:23:36.480708+0000","last_fullsized":"2024-01-15T18:23:36.480708+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:51:04.888171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080645,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":22,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":24,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.664196+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:21:01.664196+0000","last_peered":"2024-01-15T18:21:01.664196+0000","last_clean":"2024-01-15T18:21:01.664196+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:21:01.664196+0000","last_undegraded":"2024-01-15T18:21:01.664196+0000","last_fullsized":"2024-01-15T18:21:01.664196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:27:26.085701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'21","reported_seq":38,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479169+0000","last_change":"2024-01-15T18:18:41.936562+0000","last_active":"2024-01-15T18:23:36.479169+0000","last_peered":"2024-01-15T18:23:36.479169+0000","last_clean":"2024-01-15T18:23:36.479169+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:23:36.479169+0000","last_undegraded":"2024-01-15T18:23:36.479169+0000","last_fullsized":"2024-01-15T18:23:36.479169+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:23.675190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":7,"num_write_kb":28672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:57.081108+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:20:57.081108+0000","last_peered":"2024-01-15T18:20:57.081108+0000","last_clean":"2024-01-15T18:20:57.081108+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:20:57.081108+0000","last_undegraded":"2024-01-15T18:20:57.081108+0000","last_fullsized":"2024-01-15T18:20:57.081108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:01.231374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'35","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.472587+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:23:36.472587+0000","last_peered":"2024-01-15T18:23:36.472587+0000","last_clean":"2024-01-15T18:23:36.472587+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:23:36.472587+0000","last_undegraded":"2024-01-15T18:23:36.472587+0000","last_fullsized":"2024-01-15T18:23:36.472587+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:48:34.695980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663623+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:20:56.663623+0000","last_peered":"2024-01-15T18:20:56.663623+0000","last_clean":"2024-01-15T18:20:56.663623+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:20:56.663623+0000","last_undegraded":"2024-01-15T18:20:56.663623+0000","last_fullsized":"2024-01-15T18:20:56.663623+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:38.154337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'41","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.468898+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:23:36.468898+0000","last_peered":"2024-01-15T18:23:36.468898+0000","last_clean":"2024-01-15T18:23:36.468898+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:23:36.468898+0000","last_undegraded":"2024-01-15T18:23:36.468898+0000","last_fullsized":"2024-01-15T18:23:36.468898+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":7,"num_read_kb":12292,"num_write":19,"num_write_kb":61442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653254+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:20:56.653254+0000","last_peered":"2024-01-15T18:20:56.653254+0000","last_clean":"2024-01-15T18:20:56.653254+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:20:56.653254+0000","last_undegraded":"2024-01-15T18:20:56.653254+0000","last_fullsized":"2024-01-15T18:20:56.653254+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:00:59.694593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'40","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.463772+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:23:36.463772+0000","last_peered":"2024-01-15T18:23:36.463772+0000","last_clean":"2024-01-15T18:23:36.463772+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:23:36.463772+0000","last_undegraded":"2024-01-15T18:23:36.463772+0000","last_fullsized":"2024-01-15T18:23:36.463772+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:26.983917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.656234+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:20:56.656234+0000","last_peered":"2024-01-15T18:20:56.656234+0000","last_clean":"2024-01-15T18:20:56.656234+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:20:56.656234+0000","last_undegraded":"2024-01-15T18:20:56.656234+0000","last_fullsized":"2024-01-15T18:20:56.656234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:42.895411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'34","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480616+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:23:36.480616+0000","last_peered":"2024-01-15T18:23:36.480616+0000","last_clean":"2024-01-15T18:23:36.480616+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:23:36.480616+0000","last_undegraded":"2024-01-15T18:23:36.480616+0000","last_fullsized":"2024-01-15T18:23:36.480616+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'3","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663515+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:20:56.663515+0000","last_peered":"2024-01-15T18:20:56.663515+0000","last_clean":"2024-01-15T18:20:56.663515+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:20:56.663515+0000","last_undegraded":"2024-01-15T18:20:56.663515+0000","last_fullsized":"2024-01-15T18:20:56.663515+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:50.243863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'39","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.481081+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:23:36.481081+0000","last_peered":"2024-01-15T18:23:36.481081+0000","last_clean":"2024-01-15T18:23:36.481081+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:23:36.481081+0000","last_undegraded":"2024-01-15T18:23:36.481081+0000","last_fullsized":"2024-01-15T18:23:36.481081+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:14.044300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.663534+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:20:56.663534+0000","last_peered":"2024-01-15T18:20:56.663534+0000","last_clean":"2024-01-15T18:20:56.663534+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:20:56.663534+0000","last_undegraded":"2024-01-15T18:20:56.663534+0000","last_fullsized":"2024-01-15T18:20:56.663534+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:51:31.110057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'40","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.467085+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:23:36.467085+0000","last_peered":"2024-01-15T18:23:36.467085+0000","last_clean":"2024-01-15T18:23:36.467085+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:23:36.467085+0000","last_undegraded":"2024-01-15T18:23:36.467085+0000","last_fullsized":"2024-01-15T18:23:36.467085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:51.662713+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:20:51.662713+0000","last_peered":"2024-01-15T18:20:51.662713+0000","last_clean":"2024-01-15T18:20:51.662713+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:20:51.662713+0000","last_undegraded":"2024-01-15T18:20:51.662713+0000","last_fullsized":"2024-01-15T18:20:51.662713+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:39.137957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'40","reported_seq":58,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.462441+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:23:36.462441+0000","last_peered":"2024-01-15T18:23:36.462441+0000","last_clean":"2024-01-15T18:23:36.462441+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:23:36.462441+0000","last_undegraded":"2024-01-15T18:23:36.462441+0000","last_fullsized":"2024-01-15T18:23:36.462441+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720517,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":15,"num_write_kb":57345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:52.080245+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:20:52.080245+0000","last_peered":"2024-01-15T18:20:52.080245+0000","last_clean":"2024-01-15T18:20:52.080245+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:20:52.080245+0000","last_undegraded":"2024-01-15T18:20:52.080245+0000","last_fullsized":"2024-01-15T18:20:52.080245+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:11.659285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'33","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.461910+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:23:36.461910+0000","last_peered":"2024-01-15T18:23:36.461910+0000","last_clean":"2024-01-15T18:23:36.461910+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:23:36.461910+0000","last_undegraded":"2024-01-15T18:23:36.461910+0000","last_fullsized":"2024-01-15T18:23:36.461910+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:52.284956+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:20:52.284956+0000","last_peered":"2024-01-15T18:20:52.284956+0000","last_clean":"2024-01-15T18:20:52.284956+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:20:52.284956+0000","last_undegraded":"2024-01-15T18:20:52.284956+0000","last_fullsized":"2024-01-15T18:20:52.284956+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:57:15.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'37","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.490017+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:23:36.490017+0000","last_peered":"2024-01-15T18:23:36.490017+0000","last_clean":"2024-01-15T18:23:36.490017+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:23:36.490017+0000","last_undegraded":"2024-01-15T18:23:36.490017+0000","last_fullsized":"2024-01-15T18:23:36.490017+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:38:55.463538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2,"num_read_kb":8192,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'15","reported_seq":31,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:42.035206+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:22:42.035206+0000","last_peered":"2024-01-15T18:22:42.035206+0000","last_clean":"2024-01-15T18:22:42.035206+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:22:42.035206+0000","last_undegraded":"2024-01-15T18:22:42.035206+0000","last_fullsized":"2024-01-15T18:22:42.035206+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:51.313218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'42","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.479751+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:23:36.479751+0000","last_peered":"2024-01-15T18:23:36.479751+0000","last_clean":"2024-01-15T18:23:36.479751+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:23:36.479751+0000","last_undegraded":"2024-01-15T18:23:36.479751+0000","last_fullsized":"2024-01-15T18:23:36.479751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:47.175404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:47.079647+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:20:47.079647+0000","last_peered":"2024-01-15T18:20:47.079647+0000","last_clean":"2024-01-15T18:20:47.079647+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:20:47.079647+0000","last_undegraded":"2024-01-15T18:20:47.079647+0000","last_fullsized":"2024-01-15T18:20:47.079647+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:31.745491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'34","reported_seq":53,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.460078+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:23:36.460078+0000","last_peered":"2024-01-15T18:23:36.460078+0000","last_clean":"2024-01-15T18:23:36.460078+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:23:36.460078+0000","last_undegraded":"2024-01-15T18:23:36.460078+0000","last_fullsized":"2024-01-15T18:23:36.460078+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:27.713450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331770,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":5,"num_read_kb":20480,"num_write":14,"num_write_kb":49153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:47.284362+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:20:47.284362+0000","last_peered":"2024-01-15T18:20:47.284362+0000","last_clean":"2024-01-15T18:20:47.284362+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:20:47.284362+0000","last_undegraded":"2024-01-15T18:20:47.284362+0000","last_fullsized":"2024-01-15T18:20:47.284362+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'44","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480400+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:23:36.480400+0000","last_peered":"2024-01-15T18:23:36.480400+0000","last_clean":"2024-01-15T18:23:36.480400+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:23:36.480400+0000","last_undegraded":"2024-01-15T18:23:36.480400+0000","last_fullsized":"2024-01-15T18:23:36.480400+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":1,"num_read_kb":4096,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655069+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:20:46.655069+0000","last_peered":"2024-01-15T18:20:46.655069+0000","last_clean":"2024-01-15T18:20:46.655069+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:20:46.655069+0000","last_undegraded":"2024-01-15T18:20:46.655069+0000","last_fullsized":"2024-01-15T18:20:46.655069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:54.586280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'51","reported_seq":72,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.480979+0000","last_change":"2024-01-15T18:18:41.935993+0000","last_active":"2024-01-15T18:23:36.480979+0000","last_peered":"2024-01-15T18:23:36.480979+0000","last_clean":"2024-01-15T18:23:36.480979+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:23:36.480979+0000","last_undegraded":"2024-01-15T18:23:36.480979+0000","last_fullsized":"2024-01-15T18:23:36.480979+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652145+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:20:46.652145+0000","last_peered":"2024-01-15T18:20:46.652145+0000","last_clean":"2024-01-15T18:20:46.652145+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:20:46.652145+0000","last_undegraded":"2024-01-15T18:20:46.652145+0000","last_fullsized":"2024-01-15T18:20:46.652145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:26.705662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'38","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.466662+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:23:36.466662+0000","last_peered":"2024-01-15T18:23:36.466662+0000","last_clean":"2024-01-15T18:23:36.466662+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:23:36.466662+0000","last_undegraded":"2024-01-15T18:23:36.466662+0000","last_fullsized":"2024-01-15T18:23:36.466662+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'10","reported_seq":35,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:40.067251+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:21:40.067251+0000","last_peered":"2024-01-15T18:21:40.067251+0000","last_clean":"2024-01-15T18:21:40.067251+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:21:40.067251+0000","last_undegraded":"2024-01-15T18:21:40.067251+0000","last_fullsized":"2024-01-15T18:21:40.067251+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:03:44.404981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'36","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475826+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:23:36.475826+0000","last_peered":"2024-01-15T18:23:36.475826+0000","last_clean":"2024-01-15T18:23:36.475826+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:36.475826+0000","last_undegraded":"2024-01-15T18:23:36.475826+0000","last_fullsized":"2024-01-15T18:23:36.475826+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:52.187345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:42.078947+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:20:42.078947+0000","last_peered":"2024-01-15T18:20:42.078947+0000","last_clean":"2024-01-15T18:20:42.078947+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:20:42.078947+0000","last_undegraded":"2024-01-15T18:20:42.078947+0000","last_fullsized":"2024-01-15T18:20:42.078947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:19.414580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'30","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:36.475835+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:23:36.475835+0000","last_peered":"2024-01-15T18:23:36.475835+0000","last_clean":"2024-01-15T18:23:36.475835+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:23:36.475835+0000","last_undegraded":"2024-01-15T18:23:36.475835+0000","last_fullsized":"2024-01-15T18:23:36.475835+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.656696+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:21:01.656696+0000","last_peered":"2024-01-15T18:21:01.656696+0000","last_clean":"2024-01-15T18:21:01.656696+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:21:01.656696+0000","last_undegraded":"2024-01-15T18:21:01.656696+0000","last_fullsized":"2024-01-15T18:21:01.656696+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:35.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":246,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653639+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:20:56.653639+0000","last_peered":"2024-01-15T18:20:56.653639+0000","last_clean":"2024-01-15T18:20:56.653639+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:20:56.653639+0000","last_undegraded":"2024-01-15T18:20:56.653639+0000","last_fullsized":"2024-01-15T18:20:56.653639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":4294968312,"num_objects":1030,"num_object_clones":1,"num_object_copies":2060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1030,"num_whiteouts":1,"num_read":269,"num_read_kb":1048589,"num_write":1050,"num_write_kb":4194319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8589975552,"data_stored":8589936624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2554,"ondisk_log_size":2554,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":571494,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":256,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1204224,"data_stored":1142988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461744,"kb_used_data":1398696,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92910096,"statfs":{"total":96636764160,"available":95139938304,"internally_reserved":0,"allocated":1432264704,"data_stored":1431460098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69099999999999995,"15min":0.94399999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.29999999999999999},"max":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.627},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.87},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":13.744},"last":0.56799999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":1.0840000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":16.920000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":1.05},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":1.04,"15min":31.774999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72599999999999998,"15min":1},"min":{"1min":0.57399999999999995,"5min":0.48699999999999999,"15min":0.377},"max":{"1min":1.056,"5min":1.085,"15min":30.846},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":1.0629999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":1.0229999999999999,"15min":39.695},"last":0.69499999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74099999999999999,"15min":1.1499999999999999},"min":{"1min":0.54500000000000004,"5min":0.437,"15min":0.38800000000000001},"max":{"1min":1.01,"5min":1.0109999999999999,"15min":26.463999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":1.198},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.41499999999999998},"max":{"1min":0.86699999999999999,"5min":1.0640000000000001,"15min":42.692},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86699999999999999,"5min":0.874,"15min":14.087},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.36299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":2.952},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":11.334},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.746},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91700000000000004,"5min":0.94499999999999995,"15min":8.423},"last":0.52300000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.78200000000000003},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.45500000000000002},"max":{"1min":0.95099999999999996,"5min":1.034,"15min":8.5030000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.35299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":3.3450000000000002},"last":0.752}]}]},{"osd":6,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":998316,"kb_used_data":935268,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93373524,"statfs":{"total":96636764160,"available":95614488576,"internally_reserved":0,"allocated":957714432,"data_stored":956912954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":22.754000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.82399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":1.018,"15min":9.1470000000000002},"last":0.65300000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.35299999999999998,"15min":0.34300000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":14.246},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":0.98199999999999998},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":0.97899999999999998,"15min":19.475000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":1.163},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.0549999999999999,"15min":42.825000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.747,"15min":1.107},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":1.103,"15min":42.851999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.98599999999999999},"min":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.8160000000000001,"15min":24.199000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":1.0700000000000001},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":1.03,"15min":18.791},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":1.044,"15min":19.114999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":8.3059999999999992},"last":0.53600000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.88800000000000001,"5min":1.0920000000000001,"15min":7.5880000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":0.93500000000000005,"5min":1.1120000000000001,"15min":12.945},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69099999999999995,"15min":0.80100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.38600000000000001},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":11.907},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.85399999999999998},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.86599999999999999,"5min":1.1910000000000001,"15min":27.651},"last":0.66900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793712,"kb_used_data":718184,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93578128,"statfs":{"total":96636764160,"available":95824003072,"internally_reserved":0,"allocated":735420416,"data_stored":734615446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68400000000000005,"15min":0.85199999999999998},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.29899999999999999},"max":{"1min":0.98599999999999999,"5min":1.9350000000000001,"15min":11.519},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74099999999999999,"15min":0.94599999999999995},"min":{"1min":0.626,"5min":0.371,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":2.6819999999999999,"15min":12.952},"last":0.56399999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":2.1179999999999999,"15min":13.358000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.96699999999999997},"min":{"1min":0.56999999999999995,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":1.0289999999999999,"5min":2.8050000000000002,"15min":21.856999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":3.508,"15min":18.314},"last":0.62},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":1.0449999999999999},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":1.069,"5min":3.5310000000000001,"15min":21.120000000000001},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":6.7990000000000004,"15min":13.317},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":1.0369999999999999},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":2.3380000000000001,"15min":19.638000000000002},"last":0.57799999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.96399999999999997,"15min":11.093},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.95699999999999996,"15min":3.3889999999999998},"last":0.38900000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.78900000000000003},"min":{"1min":0.5,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":5.8680000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32600000000000001},"max":{"1min":0.91300000000000003,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.316}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":1.081,"15min":3.2759999999999998},"last":0.432},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.754,"15min":0.80300000000000005},"min":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.88300000000000001,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.54200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1387432,"kb_used_data":1324384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92984408,"statfs":{"total":96636764160,"available":95216033792,"internally_reserved":0,"allocated":1356169216,"data_stored":1355371903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.94899999999999995},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":1.02,"15min":34.802},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.91300000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":0.871,"5min":1.1319999999999999,"15min":22.335999999999999},"last":0.61699999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":0.90600000000000003,"5min":1.224,"15min":38.198},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.752,"15min":0.96799999999999997},"min":{"1min":0.52400000000000002,"5min":0.438,"15min":0.372},"max":{"1min":0.875,"5min":1.0269999999999999,"15min":38.220999999999997},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.89600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.34599999999999997},"max":{"1min":0.91600000000000004,"5min":1.0509999999999999,"15min":7.6989999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76000000000000001,"15min":0.85999999999999999},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":1.0740000000000001,"15min":7.4829999999999997},"last":0.77700000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.95199999999999996},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":1.214,"15min":14.529},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.94699999999999995},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":25.774000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.68899999999999995},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.33200000000000002},"max":{"1min":0.89800000000000002,"5min":0.90300000000000002,"15min":9.9619999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.79700000000000004},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":1.1719999999999999,"15min":25.887},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":10.284000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.97299999999999998,"15min":3.6949999999999998},"last":0.73099999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":13.760999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.36199999999999999},"max":{"1min":0.96699999999999997,"5min":1.0269999999999999,"15min":11.67},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1154596,"kb_used_data":1087388,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93217244,"statfs":{"total":96636764160,"available":95454457856,"internally_reserved":0,"allocated":1113485312,"data_stored":1112671977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.66200000000000003,"5min":2.415,"15min":26.364000000000001},"last":0.44},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.95499999999999996},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.79500000000000004,"15min":28.332000000000001},"last":0.40699999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.98399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63300000000000001,"5min":0.82499999999999996,"15min":40.457999999999998},"last":0.315},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":1.1120000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":27.427},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.98799999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":23.036999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":1.081},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.873,"15min":26.404},"last":0.504}]},{"osd":4,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":1.0349999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.0529999999999999,"5min":1.1439999999999999,"15min":22.998000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76300000000000001,"15min":1.0740000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1,"5min":1.1439999999999999,"15min":20.399000000000001},"last":0.746}]},{"osd":5,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":1.0820000000000001},"min":{"1min":0.52800000000000002,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":0.97999999999999998,"5min":1.46,"15min":26.314},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":1.1040000000000001},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1,"5min":1.109,"15min":22.908999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79000000000000004,"15min":1.155},"min":{"1min":0.57399999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.105,"5min":1.105,"15min":20.445},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81399999999999995,"15min":1.1659999999999999},"min":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":1.4299999999999999,"15min":26.091999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":1.159},"min":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":23.111999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":1.3169999999999999},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.46400000000000002},"max":{"1min":1.0880000000000001,"5min":1.145,"15min":44.817999999999998},"last":0.85099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768936,"kb_used_data":705888,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93602904,"statfs":{"total":96636764160,"available":95849373696,"internally_reserved":0,"allocated":722829312,"data_stored":722033059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":1.054},"min":{"1min":0.28599999999999998,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.82899999999999996,"5min":4.2329999999999997,"15min":29.527999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":1.335},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":6.0670000000000002,"15min":34.122999999999998},"last":0.52000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":1.534},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":4.5259999999999998,"15min":113.149},"last":0.498},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59099999999999997,"15min":1.127},"min":{"1min":0.40500000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84899999999999998,"5min":6.1020000000000003,"15min":41.302},"last":0.47899999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":1.3899999999999999},"min":{"1min":0.34699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76800000000000002,"5min":4.7370000000000001,"15min":46.960000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.625,"15min":1.2809999999999999},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":3.8809999999999998,"15min":115.693},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83199999999999996,"15min":1.194},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":1.1100000000000001,"5min":3.8399999999999999,"15min":24.366},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":1.228},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.432},"max":{"1min":1.0189999999999999,"5min":2.343,"15min":20.489000000000001},"last":0.81499999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":1.5289999999999999},"min":{"1min":0.61899999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":1.175,"5min":3.927,"15min":41.683999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80400000000000005,"15min":1.2889999999999999},"min":{"1min":0.60799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98599999999999999,"5min":2.3029999999999999,"15min":26.084},"last":0.70899999999999996}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":1.319},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.159,"5min":6.0220000000000002,"15min":22.199000000000002},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":1.569},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.0860000000000001,"5min":3.9590000000000001,"15min":41.738999999999997},"last":0.621}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85599999999999998,"15min":1.262},"min":{"1min":0.621,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.1339999999999999,"5min":2.556,"15min":20.446999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":1.6060000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1599999999999999,"5min":2.5939999999999999,"15min":41.402000000000001},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":941048,"kb_used_data":873840,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93430792,"statfs":{"total":96636764160,"available":95673131008,"internally_reserved":0,"allocated":894812160,"data_stored":894000019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.439,"15min":0.86099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.63700000000000001,"5min":1.4179999999999999,"15min":39.948},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.56599999999999995,"15min":1.0649999999999999},"min":{"1min":0.30399999999999999,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":4.9329999999999998,"15min":35.790999999999997},"last":0.38900000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.57699999999999996,"15min":0.97099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.27600000000000002},"max":{"1min":0.66100000000000003,"5min":9.2609999999999992,"15min":49.229999999999997},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":1.7649999999999999,"15min":10.349},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":1.0589999999999999},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68300000000000005,"5min":0.84999999999999998,"15min":27.643999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.90800000000000003},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":1.198,"15min":15.608000000000001},"last":0.498}]},{"osd":4,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76500000000000001,"15min":1.165},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":1.1080000000000001,"5min":1.383,"15min":27.489999999999998},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":1.0349999999999999},"min":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.218,"5min":6.6950000000000003,"15min":10.038},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76900000000000002,"15min":1.1779999999999999},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1739999999999999,"5min":1.2010000000000001,"15min":27.547000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.55400000000000005},"max":{"1min":1.1599999999999999,"5min":1.266,"15min":9.9610000000000003},"last":0.90000000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80600000000000005,"15min":1.171},"min":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.496},"max":{"1min":1.147,"5min":1.984,"15min":27.579999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":1},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.42399999999999999},"max":{"1min":1.1910000000000001,"5min":5.6470000000000002,"15min":11.712999999999999},"last":0.79900000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":1.0980000000000001},"min":{"1min":0.56899999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":5.819,"15min":11.833},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.89000000000000001,"15min":1.0920000000000001},"min":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.502},"max":{"1min":1.254,"5min":6.3570000000000002,"15min":14.606},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575077,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1421360,"kb_used_data":1358312,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92950480,"statfs":{"total":96636764160,"available":95181291520,"internally_reserved":0,"allocated":1390911488,"data_stored":1390086416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46300000000000002,"15min":1.054},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":31.977},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.51300000000000001,"15min":1.0349999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.61499999999999999,"5min":1.6579999999999999,"15min":36.145000000000003},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":1.1439999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":1.1759999999999999,"15min":41.387999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":1.0089999999999999},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70399999999999996,"5min":1.7689999999999999,"15min":33.219999999999999},"last":0.503}]},{"osd":3,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":1.006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80500000000000005,"15min":31.943999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":1.107},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.746,"5min":0.95599999999999996,"15min":45.128},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.38300000000000001},"max":{"1min":1.0469999999999999,"5min":1.4550000000000001,"15min":32.932000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.77400000000000002,"15min":1.1659999999999999},"min":{"1min":0.53600000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":1.115,"15min":31.782},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":1.149},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.38600000000000001},"max":{"1min":1.0249999999999999,"5min":1.179,"15min":31.858000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.79300000000000004,"15min":1.0409999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.7869999999999999,"15min":33.151000000000003},"last":0.69899999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":1.0660000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":13.055},"last":0.621},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.47699999999999998},"max":{"1min":1.262,"5min":1.7170000000000001,"15min":33.098999999999997},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":1.097},"min":{"1min":0.68200000000000005,"5min":0.55500000000000005,"15min":0.48899999999999999},"max":{"1min":1.0029999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":1.2050000000000001},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":0.997,"5min":1.075,"15min":31.905999999999999},"last":0.59799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":585728,"data_stored":569487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":569445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1388326912,"data_stored":1388315187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":893394944,"data_stored":893387124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":721420288,"data_stored":721420288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1111498752,"data_stored":1111491082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1354764288,"data_stored":1354760453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":734007296,"data_stored":734003392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":956305408,"data_stored":956301434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1430257664,"data_stored":1430257664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:23:42.392 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:23:42.565 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 Processor -- start 2024-01-15T18:23:42.566 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 -- start start 2024-01-15T18:23:42.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e006b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27e006bb70 0x7f27e0070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.567 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e00b3370 con 0x7f27e00712a0 2024-01-15T18:23:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e00716f0 con 0x7f27e006bb70 2024-01-15T18:23:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27e4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e015d5d0 con 0x7f27e015d1f0 2024-01-15T18:23:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dd400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27e006bb70 0x7f27e0070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27de800640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f27e00712a0 0x7f27e006cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60706/0 (socket says 172.21.15.143:60706) 2024-01-15T18:23:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27de800640 1 -- 172.21.15.143:0/1851431785 learned_addr learned my addr 172.21.15.143:0/1851431785 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:42.568 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dde00640 1 --2- 172.21.15.143:0/1851431785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e006b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dd400640 1 -- 172.21.15.143:0/1851431785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 msgr2=0x7f27e006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dd400640 1 --2- 172.21.15.143:0/1851431785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e006b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dd400640 1 -- 172.21.15.143:0/1851431785 >> v1:172.21.15.143:6789/0 conn(0x7f27e00712a0 legacy=0x7f27e006cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dd400640 1 -- 172.21.15.143:0/1851431785 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e006a800 con 0x7f27e006bb70 2024-01-15T18:23:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.575+0000 7f27dca00640 1 -- 172.21.15.143:0/1851431785 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657889956 0 0) 0x7f27e00b3370 con 0x7f27e00712a0 2024-01-15T18:23:42.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27dd400640 1 --2- 172.21.15.143:0/1851431785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0070d60 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f27c80097c0 tx=0x7f27c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a10b1ba73c82893 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:42.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27dca00640 1 -- 172.21.15.143:0/1851431785 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80ab650 con 0x7f27e006bb70 2024-01-15T18:23:42.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27dca00640 1 -- 172.21.15.143:0/1851431785 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c80ab7b0 con 0x7f27e006bb70 2024-01-15T18:23:42.571 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27dca00640 1 -- 172.21.15.143:0/1851431785 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c80abb40 con 0x7f27e006bb70 2024-01-15T18:23:42.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 -- 172.21.15.143:0/1851431785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 msgr2=0x7f27e0070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 --2- 172.21.15.143:0/1851431785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0070d60 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f27c80097c0 tx=0x7f27c801c450 comp rx=0 tx=0).stop 2024-01-15T18:23:42.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 -- 172.21.15.143:0/1851431785 shutdown_connections 2024-01-15T18:23:42.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 --2- 172.21.15.143:0/1851431785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0070d60 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.572 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 --2- 172.21.15.143:0/1851431785 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e006b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 -- 172.21.15.143:0/1851431785 >> 172.21.15.143:0/1851431785 conn(0x7f27e00b1610 msgr2=0x7f27e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:42.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.579+0000 7f27e4e00640 1 -- 172.21.15.143:0/1851431785 shutdown_connections 2024-01-15T18:23:42.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 -- 172.21.15.143:0/1851431785 wait complete. 2024-01-15T18:23:42.573 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 Processor -- start 2024-01-15T18:23:42.574 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 -- start start 2024-01-15T18:23:42.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e00712a0 0x7f27e0149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.576 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e00b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e014a190 con 0x7f27e006bb70 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e014a470 con 0x7f27e00712a0 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e014a750 con 0x7f27e015d1f0 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38082/0 (socket says 172.21.15.143:38082) 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 -- 172.21.15.143:0/344101020 learned_addr learned my addr 172.21.15.143:0/344101020 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dd400640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e00712a0 0x7f27e0149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27de800640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e00b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.577 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 -- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 msgr2=0x7f27e00b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e00b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 -- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e00712a0 msgr2=0x7f27e0149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e00712a0 0x7f27e0149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 -- 172.21.15.143:0/344101020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c8009420 con 0x7f27e006bb70 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dd400640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e00712a0 0x7f27e0149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27de800640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e00b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27dde00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0149bb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f27d000e870 tx=0x7f27d000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:42.578 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0009510 con 0x7f27e006bb70 2024-01-15T18:23:42.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d000eec0 con 0x7f27e006bb70 2024-01-15T18:23:42.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.583+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e014aa30 con 0x7f27e006bb70 2024-01-15T18:23:42.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d000cb40 con 0x7f27e006bb70 2024-01-15T18:23:42.579 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e0160610 con 0x7f27e006bb70 2024-01-15T18:23:42.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e0074b00 con 0x7f27e006bb70 2024-01-15T18:23:42.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f27d000cca0 con 0x7f27e006bb70 2024-01-15T18:23:42.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27d6000640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f27b8067b80 0x7f27b806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:42.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f27d00831b0 con 0x7f27e006bb70 2024-01-15T18:23:42.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27dd400640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f27b8067b80 0x7f27b806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:42.580 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.587+0000 7f27dd400640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f27b8067b80 0x7f27b806a040 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f27c80af000 tx=0x7f27c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:42.583 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.591+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f27d0051e40 con 0x7f27e006bb70 2024-01-15T18:23:42.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.743+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27e0067500 con 0x7f27e006bb70 2024-01-15T18:23:42.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.743+0000 7f27d6000640 1 -- 172.21.15.143:0/344101020 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16745 (secure 0 0 0) 0x7f27d0021090 con 0x7f27e006bb70 2024-01-15T18:23:42.735 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:42.735 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":113,"fsid":"10305fab-1773-4952-a8b6-aa6388f5534c","created":"2024-01-15T18:04:33.678621+0000","modified":"2024-01-15T18:19:41.863194+0000","last_up_change":"2024-01-15T18:04:46.036975+0000","last_in_change":"2024-01-15T18:04:36.184250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T18:04:46.686433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:18:39.661662+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:18:40.286820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7f53e508-d995-4e4c-811b-124725163b34","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6804","nonce":2045350344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6808","nonce":2045350344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6815","nonce":2045350344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":2045350344},{"type":"v1","addr":"172.21.15.143:6812","nonce":2045350344}]},"public_addr":"172.21.15.143:6804/2045350344","cluster_addr":"172.21.15.143:6808/2045350344","heartbeat_back_addr":"172.21.15.143:6815/2045350344","heartbeat_front_addr":"172.21.15.143:6812/2045350344","state":["exists","up"]},{"osd":1,"uuid":"7066343c-fbd3-45f3-8f91-12ecd8dab094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6801","nonce":2952112662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6803","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6805","nonce":2952112662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6811","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6813","nonce":2952112662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6807","nonce":2952112662},{"type":"v1","addr":"172.21.15.143:6809","nonce":2952112662}]},"public_addr":"172.21.15.143:6801/2952112662","cluster_addr":"172.21.15.143:6805/2952112662","heartbeat_back_addr":"172.21.15.143:6813/2952112662","heartbeat_front_addr":"172.21.15.143:6809/2952112662","state":["exists","up"]},{"osd":2,"uuid":"cfa742d1-b3c0-440a-87db-8f22f23d5e79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6817","nonce":761026363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6819","nonce":761026363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6823","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6825","nonce":761026363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":761026363},{"type":"v1","addr":"172.21.15.143:6821","nonce":761026363}]},"public_addr":"172.21.15.143:6817/761026363","cluster_addr":"172.21.15.143:6819/761026363","heartbeat_back_addr":"172.21.15.143:6825/761026363","heartbeat_front_addr":"172.21.15.143:6821/761026363","state":["exists","up"]},{"osd":3,"uuid":"68992798-8430-430e-a3d4-d05044dba6a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6824","nonce":254985013}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6827","nonce":254985013}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6831","nonce":254985013}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":254985013},{"type":"v1","addr":"172.21.15.143:6829","nonce":254985013}]},"public_addr":"172.21.15.143:6824/254985013","cluster_addr":"172.21.15.143:6827/254985013","heartbeat_back_addr":"172.21.15.143:6831/254985013","heartbeat_front_addr":"172.21.15.143:6829/254985013","state":["exists","up"]},{"osd":4,"uuid":"597dd8ae-08f6-4609-9eee-19492d4f4354","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6817","nonce":1717170274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6819","nonce":1717170274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6823","nonce":1717170274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1717170274},{"type":"v1","addr":"172.21.15.204:6821","nonce":1717170274}]},"public_addr":"172.21.15.204:6817/1717170274","cluster_addr":"172.21.15.204:6819/1717170274","heartbeat_back_addr":"172.21.15.204:6823/1717170274","heartbeat_front_addr":"172.21.15.204:6821/1717170274","state":["exists","up"]},{"osd":5,"uuid":"644e82b5-0dbb-46af-9320-65cf68f8e9b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6801","nonce":1020370322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6803","nonce":1020370322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6807","nonce":1020370322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1020370322},{"type":"v1","addr":"172.21.15.204:6805","nonce":1020370322}]},"public_addr":"172.21.15.204:6801/1020370322","cluster_addr":"172.21.15.204:6803/1020370322","heartbeat_back_addr":"172.21.15.204:6807/1020370322","heartbeat_front_addr":"172.21.15.204:6805/1020370322","state":["exists","up"]},{"osd":6,"uuid":"1e9a0e87-6c4a-48e9-9e2c-9c2e9951ab64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6809","nonce":1677254362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6811","nonce":1677254362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6815","nonce":1677254362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":1677254362},{"type":"v1","addr":"172.21.15.204:6813","nonce":1677254362}]},"public_addr":"172.21.15.204:6809/1677254362","cluster_addr":"172.21.15.204:6811/1677254362","heartbeat_back_addr":"172.21.15.204:6815/1677254362","heartbeat_front_addr":"172.21.15.204:6813/1677254362","state":["exists","up"]},{"osd":7,"uuid":"80584af3-3415-4da6-b726-fa846967621e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6825","nonce":2462757405}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6827","nonce":2462757405}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6831","nonce":2462757405}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":2462757405},{"type":"v1","addr":"172.21.15.204:6829","nonce":2462757405}]},"public_addr":"172.21.15.204:6825/2462757405","cluster_addr":"172.21.15.204:6827/2462757405","heartbeat_back_addr":"172.21.15.204:6831/2462757405","heartbeat_front_addr":"172.21.15.204:6829/2462757405","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.700701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.863549+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.927585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.926050+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.975885+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.775623+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:42.041325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T18:04:41.835308+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":5}]},{"pgid":"17.26","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:23:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f27b8067b80 msgr2=0x7f27b806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f27b8067b80 0x7f27b806a040 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f27c80af000 tx=0x7f27c8008040 comp rx=0 tx=0).stop 2024-01-15T18:23:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 msgr2=0x7f27e0149bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:42.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0149bb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f27d000e870 tx=0x7f27d000ec70 comp rx=0 tx=0).stop 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 shutdown_connections 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f27b8067b80 0x7f27b806a040 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f27e015d1f0 0x7f27e00b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f27e00712a0 0x7f27e0149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 --2- 172.21.15.143:0/344101020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27e006bb70 0x7f27e0149bb0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 >> 172.21.15.143:0/344101020 conn(0x7f27e00b1610 msgr2=0x7f27e015b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 shutdown_connections 2024-01-15T18:23:42.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:42.747+0000 7f27e4e00640 1 -- 172.21.15.143:0/344101020 wait complete. 2024-01-15T18:23:43.757 INFO:tasks.ceph:Scrubbing osd.0 2024-01-15T18:23:43.758 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:43.904 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 Processor -- start 2024-01-15T18:23:43.905 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 -- start start 2024-01-15T18:23:43.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fc5c80721b0 0x7fc5c80b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc5c80af860 0x7fc5c80b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.906 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5c80b3a10 con 0x7fc5c8159360 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c806a490 con 0x7fc5c80af860 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cfa00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5c806a750 con 0x7fc5c80721b0 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5ce400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc5c80af860 0x7fc5c80b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cd000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc5c8159360 0x7fc5c8071aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60722/0 (socket says 172.21.15.143:60722) 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5cd000640 1 -- 172.21.15.143:0/1099994797 learned_addr learned my addr 172.21.15.143:0/1099994797 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5c7e00640 1 --2- 172.21.15.143:0/1099994797 >> v2:172.21.15.204:3300/0 conn(0x7fc5c80721b0 0x7fc5c80b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5ce400640 1 -- 172.21.15.143:0/1099994797 >> v2:172.21.15.204:3300/0 conn(0x7fc5c80721b0 msgr2=0x7fc5c80b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:43.907 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5ce400640 1 --2- 172.21.15.143:0/1099994797 >> v2:172.21.15.204:3300/0 conn(0x7fc5c80721b0 0x7fc5c80b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5ce400640 1 -- 172.21.15.143:0/1099994797 >> v1:172.21.15.143:6789/0 conn(0x7fc5c8159360 legacy=0x7fc5c8071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5ce400640 1 -- 172.21.15.143:0/1099994797 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c806aa30 con 0x7fc5c80af860 2024-01-15T18:23:43.908 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.915+0000 7fc5c7400640 1 -- 172.21.15.143:0/1099994797 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876633666 0 0) 0x7fc5c80b3a10 con 0x7fc5c8159360 2024-01-15T18:23:43.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5ce400640 1 --2- 172.21.15.143:0/1099994797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c80b43e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009d90 tx=0x7fc5c001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db63ee258dd7bb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5c7400640 1 -- 172.21.15.143:0/1099994797 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00a59c0 con 0x7fc5c80af860 2024-01-15T18:23:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5c7400640 1 -- 172.21.15.143:0/1099994797 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c00a5b20 con 0x7fc5c80af860 2024-01-15T18:23:43.910 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5c7400640 1 -- 172.21.15.143:0/1099994797 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00ad440 con 0x7fc5c80af860 2024-01-15T18:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1099994797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 msgr2=0x7fc5c80b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 --2- 172.21.15.143:0/1099994797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c80b43e0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009d90 tx=0x7fc5c001c8e0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1099994797 shutdown_connections 2024-01-15T18:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 --2- 172.21.15.143:0/1099994797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c80b43e0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 --2- 172.21.15.143:0/1099994797 >> v2:172.21.15.204:3300/0 conn(0x7fc5c80721b0 0x7fc5c80b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.911 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1099994797 >> 172.21.15.143:0/1099994797 conn(0x7fc5c8066720 msgr2=0x7fc5c806dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1099994797 shutdown_connections 2024-01-15T18:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.919+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1099994797 wait complete. 2024-01-15T18:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 Processor -- start 2024-01-15T18:23:43.912 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- start start 2024-01-15T18:23:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5c80721b0 0x7fc5c8144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c8146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5c8159360 0x7fc5c8151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c80cdd80 con 0x7fc5c80af860 2024-01-15T18:23:43.915 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c81450e0 con 0x7fc5c80721b0 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c8146750 con 0x7fc5c8159360 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c8146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c8146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38092/0 (socket says 172.21.15.143:38092) 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 -- 172.21.15.143:0/1478299027 learned_addr learned my addr 172.21.15.143:0/1478299027 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cd000640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5c80721b0 0x7fc5c8144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5ce400640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5c8159360 0x7fc5c8151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.916 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 -- 172.21.15.143:0/1478299027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5c8159360 msgr2=0x7fc5c8151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fc5c8159360 0x7fc5c8151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 -- 172.21.15.143:0/1478299027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5c80721b0 msgr2=0x7fc5c8144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5c80721b0 0x7fc5c8144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 -- 172.21.15.143:0/1478299027 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c0009a70 con 0x7fc5c80af860 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c7e00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c8146210 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000d830 tx=0x7fc5b000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000b6c0 con 0x7fc5c80af860 2024-01-15T18:23:43.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b000b820 con 0x7fc5c80af860 2024-01-15T18:23:43.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c81448c0 con 0x7fc5c80af860 2024-01-15T18:23:43.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000bb90 con 0x7fc5c80af860 2024-01-15T18:23:43.918 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c8157a40 con 0x7fc5c80af860 2024-01-15T18:23:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.923+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc5c8067680 con 0x7fc5c80af860 2024-01-15T18:23:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fc5b001e460 con 0x7fc5c80af860 2024-01-15T18:23:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5a0067b80 0x7fc5a006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fc5b0083be0 con 0x7fc5c80af860 2024-01-15T18:23:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5cd000640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5a0067b80 0x7fc5a006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.920 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fc5a006fea0 0x7fc5a0072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 --> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc5a00729f0 con 0x7fc5a006fea0 2024-01-15T18:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5b0084770 con 0x7fc5c80af860 2024-01-15T18:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5ce400640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fc5a006fea0 0x7fc5a0072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5cd000640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5a0067b80 0x7fc5a006a040 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000f70 tx=0x7fc5b8007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5ce400640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fc5a006fea0 0x7fc5a0072320 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:43.921 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.927+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== osd.0 v2:172.21.15.143:6802/2045350344 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc5a00729f0 con 0x7fc5a006fea0 2024-01-15T18:23:43.932 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 --> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc5c815cb30 con 0x7fc5a006fea0 2024-01-15T18:23:43.933 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5c4c00640 1 -- 172.21.15.143:0/1478299027 <== osd.0 v2:172.21.15.143:6802/2045350344 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc5c815cb30 con 0x7fc5a006fea0 2024-01-15T18:23:43.933 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:43.933 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:43.934 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fc5a006fea0 msgr2=0x7fc5a0072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6802/2045350344,v1:172.21.15.143:6804/2045350344] conn(0x7fc5a006fea0 0x7fc5a0072320 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5a0067b80 msgr2=0x7fc5a006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fc5a0067b80 0x7fc5a006a040 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000f70 tx=0x7fc5b8007c10 comp rx=0 tx=0).stop 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 msgr2=0x7fc5c8146210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 --2- 172.21.15.143:0/1478299027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5c80af860 0x7fc5c8146210 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000d830 tx=0x7fc5b000dcf0 comp rx=0 tx=0).stop 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5ce400640 1 -- 172.21.15.143:0/1478299027 reap_dead start 2024-01-15T18:23:43.935 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 shutdown_connections 2024-01-15T18:23:43.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 >> 172.21.15.143:0/1478299027 conn(0x7fc5c8066720 msgr2=0x7fc5c815b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:43.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 shutdown_connections 2024-01-15T18:23:43.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:43.943+0000 7fc5cfa00640 1 -- 172.21.15.143:0/1478299027 wait complete. 2024-01-15T18:23:43.952 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-15T18:23:44.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a18a00640 1 Processor -- start 2024-01-15T18:23:44.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a18a00640 1 -- start start 2024-01-15T18:23:44.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 0x7f7a100d1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a18a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1015b6b0 con 0x7f7a1015b2d0 2024-01-15T18:23:44.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a18a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a100b0620 con 0x7f7a100cfb00 2024-01-15T18:23:44.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a18a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100b0320 con 0x7f7a100cebf0 2024-01-15T18:23:44.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a16000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7a1015b2d0 0x7f7a100cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60736/0 (socket says 172.21.15.143:60736) 2024-01-15T18:23:44.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a16000640 1 -- 172.21.15.143:0/630167218 learned_addr learned my addr 172.21.15.143:0/630167218 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606911260 0 0) 0x7f7a1015b6b0 con 0x7f7a1015b2d0 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a16a00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 0x7f7a100d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.099+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f0003530 con 0x7f7a1015b2d0 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367909748 0 0) 0x7f79f0003530 con 0x7f7a1015b2d0 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 msgr2=0x7f7a100d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 0x7f7a100d1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 >> v1:172.21.15.204:6789/0 conn(0x7f7a100cfb00 legacy=0x7f7a100b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a100cc3c0 con 0x7f7a1015b2d0 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f7a00001680 con 0x7f7a1015b2d0 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 >> v1:172.21.15.143:6789/0 conn(0x7f7a1015b2d0 legacy=0x7f7a100cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79f00039b0 0x7f79f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 0x7f79f00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0004d10 0x7f79f0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100b0320 con 0x7f79f0004d10 2024-01-15T18:23:44.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f000ba70 con 0x7f79f00039b0 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f000bc30 con 0x7f7a100cebf0 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16000640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79f00039b0 0x7f79f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16a00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0004d10 0x7f79f0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a15600640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 0x7f79f00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16a00640 1 -- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 msgr2=0x7f79f00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16a00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f7a100cebf0 0x7f79f00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16a00640 1 -- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79f00039b0 msgr2=0x7f79f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16a00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79f00039b0 0x7f79f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16a00640 1 -- 172.21.15.143:0/630167218 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0003530 con 0x7f79f0004d10 2024-01-15T18:23:44.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.103+0000 7f7a16000640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f79f00039b0 0x7f79f0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:44.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a16a00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0004d10 0x7f79f0005190 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009f90 tx=0x7f7a0801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5884e67f9894d95 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080a5c30 con 0x7f79f0004d10 2024-01-15T18:23:44.098 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0800f040 con 0x7f79f0004d10 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a14c00640 1 -- 172.21.15.143:0/630167218 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080ad440 con 0x7f79f0004d10 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 -- 172.21.15.143:0/630167218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0004d10 msgr2=0x7f79f0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 --2- 172.21.15.143:0/630167218 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f79f0004d10 0x7f79f0005190 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009f90 tx=0x7f7a0801cc20 comp rx=0 tx=0).stop 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a16a00640 1 -- 172.21.15.143:0/630167218 reap_dead start 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 -- 172.21.15.143:0/630167218 shutdown_connections 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 -- 172.21.15.143:0/630167218 >> 172.21.15.143:0/630167218 conn(0x7f7a100d0e10 msgr2=0x7f7a100d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 -- 172.21.15.143:0/630167218 shutdown_connections 2024-01-15T18:23:44.099 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 -- 172.21.15.143:0/630167218 wait complete. 2024-01-15T18:23:44.101 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 Processor -- start 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 -- start start 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.107+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79f0004d10 0x7f7a10159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 0x7f7a100c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a100c7110 0x7f7a101f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a101597b0 con 0x7f7a100c8800 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a101f3c00 con 0x7f7a100c7110 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1015c950 con 0x7f79f0004d10 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 0x7f7a100c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 0x7f7a100c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38112/0 (socket says 172.21.15.143:38112) 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 -- 172.21.15.143:0/462059527 learned_addr learned my addr 172.21.15.143:0/462059527 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a16a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a100c7110 0x7f7a101f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a16000640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79f0004d10 0x7f7a10159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 -- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79f0004d10 msgr2=0x7f7a10159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79f0004d10 0x7f7a10159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 -- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a100c7110 msgr2=0x7f7a101f2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a100c7110 0x7f7a101f2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 -- 172.21.15.143:0/462059527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1015cc30 con 0x7f7a100c8800 2024-01-15T18:23:44.103 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a16000640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79f0004d10 0x7f7a10159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a16a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a100c7110 0x7f7a101f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a15600640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 0x7f7a100c6bd0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f79f800f5c0 tx=0x7f79f800fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8005320 con 0x7f7a100c8800 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8005480 con 0x7f7a100c8800 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1015cf10 con 0x7f7a100c8800 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8007250 con 0x7f7a100c8800 2024-01-15T18:23:44.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a100c7590 con 0x7f7a100c8800 2024-01-15T18:23:44.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a100ccbc0 con 0x7f7a100c8800 2024-01-15T18:23:44.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.111+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f79f80073b0 con 0x7f7a100c8800 2024-01-15T18:23:44.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.115+0000 7f7a0e000640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79e8067b80 0x7f79e806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.115+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f79f8082a10 con 0x7f7a100c8800 2024-01-15T18:23:44.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.115+0000 7f7a16000640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79e8067b80 0x7f79e806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.115+0000 7f7a16000640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79e8067b80 0x7f79e806a040 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7a00007fd0 tx=0x7f7a00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.119+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f79f8051700 con 0x7f7a100c8800 2024-01-15T18:23:44.307 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.315+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a100cc920 con 0x7f79e8067b80 2024-01-15T18:23:44.308 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.319+0000 7f7a0e000640 1 -- 172.21.15.143:0/462059527 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7a000073b0 con 0x7f79e8067b80 2024-01-15T18:23:44.309 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 0 to deep-scrub 2024-01-15T18:23:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.323+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79e8067b80 msgr2=0x7f79e806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.323+0000 7f7a18a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79e8067b80 0x7f79e806a040 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7a00007fd0 tx=0x7f7a00009040 comp rx=0 tx=0).stop 2024-01-15T18:23:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.323+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 msgr2=0x7f7a100c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.316 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.323+0000 7f7a18a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 0x7f7a100c6bd0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f79f800f5c0 tx=0x7f79f800fa80 comp rx=0 tx=0).stop 2024-01-15T18:23:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 shutdown_connections 2024-01-15T18:23:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f79e8067b80 0x7f79e806a040 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a100c7110 0x7f7a101f2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7a100c8800 0x7f7a100c6bd0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 --2- 172.21.15.143:0/462059527 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f79f0004d10 0x7f7a10159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 >> 172.21.15.143:0/462059527 conn(0x7f7a100d0e10 msgr2=0x7f7a100d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 shutdown_connections 2024-01-15T18:23:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.327+0000 7f7a18a00640 1 -- 172.21.15.143:0/462059527 wait complete. 2024-01-15T18:23:44.390 INFO:tasks.ceph:Scrubbing osd.1 2024-01-15T18:23:44.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:44.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.675+0000 7f60e0400640 1 Processor -- start 2024-01-15T18:23:44.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.675+0000 7f60e0400640 1 -- start start 2024-01-15T18:23:44.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60e0400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60d80ac060 0x7f60d80725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60e0400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d80b3d90 0x7f60d80ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60e0400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80b4200 con 0x7f60d8161b20 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60e0400640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80ae0d0 con 0x7f60d80ac060 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60e0400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80addf0 con 0x7f60d80b3d90 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60d80ac060 0x7f60d80725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60d80ac060 0x7f60d80725b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38116/0 (socket says 172.21.15.143:38116) 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 -- 172.21.15.143:0/1194774491 learned_addr learned my addr 172.21.15.143:0/1194774491 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:44.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60de400640 1 --2- 172.21.15.143:0/1194774491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d80b3d90 0x7f60d80ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 -- 172.21.15.143:0/1194774491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d80b3d90 msgr2=0x7f60d80ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:44.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 --2- 172.21.15.143:0/1194774491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d80b3d90 0x7f60d80ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 -- 172.21.15.143:0/1194774491 >> v1:172.21.15.143:6789/0 conn(0x7f60d8161b20 legacy=0x7f60d80af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.679+0000 7f60dda00640 1 -- 172.21.15.143:0/1194774491 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80ae930 con 0x7f60d80ac060 2024-01-15T18:23:44.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60dda00640 1 --2- 172.21.15.143:0/1194774491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80ac060 0x7f60d80725b0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009870 tx=0x7f60c801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad75d38b6f5dd22 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60d7e00640 1 -- 172.21.15.143:0/1194774491 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c80ab610 con 0x7f60d80ac060 2024-01-15T18:23:44.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60d7e00640 1 -- 172.21.15.143:0/1194774491 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c80ab770 con 0x7f60d80ac060 2024-01-15T18:23:44.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60d7e00640 1 -- 172.21.15.143:0/1194774491 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c80abb00 con 0x7f60d80ac060 2024-01-15T18:23:44.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60e0400640 1 -- 172.21.15.143:0/1194774491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80ac060 msgr2=0x7f60d80725b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60e0400640 1 --2- 172.21.15.143:0/1194774491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80ac060 0x7f60d80725b0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009870 tx=0x7f60c801c480 comp rx=0 tx=0).stop 2024-01-15T18:23:44.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60e0400640 1 -- 172.21.15.143:0/1194774491 shutdown_connections 2024-01-15T18:23:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60e0400640 1 --2- 172.21.15.143:0/1194774491 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d80b3d90 0x7f60d80ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60e0400640 1 --2- 172.21.15.143:0/1194774491 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80ac060 0x7f60d80725b0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.683+0000 7f60e0400640 1 -- 172.21.15.143:0/1194774491 >> 172.21.15.143:0/1194774491 conn(0x7f60d8071a90 msgr2=0x7f60d8071ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.687+0000 7f60e0400640 1 -- 172.21.15.143:0/1194774491 shutdown_connections 2024-01-15T18:23:44.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.687+0000 7f60e0400640 1 -- 172.21.15.143:0/1194774491 wait complete. 2024-01-15T18:23:44.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.687+0000 7f60e0400640 1 Processor -- start 2024-01-15T18:23:44.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.687+0000 7f60e0400640 1 -- start start 2024-01-15T18:23:44.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60e0400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d80ac060 0x7f60d8164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60e0400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80b3d90 0x7f60d815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60e0400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d815cb80 0x7f60d815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60e0400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8157850 con 0x7f60d80b3d90 2024-01-15T18:23:44.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60e0400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8157b30 con 0x7f60d80ac060 2024-01-15T18:23:44.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60e0400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81621d0 con 0x7f60d815cb80 2024-01-15T18:23:44.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d815cb80 0x7f60d815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d815cb80 0x7f60d815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:44980/0 (socket says 172.21.15.143:44980) 2024-01-15T18:23:44.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 -- 172.21.15.143:0/1955935170 learned_addr learned my addr 172.21.15.143:0/1955935170 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:44.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60dda00640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d80ac060 0x7f60d8164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60dd000640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80b3d90 0x7f60d815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 -- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d80ac060 msgr2=0x7f60d8164900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d80ac060 0x7f60d8164900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 -- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80b3d90 msgr2=0x7f60d815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60d80b3d90 0x7f60d815c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60de400640 1 -- 172.21.15.143:0/1955935170 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c80094d0 con 0x7f60d815cb80 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.691+0000 7f60dda00640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60d80ac060 0x7f60d8164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60de400640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d815cb80 0x7f60d815d010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60d0007e40 tx=0x7f60d000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0014620 con 0x7f60d815cb80 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d000ee90 con 0x7f60d815cb80 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d00149e0 con 0x7f60d815cb80 2024-01-15T18:23:44.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d81624b0 con 0x7f60d815cb80 2024-01-15T18:23:44.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d8156970 con 0x7f60d815cb80 2024-01-15T18:23:44.687 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f60d8162050 con 0x7f60d815cb80 2024-01-15T18:23:44.688 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.695+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f60d0014b40 con 0x7f60d815cb80 2024-01-15T18:23:44.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60d5600640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b0067b80 0x7f60b006a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f60d00843a0 con 0x7f60d815cb80 2024-01-15T18:23:44.690 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60d5600640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7f60b006fea0 0x7f60b0072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:44.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 --> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60b00729f0 con 0x7f60b006fea0 2024-01-15T18:23:44.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60d0084710 con 0x7f60d815cb80 2024-01-15T18:23:44.691 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60dd000640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7f60b006fea0 0x7f60b0072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.699+0000 7f60dda00640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b0067b80 0x7f60b006a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:44.692 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.703+0000 7f60dd000640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7f60b006fea0 0x7f60b0072320 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.694 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.703+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== osd.1 v2:172.21.15.143:6800/2952112662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f60b00729f0 con 0x7f60b006fea0 2024-01-15T18:23:44.699 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.707+0000 7f60dda00640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b0067b80 0x7f60b006a040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f60c80af000 tx=0x7f60c80b6040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:44.732 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:44.732 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:44.732 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.739+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 --> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f60d80adba0 con 0x7f60b006fea0 2024-01-15T18:23:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.739+0000 7f60d5600640 1 -- 172.21.15.143:0/1955935170 <== osd.1 v2:172.21.15.143:6800/2952112662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f60c00050e0 con 0x7f60b006fea0 2024-01-15T18:23:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7f60b006fea0 msgr2=0x7f60b0072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6800/2952112662,v1:172.21.15.143:6801/2952112662] conn(0x7f60b006fea0 0x7f60b0072320 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b0067b80 msgr2=0x7f60b006a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f60b0067b80 0x7f60b006a040 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f60c80af000 tx=0x7f60c80b6040 comp rx=0 tx=0).stop 2024-01-15T18:23:44.734 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d815cb80 msgr2=0x7f60d815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:44.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 --2- 172.21.15.143:0/1955935170 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f60d815cb80 0x7f60d815d010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f60d0007e40 tx=0x7f60d000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:23:44.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60de400640 1 -- 172.21.15.143:0/1955935170 reap_dead start 2024-01-15T18:23:44.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 shutdown_connections 2024-01-15T18:23:44.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 >> 172.21.15.143:0/1955935170 conn(0x7f60d8071a90 msgr2=0x7f60d806ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:44.735 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 shutdown_connections 2024-01-15T18:23:44.736 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:44.743+0000 7f60e0400640 1 -- 172.21.15.143:0/1955935170 wait complete. 2024-01-15T18:23:44.789 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-15T18:23:45.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.111+0000 7f492b200640 1 Processor -- start 2024-01-15T18:23:45.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.111+0000 7f492b200640 1 -- start start 2024-01-15T18:23:45.105 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f492b200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f49241614e0 0x7f492406d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f492b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 0x7f49240ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f492b200640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49240cda20 con 0x7f49240caf20 2024-01-15T18:23:45.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f492b200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49240cdd00 con 0x7f49241614e0 2024-01-15T18:23:45.106 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f492b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49240cdfe0 con 0x7f4924165ec0 2024-01-15T18:23:45.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f4923400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 0x7f49240ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f4923400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 0x7f49240ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:44996/0 (socket says 172.21.15.143:44996) 2024-01-15T18:23:45.108 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.115+0000 7f4923400640 1 -- 172.21.15.143:0/3746816387 learned_addr learned my addr 172.21.15.143:0/3746816387 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:45.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.119+0000 7f4922a00640 1 -- 172.21.15.143:0/3746816387 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495628499 0 0) 0x7f49240cda20 con 0x7f49240caf20 2024-01-15T18:23:45.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.119+0000 7f4922a00640 1 -- 172.21.15.143:0/3746816387 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4904003530 con 0x7f49240caf20 2024-01-15T18:23:45.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.119+0000 7f4923400640 1 -- 172.21.15.143:0/3746816387 >> v2:172.21.15.143:3300/0 conn(0x7f49241614e0 msgr2=0x7f492406d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:45.109 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.119+0000 7f4923400640 1 --2- 172.21.15.143:0/3746816387 >> v2:172.21.15.143:3300/0 conn(0x7f49241614e0 0x7f492406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.119+0000 7f4923400640 1 -- 172.21.15.143:0/3746816387 >> v1:172.21.15.143:6789/0 conn(0x7f49240caf20 legacy=0x7f492406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.110 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.119+0000 7f4923400640 1 -- 172.21.15.143:0/3746816387 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49240ce2c0 con 0x7f4924165ec0 2024-01-15T18:23:45.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.123+0000 7f4923400640 1 --2- 172.21.15.143:0/3746816387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 0x7f49240ca9e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f490c009870 tx=0x7f490c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ed42f35fdb7139 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.123+0000 7f4922a00640 1 -- 172.21.15.143:0/3746816387 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0a6040 con 0x7f4924165ec0 2024-01-15T18:23:45.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.123+0000 7f4922a00640 1 -- 172.21.15.143:0/3746816387 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0aa600 con 0x7f4924165ec0 2024-01-15T18:23:45.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.123+0000 7f4922a00640 1 -- 172.21.15.143:0/3746816387 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0aa990 con 0x7f4924165ec0 2024-01-15T18:23:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 -- 172.21.15.143:0/3746816387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 msgr2=0x7f49240ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 --2- 172.21.15.143:0/3746816387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 0x7f49240ca9e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f490c009870 tx=0x7f490c01c4a0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 -- 172.21.15.143:0/3746816387 shutdown_connections 2024-01-15T18:23:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 --2- 172.21.15.143:0/3746816387 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4924165ec0 0x7f49240ca9e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 --2- 172.21.15.143:0/3746816387 >> v2:172.21.15.143:3300/0 conn(0x7f49241614e0 0x7f492406d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 -- 172.21.15.143:0/3746816387 >> 172.21.15.143:0/3746816387 conn(0x7f49240cfd20 msgr2=0x7f49240d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:45.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 -- 172.21.15.143:0/3746816387 shutdown_connections 2024-01-15T18:23:45.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.127+0000 7f492b200640 1 -- 172.21.15.143:0/3746816387 wait complete. 2024-01-15T18:23:45.122 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.131+0000 7f492b200640 1 Processor -- start 2024-01-15T18:23:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.131+0000 7f492b200640 1 -- start start 2024-01-15T18:23:45.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f492b200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49240caf20 0x7f49241562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f492b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 0x7f49241567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f492b200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4924165ec0 0x7f49241587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f492b200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924159ce0 con 0x7f4924165ec0 2024-01-15T18:23:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f492b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924159fc0 con 0x7f49241614e0 2024-01-15T18:23:45.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f492b200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492415a2a0 con 0x7f49240caf20 2024-01-15T18:23:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 0x7f49241567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 0x7f49241567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:44190/0 (socket says 172.21.15.143:44190) 2024-01-15T18:23:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 -- 172.21.15.143:0/33346248 learned_addr learned my addr 172.21.15.143:0/33346248 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4929200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4924165ec0 0x7f49241587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 -- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49240caf20 msgr2=0x7f49241562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49240caf20 0x7f49241562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49240caf20 0x7f49241562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 -- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4924165ec0 msgr2=0x7f49241587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4924165ec0 0x7f49241587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923400640 1 -- 172.21.15.143:0/33346248 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0094d0 con 0x7f49241614e0 2024-01-15T18:23:45.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4929200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4924165ec0 0x7f49241587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:45.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.135+0000 7f4923e00640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49240caf20 0x7f49241562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:45.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.139+0000 7f4923400640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 0x7f49241567e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f490c01ab50 tx=0x7f490c019240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.139+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0aadc0 con 0x7f49241614e0 2024-01-15T18:23:45.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.139+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0b7070 con 0x7f49241614e0 2024-01-15T18:23:45.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.139+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0bd640 con 0x7f49241614e0 2024-01-15T18:23:45.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.143+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492415a580 con 0x7f49241614e0 2024-01-15T18:23:45.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.143+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492406dff0 con 0x7f49241614e0 2024-01-15T18:23:45.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.147+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f490c0c53f0 con 0x7f49241614e0 2024-01-15T18:23:45.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.147+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492415a710 con 0x7f49241614e0 2024-01-15T18:23:45.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.147+0000 7f491be00640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48fc067bd0 0x7f48fc06a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.147+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f490c11cf50 con 0x7f49241614e0 2024-01-15T18:23:45.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.151+0000 7f4923e00640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48fc067bd0 0x7f48fc06a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.141 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.151+0000 7f4923e00640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48fc067bd0 0x7f48fc06a090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f492415b790 tx=0x7f4914008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.144 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.155+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f490c0ebbe0 con 0x7f49241614e0 2024-01-15T18:23:45.437 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.447+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f492406adf0 con 0x7f48fc067bd0 2024-01-15T18:23:45.438 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.447+0000 7f491be00640 1 -- 172.21.15.143:0/33346248 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f491400a4a0 con 0x7f48fc067bd0 2024-01-15T18:23:45.438 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 1 to deep-scrub 2024-01-15T18:23:45.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48fc067bd0 msgr2=0x7f48fc06a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.441 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48fc067bd0 0x7f48fc06a090 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f492415b790 tx=0x7f4914008040 comp rx=0 tx=0).stop 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 msgr2=0x7f49241567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 0x7f49241567e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f490c01ab50 tx=0x7f490c019240 comp rx=0 tx=0).stop 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 shutdown_connections 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f48fc067bd0 0x7f48fc06a090 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4924165ec0 0x7f49241587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f49241614e0 0x7f49241567e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 --2- 172.21.15.143:0/33346248 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f49240caf20 0x7f49241562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 >> 172.21.15.143:0/33346248 conn(0x7f49240cfd20 msgr2=0x7f492415f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:45.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 shutdown_connections 2024-01-15T18:23:45.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.451+0000 7f492b200640 1 -- 172.21.15.143:0/33346248 wait complete. 2024-01-15T18:23:45.472 INFO:tasks.ceph:Scrubbing osd.2 2024-01-15T18:23:45.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:45.728 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 Processor -- start 2024-01-15T18:23:45.729 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 -- start start 2024-01-15T18:23:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fb4600b3920 0x7fb46015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb46006df80 0x7fb4600b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4600b2cd0 con 0x7fb46006b830 2024-01-15T18:23:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46006db60 con 0x7fb46006df80 2024-01-15T18:23:45.731 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb465e00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46006d200 con 0x7fb4600b3920 2024-01-15T18:23:45.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45ea00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb46006b830 0x7fb4600b3210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60756/0 (socket says 172.21.15.143:60756) 2024-01-15T18:23:45.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45ea00640 1 -- 172.21.15.143:0/1253037225 learned_addr learned my addr 172.21.15.143:0/1253037225 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:45.732 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45f400640 1 --2- 172.21.15.143:0/1253037225 >> v2:172.21.15.143:3300/0 conn(0x7fb46006df80 0x7fb4600b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45d600640 1 -- 172.21.15.143:0/1253037225 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715128257 0 0) 0x7fb4600b2cd0 con 0x7fb46006b830 2024-01-15T18:23:45.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45d600640 1 -- 172.21.15.143:0/1253037225 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb440003530 con 0x7fb46006b830 2024-01-15T18:23:45.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45f400640 1 -- 172.21.15.143:0/1253037225 >> v2:172.21.15.204:3300/0 conn(0x7fb4600b3920 msgr2=0x7fb46015a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:45.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45f400640 1 --2- 172.21.15.143:0/1253037225 >> v2:172.21.15.204:3300/0 conn(0x7fb4600b3920 0x7fb46015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45f400640 1 -- 172.21.15.143:0/1253037225 >> v1:172.21.15.143:6789/0 conn(0x7fb46006b830 legacy=0x7fb4600b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.733 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.739+0000 7fb45f400640 1 -- 172.21.15.143:0/1253037225 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600b2f30 con 0x7fb46006df80 2024-01-15T18:23:45.737 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.747+0000 7fb45f400640 1 --2- 172.21.15.143:0/1253037225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006df80 0x7fb4600b2790 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b2060 tx=0x7fb45801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7680c4dba5f6fbbf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.747+0000 7fb45d600640 1 -- 172.21.15.143:0/1253037225 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580a5890 con 0x7fb46006df80 2024-01-15T18:23:45.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.747+0000 7fb45d600640 1 -- 172.21.15.143:0/1253037225 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580a59f0 con 0x7fb46006df80 2024-01-15T18:23:45.738 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.747+0000 7fb45d600640 1 -- 172.21.15.143:0/1253037225 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580a5d60 con 0x7fb46006df80 2024-01-15T18:23:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 -- 172.21.15.143:0/1253037225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006df80 msgr2=0x7fb4600b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 --2- 172.21.15.143:0/1253037225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006df80 0x7fb4600b2790 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fb4600b2060 tx=0x7fb45801c850 comp rx=0 tx=0).stop 2024-01-15T18:23:45.742 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 -- 172.21.15.143:0/1253037225 shutdown_connections 2024-01-15T18:23:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 --2- 172.21.15.143:0/1253037225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006df80 0x7fb4600b2790 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 --2- 172.21.15.143:0/1253037225 >> v2:172.21.15.204:3300/0 conn(0x7fb4600b3920 0x7fb46015a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 -- 172.21.15.143:0/1253037225 >> 172.21.15.143:0/1253037225 conn(0x7fb4600b1a90 msgr2=0x7fb4600cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:45.743 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 -- 172.21.15.143:0/1253037225 shutdown_connections 2024-01-15T18:23:45.744 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.751+0000 7fb465e00640 1 -- 172.21.15.143:0/1253037225 wait complete. 2024-01-15T18:23:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.755+0000 7fb465e00640 1 Processor -- start 2024-01-15T18:23:45.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 -- start start 2024-01-15T18:23:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006b830 0x7fb460147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46006df80 0x7fb460133550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4600b3920 0x7fb460149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.750 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460130b00 con 0x7fb46006b830 2024-01-15T18:23:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460130d10 con 0x7fb46006df80 2024-01-15T18:23:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb465e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb460130ff0 con 0x7fb4600b3920 2024-01-15T18:23:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb45ea00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006b830 0x7fb460147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb45f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4600b3920 0x7fb460149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb45f400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4600b3920 0x7fb460149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:45008/0 (socket says 172.21.15.143:45008) 2024-01-15T18:23:45.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb45e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46006df80 0x7fb460133550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.759+0000 7fb45f400640 1 -- 172.21.15.143:0/3750262665 learned_addr learned my addr 172.21.15.143:0/3750262665 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45f400640 1 -- 172.21.15.143:0/3750262665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46006df80 msgr2=0x7fb460133550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45f400640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46006df80 0x7fb460133550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45f400640 1 -- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006b830 msgr2=0x7fb460147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45f400640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb46006b830 0x7fb460147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45f400640 1 -- 172.21.15.143:0/3750262665 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458009a70 con 0x7fb4600b3920 2024-01-15T18:23:45.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45e000640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb46006df80 0x7fb460133550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:45.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb45f400640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4600b3920 0x7fb460149820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb458019460 tx=0x7fb4580195f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580ad850 con 0x7fb4600b3920 2024-01-15T18:23:45.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4580ad9b0 con 0x7fb4600b3920 2024-01-15T18:23:45.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4580add20 con 0x7fb4600b3920 2024-01-15T18:23:45.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4601312d0 con 0x7fb4600b3920 2024-01-15T18:23:45.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb460131890 con 0x7fb4600b3920 2024-01-15T18:23:45.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.763+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb46006a490 con 0x7fb4600b3920 2024-01-15T18:23:45.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== mon.2 v2:172.21.15.204:3301/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fb4580b4070 con 0x7fb4600b3920 2024-01-15T18:23:45.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb457400640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb438067b80 0x7fb43806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== mon.2 v2:172.21.15.204:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fb45811e0f0 con 0x7fb4600b3920 2024-01-15T18:23:45.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb457400640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb43806fea0 0x7fb438072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:45.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 --> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4380729f0 con 0x7fb43806fea0 2024-01-15T18:23:45.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== mon.2 v2:172.21.15.204:3301/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb458123c70 con 0x7fb4600b3920 2024-01-15T18:23:45.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.767+0000 7fb45e000640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb43806fea0 0x7fb438072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.762 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.771+0000 7fb45ea00640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb438067b80 0x7fb43806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:45.765 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.771+0000 7fb45e000640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb43806fea0 0x7fb438072320 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.779+0000 7fb45ea00640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb438067b80 0x7fb43806a040 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4600ce900 tx=0x7fb45000d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:45.769 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.779+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== osd.2 v2:172.21.15.143:6816/761026363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb4380729f0 con 0x7fb43806fea0 2024-01-15T18:23:45.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.803+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 --> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb460067c10 con 0x7fb43806fea0 2024-01-15T18:23:45.796 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:45.797 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:45.797 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb457400640 1 -- 172.21.15.143:0/3750262665 <== osd.2 v2:172.21.15.143:6816/761026363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb460067c10 con 0x7fb43806fea0 2024-01-15T18:23:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb43806fea0 msgr2=0x7fb438072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6816/761026363,v1:172.21.15.143:6817/761026363] conn(0x7fb43806fea0 0x7fb438072320 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb438067b80 msgr2=0x7fb43806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fb438067b80 0x7fb43806a040 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb4600ce900 tx=0x7fb45000d040 comp rx=0 tx=0).stop 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4600b3920 msgr2=0x7fb460149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 --2- 172.21.15.143:0/3750262665 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fb4600b3920 0x7fb460149820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb458019460 tx=0x7fb4580195f0 comp rx=0 tx=0).stop 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb45f400640 1 -- 172.21.15.143:0/3750262665 reap_dead start 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 shutdown_connections 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 >> 172.21.15.143:0/3750262665 conn(0x7fb4600b1a90 msgr2=0x7fb460164b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 shutdown_connections 2024-01-15T18:23:45.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:45.807+0000 7fb465e00640 1 -- 172.21.15.143:0/3750262665 wait complete. 2024-01-15T18:23:45.864 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-15T18:23:46.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.087+0000 7f1093a00640 1 Processor -- start 2024-01-15T18:23:46.079 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.087+0000 7f1093a00640 1 -- start start 2024-01-15T18:23:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1093a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f108c15b610 0x7f108c140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1093a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c071920 0x7f108c071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1093a00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c140e00 con 0x7f108c06a490 2024-01-15T18:23:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1093a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c13e420 con 0x7f108c15b610 2024-01-15T18:23:46.081 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1093a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c1416a0 con 0x7f108c071920 2024-01-15T18:23:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f108c15b610 0x7f108c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091a00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f108c06a490 0x7f108c15b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60770/0 (socket says 172.21.15.143:60770) 2024-01-15T18:23:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091a00640 1 -- 172.21.15.143:0/3822659241 learned_addr learned my addr 172.21.15.143:0/3822659241 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f108be00640 1 --2- 172.21.15.143:0/3822659241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c071920 0x7f108c071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091000640 1 -- 172.21.15.143:0/3822659241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c071920 msgr2=0x7f108c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:46.082 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091000640 1 --2- 172.21.15.143:0/3822659241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c071920 0x7f108c071db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091000640 1 -- 172.21.15.143:0/3822659241 >> v1:172.21.15.143:6789/0 conn(0x7f108c06a490 legacy=0x7f108c15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.083 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.091+0000 7f1091000640 1 -- 172.21.15.143:0/3822659241 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c1410e0 con 0x7f108c15b610 2024-01-15T18:23:46.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1091000640 1 --2- 172.21.15.143:0/3822659241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c15b610 0x7f108c140700 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f107c0097c0 tx=0x7f107c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b98c28285b35bf5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f108b400640 1 -- 172.21.15.143:0/3822659241 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0ad640 con 0x7f108c15b610 2024-01-15T18:23:46.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f108b400640 1 -- 172.21.15.143:0/3822659241 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c0ad7a0 con 0x7f108c15b610 2024-01-15T18:23:46.085 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f108b400640 1 -- 172.21.15.143:0/3822659241 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c0adb30 con 0x7f108c15b610 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 -- 172.21.15.143:0/3822659241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c15b610 msgr2=0x7f108c140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 --2- 172.21.15.143:0/3822659241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c15b610 0x7f108c140700 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f107c0097c0 tx=0x7f107c01c480 comp rx=0 tx=0).stop 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 -- 172.21.15.143:0/3822659241 shutdown_connections 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 --2- 172.21.15.143:0/3822659241 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c071920 0x7f108c071db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 --2- 172.21.15.143:0/3822659241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c15b610 0x7f108c140700 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 -- 172.21.15.143:0/3822659241 >> 172.21.15.143:0/3822659241 conn(0x7f108c0b1b90 msgr2=0x7f108c0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.086 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 -- 172.21.15.143:0/3822659241 shutdown_connections 2024-01-15T18:23:46.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 -- 172.21.15.143:0/3822659241 wait complete. 2024-01-15T18:23:46.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 Processor -- start 2024-01-15T18:23:46.087 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.095+0000 7f1093a00640 1 -- start start 2024-01-15T18:23:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c06a490 0x7f108c1377e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 0x7f108c06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.089 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f108c15b610 0x7f108c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c06e830 con 0x7f108c071920 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c13c0e0 con 0x7f108c15b610 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c06dbb0 con 0x7f108c06a490 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 0x7f108c06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 0x7f108c06d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38168/0 (socket says 172.21.15.143:38168) 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 -- 172.21.15.143:0/2369847716 learned_addr learned my addr 172.21.15.143:0/2369847716 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1091000640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c06a490 0x7f108c1377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1091a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f108c15b610 0x7f108c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.090 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 -- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c06a490 msgr2=0x7f108c1377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c06a490 0x7f108c1377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 -- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f108c15b610 msgr2=0x7f108c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f108c15b610 0x7f108c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 -- 172.21.15.143:0/2369847716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c009420 con 0x7f108c071920 2024-01-15T18:23:46.091 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f108be00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 0x7f108c06d4b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f107400e930 tx=0x7f107400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074009690 con 0x7f108c071920 2024-01-15T18:23:46.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107400ca00 con 0x7f108c071920 2024-01-15T18:23:46.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400cd70 con 0x7f108c071920 2024-01-15T18:23:46.092 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c06de90 con 0x7f108c071920 2024-01-15T18:23:46.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c06e450 con 0x7f108c071920 2024-01-15T18:23:46.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c067500 con 0x7f108c071920 2024-01-15T18:23:46.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.099+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f1074013470 con 0x7f108c071920 2024-01-15T18:23:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.103+0000 7f1088c00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1064067b80 0x7f106406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.103+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f10740832b0 con 0x7f108c071920 2024-01-15T18:23:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.103+0000 7f1091000640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1064067b80 0x7f106406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.103+0000 7f1091000640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1064067b80 0x7f106406a040 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f107c0af000 tx=0x7f107c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.107+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f1074052000 con 0x7f108c071920 2024-01-15T18:23:46.248 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f108c140a00 con 0x7f1064067b80 2024-01-15T18:23:46.249 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1088c00640 1 -- 172.21.15.143:0/2369847716 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f108c140a00 con 0x7f1064067b80 2024-01-15T18:23:46.249 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 2 to deep-scrub 2024-01-15T18:23:46.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1064067b80 msgr2=0x7f106406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1064067b80 0x7f106406a040 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f107c0af000 tx=0x7f107c008040 comp rx=0 tx=0).stop 2024-01-15T18:23:46.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 msgr2=0x7f108c06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.251 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 0x7f108c06d4b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f107400e930 tx=0x7f107400edf0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 shutdown_connections 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1064067b80 0x7f106406a040 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f108c15b610 0x7f108c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f108c071920 0x7f108c06d4b0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 --2- 172.21.15.143:0/2369847716 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f108c06a490 0x7f108c1377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 >> 172.21.15.143:0/2369847716 conn(0x7f108c0b1b90 msgr2=0x7f108c0b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 shutdown_connections 2024-01-15T18:23:46.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.259+0000 7f1093a00640 1 -- 172.21.15.143:0/2369847716 wait complete. 2024-01-15T18:23:46.271 INFO:tasks.ceph:Scrubbing osd.3 2024-01-15T18:23:46.272 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:46.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.419+0000 7fd794e00640 1 Processor -- start 2024-01-15T18:23:46.412 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd794e00640 1 -- start start 2024-01-15T18:23:46.413 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd794e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd790157300 0x7fd790159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd794e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd794e00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7900b2a20 con 0x7fd7900b48f0 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd794e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7900b35a0 con 0x7fd790157300 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd794e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900b4d80 con 0x7fd7900b21d0 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd790157300 0x7fd790159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd790157300 0x7fd790159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38182/0 (socket says 172.21.15.143:38182) 2024-01-15T18:23:46.414 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 -- 172.21.15.143:0/1226925078 learned_addr learned my addr 172.21.15.143:0/1226925078 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 -- 172.21.15.143:0/1226925078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 msgr2=0x7fd7900b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 --2- 172.21.15.143:0/1226925078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 -- 172.21.15.143:0/1226925078 >> v1:172.21.15.204:6789/0 conn(0x7fd7900b48f0 legacy=0x7fd79015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:46.415 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.423+0000 7fd78de00640 1 -- 172.21.15.143:0/1226925078 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7900b2d00 con 0x7fd790157300 2024-01-15T18:23:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd78de00640 1 --2- 172.21.15.143:0/1226925078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd790159b70 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd780009870 tx=0x7fd78001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862aa18098f9c5cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd78ca00640 1 -- 172.21.15.143:0/1226925078 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7800aa5f0 con 0x7fd790157300 2024-01-15T18:23:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd78ca00640 1 -- 172.21.15.143:0/1226925078 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7800aa750 con 0x7fd790157300 2024-01-15T18:23:46.417 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd78ca00640 1 -- 172.21.15.143:0/1226925078 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7800aaae0 con 0x7fd790157300 2024-01-15T18:23:46.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 -- 172.21.15.143:0/1226925078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 msgr2=0x7fd790159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 --2- 172.21.15.143:0/1226925078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd790159b70 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd780009870 tx=0x7fd78001c4d0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.418 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 -- 172.21.15.143:0/1226925078 shutdown_connections 2024-01-15T18:23:46.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 --2- 172.21.15.143:0/1226925078 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 --2- 172.21.15.143:0/1226925078 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd790159b70 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 -- 172.21.15.143:0/1226925078 >> 172.21.15.143:0/1226925078 conn(0x7fd79006c3f0 msgr2=0x7fd79006c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 -- 172.21.15.143:0/1226925078 shutdown_connections 2024-01-15T18:23:46.419 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 -- 172.21.15.143:0/1226925078 wait complete. 2024-01-15T18:23:46.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.427+0000 7fd794e00640 1 Processor -- start 2024-01-15T18:23:46.420 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- start start 2024-01-15T18:23:46.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.421 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7900b48f0 0x7fd79014a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd7901f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900ace20 con 0x7fd790157300 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900ad0c0 con 0x7fd7900b48f0 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7900ad3a0 con 0x7fd7900b21d0 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd7901f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7900b48f0 0x7fd79014a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.422 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78de00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3301/0 says I am v2:172.21.15.143:45050/0 (socket says 172.21.15.143:45050) 2024-01-15T18:23:46.423 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78de00640 1 -- 172.21.15.143:0/4082254060 learned_addr learned my addr 172.21.15.143:0/4082254060 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 -- 172.21.15.143:0/4082254060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 msgr2=0x7fd7900ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 -- 172.21.15.143:0/4082254060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7900b48f0 msgr2=0x7fd79014a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7900b48f0 0x7fd79014a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 -- 172.21.15.143:0/4082254060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7800094d0 con 0x7fd790157300 2024-01-15T18:23:46.424 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78de00640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd7900b21d0 0x7fd7900ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78d400640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7900b48f0 0x7fd79014a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd78e800640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd7901f6220 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c780 tx=0x7fd78800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800a840 con 0x7fd790157300 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800a9a0 con 0x7fd790157300 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ad90 con 0x7fd790157300 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7900ad680 con 0x7fd790157300 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7900adc40 con 0x7fd790157300 2024-01-15T18:23:46.425 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.431+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd79006e340 con 0x7fd790157300 2024-01-15T18:23:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fd788012420 con 0x7fd790157300 2024-01-15T18:23:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd768067b80 0x7fd76806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd788083380 con 0x7fd790157300 2024-01-15T18:23:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd78de00640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd768067b80 0x7fd76806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7fd76806fea0 0x7fd768072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.427 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 --> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7680729f0 con 0x7fd76806fea0 2024-01-15T18:23:46.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd788083f10 con 0x7fd790157300 2024-01-15T18:23:46.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd78d400640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7fd76806fea0 0x7fd768072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd78de00640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd768067b80 0x7fd76806a040 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd78001c510 tx=0x7fd7800b5040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd78d400640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7fd76806fea0 0x7fd768072320 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.428 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.435+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== osd.3 v2:172.21.15.143:6822/254985013 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd7680729f0 con 0x7fd76806fea0 2024-01-15T18:23:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.447+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 --> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd7900ade30 con 0x7fd76806fea0 2024-01-15T18:23:46.440 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd786000640 1 -- 172.21.15.143:0/4082254060 <== osd.3 v2:172.21.15.143:6822/254985013 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd7900ade30 con 0x7fd76806fea0 2024-01-15T18:23:46.441 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:46.441 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:46.441 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7fd76806fea0 msgr2=0x7fd768072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6822/254985013,v1:172.21.15.143:6824/254985013] conn(0x7fd76806fea0 0x7fd768072320 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd768067b80 msgr2=0x7fd76806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd768067b80 0x7fd76806a040 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fd78001c510 tx=0x7fd7800b5040 comp rx=0 tx=0).stop 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 msgr2=0x7fd7901f6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 --2- 172.21.15.143:0/4082254060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd790157300 0x7fd7901f6220 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd78800c780 tx=0x7fd78800cc40 comp rx=0 tx=0).stop 2024-01-15T18:23:46.442 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd78e800640 1 -- 172.21.15.143:0/4082254060 reap_dead start 2024-01-15T18:23:46.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 shutdown_connections 2024-01-15T18:23:46.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 >> 172.21.15.143:0/4082254060 conn(0x7fd79006c3f0 msgr2=0x7fd7901565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 shutdown_connections 2024-01-15T18:23:46.443 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.451+0000 7fd794e00640 1 -- 172.21.15.143:0/4082254060 wait complete. 2024-01-15T18:23:46.460 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-15T18:23:46.598 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c21c00640 1 Processor -- start 2024-01-15T18:23:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c21c00640 1 -- start start 2024-01-15T18:23:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c21c00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c1c1607c0 0x7f4c1c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c21c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0cffd0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c21c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1c0d02b0 con 0x7f4c1c0cbf50 2024-01-15T18:23:46.599 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c21c00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0d1ce0 con 0x7f4c1c1607c0 2024-01-15T18:23:46.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c1a000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4c1c0cfb30 0x7f4c1c06bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60786/0 (socket says 172.21.15.143:60786) 2024-01-15T18:23:46.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c1a000640 1 -- 172.21.15.143:0/409041419 learned_addr learned my addr 172.21.15.143:0/409041419 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:46.600 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c1aa00640 1 --2- 172.21.15.143:0/409041419 >> v2:172.21.15.143:3300/0 conn(0x7f4c1c1607c0 0x7f4c1c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814854284 0 0) 0x7f4c1c0cffd0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bfc003530 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477748142 0 0) 0x7f4c1c0d02b0 con 0x7f4c1c0cbf50 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c1c0cffd0 con 0x7f4c1c0cbf50 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c1aa00640 1 -- 172.21.15.143:0/409041419 >> v1:172.21.15.204:6789/0 conn(0x7f4c1c0cbf50 legacy=0x7f4c1c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c1aa00640 1 -- 172.21.15.143:0/409041419 >> v1:172.21.15.143:6789/0 conn(0x7f4c1c0cfb30 legacy=0x7f4c1c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.601 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.607+0000 7f4c1aa00640 1 -- 172.21.15.143:0/409041419 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c0cb470 con 0x7f4c1c1607c0 2024-01-15T18:23:46.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c1aa00640 1 --2- 172.21.15.143:0/409041419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c06b7b0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0097c0 tx=0x7f4c0c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607ecdc4c7f4b386 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0a8970 con 0x7f4c1c1607c0 2024-01-15T18:23:46.602 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0c0a8ad0 con 0x7f4c1c1607c0 2024-01-15T18:23:46.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c19600640 1 -- 172.21.15.143:0/409041419 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0a8e40 con 0x7f4c1c1607c0 2024-01-15T18:23:46.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 -- 172.21.15.143:0/409041419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 msgr2=0x7f4c1c06b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.603 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 --2- 172.21.15.143:0/409041419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c06b7b0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0097c0 tx=0x7f4c0c01ec70 comp rx=0 tx=0).stop 2024-01-15T18:23:46.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 -- 172.21.15.143:0/409041419 shutdown_connections 2024-01-15T18:23:46.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 --2- 172.21.15.143:0/409041419 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c06b7b0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 -- 172.21.15.143:0/409041419 >> 172.21.15.143:0/409041419 conn(0x7f4c1c0d0e20 msgr2=0x7f4c1c0d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 -- 172.21.15.143:0/409041419 shutdown_connections 2024-01-15T18:23:46.604 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.611+0000 7f4c21c00640 1 -- 172.21.15.143:0/409041419 wait complete. 2024-01-15T18:23:46.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 Processor -- start 2024-01-15T18:23:46.605 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 -- start start 2024-01-15T18:23:46.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c1c0cbf50 0x7f4c1c0c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 0x7f4c1c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.606 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c5e10 con 0x7f4c1c1607c0 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c6000 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c62e0 con 0x7f4c1c0cbf50 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 0x7f4c1c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 0x7f4c1c0c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.143:44230/0 (socket says 172.21.15.143:44230) 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1b400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38200/0 (socket says 172.21.15.143:38200) 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 -- 172.21.15.143:0/3566920212 learned_addr learned my addr 172.21.15.143:0/3566920212 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:46.607 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1aa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c1c0cbf50 0x7f4c1c0c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 -- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c1c0cbf50 msgr2=0x7f4c1c0c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c1c0cbf50 0x7f4c1c0c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 -- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 msgr2=0x7f4c1c147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 -- 172.21.15.143:0/3566920212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c009420 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1b400640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:46.609 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c1a000640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 0x7f4c1c0c58d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c4d0 tx=0x7f4c0400c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04005cf0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c04005e50 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c04007270 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c0c65c0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.615+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c153ea0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.610 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.619+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c06d4b0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.611 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.619+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f4c040073d0 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.619+0000 7f4c12a00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4bf4067bb0 0x7f4bf406a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.619+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f4c04082840 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.619+0000 7f4c1aa00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4bf4067bb0 0x7f4bf406a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.612 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.619+0000 7f4c1aa00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4bf4067bb0 0x7f4bf406a070 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0b2000 tx=0x7f4c0c0aa070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.614 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.623+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4c04051200 con 0x7f4c1c0cfb30 2024-01-15T18:23:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c1c0ce6b0 con 0x7f4bf4067bb0 2024-01-15T18:23:46.768 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c12a00640 1 -- 172.21.15.143:0/3566920212 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4c0c0c0090 con 0x7f4bf4067bb0 2024-01-15T18:23:46.769 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 3 to deep-scrub 2024-01-15T18:23:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4bf4067bb0 msgr2=0x7f4bf406a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4bf4067bb0 0x7f4bf406a070 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0b2000 tx=0x7f4c0c0aa070 comp rx=0 tx=0).stop 2024-01-15T18:23:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 msgr2=0x7f4c1c0c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 0x7f4c1c0c58d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4c0400c4d0 tx=0x7f4c0400c990 comp rx=0 tx=0).stop 2024-01-15T18:23:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 shutdown_connections 2024-01-15T18:23:46.771 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4bf4067bb0 0x7f4bf406a070 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c1c1607c0 0x7f4c1c147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4c1c0cfb30 0x7f4c1c0c58d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 --2- 172.21.15.143:0/3566920212 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4c1c0cbf50 0x7f4c1c0c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 >> 172.21.15.143:0/3566920212 conn(0x7f4c1c0d0e20 msgr2=0x7f4c1c15e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 shutdown_connections 2024-01-15T18:23:46.772 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.779+0000 7f4c21c00640 1 -- 172.21.15.143:0/3566920212 wait complete. 2024-01-15T18:23:46.792 INFO:tasks.ceph:Scrubbing osd.4 2024-01-15T18:23:46.792 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 Processor -- start 2024-01-15T18:23:46.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 -- start start 2024-01-15T18:23:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1aec141660 0x7f1aec0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec071d00 0x7f1aec072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec15f310 con 0x7f1aec15ec80 2024-01-15T18:23:46.936 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec15fbb0 con 0x7f1aec141660 2024-01-15T18:23:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15f130 con 0x7f1aec071d00 2024-01-15T18:23:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1aec141660 0x7f1aec0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.943+0000 7f1af1000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1aec141660 0x7f1aec0ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38210/0 (socket says 172.21.15.143:38210) 2024-01-15T18:23:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1000640 1 -- 172.21.15.143:0/3562336424 learned_addr learned my addr 172.21.15.143:0/3562336424 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:46.937 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1a00640 1 --2- 172.21.15.143:0/3562336424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec071d00 0x7f1aec072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1000640 1 -- 172.21.15.143:0/3562336424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec071d00 msgr2=0x7f1aec072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1000640 1 --2- 172.21.15.143:0/3562336424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec071d00 0x7f1aec072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1000640 1 -- 172.21.15.143:0/3562336424 >> v1:172.21.15.204:6789/0 conn(0x7f1aec15ec80 legacy=0x7f1aec141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1000640 1 -- 172.21.15.143:0/3562336424 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec15f5f0 con 0x7f1aec141660 2024-01-15T18:23:46.938 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1aeb400640 1 -- 172.21.15.143:0/3562336424 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637118759 0 0) 0x7f1aec15f310 con 0x7f1aec15ec80 2024-01-15T18:23:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1af1000640 1 --2- 172.21.15.143:0/3562336424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec141660 0x7f1aec0ce560 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009870 tx=0x7f1adc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e9f548f827fa81 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.939 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1aeb400640 1 -- 172.21.15.143:0/3562336424 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0a7070 con 0x7f1aec141660 2024-01-15T18:23:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1aeb400640 1 -- 172.21.15.143:0/3562336424 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc0ab430 con 0x7f1aec141660 2024-01-15T18:23:46.940 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.947+0000 7f1aeb400640 1 -- 172.21.15.143:0/3562336424 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0ab7c0 con 0x7f1aec141660 2024-01-15T18:23:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- 172.21.15.143:0/3562336424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec141660 msgr2=0x7f1aec0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 --2- 172.21.15.143:0/3562336424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec141660 0x7f1aec0ce560 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009870 tx=0x7f1adc01c540 comp rx=0 tx=0).stop 2024-01-15T18:23:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- 172.21.15.143:0/3562336424 shutdown_connections 2024-01-15T18:23:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 --2- 172.21.15.143:0/3562336424 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec071d00 0x7f1aec072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 --2- 172.21.15.143:0/3562336424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec141660 0x7f1aec0ce560 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.941 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- 172.21.15.143:0/3562336424 >> 172.21.15.143:0/3562336424 conn(0x7f1aec06c3f0 msgr2=0x7f1aec06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- 172.21.15.143:0/3562336424 shutdown_connections 2024-01-15T18:23:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- 172.21.15.143:0/3562336424 wait complete. 2024-01-15T18:23:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 Processor -- start 2024-01-15T18:23:46.942 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- start start 2024-01-15T18:23:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec071d00 0x7f1aec13b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec141660 0x7f1aec15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.948 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aec15ec80 0x7f1aec157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec1522a0 con 0x7f1aec071d00 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15bea0 con 0x7f1aec15ec80 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af3a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15e570 con 0x7f1aec141660 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec071d00 0x7f1aec13b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af1000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec071d00 0x7f1aec13b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38222/0 (socket says 172.21.15.143:38222) 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af1000640 1 -- 172.21.15.143:0/3474317706 learned_addr learned my addr 172.21.15.143:0/3474317706 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1af1a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aec15ec80 0x7f1aec157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.951+0000 7f1aebe00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec141660 0x7f1aec15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.949 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1000640 1 -- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec141660 msgr2=0x7f1aec15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1000640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f1aec141660 0x7f1aec15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1000640 1 -- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aec15ec80 msgr2=0x7f1aec157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1000640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aec15ec80 0x7f1aec157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1000640 1 -- 172.21.15.143:0/3474317706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0094d0 con 0x7f1aec071d00 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1aec15ec80 0x7f1aec157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af1000640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec071d00 0x7f1aec13b720 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0ad040 tx=0x7f1adc0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0b4620 con 0x7f1aec071d00 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc0b4780 con 0x7f1aec071d00 2024-01-15T18:23:46.950 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0b4b30 con 0x7f1aec071d00 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec0cd150 con 0x7f1aec071d00 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec15e290 con 0x7f1aec071d00 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1aec0ccfd0 con 0x7f1aec071d00 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f1adc0b4c90 con 0x7f1aec071d00 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1ae8c00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ac4067b80 0x7f1ac406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f1adc11d8a0 con 0x7f1aec071d00 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.955+0000 7f1aebe00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ac4067b80 0x7f1ac406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1ae8c00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f1ac406fea0 0x7f1ac4072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:46.951 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 --> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1ac40729f0 con 0x7f1ac406fea0 2024-01-15T18:23:46.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1adc11dc50 con 0x7f1aec071d00 2024-01-15T18:23:46.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1af1a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f1ac406fea0 0x7f1ac4072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:46.952 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1aebe00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ac4067b80 0x7f1ac406a040 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1aec1f9540 tx=0x7f1ad4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1af1a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f1ac406fea0 0x7f1ac4072320 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:46.953 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.959+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== osd.4 v2:172.21.15.204:6816/1717170274 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1ac40729f0 con 0x7f1ac406fea0 2024-01-15T18:23:46.962 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.971+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 --> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1aec142640 con 0x7f1ac406fea0 2024-01-15T18:23:46.964 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.971+0000 7f1ae8c00640 1 -- 172.21.15.143:0/3474317706 <== osd.4 v2:172.21.15.204:6816/1717170274 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1ae400f040 con 0x7f1ac406fea0 2024-01-15T18:23:46.964 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:46.964 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:46.964 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.971+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f1ac406fea0 msgr2=0x7f1ac4072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.965 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.971+0000 7f1af3a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.204:6816/1717170274,v1:172.21.15.204:6817/1717170274] conn(0x7f1ac406fea0 0x7f1ac4072320 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ac4067b80 msgr2=0x7f1ac406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1ac4067b80 0x7f1ac406a040 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1aec1f9540 tx=0x7f1ad4009040 comp rx=0 tx=0).stop 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec071d00 msgr2=0x7f1aec13b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 --2- 172.21.15.143:0/3474317706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1aec071d00 0x7f1aec13b720 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0ad040 tx=0x7f1adc0192e0 comp rx=0 tx=0).stop 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af1a00640 1 -- 172.21.15.143:0/3474317706 reap_dead start 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 shutdown_connections 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 >> 172.21.15.143:0/3474317706 conn(0x7f1aec06c3f0 msgr2=0x7f1aec157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 shutdown_connections 2024-01-15T18:23:46.966 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:46.975+0000 7f1af3a00640 1 -- 172.21.15.143:0/3474317706 wait complete. 2024-01-15T18:23:46.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-15T18:23:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 Processor -- start 2024-01-15T18:23:47.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 -- start start 2024-01-15T18:23:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f912c0ce560 0x7f912c15ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c15f310 con 0x7f912c141660 2024-01-15T18:23:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c15fbb0 con 0x7f912c0ce560 2024-01-15T18:23:47.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9133e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c072690 con 0x7f912c071d00 2024-01-15T18:23:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9131400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f912c141660 0x7f912c141f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60792/0 (socket says 172.21.15.143:60792) 2024-01-15T18:23:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9131400640 1 -- 172.21.15.143:0/3216389179 learned_addr learned my addr 172.21.15.143:0/3216389179 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9130a00640 1 --2- 172.21.15.143:0/3216389179 >> v2:172.21.15.143:3300/0 conn(0x7f912c0ce560 0x7f912c15ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9131e00640 1 --2- 172.21.15.143:0/3216389179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f912be00640 1 -- 172.21.15.143:0/3216389179 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330154375 0 0) 0x7f912c15f310 con 0x7f912c141660 2024-01-15T18:23:47.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f912be00640 1 -- 172.21.15.143:0/3216389179 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f910c003530 con 0x7f912c141660 2024-01-15T18:23:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9130a00640 1 -- 172.21.15.143:0/3216389179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 msgr2=0x7f912c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9130a00640 1 --2- 172.21.15.143:0/3216389179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9130a00640 1 -- 172.21.15.143:0/3216389179 >> v1:172.21.15.143:6789/0 conn(0x7f912c141660 legacy=0x7f912c141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.129 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.135+0000 7f9130a00640 1 -- 172.21.15.143:0/3216389179 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c15f5f0 con 0x7f912c0ce560 2024-01-15T18:23:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9130a00640 1 --2- 172.21.15.143:0/3216389179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15ec80 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f91140097c0 tx=0x7f911401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e5da185f0cf13d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f912be00640 1 -- 172.21.15.143:0/3216389179 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140ab600 con 0x7f912c0ce560 2024-01-15T18:23:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f912be00640 1 -- 172.21.15.143:0/3216389179 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91140a7050 con 0x7f912c0ce560 2024-01-15T18:23:47.130 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f912be00640 1 -- 172.21.15.143:0/3216389179 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140ab960 con 0x7f912c0ce560 2024-01-15T18:23:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 -- 172.21.15.143:0/3216389179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 msgr2=0x7f912c15ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 --2- 172.21.15.143:0/3216389179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15ec80 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f91140097c0 tx=0x7f911401c480 comp rx=0 tx=0).stop 2024-01-15T18:23:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 -- 172.21.15.143:0/3216389179 shutdown_connections 2024-01-15T18:23:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 --2- 172.21.15.143:0/3216389179 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 --2- 172.21.15.143:0/3216389179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15ec80 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 -- 172.21.15.143:0/3216389179 >> 172.21.15.143:0/3216389179 conn(0x7f912c06c3f0 msgr2=0x7f912c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 -- 172.21.15.143:0/3216389179 shutdown_connections 2024-01-15T18:23:47.132 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.139+0000 7f9133e00640 1 -- 172.21.15.143:0/3216389179 wait complete. 2024-01-15T18:23:47.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 Processor -- start 2024-01-15T18:23:47.133 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 -- start start 2024-01-15T18:23:47.135 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c13da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c141660 0x7f912c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c13dfb0 con 0x7f912c0ce560 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c15d120 con 0x7f912c141660 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c15c750 con 0x7f912c071d00 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38242/0 (socket says 172.21.15.143:38242) 2024-01-15T18:23:47.136 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 -- 172.21.15.143:0/1794543710 learned_addr learned my addr 172.21.15.143:0/1794543710 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9131400640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c13da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9131e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c141660 0x7f912c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 -- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 msgr2=0x7f912c13da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c13da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 -- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c141660 msgr2=0x7f912c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c141660 0x7f912c161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 -- 172.21.15.143:0/1794543710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9114009420 con 0x7f912c0ce560 2024-01-15T18:23:47.137 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9130a00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15cbe0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f911401ab30 tx=0x7f9114019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140b9070 con 0x7f912c0ce560 2024-01-15T18:23:47.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91140abd10 con 0x7f912c0ce560 2024-01-15T18:23:47.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140b4620 con 0x7f912c0ce560 2024-01-15T18:23:47.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c0cd150 con 0x7f912c0ce560 2024-01-15T18:23:47.138 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.143+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c15be20 con 0x7f912c0ce560 2024-01-15T18:23:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.147+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c067500 con 0x7f912c0ce560 2024-01-15T18:23:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.147+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f91140b2040 con 0x7f912c0ce560 2024-01-15T18:23:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.147+0000 7f9129600640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9104067bb0 0x7f910406a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.147+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f911411c810 con 0x7f912c0ce560 2024-01-15T18:23:47.139 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.147+0000 7f9131400640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9104067bb0 0x7f910406a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.140 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.147+0000 7f9131400640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9104067bb0 0x7f910406a070 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f911c007fd0 tx=0x7f911c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.142 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.151+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f91140eb210 con 0x7f912c0ce560 2024-01-15T18:23:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f912c0ccfd0 con 0x7f9104067bb0 2024-01-15T18:23:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9129600640 1 -- 172.21.15.143:0/1794543710 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f911c001480 con 0x7f9104067bb0 2024-01-15T18:23:47.293 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 4 to deep-scrub 2024-01-15T18:23:47.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9104067bb0 msgr2=0x7f910406a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9104067bb0 0x7f910406a070 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f911c007fd0 tx=0x7f911c009040 comp rx=0 tx=0).stop 2024-01-15T18:23:47.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 msgr2=0x7f912c15cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.295 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15cbe0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f911401ab30 tx=0x7f9114019220 comp rx=0 tx=0).stop 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 shutdown_connections 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9104067bb0 0x7f910406a070 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f912c141660 0x7f912c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f912c0ce560 0x7f912c15cbe0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 --2- 172.21.15.143:0/1794543710 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f912c071d00 0x7f912c13da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 >> 172.21.15.143:0/1794543710 conn(0x7f912c06c3f0 msgr2=0x7f912c13ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 shutdown_connections 2024-01-15T18:23:47.296 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.303+0000 7f9133e00640 1 -- 172.21.15.143:0/1794543710 wait complete. 2024-01-15T18:23:47.316 INFO:tasks.ceph:Scrubbing osd.5 2024-01-15T18:23:47.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:47.455 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.463+0000 7f9f32800640 1 Processor -- start 2024-01-15T18:23:47.456 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.463+0000 7f9f32800640 1 -- start start 2024-01-15T18:23:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f32800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c143780 0x7f9f2c0cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f32800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f9f2c0ce520 0x7f9f2c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f32800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f2c0720f0 0x7f9f2c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f32800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c072580 con 0x7f9f2c0720f0 2024-01-15T18:23:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f32800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2c0ceb50 con 0x7f9f2c0ce520 2024-01-15T18:23:47.457 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f32800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c15c110 con 0x7f9f2c143780 2024-01-15T18:23:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f2c0720f0 0x7f9f2c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9f2c0720f0 0x7f9f2c15a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33802/0 (socket says 172.21.15.143:33802) 2024-01-15T18:23:47.458 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 -- 172.21.15.143:0/1940354932 learned_addr learned my addr 172.21.15.143:0/1940354932 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2b400640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c143780 0x7f9f2c0cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2aa00640 1 --2- 172.21.15.143:0/1940354932 >> v2:172.21.15.204:3300/0 conn(0x7f9f2c0ce520 0x7f9f2c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 -- 172.21.15.143:0/1940354932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c143780 msgr2=0x7f9f2c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c143780 0x7f9f2c0cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 -- 172.21.15.143:0/1940354932 >> v2:172.21.15.204:3300/0 conn(0x7f9f2c0ce520 msgr2=0x7f9f2c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 --2- 172.21.15.143:0/1940354932 >> v2:172.21.15.204:3300/0 conn(0x7f9f2c0ce520 0x7f9f2c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2be00640 1 -- 172.21.15.143:0/1940354932 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c15b7e0 con 0x7f9f2c0720f0 2024-01-15T18:23:47.459 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2b400640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c143780 0x7f9f2c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.467+0000 7f9f2aa00640 1 --2- 172.21.15.143:0/1940354932 >> v2:172.21.15.204:3300/0 conn(0x7f9f2c0ce520 0x7f9f2c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:23:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f2be00640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c0720f0 0x7f9f2c15a450 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c06bee0 tx=0x7f9f2401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941485e4ce7546a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.460 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f2a000640 1 -- 172.21.15.143:0/1940354932 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240a5760 con 0x7f9f2c0720f0 2024-01-15T18:23:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f2a000640 1 -- 172.21.15.143:0/1940354932 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f240a58c0 con 0x7f9f2c0720f0 2024-01-15T18:23:47.461 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f2a000640 1 -- 172.21.15.143:0/1940354932 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240a5c50 con 0x7f9f2c0720f0 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 -- 172.21.15.143:0/1940354932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c0720f0 msgr2=0x7f9f2c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c0720f0 0x7f9f2c15a450 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c06bee0 tx=0x7f9f2401c9d0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 -- 172.21.15.143:0/1940354932 shutdown_connections 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c0720f0 0x7f9f2c15a450 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 --2- 172.21.15.143:0/1940354932 >> v2:172.21.15.204:3300/0 conn(0x7f9f2c0ce520 0x7f9f2c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 --2- 172.21.15.143:0/1940354932 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c143780 0x7f9f2c0cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 -- 172.21.15.143:0/1940354932 >> 172.21.15.143:0/1940354932 conn(0x7f9f2c06c3f0 msgr2=0x7f9f2c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.462 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 -- 172.21.15.143:0/1940354932 shutdown_connections 2024-01-15T18:23:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 -- 172.21.15.143:0/1940354932 wait complete. 2024-01-15T18:23:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 Processor -- start 2024-01-15T18:23:47.463 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.471+0000 7f9f32800640 1 -- start start 2024-01-15T18:23:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c0720f0 0x7f9f2c1386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2c0ce520 0x7f9f2c13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.464 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c143780 0x7f9f2c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c06d3a0 con 0x7f9f2c143780 2024-01-15T18:23:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c138c20 con 0x7f9f2c0ce520 2024-01-15T18:23:47.465 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2c06c7d0 con 0x7f9f2c0720f0 2024-01-15T18:23:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c143780 0x7f9f2c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c0720f0 0x7f9f2c1386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.466 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2c0ce520 0x7f9f2c13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c143780 0x7f9f2c06d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33810/0 (socket says 172.21.15.143:33810) 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 -- 172.21.15.143:0/2218242929 learned_addr learned my addr 172.21.15.143:0/2218242929 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 -- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c0720f0 msgr2=0x7f9f2c1386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f9f2c0720f0 0x7f9f2c1386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 -- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2c0ce520 msgr2=0x7f9f2c13e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f2c0ce520 0x7f9f2c13e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 -- 172.21.15.143:0/2218242929 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f24009b30 con 0x7f9f2c143780 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f2be00640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c143780 0x7f9f2c06d830 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401b080 tx=0x7f9f24019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240b8670 con 0x7f9f2c143780 2024-01-15T18:23:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f240b87f0 con 0x7f9f2c143780 2024-01-15T18:23:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240b8ba0 con 0x7f9f2c143780 2024-01-15T18:23:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2c06ca70 con 0x7f9f2c143780 2024-01-15T18:23:47.468 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c06d030 con 0x7f9f2c143780 2024-01-15T18:23:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9f2c067500 con 0x7f9f2c143780 2024-01-15T18:23:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.475+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f9f240b8d00 con 0x7f9f2c143780 2024-01-15T18:23:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f23400640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f04067b80 0x7f9f0406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f9f2411ee30 con 0x7f9f2c143780 2024-01-15T18:23:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f2b400640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f04067b80 0x7f9f0406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.469 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f23400640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f9f0406fea0 0x7f9f04072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 --> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9f040729f0 con 0x7f9f0406fea0 2024-01-15T18:23:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f2411f1a0 con 0x7f9f2c143780 2024-01-15T18:23:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f2b400640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f04067b80 0x7f9f0406a040 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006fd0 tx=0x7f9f1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f2aa00640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f9f0406fea0 0x7f9f04072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.470 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f2aa00640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f9f0406fea0 0x7f9f04072320 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.471 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.479+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== osd.5 v2:172.21.15.204:6800/1020370322 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9f040729f0 con 0x7f9f0406fea0 2024-01-15T18:23:47.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.491+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 --> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9f2c06d550 con 0x7f9f0406fea0 2024-01-15T18:23:47.484 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f23400640 1 -- 172.21.15.143:0/2218242929 <== osd.5 v2:172.21.15.204:6800/1020370322 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9f2c06d550 con 0x7f9f0406fea0 2024-01-15T18:23:47.485 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:47.485 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:47.485 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f9f0406fea0 msgr2=0x7f9f04072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.204:6800/1020370322,v1:172.21.15.204:6801/1020370322] conn(0x7f9f0406fea0 0x7f9f04072320 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f04067b80 msgr2=0x7f9f0406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f9f04067b80 0x7f9f0406a040 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006fd0 tx=0x7f9f1c008040 comp rx=0 tx=0).stop 2024-01-15T18:23:47.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c143780 msgr2=0x7f9f2c06d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 --2- 172.21.15.143:0/2218242929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f2c143780 0x7f9f2c06d830 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401b080 tx=0x7f9f24019770 comp rx=0 tx=0).stop 2024-01-15T18:23:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f2be00640 1 -- 172.21.15.143:0/2218242929 reap_dead start 2024-01-15T18:23:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 shutdown_connections 2024-01-15T18:23:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 >> 172.21.15.143:0/2218242929 conn(0x7f9f2c06c3f0 msgr2=0x7f9f2c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 shutdown_connections 2024-01-15T18:23:47.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.495+0000 7f9f32800640 1 -- 172.21.15.143:0/2218242929 wait complete. 2024-01-15T18:23:47.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-15T18:23:47.639 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.647+0000 7f8e15a00640 1 Processor -- start 2024-01-15T18:23:47.640 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.647+0000 7f8e15a00640 1 -- start start 2024-01-15T18:23:47.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e15a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e100b2c30 0x7f8e1006ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e15a00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f8e1006bab0 0x7f8e10070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e15a00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8e1006e5c0 0x7f8e10071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.641 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e15a00640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10160c90 con 0x7f8e1006e5c0 2024-01-15T18:23:47.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e15a00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e10160f70 con 0x7f8e1006bab0 2024-01-15T18:23:47.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e15a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e10161250 con 0x7f8e100b2c30 2024-01-15T18:23:47.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8e1006e5c0 0x7f8e10071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8e1006e5c0 0x7f8e10071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33812/0 (socket says 172.21.15.143:33812) 2024-01-15T18:23:47.642 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 -- 172.21.15.143:0/1792442726 learned_addr learned my addr 172.21.15.143:0/1792442726 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0ea00640 1 --2- 172.21.15.143:0/1792442726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e100b2c30 0x7f8e1006ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0e000640 1 --2- 172.21.15.143:0/1792442726 >> v2:172.21.15.204:3300/0 conn(0x7f8e1006bab0 0x7f8e10070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 -- 172.21.15.143:0/1792442726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e100b2c30 msgr2=0x7f8e1006ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 --2- 172.21.15.143:0/1792442726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e100b2c30 0x7f8e1006ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 -- 172.21.15.143:0/1792442726 >> v2:172.21.15.204:3300/0 conn(0x7f8e1006bab0 msgr2=0x7f8e10070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 --2- 172.21.15.143:0/1792442726 >> v2:172.21.15.204:3300/0 conn(0x7f8e1006bab0 0x7f8e10070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.643 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.651+0000 7f8e0f400640 1 -- 172.21.15.143:0/1792442726 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10161530 con 0x7f8e1006e5c0 2024-01-15T18:23:47.644 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e0f400640 1 --2- 172.21.15.143:0/1792442726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e1006e5c0 0x7f8e10071200 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f8e100718c0 tx=0x7f8e0801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8562e20f284559b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e0d600640 1 -- 172.21.15.143:0/1792442726 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080a5730 con 0x7f8e1006e5c0 2024-01-15T18:23:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e0d600640 1 -- 172.21.15.143:0/1792442726 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e080a5890 con 0x7f8e1006e5c0 2024-01-15T18:23:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e0d600640 1 -- 172.21.15.143:0/1792442726 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080a5c20 con 0x7f8e1006e5c0 2024-01-15T18:23:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 -- 172.21.15.143:0/1792442726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e1006e5c0 msgr2=0x7f8e10071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.645 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 --2- 172.21.15.143:0/1792442726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e1006e5c0 0x7f8e10071200 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f8e100718c0 tx=0x7f8e0801c9a0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 -- 172.21.15.143:0/1792442726 shutdown_connections 2024-01-15T18:23:47.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 --2- 172.21.15.143:0/1792442726 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e1006e5c0 0x7f8e10071200 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 --2- 172.21.15.143:0/1792442726 >> v2:172.21.15.204:3300/0 conn(0x7f8e1006bab0 0x7f8e10070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 --2- 172.21.15.143:0/1792442726 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e100b2c30 0x7f8e1006ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 -- 172.21.15.143:0/1792442726 >> 172.21.15.143:0/1792442726 conn(0x7f8e100b1610 msgr2=0x7f8e100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.646 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 -- 172.21.15.143:0/1792442726 shutdown_connections 2024-01-15T18:23:47.647 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 -- 172.21.15.143:0/1792442726 wait complete. 2024-01-15T18:23:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.655+0000 7f8e15a00640 1 Processor -- start 2024-01-15T18:23:47.648 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- start start 2024-01-15T18:23:47.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1006bab0 0x7f8e1014cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.649 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e1006e5c0 0x7f8e100aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 0x7f8e1014e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1014e270 con 0x7f8e100b2c30 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1014d150 con 0x7f8e1006bab0 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100af490 con 0x7f8e1006e5c0 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 0x7f8e1014e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 0x7f8e1014e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33822/0 (socket says 172.21.15.143:33822) 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 -- 172.21.15.143:0/4260957638 learned_addr learned my addr 172.21.15.143:0/4260957638 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0ea00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1006bab0 0x7f8e1014cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.650 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0e000640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e1006e5c0 0x7f8e100aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 -- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e1006e5c0 msgr2=0x7f8e100aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e1006e5c0 0x7f8e100aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 -- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1006bab0 msgr2=0x7f8e1014cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1006bab0 0x7f8e1014cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 -- 172.21.15.143:0/4260957638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e08009b30 con 0x7f8e100b2c30 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e0f400640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 0x7f8e1014e690 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f8e0801b050 tx=0x7f8e08019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080b8670 con 0x7f8e100b2c30 2024-01-15T18:23:47.652 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e080b87f0 con 0x7f8e100b2c30 2024-01-15T18:23:47.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e080b8ba0 con 0x7f8e100b2c30 2024-01-15T18:23:47.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e100af770 con 0x7f8e100b2c30 2024-01-15T18:23:47.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1014de90 con 0x7f8e100b2c30 2024-01-15T18:23:47.653 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.659+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e10074b00 con 0x7f8e100b2c30 2024-01-15T18:23:47.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.663+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f8e080b8d00 con 0x7f8e100b2c30 2024-01-15T18:23:47.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.663+0000 7f8e06a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8de8067b80 0x7f8de806a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.663+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f8e0811e410 con 0x7f8e100b2c30 2024-01-15T18:23:47.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.663+0000 7f8e0ea00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8de8067b80 0x7f8de806a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.654 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.663+0000 7f8e0ea00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8de8067b80 0x7f8de806a040 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8e00007940 tx=0x7f8e00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.657 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.667+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f8e080ed270 con 0x7f8e100b2c30 2024-01-15T18:23:47.806 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.815+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e10067c80 con 0x7f8de8067b80 2024-01-15T18:23:47.807 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.815+0000 7f8e06a00640 1 -- 172.21.15.143:0/4260957638 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e10067c80 con 0x7f8de8067b80 2024-01-15T18:23:47.807 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 5 to deep-scrub 2024-01-15T18:23:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8de8067b80 msgr2=0x7f8de806a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8de8067b80 0x7f8de806a040 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8e00007940 tx=0x7f8e00008040 comp rx=0 tx=0).stop 2024-01-15T18:23:47.809 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 msgr2=0x7f8e1014e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 0x7f8e1014e690 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f8e0801b050 tx=0x7f8e08019740 comp rx=0 tx=0).stop 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 shutdown_connections 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8de8067b80 0x7f8de806a040 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e100b2c30 0x7f8e1014e690 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f8e1006e5c0 0x7f8e100aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 --2- 172.21.15.143:0/4260957638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e1006bab0 0x7f8e1014cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 >> 172.21.15.143:0/4260957638 conn(0x7f8e100b1610 msgr2=0x7f8e100b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 shutdown_connections 2024-01-15T18:23:47.810 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.819+0000 7f8e15a00640 1 -- 172.21.15.143:0/4260957638 wait complete. 2024-01-15T18:23:47.829 INFO:tasks.ceph:Scrubbing osd.6 2024-01-15T18:23:47.829 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:47.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68c1600640 1 Processor -- start 2024-01-15T18:23:47.969 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68c1600640 1 -- start start 2024-01-15T18:23:47.970 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68c1600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f68bc141660 0x7f68bc0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68c1600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc15f310 con 0x7f68bc071d00 2024-01-15T18:23:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68c1600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc15f5f0 con 0x7f68bc15ec80 2024-01-15T18:23:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68c1600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68bc15fbb0 con 0x7f68bc141660 2024-01-15T18:23:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68baa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f68bc141660 0x7f68bc0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68bb400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f68bc071d00 0x7f68bc0b40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50810/0 (socket says 172.21.15.143:50810) 2024-01-15T18:23:47.971 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68bb400640 1 -- 172.21.15.143:0/3170090328 learned_addr learned my addr 172.21.15.143:0/3170090328 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68baa00640 1 -- 172.21.15.143:0/3170090328 >> v1:172.21.15.204:6789/0 conn(0x7f68bc15ec80 legacy=0x7f68bc141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68baa00640 1 -- 172.21.15.143:0/3170090328 >> v1:172.21.15.143:6789/0 conn(0x7f68bc071d00 legacy=0x7f68bc0b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68baa00640 1 -- 172.21.15.143:0/3170090328 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc15f8d0 con 0x7f68bc141660 2024-01-15T18:23:47.972 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.979+0000 7f68b9600640 1 -- 172.21.15.143:0/3170090328 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084344578 0 0) 0x7f68bc15f310 con 0x7f68bc071d00 2024-01-15T18:23:47.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68baa00640 1 --2- 172.21.15.143:0/3170090328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc141660 0x7f68bc0ce560 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0097c0 tx=0x7f68ac01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440ef928d5d50b9d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.974 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68b9600640 1 -- 172.21.15.143:0/3170090328 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0a7070 con 0x7f68bc141660 2024-01-15T18:23:47.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68b9600640 1 -- 172.21.15.143:0/3170090328 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac0ab430 con 0x7f68bc141660 2024-01-15T18:23:47.975 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68b9600640 1 -- 172.21.15.143:0/3170090328 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ab7c0 con 0x7f68bc141660 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 -- 172.21.15.143:0/3170090328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc141660 msgr2=0x7f68bc0ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 --2- 172.21.15.143:0/3170090328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc141660 0x7f68bc0ce560 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0097c0 tx=0x7f68ac01c410 comp rx=0 tx=0).stop 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 -- 172.21.15.143:0/3170090328 shutdown_connections 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 --2- 172.21.15.143:0/3170090328 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc141660 0x7f68bc0ce560 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 -- 172.21.15.143:0/3170090328 >> 172.21.15.143:0/3170090328 conn(0x7f68bc06c3f0 msgr2=0x7f68bc06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 -- 172.21.15.143:0/3170090328 shutdown_connections 2024-01-15T18:23:47.976 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.983+0000 7f68c1600640 1 -- 172.21.15.143:0/3170090328 wait complete. 2024-01-15T18:23:47.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 Processor -- start 2024-01-15T18:23:47.977 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 -- start start 2024-01-15T18:23:47.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68bc071d00 0x7f68bc0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68bc141660 0x7f68bc143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.978 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc15ec80 0x7f68bc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc15f130 con 0x7f68bc15ec80 2024-01-15T18:23:47.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc156730 con 0x7f68bc141660 2024-01-15T18:23:47.979 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc1570d0 con 0x7f68bc071d00 2024-01-15T18:23:47.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc15ec80 0x7f68bc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc15ec80 0x7f68bc0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33840/0 (socket says 172.21.15.143:33840) 2024-01-15T18:23:47.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 -- 172.21.15.143:0/3025767096 learned_addr learned my addr 172.21.15.143:0/3025767096 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:47.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68baa00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68bc071d00 0x7f68bc0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.981 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68ba000640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68bc141660 0x7f68bc143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 -- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68bc071d00 msgr2=0x7f68bc0b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f68bc071d00 0x7f68bc0b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 -- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68bc141660 msgr2=0x7f68bc143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f68bc141660 0x7f68bc143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 -- 172.21.15.143:0/3025767096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac009420 con 0x7f68bc15ec80 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68bb400640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc15ec80 0x7f68bc0ac8e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f68b4001570 tx=0x7f68b400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.982 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400cea0 con 0x7f68bc15ec80 2024-01-15T18:23:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b400a5e0 con 0x7f68bc15ec80 2024-01-15T18:23:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc0cd150 con 0x7f68bc15ec80 2024-01-15T18:23:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400a990 con 0x7f68bc15ec80 2024-01-15T18:23:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.987+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc156940 con 0x7f68bc15ec80 2024-01-15T18:23:47.983 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68bc141ed0 con 0x7f68bc15ec80 2024-01-15T18:23:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f68b400aaf0 con 0x7f68bc15ec80 2024-01-15T18:23:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68b2a00640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6894067b80 0x7f689406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f68b40828d0 con 0x7f68bc15ec80 2024-01-15T18:23:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68baa00640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6894067b80 0x7f689406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68b2a00640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f689406fea0 0x7f6894072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:47.985 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 --> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68940729f0 con 0x7f689406fea0 2024-01-15T18:23:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68b4083460 con 0x7f68bc15ec80 2024-01-15T18:23:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68ba000640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f689406fea0 0x7f6894072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.991+0000 7f68baa00640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6894067b80 0x7f689406a040 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0af000 tx=0x7f68ac0a7000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.995+0000 7f68ba000640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f689406fea0 0x7f6894072320 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:47.986 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:47.995+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== osd.6 v2:172.21.15.204:6808/1677254362 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f68940729f0 con 0x7f689406fea0 2024-01-15T18:23:47.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 --> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f68bc0ccfd0 con 0x7f689406fea0 2024-01-15T18:23:47.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68b2a00640 1 -- 172.21.15.143:0/3025767096 <== osd.6 v2:172.21.15.204:6808/1677254362 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f68a4009a10 con 0x7f689406fea0 2024-01-15T18:23:47.999 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:47.999 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:47.999 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f689406fea0 msgr2=0x7f6894072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.204:6808/1677254362,v1:172.21.15.204:6809/1677254362] conn(0x7f689406fea0 0x7f6894072320 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6894067b80 msgr2=0x7f689406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6894067b80 0x7f689406a040 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0af000 tx=0x7f68ac0a7000 comp rx=0 tx=0).stop 2024-01-15T18:23:48.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc15ec80 msgr2=0x7f68bc0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 --2- 172.21.15.143:0/3025767096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f68bc15ec80 0x7f68bc0ac8e0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f68b4001570 tx=0x7f68b400cc50 comp rx=0 tx=0).stop 2024-01-15T18:23:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68bb400640 1 -- 172.21.15.143:0/3025767096 reap_dead start 2024-01-15T18:23:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 shutdown_connections 2024-01-15T18:23:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 >> 172.21.15.143:0/3025767096 conn(0x7f68bc06c3f0 msgr2=0x7f68bc0af4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 shutdown_connections 2024-01-15T18:23:48.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.007+0000 7f68c1600640 1 -- 172.21.15.143:0/3025767096 wait complete. 2024-01-15T18:23:48.018 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-15T18:23:48.155 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.163+0000 7f6851000640 1 Processor -- start 2024-01-15T18:23:48.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.163+0000 7f6851000640 1 -- start start 2024-01-15T18:23:48.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6851000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f684c071bb0 0x7f684c072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6851000640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f684c15c5a0 0x7f684c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6851000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0ceb50 con 0x7f684c143780 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6851000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c072570 con 0x7f684c071bb0 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6851000640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c15c110 con 0x7f684c15c5a0 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f684c071bb0 0x7f684c072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f684c071bb0 0x7f684c072030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33842/0 (socket says 172.21.15.143:33842) 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 -- 172.21.15.143:0/3862424819 learned_addr learned my addr 172.21.15.143:0/3862424819 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f684a800640 1 --2- 172.21.15.143:0/3862424819 >> v2:172.21.15.204:3300/0 conn(0x7f684c15c5a0 0x7f684c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 -- 172.21.15.143:0/3862424819 >> v2:172.21.15.204:3300/0 conn(0x7f684c15c5a0 msgr2=0x7f684c15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 --2- 172.21.15.143:0/3862424819 >> v2:172.21.15.204:3300/0 conn(0x7f684c15c5a0 0x7f684c15c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 -- 172.21.15.143:0/3862424819 >> v1:172.21.15.204:6789/0 conn(0x7f684c143780 legacy=0x7f684c0cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.167+0000 7f6849400640 1 -- 172.21.15.143:0/3862424819 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c15b7e0 con 0x7f684c071bb0 2024-01-15T18:23:48.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6849400640 1 --2- 172.21.15.143:0/3862424819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c071bb0 0x7f684c072030 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f684c070a20 tx=0x7f683401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4bd2dde4d653b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6848a00640 1 -- 172.21.15.143:0/3862424819 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68340a7ca0 con 0x7f684c071bb0 2024-01-15T18:23:48.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6848a00640 1 -- 172.21.15.143:0/3862424819 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68340a7e00 con 0x7f684c071bb0 2024-01-15T18:23:48.161 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6848a00640 1 -- 172.21.15.143:0/3862424819 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68340af660 con 0x7f684c071bb0 2024-01-15T18:23:48.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 -- 172.21.15.143:0/3862424819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c071bb0 msgr2=0x7f684c072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 --2- 172.21.15.143:0/3862424819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c071bb0 0x7f684c072030 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f684c070a20 tx=0x7f683401ebd0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 -- 172.21.15.143:0/3862424819 shutdown_connections 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 --2- 172.21.15.143:0/3862424819 >> v2:172.21.15.204:3300/0 conn(0x7f684c15c5a0 0x7f684c15c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 --2- 172.21.15.143:0/3862424819 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c071bb0 0x7f684c072030 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 -- 172.21.15.143:0/3862424819 >> 172.21.15.143:0/3862424819 conn(0x7f684c06c3f0 msgr2=0x7f684c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 -- 172.21.15.143:0/3862424819 shutdown_connections 2024-01-15T18:23:48.163 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 -- 172.21.15.143:0/3862424819 wait complete. 2024-01-15T18:23:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 Processor -- start 2024-01-15T18:23:48.164 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.171+0000 7f6851000640 1 -- start start 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c071bb0 0x7f684c142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f684c143780 0x7f684c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 0x7f684c152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c143090 con 0x7f684c15c5a0 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c156be0 con 0x7f684c071bb0 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c156060 con 0x7f684c143780 2024-01-15T18:23:48.166 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 0x7f684c152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 0x7f684c152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33846/0 (socket says 172.21.15.143:33846) 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 -- 172.21.15.143:0/1604866145 learned_addr learned my addr 172.21.15.143:0/1604866145 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6849e00640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c071bb0 0x7f684c142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6849400640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f684c143780 0x7f684c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 -- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f684c143780 msgr2=0x7f684c141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f684c143780 0x7f684c141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 -- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c071bb0 msgr2=0x7f684c142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c071bb0 0x7f684c142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 -- 172.21.15.143:0/1604866145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683400b770 con 0x7f684c15c5a0 2024-01-15T18:23:48.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f684a800640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 0x7f684c152570 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f6844007cf0 tx=0x7f684400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844014420 con 0x7f684c15c5a0 2024-01-15T18:23:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68440145a0 con 0x7f684c15c5a0 2024-01-15T18:23:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844014950 con 0x7f684c15c5a0 2024-01-15T18:23:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c156340 con 0x7f684c15c5a0 2024-01-15T18:23:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c156900 con 0x7f684c15c5a0 2024-01-15T18:23:48.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.175+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c138d70 con 0x7f684c15c5a0 2024-01-15T18:23:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.179+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f6844014ab0 con 0x7f684c15c5a0 2024-01-15T18:23:48.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.179+0000 7f6842000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6824067b80 0x7f682406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.179+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f6844083af0 con 0x7f684c15c5a0 2024-01-15T18:23:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.179+0000 7f6849e00640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6824067b80 0x7f682406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.171 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.179+0000 7f6849e00640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6824067b80 0x7f682406a040 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f684c0b3540 tx=0x7f683c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.173 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.183+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f68440524b0 con 0x7f684c15c5a0 2024-01-15T18:23:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f684c070aa0 con 0x7f6824067b80 2024-01-15T18:23:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6842000640 1 -- 172.21.15.143:0/1604866145 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f683c001510 con 0x7f6824067b80 2024-01-15T18:23:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 6 to deep-scrub 2024-01-15T18:23:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6824067b80 msgr2=0x7f682406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6824067b80 0x7f682406a040 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f684c0b3540 tx=0x7f683c009040 comp rx=0 tx=0).stop 2024-01-15T18:23:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 msgr2=0x7f684c152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 0x7f684c152570 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f6844007cf0 tx=0x7f684400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 shutdown_connections 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6824067b80 0x7f682406a040 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f684c15c5a0 0x7f684c152570 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f684c143780 0x7f684c141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 --2- 172.21.15.143:0/1604866145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f684c071bb0 0x7f684c142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 >> 172.21.15.143:0/1604866145 conn(0x7f684c06c3f0 msgr2=0x7f684c06dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 shutdown_connections 2024-01-15T18:23:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.331+0000 7f6851000640 1 -- 172.21.15.143:0/1604866145 wait complete. 2024-01-15T18:23:48.343 INFO:tasks.ceph:Scrubbing osd.7 2024-01-15T18:23:48.343 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:23:48.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 Processor -- start 2024-01-15T18:23:48.482 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 -- start start 2024-01-15T18:23:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 0x7f607c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.485 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f607c0b1da0 0x7f607c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c070bf0 con 0x7f607c0b3bf0 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607c15dba0 con 0x7f607c0b1da0 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6082c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0b2740 con 0x7f607c0ce690 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6080c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f607c0b3bf0 0x7f607c0b4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50816/0 (socket says 172.21.15.143:50816) 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.491+0000 7f6080c00640 1 -- 172.21.15.143:0/2894881334 learned_addr learned my addr 172.21.15.143:0/2894881334 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607b400640 1 --2- 172.21.15.143:0/2894881334 >> v2:172.21.15.204:3300/0 conn(0x7f607c0b1da0 0x7f607c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.486 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 0x7f607c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877682451 0 0) 0x7f607c070bf0 con 0x7f607c0b3bf0 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f605c003530 con 0x7f607c0b3bf0 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554643343 0 0) 0x7f605c003530 con 0x7f607c0b3bf0 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 msgr2=0x7f607c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 0x7f607c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 >> v2:172.21.15.204:3300/0 conn(0x7f607c0b1da0 msgr2=0x7f607c0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 --2- 172.21.15.143:0/2894881334 >> v2:172.21.15.204:3300/0 conn(0x7f607c0b1da0 0x7f607c0b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0b4920 con 0x7f607c0b3bf0 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f6074001a00 con 0x7f607c0b3bf0 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 >> v1:172.21.15.143:6789/0 conn(0x7f607c0b3bf0 legacy=0x7f607c0b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f605c003980 0x7f605c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c0047d0 0x7f605c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 0x7f605c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c15dba0 con 0x7f605c003980 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0b2740 con 0x7f605c0047d0 2024-01-15T18:23:48.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c00b990 con 0x7f607c0ce690 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f605c003980 0x7f605c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607b400640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c0047d0 0x7f605c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f6080c00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 0x7f605c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 -- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 msgr2=0x7f605c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c0ce690 0x7f605c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 -- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c0047d0 msgr2=0x7f605c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c0047d0 0x7f605c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.495+0000 7f607be00640 1 -- 172.21.15.143:0/2894881334 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c003530 con 0x7f605c003980 2024-01-15T18:23:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f607be00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f605c003980 0x7f605c003da0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f606c009ae0 tx=0x7f606c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f724701f19a1ae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0a6430 con 0x7f605c003980 2024-01-15T18:23:48.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c0a6590 con 0x7f605c003980 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f607aa00640 1 -- 172.21.15.143:0/2894881334 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0a6940 con 0x7f605c003980 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 -- 172.21.15.143:0/2894881334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f605c003980 msgr2=0x7f605c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 --2- 172.21.15.143:0/2894881334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f605c003980 0x7f605c003da0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f606c009ae0 tx=0x7f606c01ca20 comp rx=0 tx=0).stop 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6080c00640 1 -- 172.21.15.143:0/2894881334 reap_dead start 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 -- 172.21.15.143:0/2894881334 shutdown_connections 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 -- 172.21.15.143:0/2894881334 >> 172.21.15.143:0/2894881334 conn(0x7f607c06c3f0 msgr2=0x7f607c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 -- 172.21.15.143:0/2894881334 shutdown_connections 2024-01-15T18:23:48.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 -- 172.21.15.143:0/2894881334 wait complete. 2024-01-15T18:23:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 Processor -- start 2024-01-15T18:23:48.493 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.499+0000 7f6082c00640 1 -- start start 2024-01-15T18:23:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c003980 0x7f607c13fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c140940 0x7f607c1f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f607c13c110 0x7f607c13c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c13eed0 con 0x7f607c13c110 2024-01-15T18:23:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c13f2f0 con 0x7f605c003980 2024-01-15T18:23:48.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c13eb30 con 0x7f607c140940 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f607c13c110 0x7f607c13c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f607c13c110 0x7f607c13c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33858/0 (socket says 172.21.15.143:33858) 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 -- 172.21.15.143:0/1775199385 learned_addr learned my addr 172.21.15.143:0/1775199385 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f607be00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c003980 0x7f607c13fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f607b400640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c140940 0x7f607c1f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 -- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c140940 msgr2=0x7f607c1f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c140940 0x7f607c1f9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.496 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 -- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c003980 msgr2=0x7f607c13fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f605c003980 0x7f607c13fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 -- 172.21.15.143:0/1775199385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c13dd50 con 0x7f607c13c110 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f607b400640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f607c140940 0x7f607c1f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6080c00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f607c13c110 0x7f607c13c590 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f607c15b2d0 tx=0x7f6074007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400ab60 con 0x7f607c13c110 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607400acc0 con 0x7f607c13c110 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074013630 con 0x7f607c13c110 2024-01-15T18:23:48.497 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c13d790 con 0x7f607c13c110 2024-01-15T18:23:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c13cad0 con 0x7f607c13c110 2024-01-15T18:23:48.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f607c067500 con 0x7f607c13c110 2024-01-15T18:23:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.503+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f6074013790 con 0x7f607c13c110 2024-01-15T18:23:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f6073e00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6054067b80 0x7f605406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f60740839b0 con 0x7f607c13c110 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f607be00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6054067b80 0x7f605406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f6073e00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f605406fea0 0x7f6054072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 --> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60540729f0 con 0x7f605406fea0 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6074084540 con 0x7f607c13c110 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f607b400640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f605406fea0 0x7f6054072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f607be00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6054067b80 0x7f605406a040 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f606c0ad000 tx=0x7f606c0b3040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f607b400640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f605406fea0 0x7f6054072320 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.507+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== osd.7 v2:172.21.15.204:6824/2462757405 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f60540729f0 con 0x7f605406fea0 2024-01-15T18:23:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.519+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 --> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f607c13f010 con 0x7f605406fea0 2024-01-15T18:23:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6073e00640 1 -- 172.21.15.143:0/1775199385 <== osd.7 v2:172.21.15.204:6824/2462757405 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f607c13f010 con 0x7f605406fea0 2024-01-15T18:23:48.512 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-01-15T18:23:48.513 INFO:teuthology.orchestra.run.smithi143.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:23:48.513 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f605406fea0 msgr2=0x7f6054072320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.204:6824/2462757405,v1:172.21.15.204:6825/2462757405] conn(0x7f605406fea0 0x7f6054072320 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6054067b80 msgr2=0x7f605406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f6054067b80 0x7f605406a040 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f606c0ad000 tx=0x7f606c0b3040 comp rx=0 tx=0).stop 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f607c13c110 msgr2=0x7f607c13c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 --2- 172.21.15.143:0/1775199385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f607c13c110 0x7f607c13c590 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f607c15b2d0 tx=0x7f6074007a20 comp rx=0 tx=0).stop 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6080c00640 1 -- 172.21.15.143:0/1775199385 reap_dead start 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 shutdown_connections 2024-01-15T18:23:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 >> 172.21.15.143:0/1775199385 conn(0x7f607c06c3f0 msgr2=0x7f607c15dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 shutdown_connections 2024-01-15T18:23:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.523+0000 7f6082c00640 1 -- 172.21.15.143:0/1775199385 wait complete. 2024-01-15T18:23:48.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-15T18:23:48.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.679+0000 7f81a2200640 1 Processor -- start 2024-01-15T18:23:48.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.679+0000 7f81a2200640 1 -- start start 2024-01-15T18:23:48.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f81a2200640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f819c141660 0x7f819c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f81a2200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f819c15ec80 0x7f819c141f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.673 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f81a2200640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c15f310 con 0x7f819c142470 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f81a2200640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c15fbb0 con 0x7f819c15ec80 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f81a2200640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c15f130 con 0x7f819c141660 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f819c15ec80 0x7f819c141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f819c15ec80 0x7f819c141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33870/0 (socket says 172.21.15.143:33870) 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 -- 172.21.15.143:0/2394483145 learned_addr learned my addr 172.21.15.143:0/2394483145 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819b400640 1 --2- 172.21.15.143:0/2394483145 >> v2:172.21.15.204:3300/0 conn(0x7f819c141660 0x7f819c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 -- 172.21.15.143:0/2394483145 >> v2:172.21.15.204:3300/0 conn(0x7f819c141660 msgr2=0x7f819c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.674 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 --2- 172.21.15.143:0/2394483145 >> v2:172.21.15.204:3300/0 conn(0x7f819c141660 0x7f819c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 -- 172.21.15.143:0/2394483145 >> v1:172.21.15.204:6789/0 conn(0x7f819c142470 legacy=0x7f819c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:23:48.675 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.683+0000 7f819aa00640 1 -- 172.21.15.143:0/2394483145 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c15f5f0 con 0x7f819c15ec80 2024-01-15T18:23:48.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f819aa00640 1 --2- 172.21.15.143:0/2394483145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15ec80 0x7f819c141f30 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f819c0b1aa0 tx=0x7f818401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f44160a012b45f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.676 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f819a000640 1 -- 172.21.15.143:0/2394483145 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840aa5f0 con 0x7f819c15ec80 2024-01-15T18:23:48.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f819a000640 1 -- 172.21.15.143:0/2394483145 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81840aa750 con 0x7f819c15ec80 2024-01-15T18:23:48.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f819a000640 1 -- 172.21.15.143:0/2394483145 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840aaae0 con 0x7f819c15ec80 2024-01-15T18:23:48.677 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 -- 172.21.15.143:0/2394483145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15ec80 msgr2=0x7f819c141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 --2- 172.21.15.143:0/2394483145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15ec80 0x7f819c141f30 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f819c0b1aa0 tx=0x7f818401c490 comp rx=0 tx=0).stop 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 -- 172.21.15.143:0/2394483145 shutdown_connections 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 --2- 172.21.15.143:0/2394483145 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c15ec80 0x7f819c141f30 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 --2- 172.21.15.143:0/2394483145 >> v2:172.21.15.204:3300/0 conn(0x7f819c141660 0x7f819c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 -- 172.21.15.143:0/2394483145 >> 172.21.15.143:0/2394483145 conn(0x7f819c06c3f0 msgr2=0x7f819c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 -- 172.21.15.143:0/2394483145 shutdown_connections 2024-01-15T18:23:48.678 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 -- 172.21.15.143:0/2394483145 wait complete. 2024-01-15T18:23:48.679 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 Processor -- start 2024-01-15T18:23:48.680 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.687+0000 7f81a2200640 1 -- start start 2024-01-15T18:23:48.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f819c141660 0x7f819c139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 0x7f819c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f819c15ec80 0x7f819c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c15e480 con 0x7f819c142470 2024-01-15T18:23:48.681 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c13b6e0 con 0x7f819c15ec80 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c13a440 con 0x7f819c141660 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 0x7f819c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 0x7f819c158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33886/0 (socket says 172.21.15.143:33886) 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 -- 172.21.15.143:0/4153709329 learned_addr learned my addr 172.21.15.143:0/4153709329 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819be00640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f819c15ec80 0x7f819c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819b400640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f819c141660 0x7f819c139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 -- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f819c141660 msgr2=0x7f819c139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f819c141660 0x7f819c139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.682 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 -- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f819c15ec80 msgr2=0x7f819c158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f819c15ec80 0x7f819c158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 -- 172.21.15.143:0/4153709329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840094d0 con 0x7f819c142470 2024-01-15T18:23:48.683 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819be00640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f819c15ec80 0x7f819c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f819aa00640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 0x7f819c158960 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f818401a0c0 tx=0x7f818401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840b3800 con 0x7f819c142470 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81840b3980 con 0x7f819c142470 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840b3d30 con 0x7f819c142470 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c0cd150 con 0x7f819c142470 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c13b980 con 0x7f819c142470 2024-01-15T18:23:48.684 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.691+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c13b540 con 0x7f819c142470 2024-01-15T18:23:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.695+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f81840b2380 con 0x7f819c142470 2024-01-15T18:23:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.695+0000 7f8193400640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8174067bb0 0x7f817406a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:48.685 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.695+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f81840a7070 con 0x7f819c142470 2024-01-15T18:23:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.695+0000 7f819b400640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8174067bb0 0x7f817406a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:48.686 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.695+0000 7f819b400640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8174067bb0 0x7f817406a070 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f819c13e410 tx=0x7f818c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:48.689 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.699+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f819c13b540 con 0x7f819c142470 2024-01-15T18:23:48.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.847+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f819c1593e0 con 0x7f8174067bb0 2024-01-15T18:23:48.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.847+0000 7f8193400640 1 -- 172.21.15.143:0/4153709329 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f819c1593e0 con 0x7f8174067bb0 2024-01-15T18:23:48.839 INFO:teuthology.orchestra.run.smithi143.stderr:instructed osd(s) 7 to deep-scrub 2024-01-15T18:23:48.841 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8174067bb0 msgr2=0x7f817406a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8174067bb0 0x7f817406a070 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f819c13e410 tx=0x7f818c008040 comp rx=0 tx=0).stop 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 msgr2=0x7f819c158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 0x7f819c158960 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f818401a0c0 tx=0x7f818401acd0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 shutdown_connections 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f8174067bb0 0x7f817406a070 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f819c15ec80 0x7f819c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f819c142470 0x7f819c158960 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 --2- 172.21.15.143:0/4153709329 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f819c141660 0x7f819c139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:48.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 >> 172.21.15.143:0/4153709329 conn(0x7f819c06c3f0 msgr2=0x7f819c140590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:48.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 shutdown_connections 2024-01-15T18:23:48.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:48.851+0000 7f81a2200640 1 -- 172.21.15.143:0/4153709329 wait complete. 2024-01-15T18:23:48.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:23:48.998 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.007+0000 7f756c600640 1 Processor -- start 2024-01-15T18:23:48.999 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.007+0000 7f756c600640 1 -- start start 2024-01-15T18:23:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.007+0000 7f756c600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75640b4300 0x7f75640ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.007+0000 7f756c600640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7564070bf0 con 0x7f75640ce690 2024-01-15T18:23:49.000 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.007+0000 7f756c600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75640b4920 con 0x7f75640b24b0 2024-01-15T18:23:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.007+0000 7f756c600640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756415dba0 con 0x7f75640b4300 2024-01-15T18:23:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f756a600640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f75640b4300 0x7f75640ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f7569c00640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f75640ce690 0x7f75640b1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50818/0 (socket says 172.21.15.143:50818) 2024-01-15T18:23:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f7569c00640 1 -- 172.21.15.143:0/1504404214 learned_addr learned my addr 172.21.15.143:0/1504404214 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:23:49.001 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f756a600640 1 -- 172.21.15.143:0/1504404214 >> v1:172.21.15.204:6789/0 conn(0x7f75640b24b0 legacy=0x7f75640b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f756a600640 1 -- 172.21.15.143:0/1504404214 >> v1:172.21.15.143:6789/0 conn(0x7f75640ce690 legacy=0x7f75640b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f7563e00640 1 -- 172.21.15.143:0/1504404214 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522077867 0 0) 0x7f7564070bf0 con 0x7f75640ce690 2024-01-15T18:23:49.002 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f756a600640 1 -- 172.21.15.143:0/1504404214 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75640b4c00 con 0x7f75640b4300 2024-01-15T18:23:49.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f756a600640 1 --2- 172.21.15.143:0/1504404214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640b4300 0x7f75640ac8e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c916f9b7e270b5f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:49.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.011+0000 7f7563e00640 1 -- 172.21.15.143:0/1504404214 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0a5a00 con 0x7f75640b4300 2024-01-15T18:23:49.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f7563e00640 1 -- 172.21.15.143:0/1504404214 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c0a5b60 con 0x7f75640b4300 2024-01-15T18:23:49.004 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f7563e00640 1 -- 172.21.15.143:0/1504404214 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0ad440 con 0x7f75640b4300 2024-01-15T18:23:49.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- 172.21.15.143:0/1504404214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640b4300 msgr2=0x7f75640ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.005 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 --2- 172.21.15.143:0/1504404214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640b4300 0x7f75640ac8e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f755c009d90 tx=0x7f755c01d380 comp rx=0 tx=0).stop 2024-01-15T18:23:49.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- 172.21.15.143:0/1504404214 shutdown_connections 2024-01-15T18:23:49.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 --2- 172.21.15.143:0/1504404214 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640b4300 0x7f75640ac8e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- 172.21.15.143:0/1504404214 >> 172.21.15.143:0/1504404214 conn(0x7f756406c3f0 msgr2=0x7f756406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:49.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- 172.21.15.143:0/1504404214 shutdown_connections 2024-01-15T18:23:49.006 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- 172.21.15.143:0/1504404214 wait complete. 2024-01-15T18:23:49.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 Processor -- start 2024-01-15T18:23:49.007 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- start start 2024-01-15T18:23:49.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75640b24b0 0x7f7564138640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:49.009 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75640b4300 0x7f7564145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 0x7f756415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756413d5f0 con 0x7f75640ce690 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756413c9e0 con 0x7f75640b24b0 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.015+0000 7f756c600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756413ccc0 con 0x7f75640b4300 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 0x7f756415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 0x7f756415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:33916/0 (socket says 172.21.15.143:33916) 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 -- 172.21.15.143:0/1163503347 learned_addr learned my addr 172.21.15.143:0/1163503347 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7569c00640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75640b24b0 0x7f7564138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:49.010 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7569200640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75640b4300 0x7f7564145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 -- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75640b4300 msgr2=0x7f7564145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75640b4300 0x7f7564145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 -- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75640b24b0 msgr2=0x7f7564138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75640b24b0 0x7f7564138640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 -- 172.21.15.143:0/1163503347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009a70 con 0x7f75640ce690 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7569c00640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75640b24b0 0x7f7564138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756a600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 0x7f756415e980 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f755c019530 tx=0x7f755c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0adb10 con 0x7f75640ce690 2024-01-15T18:23:49.011 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c0adc70 con 0x7f75640ce690 2024-01-15T18:23:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0b6400 con 0x7f75640ce690 2024-01-15T18:23:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756413cfa0 con 0x7f75640ce690 2024-01-15T18:23:49.012 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7564138b80 con 0x7f75640ce690 2024-01-15T18:23:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756413d760 con 0x7f75640ce690 2024-01-15T18:23:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.019+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f755c0b4040 con 0x7f75640ce690 2024-01-15T18:23:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.023+0000 7f7561600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f753c067b80 0x7f753c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:23:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.023+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f755c11dc70 con 0x7f75640ce690 2024-01-15T18:23:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.023+0000 7f7569c00640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f753c067b80 0x7f753c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:23:49.013 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.023+0000 7f7569c00640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f753c067b80 0x7f753c06a040 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7554000f70 tx=0x7f7554007dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:23:49.016 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.027+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f755c0ec7c0 con 0x7f75640ce690 2024-01-15T18:23:49.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.175+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7564067500 con 0x7f753c067b80 2024-01-15T18:23:49.176 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.187+0000 7f7561600640 1 -- 172.21.15.143:0/1163503347 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289254 (secure 0 0 0) 0x7f7564067500 con 0x7f753c067b80 2024-01-15T18:23:49.180 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:23:49.182 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:23:49.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f753c067b80 msgr2=0x7f753c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.183 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f753c067b80 0x7f753c06a040 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7554000f70 tx=0x7f7554007dd0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 msgr2=0x7f756415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 0x7f756415e980 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f755c019530 tx=0x7f755c0196c0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 shutdown_connections 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f753c067b80 0x7f753c06a040 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f75640ce690 0x7f756415e980 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f75640b4300 0x7f7564145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 --2- 172.21.15.143:0/1163503347 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f75640b24b0 0x7f7564138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:23:49.184 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 >> 172.21.15.143:0/1163503347 conn(0x7f756406c3f0 msgr2=0x7f756415fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:23:49.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 shutdown_connections 2024-01-15T18:23:49.185 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:23:49.191+0000 7f756c600640 1 -- 172.21.15.143:0/1163503347 wait complete. 2024-01-15T18:23:49.201 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":677,"stamp":"2024-01-15T18:23:48.976369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369906514,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1736,"num_write_kb":5246020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3179,"ondisk_log_size":3179,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":10967088,"kb_used_data":10437744,"kb_used_omap":12,"kb_used_meta":529267,"kb_avail":744007632,"statfs":{"total":773094113280,"available":761863815168,"internally_reserved":0,"allocated":10688249856,"data_stored":10681790068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":541969968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":148,"apply_latency_ms":148,"commit_latency_ns":148000000,"apply_latency_ns":148000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1073776589,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":256,"num_read_kb":1048576,"num_write":274,"num_write_kb":1048616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003058"},"pg_stats":[{"pgid":"17.3f","version":"113'53","reported_seq":78,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.247295+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:23:45.247295+0000","last_peered":"2024-01-15T18:23:45.247295+0000","last_clean":"2024-01-15T18:23:45.247295+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:23:45.247295+0000","last_undegraded":"2024-01-15T18:23:45.247295+0000","last_fullsized":"2024-01-15T18:23:45.247295+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'45","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.619274+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:23:45.619274+0000","last_peered":"2024-01-15T18:23:45.619274+0000","last_clean":"2024-01-15T18:23:45.619274+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:23:45.619274+0000","last_undegraded":"2024-01-15T18:23:45.619274+0000","last_fullsized":"2024-01-15T18:23:45.619274+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:03.613689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'44","reported_seq":67,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.766636+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:23:45.766636+0000","last_peered":"2024-01-15T18:23:45.766636+0000","last_clean":"2024-01-15T18:23:45.766636+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:23:45.766636+0000","last_undegraded":"2024-01-15T18:23:45.766636+0000","last_fullsized":"2024-01-15T18:23:45.766636+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:07.807378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'45","reported_seq":67,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.700720+0000","last_change":"2024-01-15T18:18:41.932250+0000","last_active":"2024-01-15T18:23:45.700720+0000","last_peered":"2024-01-15T18:23:45.700720+0000","last_clean":"2024-01-15T18:23:45.700720+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:45.700720+0000","last_undegraded":"2024-01-15T18:23:45.700720+0000","last_fullsized":"2024-01-15T18:23:45.700720+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'27","reported_seq":50,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.281888+0000","last_change":"2024-01-15T18:18:41.951758+0000","last_active":"2024-01-15T18:23:45.281888+0000","last_peered":"2024-01-15T18:23:45.281888+0000","last_clean":"2024-01-15T18:23:45.281888+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:23:45.281888+0000","last_undegraded":"2024-01-15T18:23:45.281888+0000","last_fullsized":"2024-01-15T18:23:45.281888+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'48","reported_seq":75,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.450633+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:23:45.450633+0000","last_peered":"2024-01-15T18:23:45.450633+0000","last_clean":"2024-01-15T18:23:45.450633+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:23:45.450633+0000","last_undegraded":"2024-01-15T18:23:45.450633+0000","last_fullsized":"2024-01-15T18:23:45.450633+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'43","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.350759+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:23:45.350759+0000","last_peered":"2024-01-15T18:23:45.350759+0000","last_clean":"2024-01-15T18:23:45.350759+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:23:45.350759+0000","last_undegraded":"2024-01-15T18:23:45.350759+0000","last_fullsized":"2024-01-15T18:23:45.350759+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:17:42.095962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'44","reported_seq":67,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.337538+0000","last_change":"2024-01-15T18:18:41.932178+0000","last_active":"2024-01-15T18:23:45.337538+0000","last_peered":"2024-01-15T18:23:45.337538+0000","last_clean":"2024-01-15T18:23:45.337538+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:23:45.337538+0000","last_undegraded":"2024-01-15T18:23:45.337538+0000","last_fullsized":"2024-01-15T18:23:45.337538+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:32.753279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'47","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.838934+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:23:45.838934+0000","last_peered":"2024-01-15T18:23:45.838934+0000","last_clean":"2024-01-15T18:23:45.838934+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:23:45.838934+0000","last_undegraded":"2024-01-15T18:23:45.838934+0000","last_fullsized":"2024-01-15T18:23:45.838934+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'47","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.953471+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:23:45.953471+0000","last_peered":"2024-01-15T18:23:45.953471+0000","last_clean":"2024-01-15T18:23:45.953471+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:23:45.953471+0000","last_undegraded":"2024-01-15T18:23:45.953471+0000","last_fullsized":"2024-01-15T18:23:45.953471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'47","reported_seq":74,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.628793+0000","last_change":"2024-01-15T18:18:41.934119+0000","last_active":"2024-01-15T18:23:45.628793+0000","last_peered":"2024-01-15T18:23:45.628793+0000","last_clean":"2024-01-15T18:23:45.628793+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:23:45.628793+0000","last_undegraded":"2024-01-15T18:23:45.628793+0000","last_fullsized":"2024-01-15T18:23:45.628793+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'48","reported_seq":80,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.754935+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:23:45.754935+0000","last_peered":"2024-01-15T18:23:45.754935+0000","last_clean":"2024-01-15T18:23:45.754935+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:23:45.754935+0000","last_undegraded":"2024-01-15T18:23:45.754935+0000","last_fullsized":"2024-01-15T18:23:45.754935+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'38","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.193404+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:23:45.193404+0000","last_peered":"2024-01-15T18:23:45.193404+0000","last_clean":"2024-01-15T18:23:45.193404+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:23:45.193404+0000","last_undegraded":"2024-01-15T18:23:45.193404+0000","last_fullsized":"2024-01-15T18:23:45.193404+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:27.425200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'51","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.870383+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:23:45.870383+0000","last_peered":"2024-01-15T18:23:45.870383+0000","last_clean":"2024-01-15T18:23:45.870383+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:23:45.870383+0000","last_undegraded":"2024-01-15T18:23:45.870383+0000","last_fullsized":"2024-01-15T18:23:45.870383+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:49:06.614151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'34","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.797457+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:23:45.797457+0000","last_peered":"2024-01-15T18:23:45.797457+0000","last_clean":"2024-01-15T18:23:45.797457+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:23:45.797457+0000","last_undegraded":"2024-01-15T18:23:45.797457+0000","last_fullsized":"2024-01-15T18:23:45.797457+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'42","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.345890+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:23:45.345890+0000","last_peered":"2024-01-15T18:23:45.345890+0000","last_clean":"2024-01-15T18:23:45.345890+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:23:45.345890+0000","last_undegraded":"2024-01-15T18:23:45.345890+0000","last_fullsized":"2024-01-15T18:23:45.345890+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'44","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.554174+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:23:45.554174+0000","last_peered":"2024-01-15T18:23:45.554174+0000","last_clean":"2024-01-15T18:23:45.554174+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:23:45.554174+0000","last_undegraded":"2024-01-15T18:23:45.554174+0000","last_fullsized":"2024-01-15T18:23:45.554174+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:25.057063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'48","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.792849+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:23:45.792849+0000","last_peered":"2024-01-15T18:23:45.792849+0000","last_clean":"2024-01-15T18:23:45.792849+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:23:45.792849+0000","last_undegraded":"2024-01-15T18:23:45.792849+0000","last_fullsized":"2024-01-15T18:23:45.792849+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:45.110826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'44","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.541181+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:23:45.541181+0000","last_peered":"2024-01-15T18:23:45.541181+0000","last_clean":"2024-01-15T18:23:45.541181+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:23:45.541181+0000","last_undegraded":"2024-01-15T18:23:45.541181+0000","last_fullsized":"2024-01-15T18:23:45.541181+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:54:56.642547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.793858+0000","last_change":"2024-01-15T18:18:41.927062+0000","last_active":"2024-01-15T18:23:45.793858+0000","last_peered":"2024-01-15T18:23:45.793858+0000","last_clean":"2024-01-15T18:23:45.793858+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:45.793858+0000","last_undegraded":"2024-01-15T18:23:45.793858+0000","last_fullsized":"2024-01-15T18:23:45.793858+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:00.986466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'56","reported_seq":84,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.518015+0000","last_change":"2024-01-15T18:18:41.937906+0000","last_active":"2024-01-15T18:23:45.518015+0000","last_peered":"2024-01-15T18:23:45.518015+0000","last_clean":"2024-01-15T18:23:45.518015+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:23:45.518015+0000","last_undegraded":"2024-01-15T18:23:45.518015+0000","last_fullsized":"2024-01-15T18:23:45.518015+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:20.422368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'47","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.021793+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:23:46.021793+0000","last_peered":"2024-01-15T18:23:46.021793+0000","last_clean":"2024-01-15T18:23:46.021793+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:23:46.021793+0000","last_undegraded":"2024-01-15T18:23:46.021793+0000","last_fullsized":"2024-01-15T18:23:46.021793+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'39","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.726122+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:23:45.726122+0000","last_peered":"2024-01-15T18:23:45.726122+0000","last_clean":"2024-01-15T18:23:45.726122+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:23:45.726122+0000","last_undegraded":"2024-01-15T18:23:45.726122+0000","last_fullsized":"2024-01-15T18:23:45.726122+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:07.193242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'43","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.972411+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:23:45.972411+0000","last_peered":"2024-01-15T18:23:45.972411+0000","last_clean":"2024-01-15T18:23:45.972411+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:23:45.972411+0000","last_undegraded":"2024-01-15T18:23:45.972411+0000","last_fullsized":"2024-01-15T18:23:45.972411+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'38","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.293933+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:23:45.293933+0000","last_peered":"2024-01-15T18:23:45.293933+0000","last_clean":"2024-01-15T18:23:45.293933+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:23:45.293933+0000","last_undegraded":"2024-01-15T18:23:45.293933+0000","last_fullsized":"2024-01-15T18:23:45.293933+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:12.667072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'53","reported_seq":87,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.985430+0000","last_change":"2024-01-15T18:19:42.172768+0000","last_active":"2024-01-15T18:23:45.985430+0000","last_peered":"2024-01-15T18:23:45.985430+0000","last_clean":"2024-01-15T18:23:45.985430+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:23:45.985430+0000","last_undegraded":"2024-01-15T18:23:45.985430+0000","last_fullsized":"2024-01-15T18:23:45.985430+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'37","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.336704+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:23:45.336704+0000","last_peered":"2024-01-15T18:23:45.336704+0000","last_clean":"2024-01-15T18:23:45.336704+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:23:45.336704+0000","last_undegraded":"2024-01-15T18:23:45.336704+0000","last_fullsized":"2024-01-15T18:23:45.336704+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":28673,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'38","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.972947+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:23:45.972947+0000","last_peered":"2024-01-15T18:23:45.972947+0000","last_clean":"2024-01-15T18:23:45.972947+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:23:45.972947+0000","last_undegraded":"2024-01-15T18:23:45.972947+0000","last_fullsized":"2024-01-15T18:23:45.972947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'46","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.830710+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:23:45.830710+0000","last_peered":"2024-01-15T18:23:45.830710+0000","last_clean":"2024-01-15T18:23:45.830710+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:23:45.830710+0000","last_undegraded":"2024-01-15T18:23:45.830710+0000","last_fullsized":"2024-01-15T18:23:45.830710+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:31:11.905380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'41","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.746033+0000","last_change":"2024-01-15T18:18:41.937559+0000","last_active":"2024-01-15T18:23:45.746033+0000","last_peered":"2024-01-15T18:23:45.746033+0000","last_clean":"2024-01-15T18:23:45.746033+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:23:45.746033+0000","last_undegraded":"2024-01-15T18:23:45.746033+0000","last_fullsized":"2024-01-15T18:23:45.746033+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:27.695371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'62","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.367607+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:23:45.367607+0000","last_peered":"2024-01-15T18:23:45.367607+0000","last_clean":"2024-01-15T18:23:45.367607+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:23:45.367607+0000","last_undegraded":"2024-01-15T18:23:45.367607+0000","last_fullsized":"2024-01-15T18:23:45.367607+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'54","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.655378+0000","last_change":"2024-01-15T18:18:41.951709+0000","last_active":"2024-01-15T18:23:45.655378+0000","last_peered":"2024-01-15T18:23:45.655378+0000","last_clean":"2024-01-15T18:23:45.655378+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:23:45.655378+0000","last_undegraded":"2024-01-15T18:23:45.655378+0000","last_fullsized":"2024-01-15T18:23:45.655378+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'34","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.480145+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:23:45.480145+0000","last_peered":"2024-01-15T18:23:45.480145+0000","last_clean":"2024-01-15T18:23:45.480145+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:23:45.480145+0000","last_undegraded":"2024-01-15T18:23:45.480145+0000","last_fullsized":"2024-01-15T18:23:45.480145+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"110'4","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655110+0000","last_change":"2024-01-15T18:18:40.923701+0000","last_active":"2024-01-15T18:20:46.655110+0000","last_peered":"2024-01-15T18:20:46.655110+0000","last_clean":"2024-01-15T18:20:46.655110+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:20:46.655110+0000","last_undegraded":"2024-01-15T18:20:46.655110+0000","last_fullsized":"2024-01-15T18:20:46.655110+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:01:16.930740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'47","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.160014+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:23:45.160014+0000","last_peered":"2024-01-15T18:23:45.160014+0000","last_clean":"2024-01-15T18:23:45.160014+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:23:45.160014+0000","last_undegraded":"2024-01-15T18:23:45.160014+0000","last_fullsized":"2024-01-15T18:23:45.160014+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:53:23.755030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652300+0000","last_change":"2024-01-15T18:18:40.922293+0000","last_active":"2024-01-15T18:20:46.652300+0000","last_peered":"2024-01-15T18:20:46.652300+0000","last_clean":"2024-01-15T18:20:46.652300+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:20:46.652300+0000","last_undegraded":"2024-01-15T18:20:46.652300+0000","last_fullsized":"2024-01-15T18:20:46.652300+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:24.049276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'43","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.641664+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:23:45.641664+0000","last_peered":"2024-01-15T18:23:45.641664+0000","last_clean":"2024-01-15T18:23:45.641664+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:23:45.641664+0000","last_undegraded":"2024-01-15T18:23:45.641664+0000","last_fullsized":"2024-01-15T18:23:45.641664+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"111'3","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:41.685167+0000","last_change":"2024-01-15T18:18:40.943416+0000","last_active":"2024-01-15T18:23:41.685167+0000","last_peered":"2024-01-15T18:23:41.685167+0000","last_clean":"2024-01-15T18:23:41.685167+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:23:41.685167+0000","last_undegraded":"2024-01-15T18:23:41.685167+0000","last_fullsized":"2024-01-15T18:23:41.685167+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'56","reported_seq":82,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.301232+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:23:45.301232+0000","last_peered":"2024-01-15T18:23:45.301232+0000","last_clean":"2024-01-15T18:23:45.301232+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:23:45.301232+0000","last_undegraded":"2024-01-15T18:23:45.301232+0000","last_fullsized":"2024-01-15T18:23:45.301232+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:02.239162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"110'2","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:41.685145+0000","last_change":"2024-01-15T18:18:40.942587+0000","last_active":"2024-01-15T18:23:41.685145+0000","last_peered":"2024-01-15T18:23:41.685145+0000","last_clean":"2024-01-15T18:23:41.685145+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:23:41.685145+0000","last_undegraded":"2024-01-15T18:23:41.685145+0000","last_fullsized":"2024-01-15T18:23:41.685145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'39","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.903009+0000","last_change":"2024-01-15T18:18:41.945518+0000","last_active":"2024-01-15T18:23:45.903009+0000","last_peered":"2024-01-15T18:23:45.903009+0000","last_clean":"2024-01-15T18:23:45.903009+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:45.903009+0000","last_undegraded":"2024-01-15T18:23:45.903009+0000","last_fullsized":"2024-01-15T18:23:45.903009+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:11.657812+0000","last_change":"2024-01-15T18:18:40.931709+0000","last_active":"2024-01-15T18:21:11.657812+0000","last_peered":"2024-01-15T18:21:11.657812+0000","last_clean":"2024-01-15T18:21:11.657812+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:21:11.657812+0000","last_undegraded":"2024-01-15T18:21:11.657812+0000","last_fullsized":"2024-01-15T18:21:11.657812+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:23.630535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'49","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.718847+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:23:45.718847+0000","last_peered":"2024-01-15T18:23:45.718847+0000","last_clean":"2024-01-15T18:23:45.718847+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:23:45.718847+0000","last_undegraded":"2024-01-15T18:23:45.718847+0000","last_fullsized":"2024-01-15T18:23:45.718847+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:35:40.652548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:42.314603+0000","last_change":"2024-01-15T18:18:40.932187+0000","last_active":"2024-01-15T18:23:42.314603+0000","last_peered":"2024-01-15T18:23:42.314603+0000","last_clean":"2024-01-15T18:23:42.314603+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:23:42.314603+0000","last_undegraded":"2024-01-15T18:23:42.314603+0000","last_fullsized":"2024-01-15T18:23:42.314603+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'53","reported_seq":89,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.713853+0000","last_change":"2024-01-15T18:19:42.184955+0000","last_active":"2024-01-15T18:23:45.713853+0000","last_peered":"2024-01-15T18:23:45.713853+0000","last_clean":"2024-01-15T18:23:45.713853+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:23:45.713853+0000","last_undegraded":"2024-01-15T18:23:45.713853+0000","last_fullsized":"2024-01-15T18:23:45.713853+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:19.878103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857792,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:11.655302+0000","last_change":"2024-01-15T18:18:40.921073+0000","last_active":"2024-01-15T18:21:11.655302+0000","last_peered":"2024-01-15T18:21:11.655302+0000","last_clean":"2024-01-15T18:21:11.655302+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:21:11.655302+0000","last_undegraded":"2024-01-15T18:21:11.655302+0000","last_fullsized":"2024-01-15T18:21:11.655302+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:15:44.103039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'43","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.475158+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:23:45.475158+0000","last_peered":"2024-01-15T18:23:45.475158+0000","last_clean":"2024-01-15T18:23:45.475158+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:23:45.475158+0000","last_undegraded":"2024-01-15T18:23:45.475158+0000","last_fullsized":"2024-01-15T18:23:45.475158+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:16.197896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:32.102802+0000","last_change":"2024-01-15T18:18:40.926810+0000","last_active":"2024-01-15T18:23:32.102802+0000","last_peered":"2024-01-15T18:23:32.102802+0000","last_clean":"2024-01-15T18:23:32.102802+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:23:32.102802+0000","last_undegraded":"2024-01-15T18:23:32.102802+0000","last_fullsized":"2024-01-15T18:23:32.102802+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'42","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:44.539959+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:23:44.539959+0000","last_peered":"2024-01-15T18:23:44.539959+0000","last_clean":"2024-01-15T18:23:44.539959+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:23:44.539959+0000","last_undegraded":"2024-01-15T18:23:44.539959+0000","last_fullsized":"2024-01-15T18:23:44.539959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:58.970330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:31.683922+0000","last_change":"2024-01-15T18:18:40.942628+0000","last_active":"2024-01-15T18:23:31.683922+0000","last_peered":"2024-01-15T18:23:31.683922+0000","last_clean":"2024-01-15T18:23:31.683922+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:23:31.683922+0000","last_undegraded":"2024-01-15T18:23:31.683922+0000","last_fullsized":"2024-01-15T18:23:31.683922+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'40","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.759139+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:23:45.759139+0000","last_peered":"2024-01-15T18:23:45.759139+0000","last_clean":"2024-01-15T18:23:45.759139+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:23:45.759139+0000","last_undegraded":"2024-01-15T18:23:45.759139+0000","last_fullsized":"2024-01-15T18:23:45.759139+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:40:35.571422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:32.313643+0000","last_change":"2024-01-15T18:18:40.931027+0000","last_active":"2024-01-15T18:23:32.313643+0000","last_peered":"2024-01-15T18:23:32.313643+0000","last_clean":"2024-01-15T18:23:32.313643+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:23:32.313643+0000","last_undegraded":"2024-01-15T18:23:32.313643+0000","last_fullsized":"2024-01-15T18:23:32.313643+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'43","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.001456+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:23:46.001456+0000","last_peered":"2024-01-15T18:23:46.001456+0000","last_clean":"2024-01-15T18:23:46.001456+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:23:46.001456+0000","last_undegraded":"2024-01-15T18:23:46.001456+0000","last_fullsized":"2024-01-15T18:23:46.001456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'11","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:32.102894+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:23:32.102894+0000","last_peered":"2024-01-15T18:23:32.102894+0000","last_clean":"2024-01-15T18:23:32.102894+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:23:32.102894+0000","last_undegraded":"2024-01-15T18:23:32.102894+0000","last_fullsized":"2024-01-15T18:23:32.102894+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'43","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.482927+0000","last_change":"2024-01-15T18:18:41.931596+0000","last_active":"2024-01-15T18:23:45.482927+0000","last_peered":"2024-01-15T18:23:45.482927+0000","last_clean":"2024-01-15T18:23:45.482927+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:23:45.482927+0000","last_undegraded":"2024-01-15T18:23:45.482927+0000","last_fullsized":"2024-01-15T18:23:45.482927+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:29.737906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'183","reported_seq":200,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.508310+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:23:45.508310+0000","last_peered":"2024-01-15T18:23:45.508310+0000","last_clean":"2024-01-15T18:23:45.508310+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:23:45.508310+0000","last_undegraded":"2024-01-15T18:23:45.508310+0000","last_fullsized":"2024-01-15T18:23:45.508310+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":603891,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'44","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.021337+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:23:46.021337+0000","last_peered":"2024-01-15T18:23:46.021337+0000","last_clean":"2024-01-15T18:23:46.021337+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:23:46.021337+0000","last_undegraded":"2024-01-15T18:23:46.021337+0000","last_fullsized":"2024-01-15T18:23:46.021337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"110'2","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:21.682075+0000","last_change":"2024-01-15T18:18:40.936647+0000","last_active":"2024-01-15T18:23:21.682075+0000","last_peered":"2024-01-15T18:23:21.682075+0000","last_clean":"2024-01-15T18:23:21.682075+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:23:21.682075+0000","last_undegraded":"2024-01-15T18:23:21.682075+0000","last_fullsized":"2024-01-15T18:23:21.682075+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'52","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.833319+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:23:45.833319+0000","last_peered":"2024-01-15T18:23:45.833319+0000","last_clean":"2024-01-15T18:23:45.833319+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:23:45.833319+0000","last_undegraded":"2024-01-15T18:23:45.833319+0000","last_fullsized":"2024-01-15T18:23:45.833319+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.653868+0000","last_change":"2024-01-15T18:18:40.914553+0000","last_active":"2024-01-15T18:21:01.653868+0000","last_peered":"2024-01-15T18:21:01.653868+0000","last_clean":"2024-01-15T18:21:01.653868+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:21:01.653868+0000","last_undegraded":"2024-01-15T18:21:01.653868+0000","last_fullsized":"2024-01-15T18:21:01.653868+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:04:16.633477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.778493+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:23:45.778493+0000","last_peered":"2024-01-15T18:23:45.778493+0000","last_clean":"2024-01-15T18:23:45.778493+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:23:45.778493+0000","last_undegraded":"2024-01-15T18:23:45.778493+0000","last_fullsized":"2024-01-15T18:23:45.778493+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663557,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"110'8","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:21.682335+0000","last_change":"2024-01-15T18:18:40.936329+0000","last_active":"2024-01-15T18:23:21.682335+0000","last_peered":"2024-01-15T18:23:21.682335+0000","last_clean":"2024-01-15T18:23:21.682335+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:23:21.682335+0000","last_undegraded":"2024-01-15T18:23:21.682335+0000","last_fullsized":"2024-01-15T18:23:21.682335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'24","reported_seq":44,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.952033+0000","last_change":"2024-01-15T18:18:41.936562+0000","last_active":"2024-01-15T18:23:45.952033+0000","last_peered":"2024-01-15T18:23:45.952033+0000","last_clean":"2024-01-15T18:23:45.952033+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:23:45.952033+0000","last_undegraded":"2024-01-15T18:23:45.952033+0000","last_fullsized":"2024-01-15T18:23:45.952033+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:12.100342+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:23:12.100342+0000","last_peered":"2024-01-15T18:23:12.100342+0000","last_clean":"2024-01-15T18:23:12.100342+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:23:12.100342+0000","last_undegraded":"2024-01-15T18:23:12.100342+0000","last_fullsized":"2024-01-15T18:23:12.100342+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'36","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.756073+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:23:45.756073+0000","last_peered":"2024-01-15T18:23:45.756073+0000","last_clean":"2024-01-15T18:23:45.756073+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:23:45.756073+0000","last_undegraded":"2024-01-15T18:23:45.756073+0000","last_fullsized":"2024-01-15T18:23:45.756073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"110'2","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:11.681053+0000","last_change":"2024-01-15T18:18:40.930009+0000","last_active":"2024-01-15T18:23:11.681053+0000","last_peered":"2024-01-15T18:23:11.681053+0000","last_clean":"2024-01-15T18:23:11.681053+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:23:11.681053+0000","last_undegraded":"2024-01-15T18:23:11.681053+0000","last_fullsized":"2024-01-15T18:23:11.681053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'44","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.660550+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:23:45.660550+0000","last_peered":"2024-01-15T18:23:45.660550+0000","last_clean":"2024-01-15T18:23:45.660550+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:23:45.660550+0000","last_undegraded":"2024-01-15T18:23:45.660550+0000","last_fullsized":"2024-01-15T18:23:45.660550+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:21:50.820872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"110'1","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653254+0000","last_change":"2024-01-15T18:18:40.916786+0000","last_active":"2024-01-15T18:20:56.653254+0000","last_peered":"2024-01-15T18:20:56.653254+0000","last_clean":"2024-01-15T18:20:56.653254+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:20:56.653254+0000","last_undegraded":"2024-01-15T18:20:56.653254+0000","last_fullsized":"2024-01-15T18:20:56.653254+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:00:59.694593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'41","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.761598+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:23:45.761598+0000","last_peered":"2024-01-15T18:23:45.761598+0000","last_clean":"2024-01-15T18:23:45.761598+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:23:45.761598+0000","last_undegraded":"2024-01-15T18:23:45.761598+0000","last_fullsized":"2024-01-15T18:23:45.761598+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:26.983917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.656234+0000","last_change":"2024-01-15T18:18:40.931379+0000","last_active":"2024-01-15T18:20:56.656234+0000","last_peered":"2024-01-15T18:20:56.656234+0000","last_clean":"2024-01-15T18:20:56.656234+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:20:56.656234+0000","last_undegraded":"2024-01-15T18:20:56.656234+0000","last_fullsized":"2024-01-15T18:20:56.656234+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:42.895411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'45","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.978894+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:23:45.978894+0000","last_peered":"2024-01-15T18:23:45.978894+0000","last_clean":"2024-01-15T18:23:45.978894+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:23:45.978894+0000","last_undegraded":"2024-01-15T18:23:45.978894+0000","last_fullsized":"2024-01-15T18:23:45.978894+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:24:57.458139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"111'3","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:11.680713+0000","last_change":"2024-01-15T18:18:40.940253+0000","last_active":"2024-01-15T18:23:11.680713+0000","last_peered":"2024-01-15T18:23:11.680713+0000","last_clean":"2024-01-15T18:23:11.680713+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:23:11.680713+0000","last_undegraded":"2024-01-15T18:23:11.680713+0000","last_fullsized":"2024-01-15T18:23:11.680713+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'43","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.369497+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:23:45.369497+0000","last_peered":"2024-01-15T18:23:45.369497+0000","last_clean":"2024-01-15T18:23:45.369497+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:23:45.369497+0000","last_undegraded":"2024-01-15T18:23:45.369497+0000","last_fullsized":"2024-01-15T18:23:45.369497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:06.680318+0000","last_change":"2024-01-15T18:18:40.945601+0000","last_active":"2024-01-15T18:23:06.680318+0000","last_peered":"2024-01-15T18:23:06.680318+0000","last_clean":"2024-01-15T18:23:06.680318+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:23:06.680318+0000","last_undegraded":"2024-01-15T18:23:06.680318+0000","last_fullsized":"2024-01-15T18:23:06.680318+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'44","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.695850+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:23:45.695850+0000","last_peered":"2024-01-15T18:23:45.695850+0000","last_clean":"2024-01-15T18:23:45.695850+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:23:45.695850+0000","last_undegraded":"2024-01-15T18:23:45.695850+0000","last_fullsized":"2024-01-15T18:23:45.695850+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:03.894036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:01.679704+0000","last_change":"2024-01-15T18:18:40.944626+0000","last_active":"2024-01-15T18:23:01.679704+0000","last_peered":"2024-01-15T18:23:01.679704+0000","last_clean":"2024-01-15T18:23:01.679704+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:23:01.679704+0000","last_undegraded":"2024-01-15T18:23:01.679704+0000","last_fullsized":"2024-01-15T18:23:01.679704+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'42","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.930016+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:23:45.930016+0000","last_peered":"2024-01-15T18:23:45.930016+0000","last_clean":"2024-01-15T18:23:45.930016+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:23:45.930016+0000","last_undegraded":"2024-01-15T18:23:45.930016+0000","last_fullsized":"2024-01-15T18:23:45.930016+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109125,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:02.098685+0000","last_change":"2024-01-15T18:18:40.933609+0000","last_active":"2024-01-15T18:23:02.098685+0000","last_peered":"2024-01-15T18:23:02.098685+0000","last_clean":"2024-01-15T18:23:02.098685+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:23:02.098685+0000","last_undegraded":"2024-01-15T18:23:02.098685+0000","last_fullsized":"2024-01-15T18:23:02.098685+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'36","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.661648+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:23:45.661648+0000","last_peered":"2024-01-15T18:23:45.661648+0000","last_clean":"2024-01-15T18:23:45.661648+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:23:45.661648+0000","last_undegraded":"2024-01-15T18:23:45.661648+0000","last_fullsized":"2024-01-15T18:23:45.661648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:29:11.051295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"110'2","reported_seq":19,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:02.309575+0000","last_change":"2024-01-15T18:18:40.938969+0000","last_active":"2024-01-15T18:23:02.309575+0000","last_peered":"2024-01-15T18:23:02.309575+0000","last_clean":"2024-01-15T18:23:02.309575+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:23:02.309575+0000","last_undegraded":"2024-01-15T18:23:02.309575+0000","last_fullsized":"2024-01-15T18:23:02.309575+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'41","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.477316+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:23:45.477316+0000","last_peered":"2024-01-15T18:23:45.477316+0000","last_clean":"2024-01-15T18:23:45.477316+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:23:45.477316+0000","last_undegraded":"2024-01-15T18:23:45.477316+0000","last_fullsized":"2024-01-15T18:23:45.477316+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'16","reported_seq":32,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:01.668095+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:23:01.668095+0000","last_peered":"2024-01-15T18:23:01.668095+0000","last_clean":"2024-01-15T18:23:01.668095+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:23:01.668095+0000","last_undegraded":"2024-01-15T18:23:01.668095+0000","last_fullsized":"2024-01-15T18:23:01.668095+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:51.313218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'46","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.650341+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:23:45.650341+0000","last_peered":"2024-01-15T18:23:45.650341+0000","last_clean":"2024-01-15T18:23:45.650341+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:23:45.650341+0000","last_undegraded":"2024-01-15T18:23:45.650341+0000","last_fullsized":"2024-01-15T18:23:45.650341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:52.097449+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:22:52.097449+0000","last_peered":"2024-01-15T18:22:52.097449+0000","last_clean":"2024-01-15T18:22:52.097449+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:22:52.097449+0000","last_undegraded":"2024-01-15T18:22:52.097449+0000","last_fullsized":"2024-01-15T18:22:52.097449+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'39","reported_seq":60,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.027732+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:23:45.027732+0000","last_peered":"2024-01-15T18:23:45.027732+0000","last_clean":"2024-01-15T18:23:45.027732+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:23:45.027732+0000","last_undegraded":"2024-01-15T18:23:45.027732+0000","last_fullsized":"2024-01-15T18:23:45.027732+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:27.713450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:52.308553+0000","last_change":"2024-01-15T18:18:40.940859+0000","last_active":"2024-01-15T18:22:52.308553+0000","last_peered":"2024-01-15T18:22:52.308553+0000","last_clean":"2024-01-15T18:22:52.308553+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:22:52.308553+0000","last_undegraded":"2024-01-15T18:22:52.308553+0000","last_fullsized":"2024-01-15T18:22:52.308553+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'44","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.792783+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:23:45.792783+0000","last_peered":"2024-01-15T18:23:45.792783+0000","last_clean":"2024-01-15T18:23:45.792783+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:23:45.792783+0000","last_undegraded":"2024-01-15T18:23:45.792783+0000","last_fullsized":"2024-01-15T18:23:45.792783+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:12.843266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.655069+0000","last_change":"2024-01-15T18:18:40.924285+0000","last_active":"2024-01-15T18:20:46.655069+0000","last_peered":"2024-01-15T18:20:46.655069+0000","last_clean":"2024-01-15T18:20:46.655069+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:20:46.655069+0000","last_undegraded":"2024-01-15T18:20:46.655069+0000","last_fullsized":"2024-01-15T18:20:46.655069+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:54.586280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'56","reported_seq":79,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.871991+0000","last_change":"2024-01-15T18:18:41.935993+0000","last_active":"2024-01-15T18:23:45.871991+0000","last_peered":"2024-01-15T18:23:45.871991+0000","last_clean":"2024-01-15T18:23:45.871991+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:23:45.871991+0000","last_undegraded":"2024-01-15T18:23:45.871991+0000","last_fullsized":"2024-01-15T18:23:45.871991+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:39:49.190850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:46.652145+0000","last_change":"2024-01-15T18:18:40.935934+0000","last_active":"2024-01-15T18:20:46.652145+0000","last_peered":"2024-01-15T18:20:46.652145+0000","last_clean":"2024-01-15T18:20:46.652145+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:20:46.652145+0000","last_undegraded":"2024-01-15T18:20:46.652145+0000","last_fullsized":"2024-01-15T18:20:46.652145+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:26.705662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.846604+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:23:45.846604+0000","last_peered":"2024-01-15T18:23:45.846604+0000","last_clean":"2024-01-15T18:23:45.846604+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:23:45.846604+0000","last_undegraded":"2024-01-15T18:23:45.846604+0000","last_fullsized":"2024-01-15T18:23:45.846604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:22:50.686671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'10","reported_seq":36,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:51.678454+0000","last_change":"2024-01-15T18:18:40.942710+0000","last_active":"2024-01-15T18:22:51.678454+0000","last_peered":"2024-01-15T18:22:51.678454+0000","last_clean":"2024-01-15T18:22:51.678454+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:22:51.678454+0000","last_undegraded":"2024-01-15T18:22:51.678454+0000","last_fullsized":"2024-01-15T18:22:51.678454+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'39","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.830536+0000","last_change":"2024-01-15T18:18:41.941571+0000","last_active":"2024-01-15T18:23:45.830536+0000","last_peered":"2024-01-15T18:23:45.830536+0000","last_clean":"2024-01-15T18:23:45.830536+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:45.830536+0000","last_undegraded":"2024-01-15T18:23:45.830536+0000","last_fullsized":"2024-01-15T18:23:45.830536+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:42.096036+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:22:42.096036+0000","last_peered":"2024-01-15T18:22:42.096036+0000","last_clean":"2024-01-15T18:22:42.096036+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:22:42.096036+0000","last_undegraded":"2024-01-15T18:22:42.096036+0000","last_fullsized":"2024-01-15T18:22:42.096036+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'31","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.998585+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:23:45.998585+0000","last_peered":"2024-01-15T18:23:45.998585+0000","last_clean":"2024-01-15T18:23:45.998585+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:23:45.998585+0000","last_undegraded":"2024-01-15T18:23:45.998585+0000","last_fullsized":"2024-01-15T18:23:45.998585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:21:01.656696+0000","last_change":"2024-01-15T18:18:40.930208+0000","last_active":"2024-01-15T18:21:01.656696+0000","last_peered":"2024-01-15T18:21:01.656696+0000","last_clean":"2024-01-15T18:21:01.656696+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:21:01.656696+0000","last_undegraded":"2024-01-15T18:21:01.656696+0000","last_fullsized":"2024-01-15T18:21:01.656696+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:35.260253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":246,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:20:56.653639+0000","last_change":"2024-01-15T18:04:48.760492+0000","last_active":"2024-01-15T18:20:56.653639+0000","last_peered":"2024-01-15T18:20:56.653639+0000","last_clean":"2024-01-15T18:20:56.653639+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:20:56.653639+0000","last_undegraded":"2024-01-15T18:20:56.653639+0000","last_fullsized":"2024-01-15T18:20:56.653639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:04:47.040191+0000","last_clean_scrub_stamp":"2024-01-15T18:04:47.040191+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:22.625272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710136,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":606010,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":271,"num_write_kb":709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1277952,"data_stored":1212020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":252,"ondisk_log_size":252,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575080,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834544,"kb_used_data":1767336,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537296,"statfs":{"total":96636764160,"available":94758191104,"internally_reserved":0,"allocated":1809752064,"data_stored":1808947458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69099999999999995,"15min":0.94399999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.29999999999999999},"max":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.87},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":13.744},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":1.0840000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":16.920000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":1.05},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":1.04,"15min":31.774999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72599999999999998,"15min":1},"min":{"1min":0.57399999999999995,"5min":0.48699999999999999,"15min":0.377},"max":{"1min":1.056,"5min":1.085,"15min":30.846},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":1.0629999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":1.0229999999999999,"15min":39.695},"last":0.79000000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74099999999999999,"15min":1.1499999999999999},"min":{"1min":0.54500000000000004,"5min":0.437,"15min":0.38800000000000001},"max":{"1min":1.01,"5min":1.0109999999999999,"15min":26.463999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":1.198},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.41499999999999998},"max":{"1min":0.86699999999999999,"5min":1.0640000000000001,"15min":42.692},"last":0.746}]},{"osd":4,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86699999999999999,"5min":0.874,"15min":14.087},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.36299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":2.952},"last":0.56899999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":11.334},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.746},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91700000000000004,"5min":0.94499999999999995,"15min":8.423},"last":0.54800000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.78200000000000003},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.45500000000000002},"max":{"1min":0.95099999999999996,"5min":1.034,"15min":8.5030000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.35299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":3.3450000000000002},"last":0.72999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1256364,"kb_used_data":1193316,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93115476,"statfs":{"total":96636764160,"available":95350247424,"internally_reserved":0,"allocated":1221955584,"data_stored":1221154106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":22.754000000000001},"last":0.498},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.82399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":1.018,"15min":9.1470000000000002},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.35299999999999998,"15min":0.34300000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":14.246},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":0.98199999999999998},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":0.97899999999999998,"15min":19.475000000000001},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":1.163},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.0549999999999999,"15min":42.825000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.747,"15min":1.107},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":1.103,"15min":42.851999999999997},"last":0.52300000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.98599999999999999},"min":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.8160000000000001,"15min":24.199000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":1.0700000000000001},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":1.03,"15min":18.791},"last":0.54500000000000004}]},{"osd":4,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":1.044,"15min":19.114999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":8.3059999999999992},"last":0.39400000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.88800000000000001,"5min":1.0920000000000001,"15min":7.5880000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":0.93500000000000005,"5min":1.1120000000000001,"15min":12.945},"last":0.63200000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69099999999999995,"15min":0.80100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.38600000000000001},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":11.907},"last":0.433},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.85399999999999998},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.86599999999999999,"5min":1.1910000000000001,"15min":27.651},"last":0.55900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575080,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":912496,"kb_used_data":836968,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93459344,"statfs":{"total":96636764160,"available":95702368256,"internally_reserved":0,"allocated":857055232,"data_stored":856250262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68400000000000005,"15min":0.85199999999999998},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.29899999999999999},"max":{"1min":0.98599999999999999,"5min":1.9350000000000001,"15min":11.519},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74099999999999999,"15min":0.94599999999999995},"min":{"1min":0.626,"5min":0.371,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":2.6819999999999999,"15min":12.952},"last":0.67000000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":2.1179999999999999,"15min":13.358000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.96699999999999997},"min":{"1min":0.56999999999999995,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":1.0289999999999999,"5min":2.8050000000000002,"15min":21.856999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":3.508,"15min":18.314},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":1.0449999999999999},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":1.069,"5min":3.5310000000000001,"15min":21.120000000000001},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":6.7990000000000004,"15min":13.317},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":1.0369999999999999},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":2.3380000000000001,"15min":19.638000000000002},"last":0.76200000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.96399999999999997,"15min":11.093},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.95699999999999996,"15min":3.3889999999999998},"last":0.80400000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.78900000000000003},"min":{"1min":0.5,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":5.8680000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32600000000000001},"max":{"1min":0.91300000000000003,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.69099999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":1.081,"15min":3.2759999999999998},"last":0.87},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.754,"15min":0.80300000000000005},"min":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.88300000000000001,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.89300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1670056,"kb_used_data":1607008,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92701784,"statfs":{"total":96636764160,"available":94926626816,"internally_reserved":0,"allocated":1645576192,"data_stored":1644778879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.94899999999999995},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":1.02,"15min":34.802},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.91300000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":0.871,"5min":1.1319999999999999,"15min":22.335999999999999},"last":2.6920000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":0.90600000000000003,"5min":1.224,"15min":38.198},"last":1.4850000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.752,"15min":0.96799999999999997},"min":{"1min":0.52400000000000002,"5min":0.438,"15min":0.372},"max":{"1min":0.875,"5min":1.0269999999999999,"15min":38.220999999999997},"last":1.5049999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.89600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.34599999999999997},"max":{"1min":0.91600000000000004,"5min":1.0509999999999999,"15min":7.6989999999999998},"last":1.877},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76000000000000001,"15min":0.85999999999999999},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":1.0740000000000001,"15min":7.4829999999999997},"last":1.173}]},{"osd":3,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.95199999999999996},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":1.214,"15min":14.529},"last":1.141},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.94699999999999995},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":25.774000000000001},"last":1.6910000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.68899999999999995},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.33200000000000002},"max":{"1min":0.89800000000000002,"5min":0.90300000000000002,"15min":9.9619999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.79700000000000004},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":1.1719999999999999,"15min":25.887},"last":0.61799999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":10.284000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.97299999999999998,"15min":3.6949999999999998},"last":0.49299999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":13.760999999999999},"last":11.093},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.36199999999999999},"max":{"1min":0.96699999999999997,"5min":1.0269999999999999,"15min":11.67},"last":1.3260000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1453640,"kb_used_data":1386432,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92918200,"statfs":{"total":96636764160,"available":95148236800,"internally_reserved":0,"allocated":1419706368,"data_stored":1418890685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.66200000000000003,"5min":2.415,"15min":26.364000000000001},"last":2.4079999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.95499999999999996},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.79500000000000004,"15min":28.332000000000001},"last":2.4359999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.98399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63300000000000001,"5min":0.82499999999999996,"15min":40.457999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":1.1120000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":27.427},"last":2.335}]},{"osd":2,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.98799999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":23.036999999999999},"last":2.7930000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":1.081},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.873,"15min":26.404},"last":2.3690000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":1.0349999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.0529999999999999,"5min":1.1439999999999999,"15min":22.998000000000001},"last":1.468},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76300000000000001,"15min":1.0740000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1,"5min":1.1439999999999999,"15min":20.399000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":1.0820000000000001},"min":{"1min":0.52800000000000002,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":0.97999999999999998,"5min":1.46,"15min":26.314},"last":2.2970000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":1.1040000000000001},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1,"5min":1.109,"15min":22.908999999999999},"last":1.7549999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79000000000000004,"15min":1.155},"min":{"1min":0.57399999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.105,"5min":1.105,"15min":20.445},"last":1.7190000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81399999999999995,"15min":1.1659999999999999},"min":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":1.4299999999999999,"15min":26.091999999999999},"last":2.2509999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":1.159},"min":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":23.111999999999998},"last":1.853},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":1.3169999999999999},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.46400000000000002},"max":{"1min":1.0880000000000001,"5min":1.145,"15min":44.817999999999998},"last":1.6779999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575080,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":965544,"kb_used_data":902496,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93406296,"statfs":{"total":96636764160,"available":95648047104,"internally_reserved":0,"allocated":924155904,"data_stored":923359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":1.054},"min":{"1min":0.28599999999999998,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.82899999999999996,"5min":4.2329999999999997,"15min":29.527999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":1.335},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":6.0670000000000002,"15min":34.122999999999998},"last":0.94299999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":1.534},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":4.5259999999999998,"15min":113.149},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59099999999999997,"15min":1.127},"min":{"1min":0.40500000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84899999999999998,"5min":6.1020000000000003,"15min":41.302},"last":0.79000000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":1.3899999999999999},"min":{"1min":0.34699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76800000000000002,"5min":4.7370000000000001,"15min":46.960000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.625,"15min":1.2809999999999999},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":3.8809999999999998,"15min":115.693},"last":0.61899999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83199999999999996,"15min":1.194},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":1.1100000000000001,"5min":3.8399999999999999,"15min":24.366},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":1.228},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.432},"max":{"1min":1.0189999999999999,"5min":2.343,"15min":20.489000000000001},"last":0.84399999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":1.5289999999999999},"min":{"1min":0.61899999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":1.175,"5min":3.927,"15min":41.683999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80400000000000005,"15min":1.2889999999999999},"min":{"1min":0.60799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98599999999999999,"5min":2.3029999999999999,"15min":26.084},"last":0.73599999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":1.319},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.159,"5min":6.0220000000000002,"15min":22.199000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":1.569},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.0860000000000001,"5min":3.9590000000000001,"15min":41.738999999999997},"last":0.67600000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85599999999999998,"15min":1.262},"min":{"1min":0.621,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.1339999999999999,"5min":2.556,"15min":20.446999999999999},"last":0.876},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":1.6060000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1599999999999999,"5min":2.5939999999999999,"15min":41.402000000000001},"last":0.70899999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1133560,"kb_used_data":1066352,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93238280,"statfs":{"total":96636764160,"available":95475998720,"internally_reserved":0,"allocated":1091944448,"data_stored":1091132307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.439,"15min":0.86099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.63700000000000001,"5min":1.4179999999999999,"15min":39.948},"last":1.8700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.56599999999999995,"15min":1.0649999999999999},"min":{"1min":0.30399999999999999,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":4.9329999999999998,"15min":35.790999999999997},"last":3.8010000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.57699999999999996,"15min":0.97099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.27600000000000002},"max":{"1min":0.66100000000000003,"5min":9.2609999999999992,"15min":49.229999999999997},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":1.7649999999999999,"15min":10.349},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":1.0589999999999999},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68300000000000005,"5min":0.84999999999999998,"15min":27.643999999999998},"last":3.278},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.90800000000000003},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":1.198,"15min":15.608000000000001},"last":1.02}]},{"osd":4,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76500000000000001,"15min":1.165},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":1.1080000000000001,"5min":1.383,"15min":27.489999999999998},"last":2.4889999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":1.0349999999999999},"min":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.218,"5min":6.6950000000000003,"15min":10.038},"last":0.75600000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76900000000000002,"15min":1.1779999999999999},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1739999999999999,"5min":1.2010000000000001,"15min":27.547000000000001},"last":2.5600000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.55400000000000005},"max":{"1min":1.1599999999999999,"5min":1.266,"15min":9.9610000000000003},"last":1.337}]},{"osd":6,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80600000000000005,"15min":1.171},"min":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.496},"max":{"1min":1.147,"5min":1.984,"15min":27.579999999999998},"last":2.605},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":1},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.42399999999999999},"max":{"1min":1.1910000000000001,"5min":5.6470000000000002,"15min":11.712999999999999},"last":1.6950000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":1.0980000000000001},"min":{"1min":0.56899999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":5.819,"15min":11.833},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.89000000000000001,"15min":1.0920000000000001},"min":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.502},"max":{"1min":1.254,"5min":6.3570000000000002,"15min":14.606},"last":12.673999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1740884,"kb_used_data":1677836,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92630956,"statfs":{"total":96636764160,"available":94854098944,"internally_reserved":0,"allocated":1718104064,"data_stored":1717276720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46300000000000002,"15min":1.054},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":31.977},"last":0.497},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.51300000000000001,"15min":1.0349999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.61499999999999999,"5min":1.6579999999999999,"15min":36.145000000000003},"last":0.57299999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":1.1439999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":1.1759999999999999,"15min":41.387999999999998},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":1.0089999999999999},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70399999999999996,"5min":1.7689999999999999,"15min":33.219999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":1.006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80500000000000005,"15min":31.943999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":1.107},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.746,"5min":0.95599999999999996,"15min":45.128},"last":0.47999999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.38300000000000001},"max":{"1min":1.0469999999999999,"5min":1.4550000000000001,"15min":32.932000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.77400000000000002,"15min":1.1659999999999999},"min":{"1min":0.53600000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":1.115,"15min":31.782},"last":0.56000000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":1.149},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.38600000000000001},"max":{"1min":1.0249999999999999,"5min":1.179,"15min":31.858000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.79300000000000004,"15min":1.0409999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.7869999999999999,"15min":33.151000000000003},"last":0.629}]},{"osd":6,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":1.0660000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":13.055},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.47699999999999998},"max":{"1min":1.262,"5min":1.7170000000000001,"15min":33.098999999999997},"last":0.65200000000000002}]},{"osd":7,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":1.097},"min":{"1min":0.68200000000000005,"5min":0.55500000000000005,"15min":0.48899999999999999},"max":{"1min":1.0029999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":1.2050000000000001},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":0.997,"5min":1.075,"15min":31.905999999999999},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":622592,"data_stored":604003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":618496,"data_stored":603961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1715482624,"data_stored":1715470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":922746880,"data_stored":922746880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":880807936,"data_stored":880804032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:23:49.208 INFO:tasks.ceph:pgid 17.3f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.3e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.3d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.3c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.3b last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.3a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.39 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.209 INFO:tasks.ceph:pgid 17.38 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.210 INFO:tasks.ceph:pgid 17.37 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.210 INFO:tasks.ceph:pgid 17.36 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.210 INFO:tasks.ceph:pgid 17.35 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.210 INFO:tasks.ceph:pgid 17.34 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.210 INFO:tasks.ceph:pgid 17.33 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.210 INFO:tasks.ceph:pgid 17.32 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.31 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.30 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.2f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.2e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.2d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.2c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.211 INFO:tasks.ceph:pgid 17.2b last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.212 INFO:tasks.ceph:pgid 17.2a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.212 INFO:tasks.ceph:pgid 17.29 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.212 INFO:tasks.ceph:pgid 17.28 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.212 INFO:tasks.ceph:pgid 17.27 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.212 INFO:tasks.ceph:pgid 17.26 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.25 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.24 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.23 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.22 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.21 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.20 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.213 INFO:tasks.ceph:pgid 17.1e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 16.1f last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 17.1f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 16.1e last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 17.1c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 16.1d last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 17.1d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.214 INFO:tasks.ceph:pgid 16.1c last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 17.1a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 16.1b last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 17.1b last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 16.1a last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 17.18 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 16.19 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 17.19 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.215 INFO:tasks.ceph:pgid 16.18 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.216 INFO:tasks.ceph:pgid 17.16 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.216 INFO:tasks.ceph:pgid 16.17 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.216 INFO:tasks.ceph:pgid 17.17 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.216 INFO:tasks.ceph:pgid 16.16 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.216 INFO:tasks.ceph:pgid 17.14 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.216 INFO:tasks.ceph:pgid 16.15 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 17.15 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 16.14 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 17.12 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 16.13 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 17.13 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 16.12 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.217 INFO:tasks.ceph:pgid 17.11 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 16.10 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 17.e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 16.f last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 17.f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 16.e last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 17.c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.218 INFO:tasks.ceph:pgid 16.d last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 17.d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 16.c last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 17.a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 16.b last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 17.b last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 16.a last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 17.8 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.219 INFO:tasks.ceph:pgid 16.9 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.220 INFO:tasks.ceph:pgid 17.9 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.220 INFO:tasks.ceph:pgid 16.8 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.220 INFO:tasks.ceph:pgid 17.6 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.220 INFO:tasks.ceph:pgid 16.7 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.220 INFO:tasks.ceph:pgid 17.7 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.220 INFO:tasks.ceph:pgid 16.6 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 17.4 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 16.5 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 17.5 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 16.4 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 17.2 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 16.3 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.221 INFO:tasks.ceph:pgid 17.3 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 16.2 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 17.0 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 16.1 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 17.1 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 16.0 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 17.10 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.222 INFO:tasks.ceph:pgid 16.11 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.223 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-01-15T18:04:47.040191+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=4, tm_sec=47, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:23:49.223 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-15T18:24:01.058 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:24:01.062 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:24:09.224 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:24:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 Processor -- start 2024-01-15T18:24:09.362 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 -- start start 2024-01-15T18:24:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7fd484157300 0x7fd484159b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:09.364 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4840b21d0 0x7fd4840b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4840b4d20 con 0x7fd4840b48f0 2024-01-15T18:24:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4840b35a0 con 0x7fd4840b21d0 2024-01-15T18:24:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd48c800640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4840b2a20 con 0x7fd484157300 2024-01-15T18:24:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd489400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4840b21d0 0x7fd4840b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:09.365 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd489400640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4840b21d0 0x7fd4840b43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37918/0 (socket says 172.21.15.143:37918) 2024-01-15T18:24:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.370+0000 7fd489400640 1 -- 172.21.15.143:0/3207320511 learned_addr learned my addr 172.21.15.143:0/3207320511 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:24:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489e00640 1 --2- 172.21.15.143:0/3207320511 >> v2:172.21.15.204:3300/0 conn(0x7fd484157300 0x7fd484159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489400640 1 -- 172.21.15.143:0/3207320511 >> v2:172.21.15.204:3300/0 conn(0x7fd484157300 msgr2=0x7fd484159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:24:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489400640 1 --2- 172.21.15.143:0/3207320511 >> v2:172.21.15.204:3300/0 conn(0x7fd484157300 0x7fd484159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489400640 1 -- 172.21.15.143:0/3207320511 >> v1:172.21.15.204:6789/0 conn(0x7fd4840b48f0 legacy=0x7fd48415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:09.366 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489400640 1 -- 172.21.15.143:0/3207320511 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840b2d00 con 0x7fd4840b21d0 2024-01-15T18:24:09.367 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489e00640 1 --2- 172.21.15.143:0/3207320511 >> v2:172.21.15.204:3300/0 conn(0x7fd484157300 0x7fd484159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:24:09.368 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd489400640 1 --2- 172.21.15.143:0/3207320511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b21d0 0x7fd4840b43b0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b1aa0 tx=0x7fd46c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948b835e548fb8bb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:09.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd488a00640 1 -- 172.21.15.143:0/3207320511 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0aa430 con 0x7fd4840b21d0 2024-01-15T18:24:09.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd488a00640 1 -- 172.21.15.143:0/3207320511 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c0aa590 con 0x7fd4840b21d0 2024-01-15T18:24:09.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.374+0000 7fd488a00640 1 -- 172.21.15.143:0/3207320511 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0aa920 con 0x7fd4840b21d0 2024-01-15T18:24:09.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- 172.21.15.143:0/3207320511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b21d0 msgr2=0x7fd4840b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:09.369 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 --2- 172.21.15.143:0/3207320511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b21d0 0x7fd4840b43b0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd4840b1aa0 tx=0x7fd46c01c4d0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- 172.21.15.143:0/3207320511 shutdown_connections 2024-01-15T18:24:09.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 --2- 172.21.15.143:0/3207320511 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b21d0 0x7fd4840b43b0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 --2- 172.21.15.143:0/3207320511 >> v2:172.21.15.204:3300/0 conn(0x7fd484157300 0x7fd484159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- 172.21.15.143:0/3207320511 >> 172.21.15.143:0/3207320511 conn(0x7fd48406c3f0 msgr2=0x7fd48406c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:09.370 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- 172.21.15.143:0/3207320511 shutdown_connections 2024-01-15T18:24:09.371 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- 172.21.15.143:0/3207320511 wait complete. 2024-01-15T18:24:09.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 Processor -- start 2024-01-15T18:24:09.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- start start 2024-01-15T18:24:09.374 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4840b21d0 0x7fd4841f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:09.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 0x7fd48414ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:09.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd484157300 0x7fd4840ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:09.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48414f4e0 con 0x7fd4840b48f0 2024-01-15T18:24:09.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48415ba70 con 0x7fd4840b21d0 2024-01-15T18:24:09.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48c800640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48415bd50 con 0x7fd484157300 2024-01-15T18:24:09.375 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd489400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 0x7fd48414ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd489400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 0x7fd48414ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37928/0 (socket says 172.21.15.143:37928) 2024-01-15T18:24:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd489400640 1 -- 172.21.15.143:0/3881912472 learned_addr learned my addr 172.21.15.143:0/3881912472 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:24:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd489e00640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4840b21d0 0x7fd4841f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.378+0000 7fd48a800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd484157300 0x7fd4840ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489400640 1 -- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd484157300 msgr2=0x7fd4840ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:09.376 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489400640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd484157300 0x7fd4840ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489400640 1 -- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4840b21d0 msgr2=0x7fd4841f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489400640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4840b21d0 0x7fd4841f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489400640 1 -- 172.21.15.143:0/3881912472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0094d0 con 0x7fd4840b48f0 2024-01-15T18:24:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489400640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 0x7fd48414ec20 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0190e0 tx=0x7fd46c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0b3800 con 0x7fd4840b48f0 2024-01-15T18:24:09.377 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c0b3960 con 0x7fd4840b48f0 2024-01-15T18:24:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48415c030 con 0x7fd4840b48f0 2024-01-15T18:24:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0b3d10 con 0x7fd4840b48f0 2024-01-15T18:24:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48414f7c0 con 0x7fd4840b48f0 2024-01-15T18:24:09.378 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48415c250 con 0x7fd4840b48f0 2024-01-15T18:24:09.379 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7fd46c0b2380 con 0x7fd4840b48f0 2024-01-15T18:24:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd482000640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd45c067b80 0x7fd45c06a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fd46c11d7c0 con 0x7fd4840b48f0 2024-01-15T18:24:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.382+0000 7fd489e00640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd45c067b80 0x7fd45c06a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.386+0000 7fd489e00640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd45c067b80 0x7fd45c06a040 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd474006e90 tx=0x7fd474008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:09.380 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.386+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7fd46c0ec480 con 0x7fd4840b48f0 2024-01-15T18:24:09.528 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.538+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd484067500 con 0x7fd45c067b80 2024-01-15T18:24:09.536 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.546+0000 7fd482000640 1 -- 172.21.15.143:0/3881912472 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289702 (secure 0 0 0) 0x7fd484067500 con 0x7fd45c067b80 2024-01-15T18:24:09.540 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:24:09.542 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:24:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd45c067b80 msgr2=0x7fd45c06a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd45c067b80 0x7fd45c06a040 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd474006e90 tx=0x7fd474008040 comp rx=0 tx=0).stop 2024-01-15T18:24:09.544 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 msgr2=0x7fd48414ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 0x7fd48414ec20 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0190e0 tx=0x7fd46c01ad10 comp rx=0 tx=0).stop 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 shutdown_connections 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7fd45c067b80 0x7fd45c06a040 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7fd484157300 0x7fd4840ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4840b48f0 0x7fd48414ec20 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 --2- 172.21.15.143:0/3881912472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd4840b21d0 0x7fd4841f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:09.545 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 >> 172.21.15.143:0/3881912472 conn(0x7fd48406c3f0 msgr2=0x7fd4840b0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 shutdown_connections 2024-01-15T18:24:09.546 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:09.550+0000 7fd48c800640 1 -- 172.21.15.143:0/3881912472 wait complete. 2024-01-15T18:24:09.561 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":687,"stamp":"2024-01-15T18:24:08.982358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369918897,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1744,"num_write_kb":5246035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3409,"ondisk_log_size":3409,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11028552,"kb_used_data":10499208,"kb_used_omap":12,"kb_used_meta":529267,"kb_avail":743946168,"statfs":{"total":773094113280,"available":761800876032,"internally_reserved":0,"allocated":10751188992,"data_stored":10744729394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":541969968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2942,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003567"},"pg_stats":[{"pgid":"17.3f","version":"113'53","reported_seq":78,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.247295+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:23:45.247295+0000","last_peered":"2024-01-15T18:23:45.247295+0000","last_clean":"2024-01-15T18:23:45.247295+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:23:45.247295+0000","last_undegraded":"2024-01-15T18:23:45.247295+0000","last_fullsized":"2024-01-15T18:23:45.247295+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'45","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.628291+0000","last_change":"2024-01-15T18:18:41.926378+0000","last_active":"2024-01-15T18:23:45.628291+0000","last_peered":"2024-01-15T18:23:45.628291+0000","last_clean":"2024-01-15T18:23:45.628291+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:23:45.628291+0000","last_undegraded":"2024-01-15T18:23:45.628291+0000","last_fullsized":"2024-01-15T18:23:45.628291+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'44","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.767210+0000","last_change":"2024-01-15T18:18:41.928574+0000","last_active":"2024-01-15T18:23:45.767210+0000","last_peered":"2024-01-15T18:23:45.767210+0000","last_clean":"2024-01-15T18:23:45.767210+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:23:45.767210+0000","last_undegraded":"2024-01-15T18:23:45.767210+0000","last_fullsized":"2024-01-15T18:23:45.767210+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'62","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:54.586712+0000","last_change":"2024-01-15T18:23:54.586767+0000","last_active":"2024-01-15T18:23:54.586712+0000","last_peered":"2024-01-15T18:23:54.586712+0000","last_clean":"2024-01-15T18:23:54.586712+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:54.586712+0000","last_undegraded":"2024-01-15T18:23:54.586712+0000","last_fullsized":"2024-01-15T18:23:54.586712+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:23:54.586620+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:23:54.586620+0000","last_clean_scrub_stamp":"2024-01-15T18:23:54.586620+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:04.906558+0000","scrub_duration":0.14577402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'35","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:03.545628+0000","last_change":"2024-01-15T18:24:03.545686+0000","last_active":"2024-01-15T18:24:03.545628+0000","last_peered":"2024-01-15T18:24:03.545628+0000","last_clean":"2024-01-15T18:24:03.545628+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:24:03.545628+0000","last_undegraded":"2024-01-15T18:24:03.545628+0000","last_fullsized":"2024-01-15T18:24:03.545628+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'35","last_scrub_stamp":"2024-01-15T18:24:03.545528+0000","last_deep_scrub":"113'35","last_deep_scrub_stamp":"2024-01-15T18:24:03.545528+0000","last_clean_scrub_stamp":"2024-01-15T18:24:03.545528+0000","objects_scrubbed":10,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:24:34.025807+0000","scrub_duration":0.072323520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'48","reported_seq":75,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.450633+0000","last_change":"2024-01-15T18:18:41.944214+0000","last_active":"2024-01-15T18:23:45.450633+0000","last_peered":"2024-01-15T18:23:45.450633+0000","last_clean":"2024-01-15T18:23:45.450633+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:23:45.450633+0000","last_undegraded":"2024-01-15T18:23:45.450633+0000","last_fullsized":"2024-01-15T18:23:45.450633+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'43","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.478924+0000","last_change":"2024-01-15T18:18:41.942410+0000","last_active":"2024-01-15T18:23:45.478924+0000","last_peered":"2024-01-15T18:23:45.478924+0000","last_clean":"2024-01-15T18:23:45.478924+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:23:45.478924+0000","last_undegraded":"2024-01-15T18:23:45.478924+0000","last_fullsized":"2024-01-15T18:23:45.478924+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'61","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:06.128647+0000","last_change":"2024-01-15T18:24:06.128704+0000","last_active":"2024-01-15T18:24:06.128647+0000","last_peered":"2024-01-15T18:24:06.128647+0000","last_clean":"2024-01-15T18:24:06.128647+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:24:06.128647+0000","last_undegraded":"2024-01-15T18:24:06.128647+0000","last_fullsized":"2024-01-15T18:24:06.128647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'61","last_scrub_stamp":"2024-01-15T18:24:06.128559+0000","last_deep_scrub":"113'61","last_deep_scrub_stamp":"2024-01-15T18:24:06.128559+0000","last_clean_scrub_stamp":"2024-01-15T18:24:06.128559+0000","objects_scrubbed":19,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:03.610416+0000","scrub_duration":0.116078944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'47","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.838934+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:23:45.838934+0000","last_peered":"2024-01-15T18:23:45.838934+0000","last_clean":"2024-01-15T18:23:45.838934+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:23:45.838934+0000","last_undegraded":"2024-01-15T18:23:45.838934+0000","last_fullsized":"2024-01-15T18:23:45.838934+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'47","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.953471+0000","last_change":"2024-01-15T18:18:41.940262+0000","last_active":"2024-01-15T18:23:45.953471+0000","last_peered":"2024-01-15T18:23:45.953471+0000","last_clean":"2024-01-15T18:23:45.953471+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:23:45.953471+0000","last_undegraded":"2024-01-15T18:23:45.953471+0000","last_fullsized":"2024-01-15T18:23:45.953471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'69","reported_seq":106,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.920167+0000","last_change":"2024-01-15T18:24:01.920191+0000","last_active":"2024-01-15T18:24:01.920167+0000","last_peered":"2024-01-15T18:24:01.920167+0000","last_clean":"2024-01-15T18:24:01.920167+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:24:01.920167+0000","last_undegraded":"2024-01-15T18:24:01.920167+0000","last_fullsized":"2024-01-15T18:24:01.920167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'69","last_scrub_stamp":"2024-01-15T18:24:01.920080+0000","last_deep_scrub":"113'69","last_deep_scrub_stamp":"2024-01-15T18:24:01.920080+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.920080+0000","objects_scrubbed":28,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:15.623343+0000","scrub_duration":0.20686890199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'48","reported_seq":80,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.754935+0000","last_change":"2024-01-15T18:18:41.937693+0000","last_active":"2024-01-15T18:23:45.754935+0000","last_peered":"2024-01-15T18:23:45.754935+0000","last_clean":"2024-01-15T18:23:45.754935+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:23:45.754935+0000","last_undegraded":"2024-01-15T18:23:45.754935+0000","last_fullsized":"2024-01-15T18:23:45.754935+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'38","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.471162+0000","last_change":"2024-01-15T18:18:41.931433+0000","last_active":"2024-01-15T18:23:45.471162+0000","last_peered":"2024-01-15T18:23:45.471162+0000","last_clean":"2024-01-15T18:23:45.471162+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:23:45.471162+0000","last_undegraded":"2024-01-15T18:23:45.471162+0000","last_fullsized":"2024-01-15T18:23:45.471162+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'51","reported_seq":74,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.957743+0000","last_change":"2024-01-15T18:18:41.939828+0000","last_active":"2024-01-15T18:23:45.957743+0000","last_peered":"2024-01-15T18:23:45.957743+0000","last_clean":"2024-01-15T18:23:45.957743+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:23:45.957743+0000","last_undegraded":"2024-01-15T18:23:45.957743+0000","last_fullsized":"2024-01-15T18:23:45.957743+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'34","reported_seq":54,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.797457+0000","last_change":"2024-01-15T18:18:41.951030+0000","last_active":"2024-01-15T18:23:45.797457+0000","last_peered":"2024-01-15T18:23:45.797457+0000","last_clean":"2024-01-15T18:23:45.797457+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:23:45.797457+0000","last_undegraded":"2024-01-15T18:23:45.797457+0000","last_fullsized":"2024-01-15T18:23:45.797457+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'42","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.345890+0000","last_change":"2024-01-15T18:18:41.931633+0000","last_active":"2024-01-15T18:23:45.345890+0000","last_peered":"2024-01-15T18:23:45.345890+0000","last_clean":"2024-01-15T18:23:45.345890+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:23:45.345890+0000","last_undegraded":"2024-01-15T18:23:45.345890+0000","last_fullsized":"2024-01-15T18:23:45.345890+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'44","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.849520+0000","last_change":"2024-01-15T18:18:41.939092+0000","last_active":"2024-01-15T18:23:45.849520+0000","last_peered":"2024-01-15T18:23:45.849520+0000","last_clean":"2024-01-15T18:23:45.849520+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:23:45.849520+0000","last_undegraded":"2024-01-15T18:23:45.849520+0000","last_fullsized":"2024-01-15T18:23:45.849520+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'48","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.898185+0000","last_change":"2024-01-15T18:18:41.925915+0000","last_active":"2024-01-15T18:23:45.898185+0000","last_peered":"2024-01-15T18:23:45.898185+0000","last_clean":"2024-01-15T18:23:45.898185+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:23:45.898185+0000","last_undegraded":"2024-01-15T18:23:45.898185+0000","last_fullsized":"2024-01-15T18:23:45.898185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'44","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.642989+0000","last_change":"2024-01-15T18:18:41.942676+0000","last_active":"2024-01-15T18:23:45.642989+0000","last_peered":"2024-01-15T18:23:45.642989+0000","last_clean":"2024-01-15T18:23:45.642989+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:23:45.642989+0000","last_undegraded":"2024-01-15T18:23:45.642989+0000","last_fullsized":"2024-01-15T18:23:45.642989+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'63","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.808942+0000","last_change":"2024-01-15T18:23:58.808978+0000","last_active":"2024-01-15T18:23:58.808942+0000","last_peered":"2024-01-15T18:23:58.808942+0000","last_clean":"2024-01-15T18:23:58.808942+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:58.808942+0000","last_undegraded":"2024-01-15T18:23:58.808942+0000","last_fullsized":"2024-01-15T18:23:58.808942+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:23:58.808865+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:23:58.808865+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.808865+0000","objects_scrubbed":17,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:01.715099+0000","scrub_duration":0.125894276,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'76","reported_seq":114,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:02.904727+0000","last_change":"2024-01-15T18:24:02.904748+0000","last_active":"2024-01-15T18:24:02.904727+0000","last_peered":"2024-01-15T18:24:02.904727+0000","last_clean":"2024-01-15T18:24:02.904727+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:24:02.904727+0000","last_undegraded":"2024-01-15T18:24:02.904727+0000","last_fullsized":"2024-01-15T18:24:02.904727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'76","last_scrub_stamp":"2024-01-15T18:24:02.904627+0000","last_deep_scrub":"113'76","last_deep_scrub_stamp":"2024-01-15T18:24:02.904627+0000","last_clean_scrub_stamp":"2024-01-15T18:24:02.904627+0000","objects_scrubbed":26,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:21.142713+0000","scrub_duration":0.16157696399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'47","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.021793+0000","last_change":"2024-01-15T18:18:41.942381+0000","last_active":"2024-01-15T18:23:46.021793+0000","last_peered":"2024-01-15T18:23:46.021793+0000","last_clean":"2024-01-15T18:23:46.021793+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:23:46.021793+0000","last_undegraded":"2024-01-15T18:23:46.021793+0000","last_fullsized":"2024-01-15T18:23:46.021793+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'39","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.775165+0000","last_change":"2024-01-15T18:18:41.926852+0000","last_active":"2024-01-15T18:23:45.775165+0000","last_peered":"2024-01-15T18:23:45.775165+0000","last_clean":"2024-01-15T18:23:45.775165+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:23:45.775165+0000","last_undegraded":"2024-01-15T18:23:45.775165+0000","last_fullsized":"2024-01-15T18:23:45.775165+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'43","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.972411+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:23:45.972411+0000","last_peered":"2024-01-15T18:23:45.972411+0000","last_clean":"2024-01-15T18:23:45.972411+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:23:45.972411+0000","last_undegraded":"2024-01-15T18:23:45.972411+0000","last_fullsized":"2024-01-15T18:23:45.972411+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'38","reported_seq":64,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.923097+0000","last_change":"2024-01-15T18:18:41.934159+0000","last_active":"2024-01-15T18:23:45.923097+0000","last_peered":"2024-01-15T18:23:45.923097+0000","last_clean":"2024-01-15T18:23:45.923097+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:23:45.923097+0000","last_undegraded":"2024-01-15T18:23:45.923097+0000","last_fullsized":"2024-01-15T18:23:45.923097+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'53","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.261134+0000","last_change":"2024-01-15T18:19:42.172768+0000","last_active":"2024-01-15T18:23:46.261134+0000","last_peered":"2024-01-15T18:23:46.261134+0000","last_clean":"2024-01-15T18:23:46.261134+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:23:46.261134+0000","last_undegraded":"2024-01-15T18:23:46.261134+0000","last_fullsized":"2024-01-15T18:23:46.261134+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'37","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.336704+0000","last_change":"2024-01-15T18:18:41.931705+0000","last_active":"2024-01-15T18:23:45.336704+0000","last_peered":"2024-01-15T18:23:45.336704+0000","last_clean":"2024-01-15T18:23:45.336704+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:23:45.336704+0000","last_undegraded":"2024-01-15T18:23:45.336704+0000","last_fullsized":"2024-01-15T18:23:45.336704+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":28673,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'38","reported_seq":57,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.972947+0000","last_change":"2024-01-15T18:18:41.953020+0000","last_active":"2024-01-15T18:23:45.972947+0000","last_peered":"2024-01-15T18:23:45.972947+0000","last_clean":"2024-01-15T18:23:45.972947+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:23:45.972947+0000","last_undegraded":"2024-01-15T18:23:45.972947+0000","last_fullsized":"2024-01-15T18:23:45.972947+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'46","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.886066+0000","last_change":"2024-01-15T18:18:41.930844+0000","last_active":"2024-01-15T18:23:45.886066+0000","last_peered":"2024-01-15T18:23:45.886066+0000","last_clean":"2024-01-15T18:23:45.886066+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:23:45.886066+0000","last_undegraded":"2024-01-15T18:23:45.886066+0000","last_fullsized":"2024-01-15T18:23:45.886066+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'54","reported_seq":87,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:00.920221+0000","last_change":"2024-01-15T18:24:00.920252+0000","last_active":"2024-01-15T18:24:00.920221+0000","last_peered":"2024-01-15T18:24:00.920221+0000","last_clean":"2024-01-15T18:24:00.920221+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:24:00.920221+0000","last_undegraded":"2024-01-15T18:24:00.920221+0000","last_fullsized":"2024-01-15T18:24:00.920221+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'54","last_scrub_stamp":"2024-01-15T18:24:00.920149+0000","last_deep_scrub":"113'54","last_deep_scrub_stamp":"2024-01-15T18:24:00.920149+0000","last_clean_scrub_stamp":"2024-01-15T18:24:00.920149+0000","objects_scrubbed":16,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:12:31.167250+0000","scrub_duration":0.10608548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'62","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.367607+0000","last_change":"2024-01-15T18:18:41.948603+0000","last_active":"2024-01-15T18:23:45.367607+0000","last_peered":"2024-01-15T18:23:45.367607+0000","last_clean":"2024-01-15T18:23:45.367607+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:23:45.367607+0000","last_undegraded":"2024-01-15T18:23:45.367607+0000","last_fullsized":"2024-01-15T18:23:45.367607+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'72","reported_seq":103,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.607585+0000","last_change":"2024-01-15T18:24:01.607622+0000","last_active":"2024-01-15T18:24:01.607585+0000","last_peered":"2024-01-15T18:24:01.607585+0000","last_clean":"2024-01-15T18:24:01.607585+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:24:01.607585+0000","last_undegraded":"2024-01-15T18:24:01.607585+0000","last_fullsized":"2024-01-15T18:24:01.607585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'72","last_scrub_stamp":"2024-01-15T18:24:01.607513+0000","last_deep_scrub":"113'72","last_deep_scrub_stamp":"2024-01-15T18:24:01.607513+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.607513+0000","objects_scrubbed":22,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:02.388269+0000","scrub_duration":0.15048887599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'34","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.480145+0000","last_change":"2024-01-15T18:18:41.941919+0000","last_active":"2024-01-15T18:23:45.480145+0000","last_peered":"2024-01-15T18:23:45.480145+0000","last_clean":"2024-01-15T18:23:45.480145+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:23:45.480145+0000","last_undegraded":"2024-01-15T18:23:45.480145+0000","last_fullsized":"2024-01-15T18:23:45.480145+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'6","reported_seq":31,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.867891+0000","last_change":"2024-01-15T18:23:57.867914+0000","last_active":"2024-01-15T18:23:57.867891+0000","last_peered":"2024-01-15T18:23:57.867891+0000","last_clean":"2024-01-15T18:23:57.867891+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:23:57.867891+0000","last_undegraded":"2024-01-15T18:23:57.867891+0000","last_fullsized":"2024-01-15T18:23:57.867891+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'6","last_scrub_stamp":"2024-01-15T18:23:57.867799+0000","last_deep_scrub":"113'6","last_deep_scrub_stamp":"2024-01-15T18:23:57.867799+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.867799+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:03:18.594537+0000","scrub_duration":0.0040798709999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'47","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.603258+0000","last_change":"2024-01-15T18:18:41.927754+0000","last_active":"2024-01-15T18:23:45.603258+0000","last_peered":"2024-01-15T18:23:45.603258+0000","last_clean":"2024-01-15T18:23:45.603258+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:23:45.603258+0000","last_undegraded":"2024-01-15T18:23:45.603258+0000","last_fullsized":"2024-01-15T18:23:45.603258+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.978967+0000","last_change":"2024-01-15T18:23:59.978990+0000","last_active":"2024-01-15T18:23:59.978967+0000","last_peered":"2024-01-15T18:23:59.978967+0000","last_clean":"2024-01-15T18:23:59.978967+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:23:59.978967+0000","last_undegraded":"2024-01-15T18:23:59.978967+0000","last_fullsized":"2024-01-15T18:23:59.978967+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:59.978827+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:59.978827+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.978827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:35.915671+0000","scrub_duration":0.0020127669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'43","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.641664+0000","last_change":"2024-01-15T18:18:41.943508+0000","last_active":"2024-01-15T18:23:45.641664+0000","last_peered":"2024-01-15T18:23:45.641664+0000","last_clean":"2024-01-15T18:23:45.641664+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:23:45.641664+0000","last_undegraded":"2024-01-15T18:23:45.641664+0000","last_fullsized":"2024-01-15T18:23:45.641664+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:51.632037+0000","last_change":"2024-01-15T18:23:51.632064+0000","last_active":"2024-01-15T18:23:51.632037+0000","last_peered":"2024-01-15T18:23:51.632037+0000","last_clean":"2024-01-15T18:23:51.632037+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:23:51.632037+0000","last_undegraded":"2024-01-15T18:23:51.632037+0000","last_fullsized":"2024-01-15T18:23:51.632037+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:51.631949+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:51.631949+0000","last_clean_scrub_stamp":"2024-01-15T18:23:51.631949+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:46:13.718266+0000","scrub_duration":0.003650155,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'56","reported_seq":83,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.830270+0000","last_change":"2024-01-15T18:18:41.947744+0000","last_active":"2024-01-15T18:23:45.830270+0000","last_peered":"2024-01-15T18:23:45.830270+0000","last_clean":"2024-01-15T18:23:45.830270+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:23:45.830270+0000","last_undegraded":"2024-01-15T18:23:45.830270+0000","last_fullsized":"2024-01-15T18:23:45.830270+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:02.499567+0000","last_change":"2024-01-15T18:24:02.499587+0000","last_active":"2024-01-15T18:24:02.499567+0000","last_peered":"2024-01-15T18:24:02.499567+0000","last_clean":"2024-01-15T18:24:02.499567+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:24:02.499567+0000","last_undegraded":"2024-01-15T18:24:02.499567+0000","last_fullsized":"2024-01-15T18:24:02.499567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:24:02.499486+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:24:02.499486+0000","last_clean_scrub_stamp":"2024-01-15T18:24:02.499486+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:00.198981+0000","scrub_duration":0.0029058650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'53","reported_seq":85,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.535064+0000","last_change":"2024-01-15T18:23:57.535097+0000","last_active":"2024-01-15T18:23:57.535064+0000","last_peered":"2024-01-15T18:23:57.535064+0000","last_clean":"2024-01-15T18:23:57.535064+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:57.535064+0000","last_undegraded":"2024-01-15T18:23:57.535064+0000","last_fullsized":"2024-01-15T18:23:57.535064+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:23:57.534962+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:23:57.534962+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.534962+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:34.769272+0000","scrub_duration":0.11579978,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:55.839729+0000","last_change":"2024-01-15T18:23:55.839753+0000","last_active":"2024-01-15T18:23:55.839729+0000","last_peered":"2024-01-15T18:23:55.839729+0000","last_clean":"2024-01-15T18:23:55.839729+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:23:55.839729+0000","last_undegraded":"2024-01-15T18:23:55.839729+0000","last_fullsized":"2024-01-15T18:23:55.839729+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:55.839629+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:55.839629+0000","last_clean_scrub_stamp":"2024-01-15T18:23:55.839629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:12.580092+0000","scrub_duration":0.0024064479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'49","reported_seq":78,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.870248+0000","last_change":"2024-01-15T18:18:41.936023+0000","last_active":"2024-01-15T18:23:45.870248+0000","last_peered":"2024-01-15T18:23:45.870248+0000","last_clean":"2024-01-15T18:23:45.870248+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:23:45.870248+0000","last_undegraded":"2024-01-15T18:23:45.870248+0000","last_fullsized":"2024-01-15T18:23:45.870248+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:47.658985+0000","last_change":"2024-01-15T18:23:47.659004+0000","last_active":"2024-01-15T18:23:47.658985+0000","last_peered":"2024-01-15T18:23:47.658985+0000","last_clean":"2024-01-15T18:23:47.658985+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:23:47.658985+0000","last_undegraded":"2024-01-15T18:23:47.658985+0000","last_fullsized":"2024-01-15T18:23:47.658985+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:47.658903+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:47.658903+0000","last_clean_scrub_stamp":"2024-01-15T18:23:47.658903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:12:17.906004+0000","scrub_duration":0.002056914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'53","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.872894+0000","last_change":"2024-01-15T18:19:42.184955+0000","last_active":"2024-01-15T18:23:45.872894+0000","last_peered":"2024-01-15T18:23:45.872894+0000","last_clean":"2024-01-15T18:23:45.872894+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:23:45.872894+0000","last_undegraded":"2024-01-15T18:23:45.872894+0000","last_fullsized":"2024-01-15T18:23:45.872894+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857792,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:55.766357+0000","last_change":"2024-01-15T18:23:55.766378+0000","last_active":"2024-01-15T18:23:55.766357+0000","last_peered":"2024-01-15T18:23:55.766357+0000","last_clean":"2024-01-15T18:23:55.766357+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:23:55.766357+0000","last_undegraded":"2024-01-15T18:23:55.766357+0000","last_fullsized":"2024-01-15T18:23:55.766357+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:55.766236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:55.766236+0000","last_clean_scrub_stamp":"2024-01-15T18:23:55.766236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:28.192190+0000","scrub_duration":0.0029978629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'43","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.872348+0000","last_change":"2024-01-15T18:18:41.941920+0000","last_active":"2024-01-15T18:23:45.872348+0000","last_peered":"2024-01-15T18:23:45.872348+0000","last_clean":"2024-01-15T18:23:45.872348+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:23:45.872348+0000","last_undegraded":"2024-01-15T18:23:45.872348+0000","last_fullsized":"2024-01-15T18:23:45.872348+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.457759+0000","last_change":"2024-01-15T18:23:59.457780+0000","last_active":"2024-01-15T18:23:59.457759+0000","last_peered":"2024-01-15T18:23:59.457759+0000","last_clean":"2024-01-15T18:23:59.457759+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:23:59.457759+0000","last_undegraded":"2024-01-15T18:23:59.457759+0000","last_fullsized":"2024-01-15T18:23:59.457759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.457652+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:36.928230+0000","scrub_duration":0.0021356639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'42","reported_seq":62,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.040144+0000","last_change":"2024-01-15T18:18:41.931360+0000","last_active":"2024-01-15T18:23:45.040144+0000","last_peered":"2024-01-15T18:23:45.040144+0000","last_clean":"2024-01-15T18:23:45.040144+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:23:45.040144+0000","last_undegraded":"2024-01-15T18:23:45.040144+0000","last_fullsized":"2024-01-15T18:23:45.040144+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.513102+0000","last_change":"2024-01-15T18:23:59.513138+0000","last_active":"2024-01-15T18:23:59.513102+0000","last_peered":"2024-01-15T18:23:59.513102+0000","last_clean":"2024-01-15T18:23:59.513102+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:23:59.513102+0000","last_undegraded":"2024-01-15T18:23:59.513102+0000","last_fullsized":"2024-01-15T18:23:59.513102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'1","last_scrub_stamp":"2024-01-15T18:23:59.513003+0000","last_deep_scrub":"111'1","last_deep_scrub_stamp":"2024-01-15T18:23:59.513003+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.513003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:34:17.358881+0000","scrub_duration":0.001554124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'40","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.837766+0000","last_change":"2024-01-15T18:18:41.925305+0000","last_active":"2024-01-15T18:23:45.837766+0000","last_peered":"2024-01-15T18:23:45.837766+0000","last_clean":"2024-01-15T18:23:45.837766+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:23:45.837766+0000","last_undegraded":"2024-01-15T18:23:45.837766+0000","last_fullsized":"2024-01-15T18:23:45.837766+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.756143+0000","last_change":"2024-01-15T18:23:59.756175+0000","last_active":"2024-01-15T18:23:59.756143+0000","last_peered":"2024-01-15T18:23:59.756143+0000","last_clean":"2024-01-15T18:23:59.756143+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:23:59.756143+0000","last_undegraded":"2024-01-15T18:23:59.756143+0000","last_fullsized":"2024-01-15T18:23:59.756143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.756033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.756033+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.756033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:34.482493+0000","scrub_duration":0.0022605020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'43","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.001456+0000","last_change":"2024-01-15T18:18:41.943046+0000","last_active":"2024-01-15T18:23:46.001456+0000","last_peered":"2024-01-15T18:23:46.001456+0000","last_clean":"2024-01-15T18:23:46.001456+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:23:46.001456+0000","last_undegraded":"2024-01-15T18:23:46.001456+0000","last_fullsized":"2024-01-15T18:23:46.001456+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'11","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:32.102894+0000","last_change":"2024-01-15T18:18:40.937973+0000","last_active":"2024-01-15T18:23:32.102894+0000","last_peered":"2024-01-15T18:23:32.102894+0000","last_clean":"2024-01-15T18:23:32.102894+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:23:32.102894+0000","last_undegraded":"2024-01-15T18:23:32.102894+0000","last_fullsized":"2024-01-15T18:23:32.102894+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'60","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:05.973582+0000","last_change":"2024-01-15T18:24:05.973644+0000","last_active":"2024-01-15T18:24:05.973582+0000","last_peered":"2024-01-15T18:24:05.973582+0000","last_clean":"2024-01-15T18:24:05.973582+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:24:05.973582+0000","last_undegraded":"2024-01-15T18:24:05.973582+0000","last_fullsized":"2024-01-15T18:24:05.973582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:05.973484+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:05.973484+0000","last_clean_scrub_stamp":"2024-01-15T18:24:05.973484+0000","objects_scrubbed":20,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:18:40.025089+0000","scrub_duration":0.14927474800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'187","reported_seq":204,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:52.052478+0000","last_change":"2024-01-15T18:18:40.927368+0000","last_active":"2024-01-15T18:23:52.052478+0000","last_peered":"2024-01-15T18:23:52.052478+0000","last_clean":"2024-01-15T18:23:52.052478+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:23:52.052478+0000","last_undegraded":"2024-01-15T18:23:52.052478+0000","last_fullsized":"2024-01-15T18:23:52.052478+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":616274,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'44","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.021337+0000","last_change":"2024-01-15T18:18:41.942190+0000","last_active":"2024-01-15T18:23:46.021337+0000","last_peered":"2024-01-15T18:23:46.021337+0000","last_clean":"2024-01-15T18:23:46.021337+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:23:46.021337+0000","last_undegraded":"2024-01-15T18:23:46.021337+0000","last_fullsized":"2024-01-15T18:23:46.021337+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.459728+0000","last_change":"2024-01-15T18:23:57.459775+0000","last_active":"2024-01-15T18:23:57.459728+0000","last_peered":"2024-01-15T18:23:57.459728+0000","last_clean":"2024-01-15T18:23:57.459728+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:23:57.459728+0000","last_undegraded":"2024-01-15T18:23:57.459728+0000","last_fullsized":"2024-01-15T18:23:57.459728+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:57.459625+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:57.459625+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.459625+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:51.586938+0000","scrub_duration":0.0053587779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'52","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.833319+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:23:45.833319+0000","last_peered":"2024-01-15T18:23:45.833319+0000","last_clean":"2024-01-15T18:23:45.833319+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:23:45.833319+0000","last_undegraded":"2024-01-15T18:23:45.833319+0000","last_fullsized":"2024-01-15T18:23:45.833319+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:54.393613+0000","last_change":"2024-01-15T18:23:54.393632+0000","last_active":"2024-01-15T18:23:54.393613+0000","last_peered":"2024-01-15T18:23:54.393613+0000","last_clean":"2024-01-15T18:23:54.393613+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:23:54.393613+0000","last_undegraded":"2024-01-15T18:23:54.393613+0000","last_fullsized":"2024-01-15T18:23:54.393613+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:54.393495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:54.393495+0000","last_clean_scrub_stamp":"2024-01-15T18:23:54.393495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:48:38.527529+0000","scrub_duration":0.0024808400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'46","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.778493+0000","last_change":"2024-01-15T18:18:41.944333+0000","last_active":"2024-01-15T18:23:45.778493+0000","last_peered":"2024-01-15T18:23:45.778493+0000","last_clean":"2024-01-15T18:23:45.778493+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:23:45.778493+0000","last_undegraded":"2024-01-15T18:23:45.778493+0000","last_fullsized":"2024-01-15T18:23:45.778493+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663557,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'12","reported_seq":37,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.559825+0000","last_change":"2024-01-15T18:23:58.559858+0000","last_active":"2024-01-15T18:23:58.559825+0000","last_peered":"2024-01-15T18:23:58.559825+0000","last_clean":"2024-01-15T18:23:58.559825+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:23:58.559825+0000","last_undegraded":"2024-01-15T18:23:58.559825+0000","last_fullsized":"2024-01-15T18:23:58.559825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'12","last_scrub_stamp":"2024-01-15T18:23:58.559722+0000","last_deep_scrub":"113'12","last_deep_scrub_stamp":"2024-01-15T18:23:58.559722+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.559722+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:38.037968+0000","scrub_duration":0.0054070120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'24","reported_seq":44,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.952033+0000","last_change":"2024-01-15T18:18:41.936562+0000","last_active":"2024-01-15T18:23:45.952033+0000","last_peered":"2024-01-15T18:23:45.952033+0000","last_clean":"2024-01-15T18:23:45.952033+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:23:45.952033+0000","last_undegraded":"2024-01-15T18:23:45.952033+0000","last_fullsized":"2024-01-15T18:23:45.952033+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"110'1","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:12.100342+0000","last_change":"2024-01-15T18:18:40.937846+0000","last_active":"2024-01-15T18:23:12.100342+0000","last_peered":"2024-01-15T18:23:12.100342+0000","last_clean":"2024-01-15T18:23:12.100342+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:23:12.100342+0000","last_undegraded":"2024-01-15T18:23:12.100342+0000","last_fullsized":"2024-01-15T18:23:12.100342+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'36","reported_seq":63,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.756073+0000","last_change":"2024-01-15T18:18:41.932198+0000","last_active":"2024-01-15T18:23:45.756073+0000","last_peered":"2024-01-15T18:23:45.756073+0000","last_clean":"2024-01-15T18:23:45.756073+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:23:45.756073+0000","last_undegraded":"2024-01-15T18:23:45.756073+0000","last_fullsized":"2024-01-15T18:23:45.756073+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.632496+0000","last_change":"2024-01-15T18:23:48.632517+0000","last_active":"2024-01-15T18:23:48.632496+0000","last_peered":"2024-01-15T18:23:48.632496+0000","last_clean":"2024-01-15T18:23:48.632496+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:23:48.632496+0000","last_undegraded":"2024-01-15T18:23:48.632496+0000","last_fullsized":"2024-01-15T18:23:48.632496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:23:48.632420+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:23:48.632420+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.632420+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:03.214903+0000","scrub_duration":0.0038853939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'44","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.674067+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:23:45.674067+0000","last_peered":"2024-01-15T18:23:45.674067+0000","last_clean":"2024-01-15T18:23:45.674067+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:23:45.674067+0000","last_undegraded":"2024-01-15T18:23:45.674067+0000","last_fullsized":"2024-01-15T18:23:45.674067+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'2","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:04.477880+0000","last_change":"2024-01-15T18:24:04.477901+0000","last_active":"2024-01-15T18:24:04.477880+0000","last_peered":"2024-01-15T18:24:04.477880+0000","last_clean":"2024-01-15T18:24:04.477880+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:24:04.477880+0000","last_undegraded":"2024-01-15T18:24:04.477880+0000","last_fullsized":"2024-01-15T18:24:04.477880+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-01-15T18:24:04.477798+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-01-15T18:24:04.477798+0000","last_clean_scrub_stamp":"2024-01-15T18:24:04.477798+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:31.016306+0000","scrub_duration":0.0041278030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'41","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.845717+0000","last_change":"2024-01-15T18:18:41.933686+0000","last_active":"2024-01-15T18:23:45.845717+0000","last_peered":"2024-01-15T18:23:45.845717+0000","last_clean":"2024-01-15T18:23:45.845717+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:23:45.845717+0000","last_undegraded":"2024-01-15T18:23:45.845717+0000","last_fullsized":"2024-01-15T18:23:45.845717+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.819756+0000","last_change":"2024-01-15T18:23:58.819772+0000","last_active":"2024-01-15T18:23:58.819756+0000","last_peered":"2024-01-15T18:23:58.819756+0000","last_clean":"2024-01-15T18:23:58.819756+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:23:58.819756+0000","last_undegraded":"2024-01-15T18:23:58.819756+0000","last_fullsized":"2024-01-15T18:23:58.819756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:58.819676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:58.819676+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.819676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:01.186201+0000","scrub_duration":0.0020202330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'45","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.006235+0000","last_change":"2024-01-15T18:18:41.929500+0000","last_active":"2024-01-15T18:23:46.006235+0000","last_peered":"2024-01-15T18:23:46.006235+0000","last_clean":"2024-01-15T18:23:46.006235+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:23:46.006235+0000","last_undegraded":"2024-01-15T18:23:46.006235+0000","last_fullsized":"2024-01-15T18:23:46.006235+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:50.448326+0000","last_change":"2024-01-15T18:23:50.448372+0000","last_active":"2024-01-15T18:23:50.448326+0000","last_peered":"2024-01-15T18:23:50.448326+0000","last_clean":"2024-01-15T18:23:50.448326+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:23:50.448326+0000","last_undegraded":"2024-01-15T18:23:50.448326+0000","last_fullsized":"2024-01-15T18:23:50.448326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:50.448207+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:50.448207+0000","last_clean_scrub_stamp":"2024-01-15T18:23:50.448207+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:29.926453+0000","scrub_duration":0.0039608999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'43","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.369497+0000","last_change":"2024-01-15T18:18:41.931595+0000","last_active":"2024-01-15T18:23:45.369497+0000","last_peered":"2024-01-15T18:23:45.369497+0000","last_clean":"2024-01-15T18:23:45.369497+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:23:45.369497+0000","last_undegraded":"2024-01-15T18:23:45.369497+0000","last_fullsized":"2024-01-15T18:23:45.369497+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:52.641865+0000","last_change":"2024-01-15T18:23:52.641887+0000","last_active":"2024-01-15T18:23:52.641865+0000","last_peered":"2024-01-15T18:23:52.641865+0000","last_clean":"2024-01-15T18:23:52.641865+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:23:52.641865+0000","last_undegraded":"2024-01-15T18:23:52.641865+0000","last_fullsized":"2024-01-15T18:23:52.641865+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:52.641753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:52.641753+0000","last_clean_scrub_stamp":"2024-01-15T18:23:52.641753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:20:47.869635+0000","scrub_duration":0.0018539540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'44","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.895856+0000","last_change":"2024-01-15T18:18:41.950295+0000","last_active":"2024-01-15T18:23:45.895856+0000","last_peered":"2024-01-15T18:23:45.895856+0000","last_clean":"2024-01-15T18:23:45.895856+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:23:45.895856+0000","last_undegraded":"2024-01-15T18:23:45.895856+0000","last_fullsized":"2024-01-15T18:23:45.895856+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.472252+0000","last_change":"2024-01-15T18:23:59.472271+0000","last_active":"2024-01-15T18:23:59.472252+0000","last_peered":"2024-01-15T18:23:59.472252+0000","last_clean":"2024-01-15T18:23:59.472252+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:23:59.472252+0000","last_undegraded":"2024-01-15T18:23:59.472252+0000","last_fullsized":"2024-01-15T18:23:59.472252+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.472157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.472157+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.472157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:16:00.507773+0000","scrub_duration":0.00163682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'42","reported_seq":65,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.930016+0000","last_change":"2024-01-15T18:18:41.936455+0000","last_active":"2024-01-15T18:23:45.930016+0000","last_peered":"2024-01-15T18:23:45.930016+0000","last_clean":"2024-01-15T18:23:45.930016+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:23:45.930016+0000","last_undegraded":"2024-01-15T18:23:45.930016+0000","last_fullsized":"2024-01-15T18:23:45.930016+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109125,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.426929+0000","last_change":"2024-01-15T18:24:01.426948+0000","last_active":"2024-01-15T18:24:01.426929+0000","last_peered":"2024-01-15T18:24:01.426929+0000","last_clean":"2024-01-15T18:24:01.426929+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:24:01.426929+0000","last_undegraded":"2024-01-15T18:24:01.426929+0000","last_fullsized":"2024-01-15T18:24:01.426929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:01.426834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:01.426834+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.426834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:39.092898+0000","scrub_duration":0.0023004760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'36","reported_seq":56,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.840025+0000","last_change":"2024-01-15T18:18:41.931326+0000","last_active":"2024-01-15T18:23:45.840025+0000","last_peered":"2024-01-15T18:23:45.840025+0000","last_clean":"2024-01-15T18:23:45.840025+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:23:45.840025+0000","last_undegraded":"2024-01-15T18:23:45.840025+0000","last_fullsized":"2024-01-15T18:23:45.840025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.755286+0000","last_change":"2024-01-15T18:24:01.755309+0000","last_active":"2024-01-15T18:24:01.755286+0000","last_peered":"2024-01-15T18:24:01.755286+0000","last_clean":"2024-01-15T18:24:01.755286+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:24:01.755286+0000","last_undegraded":"2024-01-15T18:24:01.755286+0000","last_fullsized":"2024-01-15T18:24:01.755286+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:24:01.755211+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:24:01.755211+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.755211+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:34.181165+0000","scrub_duration":0.002739652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'41","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.477316+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:23:45.477316+0000","last_peered":"2024-01-15T18:23:45.477316+0000","last_clean":"2024-01-15T18:23:45.477316+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:23:45.477316+0000","last_undegraded":"2024-01-15T18:23:45.477316+0000","last_fullsized":"2024-01-15T18:23:45.477316+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'18","reported_seq":35,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.321628+0000","last_change":"2024-01-15T18:18:40.922156+0000","last_active":"2024-01-15T18:23:48.321628+0000","last_peered":"2024-01-15T18:23:48.321628+0000","last_clean":"2024-01-15T18:23:48.321628+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:23:48.321628+0000","last_undegraded":"2024-01-15T18:23:48.321628+0000","last_fullsized":"2024-01-15T18:23:48.321628+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'46","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.650341+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:23:45.650341+0000","last_peered":"2024-01-15T18:23:45.650341+0000","last_clean":"2024-01-15T18:23:45.650341+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:23:45.650341+0000","last_undegraded":"2024-01-15T18:23:45.650341+0000","last_fullsized":"2024-01-15T18:23:45.650341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:52.097449+0000","last_change":"2024-01-15T18:18:40.935391+0000","last_active":"2024-01-15T18:22:52.097449+0000","last_peered":"2024-01-15T18:22:52.097449+0000","last_clean":"2024-01-15T18:22:52.097449+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:22:52.097449+0000","last_undegraded":"2024-01-15T18:22:52.097449+0000","last_fullsized":"2024-01-15T18:22:52.097449+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'39","reported_seq":61,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.174085+0000","last_change":"2024-01-15T18:18:41.928769+0000","last_active":"2024-01-15T18:23:45.174085+0000","last_peered":"2024-01-15T18:23:45.174085+0000","last_clean":"2024-01-15T18:23:45.174085+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:23:45.174085+0000","last_undegraded":"2024-01-15T18:23:45.174085+0000","last_fullsized":"2024-01-15T18:23:45.174085+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.770011+0000","last_change":"2024-01-15T18:23:58.770042+0000","last_active":"2024-01-15T18:23:58.770011+0000","last_peered":"2024-01-15T18:23:58.770011+0000","last_clean":"2024-01-15T18:23:58.770011+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:23:58.770011+0000","last_undegraded":"2024-01-15T18:23:58.770011+0000","last_fullsized":"2024-01-15T18:23:58.770011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:58.769909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:58.769909+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.769909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:46.967528+0000","scrub_duration":0.0018566699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'44","reported_seq":67,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.955653+0000","last_change":"2024-01-15T18:18:41.937145+0000","last_active":"2024-01-15T18:23:45.955653+0000","last_peered":"2024-01-15T18:23:45.955653+0000","last_clean":"2024-01-15T18:23:45.955653+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:23:45.955653+0000","last_undegraded":"2024-01-15T18:23:45.955653+0000","last_fullsized":"2024-01-15T18:23:45.955653+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:49.798222+0000","last_change":"2024-01-15T18:23:49.798239+0000","last_active":"2024-01-15T18:23:49.798222+0000","last_peered":"2024-01-15T18:23:49.798222+0000","last_clean":"2024-01-15T18:23:49.798222+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:23:49.798222+0000","last_undegraded":"2024-01-15T18:23:49.798222+0000","last_fullsized":"2024-01-15T18:23:49.798222+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:49.798143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:49.798143+0000","last_clean_scrub_stamp":"2024-01-15T18:23:49.798143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:43.688382+0000","scrub_duration":0.0014863820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'80","reported_seq":113,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:03.884188+0000","last_change":"2024-01-15T18:24:03.884213+0000","last_active":"2024-01-15T18:24:03.884188+0000","last_peered":"2024-01-15T18:24:03.884188+0000","last_clean":"2024-01-15T18:24:03.884188+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:24:03.884188+0000","last_undegraded":"2024-01-15T18:24:03.884188+0000","last_fullsized":"2024-01-15T18:24:03.884188+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'80","last_scrub_stamp":"2024-01-15T18:24:03.884114+0000","last_deep_scrub":"113'80","last_deep_scrub_stamp":"2024-01-15T18:24:03.884114+0000","last_clean_scrub_stamp":"2024-01-15T18:24:03.884114+0000","objects_scrubbed":29,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:58.010620+0000","scrub_duration":0.17135824699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.761653+0000","last_change":"2024-01-15T18:23:48.761672+0000","last_active":"2024-01-15T18:23:48.761653+0000","last_peered":"2024-01-15T18:23:48.761653+0000","last_clean":"2024-01-15T18:23:48.761653+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:23:48.761653+0000","last_undegraded":"2024-01-15T18:23:48.761653+0000","last_fullsized":"2024-01-15T18:23:48.761653+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:48.761549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:48.761549+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.761549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:23.488009+0000","scrub_duration":0.0019591489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'46","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.009723+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:23:46.009723+0000","last_peered":"2024-01-15T18:23:46.009723+0000","last_clean":"2024-01-15T18:23:46.009723+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:23:46.009723+0000","last_undegraded":"2024-01-15T18:23:46.009723+0000","last_fullsized":"2024-01-15T18:23:46.009723+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'11","reported_seq":45,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:53.596663+0000","last_change":"2024-01-15T18:23:53.596683+0000","last_active":"2024-01-15T18:23:53.596663+0000","last_peered":"2024-01-15T18:23:53.596663+0000","last_clean":"2024-01-15T18:23:53.596663+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:23:53.596663+0000","last_undegraded":"2024-01-15T18:23:53.596663+0000","last_fullsized":"2024-01-15T18:23:53.596663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'11","last_scrub_stamp":"2024-01-15T18:23:53.596595+0000","last_deep_scrub":"113'11","last_deep_scrub_stamp":"2024-01-15T18:23:53.596595+0000","last_clean_scrub_stamp":"2024-01-15T18:23:53.596595+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:48.256546+0000","scrub_duration":0.0028069950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'59","reported_seq":91,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:53.558536+0000","last_change":"2024-01-15T18:23:53.558572+0000","last_active":"2024-01-15T18:23:53.558536+0000","last_peered":"2024-01-15T18:23:53.558536+0000","last_clean":"2024-01-15T18:23:53.558536+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:53.558536+0000","last_undegraded":"2024-01-15T18:23:53.558536+0000","last_fullsized":"2024-01-15T18:23:53.558536+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'59","last_scrub_stamp":"2024-01-15T18:23:53.558463+0000","last_deep_scrub":"113'59","last_deep_scrub_stamp":"2024-01-15T18:23:53.558463+0000","last_clean_scrub_stamp":"2024-01-15T18:23:53.558463+0000","objects_scrubbed":23,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:58.192714+0000","scrub_duration":0.163912535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:22:42.096036+0000","last_change":"2024-01-15T18:18:40.934568+0000","last_active":"2024-01-15T18:22:42.096036+0000","last_peered":"2024-01-15T18:22:42.096036+0000","last_clean":"2024-01-15T18:22:42.096036+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:22:42.096036+0000","last_undegraded":"2024-01-15T18:22:42.096036+0000","last_fullsized":"2024-01-15T18:22:42.096036+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:39.894949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.894949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'31","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.998585+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:23:45.998585+0000","last_peered":"2024-01-15T18:23:45.998585+0000","last_clean":"2024-01-15T18:23:45.998585+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:23:45.998585+0000","last_undegraded":"2024-01-15T18:23:45.998585+0000","last_fullsized":"2024-01-15T18:23:45.998585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.772874+0000","last_change":"2024-01-15T18:23:48.772893+0000","last_active":"2024-01-15T18:23:48.772874+0000","last_peered":"2024-01-15T18:23:48.772874+0000","last_clean":"2024-01-15T18:23:48.772874+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:23:48.772874+0000","last_undegraded":"2024-01-15T18:23:48.772874+0000","last_fullsized":"2024-01-15T18:23:48.772874+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:48.772769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:48.772769+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.772769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:41.149208+0000","scrub_duration":0.0017691320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"113'119","reported_seq":257,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:49.271682+0000","last_change":"2024-01-15T18:23:49.271712+0000","last_active":"2024-01-15T18:23:49.271682+0000","last_peered":"2024-01-15T18:23:49.271682+0000","last_clean":"2024-01-15T18:23:49.271682+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:23:49.271682+0000","last_undegraded":"2024-01-15T18:23:49.271682+0000","last_fullsized":"2024-01-15T18:23:49.271682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'119","last_scrub_stamp":"2024-01-15T18:23:49.271604+0000","last_deep_scrub":"113'119","last_deep_scrub_stamp":"2024-01-15T18:23:49.271604+0000","last_clean_scrub_stamp":"2024-01-15T18:23:49.271604+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:57.760381+0000","scrub_duration":0.19866204600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710136,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3017,"ondisk_log_size":3017,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":618393,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279,"num_write_kb":724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1302528,"data_stored":1236786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575084,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834544,"kb_used_data":1767336,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537296,"statfs":{"total":96636764160,"available":94758191104,"internally_reserved":0,"allocated":1809752064,"data_stored":1808947458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69099999999999995,"15min":0.94399999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.29999999999999999},"max":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.87},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":13.744},"last":0.72399999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":1.0840000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":16.920000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":1.05},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":1.04,"15min":31.774999999999999},"last":0.629}]},{"osd":2,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72599999999999998,"15min":1},"min":{"1min":0.57399999999999995,"5min":0.48699999999999999,"15min":0.377},"max":{"1min":1.056,"5min":1.085,"15min":30.846},"last":0.622},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":1.0629999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":1.0229999999999999,"15min":39.695},"last":0.66400000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74099999999999999,"15min":1.1499999999999999},"min":{"1min":0.54500000000000004,"5min":0.437,"15min":0.38800000000000001},"max":{"1min":1.01,"5min":1.0109999999999999,"15min":26.463999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":1.198},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.41499999999999998},"max":{"1min":0.86699999999999999,"5min":1.0640000000000001,"15min":42.692},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86699999999999999,"5min":0.874,"15min":14.087},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.36299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":2.952},"last":0.42199999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":11.334},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.746},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91700000000000004,"5min":0.94499999999999995,"15min":8.423},"last":0.432}]},{"osd":6,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.78200000000000003},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.45500000000000002},"max":{"1min":0.95099999999999996,"5min":1.034,"15min":8.5030000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.35299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":3.3450000000000002},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575083,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1256364,"kb_used_data":1193316,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93115476,"statfs":{"total":96636764160,"available":95350247424,"internally_reserved":0,"allocated":1221955584,"data_stored":1221154106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67700000000000005,"15min":0.93100000000000005},"min":{"1min":0.501,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89900000000000002,"5min":0.94599999999999995,"15min":22.754000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68600000000000005,"15min":0.82399999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.92900000000000005,"5min":1.018,"15min":9.1470000000000002},"last":0.749}]},{"osd":1,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.86399999999999999},"min":{"1min":0.498,"5min":0.35299999999999998,"15min":0.34300000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":14.246},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70199999999999996,"15min":0.98199999999999998},"min":{"1min":0.47499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85599999999999998,"5min":0.97899999999999998,"15min":19.475000000000001},"last":0.92900000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":1.163},"min":{"1min":0.45600000000000002,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":0.91100000000000003,"5min":1.0549999999999999,"15min":42.825000000000003},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.747,"15min":1.107},"min":{"1min":0.442,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.96699999999999997,"5min":1.103,"15min":42.851999999999997},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.98599999999999999},"min":{"1min":0.54400000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.8160000000000001,"15min":24.199000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.745,"15min":1.0700000000000001},"min":{"1min":0.58799999999999997,"5min":0.53300000000000003,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":1.03,"15min":18.791},"last":0.88600000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58899999999999997,"15min":0.72499999999999998},"min":{"1min":0.47899999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":1.044,"15min":19.114999999999998},"last":0.495},{"interface":"front","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":8.3059999999999992},"last":0.51900000000000002}]},{"osd":5,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65200000000000002,"15min":0.69399999999999995},"min":{"1min":0.52600000000000002,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.88800000000000001,"5min":1.0920000000000001,"15min":7.5880000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73099999999999998,"15min":0.79500000000000004},"min":{"1min":0.58299999999999996,"5min":0.51800000000000002,"15min":0.42099999999999999},"max":{"1min":0.93500000000000005,"5min":1.1120000000000001,"15min":12.945},"last":0.66700000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:23:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69099999999999995,"15min":0.80100000000000005},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.38600000000000001},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":11.907},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.65900000000000003,"15min":0.85399999999999998},"min":{"1min":0.496,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.86599999999999999,"5min":1.1910000000000001,"15min":27.651},"last":0.97799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575084,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":937072,"kb_used_data":861544,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93434768,"statfs":{"total":96636764160,"available":95677202432,"internally_reserved":0,"allocated":882221056,"data_stored":881416086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68400000000000005,"15min":0.85199999999999998},"min":{"1min":0.43099999999999999,"5min":0.39200000000000002,"15min":0.29899999999999999},"max":{"1min":0.98599999999999999,"5min":1.9350000000000001,"15min":11.519},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74099999999999999,"15min":0.94599999999999995},"min":{"1min":0.626,"5min":0.371,"15min":0.371},"max":{"1min":0.96599999999999997,"5min":2.6819999999999999,"15min":12.952},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.749,"15min":0.91700000000000004},"min":{"1min":0.51100000000000001,"5min":0.43099999999999999,"15min":0.37},"max":{"1min":0.92100000000000004,"5min":2.1179999999999999,"15min":13.358000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.96699999999999997},"min":{"1min":0.56999999999999995,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":1.0289999999999999,"5min":2.8050000000000002,"15min":21.856999999999999},"last":0.48899999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.97299999999999998},"min":{"1min":0.70099999999999996,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.94999999999999996,"5min":3.508,"15min":18.314},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":1.0449999999999999},"min":{"1min":0.54700000000000004,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":1.069,"5min":3.5310000000000001,"15min":21.120000000000001},"last":0.69399999999999995}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.95999999999999996},"min":{"1min":0.54300000000000004,"5min":0.46600000000000003,"15min":0.441},"max":{"1min":0.92400000000000004,"5min":6.7990000000000004,"15min":13.317},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":1.0369999999999999},"min":{"1min":0.56799999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":2.3380000000000001,"15min":19.638000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.621,"15min":0.69799999999999995},"min":{"1min":0.436,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.80000000000000004,"5min":0.96399999999999997,"15min":11.093},"last":0.37},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.66300000000000003},"min":{"1min":0.437,"5min":0.35299999999999998,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.95699999999999996,"15min":3.3889999999999998},"last":0.441}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.78900000000000003},"min":{"1min":0.5,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.89700000000000002,"5min":1.008,"15min":5.8680000000000003},"last":0.748},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.629,"15min":0.70499999999999996},"min":{"1min":0.40300000000000002,"5min":0.33800000000000002,"15min":0.32600000000000001},"max":{"1min":0.91300000000000003,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.72299999999999998},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.86699999999999999,"5min":1.081,"15min":3.2759999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.754,"15min":0.80300000000000005},"min":{"1min":0.52100000000000002,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.88300000000000001,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.62}]}]},{"osd":4,"up_from":13,"seq":55834575083,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1706920,"kb_used_data":1643872,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92664920,"statfs":{"total":96636764160,"available":94888878080,"internally_reserved":0,"allocated":1683324928,"data_stored":1682527615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.94899999999999995},"min":{"1min":0.372,"5min":0.36599999999999999,"15min":0.33300000000000002},"max":{"1min":0.85899999999999999,"5min":1.02,"15min":34.802},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69899999999999995,"15min":0.91300000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":0.871,"5min":1.1319999999999999,"15min":22.335999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75,"15min":0.96199999999999997},"min":{"1min":0.44900000000000001,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":0.90600000000000003,"5min":1.224,"15min":38.198},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.752,"15min":0.96799999999999997},"min":{"1min":0.52400000000000002,"5min":0.438,"15min":0.372},"max":{"1min":0.875,"5min":1.0269999999999999,"15min":38.220999999999997},"last":0.77200000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78500000000000003,"15min":0.89600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.34599999999999997},"max":{"1min":0.91600000000000004,"5min":1.0509999999999999,"15min":7.6989999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76000000000000001,"15min":0.85999999999999999},"min":{"1min":0.62,"5min":0.41299999999999998,"15min":0.38700000000000001},"max":{"1min":0.94999999999999996,"5min":1.0740000000000001,"15min":7.4829999999999997},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73299999999999998,"15min":0.95199999999999996},"min":{"1min":0.55400000000000005,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":0.90000000000000002,"5min":1.214,"15min":14.529},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.94699999999999995},"min":{"1min":0.53100000000000003,"5min":0.498,"15min":0.41999999999999998},"max":{"1min":0.94499999999999995,"5min":0.95399999999999996,"15min":25.774000000000001},"last":0.72999999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65300000000000002,"15min":0.68899999999999995},"min":{"1min":0.42799999999999999,"5min":0.40899999999999997,"15min":0.33200000000000002},"max":{"1min":0.89800000000000002,"5min":0.90300000000000002,"15min":9.9619999999999997},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.79700000000000004},"min":{"1min":0.41999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86499999999999999,"5min":1.1719999999999999,"15min":25.887},"last":0.84499999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63200000000000001,"15min":0.70399999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.93899999999999995,"15min":10.284000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67300000000000004,"15min":0.68300000000000005},"min":{"1min":0.46700000000000003,"5min":0.435,"15min":0.33900000000000002},"max":{"1min":0.86799999999999999,"5min":0.97299999999999998,"15min":3.6949999999999998},"last":0.86399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65500000000000003,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.38400000000000001,"15min":0.30399999999999999},"max":{"1min":0.93700000000000006,"5min":1.194,"15min":13.760999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71499999999999997,"15min":0.75800000000000001},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.36199999999999999},"max":{"1min":0.96699999999999997,"5min":1.0269999999999999,"15min":11.67},"last":0.88100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575083,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1453652,"kb_used_data":1386444,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92918188,"statfs":{"total":96636764160,"available":95148224512,"internally_reserved":0,"allocated":1419718656,"data_stored":1418903068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.47399999999999998,"15min":0.93799999999999994},"min":{"1min":0.29999999999999999,"5min":0.251,"15min":0.251},"max":{"1min":0.66200000000000003,"5min":2.415,"15min":26.364000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.499,"15min":0.95499999999999996},"min":{"1min":0.33300000000000002,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":0.73399999999999999,"5min":0.79500000000000004,"15min":28.332000000000001},"last":0.57899999999999996}]},{"osd":1,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48199999999999998,"15min":0.98399999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.63300000000000001,"5min":0.82499999999999996,"15min":40.457999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":1.1120000000000001},"min":{"1min":0.34399999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70399999999999996,"5min":0.84599999999999997,"15min":27.427},"last":0.45000000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.98799999999999999},"min":{"1min":0.40400000000000003,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":0.76100000000000001,"5min":0.90700000000000003,"15min":23.036999999999999},"last":0.505},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":1.081},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":0.76900000000000002,"5min":0.873,"15min":26.404},"last":0.52200000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.76900000000000002,"15min":1.0349999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.0529999999999999,"5min":1.1439999999999999,"15min":22.998000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76300000000000001,"15min":1.0740000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1,"5min":1.1439999999999999,"15min":20.399000000000001},"last":0.80300000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76700000000000002,"15min":1.0820000000000001},"min":{"1min":0.52800000000000002,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":0.97999999999999998,"5min":1.46,"15min":26.314},"last":1.01},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":1.1040000000000001},"min":{"1min":0.55400000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1,"5min":1.109,"15min":22.908999999999999},"last":0.90900000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.79000000000000004,"15min":1.155},"min":{"1min":0.57399999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.105,"5min":1.105,"15min":20.445},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81399999999999995,"15min":1.1659999999999999},"min":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":1.0800000000000001,"5min":1.4299999999999999,"15min":26.091999999999999},"last":1.0589999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78600000000000003,"15min":1.159},"min":{"1min":0.54700000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.99299999999999999,"5min":1.155,"15min":23.111999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80200000000000005,"15min":1.3169999999999999},"min":{"1min":0.56200000000000006,"5min":0.53600000000000003,"15min":0.46400000000000002},"max":{"1min":1.0880000000000001,"5min":1.145,"15min":44.817999999999998},"last":0.83699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575084,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":965544,"kb_used_data":902496,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93406296,"statfs":{"total":96636764160,"available":95648047104,"internally_reserved":0,"allocated":924155904,"data_stored":923359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52400000000000002,"15min":1.054},"min":{"1min":0.28599999999999998,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.82899999999999996,"5min":4.2329999999999997,"15min":29.527999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":1.335},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73499999999999999,"5min":6.0670000000000002,"15min":34.122999999999998},"last":0.436}]},{"osd":1,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56200000000000006,"15min":1.534},"min":{"1min":0.33200000000000002,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":0.71199999999999997,"5min":4.5259999999999998,"15min":113.149},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.59099999999999997,"15min":1.127},"min":{"1min":0.40500000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.84899999999999998,"5min":6.1020000000000003,"15min":41.302},"last":0.36699999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54900000000000004,"15min":1.3899999999999999},"min":{"1min":0.34699999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76800000000000002,"5min":4.7370000000000001,"15min":46.960000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.625,"15min":1.2809999999999999},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83299999999999996,"5min":3.8809999999999998,"15min":115.693},"last":0.38900000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.83199999999999996,"15min":1.194},"min":{"1min":0.55900000000000005,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":1.1100000000000001,"5min":3.8399999999999999,"15min":24.366},"last":0.626},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":1.228},"min":{"1min":0.57999999999999996,"5min":0.45600000000000002,"15min":0.432},"max":{"1min":1.0189999999999999,"5min":2.343,"15min":20.489000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.82599999999999996,"15min":1.5289999999999999},"min":{"1min":0.61899999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":1.175,"5min":3.927,"15min":41.683999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80400000000000005,"15min":1.2889999999999999},"min":{"1min":0.60799999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98599999999999999,"5min":2.3029999999999999,"15min":26.084},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":1.319},"min":{"1min":0.57399999999999995,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.159,"5min":6.0220000000000002,"15min":22.199000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.82099999999999995,"15min":1.569},"min":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.0860000000000001,"5min":3.9590000000000001,"15min":41.738999999999997},"last":0.72199999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:23:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.85599999999999998,"15min":1.262},"min":{"1min":0.621,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.1339999999999999,"5min":2.556,"15min":20.446999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":1.6060000000000001},"min":{"1min":0.60799999999999998,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1599999999999999,"5min":2.5939999999999999,"15min":41.402000000000001},"last":0.60699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575084,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1133560,"kb_used_data":1066352,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93238280,"statfs":{"total":96636764160,"available":95475998720,"internally_reserved":0,"allocated":1091944448,"data_stored":1091132307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.439,"15min":0.86099999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":0.63700000000000001,"5min":1.4179999999999999,"15min":39.948},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.56599999999999995,"15min":1.0649999999999999},"min":{"1min":0.30399999999999999,"5min":0.28100000000000003,"15min":0.27600000000000002},"max":{"1min":0.66700000000000004,"5min":4.9329999999999998,"15min":35.790999999999997},"last":0.29499999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.57699999999999996,"15min":0.97099999999999997},"min":{"1min":0.39400000000000002,"5min":0.32300000000000001,"15min":0.27600000000000002},"max":{"1min":0.66100000000000003,"5min":9.2609999999999992,"15min":49.229999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.81100000000000005},"min":{"1min":0.34499999999999997,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.72999999999999998,"5min":1.7649999999999999,"15min":10.349},"last":0.44}]},{"osd":3,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":1.0589999999999999},"min":{"1min":0.38800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.68300000000000005,"5min":0.84999999999999998,"15min":27.643999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57199999999999995,"15min":0.90800000000000003},"min":{"1min":0.38600000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":1.198,"15min":15.608000000000001},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.76500000000000001,"15min":1.165},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45100000000000001},"max":{"1min":1.1080000000000001,"5min":1.383,"15min":27.489999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.85599999999999998,"15min":1.0349999999999999},"min":{"1min":0.495,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.218,"5min":6.6950000000000003,"15min":10.038},"last":0.623}]},{"osd":5,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.76900000000000002,"15min":1.1779999999999999},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.1739999999999999,"5min":1.2010000000000001,"15min":27.547000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.84599999999999997,"15min":1.0469999999999999},"min":{"1min":0.61699999999999999,"5min":0.55800000000000005,"15min":0.55400000000000005},"max":{"1min":1.1599999999999999,"5min":1.266,"15min":9.9610000000000003},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80600000000000005,"15min":1.171},"min":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.496},"max":{"1min":1.147,"5min":1.984,"15min":27.579999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82899999999999996,"15min":1},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.42399999999999999},"max":{"1min":1.1910000000000001,"5min":5.6470000000000002,"15min":11.712999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:23:26 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.85399999999999998,"15min":1.0980000000000001},"min":{"1min":0.56899999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1299999999999999,"5min":5.819,"15min":11.833},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.89000000000000001,"15min":1.0920000000000001},"min":{"1min":0.59899999999999998,"5min":0.58499999999999996,"15min":0.502},"max":{"1min":1.254,"5min":6.3570000000000002,"15min":14.606},"last":0.57299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575083,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1740896,"kb_used_data":1677848,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92630944,"statfs":{"total":96636764160,"available":94854086656,"internally_reserved":0,"allocated":1718116352,"data_stored":1717289103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46300000000000002,"15min":1.054},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":31.977},"last":0.371},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.51300000000000001,"15min":1.0349999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.61499999999999999,"5min":1.6579999999999999,"15min":36.145000000000003},"last":0.504}]},{"osd":2,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":1.1439999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":1.1759999999999999,"15min":41.387999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.50600000000000001,"15min":1.0089999999999999},"min":{"1min":0.33300000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.70399999999999996,"5min":1.7689999999999999,"15min":33.219999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":1.006},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.68200000000000005,"5min":0.80500000000000005,"15min":31.943999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":1.107},"min":{"1min":0.44700000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.746,"5min":0.95599999999999996,"15min":45.128},"last":0.56399999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76200000000000001,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.38300000000000001},"max":{"1min":1.0469999999999999,"5min":1.4550000000000001,"15min":32.932000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.77400000000000002,"15min":1.1659999999999999},"min":{"1min":0.53600000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":0.92700000000000005,"5min":1.115,"15min":31.782},"last":0.69799999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":1.149},"min":{"1min":0.57699999999999996,"5min":0.503,"15min":0.38600000000000001},"max":{"1min":1.0249999999999999,"5min":1.179,"15min":31.858000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.79300000000000004,"15min":1.0409999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":1.7869999999999999,"15min":33.151000000000003},"last":0.77900000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79300000000000004,"15min":1.0660000000000001},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":13.055},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79300000000000004,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.47699999999999998},"max":{"1min":1.262,"5min":1.7170000000000001,"15min":33.098999999999997},"last":0.746}]},{"osd":7,"last update":"Mon Jan 15 18:23:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":1.097},"min":{"1min":0.68200000000000005,"5min":0.55500000000000005,"15min":0.48899999999999999},"max":{"1min":1.0029999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":1.2050000000000001},"min":{"1min":0.61799999999999999,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":0.997,"5min":1.075,"15min":31.905999999999999},"last":0.76300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":634880,"data_stored":616386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":616344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1715482624,"data_stored":1715470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":922746880,"data_stored":922746880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":880807936,"data_stored":880804032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:24:09.567 INFO:tasks.ceph:pgid 17.3f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.567 INFO:tasks.ceph:pgid 17.3e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.567 INFO:tasks.ceph:pgid 17.3d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.568 INFO:tasks.ceph:pgid 17.3a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.568 INFO:tasks.ceph:pgid 17.39 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.568 INFO:tasks.ceph:pgid 17.37 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.568 INFO:tasks.ceph:pgid 17.36 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.569 INFO:tasks.ceph:pgid 17.34 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.569 INFO:tasks.ceph:pgid 17.33 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.569 INFO:tasks.ceph:pgid 17.32 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.569 INFO:tasks.ceph:pgid 17.31 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.569 INFO:tasks.ceph:pgid 17.30 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.570 INFO:tasks.ceph:pgid 17.2f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.570 INFO:tasks.ceph:pgid 17.2e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.570 INFO:tasks.ceph:pgid 17.2d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.570 INFO:tasks.ceph:pgid 17.2a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.570 INFO:tasks.ceph:pgid 17.29 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.570 INFO:tasks.ceph:pgid 17.28 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.571 INFO:tasks.ceph:pgid 17.27 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.571 INFO:tasks.ceph:pgid 17.26 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.571 INFO:tasks.ceph:pgid 17.25 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.571 INFO:tasks.ceph:pgid 17.24 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.571 INFO:tasks.ceph:pgid 17.23 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.572 INFO:tasks.ceph:pgid 17.21 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.572 INFO:tasks.ceph:pgid 17.1e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.572 INFO:tasks.ceph:pgid 17.1f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.572 INFO:tasks.ceph:pgid 17.1c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.572 INFO:tasks.ceph:pgid 17.1d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.573 INFO:tasks.ceph:pgid 17.1b last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.573 INFO:tasks.ceph:pgid 17.18 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.573 INFO:tasks.ceph:pgid 17.19 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.573 INFO:tasks.ceph:pgid 17.16 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.574 INFO:tasks.ceph:pgid 17.17 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.574 INFO:tasks.ceph:pgid 17.14 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.574 INFO:tasks.ceph:pgid 16.15 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.574 INFO:tasks.ceph:pgid 16.14 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.574 INFO:tasks.ceph:pgid 17.12 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.574 INFO:tasks.ceph:pgid 17.13 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.575 INFO:tasks.ceph:pgid 17.11 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.575 INFO:tasks.ceph:pgid 17.e last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.575 INFO:tasks.ceph:pgid 16.f last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.575 INFO:tasks.ceph:pgid 17.f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.575 INFO:tasks.ceph:pgid 17.c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.576 INFO:tasks.ceph:pgid 17.d last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.576 INFO:tasks.ceph:pgid 17.a last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.576 INFO:tasks.ceph:pgid 17.b last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.576 INFO:tasks.ceph:pgid 17.8 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.576 INFO:tasks.ceph:pgid 17.9 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.577 INFO:tasks.ceph:pgid 17.6 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.577 INFO:tasks.ceph:pgid 17.7 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.577 INFO:tasks.ceph:pgid 16.6 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.577 INFO:tasks.ceph:pgid 17.4 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.577 INFO:tasks.ceph:pgid 16.5 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.578 INFO:tasks.ceph:pgid 17.5 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.578 INFO:tasks.ceph:pgid 17.2 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.578 INFO:tasks.ceph:pgid 17.0 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.578 INFO:tasks.ceph:pgid 16.0 last_scrub_stamp 2024-01-15T18:18:39.894949+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=39, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.579 INFO:tasks.ceph:pgid 17.10 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:09.579 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-15T18:24:29.580 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:24:29.745 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f0a01000640 1 Processor -- start 2024-01-15T18:24:29.746 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f0a01000640 1 -- start start 2024-01-15T18:24:29.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f0a01000640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f09fc0b4410 0x7f09fc0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:29.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f0a01000640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc06c2c0 con 0x7f09fc0b3f60 2024-01-15T18:24:29.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f0a01000640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09fc0b3780 con 0x7f09fc06bee0 2024-01-15T18:24:29.747 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f0a01000640 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09fc15aed0 con 0x7f09fc0b4410 2024-01-15T18:24:29.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09fb200640 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f09fc0b4410 0x7f09fc0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:29.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09f9e00640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f09fc06bee0 0x7f09fc15a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:43900/0 (socket says 172.21.15.143:43900) 2024-01-15T18:24:29.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09f9400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f09fc0b3f60 0x7f09fc15b5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36304/0 (socket says 172.21.15.143:36304) 2024-01-15T18:24:29.748 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09f9e00640 1 -- 172.21.15.143:0/2153362362 learned_addr learned my addr 172.21.15.143:0/2153362362 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:24:29.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09fb200640 1 -- 172.21.15.143:0/2153362362 >> v1:172.21.15.204:6789/0 conn(0x7f09fc06bee0 legacy=0x7f09fc15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09fb200640 1 -- 172.21.15.143:0/2153362362 >> v1:172.21.15.143:6789/0 conn(0x7f09fc0b3f60 legacy=0x7f09fc15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09f8a00640 1 -- 172.21.15.143:0/2153362362 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3426342565 0 0) 0x7f09fc0b3780 con 0x7f09fc06bee0 2024-01-15T18:24:29.749 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.753+0000 7f09fb200640 1 -- 172.21.15.143:0/2153362362 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc15b1b0 con 0x7f09fc0b4410 2024-01-15T18:24:29.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f09fb200640 1 --2- 172.21.15.143:0/2153362362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc0b4410 0x7f09fc0b2290 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009d90 tx=0x7f09f401c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c26e9c38ce108ba server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:29.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f09f8a00640 1 -- 172.21.15.143:0/2153362362 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40a5840 con 0x7f09fc0b4410 2024-01-15T18:24:29.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f09f8a00640 1 -- 172.21.15.143:0/2153362362 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f40a59a0 con 0x7f09fc0b4410 2024-01-15T18:24:29.751 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f09f8a00640 1 -- 172.21.15.143:0/2153362362 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f40a5d30 con 0x7f09fc0b4410 2024-01-15T18:24:29.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f0a01000640 1 -- 172.21.15.143:0/2153362362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc0b4410 msgr2=0x7f09fc0b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f0a01000640 1 --2- 172.21.15.143:0/2153362362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc0b4410 0x7f09fc0b2290 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009d90 tx=0x7f09f401c7e0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f0a01000640 1 -- 172.21.15.143:0/2153362362 shutdown_connections 2024-01-15T18:24:29.752 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f0a01000640 1 --2- 172.21.15.143:0/2153362362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc0b4410 0x7f09fc0b2290 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.757+0000 7f0a01000640 1 -- 172.21.15.143:0/2153362362 >> 172.21.15.143:0/2153362362 conn(0x7f09fc066720 msgr2=0x7f09fc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:29.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 -- 172.21.15.143:0/2153362362 shutdown_connections 2024-01-15T18:24:29.753 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 -- 172.21.15.143:0/2153362362 wait complete. 2024-01-15T18:24:29.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 Processor -- start 2024-01-15T18:24:29.754 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 -- start start 2024-01-15T18:24:29.755 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 0x7f09fc14f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09fc0b3f60 0x7f09fc151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09fc0b4410 0x7f09fc146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc150ac0 con 0x7f09fc06bee0 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc14ff40 con 0x7f09fc0b3f60 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f0a01000640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc150220 con 0x7f09fc0b4410 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 0x7f09fc14f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:29.756 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09fc0b3f60 0x7f09fc151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 0x7f09fc14f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58878/0 (socket says 172.21.15.143:58878) 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 -- 172.21.15.143:0/160743035 learned_addr learned my addr 172.21.15.143:0/160743035 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09fb200640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09fc0b4410 0x7f09fc146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 -- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09fc0b4410 msgr2=0x7f09fc146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09fc0b4410 0x7f09fc146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 -- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09fc0b3f60 msgr2=0x7f09fc151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09fc0b3f60 0x7f09fc151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 -- 172.21.15.143:0/160743035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f4009a70 con 0x7f09fc06bee0 2024-01-15T18:24:29.757 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.761+0000 7f09f9e00640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 0x7f09fc14f960 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0011f0 tx=0x7f09ec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:29.758 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0074e0 con 0x7f09fc06bee0 2024-01-15T18:24:29.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec007640 con 0x7f09fc06bee0 2024-01-15T18:24:29.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec004a10 con 0x7f09fc06bee0 2024-01-15T18:24:29.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc15aca0 con 0x7f09fc06bee0 2024-01-15T18:24:29.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc1507e0 con 0x7f09fc06bee0 2024-01-15T18:24:29.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc067680 con 0x7f09fc06bee0 2024-01-15T18:24:29.759 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 45) v1 ==== 87843+0+0 (secure 0 0 0) 0x7f09ec004b70 con 0x7f09fc06bee0 2024-01-15T18:24:29.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f2000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f09d4067b80 0x7f09d406a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:29.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f09ec082610 con 0x7f09fc06bee0 2024-01-15T18:24:29.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f9400640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f09d4067b80 0x7f09d406a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:29.760 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.765+0000 7f09f9400640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f09d4067b80 0x7f09d406a040 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007880 tx=0x7f09e4005730 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:29.763 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.769+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f09ec051010 con 0x7f09fc06bee0 2024-01-15T18:24:29.909 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.917+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09fc15ab20 con 0x7f09d4067b80 2024-01-15T18:24:29.917 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.925+0000 7f09f2000640 1 -- 172.21.15.143:0/160743035 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292590 (secure 0 0 0) 0x7f09e4007d40 con 0x7f09d4067b80 2024-01-15T18:24:29.920 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:24:29.922 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:24:29.923 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f09d4067b80 msgr2=0x7f09d406a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f09d4067b80 0x7f09d406a040 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007880 tx=0x7f09e4005730 comp rx=0 tx=0).stop 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 msgr2=0x7f09fc14f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 0x7f09fc14f960 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f09ec0011f0 tx=0x7f09ec00c9b0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 shutdown_connections 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f09d4067b80 0x7f09d406a040 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f09fc0b4410 0x7f09fc146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f09fc0b3f60 0x7f09fc151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.924 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 --2- 172.21.15.143:0/160743035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09fc06bee0 0x7f09fc14f960 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 >> 172.21.15.143:0/160743035 conn(0x7f09fc066720 msgr2=0x7f09fc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 shutdown_connections 2024-01-15T18:24:29.925 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:29.929+0000 7f0a01000640 1 -- 172.21.15.143:0/160743035 wait complete. 2024-01-15T18:24:29.943 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":697,"stamp":"2024-01-15T18:24:28.986872+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369918897,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1744,"num_write_kb":5246035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5506,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4193,"ondisk_log_size":4193,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11036872,"kb_used_data":10499208,"kb_used_omap":12,"kb_used_meta":537587,"kb_avail":743937848,"statfs":{"total":773094113280,"available":761792356352,"internally_reserved":0,"allocated":10751188992,"data_stored":10744729394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":550489648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"17.3f","version":"113'53","reported_seq":78,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.247295+0000","last_change":"2024-01-15T18:18:41.946270+0000","last_active":"2024-01-15T18:23:45.247295+0000","last_peered":"2024-01-15T18:23:45.247295+0000","last_clean":"2024-01-15T18:23:45.247295+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:23:45.247295+0000","last_undegraded":"2024-01-15T18:23:45.247295+0000","last_fullsized":"2024-01-15T18:23:45.247295+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'64","reported_seq":96,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:19.014535+0000","last_change":"2024-01-15T18:24:19.014568+0000","last_active":"2024-01-15T18:24:19.014535+0000","last_peered":"2024-01-15T18:24:19.014535+0000","last_clean":"2024-01-15T18:24:19.014535+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:24:19.014535+0000","last_undegraded":"2024-01-15T18:24:19.014535+0000","last_fullsized":"2024-01-15T18:24:19.014535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:19.014459+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:19.014459+0000","last_clean_scrub_stamp":"2024-01-15T18:24:19.014459+0000","objects_scrubbed":19,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:41.309827+0000","scrub_duration":0.123762099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'63","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:17.134552+0000","last_change":"2024-01-15T18:24:17.134593+0000","last_active":"2024-01-15T18:24:17.134552+0000","last_peered":"2024-01-15T18:24:17.134552+0000","last_clean":"2024-01-15T18:24:17.134552+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:24:17.134552+0000","last_undegraded":"2024-01-15T18:24:17.134552+0000","last_fullsized":"2024-01-15T18:24:17.134552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:24:17.134470+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:24:17.134470+0000","last_clean_scrub_stamp":"2024-01-15T18:24:17.134470+0000","objects_scrubbed":19,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:27.454408+0000","scrub_duration":0.131278227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'62","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:54.586712+0000","last_change":"2024-01-15T18:23:54.586767+0000","last_active":"2024-01-15T18:23:54.586712+0000","last_peered":"2024-01-15T18:23:54.586712+0000","last_clean":"2024-01-15T18:23:54.586712+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:54.586712+0000","last_undegraded":"2024-01-15T18:23:54.586712+0000","last_fullsized":"2024-01-15T18:23:54.586712+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:23:54.586620+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:23:54.586620+0000","last_clean_scrub_stamp":"2024-01-15T18:23:54.586620+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:04.906558+0000","scrub_duration":0.14577402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'35","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:03.545628+0000","last_change":"2024-01-15T18:24:03.545686+0000","last_active":"2024-01-15T18:24:03.545628+0000","last_peered":"2024-01-15T18:24:03.545628+0000","last_clean":"2024-01-15T18:24:03.545628+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:24:03.545628+0000","last_undegraded":"2024-01-15T18:24:03.545628+0000","last_fullsized":"2024-01-15T18:24:03.545628+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'35","last_scrub_stamp":"2024-01-15T18:24:03.545528+0000","last_deep_scrub":"113'35","last_deep_scrub_stamp":"2024-01-15T18:24:03.545528+0000","last_clean_scrub_stamp":"2024-01-15T18:24:03.545528+0000","objects_scrubbed":10,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:24:34.025807+0000","scrub_duration":0.072323520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'66","reported_seq":101,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.570290+0000","last_change":"2024-01-15T18:24:10.570328+0000","last_active":"2024-01-15T18:24:10.570290+0000","last_peered":"2024-01-15T18:24:10.570290+0000","last_clean":"2024-01-15T18:24:10.570290+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:24:10.570290+0000","last_undegraded":"2024-01-15T18:24:10.570290+0000","last_fullsized":"2024-01-15T18:24:10.570290+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'66","last_scrub_stamp":"2024-01-15T18:24:10.570213+0000","last_deep_scrub":"113'66","last_deep_scrub_stamp":"2024-01-15T18:24:10.570213+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.570213+0000","objects_scrubbed":21,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:04:00.820904+0000","scrub_duration":0.14447458199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'58","reported_seq":86,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.988814+0000","last_change":"2024-01-15T18:24:08.988849+0000","last_active":"2024-01-15T18:24:08.988814+0000","last_peered":"2024-01-15T18:24:08.988814+0000","last_clean":"2024-01-15T18:24:08.988814+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:24:08.988814+0000","last_undegraded":"2024-01-15T18:24:08.988814+0000","last_fullsized":"2024-01-15T18:24:08.988814+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'58","last_scrub_stamp":"2024-01-15T18:24:08.988737+0000","last_deep_scrub":"113'58","last_deep_scrub_stamp":"2024-01-15T18:24:08.988737+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.988737+0000","objects_scrubbed":17,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:16.645900+0000","scrub_duration":0.120076418,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'61","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:06.128647+0000","last_change":"2024-01-15T18:24:06.128704+0000","last_active":"2024-01-15T18:24:06.128647+0000","last_peered":"2024-01-15T18:24:06.128647+0000","last_clean":"2024-01-15T18:24:06.128647+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:24:06.128647+0000","last_undegraded":"2024-01-15T18:24:06.128647+0000","last_fullsized":"2024-01-15T18:24:06.128647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'61","last_scrub_stamp":"2024-01-15T18:24:06.128559+0000","last_deep_scrub":"113'61","last_deep_scrub_stamp":"2024-01-15T18:24:06.128559+0000","last_clean_scrub_stamp":"2024-01-15T18:24:06.128559+0000","objects_scrubbed":19,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:03.610416+0000","scrub_duration":0.116078944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'47","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.838934+0000","last_change":"2024-01-15T18:18:41.946926+0000","last_active":"2024-01-15T18:23:45.838934+0000","last_peered":"2024-01-15T18:23:45.838934+0000","last_clean":"2024-01-15T18:23:45.838934+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:23:45.838934+0000","last_undegraded":"2024-01-15T18:23:45.838934+0000","last_fullsized":"2024-01-15T18:23:45.838934+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'64","reported_seq":98,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:21.670959+0000","last_change":"2024-01-15T18:24:21.671012+0000","last_active":"2024-01-15T18:24:21.670959+0000","last_peered":"2024-01-15T18:24:21.670959+0000","last_clean":"2024-01-15T18:24:21.670959+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:24:21.670959+0000","last_undegraded":"2024-01-15T18:24:21.670959+0000","last_fullsized":"2024-01-15T18:24:21.670959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:21.670862+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:21.670862+0000","last_clean_scrub_stamp":"2024-01-15T18:24:21.670862+0000","objects_scrubbed":17,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:27.636410+0000","scrub_duration":0.117922286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'69","reported_seq":106,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.920167+0000","last_change":"2024-01-15T18:24:01.920191+0000","last_active":"2024-01-15T18:24:01.920167+0000","last_peered":"2024-01-15T18:24:01.920167+0000","last_clean":"2024-01-15T18:24:01.920167+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:24:01.920167+0000","last_undegraded":"2024-01-15T18:24:01.920167+0000","last_fullsized":"2024-01-15T18:24:01.920167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'69","last_scrub_stamp":"2024-01-15T18:24:01.920080+0000","last_deep_scrub":"113'69","last_deep_scrub_stamp":"2024-01-15T18:24:01.920080+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.920080+0000","objects_scrubbed":28,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:15.623343+0000","scrub_duration":0.20686890199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'76","reported_seq":117,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:23.822318+0000","last_change":"2024-01-15T18:24:23.822342+0000","last_active":"2024-01-15T18:24:23.822318+0000","last_peered":"2024-01-15T18:24:23.822318+0000","last_clean":"2024-01-15T18:24:23.822318+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:24:23.822318+0000","last_undegraded":"2024-01-15T18:24:23.822318+0000","last_fullsized":"2024-01-15T18:24:23.822318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'76","last_scrub_stamp":"2024-01-15T18:24:23.822246+0000","last_deep_scrub":"113'76","last_deep_scrub_stamp":"2024-01-15T18:24:23.822246+0000","last_clean_scrub_stamp":"2024-01-15T18:24:23.822246+0000","objects_scrubbed":28,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:17:07.022123+0000","scrub_duration":0.220839438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'57","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:18.161646+0000","last_change":"2024-01-15T18:24:18.161677+0000","last_active":"2024-01-15T18:24:18.161646+0000","last_peered":"2024-01-15T18:24:18.161646+0000","last_clean":"2024-01-15T18:24:18.161646+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:24:18.161646+0000","last_undegraded":"2024-01-15T18:24:18.161646+0000","last_fullsized":"2024-01-15T18:24:18.161646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'57","last_scrub_stamp":"2024-01-15T18:24:18.161575+0000","last_deep_scrub":"113'57","last_deep_scrub_stamp":"2024-01-15T18:24:18.161575+0000","last_clean_scrub_stamp":"2024-01-15T18:24:18.161575+0000","objects_scrubbed":19,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:44:42.184776+0000","scrub_duration":0.114720246,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'70","reported_seq":102,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.156856+0000","last_change":"2024-01-15T18:24:10.156878+0000","last_active":"2024-01-15T18:24:10.156856+0000","last_peered":"2024-01-15T18:24:10.156856+0000","last_clean":"2024-01-15T18:24:10.156856+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:24:10.156856+0000","last_undegraded":"2024-01-15T18:24:10.156856+0000","last_fullsized":"2024-01-15T18:24:10.156856+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'70","last_scrub_stamp":"2024-01-15T18:24:10.156783+0000","last_deep_scrub":"113'70","last_deep_scrub_stamp":"2024-01-15T18:24:10.156783+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.156783+0000","objects_scrubbed":27,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:39:42.365100+0000","scrub_duration":0.18950409400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'43","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:15.514128+0000","last_change":"2024-01-15T18:24:15.514162+0000","last_active":"2024-01-15T18:24:15.514128+0000","last_peered":"2024-01-15T18:24:15.514128+0000","last_clean":"2024-01-15T18:24:15.514128+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:24:15.514128+0000","last_undegraded":"2024-01-15T18:24:15.514128+0000","last_fullsized":"2024-01-15T18:24:15.514128+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'43","last_scrub_stamp":"2024-01-15T18:24:15.514051+0000","last_deep_scrub":"113'43","last_deep_scrub_stamp":"2024-01-15T18:24:15.514051+0000","last_clean_scrub_stamp":"2024-01-15T18:24:15.514051+0000","objects_scrubbed":12,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:37.561789+0000","scrub_duration":0.093520221000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'60","reported_seq":87,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:16.630070+0000","last_change":"2024-01-15T18:24:16.630104+0000","last_active":"2024-01-15T18:24:16.630070+0000","last_peered":"2024-01-15T18:24:16.630070+0000","last_clean":"2024-01-15T18:24:16.630070+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:24:16.630070+0000","last_undegraded":"2024-01-15T18:24:16.630070+0000","last_fullsized":"2024-01-15T18:24:16.630070+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:16.629993+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:16.629993+0000","last_clean_scrub_stamp":"2024-01-15T18:24:16.629993+0000","objects_scrubbed":18,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:28:37.606419+0000","scrub_duration":0.115176582,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'66","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:20.008135+0000","last_change":"2024-01-15T18:24:20.008183+0000","last_active":"2024-01-15T18:24:20.008135+0000","last_peered":"2024-01-15T18:24:20.008135+0000","last_clean":"2024-01-15T18:24:20.008135+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:24:20.008135+0000","last_undegraded":"2024-01-15T18:24:20.008135+0000","last_fullsized":"2024-01-15T18:24:20.008135+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'66","last_scrub_stamp":"2024-01-15T18:24:20.008049+0000","last_deep_scrub":"113'66","last_deep_scrub_stamp":"2024-01-15T18:24:20.008049+0000","last_clean_scrub_stamp":"2024-01-15T18:24:20.008049+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:23:24.642299+0000","scrub_duration":0.158380297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'67","reported_seq":105,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:11.726908+0000","last_change":"2024-01-15T18:24:11.726947+0000","last_active":"2024-01-15T18:24:11.726908+0000","last_peered":"2024-01-15T18:24:11.726908+0000","last_clean":"2024-01-15T18:24:11.726908+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:24:11.726908+0000","last_undegraded":"2024-01-15T18:24:11.726908+0000","last_fullsized":"2024-01-15T18:24:11.726908+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'67","last_scrub_stamp":"2024-01-15T18:24:11.726822+0000","last_deep_scrub":"113'67","last_deep_scrub_stamp":"2024-01-15T18:24:11.726822+0000","last_clean_scrub_stamp":"2024-01-15T18:24:11.726822+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:19.383985+0000","scrub_duration":0.14728950399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'65","reported_seq":94,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:19.209516+0000","last_change":"2024-01-15T18:24:19.209575+0000","last_active":"2024-01-15T18:24:19.209516+0000","last_peered":"2024-01-15T18:24:19.209516+0000","last_clean":"2024-01-15T18:24:19.209516+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:24:19.209516+0000","last_undegraded":"2024-01-15T18:24:19.209516+0000","last_fullsized":"2024-01-15T18:24:19.209516+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'65","last_scrub_stamp":"2024-01-15T18:24:19.209430+0000","last_deep_scrub":"113'65","last_deep_scrub_stamp":"2024-01-15T18:24:19.209430+0000","last_clean_scrub_stamp":"2024-01-15T18:24:19.209430+0000","objects_scrubbed":21,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:56.680008+0000","scrub_duration":0.13384733800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'63","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.808942+0000","last_change":"2024-01-15T18:23:58.808978+0000","last_active":"2024-01-15T18:23:58.808942+0000","last_peered":"2024-01-15T18:23:58.808942+0000","last_clean":"2024-01-15T18:23:58.808942+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:58.808942+0000","last_undegraded":"2024-01-15T18:23:58.808942+0000","last_fullsized":"2024-01-15T18:23:58.808942+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:23:58.808865+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:23:58.808865+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.808865+0000","objects_scrubbed":17,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:01.715099+0000","scrub_duration":0.125894276,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'76","reported_seq":114,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:02.904727+0000","last_change":"2024-01-15T18:24:02.904748+0000","last_active":"2024-01-15T18:24:02.904727+0000","last_peered":"2024-01-15T18:24:02.904727+0000","last_clean":"2024-01-15T18:24:02.904727+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:24:02.904727+0000","last_undegraded":"2024-01-15T18:24:02.904727+0000","last_fullsized":"2024-01-15T18:24:02.904727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'76","last_scrub_stamp":"2024-01-15T18:24:02.904627+0000","last_deep_scrub":"113'76","last_deep_scrub_stamp":"2024-01-15T18:24:02.904627+0000","last_clean_scrub_stamp":"2024-01-15T18:24:02.904627+0000","objects_scrubbed":26,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:21.142713+0000","scrub_duration":0.16157696399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'68","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:23.807206+0000","last_change":"2024-01-15T18:24:23.807266+0000","last_active":"2024-01-15T18:24:23.807206+0000","last_peered":"2024-01-15T18:24:23.807206+0000","last_clean":"2024-01-15T18:24:23.807206+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:24:23.807206+0000","last_undegraded":"2024-01-15T18:24:23.807206+0000","last_fullsized":"2024-01-15T18:24:23.807206+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:23.807112+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:23.807112+0000","last_clean_scrub_stamp":"2024-01-15T18:24:23.807112+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:38.477642+0000","scrub_duration":0.168180996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'50","reported_seq":83,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:11.893753+0000","last_change":"2024-01-15T18:24:11.893787+0000","last_active":"2024-01-15T18:24:11.893753+0000","last_peered":"2024-01-15T18:24:11.893753+0000","last_clean":"2024-01-15T18:24:11.893753+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:24:11.893753+0000","last_undegraded":"2024-01-15T18:24:11.893753+0000","last_fullsized":"2024-01-15T18:24:11.893753+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'50","last_scrub_stamp":"2024-01-15T18:24:11.893670+0000","last_deep_scrub":"113'50","last_deep_scrub_stamp":"2024-01-15T18:24:11.893670+0000","last_clean_scrub_stamp":"2024-01-15T18:24:11.893670+0000","objects_scrubbed":15,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:47.830514+0000","scrub_duration":0.110496677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'43","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.972411+0000","last_change":"2024-01-15T18:18:41.942491+0000","last_active":"2024-01-15T18:23:45.972411+0000","last_peered":"2024-01-15T18:23:45.972411+0000","last_clean":"2024-01-15T18:23:45.972411+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:23:45.972411+0000","last_undegraded":"2024-01-15T18:23:45.972411+0000","last_fullsized":"2024-01-15T18:23:45.972411+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'56","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:17.996721+0000","last_change":"2024-01-15T18:24:17.996777+0000","last_active":"2024-01-15T18:24:17.996721+0000","last_peered":"2024-01-15T18:24:17.996721+0000","last_clean":"2024-01-15T18:24:17.996721+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:24:17.996721+0000","last_undegraded":"2024-01-15T18:24:17.996721+0000","last_fullsized":"2024-01-15T18:24:17.996721+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'56","last_scrub_stamp":"2024-01-15T18:24:17.996618+0000","last_deep_scrub":"113'56","last_deep_scrub_stamp":"2024-01-15T18:24:17.996618+0000","last_clean_scrub_stamp":"2024-01-15T18:24:17.996618+0000","objects_scrubbed":18,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:41.514918+0000","scrub_duration":0.132082754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'73","reported_seq":116,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.623718+0000","last_change":"2024-01-15T18:24:08.623783+0000","last_active":"2024-01-15T18:24:08.623718+0000","last_peered":"2024-01-15T18:24:08.623718+0000","last_clean":"2024-01-15T18:24:08.623718+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:24:08.623718+0000","last_undegraded":"2024-01-15T18:24:08.623718+0000","last_fullsized":"2024-01-15T18:24:08.623718+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'73","last_scrub_stamp":"2024-01-15T18:24:08.623631+0000","last_deep_scrub":"113'73","last_deep_scrub_stamp":"2024-01-15T18:24:08.623631+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.623631+0000","objects_scrubbed":23,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:32:40.432035+0000","scrub_duration":0.13678316800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'58","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:07.616751+0000","last_change":"2024-01-15T18:24:07.616784+0000","last_active":"2024-01-15T18:24:07.616751+0000","last_peered":"2024-01-15T18:24:07.616751+0000","last_clean":"2024-01-15T18:24:07.616751+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:24:07.616751+0000","last_undegraded":"2024-01-15T18:24:07.616751+0000","last_fullsized":"2024-01-15T18:24:07.616751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'58","last_scrub_stamp":"2024-01-15T18:24:07.616679+0000","last_deep_scrub":"113'58","last_deep_scrub_stamp":"2024-01-15T18:24:07.616679+0000","last_clean_scrub_stamp":"2024-01-15T18:24:07.616679+0000","objects_scrubbed":21,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:59.993118+0000","scrub_duration":0.170067107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":28673,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'55","reported_seq":82,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:22.761552+0000","last_change":"2024-01-15T18:24:22.761587+0000","last_active":"2024-01-15T18:24:22.761552+0000","last_peered":"2024-01-15T18:24:22.761552+0000","last_clean":"2024-01-15T18:24:22.761552+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:24:22.761552+0000","last_undegraded":"2024-01-15T18:24:22.761552+0000","last_fullsized":"2024-01-15T18:24:22.761552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'55","last_scrub_stamp":"2024-01-15T18:24:22.761479+0000","last_deep_scrub":"113'55","last_deep_scrub_stamp":"2024-01-15T18:24:22.761479+0000","last_clean_scrub_stamp":"2024-01-15T18:24:22.761479+0000","objects_scrubbed":17,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:43:33.628429+0000","scrub_duration":0.122328293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'64","reported_seq":98,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.964719+0000","last_change":"2024-01-15T18:24:10.964744+0000","last_active":"2024-01-15T18:24:10.964719+0000","last_peered":"2024-01-15T18:24:10.964719+0000","last_clean":"2024-01-15T18:24:10.964719+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:24:10.964719+0000","last_undegraded":"2024-01-15T18:24:10.964719+0000","last_fullsized":"2024-01-15T18:24:10.964719+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:10.964636+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:10.964636+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.964636+0000","objects_scrubbed":26,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:25.635165+0000","scrub_duration":0.19124144400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'54","reported_seq":87,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:00.920221+0000","last_change":"2024-01-15T18:24:00.920252+0000","last_active":"2024-01-15T18:24:00.920221+0000","last_peered":"2024-01-15T18:24:00.920221+0000","last_clean":"2024-01-15T18:24:00.920221+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:24:00.920221+0000","last_undegraded":"2024-01-15T18:24:00.920221+0000","last_fullsized":"2024-01-15T18:24:00.920221+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'54","last_scrub_stamp":"2024-01-15T18:24:00.920149+0000","last_deep_scrub":"113'54","last_deep_scrub_stamp":"2024-01-15T18:24:00.920149+0000","last_clean_scrub_stamp":"2024-01-15T18:24:00.920149+0000","objects_scrubbed":16,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:12:31.167250+0000","scrub_duration":0.10608548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'88","reported_seq":125,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:20.670931+0000","last_change":"2024-01-15T18:24:20.670953+0000","last_active":"2024-01-15T18:24:20.670931+0000","last_peered":"2024-01-15T18:24:20.670931+0000","last_clean":"2024-01-15T18:24:20.670931+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:24:20.670931+0000","last_undegraded":"2024-01-15T18:24:20.670931+0000","last_fullsized":"2024-01-15T18:24:20.670931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'88","last_scrub_stamp":"2024-01-15T18:24:20.670863+0000","last_deep_scrub":"113'88","last_deep_scrub_stamp":"2024-01-15T18:24:20.670863+0000","last_clean_scrub_stamp":"2024-01-15T18:24:20.670863+0000","objects_scrubbed":26,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:12.822752+0000","scrub_duration":0.15657138600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'72","reported_seq":103,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.607585+0000","last_change":"2024-01-15T18:24:01.607622+0000","last_active":"2024-01-15T18:24:01.607585+0000","last_peered":"2024-01-15T18:24:01.607585+0000","last_clean":"2024-01-15T18:24:01.607585+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:24:01.607585+0000","last_undegraded":"2024-01-15T18:24:01.607585+0000","last_fullsized":"2024-01-15T18:24:01.607585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'72","last_scrub_stamp":"2024-01-15T18:24:01.607513+0000","last_deep_scrub":"113'72","last_deep_scrub_stamp":"2024-01-15T18:24:01.607513+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.607513+0000","objects_scrubbed":22,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:02.388269+0000","scrub_duration":0.15048887599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'49","reported_seq":79,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:26.577409+0000","last_change":"2024-01-15T18:24:26.577457+0000","last_active":"2024-01-15T18:24:26.577409+0000","last_peered":"2024-01-15T18:24:26.577409+0000","last_clean":"2024-01-15T18:24:26.577409+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:24:26.577409+0000","last_undegraded":"2024-01-15T18:24:26.577409+0000","last_fullsized":"2024-01-15T18:24:26.577409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'49","last_scrub_stamp":"2024-01-15T18:24:26.577322+0000","last_deep_scrub":"113'49","last_deep_scrub_stamp":"2024-01-15T18:24:26.577322+0000","last_clean_scrub_stamp":"2024-01-15T18:24:26.577322+0000","objects_scrubbed":15,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:00.599189+0000","scrub_duration":0.109099811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'6","reported_seq":31,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.867891+0000","last_change":"2024-01-15T18:23:57.867914+0000","last_active":"2024-01-15T18:23:57.867891+0000","last_peered":"2024-01-15T18:23:57.867891+0000","last_clean":"2024-01-15T18:23:57.867891+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:23:57.867891+0000","last_undegraded":"2024-01-15T18:23:57.867891+0000","last_fullsized":"2024-01-15T18:23:57.867891+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'6","last_scrub_stamp":"2024-01-15T18:23:57.867799+0000","last_deep_scrub":"113'6","last_deep_scrub_stamp":"2024-01-15T18:23:57.867799+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.867799+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:03:18.594537+0000","scrub_duration":0.0040798709999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'69","reported_seq":101,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:18.847983+0000","last_change":"2024-01-15T18:24:18.848023+0000","last_active":"2024-01-15T18:24:18.847983+0000","last_peered":"2024-01-15T18:24:18.847983+0000","last_clean":"2024-01-15T18:24:18.847983+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:24:18.847983+0000","last_undegraded":"2024-01-15T18:24:18.847983+0000","last_fullsized":"2024-01-15T18:24:18.847983+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'69","last_scrub_stamp":"2024-01-15T18:24:18.847888+0000","last_deep_scrub":"113'69","last_deep_scrub_stamp":"2024-01-15T18:24:18.847888+0000","last_clean_scrub_stamp":"2024-01-15T18:24:18.847888+0000","objects_scrubbed":22,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:48.006506+0000","scrub_duration":0.15792041500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.978967+0000","last_change":"2024-01-15T18:23:59.978990+0000","last_active":"2024-01-15T18:23:59.978967+0000","last_peered":"2024-01-15T18:23:59.978967+0000","last_clean":"2024-01-15T18:23:59.978967+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:23:59.978967+0000","last_undegraded":"2024-01-15T18:23:59.978967+0000","last_fullsized":"2024-01-15T18:23:59.978967+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:59.978827+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:59.978827+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.978827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:35.915671+0000","scrub_duration":0.0020127669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'62","reported_seq":97,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.582626+0000","last_change":"2024-01-15T18:24:25.582659+0000","last_active":"2024-01-15T18:24:25.582626+0000","last_peered":"2024-01-15T18:24:25.582626+0000","last_clean":"2024-01-15T18:24:25.582626+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:24:25.582626+0000","last_undegraded":"2024-01-15T18:24:25.582626+0000","last_fullsized":"2024-01-15T18:24:25.582626+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:24:25.582553+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:24:25.582553+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.582553+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:46:04.944469+0000","scrub_duration":0.12929829300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:51.632037+0000","last_change":"2024-01-15T18:23:51.632064+0000","last_active":"2024-01-15T18:23:51.632037+0000","last_peered":"2024-01-15T18:23:51.632037+0000","last_clean":"2024-01-15T18:23:51.632037+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:23:51.632037+0000","last_undegraded":"2024-01-15T18:23:51.632037+0000","last_fullsized":"2024-01-15T18:23:51.632037+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:51.631949+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:51.631949+0000","last_clean_scrub_stamp":"2024-01-15T18:23:51.631949+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:46:13.718266+0000","scrub_duration":0.003650155,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'79","reported_seq":114,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:20.252089+0000","last_change":"2024-01-15T18:24:20.252134+0000","last_active":"2024-01-15T18:24:20.252089+0000","last_peered":"2024-01-15T18:24:20.252089+0000","last_clean":"2024-01-15T18:24:20.252089+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:24:20.252089+0000","last_undegraded":"2024-01-15T18:24:20.252089+0000","last_fullsized":"2024-01-15T18:24:20.252089+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'79","last_scrub_stamp":"2024-01-15T18:24:20.252006+0000","last_deep_scrub":"113'79","last_deep_scrub_stamp":"2024-01-15T18:24:20.252006+0000","last_clean_scrub_stamp":"2024-01-15T18:24:20.252006+0000","objects_scrubbed":23,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:57.918070+0000","scrub_duration":0.144693926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:02.499567+0000","last_change":"2024-01-15T18:24:02.499587+0000","last_active":"2024-01-15T18:24:02.499567+0000","last_peered":"2024-01-15T18:24:02.499567+0000","last_clean":"2024-01-15T18:24:02.499567+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:24:02.499567+0000","last_undegraded":"2024-01-15T18:24:02.499567+0000","last_fullsized":"2024-01-15T18:24:02.499567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:24:02.499486+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:24:02.499486+0000","last_clean_scrub_stamp":"2024-01-15T18:24:02.499486+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:00.198981+0000","scrub_duration":0.0029058650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'53","reported_seq":85,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.535064+0000","last_change":"2024-01-15T18:23:57.535097+0000","last_active":"2024-01-15T18:23:57.535064+0000","last_peered":"2024-01-15T18:23:57.535064+0000","last_clean":"2024-01-15T18:23:57.535064+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:57.535064+0000","last_undegraded":"2024-01-15T18:23:57.535064+0000","last_fullsized":"2024-01-15T18:23:57.535064+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:23:57.534962+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:23:57.534962+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.534962+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:34.769272+0000","scrub_duration":0.11579978,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:55.839729+0000","last_change":"2024-01-15T18:23:55.839753+0000","last_active":"2024-01-15T18:23:55.839729+0000","last_peered":"2024-01-15T18:23:55.839729+0000","last_clean":"2024-01-15T18:23:55.839729+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:23:55.839729+0000","last_undegraded":"2024-01-15T18:23:55.839729+0000","last_fullsized":"2024-01-15T18:23:55.839729+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:55.839629+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:55.839629+0000","last_clean_scrub_stamp":"2024-01-15T18:23:55.839629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:12.580092+0000","scrub_duration":0.0024064479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'68","reported_seq":105,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:09.868471+0000","last_change":"2024-01-15T18:24:09.868505+0000","last_active":"2024-01-15T18:24:09.868471+0000","last_peered":"2024-01-15T18:24:09.868471+0000","last_clean":"2024-01-15T18:24:09.868471+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:24:09.868471+0000","last_undegraded":"2024-01-15T18:24:09.868471+0000","last_fullsized":"2024-01-15T18:24:09.868471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:09.868399+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:09.868399+0000","last_clean_scrub_stamp":"2024-01-15T18:24:09.868399+0000","objects_scrubbed":24,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:40.404894+0000","scrub_duration":0.139674142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:47.658985+0000","last_change":"2024-01-15T18:23:47.659004+0000","last_active":"2024-01-15T18:23:47.658985+0000","last_peered":"2024-01-15T18:23:47.658985+0000","last_clean":"2024-01-15T18:23:47.658985+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:23:47.658985+0000","last_undegraded":"2024-01-15T18:23:47.658985+0000","last_fullsized":"2024-01-15T18:23:47.658985+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:47.658903+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:47.658903+0000","last_clean_scrub_stamp":"2024-01-15T18:23:47.658903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:12:17.906004+0000","scrub_duration":0.002056914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'71","reported_seq":117,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:14.006722+0000","last_change":"2024-01-15T18:24:14.006760+0000","last_active":"2024-01-15T18:24:14.006722+0000","last_peered":"2024-01-15T18:24:14.006722+0000","last_clean":"2024-01-15T18:24:14.006722+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:24:14.006722+0000","last_undegraded":"2024-01-15T18:24:14.006722+0000","last_fullsized":"2024-01-15T18:24:14.006722+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'71","last_scrub_stamp":"2024-01-15T18:24:14.006641+0000","last_deep_scrub":"113'71","last_deep_scrub_stamp":"2024-01-15T18:24:14.006641+0000","last_clean_scrub_stamp":"2024-01-15T18:24:14.006641+0000","objects_scrubbed":26,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:27.709904+0000","scrub_duration":0.195203722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857792,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:55.766357+0000","last_change":"2024-01-15T18:23:55.766378+0000","last_active":"2024-01-15T18:23:55.766357+0000","last_peered":"2024-01-15T18:23:55.766357+0000","last_clean":"2024-01-15T18:23:55.766357+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:23:55.766357+0000","last_undegraded":"2024-01-15T18:23:55.766357+0000","last_fullsized":"2024-01-15T18:23:55.766357+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:55.766236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:55.766236+0000","last_clean_scrub_stamp":"2024-01-15T18:23:55.766236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:28.192190+0000","scrub_duration":0.0029978629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'56","reported_seq":86,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:06.984054+0000","last_change":"2024-01-15T18:24:06.984094+0000","last_active":"2024-01-15T18:24:06.984054+0000","last_peered":"2024-01-15T18:24:06.984054+0000","last_clean":"2024-01-15T18:24:06.984054+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:24:06.984054+0000","last_undegraded":"2024-01-15T18:24:06.984054+0000","last_fullsized":"2024-01-15T18:24:06.984054+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'56","last_scrub_stamp":"2024-01-15T18:24:06.983972+0000","last_deep_scrub":"113'56","last_deep_scrub_stamp":"2024-01-15T18:24:06.983972+0000","last_clean_scrub_stamp":"2024-01-15T18:24:06.983972+0000","objects_scrubbed":16,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:55.181591+0000","scrub_duration":0.11652866200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.457759+0000","last_change":"2024-01-15T18:23:59.457780+0000","last_active":"2024-01-15T18:23:59.457759+0000","last_peered":"2024-01-15T18:23:59.457759+0000","last_clean":"2024-01-15T18:23:59.457759+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:23:59.457759+0000","last_undegraded":"2024-01-15T18:23:59.457759+0000","last_fullsized":"2024-01-15T18:23:59.457759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.457652+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:36.928230+0000","scrub_duration":0.0021356639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'49","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:12.825576+0000","last_change":"2024-01-15T18:24:12.825601+0000","last_active":"2024-01-15T18:24:12.825576+0000","last_peered":"2024-01-15T18:24:12.825576+0000","last_clean":"2024-01-15T18:24:12.825576+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:24:12.825576+0000","last_undegraded":"2024-01-15T18:24:12.825576+0000","last_fullsized":"2024-01-15T18:24:12.825576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'49","last_scrub_stamp":"2024-01-15T18:24:12.825497+0000","last_deep_scrub":"113'49","last_deep_scrub_stamp":"2024-01-15T18:24:12.825497+0000","last_clean_scrub_stamp":"2024-01-15T18:24:12.825497+0000","objects_scrubbed":8,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:06:51.969992+0000","scrub_duration":0.049176540999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.513102+0000","last_change":"2024-01-15T18:23:59.513138+0000","last_active":"2024-01-15T18:23:59.513102+0000","last_peered":"2024-01-15T18:23:59.513102+0000","last_clean":"2024-01-15T18:23:59.513102+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:23:59.513102+0000","last_undegraded":"2024-01-15T18:23:59.513102+0000","last_fullsized":"2024-01-15T18:23:59.513102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'1","last_scrub_stamp":"2024-01-15T18:23:59.513003+0000","last_deep_scrub":"111'1","last_deep_scrub_stamp":"2024-01-15T18:23:59.513003+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.513003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:34:17.358881+0000","scrub_duration":0.001554124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'53","reported_seq":82,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:12.721675+0000","last_change":"2024-01-15T18:24:12.721709+0000","last_active":"2024-01-15T18:24:12.721675+0000","last_peered":"2024-01-15T18:24:12.721675+0000","last_clean":"2024-01-15T18:24:12.721675+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:24:12.721675+0000","last_undegraded":"2024-01-15T18:24:12.721675+0000","last_fullsized":"2024-01-15T18:24:12.721675+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:24:12.721589+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:24:12.721589+0000","last_clean_scrub_stamp":"2024-01-15T18:24:12.721589+0000","objects_scrubbed":18,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:19.800682+0000","scrub_duration":0.117736047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.756143+0000","last_change":"2024-01-15T18:23:59.756175+0000","last_active":"2024-01-15T18:23:59.756143+0000","last_peered":"2024-01-15T18:23:59.756143+0000","last_clean":"2024-01-15T18:23:59.756143+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:23:59.756143+0000","last_undegraded":"2024-01-15T18:23:59.756143+0000","last_fullsized":"2024-01-15T18:23:59.756143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.756033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.756033+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.756033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:34.482493+0000","scrub_duration":0.0022605020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'65","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:19.767777+0000","last_change":"2024-01-15T18:24:19.767819+0000","last_active":"2024-01-15T18:24:19.767777+0000","last_peered":"2024-01-15T18:24:19.767777+0000","last_clean":"2024-01-15T18:24:19.767777+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:24:19.767777+0000","last_undegraded":"2024-01-15T18:24:19.767777+0000","last_fullsized":"2024-01-15T18:24:19.767777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'65","last_scrub_stamp":"2024-01-15T18:24:19.767697+0000","last_deep_scrub":"113'65","last_deep_scrub_stamp":"2024-01-15T18:24:19.767697+0000","last_clean_scrub_stamp":"2024-01-15T18:24:19.767697+0000","objects_scrubbed":22,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:43:28.256474+0000","scrub_duration":0.14783215899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'13","reported_seq":38,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:09.423971+0000","last_change":"2024-01-15T18:24:09.423992+0000","last_active":"2024-01-15T18:24:09.423971+0000","last_peered":"2024-01-15T18:24:09.423971+0000","last_clean":"2024-01-15T18:24:09.423971+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:24:09.423971+0000","last_undegraded":"2024-01-15T18:24:09.423971+0000","last_fullsized":"2024-01-15T18:24:09.423971+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'13","last_scrub_stamp":"2024-01-15T18:24:09.423875+0000","last_deep_scrub":"113'13","last_deep_scrub_stamp":"2024-01-15T18:24:09.423875+0000","last_clean_scrub_stamp":"2024-01-15T18:24:09.423875+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:23.481555+0000","scrub_duration":0.0038319389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'60","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:05.973582+0000","last_change":"2024-01-15T18:24:05.973644+0000","last_active":"2024-01-15T18:24:05.973582+0000","last_peered":"2024-01-15T18:24:05.973582+0000","last_clean":"2024-01-15T18:24:05.973582+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:24:05.973582+0000","last_undegraded":"2024-01-15T18:24:05.973582+0000","last_fullsized":"2024-01-15T18:24:05.973582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:05.973484+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:05.973484+0000","last_clean_scrub_stamp":"2024-01-15T18:24:05.973484+0000","objects_scrubbed":20,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:18:40.025089+0000","scrub_duration":0.14927474800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'189","reported_seq":214,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.457917+0000","last_change":"2024-01-15T18:24:08.457943+0000","last_active":"2024-01-15T18:24:08.457917+0000","last_peered":"2024-01-15T18:24:08.457917+0000","last_clean":"2024-01-15T18:24:08.457917+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:24:08.457917+0000","last_undegraded":"2024-01-15T18:24:08.457917+0000","last_fullsized":"2024-01-15T18:24:08.457917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'189","last_scrub_stamp":"2024-01-15T18:24:08.457835+0000","last_deep_scrub":"113'189","last_deep_scrub_stamp":"2024-01-15T18:24:08.457835+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.457835+0000","objects_scrubbed":4,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:00:50.921849+0000","scrub_duration":0.0067328329999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":616274,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'64","reported_seq":94,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:15.755020+0000","last_change":"2024-01-15T18:24:15.755054+0000","last_active":"2024-01-15T18:24:15.755020+0000","last_peered":"2024-01-15T18:24:15.755020+0000","last_clean":"2024-01-15T18:24:15.755020+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:24:15.755020+0000","last_undegraded":"2024-01-15T18:24:15.755020+0000","last_fullsized":"2024-01-15T18:24:15.755020+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:15.754943+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:15.754943+0000","last_clean_scrub_stamp":"2024-01-15T18:24:15.754943+0000","objects_scrubbed":21,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:33.993030+0000","scrub_duration":0.13024677200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.459728+0000","last_change":"2024-01-15T18:23:57.459775+0000","last_active":"2024-01-15T18:23:57.459728+0000","last_peered":"2024-01-15T18:23:57.459728+0000","last_clean":"2024-01-15T18:23:57.459728+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:23:57.459728+0000","last_undegraded":"2024-01-15T18:23:57.459728+0000","last_fullsized":"2024-01-15T18:23:57.459728+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:57.459625+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:57.459625+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.459625+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:51.586938+0000","scrub_duration":0.0053587779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'52","reported_seq":73,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.833319+0000","last_change":"2024-01-15T18:18:41.942172+0000","last_active":"2024-01-15T18:23:45.833319+0000","last_peered":"2024-01-15T18:23:45.833319+0000","last_clean":"2024-01-15T18:23:45.833319+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:23:45.833319+0000","last_undegraded":"2024-01-15T18:23:45.833319+0000","last_fullsized":"2024-01-15T18:23:45.833319+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:54.393613+0000","last_change":"2024-01-15T18:23:54.393632+0000","last_active":"2024-01-15T18:23:54.393613+0000","last_peered":"2024-01-15T18:23:54.393613+0000","last_clean":"2024-01-15T18:23:54.393613+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:23:54.393613+0000","last_undegraded":"2024-01-15T18:23:54.393613+0000","last_fullsized":"2024-01-15T18:23:54.393613+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:54.393495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:54.393495+0000","last_clean_scrub_stamp":"2024-01-15T18:23:54.393495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:48:38.527529+0000","scrub_duration":0.0024808400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'68","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.818049+0000","last_change":"2024-01-15T18:24:10.818070+0000","last_active":"2024-01-15T18:24:10.818049+0000","last_peered":"2024-01-15T18:24:10.818049+0000","last_clean":"2024-01-15T18:24:10.818049+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:24:10.818049+0000","last_undegraded":"2024-01-15T18:24:10.818049+0000","last_fullsized":"2024-01-15T18:24:10.818049+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:10.817981+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:10.817981+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.817981+0000","objects_scrubbed":25,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:24.521244+0000","scrub_duration":0.14952846,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663557,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'12","reported_seq":37,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.559825+0000","last_change":"2024-01-15T18:23:58.559858+0000","last_active":"2024-01-15T18:23:58.559825+0000","last_peered":"2024-01-15T18:23:58.559825+0000","last_clean":"2024-01-15T18:23:58.559825+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:23:58.559825+0000","last_undegraded":"2024-01-15T18:23:58.559825+0000","last_fullsized":"2024-01-15T18:23:58.559825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'12","last_scrub_stamp":"2024-01-15T18:23:58.559722+0000","last_deep_scrub":"113'12","last_deep_scrub_stamp":"2024-01-15T18:23:58.559722+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.559722+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:38.037968+0000","scrub_duration":0.0054070120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'31","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:07.559609+0000","last_change":"2024-01-15T18:24:07.559659+0000","last_active":"2024-01-15T18:24:07.559609+0000","last_peered":"2024-01-15T18:24:07.559609+0000","last_clean":"2024-01-15T18:24:07.559609+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:24:07.559609+0000","last_undegraded":"2024-01-15T18:24:07.559609+0000","last_fullsized":"2024-01-15T18:24:07.559609+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'31","last_scrub_stamp":"2024-01-15T18:24:07.559498+0000","last_deep_scrub":"113'31","last_deep_scrub_stamp":"2024-01-15T18:24:07.559498+0000","last_clean_scrub_stamp":"2024-01-15T18:24:07.559498+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:56:33.811882+0000","scrub_duration":0.090809666999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'2","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:11.491658+0000","last_change":"2024-01-15T18:24:11.491681+0000","last_active":"2024-01-15T18:24:11.491658+0000","last_peered":"2024-01-15T18:24:11.491658+0000","last_clean":"2024-01-15T18:24:11.491658+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:24:11.491658+0000","last_undegraded":"2024-01-15T18:24:11.491658+0000","last_fullsized":"2024-01-15T18:24:11.491658+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-01-15T18:24:11.491563+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-01-15T18:24:11.491563+0000","last_clean_scrub_stamp":"2024-01-15T18:24:11.491563+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:12:58.961260+0000","scrub_duration":0.004071089,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'53","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:23.558948+0000","last_change":"2024-01-15T18:24:23.558977+0000","last_active":"2024-01-15T18:24:23.558948+0000","last_peered":"2024-01-15T18:24:23.558948+0000","last_clean":"2024-01-15T18:24:23.558948+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:24:23.558948+0000","last_undegraded":"2024-01-15T18:24:23.558948+0000","last_fullsized":"2024-01-15T18:24:23.558948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:24:23.558876+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:24:23.558876+0000","last_clean_scrub_stamp":"2024-01-15T18:24:23.558876+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:25.925401+0000","scrub_duration":0.123621357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.632496+0000","last_change":"2024-01-15T18:23:48.632517+0000","last_active":"2024-01-15T18:23:48.632496+0000","last_peered":"2024-01-15T18:23:48.632496+0000","last_clean":"2024-01-15T18:23:48.632496+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:23:48.632496+0000","last_undegraded":"2024-01-15T18:23:48.632496+0000","last_fullsized":"2024-01-15T18:23:48.632496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:23:48.632420+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:23:48.632420+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.632420+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:03.214903+0000","scrub_duration":0.0038853939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'44","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.674067+0000","last_change":"2024-01-15T18:18:41.939791+0000","last_active":"2024-01-15T18:23:45.674067+0000","last_peered":"2024-01-15T18:23:45.674067+0000","last_clean":"2024-01-15T18:23:45.674067+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:23:45.674067+0000","last_undegraded":"2024-01-15T18:23:45.674067+0000","last_fullsized":"2024-01-15T18:23:45.674067+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'2","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:04.477880+0000","last_change":"2024-01-15T18:24:04.477901+0000","last_active":"2024-01-15T18:24:04.477880+0000","last_peered":"2024-01-15T18:24:04.477880+0000","last_clean":"2024-01-15T18:24:04.477880+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:24:04.477880+0000","last_undegraded":"2024-01-15T18:24:04.477880+0000","last_fullsized":"2024-01-15T18:24:04.477880+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-01-15T18:24:04.477798+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-01-15T18:24:04.477798+0000","last_clean_scrub_stamp":"2024-01-15T18:24:04.477798+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:31.016306+0000","scrub_duration":0.0041278030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'63","reported_seq":99,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:26.250983+0000","last_change":"2024-01-15T18:24:26.251023+0000","last_active":"2024-01-15T18:24:26.250983+0000","last_peered":"2024-01-15T18:24:26.250983+0000","last_clean":"2024-01-15T18:24:26.250983+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:24:26.250983+0000","last_undegraded":"2024-01-15T18:24:26.250983+0000","last_fullsized":"2024-01-15T18:24:26.250983+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:24:26.250910+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:24:26.250910+0000","last_clean_scrub_stamp":"2024-01-15T18:24:26.250910+0000","objects_scrubbed":22,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:16:32.094041+0000","scrub_duration":0.15294004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.819756+0000","last_change":"2024-01-15T18:23:58.819772+0000","last_active":"2024-01-15T18:23:58.819756+0000","last_peered":"2024-01-15T18:23:58.819756+0000","last_clean":"2024-01-15T18:23:58.819756+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:23:58.819756+0000","last_undegraded":"2024-01-15T18:23:58.819756+0000","last_fullsized":"2024-01-15T18:23:58.819756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:58.819676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:58.819676+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.819676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:01.186201+0000","scrub_duration":0.0020202330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'74","reported_seq":107,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:16.189204+0000","last_change":"2024-01-15T18:24:16.189230+0000","last_active":"2024-01-15T18:24:16.189204+0000","last_peered":"2024-01-15T18:24:16.189204+0000","last_clean":"2024-01-15T18:24:16.189204+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:24:16.189204+0000","last_undegraded":"2024-01-15T18:24:16.189204+0000","last_fullsized":"2024-01-15T18:24:16.189204+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'74","last_scrub_stamp":"2024-01-15T18:24:16.189126+0000","last_deep_scrub":"113'74","last_deep_scrub_stamp":"2024-01-15T18:24:16.189126+0000","last_clean_scrub_stamp":"2024-01-15T18:24:16.189126+0000","objects_scrubbed":30,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:23:20.823376+0000","scrub_duration":0.21666178899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:50.448326+0000","last_change":"2024-01-15T18:23:50.448372+0000","last_active":"2024-01-15T18:23:50.448326+0000","last_peered":"2024-01-15T18:23:50.448326+0000","last_clean":"2024-01-15T18:23:50.448326+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:23:50.448326+0000","last_undegraded":"2024-01-15T18:23:50.448326+0000","last_fullsized":"2024-01-15T18:23:50.448326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:50.448207+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:50.448207+0000","last_clean_scrub_stamp":"2024-01-15T18:23:50.448207+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:29.926453+0000","scrub_duration":0.0039608999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'62","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.710805+0000","last_change":"2024-01-15T18:24:25.710836+0000","last_active":"2024-01-15T18:24:25.710805+0000","last_peered":"2024-01-15T18:24:25.710805+0000","last_clean":"2024-01-15T18:24:25.710805+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:24:25.710805+0000","last_undegraded":"2024-01-15T18:24:25.710805+0000","last_fullsized":"2024-01-15T18:24:25.710805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:24:25.710709+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:24:25.710709+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.710709+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:46.381145+0000","scrub_duration":0.114989034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:52.641865+0000","last_change":"2024-01-15T18:23:52.641887+0000","last_active":"2024-01-15T18:23:52.641865+0000","last_peered":"2024-01-15T18:23:52.641865+0000","last_clean":"2024-01-15T18:23:52.641865+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:23:52.641865+0000","last_undegraded":"2024-01-15T18:23:52.641865+0000","last_fullsized":"2024-01-15T18:23:52.641865+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:52.641753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:52.641753+0000","last_clean_scrub_stamp":"2024-01-15T18:23:52.641753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:20:47.869635+0000","scrub_duration":0.0018539540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'61","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.238170+0000","last_change":"2024-01-15T18:24:25.238202+0000","last_active":"2024-01-15T18:24:25.238170+0000","last_peered":"2024-01-15T18:24:25.238170+0000","last_clean":"2024-01-15T18:24:25.238170+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:24:25.238170+0000","last_undegraded":"2024-01-15T18:24:25.238170+0000","last_fullsized":"2024-01-15T18:24:25.238170+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'61","last_scrub_stamp":"2024-01-15T18:24:25.238094+0000","last_deep_scrub":"113'61","last_deep_scrub_stamp":"2024-01-15T18:24:25.238094+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.238094+0000","objects_scrubbed":17,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:07.702107+0000","scrub_duration":0.105860971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.472252+0000","last_change":"2024-01-15T18:23:59.472271+0000","last_active":"2024-01-15T18:23:59.472252+0000","last_peered":"2024-01-15T18:23:59.472252+0000","last_clean":"2024-01-15T18:23:59.472252+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:23:59.472252+0000","last_undegraded":"2024-01-15T18:23:59.472252+0000","last_fullsized":"2024-01-15T18:23:59.472252+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.472157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.472157+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.472157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:16:00.507773+0000","scrub_duration":0.00163682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'57","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.758838+0000","last_change":"2024-01-15T18:24:08.758873+0000","last_active":"2024-01-15T18:24:08.758838+0000","last_peered":"2024-01-15T18:24:08.758838+0000","last_clean":"2024-01-15T18:24:08.758838+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:24:08.758838+0000","last_undegraded":"2024-01-15T18:24:08.758838+0000","last_fullsized":"2024-01-15T18:24:08.758838+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'57","last_scrub_stamp":"2024-01-15T18:24:08.758766+0000","last_deep_scrub":"113'57","last_deep_scrub_stamp":"2024-01-15T18:24:08.758766+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.758766+0000","objects_scrubbed":17,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:11.665000+0000","scrub_duration":0.120732729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109125,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.426929+0000","last_change":"2024-01-15T18:24:01.426948+0000","last_active":"2024-01-15T18:24:01.426929+0000","last_peered":"2024-01-15T18:24:01.426929+0000","last_clean":"2024-01-15T18:24:01.426929+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:24:01.426929+0000","last_undegraded":"2024-01-15T18:24:01.426929+0000","last_fullsized":"2024-01-15T18:24:01.426929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:01.426834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:01.426834+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.426834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:39.092898+0000","scrub_duration":0.0023004760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'46","reported_seq":74,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.614302+0000","last_change":"2024-01-15T18:24:10.614346+0000","last_active":"2024-01-15T18:24:10.614302+0000","last_peered":"2024-01-15T18:24:10.614302+0000","last_clean":"2024-01-15T18:24:10.614302+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:24:10.614302+0000","last_undegraded":"2024-01-15T18:24:10.614302+0000","last_fullsized":"2024-01-15T18:24:10.614302+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'46","last_scrub_stamp":"2024-01-15T18:24:10.614210+0000","last_deep_scrub":"113'46","last_deep_scrub_stamp":"2024-01-15T18:24:10.614210+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.614210+0000","objects_scrubbed":13,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:45.340669+0000","scrub_duration":0.080420363999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.755286+0000","last_change":"2024-01-15T18:24:01.755309+0000","last_active":"2024-01-15T18:24:01.755286+0000","last_peered":"2024-01-15T18:24:01.755286+0000","last_clean":"2024-01-15T18:24:01.755286+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:24:01.755286+0000","last_undegraded":"2024-01-15T18:24:01.755286+0000","last_fullsized":"2024-01-15T18:24:01.755286+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:24:01.755211+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:24:01.755211+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.755211+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:34.181165+0000","scrub_duration":0.002739652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'41","reported_seq":69,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.477316+0000","last_change":"2024-01-15T18:18:41.935519+0000","last_active":"2024-01-15T18:23:45.477316+0000","last_peered":"2024-01-15T18:23:45.477316+0000","last_clean":"2024-01-15T18:23:45.477316+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:23:45.477316+0000","last_undegraded":"2024-01-15T18:23:45.477316+0000","last_fullsized":"2024-01-15T18:23:45.477316+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'18","reported_seq":43,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:07.473108+0000","last_change":"2024-01-15T18:24:07.473129+0000","last_active":"2024-01-15T18:24:07.473108+0000","last_peered":"2024-01-15T18:24:07.473108+0000","last_clean":"2024-01-15T18:24:07.473108+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:24:07.473108+0000","last_undegraded":"2024-01-15T18:24:07.473108+0000","last_fullsized":"2024-01-15T18:24:07.473108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'18","last_scrub_stamp":"2024-01-15T18:24:07.473023+0000","last_deep_scrub":"113'18","last_deep_scrub_stamp":"2024-01-15T18:24:07.473023+0000","last_clean_scrub_stamp":"2024-01-15T18:24:07.473023+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:30.825105+0000","scrub_duration":0.0022104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":506,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'46","reported_seq":76,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.650341+0000","last_change":"2024-01-15T18:18:41.937525+0000","last_active":"2024-01-15T18:23:45.650341+0000","last_peered":"2024-01-15T18:23:45.650341+0000","last_clean":"2024-01-15T18:23:45.650341+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:23:45.650341+0000","last_undegraded":"2024-01-15T18:23:45.650341+0000","last_fullsized":"2024-01-15T18:23:45.650341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:13.526357+0000","last_change":"2024-01-15T18:24:13.526376+0000","last_active":"2024-01-15T18:24:13.526357+0000","last_peered":"2024-01-15T18:24:13.526357+0000","last_clean":"2024-01-15T18:24:13.526357+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:24:13.526357+0000","last_undegraded":"2024-01-15T18:24:13.526357+0000","last_fullsized":"2024-01-15T18:24:13.526357+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:13.526263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:13.526263+0000","last_clean_scrub_stamp":"2024-01-15T18:24:13.526263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:30:10.984389+0000","scrub_duration":0.0022385819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'58","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.855949+0000","last_change":"2024-01-15T18:24:25.855988+0000","last_active":"2024-01-15T18:24:25.855949+0000","last_peered":"2024-01-15T18:24:25.855949+0000","last_clean":"2024-01-15T18:24:25.855949+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:24:25.855949+0000","last_undegraded":"2024-01-15T18:24:25.855949+0000","last_fullsized":"2024-01-15T18:24:25.855949+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'58","last_scrub_stamp":"2024-01-15T18:24:25.855865+0000","last_deep_scrub":"113'58","last_deep_scrub_stamp":"2024-01-15T18:24:25.855865+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.855865+0000","objects_scrubbed":19,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:26.822867+0000","scrub_duration":0.11873668900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.770011+0000","last_change":"2024-01-15T18:23:58.770042+0000","last_active":"2024-01-15T18:23:58.770011+0000","last_peered":"2024-01-15T18:23:58.770011+0000","last_clean":"2024-01-15T18:23:58.770011+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:23:58.770011+0000","last_undegraded":"2024-01-15T18:23:58.770011+0000","last_fullsized":"2024-01-15T18:23:58.770011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:58.769909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:58.769909+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.769909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:46.967528+0000","scrub_duration":0.0018566699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'60","reported_seq":91,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:13.978996+0000","last_change":"2024-01-15T18:24:13.979039+0000","last_active":"2024-01-15T18:24:13.978996+0000","last_peered":"2024-01-15T18:24:13.978996+0000","last_clean":"2024-01-15T18:24:13.978996+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:24:13.978996+0000","last_undegraded":"2024-01-15T18:24:13.978996+0000","last_fullsized":"2024-01-15T18:24:13.978996+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:13.978917+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:13.978917+0000","last_clean_scrub_stamp":"2024-01-15T18:24:13.978917+0000","objects_scrubbed":16,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:41:23.967366+0000","scrub_duration":0.13694841599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:49.798222+0000","last_change":"2024-01-15T18:23:49.798239+0000","last_active":"2024-01-15T18:23:49.798222+0000","last_peered":"2024-01-15T18:23:49.798222+0000","last_clean":"2024-01-15T18:23:49.798222+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:23:49.798222+0000","last_undegraded":"2024-01-15T18:23:49.798222+0000","last_fullsized":"2024-01-15T18:23:49.798222+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:49.798143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:49.798143+0000","last_clean_scrub_stamp":"2024-01-15T18:23:49.798143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:43.688382+0000","scrub_duration":0.0014863820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'80","reported_seq":113,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:03.884188+0000","last_change":"2024-01-15T18:24:03.884213+0000","last_active":"2024-01-15T18:24:03.884188+0000","last_peered":"2024-01-15T18:24:03.884188+0000","last_clean":"2024-01-15T18:24:03.884188+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:24:03.884188+0000","last_undegraded":"2024-01-15T18:24:03.884188+0000","last_fullsized":"2024-01-15T18:24:03.884188+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'80","last_scrub_stamp":"2024-01-15T18:24:03.884114+0000","last_deep_scrub":"113'80","last_deep_scrub_stamp":"2024-01-15T18:24:03.884114+0000","last_clean_scrub_stamp":"2024-01-15T18:24:03.884114+0000","objects_scrubbed":29,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:58.010620+0000","scrub_duration":0.17135824699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.761653+0000","last_change":"2024-01-15T18:23:48.761672+0000","last_active":"2024-01-15T18:23:48.761653+0000","last_peered":"2024-01-15T18:23:48.761653+0000","last_clean":"2024-01-15T18:23:48.761653+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:23:48.761653+0000","last_undegraded":"2024-01-15T18:23:48.761653+0000","last_fullsized":"2024-01-15T18:23:48.761653+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:48.761549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:48.761549+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.761549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:23.488009+0000","scrub_duration":0.0019591489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'46","reported_seq":70,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:46.009723+0000","last_change":"2024-01-15T18:18:41.942871+0000","last_active":"2024-01-15T18:23:46.009723+0000","last_peered":"2024-01-15T18:23:46.009723+0000","last_clean":"2024-01-15T18:23:46.009723+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:23:46.009723+0000","last_undegraded":"2024-01-15T18:23:46.009723+0000","last_fullsized":"2024-01-15T18:23:46.009723+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'11","reported_seq":45,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:53.596663+0000","last_change":"2024-01-15T18:23:53.596683+0000","last_active":"2024-01-15T18:23:53.596663+0000","last_peered":"2024-01-15T18:23:53.596663+0000","last_clean":"2024-01-15T18:23:53.596663+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:23:53.596663+0000","last_undegraded":"2024-01-15T18:23:53.596663+0000","last_fullsized":"2024-01-15T18:23:53.596663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'11","last_scrub_stamp":"2024-01-15T18:23:53.596595+0000","last_deep_scrub":"113'11","last_deep_scrub_stamp":"2024-01-15T18:23:53.596595+0000","last_clean_scrub_stamp":"2024-01-15T18:23:53.596595+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:48.256546+0000","scrub_duration":0.0028069950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'59","reported_seq":91,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:53.558536+0000","last_change":"2024-01-15T18:23:53.558572+0000","last_active":"2024-01-15T18:23:53.558536+0000","last_peered":"2024-01-15T18:23:53.558536+0000","last_clean":"2024-01-15T18:23:53.558536+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:53.558536+0000","last_undegraded":"2024-01-15T18:23:53.558536+0000","last_fullsized":"2024-01-15T18:23:53.558536+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'59","last_scrub_stamp":"2024-01-15T18:23:53.558463+0000","last_deep_scrub":"113'59","last_deep_scrub_stamp":"2024-01-15T18:23:53.558463+0000","last_clean_scrub_stamp":"2024-01-15T18:23:53.558463+0000","objects_scrubbed":23,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:58.192714+0000","scrub_duration":0.163912535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.443830+0000","last_change":"2024-01-15T18:24:10.443848+0000","last_active":"2024-01-15T18:24:10.443830+0000","last_peered":"2024-01-15T18:24:10.443830+0000","last_clean":"2024-01-15T18:24:10.443830+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:24:10.443830+0000","last_undegraded":"2024-01-15T18:24:10.443830+0000","last_fullsized":"2024-01-15T18:24:10.443830+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:10.443717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:10.443717+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.443717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:05:32.092645+0000","scrub_duration":0.0019560620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'31","reported_seq":55,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:45.998585+0000","last_change":"2024-01-15T18:18:41.941772+0000","last_active":"2024-01-15T18:23:45.998585+0000","last_peered":"2024-01-15T18:23:45.998585+0000","last_clean":"2024-01-15T18:23:45.998585+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:23:45.998585+0000","last_undegraded":"2024-01-15T18:23:45.998585+0000","last_fullsized":"2024-01-15T18:23:45.998585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:40.902736+0000","last_clean_scrub_stamp":"2024-01-15T18:18:40.902736+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.772874+0000","last_change":"2024-01-15T18:23:48.772893+0000","last_active":"2024-01-15T18:23:48.772874+0000","last_peered":"2024-01-15T18:23:48.772874+0000","last_clean":"2024-01-15T18:23:48.772874+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:23:48.772874+0000","last_undegraded":"2024-01-15T18:23:48.772874+0000","last_fullsized":"2024-01-15T18:23:48.772874+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:48.772769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:48.772769+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.772769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:41.149208+0000","scrub_duration":0.0017691320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"113'119","reported_seq":257,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:49.271682+0000","last_change":"2024-01-15T18:23:49.271712+0000","last_active":"2024-01-15T18:23:49.271682+0000","last_peered":"2024-01-15T18:23:49.271682+0000","last_clean":"2024-01-15T18:23:49.271682+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:23:49.271682+0000","last_undegraded":"2024-01-15T18:23:49.271682+0000","last_fullsized":"2024-01-15T18:23:49.271682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'119","last_scrub_stamp":"2024-01-15T18:23:49.271604+0000","last_deep_scrub":"113'119","last_deep_scrub_stamp":"2024-01-15T18:23:49.271604+0000","last_clean_scrub_stamp":"2024-01-15T18:23:49.271604+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:57.760381+0000","scrub_duration":0.19866204600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710136,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3796,"ondisk_log_size":3796,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":618393,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279,"num_write_kb":724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5506,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1302528,"data_stored":1236786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575088,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834544,"kb_used_data":1767336,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537296,"statfs":{"total":96636764160,"available":94758191104,"internally_reserved":0,"allocated":1809752064,"data_stored":1808947458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.69099999999999995,"15min":0.94399999999999995},"min":{"1min":0.48999999999999999,"5min":0.38,"15min":0.29999999999999999},"max":{"1min":0.85599999999999998,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.379},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":0.87},"min":{"1min":0.52700000000000002,"5min":0.37,"15min":0.28399999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":13.744},"last":0.439}]},{"osd":1,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.75,"15min":1.0840000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":0.99299999999999999,"5min":1.024,"15min":16.920000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":1.05},"min":{"1min":0.48899999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93200000000000005,"5min":1.04,"15min":31.774999999999999},"last":0.52600000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72599999999999998,"15min":1},"min":{"1min":0.57399999999999995,"5min":0.48699999999999999,"15min":0.377},"max":{"1min":1.056,"5min":1.085,"15min":30.846},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.749,"15min":1.0629999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.92600000000000005,"5min":1.0229999999999999,"15min":39.695},"last":0.498}]},{"osd":3,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.74099999999999999,"15min":1.1499999999999999},"min":{"1min":0.54500000000000004,"5min":0.437,"15min":0.38800000000000001},"max":{"1min":1.01,"5min":1.0109999999999999,"15min":26.463999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76600000000000001,"15min":1.198},"min":{"1min":0.57299999999999995,"5min":0.47099999999999997,"15min":0.41499999999999998},"max":{"1min":0.86699999999999999,"5min":1.0640000000000001,"15min":42.692},"last":0.53200000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58599999999999997,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.31,"15min":0.31},"max":{"1min":0.86699999999999999,"5min":0.874,"15min":14.087},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.63100000000000001,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.38700000000000001,"15min":0.36299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":2.952},"last":0.41699999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.69499999999999995},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.83099999999999996,"5min":1.0369999999999999,"15min":11.334},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.746},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91700000000000004,"5min":0.94499999999999995,"15min":8.423},"last":0.48599999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:23:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72299999999999998,"15min":0.78200000000000003},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.45500000000000002},"max":{"1min":0.95099999999999996,"5min":1.034,"15min":8.5030000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.71499999999999997,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.35299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":3.3450000000000002},"last":0.54000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1260524,"kb_used_data":1193316,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93111316,"statfs":{"total":96636764160,"available":95345987584,"internally_reserved":0,"allocated":1221955584,"data_stored":1221154106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.84199999999999997},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":15.257999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68400000000000005,"15min":0.753},"min":{"1min":0.501,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.93700000000000006,"5min":1.018,"15min":8.8030000000000008},"last":0.33100000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.80400000000000005},"min":{"1min":0.51000000000000001,"5min":0.47199999999999998,"15min":0.34300000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":6.8239999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.92100000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.32300000000000001},"max":{"1min":0.92900000000000005,"5min":0.96299999999999997,"15min":19.475000000000001},"last":0.53500000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.749,"15min":0.95199999999999996},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":1.028,"5min":1.0389999999999999,"15min":14.743},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.747,"15min":0.89300000000000002},"min":{"1min":0.52300000000000002,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.048,"5min":1.103,"15min":14.340999999999999},"last":0.39800000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72699999999999998,"15min":0.88200000000000001},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.0620000000000001,"15min":17.359999999999999},"last":0.441},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73999999999999999,"15min":1.016},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.45800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":18.791},"last":0.77700000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.70399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.32400000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":19.114999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.90700000000000003,"5min":0.93899999999999995,"15min":2.444},"last":0.47999999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63900000000000001,"15min":0.69299999999999995},"min":{"1min":0.42399999999999999,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":7.5880000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":0.96299999999999997,"15min":2.5259999999999998},"last":0.50600000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68100000000000005,"15min":0.80200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.38600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":11.907},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.83299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":27.651},"last":0.82599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575088,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":937072,"kb_used_data":861544,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93434768,"statfs":{"total":96636764160,"available":95677202432,"internally_reserved":0,"allocated":882221056,"data_stored":881416086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.67900000000000005,"15min":0.83699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":1.9350000000000001,"15min":11.519},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.93600000000000005},"min":{"1min":0.49199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85399999999999998,"5min":0.96999999999999997,"15min":12.952},"last":0.58899999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.74299999999999999,"15min":0.89100000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":1.0720000000000001,"15min":13.358000000000001},"last":0.627},{"interface":"front","average":{"1min":0.628,"5min":0.71599999999999997,"15min":0.91200000000000003},"min":{"1min":0.45900000000000002,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":1.6399999999999999,"15min":21.856999999999999},"last":0.84599999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.77100000000000002,"15min":0.94899999999999995},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":1.734,"15min":18.314},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.77400000000000002,"15min":1.0209999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":0.88300000000000001,"5min":1.762,"15min":21.120000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76800000000000002,"15min":0.94599999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.86499999999999999,"5min":1.956,"15min":13.317},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.75700000000000001,"15min":1.0109999999999999},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84399999999999997,"5min":1.6870000000000001,"15min":19.638000000000002},"last":0.78500000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.621,"15min":0.67300000000000004},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.90900000000000003,"5min":0.96399999999999997,"15min":5.899},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.60699999999999998,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.35299999999999998},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":3.3889999999999998},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.72799999999999998,"15min":0.78300000000000003},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":5.8680000000000003},"last":0.752},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.628,"15min":0.70299999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.95599999999999996,"5min":1.081,"15min":3.2759999999999998},"last":0.501},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.753,"15min":0.79800000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.56299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834575087,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1706920,"kb_used_data":1643872,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92664920,"statfs":{"total":96636764160,"available":94888878080,"internally_reserved":0,"allocated":1683324928,"data_stored":1682527615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69999999999999996,"15min":0.80600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93100000000000005,"5min":1.02,"15min":11.374000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71099999999999997,"15min":0.83399999999999996},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":14.488},"last":0.51300000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.46999999999999997,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":7.625},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76000000000000001,"15min":0.79000000000000004},"min":{"1min":0.499,"5min":0.497,"15min":0.372},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":4.4550000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.85999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.34599999999999997},"max":{"1min":1.877,"5min":1.877,"15min":7.6989999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76700000000000002,"15min":0.82299999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":1.173,"5min":1.173,"15min":7.4829999999999997},"last":0.67900000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73199999999999998,"15min":0.90100000000000002},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":1.141,"5min":1.141,"15min":14.529},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":25.774000000000001},"last":0.621}]},{"osd":5,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.33200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":9.9619999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68600000000000005,"15min":0.78200000000000003},"min":{"1min":0.45500000000000002,"5min":0.39400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93799999999999994,"5min":1.0109999999999999,"15min":25.887},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.67500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":7.5439999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.48999999999999999,"5min":0.46200000000000002,"15min":0.33900000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":3.6949999999999998},"last":0.58199999999999996}]},{"osd":7,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":0.77100000000000002,"15min":0.76500000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.30399999999999999},"max":{"1min":11.093,"5min":11.093,"15min":13.760999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.377},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":11.67},"last":0.69099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575087,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1453652,"kb_used_data":1386444,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92918188,"statfs":{"total":96636764160,"available":95148224512,"internally_reserved":0,"allocated":1419718656,"data_stored":1418903068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.48299999999999998,"15min":0.84499999999999997},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":26.364000000000001},"last":0.375},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53600000000000003,"15min":0.82499999999999996},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":23.074999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48999999999999999,"15min":0.82299999999999995},"min":{"1min":0.315,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":20.321000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.86599999999999999},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.335,"5min":2.335,"15min":26.260999999999999},"last":0.45900000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.90500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":23.036999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.628,"15min":1.01},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":26.404},"last":0.61899999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.79500000000000004,"15min":0.97299999999999998},"min":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.468,"5min":1.468,"15min":22.998000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":1.038},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1.0529999999999999,"5min":1.0960000000000001,"15min":20.399000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.79900000000000004,"15min":1.056},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":26.314},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80400000000000005,"15min":1.032},"min":{"1min":0.68100000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":22.908999999999999},"last":0.93600000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.82499999999999996,"15min":1.107},"min":{"1min":0.56899999999999995,"5min":0.51000000000000001,"15min":0.45600000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":20.445},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.84299999999999997,"15min":1.141},"min":{"1min":0.61099999999999999,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":26.091999999999999},"last":0.74399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.82499999999999996,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.853,"5min":1.853,"15min":23.111999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82899999999999996,"15min":1.2609999999999999},"min":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.46400000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":44.817999999999998},"last":0.96099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":969704,"kb_used_data":902496,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93402136,"statfs":{"total":96636764160,"available":95643787264,"internally_reserved":0,"allocated":924155904,"data_stored":923359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.5,"15min":0.75},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":12.695},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.94299999999999995},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":23.792999999999999},"last":0.40999999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.81899999999999995},"min":{"1min":0.36599999999999999,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":8.4380000000000006,"5min":8.4380000000000006,"15min":34.029000000000003},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54300000000000004,"15min":0.83599999999999997},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.627,"5min":1.627,"15min":20.619},"last":0.36799999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":1.0669999999999999},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":39.945999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.71399999999999997},"min":{"1min":0.38900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.84299999999999997,"15min":7.3609999999999998},"last":0.44500000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79100000000000004,"15min":0.91500000000000004},"min":{"1min":0.54100000000000004,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":0.94899999999999995,"5min":1.1100000000000001,"15min":7.4169999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.98799999999999999},"min":{"1min":0.56899999999999995,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.673,"5min":1.673,"15min":20.053000000000001},"last":0.59099999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":1.292},"min":{"1min":0.56100000000000005,"5min":0.502,"15min":0.45200000000000001},"max":{"1min":1.224,"5min":1.246,"15min":41.683999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":1.0349999999999999},"min":{"1min":0.57099999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.649,"5min":1.649,"15min":26.084},"last":0.55200000000000005}]},{"osd":6,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":1.0669999999999999},"min":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":22.199000000000002},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.78600000000000003,"15min":1.329},"min":{"1min":0.58599999999999997,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.3200000000000001,"5min":1.369,"15min":41.738999999999997},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84899999999999998,"15min":1.075},"min":{"1min":0.64900000000000002,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":20.158000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":1.3540000000000001},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.44800000000000001},"max":{"1min":1.2669999999999999,"5min":1.3,"15min":41.402000000000001},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575088,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1133560,"kb_used_data":1066352,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93238280,"statfs":{"total":96636764160,"available":95475998720,"internally_reserved":0,"allocated":1091944448,"data_stored":1091132307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":9.452},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60099999999999998,"15min":0.91800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.8010000000000002,"5min":4.9329999999999998,"15min":35.790999999999997},"last":0.44}]},{"osd":2,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.58199999999999996,"15min":0.749},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27600000000000002},"max":{"1min":0.91600000000000004,"5min":9.2609999999999992,"15min":15.137},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.77800000000000002},"min":{"1min":0.374,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":1.7649999999999999,"15min":9.8469999999999995},"last":0.56000000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.56699999999999995,"15min":0.91300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":3.278,"5min":3.278,"15min":27.643999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.874},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":1.02,"5min":1.198,"15min":15.608000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":1.069},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.45100000000000001},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":27.489999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.85499999999999998,"15min":1.0229999999999999},"min":{"1min":0.51300000000000001,"5min":0.495,"15min":0.46899999999999997},"max":{"1min":1.131,"5min":6.6950000000000003,"15min":10.038},"last":0.65800000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80200000000000005,"15min":1.095},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.5600000000000001,"5min":2.5600000000000001,"15min":27.547000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.85099999999999998,"15min":1.0389999999999999},"min":{"1min":0.63,"5min":0.59199999999999997,"15min":0.55400000000000005},"max":{"1min":1.337,"5min":1.337,"15min":9.9610000000000003},"last":0.63400000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":1.0960000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.605,"5min":2.605,"15min":27.579999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.83799999999999997,"15min":0.94599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42399999999999999},"max":{"1min":1.6950000000000001,"5min":5.6470000000000002,"15min":11.643000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.83699999999999997,"15min":1.04},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0149999999999999,"5min":5.819,"15min":11.833},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.018,"15min":1.1220000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.502},"max":{"1min":12.673999999999999,"5min":12.673999999999999,"15min":14.606},"last":0.70099999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575087,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1740896,"kb_used_data":1677848,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92630944,"statfs":{"total":96636764160,"available":94854086656,"internally_reserved":0,"allocated":1718116352,"data_stored":1717289103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.83699999999999997},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":16.149000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50700000000000001,"15min":0.82199999999999995},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":1.6579999999999999,"15min":13.018000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54800000000000004,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.307},"max":{"1min":0.73399999999999999,"5min":1.1759999999999999,"15min":36.386000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.78700000000000003},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68500000000000005,"5min":1.7689999999999999,"15min":16.556000000000001},"last":0.45000000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.84499999999999997},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80500000000000005,"15min":11.391999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":1.012},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.95599999999999996,"15min":45.128},"last":0.58399999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.92300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":1.4550000000000001,"15min":11.772},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":1.0129999999999999},"min":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":1.0589999999999999,"5min":1.115,"15min":14.831},"last":0.55300000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.55700000000000005,"15min":0.38600000000000001},"max":{"1min":1.008,"5min":1.179,"15min":11.287000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.94199999999999995},"min":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.7869999999999999,"15min":11.958},"last":0.57099999999999995}]},{"osd":6,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.99099999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46999999999999997},"max":{"1min":1.073,"5min":1.3859999999999999,"15min":13.055},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.96499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47699999999999998},"max":{"1min":1.0089999999999999,"5min":1.7170000000000001,"15min":11.833},"last":0.72099999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.83399999999999996,"15min":1.024},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80000000000000004,"15min":1.054},"min":{"1min":0.59799999999999998,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":11.321999999999999},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":634880,"data_stored":616386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":616344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1715482624,"data_stored":1715470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":922746880,"data_stored":922746880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":880807936,"data_stored":880804032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:24:29.948 INFO:tasks.ceph:pgid 17.3f last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.948 INFO:tasks.ceph:pgid 17.37 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.949 INFO:tasks.ceph:pgid 17.28 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.949 INFO:tasks.ceph:pgid 17.13 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.949 INFO:tasks.ceph:pgid 17.c last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.950 INFO:tasks.ceph:pgid 17.7 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.950 INFO:tasks.ceph:pgid 17.4 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.950 INFO:tasks.ceph:pgid 17.0 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.950 INFO:tasks.ceph:pgid 17.10 last_scrub_stamp 2024-01-15T18:18:40.902736+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=18, tm_sec=40, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=23, tm_sec=42, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:24:29.950 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-15T18:24:31.107 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:24:31.111 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:24:49.951 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:24:50.112 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.116+0000 7f4b14400640 1 Processor -- start 2024-01-15T18:24:50.113 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.116+0000 7f4b14400640 1 -- start start 2024-01-15T18:24:50.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b14400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c0c6230 0x7f4b0c0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b14400640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0c0c4d90 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b14400640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0c15b640 con 0x7f4b0c15b260 2024-01-15T18:24:50.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b14400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0cf800 con 0x7f4b0c0c6230 2024-01-15T18:24:50.114 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b11000640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4b0c0cdf20 0x7f4b0c0cbae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55988/0 (socket says 172.21.15.143:55988) 2024-01-15T18:24:50.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b11000640 1 -- 172.21.15.143:0/870723609 learned_addr learned my addr 172.21.15.143:0/870723609 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:24:50.115 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b12400640 1 --2- 172.21.15.143:0/870723609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c0c6230 0x7f4b0c0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655014375 0 0) 0x7f4b0c0c4d90 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aec003530 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821225507 0 0) 0x7f4b0c15b640 con 0x7f4b0c15b260 2024-01-15T18:24:50.116 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c0c4d90 con 0x7f4b0c15b260 2024-01-15T18:24:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b12400640 1 -- 172.21.15.143:0/870723609 >> v1:172.21.15.204:6789/0 conn(0x7f4b0c15b260 legacy=0x7f4b0c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b12400640 1 -- 172.21.15.143:0/870723609 >> v1:172.21.15.143:6789/0 conn(0x7f4b0c0cdf20 legacy=0x7f4b0c0cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.117 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.120+0000 7f4b12400640 1 -- 172.21.15.143:0/870723609 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c0cfae0 con 0x7f4b0c0c6230 2024-01-15T18:24:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b12400640 1 --2- 172.21.15.143:0/870723609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c0c6230 0x7f4b0c0cc1f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009dc0 tx=0x7f4b0401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57b1041c6e472649 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b040a5a00 con 0x7f4b0c0c6230 2024-01-15T18:24:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c0ccf40 con 0x7f4b0c0c6230 2024-01-15T18:24:50.118 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b0be00640 1 -- 172.21.15.143:0/870723609 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b040a5d60 con 0x7f4b0c0c6230 2024-01-15T18:24:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 -- 172.21.15.143:0/870723609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c0c6230 msgr2=0x7f4b0c0cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 --2- 172.21.15.143:0/870723609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c0c6230 0x7f4b0c0cc1f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009dc0 tx=0x7f4b0401c940 comp rx=0 tx=0).stop 2024-01-15T18:24:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 -- 172.21.15.143:0/870723609 shutdown_connections 2024-01-15T18:24:50.119 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 --2- 172.21.15.143:0/870723609 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c0c6230 0x7f4b0c0cc1f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 -- 172.21.15.143:0/870723609 >> 172.21.15.143:0/870723609 conn(0x7f4b0c0d0ed0 msgr2=0x7f4b0c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 -- 172.21.15.143:0/870723609 shutdown_connections 2024-01-15T18:24:50.120 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.124+0000 7f4b14400640 1 -- 172.21.15.143:0/870723609 wait complete. 2024-01-15T18:24:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 Processor -- start 2024-01-15T18:24:50.121 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- start start 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b0c0c6230 0x7f4b0c153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 0x7f4b0c1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c15b260 0x7f4b0c154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c156520 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c0c8140 con 0x7f4b0c0c6230 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c153d60 con 0x7f4b0c15b260 2024-01-15T18:24:50.123 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 0x7f4b0c1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 0x7f4b0c1546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55300/0 (socket says 172.21.15.143:55300) 2024-01-15T18:24:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 -- 172.21.15.143:0/3457569985 learned_addr learned my addr 172.21.15.143:0/3457569985 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:24:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b12400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c15b260 0x7f4b0c154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11a00640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b0c0c6230 0x7f4b0c153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 -- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c15b260 msgr2=0x7f4b0c154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.124 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c15b260 0x7f4b0c154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 -- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b0c0c6230 msgr2=0x7f4b0c153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b0c0c6230 0x7f4b0c153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 -- 172.21.15.143:0/3457569985 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04009a70 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11a00640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b0c0c6230 0x7f4b0c153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:24:50.125 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b11000640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 0x7f4b0c1546f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c420 tx=0x7f4af400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.126 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4005cf0 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4005e50 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4007310 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0c159710 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0c158e70 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.128+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0c0c8280 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.132+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 46) v1 ==== 87792+0+0 (secure 0 0 0) 0x7f4af4007470 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.132+0000 7f4b09600640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ae4067af0 0x7f4ae4069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.127 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.132+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f4af4082980 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.132+0000 7f4b11a00640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ae4067af0 0x7f4ae4069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.128 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.132+0000 7f4b11a00640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ae4067af0 0x7f4ae4069fb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4afc001740 tx=0x7f4afc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.131 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.136+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f4af40512c0 con 0x7f4b0c0cdf20 2024-01-15T18:24:50.276 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.280+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 --> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b0c067500 con 0x7f4ae4067af0 2024-01-15T18:24:50.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.288+0000 7f4b09600640 1 -- 172.21.15.143:0/3457569985 <== mgr.4107 v2:172.21.15.143:6832/3137793172 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+293201 (secure 0 0 0) 0x7f4b0c067500 con 0x7f4ae4067af0 2024-01-15T18:24:50.288 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:24:50.290 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2024-01-15T18:24:50.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ae4067af0 msgr2=0x7f4ae4069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.291 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ae4067af0 0x7f4ae4069fb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4afc001740 tx=0x7f4afc009040 comp rx=0 tx=0).stop 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 msgr2=0x7f4b0c1546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 0x7f4b0c1546f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c420 tx=0x7f4af400c8e0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 shutdown_connections 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f4ae4067af0 0x7f4ae4069fb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f4b0c15b260 0x7f4b0c154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4b0c0cdf20 0x7f4b0c1546f0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 --2- 172.21.15.143:0/3457569985 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4b0c0c6230 0x7f4b0c153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.292 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.292+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 >> 172.21.15.143:0/3457569985 conn(0x7f4b0c0d0ed0 msgr2=0x7f4b0c06fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:50.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.296+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 shutdown_connections 2024-01-15T18:24:50.293 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.296+0000 7f4b14400640 1 -- 172.21.15.143:0/3457569985 wait complete. 2024-01-15T18:24:50.309 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":707,"stamp":"2024-01-15T18:24:48.992806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369919038,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1746,"num_write_kb":5246037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5506,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4384,"ondisk_log_size":4384,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11036872,"kb_used_data":10499208,"kb_used_omap":12,"kb_used_meta":537587,"kb_avail":743937848,"statfs":{"total":773094113280,"available":761792356352,"internally_reserved":0,"allocated":10751188992,"data_stored":10744729676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":550489648},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003758"},"pg_stats":[{"pgid":"17.3f","version":"113'78","reported_seq":112,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:29.767758+0000","last_change":"2024-01-15T18:24:29.767796+0000","last_active":"2024-01-15T18:24:29.767758+0000","last_peered":"2024-01-15T18:24:29.767758+0000","last_clean":"2024-01-15T18:24:29.767758+0000","last_became_active":"2024-01-15T18:18:41.945985+0000","last_became_peered":"2024-01-15T18:18:41.945985+0000","last_unstale":"2024-01-15T18:24:29.767758+0000","last_undegraded":"2024-01-15T18:24:29.767758+0000","last_fullsized":"2024-01-15T18:24:29.767758+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'78","last_scrub_stamp":"2024-01-15T18:24:29.767629+0000","last_deep_scrub":"113'78","last_deep_scrub_stamp":"2024-01-15T18:24:29.767629+0000","last_clean_scrub_stamp":"2024-01-15T18:24:29.767629+0000","objects_scrubbed":25,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:25:24.296278+0000","scrub_duration":0.17294203699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"113'64","reported_seq":96,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:19.014535+0000","last_change":"2024-01-15T18:24:19.014568+0000","last_active":"2024-01-15T18:24:19.014535+0000","last_peered":"2024-01-15T18:24:19.014535+0000","last_clean":"2024-01-15T18:24:19.014535+0000","last_became_active":"2024-01-15T18:18:41.925961+0000","last_became_peered":"2024-01-15T18:18:41.925961+0000","last_unstale":"2024-01-15T18:24:19.014535+0000","last_undegraded":"2024-01-15T18:24:19.014535+0000","last_fullsized":"2024-01-15T18:24:19.014535+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:19.014459+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:19.014459+0000","last_clean_scrub_stamp":"2024-01-15T18:24:19.014459+0000","objects_scrubbed":19,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:22:41.309827+0000","scrub_duration":0.123762099,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"113'63","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:17.134552+0000","last_change":"2024-01-15T18:24:17.134593+0000","last_active":"2024-01-15T18:24:17.134552+0000","last_peered":"2024-01-15T18:24:17.134552+0000","last_clean":"2024-01-15T18:24:17.134552+0000","last_became_active":"2024-01-15T18:18:41.928260+0000","last_became_peered":"2024-01-15T18:18:41.928260+0000","last_unstale":"2024-01-15T18:24:17.134552+0000","last_undegraded":"2024-01-15T18:24:17.134552+0000","last_fullsized":"2024-01-15T18:24:17.134552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:24:17.134470+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:24:17.134470+0000","last_clean_scrub_stamp":"2024-01-15T18:24:17.134470+0000","objects_scrubbed":19,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:27.454408+0000","scrub_duration":0.131278227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"113'62","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:54.586767+0000","last_change":"2024-01-15T18:23:54.586767+0000","last_active":"2024-01-15T18:23:54.586767+0000","last_peered":"2024-01-15T18:23:54.586767+0000","last_clean":"2024-01-15T18:23:54.586767+0000","last_became_active":"2024-01-15T18:18:41.931981+0000","last_became_peered":"2024-01-15T18:18:41.931981+0000","last_unstale":"2024-01-15T18:23:54.586767+0000","last_undegraded":"2024-01-15T18:23:54.586767+0000","last_fullsized":"2024-01-15T18:23:54.586767+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:23:54.586620+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:23:54.586620+0000","last_clean_scrub_stamp":"2024-01-15T18:23:54.586620+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:04.906558+0000","scrub_duration":0.14577402,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"113'35","reported_seq":66,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:03.545628+0000","last_change":"2024-01-15T18:24:03.545686+0000","last_active":"2024-01-15T18:24:03.545628+0000","last_peered":"2024-01-15T18:24:03.545628+0000","last_clean":"2024-01-15T18:24:03.545628+0000","last_became_active":"2024-01-15T18:18:41.951504+0000","last_became_peered":"2024-01-15T18:18:41.951504+0000","last_unstale":"2024-01-15T18:24:03.545628+0000","last_undegraded":"2024-01-15T18:24:03.545628+0000","last_fullsized":"2024-01-15T18:24:03.545628+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'35","last_scrub_stamp":"2024-01-15T18:24:03.545528+0000","last_deep_scrub":"113'35","last_deep_scrub_stamp":"2024-01-15T18:24:03.545528+0000","last_clean_scrub_stamp":"2024-01-15T18:24:03.545528+0000","objects_scrubbed":10,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:24:34.025807+0000","scrub_duration":0.072323520000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"113'66","reported_seq":101,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.570290+0000","last_change":"2024-01-15T18:24:10.570328+0000","last_active":"2024-01-15T18:24:10.570290+0000","last_peered":"2024-01-15T18:24:10.570290+0000","last_clean":"2024-01-15T18:24:10.570290+0000","last_became_active":"2024-01-15T18:18:41.943955+0000","last_became_peered":"2024-01-15T18:18:41.943955+0000","last_unstale":"2024-01-15T18:24:10.570290+0000","last_undegraded":"2024-01-15T18:24:10.570290+0000","last_fullsized":"2024-01-15T18:24:10.570290+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'66","last_scrub_stamp":"2024-01-15T18:24:10.570213+0000","last_deep_scrub":"113'66","last_deep_scrub_stamp":"2024-01-15T18:24:10.570213+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.570213+0000","objects_scrubbed":21,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:04:00.820904+0000","scrub_duration":0.14447458199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"113'58","reported_seq":86,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.988814+0000","last_change":"2024-01-15T18:24:08.988849+0000","last_active":"2024-01-15T18:24:08.988814+0000","last_peered":"2024-01-15T18:24:08.988814+0000","last_clean":"2024-01-15T18:24:08.988814+0000","last_became_active":"2024-01-15T18:18:41.942112+0000","last_became_peered":"2024-01-15T18:18:41.942112+0000","last_unstale":"2024-01-15T18:24:08.988814+0000","last_undegraded":"2024-01-15T18:24:08.988814+0000","last_fullsized":"2024-01-15T18:24:08.988814+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'58","last_scrub_stamp":"2024-01-15T18:24:08.988737+0000","last_deep_scrub":"113'58","last_deep_scrub_stamp":"2024-01-15T18:24:08.988737+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.988737+0000","objects_scrubbed":17,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:16.645900+0000","scrub_duration":0.120076418,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"113'61","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:06.128647+0000","last_change":"2024-01-15T18:24:06.128704+0000","last_active":"2024-01-15T18:24:06.128647+0000","last_peered":"2024-01-15T18:24:06.128647+0000","last_clean":"2024-01-15T18:24:06.128647+0000","last_became_active":"2024-01-15T18:18:41.931889+0000","last_became_peered":"2024-01-15T18:18:41.931889+0000","last_unstale":"2024-01-15T18:24:06.128647+0000","last_undegraded":"2024-01-15T18:24:06.128647+0000","last_fullsized":"2024-01-15T18:24:06.128647+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'61","last_scrub_stamp":"2024-01-15T18:24:06.128559+0000","last_deep_scrub":"113'61","last_deep_scrub_stamp":"2024-01-15T18:24:06.128559+0000","last_clean_scrub_stamp":"2024-01-15T18:24:06.128559+0000","objects_scrubbed":19,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:03.610416+0000","scrub_duration":0.116078944,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"113'68","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:30.798541+0000","last_change":"2024-01-15T18:24:30.798581+0000","last_active":"2024-01-15T18:24:30.798541+0000","last_peered":"2024-01-15T18:24:30.798541+0000","last_clean":"2024-01-15T18:24:30.798541+0000","last_became_active":"2024-01-15T18:18:41.946668+0000","last_became_peered":"2024-01-15T18:18:41.946668+0000","last_unstale":"2024-01-15T18:24:30.798541+0000","last_undegraded":"2024-01-15T18:24:30.798541+0000","last_fullsized":"2024-01-15T18:24:30.798541+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:30.798475+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:30.798475+0000","last_clean_scrub_stamp":"2024-01-15T18:24:30.798475+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:07:51.591375+0000","scrub_duration":0.159035502,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"113'64","reported_seq":98,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:21.670959+0000","last_change":"2024-01-15T18:24:21.671012+0000","last_active":"2024-01-15T18:24:21.670959+0000","last_peered":"2024-01-15T18:24:21.670959+0000","last_clean":"2024-01-15T18:24:21.670959+0000","last_became_active":"2024-01-15T18:18:41.939907+0000","last_became_peered":"2024-01-15T18:18:41.939907+0000","last_unstale":"2024-01-15T18:24:21.670959+0000","last_undegraded":"2024-01-15T18:24:21.670959+0000","last_fullsized":"2024-01-15T18:24:21.670959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:21.670862+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:21.670862+0000","last_clean_scrub_stamp":"2024-01-15T18:24:21.670862+0000","objects_scrubbed":17,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:27.636410+0000","scrub_duration":0.117922286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"113'69","reported_seq":106,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.920167+0000","last_change":"2024-01-15T18:24:01.920191+0000","last_active":"2024-01-15T18:24:01.920167+0000","last_peered":"2024-01-15T18:24:01.920167+0000","last_clean":"2024-01-15T18:24:01.920167+0000","last_became_active":"2024-01-15T18:18:41.933811+0000","last_became_peered":"2024-01-15T18:18:41.933811+0000","last_unstale":"2024-01-15T18:24:01.920167+0000","last_undegraded":"2024-01-15T18:24:01.920167+0000","last_fullsized":"2024-01-15T18:24:01.920167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'69","last_scrub_stamp":"2024-01-15T18:24:01.920080+0000","last_deep_scrub":"113'69","last_deep_scrub_stamp":"2024-01-15T18:24:01.920080+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.920080+0000","objects_scrubbed":28,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:15.623343+0000","scrub_duration":0.20686890199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"113'76","reported_seq":117,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:23.822318+0000","last_change":"2024-01-15T18:24:23.822342+0000","last_active":"2024-01-15T18:24:23.822318+0000","last_peered":"2024-01-15T18:24:23.822318+0000","last_clean":"2024-01-15T18:24:23.822318+0000","last_became_active":"2024-01-15T18:18:41.937249+0000","last_became_peered":"2024-01-15T18:18:41.937249+0000","last_unstale":"2024-01-15T18:24:23.822318+0000","last_undegraded":"2024-01-15T18:24:23.822318+0000","last_fullsized":"2024-01-15T18:24:23.822318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'76","last_scrub_stamp":"2024-01-15T18:24:23.822246+0000","last_deep_scrub":"113'76","last_deep_scrub_stamp":"2024-01-15T18:24:23.822246+0000","last_clean_scrub_stamp":"2024-01-15T18:24:23.822246+0000","objects_scrubbed":28,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:17:07.022123+0000","scrub_duration":0.220839438,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"113'57","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:18.161646+0000","last_change":"2024-01-15T18:24:18.161677+0000","last_active":"2024-01-15T18:24:18.161646+0000","last_peered":"2024-01-15T18:24:18.161646+0000","last_clean":"2024-01-15T18:24:18.161646+0000","last_became_active":"2024-01-15T18:18:41.931167+0000","last_became_peered":"2024-01-15T18:18:41.931167+0000","last_unstale":"2024-01-15T18:24:18.161646+0000","last_undegraded":"2024-01-15T18:24:18.161646+0000","last_fullsized":"2024-01-15T18:24:18.161646+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'57","last_scrub_stamp":"2024-01-15T18:24:18.161575+0000","last_deep_scrub":"113'57","last_deep_scrub_stamp":"2024-01-15T18:24:18.161575+0000","last_clean_scrub_stamp":"2024-01-15T18:24:18.161575+0000","objects_scrubbed":19,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:44:42.184776+0000","scrub_duration":0.114720246,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"113'70","reported_seq":102,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.156856+0000","last_change":"2024-01-15T18:24:10.156878+0000","last_active":"2024-01-15T18:24:10.156856+0000","last_peered":"2024-01-15T18:24:10.156856+0000","last_clean":"2024-01-15T18:24:10.156856+0000","last_became_active":"2024-01-15T18:18:41.939552+0000","last_became_peered":"2024-01-15T18:18:41.939552+0000","last_unstale":"2024-01-15T18:24:10.156856+0000","last_undegraded":"2024-01-15T18:24:10.156856+0000","last_fullsized":"2024-01-15T18:24:10.156856+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'70","last_scrub_stamp":"2024-01-15T18:24:10.156783+0000","last_deep_scrub":"113'70","last_deep_scrub_stamp":"2024-01-15T18:24:10.156783+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.156783+0000","objects_scrubbed":27,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:39:42.365100+0000","scrub_duration":0.18950409400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"113'43","reported_seq":71,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:15.514128+0000","last_change":"2024-01-15T18:24:15.514162+0000","last_active":"2024-01-15T18:24:15.514128+0000","last_peered":"2024-01-15T18:24:15.514128+0000","last_clean":"2024-01-15T18:24:15.514128+0000","last_became_active":"2024-01-15T18:18:41.950757+0000","last_became_peered":"2024-01-15T18:18:41.950757+0000","last_unstale":"2024-01-15T18:24:15.514128+0000","last_undegraded":"2024-01-15T18:24:15.514128+0000","last_fullsized":"2024-01-15T18:24:15.514128+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'43","last_scrub_stamp":"2024-01-15T18:24:15.514051+0000","last_deep_scrub":"113'43","last_deep_scrub_stamp":"2024-01-15T18:24:15.514051+0000","last_clean_scrub_stamp":"2024-01-15T18:24:15.514051+0000","objects_scrubbed":12,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:37.561789+0000","scrub_duration":0.093520221000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"113'60","reported_seq":87,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:16.630070+0000","last_change":"2024-01-15T18:24:16.630104+0000","last_active":"2024-01-15T18:24:16.630070+0000","last_peered":"2024-01-15T18:24:16.630070+0000","last_clean":"2024-01-15T18:24:16.630070+0000","last_became_active":"2024-01-15T18:18:41.931101+0000","last_became_peered":"2024-01-15T18:18:41.931101+0000","last_unstale":"2024-01-15T18:24:16.630070+0000","last_undegraded":"2024-01-15T18:24:16.630070+0000","last_fullsized":"2024-01-15T18:24:16.630070+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:16.629993+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:16.629993+0000","last_clean_scrub_stamp":"2024-01-15T18:24:16.629993+0000","objects_scrubbed":18,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:28:37.606419+0000","scrub_duration":0.115176582,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"113'66","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:20.008135+0000","last_change":"2024-01-15T18:24:20.008183+0000","last_active":"2024-01-15T18:24:20.008135+0000","last_peered":"2024-01-15T18:24:20.008135+0000","last_clean":"2024-01-15T18:24:20.008135+0000","last_became_active":"2024-01-15T18:18:41.938783+0000","last_became_peered":"2024-01-15T18:18:41.938783+0000","last_unstale":"2024-01-15T18:24:20.008135+0000","last_undegraded":"2024-01-15T18:24:20.008135+0000","last_fullsized":"2024-01-15T18:24:20.008135+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'66","last_scrub_stamp":"2024-01-15T18:24:20.008049+0000","last_deep_scrub":"113'66","last_deep_scrub_stamp":"2024-01-15T18:24:20.008049+0000","last_clean_scrub_stamp":"2024-01-15T18:24:20.008049+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:23:24.642299+0000","scrub_duration":0.158380297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"113'67","reported_seq":105,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:11.726908+0000","last_change":"2024-01-15T18:24:11.726947+0000","last_active":"2024-01-15T18:24:11.726908+0000","last_peered":"2024-01-15T18:24:11.726908+0000","last_clean":"2024-01-15T18:24:11.726908+0000","last_became_active":"2024-01-15T18:18:41.925261+0000","last_became_peered":"2024-01-15T18:18:41.925261+0000","last_unstale":"2024-01-15T18:24:11.726908+0000","last_undegraded":"2024-01-15T18:24:11.726908+0000","last_fullsized":"2024-01-15T18:24:11.726908+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'67","last_scrub_stamp":"2024-01-15T18:24:11.726822+0000","last_deep_scrub":"113'67","last_deep_scrub_stamp":"2024-01-15T18:24:11.726822+0000","last_clean_scrub_stamp":"2024-01-15T18:24:11.726822+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:19.383985+0000","scrub_duration":0.14728950399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"113'65","reported_seq":94,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:19.209516+0000","last_change":"2024-01-15T18:24:19.209575+0000","last_active":"2024-01-15T18:24:19.209516+0000","last_peered":"2024-01-15T18:24:19.209516+0000","last_clean":"2024-01-15T18:24:19.209516+0000","last_became_active":"2024-01-15T18:18:41.942396+0000","last_became_peered":"2024-01-15T18:18:41.942396+0000","last_unstale":"2024-01-15T18:24:19.209516+0000","last_undegraded":"2024-01-15T18:24:19.209516+0000","last_fullsized":"2024-01-15T18:24:19.209516+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'65","last_scrub_stamp":"2024-01-15T18:24:19.209430+0000","last_deep_scrub":"113'65","last_deep_scrub_stamp":"2024-01-15T18:24:19.209430+0000","last_clean_scrub_stamp":"2024-01-15T18:24:19.209430+0000","objects_scrubbed":21,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:56.680008+0000","scrub_duration":0.13384733800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"113'63","reported_seq":95,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.808942+0000","last_change":"2024-01-15T18:23:58.808978+0000","last_active":"2024-01-15T18:23:58.808942+0000","last_peered":"2024-01-15T18:23:58.808942+0000","last_clean":"2024-01-15T18:23:58.808942+0000","last_became_active":"2024-01-15T18:18:41.926718+0000","last_became_peered":"2024-01-15T18:18:41.926718+0000","last_unstale":"2024-01-15T18:23:58.808942+0000","last_undegraded":"2024-01-15T18:23:58.808942+0000","last_fullsized":"2024-01-15T18:23:58.808942+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:23:58.808865+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:23:58.808865+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.808865+0000","objects_scrubbed":17,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:01.715099+0000","scrub_duration":0.125894276,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"113'76","reported_seq":114,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:02.904727+0000","last_change":"2024-01-15T18:24:02.904748+0000","last_active":"2024-01-15T18:24:02.904727+0000","last_peered":"2024-01-15T18:24:02.904727+0000","last_clean":"2024-01-15T18:24:02.904727+0000","last_became_active":"2024-01-15T18:18:41.937502+0000","last_became_peered":"2024-01-15T18:18:41.937502+0000","last_unstale":"2024-01-15T18:24:02.904727+0000","last_undegraded":"2024-01-15T18:24:02.904727+0000","last_fullsized":"2024-01-15T18:24:02.904727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'76","last_scrub_stamp":"2024-01-15T18:24:02.904627+0000","last_deep_scrub":"113'76","last_deep_scrub_stamp":"2024-01-15T18:24:02.904627+0000","last_clean_scrub_stamp":"2024-01-15T18:24:02.904627+0000","objects_scrubbed":26,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:21.142713+0000","scrub_duration":0.16157696399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"113'68","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:23.807206+0000","last_change":"2024-01-15T18:24:23.807266+0000","last_active":"2024-01-15T18:24:23.807206+0000","last_peered":"2024-01-15T18:24:23.807206+0000","last_clean":"2024-01-15T18:24:23.807206+0000","last_became_active":"2024-01-15T18:18:41.942006+0000","last_became_peered":"2024-01-15T18:18:41.942006+0000","last_unstale":"2024-01-15T18:24:23.807206+0000","last_undegraded":"2024-01-15T18:24:23.807206+0000","last_fullsized":"2024-01-15T18:24:23.807206+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:23.807112+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:23.807112+0000","last_clean_scrub_stamp":"2024-01-15T18:24:23.807112+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:38.477642+0000","scrub_duration":0.168180996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"113'50","reported_seq":83,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:11.893753+0000","last_change":"2024-01-15T18:24:11.893787+0000","last_active":"2024-01-15T18:24:11.893753+0000","last_peered":"2024-01-15T18:24:11.893753+0000","last_clean":"2024-01-15T18:24:11.893753+0000","last_became_active":"2024-01-15T18:18:41.926569+0000","last_became_peered":"2024-01-15T18:18:41.926569+0000","last_unstale":"2024-01-15T18:24:11.893753+0000","last_undegraded":"2024-01-15T18:24:11.893753+0000","last_fullsized":"2024-01-15T18:24:11.893753+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'50","last_scrub_stamp":"2024-01-15T18:24:11.893670+0000","last_deep_scrub":"113'50","last_deep_scrub_stamp":"2024-01-15T18:24:11.893670+0000","last_clean_scrub_stamp":"2024-01-15T18:24:11.893670+0000","objects_scrubbed":15,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:47.830514+0000","scrub_duration":0.110496677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"113'61","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:27.802341+0000","last_change":"2024-01-15T18:24:27.802374+0000","last_active":"2024-01-15T18:24:27.802341+0000","last_peered":"2024-01-15T18:24:27.802341+0000","last_clean":"2024-01-15T18:24:27.802341+0000","last_became_active":"2024-01-15T18:18:41.942197+0000","last_became_peered":"2024-01-15T18:18:41.942197+0000","last_unstale":"2024-01-15T18:24:27.802341+0000","last_undegraded":"2024-01-15T18:24:27.802341+0000","last_fullsized":"2024-01-15T18:24:27.802341+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'61","last_scrub_stamp":"2024-01-15T18:24:27.802269+0000","last_deep_scrub":"113'61","last_deep_scrub_stamp":"2024-01-15T18:24:27.802269+0000","last_clean_scrub_stamp":"2024-01-15T18:24:27.802269+0000","objects_scrubbed":18,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:10:55.856582+0000","scrub_duration":0.1228257,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"113'56","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:17.996721+0000","last_change":"2024-01-15T18:24:17.996777+0000","last_active":"2024-01-15T18:24:17.996721+0000","last_peered":"2024-01-15T18:24:17.996721+0000","last_clean":"2024-01-15T18:24:17.996721+0000","last_became_active":"2024-01-15T18:18:41.933804+0000","last_became_peered":"2024-01-15T18:18:41.933804+0000","last_unstale":"2024-01-15T18:24:17.996721+0000","last_undegraded":"2024-01-15T18:24:17.996721+0000","last_fullsized":"2024-01-15T18:24:17.996721+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'56","last_scrub_stamp":"2024-01-15T18:24:17.996618+0000","last_deep_scrub":"113'56","last_deep_scrub_stamp":"2024-01-15T18:24:17.996618+0000","last_clean_scrub_stamp":"2024-01-15T18:24:17.996618+0000","objects_scrubbed":18,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:41.514918+0000","scrub_duration":0.132082754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"113'73","reported_seq":116,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.623718+0000","last_change":"2024-01-15T18:24:08.623783+0000","last_active":"2024-01-15T18:24:08.623718+0000","last_peered":"2024-01-15T18:24:08.623718+0000","last_clean":"2024-01-15T18:24:08.623718+0000","last_became_active":"2024-01-15T18:19:41.881774+0000","last_became_peered":"2024-01-15T18:19:41.881774+0000","last_unstale":"2024-01-15T18:24:08.623718+0000","last_undegraded":"2024-01-15T18:24:08.623718+0000","last_fullsized":"2024-01-15T18:24:08.623718+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'73","last_scrub_stamp":"2024-01-15T18:24:08.623631+0000","last_deep_scrub":"113'73","last_deep_scrub_stamp":"2024-01-15T18:24:08.623631+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.623631+0000","objects_scrubbed":23,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:32:40.432035+0000","scrub_duration":0.13678316800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":54525952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"113'58","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:07.616751+0000","last_change":"2024-01-15T18:24:07.616784+0000","last_active":"2024-01-15T18:24:07.616751+0000","last_peered":"2024-01-15T18:24:07.616751+0000","last_clean":"2024-01-15T18:24:07.616751+0000","last_became_active":"2024-01-15T18:18:41.931429+0000","last_became_peered":"2024-01-15T18:18:41.931429+0000","last_unstale":"2024-01-15T18:24:07.616751+0000","last_undegraded":"2024-01-15T18:24:07.616751+0000","last_fullsized":"2024-01-15T18:24:07.616751+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'58","last_scrub_stamp":"2024-01-15T18:24:07.616679+0000","last_deep_scrub":"113'58","last_deep_scrub_stamp":"2024-01-15T18:24:07.616679+0000","last_clean_scrub_stamp":"2024-01-15T18:24:07.616679+0000","objects_scrubbed":21,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:59.993118+0000","scrub_duration":0.170067107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":28673,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"113'55","reported_seq":82,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:22.761552+0000","last_change":"2024-01-15T18:24:22.761587+0000","last_active":"2024-01-15T18:24:22.761552+0000","last_peered":"2024-01-15T18:24:22.761552+0000","last_clean":"2024-01-15T18:24:22.761552+0000","last_became_active":"2024-01-15T18:18:41.952738+0000","last_became_peered":"2024-01-15T18:18:41.952738+0000","last_unstale":"2024-01-15T18:24:22.761552+0000","last_undegraded":"2024-01-15T18:24:22.761552+0000","last_fullsized":"2024-01-15T18:24:22.761552+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'55","last_scrub_stamp":"2024-01-15T18:24:22.761479+0000","last_deep_scrub":"113'55","last_deep_scrub_stamp":"2024-01-15T18:24:22.761479+0000","last_clean_scrub_stamp":"2024-01-15T18:24:22.761479+0000","objects_scrubbed":17,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:43:33.628429+0000","scrub_duration":0.122328293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"113'64","reported_seq":98,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.964719+0000","last_change":"2024-01-15T18:24:10.964744+0000","last_active":"2024-01-15T18:24:10.964719+0000","last_peered":"2024-01-15T18:24:10.964719+0000","last_clean":"2024-01-15T18:24:10.964719+0000","last_became_active":"2024-01-15T18:18:41.930566+0000","last_became_peered":"2024-01-15T18:18:41.930566+0000","last_unstale":"2024-01-15T18:24:10.964719+0000","last_undegraded":"2024-01-15T18:24:10.964719+0000","last_fullsized":"2024-01-15T18:24:10.964719+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:10.964636+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:10.964636+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.964636+0000","objects_scrubbed":26,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:25.635165+0000","scrub_duration":0.19124144400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"113'54","reported_seq":87,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:00.920221+0000","last_change":"2024-01-15T18:24:00.920252+0000","last_active":"2024-01-15T18:24:00.920221+0000","last_peered":"2024-01-15T18:24:00.920221+0000","last_clean":"2024-01-15T18:24:00.920221+0000","last_became_active":"2024-01-15T18:18:41.937212+0000","last_became_peered":"2024-01-15T18:18:41.937212+0000","last_unstale":"2024-01-15T18:24:00.920221+0000","last_undegraded":"2024-01-15T18:24:00.920221+0000","last_fullsized":"2024-01-15T18:24:00.920221+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'54","last_scrub_stamp":"2024-01-15T18:24:00.920149+0000","last_deep_scrub":"113'54","last_deep_scrub_stamp":"2024-01-15T18:24:00.920149+0000","last_clean_scrub_stamp":"2024-01-15T18:24:00.920149+0000","objects_scrubbed":16,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:12:31.167250+0000","scrub_duration":0.10608548,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"113'88","reported_seq":125,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:20.670931+0000","last_change":"2024-01-15T18:24:20.670953+0000","last_active":"2024-01-15T18:24:20.670931+0000","last_peered":"2024-01-15T18:24:20.670931+0000","last_clean":"2024-01-15T18:24:20.670931+0000","last_became_active":"2024-01-15T18:18:41.941395+0000","last_became_peered":"2024-01-15T18:18:41.941395+0000","last_unstale":"2024-01-15T18:24:20.670931+0000","last_undegraded":"2024-01-15T18:24:20.670931+0000","last_fullsized":"2024-01-15T18:24:20.670931+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'88","last_scrub_stamp":"2024-01-15T18:24:20.670863+0000","last_deep_scrub":"113'88","last_deep_scrub_stamp":"2024-01-15T18:24:20.670863+0000","last_clean_scrub_stamp":"2024-01-15T18:24:20.670863+0000","objects_scrubbed":26,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:12.822752+0000","scrub_duration":0.15657138600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"113'72","reported_seq":103,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.607585+0000","last_change":"2024-01-15T18:24:01.607622+0000","last_active":"2024-01-15T18:24:01.607585+0000","last_peered":"2024-01-15T18:24:01.607585+0000","last_clean":"2024-01-15T18:24:01.607585+0000","last_became_active":"2024-01-15T18:18:41.951412+0000","last_became_peered":"2024-01-15T18:18:41.951412+0000","last_unstale":"2024-01-15T18:24:01.607585+0000","last_undegraded":"2024-01-15T18:24:01.607585+0000","last_fullsized":"2024-01-15T18:24:01.607585+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'72","last_scrub_stamp":"2024-01-15T18:24:01.607513+0000","last_deep_scrub":"113'72","last_deep_scrub_stamp":"2024-01-15T18:24:01.607513+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.607513+0000","objects_scrubbed":22,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:02.388269+0000","scrub_duration":0.15048887599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"113'49","reported_seq":80,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:26.577457+0000","last_change":"2024-01-15T18:24:26.577457+0000","last_active":"2024-01-15T18:24:26.577457+0000","last_peered":"2024-01-15T18:24:26.577457+0000","last_clean":"2024-01-15T18:24:26.577457+0000","last_became_active":"2024-01-15T18:18:41.939052+0000","last_became_peered":"2024-01-15T18:18:41.939052+0000","last_unstale":"2024-01-15T18:24:26.577457+0000","last_undegraded":"2024-01-15T18:24:26.577457+0000","last_fullsized":"2024-01-15T18:24:26.577457+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'49","last_scrub_stamp":"2024-01-15T18:24:26.577322+0000","last_deep_scrub":"113'49","last_deep_scrub_stamp":"2024-01-15T18:24:26.577322+0000","last_clean_scrub_stamp":"2024-01-15T18:24:26.577322+0000","objects_scrubbed":15,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:00.599189+0000","scrub_duration":0.109099811,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'6","reported_seq":31,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.867891+0000","last_change":"2024-01-15T18:23:57.867914+0000","last_active":"2024-01-15T18:23:57.867891+0000","last_peered":"2024-01-15T18:23:57.867891+0000","last_clean":"2024-01-15T18:23:57.867891+0000","last_became_active":"2024-01-15T18:18:40.923377+0000","last_became_peered":"2024-01-15T18:18:40.923377+0000","last_unstale":"2024-01-15T18:23:57.867891+0000","last_undegraded":"2024-01-15T18:23:57.867891+0000","last_fullsized":"2024-01-15T18:23:57.867891+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'6","last_scrub_stamp":"2024-01-15T18:23:57.867799+0000","last_deep_scrub":"113'6","last_deep_scrub_stamp":"2024-01-15T18:23:57.867799+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.867799+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:03:18.594537+0000","scrub_duration":0.0040798709999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"113'69","reported_seq":101,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:18.847983+0000","last_change":"2024-01-15T18:24:18.848023+0000","last_active":"2024-01-15T18:24:18.847983+0000","last_peered":"2024-01-15T18:24:18.847983+0000","last_clean":"2024-01-15T18:24:18.847983+0000","last_became_active":"2024-01-15T18:18:41.927480+0000","last_became_peered":"2024-01-15T18:18:41.927480+0000","last_unstale":"2024-01-15T18:24:18.847983+0000","last_undegraded":"2024-01-15T18:24:18.847983+0000","last_fullsized":"2024-01-15T18:24:18.847983+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'69","last_scrub_stamp":"2024-01-15T18:24:18.847888+0000","last_deep_scrub":"113'69","last_deep_scrub_stamp":"2024-01-15T18:24:18.847888+0000","last_clean_scrub_stamp":"2024-01-15T18:24:18.847888+0000","objects_scrubbed":22,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:48.006506+0000","scrub_duration":0.15792041500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"110'1","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.978990+0000","last_change":"2024-01-15T18:23:59.978990+0000","last_active":"2024-01-15T18:23:59.978990+0000","last_peered":"2024-01-15T18:23:59.978990+0000","last_clean":"2024-01-15T18:23:59.978990+0000","last_became_active":"2024-01-15T18:18:40.921341+0000","last_became_peered":"2024-01-15T18:18:40.921341+0000","last_unstale":"2024-01-15T18:23:59.978990+0000","last_undegraded":"2024-01-15T18:23:59.978990+0000","last_fullsized":"2024-01-15T18:23:59.978990+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:59.978827+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:59.978827+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.978827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:35.915671+0000","scrub_duration":0.0020127669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"113'62","reported_seq":97,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.582626+0000","last_change":"2024-01-15T18:24:25.582659+0000","last_active":"2024-01-15T18:24:25.582626+0000","last_peered":"2024-01-15T18:24:25.582626+0000","last_clean":"2024-01-15T18:24:25.582626+0000","last_became_active":"2024-01-15T18:18:41.943233+0000","last_became_peered":"2024-01-15T18:18:41.943233+0000","last_unstale":"2024-01-15T18:24:25.582626+0000","last_undegraded":"2024-01-15T18:24:25.582626+0000","last_fullsized":"2024-01-15T18:24:25.582626+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:24:25.582553+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:24:25.582553+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.582553+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:46:04.944469+0000","scrub_duration":0.12929829300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:51.632037+0000","last_change":"2024-01-15T18:23:51.632064+0000","last_active":"2024-01-15T18:23:51.632037+0000","last_peered":"2024-01-15T18:23:51.632037+0000","last_clean":"2024-01-15T18:23:51.632037+0000","last_became_active":"2024-01-15T18:18:40.941985+0000","last_became_peered":"2024-01-15T18:18:40.941985+0000","last_unstale":"2024-01-15T18:23:51.632037+0000","last_undegraded":"2024-01-15T18:23:51.632037+0000","last_fullsized":"2024-01-15T18:23:51.632037+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:51.631949+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:51.631949+0000","last_clean_scrub_stamp":"2024-01-15T18:23:51.631949+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:46:13.718266+0000","scrub_duration":0.003650155,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"113'79","reported_seq":114,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:20.252089+0000","last_change":"2024-01-15T18:24:20.252134+0000","last_active":"2024-01-15T18:24:20.252089+0000","last_peered":"2024-01-15T18:24:20.252089+0000","last_clean":"2024-01-15T18:24:20.252089+0000","last_became_active":"2024-01-15T18:18:41.947456+0000","last_became_peered":"2024-01-15T18:18:41.947456+0000","last_unstale":"2024-01-15T18:24:20.252089+0000","last_undegraded":"2024-01-15T18:24:20.252089+0000","last_fullsized":"2024-01-15T18:24:20.252089+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'79","last_scrub_stamp":"2024-01-15T18:24:20.252006+0000","last_deep_scrub":"113'79","last_deep_scrub_stamp":"2024-01-15T18:24:20.252006+0000","last_clean_scrub_stamp":"2024-01-15T18:24:20.252006+0000","objects_scrubbed":23,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:57.918070+0000","scrub_duration":0.144693926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:02.499567+0000","last_change":"2024-01-15T18:24:02.499587+0000","last_active":"2024-01-15T18:24:02.499567+0000","last_peered":"2024-01-15T18:24:02.499567+0000","last_clean":"2024-01-15T18:24:02.499567+0000","last_became_active":"2024-01-15T18:18:40.942267+0000","last_became_peered":"2024-01-15T18:18:40.942267+0000","last_unstale":"2024-01-15T18:24:02.499567+0000","last_undegraded":"2024-01-15T18:24:02.499567+0000","last_fullsized":"2024-01-15T18:24:02.499567+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:24:02.499486+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:24:02.499486+0000","last_clean_scrub_stamp":"2024-01-15T18:24:02.499486+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:57:00.198981+0000","scrub_duration":0.0029058650000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"113'53","reported_seq":85,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.535064+0000","last_change":"2024-01-15T18:23:57.535097+0000","last_active":"2024-01-15T18:23:57.535064+0000","last_peered":"2024-01-15T18:23:57.535064+0000","last_clean":"2024-01-15T18:23:57.535064+0000","last_became_active":"2024-01-15T18:18:41.945219+0000","last_became_peered":"2024-01-15T18:18:41.945219+0000","last_unstale":"2024-01-15T18:23:57.535064+0000","last_undegraded":"2024-01-15T18:23:57.535064+0000","last_fullsized":"2024-01-15T18:23:57.535064+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:23:57.534962+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:23:57.534962+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.534962+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:34.769272+0000","scrub_duration":0.11579978,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:55.839729+0000","last_change":"2024-01-15T18:23:55.839753+0000","last_active":"2024-01-15T18:23:55.839729+0000","last_peered":"2024-01-15T18:23:55.839729+0000","last_clean":"2024-01-15T18:23:55.839729+0000","last_became_active":"2024-01-15T18:18:40.931477+0000","last_became_peered":"2024-01-15T18:18:40.931477+0000","last_unstale":"2024-01-15T18:23:55.839729+0000","last_undegraded":"2024-01-15T18:23:55.839729+0000","last_fullsized":"2024-01-15T18:23:55.839729+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:55.839629+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:55.839629+0000","last_clean_scrub_stamp":"2024-01-15T18:23:55.839629+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:12.580092+0000","scrub_duration":0.0024064479999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"113'68","reported_seq":105,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:09.868471+0000","last_change":"2024-01-15T18:24:09.868505+0000","last_active":"2024-01-15T18:24:09.868471+0000","last_peered":"2024-01-15T18:24:09.868471+0000","last_clean":"2024-01-15T18:24:09.868471+0000","last_became_active":"2024-01-15T18:18:41.935603+0000","last_became_peered":"2024-01-15T18:18:41.935603+0000","last_unstale":"2024-01-15T18:24:09.868471+0000","last_undegraded":"2024-01-15T18:24:09.868471+0000","last_fullsized":"2024-01-15T18:24:09.868471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:09.868399+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:09.868399+0000","last_clean_scrub_stamp":"2024-01-15T18:24:09.868399+0000","objects_scrubbed":24,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:40.404894+0000","scrub_duration":0.139674142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"110'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:47.658985+0000","last_change":"2024-01-15T18:23:47.659004+0000","last_active":"2024-01-15T18:23:47.658985+0000","last_peered":"2024-01-15T18:23:47.658985+0000","last_clean":"2024-01-15T18:23:47.658985+0000","last_became_active":"2024-01-15T18:18:40.931887+0000","last_became_peered":"2024-01-15T18:18:40.931887+0000","last_unstale":"2024-01-15T18:23:47.658985+0000","last_undegraded":"2024-01-15T18:23:47.658985+0000","last_fullsized":"2024-01-15T18:23:47.658985+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'1","last_scrub_stamp":"2024-01-15T18:23:47.658903+0000","last_deep_scrub":"110'1","last_deep_scrub_stamp":"2024-01-15T18:23:47.658903+0000","last_clean_scrub_stamp":"2024-01-15T18:23:47.658903+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:12:17.906004+0000","scrub_duration":0.002056914,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"113'71","reported_seq":118,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:14.006760+0000","last_change":"2024-01-15T18:24:14.006760+0000","last_active":"2024-01-15T18:24:14.006760+0000","last_peered":"2024-01-15T18:24:14.006760+0000","last_clean":"2024-01-15T18:24:14.006760+0000","last_became_active":"2024-01-15T18:19:41.873080+0000","last_became_peered":"2024-01-15T18:19:41.873080+0000","last_unstale":"2024-01-15T18:24:14.006760+0000","last_undegraded":"2024-01-15T18:24:14.006760+0000","last_fullsized":"2024-01-15T18:24:14.006760+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'71","last_scrub_stamp":"2024-01-15T18:24:14.006641+0000","last_deep_scrub":"113'71","last_deep_scrub_stamp":"2024-01-15T18:24:14.006641+0000","last_clean_scrub_stamp":"2024-01-15T18:24:14.006641+0000","objects_scrubbed":26,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:27.709904+0000","scrub_duration":0.195203722,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857792,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":58720256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:55.766357+0000","last_change":"2024-01-15T18:23:55.766378+0000","last_active":"2024-01-15T18:23:55.766357+0000","last_peered":"2024-01-15T18:23:55.766357+0000","last_clean":"2024-01-15T18:23:55.766357+0000","last_became_active":"2024-01-15T18:18:40.920722+0000","last_became_peered":"2024-01-15T18:18:40.920722+0000","last_unstale":"2024-01-15T18:23:55.766357+0000","last_undegraded":"2024-01-15T18:23:55.766357+0000","last_fullsized":"2024-01-15T18:23:55.766357+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:55.766236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:55.766236+0000","last_clean_scrub_stamp":"2024-01-15T18:23:55.766236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:28.192190+0000","scrub_duration":0.0029978629999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"113'56","reported_seq":86,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:06.984054+0000","last_change":"2024-01-15T18:24:06.984094+0000","last_active":"2024-01-15T18:24:06.984054+0000","last_peered":"2024-01-15T18:24:06.984054+0000","last_clean":"2024-01-15T18:24:06.984054+0000","last_became_active":"2024-01-15T18:18:41.941646+0000","last_became_peered":"2024-01-15T18:18:41.941646+0000","last_unstale":"2024-01-15T18:24:06.984054+0000","last_undegraded":"2024-01-15T18:24:06.984054+0000","last_fullsized":"2024-01-15T18:24:06.984054+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'56","last_scrub_stamp":"2024-01-15T18:24:06.983972+0000","last_deep_scrub":"113'56","last_deep_scrub_stamp":"2024-01-15T18:24:06.983972+0000","last_clean_scrub_stamp":"2024-01-15T18:24:06.983972+0000","objects_scrubbed":16,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:55.181591+0000","scrub_duration":0.11652866200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.457759+0000","last_change":"2024-01-15T18:23:59.457780+0000","last_active":"2024-01-15T18:23:59.457759+0000","last_peered":"2024-01-15T18:23:59.457759+0000","last_clean":"2024-01-15T18:23:59.457759+0000","last_became_active":"2024-01-15T18:18:40.926495+0000","last_became_peered":"2024-01-15T18:18:40.926495+0000","last_unstale":"2024-01-15T18:23:59.457759+0000","last_undegraded":"2024-01-15T18:23:59.457759+0000","last_fullsized":"2024-01-15T18:23:59.457759+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.457652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.457652+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.457652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:47:36.928230+0000","scrub_duration":0.0021356639999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"113'49","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:12.825576+0000","last_change":"2024-01-15T18:24:12.825601+0000","last_active":"2024-01-15T18:24:12.825576+0000","last_peered":"2024-01-15T18:24:12.825576+0000","last_clean":"2024-01-15T18:24:12.825576+0000","last_became_active":"2024-01-15T18:18:41.930802+0000","last_became_peered":"2024-01-15T18:18:41.930802+0000","last_unstale":"2024-01-15T18:24:12.825576+0000","last_undegraded":"2024-01-15T18:24:12.825576+0000","last_fullsized":"2024-01-15T18:24:12.825576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'49","last_scrub_stamp":"2024-01-15T18:24:12.825497+0000","last_deep_scrub":"113'49","last_deep_scrub_stamp":"2024-01-15T18:24:12.825497+0000","last_clean_scrub_stamp":"2024-01-15T18:24:12.825497+0000","objects_scrubbed":8,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:06:51.969992+0000","scrub_duration":0.049176540999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"111'1","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.513102+0000","last_change":"2024-01-15T18:23:59.513138+0000","last_active":"2024-01-15T18:23:59.513102+0000","last_peered":"2024-01-15T18:23:59.513102+0000","last_clean":"2024-01-15T18:23:59.513102+0000","last_became_active":"2024-01-15T18:18:40.942342+0000","last_became_peered":"2024-01-15T18:18:40.942342+0000","last_unstale":"2024-01-15T18:23:59.513102+0000","last_undegraded":"2024-01-15T18:23:59.513102+0000","last_fullsized":"2024-01-15T18:23:59.513102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'1","last_scrub_stamp":"2024-01-15T18:23:59.513003+0000","last_deep_scrub":"111'1","last_deep_scrub_stamp":"2024-01-15T18:23:59.513003+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.513003+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:34:17.358881+0000","scrub_duration":0.001554124,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"113'53","reported_seq":82,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:12.721675+0000","last_change":"2024-01-15T18:24:12.721709+0000","last_active":"2024-01-15T18:24:12.721675+0000","last_peered":"2024-01-15T18:24:12.721675+0000","last_clean":"2024-01-15T18:24:12.721675+0000","last_became_active":"2024-01-15T18:18:41.924987+0000","last_became_peered":"2024-01-15T18:18:41.924987+0000","last_unstale":"2024-01-15T18:24:12.721675+0000","last_undegraded":"2024-01-15T18:24:12.721675+0000","last_fullsized":"2024-01-15T18:24:12.721675+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:24:12.721589+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:24:12.721589+0000","last_clean_scrub_stamp":"2024-01-15T18:24:12.721589+0000","objects_scrubbed":18,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:19.800682+0000","scrub_duration":0.117736047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.756143+0000","last_change":"2024-01-15T18:23:59.756175+0000","last_active":"2024-01-15T18:23:59.756143+0000","last_peered":"2024-01-15T18:23:59.756143+0000","last_clean":"2024-01-15T18:23:59.756143+0000","last_became_active":"2024-01-15T18:18:40.929383+0000","last_became_peered":"2024-01-15T18:18:40.929383+0000","last_unstale":"2024-01-15T18:23:59.756143+0000","last_undegraded":"2024-01-15T18:23:59.756143+0000","last_fullsized":"2024-01-15T18:23:59.756143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.756033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.756033+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.756033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:34.482493+0000","scrub_duration":0.0022605020000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"113'65","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:19.767777+0000","last_change":"2024-01-15T18:24:19.767819+0000","last_active":"2024-01-15T18:24:19.767777+0000","last_peered":"2024-01-15T18:24:19.767777+0000","last_clean":"2024-01-15T18:24:19.767777+0000","last_became_active":"2024-01-15T18:18:41.942749+0000","last_became_peered":"2024-01-15T18:18:41.942749+0000","last_unstale":"2024-01-15T18:24:19.767777+0000","last_undegraded":"2024-01-15T18:24:19.767777+0000","last_fullsized":"2024-01-15T18:24:19.767777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'65","last_scrub_stamp":"2024-01-15T18:24:19.767697+0000","last_deep_scrub":"113'65","last_deep_scrub_stamp":"2024-01-15T18:24:19.767697+0000","last_clean_scrub_stamp":"2024-01-15T18:24:19.767697+0000","objects_scrubbed":22,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:43:28.256474+0000","scrub_duration":0.14783215899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'13","reported_seq":38,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:09.423971+0000","last_change":"2024-01-15T18:24:09.423992+0000","last_active":"2024-01-15T18:24:09.423971+0000","last_peered":"2024-01-15T18:24:09.423971+0000","last_clean":"2024-01-15T18:24:09.423971+0000","last_became_active":"2024-01-15T18:18:40.936384+0000","last_became_peered":"2024-01-15T18:18:40.936384+0000","last_unstale":"2024-01-15T18:24:09.423971+0000","last_undegraded":"2024-01-15T18:24:09.423971+0000","last_fullsized":"2024-01-15T18:24:09.423971+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'13","last_scrub_stamp":"2024-01-15T18:24:09.423875+0000","last_deep_scrub":"113'13","last_deep_scrub_stamp":"2024-01-15T18:24:09.423875+0000","last_clean_scrub_stamp":"2024-01-15T18:24:09.423875+0000","objects_scrubbed":2,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:23.481555+0000","scrub_duration":0.0038319389999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"113'60","reported_seq":92,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:05.973582+0000","last_change":"2024-01-15T18:24:05.973644+0000","last_active":"2024-01-15T18:24:05.973582+0000","last_peered":"2024-01-15T18:24:05.973582+0000","last_clean":"2024-01-15T18:24:05.973582+0000","last_became_active":"2024-01-15T18:18:41.931286+0000","last_became_peered":"2024-01-15T18:18:41.931286+0000","last_unstale":"2024-01-15T18:24:05.973582+0000","last_undegraded":"2024-01-15T18:24:05.973582+0000","last_fullsized":"2024-01-15T18:24:05.973582+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:05.973484+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:05.973484+0000","last_clean_scrub_stamp":"2024-01-15T18:24:05.973484+0000","objects_scrubbed":20,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:18:40.025089+0000","scrub_duration":0.14927474800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'191","reported_seq":216,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:39.944451+0000","last_change":"2024-01-15T18:24:08.457943+0000","last_active":"2024-01-15T18:24:39.944451+0000","last_peered":"2024-01-15T18:24:39.944451+0000","last_clean":"2024-01-15T18:24:39.944451+0000","last_became_active":"2024-01-15T18:18:40.927108+0000","last_became_peered":"2024-01-15T18:18:40.927108+0000","last_unstale":"2024-01-15T18:24:39.944451+0000","last_undegraded":"2024-01-15T18:24:39.944451+0000","last_fullsized":"2024-01-15T18:24:39.944451+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'189","last_scrub_stamp":"2024-01-15T18:24:08.457835+0000","last_deep_scrub":"113'189","last_deep_scrub_stamp":"2024-01-15T18:24:08.457835+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.457835+0000","objects_scrubbed":4,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:00:50.921849+0000","scrub_duration":0.0067328329999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":616415,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":188,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"113'64","reported_seq":94,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:15.755020+0000","last_change":"2024-01-15T18:24:15.755054+0000","last_active":"2024-01-15T18:24:15.755020+0000","last_peered":"2024-01-15T18:24:15.755020+0000","last_clean":"2024-01-15T18:24:15.755020+0000","last_became_active":"2024-01-15T18:18:41.941602+0000","last_became_peered":"2024-01-15T18:18:41.941602+0000","last_unstale":"2024-01-15T18:24:15.755020+0000","last_undegraded":"2024-01-15T18:24:15.755020+0000","last_fullsized":"2024-01-15T18:24:15.755020+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'64","last_scrub_stamp":"2024-01-15T18:24:15.754943+0000","last_deep_scrub":"113'64","last_deep_scrub_stamp":"2024-01-15T18:24:15.754943+0000","last_clean_scrub_stamp":"2024-01-15T18:24:15.754943+0000","objects_scrubbed":21,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:33.993030+0000","scrub_duration":0.13024677200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:57.459728+0000","last_change":"2024-01-15T18:23:57.459775+0000","last_active":"2024-01-15T18:23:57.459728+0000","last_peered":"2024-01-15T18:23:57.459728+0000","last_clean":"2024-01-15T18:23:57.459728+0000","last_became_active":"2024-01-15T18:18:40.936376+0000","last_became_peered":"2024-01-15T18:18:40.936376+0000","last_unstale":"2024-01-15T18:23:57.459728+0000","last_undegraded":"2024-01-15T18:23:57.459728+0000","last_fullsized":"2024-01-15T18:23:57.459728+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:57.459625+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:57.459625+0000","last_clean_scrub_stamp":"2024-01-15T18:23:57.459625+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:51.586938+0000","scrub_duration":0.0053587779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"113'73","reported_seq":102,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:30.849668+0000","last_change":"2024-01-15T18:24:30.849707+0000","last_active":"2024-01-15T18:24:30.849668+0000","last_peered":"2024-01-15T18:24:30.849668+0000","last_clean":"2024-01-15T18:24:30.849668+0000","last_became_active":"2024-01-15T18:18:41.941884+0000","last_became_peered":"2024-01-15T18:18:41.941884+0000","last_unstale":"2024-01-15T18:24:30.849668+0000","last_undegraded":"2024-01-15T18:24:30.849668+0000","last_fullsized":"2024-01-15T18:24:30.849668+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'73","last_scrub_stamp":"2024-01-15T18:24:30.849590+0000","last_deep_scrub":"113'73","last_deep_scrub_stamp":"2024-01-15T18:24:30.849590+0000","last_clean_scrub_stamp":"2024-01-15T18:24:30.849590+0000","objects_scrubbed":21,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:54.367890+0000","scrub_duration":0.15187130600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:54.393613+0000","last_change":"2024-01-15T18:23:54.393632+0000","last_active":"2024-01-15T18:23:54.393613+0000","last_peered":"2024-01-15T18:23:54.393613+0000","last_clean":"2024-01-15T18:23:54.393613+0000","last_became_active":"2024-01-15T18:18:40.913755+0000","last_became_peered":"2024-01-15T18:18:40.913755+0000","last_unstale":"2024-01-15T18:23:54.393613+0000","last_undegraded":"2024-01-15T18:23:54.393613+0000","last_fullsized":"2024-01-15T18:23:54.393613+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:54.393495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:54.393495+0000","last_clean_scrub_stamp":"2024-01-15T18:23:54.393495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:48:38.527529+0000","scrub_duration":0.0024808400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"113'68","reported_seq":100,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.818049+0000","last_change":"2024-01-15T18:24:10.818070+0000","last_active":"2024-01-15T18:24:10.818049+0000","last_peered":"2024-01-15T18:24:10.818049+0000","last_clean":"2024-01-15T18:24:10.818049+0000","last_became_active":"2024-01-15T18:18:41.943477+0000","last_became_peered":"2024-01-15T18:18:41.943477+0000","last_unstale":"2024-01-15T18:24:10.818049+0000","last_undegraded":"2024-01-15T18:24:10.818049+0000","last_fullsized":"2024-01-15T18:24:10.818049+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'68","last_scrub_stamp":"2024-01-15T18:24:10.817981+0000","last_deep_scrub":"113'68","last_deep_scrub_stamp":"2024-01-15T18:24:10.817981+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.817981+0000","objects_scrubbed":25,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:43:24.521244+0000","scrub_duration":0.14952846,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663557,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'12","reported_seq":37,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.559825+0000","last_change":"2024-01-15T18:23:58.559858+0000","last_active":"2024-01-15T18:23:58.559825+0000","last_peered":"2024-01-15T18:23:58.559825+0000","last_clean":"2024-01-15T18:23:58.559825+0000","last_became_active":"2024-01-15T18:18:40.933678+0000","last_became_peered":"2024-01-15T18:18:40.933678+0000","last_unstale":"2024-01-15T18:23:58.559825+0000","last_undegraded":"2024-01-15T18:23:58.559825+0000","last_fullsized":"2024-01-15T18:23:58.559825+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'12","last_scrub_stamp":"2024-01-15T18:23:58.559722+0000","last_deep_scrub":"113'12","last_deep_scrub_stamp":"2024-01-15T18:23:58.559722+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.559722+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:38.037968+0000","scrub_duration":0.0054070120000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"113'31","reported_seq":59,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:07.559609+0000","last_change":"2024-01-15T18:24:07.559659+0000","last_active":"2024-01-15T18:24:07.559609+0000","last_peered":"2024-01-15T18:24:07.559609+0000","last_clean":"2024-01-15T18:24:07.559609+0000","last_became_active":"2024-01-15T18:18:41.933777+0000","last_became_peered":"2024-01-15T18:18:41.933777+0000","last_unstale":"2024-01-15T18:24:07.559609+0000","last_undegraded":"2024-01-15T18:24:07.559609+0000","last_fullsized":"2024-01-15T18:24:07.559609+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'31","last_scrub_stamp":"2024-01-15T18:24:07.559498+0000","last_deep_scrub":"113'31","last_deep_scrub_stamp":"2024-01-15T18:24:07.559498+0000","last_clean_scrub_stamp":"2024-01-15T18:24:07.559498+0000","objects_scrubbed":10,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:56:33.811882+0000","scrub_duration":0.090809666999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'2","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:11.491658+0000","last_change":"2024-01-15T18:24:11.491681+0000","last_active":"2024-01-15T18:24:11.491658+0000","last_peered":"2024-01-15T18:24:11.491658+0000","last_clean":"2024-01-15T18:24:11.491658+0000","last_became_active":"2024-01-15T18:18:40.932274+0000","last_became_peered":"2024-01-15T18:18:40.932274+0000","last_unstale":"2024-01-15T18:24:11.491658+0000","last_undegraded":"2024-01-15T18:24:11.491658+0000","last_fullsized":"2024-01-15T18:24:11.491658+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-01-15T18:24:11.491563+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-01-15T18:24:11.491563+0000","last_clean_scrub_stamp":"2024-01-15T18:24:11.491563+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:12:58.961260+0000","scrub_duration":0.004071089,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"113'53","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:23.558948+0000","last_change":"2024-01-15T18:24:23.558977+0000","last_active":"2024-01-15T18:24:23.558948+0000","last_peered":"2024-01-15T18:24:23.558948+0000","last_clean":"2024-01-15T18:24:23.558948+0000","last_became_active":"2024-01-15T18:18:41.931114+0000","last_became_peered":"2024-01-15T18:18:41.931114+0000","last_unstale":"2024-01-15T18:24:23.558948+0000","last_undegraded":"2024-01-15T18:24:23.558948+0000","last_fullsized":"2024-01-15T18:24:23.558948+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'53","last_scrub_stamp":"2024-01-15T18:24:23.558876+0000","last_deep_scrub":"113'53","last_deep_scrub_stamp":"2024-01-15T18:24:23.558876+0000","last_clean_scrub_stamp":"2024-01-15T18:24:23.558876+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:25.925401+0000","scrub_duration":0.123621357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.632496+0000","last_change":"2024-01-15T18:23:48.632517+0000","last_active":"2024-01-15T18:23:48.632496+0000","last_peered":"2024-01-15T18:23:48.632496+0000","last_clean":"2024-01-15T18:23:48.632496+0000","last_became_active":"2024-01-15T18:18:40.929719+0000","last_became_peered":"2024-01-15T18:18:40.929719+0000","last_unstale":"2024-01-15T18:23:48.632496+0000","last_undegraded":"2024-01-15T18:23:48.632496+0000","last_fullsized":"2024-01-15T18:23:48.632496+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:23:48.632420+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:23:48.632420+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.632420+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:03.214903+0000","scrub_duration":0.0038853939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"113'62","reported_seq":97,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:30.261471+0000","last_change":"2024-01-15T18:24:30.261509+0000","last_active":"2024-01-15T18:24:30.261471+0000","last_peered":"2024-01-15T18:24:30.261471+0000","last_clean":"2024-01-15T18:24:30.261471+0000","last_became_active":"2024-01-15T18:18:41.939452+0000","last_became_peered":"2024-01-15T18:18:41.939452+0000","last_unstale":"2024-01-15T18:24:30.261471+0000","last_undegraded":"2024-01-15T18:24:30.261471+0000","last_fullsized":"2024-01-15T18:24:30.261471+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:24:30.261389+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:24:30.261389+0000","last_clean_scrub_stamp":"2024-01-15T18:24:30.261389+0000","objects_scrubbed":18,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:59:00.250903+0000","scrub_duration":0.106360553,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'2","reported_seq":27,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:04.477880+0000","last_change":"2024-01-15T18:24:04.477901+0000","last_active":"2024-01-15T18:24:04.477880+0000","last_peered":"2024-01-15T18:24:04.477880+0000","last_clean":"2024-01-15T18:24:04.477880+0000","last_became_active":"2024-01-15T18:18:40.916522+0000","last_became_peered":"2024-01-15T18:18:40.916522+0000","last_unstale":"2024-01-15T18:24:04.477880+0000","last_undegraded":"2024-01-15T18:24:04.477880+0000","last_fullsized":"2024-01-15T18:24:04.477880+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-01-15T18:24:04.477798+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-01-15T18:24:04.477798+0000","last_clean_scrub_stamp":"2024-01-15T18:24:04.477798+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:31.016306+0000","scrub_duration":0.0041278030000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"113'63","reported_seq":99,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:26.250983+0000","last_change":"2024-01-15T18:24:26.251023+0000","last_active":"2024-01-15T18:24:26.250983+0000","last_peered":"2024-01-15T18:24:26.250983+0000","last_clean":"2024-01-15T18:24:26.250983+0000","last_became_active":"2024-01-15T18:18:41.932860+0000","last_became_peered":"2024-01-15T18:18:41.932860+0000","last_unstale":"2024-01-15T18:24:26.250983+0000","last_undegraded":"2024-01-15T18:24:26.250983+0000","last_fullsized":"2024-01-15T18:24:26.250983+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:24:26.250910+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:24:26.250910+0000","last_clean_scrub_stamp":"2024-01-15T18:24:26.250910+0000","objects_scrubbed":22,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:16:32.094041+0000","scrub_duration":0.15294004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.819756+0000","last_change":"2024-01-15T18:23:58.819772+0000","last_active":"2024-01-15T18:23:58.819756+0000","last_peered":"2024-01-15T18:23:58.819756+0000","last_clean":"2024-01-15T18:23:58.819756+0000","last_became_active":"2024-01-15T18:18:40.931142+0000","last_became_peered":"2024-01-15T18:18:40.931142+0000","last_unstale":"2024-01-15T18:23:58.819756+0000","last_undegraded":"2024-01-15T18:23:58.819756+0000","last_fullsized":"2024-01-15T18:23:58.819756+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:58.819676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:58.819676+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.819676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:01.186201+0000","scrub_duration":0.0020202330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"113'74","reported_seq":107,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:16.189204+0000","last_change":"2024-01-15T18:24:16.189230+0000","last_active":"2024-01-15T18:24:16.189204+0000","last_peered":"2024-01-15T18:24:16.189204+0000","last_clean":"2024-01-15T18:24:16.189204+0000","last_became_active":"2024-01-15T18:18:41.929196+0000","last_became_peered":"2024-01-15T18:18:41.929196+0000","last_unstale":"2024-01-15T18:24:16.189204+0000","last_undegraded":"2024-01-15T18:24:16.189204+0000","last_fullsized":"2024-01-15T18:24:16.189204+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'74","last_scrub_stamp":"2024-01-15T18:24:16.189126+0000","last_deep_scrub":"113'74","last_deep_scrub_stamp":"2024-01-15T18:24:16.189126+0000","last_clean_scrub_stamp":"2024-01-15T18:24:16.189126+0000","objects_scrubbed":30,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:23:20.823376+0000","scrub_duration":0.21666178899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'4","reported_seq":29,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:50.448326+0000","last_change":"2024-01-15T18:23:50.448372+0000","last_active":"2024-01-15T18:23:50.448326+0000","last_peered":"2024-01-15T18:23:50.448326+0000","last_clean":"2024-01-15T18:23:50.448326+0000","last_became_active":"2024-01-15T18:18:40.939817+0000","last_became_peered":"2024-01-15T18:18:40.939817+0000","last_unstale":"2024-01-15T18:23:50.448326+0000","last_undegraded":"2024-01-15T18:23:50.448326+0000","last_fullsized":"2024-01-15T18:23:50.448326+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'4","last_scrub_stamp":"2024-01-15T18:23:50.448207+0000","last_deep_scrub":"113'4","last_deep_scrub_stamp":"2024-01-15T18:23:50.448207+0000","last_clean_scrub_stamp":"2024-01-15T18:23:50.448207+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:10:29.926453+0000","scrub_duration":0.0039608999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"113'62","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.710805+0000","last_change":"2024-01-15T18:24:25.710836+0000","last_active":"2024-01-15T18:24:25.710805+0000","last_peered":"2024-01-15T18:24:25.710805+0000","last_clean":"2024-01-15T18:24:25.710805+0000","last_became_active":"2024-01-15T18:18:41.930869+0000","last_became_peered":"2024-01-15T18:18:41.930869+0000","last_unstale":"2024-01-15T18:24:25.710805+0000","last_undegraded":"2024-01-15T18:24:25.710805+0000","last_fullsized":"2024-01-15T18:24:25.710805+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'62","last_scrub_stamp":"2024-01-15T18:24:25.710709+0000","last_deep_scrub":"113'62","last_deep_scrub_stamp":"2024-01-15T18:24:25.710709+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.710709+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:46.381145+0000","scrub_duration":0.114989034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:52.641865+0000","last_change":"2024-01-15T18:23:52.641887+0000","last_active":"2024-01-15T18:23:52.641865+0000","last_peered":"2024-01-15T18:23:52.641865+0000","last_clean":"2024-01-15T18:23:52.641865+0000","last_became_active":"2024-01-15T18:18:40.945313+0000","last_became_peered":"2024-01-15T18:18:40.945313+0000","last_unstale":"2024-01-15T18:23:52.641865+0000","last_undegraded":"2024-01-15T18:23:52.641865+0000","last_fullsized":"2024-01-15T18:23:52.641865+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:52.641753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:52.641753+0000","last_clean_scrub_stamp":"2024-01-15T18:23:52.641753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:20:47.869635+0000","scrub_duration":0.0018539540000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"113'61","reported_seq":90,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.238170+0000","last_change":"2024-01-15T18:24:25.238202+0000","last_active":"2024-01-15T18:24:25.238170+0000","last_peered":"2024-01-15T18:24:25.238170+0000","last_clean":"2024-01-15T18:24:25.238170+0000","last_became_active":"2024-01-15T18:18:41.949990+0000","last_became_peered":"2024-01-15T18:18:41.949990+0000","last_unstale":"2024-01-15T18:24:25.238170+0000","last_undegraded":"2024-01-15T18:24:25.238170+0000","last_fullsized":"2024-01-15T18:24:25.238170+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'61","last_scrub_stamp":"2024-01-15T18:24:25.238094+0000","last_deep_scrub":"113'61","last_deep_scrub_stamp":"2024-01-15T18:24:25.238094+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.238094+0000","objects_scrubbed":17,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:01:07.702107+0000","scrub_duration":0.105860971,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:59.472252+0000","last_change":"2024-01-15T18:23:59.472271+0000","last_active":"2024-01-15T18:23:59.472252+0000","last_peered":"2024-01-15T18:23:59.472252+0000","last_clean":"2024-01-15T18:23:59.472252+0000","last_became_active":"2024-01-15T18:18:40.944390+0000","last_became_peered":"2024-01-15T18:18:40.944390+0000","last_unstale":"2024-01-15T18:23:59.472252+0000","last_undegraded":"2024-01-15T18:23:59.472252+0000","last_fullsized":"2024-01-15T18:23:59.472252+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:59.472157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:59.472157+0000","last_clean_scrub_stamp":"2024-01-15T18:23:59.472157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:16:00.507773+0000","scrub_duration":0.00163682,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"113'57","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:08.758838+0000","last_change":"2024-01-15T18:24:08.758873+0000","last_active":"2024-01-15T18:24:08.758838+0000","last_peered":"2024-01-15T18:24:08.758838+0000","last_clean":"2024-01-15T18:24:08.758838+0000","last_became_active":"2024-01-15T18:18:41.936142+0000","last_became_peered":"2024-01-15T18:18:41.936142+0000","last_unstale":"2024-01-15T18:24:08.758838+0000","last_undegraded":"2024-01-15T18:24:08.758838+0000","last_fullsized":"2024-01-15T18:24:08.758838+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'57","last_scrub_stamp":"2024-01-15T18:24:08.758766+0000","last_deep_scrub":"113'57","last_deep_scrub_stamp":"2024-01-15T18:24:08.758766+0000","last_clean_scrub_stamp":"2024-01-15T18:24:08.758766+0000","objects_scrubbed":17,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:11.665000+0000","scrub_duration":0.120732729,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109125,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.426929+0000","last_change":"2024-01-15T18:24:01.426948+0000","last_active":"2024-01-15T18:24:01.426929+0000","last_peered":"2024-01-15T18:24:01.426929+0000","last_clean":"2024-01-15T18:24:01.426929+0000","last_became_active":"2024-01-15T18:18:40.933059+0000","last_became_peered":"2024-01-15T18:18:40.933059+0000","last_unstale":"2024-01-15T18:24:01.426929+0000","last_undegraded":"2024-01-15T18:24:01.426929+0000","last_fullsized":"2024-01-15T18:24:01.426929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:01.426834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:01.426834+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.426834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:48:39.092898+0000","scrub_duration":0.0023004760000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"113'46","reported_seq":74,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.614302+0000","last_change":"2024-01-15T18:24:10.614346+0000","last_active":"2024-01-15T18:24:10.614302+0000","last_peered":"2024-01-15T18:24:10.614302+0000","last_clean":"2024-01-15T18:24:10.614302+0000","last_became_active":"2024-01-15T18:18:41.927443+0000","last_became_peered":"2024-01-15T18:18:41.927443+0000","last_unstale":"2024-01-15T18:24:10.614302+0000","last_undegraded":"2024-01-15T18:24:10.614302+0000","last_fullsized":"2024-01-15T18:24:10.614302+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'46","last_scrub_stamp":"2024-01-15T18:24:10.614210+0000","last_deep_scrub":"113'46","last_deep_scrub_stamp":"2024-01-15T18:24:10.614210+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.614210+0000","objects_scrubbed":13,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:45.340669+0000","scrub_duration":0.080420363999999994,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'3","reported_seq":28,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:01.755286+0000","last_change":"2024-01-15T18:24:01.755309+0000","last_active":"2024-01-15T18:24:01.755286+0000","last_peered":"2024-01-15T18:24:01.755286+0000","last_clean":"2024-01-15T18:24:01.755286+0000","last_became_active":"2024-01-15T18:18:40.938679+0000","last_became_peered":"2024-01-15T18:18:40.938679+0000","last_unstale":"2024-01-15T18:24:01.755286+0000","last_undegraded":"2024-01-15T18:24:01.755286+0000","last_fullsized":"2024-01-15T18:24:01.755286+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'3","last_scrub_stamp":"2024-01-15T18:24:01.755211+0000","last_deep_scrub":"113'3","last_deep_scrub_stamp":"2024-01-15T18:24:01.755211+0000","last_clean_scrub_stamp":"2024-01-15T18:24:01.755211+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:34.181165+0000","scrub_duration":0.002739652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"113'63","reported_seq":99,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:32.794409+0000","last_change":"2024-01-15T18:24:32.794442+0000","last_active":"2024-01-15T18:24:32.794409+0000","last_peered":"2024-01-15T18:24:32.794409+0000","last_clean":"2024-01-15T18:24:32.794409+0000","last_became_active":"2024-01-15T18:18:41.935248+0000","last_became_peered":"2024-01-15T18:18:41.935248+0000","last_unstale":"2024-01-15T18:24:32.794409+0000","last_undegraded":"2024-01-15T18:24:32.794409+0000","last_fullsized":"2024-01-15T18:24:32.794409+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'63","last_scrub_stamp":"2024-01-15T18:24:32.794329+0000","last_deep_scrub":"113'63","last_deep_scrub_stamp":"2024-01-15T18:24:32.794329+0000","last_clean_scrub_stamp":"2024-01-15T18:24:32.794329+0000","objects_scrubbed":22,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:34:29.544318+0000","scrub_duration":0.134109532,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"113'18","reported_seq":43,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:07.473108+0000","last_change":"2024-01-15T18:24:07.473129+0000","last_active":"2024-01-15T18:24:07.473108+0000","last_peered":"2024-01-15T18:24:07.473108+0000","last_clean":"2024-01-15T18:24:07.473108+0000","last_became_active":"2024-01-15T18:18:40.921839+0000","last_became_peered":"2024-01-15T18:18:40.921839+0000","last_unstale":"2024-01-15T18:24:07.473108+0000","last_undegraded":"2024-01-15T18:24:07.473108+0000","last_fullsized":"2024-01-15T18:24:07.473108+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'18","last_scrub_stamp":"2024-01-15T18:24:07.473023+0000","last_deep_scrub":"113'18","last_deep_scrub_stamp":"2024-01-15T18:24:07.473023+0000","last_clean_scrub_stamp":"2024-01-15T18:24:07.473023+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:00:30.825105+0000","scrub_duration":0.0022104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":506,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"113'70","reported_seq":108,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:28.766355+0000","last_change":"2024-01-15T18:24:28.766403+0000","last_active":"2024-01-15T18:24:28.766355+0000","last_peered":"2024-01-15T18:24:28.766355+0000","last_clean":"2024-01-15T18:24:28.766355+0000","last_became_active":"2024-01-15T18:18:41.935025+0000","last_became_peered":"2024-01-15T18:18:41.935025+0000","last_unstale":"2024-01-15T18:24:28.766355+0000","last_undegraded":"2024-01-15T18:24:28.766355+0000","last_fullsized":"2024-01-15T18:24:28.766355+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'70","last_scrub_stamp":"2024-01-15T18:24:28.766273+0000","last_deep_scrub":"113'70","last_deep_scrub_stamp":"2024-01-15T18:24:28.766273+0000","last_clean_scrub_stamp":"2024-01-15T18:24:28.766273+0000","objects_scrubbed":24,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.033773+0000","scrub_duration":0.17895370999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:13.526357+0000","last_change":"2024-01-15T18:24:13.526376+0000","last_active":"2024-01-15T18:24:13.526357+0000","last_peered":"2024-01-15T18:24:13.526357+0000","last_clean":"2024-01-15T18:24:13.526357+0000","last_became_active":"2024-01-15T18:18:40.934360+0000","last_became_peered":"2024-01-15T18:18:40.934360+0000","last_unstale":"2024-01-15T18:24:13.526357+0000","last_undegraded":"2024-01-15T18:24:13.526357+0000","last_fullsized":"2024-01-15T18:24:13.526357+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:13.526263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:13.526263+0000","last_clean_scrub_stamp":"2024-01-15T18:24:13.526263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:30:10.984389+0000","scrub_duration":0.0022385819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"113'58","reported_seq":88,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:25.855949+0000","last_change":"2024-01-15T18:24:25.855988+0000","last_active":"2024-01-15T18:24:25.855949+0000","last_peered":"2024-01-15T18:24:25.855949+0000","last_clean":"2024-01-15T18:24:25.855949+0000","last_became_active":"2024-01-15T18:18:41.928443+0000","last_became_peered":"2024-01-15T18:18:41.928443+0000","last_unstale":"2024-01-15T18:24:25.855949+0000","last_undegraded":"2024-01-15T18:24:25.855949+0000","last_fullsized":"2024-01-15T18:24:25.855949+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'58","last_scrub_stamp":"2024-01-15T18:24:25.855865+0000","last_deep_scrub":"113'58","last_deep_scrub_stamp":"2024-01-15T18:24:25.855865+0000","last_clean_scrub_stamp":"2024-01-15T18:24:25.855865+0000","objects_scrubbed":19,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:26.822867+0000","scrub_duration":0.11873668900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:58.770011+0000","last_change":"2024-01-15T18:23:58.770042+0000","last_active":"2024-01-15T18:23:58.770011+0000","last_peered":"2024-01-15T18:23:58.770011+0000","last_clean":"2024-01-15T18:23:58.770011+0000","last_became_active":"2024-01-15T18:18:40.939534+0000","last_became_peered":"2024-01-15T18:18:40.939534+0000","last_unstale":"2024-01-15T18:23:58.770011+0000","last_undegraded":"2024-01-15T18:23:58.770011+0000","last_fullsized":"2024-01-15T18:23:58.770011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:58.769909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:58.769909+0000","last_clean_scrub_stamp":"2024-01-15T18:23:58.769909+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:46.967528+0000","scrub_duration":0.0018566699999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"113'60","reported_seq":91,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:13.978996+0000","last_change":"2024-01-15T18:24:13.979039+0000","last_active":"2024-01-15T18:24:13.978996+0000","last_peered":"2024-01-15T18:24:13.978996+0000","last_clean":"2024-01-15T18:24:13.978996+0000","last_became_active":"2024-01-15T18:18:41.936866+0000","last_became_peered":"2024-01-15T18:18:41.936866+0000","last_unstale":"2024-01-15T18:24:13.978996+0000","last_undegraded":"2024-01-15T18:24:13.978996+0000","last_fullsized":"2024-01-15T18:24:13.978996+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'60","last_scrub_stamp":"2024-01-15T18:24:13.978917+0000","last_deep_scrub":"113'60","last_deep_scrub_stamp":"2024-01-15T18:24:13.978917+0000","last_clean_scrub_stamp":"2024-01-15T18:24:13.978917+0000","objects_scrubbed":16,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:41:23.967366+0000","scrub_duration":0.13694841599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:49.798222+0000","last_change":"2024-01-15T18:23:49.798239+0000","last_active":"2024-01-15T18:23:49.798222+0000","last_peered":"2024-01-15T18:23:49.798222+0000","last_clean":"2024-01-15T18:23:49.798222+0000","last_became_active":"2024-01-15T18:18:40.923927+0000","last_became_peered":"2024-01-15T18:18:40.923927+0000","last_unstale":"2024-01-15T18:23:49.798222+0000","last_undegraded":"2024-01-15T18:23:49.798222+0000","last_fullsized":"2024-01-15T18:23:49.798222+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:49.798143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:49.798143+0000","last_clean_scrub_stamp":"2024-01-15T18:23:49.798143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:43.688382+0000","scrub_duration":0.0014863820000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"113'80","reported_seq":113,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:03.884188+0000","last_change":"2024-01-15T18:24:03.884213+0000","last_active":"2024-01-15T18:24:03.884188+0000","last_peered":"2024-01-15T18:24:03.884188+0000","last_clean":"2024-01-15T18:24:03.884188+0000","last_became_active":"2024-01-15T18:18:41.935255+0000","last_became_peered":"2024-01-15T18:18:41.935255+0000","last_unstale":"2024-01-15T18:24:03.884188+0000","last_undegraded":"2024-01-15T18:24:03.884188+0000","last_fullsized":"2024-01-15T18:24:03.884188+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'80","last_scrub_stamp":"2024-01-15T18:24:03.884114+0000","last_deep_scrub":"113'80","last_deep_scrub_stamp":"2024-01-15T18:24:03.884114+0000","last_clean_scrub_stamp":"2024-01-15T18:24:03.884114+0000","objects_scrubbed":29,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:17:58.010620+0000","scrub_duration":0.17135824699999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.761653+0000","last_change":"2024-01-15T18:23:48.761672+0000","last_active":"2024-01-15T18:23:48.761653+0000","last_peered":"2024-01-15T18:23:48.761653+0000","last_clean":"2024-01-15T18:23:48.761653+0000","last_became_active":"2024-01-15T18:18:40.935622+0000","last_became_peered":"2024-01-15T18:18:40.935622+0000","last_unstale":"2024-01-15T18:23:48.761653+0000","last_undegraded":"2024-01-15T18:23:48.761653+0000","last_fullsized":"2024-01-15T18:23:48.761653+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:48.761549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:48.761549+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.761549+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:34:23.488009+0000","scrub_duration":0.0019591489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"113'72","reported_seq":106,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:29.298000+0000","last_change":"2024-01-15T18:24:29.298000+0000","last_active":"2024-01-15T18:24:29.298000+0000","last_peered":"2024-01-15T18:24:29.298000+0000","last_clean":"2024-01-15T18:24:29.298000+0000","last_became_active":"2024-01-15T18:18:41.942402+0000","last_became_peered":"2024-01-15T18:18:41.942402+0000","last_unstale":"2024-01-15T18:24:29.298000+0000","last_undegraded":"2024-01-15T18:24:29.298000+0000","last_fullsized":"2024-01-15T18:24:29.298000+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'72","last_scrub_stamp":"2024-01-15T18:24:29.297891+0000","last_deep_scrub":"113'72","last_deep_scrub_stamp":"2024-01-15T18:24:29.297891+0000","last_clean_scrub_stamp":"2024-01-15T18:24:29.297891+0000","objects_scrubbed":26,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:40.830744+0000","scrub_duration":0.17798181900000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"113'11","reported_seq":45,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:53.596663+0000","last_change":"2024-01-15T18:23:53.596683+0000","last_active":"2024-01-15T18:23:53.596663+0000","last_peered":"2024-01-15T18:23:53.596663+0000","last_clean":"2024-01-15T18:23:53.596663+0000","last_became_active":"2024-01-15T18:18:40.942306+0000","last_became_peered":"2024-01-15T18:18:40.942306+0000","last_unstale":"2024-01-15T18:23:53.596663+0000","last_undegraded":"2024-01-15T18:23:53.596663+0000","last_fullsized":"2024-01-15T18:23:53.596663+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'11","last_scrub_stamp":"2024-01-15T18:23:53.596595+0000","last_deep_scrub":"113'11","last_deep_scrub_stamp":"2024-01-15T18:23:53.596595+0000","last_clean_scrub_stamp":"2024-01-15T18:23:53.596595+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:48.256546+0000","scrub_duration":0.0028069950000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"113'59","reported_seq":91,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:53.558536+0000","last_change":"2024-01-15T18:23:53.558572+0000","last_active":"2024-01-15T18:23:53.558536+0000","last_peered":"2024-01-15T18:23:53.558536+0000","last_clean":"2024-01-15T18:23:53.558536+0000","last_became_active":"2024-01-15T18:18:41.941283+0000","last_became_peered":"2024-01-15T18:18:41.941283+0000","last_unstale":"2024-01-15T18:23:53.558536+0000","last_undegraded":"2024-01-15T18:23:53.558536+0000","last_fullsized":"2024-01-15T18:23:53.558536+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'59","last_scrub_stamp":"2024-01-15T18:23:53.558463+0000","last_deep_scrub":"113'59","last_deep_scrub_stamp":"2024-01-15T18:23:53.558463+0000","last_clean_scrub_stamp":"2024-01-15T18:23:53.558463+0000","objects_scrubbed":23,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:58.192714+0000","scrub_duration":0.163912535,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":26,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:10.443848+0000","last_change":"2024-01-15T18:24:10.443848+0000","last_active":"2024-01-15T18:24:10.443848+0000","last_peered":"2024-01-15T18:24:10.443848+0000","last_clean":"2024-01-15T18:24:10.443848+0000","last_became_active":"2024-01-15T18:18:40.934274+0000","last_became_peered":"2024-01-15T18:18:40.934274+0000","last_unstale":"2024-01-15T18:24:10.443848+0000","last_undegraded":"2024-01-15T18:24:10.443848+0000","last_fullsized":"2024-01-15T18:24:10.443848+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:24:10.443717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:24:10.443717+0000","last_clean_scrub_stamp":"2024-01-15T18:24:10.443717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:05:32.092645+0000","scrub_duration":0.0019560620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"113'45","reported_seq":77,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:24:28.807806+0000","last_change":"2024-01-15T18:24:28.807835+0000","last_active":"2024-01-15T18:24:28.807806+0000","last_peered":"2024-01-15T18:24:28.807806+0000","last_clean":"2024-01-15T18:24:28.807806+0000","last_became_active":"2024-01-15T18:18:41.939830+0000","last_became_peered":"2024-01-15T18:18:41.939830+0000","last_unstale":"2024-01-15T18:24:28.807806+0000","last_undegraded":"2024-01-15T18:24:28.807806+0000","last_fullsized":"2024-01-15T18:24:28.807806+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'45","last_scrub_stamp":"2024-01-15T18:24:28.807735+0000","last_deep_scrub":"113'45","last_deep_scrub_stamp":"2024-01-15T18:24:28.807735+0000","last_clean_scrub_stamp":"2024-01-15T18:24:28.807735+0000","objects_scrubbed":14,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:29:26.289592+0000","scrub_duration":0.10102768500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":25,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:48.772874+0000","last_change":"2024-01-15T18:23:48.772893+0000","last_active":"2024-01-15T18:23:48.772874+0000","last_peered":"2024-01-15T18:23:48.772874+0000","last_clean":"2024-01-15T18:23:48.772874+0000","last_became_active":"2024-01-15T18:18:40.929883+0000","last_became_peered":"2024-01-15T18:18:40.929883+0000","last_unstale":"2024-01-15T18:23:48.772874+0000","last_undegraded":"2024-01-15T18:23:48.772874+0000","last_fullsized":"2024-01-15T18:23:48.772874+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:23:48.772769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:23:48.772769+0000","last_clean_scrub_stamp":"2024-01-15T18:23:48.772769+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:49:41.149208+0000","scrub_duration":0.0017691320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"113'119","reported_seq":257,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-01-15T18:23:49.271682+0000","last_change":"2024-01-15T18:23:49.271712+0000","last_active":"2024-01-15T18:23:49.271682+0000","last_peered":"2024-01-15T18:23:49.271682+0000","last_clean":"2024-01-15T18:23:49.271682+0000","last_became_active":"2024-01-15T18:04:48.759232+0000","last_became_peered":"2024-01-15T18:04:48.759232+0000","last_unstale":"2024-01-15T18:23:49.271682+0000","last_undegraded":"2024-01-15T18:23:49.271682+0000","last_fullsized":"2024-01-15T18:23:49.271682+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'119","last_scrub_stamp":"2024-01-15T18:23:49.271604+0000","last_deep_scrub":"113'119","last_deep_scrub_stamp":"2024-01-15T18:23:49.271604+0000","last_clean_scrub_stamp":"2024-01-15T18:23:49.271604+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:57.760381+0000","scrub_duration":0.19866204600000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710136,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":113246208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3985,"ondisk_log_size":3985,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":618534,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281,"num_write_kb":726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5506,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1302528,"data_stored":1237068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575092,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834544,"kb_used_data":1767336,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537296,"statfs":{"total":96636764160,"available":94758191104,"internally_reserved":0,"allocated":1809752064,"data_stored":1808947458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69299999999999995,"15min":0.90600000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.29999999999999999},"max":{"1min":0.81299999999999994,"5min":0.95899999999999996,"15min":16.815999999999999},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73499999999999999,"15min":0.81200000000000006},"min":{"1min":0.432,"5min":0.40400000000000003,"15min":0.33400000000000002},"max":{"1min":0.91600000000000004,"5min":0.99199999999999999,"15min":8.5419999999999998},"last":0.91800000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.749,"15min":1.0309999999999999},"min":{"1min":0.48799999999999999,"5min":0.375,"15min":0.36699999999999999},"max":{"1min":1.016,"5min":1.024,"15min":16.920000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.73099999999999998,"15min":1.012},"min":{"1min":0.49199999999999999,"5min":0.44800000000000001,"15min":0.40999999999999998},"max":{"1min":0.78800000000000003,"5min":1.04,"15min":31.774999999999999},"last":0.98899999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.72499999999999998,"15min":0.96699999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.377},"max":{"1min":0.76800000000000002,"5min":1.085,"15min":30.846},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.755,"15min":0.92000000000000004},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.39800000000000002},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":17.039999999999999},"last":1.0049999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73699999999999999,"15min":1.0409999999999999},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.38800000000000001},"max":{"1min":0.84799999999999998,"5min":1.0109999999999999,"15min":19.131},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77300000000000002,"15min":1.1519999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41499999999999998},"max":{"1min":0.97399999999999998,"5min":1.0640000000000001,"15min":42.692},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57899999999999996,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.38100000000000001,"15min":0.31},"max":{"1min":0.82999999999999996,"5min":0.874,"15min":14.087},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.622,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.40699999999999997,"15min":0.36399999999999999},"max":{"1min":0.82599999999999996,"5min":0.94799999999999995,"15min":2.952},"last":0.878}]},{"osd":5,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65600000000000003,"15min":0.69799999999999995},"min":{"1min":0.439,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":11.334},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.66700000000000004,"15min":0.746},"min":{"1min":0.432,"5min":0.42399999999999999,"15min":0.36099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":8.423},"last":0.85099999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:24:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71399999999999997,"15min":0.78000000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":8.5030000000000001},"last":1.034},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.35299999999999998},"max":{"1min":0.88700000000000001,"5min":1.194,"15min":3.3450000000000002},"last":0.747}]}]},{"osd":6,"up_from":13,"seq":55834575091,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1260524,"kb_used_data":1193316,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93111316,"statfs":{"total":96636764160,"available":95345987584,"internally_reserved":0,"allocated":1221955584,"data_stored":1221154106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.68500000000000005,"15min":0.84199999999999997},"min":{"1min":0.41499999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":15.257999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68400000000000005,"15min":0.753},"min":{"1min":0.501,"5min":0.376,"15min":0.32000000000000001},"max":{"1min":0.93700000000000006,"5min":1.018,"15min":8.8030000000000008},"last":0.77600000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.80400000000000005},"min":{"1min":0.51000000000000001,"5min":0.47199999999999998,"15min":0.34300000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":6.8239999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70499999999999996,"15min":0.92100000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.32300000000000001},"max":{"1min":0.92900000000000005,"5min":0.96299999999999997,"15min":19.475000000000001},"last":0.51700000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.749,"15min":0.95199999999999996},"min":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.40600000000000003},"max":{"1min":1.028,"5min":1.0389999999999999,"15min":14.743},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.747,"15min":0.89300000000000002},"min":{"1min":0.52300000000000002,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.048,"5min":1.103,"15min":14.340999999999999},"last":0.495}]},{"osd":3,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72699999999999998,"15min":0.88200000000000001},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.0620000000000001,"15min":17.359999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73999999999999999,"15min":1.016},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.45800000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":18.791},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56799999999999995,"15min":0.70399999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.32400000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":19.114999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.63},"min":{"1min":0.38800000000000001,"5min":0.38500000000000001,"15min":0.33700000000000002},"max":{"1min":0.90700000000000003,"5min":0.93899999999999995,"15min":2.444},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63900000000000001,"15min":0.69299999999999995},"min":{"1min":0.42399999999999999,"5min":0.375,"15min":0.33500000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":7.5880000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.75},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.42099999999999999},"max":{"1min":0.94899999999999995,"5min":0.96299999999999997,"15min":2.5259999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.68100000000000005,"15min":0.80200000000000005},"min":{"1min":0.433,"5min":0.433,"15min":0.38600000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":11.907},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65200000000000002,"15min":0.83299999999999996},"min":{"1min":0.41799999999999998,"5min":0.41299999999999998,"15min":0.34200000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":27.651},"last":0.56200000000000006}]}]},{"osd":5,"up_from":13,"seq":55834575092,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":937072,"kb_used_data":861544,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93434768,"statfs":{"total":96636764160,"available":95677202432,"internally_reserved":0,"allocated":882221056,"data_stored":881416086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.67900000000000005,"15min":0.83699999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":1.9350000000000001,"15min":11.519},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.93600000000000005},"min":{"1min":0.49199999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.85399999999999998,"5min":0.96999999999999997,"15min":12.952},"last":0.78500000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.74299999999999999,"15min":0.89100000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.37},"max":{"1min":0.93400000000000005,"5min":1.0720000000000001,"15min":13.358000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.71599999999999997,"15min":0.91200000000000003},"min":{"1min":0.45900000000000002,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":0.77500000000000002,"5min":1.6399999999999999,"15min":21.856999999999999},"last":0.442}]},{"osd":2,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.77100000000000002,"15min":0.94899999999999995},"min":{"1min":0.47599999999999998,"5min":0.41799999999999998,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":1.734,"15min":18.314},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.77400000000000002,"15min":1.0209999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.435},"max":{"1min":0.88300000000000001,"5min":1.762,"15min":21.120000000000001},"last":0.748}]},{"osd":3,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76800000000000002,"15min":0.94599999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.441},"max":{"1min":0.86499999999999999,"5min":1.956,"15min":13.317},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.75700000000000001,"15min":1.0109999999999999},"min":{"1min":0.47999999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84399999999999997,"5min":1.6870000000000001,"15min":19.638000000000002},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.621,"15min":0.67300000000000004},"min":{"1min":0.37,"5min":0.36799999999999999,"15min":0.33500000000000002},"max":{"1min":0.90900000000000003,"5min":0.96399999999999997,"15min":5.899},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.60699999999999998,"15min":0.65500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.35299999999999998},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":3.3889999999999998},"last":0.495}]},{"osd":6,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.72799999999999998,"15min":0.78300000000000003},"min":{"1min":0.48699999999999999,"5min":0.45000000000000001,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":1.008,"15min":5.8680000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.628,"15min":0.70299999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80100000000000005,"5min":0.95799999999999996,"15min":7.2240000000000002},"last":0.80800000000000005}]},{"osd":7,"last update":"Mon Jan 15 18:24:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68600000000000005,"15min":0.72499999999999998},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.39400000000000002},"max":{"1min":0.95599999999999996,"5min":1.081,"15min":3.2759999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.753,"15min":0.79800000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.41099999999999998},"max":{"1min":0.96999999999999997,"5min":3.7530000000000001,"15min":6.2060000000000004},"last":0.72399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575091,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1706920,"kb_used_data":1643872,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92664920,"statfs":{"total":96636764160,"available":94888878080,"internally_reserved":0,"allocated":1683324928,"data_stored":1682527615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69999999999999996,"15min":0.80600000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.93100000000000005,"5min":1.02,"15min":11.374000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.71099999999999997,"15min":0.83399999999999996},"min":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.32600000000000001},"max":{"1min":2.6920000000000002,"5min":2.6920000000000002,"15min":14.488},"last":0.51200000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.46999999999999997,"5min":0.441,"15min":0.34000000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":7.625},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.76000000000000001,"15min":0.79000000000000004},"min":{"1min":0.499,"5min":0.497,"15min":0.372},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":4.4550000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79900000000000004,"15min":0.85999999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.34599999999999997},"max":{"1min":1.877,"5min":1.877,"15min":7.6989999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76700000000000002,"15min":0.82299999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":1.173,"5min":1.173,"15min":7.4829999999999997},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.73199999999999998,"15min":0.90100000000000002},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.377},"max":{"1min":1.141,"5min":1.141,"15min":14.529},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.747,"15min":0.88800000000000001},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":25.774000000000001},"last":0.79300000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66400000000000003,"15min":0.68300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.33200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":9.9619999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.68600000000000005,"15min":0.78200000000000003},"min":{"1min":0.45500000000000002,"5min":0.39400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93799999999999994,"5min":1.0109999999999999,"15min":25.887},"last":0.61299999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.67500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":7.5439999999999996},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68600000000000005,"15min":0.67600000000000005},"min":{"1min":0.48999999999999999,"5min":0.46200000000000002,"15min":0.33900000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":3.6949999999999998},"last":0.58999999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":0.77100000000000002,"15min":0.76500000000000001},"min":{"1min":0.48499999999999999,"5min":0.45100000000000001,"15min":0.30399999999999999},"max":{"1min":11.093,"5min":11.093,"15min":13.760999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72899999999999998,"15min":0.74399999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.377},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":11.67},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834575091,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1453652,"kb_used_data":1386444,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92918188,"statfs":{"total":96636764160,"available":95148224512,"internally_reserved":0,"allocated":1419718656,"data_stored":1418903209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.48299999999999998,"15min":0.84499999999999997},"min":{"1min":0.309,"5min":0.251,"15min":0.251},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":26.364000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.53600000000000003,"15min":0.82499999999999996},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.25},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":23.074999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48999999999999999,"15min":0.82299999999999995},"min":{"1min":0.315,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":20.321000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.86599999999999999},"min":{"1min":0.35499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.335,"5min":2.335,"15min":26.260999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.90500000000000003},"min":{"1min":0.38900000000000001,"5min":0.38500000000000001,"15min":0.314},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":23.036999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.628,"15min":1.01},"min":{"1min":0.41699999999999998,"5min":0.41299999999999998,"15min":0.34300000000000003},"max":{"1min":2.3690000000000002,"5min":2.3690000000000002,"15min":26.404},"last":0.61799999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.79500000000000004,"15min":0.97299999999999998},"min":{"1min":0.51300000000000001,"5min":0.50700000000000001,"15min":0.42699999999999999},"max":{"1min":1.468,"5min":1.468,"15min":22.998000000000001},"last":0.751},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":1.038},"min":{"1min":0.58099999999999996,"5min":0.46300000000000002,"15min":0.372},"max":{"1min":1.0529999999999999,"5min":1.0960000000000001,"15min":20.399000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.79900000000000004,"15min":1.056},"min":{"1min":0.59999999999999998,"5min":0.498,"15min":0.45000000000000001},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":26.314},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.80400000000000005,"15min":1.032},"min":{"1min":0.68100000000000005,"5min":0.52000000000000002,"15min":0.42399999999999999},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":22.908999999999999},"last":0.89300000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.82499999999999996,"15min":1.107},"min":{"1min":0.56899999999999995,"5min":0.51000000000000001,"15min":0.45600000000000002},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":20.445},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.84299999999999997,"15min":1.141},"min":{"1min":0.61099999999999999,"5min":0.53400000000000003,"15min":0.53200000000000003},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":26.091999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Jan 15 18:24:23 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.82499999999999996,"15min":1.091},"min":{"1min":0.61699999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.853,"5min":1.853,"15min":23.111999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82899999999999996,"15min":1.2609999999999999},"min":{"1min":0.60999999999999999,"5min":0.56200000000000006,"15min":0.46400000000000002},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":44.817999999999998},"last":0.92300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575092,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":969704,"kb_used_data":902496,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93402136,"statfs":{"total":96636764160,"available":95643787264,"internally_reserved":0,"allocated":924155904,"data_stored":923359651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.5,"15min":0.75},"min":{"1min":0.30099999999999999,"5min":0.26200000000000001,"15min":0.26000000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":12.695},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.56999999999999995,"15min":0.94299999999999995},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":23.792999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.59699999999999998,"15min":0.81899999999999995},"min":{"1min":0.36599999999999999,"5min":0.32100000000000001,"15min":0.30499999999999999},"max":{"1min":8.4380000000000006,"5min":8.4380000000000006,"15min":34.029000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54300000000000004,"15min":0.83599999999999997},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.627,"5min":1.627,"15min":20.619},"last":0.53000000000000003}]},{"osd":3,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.51600000000000001,"15min":1.0669999999999999},"min":{"1min":0.34399999999999997,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":39.945999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.58799999999999997,"15min":0.71399999999999997},"min":{"1min":0.38900000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81999999999999995,"5min":0.84299999999999997,"15min":7.3609999999999998},"last":0.50700000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.79100000000000004,"15min":0.91500000000000004},"min":{"1min":0.54100000000000004,"5min":0.49099999999999999,"15min":0.47899999999999998},"max":{"1min":0.94899999999999995,"5min":1.1100000000000001,"15min":7.4169999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.76900000000000002,"15min":0.98799999999999999},"min":{"1min":0.56899999999999995,"5min":0.47499999999999998,"15min":0.432},"max":{"1min":1.673,"5min":1.673,"15min":20.053000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79400000000000004,"15min":1.292},"min":{"1min":0.56100000000000005,"5min":0.502,"15min":0.45200000000000001},"max":{"1min":1.224,"5min":1.246,"15min":41.683999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79200000000000004,"15min":1.0349999999999999},"min":{"1min":0.57099999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.649,"5min":1.649,"15min":26.084},"last":0.78000000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":1.0669999999999999},"min":{"1min":0.58299999999999996,"5min":0.47299999999999998,"15min":0.45900000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":22.199000000000002},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.78600000000000003,"15min":1.329},"min":{"1min":0.58599999999999997,"5min":0.51400000000000001,"15min":0.47699999999999998},"max":{"1min":1.3200000000000001,"5min":1.369,"15min":41.738999999999997},"last":0.75800000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:24:16 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.84899999999999998,"15min":1.075},"min":{"1min":0.64900000000000002,"5min":0.47699999999999998,"15min":0.45100000000000001},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":20.158000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":1.3540000000000001},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.44800000000000001},"max":{"1min":1.2669999999999999,"5min":1.3,"15min":41.402000000000001},"last":0.79900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575092,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1133560,"kb_used_data":1066352,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93238280,"statfs":{"total":96636764160,"available":95475998720,"internally_reserved":0,"allocated":1091944448,"data_stored":1091132307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.46100000000000002,"15min":0.65900000000000003},"min":{"1min":0.29199999999999998,"5min":0.247,"15min":0.23599999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":9.452},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60099999999999998,"15min":0.91800000000000004},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":3.8010000000000002,"5min":4.9329999999999998,"15min":35.790999999999997},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.58199999999999996,"15min":0.749},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.27600000000000002},"max":{"1min":0.91600000000000004,"5min":9.2609999999999992,"15min":15.137},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.77800000000000002},"min":{"1min":0.374,"5min":0.32900000000000001,"15min":0.29999999999999999},"max":{"1min":0.80700000000000005,"5min":1.7649999999999999,"15min":9.8469999999999995},"last":0.40000000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.56699999999999995,"15min":0.91300000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.311},"max":{"1min":3.278,"5min":3.278,"15min":27.643999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.57799999999999996,"15min":0.874},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":1.02,"5min":1.198,"15min":15.608000000000001},"last":0.56499999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77100000000000002,"15min":1.069},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.45100000000000001},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":27.489999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.85499999999999998,"15min":1.0229999999999999},"min":{"1min":0.51300000000000001,"5min":0.495,"15min":0.46899999999999997},"max":{"1min":1.131,"5min":6.6950000000000003,"15min":10.038},"last":0.93600000000000005}]},{"osd":5,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80200000000000005,"15min":1.095},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.5600000000000001,"5min":2.5600000000000001,"15min":27.547000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.85099999999999998,"15min":1.0389999999999999},"min":{"1min":0.63,"5min":0.59199999999999997,"15min":0.55400000000000005},"max":{"1min":1.337,"5min":1.337,"15min":9.9610000000000003},"last":0.89600000000000002}]},{"osd":6,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81499999999999995,"15min":1.0960000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.605,"5min":2.605,"15min":27.579999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.83799999999999997,"15min":0.94599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.42399999999999999},"max":{"1min":1.6950000000000001,"5min":5.6470000000000002,"15min":11.643000000000001},"last":0.69599999999999995}]},{"osd":7,"last update":"Mon Jan 15 18:24:26 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.83699999999999997,"15min":1.04},"min":{"1min":0.57999999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0149999999999999,"5min":5.819,"15min":11.833},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.018,"15min":1.1220000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.502},"max":{"1min":12.673999999999999,"5min":12.673999999999999,"15min":14.606},"last":0.85199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575091,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1740896,"kb_used_data":1677848,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":92630944,"statfs":{"total":96636764160,"available":94854086656,"internally_reserved":0,"allocated":1718116352,"data_stored":1717289244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.83699999999999997},"min":{"1min":0.316,"5min":0.27400000000000002,"15min":0.23999999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":16.149000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.50700000000000001,"15min":0.82199999999999995},"min":{"1min":0.34599999999999997,"5min":0.28299999999999997,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":1.6579999999999999,"15min":13.018000000000001},"last":0.44600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54800000000000004,"15min":0.96299999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.307},"max":{"1min":0.73399999999999999,"5min":1.1759999999999999,"15min":36.386000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51300000000000001,"15min":0.78700000000000003},"min":{"1min":0.36699999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68500000000000005,"5min":1.7689999999999999,"15min":16.556000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.84499999999999997},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.29899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80500000000000005,"15min":11.391999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":1.012},"min":{"1min":0.41199999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.80200000000000005,"5min":0.95599999999999996,"15min":45.128},"last":0.86799999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76200000000000001,"15min":0.92300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":1.4550000000000001,"15min":11.772},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78100000000000003,"15min":1.0129999999999999},"min":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.45100000000000001},"max":{"1min":1.0589999999999999,"5min":1.115,"15min":14.831},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.98999999999999999},"min":{"1min":0.57099999999999995,"5min":0.55700000000000005,"15min":0.38600000000000001},"max":{"1min":1.008,"5min":1.179,"15min":11.287000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.94199999999999995},"min":{"1min":0.54500000000000004,"5min":0.48699999999999999,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.7869999999999999,"15min":11.958},"last":0.749}]},{"osd":6,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.80200000000000005,"15min":0.99099999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46999999999999997},"max":{"1min":1.073,"5min":1.3859999999999999,"15min":13.055},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.79900000000000004,"15min":0.96499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47699999999999998},"max":{"1min":1.0089999999999999,"5min":1.7170000000000001,"15min":11.833},"last":0.90400000000000003}]},{"osd":7,"last update":"Mon Jan 15 18:24:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.83399999999999996,"15min":1.024},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":2.0350000000000001,"15min":13.090999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80000000000000004,"15min":1.054},"min":{"1min":0.59799999999999998,"5min":0.52200000000000002,"15min":0.51000000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":11.321999999999999},"last":0.83699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":634880,"data_stored":616527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":630784,"data_stored":616485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1715482624,"data_stored":1715470899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":922746880,"data_stored":922746880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":880807936,"data_stored":880804032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:24:50.316 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-15T18:24:50.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 Processor -- start 2024-01-15T18:24:50.487 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 -- start start 2024-01-15T18:24:50.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f275c15b260 0x7f275c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.488 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c0cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275c0c4d90 con 0x7f275c0d19f0 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c15b640 con 0x7f275c15b260 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f2761600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0cf800 con 0x7f275c0ccaf0 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f275b400640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c0cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f275a000640 1 --1- >> v1:172.21.15.204:6789/0 conn(0x7f275c0d19f0 0x7f275c0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.204:6789/0 says I am v1:172.21.15.143:58540/0 (socket says 172.21.15.143:58540) 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f275a000640 1 -- 172.21.15.143:0/1456637255 learned_addr learned my addr 172.21.15.143:0/1456637255 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:24:50.489 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.492+0000 7f275aa00640 1 --2- 172.21.15.143:0/1456637255 >> v2:172.21.15.204:3300/0 conn(0x7f275c15b260 0x7f275c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f275b400640 1 -- 172.21.15.143:0/1456637255 >> v2:172.21.15.204:3300/0 conn(0x7f275c15b260 msgr2=0x7f275c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:24:50.490 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f275b400640 1 --2- 172.21.15.143:0/1456637255 >> v2:172.21.15.204:3300/0 conn(0x7f275c15b260 0x7f275c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f275b400640 1 -- 172.21.15.143:0/1456637255 >> v1:172.21.15.204:6789/0 conn(0x7f275c0d19f0 legacy=0x7f275c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f2759600640 1 -- 172.21.15.143:0/1456637255 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964089452 0 0) 0x7f275c0c4d90 con 0x7f275c0d19f0 2024-01-15T18:24:50.491 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f275b400640 1 -- 172.21.15.143:0/1456637255 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0cfae0 con 0x7f275c0ccaf0 2024-01-15T18:24:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f275b400640 1 --2- 172.21.15.143:0/1456637255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c0cdd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2754009d90 tx=0x7f275401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f3e4c81a94d0cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f2759600640 1 -- 172.21.15.143:0/1456637255 <== mon.2 v2:172.21.15.204:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27540a5900 con 0x7f275c0ccaf0 2024-01-15T18:24:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f2759600640 1 -- 172.21.15.143:0/1456637255 <== mon.2 v2:172.21.15.204:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27540a5a60 con 0x7f275c0ccaf0 2024-01-15T18:24:50.492 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f2759600640 1 -- 172.21.15.143:0/1456637255 <== mon.2 v2:172.21.15.204:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27540a5df0 con 0x7f275c0ccaf0 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f2761600640 1 -- 172.21.15.143:0/1456637255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 msgr2=0x7f275c0cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.496+0000 7f2761600640 1 --2- 172.21.15.143:0/1456637255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c0cdd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2754009d90 tx=0x7f275401c950 comp rx=0 tx=0).stop 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- 172.21.15.143:0/1456637255 shutdown_connections 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 --2- 172.21.15.143:0/1456637255 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c0cdd50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 --2- 172.21.15.143:0/1456637255 >> v2:172.21.15.204:3300/0 conn(0x7f275c15b260 0x7f275c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- 172.21.15.143:0/1456637255 >> 172.21.15.143:0/1456637255 conn(0x7f275c0d0ed0 msgr2=0x7f275c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:50.494 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- 172.21.15.143:0/1456637255 shutdown_connections 2024-01-15T18:24:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- 172.21.15.143:0/1456637255 wait complete. 2024-01-15T18:24:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 Processor -- start 2024-01-15T18:24:50.495 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- start start 2024-01-15T18:24:50.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.498 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 0x7f275c14b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f275c15b260 0x7f275c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c154390 con 0x7f275c0d19f0 2024-01-15T18:24:50.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0b3a60 con 0x7f275c15b260 2024-01-15T18:24:50.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f2761600640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c158510 con 0x7f275c0ccaf0 2024-01-15T18:24:50.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 0x7f275c14b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.499 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 0x7f275c14b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55312/0 (socket says 172.21.15.143:55312) 2024-01-15T18:24:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 -- 172.21.15.143:0/255008982 learned_addr learned my addr 172.21.15.143:0/255008982 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:24:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275aa00640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275b400640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f275c15b260 0x7f275c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 -- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 msgr2=0x7f275c153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.500 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 -- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f275c15b260 msgr2=0x7f275c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f275c15b260 0x7f275c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.500+0000 7f275a000640 1 -- 172.21.15.143:0/255008982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754009a70 con 0x7f275c0d19f0 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f275a000640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 0x7f275c14b620 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f274400c450 tx=0x7f274400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440074e0 con 0x7f275c0d19f0 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744007640 con 0x7f275c0d19f0 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27440049f0 con 0x7f275c0d19f0 2024-01-15T18:24:50.501 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c0b3d20 con 0x7f275c0d19f0 2024-01-15T18:24:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c0b42e0 con 0x7f275c0d19f0 2024-01-15T18:24:50.502 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c0b44d0 con 0x7f275c0d19f0 2024-01-15T18:24:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.504+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 46) v1 ==== 87792+0+0 (secure 0 0 0) 0x7f2744004b50 con 0x7f275c0d19f0 2024-01-15T18:24:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.508+0000 7f2752a00640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2734067af0 0x7f2734069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.508+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2744082510 con 0x7f275c0d19f0 2024-01-15T18:24:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.508+0000 7f275aa00640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2734067af0 0x7f2734069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.503 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.508+0000 7f275aa00640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2734067af0 0x7f2734069fb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f274c009b60 tx=0x7f274c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.504 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.508+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f2744050ee0 con 0x7f275c0d19f0 2024-01-15T18:24:50.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.664+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f275c067500 con 0x7f275c0d19f0 2024-01-15T18:24:50.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v43)=0 v43) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2744018030 con 0x7f275c0d19f0 2024-01-15T18:24:50.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2752a00640 1 -- 172.21.15.143:0/255008982 <== mon.0 v2:172.21.15.143:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2744050ee0 con 0x7f275c0d19f0 2024-01-15T18:24:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2734067af0 msgr2=0x7f2734069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2734067af0 0x7f2734069fb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f274c009b60 tx=0x7f274c009190 comp rx=0 tx=0).stop 2024-01-15T18:24:50.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 msgr2=0x7f275c14b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 0x7f275c14b620 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f274400c450 tx=0x7f274400c910 comp rx=0 tx=0).stop 2024-01-15T18:24:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 shutdown_connections 2024-01-15T18:24:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f2734067af0 0x7f2734069fb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f275c15b260 0x7f275c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f275c0d19f0 0x7f275c14b620 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.670 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 --2- 172.21.15.143:0/255008982 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f275c0ccaf0 0x7f275c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 >> 172.21.15.143:0/255008982 conn(0x7f275c0d0ed0 msgr2=0x7f275c06ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.672+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 shutdown_connections 2024-01-15T18:24:50.671 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.676+0000 7f2761600640 1 -- 172.21.15.143:0/255008982 wait complete. 2024-01-15T18:24:50.679 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:24:50.684+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:24:50.679 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:24:50.684+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:24:50.679 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:24:50.684+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:24:50.680 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:24:50.684+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:24:50.680 INFO:tasks.ceph.mgr.y.smithi143.stderr:2024-01-15T18:24:50.684+0000 7f3cafc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:24:50.694 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-15T18:24:50.694 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:24:50.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1340c00640 1 Processor -- start 2024-01-15T18:24:50.830 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1340c00640 1 -- start start 2024-01-15T18:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1340c00640 1 --2- >> v2:172.21.15.204:3300/0 conn(0x7f133c15b260 0x7f133c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1340c00640 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c0c4d90 con 0x7f133c0d19f0 2024-01-15T18:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1340c00640 1 -- --> v1:172.21.15.204:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c15b640 con 0x7f133c0ccaf0 2024-01-15T18:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1340c00640 1 -- --> v2:172.21.15.204:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c0cf800 con 0x7f133c15b260 2024-01-15T18:24:50.834 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1339400640 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f133c0d19f0 0x7f133c0c5220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55998/0 (socket says 172.21.15.143:55998) 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1339400640 1 -- 172.21.15.143:0/2897175767 learned_addr learned my addr 172.21.15.143:0/2897175767 (peer_addr_for_me v1:172.21.15.143:0/0) 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1339e00640 1 --2- 172.21.15.143:0/2897175767 >> v2:172.21.15.204:3300/0 conn(0x7f133c15b260 0x7f133c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330256560 0 0) 0x7f133c0c4d90 con 0x7f133c0d19f0 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c003530 con 0x7f133c0d19f0 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.1 v1:172.21.15.204:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474311921 0 0) 0x7f133c15b640 con 0x7f133c0ccaf0 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 --> v1:172.21.15.204:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f133c0c4d90 con 0x7f133c0ccaf0 2024-01-15T18:24:50.835 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548355112 0 0) 0x7f131c003530 con 0x7f133c0d19f0 2024-01-15T18:24:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 >> v2:172.21.15.204:3300/0 conn(0x7f133c15b260 msgr2=0x7f133c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:24:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 --2- 172.21.15.143:0/2897175767 >> v2:172.21.15.204:3300/0 conn(0x7f133c15b260 0x7f133c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 >> v1:172.21.15.204:6789/0 conn(0x7f133c0ccaf0 legacy=0x7f133c0c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0cfae0 con 0x7f133c0d19f0 2024-01-15T18:24:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4132338102 0 0) 0x7f1324001680 con 0x7f133c0d19f0 2024-01-15T18:24:50.836 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 >> v1:172.21.15.143:6789/0 conn(0x7f133c0d19f0 legacy=0x7f133c0c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f131c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f131c0047d0 0x7f131c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f131c005190 0x7f131c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0cf800 con 0x7f131c003b10 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00c010 con 0x7f131c005190 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.836+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00c1d0 con 0x7f131c0047d0 2024-01-15T18:24:50.837 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f131c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f133a800640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f131c005190 0x7f131c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339400640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f131c0047d0 0x7f131c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 -- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f131c0047d0 msgr2=0x7f131c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f131c0047d0 0x7f131c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 -- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f131c005190 msgr2=0x7f131c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f131c005190 0x7f131c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.838 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 -- 172.21.15.143:0/2897175767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c003530 con 0x7f131c003b10 2024-01-15T18:24:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f133a800640 1 -- 172.21.15.143:0/2897175767 reap_dead start 2024-01-15T18:24:50.839 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339400640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f131c0047d0 0x7f131c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:24:50.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1339e00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f131c003f10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ae0 tx=0x7f132c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b04f318b7e922c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.842 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0a55c0 con 0x7f131c003b10 2024-01-15T18:24:50.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f132c0a5720 con 0x7f131c003b10 2024-01-15T18:24:50.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.840+0000 7f1338a00640 1 -- 172.21.15.143:0/2897175767 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0a5af0 con 0x7f131c003b10 2024-01-15T18:24:50.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- 172.21.15.143:0/2897175767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 msgr2=0x7f131c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f131c003f10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ae0 tx=0x7f132c01ca50 comp rx=0 tx=0).stop 2024-01-15T18:24:50.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- 172.21.15.143:0/2897175767 shutdown_connections 2024-01-15T18:24:50.843 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 --2- 172.21.15.143:0/2897175767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f131c003f10 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- 172.21.15.143:0/2897175767 >> 172.21.15.143:0/2897175767 conn(0x7f133c0d0ed0 msgr2=0x7f133c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:50.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- 172.21.15.143:0/2897175767 shutdown_connections 2024-01-15T18:24:50.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- 172.21.15.143:0/2897175767 wait complete. 2024-01-15T18:24:50.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 Processor -- start 2024-01-15T18:24:50.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- start start 2024-01-15T18:24:50.844 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f133c1f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 --2- >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0c5fa0 0x7f133c0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0c63e0 0x7f133c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0cae20 con 0x7f131c003b10 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0ae1e0 con 0x7f133c0c63e0 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1340c00640 1 -- --> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0c9c00 con 0x7f133c0c5fa0 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1339e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f133c1f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.845 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1339e00640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f133c1f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:55354/0 (socket says 172.21.15.143:55354) 2024-01-15T18:24:50.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1339e00640 1 -- 172.21.15.143:0/3415444456 learned_addr learned my addr 172.21.15.143:0/3415444456 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-01-15T18:24:50.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f1339400640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0c5fa0 0x7f133c0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.844+0000 7f133a800640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0c63e0 0x7f133c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339e00640 1 -- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0c5fa0 msgr2=0x7f133c0c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339e00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0c5fa0 0x7f133c0c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.846 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339e00640 1 -- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0c63e0 msgr2=0x7f133c0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339e00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0c63e0 0x7f133c0c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339e00640 1 -- 172.21.15.143:0/3415444456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0c9ee0 con 0x7f131c003b10 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339e00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f133c1f5f90 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f132c0b0040 tx=0x7f132c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0a55c0 con 0x7f131c003b10 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f132c004240 con 0x7f131c003b10 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c0adb90 con 0x7f131c003b10 2024-01-15T18:24:50.847 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c003620 con 0x7f131c003b10 2024-01-15T18:24:50.848 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c152ef0 con 0x7f131c003b10 2024-01-15T18:24:50.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c067500 con 0x7f131c003b10 2024-01-15T18:24:50.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 46) v1 ==== 87792+0+0 (secure 0 0 0) 0x7f132c0b4070 con 0x7f131c003b10 2024-01-15T18:24:50.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1332000640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1314067b20 0x7f1314069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:24:50.849 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f132c11ce60 con 0x7f131c003b10 2024-01-15T18:24:50.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.848+0000 7f1339400640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1314067b20 0x7f1314069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:24:50.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.852+0000 7f1339400640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1314067b20 0x7f1314069fe0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f133c06fe20 tx=0x7f1324007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:24:50.850 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:50.852+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172649 (secure 0 0 0) 0x7f132c0eba90 con 0x7f131c003b10 2024-01-15T18:24:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f133c0cc5e0 con 0x7f131c003b10 2024-01-15T18:24:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1332000640 1 -- 172.21.15.143:0/3415444456 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f132c0b2070 con 0x7f131c003b10 2024-01-15T18:24:51.030 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-01-15T18:24:51.030 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8776,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6834/2963172683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":2963172683},{"type":"v1","addr":"172.21.15.204:6834","nonce":2963172683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":9012,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6836/813203633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":813203633},{"type":"v1","addr":"172.21.15.143:6836","nonce":813203633}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":9015,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/3497546933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6835","nonce":3497546933},{"type":"v1","addr":"172.21.15.143:6837","nonce":3497546933}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:18:41.297206+0000","modified":"2024-01-15T18:18:44.320349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9021},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9021":{"gid":9021,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.204:6835/3419861535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6833","nonce":3419861535},{"type":"v1","addr":"172.21.15.204:6835","nonce":3419861535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-01-15T18:24:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 73 2024-01-15T18:24:51.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1314067b20 msgr2=0x7f1314069fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:51.032 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1314067b20 0x7f1314069fe0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f133c06fe20 tx=0x7f1324007040 comp rx=0 tx=0).stop 2024-01-15T18:24:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 msgr2=0x7f133c1f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:24:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f133c1f5f90 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f132c0b0040 tx=0x7f132c0197f0 comp rx=0 tx=0).stop 2024-01-15T18:24:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 shutdown_connections 2024-01-15T18:24:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:6832/3137793172,v1:172.21.15.143:6833/3137793172] conn(0x7f1314067b20 0x7f1314069fe0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f133c0c63e0 0x7f133c0c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.204:3301/0,v1:172.21.15.204:6790/0] conn(0x7f133c0c5fa0 0x7f133c0c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 --2- 172.21.15.143:0/3415444456 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f131c003b10 0x7f133c1f5f90 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:24:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 >> 172.21.15.143:0/3415444456 conn(0x7f133c0d0ed0 msgr2=0x7f133c0c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:24:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 shutdown_connections 2024-01-15T18:24:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2024-01-15T18:24:51.036+0000 7f1340c00640 1 -- 172.21.15.143:0/3415444456 wait complete. 2024-01-15T18:24:51.052 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-15T18:24:51.053 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-15T18:24:51.053 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:24:51.053 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:51.065 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:24:51.065 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:24:51.065 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:51.120 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:24:51.120 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:24:51.120 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:51.134 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:24:51.134 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:24:51.135 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:51.195 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:24:51.196 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-15T18:24:51.196 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-15T18:24:51.196 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:51.281 INFO:tasks.ceph.osd.0:Stopped 2024-01-15T18:24:51.282 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-15T18:24:51.282 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:53.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-01-15T18:24:57.384 INFO:tasks.ceph.osd.1:Stopped 2024-01-15T18:24:57.384 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-15T18:24:57.384 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:57.468 INFO:tasks.ceph.osd.2:Stopped 2024-01-15T18:24:57.468 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-15T18:24:57.469 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:24:59.246 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-01-15T18:24:59.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-01-15T18:25:01.179 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:25:01.182 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:25:03.570 INFO:tasks.ceph.osd.3:Stopped 2024-01-15T18:25:03.570 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-15T18:25:03.570 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:03.663 INFO:tasks.ceph.osd.4:Stopped 2024-01-15T18:25:03.664 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-15T18:25:03.664 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:04.952 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-01-15T18:25:04.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-01-15T18:25:09.765 INFO:tasks.ceph.osd.5:Stopped 2024-01-15T18:25:09.766 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-15T18:25:09.766 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:09.867 INFO:tasks.ceph.osd.6:Stopped 2024-01-15T18:25:09.867 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-15T18:25:09.867 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:09.952 INFO:tasks.ceph.osd.7:Stopped 2024-01-15T18:25:09.952 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-15T18:25:09.953 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-15T18:25:09.953 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:10.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-01-15T18:25:10.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-01-15T18:25:15.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~5s 2024-01-15T18:25:16.055 INFO:tasks.ceph.mgr.y:Stopped 2024-01-15T18:25:16.056 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-15T18:25:16.056 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:16.109 INFO:tasks.ceph.mgr.x:Stopped 2024-01-15T18:25:16.109 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-15T18:25:16.109 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-15T18:25:16.110 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:16.132 INFO:tasks.ceph.mon.a:Stopped 2024-01-15T18:25:16.132 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-15T18:25:16.132 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:16.149 INFO:tasks.ceph.mon.b:Stopped 2024-01-15T18:25:16.149 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-15T18:25:16.149 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:25:16.208 INFO:tasks.ceph.mon.c:Stopped 2024-01-15T18:25:16.208 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-15T18:25:16.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-01-15T18:25:16.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:25:16.243 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-15T18:25:16.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:25:16.701 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-15T18:25:16.972 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:25:16.972 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-15T18:25:17.161 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:25:17.161 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-15T18:25:17.354 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:25:17.354 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-15T18:25:17.693 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:25:17.693 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-15T18:25:17.918 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:25:17.919 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-15T18:25:18.112 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:25:18.113 DEBUG:teuthology.orchestra.run.smithi204:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-15T18:25:18.303 INFO:tasks.ceph:Archiving mon data... 2024-01-15T18:25:18.304 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/data/mon.a.tgz 2024-01-15T18:25:18.304 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2024-01-15T18:25:18.313 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.5sU3xvt0yx 2024-01-15T18:25:18.314 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.5sU3xvt0yx 2024-01-15T18:25:18.552 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.5sU3xvt0yx 2024-01-15T18:25:18.629 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.5sU3xvt0yx is 1MB 2024-01-15T18:25:18.705 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.5sU3xvt0yx 2024-01-15T18:25:18.713 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/data/mon.b.tgz 2024-01-15T18:25:18.714 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-01-15T18:25:18.722 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.WIx3perZe8 2024-01-15T18:25:18.722 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.WIx3perZe8 2024-01-15T18:25:18.963 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.WIx3perZe8 2024-01-15T18:25:19.034 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.WIx3perZe8 is 1MB 2024-01-15T18:25:19.110 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.WIx3perZe8 2024-01-15T18:25:19.118 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/data/mon.c.tgz 2024-01-15T18:25:19.118 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-01-15T18:25:19.170 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.NZuR0pszG6 2024-01-15T18:25:19.171 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NZuR0pszG6 2024-01-15T18:25:19.404 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.NZuR0pszG6 2024-01-15T18:25:19.478 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.NZuR0pszG6 is 1MB 2024-01-15T18:25:19.550 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.NZuR0pszG6 2024-01-15T18:25:19.558 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-15T18:25:19.558 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-15T18:25:19.561 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-15T18:25:19.629 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-15T18:25:19.666 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-15T18:25:19.666 INFO:tasks.ceph:Archiving crash dumps... 2024-01-15T18:25:19.668 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/remote/smithi143/crash 2024-01-15T18:25:19.669 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-15T18:25:19.693 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/remote/smithi204/crash 2024-01-15T18:25:19.694 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-15T18:25:19.723 INFO:tasks.ceph:Shutting down logrotate 2024-01-15T18:25:19.724 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:25:19.740 DEBUG:teuthology.orchestra.run.smithi204:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:25:19.826 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-15T18:25:19.846 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-15T18:25:19.864 INFO:tasks.ceph:Compressing logs... 2024-01-15T18:25:19.865 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-15T18:25:19.896 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-15T18:25:19.918 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2024-01-15T18:25:19.919 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54659.log 2024-01-15T18:25:19.920 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38377.log 2024-01-15T18:25:19.920 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2024-01-15T18:25:19.921 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43357.log 2024-01-15T18:25:19.921 INFO:teuthology.orchestra.run.smithi143.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54659.log.gz 2024-01-15T18:25:19.922 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-01-15T18:25:19.922 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38377.log.gz 2024-01-15T18:25:19.923 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42921.log 2024-01-15T18:25:19.924 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43357.log.gz 2024-01-15T18:25:19.924 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35584.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-01-15T18:25:19.925 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53685.log 2024-01-15T18:25:19.925 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45330.log 2024-01-15T18:25:19.926 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42921.log.gz 2024-01-15T18:25:19.926 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53685.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53685.log.gz 2024-01-15T18:25:19.926 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38650.log 2024-01-15T18:25:19.927 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37851.log 2024-01-15T18:25:19.927 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45330.log.gz 2024-01-15T18:25:19.928 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38650.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.35395.log -- replaced with /var/log/ceph/ceph-client.admin.38650.log.gz 2024-01-15T18:25:19.928 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:19.929 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2024-01-15T18:25:19.929 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz 2024-01-15T18:25:19.930 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40123.log 2024-01-15T18:25:19.930 INFO:teuthology.orchestra.run.smithi143.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35395.log.gz 2024-01-15T18:25:19.931 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43435.log 2024-01-15T18:25:19.931 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz 2024-01-15T18:25:19.932 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40698.log 2024-01-15T18:25:19.932 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40123.log.gz 2024-01-15T18:25:19.933 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46293.log 2024-01-15T18:25:19.933 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43435.log.gz 2024-01-15T18:25:19.934 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44991.log 2024-01-15T18:25:19.934 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40698.log.gz 2024-01-15T18:25:19.935 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36508.log 2024-01-15T18:25:19.935 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46293.log.gz 2024-01-15T18:25:19.936 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55091.log 2024-01-15T18:25:19.936 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44991.log.gz 2024-01-15T18:25:19.937 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27345.log 2024-01-15T18:25:19.937 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37930.log 2024-01-15T18:25:19.938 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36508.log.gz 2024-01-15T18:25:19.940 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27393.log 2024-01-15T18:25:19.940 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27345.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-15T18:25:19.940 INFO:teuthology.orchestra.run.smithi204.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27345.log.gz 2024-01-15T18:25:19.940 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27393.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-15T18:25:19.940 INFO:teuthology.orchestra.run.smithi204.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27393.log.gz 2024-01-15T18:25:19.941 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33172.log 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55091.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.55091.log.gz -- 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.51801.log 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48273.log 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37930.log.gz 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34663.log 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51801.log.gz 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-01-15T18:25:19.942 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48273.log.gz 2024-01-15T18:25:19.943 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.34663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37699.log 2024-01-15T18:25:19.946 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34321.log 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.33172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33935.log 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.1.33172.log.gz 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.34321.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.33935.log: 92.7% -- replaced with /var/log/ceph/ceph-client.1.34321.log.gz 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33445.log 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr:gzip/var/log/ceph/ceph-osd.5.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27249.log 2024-01-15T18:25:19.947 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.33445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34481.log 2024-01-15T18:25:19.949 INFO:teuthology.orchestra.run.smithi143.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.34663.log.gz 2024-01-15T18:25:19.949 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54304.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53875.log 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.37699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47924.log 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54279.log 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.53875.log.gz 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38247.log 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47924.log.gz 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42496.log 2024-01-15T18:25:19.950 INFO:teuthology.orchestra.run.smithi143.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54279.log.gz 2024-01-15T18:25:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38247.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-15T18:25:19.951 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38247.log.gz 2024-01-15T18:25:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42496.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42496.log.gz 2024-01-15T18:25:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2024-01-15T18:25:19.951 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43305.log 2024-01-15T18:25:19.954 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27249.log: 92.9% -- replaced with /var/log/ceph/ceph-client.1.33445.log.gz 2024-01-15T18:25:19.954 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34160.log 2024-01-15T18:25:19.954 INFO:teuthology.orchestra.run.smithi204.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27249.log.gz 2024-01-15T18:25:19.955 INFO:teuthology.orchestra.run.smithi204.stderr: 96.1% -- replaced with /var/log/ceph/ceph-client.1.33935.log.gz 2024-01-15T18:25:19.955 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.34481.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-15T18:25:19.955 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.34160.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-15T18:25:19.955 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mgr.x.log: 93.3% -- replaced with /var/log/ceph/ceph-client.1.34160.log.gz 2024-01-15T18:25:19.955 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-15T18:25:19.956 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37068.log 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43305.log.gz 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46139.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37386.log 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37068.log.gz 2024-01-15T18:25:19.957 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44810.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-01-15T18:25:19.958 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36664.log 2024-01-15T18:25:19.958 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50666.log 2024-01-15T18:25:19.958 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37386.log.gz 2024-01-15T18:25:19.962 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36664.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36664.log.gz 2024-01-15T18:25:19.962 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44040.log 2024-01-15T18:25:19.962 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39204.log 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50666.log.gz 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44040.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44040.log.gz 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33444.log 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41390.log 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39204.log.gz 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33444.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33444.log.gz 2024-01-15T18:25:19.963 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52174.log 2024-01-15T18:25:19.964 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27297.log 2024-01-15T18:25:19.964 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.c.log: 96.3% -- replaced with /var/log/ceph/ceph-client.1.34481.log.gz 2024-01-15T18:25:19.964 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33770.log 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27297.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27297.log.gz 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33616.log 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.1.33770.log: /var/log/ceph/ceph-client.1.33616.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.1.33770.log.gz 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27321.log 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.1.33616.log.gz 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27417.log 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27321.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-15T18:25:19.965 INFO:teuthology.orchestra.run.smithi204.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27321.log.gz 2024-01-15T18:25:19.966 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27417.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.27417.log.gz 2024-01-15T18:25:19.968 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35237.log 2024-01-15T18:25:19.968 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41390.log.gz 2024-01-15T18:25:19.968 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52174.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52174.log.gz 2024-01-15T18:25:19.968 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43589.log 2024-01-15T18:25:19.969 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35637.log 2024-01-15T18:25:19.969 INFO:teuthology.orchestra.run.smithi143.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.35237.log.gz 2024-01-15T18:25:19.969 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42895.log 2024-01-15T18:25:19.969 INFO:teuthology.orchestra.run.smithi143.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43589.log.gz 2024-01-15T18:25:19.969 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-01-15T18:25:19.969 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35637.log.gz 2024-01-15T18:25:19.972 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42895.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.42895.log.gz 2024-01-15T18:25:19.972 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.52226.log 2024-01-15T18:25:19.972 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54909.log 2024-01-15T18:25:19.972 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-01-15T18:25:19.972 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52226.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52226.log.gz 2024-01-15T18:25:19.972 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39527.log 2024-01-15T18:25:19.973 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27369.log 2024-01-15T18:25:19.973 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33284.log 2024-01-15T18:25:19.973 INFO:teuthology.orchestra.run.smithi204.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-15T18:25:19.973 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27369.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-15T18:25:19.973 INFO:teuthology.orchestra.run.smithi204.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27369.log.gz 2024-01-15T18:25:19.974 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.3.33284.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-15T18:25:19.974 INFO:teuthology.orchestra.run.smithi204.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.3.33284.log.gz 2024-01-15T18:25:19.974 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-15T18:25:19.974 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mds.b.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-15T18:25:19.975 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-15T18:25:19.975 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27273.log 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49770.log 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54909.log.gz 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39527.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39527.log.gz 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51669.log 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40934.log 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49770.log.gz 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51669.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51669.log.gz 2024-01-15T18:25:19.977 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-01-15T18:25:19.978 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53711.log 2024-01-15T18:25:19.979 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40934.log.gz 2024-01-15T18:25:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49429.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-01-15T18:25:19.979 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54331.log 2024-01-15T18:25:19.980 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43383.log 2024-01-15T18:25:19.980 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53711.log.gz 2024-01-15T18:25:19.981 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-osd.4.log: 92.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-15T18:25:19.981 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/ceph-client.admin.27273.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27273.log.gz 2024-01-15T18:25:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54331.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54331.log.gz 2024-01-15T18:25:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52855.log 2024-01-15T18:25:19.982 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34120.log 2024-01-15T18:25:19.982 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz 2024-01-15T18:25:19.983 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52855.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52855.log.gz 2024-01-15T18:25:19.983 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40724.log 2024-01-15T18:25:19.984 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49905.log 2024-01-15T18:25:19.984 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34120.log.gz 2024-01-15T18:25:19.985 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40724.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40724.log.gz 2024-01-15T18:25:19.985 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42716.log 2024-01-15T18:25:19.986 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45955.log 2024-01-15T18:25:19.986 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49905.log.gz 2024-01-15T18:25:19.987 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42716.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42716.log.gz 2024-01-15T18:25:19.987 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46806.log 2024-01-15T18:25:19.988 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33313.log 2024-01-15T18:25:19.988 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45955.log.gz 2024-01-15T18:25:19.989 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49561.log 2024-01-15T18:25:19.990 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46806.log.gz 2024-01-15T18:25:19.990 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40672.log 2024-01-15T18:25:19.991 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33313.log.gz 2024-01-15T18:25:19.992 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49561.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49561.log.gz 2024-01-15T18:25:19.992 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39991.log 2024-01-15T18:25:19.993 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42999.log 2024-01-15T18:25:19.993 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40672.log.gz 2024-01-15T18:25:19.994 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39991.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39991.log.gz 2024-01-15T18:25:19.994 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-15T18:25:19.995 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41279.log 2024-01-15T18:25:19.995 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42999.log.gz 2024-01-15T18:25:19.996 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38455.log 2024-01-15T18:25:19.997 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41279.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41279.log.gz 2024-01-15T18:25:19.998 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52330.log 2024-01-15T18:25:19.999 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33392.log 2024-01-15T18:25:19.999 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38455.log.gz 2024-01-15T18:25:20.000 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52330.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52330.log.gz 2024-01-15T18:25:20.000 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50406.log 2024-01-15T18:25:20.001 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-01-15T18:25:20.001 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33392.log.gz 2024-01-15T18:25:20.002 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50406.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50406.log.gz 2024-01-15T18:25:20.002 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48642.log 2024-01-15T18:25:20.003 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34302.log 2024-01-15T18:25:20.004 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-01-15T18:25:20.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48642.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48642.log.gz 2024-01-15T18:25:20.004 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38567.log 2024-01-15T18:25:20.005 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48442.log 2024-01-15T18:25:20.006 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38567.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34302.log.gz 2024-01-15T18:25:20.006 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38567.log.gz 2024-01-15T18:25:20.007 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41175.log 2024-01-15T18:25:20.007 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54582.log 2024-01-15T18:25:20.008 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48442.log.gz 2024-01-15T18:25:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41175.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41175.log.gz 2024-01-15T18:25:20.009 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50164.log 2024-01-15T18:25:20.010 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36742.log 2024-01-15T18:25:20.010 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54582.log.gz 2024-01-15T18:25:20.010 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50164.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50164.log.gz 2024-01-15T18:25:20.011 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50510.log 2024-01-15T18:25:20.012 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51827.log 2024-01-15T18:25:20.012 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36742.log.gz 2024-01-15T18:25:20.013 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50510.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50510.log.gz 2024-01-15T18:25:20.013 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2024-01-15T18:25:20.014 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2024-01-15T18:25:20.015 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51827.log.gz 2024-01-15T18:25:20.016 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46385.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2024-01-15T18:25:20.016 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38014.log 2024-01-15T18:25:20.016 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33204.log 2024-01-15T18:25:20.017 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz 2024-01-15T18:25:20.018 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38014.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38014.log.gz 2024-01-15T18:25:20.018 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-01-15T18:25:20.019 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42522.log 2024-01-15T18:25:20.020 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33204.log.gz 2024-01-15T18:25:20.020 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-01-15T18:25:20.020 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41592.log 2024-01-15T18:25:20.021 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48140.log 2024-01-15T18:25:20.022 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42522.log.gz 2024-01-15T18:25:20.022 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41592.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41592.log.gz 2024-01-15T18:25:20.023 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41540.log 2024-01-15T18:25:20.023 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47638.log 2024-01-15T18:25:20.025 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48140.log.gz 2024-01-15T18:25:20.025 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41540.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41540.log.gz 2024-01-15T18:25:20.025 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46690.log 2024-01-15T18:25:20.026 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2024-01-15T18:25:20.026 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47638.log.gz 2024-01-15T18:25:20.027 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49301.log 2024-01-15T18:25:20.027 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46690.log.gz 2024-01-15T18:25:20.028 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46033.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2024-01-15T18:25:20.028 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34925.log 2024-01-15T18:25:20.029 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-01-15T18:25:20.030 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34925.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49301.log.gz 2024-01-15T18:25:20.030 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34925.log.gz 2024-01-15T18:25:20.031 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54934.log 2024-01-15T18:25:20.032 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52008.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-01-15T18:25:20.032 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32929.log 2024-01-15T18:25:20.034 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54684.log 2024-01-15T18:25:20.034 INFO:teuthology.orchestra.run.smithi143.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54934.log.gz 2024-01-15T18:25:20.035 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.32929.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32929.log.gz 2024-01-15T18:25:20.035 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44237.log 2024-01-15T18:25:20.036 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53581.log 2024-01-15T18:25:20.036 INFO:teuthology.orchestra.run.smithi143.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54684.log.gz 2024-01-15T18:25:20.037 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44237.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44237.log.gz 2024-01-15T18:25:20.037 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54204.log 2024-01-15T18:25:20.038 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39290.log 2024-01-15T18:25:20.039 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53581.log.gz 2024-01-15T18:25:20.040 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54204.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54204.log.gz 2024-01-15T18:25:20.040 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33482.log 2024-01-15T18:25:20.041 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33492.log 2024-01-15T18:25:20.041 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39290.log.gz 2024-01-15T18:25:20.042 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33482.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33482.log.gz 2024-01-15T18:25:20.042 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-01-15T18:25:20.043 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34432.log 2024-01-15T18:25:20.044 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33492.log.gz 2024-01-15T18:25:20.044 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42947.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz 2024-01-15T18:25:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38887.log 2024-01-15T18:25:20.045 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45981.log 2024-01-15T18:25:20.046 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34432.log.gz 2024-01-15T18:25:20.047 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38887.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38887.log.gz 2024-01-15T18:25:20.047 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55332.log 2024-01-15T18:25:20.048 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45981.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45981.log.gz 2024-01-15T18:25:20.048 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48221.log 2024-01-15T18:25:20.049 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50009.log 2024-01-15T18:25:20.050 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55332.log.gz 2024-01-15T18:25:20.050 INFO:teuthology.orchestra.run.smithi143.stderr:gzip/var/log/ceph/ceph-client.admin.48221.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45251.log 2024-01-15T18:25:20.051 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48221.log.gz 2024-01-15T18:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56109.log 2024-01-15T18:25:20.052 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50009.log.gz 2024-01-15T18:25:20.053 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32763.log 2024-01-15T18:25:20.053 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45251.log.gz 2024-01-15T18:25:20.054 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35185.log 2024-01-15T18:25:20.054 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56109.log.gz 2024-01-15T18:25:20.055 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.32763.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32763.log.gz 2024-01-15T18:25:20.056 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54002.log 2024-01-15T18:25:20.056 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45408.log 2024-01-15T18:25:20.057 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35185.log.gz 2024-01-15T18:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54002.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54002.log.gz 2024-01-15T18:25:20.058 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42358.log 2024-01-15T18:25:20.059 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51408.log 2024-01-15T18:25:20.059 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45408.log.gz 2024-01-15T18:25:20.060 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42358.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42358.log.gz 2024-01-15T18:25:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47586.log 2024-01-15T18:25:20.061 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-01-15T18:25:20.062 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51408.log.gz 2024-01-15T18:25:20.062 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47586.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47586.log.gz 2024-01-15T18:25:20.063 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45225.log 2024-01-15T18:25:20.064 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33691.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22230.log 2024-01-15T18:25:20.064 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-01-15T18:25:20.065 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45225.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45225.log.gz 2024-01-15T18:25:20.065 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53191.log 2024-01-15T18:25:20.066 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.tmp-client.admin.22230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34250.log 2024-01-15T18:25:20.066 INFO:teuthology.orchestra.run.smithi143.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22230.log.gz 2024-01-15T18:25:20.067 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53191.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53191.log.gz 2024-01-15T18:25:20.067 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37094.log 2024-01-15T18:25:20.068 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-01-15T18:25:20.069 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34250.log.gz 2024-01-15T18:25:20.069 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37094.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37094.log.gz 2024-01-15T18:25:20.069 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36060.log 2024-01-15T18:25:20.071 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49744.log 2024-01-15T18:25:20.071 INFO:teuthology.orchestra.run.smithi143.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-01-15T18:25:20.071 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36060.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36060.log.gz 2024-01-15T18:25:20.072 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38913.log 2024-01-15T18:25:20.073 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34172.log 2024-01-15T18:25:20.073 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49744.log.gz 2024-01-15T18:25:20.074 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38913.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38913.log.gz 2024-01-15T18:25:20.074 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37956.log 2024-01-15T18:25:20.075 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38116.log 2024-01-15T18:25:20.075 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34172.log.gz 2024-01-15T18:25:20.076 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37956.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37956.log.gz 2024-01-15T18:25:20.076 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54431.log 2024-01-15T18:25:20.077 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48560.log 2024-01-15T18:25:20.078 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38116.log.gz 2024-01-15T18:25:20.078 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54431.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.54431.log.gz 2024-01-15T18:25:20.079 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45474.log 2024-01-15T18:25:20.079 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39449.log 2024-01-15T18:25:20.080 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48560.log.gz 2024-01-15T18:25:20.080 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45474.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45474.log.gz 2024-01-15T18:25:20.081 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-01-15T18:25:20.082 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44066.log 2024-01-15T18:25:20.082 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39449.log.gz 2024-01-15T18:25:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45017.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-01-15T18:25:20.083 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39397.log 2024-01-15T18:25:20.084 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34484.log 2024-01-15T18:25:20.084 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44066.log.gz 2024-01-15T18:25:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39397.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39397.log.gz 2024-01-15T18:25:20.085 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35558.log 2024-01-15T18:25:20.086 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-01-15T18:25:20.087 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35558.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.34484.log.gz 2024-01-15T18:25:20.088 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35558.log.gz 2024-01-15T18:25:20.088 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45903.log 2024-01-15T18:25:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36034.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-01-15T18:25:20.089 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32864.log 2024-01-15T18:25:20.090 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54079.log 2024-01-15T18:25:20.090 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45903.log.gz 2024-01-15T18:25:20.091 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.32864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52777.log 2024-01-15T18:25:20.092 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32864.log.gz 2024-01-15T18:25:20.092 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54079.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54079.log.gz 2024-01-15T18:25:20.093 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38783.log 2024-01-15T18:25:20.094 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56001.log 2024-01-15T18:25:20.094 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52777.log.gz 2024-01-15T18:25:20.095 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42626.log 2024-01-15T18:25:20.095 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38783.log.gz 2024-01-15T18:25:20.096 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56001.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56001.log.gz 2024-01-15T18:25:20.096 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35506.log 2024-01-15T18:25:20.097 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46610.log 2024-01-15T18:25:20.098 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42626.log: /var/log/ceph/ceph-client.admin.35506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42626.log.gz 2024-01-15T18:25:20.099 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35506.log.gz 2024-01-15T18:25:20.099 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33287.log 2024-01-15T18:25:20.100 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39371.log 2024-01-15T18:25:20.100 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46610.log.gz 2024-01-15T18:25:20.101 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33287.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33287.log.gz 2024-01-15T18:25:20.102 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55384.log 2024-01-15T18:25:20.102 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42834.log 2024-01-15T18:25:20.103 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39371.log.gz 2024-01-15T18:25:20.104 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55384.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55384.log.gz 2024-01-15T18:25:20.104 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38403.log 2024-01-15T18:25:20.105 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33261.log 2024-01-15T18:25:20.105 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42834.log.gz 2024-01-15T18:25:20.106 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38403.log.gz 2024-01-15T18:25:20.107 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47872.log 2024-01-15T18:25:20.107 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49718.log 2024-01-15T18:25:20.108 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33261.log.gz 2024-01-15T18:25:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47872.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47872.log.gz 2024-01-15T18:25:20.109 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42097.log 2024-01-15T18:25:20.110 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49718.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49718.log.gz 2024-01-15T18:25:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-01-15T18:25:20.111 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46528.log 2024-01-15T18:25:20.112 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42097.log.gz 2024-01-15T18:25:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-01-15T18:25:20.113 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34797.log 2024-01-15T18:25:20.114 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55117.log 2024-01-15T18:25:20.114 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46528.log.gz 2024-01-15T18:25:20.115 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.2.34797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55871.log 2024-01-15T18:25:20.116 INFO:teuthology.orchestra.run.smithi143.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.2.34797.log.gz 2024-01-15T18:25:20.116 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-01-15T18:25:20.117 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55117.log.gz 2024-01-15T18:25:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55871.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55871.log.gz 2024-01-15T18:25:20.118 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43641.log 2024-01-15T18:25:20.119 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50692.log 2024-01-15T18:25:20.120 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-01-15T18:25:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43641.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43641.log.gz 2024-01-15T18:25:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47016.log 2024-01-15T18:25:20.122 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50035.log 2024-01-15T18:25:20.122 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50692.log.gz 2024-01-15T18:25:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47016.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47016.log.gz 2024-01-15T18:25:20.123 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-01-15T18:25:20.124 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42205.log 2024-01-15T18:25:20.124 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50035.log.gz 2024-01-15T18:25:20.125 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45773.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-01-15T18:25:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-15T18:25:20.126 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36326.log 2024-01-15T18:25:20.127 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42205.log.gz 2024-01-15T18:25:20.128 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33491.log 2024-01-15T18:25:20.129 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39553.log 2024-01-15T18:25:20.130 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36326.log.gz 2024-01-15T18:25:20.130 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52699.log 2024-01-15T18:25:20.131 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33491.log.gz 2024-01-15T18:25:20.132 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39553.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39553.log.gz 2024-01-15T18:25:20.133 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49223.log 2024-01-15T18:25:20.133 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52699.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52699.log.gz 2024-01-15T18:25:20.134 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33034.log 2024-01-15T18:25:20.135 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-01-15T18:25:20.135 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49223.log.gz 2024-01-15T18:25:20.136 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35876.log 2024-01-15T18:25:20.136 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33034.log.gz 2024-01-15T18:25:20.137 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39579.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz --verbose 2024-01-15T18:25:20.138 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.37172.log 2024-01-15T18:25:20.139 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46502.log 2024-01-15T18:25:20.139 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35876.log.gz 2024-01-15T18:25:20.140 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34899.log 2024-01-15T18:25:20.140 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37172.log.gz 2024-01-15T18:25:20.142 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33901.log 2024-01-15T18:25:20.142 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46502.log.gz 2024-01-15T18:25:20.143 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55039.log 2024-01-15T18:25:20.143 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34899.log.gz 2024-01-15T18:25:20.144 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35532.log 2024-01-15T18:25:20.144 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33901.log.gz 2024-01-15T18:25:20.145 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42548.log 2024-01-15T18:25:20.146 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55039.log.gz 2024-01-15T18:25:20.147 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40541.log 2024-01-15T18:25:20.147 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35532.log.gz 2024-01-15T18:25:20.148 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40241.log 2024-01-15T18:25:20.148 INFO:teuthology.orchestra.run.smithi143.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42548.log.gz 2024-01-15T18:25:20.150 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-01-15T18:25:20.150 INFO:teuthology.orchestra.run.smithi143.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40541.log.gz 2024-01-15T18:25:20.151 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56165.log 2024-01-15T18:25:20.151 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40241.log.gz 2024-01-15T18:25:20.152 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35772.log 2024-01-15T18:25:20.152 INFO:teuthology.orchestra.run.smithi143.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-01-15T18:25:20.153 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27319.log 2024-01-15T18:25:20.153 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56165.log.gz 2024-01-15T18:25:20.155 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52093.log 2024-01-15T18:25:20.155 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35772.log.gz 2024-01-15T18:25:20.156 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.27319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37464.log 2024-01-15T18:25:20.156 INFO:teuthology.orchestra.run.smithi143.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27319.log.gz 2024-01-15T18:25:20.157 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37594.log 2024-01-15T18:25:20.157 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52093.log.gz 2024-01-15T18:25:20.158 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48614.log 2024-01-15T18:25:20.159 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37464.log.gz 2024-01-15T18:25:20.160 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46113.log 2024-01-15T18:25:20.161 INFO:teuthology.orchestra.run.smithi143.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37594.log.gz 2024-01-15T18:25:20.161 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51877.log 2024-01-15T18:25:20.161 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48614.log.gz 2024-01-15T18:25:20.163 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52460.log 2024-01-15T18:25:20.163 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46113.log.gz 2024-01-15T18:25:20.164 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2024-01-15T18:25:20.164 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51877.log.gz 2024-01-15T18:25:20.165 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-01-15T18:25:20.165 INFO:teuthology.orchestra.run.smithi143.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52460.log.gz 2024-01-15T18:25:20.167 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36560.log 2024-01-15T18:25:20.167 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz 2024-01-15T18:25:20.168 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-01-15T18:25:20.168 INFO:teuthology.orchestra.run.smithi143.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-01-15T18:25:20.169 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2024-01-15T18:25:20.169 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36560.log.gz 2024-01-15T18:25:20.171 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42179.log 2024-01-15T18:25:20.171 INFO:teuthology.orchestra.run.smithi143.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-01-15T18:25:20.172 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45746.log 2024-01-15T18:25:20.172 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gz 2024-01-15T18:25:20.173 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42179.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-15T18:25:20.174 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42179.log.gz 2024-01-15T18:25:20.174 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33481.log 2024-01-15T18:25:20.175 INFO:teuthology.orchestra.run.smithi143.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45746.log.gz 2024-01-15T18:25:20.175 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33495.log 2024-01-15T18:25:20.177 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55689.log 2024-01-15T18:25:20.177 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33481.log.gz 2024-01-15T18:25:20.179 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33086.log 2024-01-15T18:25:20.179 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33495.log.gz 2024-01-15T18:25:20.180 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55689.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55689.log.gz 2024-01-15T18:25:20.180 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46716.log 2024-01-15T18:25:20.181 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33086.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33086.log.gz 2024-01-15T18:25:20.182 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42332.log 2024-01-15T18:25:20.183 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46716.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-client.admin.55557.log 2024-01-15T18:25:20.183 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46716.log.gz 2024-01-15T18:25:20.184 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51073.log 2024-01-15T18:25:20.185 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42332.log.gz 2024-01-15T18:25:20.186 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55557.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55557.log.gz 2024-01-15T18:25:20.187 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51073.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51073.log.gz 2024-01-15T18:25:20.188 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42243.log 2024-01-15T18:25:20.189 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55611.log 2024-01-15T18:25:20.190 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38757.log 2024-01-15T18:25:20.190 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42243.log.gz 2024-01-15T18:25:20.192 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55611.log: gzip -5 88.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.55611.log.gz -- 2024-01-15T18:25:20.192 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.35317.log 2024-01-15T18:25:20.193 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-01-15T18:25:20.193 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38757.log.gz 2024-01-15T18:25:20.194 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33497.log 2024-01-15T18:25:20.195 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35317.log.gz 2024-01-15T18:25:20.196 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47979.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-01-15T18:25:20.196 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.41305.log 2024-01-15T18:25:20.197 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41620.log 2024-01-15T18:25:20.197 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33497.log.gz 2024-01-15T18:25:20.199 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54254.log 2024-01-15T18:25:20.199 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-01-15T18:25:20.200 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55462.log 2024-01-15T18:25:20.200 INFO:teuthology.orchestra.run.smithi143.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41620.log.gz 2024-01-15T18:25:20.201 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54254.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54254.log.gz 2024-01-15T18:25:20.202 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43615.log 2024-01-15T18:25:20.202 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55462.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55462.log.gz 2024-01-15T18:25:20.203 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38220.log 2024-01-15T18:25:20.204 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-01-15T18:25:20.205 INFO:teuthology.orchestra.run.smithi143.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43615.log.gz 2024-01-15T18:25:20.206 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38220.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38220.log.gz 2024-01-15T18:25:20.207 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48756.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-01-15T18:25:20.208 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52803.log 2024-01-15T18:25:20.209 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51446.log 2024-01-15T18:25:20.211 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-01-15T18:25:20.211 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52803.log.gz 2024-01-15T18:25:20.212 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51446.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51446.log.gz 2024-01-15T18:25:20.213 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47404.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-01-15T18:25:20.213 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50995.log 2024-01-15T18:25:20.215 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41892.log 2024-01-15T18:25:20.216 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27202.log 2024-01-15T18:25:20.216 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50995.log.gz 2024-01-15T18:25:20.217 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49365.log 2024-01-15T18:25:20.218 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41892.log.gz 2024-01-15T18:25:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.27202.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27202.log.gz 2024-01-15T18:25:20.219 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36924.log 2024-01-15T18:25:20.221 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50640.log 2024-01-15T18:25:20.221 INFO:teuthology.orchestra.run.smithi143.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49365.log.gz 2024-01-15T18:25:20.222 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36924.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36924.log.gz 2024-01-15T18:25:20.222 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43059.log 2024-01-15T18:25:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50640.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50640.log.gz 2024-01-15T18:25:20.223 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52067.log 2024-01-15T18:25:20.225 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38090.log 2024-01-15T18:25:20.225 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43059.log.gz 2024-01-15T18:25:20.227 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52067.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52067.log.gz 2024-01-15T18:25:20.228 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38090.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38090.log.gz 2024-01-15T18:25:20.228 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48862.log 2024-01-15T18:25:20.230 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36086.log 2024-01-15T18:25:20.231 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42049.log 2024-01-15T18:25:20.231 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48862.log.gz 2024-01-15T18:25:20.232 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46449.log 2024-01-15T18:25:20.233 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36086.log.gz 2024-01-15T18:25:20.236 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.42049.log: /var/log/ceph/ceph-client.admin.46449.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46449.log.gz 2024-01-15T18:25:20.236 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54179.log 2024-01-15T18:25:20.237 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41566.log 2024-01-15T18:25:20.238 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54179.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54179.log.gz 2024-01-15T18:25:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41566.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41566.log.gz 2024-01-15T18:25:20.240 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52673.log 2024-01-15T18:25:20.242 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33695.log 2024-01-15T18:25:20.243 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52673.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52673.log.gz 2024-01-15T18:25:20.244 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33695.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33695.log.gz 2024-01-15T18:25:20.245 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-01-15T18:25:20.246 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-01-15T18:25:20.248 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49801.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-01-15T18:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42806.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-01-15T18:25:20.249 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34094.log 2024-01-15T18:25:20.251 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52363.log 2024-01-15T18:25:20.252 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34094.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34094.log.gz 2024-01-15T18:25:20.253 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52363.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52363.log.gz 2024-01-15T18:25:20.254 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49827.log 2024-01-15T18:25:20.255 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40593.log 2024-01-15T18:25:20.257 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49827.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49827.log.gz 2024-01-15T18:25:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40593.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40593.log.gz 2024-01-15T18:25:20.258 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51204.log 2024-01-15T18:25:20.260 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45877.log 2024-01-15T18:25:20.263 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.51204.log: /var/log/ceph/ceph-client.admin.45877.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2024-01-15T18:25:20.263 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38702.log 2024-01-15T18:25:20.266 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38702.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38702.log.gz 2024-01-15T18:25:20.266 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37893.log 2024-01-15T18:25:20.269 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-01-15T18:25:20.270 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37893.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37893.log.gz 2024-01-15T18:25:20.272 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45825.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-01-15T18:25:20.273 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37542.log 2024-01-15T18:25:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37542.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37542.log.gz 2024-01-15T18:25:20.276 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54759.log 2024-01-15T18:25:20.279 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54759.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54759.log.gz 2024-01-15T18:25:20.279 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-01-15T18:25:20.282 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54809.log 2024-01-15T18:25:20.282 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48966.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-01-15T18:25:20.286 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54809.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54809.log.gz 2024-01-15T18:25:20.286 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38194.log 2024-01-15T18:25:20.288 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38194.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38194.log.gz 2024-01-15T18:25:20.289 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48888.log 2024-01-15T18:25:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48888.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48888.log.gz 2024-01-15T18:25:20.292 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-01-15T18:25:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44436.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-01-15T18:25:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55897.log 2024-01-15T18:25:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55897.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55897.log.gz 2024-01-15T18:25:20.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39758.log 2024-01-15T18:25:20.300 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54456.log 2024-01-15T18:25:20.301 INFO:teuthology.orchestra.run.smithi143.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39758.log.gz 2024-01-15T18:25:20.304 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-01-15T18:25:20.305 INFO:teuthology.orchestra.run.smithi143.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.54456.log.gz 2024-01-15T18:25:20.307 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35980.log 2024-01-15T18:25:20.312 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44550.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-01-15T18:25:20.312 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35980.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35980.log.gz 2024-01-15T18:25:20.312 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49044.log 2024-01-15T18:25:20.315 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49044.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49044.log.gz 2024-01-15T18:25:20.315 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50889.log 2024-01-15T18:25:20.318 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-01-15T18:25:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36820.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz 2024-01-15T18:25:20.321 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52304.log 2024-01-15T18:25:20.322 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50889.log.gz 2024-01-15T18:25:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52304.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52304.log.gz 2024-01-15T18:25:20.325 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-01-15T18:25:20.331 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48992.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-01-15T18:25:20.332 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-01-15T18:25:20.332 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45694.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-01-15T18:25:20.332 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37982.log 2024-01-15T18:25:20.335 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37982.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37982.log.gz 2024-01-15T18:25:20.335 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53775.log 2024-01-15T18:25:20.338 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53775.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53775.log.gz 2024-01-15T18:25:20.338 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55949.log 2024-01-15T18:25:20.339 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-01-15T18:25:20.345 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48351.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-01-15T18:25:20.345 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55949.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55949.log.gz 2024-01-15T18:25:20.346 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55663.log 2024-01-15T18:25:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55663.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55663.log.gz 2024-01-15T18:25:20.348 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40752.log 2024-01-15T18:25:20.350 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36352.log 2024-01-15T18:25:20.352 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40752.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40752.log.gz 2024-01-15T18:25:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36352.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36352.log.gz 2024-01-15T18:25:20.355 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-01-15T18:25:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-01-15T18:25:20.358 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34380.log 2024-01-15T18:25:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52881.log 2024-01-15T18:25:20.361 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34380.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34380.log.gz 2024-01-15T18:25:20.364 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52881.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52881.log.gz 2024-01-15T18:25:20.364 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52751.log 2024-01-15T18:25:20.368 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52751.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52751.log.gz 2024-01-15T18:25:20.368 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35824.log 2024-01-15T18:25:20.371 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37644.log 2024-01-15T18:25:20.371 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35824.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35824.log.gz 2024-01-15T18:25:20.374 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37644.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37644.log.gz 2024-01-15T18:25:20.374 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53165.log 2024-01-15T18:25:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53165.log.gz 2024-01-15T18:25:20.377 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40151.log 2024-01-15T18:25:20.380 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40151.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40151.log.gz 2024-01-15T18:25:20.381 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35421.log 2024-01-15T18:25:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34977.log 2024-01-15T18:25:20.384 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35421.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35421.log.gz 2024-01-15T18:25:20.386 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53117.log 2024-01-15T18:25:20.387 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34977.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34977.log.gz 2024-01-15T18:25:20.394 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.53117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40047.log 2024-01-15T18:25:20.399 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33682.log 2024-01-15T18:25:20.400 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40047.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40047.log.gz 2024-01-15T18:25:20.402 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47146.log 2024-01-15T18:25:20.406 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33682.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33682.log.gz 2024-01-15T18:25:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47146.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47146.log.gz 2024-01-15T18:25:20.409 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48083.log 2024-01-15T18:25:20.415 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46584.log 2024-01-15T18:25:20.416 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48083.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48083.log.gz 2024-01-15T18:25:20.421 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46584.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46584.log.gz -- 2024-01-15T18:25:20.422 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.36690.log 2024-01-15T18:25:20.428 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36690.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36690.log.gz 2024-01-15T18:25:20.428 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2024-01-15T18:25:20.436 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46858.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gz 2024-01-15T18:25:20.436 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38142.log 2024-01-15T18:25:20.444 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38142.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38142.log.gz 2024-01-15T18:25:20.444 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35954.log 2024-01-15T18:25:20.451 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43112.log 2024-01-15T18:25:20.452 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35954.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35954.log.gz 2024-01-15T18:25:20.455 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35003.log 2024-01-15T18:25:20.456 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43112.log.gz 2024-01-15T18:25:20.458 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-01-15T18:25:20.464 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35003.log: 88.8%/var/log/ceph/ceph-client.admin.54986.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-01-15T18:25:20.465 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35003.log.gz 2024-01-15T18:25:20.466 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54557.log 2024-01-15T18:25:20.470 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42860.log 2024-01-15T18:25:20.472 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54557.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54557.log.gz 2024-01-15T18:25:20.474 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37568.log 2024-01-15T18:25:20.476 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42860.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42860.log.gz 2024-01-15T18:25:20.477 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47042.log 2024-01-15T18:25:20.483 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37568.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37568.log.gz 2024-01-15T18:25:20.484 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47768.log 2024-01-15T18:25:20.484 INFO:teuthology.orchestra.run.smithi143.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47042.log.gz 2024-01-15T18:25:20.486 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38978.log 2024-01-15T18:25:20.488 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47768.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47768.log.gz 2024-01-15T18:25:20.493 INFO:teuthology.orchestra.run.smithi204.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-15T18:25:20.495 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38978.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38978.log.gz 2024-01-15T18:25:20.496 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34406.log 2024-01-15T18:25:20.498 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40267.log 2024-01-15T18:25:20.500 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34406.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34406.log.gz 2024-01-15T18:25:20.507 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40267.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40267.log.gz 2024-01-15T18:25:20.508 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-01-15T18:25:20.509 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-15T18:25:20.510 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-15T18:25:20.512 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43881.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-01-15T18:25:20.518 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54104.log 2024-01-15T18:25:20.523 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34841.log 2024-01-15T18:25:20.524 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54104.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54104.log.gz 2024-01-15T18:25:20.526 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53976.log 2024-01-15T18:25:20.528 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34841.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34841.log.gz 2024-01-15T18:25:20.532 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53976.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.53976.log.gz 2024-01-15T18:25:20.533 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41094.log 2024-01-15T18:25:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55358.log 2024-01-15T18:25:20.540 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41094.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41094.log.gz 2024-01-15T18:25:20.542 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56135.log 2024-01-15T18:25:20.544 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55358.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55358.log.gz 2024-01-15T18:25:20.547 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50969.log 2024-01-15T18:25:20.548 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56135.log.gz 2024-01-15T18:25:20.550 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2024-01-15T18:25:20.552 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50969.log.gz 2024-01-15T18:25:20.554 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42306.log 2024-01-15T18:25:20.556 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44734.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2024-01-15T18:25:20.558 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33339.log 2024-01-15T18:25:20.560 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42306.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42306.log.gz 2024-01-15T18:25:20.564 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33339.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33339.log.gz 2024-01-15T18:25:20.565 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2024-01-15T18:25:20.570 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37224.log 2024-01-15T18:25:20.572 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40567.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2024-01-15T18:25:20.574 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38505.log 2024-01-15T18:25:20.576 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37224.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37224.log.gz 2024-01-15T18:25:20.578 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52647.log 2024-01-15T18:25:20.580 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38505.log.gz 2024-01-15T18:25:20.582 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41011.log 2024-01-15T18:25:20.584 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52647.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52647.log.gz 2024-01-15T18:25:20.586 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47430.log 2024-01-15T18:25:20.588 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41011.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41011.log.gz 2024-01-15T18:25:20.591 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43185.log 2024-01-15T18:25:20.592 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47430.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47430.log.gz 2024-01-15T18:25:20.594 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-01-15T18:25:20.596 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43185.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43185.log.gz 2024-01-15T18:25:20.600 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36274.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-01-15T18:25:20.601 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52934.log 2024-01-15T18:25:20.606 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36612.log 2024-01-15T18:25:20.608 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52934.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52934.log.gz 2024-01-15T18:25:20.610 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32955.log 2024-01-15T18:25:20.612 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36612.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36612.log.gz 2024-01-15T18:25:20.614 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37799.log 2024-01-15T18:25:20.616 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.32955.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32955.log.gz 2024-01-15T18:25:20.618 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47250.log 2024-01-15T18:25:20.620 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37799.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37799.log.gz 2024-01-15T18:25:20.624 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47250.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47250.log.gz 2024-01-15T18:25:20.624 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36768.log 2024-01-15T18:25:20.630 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39965.log 2024-01-15T18:25:20.632 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36768.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36768.log.gz 2024-01-15T18:25:20.634 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45119.log 2024-01-15T18:25:20.636 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39965.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39965.log.gz 2024-01-15T18:25:20.638 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53243.log 2024-01-15T18:25:20.640 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45119.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45119.log.gz 2024-01-15T18:25:20.644 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53243.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53243.log.gz 2024-01-15T18:25:20.645 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50863.log 2024-01-15T18:25:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-01-15T18:25:20.653 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50863.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50863.log.gz 2024-01-15T18:25:20.654 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47068.log 2024-01-15T18:25:20.656 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48534.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-01-15T18:25:20.658 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53449.log 2024-01-15T18:25:20.660 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47068.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47068.log.gz 2024-01-15T18:25:20.662 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45145.log 2024-01-15T18:25:20.664 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53449.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53449.log.gz 2024-01-15T18:25:20.666 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46554.log 2024-01-15T18:25:20.668 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45145.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45145.log.gz 2024-01-15T18:25:20.670 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38809.log 2024-01-15T18:25:20.672 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46554.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46554.log.gz 2024-01-15T18:25:20.674 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35029.log 2024-01-15T18:25:20.676 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38809.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38809.log.gz 2024-01-15T18:25:20.680 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35029.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35029.log.gz 2024-01-15T18:25:20.682 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33904.log 2024-01-15T18:25:20.686 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-01-15T18:25:20.688 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33904.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33904.log.gz 2024-01-15T18:25:20.690 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54129.log 2024-01-15T18:25:20.692 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42780.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-01-15T18:25:20.695 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55845.log 2024-01-15T18:25:20.696 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54129.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54129.log.gz 2024-01-15T18:25:20.698 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44144.log 2024-01-15T18:25:20.700 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55845.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55845.log.gz 2024-01-15T18:25:20.702 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53295.log 2024-01-15T18:25:20.704 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44144.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44144.log.gz 2024-01-15T18:25:20.706 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2024-01-15T18:25:20.708 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53295.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53295.log.gz 2024-01-15T18:25:20.712 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51723.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2024-01-15T18:25:20.713 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42280.log 2024-01-15T18:25:20.718 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-01-15T18:25:20.720 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42280.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42280.log.gz 2024-01-15T18:25:20.722 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44170.log 2024-01-15T18:25:20.724 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-01-15T18:25:20.727 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42600.log 2024-01-15T18:25:20.728 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44170.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44170.log.gz 2024-01-15T18:25:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42600.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-01-15T18:25:20.732 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2024-01-15T18:25:20.738 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-01-15T18:25:20.740 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51695.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2024-01-15T18:25:20.741 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45278.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-01-15T18:25:20.742 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55923.log 2024-01-15T18:25:20.746 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34328.log 2024-01-15T18:25:20.748 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55923.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55923.log.gz 2024-01-15T18:25:20.752 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34328.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34328.log.gz 2024-01-15T18:25:20.753 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36865.log 2024-01-15T18:25:20.758 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55170.log 2024-01-15T18:25:20.760 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36865.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36865.log.gz 2024-01-15T18:25:20.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-01-15T18:25:20.763 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-01-15T18:25:20.764 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55170.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55170.log.gz 2024-01-15T18:25:20.767 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2024-01-15T18:25:20.768 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45851.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-01-15T18:25:20.770 INFO:teuthology.orchestra.run.smithi204.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-15T18:25:20.771 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40203.log 2024-01-15T18:25:20.772 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43225.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2024-01-15T18:25:20.776 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40203.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40203.log.gz 2024-01-15T18:25:20.777 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52122.log 2024-01-15T18:25:20.783 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35107.log 2024-01-15T18:25:20.784 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52122.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52122.log.gz 2024-01-15T18:25:20.786 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43719.log 2024-01-15T18:25:20.788 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-01-15T18:25:20.788 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35107.log.gz 2024-01-15T18:25:20.789 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43719.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43719.log.gz 2024-01-15T18:25:20.790 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42444.log 2024-01-15T18:25:20.797 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41814.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-01-15T18:25:20.797 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33913.log 2024-01-15T18:25:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42444.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42444.log.gz 2024-01-15T18:25:20.798 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33913.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33913.log.gz 2024-01-15T18:25:20.799 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54531.log 2024-01-15T18:25:20.804 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54531.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54531.log.gz 2024-01-15T18:25:20.805 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38861.log 2024-01-15T18:25:20.811 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37278.log 2024-01-15T18:25:20.812 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38861.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38861.log.gz 2024-01-15T18:25:20.814 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39345.log 2024-01-15T18:25:20.819 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37278.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51252.log 2024-01-15T18:25:20.820 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39345.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39345.log.gz 2024-01-15T18:25:20.822 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37278.log.gz 2024-01-15T18:25:20.822 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-15T18:25:20.824 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51252.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51252.log.gz 2024-01-15T18:25:20.826 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48914.log 2024-01-15T18:25:20.830 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39319.log 2024-01-15T18:25:20.832 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48914.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48914.log.gz 2024-01-15T18:25:20.836 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39319.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39319.log.gz 2024-01-15T18:25:20.837 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45434.log 2024-01-15T18:25:20.843 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55248.log 2024-01-15T18:25:20.844 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45434.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45434.log.gz 2024-01-15T18:25:20.846 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37747.log 2024-01-15T18:25:20.848 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55248.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55248.log.gz 2024-01-15T18:25:20.850 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-01-15T18:25:20.852 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37747.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37747.log.gz 2024-01-15T18:25:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47328.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2024-01-15T18:25:20.856 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54406.log 2024-01-15T18:25:20.862 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47846.log 2024-01-15T18:25:20.864 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54406.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54406.log.gz 2024-01-15T18:25:20.866 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52829.log 2024-01-15T18:25:20.868 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47846.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47846.log.gz 2024-01-15T18:25:20.870 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36456.log 2024-01-15T18:25:20.872 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52829.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52829.log.gz 2024-01-15T18:25:20.876 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36456.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36456.log.gz 2024-01-15T18:25:20.877 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52486.log 2024-01-15T18:25:20.882 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52148.log 2024-01-15T18:25:20.884 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52486.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52486.log.gz 2024-01-15T18:25:20.887 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55715.log 2024-01-15T18:25:20.888 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52148.log.gz 2024-01-15T18:25:20.890 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51304.log 2024-01-15T18:25:20.895 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55715.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55715.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44965.log 2024-01-15T18:25:20.896 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51304.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51304.log.gz 2024-01-15T18:25:20.897 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:20.900 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44965.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44965.log.gz 2024-01-15T18:25:20.901 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52539.log 2024-01-15T18:25:20.906 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33230.log 2024-01-15T18:25:20.908 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52539.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52539.log.gz 2024-01-15T18:25:20.910 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40409.log 2024-01-15T18:25:20.912 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33230.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33230.log.gz 2024-01-15T18:25:20.914 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55741.log 2024-01-15T18:25:20.916 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40409.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40409.log.gz 2024-01-15T18:25:20.918 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-01-15T18:25:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55741.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55741.log.gz 2024-01-15T18:25:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35663.log 2024-01-15T18:25:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44356.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-01-15T18:25:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35663.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35663.log.gz 2024-01-15T18:25:20.929 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51956.log 2024-01-15T18:25:20.934 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38533.log 2024-01-15T18:25:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51956.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51956.log.gz 2024-01-15T18:25:20.938 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2024-01-15T18:25:20.940 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38533.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38533.log.gz 2024-01-15T18:25:20.942 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40645.log 2024-01-15T18:25:20.944 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50773.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2024-01-15T18:25:20.946 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52512.log 2024-01-15T18:25:20.948 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40645.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40645.log.gz 2024-01-15T18:25:20.950 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41201.log 2024-01-15T18:25:20.952 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52512.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52512.log.gz 2024-01-15T18:25:20.954 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55488.log 2024-01-15T18:25:20.956 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41201.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-01-15T18:25:20.958 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41514.log 2024-01-15T18:25:20.960 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55488.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55488.log.gz 2024-01-15T18:25:20.964 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41514.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41514.log.gz 2024-01-15T18:25:20.966 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34539.log 2024-01-15T18:25:20.970 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36534.log 2024-01-15T18:25:20.972 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34539.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34539.log.gz 2024-01-15T18:25:20.974 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49983.log 2024-01-15T18:25:20.976 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36534.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36534.log.gz 2024-01-15T18:25:20.977 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42742.log 2024-01-15T18:25:20.983 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49983.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49983.log.gz 2024-01-15T18:25:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42742.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42742.log.gz 2024-01-15T18:25:20.984 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39784.log 2024-01-15T18:25:20.986 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52041.log 2024-01-15T18:25:20.989 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39784.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39784.log.gz 2024-01-15T18:25:20.990 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39423.log 2024-01-15T18:25:20.992 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52041.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52041.log.gz 2024-01-15T18:25:20.998 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49455.log 2024-01-15T18:25:21.000 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39423.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39423.log.gz 2024-01-15T18:25:21.002 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-01-15T18:25:21.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49455.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49455.log.gz 2024-01-15T18:25:21.008 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44939.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-01-15T18:25:21.009 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36139.log 2024-01-15T18:25:21.014 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-01-15T18:25:21.016 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36139.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36139.log.gz 2024-01-15T18:25:21.018 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34354.log 2024-01-15T18:25:21.021 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44760.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-01-15T18:25:21.022 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37516.log 2024-01-15T18:25:21.024 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34354.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34354.log.gz 2024-01-15T18:25:21.026 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35291.log 2024-01-15T18:25:21.028 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37516.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37516.log.gz 2024-01-15T18:25:21.030 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38619.log 2024-01-15T18:25:21.032 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35291.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35291.log.gz 2024-01-15T18:25:21.034 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51278.log 2024-01-15T18:25:21.036 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38619.log.gz 2024-01-15T18:25:21.042 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45055.log 2024-01-15T18:25:21.044 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51278.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51278.log.gz 2024-01-15T18:25:21.048 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45055.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45055.log.gz 2024-01-15T18:25:21.049 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47690.log 2024-01-15T18:25:21.054 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53951.log 2024-01-15T18:25:21.056 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47690.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47690.log.gz 2024-01-15T18:25:21.058 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53423.log 2024-01-15T18:25:21.060 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53951.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53951.log.gz 2024-01-15T18:25:21.061 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44382.log 2024-01-15T18:25:21.065 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53423.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53423.log.gz 2024-01-15T18:25:21.066 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35715.log 2024-01-15T18:25:21.068 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44382.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44382.log.gz 2024-01-15T18:25:21.070 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36243.log 2024-01-15T18:25:21.072 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35715.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35715.log.gz 2024-01-15T18:25:21.074 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40097.log/var/log/ceph/ceph-client.admin.36243.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36243.log.gz 2024-01-15T18:25:21.077 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:21.082 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46476.log 2024-01-15T18:25:21.084 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40097.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2024-01-15T18:25:21.086 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53479.log 2024-01-15T18:25:21.088 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46476.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46476.log.gz 2024-01-15T18:25:21.091 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-01-15T18:25:21.092 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53479.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53479.log.gz 2024-01-15T18:25:21.094 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-01-15T18:25:21.096 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43850.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-01-15T18:25:21.098 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56190.log 2024-01-15T18:25:21.101 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40489.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-01-15T18:25:21.103 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51472.log 2024-01-15T18:25:21.104 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56190.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56190.log.gz 2024-01-15T18:25:21.108 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51472.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51472.log.gz 2024-01-15T18:25:21.109 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2024-01-15T18:25:21.114 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51643.log 2024-01-15T18:25:21.116 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41762.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2024-01-15T18:25:21.118 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2024-01-15T18:25:21.120 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51643.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51643.log.gz 2024-01-15T18:25:21.122 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55975.log 2024-01-15T18:25:21.124 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44630.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2024-01-15T18:25:21.130 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53926.log 2024-01-15T18:25:21.132 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55975.log.gz 2024-01-15T18:25:21.134 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41840.log 2024-01-15T18:25:21.136 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53926.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53926.log.gz 2024-01-15T18:25:21.138 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55436.log 2024-01-15T18:25:21.141 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41840.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41840.log.gz 2024-01-15T18:25:21.144 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55436.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55436.log.gz 2024-01-15T18:25:21.146 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36716.log 2024-01-15T18:25:21.150 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47120.log 2024-01-15T18:25:21.152 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36716.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36716.log.gz 2024-01-15T18:25:21.154 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38731.log 2024-01-15T18:25:21.156 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47120.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47120.log.gz 2024-01-15T18:25:21.157 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49640.log 2024-01-15T18:25:21.162 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38731.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38731.log.gz 2024-01-15T18:25:21.163 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43251.log 2024-01-15T18:25:21.164 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49640.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49640.log.gz 2024-01-15T18:25:21.166 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37120.log 2024-01-15T18:25:21.168 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43251.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43251.log.gz 2024-01-15T18:25:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37120.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37120.log.gz 2024-01-15T18:25:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55306.log 2024-01-15T18:25:21.178 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45642.log 2024-01-15T18:25:21.180 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55306.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55306.log.gz 2024-01-15T18:25:21.183 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35369.log 2024-01-15T18:25:21.184 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45642.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45642.log.gz 2024-01-15T18:25:21.186 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52986.log 2024-01-15T18:25:21.188 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35369.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35369.log.gz 2024-01-15T18:25:21.194 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36300.log 2024-01-15T18:25:21.196 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52986.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52986.log.gz 2024-01-15T18:25:21.198 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-01-15T18:25:21.200 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36300.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36300.log.gz 2024-01-15T18:25:21.202 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-01-15T18:25:21.204 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35850.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-01-15T18:25:21.208 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40828.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-01-15T18:25:21.209 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39475.log 2024-01-15T18:25:21.214 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52960.log 2024-01-15T18:25:21.216 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39475.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39475.log.gz 2024-01-15T18:25:21.218 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51099.log 2024-01-15T18:25:21.220 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52960.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52960.log.gz 2024-01-15T18:25:21.222 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-15T18:25:21.224 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51099.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51099.log.gz 2024-01-15T18:25:21.231 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33702.log 2024-01-15T18:25:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48586.log 2024-01-15T18:25:21.235 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33702.log: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-15T18:25:21.236 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33702.log.gz 2024-01-15T18:25:21.238 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32831.log 2024-01-15T18:25:21.240 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48586.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48586.log.gz 2024-01-15T18:25:21.242 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2024-01-15T18:25:21.249 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.32831.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32831.log.gz 2024-01-15T18:25:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46962.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2024-01-15T18:25:21.250 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27084.log 2024-01-15T18:25:21.254 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36112.log 2024-01-15T18:25:21.256 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.27084.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27084.log.gz 2024-01-15T18:25:21.258 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41253.log 2024-01-15T18:25:21.260 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36112.log.gz 2024-01-15T18:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39501.log 2024-01-15T18:25:21.262 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41253.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41253.log.gz 2024-01-15T18:25:21.267 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-01-15T18:25:21.268 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39501.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39501.log.gz 2024-01-15T18:25:21.275 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32982.log 2024-01-15T18:25:21.276 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50299.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-01-15T18:25:21.279 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49249.log 2024-01-15T18:25:21.284 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.32982.log: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46241.log 2024-01-15T18:25:21.285 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49249.log: -- replaced with /var/log/ceph/ceph-client.admin.32982.log.gz 2024-01-15T18:25:21.286 INFO:teuthology.orchestra.run.smithi143.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49249.log.gz 2024-01-15T18:25:21.286 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37041.log 2024-01-15T18:25:21.288 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46241.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46241.log.gz 2024-01-15T18:25:21.290 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37041.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37041.log.gz 2024-01-15T18:25:21.293 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47276.log 2024-01-15T18:25:21.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33008.log 2024-01-15T18:25:21.300 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47276.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47276.log.gz 2024-01-15T18:25:21.302 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40619.log 2024-01-15T18:25:21.304 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33008.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33008.log.gz 2024-01-15T18:25:21.307 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33914.log 2024-01-15T18:25:21.309 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40619.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40619.log.gz 2024-01-15T18:25:21.311 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41068.log 2024-01-15T18:25:21.318 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33914.log: 87.7%gzip/var/log/ceph/ceph-client.admin.41068.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33914.log.gz -5 2024-01-15T18:25:21.318 INFO:teuthology.orchestra.run.smithi143.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42574.log 2024-01-15T18:25:21.319 INFO:teuthology.orchestra.run.smithi143.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.37699.log.gz 2024-01-15T18:25:21.321 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41068.log.gz 2024-01-15T18:25:21.322 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2024-01-15T18:25:21.324 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42574.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42574.log.gz 2024-01-15T18:25:21.325 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41422.log 2024-01-15T18:25:21.327 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38351.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2024-01-15T18:25:21.330 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54859.log 2024-01-15T18:25:21.332 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41422.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41422.log.gz 2024-01-15T18:25:21.334 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48940.log 2024-01-15T18:25:21.339 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54859.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54859.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-01-15T18:25:21.340 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48940.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48940.log.gz 2024-01-15T18:25:21.341 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:21.342 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36378.log 2024-01-15T18:25:21.344 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41918.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-01-15T18:25:21.346 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54054.log 2024-01-15T18:25:21.348 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36378.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36378.log.gz 2024-01-15T18:25:21.352 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54054.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54054.log.gz 2024-01-15T18:25:21.353 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33906.log 2024-01-15T18:25:21.358 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38835.log 2024-01-15T18:25:21.360 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33906.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33906.log.gz 2024-01-15T18:25:21.362 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40017.log 2024-01-15T18:25:21.364 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38835.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38835.log.gz 2024-01-15T18:25:21.367 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44656.log 2024-01-15T18:25:21.371 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40017.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52907.log 2024-01-15T18:25:21.373 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44656.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40017.log.gz 2024-01-15T18:25:21.375 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38429.log 2024-01-15T18:25:21.376 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52907.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52907.log.gz 2024-01-15T18:25:21.378 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44656.log.gz 2024-01-15T18:25:21.378 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38325.log 2024-01-15T18:25:21.381 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38429.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38429.log.gz 2024-01-15T18:25:21.384 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38325.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38325.log.gz 2024-01-15T18:25:21.385 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46007.log 2024-01-15T18:25:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54632.log 2024-01-15T18:25:21.391 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49485.log 2024-01-15T18:25:21.394 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54632.log: 86.3% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46007.log.gz 2024-01-15T18:25:21.394 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-01-15T18:25:21.397 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49485.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54632.log.gz 2024-01-15T18:25:21.398 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42127.log 2024-01-15T18:25:21.400 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38946.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-01-15T18:25:21.401 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49485.log.gz 2024-01-15T18:25:21.406 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33706.log 2024-01-15T18:25:21.408 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42127.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42127.log.gz 2024-01-15T18:25:21.410 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50432.log 2024-01-15T18:25:21.412 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33706.log.gz 2024-01-15T18:25:21.416 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50432.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50432.log.gz 2024-01-15T18:25:21.417 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-01-15T18:25:21.423 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45564.log 2024-01-15T18:25:21.424 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44118.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-01-15T18:25:21.428 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45564.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45564.log.gz 2024-01-15T18:25:21.429 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46359.log 2024-01-15T18:25:21.434 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40357.log 2024-01-15T18:25:21.436 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46359.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46359.log.gz 2024-01-15T18:25:21.440 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40357.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40357.log.gz 2024-01-15T18:25:21.441 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34198.log 2024-01-15T18:25:21.447 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55793.log 2024-01-15T18:25:21.448 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34198.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34198.log.gz 2024-01-15T18:25:21.450 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2024-01-15T18:25:21.452 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55793.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55793.log.gz 2024-01-15T18:25:21.456 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44708.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2024-01-15T18:25:21.457 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45538.log 2024-01-15T18:25:21.463 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55013.log 2024-01-15T18:25:21.464 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45538.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45538.log.gz 2024-01-15T18:25:21.466 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54834.log 2024-01-15T18:25:21.468 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55013.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55013.log.gz 2024-01-15T18:25:21.470 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-01-15T18:25:21.472 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54834.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54834.log.gz 2024-01-15T18:25:21.476 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42973.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz 2024-01-15T18:25:21.477 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49175.log 2024-01-15T18:25:21.482 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47794.log 2024-01-15T18:25:21.487 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.49175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2024-01-15T18:25:21.488 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47794.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-01-15T18:25:21.494 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55196.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51567.log 2024-01-15T18:25:21.494 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2024-01-15T18:25:21.499 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51021.log 2024-01-15T18:25:21.501 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51567.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51567.log.gz 2024-01-15T18:25:21.504 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40383.log/var/log/ceph/ceph-client.admin.51021.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51021.log.gz 2024-01-15T18:25:21.505 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:21.510 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51537.log 2024-01-15T18:25:21.512 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40383.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40383.log.gz 2024-01-15T18:25:21.515 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33899.log 2024-01-15T18:25:21.516 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51537.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51537.log.gz 2024-01-15T18:25:21.519 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50458.log 2024-01-15T18:25:21.520 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33899.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33899.log.gz 2024-01-15T18:25:21.526 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50458.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48508.log 2024-01-15T18:25:21.530 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50458.log.gz 2024-01-15T18:25:21.531 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37308.log 2024-01-15T18:25:21.532 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48508.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48508.log.gz 2024-01-15T18:25:21.535 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48784.log 2024-01-15T18:25:21.537 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37308.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37308.log.gz 2024-01-15T18:25:21.540 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50354.log 2024-01-15T18:25:21.543 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% 96.4% -- replaced with /var/log/ceph/ceph-client.0.51204.log.gz 2024-01-15T18:25:21.543 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48784.log.gz 2024-01-15T18:25:21.544 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50354.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-01-15T18:25:21.548 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33418.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-01-15T18:25:21.548 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50354.log.gzgzip 2024-01-15T18:25:21.548 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49403.log 2024-01-15T18:25:21.551 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50745.log 2024-01-15T18:25:21.554 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50745.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50745.log.gz 2024-01-15T18:25:21.555 INFO:teuthology.orchestra.run.smithi143.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49403.log.gz 2024-01-15T18:25:21.555 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33689.log 2024-01-15T18:25:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33689.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33689.log.gz 2024-01-15T18:25:21.558 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-01-15T18:25:21.561 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46216.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-01-15T18:25:21.561 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36217.log 2024-01-15T18:25:21.568 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36217.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36217.log.gz 2024-01-15T18:25:21.568 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49327.log 2024-01-15T18:25:21.568 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49327.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49327.log.gz 2024-01-15T18:25:21.568 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41866.log 2024-01-15T18:25:21.571 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41866.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41866.log.gz 2024-01-15T18:25:21.571 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38064.log 2024-01-15T18:25:21.577 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38064.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38064.log.gz 2024-01-15T18:25:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49587.log 2024-01-15T18:25:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49587.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49587.log.gz 2024-01-15T18:25:21.578 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44205.log 2024-01-15T18:25:21.581 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44205.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44205.log.gz 2024-01-15T18:25:21.581 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34458.log 2024-01-15T18:25:21.585 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34458.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34458.log.gz 2024-01-15T18:25:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41357.log 2024-01-15T18:25:21.586 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53269.log 2024-01-15T18:25:21.587 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41357.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41357.log.gz 2024-01-15T18:25:21.590 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43511.log 2024-01-15T18:25:21.594 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43511.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43511.log.gz 2024-01-15T18:25:21.594 INFO:teuthology.orchestra.run.smithi143.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53269.log.gz 2024-01-15T18:25:21.594 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39710.log 2024-01-15T18:25:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.39710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37825.log 2024-01-15T18:25:21.607 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42470.log 2024-01-15T18:25:21.608 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37825.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37825.log.gz 2024-01-15T18:25:21.612 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42470.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42470.log.gz 2024-01-15T18:25:21.612 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39605.log 2024-01-15T18:25:21.620 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39605.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39605.log.gz 2024-01-15T18:25:21.621 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48410.log 2024-01-15T18:25:21.628 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48410.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48410.log.gz 2024-01-15T18:25:21.628 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55410.log 2024-01-15T18:25:21.630 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33366.log 2024-01-15T18:25:21.635 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55410.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55410.log.gz 2024-01-15T18:25:21.635 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5/var/log/ceph/ceph-client.admin.33366.log: --verbose -- /var/log/ceph/ceph-client.admin.39238.log 2024-01-15T18:25:21.636 INFO:teuthology.orchestra.run.smithi143.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33366.log.gz 2024-01-15T18:25:21.638 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43331.log 2024-01-15T18:25:21.640 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39238.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39238.log.gz 2024-01-15T18:25:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43331.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43331.log.gz 2024-01-15T18:25:21.644 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54959.log 2024-01-15T18:25:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54959.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54959.log.gz 2024-01-15T18:25:21.652 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46059.log 2024-01-15T18:25:21.654 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53901.log 2024-01-15T18:25:21.659 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46059.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46059.log.gz 2024-01-15T18:25:21.660 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39176.log 2024-01-15T18:25:21.660 INFO:teuthology.orchestra.run.smithi143.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.53901.log.gz 2024-01-15T18:25:21.662 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33886.log 2024-01-15T18:25:21.664 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39176.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39176.log.gz 2024-01-15T18:25:21.671 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33886.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33886.log.gz 2024-01-15T18:25:21.672 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-01-15T18:25:21.674 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50803.log 2024-01-15T18:25:21.676 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50575.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-01-15T18:25:21.682 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48810.log 2024-01-15T18:25:21.684 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50803.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50803.log.gz 2024-01-15T18:25:21.686 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43667.log 2024-01-15T18:25:21.688 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48810.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48810.log.gz 2024-01-15T18:25:21.689 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43798.log 2024-01-15T18:25:21.690 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43667.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43667.log.gz 2024-01-15T18:25:21.694 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47172.log 2024-01-15T18:25:21.695 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43798.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43798.log.gz 2024-01-15T18:25:21.698 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54154.log 2024-01-15T18:25:21.700 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47172.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz 2024-01-15T18:25:21.703 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47820.log 2024-01-15T18:25:21.704 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54154.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54154.log.gz 2024-01-15T18:25:21.708 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47820.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47820.log.gz 2024-01-15T18:25:21.710 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42652.log 2024-01-15T18:25:21.715 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46936.log 2024-01-15T18:25:21.716 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42652.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42652.log.gz 2024-01-15T18:25:21.718 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35746.log 2024-01-15T18:25:21.720 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37490.log 2024-01-15T18:25:21.720 INFO:teuthology.orchestra.run.smithi143.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46936.log.gz 2024-01-15T18:25:21.724 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35746.log: /var/log/ceph/ceph-client.admin.37490.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37490.log.gz 88.5% 2024-01-15T18:25:21.725 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35746.log.gz 2024-01-15T18:25:21.725 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43824.log 2024-01-15T18:25:21.730 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50943.log 2024-01-15T18:25:21.732 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43824.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43824.log.gz 2024-01-15T18:25:21.734 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46742.log 2024-01-15T18:25:21.736 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50943.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50943.log.gz 2024-01-15T18:25:21.738 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46742.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46742.log.gz 2024-01-15T18:25:21.742 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55280.log 2024-01-15T18:25:21.746 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-01-15T18:25:21.748 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55280.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55280.log.gz 2024-01-15T18:25:21.750 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52621.log 2024-01-15T18:25:21.752 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47664.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-01-15T18:25:21.753 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51382.log 2024-01-15T18:25:21.757 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52621.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52621.log.gz 2024-01-15T18:25:21.758 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-01-15T18:25:21.760 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51382.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51382.log.gz 2024-01-15T18:25:21.762 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-01-15T18:25:21.764 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48031.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-01-15T18:25:21.770 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43085.log 2024-01-15T18:25:21.771 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45304.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-01-15T18:25:21.774 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33701.log 2024-01-15T18:25:21.776 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43085.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43085.log.gz 2024-01-15T18:25:21.778 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45093.log 2024-01-15T18:25:21.780 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33701.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33701.log.gz 2024-01-15T18:25:21.782 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-01-15T18:25:21.784 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45093.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45093.log.gz 2024-01-15T18:25:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39148.log 2024-01-15T18:25:21.789 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41944.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 2024-01-15T18:25:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48325.log 2024-01-15T18:25:21.792 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39148.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39148.log.gz 2024-01-15T18:25:21.794 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49853.log 2024-01-15T18:25:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48325.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48325.log.gz 2024-01-15T18:25:21.804 INFO:teuthology.orchestra.run.smithi143.stderr:gzip/var/log/ceph/ceph-client.admin.49853.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49853.log.gz 2024-01-15T18:25:21.804 INFO:teuthology.orchestra.run.smithi143.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47534.log 2024-01-15T18:25:21.806 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36794.log 2024-01-15T18:25:21.808 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47534.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47534.log.gz 2024-01-15T18:25:21.810 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53649.log 2024-01-15T18:25:21.814 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36794.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36794.log.gz 2024-01-15T18:25:21.814 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47950.log 2024-01-15T18:25:21.820 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53649.log: 88.2%gzip/var/log/ceph/ceph-client.admin.47950.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47950.log.gz 2024-01-15T18:25:21.821 INFO:teuthology.orchestra.run.smithi143.stderr: -5 -- replaced with /var/log/ceph/ceph-client.admin.53649.log.gz --verbose 2024-01-15T18:25:21.822 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.54356.log 2024-01-15T18:25:21.826 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38299.log 2024-01-15T18:25:21.828 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54356.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54356.log.gz 2024-01-15T18:25:21.831 INFO:teuthology.orchestra.run.smithi143.stderr: 96.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37198.log 2024-01-15T18:25:21.831 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.0.42049.log.gz 2024-01-15T18:25:21.832 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38299.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38299.log.gz 2024-01-15T18:25:21.834 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44865.log 2024-01-15T18:25:21.836 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37198.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37198.log.gz 2024-01-15T18:25:21.842 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34224.log 2024-01-15T18:25:21.846 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.0.44865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43907.log 2024-01-15T18:25:21.848 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34224.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34224.log.gz 2024-01-15T18:25:21.850 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35133.log 2024-01-15T18:25:21.852 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43907.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43907.log.gz 2024-01-15T18:25:21.856 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35133.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35133.log.gz 2024-01-15T18:25:21.857 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49535.log 2024-01-15T18:25:21.862 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47302.log 2024-01-15T18:25:21.864 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49535.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49535.log.gz 2024-01-15T18:25:21.866 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52725.log 2024-01-15T18:25:21.869 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50247.log 2024-01-15T18:25:21.869 INFO:teuthology.orchestra.run.smithi143.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47302.log.gz 2024-01-15T18:25:21.872 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50247.log: /var/log/ceph/ceph-client.admin.52725.log: 88.3% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50247.log.gz -- replaced with /var/log/ceph/ceph-client.admin.52725.log.gz 2024-01-15T18:25:21.872 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:21.872 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-01-15T18:25:21.874 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43693.log 2024-01-15T18:25:21.876 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36892.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-01-15T18:25:21.882 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44498.log 2024-01-15T18:25:21.884 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43693.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43693.log.gz 2024-01-15T18:25:21.886 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39096.log 2024-01-15T18:25:21.888 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44498.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44498.log.gz 2024-01-15T18:25:21.893 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39096.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39096.log.gz 2024-01-15T18:25:21.894 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37412.log 2024-01-15T18:25:21.899 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36482.log 2024-01-15T18:25:21.899 INFO:teuthology.orchestra.run.smithi143.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-15T18:25:21.900 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37412.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37412.log.gz 2024-01-15T18:25:21.904 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36482.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36482.log.gz 2024-01-15T18:25:21.906 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51511.log 2024-01-15T18:25:21.911 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47482.log 2024-01-15T18:25:21.912 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51511.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51511.log.gz 2024-01-15T18:25:21.915 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35265.log 2024-01-15T18:25:21.919 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47482.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45356.log 2024-01-15T18:25:21.920 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35265.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35265.log.gz 2024-01-15T18:25:21.922 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47482.log.gz 2024-01-15T18:25:21.922 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-15T18:25:21.924 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45356.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45356.log.gz 2024-01-15T18:25:21.929 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53012.log 2024-01-15T18:25:21.934 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-15T18:25:21.936 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53012.log: 90.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-15T18:25:21.937 INFO:teuthology.orchestra.run.smithi143.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53012.log.gz 2024-01-15T18:25:21.938 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50216.log 2024-01-15T18:25:21.942 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39939.log 2024-01-15T18:25:21.944 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50216.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50216.log.gz 2024-01-15T18:25:21.946 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44913.log 2024-01-15T18:25:21.948 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39939.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39939.log.gz 2024-01-15T18:25:21.951 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44913.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44913.log.gz 2024-01-15T18:25:21.951 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40435.log 2024-01-15T18:25:21.956 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40435.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40435.log.gz 2024-01-15T18:25:21.957 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56053.log 2024-01-15T18:25:21.963 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46085.log 2024-01-15T18:25:21.964 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56053.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56053.log.gz 2024-01-15T18:25:21.966 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47508.log 2024-01-15T18:25:21.968 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46085.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46085.log.gz 2024-01-15T18:25:21.969 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37250.log 2024-01-15T18:25:21.971 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47508.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47508.log.gz 2024-01-15T18:25:21.974 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-15T18:25:21.976 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37250.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37250.log.gz 2024-01-15T18:25:21.983 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39122.log 2024-01-15T18:25:21.987 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36165.log 2024-01-15T18:25:21.994 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39122.log: 89.2%/var/log/ceph/ceph-client.admin.36165.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39122.log.gz 2024-01-15T18:25:21.994 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40985.log 2024-01-15T18:25:21.998 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36165.log.gz 2024-01-15T18:25:21.998 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36430.log 2024-01-15T18:25:22.000 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40985.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40985.log.gz 2024-01-15T18:25:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43988.log 2024-01-15T18:25:22.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36430.log: 88.2% 96.4% -- replaced with /var/log/ceph/ceph-client.0.53117.log.gz 2024-01-15T18:25:22.005 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36430.log.gz 2024-01-15T18:25:22.006 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-01-15T18:25:22.008 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45590.log 2024-01-15T18:25:22.009 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46412.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43988.log.gz 2024-01-15T18:25:22.010 INFO:teuthology.orchestra.run.smithi143.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-01-15T18:25:22.011 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48247.log 2024-01-15T18:25:22.015 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43537.log 2024-01-15T18:25:22.018 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48247.log: 89.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45590.log.gz 2024-01-15T18:25:22.020 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43537.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43537.log.gz 2024-01-15T18:25:22.021 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48247.log.gz 2024-01-15T18:25:22.021 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41149.log 2024-01-15T18:25:22.026 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-01-15T18:25:22.028 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41149.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41149.log.gz 2024-01-15T18:25:22.030 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54481.log 2024-01-15T18:25:22.032 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45799.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-01-15T18:25:22.034 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-01-15T18:25:22.035 INFO:teuthology.orchestra.run.smithi204.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-15T18:25:22.036 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54481.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.54481.log.gz 2024-01-15T18:25:22.042 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53555.log 2024-01-15T18:25:22.044 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40880.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-01-15T18:25:22.046 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-01-15T18:25:22.048 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53555.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53555.log.gz 2024-01-15T18:25:22.050 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51617.log 2024-01-15T18:25:22.052 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43025.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-01-15T18:25:22.054 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56079.log 2024-01-15T18:25:22.056 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51617.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51617.log.gz 2024-01-15T18:25:22.060 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56079.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56079.log.gz 2024-01-15T18:25:22.061 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54784.log 2024-01-15T18:25:22.066 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2024-01-15T18:25:22.068 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54784.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.54784.log.gz 2024-01-15T18:25:22.072 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46191.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2024-01-15T18:25:22.073 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41682.log 2024-01-15T18:25:22.077 INFO:teuthology.orchestra.run.smithi204.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-15T18:25:22.079 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43747.log 2024-01-15T18:25:22.080 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41682.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41682.log.gz 2024-01-15T18:25:22.083 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53825.log 2024-01-15T18:25:22.084 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43747.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43747.log.gz 2024-01-15T18:25:22.088 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53825.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.53825.log.gz 2024-01-15T18:25:22.089 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34867.log 2024-01-15T18:25:22.094 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-01-15T18:25:22.096 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34867.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34867.log.gz 2024-01-15T18:25:22.099 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56027.log 2024-01-15T18:25:22.100 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48836.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-01-15T18:25:22.102 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48195.log 2024-01-15T18:25:22.104 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56027.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56027.log.gz 2024-01-15T18:25:22.106 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-01-15T18:25:22.107 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48195.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48195.log.gz 2024-01-15T18:25:22.110 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39264.log 2024-01-15T18:25:22.112 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45175.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-01-15T18:25:22.116 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39264.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39264.log.gz 2024-01-15T18:25:22.118 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55585.log 2024-01-15T18:25:22.122 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55514.log 2024-01-15T18:25:22.124 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55585.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55585.log.gz 2024-01-15T18:25:22.126 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45668.log 2024-01-15T18:25:22.128 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55514.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55514.log.gz 2024-01-15T18:25:22.130 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39902.log 2024-01-15T18:25:22.132 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45668.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45668.log.gz 2024-01-15T18:25:22.133 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33698.log 2024-01-15T18:25:22.135 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39902.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39902.log.gz 2024-01-15T18:25:22.138 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43933.log 2024-01-15T18:25:22.141 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33698.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33698.log.gz 2024-01-15T18:25:22.142 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47742.log 2024-01-15T18:25:22.144 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43933.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43933.log.gz 2024-01-15T18:25:22.147 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47742.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47742.log.gz 2024-01-15T18:25:22.149 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50719.log 2024-01-15T18:25:22.154 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log 2024-01-15T18:25:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50719.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50719.log.gz 2024-01-15T18:25:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56228.log 2024-01-15T18:25:22.160 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35480.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-01-15T18:25:22.161 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35798.log 2024-01-15T18:25:22.166 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.56228.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56228.log.gz 2024-01-15T18:25:22.168 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35798.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35798.log.gz 2024-01-15T18:25:22.169 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-01-15T18:25:22.174 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-01-15T18:25:22.176 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37334.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-01-15T18:25:22.178 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-01-15T18:25:22.180 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48730.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-01-15T18:25:22.182 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44408.log 2024-01-15T18:25:22.184 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43461.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-01-15T18:25:22.186 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36008.log 2024-01-15T18:25:22.191 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44408.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44408.log.gz 2024-01-15T18:25:22.192 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36008.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36008.log.gz 2024-01-15T18:25:22.193 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33178.log 2024-01-15T18:25:22.199 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2024-01-15T18:25:22.200 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33178.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33178.log.gz 2024-01-15T18:25:22.202 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53529.log 2024-01-15T18:25:22.204 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52395.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-01-15T18:25:22.206 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36191.log 2024-01-15T18:25:22.208 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53529.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53529.log.gz 2024-01-15T18:25:22.209 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53062.log 2024-01-15T18:25:22.211 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36191.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36191.log.gz 2024-01-15T18:25:22.214 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48704.log 2024-01-15T18:25:22.216 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53062.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53062.log.gz 2024-01-15T18:25:22.224 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48704.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48704.log.gz 2024-01-15T18:25:22.225 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53217.log 2024-01-15T18:25:22.231 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-01-15T18:25:22.232 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53217.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53217.log.gz 2024-01-15T18:25:22.235 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54229.log 2024-01-15T18:25:22.236 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45720.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-01-15T18:25:22.238 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-01-15T18:25:22.244 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54229.log: 86.8%gzip -5 --verbose/var/log/ceph/ceph-client.admin.41037.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-01-15T18:25:22.246 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54229.log.gz -- 2024-01-15T18:25:22.246 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.43962.log 2024-01-15T18:25:22.251 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35211.log 2024-01-15T18:25:22.252 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43962.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43962.log.gz 2024-01-15T18:25:22.254 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54381.log 2024-01-15T18:25:22.256 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35211.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35211.log.gz 2024-01-15T18:25:22.260 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54381.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54381.log.gz 2024-01-15T18:25:22.261 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41734.log 2024-01-15T18:25:22.266 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46910.log 2024-01-15T18:25:22.268 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41734.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41734.log.gz 2024-01-15T18:25:22.270 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35928.log 2024-01-15T18:25:22.272 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46910.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46910.log.gz 2024-01-15T18:25:22.274 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48299.log 2024-01-15T18:25:22.276 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35928.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35928.log.gz 2024-01-15T18:25:22.278 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2024-01-15T18:25:22.280 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48299.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48299.log.gz 2024-01-15T18:25:22.282 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-01-15T18:25:22.284 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50190.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2024-01-15T18:25:22.286 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38168.log 2024-01-15T18:25:22.288 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40319.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz 2024-01-15T18:25:22.292 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38168.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38168.log.gz 2024-01-15T18:25:22.293 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51775.log 2024-01-15T18:25:22.298 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39655.log 2024-01-15T18:25:22.300 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51775.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51775.log.gz 2024-01-15T18:25:22.302 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50113.log 2024-01-15T18:25:22.304 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39655.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39655.log.gz 2024-01-15T18:25:22.306 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33498.log 2024-01-15T18:25:22.308 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50113.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50113.log.gz 2024-01-15T18:25:22.310 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33060.log 2024-01-15T18:25:22.312 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33498.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33498.log.gz 2024-01-15T18:25:22.314 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36586.log 2024-01-15T18:25:22.316 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33060.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33060.log.gz 2024-01-15T18:25:22.318 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44330.log 2024-01-15T18:25:22.320 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36586.log.gz 2024-01-15T18:25:22.324 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44330.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44330.log.gz 2024-01-15T18:25:22.325 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50837.log 2024-01-15T18:25:22.330 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50380.log 2024-01-15T18:25:22.332 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50837.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50837.log.gz 2024-01-15T18:25:22.334 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-01-15T18:25:22.336 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50380.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50380.log.gz 2024-01-15T18:25:22.338 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36404.log 2024-01-15T18:25:22.340 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46267.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-01-15T18:25:22.342 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2024-01-15T18:25:22.344 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36404.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36404.log.gz 2024-01-15T18:25:22.347 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54734.log 2024-01-15T18:25:22.348 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47560.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2024-01-15T18:25:22.350 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35055.log 2024-01-15T18:25:22.352 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54734.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.54734.log.gz 2024-01-15T18:25:22.361 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35055.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35055.log.gz 2024-01-15T18:25:22.361 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-01-15T18:25:22.362 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35343.log 2024-01-15T18:25:22.364 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43279.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-01-15T18:25:22.366 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48166.log 2024-01-15T18:25:22.368 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35343.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35343.log.gz 2024-01-15T18:25:22.370 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40177.log 2024-01-15T18:25:22.372 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48166.log.gz 2024-01-15T18:25:22.374 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52565.log 2024-01-15T18:25:22.376 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40177.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40177.log.gz 2024-01-15T18:25:22.378 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52278.log 2024-01-15T18:25:22.380 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52565.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52565.log.gz 2024-01-15T18:25:22.382 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54607.log 2024-01-15T18:25:22.384 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52278.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52278.log.gz 2024-01-15T18:25:22.388 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54607.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54607.log.gz 2024-01-15T18:25:22.389 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53800.log 2024-01-15T18:25:22.394 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-15T18:25:22.395 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46990.log 2024-01-15T18:25:22.396 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53800.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.53800.log.gz 2024-01-15T18:25:22.398 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55819.log 2024-01-15T18:25:22.400 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53385.log 2024-01-15T18:25:22.400 INFO:teuthology.orchestra.run.smithi143.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46990.log.gz 2024-01-15T18:25:22.403 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46319.log 2024-01-15T18:25:22.404 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53385.log: /var/log/ceph/ceph-client.admin.55819.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53385.log.gz 2024-01-15T18:25:22.404 INFO:teuthology.orchestra.run.smithi143.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55819.log.gz 2024-01-15T18:25:22.406 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48377.log 2024-01-15T18:25:22.408 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46319.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46319.log.gz 2024-01-15T18:25:22.409 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48377.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.48377.log.gz 2024-01-15T18:25:22.413 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47224.log 2024-01-15T18:25:22.418 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54029.log 2024-01-15T18:25:22.420 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47224.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-01-15T18:25:22.422 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52252.log 2024-01-15T18:25:22.424 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54029.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54029.log.gz 2024-01-15T18:25:22.426 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35689.log 2024-01-15T18:25:22.428 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52252.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52252.log.gz 2024-01-15T18:25:22.430 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39070.log 2024-01-15T18:25:22.432 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35689.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35689.log.gz 2024-01-15T18:25:22.434 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47612.log 2024-01-15T18:25:22.436 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39070.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39070.log.gz 2024-01-15T18:25:22.438 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48670.log 2024-01-15T18:25:22.440 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47612.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47612.log.gz 2024-01-15T18:25:22.442 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54884.log 2024-01-15T18:25:22.448 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48670.log: 87.8%gzip -5 --verbose/var/log/ceph/ceph-client.admin.54884.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54884.log.gz 2024-01-15T18:25:22.450 INFO:teuthology.orchestra.run.smithi143.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48670.log.gz -- 2024-01-15T18:25:22.450 INFO:teuthology.orchestra.run.smithi143.stderr: /var/log/ceph/ceph-client.admin.39812.log 2024-01-15T18:25:22.454 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-01-15T18:25:22.456 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39812.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39812.log.gz 2024-01-15T18:25:22.458 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43138.log 2024-01-15T18:25:22.460 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35902.log: gzip -5 --verbose 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-01-15T18:25:22.461 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.51047.log 2024-01-15T18:25:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43138.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43138.log.gz 2024-01-15T18:25:22.467 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54709.log 2024-01-15T18:25:22.468 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51047.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gz 2024-01-15T18:25:22.470 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43563.log 2024-01-15T18:25:22.472 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54709.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54709.log.gz 2024-01-15T18:25:22.474 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-01-15T18:25:22.476 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43563.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43563.log.gz 2024-01-15T18:25:22.480 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44682.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-01-15T18:25:22.480 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41994.log 2024-01-15T18:25:22.487 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-01-15T18:25:22.488 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41994.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41994.log.gz 2024-01-15T18:25:22.493 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50328.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-01-15T18:25:22.494 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55222.log 2024-01-15T18:25:22.496 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45382.log 2024-01-15T18:25:22.499 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2024-01-15T18:25:22.502 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45382.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55222.log.gz 2024-01-15T18:25:22.503 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41120.log 2024-01-15T18:25:22.504 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51749.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2024-01-15T18:25:22.506 INFO:teuthology.orchestra.run.smithi143.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45382.log.gz 2024-01-15T18:25:22.506 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49613.log 2024-01-15T18:25:22.508 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41120.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41120.log.gz 2024-01-15T18:25:22.512 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49613.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49613.log.gz 2024-01-15T18:25:22.513 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55065.log 2024-01-15T18:25:22.519 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46832.log 2024-01-15T18:25:22.520 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55065.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55065.log.gz 2024-01-15T18:25:22.522 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-01-15T18:25:22.524 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46832.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46832.log.gz 2024-01-15T18:25:22.526 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38593.log 2024-01-15T18:25:22.528 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35611.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-01-15T18:25:22.534 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40778.log 2024-01-15T18:25:22.536 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38593.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38593.log.gz 2024-01-15T18:25:22.539 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-01-15T18:25:22.540 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40778.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40778.log.gz 2024-01-15T18:25:22.542 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2024-01-15T18:25:22.548 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.48057.log: 89.2%/var/log/ceph/ceph-client.admin.50543.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2024-01-15T18:25:22.549 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz --verbose 2024-01-15T18:25:22.550 INFO:teuthology.orchestra.run.smithi143.stderr: -- /var/log/ceph/ceph-client.admin.43409.log 2024-01-15T18:25:22.554 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-01-15T18:25:22.556 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.43409.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43409.log.gz 2024-01-15T18:25:22.558 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33116.log 2024-01-15T18:25:22.560 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44014.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-01-15T18:25:22.562 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53359.log 2024-01-15T18:25:22.564 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38273.log 2024-01-15T18:25:22.564 INFO:teuthology.orchestra.run.smithi143.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33116.log.gz 2024-01-15T18:25:22.571 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53359.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53359.log.gz 2024-01-15T18:25:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38273.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38273.log.gz 2024-01-15T18:25:22.572 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33144.log 2024-01-15T18:25:22.574 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-01-15T18:25:22.576 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33144.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33144.log.gz 2024-01-15T18:25:22.578 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35159.log 2024-01-15T18:25:22.580 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47354.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-01-15T18:25:22.586 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37438.log 2024-01-15T18:25:22.588 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35159.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35159.log.gz 2024-01-15T18:25:22.590 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52200.log 2024-01-15T18:25:22.592 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37438.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37438.log.gz 2024-01-15T18:25:22.594 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41788.log 2024-01-15T18:25:22.596 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52200.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52200.log.gz 2024-01-15T18:25:22.599 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55144.log 2024-01-15T18:25:22.600 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41788.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41788.log.gz 2024-01-15T18:25:22.604 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55144.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55144.log.gz 2024-01-15T18:25:22.605 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2024-01-15T18:25:22.610 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-01-15T18:25:22.612 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41227.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2024-01-15T18:25:22.614 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53850.log 2024-01-15T18:25:22.616 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51356.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-01-15T18:25:22.618 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50273.log 2024-01-15T18:25:22.620 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53850.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.53850.log.gz 2024-01-15T18:25:22.626 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53611.log 2024-01-15T18:25:22.628 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50273.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50273.log.gz 2024-01-15T18:25:22.631 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51905.log 2024-01-15T18:25:22.632 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53611.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53611.log.gz 2024-01-15T18:25:22.634 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54506.log 2024-01-15T18:25:22.634 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51905.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51905.log.gz 2024-01-15T18:25:22.640 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.54506.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.54506.log.gz 2024-01-15T18:25:22.645 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38676.log 2024-01-15T18:25:22.650 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49120.log 2024-01-15T18:25:22.652 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.38676.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38676.log.gz 2024-01-15T18:25:22.654 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2024-01-15T18:25:22.656 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49120.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49120.log.gz 2024-01-15T18:25:22.658 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39864.log 2024-01-15T18:25:22.660 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40515.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-01-15T18:25:22.666 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34276.log 2024-01-15T18:25:22.668 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39864.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39864.log.gz 2024-01-15T18:25:22.670 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-01-15T18:25:22.672 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34276.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34276.log.gz 2024-01-15T18:25:22.674 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46165.log 2024-01-15T18:25:22.676 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49018.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-01-15T18:25:22.679 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55637.log 2024-01-15T18:25:22.679 INFO:teuthology.orchestra.run.smithi204.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-15T18:25:22.680 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46165.log.gz 2024-01-15T18:25:22.681 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-01-15T18:25:22.681 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m2.769s 2024-01-15T18:25:22.682 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m10.487s 2024-01-15T18:25:22.682 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.921s 2024-01-15T18:25:22.683 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47898.log 2024-01-15T18:25:22.684 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55637.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55637.log.gz 2024-01-15T18:25:22.686 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45929.log 2024-01-15T18:25:22.688 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47898.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47898.log.gz 2024-01-15T18:25:22.690 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49931.log 2024-01-15T18:25:22.692 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45929.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45929.log.gz 2024-01-15T18:25:22.693 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46636.log 2024-01-15T18:25:22.695 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49931.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49931.log.gz 2024-01-15T18:25:22.698 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49666.log 2024-01-15T18:25:22.700 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46636.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46636.log.gz 2024-01-15T18:25:22.702 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50085.log 2024-01-15T18:25:22.704 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49666.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49666.log.gz 2024-01-15T18:25:22.706 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41708.log 2024-01-15T18:25:22.708 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50085.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50085.log.gz 2024-01-15T18:25:22.710 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47456.log 2024-01-15T18:25:22.712 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41708.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41708.log.gz 2024-01-15T18:25:22.714 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35081.log 2024-01-15T18:25:22.717 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47456.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47456.log.gz 2024-01-15T18:25:22.720 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35081.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35081.log.gz 2024-01-15T18:25:22.721 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55767.log 2024-01-15T18:25:22.727 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34146.log 2024-01-15T18:25:22.728 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.55767.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55767.log.gz 2024-01-15T18:25:22.730 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-01-15T18:25:22.732 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34146.log.gz 2024-01-15T18:25:22.734 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2024-01-15T18:25:22.736 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.42153.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2024-01-15T18:25:22.742 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40906.log 2024-01-15T18:25:22.744 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46884.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2024-01-15T18:25:22.746 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34951.log 2024-01-15T18:25:22.748 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40906.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40906.log.gz 2024-01-15T18:25:22.750 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46768.log 2024-01-15T18:25:22.752 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.34951.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34951.log.gz 2024-01-15T18:25:22.756 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.46768.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46768.log.gz 2024-01-15T18:25:22.757 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-01-15T18:25:22.761 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2024-01-15T18:25:22.762 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49070.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-01-15T18:25:22.766 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33897.log 2024-01-15T18:25:22.768 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz 2024-01-15T18:25:22.770 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53321.log 2024-01-15T18:25:22.772 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33897.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33897.log.gz 2024-01-15T18:25:22.774 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35447.log 2024-01-15T18:25:22.776 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.53321.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53321.log.gz 2024-01-15T18:25:22.778 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49957.log 2024-01-15T18:25:22.780 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.35447.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35447.log.gz 2024-01-15T18:25:22.781 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49879.log 2024-01-15T18:25:22.786 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49957.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49957.log.gz 2024-01-15T18:25:22.787 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44578.log 2024-01-15T18:25:22.788 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49879.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49879.log.gz 2024-01-15T18:25:22.792 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44578.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44578.log.gz 2024-01-15T18:25:22.794 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-01-15T18:25:22.799 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36638.log 2024-01-15T18:25:22.800 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37015.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-01-15T18:25:22.802 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50917.log 2024-01-15T18:25:22.804 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.36638.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36638.log.gz 2024-01-15T18:25:22.806 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52593.log 2024-01-15T18:25:22.808 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50917.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50917.log.gz 2024-01-15T18:25:22.814 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37773.log 2024-01-15T18:25:22.816 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.52593.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52593.log.gz 2024-01-15T18:25:22.818 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45512.log 2024-01-15T18:25:22.820 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.37773.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37773.log.gz 2024-01-15T18:25:22.821 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40461.log 2024-01-15T18:25:22.827 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.45512.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45512.log.gz 2024-01-15T18:25:22.828 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.40461.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40461.log.gz 2024-01-15T18:25:22.829 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39838.log 2024-01-15T18:25:22.835 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44524.log 2024-01-15T18:25:22.836 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39838.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39838.log.gz 2024-01-15T18:25:22.838 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44464.log 2024-01-15T18:25:22.840 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44524.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44524.log.gz 2024-01-15T18:25:22.841 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51982.log 2024-01-15T18:25:22.846 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.44464.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44464.log.gz 2024-01-15T18:25:22.847 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2024-01-15T18:25:22.848 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51982.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51982.log.gz 2024-01-15T18:25:22.850 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-01-15T18:25:22.852 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.50484.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2024-01-15T18:25:22.854 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33485.log 2024-01-15T18:25:22.856 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41488.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2024-01-15T18:25:22.860 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.33485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33485.log.gz 2024-01-15T18:25:22.861 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47716.log 2024-01-15T18:25:22.867 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39044.log 2024-01-15T18:25:22.868 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.47716.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47716.log.gz 2024-01-15T18:25:22.870 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49275.log 2024-01-15T18:25:22.872 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.39044.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39044.log.gz 2024-01-15T18:25:22.874 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51330.log 2024-01-15T18:25:22.876 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.49275.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49275.log.gz 2024-01-15T18:25:22.878 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41648.log 2024-01-15T18:25:22.880 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51330.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51330.log.gz 2024-01-15T18:25:22.882 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-01-15T18:25:22.884 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.41648.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41648.log.gz 2024-01-15T18:25:22.888 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/ceph-client.admin.51149.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-01-15T18:25:23.061 INFO:teuthology.orchestra.run.smithi143.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.39710.log.gz 2024-01-15T18:25:23.254 INFO:teuthology.orchestra.run.smithi143.stderr: 96.5% -- replaced with /var/log/ceph/ceph-client.0.44865.log.gz 2024-01-15T18:25:23.357 INFO:teuthology.orchestra.run.smithi143.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-15T18:25:23.393 INFO:teuthology.orchestra.run.smithi143.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.0.49175.log.gz 2024-01-15T18:25:23.780 INFO:teuthology.orchestra.run.smithi143.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-15T18:25:24.224 INFO:teuthology.orchestra.run.smithi143.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-15T18:25:24.294 INFO:teuthology.orchestra.run.smithi143.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-15T18:25:24.419 INFO:teuthology.orchestra.run.smithi143.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-15T18:25:24.420 INFO:teuthology.orchestra.run.smithi143.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-15T18:25:25.948 INFO:teuthology.orchestra.run.smithi143.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-15T18:25:25.951 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-01-15T18:25:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m6.051s 2024-01-15T18:25:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:user 0m30.282s 2024-01-15T18:25:25.951 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m3.226s 2024-01-15T18:25:25.951 INFO:tasks.ceph:Archiving logs... 2024-01-15T18:25:25.952 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/remote/smithi143/log 2024-01-15T18:25:25.953 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-15T18:25:29.029 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/remote/smithi204/log 2024-01-15T18:25:29.029 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-15T18:25:30.226 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-15T18:25:30.235 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-15T18:25:30.235 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-15T18:25:30.239 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-15T18:25:30.318 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-15T18:25:30.326 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-15T18:25:30.327 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T18:25:30.330 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T18:25:30.342 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2024-01-15T18:25:30.342 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2024-01-15T18:25:30.342 INFO:teuthology.orchestra.run.smithi143.stdout:*hv01.front.sepi 44.4.53.2 2 u 13 64 377 0.116 -13.454 1.675 2024-01-15T18:25:30.343 INFO:teuthology.orchestra.run.smithi143.stdout:+hv02.front.sepi 63.231.80.2 3 u 24 64 377 0.073 -7.802 2.371 2024-01-15T18:25:30.343 INFO:teuthology.orchestra.run.smithi143.stdout:+hv03.front.sepi 74.6.168.72 3 u 16 64 377 0.067 -14.525 3.097 2024-01-15T18:25:30.343 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-01-15T18:25:30.345 INFO:teuthology.orchestra.run.smithi204.stdout: remote refid st t when poll reach delay offset jitter 2024-01-15T18:25:30.346 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================== 2024-01-15T18:25:30.346 INFO:teuthology.orchestra.run.smithi204.stdout:*hv01.front.sepi 44.4.53.2 2 u 12 64 377 0.073 -3.475 0.291 2024-01-15T18:25:30.346 INFO:teuthology.orchestra.run.smithi204.stdout:+hv02.front.sepi 63.231.80.2 3 u 11 64 377 0.085 +0.218 0.401 2024-01-15T18:25:30.346 INFO:teuthology.orchestra.run.smithi204.stdout:+hv03.front.sepi 74.6.168.72 3 u 22 64 377 0.096 -2.737 0.237 2024-01-15T18:25:30.346 INFO:teuthology.orchestra.run.smithi204.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-01-15T18:25:30.347 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-15T18:25:30.356 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-15T18:25:30.356 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-15T18:25:30.365 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-15T18:25:30.373 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-15T18:25:30.381 INFO:teuthology.task.internal:Duration was 1971.689862 seconds 2024-01-15T18:25:30.382 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-15T18:25:30.391 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-15T18:25:30.391 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-15T18:25:30.395 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-15T18:25:30.496 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-15T18:25:30.496 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-01-15T18:25:30.496 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-15T18:25:30.518 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-01-15T18:25:30.518 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-15T18:25:30.539 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-15T18:25:30.539 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-15T18:25:30.564 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-15T18:25:30.620 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-15T18:25:30.621 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-15T18:25:30.652 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-15T18:25:30.738 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-15T18:25:30.748 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-15T18:25:30.749 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-15T18:25:30.768 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-15T18:25:30.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-15T18:25:30.799 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-15T18:25:30.812 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-15T18:25:30.826 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-01-15T18:25:30.847 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-01-15T18:25:30.865 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-15T18:25:30.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:25:30.893 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-15T18:25:30.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:25:30.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-15T18:25:30.924 INFO:teuthology.task.internal:Transferring archived files... 2024-01-15T18:25:30.924 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/remote/smithi143 2024-01-15T18:25:30.925 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-15T18:25:30.959 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105/remote/smithi204 2024-01-15T18:25:30.960 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-15T18:25:30.985 INFO:teuthology.task.internal:Removing archive directory... 2024-01-15T18:25:30.985 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-15T18:25:31.000 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-15T18:25:31.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-15T18:25:31.041 INFO:teuthology.task.internal:Not uploading archives. 2024-01-15T18:25:31.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-15T18:25:31.051 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-15T18:25:31.051 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-15T18:25:31.054 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-15T18:25:31.059 INFO:teuthology.orchestra.run.smithi143.stdout: 658072 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 15 18:25 /home/ubuntu/cephtest 2024-01-15T18:25:31.078 INFO:teuthology.orchestra.run.smithi204.stdout: 658072 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 15 18:25 /home/ubuntu/cephtest 2024-01-15T18:25:31.079 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-15T18:25:31.090 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-15T18:25:31.110 INFO:teuthology.nuke:Checking targets against current locks 2024-01-15T18:25:31.137 DEBUG:teuthology.nuke:shortname: smithi143 2024-01-15T18:25:31.138 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-15T18:25:31.161 DEBUG:teuthology.nuke:shortname: smithi204 2024-01-15T18:25:31.161 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-15T18:25:31.165 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:54.432779', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAvuBUE6lVmsoYjKOKpKWQ7T+l0ufb15R/3UeDf8iIWTqzvGiIbD/viXR9zGGy67YZY6WEFp9Nu7+ofrtGNcZBY='} 2024-01-15T18:25:31.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:46-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517105', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:54.431240', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC0TxJjXSPdKNBhbICy8dkJxph6idNeJjN3iVR7cgJk1Al0pJ2ycofFY0WlQIAvYDBAwL2oSo0c545raLcSNHVs='} 2024-01-15T18:25:31.218 INFO:teuthology.orchestra.console.smithi143:Power off 2024-01-15T18:25:31.219 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-15T18:25:31.258 INFO:teuthology.orchestra.console.smithi204:Power off 2024-01-15T18:25:31.258 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-15T18:25:31.268 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2024-01-15T18:25:31.269 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:31.282 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-01-15T18:25:31.282 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:31.291 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-01-15T18:25:31.304 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-01-15T18:25:35.292 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:35.305 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:35.402 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-01-15T18:25:35.415 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-01-15T18:25:39.405 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:39.416 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:39.516 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2024-01-15T18:25:39.525 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-01-15T18:25:39.625 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-01-15T18:25:43.520 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:25:43.631 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2024-01-15T18:25:43.732 INFO:teuthology.orchestra.console.smithi143:Power off completed 2024-01-15T18:25:43.837 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} duration: 1971.6898624897003 failure_reason: 'Test failure: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress), test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress)' flavor: default mon.a-kernel-sha1: 2668949e0971849893a3fb7f177ce9032fc65780 mon.b-kernel-sha1: 2668949e0971849893a3fb7f177ce9032fc65780 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=52feaa06080c4043a77be1063634d8cb status: fail success: false 2024-01-15T18:25:43.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-15T18:25:43.926 INFO:teuthology.run:FAIL